Page 1 of 1

QSE future

PostPosted: Fri Jan 09, 2009 3:01 pm
by Riki
Is this project dead or what?
In the past six month I see only lordikon from the project members hanging around?

Anyway, the other day I decided to thoroughly analyze the QSE's current design & implementation.
At the very beginning I realized that there where just to many points to keep in mind so I started a maintaining document...
It all started a month or so ago when I decided to give QSE a try. At first I was pleasantly surprised until I tried to code some 'real' stuff which soon enough became 'mission impossible'.

After digging and digging while trying to learn how to get things done I got just more and more frustrated...
So forgive me my tone, wasn't trying to flame or disrespect the creators/contributors to this project.
Especially as I still want to become a contributor...so while reading my comments please note that I have a deep respect to everyone and everyone's code and I know how hard it is to do something in your free time but I am just trying to keep things rolling. Again, my comments might sound a bit rough - but it was just my frustration so ignore my tone and bear with me and the facts.
Paste out of my document I am still working on:
[spoil]

Messaging System

I wrote to much on the forum regarding this topic to do this again in detail.
Here is a short summary:
* Bloated, way to complicated, user unfriendly, inefficient, defeats all OOP concepts throughout the QSE.
* The main reason having this system was that it would make multithreading easier - this just not true. Xna gives you basically an Update() and a Draw() per frame. The Draw() is practically impossible to split into multiple threads except for some very special tasks like culling - which can't utilize the messaging. Update() normally implies input processing, physics update and collision detection, although the last two tasks can be multitasked I don't see an easy way to utilize the QSE messaging system. Even if we could, the parallel extensions for .NET would be probably a much better choice. The only really useful background task would be loading content especially in a paging environment - again no use of the messaging system.
A small side note:
I converted one of my scenes to QSE; classical RPG scene, big terrain, lots of static outdoor objects (no vegetation yet), few dynamic meshes...The most common condition in games (RPG, FPS, RTS whatever) is that the user moves the mouse and has at least one key/button pressed, so the point here is that we have at least one mousemove and more often the mousemove + keypress per frame. QSE generates some internal messages per frame as well (like 'getheight'). So we end having 2 or easy 3-4 messages per frame. The invocation list of the message handling EngineMessageHandler delegate had about 3000 entries in its invocation list - so it had to invoke over 3000 functions per frame! Not to mention that the invoked functions (mostly handlers inside BaseEntity) have a function traversal chain with switch statements and enumerators, so in total we have 7-8K function invocations per frame per message! If we have say 3 messages it sums up to >20K function invocations per frame!

I suggest:
1) Don't generate automatically any messages at all. Have initially an empty delegate and let potential message consumers register them.
2) Don't let the BaseEntity handle any messages at all; instead let specialized entities which need some messages register a handler.
3) Optionally but not important, we could additionally divide all messages into categories and have one handler per category (input system, camera, etc)



Entity System and overall QSE parts interaction

* Some very specialized behavior is implemented in the very base class. The functionality there can really mess up the end user application e.g. if somewhere a message is generated which is consumed by the BaseEntity one could get unexpected behavior of its specialized entity which received this message.
I suggest: move the specialized implementation to specialized entities/components which really need it.

* The current Entity-Component System is not very intuitive: Everything is an entity except if its a component in which case it has to be attached to an entity.
* Its OK to have a basic 'glue' holding our system together but there must be clear definitions what other parts are or should do.
I suggest: rethink do we really need components? Do we have to attach components to entities in order to apply special behavior? Cant we have 'controlers' instead which have attached entities and control/change their behavior? Its also desirable to have some elements in the engine which are not entities like the camera which first crosses my mind.

* there are QueryForRenderChunks()which apparently do nothing all over the place although there are also few places where it is really used
* The same function is implemented in multiple classes without a common interface
* Therefore it is not clear how to approach Rendering, through a rendercomponent or simply by accessing the AllocateRenderChunk() of the GraphicsSystem. Just look at the mess inside BaseEntity/QuadTree/Terrain QueryForRenderChunks implementation to get the picture.
* Why do we have QueryForRenderChunks()at all if the implementation violently accesses the GraphicsSystem? So either drop the function or force updating GraphiscSystem internall data only through this function by making the data private and using some param or return value of the render chunks query.
* The standard Xna DrawableGameComponent pattern is not used so each component producing some visuals must send the vertices to the QSE graphics system which accumulates them and sends to GPU during Draw(). Although the RenderChunk is recycled from a 'pool' the vertex List holding the vertex buffers is recreated/cleared per chunk per frame!
This approach also makes extremely hard for a component to control anything except the vertices information (e.g. hardware/shader instancing, static vs dynamic buffers etc.)
I suggest: Unify the QeryForRenderChunks through an interface and pass a RenderChunk as a parameter or return value. Don't allow anyone outside the graphicssystem to access the renderchunks. That said one could see that in such a case we would again have the very same Xna's DrawableGameComponent pattern so maybe the best solution would be to revert to this?

* some parts of the engine are deciding instead of the user about certain application behavior and don’t even allow to change things. MouseHandler comes first to my mind. This class sets the mouse pointer to invisible and resets its position to 100, 100 every frame? What if I want to use a hardware cursor on screen? Is it hard to imagine I want let the end user (player) click on something on the screen, press a button, pick up a game asset, draw selection rectangles etc…???
I suggest: If it’s in the engine – it has to be (re)usable and it should be as generic as possible.
If it is something exclusive it must be parameterized or/and changeable. We have only one mouse and keyboard handler in the QSE so the user expects he could use them as they are and not only in one specialized way.


Input handling

I see two problems here:
1. The Xna framework already has build in access to Mouse, Keyboard and Gamepad inputs (not sure if Joysticks are interfaced through the GamePad class though). So why bother with the complicated input system when a user can pool the input state with something as simple as Keyboard.GetState().IsKeyDown(key)?
2. The whole QSE input system is user unfriendly:
* you first register a handler through the config Xml,
* you manually add input listeners (this.inputs.AddInputListener(MouseButton.Right);) etc. To do so one must have a central place for this code to make it maintainable
* If you miss adding a listener but query for it you will either get a runtime exception or not get the input message
* Because of the current QSE messaging system we get far to many function invocations per frame
* you need to override OnGameMessage() to process the inputs and know about the insides of the QSE entities and components since they can consume the very same messages and mess up your game, or one has to make workarounds like:
- process inputs inside the Update() where you would again pool the states like this.inputs.IsDown()- identically as with the pure Xna approach !
- Of course the risk of using same Messages which have implemented consumers inside the QSE still exists :-( so you have to override and never OnGame_Message and never call base.OnGame_Message(). Isn’t it funny that in order to get something done you absolutely must override the base class functionality even if your override is an empty function?

What the end user really needs is:
a) not to register InputListeners but to have automatically all 'listeners' running so he could simply pool the input states for any key/button needed.
b) Registering a delegate for specifc listeners would be nice (read as: connecting keybindings to function pointers).
c) A 'listener' should never ever generate and dispatch messages to the whole system, instead 'dispatch mesages' only to entities/components which requested them which is actually the same as under b)

I suggest:
1) Refactor and reduce the whole Input system so it Don't generate automatically any messages at all. Have initialy an empty delegate and let potentiall message consumers register them.
2) Don't let the BaseEntity handle any messages at all, instead let specialized entities which need some messages register a handler.
3) Optionally but not important, we could additionally divide all messages into categories and have one handler per category (input, camera, etc)

[/spoil]

As few days ago lordikon gave me access to the developers only forum part I found a lot of posts with complaints about the very same things I wrote above.
Since almost each of my topics has been already mentioned in one way or the other I think it would be reasonable to at least rethink about all those issues.
I am willing to invest a lot into this project even if it means I have to branch out into a totally different direction but I'd definitively like to work with the community instead "against" it.

My primary concern right now is the question is QSE dead? Next comes is there will enough to make it better?

P.S. I did a lot of changes locally and bug fixes there is a big one in the terrain entity - I can post it on demand

PostPosted: Fri Jan 09, 2009 6:26 pm
by lordikon
The project is as close to dead as you can get without being dead. I'm the only active developer, and I'm barely active because I work 60 hours/week at my job.

As to your document:
- Terrain quadtree nodes currently have their own QueryForRenderChunks because terrain hasn't been converted to the component system. I wasn't sure how to best do this, or if it would even be best to do it at all. If I followed the way the rest of the engine did it each Quadtree node would be its own entity with a render component. Having a bunch of entities like that seems to be a waste though.

- The input system doesn't require that you add input listeners unless you want to poll for keys. For example, if you want an action to occur anytime the right mouse button is clicked, then you would just listen for that message and you'd be done with it. However, for things like character/camera movement you will often care about the state of multiple different keys. For example, in WoW a character will run if both mouse buttons are held down, and to do this through messages you'd have to listen for left and right mouse buttons seperately and store some kind of bool that was set to true only if both buttons were pressed, which is a huge pain in the ass. Instead the input poller class will store any buttons that you have requested it listen for. You can then query these keys/buttons at any time. The reason an input poller class is used is to prevent input systems from requesting the state of specific keys through messages to the input system. This prevents systems from sending a message to the input system for every state of every key/button it cares about, when instead it can just request the info from the poller. Also, I don't remember ever registering handlers in the config.xml, although it has been awhile.

- BaseEntity already handles messages (simply movement messages), where would you suggest that these messages be handled?

- I agree with having message categories, or rather, categories of game systems, and messages get sent to these systems.

Sorry if I didn't address all questions, I'm pretty busy ATM.

PostPosted: Fri Jan 09, 2009 8:23 pm
by Riki
I am sorry to hear that, this is unbelievable L :beer: I just bailed out of the Irrlicht CP project after getting involved for a couple of years for the same reason - no activity.

Regarding the engine state I have the impression almost all effort was aimed towards bringing up & running the SampleScene project, and that's exactly the problem nothing else really works except the SampleScene project :-(
To be totally honest...after playing with Xna for the last four weeks I am not really sure if an graphics engine is really needed (a complete game engine sure but QSE is years away from being a game engine and I don't even think it should head that way until the graphics part is stable).

regarding your replies:
- Yes the terrain/quad/render system is a pretty mess, but as I just a few minutes before found out the design issues inside the rendering system are much bigger. I really don't think that this could be used for any except the most rudimentary demo projects. Materials are loaded, parsed and cached at init time with no way to change them during runtime, the user could at best influence the vertices being sent to the rendering system but nothing else...there are really lot of flaws with the current concept


- Input system: i told that you can not poll keys if you forgot to add listeners - the fact that I have to explicitly add them is that bothers me. e.g. I had listeners for keys and decided to pool the mouse button...runtime exception since no mouse listeners where defined. I would see a point into explicitly defining listeners if I could directly bind them with some actions (e.g. a keybinding system). BTW to get a handler you have to utilize the config:
<inputmanager>
<handler value="QuickStart.KeyboardHandler" />
<handler value="QuickStart.MouseHandler" />
<!--<handler value="QuickStart.GamePadHandler" />-->
</inputmanager>

- BaseEntity: IMHO one of the worst problems is that the BaseEntity handles messages! It's a Base for everything, why would everything in my game react to messages? Even worse, since it's in the base class everything would react the same way on the messages. No one needs this, it actually is undesired and the user has to override the base behavior. I know that this was not on purpose but rather historically evolved: there was an entity, someone created the camera, it was natural to give the camera an entity as parent, there was this messaging system so why not use it to control the camera...but wait ... since the camera is an entity we can implement movement, rotations and various stuff in the BaseEntity and have it in our camera for free. Then one day came the TerrainEntity ...just to bad to realize that if you issue a SetPosition message not only would the camera be placed on that position but the terrain as well. In fact all all objects in the scene deriving from BaseEntity (are there any others?) would be placed on the very same position :lol: I could even understand that the messages get passed through the entities just to have one common infrastructure but having an implementation inside the BaseEntity is a NO GO. I already wrote in the forum and in that document in more detail what would I suggest: No messages until someone explicitly requests for them, no message response implementation unless inside very specialized entities/components!

PostPosted: Tue Jan 13, 2009 9:41 pm
by lordikon
Input system - The reason you have to register a listener for specific buttons/keys is because it must retain the information for that button/key. There is no reason every poller should store all information for all keys and buttons. At that point you might as well be polling directly from XNA/XInput/DirectInput/etc...

Rendering system - It is largely unfinished and in progress. Progress was halted on it to get a physics wrapper for PhysX, and then we decided to add JigLibX so we could support the Xbox360. The content processors for models was going to be pretty extensible. But I didn't write the system so I know little about it.

We used to have Dynamic entity, which I had intended for base entities that would allow themselves to receive input for movement, but I was overruled so it went in BaseEntity.

PostPosted: Sat Apr 04, 2009 9:03 am
by Knight_Christian
Hi folks,

sorry but unfortunately I forgot about this place, sorry :(
But like lordikon, im working 40 hours the week and when I come home i don't really feel like coding again (because i'm coding all day at work).
Also, unfortunately this cool project lost attention, so it does seem to me.
And the reason why i stopped the editor, was that i wasn't really happy with it and soon i started working so i didn't have much time either, so thats the reason why the editor didn't make any progress.

but we could really have more developers who would work on the engine, that would help the engine getting onward.

regards,
chris