Major refactoring

I've just pushed a major refactoring I've been working on the last couple of weeks to the master repo. In the end it turned out really well.
The main part of the refactoring was to totally revamp the whole entity hierarchy in Ember, and how the entities are tied to graphical Ogre representations (meshes, lights, particle systems etc.). As with most refactorization, I had made some assumptions about how things worked a couple of years back which didn't hold anymore, and the code was beginning to creak from workarounds and special cases.
One of the assumptions made was that all entities in the entity hierarchy tree would be represented as an Ogre scene node. Another was that all graphical entities would be shown using the Model class. As one of our GSoC students, Amey, began to implement his task and needed some client extension for this it became apparent that these assumptions do not always hold. It's not always certain that an entity will be arranged in the Ogre scene graph through the use of a scene node. The most common instance is when something is attached to a skeleton (like a person wielding something). In addition, there are many kinds of entities which need graphical representation, but not through the use of a Model. In Amey's case it was the ocean.
Furthermore, when I first made the entity class system I used inheritance, so that each different entity would be of either just a normal EmberEntity class, or a PhysicalEmberEntity class, depending on whether it had a graphical representation in the world. However, as I work more and more with software I've come to realize some of the issues with depending too much on inheritance. Instead I now more often find myself component based design. In such a system the behaviour of instances are mainly determined by the interactions between the components that make up them. This of course leads to more complex systems, where it's a bit harder for a newcomer to get the full picture of how any given part should work. But it also leads to much more powerful systems, without many of the incorrect inheritances one tend to end up with when over using inheritance.

All this lead to me totally redesigning the entity class system. There's now only one entity class (well, except for the world entity, but that is a truly unique instance). The way something is shown is now handled completely separate in a graphical component, owned by the entity. And the way it's arranged in the Ogre scene graph is also handled by another distinct component. The end result is that there's now a much more robust system in place which more correctly maps how things really are arranged. This means that we now have a much more cleaner design, which is also extensible enough to handle more advanced cases than the previous system could. I've already pushed code which will connect an ocean entity to a water representation, something which in the old system would have required all kinds of hacks.

ATI graphics fixes

The last month saw both the release of Ember 0.5.6 and the start of the Google Summer of Code.
The Ember release included the really nice addition of real time shadows, thanks to the work done by Alexey. It also included some really nifty area editing tools for world authors, as well as the usual slew of polish and bug fixes.
After the release was done Alexey and me did some additional work on the shadow system, to fix some of the bugs and artifacts that existed in the release. Amongs them was some issues with shadow texture fidelity at certain distances and a bug where lights that were created in the world never disappeared. Another issue we found was that lights which were attached to entities which a character was wielding did't get updated as the character moved. The prime example here was the torch, which in turn had a "fire" child entity.
All of these issues has now however been fixed in git master.
I've also added a new nifty snap-to-move feature. It's activated when you move an entity in the world, and allows you to easily make it snap to another entity. The snapping is done by matching the bounding boxes of the entities. This is really useful when building fences or walls, since each wall section then can snap into the other.

However, the bulk of my time the last couple of weeks has been spent on graphic glitches, mainly those found on ATI cards. We've known for a little while that there are a couple of artifacts on ATI cards, but since none of the main developers use ATI cards (we use Nvidia, since at least historically their Linux support has been orders of magnitudes better than ATI) we haven't really known how bad these artifacts are.
After upgrading to Mandriva 2009.1 I began to run into X server segfaults when running OpenGL applications. I therefore had to abandon my Nvidia card until the issue has been fixed, and it just so happened that I was able to borrow an ATI cards from work.
Turns out there were all kinds of issues with ATI cards, amongs them the issue with random segfaults within the OpenGL driver. Just this issue had me spend more than a week trying to track down. Intially it wasn't clear that the crashes were random, since they seemed to occur near certain materials. However, after disabling more and more features it became apparent that they were indeed completely random. Thanks to cdleonard, developer of the Caelum environment system, we concluded that the issue was that using hardware generated mipmapping on ATI cards, as supplied by the SGIS_generate_mipmap extension, resulted in random memory corruption withing the (closed source) GL stack. Yep, pretty much the worst possible kind of bug to track down. Once we did track it down it down and disabled the crashes went away.
There are however still some artifacts on ATI cards. The paged geometry has some issues, since it in places mixes vertex shaders with fixed function fragment processing. And the high detail glsl shaders doesn't produce any output, so when running Ember in high detail level all meshes are hidden when viewed up close. But all of these issues are easily fixable and it shouldn't take long until we can do a new release, which will work much better on ATI cards.

The Summer of Code is also progressing nicely. Manuel reports on the progress over at the wiki. We're now mainly in the discovery stage, where we're identifying where in the code base it would be suitable to add threading support. Nonetheless Manuel has already submitted a series of patches for various code cleanup issues which I've merged into Ember trunk. Git is truly a wonderful tool!

Realtime correct shadows

Finally, Ember has complete real time shadows. It took some time, and it wasn't until Alexey some months ago dove into it that we finally got a working implementation. But not only did he add shadows to the world, he also added a complete graphics level framework, where Ember will itself detect the kind of graphics card the user has and tune the graphics accordingly. This means that if you have low end card you'll still be able to run the client at playable frame rates, while if you have a newer card you'll also get real time shadows and other nice effects. One of the reasons we couldn't do this before was that it wasn't until Ogre 1.6 that we got easy access to the techniques needed to do shadows on the larger scale that our world presents (it's a bit harder to do outdoor shadows than shadows in an enclosed space).
Alexey did most of the work, and I got in at the end and fixed some issues with how the shadow system differentiated between semi-transparent and opaque materials. It turned out that Ogre doesn't have functionality where you can specify a template shadow material per technique (for example one shadow material for the "High" technique and another one for the "Medium" technique) and then have Ogre automatically supply all of the material settings from the regular material (alpha settings, textures to use, etc.). It can do that if you specify a scene manager general shadow material, but if you specify a shadow material per technique it expects you to provide all of the information needed in that material. Given the amount of materials we use that's not a viable solution; it's much more easier if we used a material template which would for each shadow pass get the correct data set. Thanks to the modular design of Ogre this wasn't hard to do, as can be seen in this commit. The result is both working shadows and a material setup that's both simple and flexible.

I've also put some work into improving the way the minimap for the compass is rendered. Previously it used the same lightning as the regular world, which mean that it was completely dark during the night. That's been fixed now, so it's always rendering it with a nice midday light. I also made it use a simpler material. This fixes some stuttering which presented itself when you moved rapidly through the world, as well as fixed the issue with ugly shadows appearing on the minimap.

A demonstration of the correct shadows, the runtime graphical settings changes and the new minimap lightning is shown here.

And a higher quality version is to be found here.