Memory usage

My development machine is quite beefy and has 4Gb of memory, so I seldom run the risk of running out of memory. The downside though is that I've been quite lax with regards to memory consumtion, and as a result Ember 0.5.2 contains both some memory leaks as well as uses too much memory in the way it preloads all available resources independent of whether they are used or not. The resource preloading hasn't been a problem previously, but since Jayr has been adding so much more cool media, with normal and specular maps, the memory usage for all media is now quite high.
I've therefore the last week gone through the codebase with memory profiling tools to try to find both unnecessary memory usage as well as memory leaks. The main tool I've used is Valgrind, which is really an excellent tool for all kinds of profiling. KDevelop comes with built in support for the memcheck tool, which finds memory leaks. It will keep tabs on all memory that is allocated and find those places where there's no more references left to unfreed memory. The valgrind massif tool is also very handy. It will track the heap of the application, and with regular interval report how large it is, and from where different sections of the heap has been allocated. The massif tool by default outputs its info in the console, which isn't very easy to navigate. I've however found an excellent tool for displaying the data in a much better graphical view over here.
Using these tools I found an issue with how the terrain splatting materials were generated, which in normal use amounted to a memory leak of ~100Mb. Quite a lot of data. Fixing it was trivial.
I also found some issues with how Ember was shut down, and how the main scene manager failed to be deleted. While this doesn't amount to any in game memory leak (the application is shutting down anyway, so unfreed memory will be freed a second later when the process exits) it did clutter the valgring output, and we should of course always strive to have components that correctly handles the allocated resources.
I'm now trying to find the cause of a problem I have with ModelDefinitions not being correctly deleted. They use the Ogre SharedPtr, which makes it possible for them to keep on living a little longer, even though the ModelDefinitionManager has been deleted, as long as something else is holding on to a reference. I'll have to investigate further.

I've also removed the preloading of all assets. This will make sure that only those assets that are needed are loaded, but the main (and quite big) drawback will be that the game will stutter as the player enters and moves through the world. This is because Ember so far is single threaded, and the main thread will have to wait while new assets are loaded as they are encountered. We're planning to move to a multithreaded resource loading solution, but we haven't started on that blueprint yet.

Much of this work is a result of the work I've done in trying to create a new island world. Since the island world is quite larger than the current world the memory usage issue became much more obvious. The plan is to release 0.5.3 with the new world. However, I'm leaning towards perhaps trying to get a 0.5.3 release out without the new world. One of many reasons is that Ogre 1.4.8 which was released after the Ember 0.5.2 release changes how resources with absolute paths are loaded. This clashes with how we load our terrain config file, and the result is that while Ember 0.5.2 compiles without problem with Ogre 1.4.8 it won't be able to run. We've already had some issues with people encountering this problem, and we'll sure have more and more. In addition, Nvidia releases a new driver version not long ago which contains some bugs which will either hang the application at startup or produce messed up mipmaps when used with Ogre 1.4.8 and lower. As a response Ogre therefore did a 1.4.9 release yesterday which contains fixes for these issues. As more and more people upgrade to the newer nvidia drivers we're sure to see more of this issue. I'm therefore thinking about trying to push a 0.5.3 release out as soon as possible. Before I do that I would however very much like to see all major memory issues resolved.
There's currently some kind of memory leak which leaks ~4kb/s when just running Ember, without even being connected to a server. That's not ok.