Terrain shaping up

The last couple of days I've been toying with a new terrain. I'm thinking of an island with some far away mountains and some harbours. I've put up some preliminary screen shots here. With the new terrain editing features it's really easy to quickly create some nice looking terrain. I should really record some movies of it...
When working on that I found a couple of problems in Ember though. The first was that it used massive amounts of memory. A prime source for this seemed to be the fact that it created a lot of both bitmaps and textures for all of the terrain, which it then never really used, but had lying around anyway. That prompted a refactoring of how the terrain worked which removed most of these instances. While doing that I also realized that we're quite wasteful in how we use full 8 bit bitmaps for all of the layers in the terrain, even if one layer might be the snow, where the 256 steps allowed by 8 bits never are used. To improve this we should combine many layers into one bitmap, where perhaps the grass layer is allocated 64 values, while the snow only is allocated 8 values, and so on. This would only work with shaders though.
After refactoring the terrain texture system I found that it still gobbled up memory, but I haven't run any good benchmark yet. I'm however guessing that it's the Mercator system which is initialized in full when the world is loaded, and then never has it's data removed. Preferrably, we should only load the Mercator segments on demand, and we should only keep those most used around and unload the others (perhaps by keeping a time stamped cache). It should work very much like the Ogre PagingLandscape system works.

I also quite alarmingly found that Ember currently is leaking memory. Just by starting it up and having it running without connecting to any server makes it allocate 2-6Kb/s. Something is seriously wrong here. I need to hunt that down and fix it.

When searching for the cause of these things I did some alterations to the logging system, making it not shut it down so early in the shutdown process. I also cleaned up the headers and moved some classes to separate headers. The logging service is included in pretty much every other file in Ember, so making it slim is good for compilation performance. Unfortunately the changes I did to it will force an almost full recompile after updating...

I also ran into a really strange problem with mouse picking in the terrain when testing out my new island. The picking is simply off. It's however working as it should the Mason world, which makes me think that it's either something to do with the fact the the island world is much larger, or that it's in a non-square form. I'll have to investigate it further.

This week also marks the start of the coding period for the students, which is really exciting. I'm looking forward to see how they'll evolve as they more and more into the WF system.

Shaping worlds

For the last couple of days I've worked on the tools for shaping the world. As my goal for 0.5.3 is to have a better world which shows off all the new media the first step is to alter the world to allow for more expansive forests and far away mountains. There's been a terrain editor in Ember for some time, but it has been very rudimentary and lacked some features which would make it easier to alter the world. One of these features was the ability to alter more than one terrain point at once. I've now added the ability to set a "radius" in the editor. That will alter all terrain points around the one that the user interacts with. This is very helpful since it allows us to quickly raise and lower large swaths of land.

When I started fiddling with that I found a couple of bugs in the terrain code which prevented it from working as smooth as I wanted. As I've squashed those bugs the editing is becoming much more smoother. I also found that the way the terrain is reloaded is very inefficient. Currently we unload the whole terrain page and then load it again. However, all that we change is just the height data, so it would be much more efficient to tell the engine just to recalculate the affected meshes from updated data. I've done some changes to the code which allows for this much more efficient approach, and it's beginning to look good. There's still a problem with the LOD calculation not being correctly updated, but that shouldn't be too hard to fix. All in all this will allow for the ability to quickly create a whole new world from a blank slate of terrain. Very exciting stuff.

Building worlds

The start of the Summer of Code is getting closer and closer and our three students have already started getting involved in the community. They've all got their git repos set up, and Alexey which has planned some downtime during the summer has already started committing code.
There's also two pages on the wiki about two of the projects. All in all it looks very promising.

I sat down and thought about what I wanted to see in the next release. It didn't take long to realize that the major focus must be on making sure that all the new media that Jayr has created must better be shown within the game. Just by looking through the commits one can see that he's put a lot of work into the media repository, and it's a real shame that not much of it is shown in the game. This mainly has to do with the fact that there's either no corresponding entity types on the server, or that they aren't used in the default world.
In addition, with Ember 0.5.2 we now have support for imposters, which means that we can display much more entities on the screen at once. This is mainly of use when rending foliage and forests, of which we would like to have much more. Jayr has created a couple of really nice birch trees which can be seen here:
He's also added some bushes as well as a whole collection of new nice meshes and textures. So the focus for Ember 0.5.3 is to update the world to make sure that all of the new media is shown. All other blueprints will be put on hold to favour the push. The first thing we need to do is to update the default terrain. Ember can handle quite large terrains without problems, but have a harder time with really steep cliffs, due to LOD issues. This is pretty much the opposite the current layout of the world. We therefore need to expand the world, make it larger and with more distant mountains. Making it larger will also allow for more expansive forests. In addition, we have quite a lot of meshes for both a village and a castle, so we need to provide space for that as well, just as we also need to make sure that the village and castle look good and are naturally laid out. There's already some maps of a proposed world, as seen here, but the current world isn't following it, and it presents a major issue in that it's hard to do world boundaries, where the world simply ends, without using water. In the current setup there's a couple of very large mountains to two sides, and water on two other sides. I'm thinking of going with an island completely, just because it's so much easier then to have a natural world boundary. There's nothing that says the world needs to be in "Moraf" either.

The first step towards a new world is creating the terrain, so the last couple of days I've been focusing on getting better terrain editing functionality into Ember, as well as finding and fixing some bugs related to the real time terrain editing. We now have the ability to alter many base points at once, allowing us to much quicker and easier alter large expanses of the terrain. The next couple of days I plan on experimenting with a new world. Once that's done we should start on making sure that all meshes and textures are represented by ingame entity types. There's now a nifty export functionality in the Model Editor which will automatically create and export an Atlas definition from a pre-existing mesh. This has already proven to be very useful in quickly generate atlas types for missing media.

Last week I also moved all Ogre material definitions from the Ember git repo to the media svn repo, where they much better belong. The idea here is that they should be used as reference when working with the media, so that it's possible to use the exact same materials both when authoring media and when showing it in Ember. We're not there yet however as we also need a good mesh viewer application that can work with the way the media repository is set up.