New skies opening up

I've spent the last couple of days updating the version of the sky system Caelum that Ember uses. We've been using an almost two year old version, and there's been some nice recent improvements to it. It turned out to be fairly easy to do, some classes and interfaces had changed, but all in all it was a straight forward update process. The end result is a better looking world, if I might say so myself. The sky model has been refined, and support for clouds and sky transitions is more smooth and robust. One interesting addition is that it now uses an astronomically correct earth model, where both earth position and absolute day must be specified. This opens up some nice possibilities for virtual worlds, since it then is possible to specify where on the earth sphere your world should be placed, and get a different lighting model. A world placed at the north pole would have a very different day-night cycle than one placed at the equator. Alternatively, the server shouldn't assume that we're on an earth sphere and could instead just specify direct values for night and day cycle, as well as the sun's trajectory over the sky. That would require us to create our own solar system model classes to be used by caelum, but it's certainly doable. But there's a certain simplicity to just letting the server specify the longitude and latitude that's quite alluring.
Another issue is how to present the world during night. In the current caelum implementation it just gets darker, until you can't pretty much see anything. This isn't very nice for players, and would probably force them to always manually set the time to noon to be able to see something. Instead we want to provide a world that's as clear during the night as it is during the day. WoW does this splendidly, and we need to adapt something similiar. But until then we'll use the current hack, where the time of day on the client always is adjusted so it's sometime around noon, even when it's in the middle of the night on the server.

Another thing that's I've committed is some updates to the PagedGeometry system where it now uses classes specifically tailored for Ember, which renders near objects as entities while switching to imposter rendering for far away ones. Previously we had to use the DummyPage class and use the Entity::renderDistance functionality for hiding far away entities, but now that's all smoothly taken care of by classes which work directly with the PagedGeometry engine. There are still some issues with both performance and updates that needs to be taken care of, but it works much better now.

On the Worldforge front, there's two new exciting developements which has unfolded the last couple of weeks. One is that we now have a new Ember developer, Sean Ryan, who's just starting out getting a feel of the Ember code base. He's already taken upon himself to implement event logging as an initial task which is an excellent way to start.
The other development is that we're applying for the Google Summer of Code this year. We've already set up a wiki page for this purpose and will gladly accept any applicants or project ideas. If you think this is something for you, or if you know someone that might be suited, please get in touch with us, preferrably by using the mailing lists.

The grass is greener

Work on implementing a better foliage system has continued, and I'm getting more and more pleased with the results. Here are some recent screen shots.
As with all other functionality in Ember I've strived to make it as general and modifiable as possible. It should be trivial to both change the behaviour and to extend it. The way it's implemented is through the use of the terrain layers as a base. This is how the xml definition for the grass terrain layer looks like (I've removed some texture definitions):

<?xml version='1.0' encoding='UTF-8'?>
<terrain>
 <layers>
  <layer shadername="grass">
   <foliage planttype="grass" populationtechnique="cluster" rendertechnique="grass">
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">10</param>
    <param key="clusterDistance">25</param>
    <param key="density">2</param>
    <param key="falloff">0.3</param>
    <param key="material">/global/plants/grass/bittergrass/single</param>
   </foliage>
   <foliage planttype="fern" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/shrubs/models/fern/fern.mesh</param>
    <param key="minScale">0.7</param>
    <param key="maxScale">2.0</param>
    
    <param key="minClusterRadius">1</param>
    <param key="maxClusterRadius">3</param>
    <param key="clusterDistance">30</param>
    <param key="density">2</param>
    <param key="falloff">0.7</param>
   </foliage>
   <foliage planttype="camelia" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/flowers/models/camellia.mesh</param>
    <param key="minScale">2</param>
    <param key="maxScale">5</param>
    
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">5</param>
    <param key="clusterDistance">40</param>
    <param key="density">1</param>
    <param key="falloff">0.7</param>
   </foliage>
   <foliage planttype="leaves" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/shrubs/models/leaf/leaves.mesh</param>
    <param key="minScale">0.5</param>
    <param key="maxScale">1.2</param>
    
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">5</param>
    <param key="clusterDistance">35</param>
    <param key="density">2</param>
    <param key="falloff">0.7</param>
   </foliage>
  </layer>
 </layers>
</terrain>

Every foliage type has both a population technique and a render technique. The first one determines where to put the vegetation and the latter determines how it should be rendered. Both of these share the same set of arbitrary parameters. We currently only support the cluster population technique. This creates a series of clusters of varying sizes randomly dispersed over the map. In most cases this looks quite nice, since plants tends to grow in clusters in the nature. All of the plants positions are calculated when the terrain is loaded, and not changed thereafter. However, every time a plant is placed we check all the layers that are above the current layer. If the combined value of all the layers above are above a threshold value we don't show the plant. This has the effect that roads, fields and other areas don't get vegetation, unless there's foliage defined for that particular layer. And since we've already determined the positions for the plants we can update foliage swaths as areas are added without having the positions of existing plants getting jumbled each time.

Regarding the render techniques we currently support two different: grass and shrubbery. The former autogenerates grass meshes, and animates it whereas the latter uses preexisting meshes. So far the latter haven't got support for animations, but that's something that we'll surely add.

The population techniques are implemented in Ember so far, but the idea is to put them in Mercator as they stabilize. That way it should be possible to specify foliage on the client, very much like we now specify Mercator shaders.

All in all it's looking mighty promising, even though as can be seen in the screen shots we need to fix up the colours a little.

Impostering

I've now finally implemented the Paged Geometry engine into Ember. This is an component which allows for much more nicer looking foliage through a variety of different techniques. While we've had grass through the use of batched geometry for a while, the implementation wasn't really optimal and it didn't contain the more advanced features needed. The main thing lacking has always been a proper implementation of imposters, which are 2d billboards used in the distance. Instead of burdening the GPU with full geometry for far away trees, which in the end will end up just taking up a few pixels on screen, we render the tree once to a 2d sprite and reuse that. The end result is an enormous speed up for forests scenes, where there can be thousands of trees on the screen at any one time.
All of this is provided by the Paged Geometry engine, which is why I'm very excited to finally have it in place. So far I've partially converted the grass system to use the new engine. It's currently lacking some of the filtering features, which makes the grass appear on paths and other areas where there should be no grass. It's also just randomly dispersed, which makes it look a little bit unnatural. It would preferably be dispersed in a more cluster like fashion, to better simulate the way grass grows in the real world. All of this is currently being addressed however.
The trees now use billboards when viewed from afar. I've extended the Model format to take an additional "rendering scheme" subsection. This allows one to specify different rendering schemes for different models. Thus we can tell Ember to use the rendering scheme "forest" for our trees, which will set up everything and make them switch to imposters at far away distances, while keeping the regular entity rendering at close distance. The Paged Geometry engine allows us to use batched geometry up close, which means that the different entities are "baked" into one large mesh. However, due to the dynamic nature of Ember and the Worldforge world that's not feasible. Thus we need to render the entities normally up close. The current solution is to do that using the regular rendering system and then set max render distance to the same distance as that when imposters take over. But since the imposters work in groups the distance where a certain entity begins to be rendered by imposters isn't exact. This means there are instances where this leads to either both the imposter and the entity being rendered at the same time (not a big problem) or that none gets rendered (a bigger problem). I therefore need to extend the Page Geometry engine with some Ember specific classes which will allow it to smoothly transition between normal entity rendering and imposter rendering.
I've already made some refactoring changes to the Page Geometry, which I intend to try to push back into the main development branch.
I've put up some more screen shots of the imposters and grass over at amber.

On a different note, there have recently been some issue with the Worldforge infrastructure. I've been trying to contact Balinor about the media svn for some time, but have never gotten any response. Last week however the svn server completely broke down: when trying to update I get a message about an internal error. A quick google shows that it stems from the server running out of disc space. This is bad, since not only can no one commit to the repository, no one can even update their local copy against it. I therefore sent some emails to the mailing lists, only to find that they also have been broken. Apparently there was a purge of all the old inactive lists last week, something sorely needed. Instead of having 15+ different lists, we now only have 4, which hopefully will mean that messages won't be lost. Unfortunately this changes seems to have broken the lists, since none of my mails got to the lists. I've been trying to contact the people responsible this weekend to no avail, but hopefully it will be fixed soon.