The grass is greener

Work on implementing a better foliage system has continued, and I'm getting more and more pleased with the results. Here are some recent screen shots.
As with all other functionality in Ember I've strived to make it as general and modifiable as possible. It should be trivial to both change the behaviour and to extend it. The way it's implemented is through the use of the terrain layers as a base. This is how the xml definition for the grass terrain layer looks like (I've removed some texture definitions):

<?xml version='1.0' encoding='UTF-8'?>
<terrain>
 <layers>
  <layer shadername="grass">
   <foliage planttype="grass" populationtechnique="cluster" rendertechnique="grass">
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">10</param>
    <param key="clusterDistance">25</param>
    <param key="density">2</param>
    <param key="falloff">0.3</param>
    <param key="material">/global/plants/grass/bittergrass/single</param>
   </foliage>
   <foliage planttype="fern" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/shrubs/models/fern/fern.mesh</param>
    <param key="minScale">0.7</param>
    <param key="maxScale">2.0</param>
    
    <param key="minClusterRadius">1</param>
    <param key="maxClusterRadius">3</param>
    <param key="clusterDistance">30</param>
    <param key="density">2</param>
    <param key="falloff">0.7</param>
   </foliage>
   <foliage planttype="camelia" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/flowers/models/camellia.mesh</param>
    <param key="minScale">2</param>
    <param key="maxScale">5</param>
    
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">5</param>
    <param key="clusterDistance">40</param>
    <param key="density">1</param>
    <param key="falloff">0.7</param>
   </foliage>
   <foliage planttype="leaves" populationtechnique="cluster" rendertechnique="shrubbery">
    <param key="mesh">3d_objects/plants/shrubs/models/leaf/leaves.mesh</param>
    <param key="minScale">0.5</param>
    <param key="maxScale">1.2</param>
    
    <param key="minClusterRadius">2</param>
    <param key="maxClusterRadius">5</param>
    <param key="clusterDistance">35</param>
    <param key="density">2</param>
    <param key="falloff">0.7</param>
   </foliage>
  </layer>
 </layers>
</terrain>

Every foliage type has both a population technique and a render technique. The first one determines where to put the vegetation and the latter determines how it should be rendered. Both of these share the same set of arbitrary parameters. We currently only support the cluster population technique. This creates a series of clusters of varying sizes randomly dispersed over the map. In most cases this looks quite nice, since plants tends to grow in clusters in the nature. All of the plants positions are calculated when the terrain is loaded, and not changed thereafter. However, every time a plant is placed we check all the layers that are above the current layer. If the combined value of all the layers above are above a threshold value we don't show the plant. This has the effect that roads, fields and other areas don't get vegetation, unless there's foliage defined for that particular layer. And since we've already determined the positions for the plants we can update foliage swaths as areas are added without having the positions of existing plants getting jumbled each time.

Regarding the render techniques we currently support two different: grass and shrubbery. The former autogenerates grass meshes, and animates it whereas the latter uses preexisting meshes. So far the latter haven't got support for animations, but that's something that we'll surely add.

The population techniques are implemented in Ember so far, but the idea is to put them in Mercator as they stabilize. That way it should be possible to specify foliage on the client, very much like we now specify Mercator shaders.

All in all it's looking mighty promising, even though as can be seen in the screen shots we need to fix up the colours a little.

Impostering

I've now finally implemented the Paged Geometry engine into Ember. This is an component which allows for much more nicer looking foliage through a variety of different techniques. While we've had grass through the use of batched geometry for a while, the implementation wasn't really optimal and it didn't contain the more advanced features needed. The main thing lacking has always been a proper implementation of imposters, which are 2d billboards used in the distance. Instead of burdening the GPU with full geometry for far away trees, which in the end will end up just taking up a few pixels on screen, we render the tree once to a 2d sprite and reuse that. The end result is an enormous speed up for forests scenes, where there can be thousands of trees on the screen at any one time.
All of this is provided by the Paged Geometry engine, which is why I'm very excited to finally have it in place. So far I've partially converted the grass system to use the new engine. It's currently lacking some of the filtering features, which makes the grass appear on paths and other areas where there should be no grass. It's also just randomly dispersed, which makes it look a little bit unnatural. It would preferably be dispersed in a more cluster like fashion, to better simulate the way grass grows in the real world. All of this is currently being addressed however.
The trees now use billboards when viewed from afar. I've extended the Model format to take an additional "rendering scheme" subsection. This allows one to specify different rendering schemes for different models. Thus we can tell Ember to use the rendering scheme "forest" for our trees, which will set up everything and make them switch to imposters at far away distances, while keeping the regular entity rendering at close distance. The Paged Geometry engine allows us to use batched geometry up close, which means that the different entities are "baked" into one large mesh. However, due to the dynamic nature of Ember and the Worldforge world that's not feasible. Thus we need to render the entities normally up close. The current solution is to do that using the regular rendering system and then set max render distance to the same distance as that when imposters take over. But since the imposters work in groups the distance where a certain entity begins to be rendered by imposters isn't exact. This means there are instances where this leads to either both the imposter and the entity being rendered at the same time (not a big problem) or that none gets rendered (a bigger problem). I therefore need to extend the Page Geometry engine with some Ember specific classes which will allow it to smoothly transition between normal entity rendering and imposter rendering.
I've already made some refactoring changes to the Page Geometry, which I intend to try to push back into the main development branch.
I've put up some more screen shots of the imposters and grass over at amber.

On a different note, there have recently been some issue with the Worldforge infrastructure. I've been trying to contact Balinor about the media svn for some time, but have never gotten any response. Last week however the svn server completely broke down: when trying to update I get a message about an internal error. A quick google shows that it stems from the server running out of disc space. This is bad, since not only can no one commit to the repository, no one can even update their local copy against it. I therefore sent some emails to the mailing lists, only to find that they also have been broken. Apparently there was a purge of all the old inactive lists last week, something sorely needed. Instead of having 15+ different lists, we now only have 4, which hopefully will mean that messages won't be lost. Unfortunately this changes seems to have broken the lists, since none of my mails got to the lists. I've been trying to contact the people responsible this weekend to no avail, but hopefully it will be fixed soon.