Christmas with the Ember

The last couple of weeks have been really hectic, as it's always before the end of the year. With not much free time work on Ember has suffered, but I've now finally gotten a stable release of 0.5.1, just in time for the holidays. This release adds a lot of new features, chief amongst them a much nicer interface with inventory and status widgets. As always there's a lot of bug fixes and improvements in almost all areas of Ember. Here's as small list of some of the improvements:

• totally refactored ingame chat system with better performance and functionality
• support for both Lua 5.0 and 5.1
• a new icon system
• a new inventory system with drag and drop equipment
• status widgets for both the avatar and current targets
• update of all materials, with support for normal and shader maps for most entities
• refactored terrain system with support for both shader and fixed pipeline functionality
  

After the 0.5.0 release earlier this year I wanted to do a 0.5.1 release pretty close to it, but for various reasons it didn't happen, and the release comes almost a full haft year after the previous one. The main reason is that I want to release really stable code without any regressions and strange graphic glitches. The problem with this is however that it's really hard to test out all of the various graphical cards when I'm the only one that works on the codebase, and I only really have one development box with a Nvidia card. Previously this has led me to disable the shader terrain rendering, since I found out at release time that it didn't work with ATI cards.
This can't work in the long run, and I'm looking at trying to better embrace the "release early, release often" motto, even if it means that some releases will bring regressions with them. Since I'm the only developer I'm really dependent on good bug reporting from end users. Lately I've been seeing some increased activity on the Launchpad bug tracker, which is really great, since it's just the kind of input I need from the users.
So I will try to get a release of 0.5.2 out not too soon, even if that release might break something. I'm also looking at perhaps using the Google BreakPad for even better crash reporting.

The new release of Ember also uses a lot of Jayr's new materials with support for normal and specular maps for most surfaces. This looks really good and hold great promises for the future as I move to include a better foliage rendering mechanism into Ember (after all, the mason world is an outdoors world, so natural foliage rendering seems almost like a requirement). It also brings the used media up to 90~ Mb.

With future versions of Ember we need to refactor the media system so that world developers can specify their own media repositories in a better way. Currently Ember gets all its media from one place and places it statically in ~/.ember/ember-media-0.5.1. However, we would like to let each world provide its own media if so desired, so that when you connect to one server you might download (through wfut) media specific to just that server.

For this release I haven't had time to set up a Windows build environment and provide a working windows binary. If anyone else feel better suited for this task I would greatly appreciate it, else I will do this myself after the holidays.

In the shadows

The new terrain rendering system is working really nice and I've now fully integrated a shader based terrain material. The idea is to optimize the material by trying to reduce the number of passes needed. By combining the different coverage layers into one single texture we can keep the number of passes down. On my card with 16 texture units I can squeeze in 12 splatting layers in one pass.

Now that I've got that part done I'm working on getting some more advanced features such as normal mapped terrain working. There are a couple of performance issues to take into account though. There's no reason to render the terrain with normal mapping on when it's in the distance. I'm thinking of adding some features which will allow the engine to automatically create pre-baked textures for the distance, where all of the layers already are put together without the need for splatting.

On the media front, Jayr continues to bring out really nice media. He's reworked the two room house, the gallows, the jetty and the stalls. Most of these are already brought into Ember, with full support for normal and specular mapping.

Shadows

I've finally started redesigning the terrain system, with some very promising result. The first issue to tackle was how the shadows on the terrain were computed. Previously I left all of that to the scene manager, which used a simple vertex lit approach which didn't look good with lower LOD (and also taxed the GFX card as the shadows had to be recalculated each frame).
I now have a simple shadow generation framework in place. The idea is to provide a couple of pluggable shadow providers for different shadow techniques. Currently I only provide a very simple technique where the shadows are calculated directly from the surface normals (just as vertex lightning). It already looks much better than before though, although I intend to provide some more advanced techniques later on.
Here are some example screenshots:
I've also started on a more thorough refactoring of how the whole terrain system works. The goal is to more easily allow for more advanced features, such as normal mapped terrain, as well as being able to alter the terrain materials in a simple gui ingame. As with all refactoring the code is currently a mess, but yesterday at least I got it to compile. The blueprint for this can be found here and I've so far followed it pretty much to the letter. The material compiler works, though it's very inefficient in the way it creates new materials.

Jayr have been churning out even more great looking media with a lot of new normal maps. It's a real joy to update the material definitions and see how the world comes to life when normal mapped materials are used. It's really such a big change in how everything gets depth and shadows.

Paths

Jayr have been adding more and more excellent media as well as overhauled the media repository layout. He's been moving source files into an "art" directory, so that the "3d_objects" and "3d_skeletons" directories only contains exported formats. This is great since it means that client developers that only want to get access to the used media, without being burdened with having to download a lot of high res source media only need to check out the two latter directories.

With the new media in Ember it becomes more and more apparent that I really need to spruce up the terrain rendering code. As can be seen in this screenshot the terrain really looks awful in the distance when the LOD kicks in.
This is because it's using vertex shading, where the lightning is set by interpolating the normals of the surrounding vertices. As the number of vertices change with the LOD the lightning becomes messed up. All from the beginning the idea has been that this is a purely temporal solution and it should be replaced with a more robust lightning solution. I've just never had time to implement a better model and thus it's been this way for some time. But with jayr's great looking media it becomes more and more jarring.
I've thus started looking into how to better this. There are a couple of different techniques that can be used, with shaders, normal maps and all. Currently I'm experimenting with just creating a one channel shadow texture from the terrain normals I can get from Mercator (compared to the direction of the sun). This won't take into account occlusion though since no ray checking is done, but hopefully it would look good enough (it would actually look pretty much like normal vertex lightning). We want to update the shadow dynamically as the sun moves over the sky, so we can't use too computing intensive techniques.
With some upscaling and perhaps some gaussian blur I think the result will be good enough.

Another pet peeve of mine that's been bugging me for some time was a bug with the foliage code where it didn't take into account "negative" ares, such as the path and the sty, where there should be no foliage. The result was that foliage was added everywhere, even in places where there should be no grass growing. I've been trying to fix it for a while but couldn't find what was wrong. I finally tracked down the problem to a en error in what lookup container for terrain surfaces was used. So now we finally have clear paths! This together with the work I'm doing on nicer terrain shadows should allow for some much nicer visuals in the near future.

Normalize!

The last couple of weeks Jayr have been providing really great new textures. The biggest improvement is that he's created both diffuse, normal and specular versions. This together with a some nice shaders allows us to get some really nice visuals in Ember. There's already other media in Ember that uses normal mapping, such as the trees, but we now have a much wider variety of textures, as well as a better naming system. This together with improvements to Ogre's material framework, chief amongst them inheritance, allows us to much easier implement and handle the normal mapping.
Here are some example screenshots:


These show the new fern model (the colour is a bit off against the ground, that will be adjusted) as well as the new normal mapped texture for the tinderbox.

Getting the fern out in the world highlights the problem with the ugly lightning on the terrain. The problem is that I'm currently using vertex lightning. This has some disadvantages in that it won't work with the level of detail system, since far away terrain have fewer vertices, thus screwing up the lightning. There also seems to be a problem with the normals, such as terrain that shouldn't be in shadow is in shadow and so on. All and all it's not a good way of doing terrain lighting. The reason however that I haven't switched to a better model (such as baked texture lightmaps) is that I first need to refactor the terrain generation framework. The current framework works, but it's hard to do things like add different materials depending on whether you have shader support or not. We need a better framework. I've created a blueprint for this. Once this is done we can start working on some much better terrain. I'm pondering whether to actually put this on top of my things to do, since getting a nicer terrain will make the visuals tremendously better (since the major part of the screen is covered in terrain).

You will also see the new status widget in the upper right. He's actually also provided normal maps, though I'm not sure how to use them yet. :)

Missing files

I had some people try to build and run the Ember source by themselves. It turned out that apart from some missing media files (such as the whole gui texture file) some of the xml files for CEGUI wasn't conforming to the schema specification. The reason I haven't noticed this before is that CEGUI can use many different backends for parsing xml. It can use either expat, libxml, xerces or tinymxl. Very nice component design there and all. But I had always compiled it with tinyxml, which as the name implies is very tiny and don't have any validation support. All incorrect tags are just read as they are. However, if CEGUI was compiled with Xerces instead, which is quite large and has schema validation, the loading would fail since Xerces would throw an exception when encountering invalid xml. And this is precisely what happened when other people tried building Ember, since they had CEGUI compiled with Xerces support. I've now fixed the issue and committed. I have to do a 0.5.1rc2 release soon though, since the modeldef format has changed, disallowing the use of the 0.5.0 media with cvs ember.

On the icon front I've gotten even further with the drag-n-drop functionality. There are some small usability bugs still left, but overall I think it's quite usable. I've been starting to think of how to improve the entity context menu. Currently it's just a regular menu that pops up, but I'm thinking of having a radial menu with icons instead.

Dolls

I've further expanded on the icon code, and in the process added more and better lua connections. I'm getting better and better at writing lua code. As with most new languages, it's not hard to learn, but it takes some while to start using the advantages of the language. Usually you tend to start out writing code in the same way that you would write C++ code, and then gradually replacing that with code written in the "Lua way".

Yesterday I added a first prototype for a user doll, where it should be possible to see what items currently are equipped. By dragging and dropping items on certain slots it should be possible to change equipment. It worked out fine. The icon system contains two elements: slots and icons. Both are handled by the EntityIconManager. The script asks the manager for some slots, puts the slots above the image of the avatar and attached some scripts to them. Then later on when an icon is dragged from the inventory and dropped on the slot a signal is emitted, which triggers a script. This script calls the necessary server side methods for wielding and changing the outfit, and voila! the item has been equipped. It's very elegant so far and should be easily expandable.

When dropping items, instead of pressing the "drop" button, the icon should be dragged and dropped on the world, at what point it should appear in the world as a 3d element. Similarly, when trading the user should only need to drag and drop items from the inventory into a "trade" box.

The EntityIconManager takes care of all bookkeeping, so it's really easy for the scripts to just request an icon or an icon slot. This should all make the player experience more smooth.

Here's a couple of screenshots demonstrating how to wield an axe.

Further icons

I've continued on the icon framework and started redesigning the inventory widget. I now have a working solution where the icon management framework provides automatically rendered icons to the widget. A work in progress screen shot can be seen to the left. This shows the icons in the inventory widget to the right, as well as the asset manager to the left, showing the icon texture.
Icons for entities will probably be used all over the interface. Apart from showing basic inventory I can see them being used in equipment dolls, quick bars, trading dialogues and so on.
Therefore I need to design a more robust generic system for dragging and dropping entity icons. Thankfully CEGUI already has excellent support for draggable windows. The icons in the inventory widget can already be dragged. What's left to do however is to allow for them to also be dropped. Preferably in a way that can easily be defined or extended in a lua script. The idea is to allow for entity icons to be dragged and dropped in a generic fashion, so that you should be able to drag and drop an icon from your quickbar to a trade dialogue in the very same way that you would be able to drag and drop it from your inventory.

In addition, I quickly realized that some entities are rendered nicely by default. For example, the coins are by default rendered lying down. Since they don't have any depth, this will result in an empty icon. We therefore need some way of defining how certain models should be rendered when shown as icons. This was easily fixed by extending the .modeldef format with a new "" tag. This tag allows you to define arbitrary "views" for the models. When rendering an icon Ember looks for a view by the name of "icon", and if found sets the camera distance and orientation from the values defined. I'll have to add support for setting this to the model editor though, since it's a bit hard to guess the correct orientation values by hand.

The inventory also needs some better looking graphics. Well, the whole of the gui needs better looking graphics. I've created a blueprint for that here.

Icons galore!

After adding the status widgets and updating the entity labels the only thing left that made Ember look ugly in the default mode was the inventory widget in the lower right. It's always been meant to be a placeholder widget only showing the most basic info (a list of entities with name and type). The grand idea was to instead use small icons for the different entities, much like each and every other game in existance. I could easily have used the EntityCEGUITexture class to render individual entities into separate textures, but this being Ember I opted to go for a more robust solution. Icons is something that we're gonna need here and there in the client, not only in the inventory. I therefore though it would be prudent to provide a more generic icon manager class which uses an icon cache, handles rendering and updates of icons and does house keeping of all icons. I've added a blueprint for it to the launchpad. So that's what I've been doing the last week. I've gotten so far as to have the basic classes done for letting other components check out icons for both entities and Eris base types. They are automatically rendered in a small size (64x64) and copied to a single larger texture. This will increase performance compared to using multiple small textures.
While I've worked on this feature I've also added some much needed asset browsing features to Ember. I found that while debugging I had a great need for a widget which listed all textures used in the system and allowed me to see real time changes to them. This is something that I've felt the need for before, but never had a pressing enough issue. The first iteration allows for texture browsing only, but the idea is to build on it to allow for browsing of all different resources used, such as materials, scripts, meshes, models and so on.
All in all, some nice features for a much better looking UI are on their way.

Chatting

I've now finished the restructuring of how labels and chat bubbles are handled. The new system is much more robust and efficient. Instead of creating the chat widgets from scratch every time a npc says something I now keep a dynamically growing pool of already allocated widgets from which I check out and return widgets once they're not shown anymore. I created some utility classes for general widget pool behaviour which should hopefully be reusable for other widgets in the future ("WidgetPool.h").
Labels are now shown for all entities that inherit from the "character" entity type. That includes pretty much all npc and animals in the game world. For now it's a hard coded check, but it could either be added as an Action to the modelmapping definitions, or as a new rule format (perhaps in lua?). If an entity is controlled by an outside client (i.e. a human) the name is pre- and suffixed with "!" signs. This will be very useful as it will now be easier to see which entities that are controlled by other users.
I also use a new, bolder font for the text. And I added some gradients. The latter might look a little garish, but it helps alot with showing the text on light backgrounds (white text on white background is no good...).
I also solved a strange issue where the text previously always lagged one frame behind the camera. So when the camera moved the text "swayed" a little. It looked quite strange. However, since I now listen for the actual Ogre render call before I position the windows, that issue has disappeared.

The guys over at fedora-games have been interested in packaging WF tech for a while. They already have builds of both Cyphesis and Sear, but alas no build of Ember. There were two reasons for this:

• Ember used WFUT which in turn used Java, which was kinda cumbersome
• Ember required Lua 5.0 and didn't work with Lua 5.1 which is what's shipped with Fedora

However, since some releases we're using libwfut and have no dependency on java anymore. That leaves the Lua issue. Yesterday I took a look at how hard it would be to allow for both Lua 5.0 and 5.1 to work with Ember. Turns out it wasn't hard at all: some #ifdefs later I had a working app which used Lua 5.1. This does however require that CEGUI is built with Lua 5.1, but there are patches available for that (Fedora itself builds CEGUI with 5.1).

All in all this might make me do a 0.5.1 release pretty soon, so that the Fedora guys can get cracking on getting Ember to be included in their package.

Labelling

Yesterday I spent some time revamping the way labels are shown on entities. Currently they aren't shown at all, unless the entity speaks. Then a little "talk bubble" widget is shown. If the entity has some suggested responses, they will appear as clickable buttons. But after a while the widget will fade and disappear. This makes for very bad usability, since it's not possible to easily see the names of any near npc or other players without clicking on them. And it's of course not the way that all other virtual worlds does it.
The plans is therefore to provide a better mechanism for displaying labels on certain entities. I've added a blueprint for this to the Launchpad. The first step is to refactor the quite inefficient way of showing the labels that are currently in use. Instead of creating new CEGUI widgets every time a new label should be shown, and then deleting them afterwards we'll keep a pool of reusable widgets. And instead of doing a lookup of the active widgets every frame we'll use listeners that we attach to the entities and the models to actively listen for events that tell us directly whether to show or hide the labels. Currently it's all still in C++, but the plan is to somehow separate the actual widget creation and rendering into Lua scripts, while keeping the heavy book keeping code to C++. This should also let us mark those avatars that are currently possessed by a human players. There's already an attribute ("external") which marks these, though it's not used. By providing a marking it should be much easier to interact in the world. Currently you often start out in a world with 20+ inactive avatars, with no way to discern which ones are actual human beings (granted, one of the design goals of the AI is that the npc's should behave like humans, but we're not there yet).
So far I've gotten to the stage were a new labelling backend system is fully functional and in use. The next step would be to extend the Model format with information on what entity types to show labels for. Or perhaps it would be better to put that information in the ModelMapping format as an Action? Hmm...

Heads up

After returning from my vacation I yesterday added some status widgets to Ember. That's one of those things that's been bugging me for a while. It's not hard to do; I just never had time. Before I started I looked through most other major mmorpgs that people are playing. People generally like when they can reuse their existing knowledge between different applications. The way the interface works in almost all mmorpgs on the market today is very much standardized: you have your status widget, your map, your inventory and your message window.
So I looked through WoW, LOTR and Vanguard and used that as a base for how the default status widget looks in Ember. I reused some simple graphical elements that jayr had created earlier and think it looks ok. The left element always shows the status of the avatar, while the right element shows the status of the entity that the avatar uses something on.
I'm using the Launchpad more and more. I've put up bugs and blueprints there. My hope with the blueprints it that it will make the development process more transparent and in the end allow for other developers to chip in. Ember is still a one man project and I could sure need some help with it.

Release

The last couple of days I've been trying to get a release of Ember out. It seems I've finally succeeded. First I had to tackle my box crashing all the time. Turns out that it wasn't the SATA card after all (but not until I already had ordered a new card from the local web retailer). Apparently it was the VMWare network kernel module which didn't want to play nice with my Mandriva kernel, resulting in random kernel panics. After applying a "vmware any-to-any patch" it seems to work a little better, even though I've had the box lock up after applying it.
Moving on, of course a lot of different problems would rear their ugly head once I started testing on Windows. Some of them had to do with lousy memory handling on my part, but some seemed to originate from the Ogre GPU code. I've disabled specular normal mapping with Pixel Shader 1 for now, since it seems there's some problems in how Ogre handled my scripts for that.

After all this however I finally managed to get a working version of Ember out. Version 0.5.0. The main news is that it's using Ogre 1.4.3 and CEGUI 0.5.0. The latter means that the gui now has a lot more capabilities than before. There's also a lot of changes going into this release, such as a new entity editing framework. And a lot of internal restructuring to make for a better foundation.

I'm going on vacation now for two weeks or more, so this is pretty much the worst time to do a release. But I'm so sick of having to wait for it; I just want to get it out there. All bugs that are sure to appear as people start using it (and I know people are, even though only a very few turn up at IRC or the forums) will be handled once I get back.

Oh, and another thing. We've started a trial of Launchpad, which is the Ubuntu project management tool. I've put up some initial blueprints and bug reports there. I think the tool has a lot of potential, but it remains to be seen if we're really gonna use it. The entry for Ember can be found here. All bug reports regarding Ember should be posted there from now on.

Pleasantness

As always I end up with all kinds of problems getting a working version of Ember out. I use VMWare Server to host an Ubuntu image on which I build the linux autopackage version. The last week however I've been experienced random kernel panic locks up of my main box. I initially thought it was caused by a faulty SATA chip, but now it seems it's caused by the VMWare network module. Oh well...

All is not bleak however. I've been working with the extremely talented Jayr Kalugin with gettings some more bling into Ember. We've focused on creating some nicer trees, and using more advanced materials with normal maps. I think the initial results are very promising, even thought there's still a lot of work to be done.

Keeping up with the make-it-look-nicer theme I also went over the foliage code. Ever since I changed how the terrain was handled some time ago it's been broken, and I've been reluctant to fixing it since it's basically one big client side hack: it doesn't take into account new areas such as the path to the village, resulting in grass where there should be no grass, and it's not supported on the server. However, it's one of the things that people notice the most, and which I get a lot of questions about. So, in order to present some nice screen shots of the upcoming release I fixed the broken implementation. It looks quite good, but is very resource hungry.

Balance

After releasing the beta 1 of 0.5.0 I've gotten some nice feedback from both Al and Simon, mainly related to gui and user interaction issues. As a developer it's too easy to become accustomed to all the quirks and inconsistencies in the interface, to the point that you don't notice them any more. Therefore it's great to get some fresh outside perspective on the player experience. I've fixed most of the issues in cvs and should publish a beta 2 soon. I still haven't looked at getting it to compile on Windows though. Hopefully roger will be able to provide me with a working compilation solution.

One thing that's still bothers me is that the world doesn't look as good as it should be able to. There's still a lot of rough edges and bad lightning. The main reason for this is that I've focused on providing other functionality, such as editing and underlying frameworks for scripting, model definitions etc.. There's tons of things that could be done to improve the look, chief amongst them is to make better use of shaders. Currently there's pretty much no shaders used at all. In previous versions I've used shaders for the terrain, but since I got some strange artifacts on ATI cards I disabled that. Using shaders I would get a much better looking terrain, as well as a nice speed boost (since I don't have to do the multipass texturing I do now).
Another thing which would improve the look a lot would be some normal maps on the characters. The current vertex lightning makes them look very blocky and jagged. With normal mapping and per pixel lightning it would be possible to give them a much smoother look.
As I've gotten the entity editor in a working state I'm gonna take a closer look at some of these issues. It's a shame not to utilize the many, many nice features of Ogre.

Navigating

I've added two improvements to Ember that I think a lot of people will appreciate. The first is finally putting in checks so that the camera never dips below the terrain. In the end it wasn't that hard to do, I just had to make sure it wouldn't bring everything to a crawl due to massive ray checks. It's still not perfect; when the camera is close to the ground it will still show some of the underworld as it pokes through parts of the terrain. I'll see if I can enhance it to use a more smoother algorithm for positioning the camera a little bit above the terrain.
The other thing I added was terrain decals for showing where the avatar was heading when using the "move to" functionality. It looks really nice and adds some visual hints to the user. Some screenshots of it in action.
Right now it's used for the movement marking only, but it could be used for all kinds of nice in game markings.

Release

I finally got a beta1 of Ember 0.5.0 out. An announcement can be found here. No Windows version this time since I haven't even began looking at the windows build. And also because I got some offers on helping out with the Windows build.
It's always hard to do a release, mainly because I have to halt adding new features for a while and focus on fixing bugs and handle package peculiarities. Basically all development is put on hold until the release is out. Now, this has gotten better as I now have more helper functions in place, but it's still a break from the normal progress.
This last week I've focused on finding and fixing all of the lingering memory corruption issues. I've managed to hunt most of them down I think.
I also reworked how binreloc was integrated. This only applies to Linux builds, and basically means that Ember can either be built as a normal UNIX app with hard coded paths, or as a "relocatable" app where all paths are relative. The main issue is that the "ember" script was meant to be run if you wanted the relative paths functionality, while the "ember.bin" executable was meant to be run if you wanted the standard behavior (for example if you compile from source yourself). This is of course unnecessarily confusing, so it's now changed so that the "ember" command will always work.

Buggers

I've been spending the last couple of days finding and fixing smaller bugs in the Ember codebase. The client itself is pretty solid, I very rarely get crashes, and when I do I can most often fix the underlying problem right away. But then there's the issue of shutdown... It's so much easier to not clean up and instead just exit the application, because it doesn't really matter if there's a segfault when shutting down; the memory is about to be reclaimed anyway, right? Well, there's the problem with sockets and other data structures that aren't that nicely shut down when the app crashes. And there's the issue with shutting down partially and then restarting, like disconnecting from a server and the reconnecting. And so on...
As I wrote in this entry I started Ember as a project to teach me C++, so initially I didn't care much about proper resource handling, because there were so many other things that I had to wrap my head around. But not so any more. Which means that I have to fix all of those lingering problems with resources not being released as they should. The most common culprit is Ogre, CEGUI and Lua. The interdependencies of these libraries forces me to be very careful about how I both acquire resources and release them.

I'm thinking it's about time 0.5.0 was released. The thing is that I always dread doing releases because it means I have to put development on hold for usually more than a week while I try to package Ember for both Linux and Windows. The latter one is always the most time consuming, since I do all development on Linux and then port to Windows at release time. So far I've used gcc and the automake toolchain for Linux and Visual Studio for Windows. This means however that I always have to update the Visual Studio project every time I do a release, and more worrying; that I have to work around all the missing features in msvc (compared to gcc). I'm thinking that it might perhaps be better to set up a mingw system on Windows so I can use gcc for both platforms. If I can get the autotools to work on Windows that would be even more excellent.

Move dammit!

Yesterday I implemented the ability to double click on the terrain and have the avatar go there. It's something that users have been requesting for some time. I've put it on the back burner because I'm myself so used to moving by using mouselook and wasd, and I was hoping that it would be one of those simple things that aspiring developers could tackle to better getting to know the code. All of the functionality already existed in both Eris and Cyphesis, I just hadn't done the connecting. So in the end it took no more then 20 minutes to implement it. Double clicking in the world now makes the avatar move.
The mouse picking system is very flexible and extensible. This flexibility is what allows us to react differently when the user clicks on an entity to when clicking and dragging a world base point in terrain editing mode. It's also what makes an addition such as this so easy. Ember development sometimes feels frustrating because so much work has been put into making base architecture and functionality, without any obvious "external" improvements. But it feels good that at least some of that work pays off. :)

Collisions

Ember uses Opcode (coupled to Ogre through OgreOpcode) for mouse picking. Opcode is a generic collision detection library and could probably be useful for many different things, but currently it's only used for mouse picking. The version used is however many moons too old and also a WIP prerelease. I've been itching for some time to upgrade to a more modern and robust version. One of the things that I've missed is the ability to do raychecks without backface culling. Not being able to do that results in not being able to pick the market stalls from behind, since the fabric is a single face where the material has backface culling turned off.
This weekend I finally took care of this nagging issue by upgrading to the latest version of OgreOpcode. It took 2h to upgrade the code, and then 4h of hunting down why no symbols could be found after the update. It turned out that it's not a good idea to use "," as delimiter in Makefile.am files (instead of the preferred " " version). Ah, the joys of stupidity.
Opcode is however not actively developed, and uses some dodgy "optimization" techniques, resulting in the code being nigh unreadable. It's working for me right now though, but if we in the future get cramped by it it might be wise to pursue other options such as Bullet.

Delisting?

By chance I went over to the wikipedia entry on Worldforge to see what it said. Not so good things it turned out. Apart from devoting a full third of the entry to "Problems", it said nothing of the different clients, the feature set of servers, the media repository and so on. Instead there's outdated and totally incorrect information (branching out into other parts of entertainment?).
What's worse however is that there's a notice now that there's not enough "notability" and that the article will be taken down unless it's filled with more information. Now, I've previously haven't wanted to touch the Worldforge entry on Wikipedia because I understand their position on preventing bias and avoiding original research. As one of the developers of Worldforge I'm quite biased. And therefore I've felt that it would be better if some of the regular Wikipedia editors gathered the information needed and updated the entry.
But it obviously doesn't work that way. And faced with the prospect of disappearing from Wikipedia I had to update the entry with some correct information. I added an software info box and added a screenshot of Ember, but I'm not sure how much more I should add.
I don't want to astroturf, and I fully understand that Wikipedia wants to keep the database clean. If Worldforge isn't notable, well then it shouldn't be there, just as so much other information that's added to Wikipedia every second. But I with my full heart don't believe that, and I think it's more of a case of a neglected entry. There's so much to write about Worldforge, about the clients, the tools, the servers, the AI, the Atlas protocol, the Mercator terrain generation etc.

Adapting

Currently I've been working on the Entity Editor widget. The idea is to allow for world builders to edit every aspect of entities in the world.
The properties of an entity are very loosely defined. Apart from some standard attributes, such as position, id, name, orientation etc. WF puts no specific constraint on the attributes. So, for example, it's possible to create an attribute with the name "age" which is a map of two strings. Such an attribute would of course make no sense. So I'm trying to present some more user friendly representations to the user of the most common attributes.
An example can be see here, editing a settler. With the exception of "bbox" all attributes are shown using the low level representation widgets. So for example, the "guise" attribute is a map, which contains two other maps, named "material" and "mesh". The "material" map in turn contains three string values, identified by the names "dress", "head" and "pants".
The bbox on the other hand is presented with a "Size" widget. In essence, the "bbox" attribute is just a list of six float values, but using the regular List widget to present that would just be cumbersome. In addition, there's no point in removing or adding another value to the list, something that the List widget allows. Therefore, a Size widget is used instead.
So far I've created specific widgets for Size, Orientation, Position and 2DPosition. The latter one is used in areas where the area is defined by a list of 2d points.
By adding more specific widgets I'm hoping to make it much more simpler for world builders to update the world.

Expanding areas

Well, turns out there's still some work to be done in the area department. Mercator has some missing methods in the area handling code. For one thing, there's a Mercator::Area::setLayer method signature, but no implementation. I don't know however if it makes much sense for an area to change it's layer once it's been created; perhaps that's why it's missing?
In addition, when an area is added to the terrain, through Mercator::Terrain::addArea, there's no corresponding removeArea method. Writing such a method is however not completely straight forward since when an area is added to the terrain, it's in turn added to only those Segments that it intersects. One would however think that when removing an Area all one has to do is to iterate over all Segments and remove it when found.
The third problem is that since an area only is added to those segments that it intersects, problems arises when an area is either moved to another segment, or changed so that it now intersects another segment. The effect is that no shading will occur on the new segment. The solution for this would be to add a new method Mercator::Terrain::refreshArea(Mercator::Area* area) which will iterate through all segments, and remove the area if it no longer intersects the segment, and add it where it does.

Also, I had to restart cyphesis since all the NPCs had died. They start out at full health, but since they never eat, they eventually starve to death. When restarting I took the opportunity to update to the latest cvs version. I also added some boulders to the landscape. We have some very nice boulder meshes that Jay did, along with a nice texture, but for some reason it's not used. A normal forest landscape is filled with all kind of boulders. The result can be seen here. Note that the shading of the boulders for some reason is a little bit strange; I'll have to investigate why that is.

The areas of my expertise

I went away over the weekend so I didn't have much time for Ember, but I did manage to fix another one of those things that have been bugging me for a while.
While Ember has supported areas for a while, it hasn't allowed for dynamic updating, either of changes to the area itself, or the position of the parent entity. So yesterday I set out to try to rectify this. Turned out that it was really simple to make that happen. Some new events on the TerrainArea class, some listening glue and some TerrainGenerator logic later it all worked. It's now possible to edit an area on the fly and see the updates directly. The area will also be moved when the entity is moved.

In addition to that, I finally found and fixed a bug in the EntityEditor which made refreshing take ~5 seconds. That's quite the leap from the 0.02 seconds it used to take, so I was a bit perplexed by what might cause it. Turned out that I had accidentally created a int-wrapping loop, which looped from 1 to 2^32. Oh well...

In other news, Ogre 1.4.2 was released today. No major changes as far as I can see, but this version will contain a fixed OGRE.pc file, which will make Ember's configure.ac work as it should.

Back from vacation

I haven't posted to this blog in a long time, mainly because I didn't have anything to say and I didn't think that anyone would be interested in the things I didn't say either.
However, I'll try to rekindle the blogging flame by posting more about my work on the Ember project. I feel that it would be nice to have a record of the development process, mainly for myself. So...

I'm currently preparing an Ember 0.5.0 release. The main reason for the jump from 0.4.3 is that I'm now using Ogre 1.4 and CEGUI 0.5.0. Mainly the latter includes some main improvements.

Apart from those upgrades, I've been focusing heavily on creating intuitive authoring widgets. Ember, and indeed the whole Worldforge project, has been lacking good authoring tools. Currently, if you want to set up your own world, you need to hack the python and xml files of cyphesis by hand. And then there's no easy way to alter existing entities. Sure, there's the cycmd tool which allows for console type editing, but that's not really a smooth experience.
Thus I've now incorporated a new framework for editing entities into Ember. It's using adapter classes to bridge between CEGUI and Eris, with Ember handling the connections. I would really have wanted to go MVC all out, but there are some things in CEGUI that prevent that. Instead I've tried to take it far enough.
One problematic area has been to properly divide the structure into C++ and Lua code. The former is much more powerful, and adds compile time checks, while the latter is more dynamic and allows for run time editing. The overarching idea is that all layout related code should be handled by Lua, while most of the other logic should be handled by C++. I think I've succeeded in part, but there are still some rough edges. And it's never a clear cut case where some part of the functionality should go.

Yesterday I needed to take a break away from the work on the editor (mainly to get my bearings on where the code is heading) and work on some smaller issues. One of the things that have been bugging me for some time is that wielded entities aren't correctly aligned. This is because the hand bone to which wielded entities are attached to isn't really meant to be used for attaching entities, and thus attached entities needs to be adjusted for that. I've planned to add a rotation adjustment setting to the modeldef xml format for some time, but I never had the time. I was also hoping that this would be one of those things that would be excellent as an introductory task for a new developer. But sad to say so far no new developers have contributed anything to the project. It's been a one man show since it's start, with the exception of some greatly appreciated help from Hagen with the console.
So yesterday I finally took care of this issue. It's now possible to specify the rotation needed to get a wielded entity to align properly. The only thing that's missing now is an easy way to edit this in the ModelEditor.