Sunday, June 15, 2014

Pathfinding in Ember

I've recently done some work on adding proper pathfinding to Worldforge. For this I've chosen the Recast/Detour library. While this approach, using voxel rendering to create navmeshes from arbitrary geometry, is both complex and resource expensive, I'm convinced it's absolutely required to allow us to properly support the dynamic and changing worlds in Worldforge.

I've done the initial implementation in Ember, mainly because it's easier to visually debug. But the main benefit will come from integrating this work into Cyphesis, so that it can control AI movement.
This is an area sorely in need of improvement. See, currently there's no path finding or obstacle avoidance at all for the AI characters. The result of this is that NPC characters always move in a straight line towards their target, causing them to inevitable become stuck on other entities. No good. No good.

With proper path finding and obstacle avoidance we can instead implement more advanced AI behaviour.

Wednesday, April 09, 2014

Navigation preview

Here's a short preview of the work I've been doing the last couple of days. I'm using Recast/Detour to implement proper navigation in Worldforge, first in Ember and then in Cyphesis.



Monday, April 07, 2014

Ember 0.7.2 binaries for Linux and Windows available

We now have binaries for Linux and Windows available for Ember 0.7.2.

They can be downloaded from here:
Linux (64 bit)
Linux (32 bit)
Windows

Monday, February 24, 2014

Worldforge taking part in Google Summer of Code 2014

Once again Worldforge has had the good fortune to be selected as a mentoring organization for Google Summer of Code 2014!

This is an awesome opportunity for students to both get more involved in the project as well as get a sweet gig for the summer. If you’re a student reading this, waste no time getting over to the GSoC site and read up on how all of this works. Then swing by the Worldforge wiki to read up on how this all works specific to Worldforge. Don’t hesitate to contact us through our various channels.

As previously we’re also acting as an umbrella organization for our friends in the Ryzom and CEGUI projects.


Here is a video showing off student work from previous year, to perhaps serve as an inspiration.



Monday, February 17, 2014

Ember 0.7.2 released

The Worldforge 3d client Ember version 0.7.2 has been released and is now available from the WorldForge download site.
Ember is a client for the WorldForge project. It allows you to both play in and edit virtual worlds.This release includes a totally revamped terrain engine as well as various improvements to both game play
and authoring.
  • Revamped and improved terrain engine.
  • High level detail terrain materials.
  • Improvements to authoring.
  • Various bug fixes.
This version can be downloaded from the sourceforge download site, at the following locations:
At this time we do not provide precompiled packages.
Note that if you compile from source, you will need to install the content of ember-media-0.7.2.tar.bz2 to PREFIX/share/ember/media or use the “make releasemediarsync” make target.
More information can be found at the Ember site.
All bug reports should be posted at the Launchpad.

Eris 1.3.23 released

Eris version 1.3.23 has been released and is now available from the WorldForge download site. Eris is the WorldForge client-side session layer. This release is a minor release which primarily fixes some bugs and adds support for the new entity reference Atlas format, which is vital for proper import and export support.
  • Support for the new entity reference Atlas format.
  • Various bug fixes.
Source code can be downloaded from the following locations:

Sunday, November 10, 2013

Binary packages for Ember 0.7.1 available


Just a quick update that we're now providing binary packages of Ember 0.7.1.
We're currently providing these for 64-bit Linux, 32-bit Linux and Windows.

To run the Linux packages you must first make sure that the file is executable.

Saturday, October 26, 2013

Ember 0.7.1 released

The Worldforge 3d client Ember version 0.7.1 has been released and is now available from the WorldForge download site.
Ember is a client for the WorldForge project. It allows you to both play in and edit virtual worlds.
This release focus on improving the editing capabilities, allowing for much easier world creation.
  • Export and import of entities and worlds.
  • Much improved entity editor.
  • Ability to edit and debug entity minds.
  • Detect local servers and connect as admin directly.
  • Many bug fixes, improvements and optimizations.
This version can be downloaded from the sourceforge download site, at the following locations:
At this time we do not provide precompiled packages.
Note that if you compile from source, you will need to install the content of ember-media-0.7.1.tar.bz2 to PREFIX/share/ember/media or use the “make releasemediarsync” make target.
More information can be found at the Ember site.
All bug reports should be posted at the Launchpad.

Cyphesis 0.6.2 release

Cyphesis version 0.6.2 has been released and is now available from the WorldForge download site.
Cyphesis is a small to medium scale server for WorldForge games, with builtin AI. This version includes the demo game Mason which is currently in development. This release is intended for server administrators wishing to run a Mason server and World developers developing new worlds or game systems. It is primarily a bugfix release which fixes a serious crash issue.
Major changes in this version:
  • Fix for a possible crash related to wielded entities and destruction.
  • Addition of a new “spawner” property
Source code can be downloaded from here:

Friday, October 25, 2013

Eris 1.3.22 Release

Eris version 1.3.22 has been released and is now available from the WorldForge download site. Eris is the WorldForge client-side session layer, used by many existing clients. This is a development release, as the API may change prior to the final release of Eris 1.4.0. However, any changes should be minor and easy to incorporate into clients; testing is recommended and appreciated. Changes:
  • Ability to connect to a local socket has been added.
  • Support for connecting using a “sys” account.
  • Handling of the avatar being deleted.
Source code can be downloaded from the following locations:

Sunday, September 29, 2013

Google Summer of Code 2013 report: Samuel Kogler

The Google Summer of Code 2013 is now complete. Samuel Kogler worked during the summer on improving the terrain rendering component in Ember.
We've previously used a component called the PagedLandscapeSceneManager2 for our terrain rendering. This component however is both unmaintained and has some design deficiencies which makes it ultimately unsuitable for what we want to achieve with the worlds in Worldforge. Samuel's task over the summer has been to bring the newer and more flexible Ogre Terrain Component into Ember.
Apart from the work he's been doing on Ember he's also submitted many changes to Ogre, both to core Ogre functionality and to the aforementioned Terrain Component.
We're very pleased with the work he's done, as well as having to work alongside with him. It's been an absolute joy to see his progress.
Below is his own presentation of the work he's done.
As he's based his work on the not yet released Ogre 1.9 branch we haven't yet merged his code into Ember master; as soon as Ogre 1.9 is released we'll commence that work. Until then his code can be seen over at his Github fork.

Thursday, September 26, 2013

Cyphesis 0.6.1 released



I just recently released Cyphesis 0.6.1. The official announcement is over at the Worldforge site.

This is the first release after I took over from Alistair as maintainer. The focus, as well as with the upcoming Ember release, is fully on implementing features to make it easier for world authors.

This follows the strategy I laid out when taking over maintenance duties in April. In order for Worldforge to attract new developers and world builders we must make it much, much easier to actually create worlds. A crucial part of this is making sure that any changes done on the server actually are persisted between server restarts. Previously this wasn't true, as no "minds" were persisted. That meant that all of the behaviour and knowledge of any creature or character was wiped out when the server shut down.
That of course made it quite hard to iteratively construct an interesting world. Not so anymore!

But not only that; it's now also much easier to both alter and inspect minds. This was also a source of frustration previously, as there was no good way as a world author to interact with the minds. While a crude mechanism for inserting new goals was available, this didn't really cut it. Especially when goals malfunctioned or behaved in non-intuitive ways. Trying to figure out why the woodcutter wouldn't cut down the tree one meter in front of him was nigh impossible.
All of this is now improved as it's now possible to add, delete, alter and inspect goals. The latter feature is a god send when constructing behaviour or implementing new goals, since it's now easy to see exactly what a certain NPC is thinking.

However, in order to fully use these editing feature you must run the latest version of Ember. There will be a 0.7.1 release shortly; we're currently working out some final issues. In the meantime, if you want to try these new features out you should run Ember from the Git master branch, preferably through the Hammer script.

Monday, April 01, 2013

Exporting and importing in Ember

The last couple of weeks I've been working on improved authoring capabilities in Ember and Cyphesis. One of the most important features is the ability to both export and import entities from and into servers.

This allows a world author to save a snapshot of the world to local disk, and later on restoring the world as it was in the snapshot. There are multiple usage scenarios, but one that immediately springs to my mind is the improved ability to perform iterative world editing. By being able to reset the world back to a known working state it's much easier to try out new configurations and behaviour.

All work is currently being done on separate branches of both Ember and Cyphesis. The following video shows the status of this feature some weeks ago; it has progressed even more since.

Monday, March 19, 2012

Worldforge in Google Summer of Code 2012

We've been allowed to take part in Google Summer of Code for another year. This is as always exciting and humbling. The Summer of Code is such a wonderful program which always keeps us amazed. So many different things are awesome, from the basic ideas and soul of the program, to the people at Google making it happen, to the mentors around the world, to the students working all through summer.

There's an announcement up on the main Worldforge site with more information.  Right now we're in the period where students are encouraged to interact with the community and work on their ideas.

If working on a totally free virtual world system during the summer sounds interesting, don't hesitate: run over to the Worldforge wiki, read up on the program and don't be shy about applying.

Wednesday, June 08, 2011

Properly shaded foliage

Ember has a system for generating grass and other kind of foliage. It looks pretty good in most cases, but the shading of the grass has always been off. This is even more apparent when the time of day changes, as the grass doesn't change colour when the world does. In addition, the grass didn't receive any shadows, making it look strange when running in a higher graphics settings.

All this has changed now however, as I've altered the grass to be both properly lit and to receive shadows correctly. The effect can be seen below.

Tuesday, May 24, 2011

Ember 0.6.1 out of the door

I finally managed to get Ember 0.6.1 out of the door.
It seems to be common procedure for me to never be able to get the releases out in accordance to my plans, but this time it took even longer than would seem "normal".
The two main reasons for this were Windows and 32-bit architecture, both legacy systems (hurr hurr hurr).
I've always striven to make Ember cross platform. This can be seen in the components used and the way its internals are laid out. The main problem is that I'm not too fond of touching Windows unless I absolutely have to, and therefore the Windows releases have lagged behind the Linux ones. Fortunately for us a contributor in the Worldforge community, James Lovejoy, has been working on streamlining the process of building Ember for Windows, and with his help we were set on releasing a working Windows binary along with the 0.6.1 release.
It turned out however that some of the low level networking stuff we were doing in the Eris library didn't play to well with Windows. Ok, cue a rewrite in Eris by Alistair, some testing of that, and we got it fixed after a while.
The other issue was more subtle. I've added some mechanisms for making sure that Ember is running on a fixed fps rate (60 by default). This relies on getting and comparing millisecond timestamps from the OS. Problem here was that I was using the C++ "long" type, which on 64 bit machines (such as the one I'm developing on) is 64 bits, but on 32 bit machines is 32 bit. And 32 bits are too few, since the timestamps will then wrap around after something like a month's time. Not good. And this was of course something I didn't discover until I basically had already made the release and was testing our 32-bit precompiled binary out (we're now using AppImage since Autopackage is defunct).

Just some snags on the way, but enough to delay the release a little bit too long.

Currently we've just started up the coding part of Google Summer of Code. This year I'll be mentoring two students: Martin and Peter. They're already at work improving Ember, with some exciting weeks ahead of us all.

Oh, and here's a small movie showing how the Model Editor can be used.

Sunday, March 20, 2011

Worldforge in Google Summer of Code 2011

Worldforge has been accepted into Google Summer of Code 2011!


We've had a remarkable experience with the program from before and are very excited about this opportunity. If you're a student which are interested in working on a cool Worldforge project during the summer, don't hesitate to visit our GSoC page. The sooner you start getting involved in the project the better, so don't be shy about making yourself heard on the mailing lists or irc.

Saturday, June 19, 2010

Simplify tolua++ with autotools

In Ember we use Lua for our scripting needs. The bindings to the C++ parts of the client are provided by the tolua++ library. Tolua++ works by generating C++ source code from .pkg files, which are simplified .h files. This works out extremely well; the developer only needs to copy the .h file to a similarly named .pkg file and remove those methods and fields that shouldn't be exported. The command line tool "tolua++" is then run to produce the C++ source.

Previously in Ember we handled all of this by keeping the generated C++ source checked into the source, and requiring that each developer had to run the tolua++ command to regenerate these sources each time a lua binding was added or changed. This however had a couple of downsides, chief amongst them that the code generated differs a bit depending on the version of tolua++ used. It also resulted in some often very large Git commits of generated code, which tended to pollute the Git history.
A much better solution would be to instead automate the generation of the binding source, and bake it into the normal build system. Fortunately the Autotools already provides all the facilities for making this happen.

An example of how this is handled in Ember though a Makefile.am can be found here:

SUFFIXES: .cxx .pkg .lo .la .cpp .o .obj

.pkg.cxx:
cd $(srcdir) && TOLUAXX=${TOLUAXX} $(abs_top_srcdir)/scripts/update_lua_bindings.sh `basename $@ .cxx` `basename $@ .cxx`.pkg $(abs_builddir)/`basename $@` $<

INCLUDES = -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir)/src -DPREFIX=\"@prefix@\"

noinst_LIBRARIES = liblua_EmberServices.a
liblua_EmberServices_a_SOURCES = EmberServices.cxx

CLEANFILES = EmberServices.cxx
TOLUA_PKGS = ConfigService.pkg EmberServices.pkg IInputAdapter.pkg Input.pkg InputService.pkg LoggingService.pkg MetaserverService.pkg ScriptingService.pkg ServerService.pkg
EXTRA_DIST = $(TOLUA_PKGS)
EmberServices.cxx: $(TOLUA_PKGS)

The files references in TOLUA_PKG are the .pkg files which define the bindings. These will be fed through the update_lua_bindings.sh script to generate the file EmberServices.cxx, which is then compiled and added to the liblua_EmberService.a archive. Note that we need to add EmberServices.css to the CLEANFILES variable to make sure that it's deleted when we're cleaning up. Since it's generated through the tolua++ tool Automake can't keep track of it itself.
The update_lua_bindings.sh script looks like this:

#! /bin/sh
#tolua++ will for some reason translate "const std::string" into "const,std::string", so we need to remove these incorrect commas from the final code
#some versions will also for some unexplainable reason not correctly remove the tolua specific directive tolua_outside, so we need to clean that out also
#We'll also replace the inclusion of "tolua++.h" with our own version which has better support for building on win32.
echo "Updating lua bindings."

#If the TOLUAXX environment variable isn't set default to using the command "tolua++".
if [ x${TOLUAXX} = x ]; then
TOLUAXX=tolua++
fi
${TOLUAXX} -n $1 $2 > $3
grep -q '** tolua internal error' $3 && cat $3 && exit 1
sed -i -e 's/const,/const /g' -e 's/tolua_outside//g' -e 's/tolua++\.h/components\/lua\/tolua++\.h/' $3

This script basically runs the tolua++ command, as defined in the TOLUAXX environment variable (with "tolua++" as fallback) and then applies some replacement to fix some issues we've been having with the generated code.
The TOLUAXX environment variable is set at configuration time. The default is "tolua++", but we'll provide the option to use an alternative command. Our acinclude.m4 file has this snippet (which is called from configure.ac):

AC_DEFUN([AM_CHECK_TOLUAXX],
[
AC_ARG_WITH(tolua++,AS_HELP_STRING([--with-tolua++=CMD],[Tolua++ command (default=tolua++)]),
toluaxx_command="$withval", toluaxx_command="tolua++")

AC_CHECK_TOOL(TOLUAXX, $toluaxx_command)

if test "x$TOLUAXX" = "x"; then
AC_MSG_ERROR([Could not find a working tolua++ command (tried '$toluaxx_command'). Use the --with-tolua++ switch to set the proper command to use.])
fi
])

This setup will allow the script bindings to be generated at compile time, but only the first time compilation occurs, or if any of the .pkg files have changed. More examples of how this is used can be found in the Ember sources.

Tuesday, June 15, 2010

Using automake to generate ChangeLog from git

When deploying a GPL application it's required that you provide a change log containing information on all the changes to the code base. This file is normally named ChangeLog.
In the olden days, before today's fancy revision control systems, people often had to edit the ChangeLog files themselves for each and every change to the code. This is what we did in the Worldforge project when we used CVS for our source code needs.

However, for the modern developer used to distributed revision control systems such as Git this seems archaic. Why provide a separate list of all changes when the Git repository already contains a complete log? Instead of keeping the ChangeLog file updated with each commit, wouldn't it make more sense to generate it from the repository history when a new release is made?

When a new release needs to be created in a project using the autotools the "make dist" make target is invoked (or in reality the "make distcheck" target). This will package the source and produce tar archive of it all. The Makefile target "dist-hook" is provided to allow for us to hook into this process. So what we want to do is to provide some shell scripting which generates a ChangeLog file from the git history. It will look something like this:
cd ${top_srcdir} && git log --stat --name-only --date=short --abbrev-commit > ${distdir}/ChangeLog
This will generate a log of all the changes, in a condensed format not unlike the ones suggested by GNU.

We could be done here, but there are some things that could need improvement. For one thing, this will only work when the "make dist" target is invoked in the original git directory. Some distributions might want to take the dist release, add some distro specific patches, and then make a new dist from that. It would therefore be better to add some logic which can recognize whether there already exists a generated ChangeLog, and if so won't try to generate a new one from the git log. This will need some more logic so we'll be splitting this functionality out to a separate script and call that from the dist-hook target. We'll also add a ChangeLog with exactly one line:
This file is autogenerated from the Git history when a the "dist" make target is invoked. If you find this file in an official release something has gone wrong and you should contact [maintainer_email]. It needs to be exactly one line long in order for the ChangeLog generating script to work.
Our script will check the length of the ChangeLog file. If it's exactly one line, we know we should generate it from the git log, else we know it's already been generated and we shouldn't do anything. Our script (generate-ChangeLog.sh) will look something like this:
#! /bin/sh
top_srcdir=$1
distdir=$2
if [ `cat ${distdir}/ChangeLog | wc -l` = "0" ]; then
chmod u+w ${distdir}/ChangeLog && cd ${top_srcdir} && git log --stat --name-only --date=short --abbrev-commit > ${distdir}/ChangeLog
fi
The dist-hook will now look like this:
dist-hook:
sh $(top_srcdir)/generate-ChangeLog.sh $(top_srcdir) $(distdir)
Now we're pretty set. With Worldforge there's however still an issue with the older CVS ChangeLog. When the source was migrated from CVS to Git the log messages from CVS were directly imported into Git. These however were in a quite verbose format with the date and authors included. Since this data also is available as meta data in the Git repo, the result will be that these log entries will have a lot of redundant data, making the ChangeLog both very large and confusing. To prevent this, we'll store a copy of the old ChangeLog with the CVS entries with the source (as ChangeLog-CVS) and combine this with the Git log history. After looking at the Git history we can see that at the commit with id "f12012e7616c191a8926432faf866c8e43854062" marks where the transition from CVS to Git happened. We'll also replace the ChangeLog-CVS with a notice about it's previous use (as it's not needed anymore, but must be present in the dist) Our script will then look like this:
#! /bin/sh
top_srcdir=$1
distdir=$2
if [ `cat ${distdir}/ChangeLog | wc -l` = "0" ]; then
chmod u+w ${distdir}/ChangeLog && cd ${top_srcdir} && git log f12012e7616c191a8926432faf866c8e43854062..HEAD --stat --name-only --date=short --abbrev-commit > ${distdir}/ChangeLog && cat ${top_srcdir}/ChangeLog-CVS >> ${distdir}/ChangeLog
chmod u+w ${distdir}/ChangeLog-CVS && echo "This file was needed for generating the proper ChangeLog as an aggregate of the code held in git and older code in CVS. It's now empty, but needs to be included in the source distribution to not upset automake." > ${distdir}/ChangeLog-CVS

fi
And now we're done. We have an automated system which will generate the ChangeLog from a combination of both the old CVS provided ChangeLog and the Git log history. It will also make sure that if the "make dist" target is run again on an already generated dist the ChangeLog will be left as it is. Just remember to also include all relevant files in the EXTRA_DIST target of the Makefile.am.
Up to date versions of how this is used in Ember can be found in the Ember repository.

Thursday, May 20, 2010

Freedom of Speech

Today I'll be posting about something other than FOSS game development, something far more important (yes, such a thing exists!). Today's topic is Freedom of Speech and how we never must take that for granted.

I live in Sweden where the last couple of years there has been a couple of incidents where islamic fascists have tried through violence and threats of violence to impose their world view on other people. Central to that world view is an active aggression against the principle of Freedom of Speech which underpins our whole democratic and parliamentary system. The most prominent examples are the threats and active murder attempts on the Danish cartoonists who drew pictures of Muhammed, or the threats made against South Park. In Sweden the artist Lars Vilks has been both physically assaulted and subjected to arson as a result of his drawing of Muhammed in 2007.

It's easy to take the Freedom of Speech for granted and to think that those people that want to bring it down only operate in chaotic countries such as Pakistan or Somalia. That is unfortunately not the case, as shown in this video of Lars Vilks presenting a lecture on the topic of "Art and Freedom of Speech". Vilks suffered some bruises and a broken pair of glasses. The day after someone tried to kill him by setting his home on fire.


Do not be tempted to draw the conclusion from this video that Sweden, or any other European country is under any threat of "being overrun by Islam" or similar. That is absolutely not the case; it is in fact the opposite. When faced with a stable society and a high standard of living most people tend to distance themselves from religion and totalitarian ideologies, even if it might take a generation or two.
But it does show that Freedom of Speech is something we never can take for granted, and not something we can barter with. It is the firmament on which our whole society is built on, and we cannot let fascist or totalitarian ideologies or groups make us infringe on it one iota. As long as people feel threatened by violence for expressing their opinions there is no real Freedom of Speech. For every Lars Vilks who has the guts to criticize Islam and as a consequence gets assaulted and beaten, there are thousands of other people who hold their tongue. Some of them in Western countries such as Sweden, but most of them in countries where Freedom of Speech already is curtailed, such as Iran or Pakistan.

Therefore I'm today celebrating the annual Draw Muhammed Day by presenting my own drawing of Muhammed, the founder of Islam. By providing my own drawing of Muhammed I'm showing my support and solidarity with the artists currently under death threats. Note that the drawing is pretty tame (and badly drawn); I know this blog is syndicated so I'm putting a little restraint on myself. To more properly show my complete and total outrage of having islamic fascists trying to impose their warped totalitarian world view on me it would be more proper to show Muhammed being raped by a pack of dogs. But I'll wait with that picture for another day (not that the Internet is lacking in such depictions).


Muhammed, shown here during one of his schizophrenic episodes trying to come up with another verse for the Qur'an.