6 Oct 2009

Tools of the Trade

We have ramped up tools development at Radon Labs considerably during the development of the two Drakensang games. Traditionally we have been (and still are) a bit conservative about inhouse tool development and try to avoid re-inventing wheels as much as possible. Each new custom-tool requires permanent maintenance work and if a “standard industry tool” exists for a job it is usually better to just use this. But especially in the domain of “game-logic stuff” there are basically no standard tools, so the situation is much more dire compared to graphics or audio tools.

We have traditionally been using Excel tables and XML files compiled into an SQLite database for “game data”. This works pretty well if the number of “work items” is around a few hundred. But for complex RPG games, the number of work items is in the range of tens-of-thousands (quest tasks, dialog takes, voice over snippets, graphics objects, textures, items, NPCs, monsters, behaviour scripts, and so on…). Excel tables and raw XML files don’t scale up very well because they lack game-specific features for filtering, searching, statistics, and of course the possibility of human error is very high, and finding and fixing those errors isn’t much fun either.

 

The Texture Tool

The very first custom tool in C# (to try the waters so to say) was a simple replacement for an Excel table which defined texture attributes (stuff like the size and DXT format of a texture, mip-map quality and so on). The Excel table allowed to give each individual texture its own set of attributes. For a small project with a few hundred textures this works pretty well. Drakensang has around 15,000 textures however, and this is way beyond the territory where Excel is starting to become a pain in the ass. Thus we wrote this:

texture_tool 

The tool manages the same information as the Excel table it replaces, but has much better filtering and manipulation features. The left hand sides gives a tree-view of all texture categories, and it’s possible to either display all textures, or textures of a given category, and it’s also possible to further filter the textures by a pattern (for instance to display all normal textures of the “armor” category, click on “armor” in the tree view and type “*_bump.*” into the top most line of the “File” column. Entering a value into one of the other top-most column-entries will set this attribute value to all displayed textures. It’s a very simple tool, but it’s easy to use and scales up very well to tens-of-thousands of textures while still making it possible to find and tweak a single attribute of a specific texture with ease.

 

The Story Editor

When we started planning Drakensang, we knew that we would need new tools for creating quests, dialogs and game logic scripting. On the surface, these 3 things are something completely different, but for the game core, quests, dialogs and scripts aren’t that different (the common element is the use of Conditions (small C++ objects which check whether some condition in the game world is true) and Actions (similar C++ objects which manipulate the game world)). Thus the Story Editor was born (actually, it’s a Dialog/Quest/Script editor). The Story Editor loads and saves XML files, which are compiled into the game database during the build process, or directly from the tool to immediately check the results in the game. We couldn’t anticipate all required features for the story editor at the start of the project, thus the editor was constantly worked on during the development of Drakensang (we added a lot of features to improve localization, proof-reading or voice-over integration for instance).

Note that the Story Editor currently suffers a bit from the typical “inhouse tool UI aesthetics problem” ;)

Here’s a screenshot of the Story Editor when working on a simple quest:

story_editor_2

On the left is the simple linear task list, the panels on the right show the attributes of the currently selected task (for instance the conditions and actions associated with the task).

Here’s the screenshot of the Story Editor with a simple action script:

story_editor_1

We opted against a traditional scripting language, but instead used a “point and click” approach. A script is just a simple collection of Conditions and Actions. These could just as well be wrapped into LUA functions for instance. Our current approach is probably not as powerful as real scripting system, but definitely less error-prone and easier to control.

Finally here’s the Story Editor when working on a dialog:

story_editor_0

Dialog takes can be associated with Conditions (to show or hide dialog takes based on some in-game condition), or Actions (to manipulate the game world as the result of a conversation). The Comment tab on the right side is most useful to add instructions for audio recording sessions.

 

The Sequence Editor

In the first Drakensang, in-game cutscenes had been hand-scripted by defining a sequence of Actions in the Story Editor, basically in the most un-intuitive way imaginable. For the new Drakensang, we wrote a whole new “Sequence subsystem” along with a new tool (the Sequence Editor) which gives control over cutscenes back into the hand of the artists and provides a very intuitive workflow with instant turnaround times (resulting in a dramatic quality improvement if I may say so). Here’s a screenshot of the Sequence Editor in action on a dual-monitor setup:

 

sequence_editor_0

On the left-hand side is the actual editor, on the right-hand side is the ingame-preview. Tool and game communicate through XML messages over a TCP/IP connection, changes in the tool immediately show up in the preview. The other direction works as well, for instance it is possible to set keys for the camera or objects directly in the preview window, complete with snap-to-ground and other time-savers.

Here’s a more detailed shot of the Sequence Editor:

sequence_editor_1

“Trackbar Elements” from the bottom area (Play Sound, Depth Of Field, etc…) can be drag’n’dropped into the timeline area above. Every trackbar element comes with a number of attributes, most of them can be animated over time. Trackbar attributes are displayed in the right-side column (in the screenshot, I have selected an “Ambience Bubble” element which allows to manipulate the ingame lighting, posteffect parameters, and other stuff which is important to set the visual “mood” of a scene). Animated parameters are edited in the Graph Editor which looks and feels very similar to Maya’s graph editor window:

sequence_editor_2

It’s important to note that complex character animations are not created in the Sequence Editor, instead the sequence system will usually only trigger existing character animations from the character’s animation library. How complex character animation tracks are is up to the cutscene designer. Sometimes its better to fire a lot of small, generic animations and do the path-animation in the Sequence Editor, sometimes it’s better to generate one big motion-capture animation for the entire cutscene, complete with the actual character movement.

One very cool feature of the sequence system is its extensibility. Simple trackbar elements, like “Depth Of Field” or “Camera Shake” can be implemented in under an hour by extending an XML config file which describes the attributes of the new element to the editor tool, and by writing a very simple subclass in the engine which connects the animated values of the trackbar to another subsystem (for instance the PostEffects subsystem in case of the Depth Of Field trackbar).

 

The Table Editor

The new Table Editor is aimed at finally replacing the Excel tables for game object template data. In Drakensang (or Mangalore in general) a game object’s persistent state is simply described by a collection of typed key/value-pair attributes. Different game object categories have different sets of attributes. For instance a Monster game object has a different attribute set then a Weapon game object.

Traditionally, we had one Excel table per game object category, and one line in the table per game-object template. The value of some of the attributes (instance attributes) can be set in the Maya level editor to a different value for each game object instance.

Some of the bigger game object tables became a really critical bottleneck during the production of Drakensang, since only one person could work at a table at a time. Also even though the tables were saved in XML format, both CVS and SVN are pretty much useless for merging.

Thus the idea was born to move the game data tables into a true database backend, and then create one generic “Table Editor” and later on, several smaller, specialized Editor tools which basically provide a different view into the database. This is also the very first step of moving away from Maya as a level editor towards a truly collaborative level design environment.

Here’s a screenshot of the generic Table Editor tool:

db_frontend

On first glance, this just looks like a nifty Excel replacement, but under the hood it is much more:

There’s a complete user system with login and access rights (in the screenshot above, I can see that I am currently logged in as “floh”, and that 1 other user is currently editing the Weapon-Table).

Then there’s a complete revision control system. The table basically extends into the time dimension, and it’s possible to inspect the table at any point in the past. There’s also a very detailed change-log, which not only tells me who modified the table at what time, but also all the changes made to the table:

db_editor_changelog

Of course it’s also possible to revert the content of any cell to any point in time.

When a level designer changes values in a table, it will only happen in a local sandbox (the changed values already live in the central database, but are marked as local data of the user). This allows a level designer to mess around with the tables without affecting the other users. When everything works as expected, the changes are committed into the “global domain” and the new values are visible to all.

The generic database table editor tool is very powerful, but the whole system really shines with specialized front-end editors like this:

db_special_frontend

This is basically just a different frontend to the global game database with a specialized GUI for efficiently equipping characters and chests in the game. The general idea here is that such a specialized editor tool is written for tasks where working with the generic editor is not desirable because it would be too cumbersome, un-intuitive or error-prone.

There’s also a lot of interesting work going on in our build-pipeline at the moment, but I’ll leave this to another post :)

7 comments:

Oliver said...

Wow, thanks for this post! It's nice to see some insight into in-house tool development of other developers.
I really like the generic approach of the table editor and the pluggability of other views on top of it! Did you use a standard relational database or did you build your own backend?

Comix said...

Great post!

roger said...

That remembers me the lot of work that I still have to do in my project just related to game tools: map editor and dialog editor.

Floh said...

@Oliver: we're using MS SQL as database server since its a natural fit for C# (LINQ etc...)

Ralph said...

@floh
When can we expect the next code dump of nebula3? the last one is almost one year old

Thank you

Steven 'lazalong' Gay said...

If I remember correctly you once said you would try to have more regular code dump, yes?

Any chance of one "soon"? :)

Dursel said...

The Story Editor is very nice. Can you relase it separectly ?