In my previous news post "More about Component Systems
" I presented the central ideas and concepts about the two component systems that we are about to introduce to the Cafu Engine. Since then, a lot of work has been achieved, and it is time for an update:
The biggest challenge and the focus of my efforts has been the porting of the old DeathMatch game code to the new component system. This was very successful, and in fact the old DeathMatch game code could be entirely removed and be replaced with newly-written components that have just the right mix of C++ and Lua script code.
In fact, the successful removal of the old, intertwined and tricky DeathMatch C++ code was the key step that motivated me to write this news update. However, that doesn't mean that we're done yet. Some things had consciously (and temporarily) to be broken and must now be fixed again, and other things are not yet as they should be. Here is a summary of the technical aspects:Updates to the repository
If you're following the Cafu source code repository
, especially on branch master-vc2012
, the following recent changes are important:
- I just tagged revision 1debdc3 (of 2013-05-25, almost exactly one year old!) with name pre-entity-component-system. This is the revision where our master branch was at the time when the development of the entity component system was begun. The annotated tag message is this:
This is the last revision before the extensive work was begun to introduce a
component system for game entities to Cafu.
It is expected that the development will take a long time. Although it is
intended to keep backwards-compatibility wherever possible, it is expected
that compatibility to now existing binary files (especially `cw` world files)
and to custom Lua map scripts (e.g. `TechDemo.lua`) must and will be given up
in the process.
Contrary to the already completed introduction of a component system for GUI
windows, it is expected that the upcoming work is of a long-running nature,
comes with extensive changes to almost all parts of the Cafu Engine, and is
possibly "open-ended". It is therefore developed in the `master` branch rather
than a feature branch that would run for months and years, and in the meantime
only leave `master` back at this commit, which will sooner rather than later
become old and unsupported.
Also see http://www.cafu.de/forum/viewtopic.php?p=6212#p6212 for additional
- Next, I fast-forwarded branch master to entity-component-system (integrating entity-component-system into master), in which until now the entity component system was developed, then deleted the now no longer needed branch entity-component-system (and master-vc2012 as well).
- If you are among those who, like me, had checked out branch entity-component-system, and possibly based work on it, you can use these steps to update your working copy (the steps are shown for Windows, but are very similar under Linux):
# Fetch all the new branches, commits and tags:
> git fetch --all --prune
> git fetch --tags
# Clean up the working copy, i.e. stash or commit local uncommitted changes
# as necessary, and (on latest entity-component-system), delete possible
# leftovers from previous revisions:
> rd /s Games\DeathMatch\Code
> rd /s Games\VSWM
# Checkout the "old" master branch, and fast-forward to origin/master:
> git checkout master # old master
> git merge origin/master --ff-only
# In my case, it was sufficient to delete the now obsolete local branch,
# however if you have based own work on it, you'll rather want to rebase it
# onto or merge into master first.
> git branch -d entity-component-system
Please note that although I felt that this is the right time to continue the development of the entity component system in the master
branch rather than keeping it further artificially separated in the now gone entity-component-system
, at this time master
in a state where a release could be made from it: In fact, if you compile and run it, you will quickly find a lot of problems.
These problems are well known to me (but any feedback is welcome, don't fear duplicate reports), and the rest of this news post summarizes the problems and the plans to resolve them.Plans regarding "Player Prototype" entities
Implementing the "carried weapons" of a human player is, in all its details, surprisingly difficult. A lot more difficult than the code for monsters and other non-player entities. In fact I managed to move its previous, old implementation into the component system -- but there it is still old and inflexible code, and not in the useful and helpful spirit of the component system as everything else.
Also, moving the code necessitated some compromises, and these compromises, while they made it possible to integrate entity-component-system
, are also responsible for the "broken" look and feel of today's (May 2014's) master
Pondering this gave raise to another insight: So far, we have no concept for human player entities
in the component system at all! That is, giving the map designer
a chance to make any settings or script code adjustments for the human player entities that are supposed to spawn in the map was just not possible.
To fix all this, the idea is to introduce a new special entity that acts as a "Player Prototype". Of this new entity, exactly one instance exists in each map that the map designer can use to customize all details to his heart's contents, including that of possibly carried weapons. The Cafu Engine then spawns an entity for each human player that joins the map by cloning the "Player Prototype".
Among others, the player prototype entity has a list of components of type "CarriedWeapon", whose properties and script callbacks define one weapon each, and each such weapon would be completely self-contained, keeping information about e.g. its availability to the player ("has it been picked up earlier?"), whether it is active or holstered (more than one weapon can be active at a time, e.g left and right hands), the current, minimum and maximum ammo, and so on.
Developing this "Player Prototype", the carried weapons, and all related functionality will amount to a total rewrite of the old code, and be the immediate goal of my next steps -- this is exactly what we have introduced the component system for, after all! Even if it is not obvious from this text alone, when done you will see that the result is very very worthwhile! More work to be done
The above is crucial, but there is still a lot more important work to be done, and branch master
, despite my promoting it as described above, currently has known bugs and generally not the quality that we normally strive to achieve and are used to.
Most importantly, the carried weapons
must be fixed as described above. However, also the particle engine
needs a thorough overhaul, and we really have to re-implement the precaching
feature also for our component system, whose current lack causes noticeable lag.
Finally, the Map Editor
is the only part left that is not yet adapted to the entity component system. This is very important, as the Map Editor is often the program that new users look at first.
Post and read comments about this news item at the related forum topic.
As explained in my previous news post
, I am currently migrating the old DeathMatch game code to the new Component System architecture
. Just this afternoon I finished dealing with the "hand grenade" entities in the new system, and I found that they're noteworthy in many regards:
- Hand grenades are both dynamically created during the course of a game (when the player throws them) and dynamically removed again (after they have exploded and their traces have gone). This makes them different from many other entities that are defined in map files and often exist from the beginning to the end of the map.
- Hand grenades employ more components than other entities in order to implement their functionality:
- Model (the hand grenade model as thrown through the map),
- Physics (for computing the flight path and dealing with collisions, e.g. with walls and the floor),
- Script (for controlling the details, e.g. letting the detonation timer tick and implementing the explosion effects),
- Light (for the flare when the hand grenade explodes),
- Sound (for the sound at the time of the explosion),
- ParticleSystem (for the smoke and spark particles at the explosion).
- They very well demonstrate the distinction between Server State and Client Effects.
It is the last point that I would like to highlight in this post, because a good understanding of it is crucial for game development with Cafu:Server State
The server state is characterized by the following properties:
- It is synchronized via the game server to all clients,
- every player (client) sees the same results and consequences,
- server state is relevant for gameplay.
Typical examples are drawbridges, the position and state of elevators and doors, details about opponents, etc.
As a side note, events
are often a result of a change in the server state, and transferred to the clients as such.Client Effects
Client effects can be triggered by some event in the server state, but besides that, they:
- run independently of each other on the clients only,
- are often used for eye-candy,
- it's ok if not everyone sees exactly the same (not relevant for gameplay).
Examples include particle effects such as smoke, sparks and small pieces of debris, many fade or transition effects, model animation frames, etc.Having both
An important insight is that we sometimes need both server state and client effects combined
For example, consider a light source that illuminates an otherwise pitch black part of a labyrinth. Whether the light source is on or off clearly makes an important difference for players who try to navigate the labyrinth, and so the state of the light source is managed as a part of the server state.
However, there are also other kinds of light sources, e.g. those that are on, but steadily flicker or pulse. This is often a means to create a specific atmosphere to a map, but the exact amplitude of the flicker or pulse is really not something that needs synchronization over the network.
In fact, as a combination we may wish that the light source's on or off state is synchronized to everyone, and if it is on, have it have a client-side flicker effect for eye-candy.
This is exactly what we can (starting with commit d11c6a9
) achieve with the light source components, and what I now employ with the hand grenades: When a hand grenade explodes, the related light source is turned on in order to mimic the bright flare of the explosion. However, the exact details of how the bright light fades, changes its color from white over yellow to red, shrinks and eventually disappears (all in about half a second), needs not be synchronized over the network.
Instead, a very simple script method is used to implement this. Here is its entire code:
local clTime = 0.0
local Duration = 0.5
if not self:get("On") then
clTime = 0.0
clTime = clTime + t
if clTime >= Duration then
return true, 0, 0, 0, 0
local Amount = 1.0 - clTime/Duration
return true, Amount, Amount*Amount, Amount*Amount*Amount, 400.0
Well, this is one of the reasons why I love the new component system!
Post and read comments about this news item at the related forum topic.
When I first posted about Component Systems
exactly one year ago, things were still at the very beginning. Since then, the efforts to introduce Component Systems to the GUI window hierarchies
of 2D and 3D GUIs, and to the game entity hierarchies
of game maps made very good progress. In fact, I was so busy with and dug into my work, that I didn't get down to actually talk about them.
The good news is that it is now perfectly clear that the new Component Systems are a very important step for the future of the Cafu Engine. They give joy and pleasure every day that I work with them.
So I thought that this was a very good time to briefly reflect on what has happened during the past year, and to highlight a few related aspects in greater detail.The problems with class hierarchies
Before it sinks into oblivion for good, I would like to summarize where we came from and what the problems were, as this helps to understand what the Component Systems actually achieve.
An excerpt of the old entity class hierarchy.
Until about a year ago, we used to use "classic" C++ class hierarchies both for GUI windows as well as for game entities. You can see an excerpt of the entity class hierarchy as it used to be found in the (now revised) C++ API documentation
in the image to the right.
At the time that it was written, even though we were aware that inheritance is not a means for code reuse, we thought that a class hierarchy was a good idea, because it naturally seemed to group entities that have related features. And even though the implementation actually worked in the beginning, it soon turned out that there were plenty of shortcomings.
For example, at some point we wanted to augment entities to play 3D sounds, complete with Doppler effects and settings to control the volume, emission cone, and several other details. Our natural approach was to derive a new EntSoundT
entity class from the BaseEntityT
class, and have it implement all the features that we wanted. However, it soon turned out that we not only wanted dedicated EntSoundT
entities to play sounds, but many other entities (of different entity classes) as well. For example, an item that is picked up by the player and respawns a few seconds later makes a sound at both occasions. So we re-implemented the sound functionality in the item's entity code as well. And soon in the code for monsters, for human players, and several others.
It took me a while to realize it, but when I did, the result was pretty shocking: Even though the Cafu Sound System allows for minimal code in the game entities, not only were we left with several pieces of code that all essentially did the same, scattered across many entity classes. It was also very cumbersome to properly document all this (in fact we never did), to keep it in sync with the Map Editor and tools, and to deal with any special cases that occurred.
You can find another account on the the principal problems with class hierarchies in the blog post Cowboy Programming: Evolve Your Hierarchy
by Mick West.
In hindsight, I'd say that many of these shortcomings were so severe that they effectively hindered or even halted the further development of the Cafu Engine. The old code just didn't scale, as adding new features caused the combinatorial complexity to grow worse and worse. Also, the code in the Map Editor, the map compile tools and the Cafu Engine was not
the same, and keeping it in sync turned more and more into a pain. Not to speak of the special-cases that were needed here and there, and which we never managed to totally avoid.
These issues also burdened game developers with unnecessary complexity, e.g. the requirement to master C++ in order to make even small changes or to add small features, a burden which many game developers understandably did not want to concern themselves with.Two class hierarchies, two Component Systems
An example of game entities.
An example of GUI windows.
Before we proceed, be aware that in Cafu we have two
class hierarchies that are important for game development:
- game entities
- GUI windows
Game entities are the objects that populate our game worlds, such as the human player, monsters, weapons, items, ...
GUI windows are the basic rectangles of which our GUIs are built. GUIs are used both in 2D (for the Main Menu, the in-game console, etc.), as well as in 3D, where they are a part of the game worlds and used e.g. as lift controls, computer terminals, and so on.
These two systems are designed to work together, but regarding code design, they're largely independent of each other. By coincidence however, we find that both are structurally very similar to each other and share the same ideas of hierarchical setup and organization of concrete instances.
So both systems suffer the same problems as described above, and both happen to be candidates for migration to a Component System. In fact, as the GUI windows hierarchy was so much smaller than the game entities hierarchy, I started the initial tests and the actual migration with the GUI windows. This went very well, and now all the efforts are focused on the game entities. Read on for details.
Generally, all that is said above and below holds for both
game entities and
GUI windows, although I may often talk about and provide examples for only one, but not the other.The new design
After careful research and planning, I found that the new design can and should be as simple as this:
Entities → Components → Variables
Let's look at each of these elements in greater detail:Entities
All entity instances are now of the same common class: EntityT
. No inheritance, no class hierarchy. Essentially, entities are now nothing but lists (or "containers") of components.
A new aspect however is that entities can now have children, or "sub-entities". In fact, entity instances
, don't get confused!) can now be hierarchically organized.
For example, consider a car that consist of a chassis and four wheels, or a complex lift system that consist of a moving platform and a set of moving doors at each floor. It is now possible to have an EntityT
instance that represents the "whole thing", the entire car or the entire lift, possibly with a set of components that define features for the whole, and
to have child entities that represent the parts, where each part is another EntityT
instance that can contain components of its own.
The same is found (and possibly feels a bit more natural) in GUI window hierarchies: We often have a window that acts as a dialog frame, and child windows of it are used for adding details like the dialog message, text input fields and the "OK" and "Cancel" buttons.Components
Each component implements exactly one feature, and each feature is implemented in exactly one component.
For game entities, we have (at the time of this writing) these components:
- Basics (name, status),
- Transform (origin, orientation),
- Collision Model,
- Point Light,
- Radiosity Light,
- Particle System,
- Player Physics,
- Script (custom scripted behavior),
For GUI windows, we have these components:
- Basics (name, visibility),
- Transform (position, size, rotation),
- Border (frame),
- Image, Model,
- Choice, ListBox.
This may not look like much, but even though we're not finished yet (the lists will still get a bit longer), all previously existing functionality can be re-implemented in terms of these components. I take this as a sign that the key decision and design are powerful and correct.
Generally, components are similar to a code design pattern: They naturally group things together that belong together, and encapsulate code and features in a manner that is concise and clear. Above all, components eliminate the combinatorial complexity that occurs with adding features in classic class hierarchies. As we will see in the next section, components also break up other old weaknesses and bring very worthwhile improvements.Variables
Each component has a list of variables. This may sound like a note of trivial notability, but in fact I came up with a very nice scheme from which our Component Systems draw much of their power: Entities can iterate over their components, and components can iterate over their variables. On this basis, it was possible to implement features such as:
- automatic dialogs in the Map and GUI Editors (see example below),
- automatic serialization to and from disk,
- automatic serialization and synchronization over the network,
- automatic generation of documentation,
- automatic binding to Lua (all variables are accessible by script),
- automatic ability to interpolate all variables (of type float, double, or composites thereof).
In other words: Adding a new variable to a component imposes no extra work at all
to get all of the above!
Isn't that absolutely rocking cool??
These virtues were achieved by employing the "Visitor" code design pattern for implementing the variables (which, when you look at the source code, are actually small classes of their own).
Note that the motto "everything can be interpolated" is twofold: One aspect is "real" interpolation, where for example we move a platform from A to B, without concerning us with the intermediate positions along the track. Another aspect is the interpolation on the client, where we interpolate over client frames to fill the larger gaps between server frames, achieving a movement that is as smooth as possible, and independent of network latency that possibly affects the arrival of server updates.The Component System for GUI Windows
The components of a window in a GUI.
As mentioned above, I started with the Component System for GUI windows. The goal was to re-implement the then existing functionality (and optionally to improve and expand it) in terms of the new Component System. This goal has long been reached, and it was a huge success (see ticket #125
When I was done in March 2013, there was only one problem left: I was eager to get started doing the same with the game entities. So eager in fact, that I didn't want to waste time with anything else, such as expanding the GUI system (adding new components was so easy then that it was very tempting to add ton of new features!), or mundane tasks like writing proper documentation. (The new reference documentation was of course a piece of cake, you can see it here
. I'm talking about the introductory documentation in the Wiki, with screenshots and well-phrased prose texts...) Well, I'm sorry to say that these things are still not done, and unless someone is willing to help, are likely to stay that way at least until I'm done and happy with the Component System for game entities. More benefits
Component Systems remove the complexity from combining features, providing both more flexibility and more capabilities to game development.
But there is more: With our implementation of Component Systems, we can finally use the same code everywhere
: from the Map Editor over the map compile tools to the Cafu Engine, they all use the same
code now! For example, in the Map Editor, we must support editor-specific features such as "selections" (objects clicked with the mouse for further editing). But now, such support is only a matter of adding another Component to the selected entity! That's all. All other code is the same as that in the map compile tools and in the Cafu Engine.
A similar aspect allows for unified script states. Previously, script states had a tendency to get fragmented, that is, it could be difficult to access the state of the game world from the script code of an embedded 3D GUI. My implementation of Component Systems unifies script states as well.
Important for many game developers, there is no longer a need to write C++ code (if you don't want to). There is full
flexibility and full
power available with scripts. (And those who love C++ get their money's worth as well – hacking Cafu in C++ is more fun than ever!)
And finally, we're able to create very fine scripting reference documentation mostly automatically. As indicated above, we can iterate over components and their variables. This allows to generate complete Doxygen documentation automatically. To improve this even further, it is of course also possible to augment the generated documentation by hand.
The strings that are used to document the components and their variables are available in CaWE (e.g. the Map Editor) as well! That means that the documentation is not only available as a nice online web resource, but is readily available in CaWE, too! You can see an example of this in the "Window Inspector" screenshot at the bottom ("hor. Align"). It is even possible to query the help string for a variable at the in-game console in Cafu, or from any other script code, if you wish.References
For historical reference, here are some related forum posts where Component Systems were previously discussed:
Many thanks to all those who patiently kept explaining things to me (and in the beginning kept convincing me
) and who participated in discussions and provided suggestions!Ongoing work
Fireworks at my home town, 2014-01-01.
In summary, the new Component Systems provide elegant solutions to old problems. They make game development much easier and much faster, a lot more flexible, and more fun than ever.
However, we're not done yet: the Component System for game entities has made excellent progress, but there is still work to do with porting old DeathMatch game code to the new system. There is also a lot of documentation to write (those introductory and prose parts that don't automatically write themselves...), and many pages of the website to update (Features, Gallery, ...). And once that's done, when the new Component Systems are in place and all obsolete code is gone, a new release is due, and the real fun begins: We can then add new features that we couldn't add before, add new scripts that we didn't have before, and so much more.
If you're interested in the details, have a look at the Cafu source code repository
, where currently the active work is in the entity-component-system
branch (which will be merged back to master
I wish everyone a Happy New Year and all the best for 2014!
Post and read comments about this news item at the related forum topic.