Enzyme: Year 1
Iím going to take a short break from the coding to write up this post, because weíve hit a milestone. Enzyme turns one year old this week (in fact, as of tomorrow). I base that on the very first Subversion commit I did for the project, which happened on March 15, 2010. Of course, I did a bit of design work prior to that first commit, and itís likely there were a few days of setup and initial coding. So in actual fact, the current incarnation of Enzyme was probably born a few days earlier, but I donít have a record of what happened over that time, so weíll just call March 15 Enzymeís birthday, and leave it at that.
First, a bit of explanation on the name. Some of you may or may not know this, but an enzyme is a protein that catalyzes (i.e., increases the rate of) chemical reactions. In other words, enzymes donít do anything that wouldnít happen anyway, just at a much slower rate. Which is a perfect analogy for a game engine. Given enough time and money, a good designer will eventually create a game. Given enough time, money, and a good engine, a designer will do the same thing, only much faster. Given my background and degrees in science, I thought it was a pretty clever name.
There have been a total of 89 subversion commits for this project, which works out to roughly once every 4 days or so, on average. I probably ought to increase that some. Losing 4 days of work to a hard drive crash would be pretty nasty. So Iíll try to be a bit better about committing a little more often than I have been.
So letís check out what weíve accomplished in the year.
March 15 Ė April 3, 2010 (Revisions 1-23): Documentation
The first thing I did wasnít really part of the engine at all. At least not directly. The first system written was a documentation system. Documentation is hard, even with the best of intentions, and I decided the more it could be automated, the better chance we had of keeping it up to date. With that in mind, I wrote a program that would scan all the header files in a particular directory (and subdirectories thereof), pulling out keywords from them and turn that data into fully formatted web pages that could be uploaded en mass. All I have to do is keep the header files properly documented with the keywords, and the program does the rest (creating a Table of Contents, then section pages, then class pages for each section, then function pages for each class). This is a good little program to have, and Iím still using it. So that was our first three weeks.
April 5 - April 14, 2010 (Revisions 24-30): Linked List
The next two weeks were devoted to creating our first container, the linked list. It may seem like a kind of strange place to start, but I had a good reason. The original design for the memory manager used linked lists (in the end I didnít end up using them for the memory manager, but at the time linked lists seemed like a prereq for that system). Besides, containers are always useful to have, and they are very basic programming exercises, which usually donít have too many dependencies, so itís actually not such a strange place to start an engine.
April 17, 2010 (Revision 31): Hash
Another container, although this one was only really a stub with one public function. I havenít actually finished this container to this day (hasnít been necessary yet, though it will be sooner or later). What I really needed from it was the ability to turn a string (or really any segment of memory) into an integer for fast lookup (it is much faster to find an integer than it is to find a string). Again, this was a prereq for the memory manager (which this time I actually ended up using).
April 19 Ė May 13, 2010 (Revision 32-37): The Memory Manager
This was our first major subsystem of the engine. The purpose was to replace the OS specific heap allocator with our own custom allocator. This is a pretty dangerous thing to do, as you are replacing a well tested system with one that is completely untested, and which, if it has bugs, could be a nightmare to track down and fix. Still, the rewards are quite satisfying if you can get it right. Heap allocations happen; itís a bad idea to tell game programmers ďDonít use the HeapĒ. Even if they make a concerted effort to not use it, the engine will still use it (for instance, containers do a lot of heap allocations). So I made the decision that something had to be done about the slowness of the OS specific heap allocators.
It took about a month to put this system together, but the time spent seems to have been well worth it. I clocked the speed of the custom heap allocations at about 100x that of the Windows allocator. An extra benefit of this system is it is impossible for the engine to leak memory (as long as the programmer stays within the confines of the manager, if they use malloc, they are on their own). Once the engine is shut down, everything is guaranteed to be released from the heap. It also has a built in defragmenter, the cost of which is spread out over lots of frames, so you wonít get hitches in your frame rates (though to be honest, the defragger has yet to be adequately tested, I need screen text for that). Pretty good stuff.
May 14 Ė May 17, 2010 (Revision 38-41): Arrays
With the memory manager done, we were now free to develop more useful containers. One of the most useful is the dynamic array (those of you familiar with the Standard Template Library would know them as Vectors. I chose not to use that word, because vector in 3D programming means something else entirely, so I went with the less ambiguous Array). For those of you that donít know the STL, a dynamic array is simply an array that has no upper limit. You can keep adding more items to it until you run out of RAM (or in our case, fill the memory block). It can be used just like a regular array at need (which simply means the items in the array are stored contiguously in memory at all times), but with the ability to move itself if it needs more space for itís elements.
May 26 Ė June 28, 2010 (Revision 42-45): Strings
Wow, I worked on strings for an entire month and only did four commits for them. Bad programmer! It goes without saying strings are an incredibly useful thing to have. Strings are all overloaded to have lots of useful operator functions, like << and + and so forth. The trickiest bit for strings was the conversion between wide strings and narrow strings. Thatís all done behind the scenes within the class, so any string you set up will have both versions to use as you need them (some functions require narrow strings, so they are still needed, even though they are being phased out slowly).
July 10 Ė September 12, 2010 (Revision 46-56): Miscellaneous
I did various things during this time, which included some work on initializing the five different renderers, created the time class to handle time and timing functions, coded in the Vector, Matrix, and Color classes to handle various 3D math functions, and added the Camera class, with the ability to handle multiple cameras. I also did some preliminary work on the materials class during this stage.
I lump all this stuff together because none of it took all that long individually, they are all pretty basic classes. Put together, however, I spent two months on these guys. All of this was done in preparation for getting the renderers put together.
September 27, 2010 Ė February 7, 2011 (Revision 57-74): The Renderers
This, of course, is the big one. The renderers. According to that time line, it took me four and a half months to get them put together. That seems a little short to me. If you include the time spent on the initializations from the prior set, itís probably closer to six months in reality. We have a total of five renderers in the engine, which should allow us to support a wide range of hardware. The design philosophy behind the engine for this part of it was to wrap all the renderer specific calls in one class (so there would be five classes altogether) with a controlling manager class. The manager class should be the only class that knows which renderer is active, any class outside of these wrappers should neither know, nor care. Any class function calling into these wrappers has to do it in such a way as to be totally generic.
Originally I had a gargantuan manager class, which took calls from outside the wrapper and funneled them into the proper renderer. However, ultimately, I took out most of the manager class and replaced it with a pointer, thus allowing outside functions to call directly into the proper renderer, without them knowing which renderer they were going to end up in. Itís all pretty complex, but it works fine. You gotta love polymorphism.
February 12 Ė February 26, 2011 (Revision 75-84): Plugins
I spent a couple of weeks putting together the framework for plug-ins. This only really gets tricky because you need communication both ways (between the plug-in and the engine, and visa versa), and since the engine has to load various plug-ins, there needed to be a standard set for the plug-in interface. It wasnít really all that hard, but it was a knot to figure out how to best do this. To be honest, Iím still a little uncomfortable with one aspect of this system, which I may try to work out a little more in the future.
February 28 Ė March 6, 2011 (Revision 85-87): Font Builder
Having put together a format for the plug-ins, I turned my attention to creating the first one. The job of this plug-in is to grab a font and drop it to a bitmap for use by the renderers. Nothing particularly hard here, at least in theory. This system is mostly done, though there are a few odds and ends to clean up. But it hasnít yet been tested, that is coming soon.
March 12, 2011 - Current (Revision 88-89): Texturing
This is the system Iím working on right now (or would be, if I werenít writing this post). It is complete for four out of the five renderers, so we can call it just about done. Another day or so ought to do it. Of course, texturing is just the act of wrapping a 2D bitmap over a 3D object.
Whew. Quite a bit weíve done over the last year. Especially given just about half of it was spent on just the renderers. But we still have quite a long way to go.
So where are we going now? The next year should be a very interesting one for Enzyme. I wonít try to predict how far weíll get (we all know my track record with making predictions for features/releases/etc). But I will say that over the next year, we should be adding a LOT of new systems into the engine, and it should start to take on the appearance of a useful game development tool. I hope by the time I am writing next yearís wrap-up, the potential in the engine will be pretty apparent if perhaps not entirely realized yet.
In the short term, Iíll be finishing up texturing. Then we will have to move over to the resource manager and get at least one resource set up for load/save (the font file). Once that is done, I think we will be ready for our first UI elements (static text).
Beyond that it gets a little more murky. Iíll probably need to start designing the engineís UI. Menus, input boxes, buttons, and so forth. We shall need shapes (primitives) and ways to manipulate them. Over the course of the next year I want to try to start dealing with the high-level part of the renderer, so we can start thinking about things like portals, occlusion culling, viewport culling, sorting renderables by material, LOD, and so forth. And of course, there is far more to materials than just slapping a diffuse map over a wireframe. Weíll need to think about normal maps, specular maps, and all the other maps that make materials look good. And that is going to involve some shader manipulation, so weíll need some sort of system in place to allow the user to write their own shaders.
Thatís a pretty good laundry list of stuff, and no doubt Iíve missed things that are prereqs to some of them. But if we can clear that list in a year, Iíll be pretty pleased (and look there, I did exactly what I said I wouldnít, I tried to predict future feature timelines. Dammit).
Even with all that, would it be enough to begin a game? Probably not. Weíll, we could start one I suppose, designing static geometry for the levels (presuming there is a save system in place to save and load the levels). But without scripting, animation, sound, lights, particles, and probably a bunch of stuff I am forgetting, it would be a pretty boring place. But itís a start.
So that is where weíve been, and where we are going. I have to say, ďSo far, so goodĒ. Iím quite pleased with how everything has come together up to this point. We have a very solid foundation. If the next yearís worth of code turns out as good as this yearís (and I have no reason to believe it wonít), then we should have a damn fine engine on our hands.
"They laughed when I said I was going to be a comedian ... They're not laughing now." - Bob Monkhouse