When I released the previous version of UQE Hexen II with additional higher and widescreen resolutions people pointed out that the field of view (FOV) is still set at a fixed 90 degrees which is typical for 4:3 letterbox resolutions. When I investigated the FOV implementation in the engine I decided to port the original Quake engine’s FOV setup code over to Hexen II as it seemed as if its implementation pretty much fixed it to 90 degrees. Changing the FOV on the console did not change the field of view at all. After some work on the subject I succeeded in changing the code to actually calculate the FOV value dynamically based upon the currently used resolution.

While I was at it updating the engine’s FOV code I thought to look at a few other minor issues and upgrade parts of the engine that was getting out of date. The bloom code had quite a bit of an update since I heard that the bloom effect were too excessive. I took the liberty to get it cleaned-up a bit and lessen the strain on the eyes which I hopefully now got sorted out. UQE Hexen II generates the bloom effect using the “GL_EXT_framebuffer_object” OpenGL extension. I ended up splitting the frame buffer management code from the bloom code to allow more frame buffer effects to be implemented in the future.

One thing that bugged me a lot with the frame buffer is that I couldn’t get Multi-Sampled Anti-Aliasing (MSAA) to render at the same time as bloom. Once bloom was turned on the jaggies appeared that really broke the softening effect that bloom adds to the scene. I had to do some research on the subject and learnt that you should create an additional frame buffer using the “GL_EXT_framebuffer_multisample” extension. You are pretty much guaranteed that the “GL_EXT_framebuffer_object” extension is supported if the GPU supports “GL_EXT_framebuffer_multisample”. The process of getting MSAA-enabled output for further post processing is pretty simple… the engine firstly needs to render the actual scene to the MSAA frame buffer then the content of that buffer gets copied to the standard frame buffer where the frame buffer data gets processed to the final scene as normal.

Some third party libraries also got an update with this release.
The JPG library has been updated with the one used by Quake III Arena which for one allowed the loading of JPG data via a stream of bytes which helped a lot with the recent implementation of PK3 (zip) loading support. Did I mention that UQE Hexen II can now load game assets from either PAK or PK3? The engine can also load game assets even from a mix of PAK and PK3 files. The code responsible for loading data from PK3 files have been directly ported from Quake III Arena as well.

The FMOD middleware library implementation in UQE Hexen II also got a huge upgrade.
It got completely re-written from scratch using the at the time of writing the code version of FMOD. Theres been quite some logical changes apart from only the technology upgrade. The engine now also checks for WAV files as an alternative music playback format if it can’t find a suitable OGG or MP3 file. The dated WinMM-based CD Audio playback code have also been replaced with brand new FMOD code that now manages the playback of CD Audio and the same has been done to the native MIDI implementation. Moving all music playback functionality to third party middleware saves some legacy code from being part of the engine and probably might even help reduce the possibility of legacy issues rearing its ugly head at some point in the future as all source code tend to decay over long periods of time.

With the release of UQE Hexen II v.1.15 it will be the first time the software renderer will be completely omitted from the project as it serves no real purpose anymore and becomes pretty difficult to maintain and keep it error-free. I did however made some changes to the remaining OpenGL version of the engine to enable it to simulate the classic effect the software renderer has given by simply changing some console variable values. To help with simulating the unfiltered effect of textures in the software renderer I’ve added a “Point Sampled” texture mode that can be set from the menu. To get the best classic look and feel for the game will be to set the texture mode to “Point Sampled”, turn off the frame buffer and then starting the game at a smaller resolution like 640×480 for example.

There are numerous more additions and tweaks to UQE Hexen II, but they are too minor to write about.

Download UQE Hexen II and enjoy it!

If all goes according to plan I should be able to start with the development process between December 2011 and January 2012. All the features that are currently available in UQE Hexen II v1.14 will be ported over to UQE Quake. Obviously only the features that are compatible between the two game engines will be ported.

Once all the features intended for the initial release is ported I will look at a few other odd issues, like for example the field of view (FOV) changes that have to be made to accommodate wide screen monitors. The additional fixes and changes will also be back-ported to UQE Hexen II.

Unlike Hexen II there are a huge number of ports and modifications to the original Quake. So whats the purpose of UQE Quake?

Well, its pretty simple. When I started the UQE project back in 2006 I wanted to be able to play these older games as close as possible to its original, except for a few fixes and additions. Originality is the key, with its strengths and weaknesses. The UQE Quake project will be no different. If you were too young to remember what Quake was all about and for that matter Hexen II as well, the UQE versions of these games will set you on the right path.

Whether UQE Quake will have a software renderer just like the UQE Hexen II project is debatable since I’m not sure how many people are still actually bothered having a software renderer. It should be possible to emulate the feeling and atmosphere the software renderer had by adding point sampled texture rendering to the OpenGL renderer. If that works out well, the same could be applied to UQE Hexen II.

The UQE forums is the place to be to follow the developmental process as well as contribute to the project.
http://forums.korvinkorax.com/viewtopic.php?f=11&t=2
http://forums.korvinkorax.com/viewtopic.php?f=11&t=4

For the longest time since I created levels for DOOM and DOOM-based games I wanted to get into the technical side of the idTech1 engine, but sadly I never got around that. I don’t know why it turned out this way. Maybe because there’s never enough time, or maybe because I was so much involved in the Quake engine development scene.

I always wanted to build a DOOM-like game. I have a great appreciation for the technical design style and fast game-play. To save valuable time I decided to use DOOM’s original art assets while I develop the game engine to spare myself the extra effort and management of new original art asset development. Using original DOOM assets will further help doing a more direct comparison against the original DOOM.

Then my first anticipated problem steps into the limelight… I’m going to use Microsoft .NET and Microsoft XNA for the project. That means for XNA best practices I should use its content pipeline and not write my own WAD reading code and content management. It is totally possible technically to write my own code to load DOOM assets in the game engine, but that would make my code probably less compatible when thinking along the lines of making versions for other Microsoft platforms that are not PCs. It becomes obvious that the assets stored inside the WAD file needs to be extracted and written to disk as standard multimedia assets that can be loaded and compiled by XNA’s content pipeline to its binary XNB file format.

I searched the internet for a tool that could extract the assets and I found one developed by Terry Butler.
His tool did the job well to extract the data from the WAD, but unfortunately there were no source code available for download on his website. I realized I had to actually have an understanding how the WAD file structure functioned, because at some point in time I may need to write another tool that can package standard art assets back into a WAD for level design purposes as I’m planning to most likely use DoomBuilder as the level editor. I started doing some research on the internet regarding the WAD file format and came across a great specification document written by Matthew Fell detailing not only the WAD file structure, but also the individual lumps of data that make up the assets of the game.

The formats of the individual art assets are pretty interesting…

The digital sound format were pretty easy to get loaded. It is stored as raw sound data in mono channel at a sample rate of 11025 hz, 8-bits per sample. All we had to do is to take this raw sound data and write it to disk as a standard wave file with a custom wave writer built into the project.

The graphics format is stored in two different formats.
The floor and ceiling textures are all 64×64 pixels stored as raw 8-bit pixel data, which is also known as “flats”. All other graphics are stored in another custom format which seems to be stored as columns of pixels. My take on why this is the case is that all textures that are stored in this custom column-based format are all displayed on the screen either as 2D images or are rendered in the 3D environment vertically as opposed to “flats” that are rendered horizontally. It is probably a speed optimization feature of the original DOOM engine to allow easy access to columns of pixels in a texture to render, because the engine itself does ray-tracing which draws columns of pixels to the screen with probably little to no overdraw.

Even drawing the columns themselves are quite optimal. No CPU power is wasted on transparent pixels. Each column of pixels may be made up of one or more “posts” of pixels. A post is basically a starting offset where to start drawing pixels within the column followed by how many pixels to draw downwards. If the following byte after the last pixel of the post has a value of 255 it marks the end of the column, but if it doesn’t have a value of 255 it marks the position of the start of the next post within the column.

The level data gets loaded into a custom data structure and saved to disk as LEV files. These files consist of a header which is followed by the level data lumps and closer to the end of the file a directory structure that contains information regarding the name of each lump, its offset within the file and the length of byte data following the offset position. The lump data is saved exactly as its read from the WAD file. The level data doesn’t get visualized within the tool at the time of this writing.

Most of the lumps that are not recognized as graphics, levels or sounds are not visualized or extracted to disk at the moment, but this will be fixed with the next update to the Doom WAD Reader & Extractor.

Special thanks goes out to Matthew Fell for his excellent Doom specifications document which made this tool and source release a possibility.

Download the latest version of the Doom WAD Reader & Extractor project.
Doom Engine WAD Reader and Extractor v1.0

This marks the beginning of a new way of thinking and a new direction when it comes to the creation of art be it of a programmatic technical nature, a musical nature or even visuals. For far too long have I been trapped in a world of code only. Its time for the world of the inner self to come out and become creative… the way it always were before the code. Its time to rise above the world of the temporary, weathering world of code and also enter the more permanent, timeless world of the arts.

Most of the content currently on the site is technical of nature, but this will change soon as time progress until there is the good mix between the technical and the artistic.

Trent Reznor, Rob Zombie, Marilyn Manson as well as John Carmack have been great sources of inspiration over the years and the creation of this website and what I’m going to do with it as well as myself will be a testament towards the well of inspiration I’ve drawn from them since the start of it all.

While on vacation I thought it will be cool to quickly convert the technology from Microsoft .NET 3.5 and Microsoft XNA 3.1 to .NET 4.0 and XNA 4.0, but great was my disappointment when I realized that it won’t be a quicky and that there were significant breaking changes introduced in XNA. At least these were all good breaking changes, but none the less it still meant I need to go through the technology with a fine tooth comb and fix the now broken bits of code.

After spending days reading loads of Shawn Hargreaves blog posts regarding these breaking changes and fixing code I finally managed to restore the idTech2 XNA Renderer to a working state. Some of the most challenging issues were changing the actual rendering of some of the existing geometry as it seems they dropped support for using triangle fans which forced me to rewrite all the logic where triangle fans were used in favour of triangle lists.

The other tricky change was the way rendering effects got changed which also caused the bloom post processing to break. So after around a week or two’s work the renderer was back to its original rendering state. I also found a few minor bugs that also got fixed and quite a number of performance improvements were done.

Another problem that’s currently not fixed involves Shader Model 3.0 and the bloom post-processing effect. Since the bloom post-processing effect only uses a pixel shader and no vertex shader, running it via SM3 generates the error message “Cannot mix shader model 3.0 with earlier shader models. If either the vertex shader or pixel shader is compiled as 3.0, they must both be“. I’ve checked the internet, but at the time of finishing this version of the idTech2 XNA Renderer I couldn’t find a solution to this other than using the Shader Model 2 path.

There were quite a number of new features added to the technology, but let’s start with what got dropped-out. That stupid XML configuration file was removed and made way for a proper application configuration file. Thinking back I really don’t know why I didn’t do this from the beginning. One of the real big internal changes in the renderer is the way it loads the BSP files. This was needed because an entity system was necessary to manage the loading and rendering of MD2 models.

After doing the groundwork and researching idTech2′s entity system the renderer sports a 99% complete idTech2-based entity system which manages the loading and rendering of two out of three types of models namely brush models and alias models. Sprite models isn’t implemented yet. Brush models are any world geometry be it static worldmodel geometry or geometry the user can interact with and that isn’t static. Alias models are any MD2 models representing objects, items, monsters etc.

The MD2 model (Alias model) loading and rendering code was done in a record 24 hours, but getting it rock-solid took a further few days with the lighting being properly applied to each model only a few weeks later. The brush model loading and rendering code was significantly changed because in the previous version of the renderer it only loaded the static worldmodel and rendered that via BSP, but within the worldmodel data there are sections of geometry in the data stream that is meant to form part of an entity similarly to a MD2 model.

This geometry data is called submodels. These submodels makes up all the interactive parts of the static BSP world like breakable parts, rotating parts, moveable parts, doors, lifts etc. With some game code (logic) implemented eventually these seemingly static submodels will add some mechanical life to the BSP world.

Not only is the idTech2 XNA Renderer capable of rendering Quake2 data, it’s also capable of rendering Heretic II data. Since the Heretic II game is just a heavily modified Quake 2 engine it didn’t take much extra work to get it to render. Infact, Raven Software didn’t do any changes to the BSP file structure and it was possible to directly use the Quake 2 BSP tools to build Heretic II BSPs. Most of the work involved implementing Heretic II’s M8 texture format which is only a modified Quake 2 WAL texture format.

Adding support for Heretic II’s M8 format enabled the renderer to be able to load and render Heretic II BSP files. Heretic II’s custom model format isn’t currently implemented, but maybe the next version of the idTech2 XNA Renderer will see that done.

There are quite a number of entity effects which isn’t currently implemented in the renderer, because most entity effects relies on game code (logic) which is typically found in Quake2′s gamex86.dll file. Some examples of these effects are model animation, rotation as well as the application of gravity. All model positioning and rotations in the renderer are expressed as design-time information which means what you see on-screen is actually where they are placed in the world before the game logic takes over. At times these positions and rotations might look strange.

Another example is the static nature of submodels like doors, lifts, fans and other non-static world geometry. Since these submodels are attached to entities they are also rendering without any game code driving them.

Download the latest version of the Quake 2 XNA project.

This new and second release of UQE Hexen II came almost five years since the very first release back in 2006 and it boasts a number of really great additions and fixes to the engine. Here we will highlight the most important ones, but for a complete list of new features and changes see the change log.

When it comes to the technologies used; I have updated the DirectX libraries to the latest and compiled and linked to them as well as updated the OpenGL extensions header files to make sure the code is aware of any new extensions that are available. The engine have also been compiled with Visual Studio 2010 which in itself fixes a lot of issues with the binary engine. All 16-bit video modes have been removed from the engine to create room for higher resolutions up to HD resolutions.

The 2D rendering logic managing things like statusbars, text, menus and the likes had to be updated to allow for the proper scaling there-of since the addition of HD resolutions makes the 2D stuff to become ridiculously tiny. The 2D scaling logic in the engine has been significantly improved. Also featuring is the ability to load non-power-of-two textures which fixes that “corrupted” or “malformed” look the engine shows which is aparent with most of the 2D graphics when compared to the software version of the engine. The OpenGL renderer now match perfectly with the quality of the software renderer in that regard.

External texture loading support have been sigificantly improved. Not only can bump mapping textures be internally generated, but the ability has been added to allow authors to load their own bump mapping textures. Normal mapping would be a more correct term. External texture loading was also simplified a lot by not forcing the loading of Hexen II WAL files. External luma textures are also supported now to allow “glowing” texture overlays. It’s also possible now to load external sky textures for classic Hexen II style skies. Another nice addition is the support for colored lighting via external .LIT files to give the world a more natural feel.

A feature people have been asking for ages to be added to UQE is interpolation. Well, the good news is that its finally and fully implemented. There are three major interpolation technologies implemented in UQE Hexen II v1.14 now. Translation interpolation smooths the positional movement of entities. Rotation inperpolation smooths the rotations of entities. Animation interpolation smooths the rendering of MDL model animations attached to entities. Translation and Animation interpolation are enabled by default with only Rotation interpolation which is disabled by default since I’m not 100% satisfied with its results. The documentation that comes with the engine contains all the information on how to enable/disable any of these interpolation methods.

The nicest feature of all that has been added to the engine is frame buffer objects using the OpenGL extension “GL_EXT_framebuffer_object”. With this the engine now features an all new bloom post-processing effect, written from the ground up to replace the dated bloom method used in previous UQE releases as well as other engine mods. The full power of the GPU is harnessed to process bloom using GLSL. Not only does it look better, but it also performs better. The addition of frame buffer objects also opens the floodgates to even more impressive post processing features that could be added in the future like for example motion blur!

This new bloom technology is my contribution to the Quake community. Read on the change log for more information regarding what has been done with this version of UQE Hexen II.

Download UQE Hexen II and enjoy it!

This is the utopia, but it is not possible to accomplish this in a decent timeframe.
On the one hand you want to build technology that is competitive regarding capabilities to other engines or renderers out there, but on the other hand you want to remain faithful to what the technology should be able to do regarding the UQE project. Ultimately its impossible to build a single technology that looks into both directions without having a dated or bloated design. We decided the best was to go, as Corvus Games, is to utilize XNA for simpler arcade-like titles and for larger titles license a commercial engine like Torque3D and the likes.

So, why are we still working on the Quake2 BSP Renderer?
Since we spent so much time getting it as far as it is, we decided to finish-up the project and release the source code using the GPL license to help other like-minded people to gain access to the source code and hopefully learn from it. Its a much better proposition then stuffing it into our archives to gather dust! There is also another twist to the story; The project turned out to be an experimental renderer for our efforts over at our “idTech” community project, UQE. We are still looking at writing a .NET/XNA version of the idTech2 engine making as much as possible use of XNA to faithfully render the Quake2 game. We will also be looking at building loading and translation paths for Quake1 and Hexen2 at a later stage should the Quake2 project be successful.

The UQE project is a very personal project of mine and with it I’m sharing my passion for the idTech engines with both gamers and game developers interested in the same subject matter. UQE is not one of our primary game development objectives and its development is being managed as time permits us to work on it. The UQE project is great for expanding our knowledge on XNA and helping us build and use game subsystems we can re-use for our XNA targetted projects. It also stands as proof for anyone out there that game development and its related technologies is completely possible in Africa and in showing she is not as dark and illiterate as the media makes her out to be. As far as my research stretches at the time of this writing, Corvus Games have developed THE most complete XNA-based idTech2 BSP renderer out there. Yet, sadly, our renderer is not complete, it still lacks entity (MD2 and submodels) rendering as well as rendering static lightmap styles.

Lets get right into the technologies the renderer consists of and what you could learn from it…
When the renderer starts-up it loads the BSP level specified in its config.xml file. If this file doesn’t exist, it creates one with some default values. The configuration system implemented is a very quick and dirty one just to allow you to change a few variables that would otherwise be hardcoded.

The very first thing Q2BSP does is either load the assets directly from the disk as single files or it loads the assets from the PAK virtual file system. The code managing the PAK file is very basic and just manages the loading from a specific PAK file and is not a fully fledged virtual file system… yet. Even though optionally loading from a .PAK file is great, I think its not the most optimal solution. For a custom virtual file system to work you must be able to stream any data from it for use, but I found it difficult to near impossible to stream a wave file into a form that can be played back by XNA.

I did some reading up on the .XNB format, and it seems it can store multiple files and the XNA API can easily access that file to read data from it. Since we don’t want to re-invent a feature that seems to be present, the best thing most likely to do is to build a little conversion tool that can read PAK data and compile XNB data from it, then use the XNB as storage mechanism rather than PAK files.

For this project you’ll be able to load any BSP level contained inside a PAK. We have tested it with the pak0.pak file of Quake2 full version, but it should work fine with the demo and other custom PAK files. The only requirement for the loading from a PAK is that ALL assets that has to be loaded needs to be in that very same file, or everything must be external files if the PAK loading option is disabled. Thats how basic the PAK implementation for Q2BSP is.

With the BSP file loaded the renderer is ready to render the world. The whole world is broken up into surfaces. A surface is a set of vertices that defines a piece of flat geometry. A surface also has a fixed maximum dimensional size limit to easily be broken up for rendering by the PVS and if also an effect of BSP. This means a single flat large wall will be made up by several pieces of surfaces. A surface can have several primitives.

Because of BSP partitioning a leaf can have several surfaces attached to it that forms a piece of geometry. A leaf could for example be that complete large wall, or just a chunck of it. Next we get something we call clusters, or to put it in easier terms, leaf clusters. A cluster is simply a set of possibly visible leafs. So what happens is when you have a specific position in the world, BSP traversing is used to determine which cluster you are in. Once we know the cluster number (index) we look up the PVS and get a list of leaves visible for that cluster. This set of leaves gets drawn to the screen.

If you want to visualize this in your mind’s eye, you can imagine a cluster being a volume of space could be big or small, depending on the geometry, you could move you camera in without the geometry PVS changing. Once you move from one cluster to a neigbouring cluster the PVS needs to be re-checked and a new set of leaves needs to be drawn. What is notable is that clusters that are closer together share relatively the same set of leaves. Its very possible that 5 neigbouring clusters would share 70% of the same leaves, depending on how the world is constructed.

One of the performance drawbacks that still remains to be solved is to optimally sort the number of surfaces in the current cluster in such a way that we could make use of Index Buffers and execute draw calls using the DrawIndexedPrimitives() function rather than using the DrawPrimitives() function that we currently use. DrawPrimitives() gets called one for each surface that is visible in the current cluster, but if we could use DrawIndexedPrimitives() we could greatly reduce the number of drawing calls that we need to issue. We are still looking at a few scenarios to see where the use of DrawIndexedPrimitives() works well and where it breaks functionality.

The first priority is to sort surface rendering by texture (texinfo) then from there build a triangle list based index buffer every time the camera changes PVS cluster. This will change the scene rendering from per-surface DrawPrimitives() to per-texture DrawIndexedPrimitives(). I’m not sure if it will be jittery if the camera move from one cluster to the next. The only way to know is to code it and find out. Maybe changing the code from DrawPrimitives() to DrawIndexedPrimitives() frees the CPU enough that you don’t notice. For this release the renderer will be rendering the scene primarily using DrawPrimitives() until at a later stage we update the code for DrawIndexPrimitives() rendering.

The BSP the renderer traverse the BSP nodes and mark surfaces for rendering in two distinct groups namely “solid” surfaces and “translucent/warped” surfaces. The reasoning behind this is to render the world in two phases, phase 1 renders all the solid opague surfaces in the world, then phase 2 renders all the translucent and warped surfaces. With the solid surface rendering phase it does not really matter if we re-sort the list of surfaces according to texture (texinfo) and not according to its original depth sort order. Most of the surfaces are solid surfaces, and sorting according to texture greatly improves rendering performance. We are not so lucky with translucent surfaces, because we need to render them in the order we get them from the BSP/VIS to make sure we don’t introduce rendering anomalies because of the depth and rendering order.

Surfaces with the “warp” flag set gets broken into 64-unit sized sets of sub-surfaces which we call “polygons” within the source code. The reason why warped surfaces gets subdivided is to generate more vertices for the original warped surface to make the ST texture coordinate warping effect possible. This possible large number of primitives using the same texinfo index are being rendered by a call to DrawIndexedPrimitives().

Classic Quake2 lightmapping have also been implemented using multitexturing. In the pixel shader the texture pixels gets multiplied with the lightmap pixels to produce the final pixel for output. We have not yet implemented static lightmap styles, like strobing lights, flickering lights and the likes. There are two types of lightmaps implemented in Quake, idTech2 and idTech3. They are called “static” lightmaps and “dynamic” lightmaps.

The difference is static lightmaps are pre-processed lightmaps that are fixed and also could have a “style” pre-processed with it, like flickers and strobes and so forth. Then you get “dynamic” lightmaps which gets calculated on-the-fly and gets temporarily blended with the original “static” lightmaps. To give a quick example is when you fire a rocket, it generates a pointlight that lights up the area as it passes through the air and explodes against an object, that light it generates on-the-fly (no pun intended) is a dynamic lightmap.

This dynamic lightmapping looks pretty ugly, especially with Quake and idTech2′s low resolution lightmaps. We can remedy this by not implementing all the software routines needed to generate and blend this dynamic lightmap by generating dynamic per-pixel lights straight on the GPU… afterall, we already have access to both the texture pixels as well as the lightmap pixels. In the renderer you can switch on/off a per-pixel pointlight that is attached to your camera position. What this pointlight in reality does is it “eats” the color value of only the static lightmap pixel at the pixel shader level and returns a modified static lightmap pixel, which we then multiply with the texture pixel to get our final pixel output result. The calculations isn’t 100% perfect yet.

Just for the fun of it we implemented two types of lighting; Classic lightmapped multitexturing and actual per-pixel hardware lighting. The BSP levels are not completely compatible and designed for per-pixel hardware lighting, but its cool to see it sort-of working. Theres a section of code that I left commented that anyone can uncomment which will add per-pixel pointlights for every pointlight entity present in any given BSP level loaded.

The focus of the project was to get good Quake2 BSP rendering up and running with decent performance, not focussing on things like error-checking, good object-orientated design and great performance. When decompressing the archive you’ll find a pre-built binary in “release” mode as well as the GPL source code as it stands now at its current state. Included with the binary is a “config.xml” configuration file with a few changeable settings. The build is configured to search and load from PAK0.PAK the “base1.bsp” level. All that needs to be done is to copy the PAK0.PAK of the Quake2 demo or commercial version into the “baseq2″ folder located within the “contents” folder.

When Q2BSP starts up it generates a file called “config.txt” which contains a list of the names of all the BSP files thats part of the commercial version of Quake2. This is done to help anyone that doesn’t have a PAK reader tool at hand to have a list of BSP files that could be tried out. It is also possible to load some of the deathmatch levels if the commercial version is sufficiently patched, although a PAK editing tool will be needed because the data need to be either merged into a single PAK, or everything needs to be unpacked. This needs to be done since the Q2BSP renderer doesn’t have a fully fledged Virtual File System.

Some other features available in the Q2BSP renderer is the ability to switch between solid and wireframe fill modes. The PVS can also be locked to help developers see where the PVS ends from a selected area. Also featuring is the ability is switch the bloom post-process effect on/off. The bloom post-process effect itself is slightly over-emphasized in this experimental renderer, but it demonstrates what the effect can do to make this classic game media slightly (ever so slightly) more current or at least in the right direction.

Download the latest version of the Quake 2 XNA project.

Most of my time was spent decoding the MDL format and making sure that all data I load DOES make sense because theres no ways you can during the process know if you are doing it right or not, unless you are willing to spend the time to echo your results in a custom renderer to visualize the data. I’m too lazy for that type of effort.

The past few days I’ve been spending translating the data from uqeModel’s MDL structures to its MD2 structures and calculating missing/unknown data. Funny enough, theres quite a few bits of those going around between the two formats.

Something Missing:
Quake1 was designed and released before the time of 3D hardware which means they had their own way of rendering the model triangles. I’m assuming they did standard GL_TRIANGLES style rendering. With the introduction of GLQuake they had to generate extra data for each model to have it render more optimally using triangle strips and triangle fans. To do this they had to generate something called a “gl_commands” list and saved it for rendering. They had to do that optimization back then, since 3D hardware wasn’t all that powerful and rendering GL_TRIANGLES style would have killed game playability. If you ever wondered what that “meshing…” and .MS2 files was all about… now you know!

With the Quake2 .MD2 format the “gl_commands” data was stored right in the model format, ready to be used in the engine. Worst of all is that the “gl_commands” list used in Quake1 (which the engine generated) is not the same as that stored in the Quake2 .MD2 model format. So I ended up having 5 different Visual Studio projects open at the same time to cross reference. I was looking at Quake engine code, Quake2 engine code, the Quake2 “qdata” tool coding as well as the Quake1 “modelgen” tool coding.

The command list is basically a list (array) of integers telling the engine when to render a triangle strip and when to render a triangle fan and how long they are. The list also includes “decompressed” ST texture coordinates for each vertex.

These “compressed” vertex and texture coordinates are quite interesting. The XYZ/ST coordinates was saved as “byte” values obviously ranging from 0 to 255. These “compressed” data was used to save space on model size. To get the actual floating point coordinates they multiplied these “compressed” values with a floating point “scale” value, then added a floating point “translation” value.

Generating Quake2 style “gl_commands” was particularly tricky because I needed to generate these “gl_commands” after I translated the MDL structures into the MD2 structures, because I needed to calculate them based upon the data contained in the MD2 structures.

The biggest headache was to find the vertex seams and ST coordinates since Quake1 models uses planar texture mapping. Imagine cutting tennis ball in half.

Once a “seam” vertex was found I had to duplicate the ST coordinates and “move” the S coordinate by 0.5 to place it on the backside of the model. This I had to do correctly for all affected “seamed” vertices before attepting to generate the “gl_commands” list otherwize the back-plane skin mapping gets all messed up like one of the below screenshots will show you.

Something Extra:
One of the biggest differences between the Quake1 model format and that of the Quake2 model format is that the Quake1 MDL format has two types of animation frames where-as Quake2 has only one type. The MDL model format has a frametype SINGLE and a frametype GROUPED where-as the MD2 model format only supports the SINGLE frametype.

The difference between the two types is simple: SINGLE frames are the typical framed animations that are controlled by your game logic.. no surprizes there, but GROUPED frames are a whole different ballgame. GROUPED frames in essence are looped framed animations complete with interval timing information for each frame contained in a group header which is controlled by the engine, rendering independantly from game logic.. well as far as I could have gathered at the moment.

Currently our uqeModels tool doesn’t support converting them yet, although it does load them correctly from any MDL. Once I fully understand how I will manage this in the Quake2 engine I will start writing supporting tech for them. I’m not too keen on breaking away from the original MD2 format, so I will likely try to find examples in the Quake2 game thats acts similarly on models. Hopefully I do find something.

Over the weekend I should be able to successfully render these newly generated/converted models within the engine. Without wasting more time, below are some sweet screenshots of the work in action! All the code doing the conversion was done using C#.

The original “demon” Quake1 MDL model. Rendered here in “qME” (Quake Model Editor). This is as original as it gets.

The converted “demon” Quake2 MD2 model. Rendered here in “qME” (Quake Model Editor). No skin texture available at the time.

The converted “demon” Quake2 MD2 model. Rendered here in “qME” (Quake Model Editor). Skinned this time, but the model “seam” vertices are corruping the ST coordinates.

The converted “demon” Quake2 MD2 model. Rendered here in “qME” (Quake Model Editor). Skinned with the seams and ST coordinates corrected.

The converted “demon” Quake2 MD2 model. Showing here the internal 8-bit skin exported as a true color TGA.

Quake – Pack Extractor
Extracts all assets from any given standard PAK file from any game using the Quake or idTech2 (Quake 2) game engine. The archive includes source code as well as pre-built binaries.

Quake – Tools (model & map converter)
Converts standard Quake MDL models to Quake 2 MD2 models. Also contains a tool that can convert standard Quake MAP source files to Quake 2 MAP files. The archive includes source code as well as pre-built binaries.

What have happened?
Unlike 18 December, I have split the data structures now into two distinct parts: “disk” structures and “memory” structures. The “disk” structures are only used to store data loaded from the Quake media files, in this case BSP data, from where it is used to build the more practical “memory” structures used by the game code. Apart from all the rewriting some code the new features added to the game project is:

Changing Lists to Arrays
Lists are now only used within the “disk” structures where performance is not such a big deal, since the benefits of lists are in its ability to just add more elements as you are going about loading the data, especially when you are not sure how many pieces of information you are loading.

Once the loading of information is done the data captured in the lists are copied to the “memory” structures in the desired format in arrays which is allocated at “memory” structure load-time, sized to the exact loading capacity. Arrays are about two 3rd’s faster than using lists from what I’ve gathered while testing.

We have all the performance with no wastage.

Removing Co-linear Vertices
By removing co-linear vertices we get no visual benefit, its purely a performance benefit.
To make things clear, in Quake a “Face” is not a single triangle, its a collection of triangles that form a flat surface having a single texture assigned to it.

In this case we are using a “Triangle Fan” for each “Face”.
By removing co-linear vertices we reduce the amount of “fans”/”triangles” each face would have saving us a few vertices, triangles and UV coords for textures and lightmaps to process each frame.

In the bigger picture most vertices are shared between faces, but when we talk about face data in “memory” we have to be able to deal with them individually because soon we will have PVS/VIS implemented that renders per-face, so the connection between faces doesn’t matter in the vertex sense.

This means there is a lot of faces that will have odd vertices on their edges for no aparent reason.
Imagine it as knots in a string of wool under tension, or even better dots connected by lines, but when you are done it forms a straight line.. so why do we have five dots when two will do the same job just perfectly.

Basically what happens is we test the vectors of the previous, current and next vertice, if it appears to form a line, we remove current, then we move one vertex onwards and do the same test until we are finished with the number of vertices saved for the “face” in question. When we are done, all vertices that will remain is those changing the “vector”/”direction” of the edges of the face.

Such a co-linear vertex is formed by two faces meeting on the edge of a third face forming t-junction vertex.

Quake Sky Rendering
To get Quake skies up we need to know which textures are sky textures. This is made easy since any sky texture in Quake starts with “sky”. Once we found the sky texture we should check its size which is supposed to be 256×128 in width and height. The sky textures consist of two textures actually, the first (left) 128 block is the “alpha” sky texture and the second (right) 128 block is the “solid” texture.

The first order of action is loading the data into two seperate data containers (arrays) because there are a few operations that needs doing on the raw texture data before we upload the texture data to the graphics card.

First we need to calculate the average RGB color of the “solid” sky texture and store that temporarily.
Next we need to update the “alpha” texture, but before we continue I need to explain what was done to the alpha channel; We updated the alpha channel to a value of 0 for all pixels that has an RGB of “0 0 0″ and an alpha channel value of “255″ for anything else.

Now we update the color pixels on the alpha texture to our avarage color we calculated previously where the pixels has an RGB of “0 0 0″.
Why do we do this? Thats because every dark cloud has an average color lining!
Did I say “average color lining”? I meant “silver lining”.

Seriously…

We do this to aviod dark fringes when we blend the alpha texture over the solid one.
The fringes are formed because of linear blending of the textures on the edges of the alpha.
By doing this we just “hide” the fringes better, they are still there and the cake still remains a lie.

From here on its just blending the two together and scrolling them, but that stuff we will leave until a next time.

Animated Textures
The was probably one of the more easier things to get implemented.
If you were able to load all the texture data successfully then you are pretty much 90% there.

With quake textures we have a few identifying pieces of information we can derive directly from the texture name itself. Textures starting with “sky” means its a sky texture (duh?); Starting with “*” means its a UV warping texture, like water; Starting with “+” means its part of an animated texture set.

We can derive how many frames an animation has by using the second character of the texture name out of a maximum of 10 frames normal and an optional maximum of 10 frames alternate.
They are identifiable as “+0″ to “+9″ for normal and “+a” to “+j” for alternate animations.

Alternate animations are typically used when you trigger something, like a switch, for example normally in Quake a button would flash at you red and lightred, this is the normal animation.. once you trigger it it goes green on you, this is the alternate animation which in this case only has one frame.

For this case we created links between the different frames each being an indice to the frame in the texture information as they are stored in the “memory” structures. All that needs doing now is attaching the animations to the game code’s timing and there we have it.. animated textures!

Resampling of Non-Power2 Textures
This was quite a mission to sort out especially when you are not making the needed changes all around your code. I noticed for some time that only some of the textures in a Quake BSP rendering is coming out all squashing in any of the given dimensions.
At the time I thought its probably some odd rendering bug with the way the wireframe gets built.. definitely not the texture loading code, since then all the textures should have some odd problem, right?

Wrong!

After some fishing for it, I finally uncovered the underlying issue.. some Quake textures are not Power of 2 textures! It really started irritating me when I loaded-up start.bsp and found that the coolest part (for me) of the game is messed-up, and that would be the introductory “quake” name.

The “quake” texture is non-pow2 at a size of 288×64 where it should be either 256×64 or 512×64.
But theres a trick involved here beyond the trickiness of writing a function that successfully resamples the texture; We need to store a “scaled” version of the texture width and height and rather use that everywhere where we are working directly with the texture data. This is because we still need the original texture width and height to calculate the UV texture mapping of it.

The texture will look stretched when you are sampling up from 288 for 512, but once you render it and properly UV map it again theres not much difference you can see since theres no loss that takes place other than a bit of extra texture memory. You would rather always want to sample upwards opposed to downwards, because you will be loosing pixel information even if 256 is closer than 512.
Lets now get to the interesting business of viewing screenshots!

21 December 2007
E1M1. Perfect Quake-style sky rendering that matches the original Quake.

26 December 2007
E1M1. Pay attention to the brightness of the red coloring in the animated texture on the side of this slipgate.

26 December 2007
E1M1. Pay attention to the brightness of the red coloring in the animated texture on the side of this slipgate.

1 January 2008
START. Resampled “quake” texture, one amongst many non-power-of-2 resampled textures.

Next on my list is getting some of the client structures implemented, loading the entities into the “memory” structures and then get styled lightmaps working.

Now I need to do some more work to be ready to bring more good news on my next entry.

First question would probably be: “Why Quake?”
Its very simple actually. Quake is the most simple of the modern-day polygon-based FPS games out there, with the least or none extra logic that is not the gameplay itself in making a simple FPS game.

Your second question would probably be: “Why emulate/re-write Quake?”
To set the record straight, its an emulation of Quake “The Game” and not a re-write of Quake “The Engine”. Infact, the old engine is so outdated right now and has such a lot of porting problems and design flaws in it, it will be just as hard fixing it as it will be re-writing it.

So why re-write it, emulate the game logic and media data and render it via a modern-day engine, whatever that engine will be. Games are becoming so large these days, its easy to decide on which side you want to be, engine development, or game development.. I see myself in the latter. Given the fact that lately I’m not all that much into C/C++ and being a C# convert I would much rather spend my time honing C# skills than keep on falling back to C development when it comes to Quake and general game development.

Why do I talk about falling back all the time?
The answer is twofold:
It’s about doing work once and getting a double-payout from it. Firstly I have started the UQE project (Unofficial Quake Engine) a few years back and as the months progressed until the current day the project generated quite some following and it will be a shame and a very personal failure to stop supporting it. I’m a very passionate person when it comes to people enjoying your work.

The first step is to get them to still enjoy the old Quake series of games, but in the same breath allow me to maintain less or easier to manage projects. Once we reach full emulation of Quake (it would be cool to emulate Hexen2 and *maybe* even Quake2) we will be using our newly developed technology to create a small and very highly playable FPS title that we will be selling online either through our own portal or through another; obviously I’m leaning towards having an own portal even though there are people not agreeing with this approach.

Creating an emulated version of the Quake game will help much with regards to us writing technology that will fit and compliment a previous AAA title other than that it will help building some support for international and local people to see and appreciate South African produced technologies and hopefully generate enough interest to help sell the games we make.

With a fully emulated Quake game we have all the technology we can use to make a simple FPS while not competing directly with current AAA titles, then primarily focus on changes and fine-tuning our already developed technology enabling us to give more attention to the art side of the project.

For this project we are currently using Truevision3D version 6.5 unless we find something else along the line.
Changing engines will be fairly simple as most of the Quake interpretation is managed by pure C# code which gets passes to the engine in question in the way the engine expects it.

Now we get to the intresting part.. progress screenshots!!
Sofar the game engine project is in its second week of development.

8 December 2007
E1M1. The first successful rendering on Quake BSP level data loaded with pure C# code. The lower part of the image is the starting spawn point in the level.

8 December 2007
E1M1. Changed the vertex coordinate system from Y being depth to Z being depth and reversed the primitive edges. The shot was made in the first outdoor area with the bridge in view.

9 December 2007
E1M1. Shot made in the first room where the starting spawn point in the level is located. (where you start playing Quake)

9 December 2007
E1M1. After a little bit more work the texture information got loaded and converted from 8-bit paletted textures to true color RGB(A) textures (runtime) and mapped accordingly to the correct faces after having to calculate each face’s UV coordinates.

9 December 2007
E1M1. A little bit deeper into the level (near the finish) another rendering of the newly textured environment. This shows the camera slightly “outside” the world. In the background also visible is unfiltered brush models which must still be managed.

9 December 2007
E1M1. The equivalent fullscreen rendering from the original Quake engine, here only lightsmaps are obviously working and a bloom effect I’ve added to the engine some months ago.

12 December 2007
E1M1. After much trouble getting lightmaps loaded, converting the data into blocklights and calculating the lightmap UV coordinates we have here some lightmapping results with some bugs.

13 December 2007
E1M1. After spending another day pulling out some hair we FINALLY have fully functional static lightmapping rendered here in fullscreen mode!! This excludes lightmaps with added styles and dynamic lightmapping.

13 December 2007
E1M1. The 10 lightmap textures generated. So, now that the lightmapping textures is sorted, heres the lightmap textures being generated (not easy) at runtime (level load) and blockmaps allocated within each lightmap texture (not easy) of which the lightmap coordinates needs to be calculated (not easy) before you get to load these into memory and render them. The lightmaps above are those of E1M1.

17 December 2007
NEWMAP. Lots of days have passed and last bits of remaining hair have been pulled. Showing here in fullscreen (not easy to spot) is fully functional water warping! Look closely at the 64×64 texture seams and you’ll see the warping.

Onto some feedback on where I struggled with the tech up until now..
With the lightmapping there are such a lot of calculation work that needs to be done before you get to the point of being able to render the lightmaps. Its not as straightforward and basically a given like with the Quake III Arena BSP format. What was needed here is to load the lightmap data into lightmap images roughly estmated by lightmapsize divided by 128×128 blocks, then you know how many lightmap images there are.

Like you have seen, each lightmap image contains lots of pieces of lightmaps (for level surfaces). the lightmap’s coordinates within the texture had to be calculated, once found you would also need to calculate the UV coordinates on the surfaces to map it correctly. It was quite hard to get all these calculated and working correctly.

The waterwarping was hard too.
When I started out I thought: “Ag lets quickly do the waterwarping, should be easy!”

How wrong I was!

Firstly, I knew already how to do proper UV warping since I’ve done it before (on the vertices) of a Quake2 level viewer I developed some years back. With horror I realized that water surfaces in Quake1 wasn’t pre-subdivided for you, like they were in Quake2. Subdivision of water and sky surfaces was done at load-time and the surfaces was updated while the level loaded adding the missing information by calculating and subdividing! The reason why it wasn’t already subdivided is simple, Quake1 and its technologies was developed and designed way before hardware 3D, so waterwarping was done very differently in the original Quake’s DOS software renderer than the later OpenGL version.

So, after a day and a half of struggling I finally got the water and sky surfaces subdivided, recalculated the face extents and UV coords.. finally I was in a position to implement water warping! Warping is working now and my calculation renders a result that matches Quake’s rendering by an estimated 95% accuracy! Our implementation further uses a real Sine calculation opposed to the original Quake’s pre-calculated Sine array, which is quicker but renders more jaggy results on faster/slower framerates.

Onto the next challenge: Getting perfect Quake-like sky rendering and writing my next entry.