(Note: The parts that Thorsten wrote are colored blue.)

Overview

    There are two components to the COMP 290 Game Programming Course that I took part in. The first was the development of more advanced rendering techniques and model loading capabilities into the Wild Magic software system. The second was helping Thorsten Scheuemann in developing his game. I will describe the major components of each in this overview.
    Adding more rendering effects to the Wild Magic software involved several modifications of the system. The major components are listed below:

Image support for RGB, TGA, and TIFF images
Image resizing, flipping horizontally and vertically, normal map generation
Single Lightscape VRML object loader
Multiple object loading of Lightscape VRML
Layered effects support for:
Gloss Maps
Diffuse Texturing
View-Dependent Cubic Environment Mapping
Sphere Mapping
Animated Textures
Light Maps (and Base and Light maps paired)
Beginnings of Diffuse Bump mapping (unfortunately incomplete)

    The game work with Thorsten consisted of creating or painting models related to the game as well as some programming to create the UFO effects. The following is a gallery of models that I painted:

These are models that I created:

Terrain created in Bryce and painted with Nichimen Nendo. Houses, bushes, and gate were all modeled and painted in Nendo as well.
 
 

    The complete scene runs at around 50 fps with 20436 triangles per frame (depending on the number of cows, UFOs, and fireballs).
    In addition, I added some code to make some effects on the UFO.

    Thorsten took the raw elements and created the rest of the game from it. The main parts of the game Thorsten created were:

Integrated an OpenGL stereo camera class
Borderless windowing for game mode
Added dual Hi-Ball tracker support
Placed all objects within the world manually
Implemented deep copies for MgcNodes for duplicating geometry
Created many elements of the game functionality
Scene graph management for UFO circling and capturing of cows
Particle system for UFO destruction
Collisions between the bullets and the cows and UFOs

    In this section, I will talk about the new rendering techniques needed for the various parts of the game engine.

More on the New Rendering Methods

        A general description of the rendering effects and how to use them can be found here.

Major Changes

        The changes that were made to support various effects, image loading, and model loading will be described here.
Probably the largest change is the replacement of RGB vertex colors with RGBA colors for the whole engine. This was done because it is potentially useful for people to be able to make objects partially transparent without relying on texture effects. For example, when the user selects and object it might be useful for the object to fade in and out of the world.
The code is based on Wild Magic v0.4 (not 0.6). This is because the changes to move up to 0.6 were enough to make the game late.
The MgcOpenGLRenderer has been altered to incorporate various parts of the MgcTextureEffect and MgcTextureEffectState class. Texture binding also can occurs while an object is being draw the first time (which in hind sight might not be the best route).
There was little regard for streaming with the new files. Most of the streaming code is not correct but still compiles.
No attempt to identify which card the application is running on is attempted. All the code is assumed to be working on a Pentium II with a GeForce 2 MX or GeForce 2 GTS and no checks for missing OpenGL extensions are made.

File Altered or Created

MgcTGAImage -- raw data reader for TGA format NEW
MgcImage -- support for deciding between TGA, RGB, TIFF. However, this also uses the _splitpath function which makes the app NON-PORTABLE (but it sure beats writing that from scratch)
MgcCubeTexture -- support for cube map textures (6 images for 1 texture). This also impacted the OpenGLRenderer in its texture set up. Cube maps can also be asked to create normalization cube maps (for bump and lighting effects). NEW
MgcOpenGLIncludes -- added new includes for TextureEffect (the .pkg file was also altered)
MgcTextureEffect -- ability to specify texturing effects NEW
MgcTextureState -- this shows up as modified but it wasn't changed (maybe used more recent version from 0.5 or 0.6)
MgcOpenGLTextureState -- shows up as modified but it wasn't changed (maybe from 0.5 or 0.6 version)
MgcOpenGLTextureEffectState -- similar to MgcOpenGLTextureState but meant for keeping multiple textures and specifying blending functions NEW
MgcTexture -- added the ability to compress texture (new enums) and to create normal textures out of a given texture (from Kilgard's torus example); also anisotropic filtering can be expressed per texture
MgcMultiTriMesh -- this calls the functions in MgcVRMLGeometry to do things like load single and multiple VRML objects NEW
MgcVRMLGeometry -- contains functions to CountObjects and CountImages in Lightscape VRML objects and append texture sets NEW
MgcFileTools -- these are called by VRMLGeometry and allow searching for words or finding out if one word occurs before another NEW
MgcOpenGLStereoCamera -- Thorsten created this class to enable stereo rendering NEW
MgcRGBImage -- raw data reader for the RGB image format. Note: RGB images are loaded as RGBA with alpha = 0 NEW
MgcString -- added a function to get the raw char * pointer from the MgcString; this was used in the MgcMultiMesh functions to set the names of the object to the textures that are applied to them; so for example "couch.tga" would name the object "couch" or "couch.tga"
MgcWinApplication -- modified so that the window appears without a border and no windows can appear on top of it (pop up window)
MgcTracker -- code to interface with the HiBall tracker and get position and orientation results NEW
ButtonManager -- code to use VRPN buttons to make the gun work NEW
MgcOpenGLRenderer -- lots of modification for various effects as well as the ability of textures not bound to go bind themselves before rendering; this might cause some hiccups in the rendering when a new object is first display
MgcTriMesh -- modified to share normals and add texture sets; each object can have a char * identifier; uses _splitpath so it is NON-PORTABLE
MgcSpatial -- Thorsten changed some of the functions for copying (deep copies)
MgcGeometry -- Thorsten changed some of the functions for copying (deep copies)
MgcNode -- Thorsten changed some of the functions for copying (deep copies)
MgcWglRenderer -- Some changes for glCull calls which were in there for debugging

Other things to note:
- There is a dynamic and a static label for individual objects. All objects loaded with MgcMultiTriMesh are marked as static and will have display lists created out of them in MgcOpenGLRenderer (also texture may be bound at that time as well)
- Objects that have MgcTextureEffects associated with them are not turned into display lists
- The cube maps are assumed to be view dependent meaning that the map is stable in world space which might not be what the user wants. This can easily be altered by not putting the inverse of the affine portion of the modelview matrix in the texture matrix (which is what the renderer does now)

Known bugs
- Currently there is a problem when there are several objects and some of them use multiple textures and some use single textures. The extra textures beyond the first are not changed which means that the render state carries over to the new object being drawn.
- Also when the cows are dropped their textures are replaced by yellow textures from the UFO. I am not sure why this happens but it seems to be because the cows are added and deleted from the scene graph. I believe this is also related to the first problem and some problem with the state pushes and pops in MgcTextureState or MgcTextureEffectState.
- There have been some problems compiling in release mode. This has not happened all the time but when Browse Info was created in Visual C++ then the errors went away. I am not sure if this Visual C++ had the latest service packs though.

Future improvements
- Unfortunately the most interesting effects such as bump maps, mirrors, and shadows were not covered in this semester. Preliminary work on bump mapping was done and it seems to involve the following steps:
        - Providing the TextureEffect with 1) the closest light of interest and 2) a height field that can be mapped to the object
        - My initial idea was to turn the height field into a normal texture and then use a per-pixel dot produt operation  (Register Combiners for nVidia or the ATI
ATIX_texture_env_dot3 extension) with the direction to the light
            - Setting up the register combiners for this effect resulted in a black object and I am not sure why (debugging the register combiners is not easy)
            - Other forms of this kind of diffuse dot product bump mapping seem to use a normalization cube map rotated to face the light and then dot product with the other normal map describing the bumps on the surface.
            - Embossing involves calculating tangents per object and I added a pointer to such tangents in MgcTextureEffect but never used it. Projecting the light on to each triangle will probably make the applications too slow.
- Vertex cache aware triangle stripping code is available from nVidia but it is not obvious how to use their library.
- Sound would have been very nice or just the ability to play random .wav or .mp3 files. Unfortunately ever since A3D has gone under there seems to be a serious sound API gap. Perhaps OpenAL could fill that void but probably a commerical library would do much better.

        Thorsten did most of the work on the game logic and creation of the world. I will describe the tool chain which worked rather well.
        Most models were created with other programs or downloaded off the web. These models were them imported into Nichimen Nendo and painted. Nendo will spit out a .wrl file and a .png which looks like:

Note that the textures cannot be compressed in this form since the colors will bleed onto nearby triangles. These .png files were converted to .tga with Adobe Photoshop. The corresponding .wrl files were modified to point to .tga files. The painted the objects were imported to 3DS Max and exported as .3ds files. The 3DS files were then imported into Lightscape and exported as VRML. Then these files were imported into the Mgc code with the following commands:

 // load lookout platform
 m_spkPlatformNode = new MgcNode;
 m_spkPlatformNode->Scale() = 0.11;
 m_spkPlatformNode->Translate() = MgcVector3(2.4, 7.4, -0.20);
 m_spkScene->AttachChild(m_spkPlatformNode);
 spkMultiMesh = new MgcMultiTriMesh();
 spkMultiMesh->LoadMultiObject("plat2.wrl");
 spkMultiMesh->AttachToObject(m_spkPlatformNode);
 for(i=0; i<spkMultiMesh->GetNumObjects(); i++)
  ms_spkRenderer->Draw(*(spkMultiMesh->GetTriMesh(i)));

    The drawing loop causes the objects to be display (and hence a display list to be created) for each object.
    This should also serve as the inspiration for a direct 3DS importer (maybe ASCII format) or .X DirectX model loader.

        Thorsten was reponsible for taking all the objects created and putting them into the environment. In addition adding tracker support and stero viewing support, he was able to create the game logic and state machines necessary to make the game playable. There were state machines for the UFOs and cows which used many different aspects of the scene graphic management system to cut and paste objects and create copies of existing geometry. Thorsten also created the initial idea for the game.

The game play is based on shooting the UFOs before the steal your cows from the land. New UFOs appear if others are destroyed and after a given amount of time your "score" is roughly the number of cows remaining on the ground. This format is good because the number of UFOs or the speed of the bullets can easily make the game easier or more challenging.

With the rendering being taken care of by the game engine and Paul's extensions, I could concentrate almost entirely on the game logic and adding bits and pieces here and there that were needed to accomplish this.

Stereo Viewing:

I derived a class called MgcOpenGLStereoCamera from MgcOpenGLCamera. This class has a method to set the distance between the eyes and methods to set up the OpenGL projection matrix for each eye by offsetting the camera position to the left and to the right respectively. This is how the scene is rendered:

ms_spkRenderer->ClearBuffers();
MgcStaticCast(MgcOpenGLStereoCamera, ms_spkCamera)->SetupLeftView();
ms_spkRenderer->Draw(m_spkScene);
MgcStaticCast(MgcOpenGLStereoCamera, ms_spkCamera)->SetupRightView();
ms_spkRenderer->Draw(m_spkScene);
ms_spkRenderer->DisplayBackBuffer();

Hiball trackers and input device:

The code for interfacing with the Hiball trackers and to query the buttons on the input device was taken from the GLVU library that is used to create VR applications on SGI machines. I took over a class called ButtonManager that manages the querying of the buttons on the portable joystick with almost no changes. I tried to fit the tracking into the Wildmagic framework by creating an MgcTracker class, but right now it is a complete mess and has to be overhauled soon. All these components are placed into a seperate library called MgcTracking.

Deep copies of MgcTriMeshes:

Since the game makes it necessary to display multiple objects of the same type at once (UFOs, cows and fireballs) and the engine doesn't support sharing of geometry at this point, I implemented a copy constructor that creates a deep copy of an MgcTriMesh object. For this to work correctly I had to create custom copy constructors for the base classes MgcGeometry and MgcSpatial, too. A better approach would have been to support data sharing in the scene graph, but time was limited and creating deep copies was a quick solution.

General game logic:

The class GameApp contains all the information about the game world and it's current state. There are two lists containing the active UFOs and cows respectively. A third list holds the cows that are free to be kidnapped by a UFO.
When a game is started, the cows are placed randomly on the terrain by choosing random x and y coordinates and then calculating the z coordinate by intersecting a line in the z direction with the triangle mesh of the terrain. Right now the cows might end up standing in one of the buildings that are placed on the terrain. I want to fix this at some point.
The time till the next UFO is entered into the game is chosen randomly. When this time has passed, a new UFO object is created and again a time for the arrival of the next UFO is picked. There is a maximum number of UFOs that can be active at any time. The GameApp class leaves the task of controlling the cows and UFOs to their respective classes by calling each object's Update() method in the OnIdle() method.

Cow logic:

The cow objects are instances of a class called Cow which is derived from MgcNode. This class controls the behavior and movement of the cows. Since the cows don't move unless they are attached to a UFO or falling to the ground, there isn't that much to control. I used a state machine with the states COW_IDLE, COW_KIDNAPPED, COW_FALLING and COW_DEAD. In the COW_FALLING state, the cow is moved along the negative Z axis. This is accomplished by updating the translation of the cow object. The actual cow mesh is attached to a cow object as a child and moves accordingly. This way the mesh can easily replaced by a version depicting a deceased cow when it is shot.

UFO logic:

The UFO behavior is implemented as a state machine in a subclass of MgcNode as well. The states are:

• APPEARING: The UFO moves downwards from it's initial position until it reaches a randomly chosen target height.
• IDLE: The UFO moves on randomly chosen circular arcs. It moves along a circle for a random amount of time and chooses another circle to continue moving on so that the movement is continuous. After a randomly chosen time, the UFO tries to picks a cow from the list of idle cows stored in the GameApp class. If this list isn't empty, the state changes to APPROACHING.

The current movement pattern cannot easily prevent a UFO from moving too far away from the terrain. It might fly through the mountains or, given enough time, move beneath the sky box. A better way to handle the movement would be to pick random waypoints (that can more easily be constrained) and have the UFO move on a spline path through these points.
• APPROACHING: The UFO moves directly over the cow on a straight line and then moves straight down.
• ATTACHING: The UFO turns on the tractor beam by attaching a beam mesh as a child node. The cow node is removed from the scene graph and reattached as a child of the UFO node. This way the cow moves together with the UFO.
• FLEEING: The UFO moves straight up till it reaches a certain height, at which point the cow and the UFO are removed from the game.
• EXPLODING: When the UFO is hit by a fireball, the child nodes are removed and replaced by a particle system. If the UFO had a cow attached, the cow's state is set to COW_FALLING.

The explosion particle system moves each particle along a randomly chosen velocity vector. Each frame this velocity vector is affected by gravity to make the particle fall to the ground. I disabled changing the particle colors as they aged because this caused the UFOs to change color as well. Right now the particle don't move according to elapsed game time. It is assumed that the frame rate is high enough so that the velocity vectors are updated often enough. This will have to be fixed at some point.

Collision detection:

Right now, collisions are only detected between fireballs and cows and fireballs and UFOs. The collision detection is done every frame and tests the line segment on which a fireball travelled between the last and current frame against the bounding box and then against the triangle mesh of an object. Although the collision detection system handles the moving fireball correctly, the movement of the other objects is ignored, since the line segment is intersected with the object statically placed where it was during the previous frame.
 
 

The biggest bug by far was the rendering errors we encountered when there were multiple objects imported into the engine. Updating the push/pop functions helped but did not eliminate the problems. The main problem comes from the MgcTextureEffectState management I believe which is basically a copy of the MgcTextureState. The problem that we initially had was using animated textures (which add a time to the MgcTexture class) on the tractor beam and then attaching and detaching that beam based on whether the UFO was near the cow. When the UFO turned on its beam not only did its texture rotate but so did everyone else's! An example is given below:

The textures are all shifting about (even the sky and cows). Taking away the rotating texture seemed to solve part of the problem.

The next problem was using particle effects when blowing up the UFOs. After compensating for the MgcTextureEffect weirdness, the UFOs started to take on the colors of the particles for some parts of the game and then return to their initial state. The vertex colors might be to blame since changing the vertex colors to white also influenced how the UFOs were subsequently drawn. This is a very strange bug (along with the TextureEffectState bug) which seems to involve the pushing and popping of state or the misallocation or some addresses for textures or effects. Both of us tried several methods to correct this but did not understand the state management system enough to predict errors or to pinpoint where the trouble really lies. It could be in how the scene graph is managed, it could be in the definition or copying of textures, it could be in the texture effects pushes ans pops. It is perplexing and is still being pursued by us.

    Thorsten and I made substantial changes to the Wild Magic code during this semester and made headway in promoting VR applications on the PC. The use of the Wild Magic tool kit significantly lessened the burden and increased the scope of our plans while creating this environment. We are both looking forward to using this in future games and VR applications as well as taking advantage of more advanced features of the engine such as spline paths and controllers for animation. Although there are some bugs in the current modifications, hopefully they provide some guide as to how the engine can be extended for unique applications.