Blind Renderer

New Music: ANI

2015-03-19T23:52:00.001-07:00

As some of my friends know, I write songs time to time. After a long artist's block, I finally finished this song...

Enjoy if you can....

Tales of Multiply Map and Gamma Correction

2015-02-18T10:04:00.000-08:00

Using a detail map, or multiply map, on top of diffuse map is a somewhat popular way to eliminate the repeating pattern on tiled textures.

I've once implemented this upon an artist's request. The actual blend(?) math I used was:

DiffuseMap * MultiplyMap * 2

The reason I added * 2 at the end was to also give a detail map the ability to brighten the diffuse map. (without this, it can only darken the base map.) So the idea was if an artist paint a detail map with half grey, or 0.5, it wouldn't change the base diffuse map at all.

It had worked fine until we decided to change our rendering engine to "gamma-aware". While we were enabling sRGB texture read on certain textures, we accidentally did it on detail maps, too. Then, a month or two passed by, and a few assets were made with this new "gamma-powered" shaders.

Then a "tech-driven" artist found out painting a detail map with the value of 0.5 darkens the texture instead of keeping the base colour. OMG. This was because our new blending formula looked like this:

{ (DiffuseMap ^ 2.2) * (MultiplyMap ^ 2.2) * 2 } ^ (1/2.2)

Because of power of 2.2, it makes 0.5 smaller than 0.25 and multiplying this by 2 gives you a value less than half intensity, 0.5. So I had to fix it by disabling sRGB read on multiply maps. And this is how the blending looks like now:

{ (DiffuseMap ^ 2.2) * MultiplyMap * 2 } ^ (1/2.2)

Better, eh?

Then, I asked artists to change detail maps authored with previous shaders to comply with our new way. To me, converting them was very straightforward.

Open a detail map in PhotoShop.
From main menu, select Image > Adjustment > Exposure.
Set gamma to 0.454545(same as 1 / 2.2).
Press OK

But this time a "visually-driven" artist wanted to change base diffuse maps instead of detail maps because gamma-ing details maps makes them too dark, so it's hard to work with. He asked me what's the easiest way to do this. Unfortunately, I couldn't find a mathematically correct way that always produces the correct conversion, so he had to manually change them while visually verifying the result.

This was probably one of the few cases where mathematical correctness can make things easier than harder, but unfortunately I was not able to find the magic formula. I just don't think it's possible here.

Or am I just stupid? Does anyone know the correct math here?

[Book] Introduction to Shader Programming

2015-02-09T14:59:00.000-08:00

Apparently, I said it would happen soon-ish, but totally forgot about mentioning it when the book was out almost an year ago. I guess I was just occupied with other things for the last 10 months.

Looking back, it's kind of funny that I was able to work on books and throw some presentations. It was a great time, but I don't see myself contributing much to the computer graphics industry anymore.

So, hopefully some beginners will find good use of this book.

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Also you can preview the first half of the book on my blog.

[Intro to Shader] 06. Love Cartoons? Here's Toon Shader

2015-01-26T14:32:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 6: Love Cartoons? Here’s Toon Shader

New HLSL in this chapter

ceil(): finds the smallest integer value greater than the input parameter.

New math in this chapter

Matrix Concatenation: Once multiple transformation matrices are pre-multiplied into one, the resulting matrix can be used for vertex transformation. This is faster than multiplying multiple matrices to each vertex.
Inverse Matrix: Can be used to transform spaces in reverse order.

Background
One day, an art director who I used to work with said, “programmers always seem to pursue super-realistic 3D graphics, but most gamers get excited about non-realistic graphics with a great style.” I thought about this for a while, and now I completely agree. It is always fun to see that programmers try to find some comfort in mathematical correctness, but highly successful games often put more stress on artistic styles. Street Fighter 4, Team Fortress and Journey would be some great examples of this.

A recent trend in 3D graphics was mostly realistic graphics, but occasionally non-realistic techniques were introduced to fulfill our artistic needs. Toon shading technique which will be covered in this chapter is also one of those. Toon is a short form or cartoon. If you read a comic book, you would probably notice that the shading of an object is very abruptly changing while real-world objects have very smooth shading. The shading technique in comic books is toon shading. Still not sure what it is? Then please take a look at Figure 6.1.

Figure 6.1 Toon shader we are going to implement in this chapter

Let’s take a close look at Figure 6.1. How is it different from ordinary diffuse lighting? With diffuse lighting, the shading changes very smoothly across the surface, but, with toon shading, it changes suddenly after not changing for a while at all. In other words, it changes discretely as if you are walking down stair steps. Now, let’s turn this observation into a graph for a better understanding.

Figure 6.2 Difference between diffuse lighting and toon shading

It makes much more sense, right? No? Um, then how about a comparison table?

Diffuse Lighting	Toon Shading
0	0
0 ~ 0.2	0.2
0.2 ~ 0.4	0.4
0.4 ~ 0.6	0.6
0.6 ~ 0.8	0.8
0.8 ~ 1	1

Table 6.1 Comparison between diffuse lighting and toon shading

This table should give you a much better idea. Table 6.1 shows that we simply need to turn every range into its upper bound in order to convert diffuse lighting to toon shading. That sounds super easy, so let’s go to RenderMonkey, right away!

Initial Step-by-Step Setup
After launching RenderMonkey, add a new DirectX effect and change the name from Default_DirectX_Effect to ToonShader. You see a matrix, named matViewProjection, right? Please delete it.

Figure 6.1 was showing a teapot model, which is often used in 3D graphics because its curvature is really good for demonstrating shader techniques. Therefore, we will use it in this chapter, as well. Inside RenderMonkey, find Model in Workspace panel. Then, right-click on it and select Change Model > Teapot.3ds.

To do toon shading, we should calculate diffuse lighting first, right? So, we need the light position and normal information as we did in Chapter 4. First, we will declare a variable for the light position. Right-click on ToonShader and select Add Variable > Float > Float4. A new variable will be added. Now change the name to gWorldLightPosition and set the value to (500, 500, -500, 1). Now, it’s time to retrieve normal information from the vertex buffer. It is as simple as double-clicking on Stream Mapping and add NORMAL field. Make sure to set the data type to FLOAT3 and Index to 0.

Let’s look at Figure 6.1, again. The teapot is green, right? There are different ways of coloring the teapot, but we will simply use a global variable to specify the color. Right-click on ToonShader and select Add Variable > Float > Float3. Once a new variable is added, change the name to gSurfaceColor. Now, double-click on the variable and change the value to (0, 1, 0). You haven’t forgot that colors are represented as a percentage values between 0 and 1, right?

Now we are going to add some matrices. Until now, we always declared three separate matrices for world, view and projection matrices, but we will do something a bit different here to make things faster. If you concatenate all the matrices into one, you can just multiply the resulting matrix to a vector instead of multiplying the matrices separately. For example, we used to multiply world, view and project matrices to a vector in order. But we can just multiply these three matrices together to get another matrix, and multiply it to the vector to produce the same result. Performance-wise, this is faster because multiplying one matrix is faster than multiplying three.

For the reasons we just discussed, we will concatenate matrices here. Therefore, we only need one global variable to hold the concatenated matrix in the end. Right-click on ToonShader and select Add Variable > Matrix > Float(4x4). Then, change the variable name to gWorldViewProjectionMatrix, and right-click on it to select Variable Semantic > WorldViewProjection.

Multiplying a matrix only once sounds really great, but don’t we still need a world matrix to calculate diffuse lighting? Light position is defined in the world space, so to find a light vector, we need a vertex position in the world space. It is the same story with the normal vector, as well. So yeah, passing a world matrix to do these multiplication twice is something we should do. But if you think a bit more, you can do the same thing with only one matrix multiplication.

The reason why we transformed the vertex position and normal vector into the world space was to match the spaces between all operands. You remember that the calculation is wrong if any of these is in a different space, right? The light position is already defined in the world space, so “why not transforming other information into the world?” was our logic. But here is another way. You can also transform the light position to the local space (of the object being drawn), while leaving the vertex position and normal vector unchanged in the local space. This also produces correct calculations since all the operands are in a same space. Even better, this way is faster because we transform only one vector instead of two.

Then how do we transform something from world to local space? Simply by multiplying the inverse matrix of world matrix. To add the inverse matrix to RenderMonkey, right-click on ToonShader and select Add Variable > Matrix > Float(4x4). Then, change the variable name to gInvWorldMatrix. Finally, right-click on the variable and select Variable Semantic > WorldInverse to finish all the setup!

Figure 6.3 RenderMonkey project after the initial setup

Vertex Shader
I will show you the full source code first, and provide line-by-line explanation after.

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal: NORMAL;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float3 mDiffuse : TEXCOORD1;
};

float4x4 gWorldViewProjectionMatrix;
float4x4 gInvWorldMatrix;

float4 gWorldLightPosition;

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldViewProjectionMatrix );

float3 objectLightPosition = mul( gWorldLightPosition, gInvWorldMatrix);
float3 lightDir = normalize(Input.mPosition.xyz - objectLightPosition);

Output.mDiffuse = dot(-lightDir, normalize(Input.mNormal));

return( Output );

}

Input and Output Data of Vertex Shader
To calculate lighting, a normal vector is needed. So, the input structure will have vertex position and normal.

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal: NORMAL;
};

The output structure is nothing special, either. The vertex shader will calculate diffuse lighting and pass the result to this structure. If you are not sure what I am talking about here, please review Chapter 4 again.

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float3 mDiffuse : TEXCOORD1;
};

Now, declare three global variables explained earlier: two matrices and the light position.

float4x4 gWorldViewProjectionMatrix;
float4x4 gInvWorldMatrix;

float4 gWorldLightPosition;

I believe these were all the variables we added to RenderMonkey, so now you can move onto the vertex shader function.

Vertex Shader Function
First, we will perform the most important role of any vertex shader: transforming the vertex position to the projection space. Since world, view and projection matrices were merged into one already, this can be done with one line of code.

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldViewProjectionMatrix );

Now, it’s time to calculate diffuse lighting. As explained earlier, we will transform the light position to the local space to perform all the calculations in the local space.

float3 objectLightPosition = mul( gWorldLightPosition, gInvWorldMatrix);

Then, make a vector pointing from the light position to the current vertex position (in the local space.) Also don’t forget to make the vector’s length to 1.

float3 lightDir = normalize(Input.mPosition.xyz - objectLightPosition);

After the above code, the light and normal vectors are both in the local space, so we can calculate a dot product of these two to calculate diffuse lighting correctly.

Output.mDiffuse = dot(-lightDir, normalize(Input.mNormal));

Do you see that Input.mNormal is normalized here? Usually normalized normals are stored in a vertex buffer, but we are calling normalize() again just in case.

Now simply return Output.

return( Output );
}

There was nothing hard with the vertex shader function because we already knew everything from Chapter 4. The only difference was that we used the local space in this chapter. But I believe it’s not a hard idea to understand. Then, let’s take a look at pixel shader.

Pixel Shader
As in Vertex Shader, I’ll show you the full source code first.

float3 gSurfaceColor;

struct PS_INPUT
{
float3 mDiffuse : TEXCOORD1;
};

float4 ps_main(PS_INPUT Input) : COLOR
{
float3 diffuse = saturate(Input.mDiffuse);

diffuse = ceil(diffuse * 5) / 5.0f;

return float4( gSurfaceColor * diffuse.xyz, 1);

}

First, let’s declare global variables and input structure of the pixel shader. The surface color is declared as a global variable, and the diffuse lighting, calculated in the vertex shader, is passed to PS_INPUT.

float3 gSurfaceColor;

struct PS_INPUT
{
float3 mDiffuse : TEXCOORD1;
};

Now it’s time to look at pixel shader function. First we clamp out meaningless negative values from mDiffuse.

float4 ps_main(PS_INPUT Input) : COLOR
{
float3 diffuse = saturate(Input.mDiffuse);

Now we will divide diffuse into 5 discrete steps so that each step’s width become 0.2. We can use ceil() function to do this. ceil() function ceils the input parameter to the nearest integer, but what we need is ceiling to the nearest multiple of 0.2. The following code solves our problem:

diffuse = ceil(diffuse * 5) / 5.0f;

Let’s look at the above formula more closely. diffuse is between [0, 1], so multiplying 5 results in [0, 5]. When ceil() is applied here, the result will become one of these: 0, 1, 2, 3, 4 or 5. Now dividing the result by 5 will give us one of these values: 0, 0.2, 0.4, 0.6, 0.8 or 1. This is what we are looking for right? Figure 6.2 and Table 6.1 say so!

The last thing to do in pixel shader is multiplying the surface color as shown below.

return float4( gSurfaceColor * diffuse.xyz, 1);
}

Now press F5 to compile vertex and pixel shaders separately and see the preview window. You will see a teapot as shown in Figure 6.1. What? The background color is different? Then right-click inside the preview window and select Clear Color to change it.

(Optional) DirectX Framework
This is an optional section for readers who want to use shaders in a C++ DirectX framework.

First, make a copy of the framework used in Chapter 3 and save it into a new folder. Then, we will save the shader and 3D model that we used in RenderMonkey so that they can be used in the DirectX Framework.

From Workspace panel, find ToonShader and right-click on it. A pop-up menu will appear.
From the pop-up menu, select Export > FX Exporter.
Browse to the folder we just created and save it as ToonShader.fx.
From Workspace panel, find Model and right-click on it. A pop-up menu will appear.
From the pop-up menu, select Save > Geometry Saver.
Browse to the folder we just created and save it as Teapot.x.

Now open the solution file in Visual C++.

First, we will find global variables we don’t need anymore. Find all the instances of gpTextureMappingShader, and change them to gpToonShader. Also change all instances of gpSphere variable to gpTeapot. There’s a texture variable, gpEarthDM, too. Since we don’t use any texture in this chapter, please remove all instances of gpEarthDM variable from the code.

What new global variables did we add to shader? There were two: light position and surface color. Add the following code:

// Light Position
D3DXVECTOR4 gWorldLightPosition = D3DXVECTOR4(500.0f, 500.0f, -500.0f, 1.0f);

// Surface Color
D3DXVECTOR4 gSurfaceColor = D3DXVECTOR4(0, 1, 0, 1);

We also set the same values used in RenderMonkey to above variables.

Now go to LoadAssets() function to change the name of model and shader to Toonshader.fx and Teapot.x, respectively.

bool LoadAssets()
{
// loading textures

// loading shaders
gpToonShader = LoadShader("ToonShader.fx");
if ( !gpToonShader )
{
return false;
}

// loading models
gpTeapot = LoadModel("Teapot.x");
if ( !gpTeapot )
{
return false;
}

return true;
}

Next is RenderScene(), which actually draws the scene. We need to pass two new matrices: a concatenated world/view/projection matrix and inverse of world matrix. First, to find the inverse matrix, add the following lines below the code calculating the world matrix.

// find inverse matrix of the world matrix
D3DXMATRIXA16 matInvWorld;
D3DXMatrixTranspose(&matInvWorld, &matWorld);

D3DXMatrixTranspose() function used in above code finds the transpose matrix. The reason why we find the transpose matrix instead of the inverse matrix is because the world matrix is an orthogonal matrix. The inverse and transpose of an orthogonal matrix are same.

Now it’s time to multiply world, view and projection matrices together. To do so, we use D3DXMatrixMultiply() function like this:

// concatenate world/view/projection matrices
D3DXMATRIXA16 matWorldView;
D3DXMATRIXA16 matWorldViewProjection;
D3DXMatrixMultiply(&matWorldView, &matWorld, &matView);
D3DXMatrixMultiply(&matWorldViewProjection, &matWorldView, &matProjection);

Please note that the order of multiplication is World Matrix X View Matrix X Projection Matrix.

Now, we will pass these two matrices to shader. Delete all SetMatrix() function calls which were already in the code, and insert the following code, instead.

// set shader global variables
gpToonShader->SetMatrix("gWorldViewProjectionMatrix",
&matWorldViewProjection);
gpToonShader->SetMatrix("gInvWorldMatrix", &matInvWorld);

Finally, don’t forget to pass the light position and the surface color.

gpToonShader->SetVector("gWorldLightPosition", &gWorldLightPosition);
gpToonShader->SetVector("gSurfaceColor", &gSurfaceColor);

If you compile and execute the program, you will see a spinning teapot. With rotation, you should be able to see the toon shader effect much better, especially on the spout and handle.

Summary
A quick summary of what we learned in this chapter:

Toon shader is one of the non-realistic rendering techniques.
Toon shader is nothing more than changing diffuse shading, which is continuous, to discrete.
Vector transformation can be optimized with matrix concatenation.

[Intro to Shader] 05. Let's Add Colors to Objects: Diffuse/Specular Shader

2015-01-12T14:23:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 5: Let’s Add Color to Objects with Diffuse and Specular Mapping

Why do different objects have different colors? It is because they absorb and reflect different spectrums of incoming light. For example, a black surface looks black because it absorbs all the spectrums, and a white surface is white because it reflects everything. Likewise, a red surface reflects red spectrum while absorbing the others.

Then how do we represent this absorption property in shader? If a surface has only one uniform color across its surface, we can just use a global variable for this, right? However, most surfaces have more complicated patterns than this. This means that each pixel needs to have a different color. So let’s think it this way. We are going to “draw” an image on a surface, and this image defines which color will be reflected on each pixel. Once this image is saved as a texture, we can look it up inside a pixel shader and apply it to the lighting result.

Do you remember that we calculated diffuse and specular light separately in Chapter 4? Then do you think we need to multiply this texture to the sum of diffuse and specular light or not? As mentioned earlier, the reason why we can recognize an object is mainly due to diffuse lighting. (On the other hand, the specular light adds a tight highlight to it.) Therefore, it is good enough to apply the texture to the diffuse lighting result only. Since this texture is only applied to diffuse lighting, we will call it diffuse map.

Then how about specular lighting? It is okay to use the same diffuse map for specular lighting, too, but usually different textures are used for specular lighting for the following two reasons. First, some surfaces reflect different spectrums for diffuse and specular lighting. Second, specular maps are often used to turn off specularity on certain pixels without affecting diffuse light, which is more globally applied. One great example is our face. When you observe specular light on a human face, do you expect to see a smooth highlight without any noise as shown in Chapter 4? If you look into a mirror, you will see the forehead and nose have more specularity. Furthermore, you will see not all the part of your nose or forehead has visible specularity. It is because your face is not perfect due to pores and facial hair. With specular mapping, this effect can be simulated to some degree. So, we will be using two separate diffuse and specular maps in this chapter.

There is another thing that affects the color of an object: the light color. If you cast a red light on a white object, the object looks reddish, right? The color of light can be easily defined as a global variable.

What we discussed so far can be summed like this:

Final diffuse = light color X amount of diffuse lighting X diffuse map

Final specular = light color X amount of specular lighting X specular map

Then, with the above knowledge under our belt, let’s write diffuse/specular mapping shader now.

Initial Step-by-Step Setup
First, make a copy of the RenderMonkey project used in the last chapter, and save it inside another folder. If you forgot to save it in the last chapter, you can use samples\04_lighting\lighting.rfx file from the accompanying code samples.

Now open this file inside RenderMonkey, and change the effect name to SpecularMapping. Once the name is changed, it’s time to add images that will be used as diffuse and specular maps. Right-click on the effect name, and select Add Texture > Add 2D Texture > Fieldstone.tga file from the pop-up menu. Now you will see a texture, named Fieldstone. Change the name to DiffuseMap.

Then right-click on Pass 0 and select Add Texture Object > DiffuseMap. You will see a texture object, named Texture0. Change the name to DiffuseSampler.

We need to add a specular map now, but I was not able to find a good candidate from the RenderMonkey installation folder. So I handmade a specular map and included with the accompanying code samples. Let’s take a look at how this specular map looks like first, side by side with the diffuse map.

Figure 5.1 Diffuse map (left) and specular map (right)

Do you see the seams between stone bricks are colored in black in the specular map? These seams won’t reflect any specular light. (Still, there is diffuse light as you can guess from the diffuse map.) One thing to note here is that textures do not always contain color information, as you can see with the specular map in Figure 5.1. The information stored in a specular map is not color; instead, it defines the amount of specular light that will be reflected from each pixel. Similarly, it is very common to use a texture map if there are variables that need to be controlled on each pixel. You will see this technique again later when we are implementing normal mapping.

Tip: Use textures for the variables that need to be controlled per pixel.

Then, find Samples\05_DiffuseSpecularMapping\Fieldstone_SM.tga file from the accompanying code samples and add it to the RenderMonkey project. To do so, you can simply drag and drop the file onto the effect name. Also change the texture name to SpecularMap, and texture object name to SpecularSampler.

Next, we will add the light color. Right-click on the effect name and select Add Variable > Float > Float3. The variable name will be gLightColor. Now double-click on the variable to assign some value. To make it emit bluish light, we will use (0.7, 0.7, 1.0) for the light color.

Last up is Stream Mapping. Unlike in Chapter 4, a texture is used here, so UV coordinates are needed. Double-click on Stream Mapping to add TEXCOORD0. The data type is float2, of course.
Once all the above steps are finished, your RenderMonkey Workspace should look like Figure 5.2.

Figure 5.2 RenderMonkey project after the initial setup

Vertex Shader
Now let’s look at vertex shader. I will show you the full source code first, and explain newly added code.

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

float4 gWorldLightPosition;
float4 gWorldCameraPosition;

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal: NORMAL;
float2 mUV: TEXCOORD0;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float2 mUV: TEXCOORD0;
float3 mDiffuse : TEXCOORD1;
float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );

float3 lightDir = Output.mPosition.xyz - gWorldLightPosition.xyz;
lightDir = normalize(lightDir);

float3 viewDir = normalize(Output.mPosition.xyz - gWorldCameraPosition.xyz);
Output.mViewDir = viewDir;

Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

float3 worldNormal = mul( Input.mNormal, (float3x3)gWorldMatrix );
worldNormal = normalize(worldNormal);

Output.mDiffuse = dot(-lightDir, worldNormal);
Output.mReflection = reflect(lightDir, worldNormal);

Output.mUV = Input.mUV;

return Output;
}

Was there any new global variables that needs be added? Yes, a light color and two texture samplers. Texture samplers are used in pixel shader, so they don’t need to be declared here. Then what about the light color? It can simply be multiplied inside pixel shader, right? So, nothing to be added here!

Then how about the input/output structure? There was one thing to add: the UV coordinates. It will be used by pixel shader to sample textures. Let’s add the following line to both input and output structures of the vertex shader.

float2 mUV: TEXCOORD0;

We also need to add only one line to vertex shader function. The below line passes the UV coordinates to pixel shader.

Output.mUV = Input.mUV;

Pretty simple, right? That’s it for vertex shader.

Pixel Shader
As we did earlier, full pixel shader code is listed first, followed by explanation on newly added code.

struct PS_INPUT
{
float2 mUV : TEXCOORD0;
float3 mDiffuse : TEXCOORD1;
float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;
};

sampler2D DiffuseSampler;
sampler2D SpecularSampler;

float3 gLightColor;

float4 ps_main(PS_INPUT Input) : COLOR
{
float4 albedo = tex2D(DiffuseSampler, Input.mUV);
float3 diffuse = gLightColor * albedo.rgb * saturate(Input.mDiffuse);

float3 reflection = normalize(Input.mReflection);
float3 viewDir = normalize(Input.mViewDir);
float3 specular = 0;
if ( diffuse.x > 0 )
{
specular = saturate(dot(reflection, -viewDir ));
specular = pow(specular, 20.0f);

float4 specularIntensity = tex2D(SpecularSampler, Input.mUV);
specular *= specularIntensity.rgb * gLightColor;
}

float3 ambient = float3(0.1f, 0.1f, 0.1f) * albedo;

return float4(ambient + diffuse + specular, 1);
}

First, we will globally declare three variables which were added to RenderMonkey.

sampler2D DiffuseSampler;
sampler2D SpecularSampler;

float3 gLightColor;

Then, add UV coordinates to PS_INPUT.

float2 mUV : TEXCOORD0;

Now, it’s time to sample the diffuse map. Add the following line to the top of the pixel shader function.

float4 albedo = tex2D(DiffuseSampler, Input.mUV);

The sampling result, albedo, is the color which current pixel reflects. I said we need to multiply this to diffuse lighting amount and light color, right? Change the previous code, which was calculating diffuse, to this:

float3 diffuse = gLightColor * albedo.rgb * saturate(Input.mDiffuse);

Now press F5 twice to compile the vertex and pixel shaders, separately. Then look at the preview window.

Figure 5.3 Result with diffuse map only

You should be able to see the brick wall texture, as well as the bluish light. However, the specular lighting does not look right. Even the seams have specular light! It’s probably because we haven’t applied the specular map yet. Then let’s add the specular map.

Tip: How to Rotate an Object in Preview WindowTo cast specular light in the seams like Figure 5.3, you will need to rotate the object. To rotate an object in Preview window, move the mouse around while holding down left mouse button inside the window. If you want translation or scaling instead of rotation, click on the second right icon from the toolbar to switch to Overloaded Camera Mode.

Add the following code, which samples the specular map, below where we did pow() on specular.

float4 specularIntensity = tex2D(SpecularSampler, Input.mUV);

We need to multiply this, as well as the light color, to specular, right? The code doing it is shown below.

float4 specularIntensity = tex2D(SpecularSampler, Input.mUV);
specular *= specularIntensity.rgb * gLightColor;

Another problem with the result shown in Figure 5.3 is that the details of diffuse texture disappear when the diffuse light disappears. It is because (0.1, 0.1, 0.1) is used for the ambient light. Since ambient light is indirect light hitting the surface, it should be modulated by the diffuse map, as well. Find the code where ambient is calculated and change the code like this:

float3 ambient = float3(0.1f, 0.1f, 0.1f) * albedo;

Again, compile the vertex and pixel shaders, and see the preview window.

Figure 5.4 Result with diffuse/specular map and ambient light

You see the obvious difference from Figure 5.3, right? Specular light is not that strong anymore, and the seams look perfect, too! In addition, you can see the traits of diffuse map in the very dark pixels, too.

(Optional) DirectX Framework
This is an optional section for readers who want to use shaders in a C++ DirectX framework.

First, make a copy of the framework used in Chapter 4 and save it into a new folder. Next, save the shader and 3D model that we used in RenderMonkey into Sphere.x and SpecularMapping.fx files so that they can be used in the DirectX framework. Also copy and paste two textures used in RenderMonkey into the framework folder. Filedstone_DM.tga and Fieldstone_SM.tga will be the file names.

Now open the Visual C++ solution, and change all instances of gpLightingShader variable to gpSpecularMappingShader in the source code.

Next, go to the global variable section to declare pointers to the two textures and a variable to store the light color, as shown below:

// Textures
LPDIRECT3DTEXTURE9 gpStoneDM = NULL;
LPDIRECT3DTEXTURE9 gpStoneSM = NULL;

// light color
D3DXVECTOR4 gLightColor(0.7f, 0.7f, 1.0f, 1.0f);

You see that we are using the exact same bluish color, (0.7, 0.7, 1.0), for the light, right? Now we will add some code to release newly added 3D resources: two textures declared above. Add the following code to CleanUp() function.

// release textures
if ( gpStoneDM )
{
gpStoneDM->Release();
gpStoneDM = NULL;
}

if ( gpStoneSM )
{
gpStoneSM->Release();
gpStoneSM = NULL;
}

Now, it is time to load D3D resources. Add the next line to LoadAssets() function.

// loading textures
gpStoneDM = LoadTexture("Fieldstone_DM.tga");
if ( !gpStoneDM )
{
return false;
}

gpStoneSM = LoadTexture("Fieldstone_SM.tga");
if ( !gpStoneSM )
{
return false;
}

Don’t forget to change the shader file to SpecularMapping.fx, too.

gpSpecularMappingShader = LoadShader("SpecularMapping.fx");

Lastly, we are going to look at RenderScene() function. The shader is doing all the work already, so we can simply assign new variables. Remember where SetMatrix() function was called before? Add the following code below those function calls.

gpSpecularMappingShader->SetVector("gLightColor", &gLightColor);
gpSpecularMappingShader->SetTexture("DiffuseMap_Tex", gpStoneDM);
gpSpecularMappingShader->SetTexture("SpecularMap_Tex", gpStoneSM);

The above code passes the light color and two textures to the shader. You still remember that _Tex postfix must be added while assigning a texture map, right? It was mentioned in Chapter 3.

If you compile and execute the program, you will see the same result you saw in RenderMonkey.

As you probably know by now, there is not much to do in a DirectX Framework when shaders are used. Loading D3D resources and managing shader variables and render states are the only things that the framework should take care of.

Summary

The reason why we can recognize the color of an object is because different objects absorb and reflect different spectrums of a light.
In 3D graphics, diffuse and specular maps are used to simulate the absorption and reflection of light. Specular map mainly controls the amount of specular light reflection on each pixel.
Light color also contributes to the final color.
Textures don’t always store color information. A specular map is a good example of this exception.

This chapter was pretty easier than we thought, right? It’s because we started from the lighting shader implemented in last chapter. Like diffuse and specular mapping, there are many techniques that can be implemented very easily by extending some basic shaders.

However, do not take the material covered in this chapter lightly. Most recent games mix this technique with normal mapping to create pretty impressive visuals.

Future of Intro to Shader Programming Book

2014-02-13T09:53:00.000-08:00

I have been posting the English version of my Intro to Shader Programming book in this blog. I intended to publish it some years ago, but I had more important things to occupy myself. I thought it was a bit too late to publish this book, but I got lucky this time.

Luckily, my friend who owns an e-book only publisher wants to publish it. I'm one of people who wants all the programming books available exclusively online, removing the need for any non-standard, paper-specific layout design. So I decided to give it to him. This means I can't post all chapters on my blog, but I managed to negotiate a bit. As a result, I will be posting up to Chapter 6 on this blog, which is about the half of this book. I hope that this new "DLC scheme" won't piss you off and people who enjoyed it on my blog would buy it.

I still have rights to post all the rest chapters on my blog. But as a good will for the publisher, I won't post them unless the book becomes unavailable for some reason in the future.

The book will be sold on Amazon, iBooks and Google Play Books, starting sometime in March or April 2014.

Learning New Knowledge is Easy

2014-02-12T09:37:00.000-08:00

learning new knowledge is easy, but changing old habits is hard.

[Intro to Shader] 04.Basic Lighting Shader - Part 2

2014-02-11T09:00:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 4: Basic Lighting Shaders - Part 2

Specular Light
Background
Specular light is different from diffuse light in a way that it only reflects to one direction and the angle of incidence is same as the angle of reflection. So if you want to observe specular light in action, you will have to see the surface from the direction where the reflected rays point toward. Have you ever turned your head away because the sun glare is too bright on your monitor? If you tilt the monitor a bit, it is bearable, right? That is specular light.

Then let’s add specular light to Figure 4.2, which had only diffuse light.

Figure 4.8 Diffuse and specular light

Just like diffuse light, there are many different specular lighting models, too. In this book, we will use Phong model, which is widely used in video games. In order to calculate specular lighting, Phong model finds the cosine value of the angle between the reflect vector (light vector reflected off the surface) and camera vector (a vector from the camera position to the current position), and raises the result to the power of some exponent. Look at the picture below for better understanding.

Figure 4.9 An example of specular lighting

Finding the cosine of the angle between reflect vector, R, and camera vector, V, is not different from what we did for diffuse lighting except that R and V are used instead of normal and light vectors. By the way, why do we raise cosine to the power of an exponent? You will find the answer in Figure 4.10.

Figure 4.10 As the exponent gets bigger, the cosine graph falls faster.

You see the graph falls faster as the exponent grows, right? If you observe specular light in the real world, you will notice that radius of the light is very tight unlike diffuse light, which is rather wide. This is why we use the power function: to mimic the tightness.[1] Then, what exponent should we use? It depends on the materials of surfaces. Rougher surfaces have less tight specular light, so the smaller exponent should be used. As a general rule, start from an exponent of 20, and experiment with bigger and smaller numbers.

Now let’s write some shader code.

Initial Step-by-Step Setup
Let’s add specular light to the diffuse light shader we wrote earlier in this chapter. After all, we need both diffuse and specular to get a “correct” light effect.

What were the new things added to Figure 4.9? They were reflect and camera vectors, right? Reflect vector is a light vector which is reflected off the surface, and the angel between normal and light vectors is same as the angle between normal and reflect vectors. This means that reflect vector can be found from all the information we already have. Then, what about camera vector? Just like how we found the light vector, we can draw a line from the camera position to the current position, right? Therefore, the camera position will be a global variable. Go to RenderMonkey, and right-click on Lighting effect to add a new float4 variable. gWorldCameraPosition should be good for the name. Now right-click on it and assign ViewPosition semantic.

Now we got everything we need. Let’s look at the vertex shader.

Vertex Shader
Just like before, the full source code is listed first.

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

float4 gWorldLightPosition;
float4 gWorldCameraPosition;

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal: NORMAL;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float3 mDiffuse : TEXCOORD1;
float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );

float3 lightDir = Output.mPosition.xyz - gWorldLightPosition.xyz;
float3 lightDirUnnorm = lightDir;
lightDir = normalize(lightDir);

Output.mViewDir = Output.mPosition.xyz - gWorldCameraPosition.xyz;

Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

float3 worldNormal = mul( Input.mNormal, (float3x3)gWorldMatrix );
worldNormal = normalize(worldNormal);

Output.mDiffuse = dot(-lightDir, worldNormal);
Output.mReflection = reflect(lightDirUnnorm, worldNormal);

return Output;
}

Global Variables and Input Data of Vertex Shader
Let’s take a look at the input data to vertex shader. Do we need any extra vertex information? I can’t think of any, so there must be none. :P Let’s just use the same input structure we used earlier in this chapter.

Then how about global variables? We have to declare gWorldCameraPosition that we just added to the RenderMonkey project, right? Add the following line:

float4 gWorldCameraPosition;

Output Data from Vertex Shader
Let’s look at the vertex shader’s output data. As we did with diffuse light, can we calculate the specular light in vertex shader and pass the result to pixel shader? Unfortunately, no. To calculate the specular light, we have to raise the cosine value to the power of an exponent, but doing it so before the interpolation step produces a wrong result. This is because the power function is not linear. This means that we need to calculate specular light in pixel shader, so we will find two directional vectors, R and V in vertex shader and pass them to pixel shader. Please add the following lines to VS_OUTPUT structure.

float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;

Vertex Shader Function
K, now let’s find these two vectors. Remember how we can find the camera vector? It’s simple. Just draw a line from the camera position to current position. This is not different from finding a light vector, at all. Let’s add the camera vector code right below where we calculated the light vector.

Output.mViewDir = Output.mPosition.xyz - gWorldCameraPosition.xyz;

Now it’s time to find the reflect vector. Then what is the math formula for vector reflection? Guess what? I don’t even remember! But don’t worry. There is another magic HLSL function for this. It’s called reflect(). This function takes two parameters, light and surface normal vectors. Add the following line before the Output structure is returned

Output.mReflection = reflect(lightDirUnnorm, worldNormal);

In the above code, we used a variable that’s not defined yet: lightDirUnnorm. This is the unnormalized light vector, which we don’t have yet. It is best not to normalize vectors which will be passed to pixel shaders to avoid visual artifacts on large triangles. That’s why we did not normalize mViewDir either. But you might still see some artifacts because we calculated the reflection vector in vertex shader. In rare cases, the reflection vector might become 0 during interpolation. If you see this symptom, find the reflection vector in pixel shader. Anyways, add the following line right below where lightDir was defined to remember the unnormalized light vector:

float3 lightDirUnnorm = lightDir;

Now that we found both vectors we need, there’s nothing more to do in vertex shader.

Pixel Shader
Let’s see the full pixel shader code first.

struct PS_INPUT
{
float3 mDiffuse : TEXCOORD1;
float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;
};

float4 ps_main(PS_INPUT Input) : COLOR
{
float3 diffuse = saturate(Input.mDiffuse);

float3 reflection = normalize(Input.mReflection);
float3 viewDir = normalize(Input.mViewDir);
float3 specular = 0;
if ( diffuse.x > 0 )
{
specular = saturate(dot(reflection, -viewDir ));
specular = pow(specular, 20.0f);
}

float3 ambient = float3(0.1f, 0.1f, 0.1f);

return float4(ambient + diffuse + specular, 1);
}

First add the following two vectors to PS_INPUT structure. These are exactly same as what we added to VS_OUTPUT structure.

float3 mViewDir: TEXCOORD2;
float3 mReflection: TEXCOORD3;

We will add some new code right after where we calculated diffuse lighting earlier in this chapter. First, normalize mReflection and mViewDir.

float3 reflection = normalize(Input.mReflection);
float3 viewDir = normalize(Input.mViewDir);

Then, find the dot product of these two vectors and raise it to the 20th power.

float3 specular = 0;
if ( diffuse.x > 0 )
{
specular = saturate(dot(reflection, -viewDir ));
specular = pow(specular, 20.0f);
}

From the above code, we calculate specular light only when diffuse light is bigger than 0%. It’s because there’s no light hitting the surface if there is no diffuse light, so specular light cannot exist there, either. Also you must have noticed that –viewDir is used when calculating the dot product, right? As with diffuse light, two vectors’ tails must meet to calculate specular light correctly.

Also please note that pow() is used to raise the value to the 20th power. The exponent 20 would be different for different objects.[2] So declaring it as a global float variable would be a good idea if you need different specular tightness for different objects. I will leave this task to readers.
Now it’s time to return the result. Let’s return only specular light first. Replace the return statement with the following line.

return float4(specular, 1);

Once you compile and run the shader, you will see specular light, as shown in Figure 4.11.

Figure 4.11 Specular light has much stronger and tighter highlight than diffuse light

Now you know how specular light looks like. If we add diffuse light to this, the result will be more perfect. Please change the return code like this:

return float4(diffuse + specular, 1);

There are cases when the addition of diffuse and specular becomes bigger than 1. Luckily, you don’t need to worry about this because the result is automatically clamped to 1.[3]

If you compile both vertex and pixel shader and see the preview window, you will find a nice looking sphere with both diffuse and specular light.

Figure 4.12 Diffuse + Specular

This is already pretty good, but the bottom-left part of the sphere is too dark. In fact, it is almost invisible. As mentioned before, indirect light usually illuminates the dark area in the real world. Then, why don’t we just add simple ambient light to brighten the dark area? Let’s declare the ambient light as 10% .

float3 ambient = float3(0.1f, 0.1f, 0.1f);

Then, add this ambient amount to the final return value.

return float4(ambient + diffuse + specular, 1);

After this, you will see the result like Figure 4.13.

Figure 4.13 Ambient + Diffuse + Specular

(Optional) DirectX Framework
This is an optional section for readers who want to use shaders in a C++ DirectX framework.

First, make a copy of the framework used in Chapter 3 and save it into a new folder. Next, save the shader and 3D model that we used in RenderMonkey into Sphere.x and Lighting.fx files so that they can be used in the DirectX framework.

Then, open the solution file in Visual C++. We will look at the global variables first. Since we don’t use any texture in this chapter, delete the texture variable declared in the last chapter. Its name was gpEarthDM. Now change the name of the shader variable from gpTextureMappingShader to gpLightingShader.
Now it is time to declare new variables for the light and camera positions. Both of them are in the world space. First, we will reuse the same light position used in RenderMonkey.

// world position of the light
D3DXVECTOR4 gWorldLightPosition(500.0f, 500.0f, -500.0f, 1.0f);

For the camera position, we are using the same values defined in RenderScene() function in the last chapter.

// world position of the camera
D3DXVECTOR4 gWorldCameraPosition(0.0f, 0.0f, -200.0f, 1.0f);

Now go to CleanUp() function. Since gpEarthDM texture is not used anymore, delete the code which was releasing the texture.

Next up is LoadAssets() function. Again, delete the code which was loading gpEarthDM texture. And change the shader’s name to Lighting.fx. Don’t forget to change the variable name from gpTextureMappingShader to gpLightingShader.

// loading textures

// loading shaders
gpLightingShader = LoadShader("Lighting.fx");
if (!gpLightingShader)
{
return false;
}

Lastly, we will look at RenderScene() function. First, find all the instance of gpTextureMappingShader, and replace them with gpLightingShader. Now let’s look at the code which constructs the view matrix. There was a variable named vEyePt that we used to make the view matrix, right? This variable’s value is same as gWorldCameraPosition, so we will reuse this value.

Change below code

D3DXVECTOR3 vEyePt( 0.0f, 0.0f, -200.0f );

to this:

D3DXVECTOR3 vEyePt( gWorldCameraPosition.x, gWorldCameraPosition.y,
gWorldCameraPosition.z );

Now delete gpLightingShader->SetTexture() code. The shader in this chapter does not use a texture, so we don’t need this code. Then, pass the light and camera positions to the shader. Since the data type is D3DXVECTOR4, we will call SetVector() function.

gpLightingShader->SetVector("gWorldLightPosition", &gWorldLightPosition);
gpLightingShader->SetVector("gWorldCameraPosition", &gWorldCameraPosition);

Now compile and run the shader. You can see the same visual that you saw in RenderMonkey, right?

Other Lighting Techniques
Still the most common lighting techniques in computer games is Lambert + Phong, but now there are more games using more advanced lighting techniques. For the readers who want to learn more about advanced lighting techniques, I will mention some of them in the following list:

Blinn-Phong: a technique that is very similar to Phong.
Oren-Nayar: a diffuse lighting technique that takes account of the roughness of a surface.
Cook-Torrance: a specular lighting technique that takes account of surface roughness
Spherical Harmonics Lighting: once indirect light is preprocessed offline, it can be applied in real-time.

Summary
A quick summary of what we learned in this chapter:

Both Lambert and Phong models use cosine function.
Phong specular lighting model uses pow() function.
Once you change the vector’s length to 1, a dot product can replace cosine.
If a same calculation can be done either in vertex and pixel shader, doing it in vertex shader is a better choice.
There are more realistic, but more complicated techniques. Some of them are already used in recent computer games.

With the completion of lighting shader, now we have learned all the basic shaders. We will mix and match what we have learned so far to implement more practical techniques. So if there was anything you are unsure about from Chapter 1 to 4, please review it before coming to Chapter 5.

------
Footnotes:

This was invented as a hack, which has no physical correctness, but still is used a lot in games.
Higher exponents produce tighter specular light. Experiment with different numbers.
It is because our back buffer format is 8 bit per channel. If a floating-point texture is used, values bigger than 1 can be stored, as well.

Next Chapter

[Intro to Shader] 04.Basic Lighting Shader - Part 1

2014-02-05T09:00:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 4: Basic Lighting Shaders - Part 1

New HLSL in this chapter

NORMAL: a shader semantic used to retrieve normal data from a vertex buffer
normalize(): normalizes a vector
dot(): dot product function
saturate(): clamps a value to [0, 1] range
reflect(): vector reflection function
pow(): raises base to the power of exponent

New math in this chapter

dot product: can be used to find the cosine value of an angle quickly
normalization: converts a vector to a unit vector, which has a length of 1

If there is no light, we can see nothing. It sounds very obvious, but we often forget about it. For example, if you go into a room with no window and close the door, you cannot see anything. No matter how long you stay in the dark, you can’t see a thing… well, unless there is a seam under the door or something. The reason why we keep forgetting this obvious fact is because it is really hard to find a complete dark place in the real world. Why is it so? It’s because light reflects off objects endlessly, and the reflected light eventually reaches our eyes. This type of light is called indirect light. On the other hand, direct light is the light directly coming from a light source. An example is shown in Figure 4.1.

Figure 4.1 An example of direct and indirect light

Between direct and indirect light, which one would be easier to calculate? As the above picture hint, the answer is direct light. Indirect light goes through multiple reflections, so it is inherently harder to calculate. As one of the methods to calculate indirect light, there is a technique called Ray Tracing. Readers who are interested in 3D graphics probably heard about it, but this technique is still not widely used in computer games due to hardware limitations.[1] Therefore, most real-time 3D applications, including computer games, are still calculating only direct light “properly”, and trying to mimic indirect light. That is why this book only covers direct light.[2] By the way, the lighting techniques covered in this chapter are still widely used in most games, so make sure you understand them very well.

In computer graphics, we like to think light is made of two major components: diffuse and specular light, so we will look at these two separately in this chapter.

Diffuse Light
Background
Although most objects don’t emit light by themselves, the reason why we can still recognize them is because light emitted from other objects, such as the sun, reflects off them. When this happens, some light is evenly reflected to all directions. We call this diffuse light. Have you ever wondered why an object’s color and darkness do not change regardless where you look at it from? It is because of diffuse light, which is evenly reflected to all directions. If it is reflected to only one direction[3], you should be able to recognize the object only from a certain direction.

By the way, a rougher surface usually reflects more diffuse light.[4]

Maybe a drawing will help.

Figure 4.2 Diffuse lighting

One thing that I didn't show you in Figure 4.2 is specular light that we will learn shortly. Don’t worry about this for now, and just remember that some part of an incoming light becomes diffuse light, and some other part becomes specular light.

Well, then how do we calculate diffuse lighting? As one can guess, there are various diffuse lighting models created by many great mathematicians. Out of these, we will learn only one simple diffuse lighting model, called Lambert diffuse lighting model, which happens to be a very popular lighting model in computer games. Lambert diffuse lighting model, which was created by a mathematician named Johann Heinrich Lambert, says the amount of diffuse light at a point on a surface is same as the cosine value of the angle between the surface normal[5] and the incoming light ray. Then, let’s observe the cosine graph shown in Figure 4.3.

Figure 4.3 A graph showing y = cos(x) function

From the above graph, you can see that the result, or the value on the y-axis, is 1 when the angle is 0. And as the angle grows bigger, the result gets smaller until it becomes 0 at 90 degree. If we go further, the result even becomes negative. With this observation in mind, let’s look at Figure 4.4, which shows what happens in the real word depending on the angle of the incoming light rays?

Figure 4.4 Various angles between different light and normal vectors

Can you guess when the surface would be lit brightest? Yes, it is when the sun reaches its highest position in the sky. (Case a) As the sun lowers, the surface gradually gets darker. (Case b) And when the sun finally goes down over the horizon, the surface becomes completely dark. (Case c) Then, what happens after the sunset? The surface will remain dark because it is not getting any light. (Case d) Then let’s turn this observation into a graph. The angle between the surface normal and the sun is on the x-axis and the brightness of the surface is on the y-axis. The range of the y-axis is [0, 1]: 0 is when the surface is darkest, and 1 is when the surface is brightest.

Figure 4.5 A graph showing our observation

The reason why I put question marks between -90 ~ 90 degrees in the above figure is because it is still unknown how fast the surface darkens as the angle decreases. Now let’s compare this figure to Figure 4.3. If you clamp any negative values to 0 in Figure 4.3, it looks almost identical to our current figure, right? The only difference is that the falloff speed between -90 and 90 is a bit different. Then, can we just believe that uncle Lambert derived this cosine formula after through observations? Yes, at least I’d love to! :P

Then we should be able to calculate diffuse light with a cosine function if we use the Lambert model! However, cosine is not a cheap function, so calling it every time in pixel shader makes me feel icky. Is there any alternative? Yes. If you flip through you math book, you will find a section where it says dot product can replace cosine… well, only under certain conditions.

θ = angle between A and B
| A | = length of vector A
| B | = length of vector B
A ∙ B = cosθ | A || B |

In other words,

cosθ = (A ∙ B) ÷ (| A |ⅹ| B |);

According to the above dot product formula, the cosine value of an angle between two vectors is same as two vector’s dot product divided by the multiplication of lengths of two vectors. If we simplify this formula even further by making the lengths of both vectors to 1.

cosθ = (A' ∙ B')

The simplified formula says if you take the cosine of the angle between two vectors, it’s same as the dot product of them. But here’s a question: is it okay to change the lengths of vectors like this? In other words, the lengths of the normal and light vectors are important while diffuse light is calculated? No, not at all. What matters is only the angel between two vectors, and the lengths don’t affect the result, at all. Therefore, it sounds much better to make the lengths to 1 to simplify the formula.[6]

Do you want to know why a dot product is better than cosine? Let (a, b, c) be vector A, and (d, e, f) be vector B. Then you can find the dot product very easily like this:

A ∙ B = (a ⅹ d) + (b ⅹ e) + (c ⅹ f)

This looks much simpler than cosine function, right? If I ask you to calculate the cosine value, I’m pretty sure you will be slower than doing three multiplications followed by two additions. :P

Okay, I think we learned enough to write diffuse lighting shader code.

Initial Step-by-Step Setup

As we did in other chapters, create a new DirectX effect inside RenderMonkey, and delete all the code inside vertex and pixel shaders.
Now change the shader name to Lighting.
Don’t forget to add gWorldMatrix, gViewMatrix and gProjectionMatrix, and assign proper semantics for them. They are needed to transform vertex positions.

What information did we need to calculate diffuse lighting with the Lambert model? The light and normal vectors, right? Normal information is normally stored in each vertex,[7] so we should get it from the vertex buffer. Do you remember what extra step we had to perform to get the UV coordinates from vertex buffer in Chapter 3? From Workspace panel, double-click on Stream Mapping and add a new field named NORMAL. Normal is a direction vector that exists in a 3D world, so it will be declared as FLOAT3. Again, you don’t need to worry about Attribute Name, but make sure that Index is 0

Then, how do we find the light vector? This is not that hard. If you just a draw a line from the position of the light source to the current pixel position, that is what we are looking for. So once the light position is known, we can find the light vector very easily. Then how is the light position defined? Something like “the light is at (500, 500, -500) in the world” should be enough. This means that the light position is a global variable. From Workspace panel, right-click on Lighting, and select Add Variable > Float > Float4. Then, change the name of the newly-created variable to gWorldLightPosition. Finally, double-click the variable and set the values to (500, 500, -500, 1).

Once you are done, RenderMonkey Workspace should look like Figure 4.6.

Figure 4.6 RenderMonkey project after the initial setup

Vertex Shader
I will show you the full source code first, and provide line-by-line explanation after.

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal : NORMAL;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float3 mDiffuse : TEXCOORD1;
};

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

float4 gWorldLightPosition;

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );

float3 lightDir = Output.mPosition.xyz - gWorldLightPosition.xyz;
lightDir = normalize(lightDir);

Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

float3 worldNormal = mul( Input.mNormal, (float3x3)gWorldMatrix );
worldNormal = normalize( worldNormal );

Output.mDiffuse = dot(-lightDir, worldNormal);

return Output;
}

Input Data to Vertex Shader
We will start from the input data structure used in Chapter 2.

struct VS_INPUT
{
float4 mPosition : POSITION;
};

Now we need to add normal here. The semantic for normal is NORMAL. As mentioned earlier, normal is a direction vector in 3D space, so the data type will be float3.

struct VS_INPUT
{
float4 mPosition : POSITION;
float3 mNormal : NORMAL;
};

Vertex Shader Function
In this chapter, we will take a look at the vertex shader function before its input and output data. I believe this is an easier way to understand the lighting shaders.

First, we transform the vertex position, as usual.

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );

Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

The above code does not need explanation anymore. What other things did we need to calculate diffuse light? We need to find the light and normal vectors, but do we find these in the vertex or in pixel shader? Think about it for a second.

...

So, what do you think? There is no one absolute answer: you can do it in either shader. If we do this in vertex shader, we would calculate the dot product of these two vectors on each vertex, and return the result as part of VS_OUTPUT structure. Then, the output values will be passed to pixel shader after interpolated by interpolators, so we can just use interpolated dot product values in pixel shader.

On the other hand, if you do this in pixel shader, you would return the normal information as part of VS_OUTPUT, and pixel shader will read it to calculate the dot product.

Since there is no difference whether the calculation is done in vertex or pixel shader[8], we should select the option which is better for performance. To see which option is better, let’s count how many times each shader is executed.[9] When a triangle is drawn, how many times will the vertex shader run? A triangle consists of three vertices, so the vertex shader is executed three times. Then how about the pixel shader? It will be executed as many times as the pixels covered by this triangle on the screen. If the triangle is really tiny on the screen, thus covering only one pixel, the pixel shader will be executed only once. However, usually a triangle covers more than three pixels on the screen. So, if a same calculation can be done in either vertex or pixel shader, it is better to be done in vertex shader over pixel shader. Therefore, we will calculate diffuse lighting in vertex shader, as well.

Tip: If a calculation can be done either in vertex or pixel shader. It is usually a better idea to do so in the vertex shader for performance reasons.

Then, let’s construct the light vector, first. As mentioned earlier, you can do this by drawing a line from the light position to current position. “Drawing a line” between two points is same as subtracting a position vector from another. So, if you subtract the light position from current position, you can get the light vector. However, there is one thing that we should be careful about. To get the correct result in 3D math, all variables must be in a same space. We defined the light position in the world space, right? But, which space is the vertex position defined in? Input.mPosition is in the local space, and Output.mPosition is in the projection space, but what we really need is the position in the world space. If you look at the vertex shader code listed above, you will see some empty lines after multiplying the world matrix to the local position. Output.mPosition right after the world matrix multiplication is the position in the world space, so we can just subtract the light position from it. Now replace the empty lines with the light vector construction code shown below:

float3 lightDir = Output.mPosition.xyz - gWorldLightPosition.xyz;

K, now it is time to make the vector’s length to 1. I told you the reason for this is to use a dot product instead of a rather expensive cosine function. Did I also tell you an operation of making a vector’s length to 1 is called normalization? To manually normalize a vector, you can divide each components of the vector by the length of the vector. However, we will just use a HLSL intrinsic function, normalize(). Yay for another magic function!

lightDir = normalize(lightDir);

Now that we have prepared the light vector, it is time to find the normal vector. Can we just use the normal information from the input data as-is? To find the answer, think about in which space the vector lives. Since this data directly comes from the vertex buffer, it must be in the local space. Now we know we need to transform it to the world space to calculate diffuse lighting properly.

Caution: While performing any 3D operation, we have to make sure that all the variables are in a same space.

float3 worldNormal = mul( Input.mNormal, (float3x3)gWorldMatrix );

Do you see we are casting the world matrix to a 3-by-3 matrix? We prefixed (float3x3) to do so. In a 4-by-4 matrix, the fourth row (or column) contains translation information, so it does not affect direction vectors, at all.[10]

Don’t forget to change this vector to a unit vector, as well.

worldNormal = normalize( worldNormal );

Now we have all two vectors we need, so let’s find the dot product of them. Do you still remember what the dot product formula was? Although it was not hard, you do not need to remember it at all because we will just use another HLSL intrinsic function, dot() for this.

Output.mDiffuse = dot(-lightDir, worldNormal);

The above code assigns the dot product result to mDiffuse of the return structure. Oh, another thing! Do you see we used -lightDir instead of lightDir here? The reason why we did this is because the tails of two vector must meet to calculate the dot product correctly. So, if lightDir was used, the light vector’s header would meet the normal vector’s tail, resulting in a wrong calculation.

Also do you see that float3 is used for mDiffuse to store the dot product result, which is only a single float? If you assign a float value to a float3 variable, all three components of the variable will have the same float value. So, the above code is same as assigning dot(-lightDir, worldNormal).xxx.

Now simply returns the result.

return Output;
}

Global Variables
Have you figured out by now why I wanted explained the vertex shader function first? It is because it made no sense to say “I’m going to declare the light position as a global variable” without any explanation.

Please add these global variables at the top of the source code.

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

float4 gWorldLightPosition;

Output Data from Vertex Shader
As we saw while writing the vertex shader function, the output data is mPosition and mDiffuse. We already know float4 and POSITION semantic are used for the position. Then, what type and semantic should be used for mDiffuse? The dot product of two vectors is not a vector: it is just a single real number.[11] Therefore, using float is completely fine, but float3 is used here since we are going to return this value as the RGB values of pixels. Then how about the semantic? Do you think there is a semantic like DIFFUSELIGHTING? Unfortunately, no.[12] While programming shaders, there are often cases where you cannot find a semantic which is specifically made for your specific use. In these cases, TEXCOORD semantic is normally used. There are at least 8 TEXCOORDs[13], so they rarely run out! For this chapter, we will use TEXCOORD1.[14]

Please add the following output structure to the source code.

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float3 mDiffuse : TEXCOORD1;
};

Pixel Shader
Alright, pixel shader time! But what do we need to do? We already calculated diffuse lighting in vertex shader. So the only work left for the pixel shader is returning the interpolated diffuse result, which is passed to the pixel shader. If you recall, the dot product is used instead of cosine, so the range of the result is [-1, 1]. But, the range of diffuse lighting is [0, 1], so let’s just clamp any negative value to 0. Although we can use an if statement for this, we will instead use a faster HLSL function, saturate(). This function clamps a value to [0, 1]. And even better, it is almost free performance-wise!

struct PS_INPUT
{
float3 mDiffuse : TEXCOORD1;
};

float4 ps_main(PS_INPUT Input) : COLOR
{
float3 diffuse = saturate(Input.mDiffuse);
return float4(diffuse, 1);
}

In the above code, the return value is constructed with float4(diffuse, 1). Just remember that this is a way to construct a float4 variable.

Now press F5 to compile vertex and pixel shaders separately before seeing the preview window. You will see a sphere which is lit with very smooth diffuse light, as shown in Figure 4.7.

Figure 4.7 Our diffuse lighting effect!

----------------
Footnotes:

Especially video game console hardware is a problem.
Lighting models that only consider direct light are called local illumination models, while ones that also consider indirect light are called global illumination models.
This is specular lighting, which we will cover later in this chapter.
There are not many objects which doesn’t reflect diffuse light at all. Even very smooth surfaces reflect diffuse light because light can penetrate through and scatter below the surface until it finally comes out of the surface.
Normal is a direction vector that represents a surface’s orientation. Therefore, the normal vector of a horizontally flat surface is perpendicular to the surface facing upward.
A vector whose length is 1 is called a unit vector. And a process of making a vector to have a length of 1 is normalization.
This is not always true. We will see another way of finding normal while implementing Normal Mapping shader later in this book.
In fact, there is a subtle difference between these two approaches.
There are also other factors that might degrade the performance, so this is a guideline only.
Think it this way. The direction which an arrow is pointing to doesn’t change, even if it is moved around without any. So translation value has no meaning to a direction vector.
This is called scalar.
Some people use COLOR0 semantic for this, but this book does not. In vertex shader 2.0 spec, a value with COLOR semantic is clamped to [0, 1], and the interpolated values passed to pixel shader seem to have small errors because of this.
TEXCOORD0 ~ TEXCOORD7
The reason why TEXCOORD1 is used instead of TEXCOORD1 is because we will use TEXCOORD0 for the UV coordinates of a texture in the next chapter.

Next Chapter

[Intro to Shader] 03. Texture Mapping

2014-01-15T09:00:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 3: Texture Mapping

New HLSL in this chapter

sampler2D: a texture sampler data type which is used to get a texel from a texture
tex2D(): a HLSL function to sample a texel from a texture
swizzling: a way to access the components of a vector in an arbitrary order

What did you think about what we covered in the last chapter? Too easy? It didn't seem that useful for the game you are trying to make? Yeah, you are right. The main goal of the last chapter was learning the basic syntax of HLSL through a simple practice. Just consider it as a hello-world program in other programming languages. Now, you are going to learn something more useful in this chapter. What about wrapping the red sphere with an image? You know this is called Texture Mapping, right?

Texture Mapping and UV Coordinates
As mentioned earlier in this book, the building blocks of a 3D object are triangles. Then, what’s involved to put an image, or a texture, on a triangle? We should order the GPU like this: “Show the pixel at the right-bottom corner of that image on the left vertex of this triangle”[1] We all know that a triangle is made of three vertices, so all we need to do is mapping each of three vertices to a pixel in a texture. Then how do we specify one pixel on a texture? A texture is an image file after all, so can we just say something like “the pixel at x = 30, y = 101”? But, what happens if we doubles the width and height of the image? We will have to change it to “x = 60, y = 202”. This is not good, at all!

Let’s take a moment and think about a common sense that we learned in the last chapter. We did something very similar with the color representation. To represent a color in a uniform way regardless the number of bits per channel, we used the percentage notation [0~1]. So why don’t we just use the same method? Let’s say x = 0 points to the very left column of a texture, and x = 1 points to the very right column. Similarly, y = 0 is the top row and y = 1 is the bottom row. By the way, the UV notation is normally used instead of XY for texture mapping; there is no special reason, it’s just to avoid any confusion since XY is normally associated with positions. Figure 3.1 shows what we just discussed here:

Figure 3.1 UV layout on a texture

Now let’s see some examples of how different UV coordinates change the visuals. Please look at Figure 3.2.

Figure 3.2 Various examples of texture mapping

(a) 2 triangles with no texture. Vertices v0, v1, v2 and v0, v2, v3 are making up one triangle each.
(b) The range of UV coordinates is [0, 0] ~ [1, 1]. It shows a full texture.
(c) The range of UV coordinates is [0, 0] ~ [0.5, 1]. It shows only the left half of the texture. 0.5 means 50%, so it’s halfway, right?
(d) The range of UV coordinates is [0, 0] ~ [0.5, 0.5]. So it only shows the top left quarter of the image.
(e) The range of UV coordinates is [0, 0] ~ [1, 2]. It repeats the texture twice vertically. [2]
(f) The range of UV coordinates is [0, 0] ~ [2, 2]. It repeats the texture twice vertically and twice horizontally. [3]

Additionally, you can flip the texture horizontally if the range of UV coordinates is set to [1, 0] ~ [0, 1]. I believe it’s enough for you to understand how UV coordinates work. Then, it is about time to write Texture Mapping shader, finally!

Initial Step-by-Step Setup

As we did in Chapter 2, open RenderMonkey to make a new DirectX effect. Then, delete all the code inside the vertex and pixel shaders.
Now, change the name of shader to TextureMapping.
Don’t forget to add gWorldMatrix, gViewMatrix and gProjectionMatrix variables that are needed to transform vertex positions. You still remember how to use variable semantics to pass the data, right?
Next, we will add an image that is going to be used as the texture. Right-click on TextureMapping shader and select Add Texture > Add 2D Texture > [RenderMonkey installation folder]\examples\media\textures\earth.jpg. Now you will see a texture, named Earth, is added.
Change the name of texture to DiffuseMap.
Now, right-click on Pass 0 and select Add Texture Object > DiffuseMap. You should be able to see a newly added texture object, named Texture0.
Change the name from Texture0 to DiffuseSampler.

Once you finish these steps, the Workspace panel should look like Figure 3.3.

Figure 3.3 RenderMonkey project after the initial setup

Vertex Shader
The full source code is listed below, followed by line-by-line explanation.

struct VS_INPUT
{
float4 mPosition : POSITION;
float2 mTexCoord : TEXCOORD0;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float2 mTexCoord : TEXCOORD0;
};

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

VS_OUTPUT vs_main(VS_INPUT Input)
{
VS_OUTPUT Output;

Output.mPosition = mul(Input.mPosition, gWorldMatrix);
Output.mPosition = mul(Output.mPosition, gViewMatrix);
Output.mPosition = mul(Output.mPosition, gProjectionMatrix);

Output.mTexCoord = Input.mTexCoord;

return Output;
}

Before walking through the vertex shader code, let’s take a moment and think about what kind of new data is needed to perform texture mapping. Obviously, we need an image, which is going to be used as the texture. Then where should we perform the actual texture mapping between the vertex and pixel shaders? If you think about where vertex and pixel shaders are executed, you can find the answer easily. A vertex shader is executed for each vertices, but where the texture will be shown? Is it on vertices? No, it’s not. We want to see the texture on all the pixels inside of a triangle, so texture mapping got to be performed inside the pixel shader, which is executed for each pixels. Then, now we know that it’s unnecessary to declare a texture variable inside of vertex shaders.

Then, is there any other information required for texture mapping? It was mentioned earlier in this chapter. Yes, you need the UV coordinates. Do you remember where the UV coordinates are stored? They are stored in vertex data since they can differ across vertices. Therefore, the UV coordinates are passed via vertex data instead of global variables. Now, with this knowledge, let’s take a look at the input and output data of the vertex shader.

Input Data to Vertex Shader
We start from the input data structure used in Chapter 2.

struct VS_INPUT
{
float4 mPosition : POSITION;
};

We will add the UV coordinates to this structure. The UV coordinates have two components, U and V, so the data type should be float2. Then which semantic must be used to retrieve the UV information from the vertex buffer? Just like how the position information was retrieved via POSITION semantic, UV coordinates have their own semantic: TEXCOORD.[4] After adding the data field for UV to the structure, it looks like below:

struct VS_INPUT
{
float4 mPosition : POSITION;
float2 mTexCoord : TEXCOORD0;
};

The reason why the number 0 follows TEXCOORD is because multiple TEXCOORDs are supported by HLSL. There are cases where multiple textures are used in a shader. In those cases, you would use different semantics, such as TEXCOORD0, TEXCOORD1 and so on.

Output Data from Vertex Shader
Again, we start from the output structure used in Chapter 2.

struct VS_OUTPUT
{
float4 mPosition : POSITION;
};

Can you guess if we need to add another information here? One of those things that were not explained in Chapter 2 is that a vertex shader can return more than just the vertex position. The reason why a vertex shader must return a vertex position was to allow the rasterizer to find pixels. However, this is not the reason why a vertex shader returns information other than the position. It does so solely for the pixel shader, and a good example is the UV coordinates.

Pixel shaders cannot directly access the vertex buffer data. Therefore, any data that needs to be accessed by pixel shaders (e.g., UV coordinates) must be passed through vertex shaders. Does it feel like an unnecessary restriction? Once you look at Figure 3.4, you will understand why this restriction exists.

Figure 3.4 What would be the UV coordinates of this pixel?

Where the UV coordinates are defined is on each vertices, but as you can see in Figure 3.4, most pixels’ UV coordinates are different from any vertices UV coordinates. [5] Therefore, the right way of finding the correct UV coordinates of a pixel is smoothly blending the UV coordinates defined on three vertices based on the distance from the pixel to each vertices. Luckily, you do not have to do this calculation manually. Just like vertex positions, any other data is automatically handled by a device called interpolator. Let’s add the interpolator to the figure of a GPU pipeline presented in Chapter 1.

Figure 3.5 Still pretty simple 3D pipeline after adding the interpolator

By the way, this device doesn't stop at interpolating[6] the UV coordinates. It interpolates any data that is returned from vertex shaders and pass the result to pixel shaders.

By now, you should know that the UV coordinates need to be returned from this vertex shader. Let’s add the data field.

struct VS_OUTPUT
{
float4 mPosition : POSITION;
float2 mTexCoord : TEXCOORD0;
};

Global Variables
We don’t need any extra global variables other than what we already used in Chapter 2. So, I’ll just show the code again and skip the explanation.

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

Vertex Shader Function
You heard it enough. The most important responsibility of a vertex shader is transforming vertex positions into the projection space. The below code is identical to the one used in Chapter 2.

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );
Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

Now, it’s time to pass through the UV coordinates, but do we need to apply any transformation before assigning the UV coordinates to Output structure? The answer is no. UV coordinates do not exist in any 3D spaces discussed in this book. Therefore, we will simply pass the UV coordinates without any transformation.

Output.mTexCoord = Input.mTexCoord;

I cannot think of any other data that needs to be handled here, so I’ll finish this function by returning Output.

return Output;
}

Pixel Shader
As done in Vertex Shader section, the full source code is listed first below:

sampler2D DiffuseSampler;

struct PS_INPUT
{
float2 mTexCoord : TEXCOORD0;
};

float4 ps_main( PS_INPUT Input ) : COLOR
{
float4 albedo = tex2D(DiffuseSampler, Input.mTexCoord);
return albedo.rgba;
}

Input Data to Pixel Shader and Global Variables
It is time to look at the pixel shader. What we need to do here is retrieving a texel [7] from a texture image and output its color on the screen. Then, we need a texture and current pixels’ UV coordinates, right? A texture image is uniform for all the pixels, so it would be a global variable. Unlikely, the UV coordinates are part of the input data sent from the vertex shader and passed through the interpolator. First, let’s declare the input structure of the pixel shader.

struct PS_INPUT
{
float2 mTexCoord : TEXCOORD0;
};

Wait. We saw something like this before. It is almost identical to the VS_OUTPUT structure except it is missing mPosition. In fact, the input structure of a pixel shader should match the output structure of its counter-part vertex shader. After all, the pixel shader is getting what is returned from the vertex shader, right?

The next step is texture declaration. Do you remember that we made a texture object named DiffuseSampler while setting up the RenderMonkey project earlier in this chapter? This object is the texture sampler and will be used to retrieve a texel. Therefore, the name of the texture sampler in HLSL must be DiffuseSampler, as well.

sampler2D DiffuseSampler;

sampler2D is another data type that is supported in HLSL, and is used to sample a texel from a 2D texture. There are also other samplers, such as sampler1D, sampler3D and samplerCUBE.

Now, we are ready to write the pixel shader function.

Pixel Shader Function
Let’s take a look at the function header first

float4 ps_main( PS_INPUT Input ) : COLOR
{

The only difference from previous pixel shader function headers is that it takes a parameter, and its type is PS_INPUT. This is to receive the UV coordinates the interpolator calculated for us. Equipped with the texture sampler and UV coordinates, we can get the value of the texel. A HLSL built-in function, tex2D() can do the magic. tex2D takes two parameters: a texture sampler and UV coordinates, in order.

float4 albedo = tex2D(DiffuseSampler, Input.mTexCoord);

The above code reads a texel which is located at the coordinates which Input.mTexCoord specifies from DiffuseSampler. And the value will be stored in a variable named albedo. Now what do we do with this value? Well, we wanted to show the texture as is, so let’s just return it.

return albedo.rgba;
}

If we press F5 to compile the vertex and pixel shaders and see the preview window…… uh… it’s messed up!

Figure 3.6 Something is messed up here!

Why? It is because that we forgot to map the UV coordinates element in the vertex buffer to the TEXCOORD semantic. To map it properly, go to Workspace panel and left-click on Stream Mapping. There is currently only one entry: POSITION. Now click on Add button to add a new entry, and then change Usage to TEXCOORD. Make sure Index is 0 and Data Type is FLOAT2. You do not need to change Attribute Name. Once you click on OK button, you will see a proper globe as shown in Figure 3.7.

Figure 3.7 A nice looking globe

By the way, did you notice that I used return albedo.rgba; instead of return albedo; while returning the final color? Although it is completely valid to use return albedo;, I intentionally did so to show you something new.

In HLSL, you can attach a postfix, such as .xyzw or .rgba to a vector variable to access the vector’s components with ease. For example, if we are dealing with a float4 variable, which has a four components, you can think it as an array of four floats. So if you add .x (or .r), it accesses the first component. Likewise, .y (or .g), .z (or .b) and .w (or .a) point to the second, third and fourth components, respectively. So, if you want to get only the rgb value from albedo, you would do something like this:

float3 rgb = albedo.rgb;

Neat, right? But it does not stop here. You can even change the order of the postfix to access vector components in an arbitrary order. Below example shows how to create a new vector with the same components, but in reverse order.

float4 newAlbedo = albedo.bgra;

Or you can even repeat only one channel three times like this:

float4 newAlbedo = albedo.rrra;

Pretty rad. We refer this technique, which allows us to access vectors’ components in any arbitrary order, swizzling.

Maybe you can do some practice here. How about switching the red and blue channels of the globe? Go ahead and try it. It should be a piece of cake for you. :-)

(Optional): DirectX Framework
This is an optional section for readers who want to use shaders in a C++ DirectX framework.

First, make a copy of the framework that we used in Chapter 2 into a new directory. Then, save the shader and 3D model into TextureMapping.fx and Sphere.x respectively so that they can be used in the DirectX framework. Also make a copy of earth.jpg texture file that we used in RenderMonkey. You can find this file in \Examples\Media\Textures folder from RenderMonkey installation folder.

First, let’s look at the global variables. In Chapter 2, we used gpColorShader variable for the shader. Change the name to gpTextureMappingShader:

// Shaders
LPD3DXEFFECT gpTextureMappingShader = NULL;

Also, we need to declare a texture pointer, which will be used to store the globe texture.

// Textures
LPDIRECT3DTEXTURE9 gpEarthDM = NULL;

Don’t forget to release D3D resources that we just declared. Go to CleanUp() function to do so. Doing so makes you a good programmer. You know that, right? ;) Also don’t forget to change the name of gpColourShader.

// release shaders
if (gpTextureMappingShader)
{
gpTextureMappingShader->Release();
gpTextureMappingShader = NULL;
}

// release textures
if (gpEarthDM)
{
gpEarthDM->Release();
gpEarthDM = NULL;
}

Now we will load the texture and shader. Of course, we do this in LoadAssets() function.

First, change the name of shader variable and file to gpTextureMappingShader and TextureMapping.fx, respectively.

// loading shaders
gpTextureMappingShader = LoadShader("TextureMapping.fx");
if (!gpTextureMappingShader)
{
return false;
}

Then, load earth.jpg file by using LoadTexture() function that we implemented earlier in this book.

// loading textures
gpEarthDM = LoadTexture("Earth.jpg");
if (!gpEarthDM)
{
return false;
}

Now go to RenderScene() function which takes care of all the drawings. There are multiple places where gpColorShader variable is used. Find and replace them all to gpTextureMappingShader.

There was a newly added global variable in the texture mapping shader, right? Yes, the texture sampler. But we can’t just assign the texture to the sampler directly in the D3D framework; instead, we have to assign it to a texture variable. Do you remember there was something called DiffuseMap? That was the texture variable. Then, you would think we should be able to assign the texture to a shader variable named DiffuseMap, right? Well that’s the most sensible thing to do, but guess what? RenderMonkey changed the texture variable’s name to something else. If you open TextureMapping.fx file in Notepad, you will see there’s only one variable which data type is texture, and apparently RenderMonkey added _Tex postfix to it. Bad, Bad Monkey!

texture DiffuseMap_Tex

Well, complaining does not solve anything. So we will just use this variable name. In order to pass a texture to a shader, we use SetTexture() function. Like SetMatrix() function, it takes the variable name in the shader as the first parameter.

gpTextureMappingShader->SetTexture("DiffuseMap_Tex", gpEarthDM);

Now, compile and run the program. You should be able to see the same visual as RenderMonkey showed us. Hey! I have an idea. Why don’t we do something cooler? Let’s make it rotate! After all, it is the earth!

First, add a global variable which will remember the current rotation angle.

// Rotation around UP vector
float gRotationY = 0.0f;

The rotation and position of a 3D object are part of the world matrix. So, let’s go back to RenderScene() function and change the world matrix construction code like this:

// for each frame, we rotate 0.4 degree
gRotationY += 0.4f * PI / 180.0f;
if (gRotationY > 2 * PI)
{
gRotationY -= 2 * PI;
}

// world matrix
D3DXMATRIXA16 matWorld;
D3DXMatrixRotationY(&matWorld, gRotationY);

The above code keeps adding 0.4 degree to the rotation each frame. Depending on the computer you are using, this might make the globe to rotate too fast or slow. Change the value appropriately. [8]

Run the code again. You can see the rotating earth, right?

Summary
A quick summary of what we learned in this chapter:

UV coordinates are required for texture mapping.
UV coordinates are varying values across vertices, thus defined on each vertex.
A pixel shader requires a vertex shader’s help to access vertex data.
Any data returned by a vertex shader goes through the interpolator.
tex2D() function is a magic function for texture sampling.

I cannot think of an advanced shading technique which doesn’t rely on texture mapping. So, texture mapping is very crucial in shader programming. Fortunately, performing a texture lookup is very easy with HLSL, so please practice it enough so that you can use it anytime!

Congratulations! You just finished texture mapping. Now take some break, and see you in Chapter 4. :D

----------------
Footnotes:

You are basically mapping a pixel to a vertex.
There are different ways of handling the UV coordinates outside 0~1 range. The current explanation is only valid when texture wrapping mode is used. Other modes, such as mirror and clamp, are also available.
Again, this explanation is only correct with wrap mode.
An abbreviation for texture coordinates.
UV coordinates are same only when the positions of pixels are same as the vertices’.
If you are having a hard time understanding this term just think it this way: it blends the values defined on three vertices. But by how much? Based on the distances to the vertices.
As a pixel is the smallest element in a picture, a texel is the smallest element in a texture.
For a real game, you would measure the elapsed time since the last frame and use it to calculate the proper rotation delta. This book’s code is definitely not ready for real-world applications. :P

Next Chapter

[Intro to Shader] 02. Red Shader

2014-01-07T09:45:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Chapter 2. It’s Really Easy! Red Shader

New HLSL in this chapter

float4: a vector data type with 4 components
float4x4: 4 X 4 matrix data type
mul(): multiplication built-in function. Can handle almost all data types
POSITION: Semantic for vertex position. Useful to read only the position info from vertex data

New math in this chapter

3D space transformation: uses matrix multiplication

In the previous chapter, we defined shaders as functions that calculate the positions and colors of pixels. Then, we should try to write a shader that actually does the job, right? We are going to write a very simple shader here so that even readers with no shader programming experience can follow easily. What about a shader that draws a red sphere? [1] We are going to use RenderMoney for this and it will be your first time seeing any HLSL syntax. Yay! Excited? Once you write a shader in RenderMonkey, you can export it to a .FX file, which can be loaded directly into the DirectX framework we prepared in the last chapter.

Initial Step-by-Step Setup
Please follow these steps in order to start writing this shader

Launch RenderMonkey. A quick scary-looking monkey will welcome you for a moment, and there will be an empty workspace.
Inside Workspace panel, click the right mouse button on Effect. You will see a pop-up menu.
From the pop-up menu, select Add Default Effect > DirectX > DirectX, in order. Can you see a red sphere in the preview window?
You will also see a new shader named Deafult_DirectX_Effect in Workspace panel. Change the name to ColorShader.

Now the screen should look like Figure 2.1.

Figure 2.1 Our RenderMonkey project after initial setup

Vertex Shader
Now, click on the plus(+) sign right next to ColorShader. Can you see Pass 0 at the very bottom? Again, click on the plus sign next to it and double-click on Vertex Shader. You will see the code for vertex shader in the shader editor window on the right. Well, this code is actually what we want: it draws a red ball! But we really need to practice, so let’s just delete all the code in it.

Did you delete the code? If so, let’s get it started! *plays music* First, I’ll show you the full source code for vertex shader below, and explain it line by line after.

struct VS_INPUT
{
float4 mPosition : POSITION;
};

struct VS_OUTPUT
{
float4 mPosition : POSITION;
};

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

VS_OUTPUT vs_main( VS_INPUT Input )
{
VS_OUTPUT Output;

Output.mPosition = mul( Input.mPosition, gWorldMatrix );
Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

return Output;
}

Global Variables vs Vertex Data
There are two types of input values for shaders: global variables and vertex data. Where to put your input values between these two totally depends on whether all vertices in a mesh can use a same value or not. If a same value is used, you should pass it through a global variable. Otherwise, you cannot use a global variable: you have to pass it as part of the vertex data. (i.e. vertex buffer). [2]

Good candidates for global variables include world matrix and camera position. On the other hand, the position and UV coordinates of each vertex are good examples of vertex data.

Input Data to Vertex Shader
First, we are going to declare vertex input data as VS_INPUT structure.

struct VS_INPUT
{
float4 mPosition : POSITION;
};

Do you remember that the most important role of a vertex shader is transforming each vertex’s position from one space to another? It was mentioned in Chapter 1: What is Shader. To do so, you need the vertex position as your input. That is why the above structure retrieves the vertex position via the member variable, mPosition. The reason why this variable is able to retrieve the position info from a vertex buffer is because of POSITION semantic.[3] Often a vertex buffer contains many different data, such as the position, UV coordinates and normal vector, for each vertex. Semantics help you to extract only attributes that are meaningful to you.

So when you see float4 mPosition : POSITION;, it is actually an order to your graphics hardware saying “extract the position information from my vertex data and assign it to mPosition!”

Oh, right. I almost forgot. Then what is float4? This is the variable’s data type. float4, a built-in type supported by HLSL, is nothing more than a vector that has four components: x, y, z and w. Of course, each component is a floating-point type as the type name suggests. HLSL also supports other data types, such as float, float2, float3. [4]

Output Data from Vertex Shader
Now that the input data for vertex shader is declared, we need to turn our eyes to output data. When something goes in, something else should come out, right? Do you still remember the super simplified diagram of a GPU pipeline from <<<Chapter 1? In that picture, vertex shader had to output transformed vertex positions so that rasterizer can figure out each pixel’s position from them. A key point here: a vertex shader must return a transformed vertex position! Okay, that was enough nagging. Now let’s declare the output data as a structure named VS_OUTPUT.

struct VS_OUTPUT
{
float4 mPosition : POSITION;
};

It looks very familiar, right? We are returning the position data as float4 type. How do you(and your GPU) know its position? You see the semantic, POSITION? Yes! That’s how.

Global Variables
We need to declare a number of global variables that will be used in the vertex shader, but doing so without understanding what space transformation is sounds dumb to me.

3D Space Transformation
I said that we need to transform vertex positions into different spaces to draw a 3D object on the monitor. Then, what kind of spaces should we go through to show it? Do you like apples? Let’s use an apple as an example.

Object Space
Let’s assume we have an apple in our left hand. The center of the apple is the origin point and from this point we can make 3 axes: one to the right direction (+x), one to the up direction (+y), and the other to the forward direction (+z). If you measure every single points on the surface of the apple, you can represent each points in (x, y, z) coordinates, right? Now you group every 3 points to build triangles. The result is an apple model!

Now we move the arm here and there while holding the apple. Even if the hand’s position is different, the distance from the origin to each points on the surface of the apple is same, right? This is the object space, or local space. In the object space, every objects (3D models) has its own coordinate system. If you think about it, it’s kind of neat. But if you want to handle different objects in a same manner, it’s a bit challenging because everyone has its own space! That is where the world space comes in.

Figure 2.2 An example of object space

World Space
Now why don’t you leave the apple right next to your monitor? The monitor is also an object, so it should have its own object space, too. Now we want to handle these two objects in a same manner. So what should we do? It’s simple. We just need to bring those two objects into a same space. To do so, we need to make a new space. Do you have a door in the room you are in now? (I really hope you do! :-P ) Let’s put an origin at the door and build 3 axes, +x, +y and +z, to right, up and forward directions, respectively. Now, from this origin, can you build new (x, y, z) coordinates for each vertices on the surface of the monitor? You should be able to do the same with the apple, too. We can call this space the world space.

Figure 2.3 An example of world space

View Space
Now, bring out your camera and take two pictures. Make sure the first picture has all these two objects in it, and the second picture doesn’t have any of those in it. These two pictures are totally different, right? In the first space, you can see the objects, while you can’t at all in the second picture. It means that there got to be positional difference, but positions of the objects in the world space didn’t change at all. A-ha! The camera must be using another space! We call this space view space. The origin of the view space is at the center of camera lens, and you can again make 3 axes to the right, up and forward directions.

Figure 2.4 An example of view space. Objects are inside of the camera’s view

Figure 2.5 An example of view space. Objects are outside of camera’s view.

Projection Space
When you see a picture that’s taken by your everyday cameras, far-away objects look smaller than ones close by the lens. Just like how you see the world through your eyes. Do you know why our eyes work this way? It’s because we humans have a field of view of roughly 100 degree horizontally and 75 degrees vertically. So you get to see more stuff as the distance increases but you are squeezing the “more stuff” into your fixed-size retinas. Your everyday camera does exactly same thing, but there is a special type of cameras called orthogonal cameras. These cameras don’t have field of views: instead, they always look straight forward. So if you use these cameras, you can get the consistent object sizes regardless of the distance.

Well, then we should be able to break down these photo-taking steps into two. First step is transforming objects from the world space to the camera space by applying scale, rotation and translation. Second step is projecting these objects onto a 2D image. (i.e., retina in the previous example) So can you distinguish the spaces we used for each of these two steps? Yes, they are view and projection spaces, respectively. With this separation, your view space is independent from the types of projections, such as orthogonal and perspective projections, you are using.

Once you apply the final projection transformation, the transformed result is the final image shown on the screen.

Summary
A usual way of transforming a vertex’s space in 3D graphics is matrix multiplication. The number of spaces that an object goes through in order to be displayed on the screen is three: world, view and projection spaces. So you need three matrices, as well. By the way, if you know any space’s origin and three axes, you can easily make a matrix that represents that space. [5]

K, now let’s sum it up. These are the space transformations that an object is going through:

Object --------> World --------> View ------> Projection
ⅹWorld Mat ⅹView Matrix ⅹProjection Matrix

Since all these matrices are uniform for all the vertices in an object, global variables should be used.

Global Variable Declarations
Now you should have a very clear idea which global variables are needed, right? We need world, view and projection matrices in order to transform vertices. Let’s insert add the following three lines in the vertex shader code.

float4x4 gWorldMatrix;
float4x4 gViewMatrix;
float4x4 gProjectionMatrix;

Hey, this is something new! float4x4! This is another data type supported by HLSL. It’s very straight forward, right? Yup, this represents a 4 X 4 matrix. As you can guess, there are also similar data types like float2x2 and float3x3.

Now we have these matrices declared. But, who is in charge of passing the values to these variables? Usually the graphics engine in a game takes care of this. But we are using RenderMonkey, so we should follow this monkey’s rule. RenderMonkey uses something called variable semantics to pass values to global variables.

Please follow these steps to set the values to the globals:

In Workspace panel, find ColorShader and right-click on it.
From the pop-up menu, select Add Variable > Matrix > Float(4x4) in order. It will add a new variable named f4x4Matrix.
Change this variable’s name to gWorldMatrix.
Now, right-click on gWorldMatrix to select Variable Semantic > World. This is how you pass a value to a variable in RenderMonkey.
Repeat above steps to make view and projection matrices too. Name the variables as gViewMatrixand and gProjectionMatrix, respectively. Also don’t forget to assign variables semantics of View and Projection.
Lastly, delete matViewProjection variable. This was added by default when we were adding the effect. We do not need this now because we are use gViewMatrix and gProjectionMatrix, instead.

If you finished all the above steps, your Workspace should look like Figure 2.6:

Figure 2.6 Workspace panel after assigning variable semantics

Vertex Shader Function
Finally, all prep works are done. It’s time to write the vertex shader function! First up! The function header!

VS_OUTPUT vs_main( VS_INPUT Input )
{

What the function header means are:

This function’s name is vs_main.
The name of the input parameter is Input and its type is VS_INPUT structure.
The return type of this function is VS_OUPUT structure.

This is not different from how you define a function in C, right? As mentioned before, HLSL uses a C-like syntax. Now, let’s look at the next line.

VS_OUTPUT Output;

This is nothing more than declaring a variable of VS_OUTPUT type that we are going to return at the end of the function. Do you remember what the member of VS_OUTPUT structure was? There was only one member: mPosition, which is in the projection space. That means we are finally apply space transformations! First, we transform the object-space position, stored in Input.mPosition, to the world space. How do you transform a vertex? Yes! You multiply a matrix to it. Since the position vector is a float4, we should multiply a float4x4 matrix, right? Wait. You don’t need to flip through your math book to find a way to do this. HLSL already provides an almighty built-in mul() function that can multiply so many different types together. So, you can simply transform the position by calling this function like below:

Output.mPosition = mul( Input.mPosition, gWorldMatrix );

Above code multiplies the world matrix, gWorldMatrix, to an object-space vertex position, Input.mPosition, and assign the result, which is the world-space position, to Output.mPosition. And you do almost exactly same thing to transform the position into the view and projection spaces.

Output.mPosition = mul( Output.mPosition, gViewMatrix );
Output.mPosition = mul( Output.mPosition, gProjectionMatrix );

Nothing complicated, right? Then what do we need to do now? Well, the most important role of a vertex shader is transforming a vertex’s position, which is originally in the object-space, to the projection space…. Um…. I think we just did it, right? Then, let’s just return the result to finish this vertex shader section.

return Output;
}

Take a moment and press F5 key to compile the vertex shader. You see a red sphere, right? This means we finished the vertex shader section with a great success! If you see any compiler error, please review the code again to see if there is any mistake.

Tip: Got a Shader Compile Error?
If RenderMonkey fails at compiling your code because of typo or invalid syntax, you will see the error messages in the preview window. To see the details about the error, take a look at the output window at the very bottom of RenderMonkey. It should display detailed error messages as well as exact line and column numbers of where the problems are.

Pixel Shader
Now it’s time to write the pixel shader. As we did in the Vertex Shader section, find Workspace window and double-click on Pixel Shader. Now, please delete all the code in it. You should type code with your fingers to learn how to code, so please delete all the code in it.

Let’s take a look at the full source code, which is only 4-lines long, and then I’ll explain it line by line.

float4 ps_main() : COLOR
{
return float4( 1.0f, 0.0f, 0.0f, 1.0f );
}

The most important role of a pixel shader is returning a color value and we want to draw a red sphere in this chapter. So we can just return red here. But, here’s a question: how do you represent red in RGB values? If you are thinking RGB(255, 0, 0), you need to read the following section before writing any pixel shader code.

How to Represent a Color
The reason why most beginners think (255, 0, 0) for the RGB values of the red color is because we are so used to a 8-bit-per-channel format to store an image. An 8-bit integer can represent 256 distinct values. (2^8 = 256) So if you start from 0, you can make 256 integer numbers ranging from 0 to 255. Then, what happens if 5 bits are used per channel instead of 8? 2^5 equals to 32, so 31 will be the maximum value this time. This means that the red color is (255, 0, 0) in a 8-bit format, while it is (31, 0, 0) in a 5-bit format. What a bummer!

So now we know what the problem is. Then, is there any way to represent colors uniformly regardless how many bits are used per channel? If you have played with HDR images in an image editing software, such as Adobe Photoshop, you probably know the answer already. Yes. You can use percentage (%) notation. With this notation, the RGB values of the red color always become (100%, 0%, 0%). This is how shaders represent colors, too. Well almost. You know 0~100% is same as 0.0~1.0 right? So, shaders represent this color as RGB (1.0, 0.0, 0.0).

Pixel Shader Function
Now we know what RGB values need to be output from the pixel shader. So let’s write the function now. First is the function header:

float4 ps_main() : COLOR
{

What this line of code means are:

the function’s name is ps_main
this function doesn’t take any parameters
this function returns a float4
the return value will be treated as COLOR

One thing to note here is that float4 is used for the return type instead of float3. 4th component is the alpha channel, which is normally used for transparency effect. [6]

By the way, what did we say we need to do in this function? Oh right, we need to return red. The code should be as simple as this:

return float4( 1.0f, 0.0f, 0.0f, 1.0f );
}

Two things worth mentioning here:

a color is encoded in a float4 vector in float4(r, g, b, a) form; and
the value of the alpha channel is 1.0, or 100%, so the pixel is completely opaque.

Now press the F5 key inside the shader editing window to compile vertex and pixel shaders. You will have to do it twice. Once for the vertex shader, and once for the pixel shader. Then as shown in Figure 2.7, you will see a red sphere in the preview window.

Tip: How to Compile a Shader in RenderMonkey
You need to compile vertex and pixel shaders separately in RenderMonkey. Open up each shaders in the shader editor and press F5. When the preview window is about to open, both shaders get compiled, too.

Figure 2.7 Our very first craft! So bloody red!

It was really simple, right? What if you want to show a blue ball instead of a bloody one? Returning float4(0.0, 0.0, 1.0, 1.0) would do it, right? What about green? What about yellow? Yellow is basically a mix of green and red, so…. Oh well, you should be smart enough to know. So, I will stop bothering you here. :)

Now, make sure to save this RenderMonkey project somewhere safe. Actually, save your RenderMonkey project at the end of every chapter because you will re-use them in the following chapters.

(Optional) DirectX Framework
This is an optional section for readers who want to use shaders in a C++ DirectX framework.

First, make a copy of the framework that we made in Chapter 1: What is Shader into a new directory. The reason why we make a copy of the framework for each chapter is because we will extend this framework for each chapter.

Next, it is time to save the shader and 3D model we used in RenderMonkey into files so that they can be used in the DirectX framework.

From Workspace panel, find ColorShader and right-click on it.
From the pop-up menu, select Export > FX Exporter.
Find the folder where we saved the DirectX framework, and save the shader as ColorShader.fx.
Again, from Workspace panel, right-click on Model.
From the pop-up menu, select Save > Geometry Saver.
Again, find the DirectX framework folder, and save it as Sphere.x.

Okay, now go ahead and open the framework’s solution file in Visual C++. We are going to add the following code in ShaderFramework.cpp file.

First, we will #define some constants that will be used for the projection matrix.

#define PI 3.14159265f
// Field of View
#define FOV (PI/4.0f)
// aspect ratio of screen
#define ASPECT_RATIO (WIN_WIDTH/(float)WIN_HEIGHT)
#define NEAR_PLANE 1
#define FAR_PLANE 10000

Then we declare two pointers that will store Sphere.x and ColorShader.fx files after loading.

// Models
LPD3DXMESH gpSphere = NULL;

// Shaders
LPD3DXEFFECT gpColorShader = NULL;

Don’t you think it’s time to load the model and shader files now? We will add some code to LoadAssets() function that we left empty in <<<Chapter 1.

// loading shaders
gpColorShader = LoadShader("ColorShader.fx");
if (!gpColorShader)
{
return false;
}

// loading models
gpSphere = LoadModel("sphere.x");
if (!gpSphere)
{
return false;
}

To load files, the above code calls LoadShader() and LoadModel() functions, which were implemented in Chapter 1: What is Shader. If any of these results in a NULL pointer, it returns false, meaning “fail to load.” When this happens, there should be some error messages in the output window of Visual C++, so please take a look.

Whenever you load new resources, don’t forget to add code to release D3D resources, too. It is to prevent GPU memory leaks. Go to CleanUp() function and insert the following code right before releasing D3D.

// release models
if (gpSphere)
{
gpSphere->Release();
gpSphere = NULL;
}

// release shaders
if (gpColorShader)
{
gpColorShader->Release();
gpColorShader = NULL;
}

Alright, it’s almost done. The last step is drawing the 3D object with our shader. We said we will put 3D drawing code inside RenderScene() function, right? Let’s go to RenderScene() function.

// draw 3D objects and so on
void RenderScene()
{

Do you remember we used some global variables in the shader? RenderMonkey used something called variable semantics to assign the values to these variables, but we don’t have that luxury in our framework. Instead, we have to construct those values and manually pass them to the shader. K, construction time! View matrix is first!

// make the view matrix
D3DXMATRIXA16 matView;
D3DXVECTOR3 vEyePt(0.0f, 0.0f, -200.0f);
D3DXVECTOR3 vLookatPt(0.0f, 0.0f, 0.0f);
D3DXVECTOR3 vUpVec(0.0f, 1.0f, 0.0f);
D3DXMatrixLookAtLH(&matView, &vEyePt, &vLookatPt, &vUpVec);

As shown in the above code snippet, a view matrix can be constructed with D3DXMatrixLookAtLH() function once we have these three information:

The position of a camera,
The position where the camera is looking at, and
The upward direction of the camera. (also known as the up vector)

In this chapter, we assume the camera’s current position is at (0, 0, -200) and is looking at the origin (0, 0, 0). In a real game, you would retrieve these information from your camera class.

Next is the projection matrix. Depending on projection techniques being used, we need to call different functions with different parameters. Remember there are two different projection techniques? Yes, perspective and orthogonal. We will use perspective projection here, so the function of choice is D3DXMatrixPerspectiveFOVLH(). [7]

// projection matrix
D3DXMATRIXA16 matProjection;
D3DXMatrixPerspectiveFovLH(&matProjection, FOV, ASPECT_RATIO, NEAR_PLANE, FAR_PLANE);

Yay! One more matrix to go! World Matrix! A world matrix is combination of the following three properties of an object:

position,
orientation, and
scale

What this means is that each object should have its own world matrix. For this example, we assume the object is at the origin (0, 0, 0) without any rotation or scale, so we will just leave our matrix as an identity matrix.

// world matrix
D3DXMATRIXA16 matWorld;
D3DXMatrixIdentity(&matWorld);

Now that we constructed all three global variables, we can pass these values to the shader. You can do this very easily by using the shader class’ SetMatrix() function. First parameter of SetMatrix() function is the name of variable in the shader, and the second is a D3DXMATRIXA16 variable declared above in the framework.

// set shader global variables
gpColorShader->SetMatrix("gWorldMatrix", &matWorld);
gpColorShader->SetMatrix("gViewMatrix", &matView);
gpColorShader->SetMatrix("gProjectionMatrix", &matProjection);

Once all necessary values are passed to the shader, it is time to order the GPU: “Use this shader for anything being drawn from now on!” To give this order, use Begin() / BeginPass() and EndPass() / End() functions. Any meshes drawn between BeginPass() and EndPass() calls will use the shader. Let’s look at the below code first.

// start a shader
UINT numPasses = 0;
gpColorShader->Begin(&numPasses, NULL);
{
for (UINT i = 0; i < numPasses; ++i)
{
gpColorShader->BeginPass(i);
{
// draw a sphere
gpSphere->DrawSubset(0);
}
gpColorShader->EndPass();
}
}
gpColorShader->End();

Do you see DrawSubset() call which is wrapped by BeginPass() and EndPass() calls, which are again wrapped by Begin() and End() calls? This will make the GPU to draw gpSphere object with gpColorShader shader.

Some readers might wonder “Why is there BeginPass() function call after Begin()? What is a pass?” Confusing, right? Well, here’s a good news; you do not need to worry about it. Passes are only useful when you draw a same object multiple times at once, but we barely do this in real world, so let’s just ignore it for now. Just remember that the address of numPasses variable is passed to Begin() function to get the number of passes that exist in the shader. Most of the time, the number is 1. If there are two or more passes in the shader, that means there are more than one vertex/pixel shader pairs, too. So you just need to call BeginPass() / EndPass() as many times as the number of passes.

Now, compile and run the program. You will see the exact same red sphere that you saw in RenderMonkey.

Summary
A quick summary of what we learned in this chapter:

Per-vertex data is passed as member variables in vertex data.
Shared data between all vertices is passed as global variables.
HLSL provides vector-operation-friendly data types, such as float4 and float4x4.
When transforming vertices into different spaces, matrices are used. To multiply a matrix to a vector, use HLSL intrinsic function, mul().
HLSL represents a color in a normalized form. [0 ~ 1]

What we learned here is the basics of basics. So unless you can write this simple shader effortlessly, you shouldn’t attempt to write other shaders. While I was teaching at a college, I saw some students who did not bother to write this red shader because they thought it was too easy, but later they had hard times with other shaders. It was not because the other shaders were hard, but because those students failed to learn very basic HLSL syntax with this red shader. Therefore, please take your time to write this simple shader once or twice before moving to the next chapter.

------
Footnotes:

You will be surprised to see how often graphics programmers use this one-color shader for debugging purposes.
Not entirely true. Textures can be used for this, too.
What the heck is a semantic? Just think it as a tag.
By the way, GPUs are optimized to handle floating-pointing vectors. So, don’t worry about using floats over ints. floats are often faster than ints on GPUs.
For more details on how to manually construct these matrices, please refer to your 3D math book.
If this value is 1, the pixel is opaque, and 0 means the pixel is completely transparent.
Please use D3DXMatrixOrthoLH() for orthogonal projection.

Next Chapter

Snow Effect in Google+ Photo

2013-12-23T10:39:00.002-08:00

I didn't know Google+'s Auto Awesome mode does snow effects too. It was a pleasant surprise to find out what it did to a picture I took recently. It would make a good x-mas card :)

CUDA Compile Error MSB3721 with Exit Code 1

2013-12-16T09:45:00.000-08:00

So I finally decided to play with CUDA a bit. I love Visual Studio, so I created a CUDA project in VS2012(CUDA SDK doesn't support VS2013 yet) and tried to compile it. But uhoh.. I got this error:

error MSB3721: The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\bin\nvcc.exe" -gencode=arch=compute_10,code=\"sm_10,compute_10\" --use-local-env --cl-version 2012 -ccbin "C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\bin" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include" -G --keep-dir Debug -maxrregcount=0 --machine 32 --compile -cudart static -g -DWIN32 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Zi /RTC1 /MDd " -o Debug\kernel.cu.obj "C:\test\kernel.cu"" exited with code 1.

I love how the error message doesn't tell me anything about the error. Yay error code 1..

Anyways, It turns out that COMODO's auto-sandbox is sandboxing it. I added nvcc.exe and msbuild.exe to the sandbox exception list, but I was still getting the same error. Eh.. I wish I could have found the permanant solution for this, but I ran out of ideas.

For now, temporarily disabling the autosandbox mode in COMODO whenever I wanna play with CUDA would cut it. Why can't COMODO and NVIDIA work it out together?

How Much Canadian Programmers Make

2013-12-10T09:30:00.000-08:00

I didn't know that Canada government publishes the salary survey information for programmers until today. But here it is.. this shows how much we, Canadian programmers, make for each region.

http://www.workingincanada.gc.ca/LMI_report_bynoc.do?&noc=2173&reportOption=wage

This shows low, median, high hourly wages and the data is pretty up-to-date. (for example, BC stat is from 2011-2012). To calculate the annual salary out of this, simply use this formula:

hourly wage * 40 (hr/wk) * 52 (wk) =

Alberta, the oil nation, comes in first, followed by BC and ON. If you are my neighbour living in BC, you should make:

low: $44,990
median: $79,997
high: $131,997

So are you making enough? :P

Lessons Learned from Voxel Editing

2013-12-02T09:30:00.000-08:00

I have been looking into voxel editing stuff on and off recently, and this is a short(or long) list of practical lessons I learned. All the factual information presented here is already available online. I'm just gathering and summarizing them here with my 2 cents.

1. Understand how isosurface works first
One of the most practical way of manipulating voxel data is by assigning an iso value for each voxel. I have tried some other ways (and even a very original way I came up with), most of them came short. After all, reconstructing meshes from iso values is the most practical and memory-smart way. There are a lot of papers explaining iso surface extraction and what not, but none of those really explains what isosurface is.

I found this tutorial. It's good and it will help you understand the concept of isosurface.

Also the developer(s) of 3D Coat kindly shared how they store their voxel data. You will be surprised how much you can learn about voxel editing by looking at their data structure.

If your interest is not so much of voxel editing, but procedurally generating large terrains with some help from temporary voxel data, read VoxelFarm inc's blog. He's genius.

2. You can "only" modify voxel data implicitly
Once you use iso values to represent your volume data. The only way you can modify your voxel data directly is by changing the iso values. The operations would be very mathematical like increase/decreasing each iso values or averaging across neighbours unless you stamp a new isosurface shape directly by evaluating a distance function(e.g, functions you can find in the tutorial linked above).

There's no one-to-one relationship between your iso value and the final mesh shape, you can't simply say "I want this edge to extend about 0.2 meter, so let me change this voxel value to 0.752" In other words, you don't have a fine control over the final shape. This is still true even if you increase your voxel density crazyily high(e.g, a voxel per a centimeter). So that's why I call it implicit editing.

There are some newer mesh construction methods that can give you very shape edges, but these are not really made for voxel editing as explained in the next point.

3. Traditional marching cube is sufficient
As I just said, there have been new researches on how to generate meshes out of iso data. Particularly, Cubical Marching Square and Efficient Voxel Octree were something I really enjoyed reading and playing with.

CMS can give you a sharp edge IF you capture your data from polygons. It encodes where polygon edges intersect with on a cube surface. But good luck with editing this data in real-time. (e.g, voxel editing like C4 engine or 3D Coat). I just couldn't find any practical way to edit this data while preserving the edge intersection info correctly.

EVO is not about polygon generation. It's a actually a ray-tracing voxel visualizer running on CUDA. It's fast enough to be real-time(very impressive! check out the demo!), and its genius way to store voxel data as a surface(so, sorta 2D) instead of volume needs some recognition. But still same problem here. I don't see any easy way to edit this data.

So basically all these new methods are good for capturing polygons as voxel data and visualize them without any alternation.

Sure, there can be some difference in final meshes generated from traditional MC and CMS, but if your intention is editing voxel data instead of replicating the polygon data precisely, the difference is negligible. As long as there's WYSIWYG in your editing tool, either method is fine. But MC is about 30% faster with my test. (both unoptimized and implemented in a multi-threaded C# application)

4. Optimize your vertex buffer
It's very common to have 1000 ~ 2000 triangles for a mesh generated from 10 x 10 x 10 voxels. If your voxel density is high and your world is large, it can easily give you 15 million triangles. Sending this much vertex data to GPU is surely a bottleneck. I tested on two different GPUs, one very powerful and the other okay-ish. Both were choking.

So at least do some basic vertex buffer optimization: sharing vertices between triangles and average normals from the neighbouring surfaces. You will see 30~50% reduction on vertex buffer size.

5. Use level-of-details
Simply sharing vertices wouldn't be enough. 30~50% save from 15 million is still 8~10 million triangles. Still not good enough! Now you need to generate LOD meshes. If you want to do it in voxel space(and probably you should), you simply sample every n-th voxels for LOD meshes. (e.g, for 1st LOD every 2nd, for 2nd LOD every 4th and so on)

If you use CMS, doing LODs is not that hard. This algorithm has intersection point information, so there will be no discontinuity between lower and higher LODs(if you have a full octree data, that is). But if you use MC as I suggested, you will see gaps. I haven't tried to fill in the gaps yet, but if I do, I will probably try TransVoxel algorithm first. Also Don Williams had a great success with this.

You can also generate lower LODs from your higher LOD polygoins instead of using voxel data directly. That's how Voxel Farm does it. This approach has a nice benefit: you can reproject higher LOD meshes and generate normal maps for the lower LOD mesh. But I'm not a big fan of mesh simplification for the reasons I'm gonna explain below.

6. Mesh simplification might not satisfy you
Another way of optimizing your vertex data is mesh simplification. Voxel Farm is doing it and it looks good enough for them, but with my test case, it was not giving me satisfactory results. I just didn't like the fact that I didn't have too much control over what triangles will be merged and what will not. It's all based on error calculation method you use, but I just was not be able to find an one-fits-all function for this.

Again, you might find it working fine for your use, but it was not the case for me.

K, that's all I can think of for now.. so happy voxeling... lol

Blendable RGBM

2013-11-25T10:00:00.000-08:00

What is RGBM?

Using a fat buffer(16 bit-per-channel) is the most intuitive way to store a HDR render target, but it was too slow and it used too much memory, so we decided to encode(pack) it into a 8 bit-per-channel buffer. There are different packing methods, but I believe the most widely used method was RGBM. (also this link has a nice summary of LogLUV too, so give it a read if you are not familiar with this topic at all)

Packing.. Yum.... It's nice: it saves bandwidth and memory...

RGBM Limitations

But... YUCK we couldn't do one thing that artists absolutely love to see.... ALPHA BLENDING..

We simply can't do alpha blending on a RGBM buffer. In other words, you shouldn't have semi-transparent objects. Why? With RGBM, you need this formula to blend pixels properly.

( DestColour * DestMultiplier * InvSrcAlpha + SrcColor * SrcAlpha ) / SrcMultiplier

Let's look at some not-too-obvious parameters:

DestMultiplier: stored in DestAlpha channel, which is easily available to our blend unit
SrcMultiplier: alpha channel output from shader
SrcAlpha: transparency value. but where do we store this? Usually in alpha channel. but with RGBM, alpha channel is used for the multiplier... eh... eh... oh yeah! there is something called "premultiplied alpha." by multiplying alpha value to SrcColor in shader, you can still get this working. yay?
InvSrcAlpha: but this is where we fail miserably. We don't have transparency value anymore, and there's no easy-n-efficient way to pre-multiply InvSrcAlpha to DestColour....... yes.... we are doomed.

Existing Solution

So what did we do to solve this problem? eh.. nothing... We simply bypassed this problem by introducing a separate off-screen buffer. We drew all semi-transparent objects onto this buffer with no HDR encoding, and merged the result onto the scene buffer later after tone mapping was done on the HDR(opaque) buffer

So basically you get HDR only on opaque objects and all semi transparent objects will be in LDR. But merging, or running an extra full-screen pass, was also slow, so we experimented with half-res or quarter-res approaches here to counter-balance the extra cost of keeping another render target.

Well, but it still use more memory than simply using only a RGBM buffer.

Blendable RGBM

So here I present my hack. Blendable RGBM. Here is a short summary.

it blends transparent objects in a mathematically correct way

so it doesn't use any extra memory

transparent pixels reuses the multipliers from the opaque pixels. which are already in the render target
but it often suffers from lower precision, so if there is a huge difference in range between the opaque and alpha pixels

you might see the bending effect (when alpha objects are darkening), and
the alpha objects might not be able to brighten pixels over what the opaque multiplier can allow

Okay, now some more details....

Making Math Work

Let's see the RGBM blending formula again:

( DestColour * DestMultiplier * InvSrcAlpha + SrcColor * SrcAlpha ) / SrcMultiplier

I said we are failing miserably because of InvSrcAlpha. And this was because we were filling shader's alpha output with SrcMultiplier. So what are we going to do about it? Just throw away SrcMultiplier... LOL.... Instead, we will simply match SrcMultiplier to DestMultiplier.. After this hackery hack, the formula becomes this:

( DestColour * DestMultiplier * InvSrcAlpha + SrcColor * SrcAlpha ) / DestMultiplier

Once simplified, it becomes like this:

DestColour * InvSrcAlpha + SrcColor * SrcAlpha / DestMultiplier

Alright, so now alpha channel is not used, so we can output our SrcAlpha here.

Setting Up Correct Blending Render States

I love to mess with hardware blending operations to make a new thing. I've done this once with Screen Space Decals before. And I did it again here!

To make this magic work, we need to change render states like this:

SrcBlend: DestAlpha
DestBlend: InvSrcAlpha

After this change, the blending becomes very similar to the above formula with one exception: multiplying of SrcAlpha is missing. Since we are already setting DestAlpha as SrcBlend, we can't set SrcAlpha to the blending unit. The solution? Premultiplied alpha again. :-) Premultiply SrcAlpha to SrcColor in shader.

So are we good? no.. not really. With our current blending states setup, we get this:

DestColour * InvSrcAlpha + SrcColor * SrcAlpha * DestMultiplier

Doh! We are multiplying DestMultiplier!! But we have to divide it :( To fix this, we will change RGBM encoding and decoding a bit.

Changing RGBM Encoding/Decoding

This is essentially how "original" RGBM encoding works:

M: max(RGB)
encoding: RGB / M
decoding: RGB * M
alpha encoding: M / 6

To make it work with our new way, we simply reverse M, so that we multiply while encoding and divide while decoding. This will be something like this.

M: 1 / clamp( length(RGB), 1, 6 )
encoding: RGB * M
decoding: RGB / M
alpha encoding: M

The reason why I used length(RGB) instead of max(RGB) was to give it some extra room which alpha objects later can use to brighten the pixels. How I get M is very hacky. Maybe I can replace two 1s with much lower value like 0.125 to allow better precision in LDR. But keeping the min range to 1 guarantees that full LDR range is available for the alpha pass. But, I'm pretty sure someone can come up with a better way of calculating M here. :)

Finally New Blending Formula

With all the changes up to here, our new blending formula looks like this:

( DestColour / DestMultiplier * InvSrcAlpha + SrcColor * SrcAlpha ) * DestMultiplier

Once simplified, it becomes this:

DestColour * InvSrcAlpha + SrcColor * SrcAlpha * DestMultiplier

Oh, hello there! This is exactly what our blending render states setup gave us! Yay! it works!

Disabling Alpha Write

But wait.. if we output our transparency value in alpha channel, the multiplier in HDR buffer will be overwritten, right? We don't want this to happen. We need the alpha value only for blending, so we have to make sure this value is not written to the render target.

We can do it easily by masking alpha from color write. Another render state change will do the magic.

Additive Blending

My way also works with additive blending too. Set this blending states:

SrcBlend: DestAlpha
DestBlend: One

Caution

As I mentioned earlier, this approach suffers from two side effects.

When the transparent objects darkens pixels too much, you will see bending effects due to the lack of precision with this approach. This happens only when the dest pixel was in HDR range.
Transparent objects cannot brighten what's already written in the opaque pass too much.

Using blendable RGBM would make sense only in certain conditions. I have seen those conditions personally. So use it only when you can't afford separate offscreen buffer for memory or performance reasons and when you are willing to sacrifice visual qualities in return. After all, it's a trade-off between visual quality and memory use.

p.s.

If you are working on newer consoles, HDR packing might not be needed anymore. But I'm pretty sure there are still a lot of developers who are stuck with older machines, so hopefully they will find this post helpful. :)

Pope

Mipmap Quality

2013-06-24T09:30:00.000-07:00

So mipmaps are often blurry and artists think we are bunch of morons making their awesome artwork so blurry.. :-) You know, most of the time we see lower mips all the time in the game. It depends on camera settings, but I found it's very rare that we see highest mip in most of the games out there.

3 Widely Used Tricks

I know there are 3 widely used ways: 1) increased texture size, 2) mipmap bias or 3) use of kaiser filter when generating mip chain.

First way can be only used selectively because it increase the memory footprint. Probably this is more viable with next-gen consoles with way bigger memories.

Second way usually introduces too much aliasing because it only pushes back when the mip starts to change. If there's a way to control the curve when consecutive mips kick in, I would love it more. (I can do all this in HLSL of course)

Third way is something I didn't see any improvement personally. Some people say kaiser filter would make mipmap so much better than box filter. But when I tired it, i didn't really see much improvement.

Less Used But More Effective Trick - Sharpening Filter

And there's another way that is not as widely used, but I found this works really well: apply sharpening. (This is why I contributed a generic convolution filter to Nvidia Texture Tools 2. There were a couple of people requesting this feature, but the maintainer didn't see this feature important, so I had to make it by myself)

The concept is very simple. You just do this for a few top mips while generating the mipchain, hopefully in your texture baker.

Generate mip 1 from original texture(mip 0) by using box filter
Apply sharpening filter on mip 1
Generate mip 2 from the result from Step 2 by using box filter
Apply sharpening filter on mip 2
Repeat....

So which technique do you use to increase mipmap quality? Or is there any other trick that I'm not aware of?

[Curiosity Research] Pet Fox

2013-06-10T09:44:00.000-07:00

During this weeked, I stumbled into this video showing a pet fox.

I was like.. "Really? You can have a pet fox?". So I called my brother, Google, and did some reasearch. And yeah it's possible while it's quite restrictive. This PopSci article, posted in Oct 2012, explains it very well.

Popular Science Article: Can I Have A Pet Fox?

Short Summary:

There's distinction between tamed and domesticated animals
You can tame animals by training them since they were young. So if you tame an wild fox, she won't bite you, but she won't play with you. In other words, no ball chasing.. no belly rubbing... no wiggling. Most importantly, the offsprings of tamed animals need to be tamed again.
Domesticated animals are the result from very selective breeding of "human-friendly" wild foxes. They act more like dogs. It's in their DNA, so their children will love you too :)
The only way you can buy domesticated foxes are from Russia. They've been doing this selective breeding since 1950s, so after tens of generations those foxes are really dog-like.
The price is about $6,000
One interesting thing though: those domesticated foxes are showing some physical difference from pure wild counterparts.

so that is today's dose of my curiosity... :P

PIMPL Pattern - Me No Likey

2013-05-21T07:30:00.000-07:00

I haven't used the PIMPL pattern so far. I remember reading about it once long time ago, but was just not convinced it's something I want.

And now I'm working in a codebase where PIMPL patters are abundant. My verdict? Yeah I don't like it. I just don't see the benefit of it in game programming. So what are the benefits of the PIMPL pattern? I can only guess these two:

clear separation between API design and implementation: for example, the grand master designer designs API and minion programmers implement those in hidden places
less header file interdependency = faster compilation time

I don't think #1 means a lot for game programming. Just like in any entertainment business, our requirements keep changing, so are a lot of APIs. Also hiding implementation from other programmers sounds just weird in this industry. We usually have the full access to any source code. After all, we are not library vendors.

I think the main reason why game developers are using the PIMPL pattern is because of #2. C++ sucks with compilation time blaa blaa... But there are other way to make the build faster and PIMPL is definitely not the fastest one I have seen. Then is it incredibuild, which is pretty expensive? No the fastest one(I have seen) is Unity builds, which is completely free.(duh)

So I don't see benefits of PIMPL at all. Instead, I see more disadvantages:

Code is harder to read: who likes to jump around different classes to see actual implementations? Maybe you do, but I don't.
Discrete memory allocation: you need an extra memory allocation to instantiate the pimpl object. I'd much prefer to have all my members in a class, so that there's only one memory allocation and it's straight-forward to find out the size of an object.
Another pointer dereferencoing = slower: sure. some people might say "it's not that slow." but I'm just nazi about it. one additional pointer dereferencing is probably about 4 cycles extra... but with the separate memory allocation for the pimpl object, there's more chance it would trash cache lines. You can probably avoid it by carefully controlling the allocation order, but it sounds like very tedious work that I don't want to deal with.

So, me no likey.... or am I missing something here?

Random thought on 1080+P

2013-05-18T23:54:00.000-07:00

some random thought I had while sitting on a bench at a rainy park today....

I finally watched James Cameron's Avatar the other day on DVD. It was a good movie, but the DVD quality was not that great on my 1080p display. It's after all 480P right? Now people are used to 1080P, so DVD quality wouldn't cut it. (even youtube video looks better than DVD, jesus!)

So then what's next? will we need anything over 1080p? I believe the common belief was that our eyes can't process more than a few million pixels so going over 1080p would be waste.(1080P is about 2 million pixels) But apparently we can perceive way more than that.... So we will need more than 1080P.

But one day TV resolutions will overcome our eyes' capabilities right? Then can we stop adding more pixels on the TV? I don't think our eyes are the limit.. the manufacturing cost would be the deciding factor.. Say our eyes can only process 1,000 million pixels. But if we are given a TV with 4,000 million pixels, our eyes will see 4 pixels as one pixel. right? So it would be same as blending those 4 pixels into 1. Oh sweet! it's a 4X super sampling antialiasing! If it's gets 16,000 million pixels it's 16x SSAA.. I can't say no to this.. :)

But as I said, it might become too expensive to add that many pixels one day... but price of LEDs are going down everyday and smart people are making LEDs use less power. so that might not be a problem.... or it might become physically impossible to add more pixels one day? just like how we coudln't make single-core CPUs faster anymore? Maybe.... but still 576 Million pixels are long way to go....

Goodbye XNA, Hello SharpDX

2013-01-09T11:55:00.000-08:00

I've been using XNA since the beginning. I liked this platform a lot because it allows me to write something that runs both on PC and Xbox 360. And it is even possible to monetize your games on Xbox 360 Indie Games channel. However, the importance of XNA games on Xbox Live turned out to be very minimal, and MS changed XNA a lot to support Windows Phone 7.0. From that moment, (XNA 4.0 I believe), XNA became somewhat awkward. PC and Xbox 360 became second-class citizens, i think.

Still I used XNA quite often. It was a great tool to prototype anything. Its contents pipeline and ability to use DirectX in C# really gave me a huge advantage to make any demo in it. For example, I even made the demo for my SIGGRAPH presentation in XNA.

Now XNA is dead because Windows Phone 8's not gonna support it. WinRT is apparently the new winner....Also Visual Studio 2012 doesn't support XNA officially. (an unofficial way exists, but whatever?) Sure, one can still stay behind with Visual Studio 2010 and XNA 4.0, but now XNA, aftert being neglected for a while, has some limits.

no 64-bit support
only supports DirectX 9 (no DX11 support)

So now time to find an alternative.... The reasons why I liked XNA were:

C# support
contents pipeline

So I just need to find an alternative which covers these two... right?

C# Support

There are two major libraries that supports DirectX in C#, or any .NET language: SharpDX and SlimDX. Both libraries have almost identical API, thus the usage is very similar. Furthermore, both supports 64 bit. I needed 64-bit support for my recent work, so I tried both libraries, and decided to go with SharDX for following reasons:

SharpDX is faster (less overhead on API calls)
it's easier to install SharpDX(simple DLL file copies)

So with SharpDX, C# support issue is covered

Contents Pipeline

Honestly, I still don't see anything that's comparable to XNA's contents pipeline yet. Texture is not a big issue since both C# and SharpDX's DX calls support textures pretty well. When it comes to other asset types, such as audio and mesh, it's still not that easy.

One good news is that Visual Studio 2012 contains an example which loads FBX files, I heard, and I believe Microsoft will enhance this side for WinRT, if they are really betting on Windows Phone 8. So maybe it's just matter of time?

I wish I had some time to make pretty solid contents pipeline for SharpDX or something.... But by looking at my schedule for this year, I don't think I'll ever get to do it.

My Conclusion - SharpDX

So I decided to live with SharpDX for now. My main reason was the 64-bit support. My prototype's already using more memory than what 32-bit application can handle. I'm not worried too much about textures and mesh support... For other contents types, I'll solve the problems when I need them.

Update - Jan 12, 2013
Nicolas just told me his Assimp.NET can handle some asset import/export and it's actually used by SharpDX too. So if you need an asset pipeline check it out at http://code.google.com/p/assimp-net/ Apparently FBX files are not supported yet, but since there's already FBX SDK from autodesk for free, I would think it's just a matter of time.

Legendary Football Player, Dale Winners

2013-01-04T11:59:00.000-08:00

I used to live in a townhouse with a very spacious basement, which my friends and I used to hangout, drink, do some random jams/recordings and sometimes work together. And this is one of the recordings we did... (or should I say Dale did because he's the only one doing all the voice acting?)

It's been in my HDD for the last 2 years, and I finally decided to edit it to show it to the world because it's hilarious. Enjoy

Another Useless Meeting

2013-01-03T09:35:00.000-08:00

I don't remember exactly, but I think it was a presentation on how Apple works internally. A few key points I thought cool:

Every meeting must draw some decisions(action plan)
Don't even bother to do a meeting if no decision will be made
Don't include any employee who has no authority to make decisions

Personally, I don't enjoy wasting my time in useless meetings: I'd rather go back to my desk and do some coding, so I liked the idea.

Later, I actually found myself in a meeting, which followed the above rules very faithfully, but still managed to waste my time. Better, it was even a very lengthy meeting. I won't go into too much details about what was discussed in the meeting. The problem of the meeting was not that it didn't make any decisions: it actually made decisions, lots of them. And the decisions were very sensible. However, those decisions were very obvious even before the meeting to anyone and no other decisions were even possible to be made.

I think the following should be added to the above rules to save my time.

Don't even bother to do a meeting if only one decision can be made from it

[Intro to Shader] 01. What is Shader

2012-12-31T21:01:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Definition of Shader
The very first question I used to get in my class was “what the heck is shader?” Looking back, I also asked this same question when I heard about shader first few times, but no one was able to give me the one sentence explanation that made sense to a noob like me. I eventually figured it out, and I think the following sentence is the easiest way to define what shader is.

Shaders are functions which calculate the position and colour of a pixel on screen.

Still not clear enough? Maybe it is easier to understand if we look at where shaders are used in a modern graphics pipeline.

Simplified 3D Graphics Pipeline
Only vertex and pixel shaders are covered in this book. So, let me quickly show you how these shaders are used in a modern 3D graphics pipeline.

One of the reasons why 3D pipeline exists is to display a 3D object onto a 2D screen. First, take a look at Figure 1.1, which shows an overly simplified 3D graphics pipeline. [1]

Figure 1.1 Overly simplified 3D graphics pipeline

In figure 1.1, what a vertex shader takes as input is a 3D model itself. A 3D model is usually represented with polygons, which are nothing more than a collection of triangles. To make a triangle, you need 3 vertices, right? So, you can just say the input for a vertex shader is an array of vertices, instead. A-ha! Now you know why it is called vertex shader.

Then what does a vertex shader do? The most important responsibility of a vertex shader is transforming vertices of a 3D object into the screen space. You can compare this to how a painter captures real-world scenery onto a canvas. Have you heard of perspective drawing? Even if you draw exactly same two objects onto the canvas, the final sizes on the canvas can be different based on the distance between each of objects and your eyes. In other words, close-by objects look bigger and far-away objects look smaller.

In graphics, we like to say that those two objects had the same size in world space, but now they have different sizes in screen(=canvas) space. Well, do not worry about these spaces. We will learn more about it in the next chapter. I just wanted to tell you that vertex shaders must transform an object from one space to another.

Remember I told you a 3D model is basically a bunch of vertices? So if you transform all the vertices that make up a 3D model one by one, it is same as transforming the 3D object itself. This is exactly how vertex shaders work. Then, how many times a vertex shader will be executed? Exactly. Just same times as the number of vertices in the model.

We can sum up the last paragraph with the following sentence:

The main role of a vertex shader is transforming each vertex’s position into another space.

So, one thing a vertex shader must output at the end of its execution is the vertex position in screen space . Every three of these vertex positions make a triangle, also in screen space. [2]

Now, can you tell me how many pixels would be inside of each triangle? The screen is made of pixels, so we should know how many pixels we need to draw and where we need to draw them, right? This is what the rasterizer unit does. Rasterizer groups every three vertex positions a vertex shader outputs and makes a triangle to find out how many pixels are in it. So now you can guess how many times the pixel shader should be executed, right? Of course, as many times as the number of pixels the rasterizer finds out.

Then, what would be the main work of pixel shader? Here is a hint. This is the last stage of the 3D graphics pipeline before reaching to the screen.

The main responsibility of a pixel shader is to calculate the final colour on screen.

If you combine the roles of vertex and pixel shaders, you finally get the definition I mentioned earlier:

Shaders are functions which calculate the position and colour of a pixel on screen.

Even though we tried to define what shader is, I honestly do not think most beginners would get a firm grasp of it yet. You should actually write some sample codes to do so. Would it help if I say shader is a way to manipulate the positions and colours of pixels while we are drawing a 3D object? Maybe not? Don’t worry. Just keep reading, you will get your eureka moment pretty soon. :)

Shader Programming
K, now we kind of know what shader is, but what does it mean to write a shader program? Let’s look at Figure 1.1 again. In Figure 1.1, do you see that some stages are rectangular while the others are round? Round stages are what GPU (Graphics Processing Unit) does automatically for you, which means we programmers have no control over them. On the other hand, rectangular stages are what programmers can manipulate “freely”. You get to write a function for each of those rectangles. This is what we call shader programming. So, you see there are only two rectangular stages in Figure 1.1, right? Yes, vertex and pixel shaders! So when someone talks about shader programming, he means writing a function for the Vertex Shader unit and another one for the Pixel Shader unit, that’s it! [3]

Just like anything in the life, there are multiple shader languages out there, but what this book uses is HLSL(High Level Shader Language) from DirectX. HLSL uses C-like syntax and is very close to other shader languages, such as GLSL[4] and CgFX[5] . Once you learn HLSL, it is very trivial to switch to another shader languages.

The best way to learn a programming language is writing code. Debating over the philosophy and syntax of a language only makes beginners bored, uninterested or clueless. Once you feel the fun of coding in that new language, all the other things naturally follow. So I will not try to turn you off by listing all HLSL syntax at this moment. Instead, I will force you to write very easy shaders in HLSL first. If you are one of those people who cannot live without knowing all the syntax, please refer to the appendix at the very back. I really do not like to bore out people.

Well, I lied. There are still some initial setups we need to do. It is boring, I know. But you will need it to learn shader programming with this book, so please bear with me? It is not that long.

Preparation for Shader Programming
As mentioned in Introduction, the only focus of this book is shader programming. The reason why I decided not to cover anything about DirectX is because there are many good DirectX books out there, so I did not want to waste any of my time (and pages[6]) to discuss about it. Also I wanted to allow technical artists to learn HLSL programming through this book, so covering DirectX, mostly programmer-only material, was a no-no to me.

To allow technical artists to find this book useful, I separated each chapter into two steps. First step involves writing shader program in an application called Render Monkey from AMD. Both programmers and technical artists should do this step.

Second step, which is only for programmers, plugs the shaders authored in Render Monkey into a C++/DirectX framework. If you are a programmer who is not interested in C++/DirectX, feel free to skip this step, too.

Now, it is time to prepare something for these two steps.

Render Monkey
Render Monkey is a shader authoring tool provided by AMD. I found this tool great for quick prototyping. You can download version 1.82 from AMD website.

Just use the default option when install.

Optional: Simple DirectX Framework
If you are one of those braves wishing to run shaders in the C++/DirectX framework, please read this section.

First, install Visual C++ 2010 and DirectX SDK. If you do not have Visual C++, you can download the express version for free from Microsoft website. You can also download DirectX SDK from the same website.

Once you installed above two programs, open 01_DxFramework/BasicFramework.sln file from this book’s code samples. (You can download them from my blog, www.popekim.com). If you just run this program, you will see something like Figure 1.2.

Figure 1.2. Super simple framework

This framework “supports” the following “features”:

Basic window functions, such as window creation and message loop
Direct 3D device creation
Resource loading, such as textures, models and shaders
Simple game loop
Simple keyboard input handling

By the way, this stripped-down framework is made to run shader codes quickly. As a result, all functions are in a single .cpp file, and it does not use any concept of OOP(Object Oriented Programming). In other words, everything is written in C-style and all variables are globally defined. You see the problem? Yes. IF YOU ARE MAKING A REAL GAME, NEVER EVER WRITE YOUR FRAMEWORK THIS WAY. Again, this framework is intentionally made very simple to allow you to run shader demos very quickly.

Alright, that was enough warning, I think. Now, let’s take a look at the framework. First, open BasicFramework.h file.

//***************************************************************
//
// ShaderFramework.h
//
// Super simple C-style framework for Shader Demo
// (NEVER ever write framework like this when you are making real
// games.)
//
// Author: Pope Kim
//
//***************************************************************

#pragma once

#include <d3d9.h>
#include <d3dx9.h>

// ---------- constants ------------------------------------
#define WIN_WIDTH 800
#define WIN_HEIGHT 600

// ---------------- function prototype ------------------------

// Message procedure related
LRESULT WINAPI MsgProc( HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam );
void ProcessInput(HWND hWnd, WPARAM keyPress);

// Initialization-related
bool InitEverything(HWND hWnd);
bool InitD3D(HWND hWnd);
bool LoadAssets();
LPD3DXEFFECT LoadShader( const char * filename );
LPDIRECT3DTEXTURE9 LoadTexture(const char * filename);
LPD3DXMESH LoadModel(const char * filename);

// game loop related
void PlayDemo();
void Update();

// Rendering related
void RenderFrame();
void RenderScene();
void RenderInfo();

// cleanup related
void Cleanup();

This header file is very straight-forward. You probably noticed WIN_WIDTH and WIN_HEIGHT. These define the window size. All the other codes are just function declarations, and the implementations are all inside ShaderFramework.cpp. So let’s take a look at ShaderFramework.cpp.

You can see all the global variables at the top of the file.

//---------------------------------------------------------------
// Globals
//---------------------------------------------------------------

// D3D-related
LPDIRECT3D9 gpD3D = NULL; // D3D
LPDIRECT3DDEVICE9 gpD3DDevice = NULL; // D3D device

// Fonts
ID3DXFont* gpFont = NULL;

// Models

// Shaders

// Textures

// Application name
const char* gAppName = "Super Simple Shader Demo Framework";

Now time to create the window.

//---------------------------------------------------------------
// Application entry point/message loop
//---------------------------------------------------------------

// entry point
INT WINAPI WinMain( HINSTANCE hInst, HINSTANCE, LPSTR, INT )
{

To create a windows, you need to register a window class first.

// register windows class
WNDCLASSEX wc = {sizeof(WNDCLASSEX), CS_CLASSDC, MsgProc, 0L, 0L,
GetModuleHandle(NULL), NULL, NULL, NULL, NULL,
gAppName, NULL };
RegisterClassEx( &wc );

Now, it is time to create an instance of the window class that we just registered. CreateWindow() functions does this. Use WIN_WIDTH and WIN_HEIGHT for the width and height of the window, respectively.

// creates program window
DWORD style = WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_MINIMIZEBOX;
HWND hWnd = CreateWindow( gAppName, gAppName,
style, CW_USEDEFAULT, 0, WIN_WIDTH, WIN_HEIGHT,
GetDesktopWindow(), NULL, wc.hInstance, NULL );

The funny thing about a windowed program is that the actual area you can render onto is smaller than WIN_WIDTH and WIN_HEIGHT. It is because the window also has other junks like the title bar and border lines. So you need to adjust the window size once it is created so that the renderable area, or client rect, is equal to WIN_WIDTH and WIN_HEIGHT.

// Client Rect size will be same as WIN_WIDTH and WIN_HEIGHT
POINT ptDiff;
RECT rcClient, rcWindow;

GetClientRect(hWnd, &rcClient);
GetWindowRect(hWnd, &rcWindow);
ptDiff.x = (rcWindow.right - rcWindow.left) - rcClient.right;
ptDiff.y = (rcWindow.bottom - rcWindow.top) - rcClient.bottom;
MoveWindow(hWnd,rcWindow.left, rcWindow.top, WIN_WIDTH + ptDiff.x,
WIN_HEIGHT + ptDiff.y, TRUE);

Now that we got the correct windows size, let’s show the window!

ShowWindow( hWnd, SW_SHOWDEFAULT );
UpdateWindow( hWnd );

Next, we initialize Direct3D and load all D3D resources, such as textures, shaders and meshes. InitEverything() function contains all these things. If the program fails at initializing Direct3D or other stuff, it simply quits.

// Initialize everything including D3D
if( !InitEverything(hWnd) )
{
PostQuitMessage(1);
}

Once D3D initialization is completed, what is left is to keep running the demo until WM_QUIT message is sent. WM_QUIT is a window message which nicely asks us to finish the execution of the program.

// Message loop
MSG msg;
ZeroMemory(&msg, sizeof(msg));
while(msg.message!=WM_QUIT)
{
if( PeekMessage( &msg, NULL, 0U, 0U, PM_REMOVE ) )
{
TranslateMessage( &msg );
DispatchMessage( &msg );
}
else // If there's no message to handle, update and draw the game
{
PlayDemo();
}
}

When we need to finish the demo, we unregister the window class and return from the program.

UnregisterClass( gAppName, wc.hInstance );
return 0;
}

We also need to see the function that takes care of window messages.

// Message handler
LRESULT WINAPI MsgProc( HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam )
{
switch( msg )
{

keyboard input is handled by ProcessInput() function.

case WM_KEYDOWN:
ProcessInput(hWnd, wParam);
break;

When the window is being closed, all D3D resources we loaded during the initialization step should be released by calling CleanUp() function. Once it is done, “please terminate this program” message is sent.

case WM_DESTROY:
Cleanup();
PostQuitMessage(0);
return 0;
}

Any message that is not being handled by this function will be sent to the default message procedure, which will, in turn, handle them.

return DefWindowProc( hWnd, msg, wParam, lParam );
}

The only keyboard input this framework listens to at this moment is ESC key. When this key is pressed, the program will be terminated.

// Keyboard input handler
void ProcessInput( HWND hWnd, WPARAM keyPress)
{
switch(keyPress)
{
// when ESC key is pressed, quit the demo
case VK_ESCAPE:
PostMessage(hWnd, WM_DESTROY, 0L, 0L);
break;
}
}

Now, let’s look at the initialization code more closely.

//------------------------------------------------------------
// intialization code
//------------------------------------------------------------
bool InitEverything(HWND hWnd)
{

First, we initialize D3D by calling InitD3D() function. Unless it fails, we call LoadAssets() function to load D3D resources, such as textures, models and shaders.

// init D3D
if( !InitD3D(hWnd) )
{
return false;
}

// loading models, shadsers and textures
if( !LoadAssets() )
{
return false;
}

Next up is font loading. We will use this font to display debug information on screen.

// load fonts
if(FAILED(D3DXCreateFont( gpD3DDevice, 20, 10, FW_BOLD, 1, FALSE,
DEFAULT_CHARSET, OUT_DEFAULT_PRECIS,
DEFAULT_QUALITY, (DEFAULT_PITCH | FF_DONTCARE),
"Arial", &gpFont )))
{
return false;
}

return true;
}

The meanings of parameters used with D3DXCreateFont():

gpD3DDevice: D3D device
20: the height of the font
10: the width of the font
FW_BOLD: use bold style
1: mipmap level
FALSE: do not use italic style
DEFAULT_CHARSET: use default character set
OUT_DEFAULT_PRECIS: defines how close the final font properties displayed on the screen should be to the ones we are setting here
DEFAULT_QUALITY: defines how close the final font quality displayed on the screen to the one we are setting here
DEFAULT_PITCH | FF_DONTCARE: Use default pitch, and I don’t care about the font family
"Arial": the name of font to use
gpFont: stores the newly created font

Now let’s take a look at InitD3D() function, which creates a D3D object and D3D device. In order to load resources or draw with DirectX, you must create a D3D device.

// D3D and device initialization
bool InitD3D(HWND hWnd)
{

First, we create a Direct3D object.

// D3D
gpD3D = Direct3DCreate9( D3D_SDK_VERSION );
if ( !gpD3D )
{
return false;
}

Now, we need to fill in the structure to create a D3D device.

// fill in the structure needed to create a D3D device
D3DPRESENT_PARAMETERS d3dpp;
ZeroMemory( &d3dpp, sizeof(d3dpp) );

d3dpp.BackBufferWidth = WIN_WIDTH;
d3dpp.BackBufferHeight = WIN_HEIGHT;
d3dpp.BackBufferFormat = D3DFMT_X8R8G8B8;
d3dpp.BackBufferCount = 1;
d3dpp.MultiSampleType = D3DMULTISAMPLE_NONE;
d3dpp.MultiSampleQuality = 0;
d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;
d3dpp.hDeviceWindow = hWnd;
d3dpp.Windowed = TRUE;
d3dpp.EnableAutoDepthStencil = TRUE;
d3dpp.AutoDepthStencilFormat = D3DFMT_D24X8;
d3dpp.Flags = D3DPRESENTFLAG_DISCARD_DEPTHSTENCIL;
d3dpp.FullScreen_RefreshRateInHz = 0;
d3dpp.PresentationInterval = D3DPRESENT_INTERVAL_ONE;

Here are some fields worth understanding:

BackBufferWidth: the width of back buffer(rendering area)
BackBuferHeight: the height of back buffer
BackBufferFormat: the format of back buffer
AutoDepthStencilFormat: the format of depth/stencil buffer
SwapEffect: the effect of swap. For performance reasons, D3DSWAPEFFECT_DISCARD is recommended.
PresentationInterval: the relationship between the refresh rate of monitor and the frequency of swapping back buffer. D3DPRESENT_INTERVAL_ONE means back buffer will be swapped whenever monitory v-sync happens. Most computer games swap the back buffer without waiting for V-sync. (D3DPRESENT IMMEDIATE) It is mainly for performance reasons. The most noticeable downside of this mode is the screen tearing

Now that we have this structure filled, we can create a D3D device.

// create D3D device
if( FAILED( gpD3D->CreateDevice( D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL,
hWnd,
D3DCREATE_HARDWARE_VERTEXPROCESSING,
&d3dpp, &gpD3DDevice ) ) )
{
return false;
}

return true;
}

LoadAssets() function is supposed to load D3D resources, but there is no code in it at this moment. You will get to call other functions, such as LoadShader(), LoadTexture() and LoadModel(), to load resources in the later chapters.

bool LoadAssets()
{
// texture loading

// shader loading

// model loading

return true;
}

Next is LoadShader() function, which loads a shader program saved in a .fx file. A .fx file is a text file which can contain both vertex and pixel shader functions. It can be dynamically compiled and loaded via D3DXCreateEffectFromFile() function. So, if there is any syntax error in HLSL code you write, this function will encounter compiler errors. The last parameter of this function is how you retrieve the error messages. We will print out the error messages in Visual C++’s output window.

// shader loading
LPD3DXEFFECT LoadShader(const char * filename )
{
LPD3DXEFFECT ret = NULL;

LPD3DXBUFFER pError = NULL;
DWORD dwShaderFlags = 0;

#if _DEBUG
dwShaderFlags |= D3DXSHADER_DEBUG;
#endif

D3DXCreateEffectFromFile(gpD3DDevice, filename,
NULL, NULL, dwShaderFlags, NULL, &ret, &pError);

// if failed at loading shaders, display compile error
// to output window
if(!ret && pError)
{
int size = pError->GetBufferSize();
void *ack = pError->GetBufferPointer();

if(ack)
{
char* str = new char[size];
sprintf(str, (const char*)ack, size);
OutputDebugString(str);
delete [] str;
}
}

return ret;
}

The meanings of the parameters of D3DXCreateEffectFromFile() function are:

gpD3DDevice: D3D device
filename: the name of shader file to load
NULL: do not use additional #define definitions for shader compilation
NULL: do not use additional #includes
dwShaderFlags: shader compilation flags
NULL: do not use an effect pool object for shared parameters
ret: will store compiled shader
pError: will point to error messages, if any

Next is model loading code. It assumes models are stored in .x format, which is supported by DirectX natively.

// loading models
LPD3DXMESH LoadModel(const char * filename)
{
LPD3DXMESH ret = NULL;
if ( FAILED(D3DXLoadMeshFromX(filename,D3DXMESH_SYSTEMMEM, gpD3DDevice,
NULL,NULL,NULL,NULL, &ret)) )
{
OutputDebugString("failed at loading a model: ");
OutputDebugString(filename);
OutputDebugString("\n");
};

return ret;
}

Again, the meaning of the above parameters for D3DXLoadMeshFromX() function call are:

D3DXMESH_SYSTEMMEM: load the mesh to system memory
gpD3DDevice: D3D device
NULL: Don’t give me adjacency data
NULL: Don’t give me material information
NULL: Don’t give me effect instance
NULL: Don’t give me the number of materials
ret: will store loaded mesh

Finally, let’s look at LoadTexture(), which loads a texture(image) file.

// loading textures
LPDIRECT3DTEXTURE9 LoadTexture(const char * filename)
{
LPDIRECT3DTEXTURE9 ret = NULL;
if ( FAILED(D3DXCreateTextureFromFile(gpD3DDevice, filename, &ret)) )
{
OutputDebugString("failed at loading a texture: ");
OutputDebugString(filename);
OutputDebugString("\n");
}

return ret;
}

Next is our game loop function, PlayDemo(). This function is called whenever there is no window message to handle. For real games, you would calculate the elapsed time since last frame and use it for both update and rendering functions, but it is omitted here for simplicity.

//------------------------------------------------------------
// game loop
//------------------------------------------------------------
void PlayDemo()
{
Update();
RenderFrame();
}

There is nothing in Update() function yet. One day, we will add something in here.

// Game logic update
void Update()
{
}

Next is RenderFrame() function, which draws stuff onto screen.

//------------------------------------------------------------
// Rendering
//------------------------------------------------------------

void RenderFrame()
{

We first clear the back buffer with blue colour.

D3DCOLOR bgColour = 0xFF0000FF; // background colour - blue

gpD3DDevice->Clear( 0, NULL, (D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER),
bgColour, 1.0f, 0 );

Then, we draw our scene and debug info.

gpD3DDevice->BeginScene();
{
RenderScene(); // draw 3D objects and so on
RenderInfo(); // show debug info
}
gpD3DDevice->EndScene();

Once rendering is done, we simply present what is drawn onto the back buffer to the screen.

gpD3DDevice->Present( NULL, NULL, NULL, NULL );
}

Just like Update() function, there is no code in RenderScene() function yet. We will write some code here in the next chapter to draw a 3D object.

// draw 3D objects and so on
void RenderScene()
{
}

RenderInfo() function simply displays key mapping information onto the screen.

// show debug info
void RenderInfo()
{
// text colour
D3DCOLOR fontColor = D3DCOLOR_ARGB(255,255,255,255);

// location to show the text
RECT rct;
rct.left=5;
rct.right=WIN_WIDTH / 3;
rct.top=5;
rct.bottom = WIN_HEIGHT / 3;

// show debug keys
gpFont->DrawText(NULL, "Demo Framework\n\nESC: Quit demo", -1, &rct, 0,
fontColor);
}

When the program is being shut down, we must release all D3D resources to prevent memory leak. Once all resources are released, the D3D device and the D3D object need to be released too.

//------------------------------------------------------------
// cleanup code
//------------------------------------------------------------

void Cleanup()
{
// release fonts
if(gpFont)
{
gpFont->Release();
gpFont = NULL;
}

// release models

// release shaders

// release textures

// release D3D
if(gpD3DDevice)
{
gpD3DDevice->Release();
gpD3DDevice = NULL;
}

if(gpD3D)
{
gpD3D->Release();
gpD3D = NULL;
}
}

That is it. We just finished writing a very simple framework. Even if you do not understand above code very well, that is completely fine. It does not really prevent you from learning HLSL with this book. But if your dream is to be a rendering dude, I highly recommend you to learn DirectX properly after finishing this book.

Thank you so much for suffering through the boring preparation steps. After the following quick summary, you we are off to the next chapter, where you will actually have some fun making something to show up on the screen!

Summary
The short summary of what we discussed in this chapter:

Shaders are functions calculating the position and colour of each pixel.
If you think shaders in terms of a painter’s workflow, vertex shader is perspective sketch and pixel shader is colouring.
Shader programming is nothing more than writing functions which are executed by vertex and pixel shading units.
Render Monkey is a great tool for quick shader prototyping.

------
Footnotes:

I intentionally over-simplified this figure to help you understand the roles of vertex and pixel shaders. Real 3D graphics pipelines are way more complicated than this.
Vertex shaders often outputs more information than just vertex positions. You will see more of it in the following chapters.
New shader types are introduced with DirectX 10 and 11. But they are not for beginners and currently not being used enough in real-world to be included in this book.
It stands for OpenGL Shader Language. As the name suggests, it is OpenGL’s shader language, which is somewhat different from HLSL syntax-wise.
It is a shader programming language supported by NVidia. It is almost identical to HLSL except a few things.
More pages = higher price = less beer = sadder life

Next Chapter

[Intro to Shader] 00. Introduction

2012-12-29T17:14:00.000-08:00

Where to buy:

- Amazon

- iBooks

- Google Play

Source Code: GitHub, Zip

Introduction
This book is a collection of lecture materials I developed and used to teach a shader programming course at the Art Institute of Vancouver between 2007 and 2009.

I started to work on this book back in 2010 without knowing for sure if I want to publish this book or just release it online. But before finish writing this book, a South Korean publisher wanted to publish this book in South Korea, so I had to stop it to work on the Korean version first.

Now the Korean version is out, and selling pretty well: in fact, it made its way to the best seller in computer programming category and has been picked up by a number of computer science universities and game schools as the textbook for their shader programming classes. So I thought it is time to come back to where it all started and finish the unfinished business.

I am still not sure whether I want to publish this book or not. If there is any publisher who wants to publish this easy-to-follow introductory shader programming book for both programmers and technical artists, feel free to contact me.

Why I Wrote This Book
When I was starting to teach in 2007, I was looking for a decent text book that can teach college students how to program shader language. Unfortunately, I could not find one and I believe there is still none out there. All the shader books out there focus more on advanced techniques, which are good for intermediate-to-advanced graphics programmers like me. It was very obvious to me that these books would rather scare off most college students than excite them. Sure, most DirectX books contain some introductory shader programming material, but I found none of those were very useful for the following reasons:

In those books, shader is a second-class citizen; they only touch the surface
They are too academic and often put too much focus on theory and syntax
They have too many examples which have no real-world use
They consist of too many pages to cover both DirectX and Shaders, which sacrifice portability of the book and increase the price unnecessarily. 600~1000 pages? Jees.

So what did I do? I started to teach without any textbook. I truly believe that students should get their hands dirty first to find fun in any new programming language, so I somewhat ignored a lot of theory or maths and rather focused more on practical techniques or other basic materials that will eventually lead to other practical techniques. One great thing about teaching at a college is that you find things that students have hard time understanding while you have been taking them granted. This really helped me refine my teaching method again and again, and the result is this book.

My shader programming course was picked as one of the best courses offered in the school for 3 years, so I have no doubt this book will help anyone who wants to learn shader programming. Some of my students used the demo they made in my class as part of their portfolio and managed to get jobs at various game companies, including big ones like Ubisoft and Electric Arts, so I can say I did not do such a bad job there, eh? :P

I stopped teaching after December 2009 to focus on my day-time job, game programming, but it kept bothering me that this material would never see the light ever again. So, I decided to make a book out of it. I hope anyone who wants to learn the black art of shader programming finds this book very useful.

What Is Special about This Book?
As I did in my class, this book follows these simple rules:

Get your hands dirty: Of course, you cannot complete ignore maths or theories when you write shader code. But I found coding-first-and-learning-theories-later is a much better way to learn shader programming. You have to question yourself first ‘why does it work this way?’ and then get the answer by learning theories. Once you learn any theory this way, you never forget. Therefore, this book is all about code-first approach. Just follow the book as you code. It will tell you background theories here and there whenever necessary. After a while, you will find yourself be able to not only write shader codes but also understand a good amount of theories behind it.
Easy real-world explanation: I even had some game art students listening in my lectures, and I wanted them to understand my lectures easily, too! What both artists and programmers can understand is real-world examples, so I tried my best to do the same thing in this book. Keen readers might find something that is not 100% correct in theory. This is most likely intentional in order to make the material easier to understand. (But, sometimes, it will be because I do not even understand the theories 100%. I tend to think computer graphics is a big giant trick show that makes human eyes believe it is real, so as long as it is not obviously wrong and the end result “looks” same as the correct one, I do not make a big fuss out of it.)
Strictly for beginners: This book is strictly for beginners. I do not see any reason to compete against tons of great shader books already out there for advanced readers. I also want this book to be very compact: it should not cost a fortune to start to learn. Once you find shader programming fun after reading this book, go ahead and read other great books to adventure further, and if you find some amazing new techniques, do not forget to email me about it. :-)
Order matters: Another blessing of teaching at a college was that I had a full control over the order of learning. This book assumes the same; you should read this book from beginning to the end in order. I will not repeat something I explained in the previous chapters. For example, while I am at normal mapping technique, I will not re-explain how lighting works because it is already covered in the previous chapters. Please, read this book from the beginning to the end as if you are taking a college course. I do not think I am asking you too much; after all, this book is pretty slim, right?

What This Book Covers
This book covers how to implement basic and some intermediate shader techniques using Vertex and Pixel Shaders. It consists of mainly three parts

Part 1 shows the definition of shader, the hello-world of shader programming – colour shader, texture mapping and lighting techniques.
Part 2 extends what is learned in Part 1 to teach most common shader techniques used in games: specular mapping, normal mapping, shadow mapping and so on.
Part 3 introduces 2D post-processing techniques, which became more important recently.

This book doesn’t cover new shader units added in DirectX 10 or 11. Geometry, Hull and Computer Shaders are some examples. They are not really for beginners and I have yet to see more practical use of these new shaders in real-world to find out the best way to teach them. However, I don’t think it would be too hard to learn those new ones by yourself once you build a solid foundation with this book.

Target Readers
Programmers
Most of my students were in their second year in the college. Prerequisites of my class included C++, 3D Math and DirectX. I don’t think it is too different from how game programmers get to shader programming, so I would assume any programmer who reads this book is comfortable with above topics. C++ and DirectX are musts. If you know 3D math very well, that is a bonus.

Technical Artists
Who does not love Technical Artists? They make my life easier, and I bet they make artists’ life easier too! Nowadays, they are even authoring shaders, mostly by using some type of graph editors. Do you feel like graph editors are too limiting, and want to learn how to code instead? Then, this book is for you. Even some art school students were able to follow my lectures, so go for it. In this book, the only difference for technical artists is that you get to skip the last part of each chapter, which is playing with the DirectX framework. But do not worry. You get to learn exactly the same thing in another great program called Render Monkey.

If You Have A Question
If you have any question while reading this book, feel free to visit my blog. Other updates including the errata will be uploaded there too.

http://www.popekim.com

Special Thanks To
It was impossible for me to write this book without the tremendous support from so many people around me.

First, I would like to thank all my students who asked me endless questions and challenged me every day. Their cat-like curiosity really made me rethink what I thought I had already known very well, which helped me write this book easier-to-follow and easier-to-understand.

I also would like to acknowledge two test readers, who very carefully read over this book and tested every single line of code for me. Jinyoung Song from NeoPle a Nexon company and Kyle Lee from Youth HiTech, without your hawk eyes, this book would still have 100 bugs.

Also I should say thanks to a talented concept artist, David Sloan, who drew a number of figures in this book. If you find anything that does not look like a programmer art, that is his work.

Last but not least, there were my friends and ex-coworkers who encouraged me to finish this book whenever I make excuses, such as ‘is it worth writing it?’ and ‘I do not have bandwidth to do it’. They are all very talented game developers, so I figured it is fun to list their names here: Vladimir Kouznetsov, Noel Austin, Karl Schmidt, Daniel Barrero, Aaron Lake and Andreas Loebenstein.

Pope Kim
from Burnaby and Montreal, Canada

Next Chapter