head noodles

Picky, Part II

2012-03-02T20:06:00.000-08:00

This is a continuation of my last post, where I set up color picking for the red GLKit cube. In this post I'm setting up color picking for the blue 2.0 shader cube. I'm starting off right where I left off, with the changes I made in my last post already made.

I'm using the exact same strategy as last time. Namely, re-rendering the scene off screen in response to a user tap, but with lighting turned off and a unique color set. The implementation is a little more involved since we have to communicate with the shader, but on the other hand, we can use a lot of the same work that we have already done.

Just as a reminder, when we left off, we had the red cube hooked up to the color picker so it got smaller whenever it was clicked on.

Step One: Modify the vertex shader

The vertex shader cannot have a hard coded diffuse light color anymore, since we have to change it based on whether we're rendering it for the screen or for picking. We also need to be able to turn lighting on or off. We can accomplish both of those tasks by setting up uniform variables. Modify the Shader.vsh file so it looks like this:

attribute vec4 position;

attribute vec3 normal;

varying lowp vec4 colorVarying;

uniform mat4 modelViewProjectionMatrix;

uniform mat3 normalMatrix;

uniform vec4 color;

uniform bool lit;

void main()

{

if (lit)

{

vec3 eyeNormal = normalize(normalMatrix * normal);

vec3 lightPosition = vec3(0.0, 0.0, 1.0);

float nDotVP = max(0.0, dot(eyeNormal, normalize(lightPosition)));

colorVarying = color * nDotVP;

}

else

{

colorVarying = color;

}

gl_Position = modelViewProjectionMatrix * position;

}

The first thing we had to do was declare the uniform variables color and lit. The uniform keyword simply means these values remain the same for all the vertices passed to the shader during the function call.

Inside the main function, we wrapped the existing lighting code inside an if statement. I'd really rather not have control logic inside my shader, but for this example, maybe it's ok. Notice how the vec4 diffuseColor declaration went away. Now we're using the uniform color variable in its place. Finally, if lit is set to false, we simply assign colorVarying equal to color.

Step Two: Connect the uniforms in the view controller code

Now that we have the vertex shader set up, we need to pass it correct values. This involves creating and configuring the two uniform variables in our view controller.

Start by adding two new members to the uniform enum near the top of the ViewController.m file. When you're done, it should look like this:

enum

{

UNIFORM_MODELVIEWPROJECTION_MATRIX,

UNIFORM_NORMAL_MATRIX,

UNIFORM_COLOR,

UNIFORM_LIT,

NUM_UNIFORMS

};

Next, declare two new variables in the ViewController interface section. I put mine right below the _normalMatrix declaration.

GLKVector4 _color;

GLboolean _lit;

Next, initialize the two variables you just declared in the setupGL function. I put mine at the end after my _scale initialization from the last post. I also initialized the color to the same diffuse color value that used to be hard coded in the vertex shader.

_color = GLKVector4Make(0.4f, 0.4f, 1.0f, 1.0f);

_lit = true;

Now, in the loadShaders function, add the code to get the locations of the two new uniforms we added:

uniforms[UNIFORM_COLOR] = glGetUniformLocation(_program, "color");

uniforms[UNIFORM_LIT] = glGetUniformLocation(_program, "lit");

This code goes right after the two glGetUniformLocation calls that already exist near the end of the function. We are simply getting the uniform IDs from OpenGL and storing them in our uniforms array for later use.

Finally, in the renderGL function we wrote in the last post, add the following code right below the two glUniformMatrix* calls that are already there.

glUniform4fv(uniforms[UNIFORM_COLOR], 1, _color.v);

glUniform1i(uniforms[UNIFORM_LIT], _lit);

This is the code that actually assigns the values we put in _color and _lit with the uniform variables we declare and use in the vertex shader.

As a sanity check, it's a good idea to run your program in the simulator at this point. It should behave the same as it did at the beginning of this post.

Step Three: Picking

Now that we've added the ability to change the blue cube's color, we need to take advantage of it.

Go down to the action handler we created in the last post. Add the following code right before the call to [self renderGL]:

// set the shader uniform color and disable lighting

GLKVector4 oldColor = _color;

_color = GLKVector4Make(1.0f, 0.0f, 0.0f, 1.0f);

_lit = false;

Conceptually, we're doing the exact same thing we did we the red cube: we're changing the color and we're turning off lighting. The only difference here is we need to store the old color so we can change it back when we're done.

If you run the program now in the simulator you should see the blue cube turn bright red when you click on the screen:

Now let's go ahead and change the color and lighting state back to the way it was. You can add this code right after the [self renderGL] call:

// change the shader color and lighting back to normal

_color = oldColor;

_lit = true;

Run the program in the simulator again. The blue cube should remain visually unchanged, but the console output should indicate a bright red color when you click on the blue cube:

2012-03-02 22:43:57.792 Picky[7308:10103] 255, 0, 0, 255

Step Four: Reaction!

Just as before, I set the blue cube up to get smaller when you click on it. To see it happen in your program, just follow the steps from the last post.

First, create a new variable right next to your _scale variable from last time:

float _shadeScale;

Next, initialize the variable to 1 in the setupGL function:

_shadeScale = 1.0f;

Next, add another scaling transformation to the update function. This one goes right after the GLKMatrix4Rotate function call in the ES2 model view matrix section:

modelViewMatrix = GLKMatrix4Scale(modelViewMatrix, _shadeScale, _shadeScale, _shadeScale);

Finally, in your tap action function, add an else if statement to your previous if statement to check for the bright red color and modify the _shadeScale variable:

else if (pixel[0] == 255 && pixel[1] == 0 && pixel[2] == 0 && pixel[3] == 255)

_shadeScale = _shadeScale / 1.5f;

Now run the program in the simulator. Clicking on either cube causes that cube to shrink.

Step Five: Troubleshooting

Using shader uniform variables is a multistep process, which means it's easy to get something wrong. In this section I purposely introduce various bugs to repo some common error states.

In the vertex shader file, I replaced all instances of the color variable with _color. This compiled and ran with no shader compile errors in the log, but since the ViewController.m file still refers to it as "color" in this line in loadShaders, we never actually succeed in passing the color to the shader, and so the cube appears black.

uniforms[UNIFORM_COLOR] = glGetUniformLocation(_program, "color");

When I stepped through this line of code in the debugger, I saw that the glGetUniformLocation call returned -1, indicating an error.

When I commented out the declaration of the color variable in the Shader.vsh file, but left references to it, I got the following messages in the error log:

//uniform vec4 color;

2012-03-07 20:18:43.563 Picky[568:10103] Shader compile log:

ERROR: 0:28: Use of undeclared identifier 'color'

ERROR: 0:32: Use of undeclared identifier 'color'

2012-03-07 20:18:43.570 Picky[568:10103] Failed to compile vertex shader

With this compile error, the blue cube does not show up at all. The loadShaders function returns when it detects a shader compile error, and does not even get to the set of glGetUniformLocation calls.

Another way to get a black cube is to mess up the call to glUniform4fv in the renderGL function. Here's how it's supposed to look:

glUniform4fv(uniforms[UNIFORM_COLOR], 1, _color.v);

The v at the end of the function name indicates that we are passing a vector. The f indicates that the vector contains float values. The 4 indicates that it is 4 dimensional. Using 3 instead of 4 results in a black cube.

The first parameter is the id we use to refer to the shader uniform. We got this id from OpenGL when we loaded the shaders. If we pass in an incorrect id, we get a black cube.

The second parameter is the number of vectors we are sending. It is not the number of bytes or the dimensions in the vector, so '1' is the correct value here. Any other value gets you a black cube.

Calling glGetError() after any one of these incorrect function calls returns error code 1282, which means invalid operation.

Conclusion

As I said in my previous post, color picking is fine for such a simple scene, but for more complex scenes, other algorithms such as ray-casting may be more effective.

As always, please let me know if you find any mistakes in my code, or if you know of a better or easier way to solve this problem. Also, use this code at your own risk.

Picky, Part I

2012-03-02T16:00:00.000-08:00

In this post, I'm trying out color picking. In the interest of keeping these posts to a more manageable size, I am planning to just color pick the red GLKit cube. My next post will build on the work in this one and handle the blue 2.0 cube.

The basic idea behind color picking is once the user has tapped somewhere, render the objects in your scene into the back buffer using a unique color for each object, then ask OpenGL to give you the color of the pixel where the user tapped. This color tells you what object was tapped. As long as you don't present the scene to the user, the user never sees the weird colors. This is fast and efficient, since you only render the scene an extra time whenever the user taps, but generally is good only for per-object or per-triangle picking. Getting the exact point on a triangle that the user clicked on will probably require a different technique.

Step One: Create a new OpenGL Game project

For instructions how to do this, refer to my texture post. Once you get it set up, you can run it and see two cubes. The red cube is rendered using GLKit. The blue cube is rendered using OpenGL ES 2.0 shaders. In this blog I color pick the red cube.

Step Two: Set up a tap gesture recognizer and action function (event handler)

Refer to my gesture post for step by step instructions.

Step Three: Read the pixel the user tapped on

Add code to the event handler so that it looks like the following:

- (IBAction)tapThat:(id)sender {

GLubyte pixel[4]; // output array for the red, green, blue, and alpha pixel components

// get the point the user tapped on

CGPoint tapPoint = [sender locationInView:self.view];

// read the pixel at the tapped location. We use the screen height to convert

// between iOS screen coordinates, which is (0,0) at the upper left, and OpenGL

// screen coordinates, which is (0,0) at the LOWER left.

int height = [self.view bounds].size.height;

glReadPixels(tapPoint.x,height - tapPoint.y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&pixel);

// log the results.

NSLog(@"%u, %u, %u, %u",pixel[0],(int)pixel[1],pixel[2],pixel[3]);

}

This just outputs the color of the pixel that the user tapped on to the log. Go ahead and run the program in the simulator and start clicking around on the simulated screen. If you hooked everything up right you should see something like this:

2012-03-02 16:16:51.505 Picky[5480:10103] 84, 84, 210, 210

2012-03-02 16:16:51.912 Picky[5480:10103] 81, 81, 203, 203

2012-03-02 16:16:52.104 Picky[5480:10103] 89, 89, 224, 224

2012-03-02 16:16:52.279 Picky[5480:10103] 94, 94, 236, 236

2012-03-02 16:16:52.831 Picky[5480:10103] 202, 87, 87, 255

2012-03-02 16:16:53.004 Picky[5480:10103] 194, 83, 83, 255

2012-03-02 16:16:53.507 Picky[5480:10103] 147, 65, 65, 255

2012-03-02 16:16:54.001 Picky[5480:10103] 185, 80, 80, 255

2012-03-02 16:16:54.184 Picky[5480:10103] 192, 83, 83, 255

2012-03-02 16:16:54.734 Picky[5480:10103] 77, 37, 37, 255

2012-03-02 16:16:54.932 Picky[5480:10103] 109, 49, 49, 255

2012-03-02 16:16:55.140 Picky[5480:10103] 138, 61, 61, 255

2012-03-02 16:16:56.104 Picky[5480:10103] 100, 100, 250, 250

The four comma-separated numbers at the end of each line are the red, green, blue, and alpha values of the pixel the user clicked on. Note that the data type that openGL returns is unsigned bytes, which go from 0 to 255. You generally send colors to OpenGL as floating point numbers between 0 and 1, so be aware that you may have to convert by dividing or multiplying by 255.

Step Four: Extract the render code

Of course, this function isn't very useful in and of itself. We need to add the code to render our object in a different color. To do this, We're going to pull out the render code into a separate function. We could just copy and paste the code from the view function to the tap action function, but that would be fragile, dangerous code. If the render functions aren't exactly the same except for the colors of the objects, we run the risk of rendering the objects in different locations when we are picking them. The user would click on a cube, but the action function would render it somewhere else, and our program wouldn't register the tap.

So create a new function declaration in the interface section of your ViewController.m file:

- (void)renderGL;

Then create the function in the implementation section, cut and paste the render code from the view function, and call the new render function inside the view function. When you are done, the view function and render function should look like this:

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect

{

[self renderGL];

}

- (void)renderGL

{

glClearColor(0.65f, 0.65f, 0.65f, 1.0f);

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glBindVertexArrayOES(_vertexArray);

// Render the object with GLKit

[self.effect prepareToDraw];

glDrawArrays(GL_TRIANGLES, 0, 36);

// Render the object again with ES2

glUseProgram(_program);

glUniformMatrix4fv(uniforms[UNIFORM_MODELVIEWPROJECTION_MATRIX], 1, 0, _modelViewProjectionMatrix.m);

glUniformMatrix3fv(uniforms[UNIFORM_NORMAL_MATRIX], 1, 0, _normalMatrix.m);

glDrawArrays(GL_TRIANGLES, 0, 36);

}

As a sanity check, go ahead and run your program in the simulator to make sure it still renders correctly.

Step Five: Render with a different color on tap

Now we need to call our render function from within the tap action function, but using a different color to draw our object. Add code to your tap action function so that it looks like this:

- (IBAction)tapThat:(id)sender {

GLubyte pixel[4]; // output array for the red, green, blue, and alpha pixel components

// turn off lighting and turn on constant color for picking

self.effect.useConstantColor = GL_TRUE;

self.effect.light0.enabled = GL_FALSE;

self.effect.constantColor = GLKVector4Make(0.0f, 0.0f, 1.0f, 1.0f);

// render the scene

[self renderGL];

// set lighting and the constant color usage back to the way they were

self.effect.useConstantColor = GL_FALSE;

self.effect.light0.enabled = GL_TRUE;

// get the point the user tapped on

CGPoint tapPoint = [sender locationInView:self.view];

// read the pixel at the tapped location. We use the screen height to convert

// between iOS screen coordinates, which is (0,0) at the upper left, and OpenGL

// screen coordinates, which is (0,0) at the LOWER left.

int height = [self.view bounds].size.height;

glReadPixels(tapPoint.x,height - tapPoint.y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&pixel);

// log the results.

NSLog(@"%u, %u, %u, %u",pixel[0],(int)pixel[1],pixel[2],pixel[3]);

}

In the code we added, we turn off lighting and turn on a constant, bright blue color. Then we render as normal. Afterwards, we turn the light back on and the constant color back off.

If you run this and click on the red cube, you will see numbers representing bright blue in the console:

2012-03-02 17:01:56.106 Picky[5746:10103] 0, 0, 255, 255

As an interesting exercise, you can comment out the two lines setting the OpenGL state back to the way it was and run the simulator. When you click on the screen, you'll see the object turn bright blue.

//self.effect.useConstantColor = GL_FALSE;

//self.effect.light0.enabled = GL_TRUE;

Step Six: Add a visual reaction

Now we just need to react to the user clicking on the cube. For this example, I arbitrarily decided to show the click by making the cube smaller. Presumably you will have your own application logic.

I started by adding a scale value to the view controller .m file

float _scale;

Next, I initialized it to 1 in the setupGL function.

_scale = 1.0f;

Next, I added a new scale transformation to the GLKit cube in the update function. Now it looks like this:

- (void)update

{

float aspect = fabsf(self.view.bounds.size.width / self.view.bounds.size.height);

GLKMatrix4 projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(65.0f), aspect, 0.1f, 100.0f);

self.effect.transform.projectionMatrix = projectionMatrix;

GLKMatrix4 baseModelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -4.0f);

baseModelViewMatrix = GLKMatrix4Rotate(baseModelViewMatrix, _rotation, 0.0f, 1.0f, 0.0f);

// Compute the model view matrix for the object rendered with GLKit

GLKMatrix4 modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -1.5f);

modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, _rotation, 1.0f, 1.0f, 1.0f);

modelViewMatrix = GLKMatrix4Scale(modelViewMatrix, _scale, _scale, _scale);

modelViewMatrix = GLKMatrix4Multiply(baseModelViewMatrix, modelViewMatrix);

self.effect.transform.modelviewMatrix = modelViewMatrix;

// Compute the model view matrix for the object rendered with ES2

modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, 1.5f);

modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, _rotation, 1.0f, 1.0f, 1.0f);

modelViewMatrix = GLKMatrix4Multiply(baseModelViewMatrix, modelViewMatrix);

_normalMatrix = GLKMatrix3InvertAndTranspose(GLKMatrix4GetMatrix3(modelViewMatrix), NULL);

_modelViewProjectionMatrix = GLKMatrix4Multiply(projectionMatrix, modelViewMatrix);

_rotation += self.timeSinceLastUpdate * 0.5f;

}

Finally, I added the logic that checks to see if we clicked on our cube and change the scale. Now the tap action function looks like this:

- (IBAction)tapThat:(id)sender {

GLubyte pixel[4]; // output array for the red, green, blue, and alpha pixel components

// turn off lighting and turn on constant color for picking

self.effect.useConstantColor = GL_TRUE;

self.effect.light0.enabled = GL_FALSE;

self.effect.constantColor = GLKVector4Make(0.0f, 0.0f, 1.0f, 1.0f);

// render the scene

[self renderGL];

// set lighting and the constant color usage back to the way they were

self.effect.useConstantColor = GL_FALSE;

self.effect.light0.enabled = GL_TRUE;

// get the point the user tapped on

CGPoint tapPoint = [sender locationInView:self.view];

// read the pixel at the tapped location. We use the screen height to convert

// between iOS screen coordinates, which is (0,0) at the upper left, and OpenGL

// screen coordinates, which is (0,0) at the LOWER left.

int height = [self.view bounds].size.height;

glReadPixels(tapPoint.x,height - tapPoint.y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&pixel);

// did we click on our cube? If so, make it smaller

if (pixel[0] == 0 && pixel[1] == 0 && pixel[2] == 255 && pixel[3] == 255)

_scale = _scale / 1.5f;

// log the results.

NSLog(@"%u, %u, %u, %u",pixel[0],(int)pixel[1],pixel[2],pixel[3]);

}

Go ahead and run the program in the simulator. If you click on the background or the blue cube, nothing happens. Clicking on the red cube, however, makes it shrink!

I chose this particular method of setting the color of our object because it was the simplest way I could think of that would demonstrate the technique. More complex scenes would require a more complex method. To implement per-triangle color picking, you could add a per-vertex color attribute to the vertex array, then use that color in your test. As I said above, my next post will add color picking to the blue 2.0 cube.

And as always, this is code I wrote while trying to figure this stuff out. It is probably not suitable for use in any production environment, and I strongly advise against using it.

PS -- I may be the only blogger on the internet who did not know how to automatically apply formatting to my source code in my blog, but just in case anyone else out there is as derp as I am, I found that if you copy your source from Xcode to TextEdit, then save as html, TextEdit will apply the color and font styles for you. Then you can just paste the html into your blog editor. Whatever you do, don't click back to compose mode once you have pasted your HTML. Oh, I also had to change the 'Apple-tab-span' style to 'Apple-converted-space' so it actually matched the style in the rest of the document.

The Texturing

2012-02-20T23:55:00.000-08:00

In this post I'm changing the default Xcode 4 OpenGL Game project to load a texture file and use it when drawing the two cubes.

I started with a brand new OpenGL Game project. See my previous post if you don't know how to create one. If you run the program after creating the project you will see two cubes orbiting about the Y-axis. The blue one is drawn using the vertex and fragment shaders, and the red one is drawn using Apple's fixed functionality pipeline. Our goal is to overlay a texture on both cubes.

The first thing we need to do is find / create our texture and load it into our project. For my texture, I got a screenshot of the Google blogger loading icon and used Mac Preview to crop out the rounded edges and resize it to 64 x 64 (I tried an 84 by 84 texture and it didn't work. Apparently, in iOS textures are still required to have lengths and widths that are powers of two). I saved it as bloogle.png on my desktop.

To load the texture into your project just drag the file from wherever it is on your computer into the Supporting Files folder in the Project navigator in Xcode. Make sure "Copy items into destination group's folder (if needed) is checked, then click the "Finish" button.

Now the file is in the project, but we still need our program to load the image data into memory so OpenGL can use it. To do this we will create a couple of new functions and a member variable in the view controller .m file.

In the interface section of the view controller .m file, add the following line:

GLKTextureInfo * _texture;

This object will end up holding all the information OpenGL needs about our texture.

In the function declarations section add the following lines:

- (void)loadTextures;

- (void)logError:(NSError *) error;

This declares our loadTextures function, which we will use to load our texture data into memory, and a logger function that will help use figure out what we are doing wrong. I added error logging after I googled my own errors and found that a lot of other people were posting on Stack Overflow with inscrutable texture loading failures of their own. Hopefully this will help you avoid that.

After you add the two function declarations a warning may pop up complaining about "Incomplete implementation." We are going to address that next by implementing the two functions.

At the end of the view controller .m file, right above the @end statement, add the following function implementation skeletons:

#pragma mark - Texture Loading

- (void)loadTextures

{

}

- (void)logError:(NSError *) error

{

}

The pragma mark statement makes it easy to find your functions in the bread crumb navigator at the top of the editor. Notice how your functions have their own little section in the symbol drop down.

Now add the following statements to the loadTextures function you just created:

NSError *error = nil; // stores the error message if we mess up

NSDictionary *options = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES]

forKey:GLKTextureLoaderGenerateMipmaps];

NSString *bundlepath = [[NSBundle mainBundle] pathForResource:@"bloogle" ofType:@"png"];

_texture = [GLKTextureLoader textureWithContentsOfFile:bundlepath options:options error:&error];

[self logError:error];

First we declare a NSError object to hold our error message. This isn't strictly necessary. If you want, you can pass nil to the textureWithContentsOfFile function. However, the error object is about the only way to get a meaningful error message out Xcode, and you can spend a lot of time banging your head against the wall without it.

Next we declare our options dictionary. This is also not strictly needed for our purposes. You can pass a nil value in for this as well. But I wanted to show you how to go about creating an options dictionary in case you didn't know how. This options dictionary tells the texture loader to automatically create mipmaps for the texture.

Next, we get the path of our image resource. You can't just use "bloogle.png". Also note the lack of a period in the parameters we pass to this function.

Finally we generate the texture. This one function call is equivalent to quite a few OpenGL calls. It's responsible for loading the image data, passing the image data into OpenGL, getting a texture id back, and setting a number of OpenGL parameters for the texture. Refer to any good OpenGL reference manual to get an idea of the things you don't have to do here.

After we (attempt to) generate the texture we pass the error to our logging function. If something went wrong, the logging function will output something meaningful to the console. To make this happen add the following statements to the logError function:

if (error)

{

NSString * domain = [error domain];

NSLog(@"Error loading texture: %@. Domain: %@", [error localizedDescription],domain);

NSDictionary * userInfo = [error userInfo];

if (domain == GLKTextureLoaderErrorDomain)

{

if (nil != [userInfo objectForKey:GLKTextureLoaderErrorKey])

NSLog(@"%@", [userInfo objectForKey:GLKTextureLoaderErrorKey]);

if (nil != [userInfo objectForKey:GLKTextureLoaderGLErrorKey])

NSLog(@"%@", [userInfo objectForKey:GLKTextureLoaderGLErrorKey]);

}

The outer if statement checks to see if there was an error at all. If everything went ok, our error object should still be nil.

If there was an error, the first thing we do get the error domain. Each part of the the iOS framework has its own error domain, and each domain has a different set of error codes. To find out what an error code means, you must know what domain the code is coming from.

Next we output the domain and a high level description of the error to the console. Other than the error number, the description tends to be vague and worthless.

Depending on the error domain, you may be able to pull a more helpful message out of the error object. The next lines get a userInfo dictionary out of the error object, which may more information. If the domain is from the GLKTextureLoader itself, we try to pull two additional error messages out of the dictionary and output them to the console.

I'll show you how to induce some of these error messages in a second, but first lets get our program working correctly. Add the following line of code to the setupGL function, right below the call to loadShaders:

[self loadTextures];

If you run the project now you shouldn't see any error messages in the console, which is good, except it's not testing our error logging function. To induce an error, change the textureWithContentsOfFile function call so that we are passing in the filename of texture directly instead of using the bundle path:

_texture = [GLKTextureLoader textureWithContentsOfFile:@"bloogle" options:options error:&error];

Now when we run the program we get the following error in our console:

2012-02-21 00:04:22.839 TheTexturing[7689:10103] Error loading texture: The operation couldn’t be completed. (Cocoa error 260.). Domain: NSCocoaErrorDomain

This is a generic unhelpful message from the NSCocoaErrorDomain. If you know how to get a useful message out of this error programically let me know. I ended up hunting down Apple's documentation which told me that error code 260 in the NSCocoaErrorDomain was a "NSFileReadNoSuchFileError" which makes sense since we futzed with the filename to get the error. Don't forget to change the function call back to the way it was before continuing.

Another way to induce an error is to change the loadTexture call in setupGL so that it is before the setCurrentContext call.

When you run the program now, you should get the following errors in the console:

2012-02-20 23:56:30.378 TheTexturing[7631:10103] Error loading texture: The operation couldn’t be completed. (GLKTextureLoaderErrorDomain error 17.). Domain: GLKTextureLoaderErrorDomain

2012-02-20 23:56:30.383 TheTexturing[7631:10103] Invalid EAGL context

The first error is our generic unhelpful error. But since it's from the GLKTextureLoaderErrorDomain we also get a slightly more helpful message informing us that our EAGL context is invalid. This is because we called our load texture function before we set the context. Move the load texture function call back to where it was to make this error go away. The complete list of error codes for the GLKTextureLoaderErrorDomain can be found in Apple's documentation.

Now that we've loaded the image data into a texture, we need to tell OpenGL how to use that texture. First, we will apply the texture to the red fixed function cube.

The first thing we need to do is add texture coordinates to the vertex data near the top of our view controller .m file. Find the declaration of the gCubeVertexData array and change it like so:

GLfloat gCubeVertexData[] =

{

// Data layout for each line below is:

// positionX, positionY, positionZ, normalX, normalY, normalZ, texCoordS, texCoordT

0.5f, -0.5f, -0.5f, 1.0f, 0.0f, 0.0f, 0,1,

0.5f, 0.5f, -0.5f, 1.0f, 0.0f, 0.0f, 1,1,

0.5f, -0.5f, 0.5f, 1.0f, 0.0f, 0.0f, 0,0,

0.5f, 0.5f, 0.5f, 1.0f, 0.0f, 0.0f, 1,0,

0.5f, 0.5f, -0.5f, 1.0f, 0.0f, 0.0f, 1,1,

0.5f, 0.5f, -0.5f, 0.0f, 1.0f, 0.0f, 1,1,

-0.5f, 0.5f, -0.5f, 0.0f, 1.0f, 0.0f, 0,1,

0.5f, 0.5f, 0.5f, 0.0f, 1.0f, 0.0f, 1,0,

-0.5f, 0.5f, -0.5f, 0.0f, 1.0f, 0.0f, 0,1,

-0.5f, 0.5f, 0.5f, 0.0f, 1.0f, 0.0f, 0,0,

-0.5f, 0.5f, -0.5f, -1.0f, 0.0f, 0.0f, 1,1,

-0.5f, -0.5f, -0.5f, -1.0f, 0.0f, 0.0f, 0,1,

-0.5f, 0.5f, 0.5f, -1.0f, 0.0f, 0.0f, 1,0,

-0.5f, -0.5f, -0.5f, -1.0f, 0.0f, 0.0f, 0,1,

-0.5f, -0.5f, 0.5f, -1.0f, 0.0f, 0.0f, 0,0,

-0.5f, -0.5f, -0.5f, 0.0f, -1.0f, 0.0f, 0,1,

0.5f, -0.5f, -0.5f, 0.0f, -1.0f, 0.0f, 1,1,

-0.5f, -0.5f, 0.5f, 0.0f, -1.0f, 0.0f, 0,0,

0.5f, -0.5f, -0.5f, 0.0f, -1.0f, 0.0f, 1,1,

0.5f, -0.5f, 0.5f, 0.0f, -1.0f, 0.0f, 1,0,

0.5f, 0.5f, 0.5f, 0.0f, 0.0f, 1.0f, 1,0,

-0.5f, 0.5f, 0.5f, 0.0f, 0.0f, 1.0f, 0,0,

0.5f, -0.5f, 0.5f, 0.0f, 0.0f, 1.0f, 1,1,

-0.5f, 0.5f, 0.5f, 0.0f, 0.0f, 1.0f, 0,0,

-0.5f, -0.5f, 0.5f, 0.0f, 0.0f, 1.0f, 0,1,

0.5f, -0.5f, -0.5f, 0.0f, 0.0f, -1.0f, 1,1,

-0.5f, -0.5f, -0.5f, 0.0f, 0.0f, -1.0f, 0,1,

0.5f, 0.5f, -0.5f, 0.0f, 0.0f, -1.0f, 1,0,

-0.5f, -0.5f, -0.5f, 0.0f, 0.0f, -1.0f, 0,1,

-0.5f, 0.5f, -0.5f, 0.0f, 0.0f, -1.0f, 0,0

};

A brief explanation of this array: Each line describes a single vertex. The first three numbers are the X,Y,Z position of the vertex. The second three numbers describe a normal vector that points away from the face of the cube that this vertex is part of. Normal vectors are used in lighting calculations. OpenGL uses the dot product of the normal vector and the vector from the light source to the vertex to find the angle between them. If the angle is very small, the light is shining directly on the geometry and it appears very bright. As the angle increases, the light gets dimmer.

The two numbers we added to each line are the texture coordinates for the vertex. These map the image data to the face of the geometry. For texture coordinates, 0,0 is the upper left corner of the image, and 1,1 is the lower right. Note that the up-down coordinate gets higher as you go down the texture, which is opposite of the geometry coordinate system.

Each group of six lines in the array represent one face of the cube. Each face is made up of two triangles. Notice that this array only holds information for a single cube. This data is loaded once, then used twice to draw two cubes on the screen.

If you're adding the texture coordinates one at a time like I did, don't forget to add a comma after the last normal vector. Also don't forget to remove the size of the array from the square brackets. It is unnecessary and will break your code if you add elements to the array without updating it.

After changing vertex data array, we need to go back down to the setupGL function and make some changes. First find the line that sets the light0.diffuseColor and change the 0.4s to 1.0s.

self.effect.light0.diffuseColor = GLKVector4Make(1.0f, 1.0f, 1.0f, 1.0f);

This line is setting the color of the light source for the fixed functionality cube. The four numbers represent the red, green, blue, and alpha (or opacity) values of the light. OpenGL tends to specify these values from 0 to 1. If you work with HTML at all, you may be more familiar with the hexadecimal representation of color, from 00 to FF. This is changing the light shining on the cube from red to white. This is so it looks normal when the texture is applied.

Next, add the following lines of code directly under code you just changed:

self.effect.texture2d0.enabled = GL_TRUE;

self.effect.texture2d0.envMode = GLKTextureEnvModeModulate;

self.effect.texture2d0.target = GLKTextureTarget2D;

self.effect.texture2d0.name = _texture.name;

The first line enables the texture we loaded.

The second line describes how OpenGL will apply the texture. Modulate mixes the texture color with the light color. If we had chosen Replace, we would end up with a flat, perfectly bright cube that is not impacted by light at all.

The third line and fourth line sets the active texture to the one we loaded earlier. I believe (but I could be wrong) that the third and fourth line is roughly the same as calling glBindTexture(GL_TEXTURE_2D, _texture.name).

Now we need to change the pointers to the vertex array that we pass OpenGL. Change the calls the VertexArrib* functions like so:

glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 32, BUFFER_OFFSET(0));

glEnableVertexAttribArray(GLKVertexAttribNormal);

glVertexAttribPointer(GLKVertexAttribNormal, 3, GL_FLOAT, GL_FALSE, 32, BUFFER_OFFSET(12));

glEnableVertexAttribArray(GLKVertexAttribTexCoord0);

glVertexAttribPointer(GLKVertexAttribTexCoord0, 2, GL_FLOAT, GL_FALSE, 32, BUFFER_OFFSET(24));

The glVertexAttribPointer functions are telling OpenGL where to find the position, normal, and texture coordinate information in the array we pass in.

The first parameter is an enum value that we use to refer to different information in our vertex array. By passing in these values here, we can use them later to refer to when working with OpenGL to refer to the position, normal, or texture coordinate information. The second parameter is how many dimensions to look for. The third is the type of the array. The fourth is whether the data is normalized. A normalized vector has a length of one. The fifth parameter is the number of bytes to skip when reading the information for each vertex from the array. You can also think of this as the number of bytes in a single line in the array above ((3 position coordinates + 3 normal coordinates + 2 texture coordinates) * 4 bytes per coordinate). The last parameter is where in the array to start reading (in bytes).

After making these changes, run the program. The red cube should now be texturized! Note that the shading from the light source is still applied.

Even though we've only changed one cube we are well over half way done. Most of the work to load the texture information into OpenGL applies to both types of rendering. Now all we have to do is set up the shaders to use the information we have provided.

Let's start by going to the Shader.vsh file and changing it to take advantage of our texture data.

First, add a new attribute declaration below the position and normal attributes:

attribute vec2 texcoord0;

Attributes are values that we pass to the shader from our program on a per vertex basis. In our case, the attributes are contained in the vertex array that we added texture coordinates to.

Next, add a new varying variable declaration below the colorVarying declaration:

varying mediump vec2 texcoord_varying;

These varying values will be interpolated and passed to the fragment shader.

Next, change the diffuseColor vector so that it is white instead of blue. This is changing the blue light to white for exactly the same reason we changed the red light to white earlier.

vec4 diffuseColor = vec4(1.0, 1.0, 1.0, 1.0);

Finally, add a line that assigns the texcoord value the vertex shader receives from the program to the texcood_varying value that it passes to the fragment shader.

texcoord_varying = texcoord0;

That's all we have to do to the vertex shader. It should now look like the screen shot:

Now lets go to the Shader.fsh file and modify the fragment shader. Even though it's doing the actual work of specifying each pixel's color, it is much smaller.

First, add a declaration for a varying variable to match the one in the vertex shader:

varying mediump vec2 texcoord_varying;

Next, add a uniform sampler declaration. This is the actual image data for our texture:

uniform sampler2D texture;

Finally, change the gl_FragColor assignment in the main function like so:

gl_FragColor = texture2D(texture, texcoord_varying) * colorVarying;

The texture2D function uses the interpolated texture coordinate to pull a color from a image data. We then multiply it by the colorVarying attribute which contains the lighting value as calculated in the vertex shader. If the light is shining directly on the face, then this has the effect of multiplying the texture color by one, which does not change it at all. If the light is shining on the face at an angle, this multiplies the texture color by some number between one and zero, making it darker. This is why each face of the cube gets darker as it rotates away from the light source.

Now we are done modifying the fragment shader. It should look like the screen shot below:

Now that we've modified the shaders to take advantage of the texture data, we need to connect the new variables we declared to their counterparts in our view controller. Start by adding the following line to the enum at the very top of the view controller .m file:

UNIFORM_TEXTURE_SAMPLER,

Be sure to add this above the NUM_UNIFORMS entry, so that it is still incremented correctly. This enum represents the set of identifiers that we use to refer to the variables in the shaders.

Next go to the loadShaders function and add the following call to glBindAttributeLocation below the two that are already there.

glBindAttribLocation(_program, GLKVertexAttribTexCoord0, "texcoord0");

As the function name implies, this associates the texcoord0 variable in the vertex shader with the data referred to by the GLKVertexAttribTexCoord0 constant. Whenever we need to refer to the shader variable we can use this constant.

If you look carefully you'll notice that the BindAttribute function call we added does not match the two that are already there. The other two function calls use members of an enum that is declared at the top of the file. You may be wondering why we didn't add a member to this enum and pass it to the glBindAttributeLocation function. It took me a while to realize it, but that enum is actually completely pointless, misleading, and a trap for the unwary programmer.

OpenGL works by assigning identifiers to resources. When you give OpenGL a vertex array, you also give it an identifier to use to refer to that array. In the glEnableVertexAttribArray and glVertexAttribPointer calls in the setupGL function, we pass in GLKVertexAttribPosition, GLKVertexAttribNormal, and GLKVertexAttribTexCoord0 constants to refer to the array's position, normal, and texture coordinate information. Now it just so happens that GLKVertexAttribPosition and GLKVertexAttribNormal are equal to 0 and 1 respectively. So when the anonymous author of OpenGL Game project later uses ATTRIB_VERTEX and ATTRIB_NORMAL, which are also equal to 0 and 1, to bind these resources to the shader attributes, everything works out ok. But this is just a happy coincidence due to the similar ordering of these values within their respective enum declarations. Adding an ATTRIB_TEXCOORD member to our enum would give it a value of 2 (assuming you follow standard practice and insert it above the NUM_ATTRIBUTES member.) Unfortunately the GLKVertexAttribtexCoord0 constant we used to refer to the texture coordinate data earlier has a value of 3. So if we follow the example given us in the OpenGL Game project we would end up with broken code.

Anywho, after you add the glBindAttribLocation call, go down a little further in the loadShaders function and add an additional uniform assignment:

uniforms[UNIFORM_TEXTURE_SAMPLER] = glGetUniformLocation(_program, "texture");

This follows the typical OpenGL pattern of assigning an unsigned integer identifier to a OpenGL resource.

Finally go to the drawInRect function and add the code to bind the texture sampler with the image data:

glUniform1i(uniforms[UNIFORM_TEXTURE_SAMPLER],0);

While I was modifying the drawInRect function, I took a moment to change the ClearColor from gray to white:

glClearColor(1.0f, 1.0f, 1.0f, 1.0f);

This is by no means necessary, I just think the gray background looks really ugly.

Running the program now gives you two cubes, both textured and lit:

As I have said before, I am chronicling my blind, fumbling efforts with iOS programming in the hopes that other people may avoid my mistakes (at least the ones I'm aware of.) Use this code with extreme caution. I feel confident that it still contains errors. If you happen to notice one, please let me know.

Rude Gestures

2012-02-20T13:12:00.000-08:00

In this post I'm changing the default OpenGL Game project to rotate based on user gestures instead of automatically. This is actually super easy to do once you know how, but if you're like me and you don't know much about Objective C or Xcode, there are a couple of annoying gotchas. I'm assuming you are running a Mac with Lion and already have downloaded Xcode and the iOS software development kit. If not you can find both in the Mac app store.

To create the project, open Xcode, and choose File->New->New Project... from the Xcode menu bar. Choose iOS Application on the upper left, and the OpenGL Game option in the middle of the window, then press the Next button.

On the next screen, name your project whatever you want. I have no idea how to use an Xcode storyboard, so I leave that un-checked, but I like the idea of automatic reference counting.

You should end with a new OpenGL project with the project file opened on the screen. I recommend taking a snapshot of your project now, so if you mess anything up you can revert. Go to File->Create Snapshot... in the menu bar and follow the instructions.

You can run the project (Command-R) and see two cubes rotating around each other. One is rendered using GLKit, which among other things is Apple's re-implementation of the OpenGL ES 1.x fixed functionality render pipeline, and the other is rendered using the OpenGL ES 2.0 vertex and fragment shaders.

Now to add gesture support we need to go to the interface builder and edit the nib file. The Project navigator should be opened on the left side of the screen, showing a tree view of the files and folders that make up your project. If it's not opened, you can show it by pressing Command-1 on the keyboard. Once it's opened select the file with the .xib extension by single clicking on it. You will see a big blank view. Find the tap gesture recognizer in the object library in the lower right of the window. You can filter for it by typing "gesture" in the little text box at the very bottom of the window.

Now drag the tap gesture recognizer from the object library to the blank view in the middle of the screen. It's possible to drag the recognizer other places, such as the list of objects to the right of the view, but those are bad, wrong places that will cause the tap gesture recognizer to silently fail. I say again, drag the tap gesture recognizer to the blank white area in the middle of your window, and nowhere else. Once you drag it over, it will appear in the list of objects in the nib file.

Now we need to add an action to our view controller. Activate the assistant editor by pressing option-command-enter on your keyboard, or by clicking the little tuxedo button in the upper right of the window. The view controller .h file should appear in a new pane on the right side of the window.

Now right click and drag from the tap gesture recognizer in the list of objects in the nib file (NOT from the object library) to the area in the view controller .h file between @interface and @end. In the little context dialog box that pops up, select an Action connection and name it something meaningful. I changed the type to UIPanGestureRecognizer, but I'm not sure it makes a difference.

Once you hit the Connect button you will see a new IBAction declaration, which is basically analogous to an event handler.

The skeleton of its implementation was also automatically added to the view controller .m file, and that's the file we are going to edit next.

Get rid of the assistant editor by pressing command-enter on your keyboard or by going to the View->Standard Editor->Show Standard Editor menu option in the menu bar. Then select the view controller .m file in the project navigator to the left of your window. The first thing we want to do is comment out the last line in the update function which increments the rotation value. The update function is called by the framework before every frame and it's this line that causes the cubes to automatically rotate around each other.

Now we need to update the rotation value ourselves, in the new pan gesture event handler that Xcode added for us at the bottom of the view controller file. Add these two lines to the empty function:

CGPoint translation = [sender translationInView:self.view];

_rotation = translation.x / 100.0f;

The first line gets the amount that the user panned in both the x and y directions. The second line sets our rotation value equal to the amount the user slide his finger back or forth. We divide by 100 because the rotation value is in radians. 2*pi radians (a little over 6) is equivalent to a full 360 degree rotation, so if every frame the user drags their finger 4 or 5 pixels the cubes will rotate almost all the way around. We want to slow that down so it looks better and is easier to control.

Now run the project again. When the simulator comes up the cubes will be stationary until you click and drag your mouse or trackpad across the screen. Then you make the cubes rotate back and forth.

This post is very much the chronicle of me fumbling around in the dark with Xcode and iOS development. I'm writing this in the hopes of helping other people avoid the same pitfalls and blind alleys I stumbled on. If you find any mistakes or know of a safer, saner, or more interesting way to do this, please let me know.

PS - One of the many cool things I learned how to do while writing this was how to take screenshots on the Mac. Pressing Command-Shift-4 allows you to select an area of your screen to which will automatically appear as an image on your desktop. Pressing Command-Shift-4, then the space bar will allow you to click on a window to take a screenshot of the entire window.