Richards Software Ramblings

CLRS Algorithm a Day: Mergesort

2018-08-28T02:13:39Z

So we're still not up to an algorithm a day, but I'm gaining... So far, we've been looking at relatively simple quadratic (i.e. O(n²)) sorting algorithms. These have been incremental algorithms, so that we sort one element, then we sort the next unsorted element, and so on. Another category of algorithms are those known as divide-and-conquer algorithms. In this style of algorithm, we take our entire data set and partition it into smaller, simpler portions, solve each subportion, and then integrate the subportions back together to yield the full result.

Mergesort is a simple example of a divide-and-conquer algorithm, described in Chapter 2.3.1 of CLRS. Perhaps this should really be called divide-conquer-combine, as that more accurately describes what we are doing:

Divide: Split our items in half, giving us a left set and a right set each of n/2 items.
Conquer: We then need to sort each of these smaller sets, so we recursively apply the divide step on each of the subsets. Trivially, once we have reached subsets of a single item, that subset is sorted, so that is our base case where we stop dividing.
Combine: When the two subsets are sorted, we then merge them back together to produce the full sorted set.

You can see this procedure in action in the GIF to the left. Dividing into the left and right subsets is trivial, so the real key is the merging operation. Using cards as an example, this is quite intuitive; when we are merging, what we do is look at the first card in the left set and the first card in the right set. Whichever card is smaller, we strip it off and add it to the merged set, and continue in that fashion until one of the sets is empty, then add all the remaining cards in the non-empty set.

Simple, right? Conceptually, it certainly is, and if I were going to do this with some higher-level constructs like LINQ, it would be a snap to translate almost directly into code, but we're going to do this the hard way, and so that makes it a little more complicated ☺.

CLRS Algorithm a Day: Selection Sort

2018-08-07T22:55:11Z

Selection Sort

I am not good at this whole blogging thing anymore... My carpentry and remodeling skills have been put to the test over the last four months, however, as I've completely redone my kitchen and gutted and renovated my first-floor apartment. Now back into the saddle with some more algorithms!

Two posts ago, I started out with one of the fundamental sorting algorithms, Insertion Sort. Today we'll be looking at its close cousin, Selection Sort, which is described briefly in exercise 2.2-2 of CLRS.

If you recall, with Insertion Sort, we would iterate over our array taking the current value and then inserting it into the proper place in the elements that we have already sorted, by bubbling it down until the array left of our current position is sorted.

Selection Sort works slightly differently. Our algorithm, in English, is as follows:

Scan through the array, to select the smallest value, and swap that value with the value in the first position.
Then scan through the rest of the array to find the next smallest value, and swap that value with the value in the second position.
Continue in this fashion until we reach the last element, at which point the array is sorted.

At first glance, this is very, very similar to Insertion Sort, but they are actually almost opposite. The key difference lies in the way that the comparisons are performed

In Insertion Sort, we select the next unsorted element, and scan backwards through the sorted elements to find the position that the current element should be placed.
In Selection Sort, we select the next position in the array, and scan forward through the remaining unsorted elements to find the element that should go in that position

This difference becomes important when we consider the best and worst case performance of the two algorithms.

For Insertion Sort, the best case is when we have an already sorted array and try to sort it again. In this case, we take each element, and need to only compare it against the preceding element, so that we have O(n) comparisons. The worst case is when we have a reverse sorted array, in which case, we will have to compare each new element against every previously sorted element, which gives us n(n-1)/2 comparisons, for a worst-case runtime of O(n²).
For Selection Sort, the best and worst case runtimes are the same. For every element, we have to scan all the remaining elements to find the smallest element, so that we must always perform n(n-1)/2 comparisons, so it is always O(n²).
One other difference between Insertion and Selection Sort is in the number of swaps that the two algorithms need to perform. Selection Sort will always perform O(n) swaps, whereas Insertion Sort can perform between 0, in the best case, to O(n²) swaps, due to the way it bubbles unsorted values into the sorted portion of the array.

Implementation

Now that I've done a few of these, I'm not going to go through all the different variations and C#-isms for generics and custom comparators, and just show the straightforward implementation for Selection Sort on an array of integers. If you want to see the full code, including a recursive version, it is available on GitHub.

public static void SelectionSort(this int[] a) {
    for (var i = 0; i < a.Length-1; i++) {
        var smallest = a[i];
        var smallestI = i; // index of the smallest item
        for (var j = i+1; j < a.Length; j++) {
            if (a[j] < smallest) {
                smallest = a[j];
                smallestI = j;
            }
        }
        // swap the smallest value into the final position
        a[smallestI] = a[i];
        a[i] = smallest;
    }
}

Flying Spaghetti Monster willing, I'm done with renovation projects for a little bit, so I should have a little more time on my hands, and there won't be a multimonth layover before the next entry in this series, but you never know. Next up we'll be moving on to divide and conquer sorting algorithms, starting with Mergesort. Stay tuned.

CLRS Algorithm a Day: Linear Search

2018-04-02T16:26:08Z

A birthday, crunch time at work, Easter, experiments with MonoGame, and some other life things have gotten in the way of me continuing this series as frequently as I had planned, but never fear, back into the breach I go. Last time around, we started things off with Insertion Sort, which is the focus of Chapter 2.1 in my copy of the 3rd Edition. The natural next step to continue on to would be other sorting algorithms: selection sort, merge sort, quicksort, etc, etc. And we'll get there. But first I wanted to make a little diversion into one of the exercises for this chapter; writing a linear search.

I'm going to reproduce the problem definition from the exercise here, just to get us started:

Input: A sequence of n numbers such that A = {a₁, a₂, ..., a_n} and a value v.

Output: An index i such that v = A[ i ] or the special value NIL if v does not appear in A.

There's a bit more about writing loop invariants and proving that the algorithm is correct, if you were going to do this as a homework assignment. But that's incredibly dull, especially for something as self-evident as this. Let's get right to writing some code (github); this will be a short one.

Linear Search

There's not a whole lot to get fancy with with a linear search.

We have the value we're searching for, and an array of values.
We start at the beginning and compare the first item in the array with our search value, if it matches, we return that index.
If not, we move on to the second item, and repeat
If we get to the end and haven't found the value, then it isn't present, and we return our NIL sentinel value.

Since C# has 0-based arrays, and doesn't allow (yet) for any trickery like using negative indices to slice from the end, -1 is a reasonable sentinel value (and matches with the convention of the CLR Array.IndexOf function). So that leads pretty naturally to this, where we just use a for-loop:

public static int LinearSearch(this int[] a, int find) {
    for (var i = 0; i < a.Length; i++) {
        if (a[i] == find) {
            return i;
        }
    }
    return -1;
}

CLRS Algorithm a Day: Insertion Sort

2018-03-07T06:19:54Z

The other day I was on HackerNews lurking as I do, and there were a number of people praising Introduction to Algorithms, the infamous CLRS book. Almost everybody who has been through a Computer Science degree has had to buy a copy of it and slog along through at some point. It's basically the bible of algorithms and data structures classes, and as most of that kind of book are, it's a weighty tome that costs about a week's work-study wages. Mine has mostly been gathering dust on the shelf for the past ten years...

I've got a little bit of a love-hate relationship with the book. My algorithms and data structures class in college was entirely theoretical - we didn't write a single line of code during the whole semester, we just did page after page of proofs. Moreover, the professor was one of those space-case academic types, who was probably a bang-up researcher, but couldn't teach a lick. My only enduring memories of that course are chalk scratching on the blackboard, as she transcribed the textbook word for word and symbol for symbol, and sitting in the 1902 Room in Baker Library with reams of scratch paper scattered across the table, flailing away doing induction proofs. Oh, and the week we spent going over that median-of-medians algorithm that I've still never seen a point to... I think I scratched out a B-, and it was easily the least enjoyable class of my entire degree.

On the other hand, I could see that there was a lot of really cool information in the text, if it was taught differently. I always thought that a hands-on version of the class, where you had to implement everything in C, rather than pseudocode and pencil, would be really fun, and moreover, really teach people how to program. Alas, we skipped a lot of the interesting material in the book, and overwhelmingly focused on proving run-time complexity bounds. Which I always thought was a little dumb, detached from real-life implementation details - there was very little consideration of trading space for time, or taking into account the capabilities of the hardware, or other real life concerns - but that is neither here nor there.

So this week, I decided to dust off my copy and give it another go. I'm mostly a C# programmer, and probably will remain one for the foreseeable future; it's a really nice language. However, I'm going to try to keep things pretty basic for the most part, to hew as closely to the pseudocode presented in the text.

Drawing State Machine Diagrams with Microsoft Automatic Graph Layout

2017-09-12T00:40:34Z

TLDR; the Microsoft Automatic Graph Layout library is pretty slick at automatically generating state transition diagrams.

One of the products that I work on for my day job is an instant message-based call center/helpdesk application. It's a pretty complicated beast, but at the heart of it, what it does is:

Allow users to contact a central endpoint.
Process the incoming request
Route the request to a member of a predefined group that can assist the end-user
Broker the conversation between the end-user and the helpdesk/call center agent.
Perform post-processing on the chat session once it has ended.

Internally, each incoming chat session uses a state machine to keep track of the session and its progression through the workflow. For each stage in the pipeline, we've got a State class that handles operations during that particular stage. There is also a table in our state machine which determines what the valid transitions between the states are, so that we can easily assert that the session is following the expected workflow.

This state machine started out fairly simple, with just a few states, but as always happens, we began to accrete features, and so more and more stages were added into the pipeline, and the complexity and number of state transitions grew. The state machine definition that originated as a hastily drawn scrawling on a sheet of printer paper blossomed into a Visio diagram, then eventually wilted into irrelevance as more and more states were added and documentation efforts fell behind. Like most manual processes, it became a pain point, and was neglected.

Things came to a head recently when it was discovered that a state transition had been added as a code path in certain situations, but the state machine transition table had not been updated accordingly, and so it was possible for chats to attempt to make a transition that the state machine would reject as invalid, leaving the chat session in limbo. Not ideal. Ideally, we'd have caught this in some way with compile or build-time static analysis, but getting that implemented is going to have to be an exercise for the future. Failing that, exhaustive unit-tests validating the state transition rules was the easier task, but ran into the difficulty that the hand-written design documentation had fallen out of date, which necessitated a tedious and time-consuming search through the code base to trace out the logic. Once that was complete, there remained the task of updating the Visio docs... which is less than fun to do by hand.

Earlier that week, I had run across a link to the Microsoft Automatic Graph Layout library, a nifty open-source project that makes it stupidly easy to define a graph and then either display it in a Windows Form or render to an image file. While it may not be quite as nice looking as a hand-crafted Visio doc, it's not too unattractive, and better, can be completely automated. So we can just pipe our state transition definitions into it from the source code, and right out comes a legible diagram. We can even run it as a post-build step in our build to automatically keep the documentation up-to-date. Hooray!

Mazes in C# - Part 2 - Dijkstra's Algorithm

2017-05-16T04:20:33Z

Last time, I had started working through Jamis Buck's Mazes for Programmers. I've made quite a lot of progress in the meantime, but haven't managed to keep up with posting about that progress. Today, we're going to work through the meat of Chapter 3, where we'll implement a simplified version of Dijkstra's Algorithm (really a breadth-first search), which we'll use to find paths through our mazes. We will also use this algorithm to help visualize the biases that exist in our two previous maze generation algorithms, by coloring the backgrounds of each cell according to its distance from a starting point.

You can download the code for this post by downloading or cloning the Post_2 branch from my GitHub. Going forward, I'm going to try and tag the code for each post in a new branch - I'm still used to the old SVN model, but I'm trying to figure out how to take advantage of Git a little more.

Below you can see an example of where we'll end up, drawing the longest path through a maze, and coloring each cell according to its distance along the path.

Let's get started!

Mazes in C# - Part 1

2017-04-09T19:11:38Z

A few weeks back, I picked up a copy of James Buck's Mazes for Programmers: Code Your Own Twisty Little Passages. (The title is, of course, an homage to one of the original text adventure games, Colossal Cave Adventure, or Adventure for short, which featured a maze puzzle where every room was described as "You are in a maze of twisty little passages, all alike"). I read through most of it on my veranda in the mornings, looking out over Long Bay in Antigua while I was on vacation, wishing I had brought my laptop along with me (although my girlfriend is probably happy I did not...). Once I got back, I started diving into implementing some of the ideas from the book - it's been quite a joy, and a much needed break from debugging UCWA applications at the day job.

The code in the book is implemented in Ruby, however, I'm not a Rubyist, so I decided that I'd work through the examples in my language of choice, C#. The code is pretty simple, so porting it over has been pretty straight-forward. I also figured it would be an interesting exercise for trying out some of the newer features in the more recent versions of C# - I've been stuck using C# 5.0 due to some tooling that we haven't had a chance to upgrade yet at work. Today, I'm going to mostly cover the examples from Chapter 2, which lays out the general plumbing of the framework that the rest of the book builds on, and implements two simple maze-building algorithms, the Binary Tree and Sidewinder algorithms. For the full code, you can check out my GitHub repository; things there may change slightly from the version presented here as I work through more of the book, but the general structure should remain similar.

Voronoi Diagrams

2016-08-26T16:28:17Z

A little over two years ago, I first saw Amit Patel's article on Polygonal Map Generation, and thought it was incredibly cool. The use of Voronoi regions created a very nice, slightly irregular look, compared to grid-based terrains. At the time, I had just finished up working on my DX11 random terrain code, and it looked like a fun project to try to tackle.

I then proceeded to spend several months messing around with different implementations of Fortune's Algorithm in C# to get started and generate the Voronoi polygons used to generate a terrain along the lines of Amit's example. At this point, I've lost track of all of the different versions that I've sort of melded together to produce the code that I've ended up with in the end, but some of the more influential are:

The original implementation of the algorithm by Steve Fortune
This JS version by Nicolas Garcia Belmonte
This C++ version, which is a port of the ActionScript version used in the original Amit Patel map generator.

The original goal was to create a map generator, suitable for a kind of overworld/strategic level map. But, alas, life happened, and I got bogged down before I got that far. I did, however, wind up with a fairly cool tool for generating Voronoi diagrams. Because I had spent so much time trying to iron out bugs in my implementation of the algorithm, I ended up producing a WinForms application that allows you to step through the algorithm one iteration at a time, visualizing the sites that are added to the diagram, the vertices and edges, as well as the position of the beach and sweep lines. Eventually I also worked in options to show the circles through three sites that define where a Voronoi vertex is located, as well as the Delauney triangulation of the sites.

Voronoi regions, with the edges drawn in white, and the sites as the blue points.

Delauney triangulation, with triangle edges in green.

Showing both the Voronoi regions and the Delauney triangles.

I won't pretend that this code is fantastic, but it's kind of interesting, and I worked at it for quite a while, so better to have it out here than moldering on a hard drive somewhere. If you'd like to see the source, it is available on GitHub. You can also download the executable below if you like - I make no promises that it will work everywhere, but it is a pretty standard .Net 4.5 Windows Forms application. I've also got some videos below the fold, if you'd like to see this in action.

Download Voronoi

Ray Tracing #3: Let's Get Some Actual Rays!

2016-04-10T11:01:30Z

Alright, ready for the third installment of this ray tracing series? This time, we'll get some actual rays, and start tracing them through a scene. Our scene is still going to be empty, but we're starting to get somewhere. Although the book I'm working from is titled Ray Tracing in One Weekend, it's starting to look like my project is going to be more like Ray Tracing in One Year...

Once again, I'm going to put all of the relevant new code for this segment up here, but if you want to see the bits I've missed, check out my GitHub project. We will be circling back to the Vector3 structure I created last time, since I inevitably left out some useful operations...

The core of what a ray tracer does is to trace rays from an origin, often called the eye, for obvious reasons, through each pixel in the image, and then out into our scene to whatever objects lie beyond. We don't have any objects to actually hit, yet, but we are going to lay the groundwork to start doing that next time. Below, you can see the setup of our eye, the image plane, and the rays that shoot from the eye through the image and into the scene beyond.

Ray Tracing #2: Abstractions

2016-03-06T17:31:04Z

It's going to take me considerably longer than one weekend to build out a ray tracer...

Last time, I laid the groundwork to construct a PPM image and output a simple gradient image, like the one below.

This time around, I'm going to focus on building some useful abstractions that will make work going forward easier. This is going to focus on two areas:

A Vector3 class, which will be helpful for representing 3D points, directional vector, RGB colors and offsets. We'll implement some useful operators and geometric methods in addition.
A Bitmap class, which will represent our output raster and handle the details of saving that raster out as a PPM image file.

Ultimately, we'll be producing the same image as in the last installment, but with considerably less boilerplate code, and lay the groundwork for making our lives much easier going forward when we get to some more meaty topics. As always, the full code is available on GitHub, but I'll be presenting the full code for this example in this post.

Hello Raytracing

2016-02-18T05:43:35Z

Whew, it's been a while...

A few weeks ago, I happened across a new book by Peter Shirley, Ray Tracing in One Weekend. Longer ago than I like to remember, I took a computer graphics course in college, and the bulk of our project work revolved around writing a simple ray tracer in Java. It was one of the few really code-heavy CS courses I took, and I really enjoyed it; from time to time I keep pulling down that old project and port it over to whatever new language I'm trying to learn. One of the primary textbooks for that course was Fundamentals of Computer Graphics, aka "the tiger book," of which Mr. Shirley was also an author. Since that's one of the more accessible graphics textbooks I've encountered, I figured this new offering was worth a look. It didn't disappoint.

Ray Tracing in One Weekend is, true to its title, a quick read, but it packs a punch in its just under 50 pages. Even better, it's running at around $3 (or free if you have Kindle Unlimited). I paged through it in a couple of hours, then, as I often do, set about working through the code.

If you want to follow along, I've put my code up on Github, although it's still a work in progress; we're wrapping up a new release at the day job and so I've not had a huge amount of time to work on anything extra. Fortunately, each example I'll be doing here is pretty self-contained, and so all of the code will be up here. We'll start at the beginning, with a simple Hello World a la raytracing.

Finite State Machines, Part 1

2015-08-02T18:14:56Z

One of my favorite books on AI programming for games is Matt Buckland's Programming Game AI By Example. Many AI programming books lean more towards presenting topics and theories, leaving the dirty work of implementing the techniques and algorithms up to the reader. This book takes a very different tack, with each chapter featuring one or more fully implemented examples illustrating the techniques covered. Even better, it comes in a format similar to a trade-paperback, rather than a coffee table book-sized tome, so it's relatively handy to carry around and read a bit at a time, as programming books go. It is also decidedly focused on the kinds of real-world, relatively simple techniques that one would actually use in the majority of games. And, mixed in the right combinations, these simple techniques can be very powerful, as the second half of the book displays, building an increasingly sophisticated top-down, 2D shooter game.

What I particularly like about this book though, is that while it is presented as a book for programming game AI, it may be the best practical explanation of a number of fundamental AI techniques and patterns that I have seen. The lessons that I learned reading through this book have been just as applicable in my day-job as an enterprise developer as in my hobby work programming games, much more so than what I learned toiling through a semester-long AI course based on Artificial Intelligence: A Modern Approach...

The first real chapter of Mr. Buckland's book (leaving aside the obligatory math primer chapter), is devoted to finite state machines. Finite State Machines are one of the simpler ways of organizing decision making, and are probably one of the most intuitive. I'll let Mr. Buckland's definition stand by itself:

A finite state machine is a device, or a model of a device, which has a finite number of states it can be in at any given time and can operate on input to either make transitions from one state to another or to cause an output or action to take place. A finite state machine can only be in one state at any moment in time.
Matt Buckland Programming Game AI By Example, p.44

I've been working on porting the first example from Mr. Buckland's FSM chapter from C++ to C#, featuring a mildly alcoholic, spaghetti Western gold miner named Bob. I'm going to focus mostly on the FSM-specific code here, but you can get the full code from https://github.com/ericrrichards/ai/tree/master/trunk/WestWorld1.

A template for Nancy OWIN Web Apps running as Windows Services

2015-07-12T20:55:14Z

Until recently, if you wanted to write a .NET web application and run it on Windows, that generally meant creating an ASP.NET project, and then deploying it to IIS. For most use-cases, that was not that painful - it's pretty well tuned for supporting a typical request/response dependent web site. IIS is also very powerful and very configurable, which is both a blessing and a curse.

In my day job, we've built out several enterprise applications for interfacing and extending instant messaging platforms, like Microsoft Lync and IBM SameTime. Invariably, we need some kind of web interface for these applications, for configuring settings, displaying metrics, and so forth. Until the past year, we followed the standard pattern, with ASP.NET web applications on top of IIS. In our case, however, there were some pretty serious pain-points that would repeatedly arise:

Complicated Deployments: The software we sell to our customers is distributed as an installer that they install on servers within their corporate networks. The tooling that we had to build our installers was not always the most powerful, and not well-suited to automatically installing Windows features and IIS roles required for our applications. I've lost track of the number of support calls that I have been on where a customer skipped installing prerequisite components, or missed some of the required IIS roles. Also, having to install an array of IIS roles and features for a very light-weight web application really feels like overkill.
IIS Configuration Woes: Similarly, because IIS is so configurable, it's easy enough for some of the settings to get twiddled and break our applications. Particularly when one of our applications shares a server or IIS site with other applications (the Lync Server Web UI and SharePoint are common culprits), sorting out the interaction between global IIS settings, site-level settings, and settings for our applications can be needlessly complicated.
Mismatches between standard IIS behavior and our applications' needs: ASP.NET on IIS is generally oriented around a low-persistence request/response model. Some of our applications have considerable amounts of background processing to perform their work, which doesn't fit this model very well. By default, IIS application pools are set to terminate applications if they have been idle (i.e. not served an HTTP request) in a relatively small amount of time, and also to recycle the application at set intervals. There are ways to get around this behavior, by disabling the idle timeouts and recycling intervals, or employing some other service to periodically ping a keep-alive method on the application. Or you can split the UI component off from the background processing and deploy that as an additional service, but that introduces the complexity of communicating between the two processes and requires installing and configuring an additional service. We've done both, but neither is particularly fun.

Clearly, we were not the only ones to run into these pain-points. Eventually, Microsoft released the OWIN framework, which is much more light-weight and modular web application model than traditional ASP.NET. Along with OWIN came Project Katana, which was the default implementation of OWIN. While OWIN can be integrated into ASP.NET running on IIS, one of the huge selling points of OWIN and Katana is that it makes it considerably easier to create a stand-alone light-weight web server, running out of a Windows Service or even a regular old desktop application.

Running a light-weight stand-alone web server offers a lot of benefits:

No dependencies on IIS or ASP.NET. All that is required is the .NET Framework, which drastically cuts back on the complexity of checking for and installing prerequisites at deployment.
Much simpler deployments. Since your application is basically just an .exe, deployments are much simpler, no need to register anything with IIS, setup application pools, or any of that mess. You can go as far as simply zipping up the application directory, unzip it on the target, run the executable, and have a web application up and running.
Complete control over configuration. Generally with OWIN, all the configuration for your webserver is explicitly configured in code. All of the IIS control panel and web.config fiddling goes away, and you can specify exactly the behaviors expected.
Application lifecycle is simpler. Your application can service any incoming HTTP requests using the OWIN HttpListener, which works on its own background threads, while your application performs any other processing. There aren't any idle timeouts or automatic recyclings, and your application runs until you decide to stop it.

This all sounds pretty excellent, and in practice it has been a pretty big win for us. However, one difficulty that we encountered was in trying to convert our existing ASP.NET MVC applications over to OWIN self-hosted versions. ASP.NET 5 has been available for some time as a release candidate, and should be available in a final version soon, but at the time that we were first investigating making the switch to OWIN, there was not any version of ASP.NET MVC that could be used with the self-hosted OWIN HttpListener. One option could have been to rewrite our applications to use WebAPI on the back-end and serve static HTML, with JavaScript to load in data from the API, but we had relatively mature applications that followed the MVC way of doing things, populating Razor cshtml files from model classes server-side, and so doing that kind of rewrite and all the necessary regression testing was not particularly attractive.

So we cast about for an alternative that would let us reuse more of our existing code, and came upon the Nancy web framework. While there are some differences in execution, Nancy follows roughly the same conceptual model as ASP.NET MVC, and also provides a view engine with support for Razor cshtml views. It also had a mature and simple-to-use OWIN integration package, so after some experimentation to determine feasibility, we jumped in and started converting over some of our applications, and it has been a great success for us.

With that justification out of the way, I'm going to go ahead a present a very barebones example of how to setup a self-hosted OWIN web application that uses Nancy and runs as either a console application or as a Windows service. The source code is available on GitHub, at https://github.com/ericrrichards/NancyOwinTemplate and you can also download this code as a Visual Studio 2013 project template.

Xamarin Forms Succinctly - Review

2015-06-25T01:19:59Z

Recently, I've started work on developing a mobile application to interface with one of my employer's server-side products. Because this product is primarily a C# .NET application, we decided to take a look at using Xamarin.Forms, so that we could leverage parts of our existing code-base and also make use of Xamarin's cross-platform capabilities, rather than developing separate native clients using Java on Android, Objective-C on iOS, or C# on Windows Phone. Since we are primarily a C#/HTML/CSS/JS shop, the potential time-savings of being able to reuse our current expertise in .NET programming languages, and only having to learn the Xamarin framework, as opposed to two additional languages and three mobile development frameworks was a huge plus - particularly as some of the technologies that we would use to interact with our server-side product are not officially supported from the native platforms to the same degree as their .NET implementations.

Time will tell if Xamarin.Forms is the correct long-term solution for our needs - on the one hand, our particular application does not need much in the way of device-specific capabilities and customization that Xamarin does not provide access to, but on the other, the licensing costs for Xamarin are somewhat steep. At least for the purposes of rapidly throwing together a prototype proof-of-concept and exploring the feasibility of the idea, using Xamarin.Forms has been a great success - in the equivalent of one week of developer time, we were able to produce a fully-functional prototype.

While I was able to throw together something workable in a short period of time working from the Xamarin sample code, a variety of blog posts, and trawling the Xamarin Forms development forums, the task would have been much easier, and I would have avoided some missteps, if I had taken the time to read a brief overview of the technology beforehand. About midway through the prototype, I happened upon Xamarin Forms Succinctly, which is that quick overview that I was looking for. It is a quick read, at only 148 pages in the PDF version, but it provides a good surface treatment of the concepts and techniques for building simple Xamarin Forms applications - I printed out a copy, put it in a three-ring binder and read through it over the course of a couple of evenings and lunch breaks. It helped a great deal in filling in some of the blanks that I had missed in my more haphazard initial exploration, and it was a great help in revising some of the issues and antipatterns from the inital prototype version of our app.

I would definitely recommend that anyone who is interested in investigating Xamarin Forms and seeing what the technology offers take a few hours and read through Xamarin Forms Succinctly. For the price (free, other than registering an account with SyncFusion), and the time investment required (minimal), it is hard to beat. Besides, it's a gateway into the large library of free, generally high-quality, Succinctly ebooks, which cover a vast array of technical topics in a quick, accessible format.

Setting up Chocolate Wolfenstein 3D in Visual Studio 2013

2015-05-20T20:00:00Z

For the past few weeks, I've been once again noodling on the idea of starting a .NET port of a classic Id FPS. As a kid on my first computer, an off-brand 486 with DOS, I just hit the tail end of the good old days of shareware. And amongst all the floppy disks of kiddy and educational software and sliming Gruzzles couldn't really hold a candle to exploring Indiana Jones and the Last Crusade-esque Gothic castles and knifing Nazis.

While the original source-code for Wolfenstein 3D has been available for some time, it is a bit of a slog trying to wade through C code that was written 20 years ago, with near and far pointers, blitting directly to VGA memory, and hand-rolled assembly routines, let alone build the project successfully. Consequently, converting over to C# is a bit of a struggle, particularly for some of the low-level pointer manipulation and when loading in binary assets - it is very helpful to be able to step through both codebases side by side in the debugger to figure out any discrepancies.

Because of these difficulties, I have started looking at the Chocolate Wolfenstein 3D project, by Fabien Sanglard. Mr. Sanglard is a great student of the Id Software engine source code, and has done several very nice writeups of the different engines open-sourced by Id. He was even planning on writing a book-length analysis of Wolfenstein 3D, which hopefully is still underway. Chocolate Wolfenstein 3D is a more modernized C++ conversion of the original Id code, with some of the more gnarly bits smoothed out, and using SDL for cross-platform window-management, graphics, input, and sound. Even better, it can be built and run using current versions of Visual Studio.

The only problem I had with the Chocolate Wolfenstein 3D GitHub repository is that it is missing some dependencies and requires a small amount of tweaking in order to get it to build and run on Visual Studio 2013. These steps are not particularly difficult, but if you simply clone the repo and hit F5, it doesn't work right out of the box. If you are working on a Mac, there is a very nice guide on setting up the project in XCode, but I have not found a similar guide for Windows, so I decided to document the steps that I went through and share those here.

Returning Data from the Client to the Hub with SignalR

2015-05-13T20:00:00Z

If you have not used it yet, Signalr is a real-time web communication library which is now under the ASP.NET banner. It is a very cool technology, which allows you to actively push data from a server application to your clients, without having to worry too much about the particular transport mechanism by which the content is delivered - SignalR ideally uses WebSockets, but degrades gracefully to older transport protocols (like AJAX long polling), if WebSocket connections are not supported by either the client or server. This is very powerful, since it allows you to write a single implementation, focused on your high-level needs, and SignalR handles most of the nitty gritty work for you. In addition, while SignalR can be used as something of a dumb pipe (although not that dumb, since you do get object->JSON serialization out of the box) via PersistentConnection, the real power of SignalR lies in its Hubs concept. By connecting to a SignalR Hub, a client can call methods on the server, both to execute commands, or to query for data. Similarly, the server can issue commands to the client by calling client-side callback methods.

This allows you to do some pretty cool things. Probably the canonical example is web-based instant-messaging/chat rooms, which is dead simple to implement using SignalR. But really any kind web page that polls for new data on an interval is a good candidate for SignalR, like, say, real-time dashboards, or a sports score ticker, maybe some classes of multiplayer games.

One scenario that SignalR does not support very nicely is when the server needs to query its clients for information and have that data returned to the Hub. As currently implemented, a SignalR client can only define handlers for methods called by the Hub which are Action<T,...> delegates, which have a void return signature. So, while the code below will compile if it is included in a Hub method, data is actually a Task object, which is an asynchronous wrapper for the client-side Action. In short, there is no way to directly return a value from a client-side method back to the Hub.

var data = Clients.Client(SOME_ID).GetData()

Why would you want to do this kind of action using SignalR? In my case, I recently encountered a situation where a third-party library implemented a wrapper around a COM application. This COM wrapper was implemented as a singleton, but in my particular case, I needed to execute multiple instances of the underlying COM application, in order to connect to an external service with different user accounts, but, because the wrapper was a singleton, I could only make one connection from my main application process. However, I could get around this limitation by spawning an additional process for each instance of the COM wrapper that was required (fortunately, it was simply a C# singleton, and not limited by any kind of global mutex). While each child process could for the most part operate independently performing its responsibilities, the main application needed to be able to issue commands to control the child processes, as well as retrieve information from them for various reporting features and to ensure consistency in various aspects globally across the application.

I considered using a more traditional IPC method, like bidirectional sockets, or named pipes, but the simplicity and cleanness of using SignalR was more attractive to me than rolling my own message-passing protocol and coding a message loop on both sides. At least for issuing commands to the child processes, implementing a SignalR Hub on my main application to which the clients would connect was pretty quick and easy. Then I ran into the difficulty of how to get data back out of the child processes using SignalR.

In retrospect, I could have modified the code executing on the child processes into a more event-driven form, so that I could use the normal SignalR paradigm to push information to the Hub in the main application, instead of querying the child processes for it. However, this would have involved a complete revision of the architecture of the application (there were very similar components that ran within the main process, using the same interfaces and the bulk of the implementation), as well as necessitating an extensive amount of caching in the main process.

Instead, I settled on the slightly crazy pattern illustrated here. In broad strokes, what I am doing is:

A specially-tagged SignalR client owned by the main application calls a method on the Hub requesting a piece of data from a specific child process.
The Hub resolves the requested child process to its SignalR connection Id, and calls a method to order the child process to assemble the requested data.
The child process calculates the requested data, and pushes it to a different method on the Hub.
The second Hub method stuffs the pushed data from the child process into a dictionary, keyed by the child process' identity.
While this has been happening, the first Hub method has been blocking, waiting for the requested data to appear in the dictionary.
When the data appears in the dictionary, it is popped off, removing the key from the dictionary, preparing the dictionary for the next invocation. There is also a timeout value, so that if there is some sort of interuption, a default value will be returned, rather than indefinitely blocking.
Finally, the first Hub method returns the piece of data to the main application.

The code for this example is available on GitHub at https://github.com/ericrrichards/crazy-signalr-ipc

Model Loading Code Updated to AssimpNet 3.3.1

2015-02-11T17:15:00Z

Just a quick update today. I've updated the 3D model loading code to use the latest version of AssimpNet that is on Nuget now. The latest code is updated on GitHub.

The biggest changes appear to be that the AssimpImporter/Exporter classes have been merged into a single AssimpContext class that can do both. Some of the GetXXX methods to retrieve vertex elements and textures also appear to be replaced with properties. The way that one hooks into the internal Assimp logging has also changed. I haven't discovered many other changes, besides some additional file format support yet, but I haven't gotten around to really explore it that closely. Mildly irritating that the interface has changed here, but hey, at least it is being actively worked on (which is more than I can say for some of my stuff at times...).

I've been quite busy with work lately, so I haven't had a huge amount of time to work on anything here. I'm also a little burned out on doing straight DX11 graphics stuff, so I think I'm going to try to transition into doing some more applied work for a while. I've got some stuff in the pipeline on making simple games with Direct2D, and ideally I'd like to get involved with One Game a Month, to give me a little additional motivation.

Serving HTML5 Video using Nancy

2015-01-17T11:15:00Z

Recently, I have been using OWIN a good deal for developing internal web applications. One of the chief benefits of this is that OWIN offers the ability to host its own HTTP server, which allows me to get out of the business of installing and configuring IIS on windows, which is one of the main points of pain when deploying the products I work on to our customers. Unfortunately, when I first started using OWIN, there was not a version of ASP.NET MVC available that was compatible with OWIN. Most of my previous experience with programming web servers has been based on MVC (except for briefly experiencing WebForms hell), so finding a similar framework that was compatible with OWIN was one of my first priorities.

In my search, I discovered Nancy, a fairly similar MVC-style framework which offered OWIN support. It also was capable of using the same Razor view engine as ASP.NET MVC, with some minor differences, so I was able to convert existing IIS ASP.NET MVC applications to OWIN/Nancy using most of the existing views and front-end code. At some point I plan to write an article illustrating how one would do this type of conversion, but for now, I'm going to examine one particular gotcha I discovered when converting my personal Netflix-type video application to OWIN/Nancy: serving HTML5 video files.

Geodesic Sphere Tessellation

2014-12-21T13:35:00Z

A couple of weeks ago as I was browsing HackerNews, I stumbled onto an article about creating spherical procedural maps by Andy Gainey. Procedural terrain/map generation is always something I find interesting, having grown up slightly obsessed with Civilization and its successors. Various methods of tiling a sphere in order to make a game grid have been bouncing around the back of my mind for a while now - unfortunately finding time to explore some of those ideas has been problematic. But, after reading through Mr. Gainey's piece and looking over the source for his demo, I got inspired to take a stab at something similar. My first attempt was to try to directly port his Javascript implementation, which got bogged down in the impedance mismatch between going from JS to C# and WebGL to DirectX at the same time, so I stepped back, and decided to come at the idea again, working in smaller steps.

The first step was to generate a spherical triangle mesh. For this purpose, the spherical mesh that we have previously worked with is not very well suited, as its topology is very inconsistent, particularly around the poles, as you can see in the screenshot below. Instead of this kind of lattitude/longitude subdivision, we will instead use an alternate method, which starts from an icosahedron, then subdivides each face of the icosahedron into four triangles, and finally projects the resulting vertices onto the sphere. This produces a much more regular tessellation, known as a geodesic grid, as you can see in the right screenshot below. This tessellation is not without artifacts - at the vertices of the original icosahedron, the resulting geodesic will have five edges, while all other vertices have six edges, similar to a real-life soccer ball.

Lattitude/Longitude tessellation, with artifacts at the poles

Geodesic tessellation

Frank Luna's Introduction to 3D Game Programming with Direct3D 11.0, Chapter 6, presented a method of generating of generating a geodesic sphere, which I previously skipped over. The code for this example is based upon his example, with some improvements to eliminate redundant vertices.

The code for this example can be obtained from my GitHub repository.

HLSL Cookbook: Directional Lighting

2014-11-24T17:56:00Z

Moving along with Chapter 1 of the HLSL Development Cookbook, we're on to handling directional lighting. I've covered most of the theory behind directional lighting previously, so this is going to be quite brief.

To recap, in case anyone is unfamiliar with the term, a directional light is a light which illuminates the entire scene equally from a given direction. Typically this means a light source which is so large and far away from the scene being rendered, such as the sun or moon, that any attenuation in light intensity, via the inverse-square law or variations in the direction from any point in the scene to the light source location are neglible. Thus, we can model a directional light simply by using a directional vector, representing the direction of incoming light from the source, and the color of the light emitted by the light source.

Generally, we model the light hitting a surface by breaking it into two components, diffuse and specular, according to an empirical lighting equation called the Phong reflection model. Diffuse light is the light that reflects from a surface equally in all directions which is calculated from the angle between the surface normal vector and the vector from the surface to the light source. Specular light is light that is reflected off of glossy surfaces in a view-dependant direction. This direction of this specular reflection is controlled by the surface normal, the vector from the surface to the light, and the vector from the surface to the viewer of the scene, while the size and color of the specular highlights are controlled by properties of the material being lit, the specular exponent, approximating the "smoothness" of the object, and the specular color of the material. Many objects will reflect specular reflections in all color frequencies, while others, mostly metals, will absorb some frequencies more than others. For now, we're only going to consider the first class of materials.

Below you can see a model being lit by directional light, in addition to the ambient lighting we used in the last example. Here, the directional light is coming downward and and from the lower-left to the upper-right. Surfaces with normals facing more directly towards the light source are lit more brightly than surfaces which are angled partly away from the light, while surfaces facing away from the light are lit only by ambient lighting. In addition, you can see a series of brighter spots along the back of the rabbit model, where there are specular highlights.

Full code for this example can be downloaded from my GitHub repository.

HLSL Cookbook: Hemispherical Ambient Lighting

2014-11-13T17:56:00Z

Well, it's been two rough months since the last post. Work has been crazy, and I've been wrestling with finishing up my never-ending Voronoi mapping project. So I decided to switch gears and return to the HLSL Development Cookbook. Now that I've got code in place to handle loading the .sdkmesh model files used by the book's examples, the hardest part is done. Now it is just a matter of converting plain HLSL vertex and pixel shaders over into the Effects Framework format I've been using, and porting the DXUT C++ code over to C# and my SlimDX application framework.

The first chapter of the HLSL Development Cookbook covers classic forward lighting, which is the style of lighting that we have used throughout the examples posted here thus far (see Directional, Point and Spot lights and three-point lighting). Later, the book covers some more complicated lighting schemes, but we'll get to that in due time.

Up first on the slate is ambient lighting. If you've been reading along with my posts, you may recall that we briefly discussed ambient lighting when defining our Light and Material classes. Ambient lighting is something of a hack that was invented to fake the effect of indirect lighting (light that hits a surface after bouncing off one or more other surfaces) without having to actually perform the computations to model the light rays. Historically, back in the days before programmable shaders, this ambient lighting was expressed as constant color values for the lights and materials in the scene being rendered. So far, I have followed this simple scheme in the shaders I adapted from Frank Luna's Introduction to 3D Game Programming with Direct3D 11.0. As you can see in the screenshot on the left below, this method results in a flat ambient color, which conceals the surface details of the mesh. Combined with the diffuse and specular terms in the traditional Phong reflection equation, this method works well, but on its own it is underwhelming.

HLSL Development Cookbook presents an alternative method of calculating ambient lighting, called hemispheric ambient lighting. This method takes advantage of programmable pixel shaders, and allows us to calculate a varying ambient color per pixel, based on the interpolated surface normal and two constant color values representing the color of light that is bounced off the ground and off the ceiling/sky. As you can see in the screenshot on the right below, this lighting method preserves the surface details of the mesh (so long as different colors are selected for the top and bottom colors). As we'll see, this method of computing ambient lighting is also pretty cheap to do computationally.


Old-style flat ambient lighting	Hemispherical ambient lighting

As always, the full source code for this post is available at my github repository.

C# SDKMesh Loader

2014-09-11T23:56:00Z

Many moons ago now, I picked up a copy of HLSL Development Cookbook by Doron Feinstein. I had intended to work my way through it after I finished up Luna's Introduction to 3D Game Programming with Direct3D 11.0, but winter and life kind of got in the way...

Another difficulty I had with this book was that the code samples made heavy use of DXUT which was the semi-official Direct3D sample framework. With the Windows 8 transistion, DXUT is sorta-kinda deprecated now, and besides, SlimDX doesn't really have support for it - SlimDX is more of a bare-metal binding to the underlying DirectX framework, which DXUT sits on top of.

So in any case, I was going to have to do a fair amount of rework to adapt the samples from this book to fit into the framework of code that I'd built up over the past year or so. Swapping the UI related stuff out for WinForms controls hosted in my SlimDX app didn't seem as though it would be that hard, but a bigger stumbling block was the fact that all of the sample meshes provided with the code used the .sdkmesh format. SDKMesh is a 3D model format that used to be used in a lot of the DirectX sample code, after .X models kind of fell into disfavor (Of course, now it appears they are using yet another model format, .CMO). SDKMesh is unfortunately not supported by Assimp, so I can't use Assimp.Net to load SDKMesh meshes. Fortunately, it is a relatively simple binary format. The documentation, at least that provided in the DirectX July 2010 SDK, is a little spotty, and not totally correct, but its possible to figure things out by looking at the source code for DXUT and another C# SDKMesh loader that appears to have disappeared from the internet since I first found it...

Well, this has sat in my todo pile for long enough, let's get started. As always, you can get the code for this example from my public GitHub repository.

Site Updates

2014-07-26T20:56:00Z

Unfortunately, I've been busier than expected with work stuff, so my free time hasn't been very abundant. It's looking like I'm going to be porting the Pidgin Sip plugin's Sip stack to c#, since the Microsoft UCMA Sip library isn't really compatible with Office365. The existing options for libraries to do this don't seem to be very suited to handling the Lync-specific SIP extensions... :-(.

Hopefully, more Voronoi work should be coming up next week, so I'll have some material for a real post.

In the meantime, I did find some time to work on the new website today.

Comments are back, although I haven't imported the old Blogger comments to the new format yet. Those may turn up tomorrow if I get ambitious... Parsing them out of the Atom export is not hard, it's just kind of tedious.

I've also hooked up a new RSS/Atom feed. You should now be seeing a link in the top-right corner that will let you subscribe to any new posts via feedburner. Any existing subscriptions should also work now - I've got things setup so the old feed url should point to the new feed as well, thanks to Sean McAlinden's helpful post. Looking over the logs for the last week, it looks like some of you are actually subscribed to my site. I had thought that it was just the service that cross-posts my new posts to Facebook.

Site Migration

2014-07-19T11:16:00Z

I've decided to move my blog off of Blogger. Blogger was great for getting started, but it has just become too painful to fight with going forward. I'm sick of fighting the Blogger templating to force it to display my content the way that I want it to. Blogger has a habit of absolutely mangling the html that I try to post. For posts consisting mostly of plain-text and images, this is not that big a problem, but I have to spend a ton of time trying to get code-heavy posts to render in a readable way. Over the last year, I've spent far more time tweaking my posted html to get it right than it has taken me in the last week to write a home-rolled blog engine for this new site to use, even counting extracting my existing content out of Blogger and converting it into a new format.

Beyond that, there are a number of things that I would like to do going forward that I simply cannot do easily with Blogger, but I can do very easily if I have control over the site. I'm far more comfortable writing html, javascript and server-side code than I will ever be at bending Blogger to my will.

I'm sure there will be some speedbumps, but I think this is a much better solution for me, and ultimately, for you, readers. I'm nowhere near finished, but I think I've got the essentials ironed out, so I'm going to go ahead with the switch-over.

Things that should still work:

Existing links to my page should still be valid, at least for content. I've put in quite a bit of effort trying to get the old-style Blogger urls to play nicely with my new site
All of the old content has been imported.
Images should still be fine, since those were all externally hosted and I haven't swapped those out yet. Eventually, I'd like to serve all my screenshots locally, so I can convert them to jpegs so pages will load faster (most are loss-less PNGs right now).

Things that are broken

Comments are disabled for the moment. Extracting the primary content from Blogger was enough of a pain, so I haven't bothered with the old comments yet. I also need to add server-side support for comments. If you have any questions about any of the tutorials in the meantime, at the top of each post is a link with my name that will allow you to send me an email. I'll try to get back to you as quickly as I can.
Rss feeds - From my analytics, I don't think anybody actually used the blog feed that Blogger provided, but I'm not going to bother with implementing one for the new site unless there is some demand or I have some spare time and get inspired.
Some of the older posts might look a little wonky. I'm going through them as I have time and making sure that the content I extracted from Blogger renders decently, but it is time-consuming. Particularly some of the oldest posts, when I was still using the online Blogger editor, before I standardized my workflow on Windows Live Writer, may be a little bit off.

Thanks for bearing with me. If you notice anything unusual, feel free to send me an email, it would be very helpful in pinpointing issues.

Clipping Lines to a Rectangle using the Cohen-Sutherland Algorithm

2014-07-14T18:56:00Z

For the last six or eight months, off and on, I’ve been trying to write some code that will create a Voronoi diagram from a set of random points, inspired by Amit Patel’s Polygonal Map Generation demo. In the last week or so, I had a bit of a break-through, in that I finally managed to get an implementation of Fortune’s Algorithm put together that would actually work and generate the Voronoi edges and vertices correctly so that I could render them. Since most of the existing implementations I’ve been trying to work from are either broken or an enormous pain in the ass to try to understand, I’m planning on writing up my implementation, once I’m finally happy with it. I’ve still got a fair way to go, since right now I’m only outputting the graph edges and vertices successfully, and while that renders prettily, I haven’t gotten to the point that I have all of the really useful graph connectivity information being output yet. Hopefully, if everything goes well, I’ll manage to finish that up by the end of the week, but this has proven to be a real bugger of a problem to solve.

As I mentioned, most of the implementations I’ve studied are either incomplete or broken. Particularly, I haven’t found an example yet that correctly clips the edges of the generated Voronoi polygons to a rectangular area. So long as all you care about is rendering the graph to the screen, this isn’t a big problem, since most 2D graphics libraries will happily draw lines that extend beyond the drawable area of the screen. However, if you’re trying to subdivide a rectangular region into Voronoi polygons, its kind of nice to have your edges actually clipped to that region. What I’m envisioning doing with this code eventually is using it to render 3D maps, subdivided into territories, but with an overlaid rectangular grid for moving units – think of the strategic map in Total War games or Lords of the Realm.

After I got frustrated with trying to make the broken clipping code I was working from perform correctly, I trolled Google and came across the Cohen-Sutherland line-clipping algorithm. This looked to be exactly what I needed, and wonder of wonders, the Wikipedia page actually featured a readable, reasonable example, rather than the obtuse academic pseudo-code you usually find there (see the Fortune’s Algorithm article…). The only thing I would caution you about with the Wikipedia example is that it uses a coordinate system where the origin is the lower-left bounds of a rectangle as usually encountered in mathematics, rather than the upper-left origin we commonly use in computer graphics, so some tweaking is necessary.

The code for this example can be downloaded from my github repository, at https://github.com/ericrrichards/dx11.git. The algorithm is included in the Algorithms project, while the example code is in the CohenSutherlandExample project. The example code is a quick-and-dirty mess of GDI drawing code, so I’m going to focus on the algorithm code.

After clipping:

One Year Later…

2014-07-10T11:58:00Z

Tuesday was the anniversary of my first real post on this blog. For the most part, I’ve tried to keep my content here on the technical side of things, but, what the hell, this is a good time to reflect on a year of blogging – what went well, what went poorly, and where I’m going from here.

A Dynamic ASP.NET MVC Controller using CSScript

2014-03-20T11:45:00Z

At my day job, we are working on a large enterprise system. One of the feature areas of this product is an array of different charts of performance metrics. All of these charts accept a common set of parameters, and generate some json that is used by our client side to render a chart for the requested metric.

At the moment, we are using a standard MVC controller to define the actions necessary to calculate and collate these stats. The other day, we were kicking around the idea of how we would be able to add new charts in for a client, after they already have the product installed. With the current design we are using, there isn’t really an easy way to do that without dropping in the updated build that has the updated controller. That’s not really an ideal situation, since reinstalling and reconfiguring the application is kind of a heavy-handed approach when all we want to do is add a couple of new charts in, without impacting the rest of the application. It also doesn’t give us many options if our customer, for whatever reason, wants to disable certain charts.

Probably, what we’ll end up doing, is using the database to define the chart calculation SQL, either using by storing the SQL statements to retrieve the metrics in a table, or via stored procedures. I’ll admit, I’m not crazy about this approach, since I’ve worked on a couple of other products that used this method, and I have some slight PTSD from trying to troubleshoot convoluted SQL that was poorly written and not checked into source control other than in the master database creation script. With a sane process, this will probably be an effective option; one of the reasons we are likely going to need to go this route is that we are developing parallel .NET and Java versions, due to API restrictions of the underlying technology stacks that we are targeting, while trying to use a common UI and database.

This did get me thinking about how it would be possible to dynamically populate the actions of an MVC controller. I had come across some posts on CSScript, which is a library which allows you to run arbitrary C# code interpretively. Depending on how you use it, you can load up entire classes, methods or even statements, and invoke them at run-time, without the script code being compiled and embedded in your .dll. So, theoretically, it should be possible to define the controller action that calculates the metrics for a given chart in a CSScript file, load that up, execute it using the CSScript interpreter, passing any parameters needed in, and return the results.

The other piece of this is figuring out how to get an MVC controller to accept action routes that are not explicitly defined for it. Fortunately, with the latest version of MVC, you are able to tweak the default behavior of the controller to a considerable degree, if you are willing to delve into overriding some of the virtual methods the Controller class gives you access to. So, after a little digging, I was able to figure out a slightly crazy way to wrestle MVC into doing what I wanted it to do – accepting dynamic controller actions, executing a scripted action, and then returning the result to the webpage.

This is a very quick and dirty example, really just a proof of concept prototype, so I make no promises as to its performance or robustness. But, it was cool to figure out that such a thing was possible, and I figured that I would share it as a jumping-off point in case anyone else encounters a similar crazy requirement. The code is available on my GitHub repository, at https://github.com/ericrrichards/MVCScriptedController.git.

Rendering Text using SlimDX SpriteTextRenderer

2014-02-05T17:35:00Z

Howdy. Today, I’m going to discuss rendering UI text using the SlimDX SpriteTextRenderer library. This is a very nifty and light-weight extension library for SlimDX, hosted on CodePlex. In older versions of DirectX, it used to be possible to easily render sprites and text using the ID3DXSprite and ID3DXFont interfaces, but those have been removed in newer versions of DirectX. I’ve experimented with some other approaches, such as using Direct2D and DirectWrite or the DirectX Toolkit, but wasn’t happy with the results. For whatever reason, Direct2D doesn’t interop well with DirectX 11, unless you create a shared DirectX 10 device and jump through a bunch of hoops, and even then it is kind of a PITA. Likewise, I have yet to find C# bindings for the DirectX Toolkit, so that’s kind of a non-starter for me; I’d either have to rewrite the pieces that I want to use with SlimDX, or figure out the marshaling to use the C++ dlls. So for that reason, the SpriteTextRenderer library seems to be my best option at the moment, and it turned out to be relatively simple to integrate into my application framework.

If you’ve used either the old DirectX 9 interfaces or XNA, then it’ll be pretty intuitive how to use SpriteTextRenderer. The SpriteRenderer class has some useful methods to draw 2D sprites, which I haven’t explored much yet, since I have already added code to draw scree-space quads. The TextBlockRenderer class provides some simple and handy methods to draw text up on the screen. Internally, it uses DirectWrite to generate sprite font textures at runtime, so you can use any installed system fonts, and specify the weight, style, and point size easily, without worrying about the nitty gritty details of creating the font.

One limitation of the TextBlockRenderer class is that you can only use an instance of it to render text with a single font. Thus, if you want to use different font sizes or styles, you need to create different instances for each font that you want to use. Because of this, I’ve written a simple manager class, which I’m calling FontCache, which will provide a central point to store all the fonts that are used, as well as a default font if you just want to throw some text up onto the screen.

The new code for rendering text has been added to my pathfinding demo, available at my GitHub repository, https://github.com/ericrrichards/dx11.git.

IBM Connect 2014

2014-02-02T10:00:00Z

Yikes! It’s been more than two weeks since my last post… It’s been a busy two weeks, as my employer has been gearing up for and attending IBM’s Connect 2014 conference in Orlando. So I’ve had less time than usual to work on my side projects here. Because of that, I’m going to go outside of my usual format, and recap the conference and some thoughts on it. These are my personal opinions, so bear in mind the old adage about the ubiquity and quality of opinions…

Simple Particle Physics

2014-01-18T13:05:00Z

As I mentioned last time, I’m going to move on from fiddling with my Terrain class for a little while, and start working on some physics code instead. I bought a copy of Ian Millington’s Game Physics Engine Development some months ago and skimmed through it, but was too busy with other things to really get into the accompanying source code. Now, I do have some free cycles, so I’m planning on working through the examples from the book as my next set of posts.

Once again, the original source code is in C++, rather than C# as I’ll be using. Millington’s code also uses OpenGL and GLUT, rather than DirectX. Consequently, these aren’t going to be such straight ports like I did with most of Frank Luna’s examples; I’ll be porting the core physics code, and then for the examples, I’m just going to have to make something up that showcases the same features.

In any case, we’ll start off with the simple particle physics of Chapters 3 & 4, and build a demo that simulates the ballistics of firing some different types of projectiles. You can find my source for this example on my GitHub page, at https://github.com/ericrrichards/dx11.git.

Here you can see the four projectile types: 1.) a pistol-type round, 2.) a large artillery shell, 3) a fireball, 4.) a bolt from a railgun or energy weapon

Pathfinding III: Putting it All Together

2014-01-11T12:56:00Z

Watch the intrepid red blob wind its way through the mountain slopes!

Last time, we discussed the implementation of our A* pathfinding algorithm, as well as some commonly used heuristics for A*. Now we’re going to put all of the pieces together and get a working example to showcase this pathfinding work.

We’ll need to slightly rework our mouse picking code to return the tile in our map that was hit, rather than just the bounding box center. To do this, we’re going to need to modify our QuadTree, so that the leaf nodes are tagged with the MapTile that their bounding boxes enclose.

We’ll also revisit the function that calculates which portions of the map are connected, as the original method in Part 1 was horribly inefficient on some maps. Instead, we’ll use a different method, which uses a series of depth-first searches to calculate the connected sets of MapTiles in the map. This method is much faster, particularly on maps that have more disconnected sets of tiles.

We’ll also need to develop a simple class to represent our unit, which will allow it to update and render itself, as well as maintain pathfinding information. The unit class implementation used here is based in part on material presented in Chapter 9 of Carl Granberg’s Programming an RTS Game with Direct3D .

Finally, we’ll add an additional texture map to our rendering shader, which will draw impassible terrain using a special texture, so that we can easily see the obstacles that our unit will be navigating around. You can see this in the video above; the impassible areas are shown with a slightly darker texture, with dark rifts.

The full code for this example can be found on my GitHub repository, https://github.com/ericrrichards/dx11.git, under the 33-Pathfinding project.

Pathfinding II: A* and Heuristics

2014-01-02T11:01:00Z

In our previous installment, we discussed the data structures that we will use to represent the graph which we will use for pathfinding on the terrain, as well as the initial pre-processing that was necessary to populate that graph with the information that our pathfinding algorithm will make use of. Now, we are ready to actually implement our pathfinding algorithm. We’ll be using A*, probably the most commonly used graph search algorithm for pathfinding.

A* is one of the most commonly used pathfinding algorithms in games because it is fast, flexible, and relatively simple to implement. A* was originally a refinement of Dijkstra’s graph search algorithm. Dijkstra’s algorithm is guaranteed to determine the shortest path between any two nodes in a directed graph, however, because Dijkstra’s algorithm only takes into account the cost of reaching an intermediate node from the start node, it tends to consider many nodes that are not on the optimal path. An alternative to Dijkstra’s algorithm is Greedy Best-First search. Best-First uses a heuristic function to estimate the cost of reaching the goal from a given intermediate node, without reference to the cost of reaching the current node from the start node. This means that Best-First tends to consider far fewer nodes than Dijkstra, but is not guaranteed to produce the shortest path in a graph which includes obstacles that are not predicted by the heuristic.

A* blends these two approaches, by using a cost function (f(x)) to evaluate each node based on both the cost from the start node (g(x)) and the estimated cost to the goal (h(x)). This allows A* to both find the optimum shortest path, while considering fewer nodes than pure Dijkstra’s algorithm. The number of intermediate nodes expanded by A* is somewhat dependent on the characteristics of the heuristic function used. There are generally three cases of heuristics that can be used to control A*, which result in different performance characteristics:

When h(x) underestimates the true cost of reaching the goal from the current node, A* will expand more nodes, but is guaranteed to find the shortest path.
When h(x) is exactly the true cost of reaching the goal, A* will only expand nodes along the shortest path, meaning that it runs very fast and produces the optimal path.
When h(x) overestimates the true cost of reaching the goal from the current node, A* will expand fewer intermediate nodes. Depending on how much h(x) underestimates the true cost, this may result in paths that are not the true shortest path; however, this does allow the algorithm to complete more quickly.

For games, we will generally use heuristics of the third class. It is important that we generate good paths when doing pathfinding for our units, but it is generally not necessary that they be mathematically perfect; they just need to look good enough, and the speed savings are very important when we are trying to cram all of our rendering and update code into just a few tens of milliseconds, in order to hit 30-60 frames per second.

A* uses two sets to keep track of the nodes that it is operating on. The first set is the closed set, which contains all of the nodes that A* has previously considered; this is sometimes called the interior of the search. The other set is the open set, which contains those nodes which are adjacent to nodes in the closed set, but which have not yet been processed by the A* algorithm. The open set is generally sorted by the calculated cost of the node (f(x)), so that the algorithm can easily select the most promising new node to consider. Because of this, we usually consider the open list to be a priority queue. The particular implementation of this priority queue has a large impact on the speed of A*; for best performance, we need to have a data structure that supports fast membership checks (is a node in the queue?), fast removal of the best element in the queue, and fast insertions into the queue. Amit Patel provides a good overview of the pros and cons of different data structures for the priority queue on his A* page; I will be using a priority queue derived from Blue Raja’s Priority Queue class, which is essentially a binary heap. For our closed set, the primary operations that we will perform are insertions and membership tests, which makes the .Net HashSet<T> class a good choice.

Pathfinding 1: Map Representation and Preprocessing

2013-12-30T18:19:00Z

This was originally intended to be a single post on pathfinding, but it got too long, and so I am splitting it up into three or four smaller pieces. Today,we’re going to look at the data structures that we will use to represent the nodes of our pathfinding graph, and generating that graph from our terrain class.

When we were working on our quadtree to detect mouse clicks on the terrain, we introduced the concept of logical terrain tiles; these were the smallest sections of the terrain mesh that we wanted to hit when we did mouse picking, representing a 2x2 portion of our fully-tessellated mesh. These logical terrain tiles are a good starting point for generating what I am going to call our map: the 2D grid that we will use for pathfinding, placing units and other objects, defining areas of the terrain, AI calculations, and so forth. At the moment, there isn’t really anything to these tiles, as they are simply a bounding box attached to the leaf nodes of our quad tree. That’s not terribly useful by itself, so we are going to create a data structure to represent these tiles, along with an array to contain them in our Terrain class. Once we have a structure to contain our tile information, we need to extract that information from our Terrain class and heightmap, and generate the graph representing the tiles and the connections between them, so that we can use it in our pathfinding algorithm.

The pathfinding code implemented here was originally derived from Chapter 4 of Carl Granberg’s Programming an RTS Game with Direct3D . I’ve made some heavy modifications, working from that starting point, using material from Amit Patel’s blog and BlueRaja’s C# PriorityQueue implementation. The full code for this example can be found on my GitHub repository, https://github.com/ericrrichards/dx11.git, under the 33-Pathfinding project.

OutOfMemoryException - Eliminating Temporary Allocations with Static Buffers in Effect Wrapper Code

2013-12-12T18:56:00Z

I came across an interesting bug in the wrapper classes for my HLSL shader effects today. In preparation for creating a class to represent a game unit, for the purposes of demonstrating the terrain pathfinding code that I finished up last night, I had been refactoring my BasicModel and SkinnedModel classes to inherit from a common abstract base class, and after getting everything to the state that it could compile again, I had fired up the SkinnedModels example project to make sure everything was still rendering and updating correctly. I got called away to do something else, and ended up checking back in on it a half hour or so later, to find that the example had died with an OutOfMemoryException. Looking at Task Manager, this relatively small demo program was consuming over 1.5 GB of memory!

I restarted the demo, and watched the memory allocation as it ran, and noticed that the memory used seemed to be climbing quite alarmingly, 0.5-1 MB every time Task Manager updated. Somehow, I’d never noticed this before… So I started the project in Visual Studio, using the Performance Wizard to sample the .Net memory allocation, and let the demo run for a couple of minutes. Memory usage had spiked up to about 150MB, in this simple demo that loaded maybe 35 MB of textures, models, code and external libraries…

Refactoring Rendering Code out of the Terrain Class

2013-12-12T18:07:00Z

Howdy, time for an update. I’ve mostly gotten my terrain pathfinding code first cut completed; I’m creating the navigation graph, and I’ve got an implementation of A* finished that allows me to create a list of terrain nodes that represents the path between tile A and tile B. I’m going to hold off a bit on presenting all of that, since I haven’t yet managed to get a nice looking demo to show off the pathfinding yet. I need to do some more work to create a simple unit class that can follow the path generated by A*, and between work and life stuff, I haven’t gotten the chance to round that out satisfactorily yet.

I’ve also been doing some pretty heavy refactoring on various engine components, both for design and performance reasons. After the last series of posts on augmenting the Terrain class, and in anticipation of adding even more functionality as I added pathfinding support, I decided to take some time and split out the code that handles Direct3D resources and rendering from the more agnostic logical terrain representation. I’m not looking to do this at the moment, but this might also make implementing an OpenGL rendering system less painful, potentially.

Going through this, I don’t think I am done splitting things up. I’m kind of a fan of small, tightly focused classes, but I’m not necessarily an OOP junkie. Right now, I’m pretty happy with how I have split things out. I’ve got the Terrain class, which contains mostly the rendering independent logical terrain representation, such as the quad tree and picking code, the terrain heightmap and heightmap generation code, and the global terrain state properties (world-space size, initialization information struct, etc). The rendering and DirectX resource management code has been split out into the new TerrainRenderer class, which does all of the drawing and creates all of the DirectX vertex buffers and texture resources.

I’ll spare you all the intermediate gyrations that this refactoring push put me through, and just post the resulting two classes. Resharper was invaluable in this process; if you have access to a full version of Visual Studio, I don’t think there is a better way to spend $100. I shiver to think of how difficult this would have been without access to its refactoring and renaming tools.

Terrain Tile Picking

2013-12-03T13:44:00Z

Typically, in a strategy game, in addition to the triangle mesh that we use to draw the terrain, there is an underlying logical representation, usually dividing the terrain into rectangular or hexagonal tiles. This grid is generally what is used to order units around, construct buildings, select targets and so forth. To do all this, we need to be able to select locations on the terrain using the mouse, so we will need to implement terrain/mouse-ray picking for our terrain, similar to what we have done previously, with model triangle picking.

We cannot simply use the same techniques that we used earlier for our terrain, however. For one, in our previous example, we were using a brute-force linear searching technique to find the picked triangle out of all the triangles in the mesh. That worked in that case, however, the mesh that we were trying to pick only contained 1850 triangles. I have been using a terrain in these examples that, when fully tessellated, is 2049x2049 vertices, which means that our terrain consists of more than 8 million triangles. It’s pretty unlikely that we could manage to use the same brute-force technique with that many triangles, so we need to use some kind of space partitioning data structure to reduce the portion of the terrain that we need to consider for intersection.

Additionally, we cannot really perform a per-triangle intersection test in any case, since our terrain uses a dynamic LOD system. The triangles of the terrain mesh are only generated on the GPU, in the hull shader, so we don’t have access to the terrain mesh triangles on the CPU, where we will be doing our picking. Because of these two constraints, I have decide on using a quadtree of axis-aligned bounding boxes to implement picking on the terrain. Using a quad tree speeds up our intersection testing considerably, since most of the time we will be able to exclude three-fourths of our terrain from further consideration at each level of the tree. This also maps quite nicely to the concept of a grid layout for representing our terrain, and allows us to select individual terrain tiles fairly efficiently, since the bounding boxes at the terminal leaves of the tree will thus encompass a single logical terrain tile. In the screenshot below, you can see how this works; the boxes drawn in color over the terrain are at double the size of the logical terrain tiles, since I ran out of video memory drawing the terminal bounding boxes, but you can see that the red ball is located on the upper-quadrant of the white bounding box containing it.

A Terrain Minimap with SlimDX and DirectX 11

2013-11-18T16:53:00Z

Minimaps are a common feature of many different types of games, especially those in which the game world is larger than the area the player can see on screen at once. Generally, a minimap allows the player to keep track of where they are in the larger game world, and in many games, particularly strategy and simulation games where the view camera is not tied to any specific player character, allow the player to move their viewing location more quickly than by using the direct camera controls. Often, a minimap will also provide a high-level view of unit movement, building locations, fog-of-war and other game specific information.

Today, we will look at implementing a minimap that will show us a birds-eye view of the our Terrain class. We’ll also superimpose the frustum for our main rendering camera over the terrain, so that we can easily see how much of the terrain is in view. We’ll also support moving our viewpoint by clicking on the minimap. All of this functionality will be wrapped up into a class, so that we can render multiple minimaps, and place them wherever we like within our application window.

As always, the full code for this example can be downloaded from GitHub, at https://github.com/ericrrichards/dx11.git. The relevant project is the Minimap project. The implementation of this minimap code was largely inspired by Chapter 11 of Carl Granberg’s Programming an RTS Game with Direct3D, particularly the camera frustum drawing code. If you can find a copy (it appears to be out of print, and copies are going for outrageous prices on Amazon…), I would highly recommend grabbing it.

Adding Shadow-mapping and SSAO to the Terrain

2013-11-14T15:56:00Z

Now that I’m finished up with everything that I wanted to cover from Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , I want to spend some time improving the Terrain class that we introduced earlier. My ultimate goal is to create a two tiered strategy game, with a turn-based strategic level and either a turn-based or real-time tactical level. My favorite games have always been these kinds of strategic/tactical hybrids, such as (in roughly chronological order) Centurion: Defender of Rome, Lords of the Realm, Close Combat and the Total War series. In all of these games, the tactical combat is one of the main highlights of gameplay, and so the terrain that that combat occurs upon is very important, both aesthetically and for gameplay.

Or first step will be to incorporate some of the graphical improvements that we have recently implemented into our terrain rendering. We will be adding shadow-mapping and SSAO support to the terrain in this installment. In the screenshots below, we have our light source (the sun) low on the horizon behind the mountain range. The first shot shows our current Terrain rendering result, with no shadows or ambient occlusion. In the second, shadows have been added, which in addition to just showing shadows, has dulled down a lot of the odd-looking highlights in the first shot. The final shot shows both shadow-mapping and ambient occlusion applied to the terrain. The ambient occlusion adds a little more detail to the scene; regardless of it’s accuracy, I kind of like the effect, just to noise up the textures applied to the terrain, although I may tweak it a bit to lighten the darker spots up a bit.

We are going to need to add another set of effect techniques to our shader effect, to support shadow mapping, as well as a technique to draw to the shadow map, and another technique to draw the normal/depth map for SSAO. For the latter two techniques, we will need to implement a new hull shader, since I would like to have the shadow maps and normal-depth maps match the fully-tessellated geometry; using the normal hull shader that dynamically tessellates may result in shadows that change shape as you move around the map. For the normal/depth technique, we will also need to implement a new pixel shader. Our domain shader is also going to need to be updated, so that it create the texture coordinates for sampling both the shadow map and the ssao map, and our pixel shader will need to be updated to do the shadow and ambient occlusion calculations.

This sounds like a lot of work, but really, it is mostly a matter of adapting what we have already done. As always, you can download my full code for this example from GitHub at https://github.com/ericrrichards/dx11.git. This example doesn’t really have a stand-alone project, as it came about as I was on my way to implementing a minimap, and thus these techniques are showcased as part of the Minimap project.

Basic Terrain Rendering

Shadowmapping Added

Shadowmapping and SSAO

Rendering Water with Displacement Mapping

2013-11-08T12:28:00Z

Quite a while back, I presented an example that rendered water waves by computing a wave equation and updating a polygonal mesh each frame. This method produced fairly nice graphical results, but it was very CPU-intensive, and relied on updating a vertex buffer every frame, so it had relatively poor performance.

We can use displacement mapping to approximate the wave calculation and modify the geometry all on the GPU, which can be considerably faster. At a very high level, what we will do is render a polygon grid mesh, using two height/normal maps that we will scroll in different directions and at different rates. Then, for each vertex that we create using the tessellation stages, we will sample the two heightmaps, and add the sampled offsets to the vertex’s y-coordinate. Because we are scrolling the heightmaps at different rates, small peaks and valleys will appear and disappear over time, resulting in an effect that looks like waves. Using different control parameters, we can control this wave effect, and generate either a still, calm surface, like a mountain pond at first light, or big, choppy waves, like the ocean in the midst of a tempest.

This example is based off of the final exercise of Chapter 18 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . The original code that inspired this example is not located with the other example for Chapter 18, but rather in the SelectedCodeSolutions directory. You can download my source code in full from https://github.com/ericrrichards/dx11.git, under the 29-WavesDemo project. One thing to note is that you will need to have a DirectX 11 compatible video card to execute this example, as we will be using tessellation stage shaders that are only available in DirectX 11.

SSAO with SlimDX and DirectX11

2013-11-03T16:37:00Z

In real-time lighting applications, like games, we usually only calculate direct lighting, i.e. light that originates from a light source and hits an object directly. The Phong lighting model that we have been using thus far is an example of this; we only calculate the direct diffuse and specular lighting. We either ignore indirect light (light that has bounced off of other objects in the scene), or approximate it using a fixed ambient term. This is very fast to calculate, but not terribly physically accurate. Physically accurate lighting models can model these indirect light bounces, but are typically too computationally expensive to use in a real-time application, which needs to render at least 30 frames per second. However, using the ambient lighting term to approximate indirect light has some issues, as you can see in the screenshot below. This depicts our standard skull and columns scene, rendered using only ambient lighting. Because we are using a fixed ambient color, each object is rendered as a solid color, with no definition. Essentially, we are making the assumption that indirect light bounces uniformly onto all surfaces of our objects, which is often not physically accurate.

Naturally, some portions of our scene will receive more indirect light than other portions, if we were actually modeling the way that light bounces within our scene. Some portions of the scene will receive the maximum amount of indirect light, while other portions, such as the nooks and crannies of our skull, should appear darker, since fewer indirect light rays should be able to hit those surfaces because the surrounding geometry would, realistically, block those rays from reaching the surface.

In a classical global illumination scheme, we would simulate indirect light by casting rays from the object surface point in a hemispherical pattern, checking for geometry that would prevent light from reaching the point. Assuming that our models are static, this could be a viable method, provided we performed these calculations off-line; ray tracing is very expensive, since we would need to cast a large number of rays to produce an acceptable result, and performing that many intersection tests can be very expensive. With animated models, this method very quickly becomes untenable; whenever the models in the scene move, we would need to recalculate the occlusion values, which is simply too slow to do in real-time.

Screen-Space Ambient Occlusion is a fast technique for approximating ambient occlusion, developed by Crytek for the game Crysis. We will initially draw the scene to a render target, which will contain the normal and depth information for each pixel in the scene. Then, we can sample this normal/depth surface to calculate occlusion values for each pixel, which we will save to another render target. Finally, in our usual shader effect, we can sample this occlusion map to modify the ambient term in our lighting calculation. While this method is not perfectly realistic, it is very fast, and generally produces good results. As you can see in the screen shot below, using SSAO darkens up the cavities of the skull and around the bases of the columns and spheres, providing some sense of depth.

The code for this example is based on Chapter 22 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . The example presented here has been stripped down considerably to demonstrate only the SSAO effects; lighting and texturing have been disabled, and the shadow mapping effects in Luna’s example have been removed. The full code for this example can be found at my GitHub repository, https://github.com/ericrrichards/dx11.git, under the SSAODemo2 project. A more faithful adaptation of Luna’s example can also be found in the 28-SsaoDemo project.

Windows 8.1 and SlimDX

2013-11-03T16:02:00Z

This weekend, I updated my home workstation from Windows 8 to Windows 8.1. Just before doing this, I had done a bunch of work on my SSAO implementation, which I was intending to write up here once I got back from a visit home to do some deer hunting and help my parents get their firewood in. When I got back, I fired up my machine, and loaded up VS to run the SSAO sample to grab some screenshots. Immediately, my demo application crashed, while trying to create the DirectX 11 device. I had done some work over the weekend to downgrade the vertex and pixel shaders in the example to SM4, so that they could run on my laptop, which has an older integrated Intel video card that only supports DX10.1. I figured that I had borked something up in the process, so I tried running some of my other, simpler demos. Same error message popped up; DXGI_ERROR_UNSUPPORTED. Now, I am running a GTX 560 TI, so I know Direct3D 11 should be supported.

However, I have been using Nvidia’s driver update tool to keep myself at the latest and greatest driver version, so I figured that perhaps the latest driver I downloaded had some bugs. Go to Nvidia’s site, check for any updates. Looks like I have the latest driver. Hmm…

So I turned again to google, trying to find some reason why I would suddenly be unable to create a DirectX device. The fourth result I found was this: http://stackoverflow.com/questions/18082080/d3d11-create-device-debug-on-windows-8-1. Apparently I need to download the Windows 8.1 SDK, now. I’m guessing that, since I had VS installed prior to updating, I didn’t get the latest SDK installed, and the Windows 8 SDK, which I did have installed, wouldn’t cut it anymore, at least when trying to create a debug device. So I went ahead and installed the 8.1 SDK from here. Restart VS, rebuild the project in question, and now it runs perfectly. Argh. At least it’s working again; I just wish I didn’t have to waste an hour futzing around with it…

Shadow Mapping with SlimDX and DirectX 11

2013-10-28T21:48:00Z

Shadow mapping is a technique to cast shadows from arbitrary objects onto arbitrary 3D surfaces. You may recall that we implemented planar shadows earlier using the stencil buffer. Although this technique worked well for rendering shadows onto planar (flat) surfaces, this technique does not work well when we want to cast shadows onto curved or irregular surfaces, which renders it of relatively little use. Shadow mapping gets around these limitations by rendering the scene from the perspective of a light and saving the depth information into a texture called a shadow map. Then, when we are rendering our scene to the backbuffer, in the pixel shader, we determine the depth value of the pixel being rendered, relative to the light position, and compare it to a sampled value from the shadow map. If the computed value is greater than the sampled value, then the pixel being rendered is not visible from the light, and so the pixel is in shadow, and we do not compute the diffuse and specular lighting for the pixel; otherwise, we render the pixel as normal. Using a simple point sampling technique for shadow mapping results in very hard, aliased shadows: a pixel is either in shadow or lit; therefore, we will use a sampling technique known as percentage closer filtering (PCF), which uses a box filter to determine how shadowed the pixel is. This allows us to render partially shadowed pixels, which results in softer shadow edges.

This example is based on the example from Chapter 21 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . The full source for this example can be downloaded from my GitHub repository at https://github.com/ericrrichards/dx11.git, under the ShadowDemos project.

Expanding our BasicModel Class

2013-10-25T23:40:00Z

I had promised that we would move on to discussing shadows, using the shadow mapping technique. However, when I got back into the code I had written for that example, I realized that I was really sick of handling all of the geometry for our stock columns & skull scene. So I decided that, rather than manage all of the buffer creation and litter the example app with all of the buffer counts, offsets, materials and world transforms necessary to create our primitive meshes, I would take some time and extend the BasicModel class with some factory methods to create geometric models for us, and leverage the BasicModel class to encapsulate and manage all of that data. This cleans up the example code considerably, so that next time when we do look at shadow mapping, there will be a lot less noise to deal with.

The heavy lifting for these methods is already done; our GeometryGenerator class already does the work of generating the vertex and index data for these geometric meshes. All that we have left to do is massage that geometry into our BasicModel’s MeshGeometry structure, add a default material and textures, and create a Subset for the entire mesh. As the material and textures are public, we can safely initialize the model with a default material and null textures, since we can apply a different material or apply diffuse or normal maps to the model after it is created.

The full source for this example can be downloaded from my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the ShapeModels project.

Particle Systems using Stream-Out in DirectX 11 and SlimDX

2013-10-21T10:16:00Z

Particle systems are a technique commonly used to simulate chaotic phenomena, which are not easy to render using normal polygons. Some common examples include fire, smoke, rain, snow, or sparks. The particle system implementation that we are going to develop will be general enough to support many different effects; we will be using the GPU’s StreamOut stage to update our particle systems, which means that all of the physics calculations and logic to update the particles will reside in our shader code, so that by substituting different shaders, we can achieve different effects using our base particle system implementation.

The code for this example was adapted from Chapter 20 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , ported to C# and SlimDX. The full source for the example can be found at my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the ParticlesDemo project.

Below, you can see the results of adding two particles systems to our terrain demo. At the center of the screen, we have a flame particle effect, along with a rain particle effect.

Skinned Models in DirectX 11 with SlimDX and Assimp.Net

2013-10-15T15:45:00Z

Sorry for the hiatus, I’ve been very busy with work and life the last couple weeks. Today, we’re going to look at loading meshes with skeletal animations in DirectX 11, using SlimDX and Assimp.Net in C#. This will probably be our most complicated example yet, so bear with me. This example is inspired by Chapter 25 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , although with some heavy modifications. Mr. Luna’s code uses a custom animation format, which I found less than totally useful; realistically, we would want to be able to load skinned meshes exported in one of the commonly used 3D modeling formats. To facilitate this, we will again use the .NET port of the Assimp library, Assimp.Net. The code I am using to load and interpret the animation and bone data is heavily based on Scott Lee’s Animation Importer code, ported to C#. The full source for this example can be found on my GitHub repository, at https://github.com/ericrrichards/dx11.git under the SkinnedModels project. The meshes used in the example are taken from the example code for Carl Granberg’s Programming an RTS Game with Direct3D .

Skeletal animation is the standard way to animate 3D character models. Generally, a character model will be represented by two structures: the exterior vertex mesh, or skin, and a tree of control points specifying the joints or bones that make up the skeleton of the mesh. Each vertex in the skin is associated with one or more bones, along with a weight that determines how much influence the bone should have on the final position of the skin vertex. Each bone is represented by a transformation matrix specifying the translation, rotation and scale that determines the final position of the bone. The bones are defined in a hierarchy, so that each bone’s transformation is specified relative to its parent bone. Thus, given a standard bipedal skeleton, if we rotate the upper arm bone of the model, this rotation will propagate to the lower arm and hand bones of the model, analogously to how our actual joints and bones work.

Animations are defined by a series of keyframes, each of which specifies the transformation of each bone in the skeleton at a given time. To get the appropriate transformation at a given time t, we linearly interpolate between the two closest keyframes. Because of this, we will typically store the bone transformations in a decomposed form, specifying the translation, scale and rotation components separately, building the transformation matrix at a given time from the interpolated components. A skinned model may contain many different animation sets; for instance, we’ll commonly have a walk animation, and attack animation, an idle animation, and a death animation.

The process of loading an animated mesh can be summarized as follows:

Extract the bone hierarchy of the model skeleton.
Extract the animations from the model, along with all bone keyframes for each animation.
Extract the skin vertex data, along with the vertex bone indices and weights.
Extract the model materials and textures.

To draw the skinned model, we need to advance the animation to the correct frame, then pass the bone transforms to our vertex shader, where we will use the vertex indices and weights to transform the vertex position to the proper location.

Loading 3D Models using Assimp.Net and SlimDX

2013-10-03T10:48:00Z

So far, we have either worked with procedurally generated meshes, like our boxes and cylinders, or loaded very simple text-based mesh formats. For any kind of real application, however, we will need to have the capability to load meshes created by artists using 3D modeling and animation programs, like Blender or 3DS Max. There are a number of potential solutions to this problem; we could write an importer to read the specific file format of the program we are most likely to use for our engine. This does present a new host of problems, however: unless we write another importer, we are limited to just using models created with that modeling software, or we will have to rely on or create plugins to our modeling software to reformat models created using different software. There is a myriad of different 3D model formats in more-or-less common use, some of which are open-source, some of which are proprietary; many of both are surprisingly hard to find good, authoritative documentation on. All this makes the prospects of creating a general-purpose, cross-format model importer a daunting task.

Fortunately, there exists an open-source library with support for loading most of the commonly used model formats, ASSIMP, or Open Asset Import Library. Although Assimp is written in C++, there exists a port for .NET, AssimpNet, which we will be able to use directly in our code, without the hassle of wrapping the native Assimp library ourselves. I am not a lawyer, but it looks like both the Assimp and AssimpNet have very permissive licenses that should allow one to use them in any kind of hobby or professional project.

While we can use Assimp to load our model data, we will need to create our own C# model class to manage and render the model. For that, I will be following the example of Chapter 23 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . This example will not be a straight conversion of his example code, since I will be ditching the m3d model format which he uses, and instead loading models from the standard old Microsoft DirectX X format. The models I will be using come from the example code for Chapter 4 of Carl Granberg’s Programming an RTS Game with Direct3D , although you may use any of the supported Assimp model formats if you want to use other meshes instead. The full source for this example can be found on my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the AssimpModel project.

Generating Random Terrains using Perlin Noise

2013-10-01T10:24:00Z

Previously, we have used our Terrain class solely with heightmaps that we have loaded from a file. Now, we are going to extend our Terrain class to generate random heightmaps as well, which will add variety to our examples. We will be using Perlin Noise, which is a method of generating naturalistic pseudo-random textures developed by Ken Perlin. One of the advantages of using Perlin noise is that its output is deterministic; for a given set of control parameters, we will always generate the same noise texture. This is desirable for our terrain generation algorithm because it allows us to recreate the same heightmap given the same initial seed value and control parameters.

Because we will be generating random heightmaps for our terrain, we will also need to generate an appropriate blendmap to texture the resulting terrain. We will be using a simple method that assigns the different terrain textures based on the heightmap values; a more complex simulation might model the effects of weather, temperature and moisture to assign diffuse textures, but simply using the elevation works quite well with the texture set that we are using.

The code for this example is heavily influenced by Chapter 4 of Carl Granberg’s Programming an RTS Game with Direct3D , adapted into C# and taking advantage of some performance improvements that multi-core CPUs on modern computers offer us. The full code for this example can be found on my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the RandomTerrainDemo project. In addition to the random terrain generation code, we will also look at how we can add Windows Forms controls to our application, which we will use to specify the parameters to use to create the random terrain.

Drawing a Loading Screen with Direct2D in SlimDX

2013-09-30T12:54:00Z

I know that I have been saying that I will cover random terrain generation in my next post for several posts now, and if all goes well, that post will be published today or tomorrow. First, though, we will talk about Direct2D, and using it to draw a loading screen, which we will display while the terrain is being generated to give some indication of the progress of our terrain generation algorithm.

Direct2D is a new 2D rendering API from Microsoft built on top of DirectX 10 (or DirectX 11 in Windows 8). It provides functionality for rendering bitmaps, vector graphics and text in screen-space using hardware acceleration. It is positioned as a successor to the GDI rendering interface, and, so far as I can tell, the old DirectDraw API from earlier versions of DirectX. According to the documentation, it should be possible to use Direct2D to render 2D elements to a Direct3D 11 render target, although I have had some difficulties in actually getting that use-case to work. It does appear to work excellently for pure 2D rendering, say for menu or loading screens, with a simpler syntax than using Direct3D with an orthographic projection.

We will be adding Direct2D support to our base application class D3DApp, and use Direct2D to render a progress bar with some status text during our initialization stage, as we load and generate the Direct3D resources for our scene. Please note that the implementation presented here should only be used while there is no active Direct3D rendering occurring; due to my difficulties in getting Direct2D/Direct3D interoperation to work correctly, Direct2D will be using a different backbuffer than Direct3D, so interleaving Direct2D drawing with Direct3D rendering will result in some very noticeable flickering when the backbuffers switch.

The inspiration for this example comes from Chapter 5 of Carl Granberg’s Programming an RTS Game with Direct3D , where a similar Progress screen is implemented using DirectX9 and the fixed function pipeline. With the removal of the ID3DXFont interface from newer versions of DirectX, as well as the lack of the ability to clear just a portion of a DirectX11 DeviceContext’s render target, a straight conversion of that code would require some fairly heavy lifting to implement in Direct3D 11, and so we will be using Direct2D instead. The full code for this example can be found on my GitHub repository, at https://github.com/ericrrichards/dx11.git, and is implemented in the TerrainDemo and RandomTerrainDemo projects.

Terrain LOD for DirectX 10 Graphics Cards, using SlimDX and Direct3D 11

2013-09-26T17:05:00Z

Last time, we discussed terrain rendering, using the tessellation stages of the GPU to render the terrain mesh with distance-based LOD. That method required a DX11-compliant graphics card, since the Hull and Domain shader stages are new to Direct3D11. According to the latest Steam Hardware survey, nearly 65% of gamers have a DX11 graphics card, which is certainly the majority of potential users, and only likely to increase in the future. Of the remaining 35% of gamers, 31% are still using DX10 graphics cards. While we can safely ignore the small percentage of the market that is still limping along on DX9 cards (I myself still have an old laptop with a GeForce Go 7400 in my oldest laptop, but that machine is seven years old and on its last legs), restricting ourselves to only DX 11 cards cuts out a third of potential users of your application. For that reason, I’m going to cover an alternative, CPU-based implementation of our previous LOD terrain rendering example. If you have the option, I would suggest that you only bother with the previous DX11 method, as tessellating the terrain mesh yourself on the CPU is relatively more complex, prone to error, less performant, and produces a somewhat lower quality result; if you must support DX10 graphics cards, however, this method or one similar to it will do the job, while the hull/domain shader method will not.

We will be implementing this rendering method as an additional render path in our Terrain class, if we detect that the user has a DX10 compatible graphics card. This allows us to reuse a large chunk of the previous code. For the rest, we will adapt portions of the HLSL shader code that we previously implemented into C#, as well as use some inspirations from Chapter 4 of Carl Granberg’s Programming an RTS Game with Direct3D . The full code for this example can be found at my GitHub repository, https://github.com/ericrrichards/dx11.git, under the TerrainDemo project.

Dynamic Terrain Rendering with SlimDX and Direct3D 11

2013-09-21T19:52:00Z

A common task for strategy and other games with an outdoor setting is the rendering of the terrain for the level. Probably the most convenient way to model a terrain is to create a triangular grid, and then perturb the y-coordinates of the vertices to match the desired elevations. This elevation data can be determined by using a mathematical function, as we have done in our previous examples, or by sampling an array or texture known as a heightmap. Using a heightmap to describe the terrain elevations allows us more fine-grain control over the details of our terrain, and also allows us to define the terrain easily, either using a procedural method to create random heightmaps, or by creating an image in a paint program.

Because a terrain can be very large, we will want to optimize the rendering of it as much as possible. One easy way to save rendering cycles is to only draw the vertices of the terrain that can be seen by the player, using frustum culling techniques similar to those we have already covered. Another way is to render the mesh using a variable level of detail, by using the Hull and Domain shaders to render the terrain mesh with more polygons near the camera, and fewer in the distance, in a manner similar to that we used for our Displacement mapping effect. Combining the two techniques allows us to render a very large terrain, with a very high level of detail, at a high frame rate, although it does limit us to running on DirectX 11 compliant graphics cards.

We will also use a technique called texture splatting to render our terrain with multiple textures in a single rendering call. This technique involves using a separate texture, called a blend map, in addition to the diffuse textures that are applied to the mesh, in order to define which texture is applied to which portion of the mesh.

The code for this example was adapted from Chapter 19 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , with some additional inspirations from Chapter 4 of Carl Granberg’s Programming an RTS Game with Direct3D . The full source for this example can be downloaded from my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the TerrainDemo project.

Diving into the Tessellation Stages of Direct3D 11

2013-09-16T14:49:00Z

In our last example on normal mapping and displacement mapping, we made use of the new Direct3D 11 tessellation stages when implementing our displacement mapping effect. For the purposes of the example, we did not examine too closely the concepts involved in making use of these new features, namely the Hull and Domain shaders. These new shader types are sufficiently complicated that they deserve a separate treatment of their own, particularly since we will continue to make use of them for more complicated effects in the future.

The Hull and Domain shaders are covered in Chapter 13 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , which I had previously skipped over. Rather than use the example from that chapter, I am going to use the shader effect we developed for our last example instead, so that we can dive into the details of how the hull and domain shaders work in the context of a useful example that we have some background with.

The primary motivation for using the tessellation stages is to offload work from the the CPU and main memory onto the GPU. We have already looked at a couple of the benefits of this technique in our previous post, but some of the advantages of using the tessellation stages are:

We can use a lower detail mesh, and specify additional detail using less memory-intensive techniques, like the displacement mapping technique presented earlier, to produce the final, high-quality mesh that is displayed.
We can adjust the level of detail of a mesh on-the-fly, depending on the distance of the mesh from the camera or other criteria that we define.
We can perform expensive calculations, like collisions and physics calculations, on the simplified mesh stored in main memory, and still render the highly-detailed generated mesh.

Bump and Displacement Mapping with SlimDX and Direct3D 11

2013-09-16T06:37:00Z

Today, we are going to cover a couple of additional techniques that we can use to achieve more realistic lighting in our 3D scenes. Going back to our first discussion of lighting, recall that thus far, we have been using per-pixel, Phong lighting. This style of lighting was an improvement upon the earlier method of Gourad lighting, by interpolating the vertex normals over the resulting surface pixels, and calculating the color of an object per-pixel, rather than per-vertex. Generally, the Phong model gives us good results, but it is limited, in that we can only specify the normals to be interpolated from at the vertices. For objects that should appear smooth, this is sufficient to give realistic-looking lighting; for surfaces that have more uneven textures applied to them, the illusion can break down, since the specular highlights computed from the interpolated normals will not match up with the apparent topology of the surface.

In the screenshot above, you can see that the highlights on the nearest column are very smooth, and match the geometry of the cylinder. However, the column has a texture applied that makes it appear to be constructed out of stone blocks, jointed with mortar. In real life, such a material would have all kinds of nooks and crannies and deformities that would affect the way light hits the surface and create much more irregular highlights than in the image above. Ideally, we would want to model those surface details in our scene, for the greatest realism. This is the motivation for the techniques we are going to discuss today.

One technique to improve the lighting of textured objects is called bump or normal mapping. Instead of just using the interpolated pixel normal, we will combine it with a normal sampled from a special texture, called a normal map, which allows us to match the per-pixel normal to the perceived surface texture, and achieve more believable lighting.

The other technique is called displacement mapping. Similarly, we use an additional texture to specify the per-texel surface details, but this time, rather than a surface normal, the texture, called a displacement map or heightmap, stores an offset that indicates how much the texel sticks out or is sunken in from its base position. We use this offset to modify the position of the vertices of an object along the vertex normal. For best results, we can increase the tessellation of the mesh using a domain shader, so that the vertex resolution of our mesh is as great as the resolution of our heightmap. Displacement mapping is often combined with normal mapping, for the highest level of realism.


The scene with normal mapping enabled. Note the highlights are much less regular.	The scene with displacement mapping enabled. Note that the mesh geometry is much more detailed, and the sides of the column are no longer smooth.

This example is based off of Chapter 18 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . You can download the full source for this example from my GitHub repository, athttps://github.com/ericrrichards/dx11.git, under the NormalDisplacementMaps project.

NOTE: You will need to have a DirectX 11 compatible video card in order to use the displacement mapping method presented here, as it makes use of the Domain and Hull shaders, which are new to DX 11.

Dynamic Environmental Reflections in Direct3D 11 and C#

2013-09-11T10:45:00Z

Last time, we looked at using cube maps to render a skybox around our 3D scenes, and also how to use that sky cubemap to render some environmental reflections onto our scene objects. While this method of rendering reflections is relatively cheap, performance-wise and can give an additional touch of realism to background geometry, it has some serious limitations if you look at it too closely. For one, none of our local scene geometry is captured in the sky cubemap, so, for instance, you can look at our reflective skull in the center and see the reflections of the distant mountains, which should be occluded by the surrounding columns. This deficiency can be overlooked for minor details, or for surfaces with low reflectivity, but it really sticks out if you have a large, highly reflective surface. Additionally, because we are using the same cubemap for all objects, the reflections at any object in our scene are not totally accurate, as our cubemap sampling technique does not differentiate on the position of the environment mapped object in the scene.

The solution to these issues is to render a cube map, at runtime, for each reflective object using Direct3D. By rendering the cubemap for each object on the fly, we can incorporate all of the visible scene details, (characters, geometry, particle effects, etc) in the reflection, which looks much more realistic. This is, of course, at the cost of the additional overhead involved in rendering these additional cubemaps each frame, as we have to effectively render the whole scene six times for each object that requires dynamic reflections.

This example is based on the second portion of Chapter 17 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , with the code ported to C# and SlimDX from the native C++ used in the original example. You can download the full source for this example from my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the DynamicCubeMap project.

Skyboxes and Environmental Reflections using Cube Maps in Direct3D 11 and SlimDX

2013-09-08T12:52:00Z

This time, we are going to take a look at a special class of texture, the cube map, and a couple of the common applications for cube maps, skyboxes and environment-mapped reflections. Skyboxes allow us to model far away details, like the sky or distant scenery, to create a sense that the world is more expansive than just our scene geometry, in an inexpensive way. Environment-mapped reflections allow us to model reflections on surfaces that are irregular or curved, rather than on flat, planar surfaces as in our Mirror Demo.

The code for this example is adapted from the first part of Chapter 17 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0. You can download the full source for this example from my GitHub repository, https://github.com/ericrrichards/dx11.git, under the CubeMap project.

Our skull & columns scene, with a skybox and environment-mapped reflections on the column tops

Status Update

2013-09-08T07:02:00Z

Hi everybody. This past week has been a rough one for me, as far as getting out any additional tutorials. Between helping my future sister-in-law move into a place in Cambridge, a deep sea fishing trip, a year-long project at work finally coming to a critical stage, and the release of Rome Total War II, it's been hard for me to find the hours to hammer out an article. Not to fear, next week should see an explosion of new content, as I am way ahead on the coding side. Expect articles on skyboxes and reflections using cube maps, normal and displacement mapping, terrain rendering with texture splatting and constant LOD, and loading meshes in commercial formats using Assimp.Net. Here are some teaser screenshots...

Using cube maps to render a skybox and reflections:

Real-time reflections using a dynamic cube map:

Normal and displacement mapping (click for full-size):


Flat lighting	Normal mapped lighting	Normal mapped lighting and displacement mapped geometry

Terrain rendering:


	Wire-frame mode, note the far hills are made up of far fewer polygons than the nearer details.

Loading meshes with commercial file formats using Assimp.Net:

Camera Picking in SlimDX and Direct3D 11

2013-08-30T15:18:00Z

So far, we have only been concerned with drawing a 3D scene to the 2D computer screen, by projecting the 3D positions of objects to the 2D pixels of the screen. Often, you will want to perform the reverse operation; given a pixel on the screen, which object in the 3D scene corresponds to that pixel? Probably the most common application for this kind of transformation is selecting and moving objects in the scene using the mouse, as in most modern real-time strategy games, although the concept has other applications.

The traditional method of performing this kind of object picking relies on a technique called ray-casting. We shoot a ray from the camera position through the selected point on the near-plane of our view frustum, which is obtained by converting the screen pixel location into normalized device coordinates, and then intersect the resulting ray with each object in our scene. The first object intersected by the ray is the object that is “picked.”

The code for this example is based on Chapter 16 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , with some modifications. You can download the full source from my GitHub repository at https://github.com/ericrrichards/dx11.git, under the PickingDemo project.

A Look-At Camera in SlimDX and Direct3D 11

2013-08-29T12:15:00Z

Today, we are going to reprise our Camera class from the Camera Demo. In addition to the FPS-style camera that we have already implemented, we will create a Look-At camera, a camera that remains focused on a point and pans around its target. This camera will be similar to the very basic camera we implemented for our initial examples (see the Colored Cube Demo). While our FPS camera is ideal for first-person type views, the Look-At camera can be used for third-person views, or the “birds-eye” view common in city-builder and strategy games. As part of this process, we will abstract out the common functionality that all cameras will share from our FPS camera into an abstract base camera class.

The inspiration for this example come both from Mr. Luna’s Camera Demo (Chapter 14 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 ), and the camera implemented in Chapter 5 of Carl Granberg’s Programming an RTS Game with Direct3D . You can download the full source for this example from my GitHub repository at https://github.com/ericrrichards/dx11.git under the CameraDemo project. To switch between the FPS and Look-At camera, use the F(ps) and L(ook-at) keys on your keyboard.

Hardware Instancing and Frustum Culling using SlimDX and Direct3D 11

2013-08-28T16:58:00Z

One of the main bottlenecks to the speed of a Direct3D application is the number of Draw calls that are issued to the GPU, along with the overhead of switching shader constants for each object that is drawn. Today, we are going to look at two methods of optimizing our drawing code. Hardware instancing allows us to minimize the overhead of drawing identical geometry in our scene, by batching the draw calls for our objects and utilizing per-instance data to avoid the overhead in uploading our per-object world matrices. Frustum culling enables us to determine which objects will be seen by our camera, and to skip the Draw calls for objects that will be clipped by the GPU during projection. Together, the two techniques reap a significant increase in frame rate.

The source code for this example was adapted from the InstancingAndCulling demo from Chapter 15 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . Additionally, the frustum culling code for this example was adapted from Chapter 5 of Carl Granberg’s Programming an RTS Game with Direct3D (Luna’s implementation of frustum culling relied heavily on xnacollision.h, which isn’t really included in the base SlimDX). You can download the full source for this example from my GitHub repository at https://github.com/ericrrichards/dx11.git under the InstancingAndCulling project.

An FPS Camera in SlimDX

2013-08-21T10:38:00Z

Up until now, we have been using a fixed, orbiting camera to view our demo applications. This style of camera works adequately for our purposes, but for a real game project, you would probably want a more flexible type of camera implementation. Additionally, thus far we have been including our camera-specific code directly in our main application classes, which, again, works, but does not scale well to a real game application. Therefore, we will be splitting out our camera-related code out into a new class (Camera.cs) that we will add to our Core library. This example maps to the CameraDemo example from Chapter 14 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . The full code for this example can be downloaded from my GitHub repository, https://github.com/ericrrichards/dx11.git, under the CameraDemo project.

We will be implementing a traditional First-Person style camera, as one sees in many FPS’s and RPG games. Conceptually, we can think of this style of camera as consisting of a position in our 3D world, typically located as the position of the eyes of the player character, along with a vector frame-of-reference, which defines the direction the character is looking. In most cases, this camera is constrained to only rotate about its X and Y axes, thus we can pitch up and down, or yaw left and right. For some applications, such as a space or aircraft simulation, you would also want to support rotation on the Z (roll) axis. Our camera will support two degrees of motion; back and forward in the direction of our camera’s local Z (Look) axis, and left and right strafing along our local X (Right) axis. Depending on your game type, you might also want to implement methods to move the camera up and down on its local Y axis, for instance for jumping, or climbing ladders. For now, we are not going to implement any collision detection with our 3D objects; our camera will operate very similarly to the Half-Life or Quake camera when using the noclip cheat.

Our camera class will additionally manage its view and projection matrices, as well as storing information that we can use to extract the view frustum. Below is a screenshot of our scene rendered using the viewpoint of our Camera class (This scene is the same as our scene from the LitSkull Demo, with textures applied to the shapes).

Geometry Shader Billboards with SlimDX and DirectX 11

2013-08-16T13:33:00Z

When I first learned about programming DirectX using shaders, it was back when DirectX 9 was the newest thing around. Back then, there were only two stages in the shader pipeline, the Vertex and Pixel shaders that we have been utilizing thus far. DirectX 10 introduced the geometry shader, which allows us to modify entire geometric primitives on the hardware, after they have gone through the vertex shader.

One application of this capability is rendering billboards. Billboarding is a common technique for rendering far-off objects or minor scene details, by replacing a full 3D object with a texture drawn to a quad that is oriented towards the viewer. This is much less performance-intensive, and for far-off objects and minor details, provides a good-enough approximation. As an example, many games use billboarding to render grass or other foliage, and the Total War series renders far-away units as billboard sprites (In Medieval Total War II, you can see this by zooming in and out on a unit; at a certain point, you’ll see the unit “pop”, which is the point where the Total War engine switches from rendering sprite billboards to rendering the full 3D model). The older way of rendering billboards required one to maintain a dynamic vertex buffer of the quads for the billboards, and to transform the vertices to orient towards the viewer on the CPU whenever the camera moved. Dynamic vertex buffers have a lot of overhead, because it is necessary to re-upload the geometry to the GPU every time it changes, along with the additional overhead of uploading four vertices per billboard. Using the geometry shader, we can use a static vertex buffer of 3D points, with only a single vertex per billboard, and expand the point to a camera-aligned quad in the geometry shader.

We’ll illustrate this technique by porting the TreeBillboard example from Chapter 11 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0. This demo builds upon our previous Alpha-blending example, adding some tree billboards to the scene. You can download the full code for this example from my GitHub repository, at https://github.com/ericrrichards/dx11.git under the TreeBillboardDemo project.

Planar Reflections and Shadows using the Stencil Buffer in SlimDX and Direct3D 11

2013-08-11T19:40:00Z

In this post, we are going to discuss applications of the Direct3D stencil buffer, by porting the example from Chapter 10 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 to C# and SlimDX. We will create a simple scene, consisting of an object (in our case, the skull mesh that we have used previously), and some simple room geometry, including a section which will act as a mirror and reflect the rest of the geometry in our scene. We will also implement planar shadows, so that our central object will cast shadows on the rest of our geometry when it is blocking our primary directional light. The full code for this example can be downloaded from my GitHub repository, at https://github.com/ericrrichards/dx11.git, under the MirrorDemo project.

Alpha-Blending Demo

2013-08-05T15:39:00Z

Last time, we covered some of the theory that underlies blending and distance fog. This time, we’ll go over the implementation of our demo that uses these effects, the BlendDemo. This will be based off of our previous demo, the Textured Hills Demo, with an added box mesh and a wire fence texture applied to demonstrate pixel clipping using an alpha map. We’ll need to update our Basic.fx shader code to add support for blending and clipping, as well as the fog effect, and we’ll need to define some new render states to define our blending operations. You can find the full code for this example at https://github.com/ericrrichards/dx11.git under the BlendDemo project.

Blending Theory

2013-08-02T14:04:00Z

This time around, we are going to dig into Chapter 9 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . We will be implementing the BlendDemo in the next couple of posts. We will base this demo off our previous example, the Textured Hills Demo. We will be adding transparency to our water mesh, a fog effect, and a box with a wire texture, with transparency implemented using pixel shader clipping. We’ll need to update our shader effect, Basic.fx, to support these additional effects, along with its C# wrapper class. We’ll also look at additional DirectX blend and rasterizer states, and build up a static class to manage our these for us, in the same fashion we used for our InputLayouts and Effects. The full code for this example can be found on GitHub at https://github.com/ericrrichards/dx11.git, under the BlendDemo project. Before we can get to the demo code, we need to cover some of the basics first, however.

Using a Texture Atlas for Animation

2013-08-01T21:11:00Z

We’re going to wrap up our exploration of Chapter 8 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 by implementing one of the exercises from the end of the chapter. This exercise asks us to render a cube, similar to our Crate Demo, but this time to show a succession of different textures, in order to create an animation, similar to a child’s flip book. Mr. Luna suggests that we simply load an array of separate textures and swap them based on our simulation time, but we are going to go one step beyond, and implement a texture atlas, so that we will have all of the frames of animation composited into a single texture, and we can select the individual frames by changing our texture coordinate transform matrix. We’ll wrap this functionality up into a little utility class that we can then reuse.

Textured Hills Demo

2013-07-31T16:02:00Z

This time around, we are going to revisit our old friend, the Waves Demo, and add textures to the land and water meshes. We will also be taking advantage of the gTexTransform matrix of our Basic.fx shader to tile our land texture multiple times across our mesh, to achieve more detail, and use tiling and translations on on our water mesh texture to create a simple but very visually appealing animation for our waves. This demo corresponds to the TexturedHillsAndWaves demo from Chapter 8 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . You can download the full source for this example from my GitHub repository at https://github.com/ericrrichards/dx11.git.

Here is a still of our finished project this time:

I’m also going to try to upload a video of this demo in action, as the static screenshot doesn’t quite do justice:

<\/param><\/embed><\/object><\/div>";' alt>

Texturing 101–Crate Demo

2013-07-29T07:00:00Z

This time around, we are going to begin with a simple texturing example. We’ll draw a simple cube, and apply a crate-style texture to it. We’ll need to make some changes to our Basic.fx shader code, as well as the C# wrapper class, BasicEffect. Lastly, we’ll need to create a new vertex structure, which will contain, in addition to the position and normal information we have been using, a uv texture coordinate. If you are following along in Mr. Luna’s book , this would be Chapter 8, the Crate Demo. You can see my full code for the demo at https://github.com/ericrrichards/dx11.git, under DX11/CrateDemo.

Three-point Lighting – LitSkull Demo

2013-07-25T12:53:00Z

This time, we are going to take the scene that we used for the Shapes Demo, and apply a three-point lighting shader. We’ll replace the central sphere from the scene with the skull model that we loaded from a file in the Skull Demo, to make the scene a little more interesting. We will also do some work encapsulating our shader in a C# class, as we will be using this shader effect as a basis that we will extend when we look at texturing, blending and other effects. As always, the full code for this example can be found at my Github repository https://github.com/ericrrichards/dx11.git; the project for this example can be found in the DX11 solution under Examples/LitSkull.


Rendering the LitSkull scene with 1 light (key lit)	Rendering the LitSkull scene with 2 lights (key and fill lit)	Rendering the LitSkull scene with 3 lights (key, fill and back lit)

Lit Terrain Demo

2013-07-24T10:57:00Z

Last time, we ran through a whirlwind tour of lighting and material theory, and built up our data structures and shader functions for our lighting operations. This time around, we’ll get to actually implementing our first lighting demo. We’re going to take the Waves Demo that we covered previously and light it with a directional light, a point light that will orbit the scene, and a spot light that will focus on our camera’s view point. On to the code!

Lighting, Take 1

2013-07-23T19:32:00Z

Last time, we finished up our exploration of the examples from Chapter 6 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 . This time, and for the next week or so, we’re going to be adding lighting to our repertoire. There are only two demo applications on tap in this chapter, but they are comparatively meatier, so I may take a couple posts to describe each one.

First off, we are going to modify our previous Waves Demo to render using per-pixel lighting, as opposed to the flat color we used previously. Even without any more advanced techniques, this gives us a much prettier visual. To get there, we are going to have to do a lot of legwork to implement this more realistic lighting model. We will need to implement three different varieties of lights, a material class, and a much more advanced shader effect. Along the way, we will run into a couple interesting and slightly aggravating differences between the way one would do things in C++ and native DirectX and how we need to implement the same ideas in C# with SlimDX. I would suggest that you follow along with my code from https://github.com/ericrrichards/dx11.git. You will want to examine the Examples/LightingDemo and Core projects from the solution.

Dynamic Vertex Buffers: Waves Demo

2013-07-22T09:54:00Z

We’ll wrap up the slate of examples from Chapter 6 of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 by taking a look at updating a vertex buffer each frame, as opposed to the immutable vertex buffers we have been working with previously. We will be using the Hills Demo that we created previously as a starting point, and add a second grid mesh that will represent water. We will update this mesh each frame, to get the ripple effect shown below.

Loading a Mesh From a File: Skull Demo

2013-07-20T12:42:00Z

This time around, we are going to be loading a pre-defined mesh from a simple text-file format. This will be a rather quick post, as the only thing different that we will be doing here is altering our CreateGeometryBuffer function to load the vertex and index data from a file, rather than creating it on-the-fly. If you are following along with me in Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 , this example corresponds to section 6.12, and is a port of the Skull Demo from the example code.

Cylinders, Spheres, and Boxes with Direct3D11 and SlimDX

2013-07-19T11:42:00Z

This time up, we are going to add some additional shape types to our GeometryGenerator class, and look at how to redraw the same geometry at different locations and scales in our scene. This example corresponds to the ShapesDemo from Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0.

Hills Demo with SlimDX and C#

2013-07-16T11:38:00Z

This time around, we are going to be porting the Hills Demo of Frank Luna’s Introduction to 3D Game Programming with Direct3D 11.0 from C++ to C# and SlimDX.

A Colored Cube in DirectX 11 and SlimDX

2013-07-10T12:08:00Z

Now that we have the demo framework constructed, we can move on to a more interesting example. This time, we are going to use the demo framework that we just developed to render a colored cube in DirectX 11, using a simple vertex and pixel shader. This example corresponds with the BoxDemo from Chapter 6 of Frank Luna’s Introduction to Game Programming with Direct3D 11.0 .

DirectX 11 Initialization with SlimDX and C#

2013-07-08T09:35:00Z

I’m going to skip over the first few chapters of Introduction to 3D Game Programming with Direct3D 11.0. I’ve got a basic understanding of basic linear algebra for 3D graphics, and I’m not eager to get into the weeds there, instead opting to go straight into the meat of DirectX coding. Besides the math pre-requisites, these chapters also give an overview of the XNA Math C++ library, which I am going to forgo for the SlimDX types instead. With that out of the way, we will begin with Chapter 4, Direct3D Initialization. This chapter covers basic initialization of the Direct3D device, and lays out the application framework used for the rest of the book. If you’ve read any of Mr. Luna’s other DirectX books, this framework will look very familiar, and even if you have not, it is basically a canonical example of a Windows game loop implementation. The framework provides a base class that handles creating a window, initializing DirectX, and provides some virtual methods that can be overridden in derived classes for application-specific logic. Additionally, this chapter introduces a timer class, which allows for updating the game world based on the elapsed time per frame, and for determining the frame rate, using the high-performance system timer.

Learning DirectX 11 via SlimDX

2013-07-04T05:54:00Z

Personal things...

A while back I picked up Frank Luna's Introduction to 3D Game Programming with DirectX 11. I've been meaning to start going through it for some time, but some personal things have cut into my programming time significantly. I recently acquired a two year old Australian Cattle dog, and Dixie has kept me on the dead run for the weeks... It's very difficult to muster up the courage to dive into a programming project when you feel half zombified from getting up at 4:30 to let the dog out so she will stop her adorable yet frenzied licking attacks on your fiance.

But today, I'm going to start. I have not dealt with DirectX 11, nor 10 before; it's only in the last year or so that I built myself a modern desktop to replace my badly aging laptop from 2006. I do have some experience playing around with DirectX 9, so it will be interesting to see what has changed.

As I mentioned, I'll be going through Introduction to 3D Game Programming with DirectX 11 by Frank D. Luna. Atfirst glance, it seems very similar to his previous book, Introduction to 3D Game Programming with Direct X 9.0c: A Shader Approach , which I found to be very well-written, with easy to follow code examples. One wrinkle is that I'm going to be doing this in C#, with SlimDX, rather than in C++ and straight DirectX 11. At my job, I mostly write C#, and have developed a serious fondness for it. Additionally, I've found that attempting to port C++ code over to C# is a somewhat interesting exercise, and not a bad way to learn some of the darker and less-common corners of the C# standard libraries.

I plan on doing a post on each major example. So there may be a couple per chapter, and I don't know what kind of pace my schedule will allow me. If you're interested at looking at my code as I'm going along, you can clone my Git repository at https://github.com/ericrrichards/dx11.git. I'm not going to include the original book code (I couldn't find a license specifically for the code, and I'm struggling to parse the legalese on the inside cover of the book), but you can download it from the book's website at http://www.d3dcoder.net/d3d11.htm.

Javascript Needs Default Parameters

2012-11-13T09:54:00Z

Lately, I've been doing a lot of web applications, with C# on the server and ASP MVC/Razor/Javascript clients. Javascript, especially with jQuery, is really wonderful for manipulating html, but there are a number of things that I've discovered that are unpleasant. One of the most irritating to me, thus far, is the lack of syntactic sugar for defining default values for parameters passed to functions. In other languages that don't support default parameter values, you can often get around this through function overloading - by first defining a function with all the possible arguments, then defining functions with a subset of those arguments, and then calling the ur-function with the missing arguments substituted with default values.

This example is unnecessary in C#, since the language supports the following:

There are a couple of ways I've found to do this in Javascript, neither of which is particularly appealing:

Now, it may be possible to use the C# style syntax if one uses CoffeeScript or TypeScript; I haven't done enough research to know. But it would be wonderful if javascript supported that syntax natively, just to reduce the amount of boiler-plate that you have to use. Boilerplate is almost always bad; it introduces the possibility of simple mistakes.

Calculating Distance in a Hexagonal Grid

2012-02-27T17:03:00Z

Lately, I've been working on a 2-D, hex-based strategy game, in the vein of the old classic, Civil War Generals 2 . Hex grids are a staple of wargames, like the old 5 Star SSI games (Panzer General and all of its descendants), although they've also turned up in some newer games, like Civilization 5 and Battle for Wesnoth.

Although they've been around forever, resources on coding hexmaps are somewhat scanty, in comparison to other things you might want to do with hobby game dev. A list of some canonical articles:

Naturally, a lot of the same algorithms you'd use with square grids are equally valid on hex grids. However, often times, those algorithms require some tweaking to work correctly with a hexagonal grid, and its not always easy to understand how they need to be tweaked without a good deal of iteration.

One thing that I found particularly difficult was understanding the correct way to compute distances in a hex grid. Amit's site has some valuable information, but, as much of it is really old, featuring archived newsgroup posts with ascii art and code written in nearly defunct languages, or worse, links that no longer exist, it is really more difficult to understand than it need be.

I'll start with a brief overview of hexagonal grids, and their typical representation.

Below, you'll see a hexagonal grid.