The Workbench

The past and future of game development education

2013-01-29T06:18:00.000-08:00

With most things, when you get to the end of an adventure, you start thinking about the beginning, which for me, started waaaay before our upcoming UDACITY course on HTML5 Game Development.

Education has always been an important portion of my career. Back in the early 90’s, when I started getting into game-development, education on the subject didn’t publicly exist; I had to teach myself what was needed through scraping together tutorials, Quake 1 / 2 source code, and hounding the gamedev.net forums. It was slow, painful, lonely, annoying work, and the results were often hit and miss. By the time I hit college my obsession had driven me from a novice to a self-educated semi-stable games-programmer, which caused nothing but grief for my CS professors (looking back, I greatly apologize to all of them ;)

The constant lamentation that I had during my self-education was the isolation. Honestly if it wasn't for a handful of contributors on the flipcode.org pages and GameDev.net forums, my career in Game development would never have happened.

That’s why I jumped at the opportunity to work as a part-time Adjunct professor at SMU Guildhall school for game development; Firstly I was impressed that this school even happened: you had heard of specialty schools like DigiPen and Full Sail, which were mostly specialty technical schools, but an accredited university picking up game dev? Wow.
Secondly, I couldn't help but see myself in the process; If I had some formal system of grizzled developers educating me on game dev, I might have been a better developer, faster. In some ways, this opportunity would be a way to try and help these pending game-developers with mentor-ship that I never had. I spent a great 4 years working there, teaching courses on Math, Physics, Graphics, and Concurrent programming. During that time, I was able to connect with hundreds of students and bring to them knowledge of all the crazy game-dev stuff that was stuck into my head. Quite frankly, I loved working with the SMU staff, and was really sad to have to move on from that chapter in my career.

I also found during that time, Engineering in Game Development is the most unique, difficult, and complex skill set across disciplines for programming; It’s very hard to find a set of programmers that have to produce as much content in a short millisecond frame time. Seriously, think about it; an RTS has to do pathing, network prediction, physics/collision detection, animation, visibility culling, scene setup, lighting, shadowing and finally graphics submission/rendering; in 30ms.

In contrast, a web-developer is considered as doing really good if their site loads in under 3 seconds.

Game dev is hard and most times unrewarding considering that most game-players won’t spend their time talking about your long hours optimizing the terrain rendering, and instead will complain about a 0.02s change in the cast-rate of your Druid build.... So part of teaching game developers involves a bit of ego-breaking; You have to let them know that any of their work is only part of the larger picture.

Back to the point, fast forward 3 years (Seriously? WTF?) I’m sitting in a cafe at Google’s Mountain View campus where I’ve spent the past year as a Developer Advocate, writing code, planning strategically, and largely educating developers. +Peter Lubbers, and I were chatting about how to reach more developers on best practices in web-game development, when he said:

“What if we did a UDACITY course on making an HTML5 game?”

I’m not really sure I ever agreed to do any work, or formally asked to be part of the project, I just started rambling on about curriculum, time frames, and started mail-bombing Peter with ideas and code snippets. We had recently wrapped up our exercise building the HTML5 PvP game GRITS, so there was plenty of content available, and it seemed like the perfect chance to bring that knowledge to more developers.

Fast forward 4 months to January 2013, where +Sean Bennet and +Calvin Hu are hunched over a video timeline, trying to find the best way to edit out my usage of the phrase “Your mom” from a UDACITY video snippet for our upcoming class; meanwhile Peter is syntax checking all the code snippets, grumbling about how awful our code looks. It’s a massive understatement to say that working with the UDACITY folks has been an amazing experience; Peter, Sean and Calvin are world-class superstars to work with, and have made creation of the course a fun and awesome process; Those guys built this course, I just opened my mouth and vomited out all the crazy game dev stuff locked in my head. The entire endeavor has proven something I’ve been thinking for a while now, standard-education for game-development is a difficult path to climb. Game devs are young, energetic, ambitious, impatient and generally focus on challenging the status quo; they need their attention funneled into an intense education curriculum to get them up and moving fast. SMU, DigiPen, FullSail and UDACITY are changing how developers make games, in the right ways.

So what’s the course about? Directly we've broken down the GRITS framework into a Client-side, and Server-side curriculum, and this first course focuses entirely on the client. We tried to approach the content to present all the tips, tricks, tools and ‘useless info’ that’s needed to make a fast, functional, client-side-only web game. The team has worked in overdrive mode to make sure that we present the right amount education, humor, and great games-industry stories to keep people interested. (also, trying to balance the right amount of @FAKEGRIMLOCK references...)

So if you’re looking to start making HTML5 games, sign up for the class now! If we’re lucky, and there’s enough feedback to warrant it, we just might be able to do the 2nd course, where we teach you how to write all the server-side portions of the GRITS code base; So no pressure, but if you like the course, tell your friends ;)

The class is set to go live on February 4th, followed up by a series of live-streamed weekly study groups, from the Google offices in San Francisco; So if you’re in the area, make sure you stop by for some face-to-face tech talk on a weekly basis!

Until then, here’s a nice blooper reel for you folks to sink your teeth into :)

~Main

You can find Colt McAnlis here:

What GameDevs should learn from Nate Silver

2012-12-17T06:54:00.002-08:00

One of the most critical points of Feeding the Iteration Machine is taking a numerical look at what your product is doing. You can’t truly iterate-to-improve your idea unless you know what’s wrong with it. As a game dev, you need to cut through the jargon and gut feelings and embrace big-data to develop a better game.

Viewing the future using numbers

Nate Silver recently gained fame after he accurately predicted 50/50 states for the 2012 US presidential election, (following his 2008 prediction of 49/50 states, where he gained less fame for some odd reason). This was a fantastic opportunity for big-data to get back into the light of the lives of common, everyday lives. It was amazing to see how Nate was able to use data prediction and modeling, a counter-intuitive process in a political system that is still driven by people following their gut feelings. And much like Moneyball, it’s apparent that the political system will never be the same; for those of us in Silicon valley, this was old news, but for most people, what Nate did was magic:

The reality is that big data rules the 21st century, and any business developer that’s not utilizing every statistic available is immediately at a disadvantage; you’re flying blind. That’s one of the great things about having your business be digital, you get input and feedback instantly.

This type of statistical analysis doesn't just extend to your product in the wild, but should also be used to track the information about your game in pre production too. Utilizing big-data in order to track your game’s frame rate, stability, and content-churn are all great predictors. We've seen this being used to great success for social casual games, and even hardcore FPS games. Directly, you shouldn't be asking if your game should track data, but rather what data should I track?

Big data checklist

Here’s a simple data-checklist on the things you should be tracking and how you should be responding to it.

What you should be tracking:

Stability each day

Track your crashes, against platform, region, map, callstacks etc.

Track a user click-stream

What are they clicking on the most, what aren’t they clicking on?
How do you analyze it?

Track a user purchase profile

If you’re using a freemium style, it’s important you not expecting every user to monetize the same way. Create a profile for your users and react to them properly. use location

How are users discovering your game

Do you know how you’re acquiring users? is your marketing campaign working? Should you be spending advertising money on kotaku.com, or thebumb.com?

Track player progress

How many players make it to level 7? How many players buy item Y?

What you’ll need to use the data properly

Automatically Generate daily reports

Generate reports from your data each day.
Make sure that you can easily access your reports. If it’s too difficult, then you’ll ignore it over time, or not get a chance to look the data the right way. You’ll ignore the signals before it’s too late.

Availability to drill-down as needed

Make sure that your data isn’t static, and that there’s ways to look at the information in different ways. It’s not beneficial if it takes an extra 3 days to track a new metric, or chart a graph plot against another value.

A flexible backend

Make sure your backend can easily accept new events, or discard old events.
Make sure your backend can easily be duplicated / redundantly stored; You should be able to track all the data for your game since inception.

Semi-regular “What does this mean” sessions

It’s important to schedule semi-regular sessions with the stakeholders of the product to determine what the data means. After all, there’s lies, damn lies, and statistics, and it’s important to get multiple validated viewpoints on the data coming in before making any actions

Ability to push builds quickly

We’ve already discussed how important this is for the life of your game

Big data fixes failures

Failure is not an option, it’s a requirement. If you’re a game developer, and expecting to not fail, then you’re delusional. The best studios hide these failures from being commercial by moving the iteration process into something that’s not tied to their bottom line (internal testing, beta tests etc)

The truth is, that if you fail,and don’t know why then you’re going to keep failing, and success isn’t coming any time sooner. Too many game devs will fail on a product, and not know why; they end up blaming tangential things which may not be correlated. Once the failure has occurred, you should know why, and be able to fix it next time.

“Insanity is the act of doing the same thing but expecting a different result”

Make sure you know why you’re failing, so you can stop doing it.

~Main

You can find Colt McAnlis here:

Feeding the iteration engine

2012-11-26T06:35:00.000-08:00

During my humble time in the games industry, the most prominent feedback I would receive about ‘how to make successful games’ was ‘iteration’; that is, constantly polishing, grinding and working on a video game would continue to make it better, more focused, and more qualitative. One problem with this, however, was that there was never a methodology behind it; but rather a wandering vague guideline to iterate to be great.

As with most things, the games industry didn’t create this mantra, and certainly web-companies have done a better job at embracing this concept in their battle cry “ship early, ship often.” This methodology has been proven multiple times as a cornerstone of the creative process. 37Signals’ book Rework, a clear counter-culture process is presented for product development:

Ship a product super early

Let the users tell you what’s wrong and what needs to be fixed

Fix all the stuff in #2

Repeat

Profit

Lean startups have been adopting this philosophy more aggressively lately; with their champion FakeGrimlock offering sound advice in bite size form:

Make sure that you're on FAKEGRIMLOCK's #NOEATFRIDAY list; As he says "you on the list, or you on the run."

In the modern digital age, iteration is sound advice, however once adopted, you’ll quickly find that the provided tomes contain plenty of information on the why but quite lacking on the how. There’s a disconnect between the dogmatic message and the mechanical operation. Especially in game development, where most often, devs ignore proper iteration during pre-production and production phases,focusing only on beta/post-ship feedback; This is a great shame, after all, content creators should have the ability to iterate on their craft as well.

Iterative gaming wins

Often, game developers are adverse to early feedback of their product, fearing that if exposed to the criticism of the outside world too early, will cause the game to fail. Interestingly enough, all the data shows the contrary, that games which ship early in alpha, beta, and continue to grind and make improvements in the public eye have been shown to have stronger user engagement, and have MORE success over time. Minecraft and Realm of the Mad God are perfect success stories in this space, having launched in early forms, and didn’t see success until being in development which included the public feedback loop, which quickly vaulted them into massive success stories.

While desktop and Web games have this ability directly, mobile game development exhibits this phenomena in a different way; where smaller companies, trying to power-level the limited space distribution platform, ship multiple games in a short time to attempt to stay on the top of the rankings. Implicitly, this creates a user-feedback cycle; This fast amount of iteration allowed the developer to get a great deal more feedback about their ecosystem and habits of their customers, in addition to doing a breadth first search on what content will become popular. An often difficult task to predict, given the general random nature of the marketplaces.

Poster-child success story Angry Birds is heralded as the secret sauce that caused the mobile gaming boom, but often forgotten is that developer Rovio Entertainment had shipped 30 other mobile games over the course of 3 years. And this has proven to be a process that works, with early 2012 success story Temple Run being the 10th title from game developer Imangi on the mobile platform, resulting in over 100 million downloads of the app.

Iteration means focusing on what counts

This approach to development methodology can be a breath of fresh air for most game studios who complain about feature creep. As Donald Knuth was once attributed to saying "premature optimization is the root of all evil.” And web developers / lean startups are right on target with this effort. Rather than spending years grinding internally, only focusing on internal feedback about priorities and power-struggles associated with internal politics, these developers were getting their product directly into the hands of the consumers, and prioritizing feedback based upon that first. These developers don’t have the capacity to prematurely optimize; There is such strong dialogue with their potential user base, they knew exactly what to work on, and the impact it would have on the end product. Which is the important part; after all, the customers are the ones keeping your company in business.

Your Iteration Engine Checklist:

Your train to success should be driven by the iteration engine, and like any complex system, you need to understand the parameters before building anything. As such, here’s you’re iteration engine checklist:

Pre Production:

Make a place to keep, sort, and find iterations & feedback. As you start iterating and collecting feedback, you’ll soon want a log of what didn’t work, and why. And as your project rambles on, you’ll need to refer back to that data over time, so find a good place to store your successes and failures.

Iterate the hell out of your idea. Share it with others, get their feedback; Automatically assume that you’re too close to the idea, and need others to validate it. Start with your co-developers for feedback, then slowly grow your circle to friends/family before opening up to unknown people.

Embrace a 1-off technology for prototyping. Don’t be afraid to generate tech that you’re going to throw away; You learn a lot from those efforts. Have your designers work in HTML5, Flash, Unity, or XNA to quickly build a prototype, and maintain a fork of their efforts so they aren’t blocked on product features.

Start with someone elses’ idea and make it better. Some of the most successful games of all time have been more spinoffs of already successful games. Halo, World Of Warcraft and Braid are great examples of this. They were able to duplicate the successful parts, fix the crummy parts, and add their own unique twists and improvements to make a much better product.

Make something fun without the features. Your goal out of pre production should be to have a fun game mechanic, that feels fun without the need for millions of bells and whistles. Take this time to prove your hook, interest, and goals; the extended feature set should add to your product quality, not define it.

Clearly define what your ARE / ARE NOT. During iteration, it’s easy to ramble on down a path which may not be directed towards your initial goals; while sometimes this can yield amazing success, it’s important to put an upper limit on your efforts. If you’re trying to make a platformer, don’t start programming an FPS. Your definition of what the target is should be constantly evaluated and appear in every iterative decision process.

Production:

Design in parallel. Keep designers working in the prototype build, in parallel to commercial builds. This will allow your designers to prove ideas and gameplay that won’t force new task ripples through code, art and production groups, this is a critical path that keeps feature creep at bay; You can test if something is fun without having to move heaven and earth to test it in-game.

Embrace tools for small content-creation loops. Content creators should have a tight feedback loop to getting content into the game. Optimize your content creation tools to allow Artists to get content in as fast as possible, as easily as possible. Reduce the number of steps, clicks, and exports needed.

Always Be Stable Builds (ABSB). Production builds should be continuous and easy to create. Make sure that you have a way to kick off manual builds, alongside automated builds that occur daily.

Bonus points: Make sure that the developer knows what build they are running at all times.

Double Bonus: Zip up the source code and asset trees for that build, so that you can properly roll back changes and issue tests on historic code.

Track your stats. Track crashes and automate testing in as many ways as possible; Make this part of your build system; Incorporate Google’s testing unit testing framework. Can you chart the FPS changes over the past month? Do you know if a checked-in asset is causing out-of-memory crashes? Where’s your memory going today?

Bonus Points: Embrace great data visualization tools to view your daily stats

Don’t get comfortable with your game. Once you get comfortable, you’re too close to give feedback. Don’t playtest your own game too much; instead get friends, family, close testers to keep trying your game and give you feedback. Don’t let them become familiar with the game, or their feedback becomes useless. You should listen to your user feedback and iterate quickly to update your build with better content.

Make time to clean up. By constantly iterating, your art and code can become a mangled mass of items that don’t exist any more. Make sure that on regular intervals you’ve scheduled gardening duty where everyone pitches in to make sure the garden is free of goblins that will inhibit future endeavors.

Post Ship:

Create a shippable product. Make sure you’re extending your ABSB to the customer. There comes a point which you will need to patch up all the holes and issues before shipping a polished, stable build. Shipping unstable builds reduces the user experience and does nothing but cause stress.

Ship Early. The only way to get feedback is to have other people play your game. Ship early versions of your game sooner than you’re comfortable doing; Put your ego aside and embrace the feedback.

Ship Often. Make shipping updates part of your product lifecycle. And then keep doing it. Make sure that your technology supports easy patching and content updates; make the user happy to see that a new patch is available.

Understand your user. Ensure you’re deeply tracking user-clicks and actions, mining the data for meaning. Issuing tests and checking results should be part of your normal process. How many players make it to the 3rd level? How many have purchased armor? Constantly run and A|B tests to find the best results.

All aboard the iteration train!

The iteration engine can be considered a fractal algorithm. Directly you should be focusing on how to track user feedback playing the game, and then quickly respond to it in order to make better products. Truthfully though, this action extends to production and pre production as well, where you need to iterate on your ideas and content creation processes, just at a smaller level.

The iterative engine is not about saying ‘no’, it’s about proving ‘yes’ .

More importantly, it’s about proving success.

~Main

You can find Colt McAnlis here:

Distributing your game content via Torrenting

2012-11-19T07:53:00.002-08:00

In my previous article, we covered the processes to distribute your game via CDNs and the costs involved with that process. Thankfully quite a few people read the article, and a number of them pointed out to me that torrenting is also a valid, and cost-effective way to distribute large game content to users too.

Which was a great point. There’s some upper limit that you’ll eventually run into when distributing your data from CDNs, namely the cost involved with distribution of content. As we showed previously, this is a pretty low-cost operation overall, but if you’ve got a massively successful product, then this can create un-need costs that eat into your overhead.

Torrenting is a process that creates a Peer-To-Peer network that allows your users to transfer your data files between each other, rather than solely relying on your CDN for the data transfer. This can be a great benefit to your system, as it reduces distribution costs on your side, as well as gives your users an available option if your CDN goes down: they can still get the files from each other.
Torrenting isn’t new, and large companies, like Blizzard and Microsoft have been using it over the past few years for the very same reasons I mentioned.

disclaimer : Torrenting for piracy usage is not advocated by myself or any organizations I’m associated with. It’s an interesting technology to distribute files, and that’s where this article starts.

How torrenting works

A user who wants to upload a file first creates a small torrent descriptor file that they distribute to other users by conventional means.

They then make the file itself available through a BitTorrent node acting as a seed.

Those with the torrent descriptor file can give it to their own BitTorrent nodes which, acting as peers or leechers, download it by connecting to the seed and/or other peers.

The file being distributed is divided into segments called pieces. As each peer receives a new piece of the file it becomes a source (of that piece) for other peers, relieving the original seed from having to send that piece to every computer or user wishing a copy.

Pieces are typically downloaded non-sequentially and are rearranged into the correct order by the BitTorrent Client, which monitors which pieces it needs, and which pieces it has and can upload to other peers.

Pieces are of the same size throughout a single download (for example a 10 MB file may be transmitted as ten 1 MB Pieces or as forty 256 KB Pieces). Due to the nature of this approach, the download of any file can be halted at any time and be resumed at a later date, without the loss of previously downloaded information,

When a peer completely downloads a file, it becomes an additional seed.

Utilizing torrenting for your game

Creating a private torrent network

I’m going to go out on a limb, and assume that you don’t want your game data files lounging around all over the internet; and as such, you’ll most likely want to create a private torrent system to use. I highly suggest not rolling your own code here:

Setting up a seeding server

Rather than reinventing the wheel, you can take advantage of existing work for torrenting. For example, using uTorrent will allow you to run a torrent seeder on a linux/windows box easily, which brings the question of where to host the box?

Simple web-hosting won’t suffice to create a torrent seeder, you’ll actually need a compute layer, meaning that you’ll need some machine sitting out in the intertubes. Jeff Atwood has a great article on comparative costs between EC2 Compute units ($17k/yr) vs building his own high-end server for $2,750. in which he argues:

Of course, that figure doesn't include the cost in time to build and rack the server, the cost of colocating the server, and the ongoing cost of managing and maintaining the server. But I humbly submit that the one-time cost of paying for three of these servers, plus the cost of colocation, plus a bunch of extra money on top to cover provisioning and maintenance and support, will still be significantly less than $17,000 for a single year of EC2 web application hosting.

Of which, you have a separate question: How much performance do I really need? Jeff is certainly targeting web applications which need to execute high performance computations, where a simple torrent seeder will only need a minor fraction of that performance, since the lion’s share of the file transfer process is between peers.

This limited-performance need may allow you to branch out to other types of server technologies, for example torrent-server is a torrent seeder that runs on Google App Engine and allows you to scale production, reduce costs, and avoid having to maintain a large server infrastructure. Which, as a small developer could be a great way to get started.

Adding a torrent system to your game client

As we discussed in the article about patching, you’ll most likely have some separate system for bringing new build data off the web, and caching it locally, which is a great place to put tons of heavy torrenting / transfer code that you won’t want munging up your game code. Which should be an easy process, a quick search yields hundreds of client libraries for doing torrenting, most of which are open source, and exist in whatever language you happen to be targeting. Once integrated, a simple modification to your file-transfer logic should allow your client to wait for file completion, regardless if it’s coming from a torrent network, or a CDN.

One down side is that to use this type of setup, you need the user to agree to allow their bandwidth to be used for the greater good. Otherwise you can run into some fussy legal problems; which I am not allowed to comment about, because I am not legal council.

When to torrent, when to CDN?

It’s worth noting that torrenting may provide a worse experience for distributing content to your users. Especially if the number of seeds are low, or if connections between peers is less than ideal. There’s certainly a break-even point, in which it makes more sense to utilize one over the other, or even find ways to use both where ideal.

For example, the farmville folks have two seperate servers; They found if the number of daily users is below 5 million, it was cheaper to route those games through their own servers, than using a scalable CDN and the costs involved with it.

By all means the best solution here is a combination of CDN and Torrenting. For instance, using the CDN to help seed new patches and content updates, while letting torrenting do the lions share. Or even if there’s a low-peer network for torrenting, it’s better to swap over to the CDN structure.

And if rolling-your-own isn’t your thing, I highly recommend Highwinds’ Game Delivery Network, which provides all of the items in this article in a nice, awesome package. In general, for distributing assets, the Torrent-CDN approach is a great middle ground, as it allows you the flexibility and cost structures that are used by lots of powerhouse game developers.

~Main

You can find Colt McAnlis here:

HTML5 canvas: Advanced image picking

2012-11-05T07:15:00.000-08:00

Let’s take a look at some more advanced ways to do picking with HTML5 canvas for game development. In this article, we’ll cover two techniques to address problems with pixel-perfect picking; using convex hulls, and using grid binning.

Advanced data structures can accelerate our picking code, while in some cases, sacrificing resolution perfection.

Using convex hulls for picking

As discussed in the last article, Pixel-Perfect picking can increase the memory footprint, and potential performance burden for slower devices. For example, if we had a 1024x1024 image, that may only be 64k in PNG form, but once we fetch it to main memory, it’s now 4MB. There’s really no (good) way around this, since the getImageData function on canvas returns RGBA data uncompressed, even if we pass in a grayscale image, we’d get the full pixel footprint.

Ideally, it would be great to get a lower-memory representation of the image, without having to store the whole thing in memory. And to that degree, we’re going to introduce the concept of using Convex Hulls for picking.

Effectively, a convex hull is a minimum representation of the shape of our sprite; Once the mouse is pressed, we’ll test the mouse-point against the convex hull of sprite instances to determine if it’s inside, or outside of a target object.
We will lose some resolution on this process; that is, our results won’t be pixel-perfect any longer, but there’s a whole separate discussion about how precise your picking code needs to be, especially on mobile, where pixels are (generally) smaller than peoples’ fingers.

Generating the convex hull

Generating a convex hull is should be done offline, ahead of time, so that we can reduce the loading time for our HTML5 game. As such, I threw together a simple C++ app that loads a sprite, calculates its convex hull, and outputs JSON data that we include in the HTML file. Your mileage may vary.

I’ll spare you walking through the C++ details here, as the code itself is simple. It opens an image and calculates, for each scan-line what the min/max pixels are that represent alpha boundaries. From there, we use a modified QHull algorithm to determine what the maximum convex hull is for the set of spatial points.

The hull points are dumped to individual files, which for simplicity, I've manually added the hull data to the HTML file :


var cHullData=[
{"name":"0.png", "hull":[{"x":0,"y":16}, {"x":1,"y":15}, {"x":31,"y":0}, {"x":34,"y":0}, {"x":64,"y":15}, {"x":65,"y":16}, {"x":65,"y":25}, {"x":64,"y":26}, {"x":34,"y":41}, {"x":31,"y":41}, {"x":1,"y":26}, {"x":0,"y":25}, {"x":0,"y":16}]},
//etc. etc.

Hull generation process. Per-scan-line we find the min-max pixels for that row, and toss those at a convex hull generator.

Doing picking against the convex hull

To transition from per-pixel picking to hull picking, we need to make a few small changes. Firstly, a sprite prototype needs to load the hull data, rather than fetching the image pixel data. This is an easy task, as the C++ app will spit out the image name for a hull, so that when an image loads, we can look up with the proper hull is for the image.

SpriteProto = Class.extend({
//...

load : function(filename,w,h)
{
//...

var img = new Image();
img.onload = function(){

 targetSpriteProto.imgHandle = img;
 for(var u =0; u < cHullData.length; u++)
 {
  var thull = cHullData[u];
  if(thull.name == filename)
  {
   targetSpriteProto.hullData = cHullData[u].hull;
   break;
  }
 } 
}
img.src = filename;

To determine if a mouse click (ie point) is inside of our new convex polygon hull, we utilizing a method of point in polygon testing known as ray casting which performs it’s test by casting a line through the 2D polygon, through the point in question. For each line that our ray intersects, we toggle a boolean value to determine if we’re in or out of the polygon. The simplistic algorithm I used in the source code came from here.

SpriteProto = Class.extend({
//...
//--------------
isPixelContained:function(lclx,lcly)
{
 var inPoly = false;
 var numPoints = this.hullData.length;
 var j = numPoints-2;
 var latLng={x:lclx,y:lcly};
  
 for(var i=0; i < numPoints; i++)
 {
  
  var vertex1 = this.hullData[i];
  var vertex2 = this.hullData[j];

  if ((vertex1.x < latLng.x && vertex2.x >= latLng.x) || (vertex2.x < latLng.x && vertex1.x >= latLng.x) )
  {
   if (vertex1.y + (latLng.x - vertex1.x) / (vertex2.x - vertex1.x) * (vertex2.y - vertex1.y) < latLng.y)
   {
    inPoly = !inPoly;
   }
  }
  j=i;
 }
 return inPoly;
 
}

An example of ray casting to determine if a point is in a polygon. The line through the polygon an even number of times, signaling that the point is not inside the polygon boundaries.

Caveats

As mentioned, this method will reduce the overall memory footprint for your data significantly, and on some platforms can have the added benefit of faster execution for a given pick operation.

One down side is that this process doesn't fit well into a pipeline for atlases; You’ll need to calculate the convex hulls of your loose sprites before placing them into your atlas. Of course the lower memory performance comes at the cost of less resolution, something that you should consider before adopting this technique.

Faster picking via bucketing

Let’s say you’re complex app, and have 4096 images in flight when a user clicks. Obviously, you’d want to reduce the number you did a pixel or convex-hull test on, and although a simple bounding-box test will be the quickest to implement, you still end up touching a lot of items that aren't even remotely near the point in question.

The solution to this is to introduce a spatial acceleration structure to our canvas; A process which organizes / separates our sprites spatially, such that we speed up spatial tests by only referencing items which are reasonably co-located. There’s lots of variants for SAS which have pros, cons, and lots of trade-offs, and for our purposes, we will use a very simplistic 2D binning algorithm, which will divide our canvas into a grid of cells; each cell will contain a list of objects which touch that cell, allowing a sprite to be listed in multiple cells.

Our bucket Grid class will effectively create a 2D array, of arrays, such that when a sprite is spawned, we calculate what grid cells it overlaps, and add a pointer to this instance to each of those cell’s lists.

BucketGrid = Class.extend({
 tileSize:16,
 numXTiles:0,
 numYTiles:0,
 tileData:null,
 
 init:function()
 {
  this.tileData = new Array();
  //16pixels is an arbitrary number from testing.
  this.tileSize = 16;
  this.numXTiles= 512 / this.tileSize; //512 = canvas size
  this.numYTiles= 512 / this.tileSize;
  
  this.tileData.length = this.numXTiles * this.numYTiles;
  for(var k =0; k < this.tileData.length;k++)
   this.tileData[k] = new Array();
 },

Marking a sprite instance on the grid is pretty simple, we simply run some math on the corners of the sprite to calculate the min/max boundaries of it, and what tiles those boundaries fall into; and for each grid cell, we add the sprite to the containing list.


//--------------
 markInstanceInAccelGrid:function(sp)
 {
  var gridminX = Math.floor(sp.pos.x / this.tileSize);
  var gridmaxX = Math.floor((sp.pos.x + sp.size.w) / this.tileSize);
  var gridminY = Math.floor(sp.pos.y / this.tileSize);
  var gridmaxY = Math.floor((sp.pos.y + sp.size.h) / this.tileSize);
 
  //we cheat here, knowing that our rand() placement doesn’t allow neg numbers
  if(gridmaxX >= this.numXTiles) gridmaxX = this.numXTiles-1;
  if(gridmaxY >= this.numYTiles) gridmaxY = this.numYTiles-1;
  
  for(var y = gridminY; y <=gridmaxY; y++)
  {
   var idx = y*this.numXTiles;
   for(var x = gridminX; x <=gridmaxX; x++)
   {
    this.tileData[idx+x].push(sp);
   }
  }
 },

This makes finding entities extremely quick, because we've already pre-sorted the environment. When a mouse-click comes in, we simply find the bucket it resides in, and return the list of entities that we've already calculated.


//--------------
 getEntsForPoint:function(x,y)
 {
  var gridminX = Math.floor(x / this.tileSize);
  var gridminY = Math.floor(y / this.tileSize);
  var idx = gridminX + gridminY*this.numXTiles;
  var ents = this.tileData[idx];
  return ents;
 }
});

Which makes modification to our existing code very nice and tidy

//--------------
function findClickedSprite(x,y)
{
 
 var alphaThreshold = 50;
 var pickedSprite = null;
 var tgtents = acclGrid.getEntsForPoint(x,y);

The acceleration grid reduces performance overhead by reducing the number of times the inner loop is called, regardless of how many sprites we have universally, this bucketing will only care about what sprites reside in a grid cell. An easy trade-off in bang-for-the-buck; We didn't have to derail ourselves by worrying how JavaScript is handling the math for convex hulls, or the array-access times for pixel-perfect picking. It’s a simple technique that can be used for static and dynamic environments.

Closing thoughts

Hopefully, you've got enough information from these past two articles to implement a very fast, efficient picking system in HTML5 canvas. note that there’s lots of other improvements you could continue on with; For example, having a separate Spatial Grid for dynamic vs. static objects; Compressing the Hull data for faster load times; and even caching click results for grid cells. Go forth and pick stuff!

Source Code

You can find the source code to this article on my github. It’s fairly simple, and shows how to use the polygon system, alongside bucketing.

~Main

You can find Colt McAnlis here:

HTML5 canvas: Simple per-pixel image picking

2012-10-29T12:53:00.000-07:00

If you’re writing a 2D game in HTML5, chances are that you’ll want the user to have the ability to pick an object on the screen; Since there’s no direct API for this type of notion, and no good results / examples on the web, let’s take a look at how to properly do picking with an HTML5 canvas.

Picking can be a tricky, troublesome, and socially awkward process; especially in kickball...

Pixel Perfect picking

For games that utilize picking for desktop applications, pixel-perfect response to a mouse click is crucial. Multiple images can be overlaid against each other, each one that can have varied alpha footprints which in no-way match their conservative bounding box estimates. As such, to do a pixel-perfect pick, you’ll need to be able to determine what pixel, from what image was clicked on; mainly identifying that you’ll need to keep a copy of the image data in memory so that your code can query the pixel array.

In order to do this in HTML5, we need to introduce two separate data structures, a sprite prototype, which represents a single loaded image sprite, and a sprite instance which represents an instance of the prototype on the canvas, that is, we assume that a single image is used multiple times on a canvas. Our sprite prototype then will need to contain the pixel data for it’s image element, such that we can query against it later.

Loading pixel data

The cornerstone of our picking process is assuming that your images contain alpha values, which we generally assume is loaded this way:

var img = new Image();
img.onload = function(){alert(‘loaded!”);}
img.src = filename;

Notice the problem here is that in Javascript, we don’t directly handle the pixel data; it’s handled behind the scenes on our behalf.
To get the data then, we need to do some extra work. We could start by writing a Javascript PNG decoder, but that would be massive overkill, considering PNGs support lossless compression. Since we’re really only concerned with the alpha values of an image, we could store the alpha channel in a separate .RAW file that we fetch in parallel, however this would increase the transfer and asset size of the app.
For the sake of our purposes, we ignore those two options, and instead decide to keep the code footprint low, and transfer sizes low by using the canvas element to fetch the data. To do this, we create an off-screen canvas, render our image to it, and fetch the pixels of the canvas object back to memory.

var offScreenCanvas= document.createElement('canvas');
var fetch_ctx = offScreenCanvas.getContext('2d');
offScreenCanvas.width = 128;
offScreenCanvas.height = 128;
function fetchImageData(imgObject, imgwidth,imgheight)
{
 fetch_ctx.clearRect(0,0,128,128);
 fetch_ctx.drawImage(imgObject, 0, 0);

 //note this keeps an additional in-memory copy 
 var imgDat = fetch_ctx.getImageData(0,0, imgwidth, imgheight);
 
 return imgDat;
}

This allows us to transfer a smaller asset footprint, keep using our PNGs / GIFs or whatever other compression footprint you want, and still get the RGBA data available in main memory during load time. to utilize this, once an image has been loaded, we fetch it’s image data using the function above :

var img = new Image();
img.onload = function(){

targetSpriteProto.imgHandle = img;
targetSpriteProto.imgData = fetchImageData(targetSpriteProto.imgHandle,w,h);
   
}
img.src = filename;

Testing a mouse click

Once we have the per-image data in memory, we need to test against it when a user clicks. This is broken down into a few sections of a larger function.

Firstly the findClickedSprite function will loop through all the sprite instances in memory, and do a conservative bounding box test against the picking point; we assume that array-lookup is a performance limiting action in javascript, and this bounding-box test allows an early out for items that don’t potentially intersect with the pick position.

//--------------
function findClickedSprite(x,y)
{
 var pickedSprite = null;
 var tgtents = spriteInstances;

 //loop through all sprites 
 for(var i =0; i < tgtents.length; i++)
 {
  var sp = tgtents[i];
  //pick is not intersecting with this sprite
  if( x < sp.pos.x || x > sp.pos.x + sp.size.w ||
   y < sp.pos.y || y > sp.pos.y + sp.size.h)
   continue;

Once we find a sprite instance whose bounding box intersects with the picking point, we grab the sprite prototype, and translate the canvas-relative mouse position to a sprite-instance-relative position that we use to test against. These values are passed to a function on the sprite prototype to determine if the target pixel is transparent or not. If we’re clicking an opaque pixel for this sprite, we set this as the selected sprite.

  var ps = sp.spriteHandle;
  //get local coords and find the alpha of the pixel
  var lclx = x - sp.pos.x;
  var lcly = y - sp.pos.y;
  if(ps.isPixelTransparent(lclx,lcly))
  {
   pickedSprite = sp;
  }
 }
 
return pickedSprite;
 
}

Note that an important part of isometric and top-down 2D games is the notion of zOrder, in which a separate index is used to determine and describe how to properly render the elements on the screen. Our picking algorithm needs to take this into account, so that what the user thinks is top-most, is actually represented as top-most.

if(ps.isPixelTransparent(lclx,lcly))
{
 //do depth test (if applicable)
 if(pickedSprite && sp.zIndex < pickedSprite.zIndex)
  continue;
 pickedSprite = sp;
}

The isPixelTransparentI function for the proto-sprite does very simple logic. Firstly it determines what the proper pixel is in the image data that we’re targeting; (Note that data given back from the canvas is always in RGBA form!) and tests the alpha value against some threshold. The threshold is important, as most artists can add gradient falloffs, drop shadows, and other items which increase the visual of the item, but shouldn't be considered for picking purposes.

isPixelTransparent:function(lclx,lcly)
{
 var alphaThreshold = 50;
 var idx = (lclx*4) + lcly * (this.imgData.width*4);
 var alpha = this.imgData.data[idx + 3];
 //test against a threshold
 return alpha > alphaThreshold;
}

Results

The results are quite nice. We can select the right object out of a very complex pixel coverage area.

Caveats

While this method works, and produces pixel-perfect results, it presents two primary issues:

1) Image data, which is normally stored in your javascript layer behind the scenes, now has to be duplicated in your scripts. As such, this results in a larger memory footprint; Often more than double the size, since your in-memory copy is uncompressed.

2) It’s currently unclear how an array look-up affects performance in javascript under the hood. In C++ you have the ability to avoid CPU addressing issues like L2 Cache optimization for array traversal, which is completely missing in Javascript. On my 12 core work-machine, a single pick against 4096 images takes around ~2ms. I’d imagine on a phone, that would be significantly higher.

And finally, it’s unclear if you really need pixel-perfect picking for your game; For instance, the user may benefit from a more loosely defined picking area, that is allowing an extension of the valid picking area beyond the pixel boundaries around the object, in an attempt to reduce user picking frustration.

Next Time

In the next article, we'll talk about some advanced and faster ways to do picking on an HTML5 canvas, Stay tuned!

Source Code

You can grab the working source-code from my github page. Note that you’ll have to access the page via the python server (run python httpd.py first, and browse to localhost:5103) as the getImageData function on a canvas can’t be accessed locally.

~Main

You can find Colt McAnlis here:

Costs of a cloud gaming backend

2012-10-16T10:56:00.002-07:00

There's a critical eye on Cloud Gaming right now, especially with the purchase of Gaikai by Sony and OnLive’s metamorphosis having just occurred. So the question lately has been “Is cloud-gaming still the future?” In order to answer that, we will need to have a technical look at the process involved, and hopefully understand a bit more about the cost of this process.

Mario knew what it took to get cloud-gaming working; Cutting corners at all costs.

Understanding how a video service works

Most video services, like YouTube work by caching the video data on a CDN, serving the content to clients via video streaming. A technology that will send small chunks of compressed video content to the client, which consumes and combines to display to the user while new chunks are coming in. Which takes advantage of the fact that sending 30 seconds of compressed video data to the client should be faster than the 30 seconds it takes to display it to the user. This allows video streams to buffer and load over time; that is the whole file isn't getting transferred to you, but small chunks of it, which then allow something to be displayed while the rest of the content is loading behind the scenes. This isn't a new concept, but it’s an important one to ensure that you’re not waiting for the ‘whole freaking video’ to come down before being able to see the first frame of the video.

Live video streams work in much the same way; You have a point of origin that’s generating frames, compressing them, and uploading them to a server cache, which broadcasts them to clients through the inter-tubes. Clients around the world who are watching the ‘live stream’ will be getting content from the server cache, and displaying as it can receive it. As such, there’s generally a significant delay between the live event and what’s seen on the internet to viewers; Not only does it take time to capture, compress, and upload the data to the server, but then it has to come from a single-physical location to a single location in the cloud. Which generates a large latency bottleneck for most live-streamed events.

These delays are generally forgiven by the end user. They have no way to know that their data stream is delayed, other than to compare it to a live-viewing. Most importantly, there’s no gauge to know how slow things are compared to the live event. It’s really a black box.

Interactive Video breaks the rules.

Cloud gaming is best thought of as “Interactive Video” a term which removes some of the mystique and business hype from the equation. Interactive video changes the concept of streaming video, since input from the user directly changes what the video content will be over the next few seconds. The video stream is now dependent on the user input, which the user feels it can gauge. So latency is very noticeable to the user.

The first thing interactive video breaks is that clients can no longer buffer-present the video frames to the user, as it does not know what the future frames are going to be yet, given the current state of user input. This means that the speed in which the client grabs the input video data off the wire, directly gates how fast they can render them on the screen. The best the server can do is render frames that may not be dependent or updated by the user input, waiting for that to be received.

As such, interactive video is very concerned about latency and lag, since it’s actually noticeable by the user. Most models for cloud-gaming services build an estimation of ~100ms for latency between input controller and user seeing a visual on the screen.

A standard desktop game running @ 60fps expects about 16ms for a frame of the game. For an interactive streaming service though, this gets tricky. Let’s assume you've got a 4ms ping from server to client, this already gives you only 12ms left to compute your frame. Then the server box has to compress it, let’s give that 2 ms, and this leaves your frame with about 10ms of time to compute in order to stay @ 60fps.

Understanding cloud-compute in relation to video streaming

Since interactive video needs to compute something, the data can’t just be distributed from a CDN to the masses. No, we need the power of Cloud Computing to drive this beast. Think of cloud compute as a large automated way to run chunks of code across millions of cores, resulting in awesome performance. Services like Google Compute Engine offer the ability to upload applications which can be run across thousands of cores online.

Our cloud gaming service will spin up a box on the cloud and start the game. Once a connection to the client is established, a stream of video frames will be captured, compressed and sent to the user through a socket connection. Thankfully, we can continue to send frames to the client at regular intervals, even the server hasn't received input yet.

It should be clear that the infrastructure to do this is not easy, or cheap to put together. Google already has data centers across the globe, with machines sitting around already. A private company attempting to produce the same results would have to dump hundreds of millions into setting up the same physical structure to compete in the same regard. So it’s not wise, if you were trying to build a system, to create your own hardware racks; Trust me.

The costs of doing business

Using these types of cloud compute services come with a cost however. Just to toss a dart towards a number, let’s assume that your cloud-gaming service has 3.4 million daily active users, each one receiving 1GB of video in a day. At H.265 compression should be roughly 46 minutes of play time, or around 2.6 million hours of computation a day to compute the video stream.

In this scenario, we should note that the costs of using GCE are two fold:

$1923 to transfer 1GB of data to 3.4 million users
$377,000 to hold the compute machines for 2.6 million hours

With some estimated math, these numbers might be obtainable, if you assume that each user is paying $5 a month fee, which would net you around $17,000,000 /mo.

Engineers reading this should have an immediate gut reaction to that number. Holding a single cloud-instance machine for an entire 46 minutes for a single client is obviously not utilizing all the resources on the box. If we were able to multiplex the machines, putting X users on each machine, we can cut the daily costs by X users-per-box. So, if we allowed 10 players to share a single box, that would cut our compute costs per day down to around $37,000 / day.

Skew all the numbers

And these numbers only include CPU side computation, completely ignoring GPU overhead. GPUs generally pull more wattage, and introduce more instability into a server rack, resulting in higher costs for cloud compute. In addition, trying to host multiple client sessions on a single GPU isn't an easy task; most games have a hard enough time making the GPU work properly with one instance of the game, not to mention multiple ones.

Thankfully this is getting easier, recently NVidia announced their new GeForce GRID hardware, which comes with hardware video compressors, alongside lower power wattage consumption, and the ability to virtualize, or mutli-slice the machine. This means you can run multiple game instances on a single box, reducing the cost of compute time. For instance, the Nvidia brochure denotes the ability to run 4 streams per GPU, reducing the daily cost for 3.4 million down to $94,250 / day; assuming that there’s no modifications to cost for extra GPU support (which is naive to assume.)

These modified GPU racks also have a separate optimization, reduced wattage. If you calculate the wattage of the average household GPU, * 3.4 million, you end up with about half the daily output of Hoover Dam.

Reality though is that $5 a month may not be tolerated by most gamers. Certainly the 3.4 million DAU are free players, which if you roughly estimate 2% are willing to pay $5/mo for a premium service, would result in around 68k players, paying $5 a month, resulting in ~$11k in revenue a day. Which is not exactly a slam dunk in the business sense.

However Sony did buy Gaikai, and we’re seeing other competitors popping up, including your local Cable providers. So there’s got to be some numbers that I’m not factoring in.

What’s it mean for game devs?

As we see more of these services pop up, I estimate we’ll see more service fragmentation as everyone rushes to get a critical mass of users-to-games ratio. A post-nuclear view could be a world where video-streaming services are as prolific as flash-portals are today. As noted though, the cost of investment is so high, the number of platform holders will be quite small.

For a game developer, this is generally a good thing; you’ll be able to use these streaming services to make your game easier to access, even allow users to try your game for free w/o having to modify your application, segment assets, or pay to transfer data to that user.

Cloud gaming silos should be looked at in the proper light, they are still silos, with the issues and benefits of it, attached alongside a tool for webification. Until these items truly take off the ground, it’s difficult to predict when all the planets will align, but chances are, it’s still on the horizon.

~Main

You can find Colt McAnlis here:

Indie Dev? You should be selling your game on the web.

2012-10-08T13:10:00.003-07:00

If you’re an indie developer, and you’re not selling your game on the web, you might be doing it wrong.

The Success Story

In 2009, Minecraft was launched in alpha, available from the author's website, in which you could download and try for free. Alternatively, you could donate to the development of the game, in which case when it was release, you’d get it for free. The results were staggering. People realized early on that the game was fun, and quickly payed for it, and most importantly, told their friends because they were so passionate about it. The message was easy to spread; the early builds were in java, and I remember playing a version of it in my browser. A year later, Mojang was founded after the author's PayPal account was temporarily suspended for suspicious activity when $700k appeared in the account almost overnight. That was entirely without any digital distribution ecosystem, or social sharing middle ware. It was just a great game being sold on a website.

Today, Minecraft has over 9 million sales, and has builds for console, mobile, and desktop. And it all started by selling on the web to passionate users who were willing to carry the message.

Indies gone wild

Another great example is Positech games, makers of Gratuitous Space Battles. Cliff Harris, owner/founder/indie gave a great talk at a conference back in 2009 where he described about how important it is for him to sell his games on the web. In his goal to sell a game every 52 seconds, he overviews how using tools like Google Analytics allows him to reach and understand his users.

“Advertising online is a real time strategy game that geeks like you are good at. … It’s a game that you’re playing with real money, and if you win, you make money.”

Other indies are realizing that eyeballs = profits, and are starting to branch out to find new ways of creating success. For example, Desktop Dungeons, a yet-unreleased game is looking to model it’s early development states after Minecraft’s success, alongside the the recent Kickstarter boom, by allowing users to fund the development of the game by buying early build for different tiers of investment, including a level that includes a reference to you, somewhere in the game.

And let’s not forget the massively successful Humble Indie Bundle, which allows consumers to donate an amount of their choosing to charity, and in return, gets DRM Free versions of the games provided in the bundle. To date, they've made over $19 million dollars for charity, selling directly through their website.

The barriers that don’t exist.

Many game devs I talk to shy away from selling their content online. In their minds, it’s a cumbersome process to market and handle money exchange etc etc. The truth is, these barriers are no longer relevant in 2012, and we should re-visit some of the big issues.

“Taking money for purchases is difficult online”

Back in the day, it was a tough process to sell digital goods through the interwebs. Thankfully through the rise of merchandising on the web, you've got more options than you can shake a stick at. Take a look at the page from GSB :

Cliff is an indie developer, and he goes out of his way to make it as easy as possible for a potential customer to become a paying one, because that’s how he feeds his family. In Fact, he goes the extra mile to tell you why you should buy from his site directly:

When you buy games from us, we get about 91% of the money. the 9% goes to our credit card payment provider. There is NO MIDDLEMAN. We do not have any RIAA or MPAA or other organisation that is profiting from our work. If you care about supporting people who make the kinds of games you want to play, here is your chance to ensure the money goes straight to them.

“It’s too difficult to support multiple platforms.”

Selling your game on the web means users will be able to buy your game on various types of platforms, operating systems, and hardware. This can be a daunting operation, considering the amount of time and dedication that goes into solving platform fragmentation.

Thankfully, with the influx of indie game developers coming onto the scene since 2009, technology middleware vendors have been priding themselves on creating tool-chains for cross-platform development. After all, not every user is on every gaming platform, so making your game accessible across multiple platforms and services increases the number of users that can become customers. Unity, Unreal, Moai, Blitz and Game Maker are all examples of middle ware systems that pride themselves on going multi-platform first, most of which even allow a web-specific build so the user won’t have to install the game locally. Now that’s not saying that supporting multiple platforms is automatic. You still have to deal with issues of screen resolution, input, and performance differences, but that’s a different article ;)

With these technologies available, selling your game on the web is even easier, as you can ensure that your app supports all the platforms that the web can find you on.

“It’s too difficult to market online.”

For current game developers “Discoverability!” is the lamentation of the day. The largest feedback I get from developers is about how hard it is to be successful in the modern industry. It seems everyone is content to fight for 96x128 pixels worth of screen real estate on a mobile store, working to get more eyeballs on your content by opting in to networks that will cross-promote your digital swag.

However, a recent article has discovered a great data chain about how mobile users discover new content. Not surprisingly, the two dominant forms of finding out about new content was via social channels, where on-mobile stores came in 3rd.

Going ‘viral’ has always been touted as the power of the web, and allowing your users to link, share, comment and buy your content, right there in the web-browser is a powerful metric to allow your content to be shared and consumed virally, and selling your content online is a very well researched science at this point. There’s been entire college courses devoted to this ecosystem, and all of the results of this type of intense focus is available online for you to learn from.

The big picture

It’s quite easy to toss aside this device claiming Return on Investment; saying “it’s not worth my time to support that platform” and all that jazz. The truth is that as a developer, you have no idea how your content is going to be received. There’s been really no rhyme or reason to why some games suddenly become successful lately, other than users attaching themselves to content they love, and doing everything they can to tell their friends about it.

I caught up with Cliff Harris recently where he mentioned that even though GSB has a tablet build, alongside being available on STEAM, he can still make over ~$1,000 / month for GSB through direct sales on his website. A game that’s over three years old.

~Main

You can find Colt McAnlis here:

Simulating Blocking PPAPI functions in Native Client

2012-10-04T15:06:00.001-07:00

Simulating blocking functions in Native Client

One of the largest hurdles in porting to Native Client is the Pepper API; a powerful, secure plugin api for Chrome, which NaCl utilizes to gain access to lower-level operations through the sandbox. Along with being secure, Pepper has two specific restrictions that we’ve covered before

All pepper calls must come from the main thread

All pepper calls must be non-blocking

Which, as a developer porting over to NaCl, may throw a wrench in the works, considering that most C library APIs for system access are blocking functions (fread, fopen come to mind directly). We’ve mentioned that this is a temporary restriction, and soon you’ll be able to call pepper functions off the main thread. In the meantime, however, it’d be nice to have a way to allow calls coming from the main thread to interact with Pepper in a blocking manner; this will reduce the amount of code you’ll have to change in your codebase to support the NaCl platform.

Thanks to some great insight and help from NaCl engineer Bradley Nelson, we can offer just that capability using a fancy technology : coroutines.

Sometimes, Blocking is the right thing to do.

From wikipedia :

Coroutines are computer program components that generalize subroutines to allow multiple entry points for suspending and resuming execution at certain locations. When subroutines are invoked, execution begins at the start and once a subroutine exits, it is finished; an instance of a subroutine only returns once. Coroutines are similar except they can also exit by calling other coroutines, which may later return to the point of calling in the original coroutine; from the coroutine point of view, it is not exciting at all but simply calling another coroutine.

Or for humans, coroutines are like global goto statements, that also preserve stack-state, and are exposed with a function pair : setjmp and longjmp. The setjmp function saves a stack environment, which you can subsequently restore, using longjmp. When used together, setjmp and longjmp provide a way to execute a non-local goto. They are typically used to pass execution control to error-handling or recovery code in a previously called routine without using the normal calling or return conventions.This process can be imagined to be a "jump" back to the point of program execution where setjmp saved the environment.

There’s plenty of documentation on how these APIs work, so I’ll ignore the description of walk-through here, and refer you to the resources section at the end of this article.

Using coroutines to emulate blocking Pepper Calls

Firstly, recall how Chrome interacts with Native Client; The PPAPIs are effectively plugin APIs that chrome calls back into, so execution control can either be owned by Chrome, or owned by your NaCl plugin; coprocessing is not an option. So, for us to simulate a blocking PPAPI call on the main thread, it must stop execution, store the state, and allow Chrome to regain execution control. Chrome can continue forward and process the pepper call, and in the resulting callback, we can process the result, and continue execution back where we left off. The diagram below shows this in more visual form:

When used correctly, the setjmp/longjmp pair allow us to save and restore executional state. Meaning we can exit the current program counter and stack, allow execution to occur at a different logic branch, and when the time is right, resume execution control back to where we wanted. This is perfect for what we want to accomplish with PPAPI.

The source code

You can grab the source code to a working example from my github repo; which will utilize the new, fancy Visual Studio 2010 add-in for native client. The source code wraps up most of the heavy lifting of how co-routines can be utilized in NaCl; so for the sake of brevity, I’ll show you the expected workflow using the C APIs.

static PP_Bool Instance_DidCreate(PP_Instance instance,
                                  uint32_t argc,
                                  const char* argn[],
                                  const char* argv[]) {
 appInstance_ = instance;
 //called when the app boots. Allows user to own their control loop
 coroutine::Create(userUpdate);
 return PP_TRUE;
}

When the application is created, the first thing we do is allow the co-routine system to set up the execution slicing. Simply calling coroutine::create is sufficient. You pass in as a parameter a function which will be ran during execution slicing.


void userUpdate(void) {
  for (;;) {
   
 //..some code goes here....
 
 //randomly kick off a file load call for whatever reason
 if(rand() %50 == 25)
  blockingURLRead();

 //some other code goes here....


 //IMPORTANT, at the end of the while-loop, 
 //we need to signal execution control back to chrome for processing 
 coroutine::Flush();
  }
}

One of the nice things about using this coroutine setup is that you gain the ability to treat your app updating as though you were a native application: by owning your processing loop. In the code above, we randomly kick off a call to load some file from the intertubes. At the end of the loop, we need to call coroutine::flush which will handle some of the time-slice execution that we’ll be using to make all this work.


void blockingURLRead()
{

 //rather than actually calling readFileFromURL, 
 //we simulate some arbitrary Pepper API function by just calling 'callonmainthread'
 PP_CompletionCallback fileLoadedCB = PP_MakeCompletionCallback(onFileLoaded, 0);
 ppb_core_interface->CallOnMainThread(0, fileLoadedCB, 0);

 //now that the async call has been kicked off to chrome, 
 //we yield execution control back to chrome so it can service it
 coroutine::Block();

 //once 'onFileLoaded' has been called, we resume control here.
 printf(&fileIOBuffer[0]);// AKA DO SOMETHING WITH THE FILE DATA YOU LOADED!
}

The blocking URL read will kick off the Async pepper call, which simply pushes a command into Chrome’s processing queue; Recall Chrome won’t process this event until it regains execution control sometime in the future.
Once kicked off, coroutine::Block is called, which will halt execution of the main thread at this location, yielding execution control back to Chrome to do processing.


void onFileLoaded(void* pData, int32_t dataSize)
{
 // process data returned from file loading
 strcpy(&fileIOBuffer[0], "I AM A CHEESEBURGER");

 //once all the data has been read, we can regain execution control
 coroutine::Resume();
}

Once chrome has time to process the command, it will issue the callback to your NaCl application with the results (onFileLoaded). For the sake of sanity, we move the data out of this callback system to a heap variable, and call coroutine::Resume, which will restore the stack state from where we last called coroutine:Block;

Caveats and limitations

There’s some nuances with getting this working correctly, most dominantly that we must manipulate the stack to allocate enough space such that when we resume our operations data on the stack didn’t get clobbered. For safety measures, I suggest increasing the stack size for your application (I’ve found 3MB to be sufficient for most apps) .

Also note that some IDEs (VS included) may do weird things when deep in this level of stack manipulation; so be warned that there may be dragons in the future.

But at least those dragons don’t require you to re-factor your code as much ;)

~Main

Resources:

You can find Colt McAnlis here:

Official NaCl VS2010 add-in available

2012-10-01T14:56:00.001-07:00

I'd first discussed getting a visual studio build for Native Client working a while back. Afterwards, we decided a professional version was in need, and after talking about this at Google I/O this year, it's finally hit the streets. In contrast to my previous article, with the plugin, you'll no longer need to download chromium source code, or copy the header files.

Let's get started:

If you haven't yet, download the nacl sdk (say, to c:\nacl\)
Install pepper-canary if you haven't yet

From a command line, run c:\nacl\naclsdk install pepper_canary

From a command line, run c:\nacl\naclsdk install vs_addin

which will grab the vs2010 package and bring it down onto your machine

Browse to c:\nacl\vs_addin
Double click install.bat

Once installed, you can create a new PPAPI and NaCl platform, just like we've discussed before!

As always, send us feedback!

~Main

You can find Colt McAnlis here:

NaCl & UDP Sockets

2012-09-05T13:37:00.002-07:00

One of the big pieces of feedback we've been getting from developers is that Native Client needs better networking support (read :sockets). Largely sockets have been one of the larger security issues for Native Client and the open web; Imagine if a web page could start sniffing any open packet communication on your computer :( Thankfully though, that’s all changing. With the new Platform Apps technology on the horizon, we now have the ability to play with UDP inside of Native Client, and this post will show you how to play with the early preview of the technology.

According to legend, the internet is a series of tubes. As such, we should allow you to send data through those tubes using Native Client

WARNING:

The future of this feature is in flux;

The steps below are valid as of 9.9.2012 and are subject to change as the future changes.

Please implement this functionality at your own risk.

Getting Started with the prototype UDP APIs:

UDP APIs are currently in draft form, meaning they are not available in the stable version of Chrome; In order to play around with these features, you need to be running a canary build of Chrome.

1. Getting the private headers

Copy the private headers from the Chrome repo into the SDK repo

C:\..\chromium\src\ppapi\c\private
C:\nacl\pepper_canary\include\ppapi\c\private

Since these headers are currently private, you’ll need to pull them from the Chrome source code. Once these APIs’ stabilize, and are moved out of the private repo, this step will no longer be required

Once done, you can add the following includes to your NaCl app:
#include "ppapi/c/private/ppb_udp_socket_private.h"

2. Run as a packaged app

UDP support in NaCl is ONLY provided as a Chrome extension / packaged app. You cannot develop as a pepper plugin; As such, you’ll need to create a manifest file that’s co-resident with your NaCl app, and launch chrome with --load-extension=PathToYourFolder. This will add the app to your new tab page, that you can then click-on to load. For instance --load-extention=C:/myapp/bin/

NOTE : If the above gives you a 'could not load manifest' error for whatever reason, then you can load the application as an unpacked extension, and things will still work properly.

3. Enable the experimental UDP apis

Go to about:flags and enable "NaCl Socket API"
Make sure you launch chrome with --allow-nacl-socket-api=IPAddy
Make sure you launch chrome with --enable-nacl

In the current state of things, you MUST define what IP address you're going to be connecting to (minus the port number) from native client. For instance --allow-nacl-socket-api=127.0.0.1 will allow your UDP socket to properly connect to that IP address.

4. Note the API differences

Note that the UDP API is not the same as standard socket APIs, due to the restrictions of the Pepper API, the interfaces for UDP are:

Non blocking
Required to be called from the main thread (a fix for this is in the works)

To get a feel for the differences, check out ppapi/c/private/ppb_udp_socket_private.h which generally shows how to create a socket to communicate to; Because of the PPAPI restrictions, the usage of these APIs requires some jiggl’n.

5. Some implementation notes

1) You’ll notice that Bind(..) will return -1, which is a correct, valid return, meaning ‘process pending’; see pp_errors.h for the full list.

2) Notice all the UDP functions take a callback, it’s worth noting that those callbacks will always be called, even if a function fails. If a failure occurs, the datasize param will have a value of -2

3) The IP address that you use to bind your client to should be generated via calling NetAddressPrivate::GetAnyAddress; Once done, you should call NetAddressPrivate::ReplacePort to update the address with your desired port. Doing so you will bind the UDP port on all network interfaces for the system.

4) To connect to your server, you should be using CreateFromIPv4Address . For testing purpose you can bind localhost (127.0.0.1) with some port.

5) Note that the RecvFrom takes in a buffer parameter, but due to the async nature of the call, it is possible, but incorrect to make multiple RecvFrom calls on the same socket while a previous call is in flight. Chrome can write async to this buffer at any time, causing your data to get mangled. IE. Simultaneous outstanding reads are not possible. The same principal applies with SendTo, you have to wait until previous data portion sent and buffer should stay valid until callback is called. You can, however, call RecvFrom and SendTo can be called in parallel on the same socket without any issues.

6. get ready for the roll out

Chrome Apps are still in early development phases, and when they release, will have full support for UDP sockets w/o needing flags or special params. There will be some minor hoops to jump through that are not yet documented, so stay tuned to Chrome for more!

Source Code

You can grab source code to a simple win32 client communicating to a NaCl platform app via UDP on my github repo.

The example project works by allowing the client to send a message to the server. To use, launch the server first, then launch the client. The server will receive a message that it displays in the console window.

For reference, I launch Chrome with
--allow-nacl-socket-api=127.0.0.1 --no-sandbox --load-extension="C:\nacl_examples\src\UDP_private\client" --enable-nacl

~Main

You can find Colt McAnlis here:

NaCl & Writing files outside the sandbox

2012-08-24T11:14:00.000-07:00

Wouldn't it be great if you could do some processing in NaCl and save the results to a file outside of the sandbox? Good news! This article shows you how.

Putting things in a sandbox is the easy part; Getting them out is often the difficult part; Especially when you don't want dirt all over the carpet..

In a previous post, I talked about how you can load a file into NaCl from OUTSIDE the Chrome sandbox. Which is a great thing for those of you who want to create content-editors or content-viewers inside of Native Client.This time, we'll talk about how to get those files from NaCl back out to the disk.

The bad news is that Native Client does not expose an API that allows you to write files outside of the sandbox.

The good news is that HTML5 does.

This means that if you have data inside of NaCl that you want to write to disk, you first need to pass it over to Javascript using the PostMessage API.

1. Saving a file to disk in HTML5

HTML5 exposes a download attribute which does the process of writing a file to disk from Javascript. When used on an anchor, this attribute signifies that the resource it points to should be downloaded by the browser rather than navigating to it. It requires a user click, but will allow the data to be streamed.

Here's a Javascript example of how to create a BlobBuilder, and create a link that will let you download some arbitrary data

function downloadData function(fileData, outputName){

  var bb = new BlobBuilder();
  bb.append(fileData);


  const MIME_TYPE = 'text/plain';

  var a = document.createElement('a'); //create a new element for the user to click on.
  a.download =  outputName ;
  a.href = window.URL.createObjectURL(bb.getBlob(MIME_TYPE));
  a.textContent = 'Click Here';
  a.dataset.downloadurl = [MIME_TYPE, a.download, a.href].join(':');

  document.getElementById('buttonDiv').appendChild(a);
}

You can set up this process in Javascript before needing to touch Native Client. That way you can get everything working at a functional level, and then just push the data from NaCl into it when ready.

2. Getting binary data from NaCl to Javascript

To get our binary data to Javascript from NaCl, we need to do 2 things:

Create a varArrayBuffer to hold the binary data
Push the binary data back to java script via PostMessage.

The example code below shows this process, and is using the C Interfaces for PPAPI.

PP_Var v2 = ppb_varArrayBuffer_interface->Create(dataSize);

//we map the array buffer and copy the input data into it.
void* pDst = ppb_varArrayBuffer_interface->Map(v2);
memcpy(pDst,pData,dataSize);
ppb_varArrayBuffer_interface->Unmap(v2);

//send the data to Javascript, using Postmessage
ppb_messaging_interface->PostMessage(appInstance_,v2);

3. Source Code

You can find a working example on my Github repo. This example will allow you to load data from disk, as well as write it back to disk.

Note that this uses the Visual Studio add in (which is not public yet), but you can by-pass that and compile the C files with the make data that comes inside the NaCl SDK.

~Main

You can find Colt McAnlis here:

Native Client and loading a user-chosen file from disk

2012-07-27T13:38:00.004-07:00

I get this question quite a bit: How do I load a video/image/audio from the users' disk in Native Client? The answer is 'easily' if you follow these 3 simple steps.

The LandShark teaches us great principles about how to access the world outside your bounded environment. Much like loading files from outside the NaCl sandbox.

Recall that Native Client has two primary ways to deal with files:

Fetching the data from a web service, using geturl
Reading/writing the data to the local filestore

These are the safe ways allowed through the sand boxing process to read/write files. Of course, there's also the non safe way of reading / writing files, and the way that bypasses the sandbox; but that's dark magic.

HTML5 recently exposed a method for which a user can specify a file/files from their disk (using a file-picker) that HTML5 could then load into memory, and manipulate. It turns out, that using a bit of a trick, you can open these files directly with NaCl.

1. Prompt the user to pick a file

In Javascript, allow the user to pick a file by adding this HTML to your page. (I cheated by copying some code from Eric ;)

           
<input type="file" id="files" name="file" />
<output id="list"></output>

<script>
  function handleFileSelect(evt) {
    var files = evt.target.files; // FileList object

    var f = files[0]; // the first selected file    
//.....
  document.getElementById('files').addEventListener('change', handleFileSelect, false);
</script>

This above block will add a nice little 'Choose File' button to your HTML page, that when clicked, will open a FileChooser dialog that the user can pick a file. (note, the code above is not safe, and doesn't check for the proper edge cases )

2. Generate the BlobURL

Once we have the file object, we can use a specific API to generate a URL encoded path to the file.

 function handleFileSelect(evt) {
    var files = evt.target.files; // FileList object

    var f = files[0];
    var objectURL = window.webkitURL.createObjectURL(f);
    var naclObj = document.getElementById('blob_load');
    naclObj.postMessage(objectURL);
    
  }
  document.getElementById('files').addEventListener('change', handleFileSelect, false);

The createObjectURL function will generate something like:

blob:http%3A//localhost%3A5103/6c941740-dc7e-4ec7-98e7-876b81e943a0

Which we can then pass directly to the NaCl module using the postMessage API.

3. Load and process the file using Native Client

On the NaCl side of the fence, we can receive the message from Javascript (via postMessage) by using the proper message handler function. The one secret here is properly converting the message from a PP_Var to a CString (check the NaCl examples on how to use VarToUTF8 properly). Second, we call getURL on the object to load it (see the geturl example in the SDK for the proper source code to this).

//Main.cpp module, C++ code
static void Messaging_HandleMessage(PP_Instance instance, struct PP_Var message) 
{
    uint32_t len;
    const char* blobURL = VarToCStr ( message);
    //kick off loads from our files
    readFileFromURL(blobURL,onFileLoaded );
}

Note that readFileFromURL function above is mearly a wrapper to the larger getURL opeartaion; it will properly kick off the getURL command, and respond to multiple reads and responses of grabbing the file, and calling the proper callback (onFileLoaded) when complete.

And that's it! With this simple setup, you can easily load audio, video, and images from the user's disk to do processing in Native Client!

Go forth and be awesome.

~Main

You can find Colt McAnlis here:

Running NaCl on Facebook

2012-07-09T14:28:00.001-07:00

Native Client is an amazing technology that allows you to run C++ on the web. With the current Chrome Web Store restrictions, it limits some of the functionality of getting your NaCl apps to the masses. This post will talk about a nice little work-around to the CWS install process that will allow you run your NaCl apps on Facebook

Jumping through hoops is the basic working attribute of the web-programmer. For dolphins though, it's their way of saying 'I'm better than you.'

WARNING:
The future of this feature is currently unknown; Please be warned that this functionality may be removed from Chrome in the future if it's deemed a security concern.
Please implement this functionality at your own risk.

Understanding Chrome Web Store and Native Client

Firstly, understand that the Chrome Web Store (CWS) restriction in running NaCl on the web is a multifaced problem; For our immediate needs, let's say that we use it as a safety measure, so that, through policy, we can remove any potential malicious code from the web, and also have a chance to reach out to developers when PNaCl comes out.

As such, the main hurdle running NaCl on sites like Facebook has less to do with NaCl, and more to do with getting around CWS install issues.

As a bit of background reading, I reccomend you read distributing your application on the NaCl developers page, which will highlight some terminology we'll be using.

Currently, the user has two main ways to install a Hosted CWS application

Browse to the chrome web-store, find your app, and hit 'install'
Browse to your url (blah.url.com) and click the inline install button

The inline install process is nice, because it allows the users to install right from your page, without having to leave it.

Installing a CWS app from facebook

A facebook game page, (much like a G+ games page) works by nesting an iframe on the desired page, which loads the game itself. One of the large issues with CWS inline install, is that it doesn't work when served from an iframe, because the origin calling the install is registered from the outer frame's domain not the inner one. As such, CWS install would fail, since we're trying to install an app from a different domain than what the app defines in the CWS dashboard.

There is a nice work-around though, in that you can open a popup and install from there.

As such, here's what we hope to achieve as the best-case scenario:

The user will open your games-page on facebook

ie apps.facebook.com/mainroachgame

If the user has never installed the app,

then a popup window will appear allowing the user to inline-install your app
The user will be able to play your NaCl game on facebook now.

If the user has the app installed

then it will magically load, and no one will know the crazy hoops you've had to jump through.

Inline installation from the inner-frame

Although we can't do inline-install from the inner-frame, we can, however, do inline-install from a popup window.

In the CWS dashboard, define the URL for the app to your domain

ie chrome.mainroach.net

Create a page on your domain, which holds the proper HTML for inline install.

ie chrome.mainroach.net/app_install.html

Create a page on your domain, which hosts the NaCl app.

ie chrome.mainroach.net/index.html

Your NaCl page should first test if the app is already installed before adding the embed tag to the page

you should use the new API : chrome.app.installState(callback), which will return "installed" "not_installed" or "disabled"

If the result is 'not_installed", then open a pop-up window pointing to your install url

ie chrome.mainroach.net/app_install.html
NOTE, that pop-up windows are normally blocked, unless initated from a user-action. So adding a nice 'Click here to enable this game' button works wonders

In the Facebook app dashboard, point the URL to your nacl page

ie chrome.mainroach.net/index.html

Here's some code!

file : chrome.mainroach.net/index.html

 <script>
var installStateNode = document.getElementById('install-state');
    if (chrome.app.installState) { //this API is only available in chrome 20+
      chrome.app.installState(function(installState) {
  if(installState == "not_installed")
  {   //if the app isn't installed, show the 'install' button
   document.getElementById('not_installed_btn').show();
  }
  else
  {  //otherwise, append the NaCl embed tag to things, and get running!
   appendNaClEmbedToDOM(...);
  }
      });
    } else {
      alert("this requires chrome 20");
    }

Later on, we need some control of a popup to show the user; so we define a div, containing a button which will launch our inline-install window.

  function popupInstall()
    { //please refer to the inline-install documentation for what this page should contain.
     window.create("./inline_install.html",...);
    }
    
  
</script>
  <!-- blah blah blah -->
  <div id="not_installed_btn">
<a href="" onclick="popupInstall();">Install some stuff</a>
</div>

Notes:

Please note that this is a pretty gnarly hack of the CWS install process in order to run NaCl on Facebook. Remember, Portable Native Client, or PNaCl (pronounced 'pinnacle') will allow you to run NaCl apps in the open web without the need to install from CWS. Expect more info on that, soon.
Recall that this only works for hosted NaCl apps, which, depending on the size of your assets may not be ideal, when compared to a package app, which most NaCl devs are drawn too, since hosting is free.

~Main

You can find Colt McAnlis here:

Chrome Games review at Google I/O

2012-07-02T14:31:00.003-07:00

For those of you who missed it, Google I/O was last week, and boy was it a tornado of awesome. This was the first IO (speaking at, or attending) so it a was a a rush of Google info, products, and announcements that we've been working so many months on. The conference was great, and it was an amazing time to talk with web developers in the wild doing amazing things.

New Games announced for HTML5 and Native Client:

Square Enix announced Lara Croft Guardian of Light for Native Client

Venture Beat was a fan.

Gaikai showed off their Chrome build in the sandbox

And also during the Keynote
And with some other press.

Klei Entertainment opened up Don't Starve for beta testing
Disney was there, showing off their new HTML5 version of AgentP
Halfbrick rocked the house showing off their new HTML5 game BandStars
Spryfox and crew were there with their new RPG game, Sparrow
EA showed off a new HTML5 game Strike Fortress

Which the NY Times thought was pretty cool

Talks
Native Client Live - VIDEO
In this talk, we will be porting an application to Native Client in 60 minutes, LIVE; showing the power of what Native Client can provide for traditional C++ developers looking to move to the web. In the porting process we'll cover specific tasks that a developer would need to perform during a port, and how to to address them with new tools and technologies including debugging integration with Visual Studio and a set of newly added utility libraries to the SDK.

GRITS: PvP Gaming in HTML5 - VIDEO

This session will present GRITS, a player vs. player shooter game, built entirely using Google technologies. In this talk, we'll walk through building an HTML5 canvas engine, serving the content, networking using Websockets, using NodeJS, social integration and more.

Gaming in the cloud - VIDEO

Many games developers are finding the easy development and deployment experience of Google App Engine ideal for
building cloud based state-storage, matching making services and collaborations services. When you have a hit game, the last thing you want to do is worry about your server provisioning. App Engine has an always-free tier to get you started and then scales seamlessly to any size of usage. Game developers also use Google Cloud Storage to easily store and quickly deliver media files to clients around the world.

~Main

You can find Colt McAnlis here:

Write once, run anywhere?

2012-05-22T09:24:00.000-07:00

For game developers these-days, one of the most important things is increasing the top of the user-funnel to reach more potential customers. 'Write once, run anywhere' they say. Is it a pipe-dream? Do developers have to throw-away their code investments for these new platforms?

Truth is, you can run your existing code almost anywhere.... if you're willing to get a little muddy..

Language is important for development studios, often there's a great deal of investment in a given development language, starting at the training of the employees to it (ie, you have a team of C++ veterans) or an existing code base ("We've shipped 8 games in C++, and have shared libraries!").
One consistent theme I find when talking to developers is that it's not well understood what options are available to them to get their existing product out in the wild.

So I put together a nice little table that talks about it. Below you will find languages plotted against popular distributions / markets, and if that language runs there.

When you graph your desired language against a given distribution platform, you realize that your options as a game developer are much larger than you would expect them to be.

Some notes:

HTML5

iOS - You can run HTML5 games in PhoneGap, or Safari
Android - You can run HTML5 games in PhoneGap, or Chrome
Steam - You can run HTML5 games with Chromium Embedded

It's worth noting that C++ for mobile comes with some restrictions (like UI access)
C++ can run on G+ w/o the need for a plugin (With Chrome/NaCl)

If not using Chrome, can run with a plugin

C++ can run on Facebook

If you're running chrome, then you need to jump through some inline-install hoops
If you're not, then you can run with a plugin

You can run C++ in FF/IE, but requires a plugin :(

Java

Java mostly runs everywhere as a plugin, except iOS :(
Steam/Desktop needs java installed,
Android has the SDK which is focused on java

The results are quite encouraging; the key take-away here is that multiple languages give you write-once-run-anywhere support, if you don't mind jumping through some hoops. With this type of realization, Game developers should breathe a bit easier:

Your technology isn't going to become out-dated any time soon.

~Main

You can find Colt McAnlis here:

How to auto populate the G+ Sharebox with custom event data

2012-05-17T09:17:00.000-07:00

I've been looking for this functionality in G+ for a long time: How do i generate a unique URL with unique data and allow a user to share that to their stream? Fellow Developer Advocate Paul Kinlan clued me onto some cool things he was doing with auto-share functionality, so I decided to scrape a demo together. This is pretty small code, So I'm going to let it speak for itself, rather than blabber on about this ;)

Sharing. It's the cornerstone of the social web. Web games can't survive without it.

Here's the gist:

On your server (App engine in my case) create a directory path that will render a page when accessed
The server-rendering of that page will include the proper G+ story metadata so that you can control what goes into the stream
On the client, we use the explicit parse mode for the G+ Share button to dynamically generate a url to the server location (with params)

See it live!

Client Side

Add this snippet inside of your HEAD tag:

<script src="https://apis.google.com/js/plusone.js" type="text/javascript">
   {parsetags: 'explicit'}
</script>

Add this where you want your share button to be located:

<div class="g-plus" data-action="share" data-height="24" id="sharebut">

And here's the javascript to generate the 'achievement' url and assign it to the share button


   //generate some random string data
   var url = "http://mainroach.appspot.com/sharetest";
   url += "?type=" + Math.floor(Math.random() * 2.999);
   url += "&userID=ABCDEFGHIJ";
   url += "&custom_data=Now I am going to google-land!";
   
   //add our share button
   var divshare = document.getElementById("sharebut");
   divshare.dataset["href"] = url;

   //this will call the G+ api to raster the share button with the right data  
   gapi.plus.go();

Server side

Server side, when a request comes in, we use the data to dynamically generate a server page that fills out the meta data that G+ will use to scrape for the story. Since the client passed in the type of achievement, as well as some achievement data, the server responds by displaying separate images for each achievement type.

On Google App Engine with Python:

class ShareHandler(webapp.RequestHandler):
 def get(self):
  achvtype = self.request.get('type')
  userID = self.request.get('userID')
  eventData = self.request.get('custom_data')
  self.response.headers['Access-Control-Allow-Origin'] = '*'
  self.response.out.write("<html>")
  self.response.out.write("<h1 itemprop='name'>Achievement Unlocked!</h1>")
  self.response.out.write("<img itemprop='image' src='share_test/shareimg" + achvtype + ".jpg'></img>")
  self.response.out.write("<p itemprop='description'>My Level 5 pet rock just unlocked a new mount thanks to <a href='http://mainroach.blogspot.com'>mainroach.blogspot.com</a>. " + eventData + "</p>")
  self.response.out.write("<meta http-equiv='refresh' content='0; url=http://mainroach.blogspot.com/'>");
  self.response.out.write("</body></html>")

Note that the final line in this snippet adds a refresh tag to the page. My intent is that the shared URL isn't important enough for the user to visit, and is just a story harness, so when the page loads, I redirect you to my blog.

Once the user shares, it will appear in the G+ stream just like any other proper stream post:

Once we got this working, I was excited!

And that's it! Go web-game developers. Be awesome and share!

~Main

You can find me here:

Porting to Native Client with Visual Studio pt.4

2012-05-07T07:55:00.000-07:00

This article series shows you how to port to Native Client via a Chrome Trusted Plugin. This will allow you port your platform code over to Chrome's Pepper APIs while using Visual Studio for debugging.

WARNING:

As of 10.1.2012, there's an official version of a Visual Studio 2010 plugin. Only some subset of this article series is accurate, please see the official documentation at gonacl.com for more.

Episode 4: imma chargin mah nacl!

The 4th, and final installment of this series will fill in the last piece of the Native Client developent puzzle : Actually compiling to native client. Until now, we've done lots of work to develop and target Chrome's Pepper APIs by developing as a trusted plugin. This process has allowed us to use Visual Studio's compiler and debugging process to develop on, but alas, this is not the same for Native Client.

Some enterprising young soul hacked this bottle of Rum to be awesome. If you've been following this series, you too have hacked awesome to make even more awesome; by running C++ code on the web.

NaCl provides a separate command-line GCC compiler that produces safe executable files to be used for the Native Client Plugin. So, in order to continue to develop inside of visual studio, we're going to have to make some modifications so that we can send our build arguments to NaCl's GCC compiler, rather than VS's.

Unofficial Hack-tastic Visual Studio Plugin
As expected, there's already some work in this area; Where this guy modified a version of an Android Visual studio plugin to support Native Client. That version hasn't been updated in a while, and as Developer Advocate for NaCl, I feel it's my responsibility to copy that version, make some changes, and keep it up-to-date ;)
So, as a first step, close your visual studio, and grab the native client vs2010 plugin from my github repo. Download it to a safe, secure location, and then:

Copy the 'addin-vs2010/NaCl' folder and the 'addin-vs2010/PPAPI' folder to your ms build folder:

C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Platforms
//so you now have
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Platforms\NaCl
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Platforms\PPAPI

Now, load up Visual Studio, and let's get started!

Using the plugin
What the plugin does, is allow for you to define a seperate platform to build against; As such, you'll need to create a new platform in the build->configuration manager

You can add a new "NaCl" platform that your project can target.

Depending on your project layout, how you configure your project, solution, and platform build layouts is up to you. I suggest taking a bit of time to do that now. In reality, we should be providing both a PPAPI and NACL platform targets, (so that you can choose debug/release for your build config, and win32/ppapi/nacl for your platform) but that is a bit besides the point ;)

Once you've got everything set-up, it's time to set some of the properties on the project. Opening the properties tab, you'll notice a few things are different from a standard windows build.

Set the nacl compiler to these settings. Most of them should be set up for you by the plugin, but just to be sure, I list the important ones here :

General> Configuration Type:

Application(.nexe)

General> Target Architecture:

x86 64-bit

General> Target Platform

pepper_18

Note, as more pepper versions are released, you'll need to manually change this string

Debugging > Command Arguments:

--incognito "localhost:5103"

Reminder -incognito turns off caching so we're sure to get the latest changes each time we launch the debugging process.

C/C++ > Preprocessor >Preprocessor Definitions :

PPAPI;NACL;.....

Linker > General >Output File :

$(ProjectDir)$(TargetName)_x86_64$(TargetExt)

Eventually you'll need 32 bit and 64 bit build targets, but I'll leave you to handle that ;)

Linker > General >Output File :

$(ToolchainDir)\x86_64-nacl\lib

Linker > Input >Native Client System Libraries :

ppapi_gles2;ppapi_cpp;ppapi

Removing platform-specific code
If recalled, the PPAPI define allows you to run both pepper and windows calls. Especially when getting errors, (like loading shaders and what not.) We'd like to keep this functionality so we can swap between the builds for easier debugging, so we add a new preprocessor define NACL, which should encompass any differences between PPAPI and Native Client.

The bulk of this work was directly changing the MessageBox usage in the program, and forcing it instead to target our PostMessage function.

#ifdef NACL  
     PP_Var v = CStrToVar(&str[0]);
     ppb_messaging_interface->PostMessage(appInstance_,v);
#else
     MessageBox( NULL,  &str[0],"GLERR",MB_OK | MB_ICONEXCLAMATION );
#endif

One specific instance of where the NaCl instance derived from the PPAPI instance is wrt a location of the gl2ext_ppapi.h header:

#ifdef PPAPI
. . .
#ifdef NACL
     #include "ppapi/gles2/gl2ext_ppapi.h"
#else
     #include "ppapi/lib/gl/gles2/gl2ext_ppapi.h"
#endif //NACL
. . .
#endif //PPAPI

Creating the .nmf file

Native client requires you to define a manifest file (served alongside the html file) which points to what executable to use / load for each processor type. There's some nice tools in the SDK that will auto-generate these, but for clarity sake, here is the one provided for this application. Note that the VS Plugin will NOT generate these for you!

{
  "program": {
    "x86-64": {"url": "plugin-port3_x86_64.nexe"},
    "x86-32": {"url": "plugin-port3_i386.nexe"}
  }
}

Debugging

It's worth noting that once you're running your NaCl application from visual studio, you loose Visual studio Debugging abilities. You can fix this by following the debugging steps to get WinGDB working; but hopefully you can debug everything as a trusted plugin, and don't have to worry about it. There is a Debugger for NaCl, but the integration process with visual studio still needs quite a bit of work. As such, I suggest you avoid it at most costs ;)

Notes about provided code

Please note that the Visual Studio plugin I provide is without warranty. It is not an official Google product and holds no expectations of being such. Please hack on it at your will, and if you have problems with it, toss them back at me. Also note that I've added a PPAPI platform as well, this should make it easier for your configurations, where you can keep your debug/release/testing builds against and target PPAPI and NACL as separate platforms.

Note- you can download the source code for this series @ my github repo.

The rest of the series -
Part 1 : Setting up visual studio
Part 2 : Using platform-specific code
Part 3 : Porting to Pepper
Part 4 : Compile to NaCl

Article Conclusions:
This article series has been a great chance to go through the trenches of NaCl development from the standpoint and POV of a random windows developer. Hopefully, this series has made it easier for windows developers to understand Native Client and successfully port over their projects. There's been lots of feedback in this process to the NaCl SDK team, and there's already planned changes on the horizon that will make some points of my articles obsolete. (for example, including the PPAPIs inside of the NaCl SDK, so you don't have to grab chromium!) When those changes land, I'll be sure to update this article series with the relevant warnings.
Thanks for reading. Feel free to send me feedback on my G+ page.

You can find me here:

~Main

Porting to Native Client with Visual Studio pt.3

2012-05-03T07:26:00.001-07:00

WARNING:

As of 10.1.2012, there's an official version of a Visual Studio 2010 plugin. Only some subset of this article series is accurate, please see the official documentation at gonacl.com for more.

Episode 3: Tigers Love Pepper

In the 3rd installment we're going to take the dive and port from a hybrid application to getting OpenGL rendering inside of the webpage! As we mentioned in the first article, Pepper is an API that allows plugins to communicate with resources in Chrome, so for us to get our OpenGL win32 app rendering in the Chrome webpage, we need to port all of our Platform code, and rendering code to Pepper.

The article this time is pretty code specific that's a combination of GL and FileIO, so I'm going to only highlight the important points, and assume that you're a sane programmer, who is reading code alongside my rambling ;)

Tigers love Pepper. This is well known. The Tiger above thought there was pepper under water, and decided to swim to get it. This article seeks to make your code more tiger friendly by merging it with the Pepper APIs. This will make your code less lion friendly though, as they prefer to snack on small zoo-going children.

We are going to avoid compiling Chromium
I realized I hadn't been verbose about this just yet, but my intent in this series is to avoid compiling chromium at all costs. Chrome is a massive code base with lots of tendrils, and taking time to compile them in order to develop your plugin is great deal of overhead that isn't worth dealing with.

One of the things we're doing to help with this is by using the C version of the Pepper headers. These headers won't require you to compile against a static lib in order to compile your application (for the most part) which keeps you from compiling Chrome, and helps improve iteration time.

That being said, there's some nuances with the process that we're going to highlight in this article that will require your code to behave a specific way that is unique to the PPAPI plugin process, that DOES NOT match a shipping NaCl product. Specifically how OpenGLES2 behaves differently as a Trusted Plugin than it does as a NaCl app. So make sure you weigh the code changes you'll have to make against compiling Chrome. Note that if you choose to compile Chrome, I offer no warranties on that process ;)

Laying the groundwork
Firstly, the code wraps some PPAPI specific merges in a #define PPAPI at the top of the file. Most of the windows-specific items, like creating the win32 window, is compiled out while this flag is defined to reduce craziness; My intent is to keep the windows functionality for this program so that we can create separate targets correctly.

It's also worth noting that we are intentionally allowing this project to be single threaded. As I noted in my GDC talk, it's much easier to spin up a separate thread and communicate back to the Pepper thread, but the extra code for multithreaded communication I felt was a bit outside the scope of this article.

There are a few things that I will cover however, which are specific to the Pepper API

Pepper can only be called from the main thread (I call this the 'Pepper thread')
Pepper APIs are required to be non-blocking

Note there are some NaCl calls (like dlopen) that are blocking so you have to manage them with care

There is no 'main loop' in Pepper. Pepper module functions are triggered from javascript pulses.

Loading Files with Pepper
First, we're going to replace the file-loading code. Up till now, we were defining explicit locations of files, and taking advantage of the --no-sandbox flag. Since this isn't going to be available in a shipping product, we need to migrate our fileIO calls to PPAPI. Note, that I've talked about how the file-system(s) in Native Client work I've also talked about how the Pepper APIs need to come from the main thread, and why fileIO is so difficult in this process. So make sure you check up with those before moving forward.

NOTE :Although the example code includes a solution on how to load files from disk, it by no-means should be considered best practices. In short, I'm going to discuss some of the changes I made, but skip over the larger discussion of how the APIs work.

As discussed in my article on NaCl and File systems, file data is firstly hosted from a server, and as such, we need to kick of an HTTP Get request to fetch the byte stream so that's usable for us in the code. For the specifics of this, I refer you to documentation on ppb_url_loader.h, which describes how to properly use the interfaces provided by Pepper. I will note, however, that using the API's is a 4 step dance:

You must 'open' the file, passing some sort of struct to hold request data.
Once the file is 'opened' you get a callback, and must then issue a 'read' command to it
Once you've read some data, you get another callback

You must check if the data has been read entirely.
If not, then you need to re-issue the 'read' command again.

When you're done, clear the buffers and close the file

Preloading our assets
The fact that the file APIs must come from the main thread, and that they are nonblocking presents a problem for us, as our existing code expects fread to block. There's many solutions to this problem; the one I will present, for simplicity's sake, is to preload the files before creating any objects needing the file data.

To do this, I first create a new callback loop function, called loadloop. This function will poll if the files have been loaded by checking a file-loaded counter, and re-issuing a callback if we're still waiting for files.

void loadLoop(void* pData, int32_t dataSize)
{
   //if the resources aren't loaded yet, then kick off another callback
    if(g_ResourceLoadedCounter != 3)
    {
       PP_CompletionCallback cc = PP_MakeCompletionCallback(loadLoop, 0);
       ppb_g3d_interface->SwapBuffers(graphicsContext_, cc);
       return;
    }
 ...
}

Because file IO needs a callback system, I provide a very hack-tastic solution that each of the files has a separate callback function to load it, which will copy the response data from the server into some global objects, and increase the file-load counter.

//glboally define the data; a file manager is outside the scope of this..
char* textureData;
unsigned char* pVSData;
unsigned char* pPSData;

//this function loads the vertex shader data
void load_vsCB(void* pData, int32_t dataSize)
{
   if(dataSize <=0)
    return;

    //note that the shader compiler expects this to be a null-terminated string
   pVSData = new unsigned char[dataSize+1];
   memcpy(pVSData,pData,dataSize);
   pVSData[dataSize]=0; //null terminate this string
   g_ResourceLoadedCounter++;
}

Once all the data is loaded, g_ResourceLoadedCounter will be set to the number of loaded files, and thus loadLoop will call init(), creating the GL context, shaders, and textures, and then kicking off our first render function loop


void loadLoop(void* pData, int32_t dataSize)
{
    //if the resources aren't loaded yet, then kick off another callback
    if(g_ResourceLoadedCounter != 3)
    {
      ...
    }

    //if everything's loaded, then init
    init();

    //kick off rendering loop here
    //now that everything's loaded, kick off our first render frame
    PP_CompletionCallback cc = PP_MakeCompletionCallback(render, 0);
    ppb_g3d_interface->SwapBuffers(graphicsContext_, cc);
}

Again, note that there are many solutions to this problem. Where the more advanced ones require you to modify your code less. For example, spinning up your loop to a worker thread, and then emulating fread by blocking your calling thread until all the execution on the Pepper thread finishes; this could be hidden by a nice modular interface, and not require any of the ugly hacks I've presented here.

Converting the Rendering API

Sadly, we can't 'slowly' convert from GL to GLES2, we kinda have to jump all-in; I am going to avoid as much of the conversation of GL->GLES2 as I can as this has little to do with nuances of Pepper directly, and more to do with graphics API weirdness. As such, I'm going to focus directly on the parts that are relevant to PPAPI.

Adding the right files
Although we're using the C version of the Pepper APIs, some of the GLES2 APIs that are Pepper Specific still fall through the cracks. The bad news is we need to modify our paths and add some files, the good news is that we don't have to include all of chrome ;)

Add these two files to your project:

(Some Path)\chromium\src\ppapi\lib\gl\gles2\gles2.c
(Some Path)\chromium\src\ppapi\lib\gl\gles2\gl2ext_ppapi.c

In addition, you'll need to add a new include path for the pepper GL headers, make sure you add the following path to your include directories

C/C++ > General >Additional Include Directories :

(Some Path)\chromium\src\ppapi\lib\gl\include

Note, you DO NOT need to link to the GL libraries (opengl32.lib,glu32.lib) any longer. Please remove thier linkage from this configuration

Creating a rendering callback loop
As mentioned, there is no 'main' loop with PPAPI. It's a callback driven process that is initiated from the external code into your Pepper Plugin. Luckily, the GL APIs come with a callback function that allows you to chain these render commands together using callbacks.

//was void render() 
void render(void* pData, int32_t dataSize) 
{
    ...
   //here be code that renders Dragons.
   ...

  //call a swap-buffer, and when it's done, call render again
  PP_CompletionCallback cc = PP_MakeCompletionCallback(render, 0);
  ppb_g3d_interface->SwapBuffers(graphicsContext_, cc);
}

What the above code will do, is that the render loop, when finished, will kick off a SwapBuffers command to openGL, and when it's done, submit a callback back to the render function, which will subsequently render, and callback again. This is as close to a while loop that we get when using PPAPI in a single threaded mode. Note that loadLoop kicks off the very first call to render() once all the initialization is completed.

Warning: No Client Array Support - trusted plugins only
There's some crazy nuances with Trusted Plugin development using Chrome.
Looking at the source code, there's a defined flag in gles2_implementation.cc called GLES2_SUPPORT_CLIENT_SIDE_ARRAYS which defines a certain behavior for plugins. Mainly when defined, it allows OpenGL ES2.0 to use client-side arrays; the type of arrays that allow you to call glVertexAttribPointer without having a Vertex Buffer Object bound first.

Normally, most high-performance applications use VBOs for the bulk of their rendering anyway, having dynamic rendering only used for debugging, or some UI specific code.

One way around this, which adds this functionality back into your plugin, involves compiling the portions of chrome effected by GLES2_SUPPORT_CLIENT_SIDE_ARRAYS into the plugin itself, so that it's allowed to define how the communication pattern works to the GPU command buffer. So this is a bit worse than compiling chrome, it requires you to compile a subset of chrome, and use that in your plugin. Which, hopefully, doesn't pull in all of chrome along with it.

This behavior is not the same for Native Client

The NaCl plugin properly compiles in the needed assemblies with the above flag, and thus allows client-side arrays to be used for Native Client development. Again, you should carefully weigh the pros/cons of your development

Warning: Chrome has it's own GL errors :(
Chrome, in an attempt to increase security, has a federation of checks and testing that occur on your GL calls before they make their way to the GL APIs, and report errors back to the java script console. These custom errors will throw a valid GL error, alongside some Chrome-specific debugging information. Note that if you Google for the exact phrase output in the error window, the only hits you'll get will link you directly to the chrome source code :(

Some common errors you get in the application if you don't call glBindBuffer before doing a draw call

Booyah

After porting your fileIO and Rendering code to Pepper, you should be able to see the quad rendering on the page.

Conclusion
This was one hell of an amount of heavy code to lift. Porting to Pepper accounts for a large amount of your time and energy, and shouldn't be taken lightly. In reality, you should approach Pepper APIs with the proper set of abstraction layers and code buffers to reduce the amount of in-line code that you have to change in your codebase.

It's worth noting that this article series has done quite a bit to illuminate the plight of the average NaCl developer to the teams here @ Google. As automated fixes come into the SDK for these things, I'll update this documentation.

Notes about the provided code

I've added two build configurations to the project to help toggle-swap your port efforts

debug-ppapi
debug-win32

Swapping your config between the two will either launch a win32 window, or a Chrome using PPAPI

Note- Sane, deity fearing developers would never port in the manner that I have in the provided code. They would write proper wrappers, abstraction interfaces etc. My goal is to provide as few source code files as possible to show off the intended techniques for porting to NaCl. My goal is not to show off good programming techniques, ethics, or best practices. YMMV ;)

Note- DISABLE YOUR CACHE. While you're developing, Chrome will pull in the cached shader and texture data, and sometimes even nexe files. As such, you should disable your cache while developing, or run in incognito mode (you can specify this with a command line string --incognito)

Note- that the debug-ppapi build config does not handle input to rotate the quad; I leave all 'input handling' in this article as an exercise for the reader ;)

Note- you can download the source code for this series @ my github repo.

The rest of the series -
Part 1 : Setting up visual studio
Part 2 : Using platform-specific code
Part 3 : Porting to Pepper
Part 4 : Compile to NaCl

You can find me here:

~Main

Porting to Native Client with Visual Studio pt.2

2012-04-30T07:02:00.000-07:00

WARNING:

As of 10.1.2012, there's an official version of a Visual Studio 2010 plugin. Only some subset of this article series is accurate, please see the official documentation at gonacl.com for more.

Episode 2: The platform-specific hope

In the 2nd installment of my soon-to-be-peabody-nominated-series I'll walk you through how to call platform-specific code from our trusted plugin, and get it running alongside our pepper api; This will be a setup for article 3, which will talk about how to port us from the platform specific API code to Pepper. Less pictures this time, more code ;)

I'm assuming that you've already grabbed the first article's code from github, and that you've properly read through the first article such that I don't need to repeat any of that.

The Waffle Burger. The Unholy alliance of breakfast and dinner that fueled the 200 year war.
It is from this brave delicacy that we draw our inspiration in this article.

Getting some Platform-Specific code
Most of my work with Native Client developers includes game devs specifically, and their biggest pain points revolve around file IO and rendering; as such, it makes sense that we should target something difficult, yet easy to deal with.

As such, go grab a nice openGL sample which covers both OpenGL, and file IO from Codesampler.com, download it and unpack it somewhere.

Once done, copy the entire contents of ogl_glslang_simple_vs2ps.cpp from the sample and paste it at the TOP of our existing plugin-port0\main.cpp file;

In addition to the file contents, you'll also need to copy over the following additional files:

glext.h - This is a file containing headers needed for this OpenGL example
vertex_shader.vert
fragment_shader.frag

Place these files alongside your plugin-port0\index.html file

Getting it compiling

Assuming you've pasted the code starting at line 0, you can easily make the following changes:

line 29: //#include "gluax.h"
line 30: //#include "resource.h"

Note you should set your characters from unicode to multibyte in the project settings, otherwise the first time you compile you'll get erros relating to LPCWSTR; Be aware that this isn't NaCl specific, it's just how this particular plugin is being handled.

Adding the GL libraries

OpenGL requires us to link to specific precompiled libraries to render properly; Let's link to them, and celebrate our ability to use Visual Studio in the same breath by using some nice VS magic that keeps us from having to modify the properties pages. Add these lines after the includes of your file.

#pragma comment(lib,"opengl32.lib")
#pragma comment(lib,"glu32.lib")

It compiles! Let's get it doing something!

Updating the main function

What we're going to do here, is modify the source code we have to pop up the existing code window that we would expect to see and run it side-by-side with our Chrome window. As such, we need to change the function header for the window creation online line 112:


init WINAPI WinMain(HINSTANCE hInstance,HISNTANCE hPrevInstance, LPSTR lpCmdLine, int nCMDShow)

with:

int initInstance()

"Simplicity is a dish best served with nothing on it"

Calling the main function

Now that we've modified the main function, we need to call it from somewhere. Place a call to the window creation inside of Instance_DidCreate in order to launch the window

static PP_Bool Instance_DidCreate(...){

   PP_Var v = CStrToVar("Hello a World (NEWLIB)");
   ppb_messaging_interface->;PostMessage(instance,v);

   initInstance();
   ..
}

Reading files from disk

This project reads in 3 files from disk, a texture and two shaders. There's a proper way to do this, via Pepper, but we're not to the point of caring about pepper just yet. Instead we're going to cheat here, and use a bit of dark voodoo magic. Although we're serving our html content from a webserver, we can define this nice flag --no-sandbox as a command line argument that will allow us to read files directly from the local disk, bypassing the sandbox security restrictions.

So in your project's settings set
Debugging > Command Arguments:

--no-sandbox --single-process --register-pepper-plugins=$(TargetPath);application/x-nacl "localhost:5103/index.html"

From here, we can access the files directly on disk, but it's a bit of hand-wavy issue about what directory these files exist in to access them; If you run the project a few times, and move the files around, it's difficult to pin down the exact spot that a relative path reads from.So for the sake of simplicity for this tutorial we're going to prepend all the files paths with the absolute path of the location. Yes. This works.


FILE* f = fopen("\\plugin-port1\\plugin-port1\\test.raw" ,"rb");

Note you'll need to do this for both the texture, and two shader files that are explicitly referenced in the code.

Booyah

From there, Run the Compile, httpd.py server, Run and see the awesome:

The fruits of our labor. An OpenGL window will launch alongside of our chrome module. It renders and responds to input quite well. We've done the Waffle Burger proud!

When you run, note that you webpage will not respond in parallel with your window being opened. This is because our main thread that the browser runs on is actually running a spin loop for servicing the window popup, and not allowing any events for the page to be serviced. To fix this, you can do all of the window logic on another thread, and communicate back to the main thread with a producer-consumer FIFO, but that's a bit beyond the scope of this article.

Notes about the provided code

The original source referenced gluax.h to decode the .bmp image has been removed, as the gluax library is depriciated at best, so rather than adding complexity, I instead converted the test.bmp file to a test.raw, and modified the texture loading code in the init(..) function to simply memcpy load the raw data. I figure that has little and less to do with Native Client, so I omitted the code discussion for the article ;)

Note, you can download the source code for this series @ my github repo.

The rest of the series -
Part 1 : Setting up visual studio
Part 2 : Using platform-specific code
Part 3 : Porting to Pepper
Part 4 : Compile to NaCl

You can find me here:

~Main

Porting to Native Client with Visual Studio pt.1

2012-04-26T08:33:00.000-07:00

WARNING:

As of 10.1.2012, there's an official version of a Visual Studio 2010 plugin. Only some subset of this article series is accurate, please see the official documentation at gonacl.com for more.

Thus far, if you were a Native Client developer, your development process would look something like this:

Compile your app against NaCl's GCC

Curse at all the problems of using the GCC tools

Modify your APIs to target the Pepper APIs

Throw furniture across the room while trying to printf debug all the API issues, while at the same time overhauling your codebase to deal with the nuances of Pepper's Async APIs

Debug any run time issues that come with running in a webpage

Take up alcoholism as a professional sport because it makes your printf logs look 'perty'

But this stops today! Today we learn about using Visual Studio and Chrome Plugins to develop your Native Client application.

Crazy Harry, also known as the patron saint of plugin development.

Chrome, Plugins, and Native Client
Officially, you should read the Basics of Chrome, it's a great read that's eye-opening for non-web developers. If you don't have the time, here's a condensed version:

Chrome is a web-browser
Chrome provides the ability for developers to create Plugins to extend functionality
Chrome provides a safe, secure API for plugins called Pepper (PPAPI). Plugins target the PPAPI to interact with Chrome, and the sandbox safely
Native Client is a plugin that's built into Chrome, which loads external assemblies and allows them to interact with the Pepper APIs in a safe, secure fashion.

The important take-away from the list above is that Native Client is a Chrome Plugin, meaning that they both target the Pepper APIs.

What this article series talks about, is how to port to native client by first developing as a Chrome Plugin, so that you can gradually migrate over to the Pepper APIs.

Oh, and did I mention you get the full power of visual studio?

You can read the official documentation on developing trusted plugins, which, honestly, melts my brain; so I've condensed the process into the steps below.

Grab the chromium source code
Before you can compile your plugin, you'll need to get the latest source code of Chrome (called chromium) to compile against. The source code includes the specific header files that your plugin will need to link against in order to operate correctly. Note that when you pull down chromium, the root folder is "src" so it's best if you create a parent folder "chromium" to pull it into.

Using GIT you can grab the top of the tree here:

git clone http://git.chromium.org/chromium/src.git

If you're new to using GIT as your source control solution, I highly recommend using TortiseGit, it fits really well into the Windows IDE, and is free.

This is going to take a while, so you while that's working, continue forward with the next steps ;)

Grab the latest build of Chromium
To properly debug, you'll need a build of Chromium that matches the headers from the source repo. You can choose to build it yourself, if you've got a few hours and a high tolerance for pain.

If you'd like to skip that step (which I recommend), you can download the nighly builds from the chromium build repository.

Clicking on "builds: continuous" and choosing "win" as a sub-folder lists all the archived builds of chromium that you can use directly. (or you could click here for simplicity ;) Scrolling to the bottom of the page gives you the latest successful build folder, contained within is the binaries and symbols for that build.

You only need the chrome-win32-syms.zip (symbols) and chrome-32.zip (binaries) from this directory

I unzip the contents of these files in a folder named "chrome_build" so that it's easy for me to remember which build i'm running ;)

aside - Yes, for Windows, there's only a 32 bit version. That's a separate discussion; just download it, use it, and move on.

Grabbing the Native Client SDK
Visit gonacl.com and click on the 'sdk' link on the left hand side of the page. Follow the instructions to download and install the latest version of the sdk, and update to the latest version of pepper_*

Creating the windows project

Start a new Visual Studio project
Select Win32 Project
When the win32 application wizard starts up, select "Application settings" and choose "DLL" with "Empty Project"

Continue on to create the project.

Adding the source files
Add a new "main.cpp" file to your solution, in this file, we'll write the base code for our Pepper plugin that will interact with Chrome. But rather than typing all that out by hand, we're going to cheat, and use some existing code from the Native Client SDK. Once it's downloaded and installed, copy-paste the contents of this example NaCl file into your new main.cpp

(NaCl_SDK_root)\pepper_18\examples\hello_world_newlib\hello_world.c

In addition, you'll also need an html file to host your NaCl module from.
Continue on and add a new "index.html" file to your solution. Note, depending on what version of visual studio you have, creating an html file will be under the 'web' subcategory of the 'add new item' dialog.
Copy-paste the contents of this example NaCl file into your new index.html

(NaCl_SDK_root)\pepper_18\examples\hello_world_newlib\hello_world.html

NOTE - For you advanced users out there, you most likely have noticed that you could have simply added the hello_world files to the project as remote references, rather than copy-pasting and all that jazz; I've presented the above method for the sake of conciseness; This makes it easier to hack on the files w/o ruining the files in your examples folder.

Setting the visual studio project properties
This is where the dark magic comes in. Right clicking on your project, select Properties, and fill in the following line-items.

C/C++ > General >Additional Include Directories :

(Some Path)\chromium\src\

This is the path where you downloaded your chromium source code to.

Debugging > Command:

(Some other Path)\chrome_build\chrome-win32\chrome.exe

This is the path to your downloaded build of Chromium.

Debugging > Command Arguments:

--single-process --register-pepper-plugins=$(TargetPath);application/x-nacl "localhost:5103/index.html"

this... requires some explanation...

--single-process - tells chrome to run the tabs in a single process. As the background document states, chrome is a multi-process browser, and each tab is opened in a separate process. Since our goal is to debug the open tab w/o going insane, we specify this command line argument so that we can properly attach to the running instance of Chromium to debug it. In addition, it's worth pointing out that this turns off the sand boxing in chrome. As such, I don't recommend running your main chrome.exe with this flag due to potential security issues.
--register-pepper-plugins=$(TargetPath);application/x-nacl - This is a debug flag that tells Chrome to load the specific plugin automatically. Normally a plugin would have to be distributed and installed through the Chrome Extensions UI, which would generally kill your productivity turnaround. In addition the application/x-nacl flag tells chrome to associate the usage of this plugin with any Mime type matching "application/x-nacl". So once the webpage find a mime object matching that string, it will invoke our plugin properly
localhost:5103/index.html - This ending parameter tells Chrome what URL to load when it launches. Why are we pointing it to localhost? let's chat about that a bit more.

The HTML and Web-service.

As noted before, we're working with a web-technology; that is, native client modules must be served from a web-server to operate correctly. This is generally a 'pick your poison' setup with tons of options to create web-servers on your windows desktop. For simplicity's sake, the Native Client SDK comes with some easy uses for this and provides a single-file python based web-server to use for showing off the examples. You can find this web-server here:

(NaCl_SDK_root)\pepper_18\examples\httpd.py

The httpd.py script will launch a webserver who's root is the directory it's launched from. As such, we need to copy the file to the same location as the index.html file you created above (or the location of your vcxproj, which should be the same)
Note, make sure python is properly added to your path, otherwise running httpd.py could be problematic.

Once placed, feel free to launch the webserver and minimize the window.

caveat
At the time of this writing, this python script contains a check to ensure that the webserver is running from the examples folder. This makes the script useless for our purposes, so I suggest that you remove this check by simply adding a 'return' at the beginning of the 'SanityCheckDirectory()' function.

Simply add a 'return' to this function, and you can use the httpd.py module anywhere as a quick, easy local web-server. Note, that if you're new to Python, make sure that you've indented 'return' in the proper tabs as the rest of the function.

Compiling and debugging.
In visual studio, Choose "Debug > Start Debugging" (or just hit F5). If all went well, Visual studio should fire up an instance of chromium; load the page, and pop up an alert.

Now, this is where the magic happens; If you place a break point inside visual studio in the Instance_DidCreate function (line 46ish) then it'll fire before the window pops up the alert box. This function will send a message from the plugin to the HTML page to alert. Continue past the break-point to see the HTML page pope up the alert.

YES! A break point! My IDE color scheme is Zenburn, thanks to Jeff Atwood

Want to go deeper? If you pop open the Debugging Tools inside of chrome, you can set a break point in the HTML's handleMessage function (line 24ish) which is responsible for popping up the alert. If you haven't stopped visual studio yet, then simply hit "RELOAD" in the webpage and you'll hit the javascript break point.

Conclusion
AND that's all for now! You can now create a trusted plugin for chrome, and have it communicate back and forth with the html in the page. Note that what we haven't done here is talk about NaCl; All we've done is create a trusted plugin that targets the Pepper APIs

Also Mad props to NaCl SDK Lead Noel Allen for helping plot all the tricky bits out.

Note, you can download the source code for this series @ my github repo.

The rest of the series -
Part 1 : Setting up visual studio
Part 2 : Using platform-specific code
Part 3 : Porting to Pepper
Part 4 : Compile to NaCl

You can find me here:

~Main

NaCl and Blacklisted GPU Drivers

2012-04-24T07:52:00.000-07:00

With the new flood of Native Client games coming out, 3D is hitting the web more, and more. One of the biggest issues with 3D on the web is the limiting nature of blacklisted drivers. Looking at the crash logs that come through, incorrectly handling blacklisting crashes accounts for the #1 crash report from NaCl applications lately. As such, let's take a look at how to properly respond to this issue.

Why do we have GPU Driver Blacklisting
GPU API can be malicious; more specifically, adding GPU APIs to a web browser opens a door to the drive-by-web that we never had access to before, specifically, we're talking to hardware; And if you've ever delt with hardware before, you know there's edge-cases all over the place that could be exploited incorrectly. Web needs to walk the line between functionality and safety, we need a modern web with advanced, deep interaction, and there's a massive majority of hardware out there that can handle it. The goal then, is to find a line that provides the advanced featuresets of the modern web, and the safety of it as well.

This is where we introduce the concept of a GPU Driver Blacklist. Some drivers have vulnerabilities, or direct stability issues that result in bad end-user experiences, while most of the time, we can work with the hardware vendors to fix the issues, sometimes is not that easy, and as such, we have to restrict that specific driver from running inside of the browser.

NOTE that we restrict the driver, not the GPU ! The GPU, like other hardware, is usually a straightforward device, which bends to the will of it's driver based overload. The device is the one we need to worry about.

Now, it's very important to understand that WebGL (via javascript) and OpenGL ES2 (via Native Client) both use the same back-end inside of chrome; As such, when discussing black-listing, you can use the terms pretty interchangeably.

Khronos keeps a general description of blacklisted devices, which is good to know. (The chrome list is a bit more informative)

Restricting via the chrome web store
Currently, Native Client applications must be installed through the Chrome Web Store to work properly.
CWS provides a barrier flag, which will restrict users from installing your NaCl application if their device is blacklisted.

"requirements": {
  "3D": {
    "features": ["webgl"]
  }
}

Note that there is some nuances here that CWS still needs to fix; If the device is blacklisted, and the app defines the above flag:

The app won't show up on the puzzle wall
The app WILL show up in a direct search
The app WILL show up if you link to it directly
When you reach the install page, user will see "This application is not supported on this computer. Installation has been disabled"

The intent is that CWS should filter out games that users can't play so that they don't leave bad feedback to the developer.

Detecting Blacklist via WebGL
If the user can install from CWS (or if you're a hosted app, just browse to your URL), you'll need to catch their issues directly at your webpage. Since WebGL and NaCl share the same rendering path; you can detect for a blacklisted device in javascript before even loading the NaCl module:

function initGLTest(size) {
  var canvas = document.createElement('canvas');
  var gl = canvas.getContext('webgl') ||
           canvas.getContext('experimental-webgl');

  if (gl) {
    return true;
  }

  return false;
}

You should be doing this test before you load your nacl module; More specifically, you should add the EMBED tag to the DOM only after the above test as been successful.

Yes, that means that there should be some series of startup tests you need to perform before actually adding the nacl module to the webpage. You should check my previous post on detecting user setup problems.

Detecting Blacklist via NaCl
If the above WebGL test has passed; chances are that your driver is OK, and you should be good-to-go with creating your NaCl OpenGLES2.0 GLContext. If creating your context does fail, please try a simpler version of context create to ensure that you're not incorrectly asking for features that the hardware doesn't support. Chances are, you're asking for a resolution, or init feature which is causing the problem.

Test your blacklist response
You can test your blacklist handling by running chrome with the following flags: With these, you can run through all the above scenarios, and test how you handle things.

--disable-webgl
--disable-pepper-3d-for-untrusted-use
--disable-gl-multisampling
--disable-accelerated-compositing
--disable-accelerated-2d-canvas

What to do when you've detected a blacklist response
Firstly, thank the user for using your application. ;)
Secondly, alert the user of the issue, and direct them to update their graphics drivers to a newer version.
Chrome provides a simple page describing how to do updates, which you should most likely direct your users to in such a case that you've found a blacklist error.
Thirdly, if that doesn't work, humbly apologize to your user about the situation, and try best to explain the reason for the problem. The intent here is that users shouldn't respond to your application with bad ratings because they have blacklisted hardware; You'd be amazed how far an apology goes to help with this.

Other tricky things
-Blacklisted drivers on macs are especially tricky to work with; Apple owns distribution of those drivers, and often distributes them at a seperate frequency than the PC / Linux branches. So it's difficult (nay, impossible sometimes) to direct the user to update their drivers on these devices.

-Make sure you detect feature sets as well; you need to properly test that the GPU supports the specific features your 3D app is using. For example, does this GPU support 4096 textures? Most of these tests can be done in webGL, before you even load your nacl module, but they are critical to test for, as hardware is heterogeneous, and thus your application needs to respond properly.

So, armed with this information, there should be no reason to see any more blacklist crashes coming through Google's crash logs. Right? Good. Because otherwise, I have to send the fear engine after you, or if he's booked, Deep Crow.

~Main

Your texture compression is not enough

2012-04-07T08:04:00.000-07:00

I've been pitching my DXT talk to GDC since 2010, having it getting into various forms of acceptance, and then having to drop out due to mandatory overtime, or a game shipping or whatever. So in 2012, I was super pumped to present some of my research to the masses.

In general, the point of my talk was to get people thinking about texture compression in a different way. Firstly to admit that with modern hardware and consoles, DXT is a bit dated if you're simply just coping it directly to the GPU. Secondly to publicize a different idea than the IDTech5 research, which is generally presented as the premier way to deal with texture streaming issues.
The gist of the talk is "Here's a bunch of interesting ways to molest your texture data for better compression footprint; but in reality, you should just use CRUNCH"

The famous LENA image; Generally regarded as the cornerstone of texture compression test images.

As a general graphics programmer, you should be following Charles Bloom's blog; If you ignore the more rant driven things, there's a great set of technical knowledge there; For example, Charles has been working on texture compression for a while now, and has some great articles on it.

After my GDC talk (video | slides) enough people connected the dots, and brought it to Charles' attention:

"I've been sent this GDC talk a few times now, so I will write some notes about it. (BTW thanks to all who sent it; I don't really follow game dev news much, so I rely on you all to tell me when there's something interesting I should know about)."

Charles is a smart cat. He's a top tier programmer with a great analytical sense; and he's also the top 10% when it comes to texture compression, so most of the things I talked about were old-hat to him; Which was good, because it means I'm not crazy. For the rest of the 90% who aren't on the cutting edge, and for the most part, the games industry as a whole, compression is a dark and spooky art. I was amazed when talking to developers on the show floor of GDC, how many people had no idea what LZMA was.

CBloom brought up some great points about my talk and I figured it would be worth sharing my thoughts on his thoughts in a recursive sort of way.

"whenever you are doing these things you need to consider the memory use and processor use tradeoffs"

I completely agree, and tried to frame the talk around this premise starting at slide 3. When evaluating any software for inclusion of a larger system, you should constantly evaluate the full suite of tradeoffs that you're making. Especially with texture compression, which has so many separate vectors for success.

A perfect example of this manifested itself right after my live GDC talk: some of the engineers from AMD came up and quickly pressed me about why I wasn't doing texture decoding on the GPU. My response was to avoid this type of work; Most graphics intensive games keep their GPUs maxed out for high-performance, I'd hate to have to balance that against doing texture decompression as well (and if you're not graphics intensive, you generally don't have enough texture assets to make this a problem anyhow...).
In addition, this is only available on higher end hardware; Most game developers care about reach as much as they do 'splash' so you can't ignore the fact that there's still a huge chunk of GPUs out there (laptops mostly) that can't do the fancy DX11 compute shaders. Decoding on the CPU allows a great reach of hardware. Of course, this is my use-case evaluation; On their end it makes loads of sense to use GPU decode. Again, you have to consider all the trade offs.

" Zip is ancient and has a tiny window, there's no reason to use zip. If you just use a better back end, most of what he does next is irrelevant."

I agree..kinda! Firstly, there's still lots of studios using GZIP variants; Blizzard's MOPAQ format finally introduced LZMA with Starcraft 2. (Although IIRC, WoW hadn't made the switch yet?) It comes back down to lack of technical knowledge of the games developer; throwing things in a zip format 'just works' and most people call it a day. If you want confirmation of this, check out your games install data on your smartphone..

I wouldn't say that switching to a better archive compressor makes the rest of the talk irrelevant though; The hope is that you can keep your textures in memory, hyper compressed, and decompress on demand. And that this whole process would be a smaller footprint, with a faster decompression speed than if you were using an archive format directly.

Slide 5 of my talk covers this: Most level archives include data that's only needed at load time, rather than at runtime; As such, keeping the full archive around, or having to parse to different sections of the disk to fetch just texture data is less than ideal. Switching to a compressor that has a better memory footprint doesn't mean you're going to get better decompression speeds.

LZMA is a great algorithm for compression; Where LZMA fails though, is when you try to do run time decompression of texture assets for streaming; In 2008 a few groups at Microsoft Games were trying to make a runtime decompressor for LZMA, and couldn't get anywhere close to 256 megatexels/sec . Again, it comes out to evaluating your options to find the proper solution for your project; maybe the size of LZMA beats out the decompression speed for some projects? Or was it just how the decompressor was abusing the XBOX360's L2 Cache? Or maybe there's been great advances in LZMA decompression speeds lately? As always, YMMV!

The only real idea in the talk is going to 8x8 blocks. That is in fact a valid thing to do, but here the rigor of the talk completely falls apart, and instead we get "look at this blurry slide in a terrible lighting environment, can you see the loss?"

I agree, the lighting in that video was horrible. You can see close-ups of my side-by-side textures in my slides. Note there's some introduced noise in a few of the source images from the process of converting the true side-by-side to JPG in order to put it in a talk. As such, you should focus on what the expanded blocks algorithm does, and note the blocking and quantization artifacts first. The parrot shows some good overal results; You start seeing bad quantization around the gradients, but the noise in the feathers goes mostly un-noticed; You can see more problems in the picture of the girl; Especially in the closeup. The game-texture has an interesting result, in that it's really difficult to see the overall compression artifacts. Most game textures have lots of noise, and are generally blurred out due to modern texture filtering in 3D scenes. In reality, this lets your texture compression algorithm cheat: Who cares about PSNR when you're getting chased by a guy with a sword?

Any time you start introducing loss you have to ask : how does this loss I'm injecting compare to other ways I could reduce bitrate and increase loss?

We have reached quorum! This is especially true when evaluating different texture compressors against different texture types. For example, the algorithm I presented didn't do as well with normal maps, but chewed through AO and specular maps like buttered bakliva.

Apparently enough people watched the video, and passed it along to cbloom for comment; which prompted him to write a larger blog post; and a followup post which presented even more ideas about the future of custom texture compression.

So it appears people are listening, and thinking, talking, and ranting about texture compression for games.

Mission Accomplished.

You can find me here:

Dynamic loading of code libraries with Native Client

2012-04-03T07:16:00.001-07:00

As of Pepper 17 Native Client supports dynamic loading of code libraries (A windows-based analogy would be "You can load .DLLs"). The Pepper 18 SDK came with an example code set (dlopen) that showed how to properly load the libraries via a few specific posix style calls. Proper documentation on how to link with GLIBC is missing however, so I thought I'd throw up a quick tutorial.

Native Client comes with two C run time libraries, Newlib and GLIBC; If you come from a windows / Visual studio back-ground, this may be a bit jarring to think of "Multiple C run times!?" But fear not, there's a reason for it.

Native Client support for dynamic loading is requires linking with GLIBC. GLIBC is the GNU implementation of the POSIX standard runtime library for the C programming language comprising of a set of interdependent libraries including libc, libpthreads, libdl, and others.

As such, before you can start dynamically loading assemblies, you need to compile your project with glibc.

Step 1. Compile your NaCl App using GLIBC

The Native Client SDK ships with both GLIBC and Newlib versions of the GCC toolchain. All code used by your application must be built with either one or the other toolchain. Code from these toolchains cannot be mixed. You must choose the GLIBC toolchain to use GLIBC and dynamic linking.

On windows, you can find these folders:
SDKROOT/pepper_18/toolchain/win_x86_glibc
SDKROOT/pepper_18/toolchain/win_x86_newlib

For Mac and Linux, the appropriate three letter marker replaces the "win" section of those libs.

As a simple example, suppose you are working on 64-bit Windows, and that your application is implemented as a single C++ source module, test.cpp.

In this case you might compile your application for 64-bit and GLIBC as follows:

..\x86_64-nacl-g++.exe -Wall -m64 -fPIC -o test-x86_64.nexe test.cpp -ldl -lppapi -lppapi_cpp

Note that you'll find the x86_64-nacl-g++ file in the /bin folder.
These compiler flags are described in the gcc documentation, summarized here for convenience:

-llibrary: use the indicated library, as per C library naming conventions. For example, -ldl causes the application to use libdl.so, or libdl.a if libdl.so is not available.
-fPIC: Generate position-independent-code, required for all 64-bit modules and for 32-bit shared libraries.
-Wall: Enable an optional set of compiler warnings
-m32, -m64: Specify generation of 32-bit or 64-bit code, respectively
-o: Specify the output file, e.g. test-x86_64.nexe

Step 2. Building a shared object
Now that you're compiling with GLIBC, you need to generate your .SO objects (an analogous version of a "Windows DLL") which will be loaded dynamically at runtime. Much like generation of the nexe, you'll compile this with GLIBC as well; the main difference is adding the -shared flag:

..\x86_64-nacl-g++.exe -fPIC -Wall -m64 -shared -o testlib-x86_64.so testlib.cpp

This will generate testlib-x86_64.so which an analogous version of a "Windows DLL"

Step 3. Generate a manifest file

When a dynamically loaded desktop application is launched, the operating system’s program loader determines the set of libraries it requires, reading explicit inter-module dependencies from executable file headers, and loads the libraries into the address space of a new process. Typically these libraries will have been installed on the system as a part of installing the application. Often the desktop application developer doesn’t know or think about the libraries required by the application, as these details are taken care of by the operating system.

In Native Client, dynamic linking support can’t rely in the same way on the operating system or the local file system. Instead, the application developer is responsible for understanding the set of libraries required by an application, listing them in a native client manifest file(.nmf), and delivering these libraries along with your application, either serving them from a web server (for hosted applications) or bundling them as a part of a packaged Chrome Web Store application (for packaged applications).

Step 3a. Determining required assemblies
Now that you've generated a proper .so, you'll need to explicitly list the .so files in the manifest in order for the security system to authorize the content. Before we list the exact ones that you've compiled by hand (from step 2) we'll need to get a list of what runtime libraries are going to be required for this project to run.

Given that you've produced the test_64.nexe already; It's worth noting that this file also includes a bunch of headers that describe the contents of the file and additional information on how to use it. Using the Native Client version of the GCCobjdump command, we can use the -p option to examine the file’s headers, including the “Dynamic Section” that specifies dynamic library dependencies, as in this example output:

..\x86_64-nacl-objdump.exe -p test-x86_64.nexe 

test-x86_64.nexe:     file format elf64-nacl

Program Header:
  INTERP off    0x000008c0 vaddr 0x110008c0 paddr 0x110008c0 align 2**0
         filesz 0x00000019 memsz 0x00000019 flags r--
    LOAD off    0x00000140 vaddr 0x01000140 paddr 0x01000140 align 2**16
         filesz 0x00000780 memsz 0x00000780 flags r-x
    LOAD off    0x000008c0 vaddr 0x110008c0 paddr 0x110008c0 align 2**16
         filesz 0x00000314 memsz 0x00000314 flags r--
    ...

Dynamic Section:
  NEEDED               libdl.so.40b700e4
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.40b700e4
  INIT                 0x01000140
  FINI                 0x01000840
  HASH                 0x110008dc
    ...

Version References:
  required from libdl.so.40b700e4:
    0x0d696911 0x00 04 GLIBC_2.1
  required from libc.so.40b700e4:
    0x0d696910 0x00 03 GLIBC_2.0
  required from libgcc_s.so.1:
    0x0d696910 0x00 02 GLIBC_2.0

In the Dynamic Section we’ve highlighted three items marked NEEDED. These are three of the files you will need to include in your manifest file and distribute with your application. More specifically, this process will describe to you the proper runtime code that needs to be distributed with your application. To note, you'll also need to list your custom built .so files

To get the full list of libraries, you need to compute the recursive closure of library dependencies, starting by using objdump on these files three files.

To simplify things a little bit, the Native Client SDK includes a python script, create_nmf.py, that will attempt to generate a manifest file for you. Assuming you have python properly installed, and in your path, you can use create_nmf.py as follows:

python %(SDKROOT)/pepper_18/tools/create_nmf.py test-x86_32.nexe -L %
(SDKROOT)/pepper_18/toolchain/win_x86/x86_64-nacl/lib32/

will produce

{
  "files": {
    "libc.so.40b700e4": {
      "x86-32": {
        "url": "x86-32/libc.so.40b700e4"
      }
    },
    "libdl.so.40b700e4": {
      "x86-32": {
        "url": "x86-32/libdl.so.40b700e4"
      }
    },
    "libgcc_s.so.1": {
      "x86-32": {
        "url": "x86-32/libgcc_s.so.1"
      }
    },
    "main.nexe": {
      "x86-32": {
        "url": "test-x86_32.nexe"
      }
    }
  },
  "program": {
    "x86-32": {
      "url": "x86-32/runnable-ld.so"
    }
  }
}

By default, create_nmf.py prints the NMF file contents to standard output. Notice we’ve passed it the -L option to specify a directory to search for dynamic libraries. Also note that create_nmf.py can only recognize explicit library dependencies, for example, those created with the -l option to GCC.

Step 3b. Adding custom assemblies
If your application uses dlopen() to load additional libraries, you must be sure these libraries are listed in your manifest file.

"yourefilehere.so": {
      "x86-32": {
        "url": "x86-32/yourfilehere.so"
      }
    },

The example manifest file above tells Native Client how to load a 32-bit “x86-32” version of your application, but it doesn’t say anything about x86-64 or ARM. Make sure you build the proper 64 and 32 bit versions before deploying!

Note that one downside of using create_nmf.py is that it blows away old changes you've made. So if you're using that app, and you have custom assemblies that need to be appended to your manifest, it may be worth forking the create_nmf.py script, and modify it to add in your custom assemblies at the end.

Step 4. Deploy your application for testing

Your manifest file should now explicitly list all the executable code modules required by your application, including modules from your application, modules from the Native Client SDK or middleware systems used by your application. As a part of deploying your application, you must make all of these modules available for your application.

You need to treat these .so files like any other resources. For instance, if you are a hosted application, you need them to be served off a web server; and for packaged apps, they need to be included in the .CRX file.

Opening shared libraries at runtime

Native Client supports the a version of the POSIX standard dlopen() interface for opening libraries explicitly, after the application is already running.

Note that dlopen() calls may cause a library download to occur, and will automatically load all libraries required by the named library. As dlopen() can potentially block, it must be called off the main thread. Calls from the main thread will always fail in current implementations of Native Client.

We advise using a worker thread during initialization of your application to pre-load libraries asynchronously so they will be available when needed. As per the specification, subsequent calls to dlopen() will return the a handle to the previously loaded object. Note that calling dlclose() will close the library, and a following dlopen() call to the same path may cause another fetch to occur.

Note that the DLOPEN example in the SDK properly shows how to do this process.

~Main

You can find me here:

Reporting errors from your web app

2012-03-30T06:54:00.000-07:00

One item that I got constant follow-up about from my GDC Talk on Best Practices in Developing a Web Game was error reporting. So I thought I'd take a moment and talk a bit more about it.

Error Reporting from the field is a constant term of struggle for developers; It's a wierd mix of how to properly catch the problem, what data to collect, and then where to submit it back to so they can gather it and do something about it later.

Once your application is out in the wild, errors will occur on users’ machines that you as a developer won’t have direct visibility to, and thus, have little or no chance of fixing. As such, it’s very useful to structure your JavaScript code to take advantage of various means to capture data when problems occur such that you can receive it and act accordingly.

For more manual debugging, the Console API has console.trace which will print out the presented content to the JavaScript console for viewing.

For syntax and execution errors, standard JavaScript provides a hook for window.onerror as a generic event for trapping all errors that happen on a page and provides the error string and line number of the error that happened.

V8 is a JavaScript virtual machine powering most modern web browsers, and it provides a great API for capturing stack traces for you using Error.captureStackTrace. For other browsers not using V8, you will have to manually capture your stack traces by inserting function tracking points and retrieving them during an event. Javascript Stack trace and TraceKit are great utilities for manually creating and capturing a stack trace of function calls leading up to an error, or particular event.

Besides direct JavaScript errors, there’s also a set of data you should be additionally concerned about, for instance another handy technique is to store important game-state information in a form such that when a crash, or other event occurs, the data is dumped out in a safe way for analysis. In addition other browser environment data like window.performance might also be useful for getting app state information -- and chrome.tabs.captureVisibleTab() (for packaged applications) can give more detail about the problems at hand to help you debug a solution.

Once you are able to catch and identify errors, the next step is to report them back to you, the developer, in some useful form to take action against. The two most popular versions of this are logging the error data to a database, and sending an e-mail to the developer.
jsErrLog is a great library that encapsulates error catching and automatic uploading to a database resource. Which is the exact service that jdrop provides, a place to store json data in the cloud environment.

Some other points to consider:

Enable error reporting when your app is used offline, by storing error data locally then submitting a report once the app is back online.
Take care that error reporting does not itself cause problems, for example by making repeated XMLHttpRequests, or failing because your app is offline.
Ensure error messages displayed are appropriate to the user: be friendly, avoid technical language and never report API error codes directly.
Whether or not you implement error reporting, it’s always good to provide feedback mechanisms so users can report bugs, suggest feature requests or make comments.
Be unobtrusive.
Respect your users’ privacy:
Notify the user unobtrusively when your app reports errors.
Ensure the user understands what data is reported – if possible, show them the actual data.
Allow the user to opt out of error reporting – better still, ask them to opt in.