Incremental development–grow, don’t build, software. I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash he broadened my whole view of the software process. The metaphor shift was powerful, and accurate. Today we understand how like other building processes the construction of software is, and we freely use other elements of the metaphor, such as specifications, assembly of components, and scaffolding.

The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be specified accurately in advance, and too complex to be built faultlessly, then we must take a radically different approach.

Let us turn nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and selfrenewing. The secret is that it is grown, not built.

So it must be with our software-systems. Some years ago Harlan Mills proposed that any software system should be grown by incremental development. That is, the system should first be made to run, even if it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit, it should be fleshed out, with the subprograms in turn being developed–into actions or calls to empty stubs in the level below.

I have seen most dramatic results since I began urging this technique on the project builders in my Software Engineering Laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness. The approach necessitates top-down design, for it is a top-down growing of the software. It allows easy backtracking. It lends itself to early prototypes. Each added function and new provision for more complex data or circumstances grows organically out of what is already there.

The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.

(The above is, of course, excerpted from Brooks’s essay.)

Iterative development that delivers frequent releases of working software, and uses feedback to determine the direction of growth for the next increment? And a report of empirical (if decidedly anecdotal) support for the notion?

Sounds pretty Agile to me.

(The full essay is available in The Mythical Man-Month: Essays on Software Engineering, which I recommend to anyone who writes, builds, grows, or otherwise makes software.)

It’s time to build our Backbone.js app, “Comment on the Movies”.  This tutorial will, ideally, give you something interesting to build.  Take a look at the finished app (so far) on Heroku & feel free to pull it from GitHub.  In addition, I’ve updated the code to use Backbone.js 0.5.3, Underscore 1.2.2, Mustache 0.3 & jQuery 1.7.1.  Let’s do this!

*Before we get started, check out one more resource: Charlie Robbins (Co-Founder of Nodejitsu) wrote an awesome article about Scaling Isomorphic Javascript Code.  In the writeup, he describes popular methodologies in retrieving data and presenting it to the user.  It’s a great read.  The image below from that article:

Backbone.js interactions: Model-View-ViewModel

Prerequisites:  Rotten Tomatoes API Account, Ruby, Sinatra, & Haml

Rotten Tomatoes API Account

The app we’re writing is called “Comment On The Movies”.  The RottenTomatoes API is full of well formed data (which is more than I can say about the New York Times movie api calls).  Real data is always more interesting than building dummy datasets ourselves.

The Rotten Tomatoes API registration is free, quick & painless.  Get your api_key & come back to this tutorial.

We’ll also be using the R.T. assets.  R.T. makes their images available to developers to use in their apps and we’ll be taking advantage.

Ruby (Project Site)

Realistically, if you’re reading this webpage, you probably already have it installed (even if you don’t use it) .  I chose Ruby here because I 9-5 in Ruby and I really enjoy it.  You can use ANY server side scripting language here (PHP, Asp.NET, JSP), but you need to use something. You don’t want to simply expose your R.T. API App Key to the world.  By using a server side language, you can create a proxy page that protects your API App Key.  Here’s how this app will work:

1. Your Backbone app makes an AJAX call with R.T. API parameters to the proxy page
2. The proxy page appends your API_KEY & makes the call to a R.T. servers
3. The proxy page receives the response and returns it to your Backbone app
4. The Backbone app updates it’s UI.

Sinatra (Project Site)

Since we only need an index page and a proxy, Sinatra is a perfect option here.  A full Rails app would be WAY too much.

Haml (Project Site)

I enjoy haml.  It’s really easy to create markup with it.

The “Comment on the Movies” Spec

Here’s what the app needs to accomplish:
**Items in orange will be implemented in future tutorial posts**

• Users should be able to switch between the various lists of R.T. data:
• Box Office (default view on page load)
• In Theaters
• Opening
• Upcoming
• The “Movies Table” should display:
  • Title
  • Year
  • MPAA Rating (R, PG-13, etc)
  • Runtime
  • Release Date (default sort)
  • Critic’s Score
  • If it’s ‘certified fresh’ that should be indicated.
  • Number of comments
• Clicking on a table column title should sort the current set of movies in the table.
• Clicking on a movie’s row should take you to a “detailed” view of the film.  The “detailed” view should display:
  • Links [all links that lead us away from the site should open in a new window] to…
    • Showtimes
    • Awards (if any)
    • Trailers
    • Abridged Cast
    • IMDB records
• Comments
  • Add a new comment
  • Edit an existing comment
  • Delete a comment
  • Require that the review indicates if the comment is BEFORE or AFTER you’ve seen the movie
  • Comments should persists across browser restarts
• The app needs to be crawl-able by Google.
• It needs to be well tested

Make your Backbone App Google Crawl-able via hashbang URLs

Hashbanging (/#!) your URLs let’s the Google crawling bot know that there’s AJAXed content on this page.  Fortunately, it’s really easy in Backbone.js as you can see in the live demo

In The Next Post…

I’ll explain how I implemented the app thus far.  That’ll include the filling the index view & the hashbang technique mentioned above.

(Note: I actually gave a talk on this topic some time back, having implemented this same concept in Ruby using Hadoop Streaming. That implementation contained a number of flaws, some of which I’m attempting to rectify. Also, it’s not as though I’m the first guy to work this angle. I haven’t perused the latest Hadoop/GA work in the field yet, because I wanted to maximize my first-hand learning at this stage – but I’ll be looking to other sources for inspiration as I develop and generalize this code.)

This was just done for my own edification; I haven’t put enough effort into tuning this software or studying its properties for this to be considered a serious work of research (yet). I’m not a specialist or any kind of expert with genetic algorithms, beyond my recreational reading. That said, I had some success at building a gene representation and evolutionary mechanisms that drove a population of solutions for an NP-hard problem toward (mostly) monotonic improvement.

What follows is a brief description of my approach to the problem, with code attached. If you are an expert with GAs, I’d love to hear your feedback on what I could do better.

I haven’t spent any effort in this post describing the Traveling Salesman problem, genetic algorithms, or Hadoop/MapReduce. (It was plenty long already.)  If you’re interested in any of those topics but need to brush up on the basics, check out the preceding links.

More geekery below the fold…

The Map

A “city” in my representation is an (x, y) coordinate pair, with each coordinate having a floating-point value in [0.0, 1.0). The software can take any arrangement of cities, but for testing I used an arrangement with a set of trivial solutions:

Gene Representation

When choosing a non-repeating path among N distinct cities, there are N possible choices for the first city; N-1 choices for the second city, and so on; and only one choice for the final city.

In my representation, a chromosome has N – 1 integer genes. For a gene G_n, valid values are [0, N - n) – thus, the first gene has N valid values, the second gene N – 1, and so on. Each gene corresponds to the 0-based index of a city in an ordered collection of the cities not yet accounted for in the path. For example, if we start with an ordered collection of cities {C₀, C₁, C₂}, and the chromosome {0, 0}:

• The first city in this candidate solution path is at index 0 in the collection:  C₀
• The second city in the path is at index 0 in the collection of remaining cities, {C₁, C₂}: C₁
• The final city is trivially C₂.

Likewise, the chromosome {1, 1} encodes the path C₁ -> C₂ -> C₀.

Fitness Scoring

Here I’ve implemented the symmetric Traveling Salesman problem (the distance/cost from C₁ to C₂ is the same as from C₂ to C₁), and the distance between cities is just the Cartesian distance between their coordinates. The maximum possible distance between any two cities in the landscape I’ve defined is √2, so the maximum/worst-case distance possible for an N-city map is (N – 1) * √2.

A chromosome’s score is this worst-case distance minus the actual distance of the path it encodes; thus, the greater the improvement over the theoretically pessimal case, the higher the chromomsome’s score.

public class ChromosomeScorer {
    protected ArrayList cities;
    protected ArrayList citiesUsed;
    final static double SQRT2 = Math.sqrt(2.0);
    protected double maxDistance;
 
    public ChromosomeScorer(String cityString) {
        cities = getCitiesFromString(cityString);
        citiesUsed = new ArrayList(cities.size());
        maxDistance = SQRT2 * (cities.size() - 1);
        for (int i = 0; i < cities.size(); ++i) {
            citiesUsed.add(Boolean.FALSE);
        }
    }
 
    protected double score(String chromosomeString) {
        ArrayList route = getRouteFromChromosome(chromosomeString);
        double distance = 0.0;
        for (int i = 1; i < cities.size(); ++i) {
            int city1Index = route.get(i);
            int city2Index = route.get(i - 1);
            double[] city1 = cities.get(city1Index);
            double[] city2 = cities.get(city2Index);
            double city1x = city1[0]; double city1y = city1[1];
            double city2x = city2[0]; double city2y = city2[1];
 
            distance += Math.sqrt(((city1x - city2x) * (city1x - city2x)) + ((city1y - city2y) * (city1y - city2y)));
        }
 
        return maxDistance - distance;
    }
 
    // city string parsing code elided
}

The ArrayList is a list of flags indicating whether a city has been added to the route yet; it’s for implementing the chromosome parsing I described above. In my example, I described the process in terms of pulling elements from an ordered collection, and performing further operations on the reduced collection. Regenerating the ordered list of cities and manipulating it in this way every time I wanted to score a chromosome sounded computationally expensive; flipping booleans sounded cheaper. (I could probably get cheaper still using actual bit flags, but I was trying to keep the code both readable and writable at this stage. Optimizations will come later.)

Initial Population

The following code creates an initial population of chromosomes (encoded as Strings) and dumps them to a requested file:

protected static void createInitialPopulation(FSDataOutputStream populationOutfile, final int populationSize, final int numCities) throws IOException {
    for (int i = 0; i < populationSize; ++i) {
        for (int j = 0; j < (numCities - 1); ++j) {
             if (j > 0) {
                populationOutfile.writeBytes(" ");
            }
            populationOutfile.writeBytes(String.format("%d", random.nextInt(numCities - j)));
        }
        populationOutfile.writeBytes("\n");
    }
    populationOutfile.close();
}

At a later time, I’ll experiment with using Hadoop’s ArrayWritable or other serialization schemes for more efficient representation of the chromosomes; at this stage, it was important to me to keep the data human-readable.

There’s a mapper dedicated to scoring generation 0:

public class ScoringMapper extends Mapper {
    private Text outKey = new Text();
    private DoubleWritable outValue = new DoubleWritable();
 
    protected ChromosomeScorer scorer = null;
 
    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        String incomingValue[] = value.toString().split("\t");
        String chromosome = incomingValue[0];
        double score = 0.0;
        if (incomingValue.length > 1) {
            String scoreString = incomingValue[1];
            try {
                score = Double.parseDouble(scoreString);
            } catch(NumberFormatException nfe) {
                // weird - but let's try re-generating the score
                score = scorer.score(value.toString());
            }
        } else { // we only go through the scoring process if we don't have a map
            score = scorer.score(value.toString());
        }
 
        outValue.set(score);
	outKey.set(chromosome);
	context.write(outKey, outValue);
    }
 
    // setup code elided
}

Subsequent Generations

I managed to cram the rest of the operations for each generation into one mapper and one reducer – really, into one reducer, since the map function just randomly assigns chromosomes to groups:

    @Override
    protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        outKey.set(random.nextInt(numBins));
        context.write(outKey, value); // shuffle
    }

So why am I shuffling the chromosomes thus? Most methods of population selection involve having some view over the data; the naïve method of truncation selection is an extreme example, where you need to sort the entire population, and keep some fraction of the highest-scoring individuals. This is hard to parallelize, and not especially scalable for large populations.

Other selection methods such as fitness proportionate selection, however, look like they’d retain their properties well if the total population was divided into large, random subpopulations. (I have yet to do a detailed mathematical treatment of this, but some combination of intuition and cocktail napkin calculation has convinced me of this well enough to forge ahead.) To this end, I’ve set up the system to randomly distribute the full population into a user-settable number of subpopulations; the size of these subpopulations can be tuned to fit within the memory allocated to a reducer instance. (Naturally, the ideal would be to have a subpopulation just large enough to be handled by one reducer instance.) Assuming that the subpopulations are sufficiently large, the random distribution should retain the aggregate characteristics (distribution of fitness scores, &c) of the total population, and give similar results under fitness proportionate selection.

After this subdivision of the population, a reducer instance receives a subpopulation, and is then responsible for selecting survivors from that population, breeding replacement members of the population from the survivors, and scoring the new population members.

    @Override
    protected void reduce(VIntWritable key, Iterable values, Context context) throws IOException, InterruptedException {
        TreeSet sortedChromosomes = getSortedChromosomeSet(values);
        normalizeScores(sortedChromosomes);
 
        int survivorsWanted = (int) ((double) sortedChromosomes.size() * survivorProportion);
        Set survivors = new HashSet(survivorsWanted);
 
        while (survivors.size() < survivorsWanted) {
            survivors.add(selectSurvivor(sortedChromosomes));
        }
 
        ArrayList parentPool = new ArrayList(survivors);
        while (survivors.size() < desiredPopulationSize) {
            survivors.add(makeOffspring(parentPool));
        }
 
        Iterator iter = survivors.iterator();
        while (iter.hasNext()) {
            ScoredChromosome sc = iter.next();
            outKey.set(sc.chromosome);
            outValue.set(sc.score);
            context.write(outKey, outValue);
        }
    }

Selection: The first step in fitness proportionate selection is to sort the new members according to their score, ascending. After that, survivors are selected from the population randomly, weighted by their scores (the higher a chromosome’s score, the more likely it is to survive).

Repopulation: Surviving chromosomes are used to breed back up to our original population numbers. I used single crossover and random mutations (maximum of one mutation per chromosome per generation):

    protected ScoredChromosome makeOffspring(ArrayList parentPool) throws InterruptedException {
        int parent1Index = random.nextInt(parentPool.size());
        int parent2Index = parent1Index;
        while (parent2Index == parent1Index) {
            parent2Index = random.nextInt(parentPool.size());
        }
 
        ScoredChromosome parent1 = parentPool.get(parent1Index);
        ScoredChromosome parent2 = parentPool.get(parent2Index);
 
        ScoredChromosome offspring = crossover(parent1, parent2);
        if (random.nextDouble() < mutationChance) {
            mutate(offspring);
        }
 
        offspring.score = scorer.score(offspring.chromosome);
 
        return offspring;
    }

Scoring: Finally, all unscored population members are scored in preparation for the next round.

This process is repeated for as many generations as is specified at the beginning of the run.  (Later, I plan to allow the number of generations to be unspecified, and have the algorithm terminate when the maximum score starts to settle/converge.)

Parameters

• Number of cities
• Population size
• Proportion of survivors in each generation
• Subpopulation size (see above for rationale for dividing up the population, and thoughts on tuning)
• Chance of random mutation
• Number of generations

Results

I’ve thrown the data for min/max/mean scores for each generation of each run into a Google spreadsheet, and made some simple graphs.

The first run was done with 10,000-member population, 30% survival chance, 500 generations. There’s an unmistakable fitness increase over time, and it gets pretty close to the theoretical maximum for my test city (~23.066).

The second run was done with a 100,000-member population and 50% survival chance, 500 generations. It gets off to a slower start, but there’s a burst of improvement late in the 500-generation run and it gets pretty close to the optimum.

The difference in early-stage performance of the second run had me kicking myself for changing two inputs. I switched back to a 30% survival chance but kept the larger population. This run landed in the same neighborhood as the others, and looked like it still had room to settle.

Interesting to note on all 3 runs is that all start with a few generations of quick improvement, then flatten out, then have an inflection point after which there’s a burst of quick improvement. (Punctuated equilibrium?) All runs seem to still be on the climb; I suspect that a moving average over a few generations and/or some statistical tests for convergence would give a clearer picture of what’s happening with the max trend; the average was still definitely on the rise at the end of all three runs.

Notes

Test coverage could be improved, especially with the use of mocks. I may move to CDH3 and try MRUnit to ease test design. As it stands I’m taking most of the logic out of the mappers and reducers and just using the overridden map() and reduce() functions to coordinate the calls that do the actual work, largely to make it easier to test the logic in isolation.

For the sake of speed, I could probably take out some of the square roots and do my scoring using the squares of the distances.

On my 13″ MacBook Pro with Core Duo processor running 2 mappers and 2 reducers, a population of 10,000 took 45-60 seconds to cull, breed, and score. The time per generation didn’t change appreciably when I went from 10,000 chromomsomes per generation to a population of 100,000 – which means most of my time is still spent in Hadoop overhead, which means that even on my little laptop, I could put up much bigger populations (or much bigger maps) and still run 500 generations in about 6 hours.

I haven’t bothered taking the simulation parameters in via the command line yet; this would be an obvious improvement, and probably the next thing I’ll do. I was a lot more focused on getting the GA code right and checking that it behaved as expected than I was with making a usable tool. I’ve gotten the itch to generalize and grow this thing, though, so I’ll need to start making it usable & configurable & scriptable.

Clearly, this is not a general toolkit for GA, or even GA applied to NP-hard problems. (I don’t like to generalize without at least two cases.) I do see plenty of places where it is generalizable, though, and will starting doing so as soon as I pick a second problem to attack. A long-term goal is to factor out a library/API for doing GA on Hadoop.

Genetic algorithms are not one single thing – there are multiple strategies for culling populations, doing crossover/breeding, etc. A longer-term design goal is to make each of those variable pieces of the algorithm into pluggable strategies selectable from the command line.

There’s optimization to do all over the place, e.g. with the multiple copies of data kept in the reducer. I chose the data structures I used for ease of implementing the algorithm – e.g., picking a Set to hold survivors so as to avoid picking up duplicate survivors – with the intent to get working software quickly, and optimize last.

The Code

The code, such as it is, is in a public repository on GitHub. More to come.

Jeff Atwood once blogged: “I don’t think it matters how you conduct the postmortem, as long as you do it“.  To that end, here’s my post mortem of the app I didn’t finish.  Ideally  other developers will learn from my successes and mistakes.  :)

The App/Idea: A web version of my iphone app ( Touch Playbook ) called Playbook Nodes.  I learned a lot creating TPB and I figured I’d learn a lot porting the code as well.  I emailed the contest organizers early to make sure I wasn’t going to violate the following rule:

“Of course, feel free to work on the concept for your application before the competition starts: pretty hand-drawn pictures on napkins, angry email threads between team members on two-space or four-space indentation, schema designs, finite state machines.  Just no digital assets including graphic design assets, code, etc.”

~Node Knockout Rules

They gave me the green light, so I starting researching the parts of the app I’d need to understand.  Looking back, here were my mistakes:

Mistake #1: Doing too much
Heading into the competition, my goal was to recreate the entire app.  That includes the user interactions (dragging players, passing the ball, shooting, etc), full replay animation logic, & state management.  On top of that, I wanted to add new features like user accounts and the ability to share pays.  Considering that it took me 5 months to build TPB (1 month to spec / 3 months to implement / 1 month of user testing ), this order was WAY too tall.  In hindsight, I should have simply picked another project.  Anything TPB inspired that’d be worthwhile would simply take too long.

Mistake #2: Assuming that things will simply fit together
I assumed that I’d simply figure out how to elegantly draw paths as the user dragged players (and animate the process later) despite the fact that I couldn’t find good examples of animating the drawing of paths and I’d never done it before using Raphael.  I also assumed Mongoose-Auth would be easily to implement and…well..would just work.  Both of these assumptions cost me precious time.

Mistake #3: Oversleeping
I was hoping to get about 5-6 hours of sleep each night.  Instead I got 7-8 each night.  I felt great, but I wish I had the extra hours to code.

Mistake #4: Failing to achieve a “thin line through” in the first 24 hours
This might be the most telling sign of failure in a 48 hour competition.  It’s really important to get a barebones version of your app working before you continue working on the features in your spec.  The 24 span INCLUDES server setup.

It wasn’t all bad though.  Here are some things I did right:

Success #1: Spec’d as much as I could BEFORE the competition
Mockingbird was super handy way to mock views & my trusty notebook saved my thoughts in the weeks leading to the competition.  Also, I was able to pull my notes from the creation of TPB which helped a bunch.

Success #2: Improved my overall JavaScript skills & got comfortable writing node JavaScript before the competition
I’m new to Node.js, but I’m not new to web development.  I have to shamefully admit that I used to think of JavaScript as a sub-language whose only use were silly DOM tricks.  Fortunately, I started following the works of Douglas Crockford and saw the light.  Not too long after that, I start hearing about NodeJS & I started soaking up examples around the first week in August.  I turned my knowledge harvesting into overdrive & gained the confidence I needed to enter the competition .  Here are some of the things that I did:

• Watched 7.5 hours of Douglas Crockford’s presentations
• Also kept a copy of “Javascript: The Good Parts” close by at all times
• Attending a free training by the co-founders of Nodejitsu, (@hij1nx & @indexzero)
• Watching as many NodeTuts as I possible could and coding the examples in the vids.
• Felix’s Node.js Guide
• Tons of links to helpful content & people to follow (twitter, blogs, etc).
• Watching videos by Ryan Dahl, the creator of Node.js

Success #3: I went with Heroku instead of Linode or Joyent
I didn’t want to lose time configuring a server.  The purpose of the competition is the app that I create, not testing my Sys-Admin skills.  Going with Heroku saved me the headache that many of my fellow participants faced.  Some teams never got their server setup, but I was deploying early and easily.

Success #4: Logging my actions
Every hour, on the hour, I wrote a summary of how I spent my time.  If I started writing a bunch of code, I’d shorten my logging to 30 minute intervals.  This was helpful because it drove me to get small victories at least once an hour.

Success #5: In Backbone.js apps, add a global logging function
I started doing this in every Backbone app that I create.  I add 3 simple methods to my controller/router definition.  This gives me a name-spaced way to log & benchmark events from my controllers, models & views.  It also gives me a clean way to disable logging when I deploy to production & an easy way to toggle the logging once the code is deployed to prod.

• debugEnv()
• The router/controller has a variable called ‘debug’ which is set to true/false.
• The first line of the method looks like this:
  return (this.debug && (typeof console !== ‘undefined’)); 
• logEvent(message)
• This method returns if the controller/router’s debug method is off
  if(! this.debugEnv() ){ return; }  
• Otherwise:
  console.log( (new Date()).toTimeString() + “: ” + message); 
• benchEvent(handle, operand_string)
• This method needs to use console.time() & console.timeEnd() so it makes sure that we can debug AND that the ‘time’ method exists (This is a problem on IE):
  if(! this.debugEnv() || typeof console.time === ‘undefined’ ){ return; }
• Otherwise, depending on the ‘operand_string’, …
  case ‘+’ : console.time(handle); break;
  case ‘-’: console.timeEnd(handle); break;

This makes it really easy to track what my app is doing and provides simple benchmarking.  I find it super helpful! 

Success #6: Entering Node Knockout 2011
The biggest success was taking part in the competition.   I learned so much more about ExpressJS, Jade, Stylus, Mongoose, MongoDB, CSS3 Gradients, and more because I challenged myself.  Was I ready for every single challenge I faced? Absolutely not!  However, failing with focused effort is better than never trying at all.

In Conclusion
I look forward to Node Knockout 2012.  I’m probably going to start spec’ing even earlier and I’ll be more realistic about what can be accomplished in 48 hours.  I also intend to complete PlaybookNodes.  While I didn’t get everything done in 48 hours, I put together a LOT of plumbing and I feel that I’m potentially only a few days away from a finished product.  I hope my experiences help you plan your next 48 coding session better than I did.

Oh, and now that I’m done with NKO2011, Part 2 of the Backbone.JS tutorial is on it’s way!

Till next time, Happy Coding!

Gotcha 1 – The .gems file:

The ‘.gems’ file is SUPER important.  Without it, Heroku won’t be able to install the gems that your app needs to function properly ( like…say…Sinatra ).  It’s mentioned as a single line in the linked page above, but it’s worth mentioning because it’s easy to miss.

Gotcha 2 – Heroku’s read-only Filesystem:

There are certain files that you shouldn’t put into source control, like an API key from the New York Times (hint, hint).  Historically, I’ve added files like these to the server directly (along with other sensitive keys/data).  While Heroku’s read-only file system doesn’t allow this, it does have a nice solution:  Define sensitive values via the Command Line Interface:

heroku config:add API_KEY=’ABC123′

… and reference them in code like this:

my_key = ENV['API_KEY']

Those are the two VERY simple gotchas I ran into which shouldn’t distract from how awesome Heroku is.  Create your free account and play with it.

In other news, Part 2 of the Backbone.js tutorial series is coming soon.  In fact, the site’s layout was changed to accomodate the code and text for the tutorial.  You can watch the unfinished product evolve on GitHub & play with it on Heroku.

Happy Coding!

It’s been a while since our last post.  Gabe’s been hard at work in Australia, hacking on a number of things.  Rebecca & Brad have large, awesome projects in the works.  Meanwhile  I’ve switched jobs a couple of times .  All the while, we’ve been coding like mad and learning a ton of interesting things.  While I can’t promise that we’ll “turn the posting faucet back on”, I can say that we’ll try to reduce the big draughts.  Now then, on to the show!

Backbone.js Philosophy (as I see it)

I’ve been getting pretty friendly with Backbone.js lately.  This 2-part post will teach you some backbone basics & show you how to create your own backbone.js app with multiple routes.

Before we begin, take a look at some of these other tutorials to get a sense of what we’re about to do.  I learned a LOT from them:

• Document Cloud’s backbone.js framework documentation
• Jérôme Gravel-Nique’s ”Todo list” example
• The Meta Cloud’s backbone tutorial
• Liquid Media’s backbone tutorial (3 parts)
• Elf Sternberg’s backbone tutorial
• James Yu’s tutorial
• David Richard’s thoughts on backbone.js

The most important thing to understand about a Backbone.js app is the separations of concerns.  Each of the parts below can do even more than what I’m describing, but here’s the minimum of what you need to know about them.

Models and Collections (an array of models)

• knows how to speak to the data source (both restful resources and api calls).
• can parse data before handed responses off to a view.

Views

• knows how to display your models and collections.  You define the markup rules in the render method
• handles events (clicks, changes to markup, changes to models) and changes the markup appropriately.
• can initialize your js plugins that should be attached to markup.

Gabe had this to say about views:

“I see two ways to use backbone views:

1. A view can be bound directly to a model or collection.  Whenever that model/collection changes, the view would re-call it’s render method and redraw itself. In essence, the view’s job is to represent a model’s state in the DOM now.  Also keep in mind that the model doesn’t have to be something bound to your db.

2. A view can be more of a stateful widget that remembers some of its own state.  In this widget style, the view would keep track of some state and maybe decide what parts of itself it should show/hide depending on its state.

In conclusion, the two view philosophies are very similar:

- one is more modular
- one is more monolithic

The view can be an island OR it can be obsessed with something.“

Thanks Gabe!

Controllers/Router

• handle all of your ‘#’ routes.
• the app is started by creating a new instance of this class.
• your app only needs 1 controller/router

[5/7/2011 : Amendment]  A quick word about markup

In a Backbone app you’ll have markup that lives on your page (index.html) & markup that you’ll need to generate as new elements are added (This will become clearer when you see our tutorial app in action in the next post).  While Backbone gives you view.make() to create markup, you’ll want to use a templating library to help you out with more complicated stuff.  A fellow KAL member recently how I draw the line between what markup lives on the page and what markup lives in a template.

My Rule: If the markup has an ID, then it should live on the page.  By maintaing this idea, you can create your page layout in it’s empty state.  The Views that you create will attach to ID’ed elements to dynamically fill/update them as events occur.  Also, with Backbone, you don’t need to have markup that contain the object’s ID (“foo_1″, “foo_2″, “foo_3″, etc).  The object is bound to the markup (if you do it correctly) which removes the need to parse it out of the ID in the markup (or other well known chicanery that we’ve all done at one point or another ).  It’s often better to have your template markup use the ‘class’ attribute declaration since they’re best used when you’re representing models in a collection.  This also fits with the W3C’s HTML spec which says that an ID/Name “must be unique in a document“.

Consider my rule when constructing a table (the tutorial app will do this):  The table’s markup lives on the page with an ID (id=”MoviesTable”) without any rows.  We’ll generate/update the rows (class=”movie”) of the table on the fly via a template fueled by my View’s Collection.  The Collection is filled by the response from our movie API.

That’s a lot to digest

Stay tuned for Part 2 where we’ll build our app, “Comment on the Movies”.

http://stackoverflow.com/questions/1197746/uiactionsheet-cancel-button-strange-behaviour

Long story short, if your launch an action sheet in a view that lives in a UITabBarController, the “hit” box for the cancel button gets shifted in a VERY STUPID WAY!

The solution is to reference the view you’re displaying in by the UITabBarController like this:

[sheet showInView:self.parentViewController.tabBarController.view];

Freaking WOW! Thanks a heap, Apple!

*grumble grumble*

private int MyAwesomeFunction(int pVar)
{
// coding magic
return pVar++;
}

…where pVar = 10, the returned value will NOT be 11!  The returned value will still be 10.  If you want to return an incremented value for pVar, do the following:

private int MyAwesomeFunction(int pVar)
{
// coding magic
return ++pVar; // or return pVar+1;
}

Just a case of postfix vs prefix.  In the previous example, the variable never gets incremented because the postfix operator would get evaluated after the line is processed, but that never happens because we’re returning the pVar.  By flipping to a prefix operator, we guarantee that we evaluate the opperator against the variable before we return it.

It’s also worth mentioning that being a little verbose here is also good.  Explicitly stating that you’re adding 1 to the variable (as displayed in the commented code) is the same number of characters and we eliminate the confusion over what’s getting returned.

Happy Coding!

1] Denial

It doesn’t really say that, does it?  I mean, they  must not realize what this will do to other projects not related to Adobe.  I’m sure they won’t enforce it.

2] Anger

Are you F*%#ING SERIOUS RIGHT NOW!?  Don’t they realize that some EA games use Lua!?  Are they going take those apps out of the store?  Don’t give me that “quality of apps” crap!  There are TONS of horrible apps written in native Objective-C!  “Crappy devs will make crappy apps regardless of how many layers there are & it doesn’t make sense to limit source-to-source conversion.”  This is creativity stifling!  Apple’s going to lose so many apps they’ll repeal this decision with the QUICKNESS!  I’m just going to bang out the best apps ever and we’ll see if Apple tries to take them out of the store.  I DOUBT THEY WILL!  What happened to the approval process!?  Shouldn’t that be enough!?  Don’t they understand how error prone manual memory management is!?  FURY!!!

3] Bargaining

Maybe we can team up with other developers and convince Apple to change this.  We can show them of the talent that’ll jump ship.  I mean, we’re not like those “sexy app” guys, right?

4] Depression

It’s over man!  It’s GAME OVER! Damn it!  I wasted all this time! What will become of my projects!?  I’m tossing out my iPhone & I’m definitely not buying an iPad now.  It’ll just be too painful of a reminder that my app is available on it**.  Nothing to see here except a teary-eyed developer

5] Acceptance

This sucks likes crazy, but I’ve still got time.  I’ll make updates to Touch Playbook for as long as I can.  MonoTouch is talking about giving developers the ability to create Droid apps and that will kick ass!  Windows Phone 7 is right around the corner and since my app is already written in C#, the port should be very easy.  The HP Slate runs Windows 7 and obviously Silverlight so there’s another port right there.  I’ll get started on that this weekend!  If it wasn’t for MonoTouch, I wouldn’t have gotten as far along as I did with Touch Playbook.  I don’t regret a moment.

Conclusion

I don’t know what the future of MonoTouch and iPhone development is.  Rather, I don’t want to admit what I fear the future is.  I’d love for a miracle to take place where the reported language is altered in 3.3.1 to allow MonoTouch and other quality iPhone development solutions.  However, if that doesn’t happen, MonoTouch still definitely rocks!  For the time being, it feels good to be in the final stage.  I don’t like being so angsty.

Update

I do, however, remain hopeful.  I’ve signed the developer driven Tiny Petition against Apple’s reported TOS change and you can too.  You might ask “Abel, isn’t this the barganing step?  Did you relapse?” and I would reply “Not exactly”.  In the bargaining step my thoughts came from desperation in the wake of anger.  Right now, signing the petition springs from hope and a desire to stand with my fellow MonoTouch devs and other devs effected by this.  This time the motivation is less about me and more about the community.

**I’ve already taken a stab at learning Objective-C and would rather use my time on other languages.  I don’t see myself taking up the effort again.  Life’s too short and I really like languages like C# & Ruby.

Internationalising my app

…and make sure the folders you use follow this format “[Language Abbreviation].lproj” as commenter #4 points out

If you miss one of the major languages, your app will default to English.  As of this post, the iPhone OS supports the following languages:

English (U.S), English (UK), French (France), German, Traditional Chinese, Simplified Chinese, Dutch, Italian, Spanish, Portuguese (Brazil), Portuguese (Portugal), Danish, Swedish, Finnish, Norwegian, Korean, Japanese, Russian, Polish, Turkish, Ukrainian, Arabic, Thai, Czech, Greek, Hebrew, Indonesian, Malay, Romanian, Slovak, and Croatian