We decided on a whole different approach to a responsive application. We are serving up data. The javascript takes care of the rest. Often times a single action requires multiple reactions by the interface. Yet, if you’ve abstracted your code (be pythonic, even in javascript: D.R.Y!) you can’t jump into a specific call/response, or even if you could, it’d be awful to throw a switch statement in there. The Command Pattern to the rescue!

To be fair, what we’ve implemented is a very simple execution of the pattern. Let’s take the python-end first:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

class CommandBuilder(object):
 
    def __init__(self, *args, **kwargs):
        super(CommandBuilder, self).__init__(*args, **kwargs)
        self.data = []
 
    def raw_callback(self, callback, args):
        self.data.append({ 'callback': callback, 'args': args })
        return self
 
    @property
    def js(self):
        return self.JavascriptMethods(self)
 
    def dumps(self):
        return { 'commands': self.data }
 
    class JavascriptMethods(object):
 
        def __init__(self, command):
            self.command = command
 
        def a_function(self, a_model):
            return self.command.raw_callback('a_function', [ a_model.id, a_model.name ])

Just for some syntactic-sugar I also used the builder pattern and a python property which masks the Javascript Methods inner class. All that means you can write pretty code like this:

1
2

cmds = CommandBuilder.js.a_function(a_model).a_function(a_model).a_function(a_model)
return HTTPResponse(simplejson.dumps(cmds.dumps()))

In the real world you wouldn’t be calling the same javascript function over and over, you would expose the functions you need to call. And now we need a little bit of javascript to handle this known JSON data structure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

var Commander = {
    a_function: function(id, name) {
        alert(name+" has an id of: "+id);
    }
};
 
var Engine = {
    run_command: function(data) {
        if(data == undefined || data.command == undefined || data.command.length === 0) {
            alert('This is not a command structure');
        }
        for(var i in data.command) {
            var row = data.command[i];
            if(typeof Commander[row.callback] === 'function') {
                 Commander[row.callback].apply(undefined, row.args); // undefined as this
            } else {
                 alert('Function '+row.callback+' does not exist on Commander');
            }
        }
    }
};

The javascript `apply` method is a lifesaver here. With this simple pattern we can have the server force client-side code to be executed. Now, if you’ve been a diligent developer you’ve already been naming your functions on the client-side, not just throwing around a million anonymous functions. And you’ve separated out the functions that deal with calling URLs or dealing with the DOM from those functions which deal with the data. There is a huge bonus with doing that (besides this pattern being useful for you): you can start testing your javascript!

Reflections:

• I tend to think of the `.js` in python a namespace. You can specify a specific Javascript object on the front end with a few minor changes on both ends. My `CommandBuilder` object has a few more methods on it, so the `.js` serves as a semantic separator as well.
• There is only one downside to this pattern that I have observed so far – if you make changes to function signatures in javascript, you’ll need to change the python side as well. You need to stay on your toes with this pattern.

If you missed it, or want to review, check out the slides (for the curious, we’re using html5slides for the slide template – we saw it at Google I/O and fell in love.

We’ll post our slides after the session, but definitely encourage everyone in the Boston area to come to the meetup!

We’re psyched to immerse ourselves in all things Google for a few days; here are a couple of highlights we’re especially looking forward to:

John

• Learning about ActivityFragments available in Honeycomb. These seem like an incredibly powerful way to use and organize your mobile applications. To use a web concept – this has the potential to be the native “responsive design” to tablet/phone displays.
• I am eager to see just how far Android can leverage NFC today. This isn’t something I’ve read very much about. The hardware/software rollout plan can be very much a chicken and egg problem, so I look forward to seeing Google’s plan.
• And last, but not least, they have a specifically UI building focused talk, which excites me. The app I’ve been working on is mostly functional – but obviously ugly. I hope I get some ideas and inspiration here.

Kevin

• “JavaScript Programming in the Large with Closure Tools” – JavaScript, a language with many critics but one of my personal favorites, has really come into its own of late. As we write bigger and bigger JavaScript apps, though, we run into more and more problems associated with non-trivial systems – and tools like Closure help make JavaScript more of a “real” language than the “scripting” language it started out as.
• “Writing Web Apps in Go” – I’ll be honest: I’m a bit skeptical about Go’s suitability for web apps. But how can you not be curious about a language designed by such an all-star cast?
• “Map your business, inside and out” and “Location Based App development using Google APIs” – these two talks represent just some of the cool things going on with geolocation.

We’ll be hanging around at I/O – if you’re there too, find us: @activefrequency, @kgrinberg, @jobelenus

But there are two warnings you need to pay attention to if you are considering going with 1.3; 1) CSRF is now required for all ajax calls, and there are some helpful code bits in there to make this an easy addition, and 2) 1.3 is the last version that will support python 2.4. Now let’s get on with the good stuff we are excited about:

• Backend support for Inactive users is going to be very helpful. There are some tricks, and I’ve done some of them, to create active users without proper usernames and passwords. Doing it that way is just strange, and this addition will provide the proper support for it.
• Include and With just got a lot more powerful in templates. This will allow you to create more modular and re-usable templates without worrying too much about template variables in your views.
• In the department of awesome, responses will now be available to decorators/middleware. I can think of a few places in our applications that would benefit by template switching located outside the view specifically.
• This is the feature that most people have been waiting on – and on the dev mailing list, talking about – Class Based Views. At first I wondered how much this would really change things, as the views I now write are very small and fairly boilerplate. However, in looking at the generic classes and examples they have I am excited to dig into this and have to write even less view code. With these kinds of developments all the stuff you hear about Javascript being the next language you must learn – they are right. More than half of the code I write these days is Javascript anyhow. Class-based views are only going to push development more in that direction.
• Two additions were made to i18n for helping translators in the new version (something we have worked with it quite a bit). First are Contextual Markers so you can give more information about what the word actually denotes. And second, comments built right into python comments. I wish I had these already, as we’ve already been in discussing and trying to make schemes to help the translators know exactly what we’re trying to get across.

The Django dev team is fantastic to get this much stuff out the door for version 1.3. You can see the whole changelog here. We here can only wait to see what they have in store for the future. And if you want to see what they’re working on for the future, sign up for the dev mailing list.

One of the striking organizational skills he goes over is the construction of a programming team. I have worked in plenty of different small-medium sized organizations, but I have never been involved in a genuine single-project group-development effort. It has been a single programmer working in isolation. Or, at best, two programmers splitting a project down the middle with a static and defined interface (this tactic is also described in the book).

Only now am I working in an actual group where roles are given. Some of Brook’s roles are: tool-master, architect, and specialist, and program manager. Obviously I don’t need a batch tape scheduler anymore – but the above roles are very key. The program manager represents the client and user interests. The architect represents the sole owner of technical implementation decisions. Now, in all the other organizations I’ve been a part of, these roles undoubtedly exist. There might be two or more people competing or fulfilling these roles – but they exist. What I had not previously experience was the tool-master and specialist.

The tool-master is solely responsible for creating the tools necessary for the team to do their jobs. I’ve seen this manifest as; deploy scripts, local environment tools to mock out production infrastructure, and using and contributing to open source projects (we do not suffer from NIH syndrom). Sometimes this person is me. Sometimes it is not. But the tool-master is always responsible for his tools and to keep them up to date, and to keep the team informed about how to use them. Without this last point members of the team are wasting time figuring something out that has already been done.

I have experienced the specialist to be one of the most invaluable assets to the architect. If the architect is responsible for the whole system and writing most of the code what happens when he or she hits a small thorny problem. Sometimes it is an incompatibility between two pieces of software, or server settings. But whatever it is, someone has to take the red pill and go down the rabbit hole. If your architect goes in, he’ll come out a couple of days later with the beginnings of a beard, and a solution, but features won’t have be written. It is crucial to have a specialist who can take these thorns that need fixing.

All in all the amazing discovery I am witnessing is the overall level of communication on a team that quickly figures out how to design, architect, and implement projects; figuring out the limitations and ways to solve and overcome them.

In the last post Kevin told showed us django-rosetta and a way to package and deploy the translation files. That is certainly one step. I have been a part of a team responsible for a PHP product with support for at least six languages. It got pretty hairy without a tool like django-rosetta and a sane deployment process.

In the Django world, the community is a huge resource – not just for advice and documentation, but also pluggable applications you can use in your projects. One of these we use pretty frequently is django-faq for creating an easy to update, index, and change FAQ system without a client writing tons of HTML. In case you think you are seeing a theme here, well, you are – we are working on an internationalized project. So we had another opportunity to contribute back to the community by internationalizing the django-faq app with the help of the already awesome multilingual-ng app. All in all a very small tweak for us to do – but now with one more fully internationalized app in the tool belt, it’s easier to internationalize the next project. I hope it helps more than just us!

If you’ve never worked with multilingual-ng in models, I’ll give a quick run-through here with the Topic model (but you can see all the basic changes I made in this diff).

1
2
3
4

class Topic(models.Model):
     # normally your name field would go here, but to make it translatable you do this
     class Translation(TranslationModel):
           name = models.CharField(max_length=150)

The rest of the work is magic. And there is an admin class built to handle the display and saving of this as well. It plugs right into the Django ORM. Beautiful!

The built-in tooling is great for creating message files to send to translators, and then deploying them with the application. However, what if you need a quick admin-like way to tweak a translation? Enter django-rosetta, an excellent helper tool that gives you an admin-like interface to your translations (.po/.mo files).

The one problem with django-rosetta is how it fits (or doesn’t fit) into our standard deployment workflow. We use git for source control, and the basic assumption is that we can deploy a build from master at any time.

The problem with that is that on the one hand, the locale message files (translations) clearly need to live with the source; on the other hand, translators making changes via django-rosetta aren’t in a position to learn git and commit their changes back.

The solution we’ve started using is two-fold:

1) Keep the translations in a separate repository, symlinked from the main repo for convenience.

2) Before running “makemessages” (which updates the message files with any changes from the application), we commit and push any changes the translators have made on the server using rosetta, then pull them locally (we use Fabric for deployments, so we just made a fab command for this).

Note that any changes should still go through the normal translation/release cycle, since you’ll generally want to update any translations before releasing the update! But at least this way, translators can make minor changes on the fly without you losing the benefits of DVCS.

There is a small glitch, however. How can I keep everything in the QuerySet so all the magic continues to work while performing the per-line, and total calculations that all clients absolutely need to see for their business? One could place these calculations squarely on the model the QuerySet returns – if they are simple enough. But if you have numbers coming from related objects, are calculating multiple objects in one line, or how many Starbucks there can possibly be in Cambridge, (you get the idea) it gets messy quick. All this still doesn’t solve the inevitable total row the client demands! You end up with functions on ten different models, and you can’t see them on one code-screen. The code that crunches the numbers is quite far away from the template calling it. I don’t know about you, but I forget why I was looking for something when need to grep the models.py for ten minutes to find it.

I am choosing a simple example of an imaginary Customer Report showing all the items they’ve ordered to explain the concept in code, rather than through every bullet-point of my description above. Once you get the concept I think you’ll see how flexible it is and how powerful it can be.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<table>
    <tr>
        <th>First</th>
        <th>Last</th>
        <th>Product</th>
        <th>Order #</th>
        <th>Revenue</th>
        <th>Insurance Revenue</th>
    </tr>
    {% for customer in customers %}
    {% for order in customer.customer_rental_order.all %}
    {% for order_item in order.order_items.all %}
    <tr class="{% cycle 'even' 'odd' %}">
        <td>{{ customer.first_name }}</td>
        <td>{{ customer.last_name }}</td>
        <td>{{ order_item.product.name }}</td>
        <td class="numeric">{{ order.pk }}</td>
        <td class="numeric">{{ order_item.amount }}</td>
        <td class="numeric">{{ order_item.insurance_amt }}</td>
    </tr>
    {% endfor %}
    {% empty %}
    <tr>
        <td>{{ customer.first_name }}</td>
        <td>{{ customer.last_name }}</td>
        <td colspan="4"><i>None</i></td>
    </tr>
    {% endfor %}
    <tr>
        <th colspan="4"></th>
        <th><!-- How to get the total Revenue ? --></th>
        <th><!-- How to get the total Insurance Revenue ? --></th>
    </tr>
    {% endfor %}
</table>

This is where template tags have saved my life. I decided to make some template tags to do the totaling work for me. The total rows are pain because of all the related managers in the way. I needed something abstracted that I could reuse over and over. I wish I could have written this:

1
2
3
4
5

<tr>
   <th colspan="7"></th>
   <th>{{ customer.customer_rental_order.all.order_items.amount }}</th>
   <th>{{ customer.customer_rental_order.all.order_items.insurance_amount }}</th>
</tr>

And have the template magically know to sum everything. In my dreams. Instead I cooked up a template tag I call `related`:

1
2
3
4
5

    <tr>
        <th colspan="7"></th>
        <th>{{ customer|related:"customer_rental_order"|related:"order_items"|tsum:"amount" }}</th>
        <th>{{ customer|related:"customer_rental_order"|related:"order_items"|tsum:"insurance_amt" }}</th>
    </tr>

And this is how it works:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

@register.filter
def related(value, arg):
    if isinstance(value, QuerySet) or type(value) == type([]):
        vals = []
        for v in value:
            if hasattr(v, arg):
                try:
                    for item in getattr(v, arg).all():
                        vals.append(item)
                except AttributeError as e: # arg has no all()
                    vals.append(getattr(v, arg))
        return vals
    elif hasattr(value, arg):
        return getattr(value, arg).all()

In a nutshell, `related` traverses the related managers for you. It takes a queryset, finds the related manager you’ve named, runs all(), and returns that list. When you chain it, next time it passes a list, it iterates the list, looking for the next related manager you’ve named in each one, and groups them all into a single list. If there is no `all` method to run, it just runs getattr. It always returns a list full of the “leaf” objects you wanted to get. You’ll also notice a `tsum` in there (short for “template_sum”) which works like so:

1
2
3
4
5
6
7
8
9
10

def cleannum(num):
    return num if num else 0
 
@register.filter
#tsum for template sum, sum is a built-in
def tsum(value, arg):
    try:
        return sum([cleannum(v[arg]) for v in value])
    except TypeError as e: #means that v is an object
        return sum([cleannum(v.__dict__[arg]) for v in value])

I had being using `tsum` in other reports, and the fact that `related` always returned a list meant this template tag *just worked*. I love this template tag technique for three reasons:

1. All the django template bits like pagination and regrouper which expect iterables/querysets still work. The ORM is still doing the heavy lifting.
2. The template actually shows what is being calculated rather than jumping around in models.py through several models.
3. It is highly abstracted and can work on any object in your application. You don’t need to write similar gathering functions on different models.

If you are ever writing reports (and I sincerely hope you are not) I hope this helps you overcome some pitfalls!

Here at Active Frequency we try to contribute back to the community by releasing our own open-source code, and – starting today – making direct financial contributions to projects we think are particularly valuable and under-recognized.

The first recipient of our largesse is mod_wsgi – a critical part of our Django deployment stack. This morning we donated $167.29 to Graham Dumpleton, the sole author of mod_wsgi (the funny amount is because we actually purchased a bunch of items from his Amazon wish list, as requested in mod_wsgi’s How To Contribute Back page).

So, what does mod_wsgi do, and why is it important? In the author’s own words:

The aim of mod_wsgi is to implement a simple to use Apache module which can host any Python application which supports the Python WSGI interface.

Essentially, mod_wsgi is the layer between the web server (Apache) and the application code (Python/Django), helping one “talk to” the other, and handling many of the details of HTTP transport. To be precise, mod_wsgi is the leading implementation of the WSGI standard, which is the Python-world equivalent to the CGI standard that people who did web programming circa 1996 will likely remember. James Bennett has an excellent (albeit slightly dated) write-up that indirectly delves into the details of what the various WSGI implementations actually do (by way of outlining his issues with the WSGI standard).

Tools like mod_wsgi aren’t sexy – they’re not something that consumers typically hear about (how often do you see a “powered by mod_wsgi” badge on a web site?) But they’re vital pieces of “plumbing” for the Django deployment stack, and we’re proud to support them.