Surfing in Kansas feed

Enable setup.py test in your Django apps

Mon, 29 Jun 2009 23:33:07 +0000

Setuptools comes with a way to run the tests on your application. This allows the user of your software to download it, and run python setup.py test and check to see if the tests in your application pass. This is really useful for distribution, because the user doesn't need to know or care how to run your tests (nose, django, unittest, py.test, or whatever else), and can simply see if they pass.

To do this, you simply define a test_suite variable in the setup() function of your setup.py. This argument is a callable that should return a test class. However, since Django has it's own test runner, we have to point this at a simple test runner that we construct, and allow that to run the tests. This is because we must set a couple of environmental things, like the settings module and PYTHONPATH.

I did this with my test_utils project, you can see the commit here, but basically I simply added this line to my setup.py:

test_suite = "test_project.runtests.runtests",

Then put this in the file test_project/runtests:

#This file mainly exists to allow python setup.py test to work.
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'test_project.settings'
test_dir = os.path.dirname(__file__)
sys.path.insert(0, test_dir)

from django.test.utils import get_runner
from django.conf import settings

def runtests():
    test_runner = get_runner(settings)
    failures = test_runner([], verbosity=1, interactive=True)
    sys.exit(failures)

if __name__ == '__main__':
    runtests()

Note that there is a bit of path and settings hackery, this is to enable Django to run correctly, with the test_project in the PYTHONPATH. However, now if you want to run the tests, you simple do a python setup.py test. You can also do a python runtests.py to have the same outcome. You could also have the runtests function simply be a call_command('test'), but without the explicit sys.exit, setuptools complains that it hasn't been given a TestCase back.

This has a couple drawbacks in that it simply runs the entire test suite. You can pass in a Test Suite to setuptools, but that isn't how the tests are organized in most Django apps. However, if you want to run specific tests, you can still test things the regular way through manage.py test. Presumably there is a better way to do this, but it's good to at least have a hacky way until a better way emerges.

There is a lot of value in providing a standard interface to running the tests on your application though. This allows the distribution tools (pip and setuptools) to run the tests on your application if they'd like. In the perl universe, CPAN runs the tests on apps before installing them, and quitting if they fail. If more people started making setup.py test work, we could hopefully add this ability to Python's distribution tools, and make the world a happier place with better working software.

Migrating Django Test Fixtures Using South

Thu, 11 Jun 2009 23:16:13 +0000

The Problem

Migrating test fixtures is one of the biggest pains of testing. If you create your tests too early, then change your schema, you have to go back and touch all your old test fixtures. This discourages people from writing tests until their app is relatively 'stable'. As we all know, this may never happen :) This solves half of the problem, the part where you have to manually change a bunch of fixtures to reflect changes in your schema or data.

Possible Solution

During the questions in my EuroDjangoCon talk, someone asked a question about this. I didn't have a good answer, but someone from the crowd raised their hand and suggested using South. Once your project has data migrations for it's real models (like any production site should), it should be relatively easy to then load up your test fixtures, migrate your database, and dump them back out.

I will be writing out a pretty basic tutorial on south, alongside of the example of how to migrate your test data using South. I think it's pretty fantastic. I hope that if you aren't already using south, this tutorial will show it's simplicity and power, and if you are, I hope to show you another way to use it. If you already understand south and have data migrations, you can skim the Example section and just focus on the later part.

I am using my Django Test Utils's test_project at a certain revision for this demo, so you can look there and follow along if you want to see the actual files used. Look at the next commit to see the outcome of the project :)

Example

I went ahead and made a simple little example application to test this on. It will be the common migration scenario of adding a slug to a model. We will be using the Polls app that everyone knows and loves from the Django Tutorial.

I don't currently have south installed at this point in test utils, so if you're following along, you just have to download south and add it to the installed apps before you start.

Basic Setup

So you have the basic polls models.

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(max_length=200)
    votes = models.IntegerField()

We realize that we don't have a way to show these well on the site, because they don't have a slug. So we want to add a slug to the Poll model. First off you need to have the initial migration for your app, so that we can migrate it. We are going to use south, so we need to create our initial migration.

$ ./manage.py startmigration polls --initial 
Creating migrations directory at '/Users/ericholscher/lib/django-test-utils/test_project/polls/migrations'...
Creating __init__.py in '/Users/ericholscher/lib/django-test-utils/test_project/polls/migrations'...
 + Added model 'polls.Poll'
 + Added model 'polls.Choice'
Created 0001_initial.py.

Adding your fields

As you can see, south knows how to add a migrations directory to your app and fill it up with the correct migration name. Now lets go ahead and edit our model.

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')
    slug = models.SlugField(null=True)

As you can see, we added a slug field. It has to be null=True because we will be creating a lot of them, and they must be able to be null before we can add the data to them. Now lets go ahead and create two migrations. We want to add one that creates the field, and then we want to create one that fills out the slug from the question field.

$ ./manage.py startmigration polls add_slug --auto
 + Added field 'polls.poll.slug'
Created 0002_add_slug.py.

Writing your Data Migration

This creates the migration that we want that allows us to add the field. Now we go ahead and run startmigration again, just passing the app name. This creates a stub migration for us, with the model serialized on the bottom, which allows us to just write the code we care about.

$ ./manage.py startmigration polls populate_slug_data
Created 0003_populate_slug_data.py.

Note that it is a good practice to separate your migrations that effect your table structure and things that actually migrate data. Now we go in to the migration and add in the code that migrates the data. It will end up looking something along these lines.

from south.db import db
from django.db import models
from polls.models import *
from django.template.defaultfilters import slugify

class Migration:

    def forwards(self, orm):
        for poll in orm.Poll.objects.all():
            poll.slug = slugify(poll.question)
            poll.save()

    def backwards(self, orm):
        "Write your backwards migration here"
        for poll in orm.Poll.objects.all():
            poll.slug = ""
            poll.save()

... Chopped for clarity ...

As you can see, the migration is really simple! This uses a fake Django ORM (which is just the real one, loaded a different way.) Now you can go ahead and test out your fancy new migrations.

Running the migrations on your test data.

Now as you see, you have these fancy migrations that actually haven't touched your database yet. I'm going to walk through the entire process of creating your database from the syncdb stage to the outputting of your shiny new test fixtures.

Setting up your test database

So the whole point of this exercise is to be able to migrate your test fixtures the same way you do your real database. This means that we simply load up a new version of our database with our test data, run our migrations, and serialize it back out, ready for our tests.

Go ahead and run syncdb on your project. This will do all the normal things you're used to, except that at the bottom of the output, you'll see a message about things not being synced because of south:

Synced:
 > django.contrib.auth
....

Not synced (use migrations):
 - polls
(use ./manage.py migrate to migrate these)

Now we need to go ahead and get the polls data in our database at the point where our fixtures exist. This means that we only want our initial data to be loaded. So we go ahead and tell south to migrate to our first migration.

$ ./manage.py migrate polls 0001
 - Soft matched migration 0001 to 0001_initial.
Running migrations for polls:
 - Migrating forwards to 0001_initial.
 > polls: 0001_initial
   = CREATE TABLE "polls_poll" ("id" integer NOT NULL PRIMARY KEY, "question" varchar(200) NOT NULL, "pub_date" datetime NOT NULL); []
   = CREATE TABLE "polls_choice" ("id" integer NOT NULL PRIMARY KEY, "poll_id" integer NOT NULL, "choice" varchar(200) NOT NULL, "votes" integer NOT NULL); []
   = CREATE INDEX "polls_choice_poll_id" ON "polls_choice" ("poll_id"); []
 - Sending post_syncdb signal for polls: ['Poll']
 - Sending post_syncdb signal for polls: ['Choice']

Migrating your test data

As you can see, this created out database table without the slug field. This is good, because our fixture data doesn't include the slug field. This is where things get a bit annoying. The loaddata command uses the models that are on disk to check if the data loads correctly. So you need to check out your code at the revision before the migrations were applied (in our case, we can simply comment out the slug line). Then you are able to go ahead and load your test data.

$ ./manage.py loaddata polls_testmaker
Installing json fixture 'polls_testmaker' from '/Users/ericholscher/lib/django-test-utils/test_project/polls/fixtures'.
Installed 8 object(s) from 1 fixture(s)

Then you can put the slug back in (or check out the current version of your code). Now you have your data in your database in the old un-migrated form. Let's go ahead and migrate out test fixtures :)

./manage.py migrate polls
Running migrations for polls:
 - Migrating forwards to 0003_populate_slug_data.
 > polls: 0002_add_slug
   = ALTER TABLE "polls_poll" ADD COLUMN "slug" varchar(50) NULL; []
 > polls: 0003_populate_slug_data
 - Loading initial data for polls.

Now lets see if that worked. Let's go ahead and run dumpdata and see what all you have.

./manage.py dumpdata polls --indent=4
[
    {
        "pk": 1, 
        "model": "polls.poll", 
        "fields": {
            "pub_date": "2007-04-01 00:00:00", 
            "question": "What's up?", 
            "slug": "whats-up"
        }
    }, 
... snip rest of data ...

You now have your migrated data fixture! Hopefully everything worked for you, and that this works for larger examples other than this trivial example.

Conclusion

The little bit at the end where you have to revert back to the old version of your code to use loaddata is a bit of a hack. With a bit of tinkering, you should be able to use south's serialized representation of the model instead of the models on disk in order to load the data. Doing this will make this whole process more seamless.

If you would like to see the changes to the models and fixtures, and migrations that were created, you can check out the south demo branch of test utils.

I would also like to thank Andrew Godin for creating south, of which none of this would be possible.

Thanks for reading, and I'd be curious to see what people think, and if there are some improvements that could be made.

A playground for Django Template tags and filters

Sun, 24 May 2009 13:32:59 +0000

The Problem

Any sufficiently large Django project starts to have a wide variety of Template Tags and Filters. Even Django ships with a dizzying array of them that allow you to do all sorts of fun and interesting things. Ellington, our CMS at work, has a ton, and I've been thinking about ways to make tags and filters a bit more accessible to people who are using the CMS.

I'm thinking along the lines of people who are tech savvy, but who were just hit with a huge wall of tags and filters to look at. I want them to be able to really easily play with the functionality and see what it does.

The Solution

I created a proof of concept playground for tags and filters in the django admin. It is released as a simple third party app that I have up on Github. Here is a small 1:40 minute screencast that explains what I did:

Django Admin Playground from Eric Holscher on Vimeo.

It gives you a "Play" link next to each of the tags and filters in the admin. Once you click on that, if the docstring for the tag has a code example, it attempts to parse that out. This allows you to easily test out the examples that you should have in the docstrings for your tags.

It displays the docstring above the input areas and allows you to input context variables (naively) and render the template. It uses Jquery to do an ajax post and response that is displayed on the right side of what the output of the template would be.

A simple example with the Add Template Tag:

Caveats

This is very hacky and basic code. Totally just a proof of concept and might not work for you. I think that the ideas are worthwhile, and something that could be included in Django at some point.

Currently the context values are just being parsed at the : and split into a dictionary. If anyone knows a good way to turn a basic list like this (using YAML?) into Django objects, then I would be all ears. I thought about it a little bit but couldn't think of an elegant solution.

Also the parsing of the templatetag syntax out of the templates is incredibly simplistic. If I took some time and played around with ReST I'm sure I could figure out a better way to do that (Pulling out the "code blocks" somehow?). But a basic regex worked well enough to get the idea done.

Feedback welcome.

EuroDjangoCon Talk: Testing Django

Tue, 05 May 2009 11:59:07 +0000

Just got off the stage at EuroDjangocon, which was my first real talk in front of the Django Community. I hope that people enjoyed it, and that it was informational. Here are the slides to my talk in PDF Form, and on slideshare.

Testing Slides

I hope that people have questions or constructive feedback, and I'd love to hear what everyone thought. I will be writing a full writeup at the end of the conference, but I'll say that it has been amazing thus far!

Django's Summer of Code students announced!

Mon, 20 Apr 2009 15:45:26 +0000

Today is the day that Google has announced the accepted projects for the Summer of Code. Django has 6 spots this year, with a bunch of exciting projects. I am lucky enough to be mentoring Kevin Kubasik with his project "Upgrade the Awesomness Quotient of the Django Test Utils and Regression Suite". I'm really excited for the opportunity to help improve Django by overseeing Windmill testing of the admin, and lots of other small testing improvements that will hopefully make it into trunk.

I would like to congratulate all the students that got accepted, and to everyone who didn't, you can still contribute! Also remember that contributing through the year will make the chance of you being accepted next year a lot higher! Here is the full list of accepted proposals:

Honza Král, "Model aware validation"
Mentor: Joseph Kocherhans

Kevin Kubasik, "Upgrade the Awesomness Quotient of the Django Test Utils and Regression Suite"
Mentor: Eric Holscher

Christopher Cahoon, "Improved HTTP and WSGI Support"
Mentor: Malcolm Tredinnick

Zain Memon, "UI improvements for the admin interface"
Mentor: Jacob Kaplan-Moss

Marc Albert Garcia Gonzalo, "Implementation of additional i18n features"
Mentor: Jannis Leidel

Alex Gaynor, Multiple Database Support in Django
Mentor: Russell Keith-Magee

Note that you can view the full list with abstracts on Google's site.

Looking forward to a summer full of great new features and amazing community involvement for Django and all the organizations participating. A big thank you to Google and Jannis Leidel for sponsoring and administering Summer of Code, respectively.

Testing AJAX Views in Django

Thu, 16 Apr 2009 14:16:18 +0000

A lot of the Django code we use at work has a special case for AJAX. It has been a kind of a pain to test, because the test client by default doesn't use AJAX. Luckily the is_ajax call in the Django HttpRequest object is a simple check of an HTTP Environmental variable.

An undocumented feature of the Django Test Client is that you can pass in custom HTTP ENV variables on requests. The definition of get for example is:

    def get(self, path, data={}, follow=False, **extra):

Later on in the file, the request environment is then updated with the extra keyword args: r.update(extra).

This lets us throw in arbitrary variables in our get and post requests in the test client. Like so:

  r = self.client.post('/ratings/vote/', {'value': '1',}, 
                                HTTP_X_REQUESTED_WITH='XMLHttpRequest')

Note that the custom env is outside of the dictionary of get parameters. This will now return the /ratings/vote/ view with the output that is normally called on an AJAX request.

Adding Google Analytics to Sphinx Docs

Sun, 05 Apr 2009 13:12:42 +0000

This is just a reminder for myself later, or people looking on Google. Also note, that this method is useful for putting any Javascript content into your sphinx docs, but Analytics tracking is a common use case.

Step 1: Where to put my files?

Check your conf.py on your Sphinx docs. You need to make sure your templates_path variable is pointed to a directory that exists. It is relative to your current directory. I use _templates, which I believe is the default. This is where you can override Sphinx templates. They use Jinja2, which is a relative of Django templates, so it should be pretty simple if you're used to Django templates.

Step 2: Override the default template

In your _templates directory, add a file called layout.html. The Sphinx Docs are pretty good in this area, containing a full listing of all the template that you can override. The Sphinx Source layout.html is also really nice, so you can see what it is by default.

Analytics says that you should put your analytics code "Right before the </body> tag" on your site. This means at the bottom of the footer. Google should have given you a piece of Javascript code to paste in your site, copy that below. So in your new layout.html, put in the following code:

{% extends "!layout.html" %}

{% block footer %}
{{ super() }}
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("YOUR_GOOGLE_CODE_HERE");
pageTracker._trackPageview();
} catch(err) {}</script>
{% endblock %}

Take note of the {{ super() }} call. This means that you are calling the inherited template's code, which pulls in the default Copyright notice into the footer. Then you can put in your custom code after that.

Let me know if there are any other neat Sphinx tricks and tips that you have. I'm in love with the software and learning more about it daily.

Really easy SSH tunneling

Sat, 21 Mar 2009 16:45:47 +0000

SSH Tunneling has become an invaluable tool that I probably use more than I should. I love tunneling, and use it all the time. This will be a quick tutorial on how to use the SOCKS proxy ability of SSH to allow you to tunnel your HTTP traffic through a remote server.

This is useful when you're on a connection that has a silly filter on it (school or library). Since it's a SOCKS5 proxy, it is useful for tunneling other things as well (like IM). It is also useful when browsing on public wifi or anywhere that you can't trust the network connection you're on, since it encrypts all the data that is sent over it.

SSH

The command to tunnel in SSH is really simple. You simply do: ssh -ND localhost:5555 example.com to tunnel traffic through example.com. This is a nice one off, but I actually have the configuration in my ssh config. To do that, in your ~/.ssh/config, you need to put in the settings you want your proxy to have.

Host tunnel
    Hostname example.com
    DynamicForward localhost:5555

This allows me to simply do ssh -N tunnel, and it will setup a proxy. This is basically turning my local port 5555 into a proxy that goes through the remote host. It is encrypted from my network to the remote network, which is really nice. The -N flag is used so that it doesn't create a shell on the other end, and simply creates the proxy connection.

Firefox

In firefox, you need to go into your Preferences > Advanced > Network > Connection > Settings. This is where your proxy settings live. Go down the the SOCKS host, and set it to localhost, with the port you set up above, 5555 in this case. It should look something like this:

I use the Quickproxy extension to easily turn my proxy settings on and off. It puts a small button on your bottom status bar in Firefox, and clicking it turns your proxy on and off.

Now you simply flip the switch on your QuickProxy, and you are surfing through an encrypted connection. To check if it's working, I use http://whatismyip.com to check my remote IP. If it changes between the proxy being on and off, you know the proxy is working.

This is a really easy way to simply create a two click encrypted proxy. Hope this is helpful, and I'd be curious if people have other tips and tricks in this regard.

Twitter Spam

Sun, 15 Mar 2009 22:27:41 +0000

I keep hearing people talking about how twitter is going to be over run with spam now that it is becoming mainstream. I really don't understand this viewpoint, and will take time here to outline what they could be talking about, and what can be done.

This is in reply to http://www.twine.com/item/123c9051b-g8/can-twitter-survive-what-is-about-to-happen-to-it specifically, but these ideas have been mentioned over and over.

Short Version: We need to stop worrying about spam on twitter, and start worrying about all the cool stuff we can make.

Kinds of Spam

"Hypertweeting"

A person you are following is tweeting too much? How is that spam? Simply unfollow them. This is one of the big ones I don't understand people complaining about. It's OPT IN to follow people; if you don't like what they say, unfollow them!

Hashtag Spam

The current implementation of hashtag spam is indeed a problem, because it is a publish and not a follow model. So anyone can include a #hashtag and it will get picked up by a hashtag aggregator. This is the common problem of broadcast mediums. It can be solved filtering hashtags to only certain users, or some other kind of grouping concept. (A twitter account that retweets a hashtag only from the people it follows, for example). You could also do the filtering on the web end, showing only hashtags from user X and Y.

This seems like a problem that could be solved by the hashtag aggregators. Currently they are just dumb aggregators, and adding relevancy would probably be easy. This also screams out as an area where Bayesian Filtering would be useful, since you have a tag that is presumably about a topic.

@reply Spam

This is legitimate. If a spammer gets on twitter and @replies your account, it will show up in your timeline. However, that spammer can only @reply 1 person a minute, and that kind of activity should be really easy for twitter to take care of. Alternatively, twitter could implement an option where you only receive @replies from people that you follow (like the settings to see their @replies to other people). This issue can also be solved in a client by separating @replies you get from people you follow, and those that you don't.

All in all, this is not a very worrisome method of spamming, and if it became used, it would statistically almost never happen to you.

Notification Overload

Again, this is the same as the "Hypertweeting argument". I follow @slicehoststatus because it is just updates about my connectivity. They have a @slicehost account that is more customer service oriented that I don't care about. Services will have logical separation between their feeds or they won't be used.

Solutions

The post does mention some good solutions to the problem. I will address my thoughts on those here as well.

Number of Followers as a Filter.

This seems likely to be gamed, and a trivial filter. This might be useful when combined with other metrics, such as how long a user as been on the site, how many people they are following (and have followed!). This is where the idea of metadata being important comes in.

Re-Tweeting Activity as a Filter.

Perhaps, but this needs to be formalized. I would really like Twitter to formalize RT's so that I can filter them out, because I find very little value in them personally. Twitter already has functionality in it (liking tweets) that seems like a more logical choice to use.

Social Network Analysis as a Filter.

I love the social graph, so I'm a fan of this one. This would be a very resource heavy way of validating anything, and the whole premise of the spam argument is that twitter is growing really fast. I don't think this is a viable option, at least not when applied to every message. There is a lot of interesting data to be gleaned from the social graph, however that is another post.

Metadata for Filtering.

He makes a good point that metadata is what is needed. It will be really hard to do these calculations outside of twitter (especially as it grows, it will be hard internally). I really think that the spam problem is something that is a non starter, and twitter will work just fine without any more measure of spam protection. The metadata will be really interesting for a lot of other applications.

Conclusion

I have yet to see a real post that has made me think that twitter will have a spam problem. The opt-in subscription method is really genius, and makes spam almost impossible. The model of twitter will stay spam free (I will get content from people I follow). External services (search and aggregators) will suffer from spam problems, until they get better (spam) filtering.

I think the real problem of twitter is how to find interesting people to follow, and not how to remove spam. This is where the problem of spam and filtering really come into play. Starting with a network of people you know, and branching from there is how twitter will work. The social graph is really interesting in that realm.

A lot of the conversation above leads itself into other really interesting areas of data analysis. Stopping spam is easy, let's go data mining :)

Google Summer of Code

Thu, 12 Mar 2009 10:18:27 +0000

It's that time of year again, and the Google Summer of Code is happening again. This year Django will be applying again, and there is currently a Wiki page on the Django wiki devoted to ideas and people who want to volunteer to help mentor. I think this is a great opportunity for students, mentors, and the projects involved. It is a really neat learning experience. Even if you can't participate, it's a good chance to put ideas up that some enterprising student might pick up and run with.

Last year had some really successful work done, with Django's Aggregation framework, F Expressions, and a revamped comments framework all coming out of last years efforts!

This is your time to really help give back to the community and help some neat, innovative work get done at the same time. You can also show a student the value and fun in open source work. The focus on SoC is in new feature development, so it's an exciting way to hopefully get your pony into Django. So go add some ideas to the wiki or volunteer to mentor if you have the time and knowledge. Most of all if you're a student, apply! This is a really great opportunity that Google provides you, you get paid, and you get to help make something amazing!