DjangoTricks

The Usage of the only() Method

2011-12-16T21:23:00.000+01:00

Recently I found the only() method of a queryset which can significantly improve the loading speed of forms with ModelChoiceField or ModelMultipleChoiceField. Let's say we have a big Event model with thousands of instances:


class Event(models.Model):
    title = models.CharField(_("Title"), max_length=200)
    # ... lots of other fields ...
    def __unicode__(self):
        return self.title

Then we have a form with ModelMultipleChoiceField:


class EventForm(forms.Form):
    related_events = forms.ModelMultipleChoiceField(
        queryset=Event.objects.all().only("id", "title"),
        label=_("Related Events"),
        required=False,
        )
    # ... other fields of the form ...

The used only() method loads only the primary key and the title for the selection widget.

Custom Admin Filter Specification

2011-11-21T18:25:00.000+01:00

Let's say we have a model Project with age groups as boolean fields:


class Project(models.Model):
    # ...
    age_group_0_5 = models.BooleanField(u"From 0 to 5 years")
    age_group_6_12 = models.BooleanField(u"From 6 to 12 years")
    age_group_13_18 = models.BooleanField(u"From 13 to 18 years")
    age_group_19_27 = models.BooleanField(u"Fron 19 to 27 years")
    # ...

In order to list them as one selection in the admin list filter, we need a specific filter specification. Let's create it in the admin.py:


from django.contrib.admin.filterspecs import FilterSpec

class AgeChoicesFilterSpec(FilterSpec):
    def title(self):
        return u"Age group"
        
    def __init__(self, f, request, params, model, model_admin,
                 field_path=None):
        super(AgeChoicesFilterSpec, self).__init__(f, request, params, model,
                                                model_admin,
                                                field_path=field_path)
        self.request = request
        
    def choices(self, cl):
        yield {'selected': not(
                    self.request.GET.get('age_group_0_5', None) or 
                    self.request.GET.get('age_group_6_12', None) or 
                    self.request.GET.get('age_group_13_18', None) or 
                    self.request.GET.get('age_group_19_27', None) 
                    ),
               'query_string': cl.get_query_string(
                               {},
                               ['age_group_0_5', 'age_group_6_12', 'age_group_13_18', 'age_group_19_27']),
               'display': u'All'}
        yield {'selected': self.request.GET.get('age_group_0_5', None) == 'True',
               'query_string': cl.get_query_string(
                               {'age_group_0_5': 'True'},
                               ['age_group_6_12', 'age_group_13_18', 'age_group_19_27']),
               'display': u"From 0 to 5 years"}
        yield {'selected': self.request.GET.get('age_group_6_12', None) == 'True',
               'query_string': cl.get_query_string(
                               {'age_group_6_12': 'True'},
                               ['age_group_0_5', 'age_group_13_18', 'age_group_19_27']),
               'display': u"From 6 to 12 years"}
        yield {'selected': self.request.GET.get('age_group_13_18', None) == 'True',
               'query_string': cl.get_query_string(
                               {'age_group_13_18': 'True'},
                               ['age_group_0_5', 'age_group_6_12', 'age_group_19_27']),
               'display': u"From 13 to 18 years"}
        yield {'selected': self.request.GET.get('age_group_19_27', None) == 'True',
               'query_string': cl.get_query_string(
                               {'age_group_19_27': 'True'},
                               ['age_group_0_5', 'age_group_6_12', 'age_group_13_18']),
               'display': u"From 19 to 27 years"}

FilterSpec.filter_specs.insert(0, (lambda f: f.name == 'age_group_0_5',
                               AgeChoicesFilterSpec))

Here the most important method of FilterSpec is choices. It defines the choices for the filter by dictionary with three keys:

selected is the condition when this choice should be selected.
query_string is the URL query string that will be formed to filter the list by. The query string is formed by get_query_string method of ChangeList instance. It takes two parameters: a dictionary of new parameters to add to the current query, and a list of parameters to remove from the current query.
display is the human-readable string which will be shown for this choice in the filter.

The new filter specification should be attached to a field. Here it will be applied to all fields with the name "age_group_0_5" and we have just one such field.

The last thing to do is to define admin options for the Project model:


from django.contrib import admin
from models import Project

class ProjectAdmin(admin.ModelAdmin):
    # ...
    list_filter = ("age_group_0_5", )
    # ...
    
admin.site.register(Project, ProjectAdmin)

Now you know how to create a custom admin filter specification for your model. :)

Django CMS, Haystack, and Custom Plugins

2011-11-11T21:31:00.001+01:00

When you need a full-text search for Django-CMS-based website, you can use Haystack and django-cms-search. The latter module ensures that all CMS Pages get indexed.

One important thing to mention is that if you use any custom Plugins, search_fields need to be defined for them, so that the Pages using them are indexed properly. For example, here is an Overview plugin which makes its title and description searchable:


from django.db import models
from django.utils.translation import ugettext_lazy as _
from cms.models import CMSPlugin

class Overview(CMSPlugin):
    title = models.CharField(_('Title'), max_length=200)
    description = models.TextField(_('Description'))
    url = models.URLField(_('url'), max_length=200, blank=True)
    
    search_fields = ("title", "description")
    
    def __unicode__(self):
        return self.title

For more information check the documentation online:

Deployment Using Fabric

2010-05-24T20:14:00.003+02:00

Yesterday just a night before EuroDjangoCon 2010, I was writing deployment scripts for my personal project. I have been using analogous scripts at least for a year for my work projects of different scopes. And I can prove that it's really worth to have such scripts, because they save a lot of time and let you avoid confusion and mistakes.

Most of the projects are continuously improved, so the main purpose of deployment in my case is not the initial installation, but handy and fast updating.

For each website that I am managing, there are two environments set up under different domains, where one of them is production environment and another one is staging/testing environment. The configuration of staging environment mirrors the production environment. Both have their own webserver configurations, codebases, databases, and uploaded media files. At first the newest features are tested in the staging environment with the copy of live data, and then they are published in the production environment.

I am using subversion for version control with third-party modules defined as svn:externals and put under the same codebase. Unfortunately, some of the modules have to be copied if they were originally taken from Git, Bazaar, or other places. Anyway, keeping all the codebase under SVN makes the initial installation easier and ensures that you can easily update the whole combination of module dependencies.

In addition, I am using south migrations, to make the changes to the database schema automagically.

Earlier the process of updating consisted of these manual steps:

Set up under-construction page for the staging/testing site either by changing .htaccess file or httpd.conf and restarting webserver (depending on the server configuration).
Export the dump of production database and import it into the staging database.
Move the media uploads from production environment to staging environment.
Update code from subversion trunk.
Migrate database schema and/or do any database changes.
Restart apache and memcached.
Test if everything is working as expected.
Unset under-construction page.

Then some similar instructions should be executed in the production environment:

Set up under-construction page for the production site.
Backup database in case if anything goes wrong, so that you could recover.
Update code from subversion.
Run database migrations.
Test everything.
Unset under-construction page.

And that's quite a long routine to do manually and also sometimes when the changes were small, I was always tempted to skip some steps which increased the risk to fail. But then I started using Fabric to automate the process. Fabric is python-based deployment tool providing simple API for FTP and SSH connections as well as for local shell calls.

To use Fabric, you have to install it in your local machine and then write scripts called fabfiles for each website, you are updating. For example, updating the staging environment in all my cases is as simple as running

# cd path/to/directory/with/fabfile.py/
# fab staging deploy

and then entering user passwords and answering questions like

Backup database ([y]/n)? _

The fabfiles could also be without user interaction, but I made those dialogs to be able to monitor the process and stop it at any point in case if I need some specific additional manual work.

For security reasons, you should avoid keeping passwords under subversion or in the deployment script. Much better it is to write some bash scripts in the remote servers which would do the work and then to address them from your fabfile.

Here is an example of possible fabfile. Keep in mind that different apache configurations and some bash scripts are something that you have to write yourself depending on your server configuration. Also the fabfile itself might slightly differ depending on the server specifics.

According to Thilo Fromm with whom I talked today at EuroDjangoCon, another way to create deployment scripts is to create linux-distribution-related packages which you could install using installation tools within the distribution, for example apt-get for Debian or rpm for Suse. However, in my opinion, that kind of deployment fits better to those cases when you don't have control of the production server and need to deliver a final product to corporate clients, or where you need to install the same project/product to multiple servers.

Weather App on Github

2009-10-20T18:56:00.006+02:00

As promised, I put the code of the climate_change app online. For that reason, I created an account on github trying to use Git for the first time. This is also the first time I am using a distributed version control system, i.e. you can create branches of the project, develop the forks separately on different computers, and merge them in any order. There is no main centralized repository. The code on github that I uploaded is just one initial copy.

I am still new to Git commands and workflows. You can share your knowledge with me. For example, maybe you know some good cheat sheet or writings about Git. Or you can try to fork the project I committed, change it somehow and then push for merging (not sure if I'm using correct jargon). I will share with you what I find interesting and useful about Git while learning it.

Weather App Tutorial. Part 5 of 5. Graph View

2009-10-19T02:00:00.004+02:00

I had some problems with the internet, so I am posting this entry a little later that expected. But anyway, this is it - the final part of the tutorial about the weather app. You can read the first four parts here: 1, 2, 3, 4. Today I will show you how to display a chart with statistical data from the collected weather details.

I googled a little to find out what charting system to use for this small project. And I found Open Flash Chart as an interesting solution. It's a flash-based charting system which uses Ajax requests to get the data for the graphs. That means that we'll need a view which will show the graphs and another view, which will provide the data. Each location will have its own page displaying a chart, so it would be nice to have URLs where each location would be accessed by a slug. We already have a field sysname which could be used here, but my best practice is to have separate fields for url slugs which might be changed at anytime and sysnames which should be defined once and not changed at all because the sysname binds an object with a template.

So I add this new field to the Location model in models.py:


class Location(models.Model):
    #...
    slug = models.SlugField(
        _("slug for urls"),
        max_length=200,
        unique=True,
        blank=True,
        default="",
        )
    #...

Then I create and apply a migration for it:


python manage.py startmigration add_slug --auto
python manage.py migrate climate_change

Let's run the webserver once again. Go to http://127.0.0.1:8000/admin/climate_change/location/1/, add "berlin" as a slug and save the record.

Next I need to define a view which will display the graphs. This looks as simple as this:


# -*- coding: UTF-8 -*-
from django.shortcuts import get_object_or_404, render_to_response

Location = models.get_model("climate_change", "Location")

def display_charts(request, slug):
    location = get_object_or_404(Location, slug=slug)
    return render_to_response(
        "climate_change/charts.html",
        {'location': location},
        context_instance=RequestContext(request),
        )

Whereas the template created at templates/climate_change/charts.html includes this code:


{% extends "base.html" %}

{% load i18n %}

{% block title %}{{ location.name }}{% endblock %}

{% block header %}
    </code><h1>{{ location.name }}</h1>
{% endblock %}

{% block content %}
    <div class="chart-temperature">
        <h2>{% trans "Temperature" %}</h2>
        <object type="application/x-shockwave-flash" data="/media/flash/open-flash-chart.swf?data-file={% url climate_change_temperature slug=location.slug %}" height="250" width="500">
            <param name="movie" value="/media/flash/open-flash-chart.swf?data-file={% url climate_change_temperature slug=location.slug %}">
            <param name="allowScriptAccess" value="sameDomain">
            <param name="quality" value="high">
        </object>
    </div>
    <div class="chart-humidity">
        <h2>{% trans "Humidity" %}</h2>
        <object type="application/x-shockwave-flash" data="/media/flash/open-flash-chart.swf?data-file={% url climate_change_humidity slug=location.slug %}" height="250" width="500">
            <param name="movie" value="/media/flash/open-flash-chart.swf?data-file={% url climate_change_humidity slug=location.slug %}">
            <param name="allowScriptAccess" value="sameDomain">
            <param name="quality" value="high">
        </object>
    </div>
    <div class="chart-wind-speed">
        <h2>{% trans "Wind speed" %}</h2>
        <object type="application/x-shockwave-flash" data="/media/flash/open-flash-chart.swf?data-file={% url climate_change_wind_speed slug=location.slug %}" height="250" width="500">
            <param name="movie" value="/media/flash/open-flash-chart.swf?data-file={% url climate_change_wind_speed slug=location.slug %}">
            <param name="allowScriptAccess" value="sameDomain">
            <param name="quality" value="high">
        </object>
    </div>
    <div class="chart-visibility">
        <h2>{% trans "Visibility" %}</h2>
        <object type="application/x-shockwave-flash" data="/media/flash/open-flash-chart.swf?data-file={% url climate_change_visibility slug=location.slug %}" height="250" width="500">
            <param name="movie" value="/media/flash/open-flash-chart.swf?data-file={% url climate_change_visibility slug=location.slug %}">
            <param name="allowScriptAccess" value="sameDomain">
            <param name="quality" value="high">
        </object>
    </div>
{% endblock %}

Also let's not forget to download the Open Flash Chart flash file itself which is in the archive of this package. I placed the file open-flash-chart.swf at media/flash.

Originally, Open Flash Chart uses PHP on the server side to provide data to the flash-based client side. But fortunately there is a port to python which I will be using for forming the data to the charts. I don't want to get dirty fingers by creating all this manually forming JSON object using simplejson, especially because there is not complete reference of Open-Flash-Chart API (just examples and tutorials). So I downloaded python port, extracted the content and copied OpenFlashChart.py to climate_change_env/lib/python2.5/site-packages/ which is the location of python libraries which will be installed a.k.a. under python path for the virtual environment "climate_change_env".

Using the examples found at a tutorial, I created a view for providing chart data. The views.py looks now like this:


# -*- coding: UTF-8 -*-
from datetime import datetime, timedelta
from random import random

# http://open-flash-chart-python.googlecode.com/files/Python%202.10.zip
from OpenFlashChart import Chart

from django.db import models
from django.http import HttpResponse
from django.template import RequestContext
from django.shortcuts import get_object_or_404, render_to_response
from django.utils.translation import ugettext as _
from django.utils.encoding import force_unicode

Location = models.get_model("climate_change", "Location")
WeatherLog = models.get_model("climate_change", "WeatherLog")

def display_charts(request, slug):
    location = get_object_or_404(Location, slug=slug)
    return render_to_response(
        "climate_change/charts.html",
        {'location': location},
        context_instance=RequestContext(request),
        )

def rgb_to_html_color(r, g, b):
    """ convert an R, G, B to #RRGGBB """
    return '#%02x%02x%02x' % (r, g, b)

def json_get_statistics(request, slug, field):
    location = get_object_or_404(Location, slug=slug)
    extreme_values = location.weatherlog_set.aggregate(
        models.Min(field),
        models.Max(field),
        models.Min("timestamp"),
        models.Max("timestamp"),
        )
    elements = []

    for year in range(
        extreme_values['timestamp__min'].year,
        extreme_values['timestamp__max'].year + 1,
        ):
        element = Chart()
        monthly_values = []
        for month in range (1, 13):
            monthly = location.weatherlog_set.filter(
                timestamp__year=year,
                timestamp__month=month,
                ).aggregate(models.Avg(field))
            monthly_values.append(monthly['%s__avg' % field])
        element.values = monthly_values
        element.type = "line"
        element.dot_style.type = "dot"
        element.dot_style.dot_size = 5
        random_html_color = rgb_to_html_color(
            r=63+int(random() * 128),
            g=63+int(random() * 128),
            b=63+int(random() * 128),
            )
        element.dot_style.colour = random_html_color
        element.width = 4
        element.colour = random_html_color
        element.text = str(year)
        element.font_size = 10
        elements.append(element)
    
    # Create chart
    chart = Chart()
    chart.y_axis.min = float(str(extreme_values['%s__min' % field]))
    chart.y_axis.max = float(str(extreme_values['%s__max' % field]))
    chart.y_axis.font_size = 10
    chart.title.text = force_unicode(WeatherLog._meta.get_field(field).verbose_name)
    chart.x_axis.labels.labels = [
        _("Jan"), _("Feb"), _("Mar"), _("Apr"),
        _("May"), _("Jun"), _("Jul"), _("Aug"),
        _("Sep"), _("Oct"), _("Nov"), _("Dec"),
        ]

    #
    # here we add our data sets to the chart:
    #
    chart.elements = elements
    
    return HttpResponse(chart.create())

If I run the webserver again and go to http://127.0.0.1:8000/climate-change/berlin/, I see a single dot in each graph.

We need some more data. There is Time Machine app in the Applications directory in my computer, so let's use it to move forward to year 2012 to check weather statistics. Damn it! It didn't function properly. Then I'll need to generate statistical data myself. Anyway, according to Mark Twain, there are three kinds of lies: lies, damned lies, and statistics.

I'll use python interactive development environment a.k.a. shell to prepopulate that data.


python manage.py shell
>>> from django.db import models
>>> Location = models.get_model("climate_change", "Location")
>>> location = Location.objects.get(sysname="berlin")
>>> extremes = (
...     (-5, 4), (-4, 5), (-2, 10), (2, 15),
...     (6, 20), (9, 24), (11, 25), (10, 25),
...     (7, 20), (4, 15), (0, 9), (-2, 5),
...     )
>>> from random import random
>>> from datetime import datetime
>>> for year in range(2009, 2012):
...     for month in range(1, 13):
...         temp_range = extremes[month-1][1] - extremes[month-1][0]
...         t = extremes[month-1][0] + int(random() * temp_range)
...         location.weatherlog_set.create(
...             timestamp=datetime(year, month, 1),
...             temperature=t,
...             humidity=int(random()*100),
...             wind_speed="%.2f" % (random()*372),,
...             visibility="%.2f" % (random()*100),
...             )

I defined minimal and maximal temperatures for each month in a tuple of tuples. Then I went through three years and for each month of that year I added random temperatures which would be in the range of minimal and maximal temperatures for that month. Humidity, wind speed and visibility are randomized without monthly ranges. The graph after data population looks like this:

That's it. Finally, I completed this tutorial. You had a chance to watch the process of Django app development. You saw, how virtualmin, migrations and third-party libraries can be used. Hopefully, you got some inspirations to create something cool by yourself just for fun.

I will publish the full code soon somewhere and you'll have a chance to download it and modify.

If something is too confusing, you don't understand some part of the tutorial, or you think that it could be improved somehow, don't hesitate to write comments.

Weather App Tutorial. Part 4 of 5. Template Tag

2009-10-17T00:30:00.046+02:00

If you followed the first parts of the tutorial, you should have basic understanding how to create an app with models, set up administration, and retrieve data from third-party services. This part is about displaying collected data in any template using custom template tag.

At first, we need to create a directory templatetags containing an empty __init__.py file in the climate_change directory.


mkdir -p climate_change/templatetags
touch climate_change/templatetags/__init__.py

I will call the template library weather. So I have to create a file weather.py in climate_change/templatetags and define and register the template tag in that file. The template tag get_current_weather should display the current weather for a chosen location. To define what location you choose, you could refer to id, name or location_id, but none of them is appropriate for this reason. id and location_id are not remember-able and not informative enough, whereas the name might be changed to translate the city to another language or to add some more specifics and this change would detach the template tag from the location. For those reasons, it is best to create a new field sysname for the location model which would have a unique non-changeable value as a textual humanized identifier for templates.

But wait! It's such a pain to add new fields and modify database schema... Not, if you are using south! Let's quickly install it and then put it under INSTALLED_APPS.

easy_install south


#...
INSTALLED_APPS = (
    # django core
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.sites",
    "django.contrib.admin",
    # third-party
    "south",
    # project-specific
    "climate_change",
    )
#...

Note that south was installed in the virtual environment not spoiling the global python namespace. Now we will syncronize the database to create south_history table, create the initial migration for climate_change and apply it.


# create the missing database table from south app
python manage.py syncdb
# create initial migration for climate_change app which will be used by new projects
python manage.py startmigration climate_change --initial
# fake this migration for this project
python manage.py migrate climate_change --fake

Now we can finally add the new sysname field, create migration for it and apply it.


#...
class Location(models.Model):
    sysname = models.SlugField(
        _("system name"),
        max_length=200,
        unique=True,
        blank=True,
        default="",
        help_text=_("Do not change this value"),
        )
    name = models.CharField(_("name"), max_length=200)
    location_id = models.CharField(
        _("location ID"),
        max_length=20,
        help_text=_("Location IDs can be retrieved from URLs of weather "
            "at specific cities at Yahoo! Weather, e.g. GMXX0008 from "
            "http://weather.yahoo.com/forecast/GMXX0008.html"),
        )
#...


# create a new migration called "add_sysname"
python manage.py startmigration climate_change add_sysname --auto
# apply it to the database
python manage.py migrate climate_change

I had to set the default value to empty string because otherwise south throws exception when I use sqlite3. Anyway, after running those commands, I started the built in webserver again and added the sysname "berlin" to the record of Berlin's location.

We can get back to weather.py and add the template tag there


# -*- coding: UTF-8 -*-
from django.db import models
from django import template
from django.template import loader

register = template.Library()

### TAGS ###

def do_get_current_weather(parser, token):
    """
    Returns the latest known weather information.

    Usage::

        {% get_current_weather in <location_sysname> [using <template_path>] [as <var_name>] %}
    
    Examples::

        {% get_current_weather in "berlin" using "climate_change/custom_weather.html" %}

        {% get_current_weather in "london" as current_weather %}
        var sCurrentWeather = "{{ current_weather|escapejs }}";

    """    
    bits = token.split_contents()
    tag_name = bits.pop(0)
    template_path = ""
    var_name = ""
    location_sysname = ""
    try:
        while bits:
            first_word = bits.pop(0)
            second_word = bits.pop(0)
            if first_word == "in":
                location_sysname = second_word
            elif first_word == "using":
                template_path = second_word
            elif first_word == "as":
                var_name = second_word
                    
    except ValueError:
        raise template.TemplateSyntaxError, "get_current_weather tag requires a following syntax: {% get_current_weather [using <template_path>] [as <var_name>] %}"
    return CurrentWeatherNode(tag_name, location_sysname, template_path, var_name)

class CurrentWeatherNode(template.Node):
    def __init__(self, tag_name, location_sysname, template_path, var_name):
        self.tag_name = tag_name
        self.location_sysname = location_sysname
        self.template_path = template_path
        self.var_name = var_name
    def render(self, context):
        location_sysname = template.resolve_variable(
            self.location_sysname,
            context,
            )
        template_path = ""
        if self.template_path:
            template_path = template.resolve_variable(
                self.template_path,
                context,
                )
        context.push()
        WeatherLog = models.get_model("climate_change", "WeatherLog")
        logs = WeatherLog.objects.filter(
            location__sysname=location_sysname,
            ).order_by("-timestamp")
        if logs:
            context['weather'] = logs[0]
        output = loader.render_to_string(
            [template_path, "climate_change/current_weather.html"],
            context,
            )
        context.pop()
        if self.var_name:
            context[self.var_name] = output
            return ""
        else:
            return output


register.tag("get_current_weather", do_get_current_weather)

### FILTERS ###

# none at the moment

As you might see from the code, the template tag is using a template which can be redefined by the template designer. We still need the default template itself, so I will create a directory templates/climate_change and a file current_weather.html with this content:


{% load i18n %}
<div class="current-wheather">
    <h3>{{ weather.location.name }}</h3>
    <dl>
        <dt>{% trans "Temperature" %}:</dt>
        <dd>{{ weather.temperature }}° C</dd>
        <dt>{% trans "Humidity" %}:</dt>
        <dd>{{ weather.humidity }} %</dd>
        <dt>{% trans "Wind speed" %}:</dt>
        <dd>{{ weather.wind_speed }} km/h</dd>
        <dt>{% trans "Visibility" %}:</dt>
        <dd>{{ weather.visibility }} km</dd>
    </dl>
</div>

How can I test the template tag? I will need a new page which will include it. So I will add a rule in urls.py to redirect root url to index.html which will extend from base.html.

So the base.html looks like this:


{% block doctype %}>!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">{% endblock %}

{% load i18n %}

<html>
<head>
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <title>{% block title %}simple document{% endblock %}</title>
    {% block extra_head %}{% endblock %}
</head>
<body>
    <div id="header">{% block header %}{% endblock %}</div>
    <div id="content">{% block content %}{% endblock %}</div>
    <div id="footer">{% block footer %}{% endblock %}</div>
</body>
</html>

The index.html looks like this:


{% extends "base.html" %}
{% load i18n weather %}
{% block content %}
    {% get_current_weather in "berlin" %}
{% endblock %}

And also we'll need an extension in the urls.py:


from django.conf.urls.defaults import *
from django.contrib import admin

admin.autodiscover()

urlpatterns = patterns("",
    (
        r"^$",
        "django.views.generic.simple.direct_to_template",
        {'template': "index.html"},
        ),
    (r"^admin/", include(admin.site.urls)),
    )

When I run the development server and go to http://127.0.0.1:8000/, I see this:

It's time for the graphs! The end of this tutorial will be published here tomorrow.

Weather App Tutorial. Part 3 of 5. Management Command

2009-10-16T00:30:00.015+02:00

As you might have noticed, this is a continuous tutorial about an app which regularly checks weather and reports how it changes throughout years. Previously I wrote how to start a project and define the models. The interestingness of the app starts now. I will show you how to import weather details from Yahoo! Weather using a management command.

Management commands in Django are those which are recognized by manage.py and django-admin.py, e.g. syncdb, shell, runserver, etc. It's not difficult to inject your own custom commands for your app. We'll need one that will be set as a scheduled task to check the current weather.

To create a management command, you have to create directories management/commands/ inside of your climate_change directory and create empty files __init__.py in each of them.


mkdir -p climate_change/management/commands
touch climate_change/management/__init__.py
touch climate_change/management/commands/__init__.py

Now I'll create a file climate_change/management/commands/check_weather.py which will aggregate the data from the feed. Just looking at the examples of Django core commands, you can find, that management-command modules should have a class Command with an overridden method handle_norargs which will have all the logic of the command. I found an example of importing weather from Yahoo! and will be using it here.


# -*- coding: UTF-8 -*-
import urllib
from xml.dom import minidom
from pprint import pprint
from datetime import datetime

from django.db import models
from django.core.management.base import NoArgsCommand

Location = models.get_model("climate_change", "Location")
WeatherLog = models.get_model("climate_change", "WeatherLog")

SILENT, NORMAL, VERBOSE = 0, 1, 2

WEATHER_URL = 'http://xml.weather.yahoo.com/forecastrss?p=%s&u=c'
WEATHER_NS = 'http://xml.weather.yahoo.com/ns/rss/1.0'

def weather_for_location(location_id):
    # taken from http://developer.yahoo.com/python/python-xml.html
    # and modified a little
    url = WEATHER_URL % location_id
    dom = minidom.parse(urllib.urlopen(url))
    forecasts = []
    for node in dom.getElementsByTagNameNS(WEATHER_NS, 'forecast'):
        forecasts.append({
            'date': node.getAttribute('date'),
            'low': node.getAttribute('low'),
            'high': node.getAttribute('high'),
            'condition': node.getAttribute('text')
        })
    ycondition = dom.getElementsByTagNameNS(WEATHER_NS, 'condition')[0]
    ywind = dom.getElementsByTagNameNS(WEATHER_NS, 'wind')[0]
    yatmosphere = dom.getElementsByTagNameNS(WEATHER_NS, 'atmosphere')[0]
    return {
        'current_condition': ycondition.getAttribute('text'),
        'current_temp': ycondition.getAttribute('temp'),
        'current_humidity': yatmosphere.getAttribute('humidity'),
        'current_visibility': yatmosphere.getAttribute('visibility'),
        'current_wind_speed': ywind.getAttribute('speed'),
        'forecasts': forecasts,
        'title': dom.getElementsByTagName('title')[0].firstChild.data,
        'guid': dom.getElementsByTagName('guid')[0].firstChild.data,
    }            

class Command(NoArgsCommand):
    help = "Aggregates data from weather feed"
    def handle_noargs(self, **options):
        verbosity = int(options.get('verbosity', NORMAL))
        
        created_count = 0
        for l in Location.objects.all():
            weather = weather_for_location(l.location_id)
            if verbosity > NORMAL:
                pprint(weather)
            timestamp_parts = map(int, weather['guid'].split("_")[1:-1])
            timestamp = datetime(*timestamp_parts)
            log, created = WeatherLog.objects.get_or_create(
                 location=l,
                 timestamp=timestamp,
                 defaults={
                    'temperature': weather['current_temp'],
                    'humidity': weather['current_humidity'],
                    'wind_speed': weather['current_wind_speed'],
                    'visibility': weather['current_visibility'],
                    }
                 )
            if created:
                created_count += 1
        if verbosity > NORMAL:
            print "New weather logs: %d" % created_count

Now we can try running the command from the project directory using the following:


python manage.py check_weather --verbosity=2

I get this result:


{'current_condition': u'Mostly Cloudy',
 'current_humidity': u'94',
 'current_temp': u'10',
 'current_visibility': u'9.99',
 'current_wind_speed': u'16.09',
 'forecasts': [{'condition': u'Partly Cloudy',
                'date': u'6 Oct 2009',
                'high': u'19',
                'low': u'14'},
               {'condition': u'PM Light Rain',
                'date': u'7 Oct 2009',
                'high': u'23',
                'low': u'14'}],
 'guid': u'GMXX0008_2009_10_06_10_20_CEST',
 'title': u'Yahoo! Weather - Berlin/Schonefeld, GM'}
New weather logs: 1

To define this task as a cron job running hourly, you can either type


echo 0 * * * * cd /path/to/climate_change_env/ && source bin/activate && cd blog_action_day_2009 && python manage.py check_weather && deactivate > crontab.txt
crontab crontab.txt
del crontab.txt

.. or use django-poormanscron app and set the check_weather command there.

OK. Cool. The next thing to do is the template tag which displays the details of recent weather.

Weather App Tutorial. Part 2 of 5. Models

2009-10-15T00:30:00.008+02:00

This is the second part of my tutorial how to make a weather app. This time I will show you how to create the models and set up the admin.

Open the models.py file in the climate_change directory and enter the following:


# -*- coding: UTF-8 -*-
from django.db import models
from django.utils.translation import ugettext_lazy as _
from django.conf import settings

class Location(models.Model):
    name = models.CharField(_("name"), max_length=200)
    location_id = models.CharField(
        _("location ID"),
        max_length=20,
        help_text=_("Location IDs can be retrieved from URLs of weather "
            "at specific cities at Yahoo! Weather, e.g. GMXX0008 from "
            "http://weather.yahoo.com/forecast/GMXX0008.html"),
        )
    
    class Meta:
        verbose_name=_("location")
        verbose_name_plural=_("locations")
    
    def __unicode__(self):
        return self.name

class WeatherLog(models.Model):
    location = models.ForeignKey(Location, verbose_name=_("location"))
    timestamp = models.DateTimeField(_("timestamp"))
    temperature = models.IntegerField(_("temperature (C°)"))
    humidity = models.IntegerField(_("humidity (%)"))
    wind_speed = models.DecimalField(
         _("wind speed (km/h)"),
         max_digits=5,
         decimal_places=2,
         )
    visibility = models.DecimalField(
         _("visibility (km)"),
         max_digits=5,
         decimal_places=2,
         )
    
    class Meta:
        verbose_name=_("weather log")
        verbose_name_plural=_("weather logs")
        ordering = ("-timestamp",)
    
    def __unicode__(self):
        return "%s @ %s" % (
            self.location.name,
            self.timestamp.strftime("%Y-%m-%dT%H:%M"),
            )

The models are created. Now let's create the database schema for them.


python manage.py syncdb

Also in this step when asked, I created a superuser called "demo" to be able to access django-contributed administration. Let's try it out. We'll need to add a file admin.py to climate_change with this content:


# -*- coding: utf-8 -*-
from django.db import models
from django.contrib import admin

Location = models.get_model("climate_change", "Location")
WeatherLog = models.get_model("climate_change", "WeatherLog")

admin.site.register(Location)
admin.site.register(WeatherLog)

Then uncomment admin related lines in urls.py of the project:


# -*- coding: utf-8 -*-
from django.conf.urls.defaults import *
from django.contrib import admin
admin.autodiscover()

urlpatterns = patterns("",
    (r"^admin/", include(admin.site.urls)),
    )

Finally, run the development server and check if the models really work.


python shell runserver
# now you can go to http://127.0.0.1:8000/admin/ in a browser

Yes! Everything works as expected for now. I will add my current location and its ID in Yahoo! Weather to the database. For me it's Berlin, Germany and the ID is GMXX0008. I found the ID in the URL of the page showing the weather for Berlin.

Tomorrow I will show you how to import weather details from Yahoo!

Weather App Tutorial. Part 1 of 5. Preparation

2009-10-14T00:30:00.003+02:00

It is Blog Action Day tomorrow so I decided to participate in it as a blogger again. The theme for this year is "Climate Change". After a little brainstorm, I came up with an idea to write a tutorial how to build an app which regularly checks local weather and lets you compare different weather features for months of different years.

I'll use Yahoo! Weather to check the current weather. From all information that it provides, we'll be mostly interested in temperature, humidity, wind speed, and visibility in the current location. The app will check the weather regularly, will allow you to show the current weather, and also provide a graph comparing average monthly weathers throughout years.

So let's start with the new project. I have quite a clean computer and want to do the app the nice way. So first of all, I will install virtualenv to be able to install third-party python libraries in a closed environment which will only be used for this project (I have already installed setuptools and django).


# install virtualenv
sudo easy_install virtualenv

I created a directory Projects in my home directory and cd to it.

Let's create a virtual environment and start the new project and app.


# create virtual environment "climate_change_env"
virtualenv climate_change_env
cd climate_change_env/
# activate the environment
source bin/activate

Since now I see (climate_change_env) as a prefix to each command line. I'll type deactivate at some point later to get out of this virtual environment.


# create django project "blog_action_day_2009"
django-admin.py startproject blog_action_day_2009
cd blog_action_day_2009/
# create django app "climate_change"
django-admin.py startapp climate_change
# create folders for templates and media
mkdir templates media

To get started quickly, I will use sqlite3 database now. As I am using python 2.5, I don't need to install sqlite3 module, because it's already there. So I open the settings.py and define those settings:

import os
PROJECT_DIR = os.path.dirname(__file__)
# ...
DATABASE_ENGINE = "sqlite3"
DATABASE_NAME = "blog_action_day_2009.sqlite3"
# ...
MEDIA_ROOT = os.path.join(PROJECT_DIR, "media")
MEDIA_URL = "/media/"
ADMIN_MEDIA_PREFIX = "/admin/media/"
# ...
TEMPLATE_DIRS = [
    os.path.join(PROJECT_DIR, "templates"),
    ]
# ...
INSTALLED_APPS = (
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.sites",
    "django.contrib.admin",
    "climate_change",
    )

Now we'll need those parts:

Models for locations and imported weather information.
Management command for importing data from Yahoo! Weather.
Template tag for displaying latest weather.
View for displaying a graph.

But I'll continue about that tomorrow.

To end today's post, let's watch a video about a regular guy who talks about weather when he has nothing to say:

Applying Patches in Guerrilla Way

2009-05-05T23:00:00.002+02:00

How many times did you submit patches that have never been approved? How many times did you need to patch Django for some third-party application to work? How many times did you bumped into the wall working on real-world projects just because of some bugfixes waiting for tests or nicer solution? In all those cases, you should not panic nor run into circles, because you can still apply the existing patches in guerrilla way (or so called monkey patching).

Guerrilla patches overwrite existing code not on a disk, but in memory at execution time. Everything is an object in Python. Numbers, strings, functions, methods, classes, instances, modules, etc. are objects. Each object might be conceived as an attribute of a module where it is defined. In addition, properties and methods of classes can be conceived as attributes of the class. The principle of guerrilla patching is loading a module or class which attributes need to be changed, and assigning new values to those attributes before the module or class is used. All that has to happen somewhere in the beginning of execution. Usually, I put or import guerrilla patches in models.py of some app, because all models are loaded in one of the first steps of execution. So if you need to change a full class (DjangoClass), you would import the module (djangomodule) where the class resides, create another class (NewDjangoClass) and assign it to the attribute of the module by the name of the original class (djangomodule.DjangoClass = NewDjangoClass). If you need to change just one method (django_method) of a class (DjangoClass), then you would import the class, write a new function which takes the object to which it will be assigned as the first argument (def new_django_method(self, *args, **kwargs)) and assigned it to the class by the name of the original function (DjangoClass.django_method = new_django_method).

Let's have a look at an example. You might need nested fieldsets for more organized grouping, suggested in the ticket #10590. That allows you to create fieldset definitions as items of the 'fields' value of a parent fieldset.

As you might see from the applied patch nested_fieldsets.diff, the main change of the patch is changing most of the code for classes Fieldset and Fieldline residing in django/contrib/admin/helpers.py, also changing the function flatten_fieldsets in django/contrib/admin/util.py, introducing a new template tag and finally modifying a template.

The file with guerrilla patches should import helpers, util, and all other modules that will be used in the overwritten objects. Browse through the code below and check the differences with the original patch to understand the concept better.


from django import forms
from django.contrib.admin import helpers
from django.contrib.admin import util
from django.contrib.admin import options
from django.utils.safestring import mark_safe
from django.forms.formsets import all_valid
from django.conf import settings

### Guerilla patches for nested fieldsets

def flatten_fieldsets(fieldsets):
    """Returns a list of field names from an admin fieldsets structure."""
    field_names = []
    for name, opts in fieldsets:
        for field in opts['fields']:
            if isinstance(field, (tuple, list)):
                if len(field)==2 and isinstance(field[1], dict):
                    # it's a nested fieldset
                    field_names.extend(flatten_fieldsets((field,)))
                else:
                    # it's a tuple of field names
                    field_names.extend(field)
            else:
                # it's a field name
                field_names.append(field)
    return field_names

options.flatten_fieldsets = util.flatten_fieldsets = flatten_fieldsets

class Fieldset(object):
    is_fieldset = True
    def __init__(self, form, name=None, fields=(), classes=(), description=None, level=0):
        self.form = form
        self.name, self.fields = name, fields
        self.classes = u' '.join(classes)
        self.description = description
        self.level = level

    def _media(self):
        if 'collapse' in self.classes:
            return forms.Media(js=['%sjs/admin/CollapsedFieldsets.js' % settings.ADMIN_MEDIA_PREFIX])
        return forms.Media()
    media = property(_media)

    def __iter__(self):
        for field in self.fields:
            if (len(field)==2 and isinstance(field[1], dict)):
                # nested fieldset
                yield Fieldset(self.form,
                    name=field[0],
                    fields=field[1].get("fields", ()),
                    classes=field[1].get("classes", ()),
                    description=field[1].get("description", ()),
                    level=self.level + 1,
                    )
            else:
                # field name or a tuple of field names
                yield helpers.Fieldline(self.form, field)
                
helpers.Fieldset = Fieldset

class InlineFieldset(Fieldset):
    def __init__(self, formset, *args, **kwargs):
        self.formset = formset
        super(InlineFieldset, self).__init__(*args, **kwargs)
        
    def __iter__(self):
        fk = getattr(self.formset, "fk", None)
        for field in self.fields:
            if fk and fk.name == field:
                continue
            if (len(field)==2 and isinstance(field[1], dict)):
                # nested fieldset
                yield Fieldset(self.form,
                    name=field[0],
                    fields=field[1].get("fields", ()),
                    classes=field[1].get("classes", ()),
                    description=field[1].get("description", ()),
                    level=self.level + 1,
                    )
            else:
                # field name or a tuple of field names
                yield helpers.Fieldline(self.form, field)

helpers.InlineFieldset = InlineFieldset

Further, the template can be overwritten putting your own template path before the django admin path in settings.TEMPLATE_DIRS. The missing template tag can be defined in your own app. Just don't forget to load the library in the template ({% load mytemplatetags %}). CSS might be overwritten by including a custom CSS file to the bottom of HEAD section in a modified admin/change_form.html template.

That's all what you need and you'll be controlling the situation without branching Django. You have full power, but don't overuse this technique as it might be difficult to manage the updates, especially when you are guerrilla-patching undocumented features or helping functions which change backwards-incompatibly in the future. Still guerrilla-patched code in a third-party app is better than a requirement to patch Django itself.

Guerrilla patching might also be used in a similar way to do some slight modification to other third party apps which are not scalable by design. For example, you can add additional fields or modify methods of third-party apps which will be updatable separately.

Tomorrow (Wednesday) evening I am leaving the wonderful Prague and EuroDjangoCon. If you are here as well and want to discuss some concepts which I introduced in this blog, that's the last chance to meet me live. :D

Design Pattern for Reusable Apps: Getting into Details

2009-04-12T16:00:00.001+02:00

This article is an attempt to explain a technique for creating reusable and overridable apps that were partly mentioned in my last post.

First of all, let's remember the presentation by James Bennett about reusable apps from DjangoCon 2008:

Also the upgraded version Real World Django by Jacob Kaplan-Moss presented at PyCon 2009 should be checked.

Imagine, that you have a bunch of predefined apps somewhere under your python path. Some of them have models with relations to models from other apps. You will surely want to reuse those apps for different projects. You'll want to be able to activate the necessary set of apps. Probably sometimes you'll want to use an extended version of an existing app. As shown in the presentations given above, reusable apps should allow you easy overrides. Although all apps should be either under python path directly or under a package, nothing should be set in stone. The method get_model() should be used to import models from installed apps. But apps consist of more components than just models. There are urls, views, forms, template tags, context processors, middlewares, etc.

Let's have a look at a simple example. Two reusable apps could be events and venues where events would contain an Event model with a foreign key to Venue model from the venues app. In order to make both models overridable, I'll put the definitions in abstract base classes in the files base.py. The files models.py will import from base.py and create the leaf classes which will be the models to use further in forms and elsewhere.

venues

__init__.py

base.py

from django.db import models

class VenueBase(models.Model):
    title = models.CharField(...)
    street_address = models.CharField(...)
    postal_code = models.CharField(...)
    city = models.CharField(...)
    country = models.CharField(...)

    class Meta:
        abstract = True

models.py

from venues.base import *

class Venue(VenueBase):
    pass

events

__init__.py

base.py

from django.db import models

Venue = models.get_model("venues", "Venue")

class EventBase(models.Model):
    title = models.CharField(...)
    venue = models.ForeignKey(Venue)
    from_date = models.DateField(...)
    to_date = models.DateField(...)

    class Meta:
        abstract = True

models.py

from events.base import *

class Event(EventBase):
    pass

Venues (as well as events) will probably have some urls, views, and forms which should be extensible too.

venues

forms.py

from django.db import models
from django.forms import ModelForm
Venue = models.get_model("venues", "Venue")

class VenueForm(ModelForm):
    class Meta:
        model = Venue

views.py


from django.db import models
from django.utils import importlib
from django.shortcuts import render_to_response

# importing the form depending on the path of installed app
venues_app = models.get_app("venues")
venues_forms = importlib.import_module(venues_app.__name__[:-6] + "forms")
VenueForm = venues_forms.VenueForm

def add_venue(request):
    if request.method == "POST":
        form = VenueForm(request.POST, request.FILES)
        if form.is_valid():
            form.save()
    else:
        form = VenueForm()
    return render_to_response("venues/add_venue.html", {
        "form": form,
    })

def change_venue(request, pk):
    ...

urls.py


from django.conf.urls.defaults import *
from django.db import models
from django.utils import importlib

# importing the form depending on the path of installed app
venues_app = models.get_app("venues")
venues_views = importlib.import_module(venues_app.__name__[:-6] + "views")

urlpatterns = patterns('',
    ...
    url(r'^add/$', venues_views.add_venue, name='add_venue'),
    url(r'^(\d+)/$', venues_views.change_venue, name='change_venue'),
    ...
    )

You might ask why such a strange and tricky importing is done in the views and urls. If we know the path to the app, it is not going to change or something. Even James Bennett himself didn't see any advantages of importing forms and views dynamically when closing ticket #10703 for Django:

I don't really see the utility of this – a properly-written Django application is just a Python module, and is importable the same as any other Python module. It's not like that module is suddenly going to have a different path (and if it does, you're doing something wrong).

The answer hides in the overriding apps.

The combo of current events and venues might fit most of your projects. But then you might need a specific venues app with additional features for a specific project. Note, that you probably don't want to change or recreate the events app just because of the relation to the venues. The app and model names can be still the same, just put the new app under your new project and include the new app into INSTALLED_APPS instead of the original app. Project name will serve as a namespace for distinguishing the original app and the overridden app.

If you want to extend just the model, but not the other components, your specific app might look like this:

myproject

venues

__init__.py

models.py

from venues.base import VenueBase

class Venue(VenueBase):
    description = models.TextField(...)

forms.py
```
from venues.forms import *
```

views.py
```
from venues.views import *
```

urls.py
```
from venues.urls import *
```

If the model is OK as is, but you need an additional behavior for the form, like saving an image for each venue, you can extend the form instead:

myproject

venues

__init__.py

models.py
```
from venues.models import *
```

forms.py

from django import forms
from venues.forms import VenueForm as VenueFormBase

class VenueForm(VenueFormBase):
    image = forms.ImageField(...)
    def save(self, *args, **kwargs):
        super(VenueForm, self).save(*args, **kwargs)
        ...

views.py
```
from venues.views import *
```

urls.py
```
from venues.urls import *
```

If models and forms are alright, but you want to add an additional view, you can do that too:

myproject

venues

__init__.py

models.py
```
from venues.models import *
```

forms.py
```
from venues.forms import *
```

views.py

from venues.views import *

def delete_venue(request, pk):
    ...

urls.py

from venues.urls import *

urlpatterns += patterns('',
    url(r'^(\d+)/delete/$', venues_views.delete_venue, name='delete_venue'),
    )

As you see from the case examples above, you don't have to duplicate any code of the original app. Still you can now enhance or overwrite just specific parts that you need very flexibly. The events could be extended in the same manner. If the original extensible apps live under a specific package, then the imports in the extending app should be changed appropriately.

Ufff.. That's about all what I wanted to explain this time. If you have any questions, suggestions or disagree with some concepts, please write a comment bellow.

If you reside in Berlin or going to be here next Wednesday by accident or so, don't miss the Django meetup at Schleusenkrug at 19:30.

Also have a nice Easter Holiday!

Abstract Models and Dynamicly Assigned Foreign Keys

2009-02-19T11:30:00.000+01:00

Model inheritance enables creating extensible apps. You can define a reusable core app which includes base.py with abstract models and models.py with models extending the abstract ones and inheriting all the features. In the specific project you can either use the core app directly, or create a specific app which models extend from the base abstract models of the core app and additionally introduce new features.

This is a quick example skipping all the unrelated parts like settings, urls, and templates:

core_project

apps

player

base.py

from django.db impport models
class PlayerBase(models.Model):
    name = models.CharField(max_length=100)
    class Meta:
        abstract = True

models.py

from core_project.apps.player.base import PlayerBase
class Player(PlayerBase):
    pass

specific_project

apps

player

models.py

from core_project.apps.player.base import PlayerBase
class Player(PlayerBase):
    points = models.IntegerField()

The concept works fine until you need to use foreign keys or many-to-many relations in the abstract models. As Josh Smeaton has already noticed, you can't set foreign keys to abstract models as they have no own database tables and they know nothing about the models which will extend them.

Let's say, we have the following situation: GameBase and MissionBase are abstract models and the model extending MissionBase should receive a foreign key to the model extending GameBase.

Thanks to Pro Django book by Marty Alchin, I understood how the models get created in the background. By default, all python classes are constructed by the type class. But whenever you use __metaclass__ property for your classes, you can define a different constructor. Django models are classes constructed by ModelBase class which extends the type class.

In order to solve the problem of foreign keys to the models extending the abstract classes, we can have a custom constructor extending the ModelBase class.

base.py

# -*- coding: utf-8 -*-
from django.db import models
from django.db.models.base import ModelBase
from django.db.models.fields import FieldDoesNotExist

class GameMissionCreator(ModelBase):
    """
    The model extending MissionBase should get a foreign key to the model extending GameBase
    """
    GameModel = None
    MissionModel = None
    def __new__(cls, name, bases, attrs):
        model = super(GameMissionCreator, cls).__new__(cls, name, bases, attrs)
        for b in bases:
            if b.__name__=="GameBase":
                cls.GameModel = model
            elif b.__name__=="MissionBase":
                cls.MissionModel = model
        if cls.GameModel and cls.MissionModel:
            try:
                cls.MissionModel._meta.get_field("game")
            except FieldDoesNotExist:
                cls.MissionModel.add_to_class(
                    "game",
                    models.ForeignKey(cls.GameModel),
                    )
        return model

class GameBase(models.Model):
    __metaclass__ = GameMissionCreator

    title = models.CharField(max_length=100)
    class Meta:
        abstract = True

class MissionBase(models.Model):
    __metaclass__ = GameMissionCreator
    
    title = models.CharField(max_length=100)
    class Meta:
        abstract = True

models.py

# -*- coding: utf-8 -*-
from base import *

class Game(GameBase):
    pass

class Mission(MissionBase):
    pass

GameMissionCreator is a constructor of GameBase, MissionBase, Game, and Mission classes. When it creates a class extending GameBase, the game model is registered as a property. When it creates a class extending MissionBase, the mission model is registered as a property. When both models are registered, a foreign key is added dynamically from one model to the other.

One drawback of this constructor-class example is that if there are more than one classes extending GameBase or MissionBase, then the code won't function correctly.

Anyway, the example shown illustrates the possible solution and gives a direction for further development of the idea.

Clarification for Confusion: Creating New Django Fields. Part 1

2008-12-15T06:00:00.001+01:00

Let's say you've just published your first Django-based project. You've learned a lot, you know how to create a new app, views, forms, middleware, and template tags. That's cool, but there is still much space for making your code cleaner and better-organized. One of such improvements could be joining multiple fields into a single field.

To be clear, there are several types of fields in Django web development and all of them are related to each other. First of all, there are the fields of database tables in the lowest level. Then there are model fields representing the database fields in python. Form fields go on top in the abstraction level. Then there are bound fields which bound the form fields with data from the database (or somewhere else). And finally there are HTML fields which represent the rendered bound fields and which can be spiced up with Javascript for creating new widgets.

Now imagine, that you have a Contact model which includes two phone numbers where each of them has three parts. They are the country code, area code, and the serial number. In the simplest case you would have something like this:

COUNTRY_CODE_CHOICES = getattr(settings, "COUNTRY_CODE_CHOICES", (
    ("+44", "United Kingdom (+44)"),
    ("+49", "Germany (+49)"),
    ("+370", "Lithuania (+370)"),
    ))

class Contact(models.Model):
    user = models.OneToOneField(User)
    website = models.URLField(_("Website"))
    primary_phone_country = models.CharField(
        _("Primary Country Code"),
        max_length=4,
        choices=COUNTRY_CODE_CHOICES,
        )
    primary_phone_area = models.CharField(
        _("Primary Area Code"),
        max_length=4,
        )
    primary_phone_number = models.CharField(
        _("Primary Number"),
        max_length=10,
        )
    secondary_phone_country = models.CharField(
        _("Secondary Country Code"),
        max_length=4,
        choices=COUNTRY_CODE_CHOICES,
        blank=True,
        )
    secondary_phone_area = models.CharField(
        _("Secondary Area Code"),
        max_length=4,
        blank=True,
        )
    secondary_phone_number = models.CharField(
        _("Secondary Number"),
        max_length=10,
        blank=True,
        )

But it seems to be such a mess when it could be something like this:

class Contact(models.Model):
    user = models.OneToOneField(User)
    website = models.URLField(_("Website"))
    primary_phone = PhoneField(_("Primary Phone"))
    secondary_phone = PhoneField(_("Secondary Code"), blank=True)

So the question now is how to create a model field that would be saved as three separate database fields and would be shown as three bound fields when editing. Let's have a look what information is available about custom model fields. There is a page in the documentation which might help us here. Also there are articles by David Cramer and Michael Elsdörfer which might help to solve this problem. In addition, I found a snippet by Myles Braithwaite which also deals about custom model fields.

Combining the ideas from the given sources with the analysis of the Django core code, I defined the PhoneField in the following way:

# -*- coding: utf-8 -*-
from django.db import models
from django.conf import settings
from django.utils.translation import ugettext_lazy as _

PHONE_COUNTRY_CODE_CHOICES = getattr(settings, "PHONE_COUNTRY_CODE_CHOICES", (
    ("+44", _("United Kingdom (+44)")),
    ("+49", _("Germany (+49)")),
    ("+370", _("Lithuania (+370)")),
    ))

class PhoneField(models.Field):
    def __init__(self, verbose_name=None, country_code_choices=PHONE_COUNTRY_CODE_CHOICES, *args, **kwargs):
        super(PhoneField, self).__init__(*args, **kwargs)
        self.country_code_choices = country_code_choices
        self.verbose_name = verbose_name
        
    def contribute_to_class(self, cls, name):
        self.name = name
        if self.verbose_name is None and name:
            self.verbose_name = name.replace('_', ' ')
        # creating three model fields on the fly
        models.CharField(
            _("%s Country Code") % self.verbose_name,
            max_length=4,
            choices=self.country_code_choices,
            blank=self.blank,
            ).contribute_to_class(cls, "%s_country" % name)
        models.CharField(
            _("%s Area Code") % self.verbose_name,
            max_length=4,
            blank=self.blank,
            ).contribute_to_class(cls, "%s_area" % name)
        models.CharField(
            _("%s Number") % self.verbose_name,
            max_length=10,
            blank=self.blank,
            ).contribute_to_class(cls, "%s_number" % name)
        # when accessing the phone field by original model field name,
        # we'll manage tuples of country code, area code, and number
        setattr(cls, self.name, PhoneFieldCreator(self))
        
class PhoneFieldCreator(object):
    def __init__(self, field):  
        self.field = field
   
    def __get__(self, obj, type=None):
        if obj is None:  
            raise AttributeError('Can only be accessed via an instance.')  
        country = obj.__dict__.get("%s_country" % self.field.name, None)
        area = obj.__dict__.get("%s_area" % self.field.name, None)
        number = obj.__dict__.get("%s_number" % self.field.name, None)
        return (country and area and number) and (country, area, number) or None
   
    def __set__(self, obj, value):  
        if isinstance(value, tuple) and len(value) == 3:
            setattr(obj, "%s_country" % self.field.name, value[0])
            setattr(obj, "%s_area" % self.field.name, value[1])
            setattr(obj, "%s_number" % self.field.name, value[2])

Note that the instances of the model with the PhoneField can be filtered by any part of the phone number. In addition, the field value can be accessed as a tuple of three parts of the number. For example:

>>> Contact.objects.filter(primary_phone_country="+370")[0].primary_phone
(u'+370', u'628', u'12345')

Now the phone number can be presented in different ways in templates depending on the business requirements or local flavor:


{{ obj.primary_phone|join:"" }}
{{ obj.primary_phone|join:" " }}
{{ obj.primary_phone|join:"-" }}
{{ obj.primary_phone.0 }} ({{ obj.primary_phone.1 }}) {{ obj.primary_phone.2 }}
{{ obj.primary_phone_country }} ({{ obj.primary_phone_area }}) {{ obj.primary_phone_number }}

Let's check how to create model fields which have single database table field, but multiple form fields in the Part 2.

Django Poor Man's Cron

2008-10-21T01:14:00.005+02:00

As I have promised in the last blog entry, I am sharing a link to an experimental conceptual app Poor Man's Cron which tends to exploit various bots to implement scheduled tasks.

Criticisms, praises, thanks, and suggestions are very welcome!

Exploiting Spammers, Search Engines, and Other Bots

2008-10-16T11:00:00.001+02:00

Django is a great platform for building web projects in a clean and manageable way. But when you want to find its relationship with poverty for a blog post on Blog Action Day, it becomes a difficult puzzle. But still that's a great challenge!

Django has a large community all over the world and someone who is gold at statistics could write a cool analytical report about djangoers' relations with poverty in the world or the user activity on Blog Action Day site using the maps and numbers provided in the links.

The Django community or the social groups mentioned above are just small parts of the big network. Looking at the bigger image of the world, internet, and the future, you might realize that everybody is connected to somebody and something, all those connections get defined in social networks, and the web is like a living and evolving organism. We are kind of going towards a machine that we can use and it can use us (See the Matrix).

We can look at a newly created site based on Django as at a small node in the web. The site together with its visitors forms a network. Different APIs and inter-site communication extend the network even more. One part of this network is various bots, i.e. search engine indexers and comment spambots. Thank to the error-reporting mechanism in Django, recently I found out that you can find bugs in large-scale projects not only by manual browsing or test cases. Bugs might be detected by visits of spammers and search engines. Broken pages or unexpected usage are reported by email to the administrator of the site. Moreover, I started brainstorming how to exploit spammers instead of fighting against them.

And then I remembered the Poor Man's Cron module for Drupal. The module is an alternative for projects which have no cron jobs supported. It runs scheduled tasks in approximately regular intervals of time, triggered by page views. Executing scheduled tasks via the page views for people might be annoying if that takes much time. But I wouldn't mind to waste a few seconds of machines. On one hand this idea supports the poverty theme for the Blog Action Day, because this kind of module is dedicated to those who can't afford getting a server with cron jobs. On another hand, we can punish spammers using their time for our needs. When the tasks are quite time consuming and takes 5 or more minutes to execute, let's use the time of comment spammers. Spammers might be recognized by captchas, Akismet filters, and similar technologies. When more precise intervals between executions are necessary, let's exploit the feeds-subscriber calls and search indexers. Those can be recognized by request.META['HTTP_USER_AGENT'].

I shall point you to the experimental conceptual code of this idea soon.

Similar projects:
http://code.google.com/p/django-cron/
http://www.djangosnippets.org/snippets/1126/

How would you suggest to exploit spambots? What drawbacks do you see from the suggested approach?

P.S. To those who reside in Berlin! Today at 19:00 there will be Djangoers' meetup at newthinking store, Tucholskystraße 48, 10117 Berlin.

Gotchas about Fixtures for Initial Data

2008-09-13T05:48:00.002+02:00

One part of Django testing environment is fixtures. They are pre-prepared datasets to fill into the database before running tests. Django uses a separate database for running tests, but the fixtures from the files initial_data.* are also loaded into the main database when you synchronize it.

I make fixtures for my application like this:

python manage.py dumpdata --format=json --indent=4 myapp > apps/myapp/fixtures/initial_data.json

The initial data is read out from apps/myapp/fixtures/initial_data.json and written to the main or the test database when I synchronize the database

python manage.py syncdb

or when I run the tests

python manage.py test myapp

Fixtures are great for deploying standard data like predefined categories, lists of countries and languages, default flatpages, default navigation, default user groups, and so on. However, you should be very cautious with them while developing.

When I create new models, it's common practice to me to sync db so that the new database tables are created and the database schema reflects the models. Just after creation of new tables all fixtures called initial_data from all applications will be loaded. N.B. The fixtures from initial_data overwrite all existing data while synchronizing database. So if you have some important data that differs from the defaults, better make some database backup before syncing or use sql command to print out the SQL statements and execute them for the database manually:

python manage.py sql myapp_with_new_models

You might have pre_save signal handlers or custom save methods (check an example below) which should recognize newly created objects and do something special with them, i.e. prepare PDF reports, generate images, send emails, index for global text search, or something else. Usually in such cases I checked the existence of the primary key: the object is new if it has no primary key. But this is wrong when you use fixtures, because fixtures come with primary keys. N.B. The object is new only if there is no object in the database which primary key equals to the primary key of the current object.

class MyModel(models.Model):
    ...
    def save(self, *args, **kwargs):
        is_new = True
        pk = self._get_pk_val()
        model = type(self)
        if pk and model._default_manager.filter(pk=pk):
            is_new = False
        # something before saving
        super(model, self).save(*args, **kwargs)
        # something after saving

aka

class MyModel(models.Model):
    ...
    def save(self, *args, **kwargs):
        is_new = True
        if self.id and MyModel.objects.filter(id=self.id):
            is_new = False
        # something before saving
        super(MyModel, self).save(*args, **kwargs)
        # something after saving

Another alternative for storing default data would be custom sql located at apps/myapp/sql/mymodel.sql, but I haven't tried that yet and probably won't.

BTW, happy programmer day!

A Note on Python Paths

2008-09-03T13:54:00.011+02:00

This time I decided to share some knowledge about Python paths which seemed a little bit confusing to me in the beginning of diving into Python. I am working with Django in different platforms like Mac OS X, Windows, and Linux, therefore the common patterns how to activate new python modules in all of those environments should be familiar to me.

Python modules are either *.py files or directories containing __init__.py. When defining paths to python modules, you will usually need to deal with the latter ones. A module is meant to be under python path if you can run python and import that module.

For example, if you can run the following, then django is under your python path.

python
>>> import django

Stay tuned to get deeper into python paths.

Installing modules

If a module is installable, usually all you need to do is to extract its setup directory, cd to it, and run

python setup.py install

This will copy the module into the site-packages directory of the current python installation. It might be that you have multiple Python versions on your computer. According to django documentation, you can find the currently used site-packages by

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Or you can use PEAK EasyInstall for installing python modules even faster.

But sometimes you will need the latest and greatest versions of your modules directly from version control system. To make them accessible from python you should either check them out directly to site-packages (very messy and inflexible) or keep them somewhere else and do some additional magic.

Sym-linking

You can create symbolic links (symlinks) in unix-based systems like Linux or Mac OS X. A symlink is like a shortcut to a file or directory. If you create a symlink in site-packages which points to a python module which is located somewhere else, it will work as if the module was copied into site-packages.

To create a symlink, type the following in a console/terminal:

ln -s <source> <target>

For example, if you want python to access django which is under /Library/Subversion/django_src/trunk/django, you need to write something like this (considering that your site-packages are at /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/)

ln -s /Library/Subversion/django_src/trunk/django /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/django

To delete the symlink, simply remove it (this won't delete the original module):

rm /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/django

But as I've already mentioned, this works only in unix-based environments and you can't use shortcuts in Windows for the same purpose.

`*.pth` files

Python supports *.pth files which contain the paths to the parent directories of your python modules one per line. Those files should be placed in site-packages and can be called whatever you want them to call. For example, you can create a file my-project.pth and write

/Library/Subversion/django_src/trunk
/Library/Subversion/myproject_src/trunk

C:\Subversion\django_src\trunk
C:\Subversion\myproject\trunk

into it. Then django and your project files will be importable in python.

However, you might have no permissions to create files under site-packages or you might need to activate different locations of python modules for different projects.

`PYTHONPATH` variable

The other way is to set additional paths for python just before running the externally kept modules. This is done by setting the python paths to the environment variable PYTHONPATH. Note again that python paths point not to the modules themselves, but to their parent directories!

The syntax slightly differs among different platforms.

Linux and Mac OS X:

# checking value
echo $PYTHOPATH
# setting value
export PYTHONPATH="/Library/Subversion/django_src/trunk"
# appending to the existing value
export PYTHONPATH="$PYTHONPATH;/Library/Subversion/django_src/trunk"

Windows:

# checking value
echo %PYTHOPATH%
# setting value
set PYTHONPATH="C:\\Subversion\\django_src\\trunk"
# appending to the existing value
set PYTHONPATH="%PYTHOPATH%;C:\\Subversion\\django_src\\trunk"

Multiple paths can be separated by a colon (";").

PYTHONPATH can be used in scripts and webserver configuration files, but it is not very comfortable in daily use.

Adding paths to `sys.path`

For the projects that you develop and which should run as standalone applications, you can set the required python paths relatively inside you python code.

Note that all python paths which you set in the PYTHONPATH variable or *.pth files as well as the path of default python libraries and the path of site-packages get listed in python variable sys.path. When you import a module, it is loaded from the first location which contains the required module. So if you have two paths to different django versions in your python paths and you import django, the django version from the first location will be used.

You can read the list of loaded python paths like this:

>>> import sys
>>> sys.path

You can also freely modify it, for example:

>>> import sys
>>> sys.path.append("/Library/Subversion/django_src/trunk")
>>> import django

And this is an example, how to get and use paths relative to the currently loaded file:

import os, sys

SVN_PATH = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
DJANGO_PATH = os.path.join(SVN_PATH, "django_src", "trunk")
PROJECT_PATH = os.path.join(SVN_PATH, "myproject", "trunk")

sys.path += [DJANGO_PATH, PROJECT_PATH]

I hope this introduction was useful for developers and made the big picture about the paths clearer.

Some more related information can be found at the official python documentation.

MultipleSubmitButton Widget for ChoiceField

2008-08-08T18:05:00.003+02:00

Recently I published a snippet with a widget rendering a choice field as a series of submit buttons.

So the {{ form.do }} field from the following form:

SUBMIT_CHOICES = (
    ('save', _("Save")),
    ('save-add', _("Save and Add Another")),
    )

class TestForm(forms.Form):
    do = forms.ChoiceField(
        widget=MultipleSubmitButton,
        choices=SUBMIT_CHOICES,
        )

will be rendered as:

<ul>
<li><button type="submit" name="do" value="save">Save</button></li>
<li><button type="submit" name="do" value="save-add">Save and Add Another</button></li>
</ul>

Can somebody enhance this widget so that it supports iteration through different choices and getting specific buttons by indexes in the template? My trials failed, but maybe you will succeed!

EuroPython 2008

2008-07-16T01:16:00.019+02:00

I started writing this post at the airport just before flying back from Vilnius to Berlin. EuroPython 2008 - the three-days conference and sprints afterwards - gave me loads of information, new relations, and better understanding of the big image of Python developers' community.

Some highlights from the conference follow.

During video conference the author of Python programming language Guido van Rossum encouraged using Python 2.6 just after the release, but said that Python 3.0 still shouldn't be used for production. The good future-proof practices are inheriting from object for classes, using xrange(), sorted(), and zip() functions, "//" for floor divisions, and dict.iterkeys(). The full porting to 3.0 guide will be probably written and published in the python.org site.

Clone Digger is a project from Google Summer of Code 2008 for finding similar pieces of code which could be optimized by moving that to parameterized functions.

Restrictive Python is a project which blocks the user from executing dangerous functions or methods or accessing files which might harm the system. You can try Restrictive Python out in the simulated terminal.

Python is an interpreted language and if you want to deploy a project writen in Python, you have to be sure that the end user will have Python interpreter installed there. O you can create an executable file from Python project using one of the following tools: Py2exe, PyInstaller, or BBFreeze.

There is a combined installer of Python, SQLite, and Django for Windows and it is called Instant Django.

Django DB External Scheme Evolution Branch is a separate application for doing DB scheme changes easily without manual DB field editing.

Honza Král showed the essentials of the new-forms admin to which I am going to port our Django projects the next days. Directly from him I learned that new forms will still have validators which will work only to check the value of that one specific field (not related to other fields). If some multiple field validation is necessary, the clean method for the form should be overwritten. All the global field-unrelated form errors will be passed to {{ form.non_field_errors }} variable.

Christian Scholtz presented PYOGP which is Python-based environment for testing the Open Grid Protocol used for exchanging avatars and other objects among virtual realms like different regions of Second Life.

Ricardo Quesada and Lucio Torre told us about cocos2d - a framework made in Python for presentations and sprite-based 2D games.

If you ever need to export some data from a database to Excel Sheet or import from Excel, you can do that not only in CSV (Comma Separated Values) format. You can also use the original XLS format with one of the following libraries: xlrd, xlwt, or pyExcelerator. I successfully used the latter one last week.

CVS and SVN are not the only version control management systems in the world :D. The others like Mercurial, Git and Bazaar are also worthy checking. These are all distributed systems which mean that they don't have one centralized source. Bazaar is written in Python and has API for manipulating information about revisions, committers, and other version-control-related stuff.

Some other useful links:

Django Sprint at EuroPython 2008

Talk Materials

Discount for O'Reilly Books

Others blogged about EuroPython 2008 as well:

Ville Säävuori: Day 1, Day 2, Day 3.

Olle Jonsson Morningstar

Ted Leung

Chee Ming Chew

Django Meetup Postmortem

2008-06-02T19:28:00.012+02:00

Thanks to the organizer Jannis, last Friday German djangoers met successfully, got some beer at Schleusenkrug in Tiergarten, Berlin, and became better aquainted with each other. We discussed the latest actualities like licenses, project deployment, business models, the future of Django, who from the gods of Django said what in which blog, and other geeky stuff.

If you want to get in touch with German djangoers, you can join the #django-de IRC channel on irc.freenode.net. Also you can subscribe to their blogs at django-de.org.

Django Meetup at Linuxtag

2008-05-27T19:34:00.007+02:00

Pssst. Don't tell anybody! There is an unofficial Django user meetup happening in Berlin this Friday at five.

datetime.datetime(2008, 5, 27, 17, 0)

The gathering point is at the main entrance of Linuxtag conference (Messe Berlin).

Decorating the Render Methods of New-Form Widgets

2008-05-23T23:07:00.015+02:00

Perhaps all template masters have already faced the problem of styling HTML input fields of different types. The selectors like input[type=checkbox] and similar in CSS are not supported by IE so people working with templates and CSS obviously need some other way to select and style specific types of input fields.

There are a few ugly ways to do that which violate the DRY principle:

Encompassing the form element in the template with HTML tag which has a class representing specific type of the input field.
For example:
```
<span class="form_checkbox">
    {{ form.is_privacy_policy_confirmed }}
</span>
```

Defining the specific CSS class for each form field widget in the form.
For example:

class FormExample(forms.Form):
    is_privacy_policy_confirmed = forms.BooleanField(
        required=True,
        widget=CheckboxInput(attrs={'class': 'form_checkbox'}),
        )

Extending all Fields and all Widgets which use HTML input fields and using the extended versions instead of the originals.

I don't like any of them, because they force me or the template formatters to repeat ourselves and make plenty of replacements in our existing forms.

Although "most sane developers consider it a bad idea", I see the Guerrilla patching of the Widget's render method as the nicest solution to solve this problem.

Guerrilla patch is the modification of the runtime code in dynamic languages without changing the original source code.

The method render of the Widget class draws the input field in HTML. As it takes a parameter attrs for additional input field attributes, my idea was to create a decorator which modifies the incoming parameters and adds a CSS class "form_TYPE", where TYPE is the input field type.

With a little help by Dalius Dobravolskas, I succeeded to code a decorator having an optional parameter which defines the CSS class name for the field. If the class name is not defined, the attribute input_type of the Widget class is used for forming the CSS class name (N.B. not all widgets have this attribute).

from django.newforms.widgets import Input, CheckboxInput, RadioSelect, CheckboxSelectMultiple

### adding class="form_*" to all html input fields ###
def add_css_class(css_class=""):
    def modify_input_class(function):
        _css_class = css_class
        def new_function(*args, **kwargs):
            arg_names = function.func_code.co_varnames
            new_kwargs = dict(zip(arg_names, args))
            new_kwargs.update(kwargs)
            css_class = _css_class or "form_%s" % getattr(
                new_kwargs['self'],
                "input_type",
                "undefined",
                )
            self = new_kwargs.pop("self")
            attrs = getattr(self, "attrs", None) or {}
            if "class" in attrs:
                css_classes = attrs["class"].split()
                if css_class not in css_classes:
                    css_classes.append(css_class)
                attrs["class"] = " ".join(css_classes)
            else:
                attrs["class"] = css_class
            self.attrs = attrs
            return function(self, **new_kwargs)
        return new_function
    return modify_input_class
Input.render = add_css_class()(Input.render)
CheckboxInput.render = add_css_class("form_checkbox")(CheckboxInput.render)
RadioSelect.render = add_css_class("form_radio")(RadioSelect.render)
CheckboxSelectMultiple.render = add_css_class("form_checkbox")(CheckboxSelectMultiple.render)

To use this code, just place it in some models.py file in your project.

The strange part here was that the variable css_class isn't recognized by the sub-child function new_function directly although the scope of the variable css_class should let it be accessed there. Anyway, the value got easily accessible when I reassigned it to another variable like _css_class in the child function modify_input_class.

The tricky part of this snippet was getting the attrs argument from the decorated function as it was not clear whether it would be passed as a positional or as a named argument. The first three lines of the function new_function collects all the incoming arguments to a dictionary new_kwargs. They can be modified and then passed to the original function to decorate.

Ups. I am late to the studio. So see you next time!

Tips #2

2008-04-08T20:37:00.008+02:00

What time is it now? It's time to give you some more tips about Django development!

Use batch scripts to automate manual routines. Do not repeat yourself extracting and compiling translatable strings, starting and stopping development web-servers, updating and committing your project to the version-control system in the console. Write batch scripts which you can run within one mouse click instead.

Define overwritable constants in your applications. Your applications are likely using some values that might be defined as constant values, i.e. the dimensions for avatars of users. Define those constants so, that you could overwrite them in the project settings if necessary.

from django.conf import settings
SOME_SETTING = getattr(settings, "SOME_SETTING", "default value")

Have one view for site-related JavaScript globals. Django views usually return (X)HTML-based responses, but it can return XML, JavaScript or others as well. Usually you will hold all you JavaScript functions in static files, but there might be some situation, where you need to get information related to database or project settings, for example, the MEDIA_URL.
The following view might be used to display a javascript page directly from a template. Just pass the template to the view in your urls.py file.

from datetime import datetime, timedelta
from django.views.generic.simple import direct_to_template
def direct_to_js_template(request, *args, **kwargs):
    response = direct_to_template(request, *args, **kwargs)
    response['Content-Type'] = "application/x-javascript"
    now = datetime.utcnow()
    response['Last-Modified'] = now.strftime('%a, %d %b %Y %H:%M:%S GMT')
    expires = now + timedelta(0, 2678400)
    response['Expires'] = expires.strftime('%a, %d %b %Y %H:%M:%S GMT')
    return response

And now it's time to go home and to get relaxed.

DRY While Working With Choices for Forms

2008-03-24T09:34:00.007+01:00

When creating dozens of forms with selection fields for some many-to-one or many-to-many relations, you might find that it's ineffective to create choices for the form fields from querysets formed by the relations defined by ForeignKeys and ManyToManyFields. You have to import the related models, filter the choices analogously to the limit_choices_to parameter, and form a list of tuples again and again.

To get the same choices from the model as in the admin form, you can use the following:

FIELD_CHOICES = SomeModel._meta.get_field("field_name").get_choices()

Then you can modify the text for the null-value choice, like

FIELD_CHOICES[0] = ("" _("- Choose One -"))

or even remove it:

del FIELD_CHOICES[0]

To save the selected object you can simply assign the chosen value to the foreign key, like:

new_instance = SomeModel()
new_instance.field_name_id = form.cleaned_data['field_name']
new_instance.save()

If you need to do something with the selected object, you can still live without importing specific models and filtering the entries in the same manner as limit_choices_to. To save time, you can use the following function, which returns the queryset containing all the choosable objects:

def get_related_queryset(model, field_name):
    """
        Get the queryset for the choices of the field in a model
        Example:
            objects = get_related_queryset(SomeModel, "field_name")
    """
    f = model._meta.get_field(field_name)
    qs = f.rel.to._default_manager.complex_filter(f.rel.limit_choices_to)
    return qs

Just put this function in one of you applications and import it whenever you need to work with forms. And have happy Easter!

DjangoTricks

The Usage of the only() Method

Custom Admin Filter Specification

Django CMS, Haystack, and Custom Plugins

Deployment Using Fabric

Weather App on Github

Weather App Tutorial. Part 5 of 5. Graph View

Weather App Tutorial. Part 4 of 5. Template Tag

Weather App Tutorial. Part 3 of 5. Management Command

Weather App Tutorial. Part 2 of 5. Models

Weather App Tutorial. Part 1 of 5. Preparation

Applying Patches in Guerrilla Way

Design Pattern for Reusable Apps: Getting into Details

Abstract Models and Dynamicly Assigned Foreign Keys

Clarification for Confusion: Creating New Django Fields. Part 1

Django Poor Man's Cron

Exploiting Spammers, Search Engines, and Other Bots

Gotchas about Fixtures for Initial Data

A Note on Python Paths

Installing modules

Sym-linking

*.pth files

PYTHONPATH variable

Adding paths to sys.path

MultipleSubmitButton Widget for ChoiceField

EuroPython 2008

Django Meetup Postmortem

Django Meetup at Linuxtag

Decorating the Render Methods of New-Form Widgets

Tips #2

DRY While Working With Choices for Forms

`*.pth` files

`PYTHONPATH` variable

Adding paths to `sys.path`