/usr/share

A Fairly Low Ceremony Method for Modularizing Django Settings

Sat, 04 Aug 2012 23:20:00 -0500

Updates

No longer wrapping modules in def require
Using pkgutil and inspect to get module names and constants

Every time I need to add anything to settings.py in Django, I find myself spending a good minute trying to find a logical place to put it. But it’s always the same. I give up, put it somewhere arbitrary, and move on feeling cheapened, dirty, and defeated. There must be a better way…

This wiki page provides a ton of examples that get very close. They let you override settings per environment, but you’re still arbitrarily placing unrelated settings together. I’d like something more modular. Today I worked out a method that is very modular and includes all the perks of the above examples at the cost of a small bit of ceremony.

For reference, take a look at the updated example project settings. The gist of it is moving settings into it’s own package, importing all settings in __init__.py, and placing specific settings into aptly named modules.

Version 2.0

import inspect
import pkgutil


for _, module_name, _ in pkgutil.walk_packages(__path__):
    module = __import__(module_name, globals(), locals(), [])
    for var_name, val in inspect.getmembers(module):
        if var_name.isupper():
            locals().update({var_name: val})

This version improves on the first by getting module names using pkgutil.walk_packages which means you can put settings in packages now too.

Also, instead of setting up the secrets, paths, and environment in here and passing them via require I let the settings modules import the settings they need.

import os

import env
import paths
import secrets


if env.DEV_ENV:
    database_dir = os.path.join(paths.BASE_PATH, 'databases')

    if not os.path.exists(database_dir):
        os.mkdir(database_dir)

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': os.path.join(database_dir, 'sqlite3'),
        }
    }
else:
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'USER': secrets.DATABASE_USER,
            'PASSWORD': secrets.DATABASE_PASSWORD,
            'HOST': secrets.DATABASE_HOST,
            'PORT': secrets.DATABASE_PORT,
        }
    }

Instead of import from I just import the modules directly and grab the settings from them. This makes things a bit more explicit and ensures the imported settings don’t get added twice in __init__.py.

Extending settings is exactly the same as version 1.0:

import env


INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # Uncomment the next line to enable the admin:
    # 'django.contrib.admin',
    # Uncomment the next line to enable admin documentation:
    # 'django.contrib.admindocs',
)

if env.DEV_ENV:
    INSTALLED_APPS += (
        'debug_toolbar',
    )

So in this version we’ve cleaned up __init__.py quite a bit and removed the ceremony from the settings modules. This is exactly what I wanted from the get go and I’m pretty stinkin happy with it.

Version 1.0

import os

from secrets import *

SETTINGS_PATH = os.path.abspath(os.path.dirname(__file__))
PROJECT_PATH  = os.path.dirname(SETTINGS_PATH)
BASE_PATH     = os.path.dirname(PROJECT_PATH)

ENV      = os.getenv('DJANGO_ENVIRONMENT', 'development')
DEV_ENV  = ENV == 'development'
TEST_ENV = ENV == 'staging'
PROD_ENV = ENV == 'production'


def _constants(d):
    return dict((key, val) for key, val in d.items() if key.isupper())

for settings_file in [filename.rstrip('.py') for filename in os.listdir(SETTINGS_PATH)
        if not filename.endswith('.pyc') and filename not in ('__init__.py', 'secrets.py')]:

    module = __import__(settings_file, globals(), locals(), [])
    settings = module.require(**_constants(locals()))
    locals().update(_constants(settings))

The first thing we do is import secrets.py. This is where you’ll put the SECRET_KEY, database credentials, etc, and is the standard of the wiki examples. Next, we setup our project paths and the environment variable. But from here I stray from the norm a bit. Instead of importing a module based on the environment, I go ahead and import everything in the settings package (listdir may not be the most elegant way to get each filename, let me know if there’s a better approach).

I quickly ran into a problem though. I want to use the secrets, project paths, and environment settings inside the imported modules. This is where the ceremony comes in. To get around this I wrap each settings module in a function I can call in __init__.py, passing in the settings it needs like so:

import os

def require(DEV_ENV, BASE_PATH, DATABASE_USER, DATABASE_PASSWORD,
        DATABASE_HOST, DATABASE_PORT, **kwargs):

    if DEV_ENV:
        database_dir = os.path.join(BASE_PATH, 'databases')

        if not os.path.exists(database_dir):
            os.mkdir(database_dir)

        DATABASES = {
            'default': {
                'ENGINE': 'django.db.backends.sqlite3',
                'NAME': os.path.join(database_dir, 'sqlite3'),
            }
        }
    else:
        DATABASES = {
            'default': {
                'ENGINE': 'django.db.backends.postgresql_psycopg2',
                'USER': DATABASE_USER,
                'PASSWORD': DATABASE_PASSWORD,
                'HOST': DATABASE_HOST,
                'PORT': DATABASE_PORT,
            }
        }

    return locals()

So each module must def require (hat tip to RequireJS) with **kwargs and return locals(). Not too bad. Also, the above example shows another way in which I diverge from the wiki examples. Instead of overriding per environment, I just check the environment setting and drop in the correct setting. To extend settings per environment you would do the following:

def require(DEV_ENV, **kwargs):

    INSTALLED_APPS = (
        'django.contrib.auth',
        'django.contrib.contenttypes',
        'django.contrib.sessions',
        'django.contrib.sites',
        'django.contrib.messages',
        'django.contrib.staticfiles',
        # Uncomment the next line to enable the admin:
        # 'django.contrib.admin',
        # Uncomment the next line to enable admin documentation:
        # 'django.contrib.admindocs',
    )

    if DEV_ENV:
        INSTALLED_APPS += (
            'debug_toolbar',
        )

    return locals()

All in all, I feel like this is a step in the right direction. If the settings package crammed full of sparse modules makes you queazy, than stick with one of the wiki examples. But for me, the ability to drop in a widgery.py module when I want to configure my widgery app is a breath of fresh air. Everything finally has a place.

Let me know what you think!

Sharefix vim plugin

Sat, 09 Jun 2012 23:42:00 -0500

Sharefix vim plugin:

Have you ever wanted to keep your quickfixes available when running different commands and functions in vim? Now you can.

Sass One Liners

Wed, 06 Jun 2012 07:14:00 -0500

Sass One Liners:

I always have trouble remembering the syntax for linear gradients, especially the legacy webkit version. It would be nice if compass had a repl that you could execute compass’ CSS3 mixins and functions in and get the nicely prefixed properties out. Well it does!

compass interactive

Here we go:

>> background(linear-gradient(white, black))
background(linear-gradient(#ffffff, #000000))

Ok, the colors were replaced at least. Let’s try the sass syntax for mixins:

>> +background(linear-gradient(white, black))
"+background(linear-gradient(#ffffff, #000000))"

Thanks for quoting it, but no thanks. How about scss:

>> @import background(linear-gradient(white, black))
SyntaxError: Invalid CSS after "": expected expression (e.g. 1px, bold), was "@import backgro..."

Alrighty, unless I’m missing something, the sass repl is pretty useless for compass extensions. There’s even an option for the sass command to run a repl with compass enabled:

sass -i --compass

Same results.

So as a work around I wrote a script to pipe in a line of sass to the sass command, compile it, and print the properties:

$ sass-cat -c "+background(linear-gradient(white, black))"
background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(0%, #ffffff), color-stop(100%, #000000));
background: -webkit-linear-gradient(#ffffff, #000000);
background: -moz-linear-gradient(#ffffff, #000000);
background: -o-linear-gradient(#ffffff, #000000);
background: -ms-linear-gradient(#ffffff, #000000);
background: linear-gradient(#ffffff, #000000);

Prefetch Images With Javascript

Fri, 25 May 2012 21:55:00 -0500

This one is an oldie but a goodie. If you ever need to swap out the src on an image tag you can prefetch images with javascript to make the swap instant after the page loads. All you need to do is create an image element in javascript and assign the url to the src. The browser will request the image immediately regardless of what you do with the image element. You can try this out in the network tab of your browser dev tools:

Now you can assign the same url to an img tag in your html and the image will change immediately if it has finished loading in the background. You can also prefetch and swap out background images the same way.

try it out in this fiddle

Vim Python Complete Function

Fri, 27 Apr 2012 00:00:00 -0500

I finally found a nice complete function setup for python in vim. It’s simply using rope-vim’s code assistance but there are a few settings you need to enable. I should note that I was clued in on rope and a ton of other great plugins through a nice article by John Anderson on using vim as a python ide. You can follow his setup or I use janus so I dropped them into my janus folder as git submodules. The rope-vim plugin he suggests includes all the rope packages so you don’t need to install them yourself.

In the code assist section of :h ropevim.txt, you’ll see that you can set a few globals to have rope-vim handle completion:

let g:ropevim_vim_completion=1
let g:ropevim_extended_complete=1

The first setting makes the RopeCodeAssistInsertMode function behave like a vim complete function, i.e. it display a popup menu when called. And the second adds useful bits of info to each completion: the location the completion came from {'L': 'local', 'G': 'global', 'B': 'builtin', '': 'no scope available'}, it’s type, and the first line of the doc string. It’s pretty sweet!

Other than that, you might want to remap the awkward default code assist mapping of alt+shift+comma (see the filetype commands of my vimrc for an option). Also, if you’ve never used rope before it will ask you to create or open a project each time. This stores a hidden ropeproject directory that caches info about your modules to make completions faster. It’s slightly annoying to open each time but it’s worth it for completions.

And that’s all there is to it!

Bret Victor - Inventing on Principle

Mon, 20 Feb 2012 08:02:06 -0600

Bret Victor - Inventing on Principle:

Summary

Devoting yourself to a principal isn’t confined to social services for example, you can take a stand on a principal in technology too. This transforms your view from ‘how can I turn this problem into a viable product?’ into ‘how can I prevent this problem from hindering myself and others?’.

Finding your principal can take a while, but when you do, it will last a lifetime. Experience many things to see what pikes your interest or bugs your conscience and sooner than later, you’ll be able to sum up what drives you with a concise, guiding principal.

Web Art

Sat, 17 Dec 2011 08:00:00 -0600

I recently read Paul Graham’s essay Hackers and Painters and it solidified a few reasons why I enjoy programming for the webs. Graham uses art as an analogy for different aspects of programming. A couple of his points stood out: That programming is not all theory and algorithms as most assume. And that dynamic languages are superior in that they lend themselves to the creative process.

Part of the essay breaks down the misconception that hacking is applied computer science, which implies to program you must have crazy computational theories and algorithms constantly floating around your mind. Instead, Graham paints the picture that hacking is more like art. Programming languages are our brushes and paint and the screen is our canvas.

Along with composers, architects, and writes, what hackers and painters are trying to do is make good things.

This is especially true of programming for the webs, which is where Graham is coming from. The web programmer is like a skilled craftsman. With an assortment of open source tools, he mashes up, duck tapes, and glues together the raw materials of the web: HTML, CSS, and Javascript. He is a maker and killer features are his commodity.

The point is web programming is more hands on. Just because it involves computers doesn’t mean it’s all computer sciency. Sure, web programming can be complex. Just to get started, you have to learn three languages. You’ll need a database at some point, which can involve mind bending queries and endless optimization. You’ll need to use design patterns as your programs get massive. And a sea of acronyms and jargon awaits you on the horizon. But the problems you’ll be solving are hardly theoretical. It’s just a matter of practice, not nth power thinking.

All the time I was in graduate school I had an uncomfortable feeling in the back of my mind that I ought to know more theory, and that it was very remiss of me to have forgotten all that stuff within three weeks of the final exam.

Now I realize I was mistaken. Hackers need to understand the theory of computation about as much as painters need to understand paint chemistry.

This is comforting because my little mind has a lot of trouble grasping the abstract underpinnings of programming. Fortunately, all those problems are solved by someone else, which is why Graham was able to forget the theory and focus on more important things and why I can participate.

Another point Graham makes is that languages are best served dynamic. Graham found his workflow consisted almost entirely of debugging. Instead of carefully planning out his programs on paper like he was taught in college, he started in an editor, building and debugging until he was satisfied.

For a long time I felt bad about this, just as I once felt bad that I didn’t hold my pencil the way they taught me to in elementary school. If I had only looked over at the other makers, the painters or the architects, I would have realized that there was a name for what I was doing: sketching.

He goes on to say that the programming workflow should match other creative processes like writing and painting. Dynamic, interpreted languages lend themselves to this workflow quite nicely. They’re high level and provide a lot of implicit functionality. Therefore you can express what you’re thinking rather than placate the compiler with crufty boilerplate. And you can write and test bits and pieces as you go, just like sketching.

The beauty of web programming is that it embraces dynamic interpreted languages. First of all, the browser interprets everything. I knew I loved web programming when I first learned HTML and CSS and could see immediate results with a page refresh. And Javascript is as dynamic as it gets. Second, the backend is language agnostic. As long as you eventually speak HTML, CSS, and JS, you can write your code in whatever you want (Python!).

To conclude, if you’re interested in web development, don’t let theory bog you down. Start by learning the tools while building projects. And if you’re interested in programming in general, the web is a great place to be since it gives you the freedom to express your thoughts in the language of your choice.