Just-Another.Net blog posts

PyOhio 2009: Call for Proposals

PyOhio 2009, the second annual Python programming mini-conference for Ohio and surrounding areas, will take place Saturday-Sunday, July 25-26, 2009 at the Ohio State University in Columbus, Ohio. A variety of activities are planned, including tutorials, scheduled talks, Lightning Talks, and Open Spaces.

PyOhio invites all interested people to submit proposals for scheduled talks and tutorials. PyOhio will accept abstracts on any topics of interest to Python programmers.

Standard presentations are expected to last 40 minutes with a 10 minute question-and-answer period. Other talk formats will also be considered, however; please indicate your preferred format in your proposal. Hands-on tutorial sessions are also welcomed. Tutorial instructors should indicate the expected length

PyOhio is especially interested in hosting a Beginners’ Track for those new to Python or new to programming in general. If your proposal would be suitable for inclusion in the Beginners’ Track, please indicate so. Organizers will work with speakers and instructors in the Beginners’ Track to help them coordinate their talks/tutorials into a smooth, coherent learning curve for new Python users.

All proposals should include abstracts no longer than 500 words in length. Abstracts must include the title, summary of the presentation, the expertise level targeted, and a brief description of the area of Python programming it relates to.

All proposals should be emailed to cfp@pyohio.org for review. Please submit proposals by May 15, 2009. Accepted speakers will be notified by June 1.

You can read more about the conference at http://pyohio.org

If you have questions about proposals, please email cfp@pyohio.org. You can also contact the PyOhio organizers at pyohio-organizers@python.org.

Python STOMP Implementation

Everyone else is doing it!

Quite seriously though… In my dealings with JMS Message Queuing, specifically with ActiveMQ, I felt the lack of a simple, straight-forward and easy to use Python implementation of the protocol that was also well documented and hackable.

More often than not, I found myself using Perl’s Net::Stomp module because it was so much quicker to get going. Gasp you say, a Perl module that is easier to use than a Python module?! BLASPHEMY! I implore you to believe me!

I don’t think I’ve reinvented the wheel, merely rounded it to my specifications. So, if you choose, check out my implementation of STOMP in Python over at Bitbucket: python-stomp.

It is feature-complete according to the STOMP Protocol, 100% Unit test covered, documented, comes with example usage and is simple to use.

Feedback is appreciated in the form of issues on the Bitbucket issue tracker or this blog if you want to say something nice, or mean about it!

Documentation:

Frame
Stomp

Code:

Code

Psyched for Pycon 2009

I’m jazzed to be heading to Chicago again this year for pycon. Last year I had a blast and learned a lot. I even got to apply some of what I learned in practice at work and on personal projects! I’m hoping that I’ll take away as much value from the conference this year. A quick look at the talks suggest that won’t be a problem.

I’m going to try to blog while I’m there, but most likely you’ll get more out of me via twitter, so why not follow me there?

Here’s the list of talks I really want to attend:

There is no order here, scheduling is the tricky part..

How to Give a Python Talk — Mr. Andrew M Kuchling
Paver: easy build and deployment automation for Python projects — Kevin Dangoor
Building tests for large, untested codebases — Dr. C. Titus Brown
Coverage testing, the good and the bad — Ned Batchelder
Designing Applications with Non-Relational Databases — Ben Bangert
Pinax: a platform for rapidly developing websites — James Tauber
Panel: Functional Testing Tools in Python — Panel, multiple folks.

There’s more, but those are the ones that stuck out to me.. Don’t think I’m in too much danger of missing anything this year.

A few of us at AG Interactive are planning a night of getting together for what we’re dubbing as #pyconbeer on twitter, so keep a search up: Twitter #pyconbeer I’m thinking we’re going to shoot for Friday evening, the 27th.

Hope to see many of you there and put names to faces!

Django debug toolbar FTW.

If you do any development with Django, then this is a must have.

Not only is it awesomely useful, but it also has a great abbreviation that could be a throwback to WWF days of old. Any child of the mid-to-late 80’s remember the DDT?! That’s kind of what this is does, help you drive problems with your app head first into the mat - nice.

Google code page here: DDT on google code

Github here: DDT on github

Like any other python package, this one is simple to install.

Get a tarball, unpack it, cd into it..

# python setup.py build
# sudo python setup.py install

Now, use it in your django app by adding something similar to your settings.py:

# This should be appended to MIDDLEWARE_CLASSES:
'debug_toolbar.middleware.DebugToolbarMiddleware',

# This should be appended to INSTALLED_APPS:
'debug_toolbar',

# Set this to localhost if developing locally
# If not developing locally, add your remote IP..
INTERNAL_IPS = ('127.0.0.1')

# Various apps available
DEBUG_TOOLBAR_PANELS = (
    'debug_toolbar.panels.version.VersionDebugPanel',
    'debug_toolbar.panels.timer.TimerDebugPanel',
    'debug_toolbar.panels.headers.HeaderDebugPanel',
    'debug_toolbar.panels.request_vars.RequestVarsDebugPanel',
    'debug_toolbar.panels.sql.SQLDebugPanel',
    'debug_toolbar.panels.cache.CacheDebugPanel',
    'debug_toolbar.panels.logger.LoggingPanel',
)

See here: README for more detail on usage/install.

Get it, use it, love it.

EDIT: Added some quick bits on how to install/use

byteflow/django+supervisord+nginx = WIN

Recently I made the final step in converting my website over to nginx. I decided to manage my django application, byteflow, with supervisor. I have had great success with this over at AGI and thought I should advocate the success I’ve had to the public by extending it to my own site.

I first heard about supervisor at pycon last year and thought it could be useful in many ways, especially at the office. In a nutshell, supervisor ‘supervises’ processes, and allows you to manage them with a simple interface, but I’ll go into more detail about it later. Around the same time that I discovered supervisor I had also started experimenting with nginx and fastcgi to run my blog. I ended up going with lighty and fastcgi instead, however, mainly due to familiarity. Things change, and so do my opinions on technology. Nginx sold me on it’s performance and simplistic configuration, plain and simple. So now, here’s the meat and potatoes.

To break it down, here is what we’re looking at.

INTARWEBS -> nginx -> fastcgi -> django/byteflow(managed by supervisor)

Everything below assumes you have installed nginx, django, byteflow and supervisor.

So how about a look at the actual setup.

First, let’s take a look at the code to fire up the django app. I’ve dubbed it runserver.py and put it in my byteflow code tree.

#!/usr/bin/env python
if __name__ == '__main__':
    from flup.server.fcgi_fork import WSGIServer
    from django.core.handlers.wsgi import WSGIHandler
    WSGIServer(WSGIHandler()).run()

This keeps the django devs happy by not relying on manage.py to keep up with the server stuffs.

It’s pretty simple.. Starts up flups’ WSGIServer and uses django’s WSGIHandler to do the dirty work. Not much to it, really.

Now, lets take a look at the supervisor setup. You can get a lot of what you need by running:

# sudo echo_supervisord_conf > /etc/supervisord.conf

That sets up a basic (but verbose!) supervisor config. I’ve trimmed it down to this:

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/var/log/supervisord/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
user=nobody                 ; (default is current user, required if root)
childlogdir=/var/log/supervisord/            ; ('AUTO' child log dir, default $TEMP)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

; Production setup
[fcgi-program:django_fcgi]
socket=tcp://127.0.0.1:8080  ; We reference this later in nginx
command = /home/bsmith/Dev/byteflow/runserver.py  ; Calls the above code
environment=PYTHONPATH=/home/bsmith/Dev/byteflow  ; Setup needed environment
environment=DJANGO_SETTINGS_MODULE=settings

; Development setup
[fcgi-program:django_dev_fcgi]
socket=tcp://127.0.0.1:8081
command = /home/bsmith/Dev/byteflow_new/runserver.py
environment=PYTHONPATH=/home/bsmith/Dev/byteflow_new
environment=DJANGO_SETTINGS_MODULE=settings

Simple enough, eh? Comments in the configuration should explain what’s going on.

Now you can crank it up by running:

$ sudo supervisord

and check the status…

$ sudo supervisorctl status
django_dev_fcgi:django_dev_fcgi_0 RUNNING    pid 15949, uptime 0:00:08
django_fcgi:django_fcgi_0        RUNNING    pid 15950, uptime 0:00:08

Could it be more simple? I doubt it. This barely begins to scratch the surface of what supervisor can do. Process groups/pools, XML-RPC interface for remote management, built-in web interface for process management(utilizing XML-RPC interface), tons of process management options (priority, umask, user/group, capture std* pipes, environment variables, auto restart/start, process naming), event listeners/handling and a simple configuration ta boot! All I’m sayin’ is, it’s awesome…I’m just sayin’.

Keeping with our simple but awesome theme, enter nginx…

My main config:

user www-data;
# Could vary by number of processors available.
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log      /var/log/nginx/access.log;

    sendfile        on;

    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;

    include /etc/nginx/sites/*;

}

and my various site-specific configurations that I keep /etc/nginx/sites:

server {
        # We listen on port 80
        listen 80;
        server_name just-another.net;

        # access and error logs for our site
        access_log /var/log/nginx/my_site_access_log;
        error_log /var/log/nginx/my_site_error_log;

        # Configure redirect for our fastcgi server
        # The fastcgi server later runs on localhost Port 8080
        location / {
                expires 10d;
                # to fastcgi server could use socket..
                #fastcgi_pass unix:{project_location}/log/django.sock;
                fastcgi_pass 127.0.0.1:8080;
                fastcgi_param PATH_INFO $fastcgi_script_name;
                fastcgi_param REQUEST_METHOD $request_method;
                fastcgi_param QUERY_STRING $query_string;
                fastcgi_param CONTENT_TYPE $content_type;
                fastcgi_param CONTENT_LENGTH $content_length;
                fastcgi_param REMOTE_ADDR $remote_addr;
                fastcgi_pass_header Authorization;
                fastcgi_intercept_errors on;
        }

        location /images {
    # For my own images..
            alias /home/bsmith/public_html/images;
            expires 10d;
        }

        # Alias for static content like themes
        location /static {
            alias /home/bsmith/Dev/byteflow/static;
            expires 10d;
        }

        # Alias for python contrib.admin stuff, needed for admin interface
        location /admin-media {
    # Point to my most recent install of django.
            alias /home/bsmith/Dev/django_trunk/django/contrib/admin/media;
            expires 10d;
        }
        # Point to media
        location /media {
            alias /home/bsmith/public_html/media;
            expires 10d;
        }
        # Use feedburner for my feeds.
        rewrite ^/rpc(.*) http://feeds.feedburner.com/Just-anothernetBlogPosts$1 permanent;
}

Pretty simple, eh? I have the same server setup for my ‘dev’ site, only with a volatile code base that I can hack on.

Now when I make code changes all I need to do is reload the supervisor process like so:

$ sudo supervisorctl restart django_fcgi:django_fcgi_0

for the production site or replacing the process name with django_dev_fcgi:django_dev_fcgi_0 to restart the dev site. I could also restart everything by replacing a process name with all as an argument.

The possibilities here for process management endless and I couldn’t be happier with how simple it has become to run what I have.

Basically, this setup is easy to get running and easy to keep running. Highly recommended for any lazy django site maintainer like myself!