Blogging spree

Opster

2009-09-15T17:48:35Z

Two months ago I’ve released little command line parsing library for Python called opster (actually it was called finaloption then, but I’ve renamed it because of remark from native speaker ;-)). What’s the reason to write one more command line parser when Python already has getopt and optparse in standard library and not so long time ago argparse and optfunc were released?

Well… as usually, because I think that they are going wrong way and doing wrong things. Of course, IMO (but what matters if not opinion? :P).

It started to happen when Zed Shaw wrote big article on Python warts and mentioned that Lamson has much better command line parsing solution than argparse/optparse. I was interested in this topic a little at the time and I looked at the code. It would be lie to say that I liked it. In fact I thought that this is a heresy of the same level as optparse. ;-)

I’ve written in Twitter that it’s funny to say that Lamson has superior command system and got some amount of sarcasm from Zed and clear understanding that Zed see nothing bad when:

command functions should be defined in a single module
default settings are defined by calling separate function inside a command function
specifying option in command line with mistake wouldn’t raise error
formatting of help text on options is done by hands

So I thought that world needs Mercurial’s command system. ;) And I’ve rewritten it as library, keeping main idea.

Small example of usage:

from opster import command

@command(usage='[-l HOST] DIR')
def main(dirname,
         listen=('l', 'localhost', 'ip to listen on'),
         port=('p', 8000, 'port to listen on'),
         daemonize=('d', False, 'daemonize process'),
         pid_file=('', '', 'name of file to write process ID to')):
    '''Command with option declaration as keyword arguments

    Otherwise it's the same as previous command
    '''
    print locals()

if __name__ == '__main__':
    main()

I think that you should understand what’s going on here. For example, option is required to have long name (keyword argument name), possibly short name (using '' will discard short name), some default value and help string. Default value determines what should be done to incoming data:

string — nothing happens, incoming value will remain as string
int — incoming value is parsed by calling int()
list — incoming value is appended to the list
function is called with incoming value and output is used
True/False/None — option needs no argument, just switching default value in opposite value

After wrapping with @command your function main() can be called:

without arguments at all; it will parse sys.argv in this case
with argument named argv, which needs to be list of strings (same as sys.argv[1:])
with usual arguments/keyword arguments, which are defined in function

I think it may be not obvious that you will get clean values in your function (for example, port will contain value 8000), and not some strange three-tuples.

And you get such help for free:

piranha@gto ~/dev/misc/opster>./test_opts.py --help
test_opts.py [-l HOST] DIR

Command with option declaration as keyword arguments

    Otherwise it's the same as previous command

options:

 -l --listen     ip to listen on (default: localhost)
 -p --port       port to listen on (default: 8000)
 -d --daemonize  daemonize process
    --pid-file   name of file to write process ID to
 -h --help       show help

Furthermore, underscores in argument names are converted to hyphens to support conventions of command line. ;)

I should mention that option names (and subcommand names, if you’re using them) can be shortened: i.e. you can say --pi instead of --pid-file.

If I’m going to compare opster with something, this should be optfunc by Simon Willison. Most noticeable differences are syntax of command definitions and subcommand support. Actually optfunc has subcommand support, but it’s pretty incomplete.

Opster uses getopt inside to parse options and that’s the reason why it’s somewhat bigger than optfunc (which is essentially optparse wrapper). Opster’s internal API — options are list of four-tuples (short name, long name, default value, help string) — is exactly the same as Mercurial’s API for defining options. This means that such code will work (taken from test_cmd.py):

import opster

config_opts=[('c', 'config', 'webshops.ini', 'config file to use')]

@opster.command(options=config_opts)
def initdb(config):
    """Initialize database"""
    pass

@opster.command(options=config_opts)
def runserver(listen=('l', 'localhost', 'ip to listen on'),
              port=('p', 5000, 'port to listen on'),
              **opts):
    """Run development server"""
    print locals()

if __name__ == '__main__':
    opster.dispatch()

Interesting thing happens in definition of runserver, help and output of which looks like this:

piranha@gto ~/dev/misc/opster> ./test_cmd.py help runs
test_cmd.py runserver [OPTIONS]

Run development server

options:

 -l --listen  ip to listen on (default: localhost)
 -p --port    port to listen on (default: 5000)
 -c --config  config file to use (default: webshops.ini)
 -h --help    display help

piranha@gto ~/dev/misc/opster> ./test_cmd.py runs
{'port': 5000, 'opts': {'config': 'webshops.ini'}, 'listen': 'localhost'}

You can factor out common options and pass them to @command decorator, keeping your pants DRY. ;-)

So… Read documentation and use it! :) Any feedback, questions, suggestions and patches are highly welcome. ;-)

Short urls — rev=canonical

2009-04-12T19:02:41Z

Just yesterday my feed reader fetched Simon Willison’s article about relatively new method of url shortening.

The idea is that url shorteners in fact are not very good thing, especially from URL term of life point of view. What if tinyurl.com is going to die? A lot of URLs will appear unusable, even if their destination is still alive. And discussion gave birth to thought that it could be nice to allow sites to specify their own short urls, even using their own url shorteners.

There is site with specification (if it can be named so), in short — you should specify it in your <head>...</head> section with such link:

<link rev="canonical" href="short-url-here">

In his article Simon tells about how he has implemented rev="canonical" in it’s own blog — I like his solution for not saving anything in database. ;) But hardcoding actual types of content is not that cool, so I’ve done small Django application — revcanonical, whose task is to generate and redirect such short links.

Every url is a two strings joined by a dot — it’s a base62-coded¹ identification numbers of content type and object in database. So you can use that application for any object without any configuration. README has instructions on how to install and use application.

Of course, functionality is already incorporated into Byteflow. ;-)

Digits, big and small latin letters ↩

Mercurial 1.1

2008-12-02T22:49:46Z

Mercurial 1.1 is out and list of changes takes 2.5 pages. There are some my efforts in this release, ;-) but in overall two most appealing features for me is a graph of changeset tree in webinterface and zeroconf extension — it allows hg paths to display all hg serve instances running in a local network. Very useful — now I don’t need to ask and answer about IPs. :)

Of course, there is long-awaited result of Google SoC — rebase and also bookmarks — git-like branches. Sad news is that bookmarks are only local now, but I hope that 1.2 will handle that.

FirePython - no prints?

2008-11-24T20:05:26Z

While I’m developing some web application I almost never use any debuggers or supporting tools: in 90% of cases usual print variable is enough for understanding a trouble. Of course, there are some very complicated cases, when I do import pdb; pdb.set_trace().

But I was pointed at very cool thing yesterday — FirePython. This thing consists of two parts — small library on python and plugin for Firebug. This tandem is engaged with very useful business — it displays all python logged activity¹ in Firebug tab.

This tool is really helpful — now I need only two applications for development: an editor and a browser. Good news is that I don’t need to look for print output in runserver output. Regarding usual usage of logging there are some benefits too — at least you don’t need to follow files and their paths (for creation of which you may not have rights), moreover immediately you have nice viewer for logs with ability to filter them.

I’ve contributed today to development of FirePython by development of two middlewares — one for Django and one for WSGI applications, so now its usage is just a question of few movements. ;) In short: you need to install plugin itself, which depends on Firebug 1.3 (it’s still in beta stage, but I’m using Firefox 3.1, so this is not scary for me :P). After that do easy_install firepython. Another option: clone project from github and put folder firepython from python folder in sys.path or your project directory.

After that you just need to enable middleware by adding its path in MIDDLEWARE_CLASSES: firepython.django.FirePythonDjango. WSGI middleware has another path (how strange! :D): firepython.wsgi.FirePythonWSGI, which you should use as any other WSGI middleware.

Usage of whole system in code looks like:

import logging
logging.debug('what you want to debug today?')

Naturally you can use any level instead of debug — you can filter them in FirePython interface.

¹ —

Precisely saying you will get logs which was logged while FirePython was registered as a handler in logging.

Blogging spree

2008-11-16T18:17:47Z

That’s my first post in this blog. The idea of having blog in English arose a lot of time ago but I always had some lame excuses for myself to not do that. But two weeks ago I’ve added ForeignKey to Site to blog.Post model in Byteflow (which naturally reflects my posts in blog ;-)) along with DynamicSiteMiddleware and that allows me to simultaneously have two blogs on different domain serving by single Byteflow instance, what, I think, is rather cool. :D

So… welcome to exhaust of my blogging spree, hope you will enjoy it! ;-)