Mock It!

Python, long words and the soft hyphen

noreply@blogger.com (malthe) — Thu, 22 Sep 2011 13:33:00 +0000

The HTML specification (any of them) does not require or allow a user agent to manage text hyphenation. Instead, it must rely on hints provided by the document:

The soft hyphen tells the user agent where a line break can occur.

In English, (very) long words are relatively infrequent, and as such, hyphenation is not particularly important. The lack of good hyphenation rarely breaks a page layout.

In German – and in Germanic languages in general – words are long! If you have a page layout with narrow columns (a left-hand side navigation for instance), lines might extend further than you'd like them to.

The following Python function inserts soft hyphens into a text string, guided by a hyphenation dictionary:

If you can't see the script embedded above, you can see the source here.

"Less.app" on linux

noreply@blogger.com (malthe) — Thu, 22 Sep 2011 09:05:00 +0000

From the homepage:

LESS extends CSS with dynamic behavior such as variables, mixins, operations and functions. LESS runs on both the client-side (IE 6+, Webkit, Firefox) and server-side, with Node.js.

In a production environment – unless you have special reasons not to – it's probably best to "cook" your stylesheets on the server-side. It saves browser resources and allows your site to be shown without Javascript enabled.

On the Mac, there's Less.app. The homepage looks a bit like a release announcement from Apple. What it does is to monitor a configurable set of directories for changes and invokes the "less" compiler.

On Linux, the equivalent is a few lines of bash: Note that you must have less.js (via node.js) and inotify-tools installed.

If you can't see the script embedded above, you can see the source here.

Speed up your Plone 4 site with Chameleon

noreply@blogger.com (malthe) — Thu, 14 Jul 2011 07:16:00 +0000

The fast template engine that runs on just about any Python platform version 2.5 and up now runs on Plone 4!

It's already running live in production on Enfold's Ploud infrastructure where they see a consistent 20% or higher increase in performance.

To get Chameleon running on your site, simply add a dependency on five.pt and make sure you get version 2.1. Plone's automatic package configuration does the rest.

eggs =

...

five.pt>=2.1

Support for Zope 2 security and restricted Python is included.

It's not just about speed.

Other good reasons to use Chameleon are:

Easier debugging – cuts out most of the noise from stack traces
Less dependencies – uses only the standard Python library
Flexible parser – supports non-compliant markup and SGML
Works with any framework on almost all Python platforms

Please report any issues on the bug tracker.

Writing a single-source library for Python 2 and 3

noreply@blogger.com (malthe) — Wed, 02 Mar 2011 10:34:00 +0000

The recently released Chameleon template compiler was written as a single source for all Python platforms > 2.5, including Python 3.x (and PyPy).

I'd like to share my experience in taking this approach instead of using the 2to3 tool or a similar technique (read this article by Armin Ronacher to compare).

I started out requiring Python 2.7. This is actually my recommendation if you're contemplating writing a cross-platform compatible library. This release of Python 2 supports most of the features of Python 3 and has syntax-compatibility. And most applications that run on 2.5 will also run without modification on 2.7.

However, in my case, Python 2.5 was a priority. It's somewhat tedious to support this version because the exception handler syntax is incompatible with Python 3. You have to call sys.exc_info() to get the exception instance in every handler. To remedy this, you can try and minimize the use of exception handlers; most of the standard library code offer interfaces that don't raise exceptions. The unit test module is also somewhat outdated and I ended up with a dependency on unittest2.

It's hard to avoid conditional imports and use of various aliases such as:

  try:         fast_string = str         str = unicode         bytes = fast_string     except NameError:         long = int         basestring = str         fast_string = str

Most libraries won't have to care about string encoding to the point where this is necessary, but in the case of a template engine, this stuff matters.

In terms of packaging, I decided to require distribute. I don't know what the support for Python 3 is with setuptools, but my attempts to make it work failed miserably. After adding trove classifiers for all supported versions, my package showed up in the Python 3 compatibility list.

Why not use the conversion tool? Truth is that I didn't care to try this approach because it seemed error prone and complex. I'd rather "dumb down" my library code than have to automatically translated. With any sort of code generation there's going to be questions such as "where is the generated code?" and "how to step-debug it?". With the single source approach, everything's business as usual. There's always a downside in supporting old Python releases in that you can't use all the new standard library code. But the conversion tool can't help there.

Chameleon on Python 2, 3 and PyPy

noreply@blogger.com (malthe) — Tue, 22 Feb 2011 12:53:00 +0000

This is to announce that a new installment of the Chameleon template compiler is available. A single codebase targets an all-new set of platforms: 2.5+, 3.1+ and PyPy! Currently it includes an engine for the Zope Page Templates language. There are no hard library dependencies.

To try it out, install it from PyPi:

  # easy_install Chameleon

Or get a copy from the repository:

  http://github.com/malthe/chameleon

You can report bugs on the issue tracker. Chameleon is the default template engine for the Pyramid web framework, but can be used on just about any system that uses Python. There's also support for Zope and Plone via add-on packages.

HTTP response proxy with Zope 2

noreply@blogger.com (malthe) — Thu, 27 Jan 2011 12:20:00 +0000

You sometimes want to proxy an HTTP object through your site.

If response is your urllib2 response object, then for Zope 2 to actually accept it as a proper downstream response, you need to apply this little trick:

  from zope.interface import alsoProvides
  from ZPublisher.Iterators import IStreamIterator
  alsoProvides(response, IStreamIterator)

Without the interface idiosyncrasy, Zope might have just checked for a read method on the response object and be done with it.

Getting registry settings in Plone to display in fieldsets

noreply@blogger.com (malthe) — Sun, 12 Dec 2010 16:38:00 +0000

Although still a little rough around the edges with regards to supporting less trivial fields such as rich text, Plone's new settings registry framework works really well. It basically lets you point to a Zope schema interface and have the contained fields presented as a control panel form.

I recently wanted to code up a control panel form modelled as a series of fieldsets (displayed with a tabbed interface in Plone). Each of the fieldsets corresponded in my case to its own interface declaration, but potentially I wanted to allow for any sort of nesting.

This proved to be somewhat out of the ordinary for plone.registry. It includes a records proxy class which neatly does the mapping between the interface and the registry. The following gist includes two points of interest in particular: an abstract records proxy class that takes care of this mapping for any type of schema and it also demonstrates the syntax for rendering your interfaces in different form groups – which will ultimately display as form tabs.

http://gist.github.com/738156.

There currently seems to be the issue that error messages that appear inside a tab aren't immediately visible when you post back the form. The user will for the moment have to interpret the generic error message as "please look under each tab to find the concrete error".

Rich text on a property sheet

noreply@blogger.com (malthe) — Thu, 02 Dec 2010 09:48:00 +0000

It's the Zope CMF property-sheet-counterpart to "snakes on a plane". Here's how to do it:

https://gist.github.com/725048

Why would anyone go through all that?

I had a brief e-mail chat with Martin Aspeli where it was established fairly well that certain control panel-like properties go well with the new registry for Plone and others don't go well at all, namely those that are more complex than numbers and strings. Rich text values are such complex properties.

The very same limitations apply for the venerable CMF property sheets. But on the other hand, you've got to put this stuff somewhere and hiding it away in some obscure annotation isn't exactly transparent.

So in a paean to the legacy software that the CMF is, I ended up storing the rich text value in said manner and it isn't all that awful.

Off-topic: On xenophobia or why we're looking for an apartment in Malmö

noreply@blogger.com (malthe) — Sun, 21 Nov 2010 17:44:00 +0000

There's a tide of repressive immigration policies passing through Europe, in particular in Denmark. Very long story cut short, it's difficult and (very) expensive to get to live with your spouse who is a foreign national (outside of the EU). Moreover, it takes many years to get a permanent and secure status. However, there's a loop-hole which circumvents my government's (numerous) attempts to keep out foreign spouses and all it requires is a 10-week stay in another EU-country. After that, you can literally waltz right back into your own country.

I have therefore decided to move to Sweden with my (pregnant!) fiancé for a while and thus ask for help in finding a place to stay, preferably in Malmö. We'd like to move as soon as possible!

Any help is greatly appreciated.

Getting broken objects out of ZODB

noreply@blogger.com (malthe) — Thu, 18 Nov 2010 12:20:00 +0000

Here's a small script that can help you get (broken object) data out of a ZODB database for which you don't have the right software: https://gist.github.com/704910.

As a minimum, you need to know which persistent base class the original object was based on, i.e. persistent object, mapping, list, dict etc. If you're on a Plone site, you'll probably want to use an Archetypes base class.

The technique can be used to migrate an old site to a completely new software platform without bothering to try and install the original software which may be so outdated that even the Python syntax is wrong. Or simply to get your data out of the site.

Translation and Punctuation

noreply@blogger.com (malthe) — Tue, 09 Nov 2010 12:00:00 +0000

The Plone CMS includes gettext translations for a great many languages. For some reason, most translations differ in its punctuation in comparison to the original text - typically the translation is missing a punctuation character. This makes translated text look bad in general, but may also have some side-effects such as duplicate status messages being shown because one includes punctuation while the other does not.

Armed with a script written for the occasion and a checkout of all Plone translations for both current major versions, I have made an effort to make punctuations uniform such that the punctuation used in the original text is reflected in the translation. Excluded from processing is translation files which contain Chinese punctuation (i.e. "。" and "：").

Please visit the changesets for 3.x and 4.x for reference. While I have tried my best to ensure that the processing was correct, it's worthwhile for translators to review - or revert!

You can use this script for your gettext-based translations. I use the following command-line:

  find . -name "*.po" -exec fix-po.py '{}' \;

Enjoy!

On-the-fly image scales with Archetypes

noreply@blogger.com (malthe) — Thu, 04 Nov 2010 14:49:00 +0000

Ever (used Plone and) wanted to stop worrying about whether your site content has proper image scales generated for them? Here's a bobo traverser snippet that'll generate them on-the-fly: image.py.

Drop it in place of the ATImage traverser or use in your own content types.

The mess Django's in

noreply@blogger.com (malthe) — Thu, 29 Apr 2010 10:52:00 +0000

Adam Gomaa is right that (when you don't need to care about the database) – "Django gets the model definition language right".

He continues to say "they're just using the wrong underlying library."

This was back in 2007, but the situation hasn't changed much. It's a lot easier to use the Django ORM than SQLAlchemy for simple tables and relations. It makes all the technical details simply go away and for the target audience of Django, that's probably a good thing. Django gets it just right.

Now if you don't really know how to use Django or if you want to use it slightly differently than how it was intended, and you take a peek under the hood, your impression of Django will change dramatically:

The code is awful.

I understand why people are attracted to Django. It's got a terrific story if you're willing to drink the kool-aid (basically it's Django's way or the high way and you shouldn't make any friends outside the circle of trust). You get a lot of things for free and you don't need to know all the details of all the routine problems.

What I don't understand is how so many people can keep the secret that the code is an outright horror. I sort of knew that it had to be, but I was in for a surprise.

The short story is that shouldn't try to use Django from the inside; it's a pile of spaghetti with insane indirection and module dependencies and you're lucky if you manage to import anything at all. Every other function tries to import modules for you automatically and circular imports are the norm. It's all driven by convenience and convention, but if you're coming with your own expectations about how a system should or could work, you'll face only inconvenience and odd conventions.

Even simple things like setting up and tearing down the database is awkward. You have to actually make an outside call to a command-line script to do it.

Chris McDonough wrote recently about import time side effects. I think Django would be a mental buffer overrun. It kind of is for me, but I have now managed to set up a line of import time side effect defenses that guard against most of Django's idiosyncracies.

Rewrite?

Frameworks in Uganda

noreply@blogger.com (malthe) — Wed, 31 Mar 2010 05:15:00 +0000

It's raining outside and I've still got half an hour to hold out for a window of opportunity. Traffic in this town is as bad as anywhere, but when it rains, it breaks down completely – traveling on two wheels is your only option. Yesterday, my boda-boda driver (the motorcycle taxi) failed to start his machine at first, and since he looked at bit young I asked him how old he was. "How can I know my age?" he replied.

As part of a charm offensive, our office has been tasked to deploy a demo application that runs on Django. It's not WSGI, but it's not terrible either:

$ env/bin/python demo/manage.py runserver 0.0.0.0:8080

This morning I was thinking about how much easier things are when you begin with WebOb, i.e. the raw protocol and a single layer of convenience. You're able to focus on the application environment and without much mental effort you're writing the beginnings of a test suite.

What I like about the BFG and Pylons frameworks is that they use this same basic foundation. You can evolve your application naturally from the just getting started phase to framework support. With Django, there's a ton of processing required to get set up, and it's my intuition that with all this setup, developers get late to the testing game. Test-driven development loses out to cargo-cult template-driven development.

I don't think I'll ever be happy on Django; I'm not even sure I buy into the notion of it being easier for beginners. Easier how? If it's about getting started writing your first spaghetti-application, then it's probably right on the mark. Heck, you don't even have to learn about relational databases. But this is 2010 and there's Scalaris and Redis and Cassandra. If I was just getting into programming, I'd want to use WebOb and one of these, throw in a URL dispatcher such as Routes or Otto.

Django seems like a platform; I think the only platform you really need is the operating system. You might benefit from a framework, but you don't need it. Try and hold out on the framework decision and pull it down on your application only when it's a clear benefit.

Using Chameleon in Django

noreply@blogger.com (malthe) — Tue, 23 Feb 2010 07:50:00 +0000

Using alternative template engines is a popular pastime these days and so I thought I'd give Chameleon a spin on Django.

Below is code for a template loader.

 from django.template.loader import get_template
 from django.template.loaders import filesystem
 from django.template import TemplateDoesNotExist
 from chameleon.core.loader import TemplateLoader
 from chameleon.zpt.language import Parser
 from chameleon.zpt.template import PageTemplateFile

 def lookup(name):
     return get_template(name+".pt")

 class DjangoPageTemplate(PageTemplateFile):
     def render(self, context):
         return super(DjangoPageTemplate, self).render(
             context=context,
             lookup=lookup)

 class Loader(TemplateLoader, filesystem.Loader):
     is_usable = True
     default_parser = Parser()

     def load_template(self, template_name, template_dirs=None):
         if not template_name.endswith('.pt'):
            raise TemplateDoesNotExist(template_name)
         source, path = self.load_template_source(template_name, template_dirs)
         return self.load(path, DjangoPageTemplate), path

It makes two variables available in templates, context and lookup. You can use the lookup-function to retrieve other page templates without rendering them, e.g.

  metal:use-macro="lookup('main_template').macros['master']"

Note: This code was written for Django trunk. Some imports have changed since the last release.

Don't trust the government

noreply@blogger.com (malthe) — Fri, 15 Jan 2010 14:59:00 +0000

This came as a surprise: Everything you do inside a Zope 2/3 (Five) page template is trusted in the Zope 3 sense, and untrusted in the Zope 2 sense. As such, this is nonsensical:

 <span replace="context/@@my-protected-view|default">
  You do not have access to the protected view.
 </span>

Fortunately, it's easy to restore order:

 from Products.Five.browser.pagetemplatefile import ViewPageTemplateFile
 from Products.PageTemplates.Expressions import getEngine

 class UntrustedPageTemplateFile(ViewPageTemplateFile):
     def pt_getEngine(self):
         return getEngine()

This will make sure your view security assertions are actually respected.

Default gateway

noreply@blogger.com (malthe) — Mon, 07 Dec 2009 17:15:00 +0000

An HTTP response frequently contains hyperlinks to resources available through HTTP which are somehow internal the system. Examples include documents linking to other documents or stylesheets linking to images. It is not always the case that these resources are available from the same server. Static files are often served using a content delivery network.

Most web frameworks use the incoming request to generate URLs to other resources.

This works because the request contains various metadata about the HTTP request including which host it arrived to. In setups that involve proxies, it is often necessary to provide additional metadata (e.g. "x-forwarded-for") to make it work.

The system works well for simple web applications. All resources stem from the same server, in our case we can imagine some WSGI application written in Python. We can deduce the application host URL from the request metadata (known as the environment) and the rest is internal routing (using Routes or another library that maps URLs to code or files).

Whether the request is passed around throughout the code or made implicit through the use of thread-locals, the actual call signature for generating a URL is (resource, request). I think this pattern is wrong. We need to explicitly identify the gateways that can serve up the resources internal to the system (with possible shades of gray in what's internal and remote). In a simple setup such a gateway may be created per-request from the request, but the abstraction should still hold.

The call signature then becomes (resource, gateway). Note that a local consumer would provide the equivalent of localhost and retrieve a URL that specifies the file protocol. For remote users there may be several possible gateways which we can prioritize based on accessibility (e.g. physical location) or availability (e.g. permission). An actual implementation might want to use (resource, service) where service represents some application mount point or URL spelling that is available on a particular gateway.

To relate this to the WSGI protocol, applications will want to extract the following out of the HTTP request and do away with it:

Path
Payload
Remote user (and/or other session- or cookie data)
Metadata (e.g. language- or accept data)

We can call this set of data the application-specific request. Any such request should map to a set of gateways which may or may not be able to service some future request (keep in mind the goal is to produce the URL for some would-be request).

Note that such a pattern would also help in writing tests. You do not need to produce a fake request in order to have a specific component render a response. Instead you need to define the "network reach" of your prospective client.

Some people have the opinion that none of this matters much. In turn we often see weak abstractions which are hard to explain, but work because it's Python. The advent of Python 3 brings an opportunity to rethink our frameworks and libraries (because none of them work). Good abstractions help to keep things simple even if it is non-trivial to conceive of them. They're on honking good idea and I think we should have more of them.

Don't look back in anger

noreply@blogger.com (malthe) — Sun, 22 Nov 2009 21:08:00 +0000

Next Thursday it will be a year since the release of Python 3. You've got to be out early in Christmas time which is my excuse for the untimeliness of this rant.

I thought I'd try to see what the fuss was about, i.e. does anything work? WSGI? It doesn't. While very informative, the roadmap for the WSGI specification is also very unreadable. Let's see if we can extract some knowledge out of it!

It seems to me that people sit with a copy of Essential Python 2 and try to see how to interpret the WSGI specification against an imaginary Essential Python 3, e.g. "In Python 2 this was a string; what should–right, so bytes. Next, in Python 2 this was..." – to no end. Let's cut to the chase:

Rule 1: Strings should be strings, chunks should be bytes.

The roadmap then mentions that on Jython, sockets will return unicode. That's wrong, so that can be conveniently filed as a bug. It continues with another string prediction: "The status line specified by the WSGI application must be a byte string." Why must this be a byte string? It is obviously an actual string. I think it's even a Freudian slip to call it a "byte string". Please see rule 1.

Then it gets a little unclear, but skipping ahead a handful of paragraphs, we come to this bullet point: "For the WSGI variable 'wsgi.url_scheme' contained in the WSGI environment, the value of the variable should be a native string." I think this is what the rest of the world calls the server protocol. If I can make a humble suggestion:

Rule 2: Please remove all proprietary keys from the HTTP environment.

Incidentally, the presence of these keys is probably what inspires developers to put all sorts of other proprietary keys in the HTTP environment. I have previously argued why this is wrong. It occurs to me that when the WSGI specification says environ, it means request. What the application needs is the request which holds the HTTP environment and the content body. Everybody uses the term request for this; an HTTP request. It's convenient to split a request up in an environment, which is the first few lines of the socket input and the body, which is the rest. This body might be quite large, so the interface must be an iterable of binary chunks.

Note that if we had a request object instead of this augmented environment, middlewares and frameworks that wanted to add touches of magic could wrap it or even change its class. Today they have to insert special string keys into a dictionary and explain how this isn't that wrong in their documentation (e.g. "it's better than a thread-local").

Okay, this has turned sour. All I wanted was to try out Python 3. I'm not sure the current approach is going to make this any fun. The 2to3 solution is not appealing. It's a bit like an in-law staying temporarily in the house. When does it end? When is the try-import-except dance over? Am I ever going to import anything from the functools module and feel comfortable about it? Here's a moratorium for you:

Rule 3: Forget Python 2.

From now on it's on the paying clients only list. Join the movement of full incompatibility and never look back! The downside is that we have to rewrite everything. I think it's worth it. Microsoft should have done it, Apple did it.

Who got the better uptake?

Scheduling music using Spotlight and Python

noreply@blogger.com (malthe) — Wed, 18 Nov 2009 17:24:00 +0000

Back in the last century, every decent music player had the playlist at its center. People were religious about it because it wasn't just a practical device for music listening, it was your musical identity. Today most people are trying to come to terms with their 40+ gigabyte collection and need a musical genie to manage it.

Python to the rescue.

Actually, the problem with Python is that the standard library is too rich. You end up writing a small multithreaded application complete with a command-line interface, when you merely set out to do something else for half an hour.

My favorite music player on the Mac is the small application Vox. It doesn't do anything, but it plays all formats and has an equalizer. For any given music track, I needed to know its duration, because this application wasn't going to collaborate with any notifications. I decided to try out my operating system's built-in indexing service, Spotlight. Turns out that I just needed to install some importers and then all my music was indexed. The duration was available from the query metadata.

The resulting Python script allows you to take a list of song queries (I use "artist - title" which seems to work fine), save it to a plain text file and schedule it. There is a command-line interface which lets you skip and repeat songs.

Source is available from the repository (as-is, BSD).

Emperor's new clothes

noreply@blogger.com (malthe) — Thu, 05 Nov 2009 23:29:00 +0000

The Zope Component Architecture is often touted for letting you override so-called components which have already been registered elsewhere (by a library, out of your immediate control).

It's wrong (edit: to clarify, using the overriding facility of the ZCA is wrong, and by extension, to use the ZCA because you can override components, is wrong too; I do not mean to suggest that the use of ZCA is misguided in general).

Shrink-wrap software is a circumstance. If you must change it from outside (instead of branching, which is a fine approach in many cases), just go ahead and update the functions or classes in question using the reload approach (object identity changes should be avoided at all costs).

There's nothing wrong with ZCA; it's in my view a misunderstanding to employ it for web application development at large. Rather, it's applicable for deployments that require live software component introspection. I don't know that web applications need this in general. What happens is that frameworks like Zope Toolkit and Plone use these introspection features to map requests to code. That's a bit like using RDIF tags for consumption tracking (hint: require credit card payment or identity-based rebate instead).

Most software is misunderstood. How to use it, when to use it. Sometimes it's worth the while to retrace your steps to learn why you ended up using a particular software.

New in school

noreply@blogger.com (malthe) — Tue, 03 Nov 2009 21:11:00 +0000

I've written an HTTP publisher!

The most common reaction is: why write a new framework? Well, it's not a framework. It's a little device that sits between the server and your code. It does more than bobo, less than bottle. What it does, it does better than both.

The name is Otto.

There aren't many moving parts, but the design is flexible enough to handle most use-cases. The syntax is agreeable, terse and very clear. Most importantly, it gets you going without any mental effort. There is no inversion of control. It is obvious at every step what happens, from request to response.

The design has evolved entirely through examples, setting the library requirements instead of the other way around. The documentation is tested continuously using the manuel package which keeps everything in sync and makes it possible to write out and test complete self-contained examples right in the documentation.

That said, there isn't a lot to it.

You can use the publisher directly as the basis of a web application, or integrate it into an existing framework. Suggestions or other feedback are welcome as comments are on the repository tracker.

The publisher will grow a few more features (e.g. support for virtual hosting, static file helper) before its first release, but an effort is made to keep scope narrow.

The cure

noreply@blogger.com (malthe) — Thu, 15 Oct 2009 22:53:00 +0000

A couple of years ago I wrote a small package called z3c.jbot which made life as an integrator on Zope and Plone a lot easier. This is an advertisement for the latest version 0.5.

New features:

No performance hit
It works with Chameleon
Replace images, stylesheets and javascripts too!

With this release you can actually fully customize an out-of-box Plone site without writing a single line of code; replace the site logo, customize content templates and portlets, or change the included stylesheets.

More information on the project page.

Copying music left or right

noreply@blogger.com (malthe) — Sat, 03 Oct 2009 10:24:00 +0000

I pirate music. Frequently. Most people do, too. It's not right in terms of copyright; so let's talk about copyleft.

When we talk about music piracy, we usually think of it as a free lunch. But we've been told many times over that there's no such thing. Money don't grow on trees; there are no free lunches. Won't the world stop turning if we give up copyright?

'Lo and behold free software.

I write free software, made available as is, under the BSD license. My only source of income is the hours I clock from clients. The clients are interested because I write free software.

The answer to music piracy is free music, as in copyleft. Everybody can hum, sing, dance and play along to music. To copyright music and sell it under a non-free license is artificial. It's no different than software, books or thoughts. Need a source of income? Work. Perform your music, or other people's music, sell it to advertisers (god forbid), sell merchandise to teenagers looking for an identity.

There's just one problem. Record labels aren't interested; they are afraid that free music will hurt their business model. The irony is that it probably won't. They can still issue compact discs and charge for them. We have recently seen that the LP has grown popular again. There is money to be made from selling free music goods. I think there is money to be made from printing and selling free books, too (in fact, many older books are free; they are still being printed and sold for a profit).

I think my music habits will stay illegal for the better part of this century. Things won't change. But when people complain "but how can you make yourself do it, knowing that the artists are starving from lack of income" my answer is:

I don't starve.

Picking the right nose

noreply@blogger.com (malthe) — Mon, 28 Sep 2009 15:54:00 +0000

Like or dislike, doctests formatted as restructured text (e.g. the venerable README.txt that resides in many packages) are a convenient way to document your software and make sure the documentation stays in sync with the actual codebase.

It's surprisingly awkward to run these tests using nose. After some debugging of the unittest module, the following is the workaround I've resorted to employ.

import unittest
import doctest

class MyTestCase(unittest.TestCase):
  def __new__(self, test):
    return getattr(self, test)()

  @classmethod
  def test_readme(cls):
    return doctest.DocFileSuite('README.txt')

The documentation for nose suggests using the ``--doctest-extension`` command-line flag to use non-Python files as doctests, but it seems much more convenient to define them in the actual test module.

A new object database for Python

noreply@blogger.com (malthe) — Sat, 26 Sep 2009 12:31:00 +0000

When Chris McDonough presents his BFG framework he uses the punchline "like Zope, but less". I'll take the liberty to reuse it about a new object database that I've written over the summer: "like ZODB, but less". It's called dobbin, which is a reuse from another project, incidentally also an object database.

Less code. About one twentieth. Of course, less features, too. And easier assumptions.

I'll be the first to admit that I felt caught in the act when I first read the so-called architect notes of the varnish proxy server. The author of that software claims most developers are still writing software like it was 1975. If you read that document and consider the memory management attempts of the ZODB, you might get the feeling that he's right.

Dobbin is the logical consequence of that document. It doesn't try to manage its own memory.

Less overhead. Persistent objects are read-only, until checked out. Their state is shared between threads. This significantly reduces memory usage; in fact, memory usage has always been the Achilles' heel of the object database. This reduction comes with an inconvenience: objects must always be explicitly checked out before they can be changed.

There's no inherent network support, but multiple processes can connect to the same database without configuration. There's no server process; the database is simply a file.

Less complexity. Dobbin is written all in Python.

The project page on PyPi has more information including a user's guide. It's not quite ready for serious usage yet. If you're interested in helping out, these are the next steps:

Packing (trimming away old versions from the transaction log)
Indexing (field-indexing and querying)
Persistent dictionaries and lists (buckets or similar)

To contribute code to the project you must sign the contributor agreement for the repoze repository.