Mock It!

Emperor's new clothes

noreply@blogger.com (malthe) — Thu, 05 Nov 2009 23:29:00 +0000

The Zope Component Architecture is often touted for letting you override so-called components which have already been registered elsewhere (by a library, out of your immediate control).

It's wrong (edit: to clarify, using the overriding facility of the ZCA is wrong, and by extension, to use the ZCA because you can override components, is wrong too; I do not mean to suggest that the use of ZCA is misguided in general).

Shrink-wrap software is a circumstance. If you must change it from outside (instead of branching, which is a fine approach in many cases), just go ahead and update the functions or classes in question using the reload approach (object identity changes should be avoided at all costs).

There's nothing wrong with ZCA; it's in my view a misunderstanding to employ it for web application development at large. Rather, it's applicable for deployments that require live software component introspection. I don't know that web applications need this in general. What happens is that frameworks like Zope Toolkit and Plone use these introspection features to map requests to code. That's a bit like using RDIF tags for consumption tracking (hint: require credit card payment or identity-based rebate instead).

Most software is misunderstood. How to use it, when to use it. Sometimes it's worth the while to retrace your steps to learn why you ended up using a particular software.

New in school

noreply@blogger.com (malthe) — Tue, 03 Nov 2009 21:11:00 +0000

I've written an HTTP publisher!

The most common reaction is: why write a new framework? Well, it's not a framework. It's a little device that sits between the server and your code. It does more than bobo, less than bottle. What it does, it does better than both.

The name is Otto.

There aren't many moving parts, but the design is flexible enough to handle most use-cases. The syntax is agreeable, terse and very clear. Most importantly, it gets you going without any mental effort. There is no inversion of control. It is obvious at every step what happens, from request to response.

The design has evolved entirely through examples, setting the library requirements instead of the other way around. The documentation is tested continuously using the manuel package which keeps everything in sync and makes it possible to write out and test complete self-contained examples right in the documentation.

That said, there isn't a lot to it.

You can use the publisher directly as the basis of a web application, or integrate it into an existing framework. Suggestions or other feedback are welcome as comments are on the repository tracker.

The publisher will grow a few more features (e.g. support for virtual hosting, static file helper) before its first release, but an effort is made to keep scope narrow.

The cure

noreply@blogger.com (malthe) — Thu, 15 Oct 2009 22:53:00 +0000

A couple of years ago I wrote a small package called z3c.jbot which made life as an integrator on Zope and Plone a lot easier. This is an advertisement for the latest version 0.5.

New features:

No performance hit
It works with Chameleon
Replace images, stylesheets and javascripts too!

With this release you can actually fully customize an out-of-box Plone site without writing a single line of code; replace the site logo, customize content templates and portlets, or change the included stylesheets.

More information on the project page.

Copying music left or right

noreply@blogger.com (malthe) — Sat, 03 Oct 2009 10:24:00 +0000

I pirate music. Frequently. Most people do, too. It's not right in terms of copyright; so let's talk about copyleft.

When we talk about music piracy, we usually think of it as a free lunch. But we've been told many times over that there's no such thing. Money don't grow on trees; there are no free lunches. Won't the world stop turning if we give up copyright?

'Lo and behold free software.

I write free software, made available as is, under the BSD license. My only source of income is the hours I clock from clients. The clients are interested because I write free software.

The answer to music piracy is free music, as in copyleft. Everybody can hum, sing, dance and play along to music. To copyright music and sell it under a non-free license is artificial. It's no different than software, books or thoughts. Need a source of income? Work. Perform your music, or other people's music, sell it to advertisers (god forbid), sell merchandise to teenagers looking for an identity.

There's just one problem. Record labels aren't interested; they are afraid that free music will hurt their business model. The irony is that it probably won't. They can still issue compact discs and charge for them. We have recently seen that the LP has grown popular again. There is money to be made from selling free music goods. I think there is money to be made from printing and selling free books, too (in fact, many older books are free; they are still being printed and sold for a profit).

I think my music habits will stay illegal for the better part of this century. Things won't change. But when people complain "but how can you make yourself do it, knowing that the artists are starving from lack of income" my answer is:

I don't starve.

Picking the right nose

noreply@blogger.com (malthe) — Mon, 28 Sep 2009 15:54:00 +0000

Like or dislike, doctests formatted as restructured text (e.g. the venerable README.txt that resides in many packages) are a convenient way to document your software and make sure the documentation stays in sync with the actual codebase.

It's surprisingly awkward to run these tests using nose. After some debugging of the unittest module, the following is the workaround I've resorted to employ.

import unittest
import doctest

class MyTestCase(unittest.TestCase):
  def __new__(self, test):
    return getattr(self, test)()

  @classmethod
  def test_readme(cls):
    return doctest.DocFileSuite('README.txt')

The documentation for nose suggests using the ``--doctest-extension`` command-line flag to use non-Python files as doctests, but it seems much more convenient to define them in the actual test module.

A new object database for Python

noreply@blogger.com (malthe) — Sat, 26 Sep 2009 12:31:00 +0000

When Chris McDonough presents his BFG framework he uses the punchline "like Zope, but less". I'll take the liberty to reuse it about a new object database that I've written over the summer: "like ZODB, but less". It's called dobbin, which is a reuse from another project, incidentally also an object database.

Less code. About one twentieth. Of course, less features, too. And easier assumptions.

I'll be the first to admit that I felt caught in the act when I first read the so-called architect notes of the varnish proxy server. The author of that software claims most developers are still writing software like it was 1975. If you read that document and consider the memory management attempts of the ZODB, you might get the feeling that he's right.

Dobbin is the logical consequence of that document. It doesn't try to manage its own memory.

Less overhead. Persistent objects are read-only, until checked out. Their state is shared between threads. This significantly reduces memory usage; in fact, memory usage has always been the Achilles' heel of the object database. This reduction comes with an inconvenience: objects must always be explicitly checked out before they can be changed.

There's no inherent network support, but multiple processes can connect to the same database without configuration. There's no server process; the database is simply a file.

Less complexity. Dobbin is written all in Python.

The project page on PyPi has more information including a user's guide. It's not quite ready for serious usage yet. If you're interested in helping out, these are the next steps:

Packing (trimming away old versions from the transaction log)
Indexing (field-indexing and querying)
Persistent dictionaries and lists (buckets or similar)

To contribute code to the project you must sign the contributor agreement for the repoze repository.

Where to eat in Porto

noreply@blogger.com (malthe) — Wed, 16 Sep 2009 15:03:00 +0000

It's just around the corner; you sit where you like. The waiter (empregado) will quickly bring bread, olive oil and vinegar, a tray of olives, mixed salad and two charcoal-grilled sausages (chouriço), then insist on a bottle of vinho verde, green wine. He will then take the actual order. The people known as os tripeiros are hedonists; this is at midday.

I usually ask for the picanha, which is an exquisite red meat cut from the loins. It's prepared on the grill and always served rare. Served with rice, black beans and cabbage (usually kale, prepared like the East-African sukuma wiki, quite bitter), it's not exactly an incentive to get back to work. The wine doesn't help here.

After the meal the waiter will bring the sobremesa. It's impolite to refuse. Then coffee.

The restaurant goes by the name of central dos leões, which is only apt. You needn't actually try and find any other restaurant in Porto; it's the best there is and you'll find that they open before you even wake up and serve the last picanha long after it's bedtime for Frances.

Singing and branching

noreply@blogger.com (malthe) — Mon, 31 Aug 2009 19:19:00 +0000

Let me try and say this in more than 140 characters.

Inevitably with shrink-wrap software like Plone there's a need to customize behavior. With Plone one of the first things you'll want to change is the look and feel of the retail site. Different packages give different answers to how this is best done; some give several. There are inherent limitations.

Here's one that'll work for any piece of software that is shrink-wrapped from a version control system: branch and change the software itself.

I'll argue that it's as prudent or better than any of the official customization stories. Upgrades are straight-forward, just merge the desired changesets into your branch; if there are conflicts, chances are that the burden equals that of the official story. Besides, upgrades across major revisions are rare.

Python is an interpreted language; you can actually change the source code, restart the process and see your changes take immediate effect. It's your project; think about what's right for you, not what's more clinical.

Hungry for memory

noreply@blogger.com (malthe) — Mon, 27 Jul 2009 10:58:00 +0000

The ZODB object database is characteristically hungry for memory. You can almost always trade memory for performance and it's entirely unfair to say that ZODB went all-in.

All objects are mutable at any time with changes persisted automatically at the end of the transaction. This holds for each thread to the effect that memory usage is often linear on the number of threads (data consistency during a transaction calls for strict separation of mutable data). It only adds to injury that Python has inherent difficulties with memory consumption over time.

Must we be multi-threaded? Unless we're CPU bound then yes, it's the only way to increase capacity.

If instead objects weren't mutable by default, threads could share them. When a program wanted to change an object, it would simply ask ZODB to provide a mutable instance. At the end of the transaction this instance would become the new shared object.

This would not challenge the benefits of using an object database, nor do I expect it to cost much in terms of performance. For a database that's optimized for reading, the convenience of being able to change objects without preparation is a non-goal.

To limit memory usage in general, we can configure the amount of objects ZODB keeps in memory (per thread). If this number of exceeded, ZODB automatically dereferences the objects least in use, an apparatus known as the object cache. Poul-Henning Kamp calls this "1975 programming" in his notes for the architecture of Varnish, the HTTP caching proxy. In his own words, this is how Varnish does it:

Varnish allocate some virtual memory, it tells the operating system to back this memory with space from a disk file. When it needs to send the object to a client, it simply refers to that piece of virtual memory and leaves the rest to the kernel.

Should ZODB manage its own memory consumption? Kamp would argue that it should instead let the operating system swap out objects not in active use.

The operation system must of course spend time paging where the ZODB simply dereferences objects, freeing them from memory. However, when objects need to be made available again, ZODB must unpickle them, arguably much more expensive than paging.

That said, the operating system might not be in a position to swap out the objects least in use; Python has its own memory allocation system that operates on a program-level.

Read-only objects and virtual memory–both, either or neither?

Relay chat

noreply@blogger.com (malthe) — Thu, 23 Jul 2009 14:59:00 +0000

<hydra__> Ergo^: do you use decorator.decorator to write decorators?

Fitter, happier and more productive

noreply@blogger.com (malthe) — Wed, 22 Jul 2009 12:22:00 +0000

I'm usually unhappy working.

Here's a short account of the remedies I've attempted; they are all peripherals that you can connect to your computer.

The first peripheral I acquired was a USB telephone handset. It has no buttons and does not integrate with anything; instead it provides a microphone and speaker device. I am very happy with this device. You need to physically hold it with one hand when you talk, like you would a normal telephone.

Then I bought a Kinesis Advantage keyboard. It was very difficult for me to learn how to use it; the first two weeks were awkward, progress–slow. Not unlike learning to play an instrument. Or wearing glasses. It has built-in key remapping with a memory of its own. It's a wonderful keyboard.

Mouse: Logitech Nano. I'm not a serious mouse-user by any measure, but if you're going to take someone's word for a mouse, this is one good mouse.

The next peripheral is my headphones, a pair of Grado SR 225. It's an open ear cup headphone which means that it doesn't try to silence out background noise. Good for sound reproduction quality, but you can't opt out of office chattering–open social, right? If you are in Europe, it's possible to buy them from a company called "The Sound Room" for the US-market price. Note that there is a new improved version available under the name SR 225i.

I listen to music to rest my mind every now and then while working, but the paper mache sound of the built-in soundcard doesn't resonate well with me. My laptop has optical audio output and I connect it to a Tianyun Zero D/A converter which has built-in headphone amplification. It makes a world of a difference. The 2009 edition even has USB input. It sells on ebay for $118 (shipped from Hong Kong)–robbery at broad daylight.

Not exactly a peripheral, but easily the most essential ingredient in my setup: emacs. I use it for shell work, notes, editing and internet relay chat, everything that involves typing. It took me some years before I understood how to use this editor and why. If you're on a mac, the carbon emacs package seems to be the better build. Dot emacs configuration on request.

It's all wrong

noreply@blogger.com (malthe) — Wed, 15 Jul 2009 11:55:00 +0000

It all started with an innocent post to the Python web development mailinglist: "Fun with WSGI - commenting middleware". It sounds harmless, but the idea had a viral potential. Middleware was to be the building blocks of web applications, much like LEGO. Not convinced? This was written in a blog post only two months ago:

"This way, Pylons middleware components could be discovered and loaded as they are installed on the system."

It's plug and play and it works out of the box. Want users to be able to rate content? Plug in the right middleware. Want it to share a database with the application? Use the database sharing middleware.

Now, some readers will say, "is it really that simple?" Yes. Is it good software design? No.

That's a fairly strong statement, but I don't think I will regret it.

WSGI middleware relies on a protocol in which you're given a request and an application object (which might itself be middleware) and in return, you're expected to provide a response. That's it–there's exactly nothing more to it.

How is it possible to share a database across these middlewares (or with the application since we can't know which it is)? Well, it's Python, so you just might insert the database connection object into the HTTP environment (aptly termed WSGI environment to keep up appearances). Another option is to keep it in a thread-local global–afterall, WSGI applications run in their own thread.

Is it abuse? You tell me. But wait, there's more.

Modular is good. You don't need much experience in software development to realize that. That's why it feels attractive to have the commenting apparatus in a separate module and plug it in "from the outside". Let's have a closer look at the fun commenting middleware:

The user interface for viewing and posting comments is typically presented at the end of an article. It uses simple GET and POST requests, perhaps coupled with user authentication and/or captcha. Disregarding Ajax-functionality, styling can be left to the host application. For storage we can connect to a SQL database.

Fairly self-contained, but not really.

Can we do any better? We could require that POST requests come from human beings with valiant intentions. Such users are commonly allowed to post a comment. A middleware sitting in front of the commenting apparatus could safeguard POST requests by looking at the remote user header (a successful captcha challenge would put a system string here). This header could be set by an authentication middleware (configured with the right cipher to validate an authentication cookie).

It would be difficult to get to such attributes as full name (certainly some developers will be itching to introduce a remote user full name key into the WSGI environment to work around this) but software is compromise.

To follow up on the authentication middleware story, it could in theory be responsible for setting the authentication cookie on egress, if it found that a remote user header had been set in the environment by a downstream application. The signal to clear the cookie could be the non-existance of this header. Middleware is not wrong in general.

What's wrong is the abuse of the WSGI specification to sneak around Python-objects because your middleware really isn't middleware–it's part of the application!

Abuse is like a casino; the house always wins.

The commenting middleware is wrong in a different way, too. It's a small application, but web applications share many of the same moving parts. Authentication, authorization, sessions, language negotiation, routing–these are just examples. You can't split them out to middleware because it violates the WSGI contract. The result is that every miniature application in the pipeline needs its own set of moving parts. Never mind that it's painful to merge HTML from two applications.

The promise of the semantic web might be the way out of this situation. Until it materializes we need to embrace integration and find the right abstractions to make modular programming possible.

The curry diet

noreply@blogger.com (malthe) — Tue, 02 Jun 2009 04:32:00 +0000

This might just work. That's really the best I can come up with to describe an odd little library that I've secretly been working on. It's called curry, because it's both cooking and currying.

Also referred to as partial application, currying is the transformation of a function of n variables into a chain of n functions of a single variable. That seemed a bit too mathematical for me to be of any practical use in Python, so my starting point is a bound method. The idea then is to apply partial application to this method to shift off the class instance pointer, partially evaluating the function body (recursively). This is actually a means of optimization and the motivation is to try and use it instead of code generation in libraries like chameleon. Rather than generating code, the idea is to reduce existing, working, tested code to obtain the exact same result.

There are more technical details available on the project page, while examples are included in the test suite.

Activist appears on a tortilla in mexico god

noreply@blogger.com (malthe) — Sat, 16 May 2009 15:02:00 +0000

Yesterday, while attending a poetry slam session in a Nairobi suburb, it dawned upon me that a central theme in the Christian belief goes against my morality. I find it interesting, because non-believers are often lead to believe that their morals are inherited from Christian doctrine. Appearently, not so. I suppose it's a relief to me, which is why I feel like sharing. An apology for posting this here is due being wanton off-topic, but rest assured that the word "science" does appear. For the record, the title of this piece was inspired by a friend who said, "I honestly don't even think I believe in God more than like some levelling force of good in the universe, I don't believe in this activist appears on a tortilla in mexico god". I'm not personally seeing much of this levelling force and not very compelled to spirituality in general, but the rest of the remark does make for a good title.

Spirituality is strong in Kenya and most of the poems recited in the session carried religious messages. I took note of one piece in particular, which had a female poet ask her would-be suitor if "He was your king, your leader" (he would then ask her the same). This lead me to think about the concept of judgment, e.g. the notion that after we pass away, divine judgment awaits (protecting the kingdom of heaven). Another poet introduced his piece by reminding the audience that "scientists should stick to formulas". While a fair proposition, I hope it does not exclude me from employing logic, because I will need it in the following.

You'll recall that in Christian doctrine, Purgatory is the station just before Heaven and Hell (there's a notion of being in "limbo" as well, but that's irrelevant here). I've always thought that eternal damnation was an odd concept, but I've been lead to believe that it's a scarecrow idea that will never materialize (surely if it did, it would undermine the entire foundation of Christian morality, being such as cruel punishment). As such, we all go to Heaven eventually. What's important is that we don't know this while down here and so there's still this judgment awaiting us which could prove detrimental.

I call this morality of Purgatory "The Carrot or the Chariot", inspired by Malcolm X's speech "The Ballot or The Bullet", which in 1964 warns the United States government that if African-Americans are not given full equality, armed struggle will ensue. Mine is a pun on the obvious flaw of using Heaven as a carrot, while Hell is the actual destination should anything go wrong. The opposite of a carrot must be neutral, else it's no carrot.

We must believe in reform and morality while down here and we cannot instill this using fear for divine judgment. The carrot is the joy of living a moral life with all in which entails of honesty, modesty, humility, and so forth. With the morality of divine judgment comes reasoning that leads us to such cruelties as capital punishment (the secular counterpart to eternal damnation).

Further, if Purgatory is merely a scarecrow institution, we must question whether Heaven is even possible. Can such a state of being truly be different from that which we experience here below? The same petty thiefs, thugs and defilers will dwell there, too, demanding justice.

To observe this belief of happiness in living a life in morality and accepting happiness as a carrot is thus to reject the existance of Purgatory, Heaven and Hell, all of which are central to Christian morality.

Pesky operands

noreply@blogger.com (malthe) — Mon, 11 May 2009 16:16:00 +0000

Unlike ZPT, Chameleon relies on the venerable expat parser. It's not forgiving in the slightest and won't parse a "<" character that isn't part of a tag (as often used in an inline Javascript for-loop). Often times we can't make assumptions on the document mode, which rules out CDATA. We're not spoilt for choice. Then I stumbled upon a clever workaround: Reverse the operands and invert the operator, then it just works.

Forms that don't make you tense and nervous

noreply@blogger.com (malthe) — Thu, 07 May 2009 14:20:00 +0000

Forms are tedious. I used to think that some five thousand lines of code was the remedy [1]. All the pesky details are abstracted away and in their stead is a very reasonable model based on schemas, widgets, converters, add forms, edit forms, display forms, page forms, subforms–rest assured, there's an option that fits you.

Invariably, with any framework you'll need to adjust things a bit. As it happens, this is where it gets tedious with the framework that is supposed to remedy tediousness. Endless abstraction and delegation makes it difficult to plug in custom behavior, let alone come to terms with what needs to be plugged out and how to write the code to replace it.

Typesetting documents in Latex is similar; you'll spend hours trying to align four graphs on a page, because "letting the system do it for you" doesn't get you where you want. The better you're acquainted with the system, headaches will come fewer and farther between–the learning curve pays off. Yet, arguably not all documents are best typeset in Latex.

Where's the restructured text of forms? There's Ian Bicking's FormEncode. It does some of the things I want and it does a lot of things I don't want. It also does the things I do want in a way I don't quite like.

Sanctioned by the "scratch your own itch" approach to software development (if it works for you, chances are it'll work for others), Stefan Eletzhofer and I decided to write a minimalistic form library called repoze.formapi. Part of the library was actually written with Ian present, while Jeroen Vloothuis wrote most of the error handling and validation logic. The result is a very usable library which intentionally does very little. It provides powerful field marshalling, flexible validation and a minimalistic form class framework. You write your own HTML.

Note: Narrative documentation for the library is now available.

[1] z3c.form

Book review: Practical Plone 3

noreply@blogger.com (malthe) — Wed, 15 Apr 2009 12:02:00 +0000

Let me first make the observation that printed books are here to stay; I received an electronic copy of the latest Plone book from the publisher asking for a review and it's been looking at me from its place on the desktop for about two months now. Today I received an e-mail from the publisher asking for the location of the review, so I'd better get on with it.

I haven't bought or read a technical book for about five years. The last one was Philipp's book on developing Zope applications using the component architecture. I thought that was quite a good book and certainly one that I'll revisit from time to time (okay, less and less so, but it's had a very decent life span). Why is it a good book? I think the primary reason is that it knows its audience and stays with it throughout the book. There's a learning curve, but not an audience curve.

I think the average reader will take one year to read this book (sic).

When I read through the initial chapters, I got the odd feeling that it's got an audience curve instead of a learning curve. In the first few chapters we learn what Plone looks like and we're introduced to some new vocabulary like breadcrumbs and colophon. That doesn't seem very practical; it seems introductory. To spend some fifty pages going through the main user interface may not seem like an issue (you can easily skip them), but skipping pages is something you do on Google, not when reading. In other words, skip the extended introduction–it's too likely that it will be anyway.

Parts 2 and 3 (starting on page 55) make up the rest of the book. Part 2 is essentially a user's guide (how to use Plone) while part 3 is an integrator's guide (how to extend and customize Plone). It's good writing and the material matches expectations. However, we can't get around the fact that this book addresses three different audience in three different parts. It'll take a year to read the book and by then we'll have Plone 4.

Perhaps 500+ pages isn't always the right balance for a technical book; in the case of Plone, it seems that 50/250/250 would be better, distributed in three separate volumes, like a trilogy. Perhaps 50/250/500/500, where the missing part 4 would deal with how to develop enterprise Plone apps, but luckily, this is what Martin's book is all about.

Is there a conclusion to these ramblings? I'm not sure, but knowing your audience, staying with them and appreciating the quick read are valuable points I think. As for a recommendation, I'll say that this book belongs in any office where Plone is being used, but perhaps not in every home. I look forward to the revised second edition, coming out in three separate volumes.

Time-out in Africa

noreply@blogger.com (malthe) — Fri, 27 Mar 2009 11:28:00 +0000

Yesterday, while listening to "White Car in Germany" by The Associates, I kept getting an error while running buildout:

Error: MD5 validation failed for plone.app.locales-3.2.0.zip;

"What the hell", I thought and went step-debugging into the code. First an observation: I would not expect a ``check_md5`` method to actually go ahead and unlink the downloaded file if validation fails. That aside, the error persisted and it turns out that ``urllib2`` silently times out a request if the socket does not receive data within some period of time (this happens quite often here).

The documentation suggests that it will raise an exception, but perhaps the clue is that if no time-out value has been set, it's really not a time-out (so why raise an exception). At least, setting it (at a reasonable location in the buildout-script) resolves the issue:

import socket
socket.setdefaulttimeout(3600)

Wire it up

noreply@blogger.com (malthe) — Thu, 29 Jan 2009 14:49:00 +0000

While trying to wire up an interface for managing composited content on a page, it occurred to me that it's very similar to using something like Interface Builder (IB) on the Mac and pulling the wires with XCode. The wire problem is fundamentally difficult, but with JavaScript and HTML, it's plain awkward.

Essentially there are two ways to go about creating user interfaces and this goes for both desktop applications and browser-based applications. It's either done programatically or graphically. Certain tasks can only be accomplished programatically, so there nothing's new under that sun.

Both HTML and IB are essentially based on containment. Buttons inside panes inside dialog windows aren't much different than buttons inside divs inside a document body. However, IB is object-oriented and each object corresponds to a class in Objective-C. The linker is actually tying the knots of the wires at compile-time.

It's hard to express abstract objects with HTML, because you're typically only afforded a class name, the exception being form elements and anchors, but even they carry only a minimal set of attributes. It's common to simply put binary decorators in elements in the form of multiple class names, e.g. "selected", "enabled", "current".

An alternative wiring scheme is to use invisible anchors to signal some interaction using the "href", "rel" and "name" attributes to carry information. It's not a good solution, because there's no "value" attribute (which would facilitate an RDF-like structure) and none of these attributes are indexed, making queries slow.

A third strategy is to use hidden definition lists. This can be coupled with containment to express attribute name/value pairs. The disadvantage is that it's an inappropriate container. Definition lists are part of the document content, while element attributes typically aren't. But the structure is near-perfect.

Meanwhile, XML namespace attributes may qualify as a fourth option even though browser-support is still disappointing. There's fundamental support, but no high-level API. To change this situation, I decided to put together a jQuery plugin which provides XML attribute access using James Clark's namespace notation "{ns}name" which will be familiar to most people who've been parsing XML.

By combining the HTML attributes "id" and "class" (which all elements may carry) and XML attributes, we can model the state of a class instance. In a complex situation, one could use the prototyping features of JavaScript, but in many cases it will be good enough to simply use this as a data storage.

Update: I've successfully tested the plugin in Safari, Firefox and IE7.

An lxml you can build on

noreply@blogger.com (malthe) — Fri, 28 Nov 2008 11:28:00 +0000

Last week I wrote about Chris Perkins' staticlxml package. While it did work for me in a virtualenv, it wasn't feasible for a buildout environment.

However, Stefan Eletzhofer has come to the aid with his z3c.recipe.staticlxml package which really nails this dreadful issue.

Configuration is slightly verbose, but straightforward:

  [lxml]
  recipe = z3c.recipe.staticlxml
  egg = lxml
  force = false
  build-libxml2 = true
  build-libxslt = true
  static-build = true

Put this part into your buildout configuration; note that it must come before any parts that would try to pull in an lxml egg over the wire, otherwise it won't work. Make sure to wipe any existing lxml eggs from your buildout cache before you start.

To verify that everything's gone to plan, use the bindings tool:

   bash-3.2$ find . -name "*.so" -exec otool -L '{}' \;

We don't expect to see any references to libxml or libxslt here.

Speedup

noreply@blogger.com (malthe) — Wed, 19 Nov 2008 14:03:00 +0000

You can now have your cake and eat it, too.

Disclaimer: Don't try this at home! Or actually, do try it, but only on Plone trunk. Simply pull in the five.pt egg and load its configuration. It's a whole-sale drop-in replacement of Zope Page Templates.

If you're interesting in sponsoring this on-going effort, there's an excellent sponsorship opportunity for the upcoming performance-sprint in Bristol. Please contact me by e-mail at mborch@gmail.com.

Update: CMF-bindings have been split out to the cmf.pt package.

An lxml you can bank on

noreply@blogger.com (malthe) — Fri, 07 Nov 2008 15:19:00 +0000

Chris Perkins has saved the Mac's reputation as a development platform with his staticxml package (see the discussion on the mailinglist for paver for reference). It allows you to build lxml as a statically linked egg with little effort. As most Mac-users looking to reap the benefits of lxml will know, building this egg reliable using dynamic linking is difficult; or rather, building it is easy, getting it pick up the right libraries at run-time is a science.

There's really not much more to say. Build that egg and sleep better at night, knowing that no-one can break that sacred bond between lxml and its system library dependencies.

Bringing it all back home

noreply@blogger.com (malthe) — Mon, 06 Oct 2008 14:19:00 +0000

In Zope 2, the application "skin" is comprised of a set of ordered layers, each containing a number of skin objects. These could be Python scripts (running in restricted mode), templates, form controller scripts and static files such as images.

The upshot is that you get to dump files in there and have them immediately working with no additional configuration needed (the file extension is used to infer the type); the downside is that it's a global namespace, there's no restriction on adaptation (any object can be the context and every request type is valid) and there's no permission check.

To contrast, in Zope 3, skin objects have been replaced with browser pages and resources, which are typically configured using the ZCML component configuration language. The Zope 3 way essentially boils down to configuration over convention, and a dose of "explicit is better than implicit".

We can play around with these words and relate to even other ways; one of them is "implicit configuration", and this is partly where Grok finds itself.

In this little writeup, I'd like to introduce two packages that try to be "explicit and conventional", ``repoze.bfg.skins`` and ``repoze.bfg.static``.

Although they can very well be used on their own, it makes sense to combine them. The latter provides "configure once, use everywhere" static resource serving (we're in a WSGI Paste application here).

Static resources are served from mount points that map a URL to a directory on the file-system. The mount points are just adapters registered as views, but they are also responsible for handing out a URL for a particular file-system path (if applicable).

Assuming skin_path is file-system path passed to the WSGI application constructor as a configuration option and app is the Router-object, we can use a statement such as:

  app.registry.registerAdapter(
   repoze.bfg.static.static_view_factory(skin_path),
   name='static')

Note that multiple static views are allowed; the library will go through the available views until it finds a mount point that's applicable to serve up a given static resource. We can now serve any static resources in or below the skin path using the ``repoze.bfg.static.get_url`` function.

This integrates with the skins package, but a quick introduction is in order before we continue.

The skins package provides a ZCML-directive to register a directory of "skin templates" (see the PyPi-page for a more thorough description).

  <bfg:templates
   for="..." request_type="..." permission="..."
   directory="templates" />

Skin templates are registered individually as view components. From inside a skin template (which is essentially a page template that gets passed two special symbols), it's easy to include other skin templates. And if the directory where the templates is located, lies within a mount point to serve static resources, we can link to resources local to the template like so:

  <img src="${api.static.url('logo.png')}" />

This can also be used to link to stylesheets and javascript. This way, using just these two packages, one could create the frontend for a simple web application, complete with images.

Blitting and squeezing

noreply@blogger.com (malthe) — Sat, 04 Oct 2008 21:29:00 +0000

When middleware actually sits in the middle and is not attached like a parasite to its host, it's makes for some very convenient software. After a few misguided attempts, we believe to have come up with two good middlewares.

First there's ``repoze.bitblt``. It transforms images on the fly! Give it a width and a height, and it'll resize your image. Use it in conjunction with a caching proxy for optimum performance. Ideas for future enhancements include mimetype conversion and image scale presets.

Then we have ``repoze.squeeze``. It literally squeezes your stylesheets and javascripts, all with zero configuration! How can it work?! It keeps a record of which resources appear together and keeps an internal cache of the resources as they are served; when it has enough information to determine which resources should be bundled (merged), it intercepts HTML documents and rewrites the resource links. Expiration dates and vary-headers are observed, too!

Haiku

noreply@blogger.com (malthe) — Wed, 24 Sep 2008 21:24:00 +0000

>>> class WhatTheFuck(object):
...     def __init__(self):
...         self._WhatTheFuck__answer_to_everything = 42
...         print self.__answer_to_everything
>>> metadata = WhatTheFuck()
42