>>> import sys
>>> len(sys.modules)
35
>>> foo
...
NameError: name 'foo' is not defined
>>> len(sys.modules)
225

That's 190 extra modules — merely importing them takes around 60 ms on my laptop! Where are they all coming from? And how could an exception cause so many imports, including such illustrious modules as “email”, “mimetools”, and “xml”?

After reading Ubuntu's “sitecustomize.py” file and all of its consequences, the situation became clear. Their apport crash-reporting subsystem instruments Python with an exception hook that, when invoked, discovers that my system says “enabled=0” in my “/etc/default/apport” file and so it undertakes no special crash logging. But, on the way to loading the routine that performs this simple check, it performs two quite flagrant and unnecessary imports, pulling in both “apt” (that brings with it 83 packages) and “apport” (an additional 107 packages).

The solution? I have removed the “python-apport” package, along with the “ubuntuone-client” suite that depends on it. After the uninstall, exceptions are — wonderfully enough — not causing a single import of a new module! Now, finally, I can continue writing my import hook in peace.

firefox -remote 'openURL(http://example.com/, new-tab)'

But after a few months of manually cutting and pasting URLs into Chrome — which wasn't actually that bad, since the address bar in Chrome is such a convenient and large target — I decided that I needed a real solution. After not finding anything like a -remote option, I discovered that Chrome can at least be run with a debugging port open:

google-chrome --remote-shell-port=9222

The protocol that Chrome speaks is primitive enough that it was quick work to implement a small client in Python. Rather than merely cutting and pasting its code here on my blog, or even be satisfied with making it available on bitbucket, I decided to place the code inside of a new Python package and make it generally available on PyPI as chrome_remote_shell.

Thanks to this simple package, a four-line program (not counting the shebang and comment) is now all that I need to ask Google Chrome to open a new tab:

#!/usr/bin/env python
# Name this file "google-chrome-open-url"
import sys
import chrome_remote_shell
shell = chrome_remote_shell.open()
shell.open_url(sys.argv[-1])

To teach Emacs to start using Google Chrome when I clicked on a link, I only needed to supply it with two new settings:

(setq browse-url-browser-function
      'browse-url-generic)
(setq browse-url-generic-program
      "google-chrome-open-url")  

And now everything works. I hope that these notes prove useful to someone else. Enjoy!

It was with regret that I tendered my resignation yesterday as the Editor-in-Chief of Python Magazine. While the publisher will keep producing the magazine by distributing PDFs on the web site, the transition to the new format has dragged on long enough — both for both myself and our customers — that I have run out of enthusiasm. My last responsibility will be to shepherd the February and March issues through the publishing process and safely on to the PDF readers of our subscribers.

I hope that the authors featured in the October issue will forgive me for not writing my usual blog post last year touting their achievements; I had just received the sad news that the publisher could no longer afford the rising costs of printing and shipping Python Magazine, and I did not want to further advertise the magazine until its fate was certain one way or the other.

I have by no means been a perfect editor. In particular, the publisher hoped that I would get the magazine — which was running eight weeks late — back on schedule. Instead, my bumbling first month as editor made the magazine an additional week late, and by the time I hit my stride in May it was another week behind. Although the schedule then stabilized at a steady ten weeks late, I never did manage to start reeling the fish back in. The only metric, I suppose, which I can really claim to my credit is that I oversaw a nineteen-fold increase in the number of em-dashes in the magazine — 247 appeared over the course of 2009, up from only 13 the year before!

I should express thanks to my co-workers: Arbi, Emanuela, and Cathleen are smart, helpful, and professional, and were patient with me as I learned the ropes. Doug Hellmann gave me ample training as he handed over the reins, and also supported the magazine later as an acquisitions editor. Several associate editors performed solid reviews of incoming articles. And, of course, the greatest privilege of being Editor-in-Chief was to help such a wide array of voices from the Python community find their way into print — from Steve Holden, the illustrious chair of the Python Software Foundation, to young Meran Cambpell-Hood, an eleven-year-old from New Zealand who described using Python for the first time to process data for her science fair project.

Which reminds me: the authors from the October issue never got their moment in the spotlight! The article by Meran Campbell-Hood about her science fair project was the most fun to edit, but every single article was interesting and taught me something. Steve Holden interviewed James Tauber about the secrets of a successful Python start-up; Yusdi Santoso finished his two-part series on the Python program he wrote to produce the PDF for the beautiful EuroPython brochure last year; the original editor of Python Magazine, Brian Jones, returned to talk about why he now tends to choose Django for web projects rather than PHP; and Joe Amenta introduced his "3to2" project, which will help Python programmers support their old Python 2 users while still moving ahead with the transition to Python 3. Finally, Greg Newman explained how to turn Emacs into a powerful Python IDE, and Steve Holden and myself rounded out the issue with our usual editorializing.

With many of the rest of you, I am eager to see the debut of the new Python Magazine web site. And I look forward to seeing everyone at PyCon 2010 in less than three weeks! While I will not have the joy that I did last year of walking the halls of PyCon as a newly-minted Editor-in-Chief, able to make dreams come true and grant the fame and fortune of being a published author, I will at least enjoy being a developer among developers in the best programming language community on Earth!

The September issue of Python Magazine appeared on the web late last week and only now, as a new week has started, am I finally sitting down to announce it! The articles range from technically heavy development topics to high-level thoughts about the whole Python community, with plenty in between.

I have to say that our prettiest article this month is “Using Python to Create Beautiful Documents” by Yusdi Santoso, who shares the basic secrets to document generation that he learned when building the EuroPython 2009 brochure using a Python program. Traditional typesetting and computer typography were both interests of mine when I was growing up, so it was fun to read Yusdi's introduction to using ReportLab to generate PDF documents. I look forward to his follow-up article that we will soon be publishing, on the specific techniques that he used in creating the EuroPython booklet.

The other technical articles are an introduction to using SOAP in Python; a guide to displaying objects in a Mac OS X GUI created with PyObjC; an article introducing Python's own built-in Tkinter GUI toolkit; and a small excursion of my own that attempts to explain the popular “trick” (well, it really confused me the first time I saw it!) of defining a decorator using a pair of nested functions. I should confess that my own article contains what is probably this issue's biggest mistake, as pointed out quite promptly by alert reader Emanuel Woiski: in the code sample that is its whole crux of my example, I somehow managed to omit one of the most crucial lines, shown here in bold:

def log(function):
    def log_wrapper(*args):
        print "called %s%s" % (
            function.__name__, tuple(args)
            )
        return function(*args)
    return log_wrapper

I suppose I will now need remedial cut-and-paste training of some sort.

Finally, the issue is rounded out by three articles that move back from Python coding and step out to wider vantage points. Justin Lilly provides an excellent guide to customizing your Vim setup so that it becomes a powerful Python integrated development environment. Steve Holden muses about why diveristy is so difficult and reveals some of the recent goings-on surrounding the diversity statement that the Python Software Foundation has been working on. And my own editorial seeks to point any Python Magazine readers who do not yet have a strong connection with the wider community in the direction of greater engagement with the world of Python.

All in all, I think the issue is a nice mix of fact, experience, and opinion. Please consider subscribing if you would like to hear more about what people are doing with Python, and how. I enjoy reading it; so might you.

Importing a normal, rectangular map of Middle-earth as a Google Earth overlay is too narrow toward the north.

I wanted to measure distances in Tolkien's Middle-earth. While a flat map distorts such measurements, it occured to me that Google Earth can correctly measure both lines and paths across the curved surface of the globe. I soon found excellent documentation for using image overlays with Google Earth, so I downloaded a map of Middle-earth and tried placing it on the globe.

Imagine my disappointment when I saw the result shown in the above image! At first I made the mistake of not holding down the Shift key when resizing the image in Google Earth; the Shift key is absolutely critical for the image to maintain its aspect ratio as you stretch it to the right dimensions. But even after learning this habit, it was painfully clear that the Middle-earth map's projection was different from that expected by Google Earth: the map is far too narrow at the top.

Obviously, it was time to pull out Python, my favorite programming language, and see whether the Python Imaging Library could help me make short work of converting a map from one projection to another.

My Python table at the Atlanta Linux Fest. You can also watch a short video of me demonstrating a depth-first search to some students who dropped by the table. (Thanks, Richard Davies, for the video!)

Running the Python table at the Atlanta Linux Fest this past weekend was a really incredible experience.

First, there was the great feeling that the pillars of the Python community were standing behind me as I stepped forward to represent my favorite programming language. It was Andrew Kuchling who noticed that exhibitor tables at the Fest were free for non-profits like the Python Software Foundation, and Steve Holden who forwarded me a heads-up since I live in Atlanta (the Fest had not yet made it on to my radar). The inimitable Aahz personally shipped me the promotional kit, including a huge “Python” banner and stacks of brochures, that he himself had just used at OSCON 2009. And, completing the loop, it was Andrew who followed up to ask if there were any last things that I needed, and sent me a pile of over one hundred Python stickers that wound up being very popular at the Fest. (I returned home with exactly one, which is sitting next to me on my desk as I type this!)

Here are some lessons that I learned from the experience:

I would like some advice from Zope and Plone folks about how to create forms that are not only easy for other developers to specialize, but which allow several specializations to be composed together. While I have used zope.formlib and z3c.form before for simple tasks, I have not yet been able to tell whether they support these more advanced kinds of operations.

Some background: I am doing some work on GetPaid for Plone with the generous funding of Derek Richardson who, if his dreams had not carried him away from grad school at the end of the Spring semester, would have tackled this same work as part of the 2009 Google Summer of Code.

The current mechanisms that GetPaid provides for customizing its checkout process are very primitive, and my task is to improve them. That is why I have been thinking about customizing forms.

The August issue of Python Magazine is out, and the cover issue is one of the most exciting that I have had the privilege to publish. Following up on his popular talk at PyCon 2009, professor of computer science Dr. Bill Punch has written an article for us with his colleague Dr. Richard Enbody about replacing C++ with Python in Michigan State University's introductory programming class.

The difficulty they faced was that the second and subsequent classes in Michigan State's curriculum have continued using the C++ language; only the initial course could change to Python. While non-majors taking the introductory course get to learn Python and leave, the computer science majors have to then start the second course without any specific knowledge of C++. How much, skeptics wondered, would it hurt the Python students to be a full semester behind in their C++ knowledge compared to students who took the “old” C++ version of the introductory course?

The circumstances at Michigan State wound up being perfect for making the change to Python a full-fledged, statistically valid experiment to determine how much student grades would be hurt because of the switch. Read the article for the detailed results, but I can report — with great satisfaction — that the Python students proved themselves the equals of their C++ peers, and also that very exciting and unexpected results ensued among the students whom the study was not looking at: the non-majors, for which this was their only programming course, and the seasoned grad students, who rolled up their sleeves and learned Python in order to be able to TA the course. Again, see the article for all of the details!

What else do we have this month? Articles about how easy concurrency becomes with Stackless Python; about documenting your software projects using the powerful Sphinx documentation system; about creating simple graphs with the industry-strength matplotlib plotting library; a guide to the details of hashing algorithms, and how they relate to the Python dictionary implementation; and, finally, the usual thoughts and ponderings from both myself and from Steve Holden of the Python Software Foundation. Come join us!

The problem, of course, was the chance of collision. Because my shell script names tended to be short and pithy collections of lowercase characters, just like the default system commands, there was no telling when Linux would add a new command that would happen to have the same name as one of mine. This was actually not very likely on, say, a System V Revision 3 workstation in the 1980s, but the trouble became quite a bit more acute when I moved into the world of Debian. Red Hat never really worried me, because they packaged (comparatively) so little software. But Debian today supports a huge number of commands; my modest Ubuntu laptop shows several thousand available:

$ apt-file search -x '^/usr/bin/[^/]*$' | wc -l
21733

The solution was obviously to adjust my command names in such a way that they were still easy to type, but would never be chosen as system command names. For me, “easy to type” means not having to use the shift key, and very few characters turned out to be available, unshifted, on a modern keyboard. The lower-case letters are the very characters used in system commands; brackets, backslashes, the colon, the back-tick, and the single-tick all had a special meaning to the shell; and the slash and dot characters both mean something special in a filename. (The slash divides directory names from filenames, and thus cannot appear in a filename itself, while the dot means “hide this file from normal browsing” if it leads the name, and separates a file from its extension in many other cases.)

There was but one character left: the simple, modest comma.

A quick experiment revealed in a flash that the comma was exactly the character that I had been looking for! Every tool and shell that lay in arm's reach treated the comma as a perfectly normal and unobjectionable character in a filename. By simply prefixing each of my custom commands with a comma, they became completely distinct from system commands and thus free from any chance of a collision.

And, best of all, thanks to the magic of tab-completion, it became very easy to browse my entire collection of commands. When trying to remember which of my commands are available in my ~/bin/ directory on a given system, or when simply trying to remember what some of my commands are called, I simply type a comma followed by tab and my list of commands appears:

$ ,«tab»
,complete-scp        ,go-thpgp            ,range
,complete-ssh        ,gr                  ,svn-store-password
,coreoff             ,hss                 ,umount
,coreon              ,mount-thpgp
,find                ,mount-twt

I heartily recommend this techique to anyone with their own ~/bin/ directory who wants their command names kept clean, tidy, and completely orthogonal to any commands that the future might bring to your system. The approach has worked for me for something like a decade, so you should find it immensely robust. And, finally, it's just plain fun.

I am home from a relaxing vacation to the Midwest, and while I was gone last week my excellent publishing team released the July issue Python Magazine to the world. I am particularly pleased that two of the feature articles in this issue come from important segments of the Python world that we have not heard much from in previous issues.

First, IronPython, the .NET version of Python for Windows, is the topic of Jonathan Hartley's article about acceptance testing. He illustrates that, regardless of the language in which you write your .NET application, you can deploy simple strategies to make your application testable through a Python test harness, and thereby bring Python's strong flexibility as a testing language to bear on a product that you might be writing in another .NET language.

Second, Malthe Borch, a veteran of the Zope community, shares how he has written Chameleon, one of the fastest template language implementations available for Python. By processing each template and turning it into Python bytecode before it is ever used to render a single page, Malthe eliminates a huge amount of redundant processing as that same code is used over and over again. His library is a key ingredient in the new high-efficiency web frameworks appearing in the Zope world. His work might even (fingers crossed) become one of the components that the Plone community uses as they streamline their framework and move towards a lighter and more agile “Plone 4”.

Other technical topics covered are: the Hadoop map-reduce framework; the concept of hash functions, and how they apply to Python; and the new string formatting operator which Guido hopes will replace all of the percent-signs that currently litter our code. Wrap it all up with an editorial by Steve Holden about EuroPython 2009 and an editorial by me, and you have a complete issue! If you do not find Python Magazine sitting on your local newsstand, then I hope you will avail yourself of a subscription and, as always, let me know what topics you would like to see covered in future issues. Enjoy!