Trent Mick

who knew path joining differed so between Python, Ruby, Node, Perl

2010-08-16T23:14:00.000-07:00

In Python I do a lot of path manipulations for build systems, various command-line utilities and Komodo support modules. Typically this is with Python’s os.path module. One thing I’ve come to expect of path joining, os.path.join, is this (apparently rare) detail:

If any component is an absolute path, all previous path components
will be discarded.

I say “apparently rare” because, in Python:

$ python
>>> import os.path
>>> os.path.join("/Users/trentm", "/var/log")
'/var/log'

in Ruby:

$ irb
>> File.join("/Users/trentm", "/var/log")
=> "/Users/trentm/var/log"

in Node.js:

$ node
node> var path = require('path')
node> path.join("/Users/trentm", "/var/log")
'/Users/trentm/var/log'

in Perl:

$ cat pathjoin.pl 
use File::Spec;
print File::Spec->join('/Users/trentm', '/var/log'), "\n";
$ perl pathjoin.pl 
/Users/trentm//var/log

Conclusions? Certainly none of these libraries is going to change their behaviour here, with the possible exception of Node which is young and changing very quickly. I’d say the double ‘/’ in Perl’s File::Spec is poor – though it doesn’t give in invalid path. You could certainly argue that Ruby’s and Node’s interpretation is less subtle (often a good thing). I like Python’s interpretation: os.path.join is kind of like running cd for each given path in sequence to get the resultant directory. It means I don’t need a guard against an absolute path input datum being joined to a current working directory scope.

I’d be curious to know what is typical in other languages, if there are any takers reading this post. If blog commenting isn’t your thing, you can tweet “@trentmick”.

Komodo 6.0 Beta 2: HTML 5, CSS 3, Python 3, DB Explorer, ...

2010-07-14T11:10:00.000-07:00

We (ActiveState) released Komodo 6.0 beta 2 yesterday and we want your feedback. HTML 5 autocomplete. CSS 3 autocomplete. Full Python 3 support (debugging, syntax checking, autocomplete, code browsing). A new Database Explorer tool for quickly exploring SQL databases (SQLite out of the box and extensions for MySQL and Oracle, with plans for Postgres). A new project system called “Places” that adds a file system browser (local and remote). New publishing support for syncing a directory with a remote machine. Additions to Komodo’s Hyperlinks for quickly navigating to file references. Added support for PHP, Perl, Ruby and JavaScript regular expression debugging with Komodo’s excellent Rx tool. See the Komodo 6.0 Features post for a full outline.

Komodo IDE	http://www.activestate.com/komodo-ide/downloads
Komodo Edit	http://www.activestate.com/komodo-edit/downloads

Full post here on the ActiveState blog.

eol.py 0.7.4 -- Python 3 support

2010-07-13T10:03:00.000-07:00

Where?

Project Page: http://github.com/trentm/eol
PyPI: http://pypi.python.org/pypi/eol/

What's new?

Python 3 support (not heavily tested yet)
Starter test suite

Full changelog: http://github.com/trentm/eol/tree/master/CHANGES.md#files

What is 'eol'?

eol is both a command-line script eol and a Python module eol for working with end-of-line chars in text files.

Command line usage

# list EOL-style of files
$ eol *
configure: Unix (LF)
build.bat: Windows (CRLF)
snafu.txt: Mixed, predominantly Unix (LF)

# find files with a given EOL-style
$ eol -f CRLF -x .svn -r ~/src/python
/Users/trentm/src/python/Doc/make.bat
/Users/trentm/src/python/Lib/email/test/data/msg_26.txt
/Users/trentm/src/python/Lib/encodings/cp720.py
...

# convert EOL-style of files
$ eol -c LF foo.c 
converted `foo.c' to LF EOLs

Module usage

>>> import eol
>>> eol.eol_info_from_path("configure")
('\n', '\n')         # (<detected-eols>, <suggested-eols>)
>>> eol.eol_info_from_path("build.bat")
('\r\n', '\r\n')
>>> eol.eol_info_from_path("snafu.txt")
(<class 'eol.MIXED'>, '\n')

See the README for full usage information.

quick hack how to move a part of a Mercurial (hg) repo to git

2010-07-10T00:36:00.000-07:00

My quick dirty hack to move a (small) part of a Mercurial (hg) repo to Git. In my case this was for moving my single file “testlib.py” from here on bitbucket to here on github

Dump the log of that part of the hg repo to a file.

export WORKDIR=$HOME/tmp/migrate
cd HGREPO/foo
hg log -pv . > $WORKDIR/full.patch

Create the starter git repo
```
cd $WORKDIR
git init foo
```

Break up the log into a number of “changesetNNN.patch” files with this Python code:

# parse.py
import codecs
changeset = []
i = 0

def write_changeset():
    global changeset
    if not changeset:
        return
    codecs.open("changeset%05d.patch" % i, 'w', "utf-8").write(''.join(changeset))
    i += 1
    changeset = []

for line in open("full.patch"):
    if line.startswith("changeset:"):
        write_changeset()
    changeset.append(line)
write_changeset()

and then run:

python parse.py

Apply and commit each patch with this Python script

# Usage: python apply.py TARGET-REPO-BASE-DIR
import os
from os.path import *
from glob import glob
import subprocess
from pprint import pprint

def apply_patch(target_repo_base_dir, patch_path):
    content = open(patch_path).read()
    header, diff = content.split('\n\n\n', 1)
    assert diff.startswith("diff --git a")
    fields = {}
    lines = header.splitlines(False)
    for i, line in enumerate(lines):
        key, value = line.split(':', 1)
        if key == "description":
            fields[key] = '\n'.join(lines[i+1:])
            break
        value = value.strip()
        fields[key] = value
    
    # Do any path renamings here. For example, I wanted to move from
    # "testlib/testlib.py" in the old repo to "lib/testlib.py" in the new.
    diff = diff.replace('a/testlib/testlib.py', 'a/lib/testlib.py')
    diff = diff.replace('b/testlib/testlib.py', 'b/lib/testlib.py')
    
    f = open(patch_path+".diff", 'w')
    f.write(diff)
    f.close()
    f = open(patch_path+".msg", 'w')
    f.write(fields["description"])
    f.close()
    cwd = target_repo_base_dir
    subprocess.check_call(['git', 'apply', '--whitespace=nowarn',
        abspath(patch_path+".diff")], cwd=cwd)
    subprocess.check_call(['git', 'add', 'lib/testlib.py'], cwd=cwd)
    subprocess.check_call(['git', 'commit', '--date', fields["date"],
        '-F', abspath(patch_path+".msg")], cwd=cwd)

if __name__ == "__main__":
    target_repo_base_dir = sys.argv[1]
    patches = list(sorted(glob("changeset*.patch")))
    for patch_path in patches:
        print "--", patch_path
        apply_patch(target_repo_base_dir, patch_path)

Then run:

python apply.py foo    # apply all changeset*.patch files to "foo" git repo

Now you can push this Git repo to github or whereever.

To improve on

You currently need to manually create the dir structure first.
This doesn’t currently used the parse “user” field from the hg commit log for the “git commit -a AUTHOR” command. Mainly this is because I didn’t need that, but also because the “user” value in the hg log isn’t the configured full user name and email, but just the short username. Maybe that was just me.

On a bit of confusion

2010-07-08T11:55:00.001-07:00

On a bit of confusion in ActivePython 2.7 released - and what 2.7 means for Python’s future, in particular the following paragraph. Before:

While the Python community has declared a moratorium on major 2.x releases in an effort to facilitate other Python implementations to catch up and, thus, accelerate the adoption of Python 3.x, ActiveState will continue supporting 2.7.x and adding new modules and updating revisions to existing ones as they become available.

After:

While you may have read, the Python community has declared a temporary moratorium (suspension) on the Python language syntax in an effort to facilitate other Python implementations to catch up to Python 3.x — the moratorium does not that mean that python core development has stopped or even slowed down.

On the contrary, new modules continue to be added, bugs fixed, and performance tweeked — and, as always, ActiveState will continue supporting 2.7.x with builds, extra modules and PyPM as they become available.

Jesse Noller accurately noted that the former paragraph was confusing. In particular, a possible interpretation that the Python community isn’t going to be supporting Python 2.7 – which is just not true. Python 2.7 will be supported for longer than the typical two years that a Python 2.x release is supported.

It is easy to convolve the mostly unrelated Python language moratorium and the plan that Python 2.7 is the last 2.x. The two are somewhat related in that ultimately the hope is they both lead to smoother adoption of Python 3. The issue (from ActiveState’s Product Manager’s point of view) is that an enterprise customer can get swayed away from considering Python when reading stuff like the following from Python moratorium and the future of 2.x:

On November 9, Python BDFL ("Benevolent Dictator For Life") Guido van Rossum froze the Python language’s syntax and grammar in their current form for at least the upcoming Python 2.7 and 3.2 releases, and possibly for longer still. This move is intended to slow things down, giving the larger Python community a chance to catch up with the latest Python 3.x releases.

It is very easy for the less-technical person to interpret “moratorium on Python language syntax” as a stop on all core Python development. This differentiation was the kernel of a heated debate I just had with our Product Manager recently.

The intention of the paragraph in the ActiveState blog post is basically to state that the language moratorium isn’t something that should dissuade businesses from considering Python.

That said: Mea culpa. I had the chance to catch this the first time and didn’t.

django-markdown-deux Django app

2010-06-14T23:19:00.000-07:00

Project page: http://github.com/trentm/django-markdown-deux
on PyPI: http://pypi.python.org/pypi/django-markdown-deux/

django-markdown-deux is a small Django app that provides template tags for using Markdown using the python-markdown2 library. MIT license.

What's with the "deux" in the name?

The obvious name for this project in django-markdown2. However, there already is one! and name confusion doesn't help anybody. Plus, I took French immersion in school for 12 years: might as well put it to use.

Quick Usage

`markdown` template filter

{% load markdown_deux_tags %}
...
{{ myvar|markdown:"STYLE" }}      {# convert `myvar` to HTML using the "STYLE" style #}
{{ myvar|markdown }}              {# same as `{{ myvar|markdown:"default"}}` #}

`markdown` template block tag

{% load markdown_deux_tags %}
...
{% markdown STYLE %}        {# can omit "STYLE" to use the "default" style #}
This is some **cool**
[Markdown](http://daringfireball.net/projects/markdown/)
text here.
{% endmarkdown %]

See more usage info, available settings, installation notes, etc. at the github project page. (I mention on Moz planet because Benjamin is, or was, using python-markdown2 and I've heard Mozilla is using more Django these days.)

Donating blood in downtown Vancouver

2010-06-14T09:47:00.000-07:00

I like to try to donate blood regularly (you're allow to donate every 56 days). It is an easy thing to do to help the system. But really I'm just in it for the free cookies at the end of it. There is no other way I can justify my having an Oreo at 10am. Here is how in downtown Vancouver (where I work):

upcoming schedule: http://bit.ly/vancouverblood
map: http://bit.ly/vancouverbloodmap

Some tips: You'll need ID. A driver's licence will do. Don't go at lunch time when it gets much more busy.

June Vancouver Python User Group (VanPyZ) this Tuesday at ActiveState

2010-05-27T16:06:00.000-07:00

We are hosting the June VanPyZ meeting at ActiveState this coming Tuesday evening. Come see Andy Mckay talk about Using SMS in the Developing World and chat about Python. Festivities begin at 7pm here.

More details on my work blog post and at vanpyz.org.

python-markdown2 1.0.1.17

2010-05-26T22:32:00.000-07:00

Where?

Project Page: http://code.google.com/p/python-markdown2/
PyPI: http://pypi.python.org/pypi/markdown2/

What's new?

[Issue 36] Fix "cuddled-lists" extra handling for an looks-like-a-cuddled-list-but-is-indented block. See the "test/tm-cases/cuddledlistindented.text" test case.
Experimental new "toc" extra. The returned string from conversion will have a toc_html attribute.

New "header-ids" extra that will add an id attribute to headers:

# My First Section

will become:

<h1 id="my-first-section">My First Section</h1>

An argument can be give for the extra, which will be used as a prefix for the ids:

$ cat foo.txt 
# hi there
$ python markdown2.py foo.txt 
<h1>hi there</h1>
$ python markdown2.py foo.txt -x header-ids
<h1 id="hi-there">hi there</h1>
$ python markdown2.py foo.txt -x header-ids=prefix
<h1 id="prefix-hi-there">hi there</h1>

Preliminary support for "html-classes" extra: takes a dict mapping HTML tag to the string value to use for a "class" attribute for that emitted tag. Currently just supports "pre" and "code" for code blocks.

Full changelog: http://code.google.com/p/python-markdown2/source/browse/trunk/CHANGES.txt

What is 'markdown2'?

markdown2.py is a fast and complete Python implementation of Markdown -- a text-to-HTML markup syntax.

Module usage

>>> import markdown2
>>> markdown2.markdown("*boo!*")  # or use `html = markdown_path(PATH)`
u'<p><em>boo!</em></p>\n'

>>> markdowner = Markdown()
>>> markdowner.convert("*boo!*")
u'<p><em>boo!</em></p>\n'
>>> markdowner.convert("**boom!**")
u'<p><strong>boom!</strong></p>\n'

Command line usage

$ cat hi.markdown
# Hello World!
$ markdown2 hi.markdown
<h1>Hello World!</h1>

This implementation of Markdown implements the full "core" syntax plus a number of extras (e.g., code syntax coloring, footnotes) as described on http://code.google.com/p/python-markdown2/wiki/Extras.

eol.py -- a tool for working with text file end-of-line characters

2010-05-19T00:21:00.001-07:00

Just finally released eol.py (on github, on pypi) -- a tool I've been using for a while to help find and convert EOLs in text files. It is a small Python module and command-line script. MIT license. Hopefully useful to others.

See the notes in either link above for usage details.

alternative Komodo doc icon for Mac

2010-03-24T10:29:00.000-07:00

Sweet! Trisha Cupra wasn't happy with the current Komodo Edit doc icon so she made a (cool) replacement.

March VanPyZ meeting tomorrow at ActiveState

2010-03-08T14:29:00.001-08:00

The March VanPyZ meeting will be hosted at ActiveState this coming Tuesday evening (tomorrow!). I posted details on my work blog. We'll be trying out a slightly new thing: Lightning talks, a short 5 minute talk on whatever ¹. The Lightning Talk format works so well at PyCon. See my work blog post for more details. See you there.

"VanPyZ" is the Vancouver Python user group.

In my email to the VanPyZ list I promised a beer to whoever presents. :) Send me a one-liner on what you'd like to talk about. ↩

PyCon 2010 wrap-ups

2010-03-05T15:28:00.000-08:00

I'm back from PyCon 2010 in Atlanta (actually I've been back for almost two weeks now, but I'm a lazy writer). PyCon, as usual, was a great time. I wrote a long wrap-up on my work blog here:

Trent's work PyCon 2010 wrap-up

A couple more personal additions...

I announced Python Translation Party during PyCon -- here, here, and in a Lightning Talk (video not up yet) -- to a resounding golf clap. :) It is a small site devoted to translating Python code between 2 and 3 using the lib2to3 and lib3to2 libraries until "equilibrium" is found. Mostly it was a fun little play with AppEngine. However, it has also resulted in a few bug reports to lib3to2 and lib2to3, which is cool.
I had a good chat with Catherine Devlin about her cmd2 library, because it is so similar in spirit to my cmdln library. We chatted a bit about whether a merging of the two libs would be reasonable. No concrete plans: busy busy.

It was great to hook up with people that I tend to only see at PyCon. I look forward to PyCon next year.

On fighting in hockey

2010-02-25T22:36:00.001-08:00

I love hockey. Obviously, growing up in Canada I'm biased, but: It is an active and skilled sport which has continuous play where baseball, curling and football aren't or don't. A real gem of hockey is a breathtakingly long stretch of all-out fast back and forth play. The build up to an at-bat in baseball pales. A full-field punt return in football doesn't come close. Basketball, soccer and rugby have continuous play, as well. Perhaps I prefer hockey because of the speed you don't get in the others. Undoubhtedly part of it is the general love for hockey in Canada. Perhaps part of it is the physicalness. And here is where there is a problem ¹.

By "physicalness" I mean cleanly knocking a player off the puck, a defenceman rubbing a charging forward out on the boards, the battle for positioning in front of the net. To many, including Gary Bettman, the physicalness includes fighting. To Gary it is "part of the game". That's crap. Somehow the Vancouver 2010 Olympic hockey games have provided great spectacle without punches (in the games I've seen anyway). Somehow soccer, basketball and football (heated games, all) maintain fan fanaticism with fighting rules that all but remove it from the game.

It is also sad. Sad because I have a two year old son. Why do I need to explain the inconsistency to my son: hockey is a great game, except for the immaturity of the fighting. An immaturity that is abetted and celebrated at the top-level. Hockey fighting isn't even comparable to boxing. It is ultimate fighting. It is something children should be educated away from, not implicitly taught to emulate.

Gary on fighting:

From a player safety standpoint, what happens in fighting is something we need to look at just as we need to look at hits to the head,

Certainly player safety is important, but this is a deflection. Fighting in hockey isn't about the safety issue, it is about the message to the immature -- from the 2 year-olds to the 30-something neanderthals you sometimes encouter in recreational hockey. Fighting no more needs to be part of the game, than my son needs to hit mommy when she takes away his cookie.

At least in the NHL and the Canadian minor leagues (WHL, OHL, et al) that I'm exposed to. ↩

updated Python Translation Party: fewer booms

2010-02-22T23:01:00.001-08:00

I've updated Python Translation Party to recent versions of lib2to3 and lib3to2. I spoke with Joe Amenta at the PyCon 2010 poster session (he had a poster on his lib3to2 work) and he mentioned that the best thing I could do with Python Translation Party for him was to update to the latest. :) Updating to the latest is a little difficult because App Engine runs Python 2.5.2 and lib3to2 targets Python 2.7, so some small backporting was in order.

At the least, the new update results in fewer "booms" -- i.e. when there is some sort of unexpected error in translating either way. The "boom" section of the "Crash other parties" page before:

and after:

One boom remains. I'll have to dig in to see if this is a lib3to2 bug, lib2to3 bug, or party bug.

other Python VM's upcoming Python version plans

2010-02-18T13:54:00.001-08:00

Some quick notes about the coming plans by the "other" Python implementations from today's Python Language Summit at PyCon 2010:

IronPython:
- plan is to do Python 2.7 first, focus for this year
- python 3.2 for the end of next year hopefully
- other work on IDE stuff
Pynie (i.e. Parrot) -- Allison Randall:
- about 4 major features away from pure Python syntax (did dicts last night)
- targetting py3k repo and test suite: should be on track for python 3.2
Jython:
- plan to target 2.6 (b/c 2to3 depends on 2.6)
- temporarily skip 2.7 and target 3.x (probably 3.2)
- then if 3.x adoption isn't fully there, then go back and add Python 2.7
- will require JDK 2.7 for Python 3 support (b/c of new support for dynamic languages)
PyPy (Holger):
- plan is Benjamin will port to Python 2.7 in the summer
- only have slight deviations from CPython: idea is to merge back with CPython so don't have deviations. Typcically 1 or 2 line changes in ~25 modules.

Python Translation Party

2010-02-18T09:16:00.000-08:00

or "Why Jesse Noller owes Trent Mick a beer"

Back at the Python Language Summit at PyCon 2009 we discussed the desire for a 3to2 tool for Python. 2to3 is a useful tool for helping with migration to Python 3 and getting a feel for Python 3. However, where we really want this Python 3 thing to go is for module developers to write for Python 3 and then have a tool that (ideally) makes the job of supporting a Python 2 version completely automatable.

In a GSOC project Joe Amenta, mentored by Benjamin Peterson, made that happen: lib3to2.

but this post is about fun

Soon after Joe's release, Jesse Noller posted to talk up 3to2 and said:

p.s. The first person to write an online 2to3 -> 3to2 app ala translation party gets mad props, and a bottle of something alcoholic at Pycon.

Game on. I'd been looking for an idea to play with App Engine that wasn't totally useless. Alcohol's good too.

The Translation Party idea: translate between English and Japanese (using Google Translator) until equilibrium is found or not.

Python Translation Party

The Python Translation Party idea: translate between Python 2 and Python 3 (using lib2to3 and the new lib3to2) until equilibrium is found or not.

Jesse, you owe me a beer. :) Party on.

February VanPyZ meeting at ActiveState on Tuesday

2010-01-30T14:16:00.000-08:00

The February VanPyZ meeting will be hosted at ActiveState this coming Tuesday evening. I posted details on my work blog. Myself and Michael Grünewald will be speaking on recent ActiveState Code work and Brett Cannon will be speaking on his importers project. See you there.

"VanPyZ" is the Vancouver Python user group.

nice I ♥ Komodo blog post

2010-01-14T11:02:00.000-08:00

Nice post on Komodo (comparing to other editors): Sayonara TextPad, Hello Komodo Edit

So, who makes an FOSS-friendly editor for Windows with the best UI that leverages Scintilla?

In the end, I settled on Komodo Edit.

...

What clinched the deal was (a) the tour of the UI that really appeared to be thoughtfully considered by its designers, and (b) the design team which has its roots in support of Perl and other FOSS initiatives for the Windows platform. What was clear to me was that the guys at ActiveState punched away in the same languages as I did, and clearly understood what would make a best of breed coding editor, then they just went ahead and built one!

VanPyZ meeting at ActiveState tomorrow (Tues)

2009-11-30T10:31:00.000-08:00

We are hosting the December VanPyZ meeting at my work (ActiveState) tomorrow night. Details:

What

Jamu Kakar: How to use Storm with a focus on some common patterns to common problems.
Doug Latornell: PyYAML, flickrapi, and TkInter in a Desktop Image Display App.

Where

ActiveState
1700 - 409 Granville Street (SW corner of Granville and Hastings, map)
Buzzer# 1700

When

Tuesday, December 1st, at 7PM

"VanPyZ" is the Vancouver Python user group.

Small Django patch to add full traceback for wrapper exception during template rendering

2009-11-17T09:43:00.000-08:00

Here is a patch that I'm tending to apply to my Django (currently v1.1.1) trees to give me a full traceback on the wrapped exception when getting an exception during template processing. Without this patch you only get the string summary of the underlying exception -- often far from enough info to track down the actual bug. With this patch the result isn't that pretty (a full Python traceback as the string summary of the TemplateSyntaxError), but the info sure is helpful.

Index: django/template/debug.py
===================================================================
--- django/template/debug.py    (revision 11742)
+++ django/template/debug.py    (working copy)
@@ -68,20 +68,23 @@
 class DebugNodeList(NodeList):
     def render_node(self, node, context):
         try:
             result = node.render(context)
         except TemplateSyntaxError, e:
             if not hasattr(e, 'source'):
                 e.source = node.source
             raise
         except Exception, e:
             from sys import exc_info
+            #TODO?: grab the "(at $file:$line)" suffix code, tack that on, propose for Django core
+            import traceback
+            e = traceback.format_exc()
             wrapped = TemplateSyntaxError(u'Caught an exception while rendering: %s' % force_unicode(e, errors='replace'))
             wrapped.source = node.source
             wrapped.exc_info = exc_info()
             raise wrapped
         return result

 class DebugVariableNode(VariableNode):
     def render(self, context):
         try:
             output = force_unicode(self.filter_expression.resolve(context))

using gmail for outbound email on Mac OS X

2009-11-11T00:09:00.000-08:00

Why?

At work I run a number of cron jobs: web log analysis scripts, metrics gathering, db migration for beta versions of some sites, etc. However, one thing I've been unable to use effectively on my Mac machines ¹ is the MAILTO facility of cron. Given a foo.cron script like this:

#!/bin/sh
export MAILTO=me@example.com
export SUBJECT="do some daily stuff"

...

that is run like this:

0 0 * * *  $HOME/.../foo.cron > $HOME/var/log/do-some-daily-stuff.log

any output to stderr with be mailed to "me@example.com". That is, it would if outgoing email was setup for the currently running MTA (mail transfer agent) on that machine.

How?

On Mac OS X (since Leopard, I think) the default MTA is Postfix, and Postfix's terminology for "handle outbound email" is to set the "relayhost". This post explains how to configure Postfix on Mac OS X 10.5 (Tiger). What follows is my summary of the commands -- slightly different than the other post for clarity -- plus a few thoughts. Please read the other post for extra details.

# Create `/etc/postfix/relay_password` with auth information like this:
#   smtp.gmail.com YOURNAME@gmail.com:YOURPASSWORD
$ sudo vi /etc/postfix/relay_password
$ cat /etc/postfix/relay_password
smtp.gmail.com YOURNAME@gmail.com:YOURPASSWORD

# Compile this to a Postfix lookup table...
$ sudo postmap /etc/postfix/relay_password

# ... and test it.
$ sudo postmap -q smtp.gmail.com /etc/postfix/relay_password
YOURNAME@gmail.com:YOURPASSWORD

# Download the Thawte root certificates from
# <https://www.verisign.com/support/roots.html> and setup
# a certificate for Postfix.
$ firefox https://www.verisign.com/support/roots.html
$ cd ~/Downloads
$ unzip -q roots.zip
$ sudo mkdir /etc/postfix/certs
$ sudo cp ~/Downloads/Thawte\ Root\ Certificates/Thawte\ Root\ Certificates/thawte\ Premium\ Server\ CA/Thawte\ Premium\ Server\ CA.pem /etc/postfix/certs
$ sudo c_rehash /etc/postfix/certs/
Doing /etc/postfix/certs/
Thawte Premium Server CA.pem => d44d72e8.0

# Add the following lines to `/etc/postfix/main.cf`:
    relayhost = smtp.gmail.com:587
    # auth
    smtp_sasl_auth_enable = yes
    smtp_sasl_password_maps = hash:/etc/postfix/relay_password
    smtp_sasl_security_options = noanonymous
    # tls
    smtp_tls_security_level = may
    smtp_tls_CApath = /etc/postfix/certs
    smtp_tls_session_cache_database = btree:/etc/postfix/smtp_scache
    smtp_tls_session_cache_timeout = 3600s
    smtp_tls_loglevel = 1
    tls_random_source = dev:/dev/urandom

# Re-start the Postfix service. (Note: You can also do the re-starting via
# `launchctl`.)
sudo postfix stop
sudo postfix start

# Now test this by sending email with `mail`:
$ /usr/bin/mail -s "testing 1 2 3" recipient@example.com
Howdy, from Postfix on your Mac!
.
EOT

Now recipient@example.com should have received your test email. If not check "/var/log/mail.log". Here is a successful log message:

Nov 10 23:33:26 mower postfix/smtp[56627]: ED1265BBA2A: to=<recipient@example.com>, relay=smtp.gmail.com[74.125.155.109]:587, delay=2.9, delays=0.44/0.94/0.56/0.96, dsn=2.0.0, status=sent (250 2.0.0 OK 1257924806 20sm930503pxi.15)

Here is a failing log message (in this case because I'd incorrectly mixed using "smtp.googlemail.com" instead of "smtp.gmail.com"):

Nov 10 21:32:14 mower postfix/smtp[55593]: C028D5BB6F8: to=<recipient@example.com>, relay=smtp.googlemail.com[74.125.155.16]:587, delay=0.33, delays=0.05/0.02/0.24/0.02, dsn=5.5.1, status=bounced (host smtp.googlemail.com[74.125.155.16] said: 530-5.5.1 Authentication Required. Learn more at                               530 5.5.1 http://mail.google.com/support/bin/answer.py?answer=14257 20sm817933pxi.3 (in reply to MAIL FROM command))

A couple thoughts

This setup requires putting your Gmail password in plaintext in /etc/postfix/relay_password, which I'm not too happy about. So I've created a separate Gmail account with a different password to use for this.

One of the beauties of this setup is that there is a record of all the emails that have been sent this way in that Gmail account's "Sent mail".

Update (23 Nov 2009)

Note that while I've gotten this working on my 10.5 laptop, I've been *unable* to get this working on a 10.4 Mac Pro. Initially I was getting this error (in /var/log/mail.log):

Nov 22 21:54:34 kungfu postfix/smtp[8329]: 9756661DC9F5: to=<recipient@example.com>, relay=gmail-smtp-msa.l.google.com[74.125.127.109], delay=1, status=bounced (host gmail-smtp-msa.l.google.com[74.125.127.109] said: 530 5.7.0 Must issue a STARTTLS command first. 21sm2865586pzk.7 (in reply to MAIL FROM command))

The fix for that was to add the following line to "/etc/postfix/main.cf" (and re-start postfix):

smtp_use_tls = yes

However, now I am stuck (and currenting giving up) on this error (in "/var/log/mail.log"):

Nov 22 21:56:53 kungfu postfix/smtp[8519]: warning: Only sdbm: type allowed for btree:/etc/postfix/smtp_scache
Nov 22 21:56:53 kungfu postfix/smtp[8519]: warning: Could not open session cache btree:/etc/postfix/smtp_scache
Nov 22 21:56:54 kungfu postfix/smtp[8519]: Peer verification: CommonName in certificate does not match: smtp.gmail.com != gmail-smtp-msa.l.google.com
Nov 22 21:56:54 kungfu postfix/qmgr[8516]: warning: premature end-of-input on private/smtp socket while reading input attribute name
Nov 22 21:56:54 kungfu postfix/qmgr[8516]: warning: private/smtp socket: malformed response
Nov 22 21:56:54 kungfu postfix/qmgr[8516]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description

Note that the naively setting "smtp_tls_enforce_peername = no" in "/etc/postfix/main.cf" did not work.

I have a Macbook that is my main machine and a Mac Pro that is used for running most of our Komodo builds and some of the heavy log analysis scripts that I run. ↩

moving this blog to blogger

2009-10-22T14:03:00.000-07:00

Meta note: I'm moving this blog to blogger (http://trentmick.blogspot.com/). Just test content there now. Hope to have that done in the next few days. If everything goes well, redirects should just make everything work.

Update: Mostly moved. Just working on redirects now.

Hannah Catherine Mick

2009-07-27T15:39:00.017-07:00

Hannah Catherine Mick

Originally uploaded by trento.

I've been remiss in not blogging about my new daughter until now... tada! A lovely little girl. 7lb 4oz. Born on July 4th. All the appropriate bits. Everyone healthy. At times ominous hospital experience, but all good in the end. Ewan's (her almost 2 year old brother) latest thing is to kiss her on the forehead repeatedly. Big wet ones.

pleasant things work better

2009-06-04T13:57:00.009-07:00

I watched Don Norman's TED Talk "3 ways good design makes you happy" last night:

This part of his talk really stuck out for me (transcription errors mine):

I really had the feeling that pleasant things work better and that never made any sense to me, until I finally figured it out. Look:
I'm gonna put a plank on the ground. So imagine I have a plank about 2' wide and 30' long. And I'm going to walk on it. See I can walk on it without looking, and go back and forth, and I can jump up and down. No problem. Now I'm going to put the plank 300' feet in the air... and I'm not going to go near it, thank you. Intense fear paralyzes you. It actually affects the way your brain works.
...
If you're happy, things work better because you're more creative. You get a little problem, you say "Ah, I'll figure it out. No big deal."

Don Norman's ideas may be more relevant to industrial design, but I think this can safely be applied to software design (more my domain). Don points to how being fun and beautiful can make something better -- not just seem better, but work better. That's an important point in the common trade off in software development between spending more time on features or bugs vs. spending some time on making a UI "pretty". Occassionally in work debates -- whether on Komodo or on some of the ActiveState websites I'm involved in -- that "pretty" is said dismissively. I'm happy to have Don Norman's talk as a debate point.

Miro

BTW, thanks to stephen. A few weeks ago I reinstalled my Mac book and posted a list of the software I use. Stephen suggested I add Miro to that list. Cue many evenings of watching TED talks (including the above talk) and other programs.

After installing Miro, visit https://miroguide.com/feeds/2014 to get the latest TED Talks.

Trent Mick

who knew path joining differed so between Python, Ruby, Node, Perl

Komodo 6.0 Beta 2: HTML 5, CSS 3, Python 3, DB Explorer, ...

eol.py 0.7.4 -- Python 3 support

Where?

What's new?

What is 'eol'?

Command line usage

Module usage

quick hack how to move a part of a Mercurial (hg) repo to git

To improve on

On a bit of confusion

django-markdown-deux Django app

What's with the "deux" in the name?

Quick Usage

markdown template filter

markdown template block tag

Donating blood in downtown Vancouver

June Vancouver Python User Group (VanPyZ) this Tuesday at ActiveState

python-markdown2 1.0.1.17

Where?

What's new?

What is 'markdown2'?

Module usage

Command line usage

eol.py -- a tool for working with text file end-of-line characters

alternative Komodo doc icon for Mac

March VanPyZ meeting tomorrow at ActiveState

PyCon 2010 wrap-ups

On fighting in hockey

updated Python Translation Party: fewer booms

other Python VM's upcoming Python version plans

Python Translation Party

or "Why Jesse Noller owes Trent Mick a beer"

but this post is about fun

February VanPyZ meeting at ActiveState on Tuesday

nice I ♥ Komodo blog post

VanPyZ meeting at ActiveState tomorrow (Tues)

What

Where

When

Small Django patch to add full traceback for wrapper exception during template rendering

using gmail for outbound email on Mac OS X

Why?

How?

A couple thoughts

Update (23 Nov 2009)

moving this blog to blogger

Hannah Catherine Mick

pleasant things work better

Miro

`markdown` template filter

`markdown` template block tag