Today, though, one of them emailed me, asking for what he should be studying and reading up on in the weeks before he joins us. He was planning to brush up on his Python and Django, but he was looking for suggestions. I thought about this a bit, and told him this:

First, rest up. Take a nap, or a walk around the park. Look at the grass and check out the breeze. Yes, really. It’s easy to wear yourself out in this line of work, and the first few weeks in particular are going to be challenging. You’ll be working different mental muscles that you were previously, and you’ll be meeting lots of new people. That takes a lot of energy. And it’ll be easier if you’re well rested.

When you wake up from your nap, if you still feel like doing preparatory work, here are a few suggestions:

One: Know what text editor you love and make sure your skills at using it are well honed.

You’ll be spending a lot of time in your editor, knowing it back and forth is a good thing. When I came to Eventbrite, I got to stop managing and write more code. And I realized that all the things that had been merely Emacs annoyances were suddenly really important to fix. I took the tutorial (M-x tutorial, natch), which actually was super helpful even though I’d been using it for years.

In particular, if you’re coming straight out of college, you’ll be working with more files at one time than you probably have in the past. Knowing how to find something in a file (and not necessarily knowing what file it’s in) will be useful. (And again, if you’re using Emacs, grep-find is your friend.)

[NB: If you want a longer treatise of why your editor matters, read The Pragmatic Programmer.]

Two: git.

We’ll give you some basic training on git if needed (and on how we use it here), but knowing the basics will put you ahead. If you want a reference, you could do worse than Chapters 1-3 of Pro Git. Remember: it’s not a set of files, it’s a graph of changes.

Three: Read some code

You’ll spend as much time reading code as you do writing it, if not more. Find a project and read it until you understand how it works. Better yet, find a project that’s a little, ahem, imperfect. A few suggestions based on things I’ve worked on recently-ish:

• Hieroglyph (Follow your nose down into Sphinx and figure out how .. noslides: works.)
• MediaGoblin (How is meddleware used? How’s it different than Django’s middleware?)
• OpenHatch (How would you take the new mission framework and make it work for things like the SVN mission?)

Really, just practice reading code.

And four, explore the poetry of Billy Collins, a former poet laureate. I suggest the CC-licensed album of him reading his own work, The Best Cigarette. Specifically “Another Reason I Don’t Keep a Gun In The House”, “Marginalia”, and “The Best Cigarette”. Maybe it’s just me, but listening to Collins calms me down and puts me in that space where I can be calm, creative, and make great software.

So there you have it, my advice for the new guy. If I had to sum it up, it’s “tools and skills, not technologies”. And poetry.

One of the things people don’t always know about Creative Commons is that there was a large technical component undergirding the licenses. Every license was prepared for three audiences (in talks, this is where I call them disjoint, in a lame attempt at humor): humans (the license “deed”), lawyers (the legal text), and machines. The output for machines was an RDF model of the license: it’s permissions, requirements, and prohibitions. In 2008 we had a technical all hands meeting where the tech team came to the San Francisco office for a week. At that time porting (preparing a license for a new legal jurisdiction and translating the web tools) was in full swing, and as we talked about what the pain points were, launching these new jurisdictions came up as a major source of pain. As we started drawing the model of how things worked on the site, I arrived at the following diagram.

We had at least three different “products” — the license chooser, the API for 3rd parties, and the prepared licenses (deeds and RDF). And for hysterical and historical reasons, they didn’t really use the same information. Well, they did at a certain level: they all used the same translation files, but after that all bets were off. We had the “questions” used for selecting a license modeled as an XSLT transformation (why? don’t remember; wish I knew what we were thinking when we did that), but the transformation needed to have localized content, so we would generate a new XSLT document from a ZPT template (yes, really) when we updated the translations. The license RDF was stored as static files for performance, but there was increasing pressure to provide localized data there, too, which was going to cause a world of hurt. And the chooser had a thin wrapper, cc.license, around the XSLT. Except when it went directly to the XSLT for special cases.

If you look in the upper right hand corner, you’ll see something labeled “cc.licenze”. This was a prototype library I had written when adding support for CC0 to the site. The idea was this: We claim that the RDF is the technical model for our legal tools. If that’s true, can we put enough information in it to drive the entire process, and have a single source of information? After launching CC0, signs pointed to yes. We set out to build a glorious future.

We’d build a single wrapper around our RDF and use it everywhere. We’d update one thing when we launched a new jurisdiction, and all the changes would flow to all parts of the site. It sounded amazing. The thing is, we were talking about moving our core infrastructure — our house — to a new foundation, but that foundation wasn’t built yet. We hadn’t really even figured out if it’d support the house or not.

Undeterred, an engineer set out to start building out “cc.licenze”, filling in the gaps I’d left to make it do all the things that licenses need that CC0 does not. And he got most of the work done, and then he left. So the work languished while we focused on continuing to ship new jurisdictions and do everything an understaffed technical team has to deal with.

The problem isn’t that we wanted to improve our underlying infrastructure, or that we wanted a coherent and consistent model. Those are the right goals. The problem was trying to build an entirely new foundation, with similar but not exactly the same APIs as the original one, and thinking we were going to slip it in. Starting this project today, I’d look at the three ways we were doing things, find the one that had the least debt, and rebase the other services/products onto it. By choosing one currently in use, any improvements made (either by rebasing or fixing bugs) would show immediate benefit. There’s immediate, tangible benefit to going from three ways to do something to two, and from two to one. Once everything uses the same foundation, there’s only one thing to rebuild and replace, not three, and we probably have a better idea about *everything* it needs to do.

To successfully live with technical debt, this is the sort of maneuver you often have to use. I think of this as Lateral Refactoring: you’re not refactoring to the API/design you want to wind up with, you’re tacking along an orthogonal axis until you’re at the point where you can start moving forward again. By doing this you can realize some benefit sooner, and continue shipping new features and bug fixes.

When some engineers — even good (but not great) ones — encounter technical debt, their reaction may be “I can do better”. That is, “Oh, this is terrible, I can’t possibly work with code like *this*, I’ll rewrite this part of the system, and then I can get around to what I came here to do.” Rewriting or refactoring debt away may be the right decision, but this statement contains unspoken assumptions that better code is more important than new features or bug fixes for users. This is where the paradox of living with technical debt first shows itself: living with technical debt does not mean accepting it, but it also doesn’t mean fixing it. Right now. The business, the organization, has to make decisions about what’s most important. (Engineers need input into those decisions, and the business needs to respect that input, or the best engineers will go elsewhere, where their input will be respected.) It’s up to the business to decide “can we go dark for n days/weeks/months.” Sometimes the answer may be yes, and we’re free to improve the code with abandon. I think that’s a rare situation. More often the answer is “no”, so we need to live with the debt and develop strategies for improving it (more on that later).

Another reaction that I think is all too common is “I guess one more won’t hurt”. That is, “Well, we’re stupid is these five places, what’s one more?” Living with technical debt does not mean you continue to incur it. If anything, it’s essential to stop running up the tab. This requires rigor and strength of will, not just on the part of the engineer working on the code, but on her peers. The team needs to decide that incurring additional debt is not acceptable: you can maintain or you can improve, but you can’t backslide. The danger of “one more won’t hurt” is that the problem spreads: you build new features that repeat past mistakes, instead of providing a model for future work.

Finally, sometimes we look at code and think, “I can’t go on”. I find that those are the time it’s helpful to step away from a project, take a break, come back after a good night’s sleep. You don’t always have that luxury, but feelings of despair rarely coincide with my best work. I’ve observed that indulging in the first two ways of thinking — “I can do better” and “One more won’t hurt” — often leads to the final one — “I can’t go on”. “One more won’t hurt” just digs a deeper and deeper hole, until you can’t see your way out, and “I can do better” often leaves you with a piece of “perfect” code that doesn’t quite fit into the rest of the system, leaving you to shims and scotch tape, the very things you started out trying to avoid.

In “Good to Great”, Jim Collins writes about characteristics that separate good companies from great ones. One of the principles he identifies is “Confront the brutal facts, but never lose faith.” In other words, it does no good to pretend that your code (company in his case) is something that it isn’t. Collins talks about meeting Admiral Stockdale, and asking him, “Who didn’t make it out?” “Oh, that’s easy — the optimists.” Stockdale explains that the optimists were routinely disappointed, and eventually lost faith. “I can’t go on.” Collins quotes Stockdale as saying, “You must never confuse faith that you will prevail in the end — which you can never afford to lose — with the discipline to confront the most brutal facts of your current reality, whatever they might be.” Technical debt may be a far cry from Stockdale’s situation, but the principle holds as the heart of truly living with technical debt: we must confront things as they are, not as we wish they were. And we must believe that we can make things better, that we know where we’re going.

I don’t think it’s unreasonable to start with what I mean by “technical debt”. “Technical Debt” is a euphemism, usually trotted out when we’re talking about something we don’t like about software or systems. I say “don’t like” as if the label is undeserved: it’s not always clear when someone says “technical debt” if they’re talking about code that’s obviously difficult to work with, or just makes a different set of choices than they would have made. One definition I’ve been thinking about is this: technical debt is some aspect of your system that increases the cognitive overhead of understanding, improving, and maintaining it. It’s possible there should be a clause added about “for the majority of developers”, too: I know there’s code I’ve written that absolutely minimizes cognitive overhead for me, but the things I’m used to, idiomatic Nathan, makes it harder for someone else to come and fix a bug or add functionality.

By speaking about technical debt in terms of cognitive overhead, we can start to detach ourselves from the situation emotionally. It’s pretty easy to become emotionally involved with the code we write. And usually that’s a good thing: it’s important for me to work on things that feel important, things that I feel like I can leave my mark on. I’d like to posit, however, that it’s possible to become emotionally co-dependent with your code. That may sound like a strange idea, so let me explain: whenever something I create becomes a proxy for my self — my individuality, my self worth — it is nearly impossible for me to see problems with it. It is nearly impossible for me to hear anything but glowing praise. And when I do hear glowing praise, it’s never enough. I’ve observed two different effects of these feelings. First, I start treating situations like a zero sum game: it’s not enough to succeed, others must fail. It’s not hard to see how this would lead to hypersensitivity and hypercriticality at the same time. Second, I don’t make smart decisions: I make them based on my feelings rather than on reality. I don’t know why this would be any less true of code than it is of other endeavors. So to really see technical debt in our systems, we need to detach ourselves emotionally: it’s not about who’s at fault, it’s about how we make it better.

(There’s a whole other topic around team building here; I’m going to assume for the purposes of this discussion that you have the people you want on your team, either because they’re operating at the level you need them to, or because you believe they can grow to that level.)

So what are some ways your system can add to the cognitive overhead needed to understand it? I can think of a few: inconsistency, duplication, and lack of cohesion all immediately come to mind. These all make it difficult for an engineer to understand, maintain, and improve a system. More on that later.

I was reminded of this incident recently, thinking about why sometimes things feel like they flow and I stay with it, and sometimes they don’t and I put it down. I had a long flight and had brought a book with me: “Mastery: The Keys to Success and Long-term Fulfillment”, by George Leonard. A colleague had recommended it to me as we talked about what it means to have mastery of something (in the case of our conversation, software engineering). It’s a short book, about 150 pages, and easy to read. The core thing I took away from it was that mastery is about pursuing a path for its own sake, not for the sake of a grade or a title. The path to mastery involves seeking instruction, practice, surrendering your ego, being intentional about what you want, and pushing yourself just past the edge of what’s comfortable or obvious. I think the most interesting thing about reading Mastery was how it crystallized other things I’ve been thinking about or reading since taking that writing class.

Julia Cameron, in The Artist’s Way, talks about “morning pages”: getting up and writing three pages, even if all you write is “I don’t know what to write”. Surrendering the idea of a polished product, practicing putting pen to paper, pushing yourself.

Annie Lamott, in “Bird by Bird”, talks about “shitty first drafts”: again, surrendering the idea of a polished product, of genius, and committing yourself to work.

And my print-making mentor, Katie Gilmartin, reminding me gently that Picasso didn’t make just one bull: he created the image again and again, practicing, improving his technique. In telling me that she reminds me that it’s not only acceptable to repeat yourself, it’s desirable.

Finally, when we interview engineering candidates, one of the things we look for is what they’re doing outside of their job. Some members of the team look for outside technical projects: are they interested in the field and in learning more about new things. That’s great, but I tend to look for “what are they making”. It might be software, it might be paintings, it might be improv. After reading “Mastery”, I think that I’m looking for a practice. Looking for something they’re doing just because they love it, not because they’re going to get paid, recognized, etc.

I think the path to mastery is the path of having a practice. What do I do again and again, not because I’m going to get an “A”, but because I love it and it feeds me.

The center of our week was a day spent in the DMZ. I have to start with this because I still feel the most uncertain about it. We arranged a day long tour of both the DMZ and the Joint Security Area (JSA). Going into it I had some trepidation: was it going to feel light and airy and unserious? Was the fact that two countries are basically at war going to be glossed over? What does it mean that I’m touring (literally, a tourist) a space set aside by an armistice agreement? I suspect the last question is something I’ll write about more at length, but here’s the high level: the JSA tour was terrifying (to me; R didn’t seem to have the psychological reaction quite to the degree I did). The ID checks, the barbed wire, and seeing the North Korean soldiers observing us with binoculars as we walked along the MDL was not light, was not airy, and definitely did not gloss over the fact two countries are at war. Part of this was Laura, our excellent guide for the morning. As we rode the bus north from Seoul, she sagely pointed out, “1,000,000 land mines! You stay with group today!” And then she looked directly at me. The afternoon tour of the “DMZ” — Dorasan Observatory, Dorasan Station, and the 3rd Infiltration Tunnel — was less compelling, perhaps because it was raining, but probably because we only had something like 20 minutes at each stop. It didn’t help that the afternoon guide, Justin, wasn’t operating at anywhere near Laura’s level. I should note that every stop on the tour has a gift shop. Some of them sell products cultivated in the DMZ (ginseng, rice, soy beans — chocolate covered and not), and some sell truly strange souvenirs (see: a plastic axe, which seems in questionable taste). The presence of gift shops did nothing to help me understand how I feel about touring a (theoretically) hostile line.

Another highlight for me was visiting the Bongeunsa Temple and Gyeongbokgung Palace. (R would insist I point out “gung” means “palace”, so I just wrote “Gyeongbok Palace Palace”, but the ticket says “Gyeongbokgung Palace”, so I’m sticking that.) These are two very different places; I’m grouping them together because they evoked similar feelings for me. Bongeunsa was an easy walk from our hotel (and the Samseong Station, if you’re going to check it out), and yielded some really relaxing time wandering the grounds, listening to the chanting, and smelling the trees. Gyeongbokgung, on the other hand, was closed the first time we tried to go, and when we finally went near closing time Thursday, there were lots of people wandering the grounds. Gyeongbokgung is actually like a park, with many buildings enclosed in it. You can look at the architecture, walk around a pond, or explore the nooks and crannies. Like the temple, this had a really meditative feel to me, and we took lots of interesting pictures.

The last day we hit up Dragon Hill Spa & Resort. Apparently the largest in Seoul (Korea?), I signed up for the “gold package” — body scrub, steam massage, oil acupressure, leg massage, muscle massage, facial massage, head acupressure, and, of course, shampoo. Quite possibly the best ₩110,000 I spent all week. The “scrub” felt like it used a scouring pad, but as I lay there afterward waiting for round two, my skin felt tingly and awake. The acupressure and “massage” were intense: every muscle in my body was pounded, slapped, pulled, and kneaded, and then for good measure the joints holding them together were stretched and twisted. I would say I walked away feeling renewed and refreshed, but I’m not sure I really “walked” so much as “oozed”. We went back to the hotel that evening and slept incredibly well. And when the soreness finally fade, I’m sure I’ll feel even better.

Other things we did, which I may write more about at a later date: Kimchi Field Museum (yes, really), Deoksugung Palace (to, ahem, further the “gung” discussion above, this one is really confusing: the ticket says “Deoksugung Palace”, which the brochure says “Deoksu Palace”), cat and dog cafes (cafes where you can play with cats or dogs while enjoying your beverage of choice), open air markets, random shopping, and great food.

I was putting together notes using reStructured Text and kept thinking that it’d be nice to generate both slides and longer written documentation from the same source. I’ve used docutils’ S5 generator in the past, but was looking for something a little more polished looking. Something like the HTML5 Slides.

So I wrote a hieroglyph, a Sphinx builder for generating HTML5 Slides. I presented hieroglyph at the Sunday morning lightning talks at PyCon: you can see the slides, the reStructured Text source, as well as the HTML documentation generated from the same source.

I’m really happy with the output — it looks great in the browser, projects well, and because I’m using the html5slides CSS, looks great on mobile devices, too. I’m even happier that I’m able to work on my content in plain text. You can find the source on github.

Recently I have been suffering from the delusion that making more commitments will make me more able to achieve them.

My first reaction to reading Asheeh’s reflections on commitments: “Yes, and I wish I’d learned that about ten years earlier than I did.” And then I remember that it’s not something you learn once; tending to your committments — and making them with care — is a life-long practice. Practicing is hard, but it’s preferrable to encountering the OOT killer.

Last night I decided to do a little clean-up on my Emacs configuration, and see if I could get rid of getelget.el. The documentation for el-get is great, so I started there. What I quickly realized is that the included el-get bootstrap mechanism is great if you want to ensure el-get is installed and then use el-get-install, el-get-remove, etc to manage your packages. But if you define your package list in you config file, it’s not quite enough. Specifically, when you first bootstrap your configuration, you want to defer calling (elget 'sync) until you’ve bootstrapped el-get. And on future runs, you want to go ahead and install any new packages that have been added to your list.

Luckily el-get has added support for hooks, which makes life a little easier. The new getelget.el (available here) looks something like this:

;; getelget - el-get boostrap script
;;
;; Checks to see if el-get has been checked out, and bootstraps it if
;; it has not. After bootstrapping, calls el-get to load specified
;; packages.
;;
;; el-get-packages should be defined before including this file. Any
;; definitions from el-get-sources will be appended to el-get-packages.
;;
;; Written in 2011 by Nathan R. Yergler 
;;
;; To the extent possible under law, the person who associated CC0 with
;; getelget has waived all copyright and related or neighboring rights
;; to getelget.
;;
;; You should have received a copy of the CC0 legalcode along with this
;; work.  If not, see .

;; add a hook listener for post-install el-get
(defun post-install-hook (pkg)
  ;; after installing el-get, load the local package list
  (if (string-equal pkg "el-get")
      (el-get 'sync
              (append el-get-packages
                      (mapcar 'el-get-source-name el-get-sources)))))
(add-hook 'el-get-post-install-hooks 'post-install-hook)

;; add the el-get directory to the load path
(add-to-list 'load-path
             (concat (file-name-as-directory user-emacs-directory)
                     (file-name-as-directory "el-get")
                     "el-get"))

;; try to require el-get
(if (eq (require 'el-get nil t) nil)

    ;; urp, need to bootstrap
    (url-retrieve
     "https://raw.github.com/dimitri/el-get/master/el-get-install.el"
     (lambda (s)
         (end-of-buffer)
         (eval-print-last-sexp)))

    ;; successfully required el-get, load the packages!
    (post-install-hook "el-get")

el-get also recommends splitting your package definitions from your local source recipes (which can themselves extend an included recipe). So getelget.el now expects you’ve defined two lists: el-get-packages, a list of packages to install from recipes, and el-get-sources, your local source list.

For example, I define a local recipe for magit that binds a key to magit-status and enables spell checking and fill mode for commit message editing:

(setq el-get-sources
      '((:name magit
               :after (lambda ()
                        (global-set-key "\C-x\r\r" 'magit-status)

                        ;; Enable spell checking, fill for log editing
                        (add-hook 'magit-log-edit-mode-hook
                                  (lambda()
                                    (auto-fill-mode 1)
                                    (flyspell-mode 1)))))
        ))

And my el-get-packages list is just a list of packages I’m installing from the included el-get recipes.

(setq el-get-packages
       '(el-get
         google-maps
         color-theme
         python-mode
         virtualenv
         php-mode-improved
         xml-rpc-el
         js2-mode
         org2blog))

Everything listed in both lists will be installed.

YMMV, FWIW, ZOMG, BBQ, etc.

class A(object):
    def __init__(self):
        super(self.__class__, self).__init__()

class B(A):
    def __init__(self):
        super(B, self).__init__()

The surprise comes when I try to use my sub-class, B. Instantiating B() blows up the stack with:

RuntimeError: maximum recursion depth exceeded while calling a Python object.

What?

According to the Python 2.7.2 standard library documentation, super “return[s] a proxy object that delegates method calls to a parent or sibling class of type.” So in the case of single inheritance, it delegates access to the super class, it does not return an instance of the super class. In the example above, this means that when you instantiate B, the follow happens:

1. enter B.__init__()
2. call super on B and call __init__ on the proxy object
3. enter A.__init__()
4. call super on self.__class__ and call __init__ on the proxy object

The problem is that when we get to step four, self still refers to our instance of B, so calling super points back to A again. In technical terms: Ka-bloom.

TL;DR: super(self.__class__, self) may look like a neat trick, but it’s the end of the line for sub-classing.

Further reading: Raymond Hettinger’s excellent blog post on super provides a great overview of super and shows off the improved Python 3 syntax, which removes the need to write the class name as part of the super statement. I was really pleased to find the Python standard library documentation links directly to it.