Cal Newport explores possible reasons for the inverse relationship that sometimes exists between getting stuff done and the technology that Newport quotes a Pew report describing as “the main digital artery that workers believe is important to their job”. In another post, specifically looking at the US Apollo program, Cal speculates:

[My] guess is that if we could go back in time and outfit the Apollo engineers with e-mail terminals, two things would have happened. First, their work lives would have become more convenient. Second, it would have taken them longer to get a man on the moon.

For grins, I once analyzed possible connections between email use and the productivity of another prolific brain, Donald Knuth. Over a 53-years-and-counting career (which I took to start in 1963 with the completion of his doctorate) during which, among many other things, he published several volumes of The Art Of Computer Programming (TAOCP) and developed a minor piece of software known as TeX, he used email for only 31% of that time. He started using email in 1975 but then stopped again in 1990, explaining that his work required “long hours of … uninterruptible concentration”. Knuth doesn’t say it but it’s probably relevant that 1990 marked the beginning of the AOLer invasion into and resulting demise of Usenet and the end of the early Internet’s glory days of the 1980s. In other words, if you wanted to pick a year when chucking email made sense so as to avoid the explosion of distracting mediocrity that is much of the modern Internet, 1990 was probably it.

The other 69% “non email years” were split between 12 years of his early career which saw him publishing the first three volumes of TAOCP, and the 24 years since “retiring” at age 54. That retirement has seen publication of the seminal work on Literate Programming, plus ongoing work on the rest of TAOCP. Again, correlation isn’t necessarily causation, but the fact that someone as productive as Knuth was afflicted by email for less than a third of his career can’t be pure coincidence.

[Addendum, 2017-05-11: Just noticed that I inadvertently left scope for people not getting a joke. Just in case anyone is in doubt, let me be as clear as the style of Douglas Adams allows: The piece of software known as TeX is minor in the way that Anatolia isn’t.]

[Addendum, 2017-05-12: Sigh. Too obscure? OK, fine. TeX rocks. Knuth rocks. In general, we’re not worthy. All hail Donald. (No, not that Donald!)]

So it turned out the job required two people. Technician A arrived first, to see what the problem was and I immediately picked up on two aspects of a poor initial impression. I wasn’t looking for them, but they became apparent very quickly. First, he rarely looked me in the eye. There was nothing shady about him; he just didn’t look comfortable doing what he was doing. Second, and much more important, he didn’t put on the little carpet-protecting plastic booties that all other techs from his company had done in the past. OK, small unimportant things. Maybe. He then proceeded to investigate and came up with a (correct) diagnosis. He offered me two solutions; replace a component (cheap but possibly band-aid-like), or replace the entire 20-year-old system (expensive but thorough). Unfortunately, he offered me no help in deciding, and so when I chose to go with the cheap solution (since as a non-expert I was unable to justify the more expensive route), I was left feeling uneasy (because although I’m non-expert, I like to pretend that we engineers can understand anything technical ). So much so that after a weekend of musing, I called the company to ask for a second opinion. Enter Technician B.

1. Consistency of standards is as important as height. The fact that the two techs acted differently, bootie-wise, undid some of the good impression made by Tech B. I’m not saying it would have been better had neither worn booties, but the fact that the two techs acted differently made me judge not just the bootie-less Tech A, but also the whole company. The lack of standards that are not only high but consistently so, was a black mark
2. Attitude matters. Neither of the Techs had a “bad” attitude, but Tech A’s uncertain demeanor set him up for increased criticism when he later failed to offer me advice on choosing a solution.
3. Upselling is neither good nor bad, but the client’s needs make it so. Providing a cheap soution is BAD CUSTOMER SERVICE if the best thing for the client is to buy the more expensive offering. The medical doctor analogy is useful to bear in mind as a guide. If someone goes to the Doc and complains of heartburn, the Doc is doing no one any good by not pointing out that the symptoms are more consistent with esophegal cancer (if, of course, they are).
4. Little things matter, like the bootie-wearing. In an ideal world, maybe they shouldn’t. Maybe we should never judge books by their cover. But, you know, inept design in a book cover can, by signaling lack of money through lack of serious publisher interest, act as a proxy for poor quality. Of course, a tradesman wearing booties actually does have a purpose in that it stops me getting an earful from my wife when she sees his mucky bootprints on the carpet. But more than that it, like the book cover, can act as a proxy for quality, attention to detail, and piece of mind, all of which the purchaser of high grade Professional Services deserves.

First, let’s take with a pinch of salt the current flavor-of-day idea that “10,000 hours of deliberate practice” is both necessary and sufficient to achieve “world class performance”. Even K. Anders Ericsson, one of the academics involved in the research upon which that idea tries to sit, has gone to the length of writing a rebuttal article (MS Word), to try to tone down some of the hype. But let’s at least consider, for argument’s sake, the following as unobjectionable.

To get Really Jolly Good at something (i.e. let’s forget the “being as good as Mozart”, or even the lame “world class” superlatives — we’re just talking about being good enough that the rest of the world can’t ignore us) we should bear in mind that:

1. Innate “talent” in the form of something like a gene for aptitude in Java or C++ is not, within broad limits, particularly important
2. What is important is you have to do the thing a lot. Not necessarily 10,000 hours — maybe a lot less, or a lot more — but you can’t get to be Really Jolly Good without putting in a Decent Chunk of hours
3. You can’t merely “do” the thing. This is the “deliberate practice” component. You have to push hard at it, work at the edges of your ability, get good feedback from an effective feeder-back (a.k.a. teacher/coach/mentor), and generally have an attitude that a Scottish friend of mine once called “borderline mental”.

Now at first sight, the above may appear to be blindingly obvious — work hard at something if you want to improve — but I think that is superficial. What I’m interested in here is what that last item looks like up close, because the more I look the more I reckon that the kind of deliberate-ness required for excellence is not at all like regular, common-or-garden hard work. And if there is anything innate in all this stuff — and the more I look, the more I think there is — it’s not “talent” per se, but my friend’s “borderline mentalness”. To get really good at stuff you have to do it a lot, and you have to do it in a way that most people just wouldn’t tolerate doing themselves. In other words, by definition you’re going to take diligence and hard work to a level that many would consider excessive, and just plain weird. So what exactly does that weirdness look like?

As I continue to read around this topic, and experiment with my own working on stuff, so far I can, on the surface, identify two  broad classes of weirdos. There are those who “love their work”, and those who are “acquainted with pain”. But I say “on the surface” because I think the two components typically arise together. That said, my main interest here is with the second aspect, if only because I think the first one is already being beaten to death in the popular literature to the extent  that many people are being left with the overly simple conclusion that all you have to do is “find your passion” and you’ll be happy every working hour of the rest of your life. I think that’s hokum partly because happiness is a far bigger project that can be fulfilled at work, but specifically because I think that finding one’s passion (even if such a unique-sounding thing existed) doesn’t appear to mean endless working bliss, not if the accounts given by Mason Curry in “Daily Routines” are to be believed. Many acclaimed authors, engineers, artists, and scientists report sustained pain and effort in maintaining endeavours (even though the pain is sometimes punctuated by periods of intense joy). So, if not bliss-inducing passion, then what? What is the deal with “acquainted with pain”?

And so here’s the Pressfield link. Two of the broad areas on which he writes are: how to be creative (e.g. “The War of Art“, “Do The Work” — both highly recommended); and historically accurate military fiction. One of the latter is “Gates of Fire“, a novel about the Spartan/Greek defense of Thermopylae against the Persians. I’ve often used military language in describing mastery, and Pressfield does the same in connecting actual warfare with the war he maintains goes on within an individual as they try to push themselves into their creative work. Pressfield, however, does it far better than I do.

To explain further, first observe that probably the prime reason for the popularity of the habit books, and many others on personal development in general, is that many of us desire behavioural change but that such change is simply hard. Creating a positive habit such as taking regular exercise is difficult, as is stopping an unwholesome one like eating too much, and the typical solutions to those things — the statement “so just take regular exercise and eat less” — is both correct but next to useless as practical advice[1]. The fundamental reason most overweight people have that problem is not that they eat too much but that some part of them wants to eat too much. It’s as if there are actually two persons inside the one individual body, one saying “resist” and the other dangling images of BBQ ribs in front of the eyes (and nose…). And of course that notion of the divided self is ancient and venerable. A recent treatment of it by Jonathan Haidt, in his “The Happiness Hypothesis”, in which he pictures the two selves as an elephant and rider, is superb. The key point here is that it’s possible for an individual to be intellectually aware of the benefits of performing behaviour B and not performing behaviour C, and yet in practice do precisely the opposite. The age-old battle between our higher and lower brains continues, and the amygdala often wins. Habituation is a method whereby we can make that battle go more often in our (overall) favour.

My observation is that that gap, between knowing what is the right course of action and then actually taking it, seems to occur not just with individuals but at an organizational level too. In software development, for example, certain processes and standards have been developed over the years. Now although some do constitute fairly useless overhead, many represent serious distilled wisdom and are a pretty sure path to better product development. However, even in the case of the latter, it seems it’s not enough for an organization to “know” that behaviour B is correct. For example, it’s not enough to know that certain behaviours in code configuration management — committing changes regularly, working on the appropriate branch — are wise. In our experience in Verilab, and that’s covering many hundreds of large and complex projects across a wide range of clients, project styles, and application domains, many if not most clients already know what is the “right” behaviour, but only a subset do it regularly.  Project planning is another area subject to this organization “divided self” effect. Few if any groups would say that planning your chip development or the highly complex verification sub-project was anything other than essential. But the number who actually plan effectively and, more to the point, persist in effective planning and management discipline throughout the project, are far fewer.

And a telling observation is what happens if these issues are raised with a client. In the same way that if you ask me, “Do you really think eating that extra slice of cheesecake is wise?”, I’ll hang my head, frown, and sigh a pouting begrudging “No” as I put down my fork, cheesecake piece uneaten, so too will the typical chip lead hang his head and sigh if asked about their team’s lack of good practices in their chip flow. And the thing is, he and his team already know about those practices, just as I know that eating too much cheesecake is not good practice. The problem is not that we don’t know. We act “badly” despite knowing otherwise. That’s why we sigh! Over the years of personal fighting with cheesecakes, and watching my clients fighting with complexity in chip design, I’ve concluded that neither of us are bad people. Neither of us are stupid. Our problem is our failure to deal with the fact that both as individuals and, it seems, as teams, we are divided selves. In each case there is a higher brain that knows the right behaviour, but there is also a lower brain that has its own ideas, is stubbornly immune to the higher brain’s reasoning, and is extremely powerful. Whatever the solution is, it’s going to have to deal with that fundamental nature. The higher brain merely yelling at the lower is no more effective than an elephant rider yelling at his ride when the big animal has spotted some nearby food.

So, the answer? You’ll get no such silver bullets, lies, or marketing bullshit from me here. For now, this remains simply one of many pieces of what are effectively field research that my Verilab teams and I do as we help clients across the planet, understanding these challenges common to almost everyone, seeing patterns emerge, and hunting for solutions. We also link back to academia, especially groups working on the psychology of learning and performance, and on programming mastery in particular, to ensure we’re feeding our thought processes from the theoretical and scholarly side too. So no easy answers, but here are three thoughts or pointers.

First, let’s keep in mind exactly what we’re looking for here in building a habit. A key characteristic is that it’s a behaviour where it has become more normal for us to do it than not to. It may even become uncomfortable not to do the habituated action. At the extreme end of habituation — addiction — the discomfort of not obtaining the drug or cigarette can be so intense as to drive the individual into what is clearly self-destructive behaviour. But further down the scale, and more positively perhaps, we have regular runners who report feeling “antsy” if they don’t get out for their morning 3 miles. An example we can almost all relate to is the use of seatbelts in a car (hat tip to my friend Tony Chen for that example). That feeling of unease one gets when driving off having forgotten to fasten one’s car seatbelt is a sign that seatbelting has become habituated for you. And a good thing too! So you will know you are building an organizational habit when your “organization” starts getting grumpy when the action in question doesn’t get done. What is the organizational equivalent of grumpiness in an individual? I’m not sure, but I suspect it has something to do with where even junior people are allowed, expected to and regularly do raise red flags. If when, on a late Friday afternoon, with traffic heavy outside, and everyone anxious to get home, Joe Newbie pipes up “Hang on, we didn’t check in and kick off the weekend regression” and the response from everyone else is (a non-ironic) “Darn! Thank goodness you spotted that Joe!”, then you may be heading in the right direction.

Second, related to the above, Atul Gawande reports on significant developments in this area in hospital medicine. In fact it was Atul’s books that first alerted me to the idea that organizational habituation may be a thing worthy of a name. I strongly recommend first his “The Checklist Manifesto”, and then his broader “Better”. I suggest that order only because despite me saying there are no easy answers, Gawande’s checklisting may come close to that. In particular, checklisting has proven useful to empower nurses and junior medical staff to challenge the authority of hitherto unapproachable surgeons, to the betterment of performance all around. A particularly interesting point he makes is the usefulness of “positive deviance” (PD) when trying to transform groups of people. The phrase, not coined but probably given wider exposure by Gawande, refers to groups of people, within a larger group, who perform in some way at odds with the main group but in a way that is beneficial. An important aspect of such groups in terms of organizational habituation is that they can act as fruitful “seed points” from which to spread the beneficial behaviour. As consultants, we see this at work where it’s clearly much easier to help a small group already embracing a new behaviour to spread their wisdom throughout a client, as opposed to us arriving on parachutes to tell everyone to Stop What You’re Doing And Listen To The Consultants! Even purely within a firm, I suspect a key tactic is for management to figure out ways to be able to hear of such PD behaviour, and to develop at least a tolerance for it. That goes, however, to deeper cultural issues than I can cover here.

Finally, a negative pointer if you will. Almost every business talk I’ve been to about improving some aspect of a firm talks about the “what to do” where the target is one of the common areas of business operations — finance, HR, innovation, and so on. Having been working now for over 25 years, almost half of which as a CEO, I don’t think I’m being immodest to say that the chances of someone telling me something in those areas that I haven’t heard before is not high. Despite that, I remain far from the perfect CEO. In other words, just as with the overweight cheesecake eater, the problem for the beleaguered, battle-worn CEO is not that her firm doesn’t do X, it’s that her firm doesn’t yet “want” to do X. A part of the solution is, I propose, the organizational equivalent of habituation. The solution to getting a firm to do X is rarely, even if wrapped up in cheesy fighter pilot analogies[2], merely to tell them that they should do X.

*** ADDED 2014-04-11 13:50***

Funny how despite hunting around prior to writing something you find nothing, and only afterwards get some leads. Where “organizational habituation” picks up very little on Google, “organizational habits” does. Here’s the transcript of a 2012 podcast interview with Charles Duhigg, author of the well-known “The Power of Habit: Why We Do What We Do in Life and Business.” He speaks about habits at the personal and organizational level. Still no silver bullet answers mind, but at least the problem is being recognized. (Although in that case, why do so many business speakers appear to ignore it?)

[1] I have two simple test for diet books in a store. First, I pick up the book and look at the cover. If it shows a picture of someone so ripped and chiseled they clearly have never been overweight in their life, I return the book to the shelf. Weight control lives in the mind, not in the body, and unless a person has fought and won their own mind battles I very much doubt they understand and can help my problem. If it passes that first test, I then split the book around 3/4 of the way to the end and open it. If I find recipes, I return the book to the shelf. My thinking — backed now by experience — is that the recipes are designed to be in the range of 1250 to 1750 calories per day, being the typical range people need to consume to lose weight. In other words, no matter what the front 75% of the book has said, in fact it amounts overall to yet another “all you have to do is consume fewer calories than you emit”, and that phrase fails in its first “all you have to do”. There is nothing “all“, “merely” or “simply” about behaviour change.

[2] That is a link. It is absolutely not a recommendation.

The real tragedy of Oppenheimer’s life was not the loss of his security clearance but his failure to be a great scientist. For forty years he put his heart and soul into thinking about deep scientific problems. With the single exception of the collapse of massive stars at the end of their lives, he did not solve any of these problems. Why did he not succeed in scientific research as brilliantly as he succeeded in soldiering and administration? I believe the main reason why he failed was a lack of Sitzfleisch. Sitzfleisch is a German word with no equivalent in English. The literal translation is “Sitflesh.” It means the ability to sit still and work quietly. He could never sit still long enough to do a difficult calculation. His calculations were always done hastily and often full of mistakes. In a letter to my parents quoted by Monk, I described Oppenheimer as I saw him in seminars:

He is moving around nervously all the time, never stops smoking, and I believe that his impatience is largely beyond his control.

If Dyson is right in his interpretation of Monk, it seems to suggest that an Oppenheimer today would receive a diagnosis of Attention Deficit Disorder. Of course, few would argue that Oppenheimer was anything other than a brilliant, effective, and important contributor, so did his lack of Sitzfleisch actually constitute a problem? Perhaps he succeeded in soldiering and administration to the extent Dyson acknowledges not in spite of but because of the very haste that hurt his mathematics. Perhaps a more focused scientist would have produced more physics but, crucially for the war, less fission. But Dyson’s point is, I think, one of tragedy: a great mind that could have been greater; depth that could have been deeper. Judging by the review’s closing comments, it sounds like Oppenheimer may have agreed:

Late in Oppenheimer’s life, when he was sick and depressed, his wife Kitty came to me with a cry for help. She implored me to collaborate with Robert in a piece of technical scientific work. She said he was no longer doing science and he needed a collaborator to get him started. I agreed with Kitty’s diagnosis, but I had to tell her that it was too late. I told her that I would like to sit quietly with Robert and hold his hand. His days as a scientist were over. It was too late to cure his anguish with equations.

[1] Dyson, F. (2013, August 15). Oppenheimer: The Shape of Genius. [Review of the book Robert Oppenheimer: A Life Inside the Center, by Ray Monk]. The New York Review of Books, p. 19.

So, we know that at very low costs, you may get someone who is just utterly useless for your purposes. They may not be fundamentally broken — perhaps they’re a basically smart but inexperienced graduate; we all were once — but while they’re at that early stage of learning they are barely worth the paper with which you have to regularly wipe their runny noses. However, the good thing is they improve quickly from those tenderfoot days. So start spending more, to buy more experience, and you get a little bit of improvement. Spend yet more and the increase gets even bigger. Something like this:

Now it doesn’t really matter what parameter we’re using for “contribution”. It’s just some overall measure of “rate of effective output”. I say “effective” because we all know that something as crude as, for example, “lines of code” is rarely, on its own, a useful measure in software. But despite the difficulty in putting a precise number on it, effectiveness is a real thing, it varies from person to person, and, all else being equal, it increases with time served.

The next thing to notice though is that while effectiveness versus hourly rate is superlinear at low hourly rates, the effect eventually slows. Effectiveness versus rate can even become sublinear at higher rates. Like this:

1. How tough is the proposed work? This lets the client and you, together, identify which of the family of “Optimal Hourly Rate” curves applies to their project
2. Does your your firm have resources to offer at the low point on the chosen curve?

If in investigating those you find that the optimal resource is at a cost far below where you operate, you — remember, this is “Professional” services — should be telling your client that you are not the right solution. On the other hand, if you do decide that you are in the sweet spot and they still insist on going to someone cheaper then, provided you have built sufficient rapport in your relationship with your client, you are only doing them a service to try to explain (tactfully) that if they do they will most likely end up spending more overall.

One final point. Notice the asymmetry in the final curves. Total cost increases more quickly as you go down in rate, away from the sweet spot, than it does when you go up in rate away from it. So buying cheap can be more damaging than buying expensive. But in fact, buying cheap is worse even than that. One of the biggest dangers for a project is being late to the market. Even if it still makes it in time for some profit, the damage from missing the core of the market window can be devastating. Labor that is too cheap not only costs more, but it takes a lot longer in time to complete the job. So if you have to err, it makes sense to err on the higher side of rate and capability, than on the cheapside.

[A version of this post appeared in an earlier blog]

It’s probably useful to define creativity, because it could be argued that engineers — people who “engineer” things — are creative by definition. I don’t think so. Creativity is the ability to make something, change something, or do something that creates positive surprise in one’s peers. It goes above and beyond just “the new”. It must elicit, from those who appreciate and understand your field, an appreciative “Huh!?”

Now it’s arguable I’m describing mere Regular Creativity, the kind that pretty much all of us could achieve if we wanted to. I accept that there may be a High Creativity, such as that of van Gogh, which is, by it’s nature, so surprising that even the peers of a High Creative (pun not entirely unintended) will not recognize the merits of the new work until long after its birth. But if you genuinely feel you may be a legend as yet unrecognized in your own lifetime, I haven’t much in the way of advice for you. I’ll just wish you well, and offer only the observation that the reason many of those who live their lives unrecognized as legends is simply that they aren’t.

To the rest of us, we need to understand that there’s a difference between copying and creating, and to accept that for many if not most of us what we do is the former. It’s tempting to think that because every day we sit down in front of our editors and produce code that no one else has written, that we are creating. But really, where did that code come from? Sure, we don’t copy it word for word, line by line, or even function by function. But it didn’t really come from us. Legally, we — or more likely the corporation that employs us — can stake a claim to ownership, but that’s not the kind of novelty we need if we want to call ourselves creative. The law doesn’t get to decide if we are are or aren’t. Other creatives do. And they do it by kudos — by a fist bumping, high fiving, noddingly approving “Kewl!”  If that’s missing, then so are we with respect to our creative mark.

Of course, creativity doesn’t have to be entirely new. Most creation is the combining, or building onto of prior art. That’s one of the major downsides of the tough “intellectual property” system we see arising all around us, where such combining and building upon is heavily restricted. But where it’s possible, there’s no shame in taking (while acknowledging) the work of several others, and then combining in a new way that produces new value. As I say, it may not be the higher form of creativity; it’s partly in an attempt to cultivate that higher form that schools such as SCAD spend the first year or so tearing down new students’ preconceived ideas about what constitutes art. Only once they have unlearned to colour within one set of lines are the developing creatives able to see that there needn’t be any lines, or to ask “What is a line anyway?” But even modestly, quietly, and incrementally; we can still create. It doesn’t have to rock the entire world. Besides, modest, quiet, and incremental changes may in themselves be part of the path to the creative heights.

But why is this important? Because if we, we engineers and programmers, want to create — to produce the “Huh!?” — there are things we can and should do to make it happen. For a start, we can learn from those who more commonly receive the name “creatives”: artists, writers, composers; also, designers beyond our regular “chip design” sense — those who close their eyes, open them, and then create artefacts that can, if we are careful, enhance lives. Read about them, learn from them and see that, ultimately, we and they are the same people. At our heart, we build because we love to build, the best of us code because they are poets. Whether it’s sunflowers in an old pot, or charges crossing a precisely constructed semiconductor junction, we first envisage, then mould a world.

One is mentioned by philosopher-turned-mechanic, Matthew Crawford in his “Shop Class as Soulcraft“. In chapter eight, “The Further Education of a Gearhead: From Amateur to Professional” he tells the story “Of Madness, a Magna, and Metaphysics” in which he takes on the task of bringing back to life an old and neglected-by-underuse 1983 Honda Magna V45. A key part of the repair was fixing the clutch hydraulics.

As he burrows in through the layers of grease and grime, he encounters a suspicious oil seal that he suspects is the culprit. But he can’t be sure without a lot of extra work. This triggers a debate within himself as to the sense of digging into that oil seal when he knows he can perform a reasonable if temporary fix by simply focusing on the slave cylinder:

“It occurred to me that the best decision would be to forget I’d ever seen the ambiguously buggered oil seal. With a freshly rebuilt slave cylinder, the clutch worked fine. Even if my idle speculation about the weeping oil seal causing the failure of the slave cylinder was right, so what? It would take quite a while for the problem to reappear, and who knows if this guy would still own the bike by then. If it is not likely to be his problem, I shouldn’t make it my problem.”

Note that he’s not idly trying to avoid work. His concern is for the client (who’ll have to pay for the extra work), and for sheer economic sense in to the bargain. But there’s more at work here than simple economics:

“But as I walked back into the fluorescent brightness of the shop, I wasn’t thinking about the owner, only about the bike. I just couldn’t let that oil seal go. The compulsion was setting in, and I did little to resist it.”

It’s that word “compulsion” that intrigues me. Only sentences later he mentions it again:

“There is something perverse at work here, and I would like to understand it. The oil seal was the opening to Pandora’s box: I felt compelled to get to the bottom of things, to gape them open and clean them out.”

Not money, not because the boss or client wants it; a compulsion, and thinking only about the bike. I agree with him; it’s perverse, and I too would like to understand it. And his inner conflict, and outright guilt at the perversity of it is clear:

“But this lust for thoroughness is at odds with the world of human concerns in which the bike is situated, where all that matters is that the bike works… [The] more … pragmatic view of the motorcycle … grounds the fiduciary responsibility of the mechanic to the owner. In digging at that oil seal needlessly, I was acting out of some need of my own.”

But if he didn’t intend to be ironic there, he should have. And that’s my point. It is the very compulsion he demonstrates that I believe is a Proxy For Greatness. I see it (and its sad opposite, a robotic lack of compulsion) all around in software.

The Proxy — the sign you may be standing in the presence of programming greatness, or the potential for it — is the tendency in some individuals to write clean, well-nigh poetic code not because it’s useful (although it usually is) or reusable (ditto) but because they cannot not write such code. They are compelled to do so. They avoid, where possible, writing the same lines of code in more than one place not because a coding standard tells them not to, but because they are personally compelled to be succinct. Saying the same thing twice is icky — it feels bad. Or they look on existing code, and cannot help but notice that those four “different” functions are really the same function. The itch to refactor develops, and may well become irresistible.

The correlation is not 100%. There are potentially fine programmers out there who don’t have this instinct but can learn it. And I’ve met a few who go too far to the extreme, sacrificing all real concerns in pursuit of an elusive platonic form of every program. But I reckon it’s a very strong predictor. And I’d recommend erring on the side of too much of it than too little.

[Published earlier in a previous version of this blog]