Coding Horror

Coding Without Comments

Jeff Atwood — Thu, 24 Jul 2008 17:00:00 -0800

If peppering your code with lots of comments is good, then having zillions of comments in your code must be great, right? Not quite. Excess is one way good comments go bad:

'*************************************************
' Name: CopyString
'
' Purpose: This routine copies a string from the source
' string (source) to the target string (target).
'
' Algorithm: It gets the length of "source" and then copies each
' character, one at a time, into "target". It uses
' the loop index as an array index into both "source"
' and "target" and increments the loop/array index
' after each character is copied.
'
' Inputs: input The string to be copied
'
' Outputs: output The string to receive the copy of "input"
'
' Interface Assumptions: None
'
' Modification History: None
'
' Author: Dwight K. Coder
' Date Created: 10/1/04
' Phone: (555) 222-2255
' SSN: 111-22-3333
' Eye Color: Green
' Maiden Name: None
' Blood Type: AB-
' Mother's Maiden Name: None
' Favorite Car: Pontiac Aztek
' Personalized License Plate: "Tek-ie"
'*************************************************

I'm constantly running across comments from developers who don't seem to understand that the code is already there to tell us how it works; we need the comments to tell us why it works. Code comments are so widely misunderstood and abused that you might find yourself wondering if they're worth using at all. Be careful what you wish for. Here's some code with no comments whatsoever:

r = n / 2;
while ( abs( r - (n/r) ) > t ) {
  r = 0.5 * ( r + (n/r) );
}
System.out.println( "r = " + r );

Any idea what that bit of code does? It's perfectly readable, but what the heck does it do?

Let's add a comment.

// square root of n with Newton-Raphson approximation
r = n / 2;
while ( abs( r - (n/r) ) > t ) {
  r = 0.5 * ( r + (n/r) );
}
System.out.println( "r = " + r );

That must be what I was getting at, right? Some sort of pleasant, middle-of-the-road compromise between the two polar extremes of no comments whatsoever and carefully formatted epic poems every second line of code?

Not exactly. Rather than add a comment, I'd refactor to this:

private double SquareRootApproximation(n) {
  r = n / 2;
  while ( abs( r - (n/r) ) > t ) {
    r = 0.5 * ( r + (n/r) );
  }
  return r;
}
System.out.println( "r = " + SquareRootApproximation(r) );

I haven't added a single comment, and yet this mysterious bit of code is now perfectly understandable.

While comments are neither inherently good or bad, they are frequently used as a crutch. You should always write your code as if comments didn't exist. This forces you to write your code in the simplest, plainest, most self-documenting way you can humanly come up with.

When you've rewritten, refactored, and rearchitected your code a dozen times to make it easy for your fellow developers to read and understand -- when you can't possibly imagine any conceivable way your code could be changed to become more straightforward and obvious -- then, and only then, should you feel compelled to add a comment explaining what your code does.

As Steve points out, this is one key difference between junior and senior developers:

In the old days, seeing too much code at once quite frankly exceeded my complexity threshold, and when I had to work with it I'd typically try to rewrite it or at least comment it heavily. Today, however, I just slog through it without complaining (much). When I have a specific goal in mind and a complicated piece of code to write, I spend my time making it happen rather than telling myself stories about it [in comments].

Junior developers rely on comments to tell the story when they should be relying on the code to tell the story. Comments are narrative asides; important in their own way, but in no way meant to replace plot, characterization, and setting.

Perhaps that's the dirty little secret of code comments: to write good comments you have to be a good writer. Comments aren't code meant for the compiler, they're words meant to communicate ideas to other human beings. While I do (mostly) love my fellow programmers, I can't say that effective communication with other human beings is exactly our strong suit. I've seen three-paragraph emails from developers on my teams that practically melted my brain. These are the people we're trusting to write clear, understandable comments in our code? I think maybe some of us might be better off sticking to our strengths -- that is, writing for the compiler, in as clear a way as we possibly can, and reaching for the comments only as a method of last resort.

Writing good, meaningful comments is hard. It's as much an art as writing the code itself; maybe even more so. As Sammy Larbi said in Common Excuses Used To Comment Code, if your feel your code is too complex to understand without comments, your code is probably just bad. Rewrite it until it doesn't need comments any more. If, at the end of that effort, you still feel comments are necessary, then by all means, add comments. Carefully.

It's OK to end up with comments in your code, but never, ever start with comments.

[advertisement] Peer Code Review. No meetings. No busy-work. Customizable workflows and reports. Try Jolt Award-winning Code Collaborator.

Building Tiny, Ultra Low Power PCs

Jeff Atwood — Tue, 22 Jul 2008 23:59:59 -0800

In previous posts, I've talked about building your own desktop PC, and building your own home theater PC. I'm still very much in love with that little HTPC I built. Not only does it have a modern dual-core CPU, and fantastic high-definition capable integrated video -- it's an outstanding general purpose media sharing server, too. But the real punchline is that I eventually got that box down to an insanely low 44 watts at idle. That's in the ballpark for a powerful laptop, and far better than your garden variety desktop PC, which will draw somewhere between 100 to 200 watts of power.

44 watts is impressive, but what if you want to build a PC that uses even less power -- radically less?

That's when you turn to something like AMD's Geode platform in the Nano-ITX form factor. It uses five watts of power at idle. That's almost ten times less than my HTPC build I was so proud of!

This is the JetWay J8F9 AMD Geode LX800 motherboard. I can't say "this is actual size" with a straight face without knowing the size and aspect ratio of your monitor, but it's probably darn close. The actual dimensions are just under five inches on each side. It may not look like much, but consider the specs:

500 Mhz AMD x86 Geode LX 800 CPU
200 pin SO-DIMM memory slot, 1 GB DDR-400 max
Two ATA-100 drive connections
mini-PCI expansion slot
CompactFlash memory card slot
onboard audio / VGA / fast ethernet / USB

This thing is, for all intents and purposes, a complete, standalone x86 PC that fits in the palm of your hand and sips five watts of power. Well, assuming you have an enormous hand.

You will need memory and a storage device, of course. You could pick up a laptop hard drive, but another clever thing about this board is that it allows you to use a cheap CompactFlash card as your storage medium -- for the optimal low power, no moving parts install.

So we can put together our own tiny utility PC for right at 200 bucks. Not bad. Unbox it, snap in the memory and CF card, plug in the wall wart, and you're ready to install and boot your operating system of choice. It's that simple.

Naturally, you won't get barn-burning performance, but if you remember the Pentium II 300 Mhz systems of yesteryear, you'll know what to expect. You may recall those now-ancient boxes were still able to do some pretty amazing things in their day. I would not build an ultra-lower power PC assuming it will be tolerable for day-to-day web browsing and email reading, unless you're comfortable using text mode or command-line interfaces exclusively.

This must be a market segment JetWay specializes in; they have a surprisingly large number of Mini-ITX motherboards to choose from. I don't think you'll find anything more power-efficient than the Geode LX 800 model, though, but there are some lesser expensive choices that get close. Lots of variety!

If the 5" x 5" profile of the Nano-ITX is far too large for your tastes, how do you feel about Pico-ITX? It's even smaller at 10cm x 7.2cm.

I've been following the ultra low power, tiny form factor PC segment for quite a few years now. With the emergence of Intel's Atom and "netbooks" like the ASUS Eee, it's a segment that is dangerously close to becoming mainstream. If you're interested, mini-itx.com is still one of the best sources of hands-on reviews, information, and community projects. It's fun stuff.

What could you do with a tiny, highly efficient x86 PC that boots up in under a minute?

[advertisement] Peer code review without meetings, paperwork, or stopwatches? No wonder Code Collaborator won the Jolt Award.

Web Development as Tag Soup

Jeff Atwood — Sun, 20 Jul 2008 23:59:59 -0800

As we work with ASP.NET MVC on Stack Overflow, I find myself violently thrust back into the bad old days of tag soup that I remember from my tenure as a classic ASP developer in the late 90's. If you're not careful bordering on manically fastidious in constructing your Views, you'll end up with a giant mish-mash of HTML, Javascript, and server-side code. Classic tag soup; difficult to read, difficult to maintain.

I don't mean tag soup in the sense of badly formed HTML, or the malformed world we live in. I mean tag soup in the sense of mixing HTML markup and server-side code. Now you can double your pleasure: badly formed HTML, meet badly written code.

The tag soup problem seems to be endemic to all modern web development stacks. I see that Ruby on Rails apps have the same problem; here's a slice of representative RHTML from Typo, a Ruby blogging engine.

Do you find this readable? Can you see where the code begins and the markup ends? Are you confident you could change the code structure without breaking the HTML, or change the HTML structure without breaking the code?

Sometimes editing this stuff makes me feel like I'm playing Operation. I have to ever so carefully maneuver my metal tweezers into one tiny slice of code or HTML and make my changes without touching the edges and setting off that blasted electrical buzzer.

I'm not trying to single out Rails or Typo here; I could easily show you a ASP.NET MVC view that's just as confusing (or as "clear", if you think that's perfectly readable, I guess). Tag soup is everywhere; take a look at the Python Django framework templates:

Archive for {{ year }}

{% for date in days %}
    {% ifchanged %}{{ date|date:"F" }}{% endifchanged %}
    {{ date|date:"j" }}
{% endfor %}

Perhaps when it comes to mixing HTML and server-side code, some form of soup is unavoidable, a necessary evil. The soup can be quite palatable; maybe even delicious. It's certainly possible to write good tag soup and bad tag soup.

But I have to wonder: is there a better way? Is there something beyond RHTML, Views, and Templates? What examples would you point to of web development stacks that avoided degenerating into yet more hazardous, difficult to maintain tag soup? Is there anything truly better on the horizon?

Or is this year's newer, fancier, even-more-delicious iteration of tag soup as good as it ever gets for web development?

[advertisement] Read the largest case study ever published about lightweight peer code review in Best Kept Secrets of Peer Code Review. Free book, free shipping.

Dealing With Bad Apples

Jeff Atwood — Thu, 17 Jul 2008 23:59:59 -0800

Robert Miesen sent in this story of a project pathology:

I was part of a team writing an web-based job application and screening system (a job kiosk the customer called it) and my team and our customer signed on to implementing this job kiosk using Windows, Apache, PHP5, and the ZendFramework -- everyone except one of our team members, who I will refer to as "Joe". Joe kept advocating the use of JavaScript throughout the technology deliberation phase, even though the customer made it quite clear that he expected the vast majority of the job kiosk to be implemented using a server-side technology and all the validation should be done using server-side technology.
The fact that the customer signed off on this, however, did nothing to deter Joe from advocating JavaScript -- abrasively. Every time our project hit a bump in the road, Joe would go off on some tirade on how much easier our lives would be if we were only writing this job kiosk in JavaScript. Joe would constantly bicker about how we were all doing this all wrong because we weren't doing it in JavaScript, not even bother to learn the technologies we were actually using, and, whenever fellow teammates would try and gently bring him back into the fold (usually via email), Joe would just flame the poor guy. At the height of Joe's pro-JavaScript bigotry, he would regularly belt off comments like, "Well, if we had only done it in JavaScript," to such an extent that the team would have been better off if he had just quit (or was reassigned or fired.)

After reading this story, I had to resist the urge to lean forward, hand placed thoughtfully under my chin, brow furrowed, and ask -- have you tried JavaScript?

Robert thought this story was a cautionary tale about technology dependence, but I see something else: a problem team member, a classic bad apple.

I'm sure "Joe" had the best of intentions, but at the point where you're actively campaigning against the project, and working against your teammates -- you're a liability to the project.

The cost of problem personnel on a project is severe, as noted in Chapter 12 of McConnell's Rapid Development: Taming Wild Software Schedules.

If you tolerate even one developer whom the other developers think is a problem, you'll hurt the morale of the good developers. You are implying that not only do you expect your team members to give their all; you expect them to do it when their co-workers are working against them.
In a review of 32 management teams, Larson and LaFasto found that the most consistent and intense complaint from team members was that their team leaders were unwilling to confront and resolve problems associated with poor performance by individual team members. (Larson and LaFasto 1989). They report that, "more than any other single aspect of team leadership, members are disturbed by leaders who are unwilling to deal directly and effectively with self-serving or noncontributing team members." They go on to to say that this is a significant management blind spot because managers nearly always think their teams are running more smoothly than their team members do.

How do we identify problem personnel? It's not difficult as you might think. I had a friend of mine once describe someone on his team as -- and this is a direct quote -- "a cancer". At the point which you, or anyone else on your team, are using words like cancer to describe a teammate, you have a serious project pathology. You don't have to be friends with everyone on your team, although it certainly helps, but a level of basic personal and professional respect is mandatory for any team to function normally.

Steve outlines a few warning signs that you're dealing with a bad apple on your team:

They cover up their ignorance rather than trying to learn from their teammates. "I don't know how to explain my design; I just know that it works." or "My code is too complicated to test." (These are both actual quotes.)
They have an excessive desire for privacy. "I don't need anyone to review my code."
They are territorial. "No one else can fix the bugs in my code. I'm too busy to fix them right now, but I'll get to them next week."
They grumble about team decisions and continue to revisit old discussions long after the team has moved on. "I still think we ought to go back and change the design we were talking about last month. The one we picked isn't going to work."
Other team members all make wisecracks or complain about the same person regularly. Software developers often won't complain directly, so you have to ask if there's a problem when you hear many wisecracks.
They don't pitch in on team activities. On one project I worked on, two days before our first major deadline, a developer asked for the day off. The reason? He wanted to spend the day at a men's clothing sale in a nearby city -- a clear sign he hadn't integrated with the team.

Let me be quite clear on this point: if your team leader or manager isn't dealing with the bad apples on your project, she isn't doing her job.

You should never be afraid to remove -- or even fire -- people who do not have the best interests of the team at heart. You can develop skill, but you can't develop a positive attitude. The longer these disruptive personalities stick around on a project, the worse their effects get. They'll slowly spread poison throughout your project, in the form of code, relationships, and contacts.

Removing someone from a team is painful; it's not fun for anyone. But realizing you should have removed someone six months ago is far more painful.

[advertisement] Complimentary paperback book on lightweight peer code review. 10 essays from industry experts. Free shipping. Order Best Kept Secrets of Peer Code Review.

The Ultimate Software Gold Plating

Jeff Atwood — Tue, 15 Jul 2008 23:59:59 -0800

Some developers love to gold plate their software. There are various shades of .. er, gold, I guess, but it's usually considered wasteful to fritter away time gold plating old code in the face of new features that need to be implemented, or old bugs that could be squashed.

Developers are fascinated by new technology and are sometimes anxious to try out new features of their language or environment or to create their own implementation of a slick feature they saw in another product -- whether or not it's required in their product. The effort required to design, implement, test, document, and support features that are not required lengthens the schedule.

But gold plating your code isn't all bad. Perhaps the most remarkable tale of successful developer gold plating I've ever read is the one Blake Patterson outlines:

Not long ago I purchased a new-in-box Atari Jaguar, complete with Jeff Minter's psychedelic sequel to Tempest, Tempest 2000. It's an amazing game that's been ported to many other platforms, but the consensus is that none are as solid as the Jaguar original. Having played several of the ports, I'd have to agree.

An interesting thing about "the world's first 64-bit console" -- its controller was, as the Brits would say, fairly pants. It was large, sported a calculator-button array for game overlays (like the Intellivision controller), had no shoulder buttons, and featured only a D-pad for directional control. (ed: certainly one of the weirdest members of the game console controller family tree, to be sure)

As the arcade original is controlled with a rotary spinner knob, the D-pad falls rather short of providing ideal game control.

But, of course, being such a savvy chap, Jeff Minter realized this.
Jeff wrote in support for an analog rotary controller ... that did not exist. Neither Atari nor third party manufacturers produced such a controller in the Jaguar's heyday. Jeff, as I understand it, hacked his own together by wiring an Atari paddle controller into a Jaguar controller. In the years since the Jaguar's passing, a few small operations have offered modified Jaguar controllers with spinners wired into them for purchase.

Jeff Minter's an interesting historical figure in the computer gaming community, as the author of several 8-bit computer era game classics. I've talked about his long-standing interest in audio visualization here once before. He's still creating games today; his latest is the Xbox Live downloadable title Space Giraffe. Jeff has a blog that he updates fairly regularly.

Still, I'm amazed that Jeff added code to a commercially shipped console game to support a completely optional homebrew spinner controller of his own creation. That's the very definition of "not required". This code lied dormant in the game until a handful of enthusiasts, fourteen years later, cobbled together custom controllers to play the game as it was originally intended by the author.

If that isn't the ultimate case of gold plating your software, I don't know what is. My hat is off to you, Mr. Minter.

[advertisement] Peer Code Review. No meetings. No busy-work. Customizable workflows and reports. Try Jolt Award-winning Code Collaborator.

Maybe Normalizing Isn't Normal

Jeff Atwood — Mon, 14 Jul 2008 23:59:59 -0800

One of the items we're struggling with now on Stack Overflow is how to maintain near-instantaneous performance levels in a relational database as the amount of data increases. More specifically, how to scale our tagging system. Traditional database design principles tell you that well-designed databases are always normalized, but I'm not so sure.

Dare Obasanjo had an excellent post When Not to Normalize your SQL Database wherein he helpfully provides a sample database schema for a generic social networking site. Here's what it would look like if we designed it in the accepted normalized fashion:

Normalization certainly delivers in terms of limiting duplication. Every entity is represented once, and only once -- so there's almost no risk of inconsistencies in the data. But this design also requires a whopping six joins to retrieve a single user's information.

select * from Users u
inner join UserPhoneNumbers upn
on u.user_id = upn.user_id
inner join UserScreenNames usn
on u.user_id = usn.user_id
inner join UserAffiliations ua
on u.user_id = ua.user_id
inner join Affiliations a
on a.affiliation_id = ua.affiliation_id
inner join UserWorkHistory uwh
on u.user_id = uwh.user_id
inner join Affiliations wa
on uwh.affiliation_id = wa.affiliation_id

(Update: this isn't intended as a real query; it's only here to visually illustrate the fact that you need six joins -- or six individual queries, if that's your cup of tea -- to get all the information back about the user.)

Those six joins aren't doing anything to help your system's performance, either. Full-blown normalization isn't merely difficult to understand and hard to work with -- it can also be quite slow.

As Dare points out, the obvious solution is to denormalize -- to collapse a lot of the data into a single Users table.

This works -- queries are now blindingly simple (select * from users), and probably blindingly fast, as well. But you'll have a bunch of gaping blank holes in your data, along with a slew of awkwardly named field arrays. And all those pesky data integrity problems the database used to enforce for you? Those are all your job now. Congratulations on your demotion!

Both solutions have their pros and cons. So let me put the question to you: which is better -- a normalized database, or a denormalized database?

Trick question! The answer is that it doesn't matter! Until you have millions and millions of rows of data, that is. Everything is fast for small n. Even a modest PC by today's standards -- let's say a dual-core box with 4 gigabytes of memory -- will give you near-identical performance in either case for anything but the very largest of databases. Assuming your team can write reasonably well-tuned queries, of course.

There's no shortage of fascinating database war stories from companies that made it big. I do worry that these war stories carry an implied tone of "I lost 200 pounds and so could you!"; please assume the tiny-asterisk disclaimer results may not be typical is in full effect while reading them. Here's a series that Tim O'Reilly compiled:

There's also the High Scalability blog, which has its own set of database war stories:

First, a reality check. It's partially an act of hubris to imagine your app as the next Flickr, YouTube, or Twitter. As Ted Dziuba so aptly said, scalability is not your problem, getting people to give a shit is. So when it comes to database design, do measure performance, but try to err heavily on the side of sane, simple design. Pick whatever database schema you feel is easiest to understand and work with on a daily basis. It doesn't have to be all or nothing as I've pictured above; you can partially denormalize where it makes sense to do so, and stay fully normalized in other areas where it doesn't.

Despite copious evidence that normalization rarely scales, I find that many software engineers will zealously hold on to total database normalization on principle alone, long after it has ceased to make sense.

When growing Cofax at Knight Ridder, we hit a nasty bump in the road after adding our 17th newspaper to the system. Performance wasn't what it used to be and there were times when services were unresponsive.
A project was started to resolve the issue, to look for 'the smoking gun'. The thought being that the database, being as well designed as it was, could not be of issue, even with our classic symptom being rapidly growing numbers of db connections right before a crash. So we concentrated on optimizing the application stack.
I disagreed and waged a number of arguments that it was our database that needed attention. We first needed to tune queries and indexes, and be willing to, if required, pre-calculate data upon writes and avoid joins by developing a set of denormalized tables. It was a hard pill for me to swallow since I was the original database designer. Turned out it was harder for everyone else! Consultants were called in. They declared the db design to be just right - that the problem must have been the application.
After two months of the team pushing numerous releases thought to resolve the issue, to no avail, we came back to my original arguments.

Pat Helland notes that people normalize because their professors told them to. I'm a bit more pragmatic; I think you should normalize when the data tells you to:

Normalization makes sense to your team.
Normalization provides better performance. (You're automatically measuring all the queries that flow through your software, right?)
Normalization prevents an onerous amount of duplication or avoids risk of synchronization problems that your problem domain or users are particularly sensitive to.
Normalization allows you to write simpler queries and code.

Never, never should you normalize a database out of some vague sense of duty to the ghosts of Boyce-Codd. Normalization is not magical fairy dust you sprinkle over your database to cure all ills; it often creates as many problems as it solves. Fear not the specter of denormalization. Duplicated data and synchronization problems are often overstated and relatively easy to work around with cron jobs. Disks and memory are cheap and getting cheaper every nanosecond. Measure performance on your system and decide for yourself what works, free of predispositions and bias.

As the old adage goes, normalize until it hurts, denormalize until it works.

[advertisement] Peer code review without meetings, paperwork, or stopwatches? No wonder Code Collaborator won the Jolt Award.

Monkeypatching For Humans

Jeff Atwood — Sat, 12 Jul 2008 23:59:59 -0800

Although I love strings, sometimes the String class can break your heart. For example, in C#, there is no String.Left() function. Fair enough; we can roll up our sleeves and write our own function lickety-split:

public static string Left(string s, int len)
{
    if (len == 0 || s.Length == 0)
        return "";
    else if (s.Length <= len)
        return s;
    else
        return s.Substring(0, len);
}

And call it like so:

var s = "Supercalifragilisticexpialidocious";
s = Left(s, 5);

Fairly painless, right?

But with the advent of C# 3.0, there's an even better way -- extension methods. With an extension method, we "extend" the String to add the missing function. The code is fairly similar; I'll highlight the changed parts in red.

public static string Left(this string s, int len)
{
    if (len == 0 || s.Length == 0)
        return "";
    else if (s.Length <= len)
        return s;
    else
        return s.Substring(0, len);
}

And now we can call it as if this very method existed on the String class as shipped:

var s = "Supercalifragilisticexpialidocious";
s = s.Left(5);

Pretty slick. It's difficult not to fall in love with extension methods, as they allow you to mold classes into exactly what you think they should be. This is fairly innocuous in C#, as extension methods only allow you to add new functionality to classes, not override, remove, or replace anything.

But imagine if you could.

Well, that's exactly how it is in other, more dynamic languages such as Javascript, Python, Perl, and Ruby. Something as prosaic as C# extensions is old hat to these folks. In those languages, you could redefine everything in the String class if you wanted to. This is commonly known in dynamic language circles as monkeypatching.

If the idea of monkeypatching scares you a little, it probably should. Can you imagine debugging code where the String class had subtly different behaviors from the String you've learned to use? Monkeypatching can be incredibly dangerous in the wrong hands, as Avdi Grimm notes:

Monkey patching is the new black [in the Ruby community]. It's what all the hip kids are doing. To the point that smart, experienced hackers reach for a monkey patch as their tool of first resort, even when a simpler, more traditional solution is possible.
I don't believe this situation to be sustainable. Where I work, we are already seeing subtle, difficult-to-debug problems crop up as the result of monkey patching in plugins. Patches interact in unpredictable, combinatoric ways. And by their nature, bugs caused by monkey patches are more difficult to track down than those introduced by more traditional classes and methods. As just one example: on one project, it was a known caveat that we could not rely on class inheritable attributes as provided by ActiveSupport. No one knew why. Every Model we wrote had to use awkward workarounds. Eventually we tracked it down in a plugin that generated admin consoles. It was overwriting Class.inherited(). It took us months to find this out.
This is just going to get worse if we don't do something about it. And the "something" is going to have to be a cultural shift, not a technical fix. I believe it is time for experienced Ruby programmers to wean ourselves off of monkey patching, and start demonstrating more robust techniques.

Try to imagine a world where every programmer you know is a wannabe language designer, bent on molding the language to their whims. When I close my eyes and imagine it, I have a vision of the apocalypse, a perfect, pitch-black storm of utterly incomprhensible, pathologically difficult to debug code.

I was just looking at random PHP plugin code the other day, and it was, frankly, crap. But that's because most code is crap. Including my own. It is, sadly, the statistical norm. That's why sites like The Daily WTF are guaranteed to have more material than they can possibly ever publish for the next millennia. (Note to self: invest in this website). I can only imagine what that PHP plugin code would have looked like, had its developer been granted the ability to redefine fundamental PHP keywords and classes at will. These are the sort of thoughts that drive me to drink Bawls. And that stuff is disgusting.

You might say that PHP, sans the fundamental dynamic language ability to monkeypatch, is just another crappy Blub language. But there's also a ton of incredibly useful PHP code out there. So it seems to me that the ability to monkeypatch doesn't stop people from producing a huge volume of useful code, even in a kind of.. horrible language. Some of it is even good!

While I acknowledge the power and utility of dynamic language monkeypatching, I know enough about programmers -- myself absolutely included -- to know the vast majority of us have absolutely no business whatsoever re-designing a programming language. There's a reason some of the most deeply respected computer scientists in the world end up as language designers.

Perhaps then, given the risks, monkeypatching should mean reaching for the meta-hammer as infrequently as humanly possible. This is a position that Avdi himself espouses in a followup comment:

I'm afraid a lot of people have missed the actual meat of my argument -- that dynamic extension of classes is currently overused in Ruby, in ways that are:

Needless - another technique (such as a mixin, or locally extending individual objects) would have worked as well or better.
Overcomplicated - the use of a monkey patch actually created more work for the author.
Fragile - the solution is tightly bound to third-party internals, reducing the usefulness of the plugin or gem because it is prone to breakage.
Excessively wide in scope - by hardcoding extensions to core classes, the author takes the choice to scope the change out of the plugin/gem user's hands, further limiting utility.

My point is that there are alternatives - often alternatives which are actually easier to implement and will make your plugin or gem more useful to the user.

While I enjoy the additive nature of C# extensions, even those are enough to make me a little nervous, as mild as they are. Full-blown dynamic language monkeypatching goes even further; it might even be the ultimate expression of programming power. Is there anything more pure and godlike than programming your own programming language?

But if wielding that power doesn't scare and humble you a little, too, then maybe you should leave the monkeypatching to the really smart monkeys.

[advertisement] Read the largest case study ever published about lightweight peer code review in Best Kept Secrets of Peer Code Review. Free book, free shipping.

iTunes is Anti-Web

Jeff Atwood — Wed, 09 Jul 2008 23:59:59 -0800

Ever find yourself clicking on links to music or videos and getting blasted in the face with this delightful little number?

That's right -- links to any sort of music, TV shows, movies, podcasts, audiobooks or anything else available through Apple's iTunes store requires custom software to be installed on your computer before they will display thing one to you.

Is it so unreasonable to expect links in your browser to resolve to, oh, I don't know, web pages containing information about the thing you just clicked on? Is there anything more anti-web than demanding users install custom software to display information that could have just as easily been delivered through the browser?

So here's what we know:

Apple wants us to install iTunes.
iTunes is necessary to sync media with Apple's iPods and iPhones.
iTunes is only available for Windows and OS X.
iTunes is required to browse or buy anything from the iTunes Store.

That's all well and good for people who own iPods and iPhones -- and happen to be running Windows or OS X, I suppose.

But what about the rest of the world? Why lock them out with the ultimate login barrier? We might like to browse the iTunes Store, too. At the very least, I might want some basic information about the media I just clicked on. Right here in my browser where I already am. Information like what the heck it is, some artwork, maybe some audio clips, how much it costs -- sweet talk me. Make me want to buy it through the Apple Store. Dazzle me with your simplicity and ease of use. Beguile me with your wares!

Or, you could bludgeon me with the digital equivalent of a giant stop sign.

Hey, "it just works". Except when it doesn't.

I'm certainly able to click through to eminently purchaseable media on dozens of other places on the web using nothing more than my web browser. Let's imagine, for a moment, how utterly ridiculous it would be if I had to install the Amazon application to browse and purchase media from Amazon.

And yet this is exactly how the iTunes Store works. Or doesn't work, depending on your perspective.

I can understand requiring iTunes once you want to sync your media with Apple hardware devices -- although I would argue syncing should really be a fundamental, built in function of the operating system. But I'm not trying to sync anything! All I did was click on a link. It's downright user hostile to demand installation of a special application merely to browse the store, and it is most certainly against everything the web stands for and was built on.

The last "application" I can recall needing to install to get to things online was AOL.

And we all know how great that turned out.

For all the buzz about the Apple "it just works" mystique, the current iTunes Store design surely doesn't -- at least not the same way the rest of the web does. And I, for one, can't get behind that.

[advertisement] Complimentary paperback book on lightweight peer code review. 10 essays from industry experts. Free shipping. Order Best Kept Secrets of Peer Code Review.

Spartan Programming

Jeff Atwood — Tue, 08 Jul 2008 04:00:00 -0800

As I grow older and ~~wiser~~even older as a programmer, I've found that my personal coding style has trended heavily toward minimalism.

I was pleased, then, to find many of the coding conventions I've settled on over the last 20 years codified in Spartan programming.

No, not that sort of Spartan, although it is historically related. The particular meaning of spartan I'm referring to is this one:

(adj) ascetic, ascetical, austere, spartan (practicing great self-denial) "Be systematically ascetic...do...something for no other reason than that you would rather not do it" - William James; "a desert nomad's austere life"; "a spartan diet"; "a spartan existence"

I've tried to code smaller, even going so far as to write no code at all when I can get away with it. Spartan programming aligns perfectly with these goals. You strive for simultaneous minimization of your code in many dimensions:

Horizontal complexity. The depth of nesting of control structures.
Vertical complexity. The number of lines or length of code.
Token count.
Character count.
Parameters. The number of parameters to a routine or a generic structure.
Variables.
Looping instructions. The number of iterative instructions and their nesting level.
Conditionals. The number of if and multiple branch switch statements.

The discipline of spartan programming means frugal use of variables:

Minimize number of variables. Inline variables which are used only once. Take advantage of foreach loops.
Minimize visibility of variables and other identifiers. Define variables at the smallest possible scope.
Minimize accessibility of variables. Prefer the greater encapsulation of private variables.
Minimize variability of variables. Strive to make variables final in Java and const in C++. Use annotations or restrictions whenever possible.
Minimize lifetime of variables. Prefer ephemeral variables to longer lived ones. Avoid persistent variables such as files.
Minimize names of variables. Short-lived, tightly scoped variables can use concise, terse names.
Minimize use of array variables. Replace them with collections provided by your standard libraries.

It also means frugal use of control structures, with early return whenever possible. This is probably best illustrated with an actual example, starting with raw code and refactoring it using the spartan programming techniques:

Applying Spartan programming techniques to a C File
Applying Spartan programming techniques to a Java function

I don't agree with all the rules and guidelines presented here, but I was definitely nodding along with the majority of the page. Minimalism isn't always the right choice, but it's rarely the wrong choice. You could certainly do worse than to adopt the discipline of spartan programming on your next programming project.

(hat tip to Yuval Tobias for sending this link my way)

[advertisement] Peer Code Review. No meetings. No busy-work. Customizable workflows and reports. Try Jolt Award-winning Code Collaborator.

The Problem With Code Folding

Jeff Atwood — Sun, 06 Jul 2008 23:59:59 -0800

When you join a team, it's important to bend your preferences a little to accommodate the generally accepted coding practices of that team. Not everyone has to agree on every miniscule detail of the code, of course, but it's a good idea to dicuss it with your team and decide on overall approaches and philosophy beforehand. It promotes team harmony, and more than that, it's just common courtesy. As they say, when in Rome, do as the Romans do.

I've always been wary of cowboy coders who rolled into an ongoing project on fresh horses and immediately started dictating terms. It's a very short trip indeed from there to Who Wrote This Crap, and the predictable, inevitable finger-pointing at the foolhardy programmers who came before you begins. Don't be that guy or gal. Work with your team, not against it.

Still, there are some coding preferences people may feel.. strongly.. about. If that's the case, try to clear the air and address those strong preferences up front, as early as possible. Don't let them simmer. For me, the use of #region is one of those things. I tried to make myself clear in this twitter message:

So what is #region? It's a named hint you place in C# or VB.NET code to set a code folding point. Any code placed inside that region is, by default, collapsed when you re-open it in the editor. Here's a random example from the Log4Net project:

Immediately I have a problem: I can't see anything! I have to manually expand those sections to browse any of the code in this class. It is possible to configure the Visual Studio IDE to not fold any of the regions when files are opened, but this is the out of box behavior, so that's what most developers will see. And of course there are keyboard shortcuts to deal with the regions:

Ctrl+M, Ctrl+M	Collapse or expand the block you're currently in.
Ctrl+M, Ctrl+O	Collapse all blocks in the file
Ctrl+M, Ctrl+L	Expand all blocks in the file
Ctrl+M, Ctrl+P	Stop outlining mode. (Ctrl+M, Ctrl+O resumes)

Here's the really sick part: once you expand the above log4net code there's literally three pages worth of code there! After you strip out all the massive XMLDoc comments and the dozen or so #region directives, you could have had all the code at your fingertips with a minor flick of the mouse wheel, in a simple scrollable layout.

I daresay being able to see the damn code is more important than having it meticulously segmented into six pointless little named buckets, but apparently a lot of programmers can't get enough of stuffing their code into pointless little named buckets. It's as if they've forgotten what the scroll bar -- and incremental search -- is for.

The #region directive drives me bonkers. It's not evil, per se, but I feel it is criminally overused in practice and heavily prone to abuse. I strongly urge you to think about how you're using code folding, because as I see it, there are a lot of downsides:

Folding directives are glorified comments. #region has zero meaning to the compiler; it's a hint to the editor to allow code folding. It doesn't do any namespacing or scoping. Why, exactly, are we writing code to accommodate the editor? It boggles my mind that we'd add significant lines of code to our project that do nothing but offer organizational hints to the editor. Even traditional comments are a better value for your keystroke, because they can be more expressive. And folding is certainly no substitute at all for bona-fide refactoring.
Folding is used to sweep code under the rug. Got a bunch of boring boilerplate code that makes your eyes water? A slew of ugly, gnarly code that nobody in their right mind wants to look at? Hide it in a region and fold that sucker into oblivion! Problem solved, right? Hardly. Your project is now full of crappy code that you can't see. That's worse. Much worse! Code that hides from you is code that will rot in the most putrescent and painful way possible. Your code should be front and center at all times -- exposed to as many programmers' eyes, and as much healing light, as possible.
Folding is used to mask excessive length. The presence of folded code can lull developers into a false sense of what clean code looks like. Under the cover of folding, you can end up writing long, horrible spaghetti code blocks. If the code needs the crutch of folding to look organized, it's bad code.
Folding can hide deficiencies in your editor. The presence of so-called "standard" boilerplate regions like "Public Constructors" and "Public Properties" and "Events" is not a feature. It's a bug. The editor should automatically offer to fold up these common structural blocks for you! I'm continually amazed that programmers spend time doing this scutwork when they could be writing useful code. Or at least demanding a smarter code editor.

I urge developers to write code that doesn't need folding to be readable, clear, and concise. I'm sure there are sane uses for code folding out there somewhere, but I rarely see them.

[advertisement] Peer code review without meetings, paperwork, or stopwatches? No wonder Code Collaborator won the Jolt Award.

Investing in a Quality Programming Chair

Jeff Atwood — Fri, 04 Jul 2008 23:59:59 -0800

In A Developer's Second Most Important Asset, I described how buying a quality chair may be one of the smartest investments you can make as a software developer.

In fact, after browsing chairs for the last few years of my career, I've come to one conclusion: you can't expect to get a decent chair for less than $500. If you are spending less than that on seating -- unless you are getting the deal of the century on dot-bomb bankruptcy auctions -- you're probably making a mistake.

I still believe this to be true, and I urge any programmers reading this to seriously consider the value of what you're sitting in while you're on the job. In our profession, seating matters:

Chairs are a primary part of the programming experience. Eight hours a day, every day, for the rest of your working life -- you're sitting in one. Like it or not, whatever you're sitting in has a measurable impact on your work experience.
Cheap chairs suck. Maybe I've become spoiled, but I have yet to sit in a single good, cheap chair. In my experience, the difference between the really great chairs and the cheap stuff is enormous. A quality chair is so comfortable and accommodating it effortlessly melts into the background, so you can focus on your work. A cheesy, cheap chair constantly reminds you how many hours of work you have left.
Chairs last. As I write this, I'm still sitting my original Aeron chair, which I purchased in 1998. I can't think of any other piece of equipment I use in my job that has lasted me ten full years and beyond. While the initial sticker shock of a quality chair may turn you off, try to mentally amortize that cost across the next ten years or more.

Choice of seating is as fundamental and constant as it gets in a programming career otherwise marked by relentless change. They are long term investments. Why not take the same care and consideration in selecting a chair as you would with the other strategic directions that you'll carry with you for the rest of your career? Skimping yourself on a chair just doesn't make sense.

Although I've been quite happy with my Herman Miller Aeron chair over the last 10 years, I've always been a little disenchanted with the way it became associated with dot-com excess:

In the '90s, the Aeron became an emblem of the dot-com boom; it symbolized mobility, speed, efficiency, and 24/seven work weeks. The Aeron was a must-have for hot startups precisely because it looked the least like office furniture: It was more like a piece of machinery or unadorned engineering. The black Pellide webbing was durable, and hid whatever Jolt or Red Bull stains you might get on it. Held taut by an aluminum frame, the mesh allowed air to circulate and kept your body cool. What's more, the chair came in three sizes, like a personalized tool. Assorted knobs and levers allowed you to adjust the seat height, tilt tension, tilt range, forward tilt, arm height, arm width, arm angle, lumbar depth, and lumbar height. The Aeron was high-tech but sexy -- which was how the dot-commers saw themselves.

But baby-faced CEOs weren't drawn to the Aeron only for the way it looked. The Aeron was a visual expression of the anti-corporate zeitgeist, a non-hierarchical philosophy about the workplace. An office full of Aerons implicitly rejected the Fortune 500, coat-and-tie, brick-and-mortar model in which the boss sinks back in an overpriced, oversized, leather dinosaur while his secretary perches on an Office Max toadstool taking notes.

I recently had the opportunity to sit in a newer Herman Miller Mirra chair on a trip, and I was surprised how much more comfortable it felt than my classic Aeron.

The Mirra chair was an excellent recliner, too. I've been disappointed by how poorly the Aeron reclines. I actually broke my Aeron's recline pin once and had to replace it myself. So I've retrained myself not to recline, which is awkward, as I'm a natural recliner.

All this made me wonder if I should retire my Aeron and upgrade to something better. I liked the Mirra, but the comments to my original chair post have a lot of other good seating suggestions, too. Here are pictures and links to the chairs that were most frequently mentioned as contenders, in addition to the Mirra and Aeron pictured above:

Steelcase Think Chair

Steelcase Leap Chair

Ergohuman Mesh Chair

HumanScale Freedom Chair

HumanScale Liberty Chair

There were also some lesser known recommendations, such as the Haworth Zody chair, Nightingale CXO chair, BodyBilt ergo chairs, Hag kneeling chair, NeutralPosture ergo, and something called the swopper.

Chair fit is, of course, a subjective thing. If you're investing $500+ in a chair, you'd understandably want to be sure it's "the one". The thing to do is find a local store that sells all these chairs and try them all out. Well, good luck with that. Don't even bother with your local big-box office supply chain. Your best bet seems to be back stores, as they tend to stock many of the more exotic chairs. Apparently they have a clientele of people who are willing to spend for comfort.

Reviews of individual chairs are relatively easy to find, but aren't particularly helpful in isolation. What we need is a multi-chair review roundup. The only notable roundup I know of is Slate's late 2005 Sit Happens: The Search for the Best Desk Chair. It's not as comprehensive as I would like, but it does have most of the main contenders. Notably, Slate's winner was the HumanScale Liberty. Update: I found another good multiple chair roundup at CrunchGear as well.

If this is all a bit too much furniture porn for your tastes, I understand. As for me, I'm headed off to my local friendly neighborhood back store to figure out which of these chairs will best replace my aging Aeron. By my calculations, the Aeron cost me about $7 per month over its ten year lifetime; I figure my continued health and comfort while programming are worth at least that much.

[advertisement] Read the largest case study ever published about lightweight peer code review in Best Kept Secrets of Peer Code Review. Free book, free shipping.

Why Can't Microsoft Ship Open Source Software?

Jeff Atwood — Wed, 02 Jul 2008 23:59:59 -0800

In Codeplex wastes six months reinventing wheels, Ryan Davis has a bone to pick with Microsoft:

I saw an announcement [in March, 2007] that CodePlex, Microsoft's version of Sourceforge, has released a source control client.
This infuriates me. This cool thing they spent six months (six!) writing is called Subversion, and it had a 1.0.0 release [in early 2004]. Subversion had its first beta in late 2003, so the Codeplex folks are waaay behind the state of the art on this one.
As a whole, I think the state of software is abysmal. The only way to make it better is to stop writing new code. New code is always full of bugs, and its an expensive path to get from blank screen to stable program. We need to treat programming more like math, we need to build on our results. Development tools is a special market, as our needs are all very similar, and when we need a tool, we have the skills to make it.

It's a great rant -- you should read the whole thing -- but I'm not sure I entirely agree.

While I do empathize with the overall sentiment that Ryan is expressing here, I also found myself nodding along with Addy Santo, who left this comment:

Author seems to think that all software development is done in basements and dorms. The reality is that software is an industry like any other - and follows the same simple rules of economics. How many brands of sports shoes are there? How many different MP3 players? Flavors of toothpaste ? If you can walk down the soft drink isle and not be "infuriated" by Vanilla Cherry Diet Doctor Pepper then you might just be a hypocrite.

So if you think Microsoft's particular flavor of source control is redundant, you'll really hate Diet Cherry Chocolate Dr. Pepper.

(I am now required by law to link Tay Zonday's Cherry Chocolate Rain video. My apologies in advance. And if that makes no sense to you, see here.)

Are there meaningful differences between Microsoft's Team Foundation flavor of version control and Subversion? The short answer is that there aren't -- if all you're looking for is a carbonated beverage. If all you require is run of the mill, basic centralized source control duties, they're basically the same product. So why not go with the free one?

But Team Foundation is much more than just source control. Of course there are open source equivalents to much of the functionality offered in Team System, as Ryan is quick to point out.

The Codeplex staff stated they needed to write their own client in order to integrate with the TFS server infrastructure. According to an MSDN article (Get All Your Devs In A Row With Visual Studio 2005 Team System), TFS seems to be a complicated tool to help manage your developers. Reading the description, TFS is an issue tracker, unit tester, continuous integration, source control system, and Visual Studio plugin. So, basically a combination of Trac, NUnit, CruiseControl.NET, Subversion, and a Visual Studio plugin. Why not just write the Visual Studio plugin, and hook into the tools people are already using? All those tools have rich plugin-architectures that would probably support any sensible addition you'd want to make.

The answer, of course, is that Microsoft does all that painful integration work for you -- at a price.

If you have the time to look closer, you'll find more flavorful differences between Subversion and TFS source control. Differences more akin to, say, Dr. Pepper and Mr. Pibb.

I'm not going to enumerate all the subtle and not-so-subtle differences between the two here; picking a fight between two modern centralized version control systems is not my goal. They're both great. Choose whatever modern source control system you prefer, and take the time to learn it in depth. Source control is the bedrock of modern software engineering, and I've found precious few developers that truly understand how it works. All that time we were going to spend arguing whether your source control system can beat up my source control system? I've got a radical idea: let's spend it on learning the damn stuff instead.

Still, there is a much deeper, more endemic problem here that Ryan alludes to, and it deserves to be addressed.

One of Microsoft's biggest challenges in the last few years has been that its competitors are free to ship what are, by now, fairly mature open source components as parts of their operating systems. When was the last time you ever saw any open source anything shipping in a Microsoft product? On some deep, dark corporate level, Microsoft must feel compelled to rewrite everything to completely own the source code. Sometimes -- a more cynical person might say "often" -- this results in poor quality copies instead of actual innovation, such as Microsoft's much-maligned MSTest unit test framework. It's a clone of NUnit with all new bugs and no new features, but it can be included in the box with Visual Studio and integrated into the product. It's a one step forward, two steps back sort of affair.

Everybody I know -- including our own Stack Overflow team -- who has tried to use the MSTest flavor of unit tests has eventually thrown up their arms and gone back to NUnit. It's just too painful; the commercial clone lacks the simplicity, power, and community support of the original open source version. There's simply no reason for MSTest to exist except to satisfy some bizarre corporate directive that Microsoft never ship open source code in their products. Furthermore, this blind spot hampers obvious integration points. Microsoft could build first-class integration points for NUnit into Visual Studio. But they haven't, and probably never will, because so much effort is poured into maintaining the second-rate MSTest clone.

In fact, the more I think about this, the more I think Microsoft's utter inability to integrate open source software of any kind whatsoever into their products might just end up killing them. It's a huge problem, and it's only going to get worse over time. Open source seems to evolve according to a different power law than commercial software. If I worked in the upper echelons of Microsoft, I'd be looking at the graph of open source software growth from the years of 1999 to 2008 and crapping my pants right about now.

It's a shame, because the best way to "beat" open source is to join 'em -- to integrate with and ship open source components as a part of your product. Unfortunately, that's the one route that Microsoft seems hell bent on never following.

Update: For background, do read Jon Galloway's explanation: Why Microsoft Can't Ship Open Source Code.

[advertisement] Complimentary paperback book on lightweight peer code review. 10 essays from industry experts. Free shipping. Order Best Kept Secrets of Peer Code Review.

Alan Turing, the Father of Computer Science

Jeff Atwood — Tue, 01 Jul 2008 19:15:58 -0800

Charles Petzold was kind enough to send me a copy of his new book, The Annotated Turing: A Guided Tour Through Alan Turing's Historic Paper on Computability and the Turing Machine.

One look at the original title page of Turing's paper is enough to convince me that we're fortunate to have a guide as distinguished and patient as Charles. You know you're in trouble when the very first page opens with "Entscheidungsproblem".

The computer you're using to read this post is based on the mathematical model laid out in that thirty-six page 1936 paper. As are all other computers in the world. The terms Turing Machine and Turing Complete are both derived from that one historic paper.

Needless to say, we owe Alan Turing a lot.

Not only is Alan Turing the father of all modern computer science, he also was the single individual most responsible for breaking the Enigma code during World War II, and he laid the foundation for artificial intelligence by posing the Turing Test in 1950.

Unfortunately, Alan Turing was also terribly persecuted for the "crime" of being a homosexual. He was arrested in 1952 for having sex with another man. It pains me greatly to read about the degrading and inhumane treatment one of our greatest scientific minds was subjected to. Alan Turing ultimately committed suicide not long afterwards at the age of 42.

The "nobel prize of computing" was founded in Turing's name in 1966. Reading the list of Turing Award recipients is humbling indeed, a reminder of not only how far we've come, but how far we have to go.

The Alan Turing Memorial, erected in 2001, bears this Bertrand Russell quote:

Mathematics, rightly viewed, possesses not only truth, but supreme beauty -- a beauty cold and austere, like that of sculpture.

You'll note that the statue depicts Turing holding an apple in his right hand, a reference to the way he chose to end his life -- by eating a cyanide-laced apple. That was Turing's last message to the world, with clear parallels not only to the legendary scientific knowledge of Isaac Newton, but also the biblical interpretation of forbidden love.

Petzold's Annotated Turing is a gripping testament to the amazing mind of Alan Turing. Writing the book was a nine year labor of love, and it shows. It may be his shortest book -- but it could also be his best yet.

[advertisement] Peer Code Review. No meetings. No busy-work. Customizable workflows and reports. Try Jolt Award-winning Code Collaborator.

Open Wireless and the Illusion of Security

Jeff Atwood — Sun, 29 Jun 2008 01:00:00 -0800

Bruce Schneier is something of a legend in the computer security community. He's the author of the classic, oft-cited 1994 book Applied Cryptography, as well as several well-known cryptography algorithms.

The cheeky Norris-esque design above is a reference to the actor names commonly used in examples of shared secret key exchange.

What I find most interesting about Bruce, however, is that he has moved beyond treating computer security as a problem that can be solved with increasingly clever cryptography algorithms:

Schneier now denounces his early success as a naive, mathematical, and ivory tower view of what is inherently a people problem. In Applied Cryptography, he implies that correctly implemented algorithms and technology promise safety and secrecy, and that following security protocol ensures security, regardless of the behavior of others. Schneier now argues that the incontrovertible mathematical guarantees miss the point. As he describes in Secrets and Lies, a business which uses RSA encryption to protect its data without considering how the cryptographic keys are handled by employees on "complex, unstable, buggy" computers has failed to properly protect the information. An actual security solution that includes technology must also take into account the vagaries of hardware, software, networks, people, economics, and business.

This is the programming equivalent of realizing that Peopleware is ultimately a much more important book than The Art of Computer Programming. The shift in focus from algorithms to people is even more evident if you frequent Bruce's excellent blog, or read his newest books Practical Cryptography and Beyond Fear.

As much as I respect Bruce, I was surprised to read that he intentionally keeps his wireless network open.

Whenever I talk or write about my own security setup, the one thing that surprises people -- and attracts the most criticism -- is the fact that I run an open wireless network at home. There's no password. There's no encryption. Anyone with wireless capability who can see my network can use it to access the internet.

I've advocated WiFi encryption from the day I owned my first wireless router. As I encountered fewer and fewer open WiFi access points over the years, I viewed it as tangible progress. Reading Bruce's opinion is enough to make me question those long held beliefs.

It's a strange position for a respected computer security expert to advocate. But I think I get it. Security is a tough problem. If you take the option of mindlessly flipping a WPA or WEP switch off the table, you're now forced to think more critically about the security of not only your network, but also the fundamental security of the data on your computers. By advocating the radical idea that your wireless network should be intentionally kept open, Bruce is attempting to penetrate the veil of false algorithmic security.

I may understand and even applaud this effort, but I don't agree. Not because I'm worried about the security of my data, or any of the half-dozen other completely rational security arguments you could make against intentionally keeping an open wireless network. My concerns are more prosaic. I desperately want to protect the thin sliver of upstream bandwidth my provider allows me. Some major internet providers are also talking about monthly download caps, too. Bruce's position only makes sense if you have effectively unlimited bandwidth in both directions. Basically, I'm worried about the tragedy of the bandwidth commons. As much as I might like my neighbors, they can pay for their own private sliver of bandwidth, or knock on my door and ask to share if they really need it.

So, to me at least, enabling wireless security is my way of ensuring that I get every last byte of the bandwidth I paid for that month.

It's worth realizing, however, that wireless security is no panacea, even in this limited role. Given a sufficiently motivated attacker, every wireless network is crackable.

With that in mind, here are a few guidelines.

WEP = Worthless Encryption Protocol
WEP, the original encryption protocol for wireless networks, is so fundamentally flawed and so deeply compromised it should arguably be removed from the firmware of every wireless router in the world. It's possible to crack WEP in under a minute on any vaguely modern laptop. If you choose WEP, you have effectively chosen to run an open wireless network. There's no difference.
WPA requires a very strong password
The common "personal" (PSK) variant of WPA is quite vulnerable to brute force dictionary attacks. It only takes a trivial amount of wireless sniffing to obtain enough data to attack your WPA password offline -- which means an unlimited amount of computing power could potentially be marshalled against your password. While brute force attacks are still for dummies, most people are, statistically speaking, dummies. They rarely pick good passwords. If ever there was a time to take my advice on using long passphrases, this is it. Experts recommend you shoot for a 33 character passphrase.

In the end, perhaps wireless security is more of a deterrent than anything else, another element of defense in depth. It's important to consider the underlying message Bruce was sending: if you've enabled WEP, or WPA with anything less than a truly random passphrase of 33 characters, you don't have security.

You have the illusion of security.

And that is far more dangerous than no security at all.

[advertisement] Peer code review without meetings, paperwork, or stopwatches? No wonder Code Collaborator won the Jolt Award.

Regular Expressions: Now You Have Two Problems

Jeff Atwood — Fri, 27 Jun 2008 10:00:00 -0800

I love regular expressions. No, I'm not sure you understand: I really love regular expressions.

You may find it a little odd that a hack who grew up using a language with the ain't keyword would fall so head over heels in love with something as obtuse and arcane as regular expressions. I'm not sure how that works. But it does. Regular expressions rock.They should absolutely be a key part of every modern coder's toolkit.

If you've ever talked about regular expressions with another programmer, you've invariably heard this 1997 chestnut:

Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.

The quote is from Jamie Zawinski, a world class hacker who I admire greatly. If he's telling us not to use regular expressions, should we even bother? Maybe, if you live and die by soundbites. But there's a bit more to the story than that, as evidenced by Jeffrey Friedl's exhaustive research on the Zawinski quote. Zawinski himself commented on it. Analyzing the full text of Jamie's posts in the original 1997 thread, we find the following:

Perl's nature encourages the use of regular expressions almost to the exclusion of all other techniques; they are far and away the most "obvious" (at least, to people who don't know any better) way to get from point A to point B.

The first quote is too glib to be taken seriously. But this, I completely agree with. Here's the point Jamie was trying to make: not that regular expressions are evil, per se, but that overuse of regular expressions is evil.

I couldn't agree more. Regular expressions are like a particularly spicy hot sauce -- to be used in moderation and with restraint only when appropriate. Should you try to solve every problem you encounter with a regular expression? Well, no. Then you'd be writing Perl, and I'm not sure you need those kind of headaches. If you drench your plate in hot sauce, you're going to be very, very sorry later.

In the same way that I can't imagine food without a dash of hot sauce now and then, I can't imagine programming without an occasional regular expression. It'd be a bland, unsatisfying experience.

But wait! Let me guess! The last time you had to read a regular expression and figure it out, your head nearly exploded! Why, it wasn't even code; it was just a bunch of unintelligible Q*Bert line noise!

Calm down. Take a deep breath. Relax.

Let me be very clear on this point: If you read an incredibly complex, impossible to decipher regular expression in your codebase, they did it wrong. If you write regular expressions that are difficult to read into your codebase, you are doing it wrong.

Look. Writing so that people can understand you is hard. I don't care if it's code, English, regular expressions, or Klingon. Whatever it is, I can show you an example of someone who has written something that is pretty much indistinguishable from gibberish in it. I can also show you something written in the very same medium that is so beautiful it will make your eyes water. So the argument that regular expressions are somehow fundamentally impossible to write or read, to me, holds no water. Like everything else, it just takes a modicum of skill.

Buck up, soldier. Even Ruby code is hard to read until you learn the symbols and keywords that make up the language. If you can learn to read code in whatever your language of choice is, you can absolutely handle reading a few regular expressions. It's just not that difficult. I won't bore you with a complete explanation of the dozen or so basic elements of regular expressions; Mike already covered this ground better than I can:

I'd like to illustrate with an actual example, a regular expression I recently wrote to strip out dangerous HTML from input. This is extracted from the SanitizeHtml routine I posted on RefactorMyCode.

var whitelist =
 @"||||||
  |||||
  |||||
  |||]+>|]+/?>";

What do you see here? The variable name whitelist is a strong hint. One thing I like about regular expressions is that they generally look like what they're matching. You see a list of HTML tags, right? Maybe with and without their closing tags?

Honestly, is this so hard to understand? To me it's perfectly readable. But we can do better. In most modern regex dialects, you can flip on a mode where whitespace is no longer significant. This frees you up to use whitespace and comments in your regular expression, like so.

var whitelist =
 @"|
   | (?# allow space at end)
   |
   |
   |
   |
   |
   |
   |
   |
   |
   | (?# h1,h2,h3)
   |
   |  (?# allow space at end)
   |
   |
   |
   |
   |
   ]+>|  (?# allow attribs)
   ]+/?> (?# allow attribs)
   ";

Do you understand it now? All I did was add a smattering of comments and a lot of whitespace. The same exact technique I would use on any code, really.

But how did I cook up this regular expression? How do I know it does what I think it does? How do I test it? Well, again, I do that the same way I do with all my other code: I use a tool. My tool of choice is RegexBuddy.

Now we get syntax highlighting and, more importantly, real time display of matches there at the bottom in our test data as we type. This is huge. If you're wondering why your IDE doesn't automatically do this for you with any regex strings it detects in your code, tell me about it. I've been wondering that very same thing for years.

RegexBuddy is far and away the best regex tool on the market in my estimation. Nothing else even comes close. But it does cost money. If you don't use software that costs money, there are plenty of alternatives out there. You wouldn't read or write code in notepad, right? Then why in the world would you attempt to read or write regular expressions that way? Before you complain how hard regular expressions are to deal with, get the right tools!

This trouble is worth it, because regular expressions are incredibly powerful and succinct. How powerful? I was able to write a no-nonsense, special purpose HTML sanitizer in about 25 lines of code and four regular expressions. Compare that with a general purpose HTML sanitizer which would take hundreds if not thousands of lines of procedural code to do the same thing.

I do have some tips for keeping your sanity while dealing with regular expressions, however:

Do not try to do everything in one uber-regex. I know you can do it that way, but you're not going to. It's not worth it. Break the operation down into several smaller, more understandable regular expressions, and apply each in turn. Nobody will be able to understand or debug that monster 20-line regex, but they might just have a fighting chance at understanding and debugging five mini regexes.
Use whitespace and comments. It isn't 1997 any more. A tiny ultra-condensed regex is no longer a virtue. Flip on the IgnorePatternWhitespace option, then use that whitespace to make your regex easier for us human beings to parse and understand. Comment liberally.
Get a regular expression tool. I don't stare at regular expressions and try to suss out their meaning through sheer force of will. Neither should you. It's a waste of time. I paste them into my regex tool of choice, RegexBuddy, which not only tells me what the regular expression does, but lets me step through it, and run it through some test data. All in real time as I type.
Regular expressions are not Parsers. Although you can do some amazing things with regular expressions, they are weak at balanced tag matching. Some regex variants have balanced matching, but it is clearly a hack -- and a nasty one. You can often make it kinda-sorta work, as I have in the sanitize routine. But no matter how clever your regex, don't delude yourself: it is in no way, shape or form a substitute for a real live parser.

If you're afraid of regular expressions, don't be. Start small. Used responsibly and with the right tooling they are big, powerful -- dare I say, spicy -- wins. If you make regular expressions a part of your toolkit, you'll be able to write less code that does more. It'll just.. taste batter.

You might enjoy them so much, in fact, that you completely forget about that "second problem".

[advertisement] Read the largest case study ever published about lightweight peer code review in Best Kept Secrets of Peer Code Review. Free book, free shipping.