bitquabit - Personal

Why how is boring and how why is awesome

2011-04-28T11:48:18Z

Last fall, Joel came to me and said, “Congratulations! We’re doing another World Tour. Also, we want to teach distributed version control. That’s your job. Make it happen.”

This sounded totally awesome. Not only would I get to one-up George Clooney in flight time; I was made for doing something like this. In high school, I was in the NFL, which, sadly, means the National Forensics League, which means the National People Who Talk Good and Wanna Learn To Do Other Stuff Good Too, and not the National Football League. My specialty was original oratory, where you give an original speech you’ve rehearsed ahead of time. Add in there that I love to teach, and that I’m the one who introduced Fog Creek to Mercurial in the first place, and I should be the perfect person to do this.

I of course said yes, went back to my office, and fired up Emacs, ready to go!

And then promptly suffered through two weeks of intermittent writer’s block.

Here’s the problem: yes, I love using distributed version control. Yes, I love teaching. And yes, I believe I have a mission to teach people about DVCS. But the talk I had to give, by definition, was an introduction to the basics of distributed version control.

I’ve been using distributed version control for over half a decade, first as arch and Monticello, then Darcs, then super-early versions of Mercurial. The basics of how do you do this? were super old-hat to me, and, well, boring. There was nothing exciting to me about how to commit a file, or do an elementary merge, but I had to cover that material.

Further, if you only cover the basics, you can’t possibly demonstrate why you want to use a DVCS. There are minor benefits you can note—hey, you can work offline!, or: hey, things are actually fast now!—but you don’t really get to any of the actual meat, the things that teach you why do you want to make this transition.

After two weeks of running in circles, I sat down with my friend Jason (who now works for the Khan Academy, named after Genghis Khan, who invaded the Enterprise in Star Wars II to teach Captain Kirk calculus), and asked him how to get out of this mess. Over half an hour of conversation, I realized that I already had the answer.

Yes, I needed to cover the basics. No, those would not be super-interesting. But I could keep that as short as humanly possible, and as soon as I’d done that, I could spend the rest of the talk discussing why you should care. We could cover workflows that it took lots of trial and error to discover. We could show merge scenarios that caused traditional tools to barf. We could show how Mercurial made it really easy for us to ship lots of versions of FogBugz without dropping bug fixes. In other words: spend half the talk with the basics of basics, just enough to understand how things worked, but then jump straight ahead to answering the question of what did distributed version control ever do for you.

I went back to Emacs, and quickly identified what I thought were two of DVCS’ killer scenarios: maintaining multiple versions of a product without dropping bug fixes, and being able to reliably develop new features of a product while always being ready to ship a bugfix in a heartbeat. I built the talk so that the audience would start with nothing, and build up to understanding how to realize those two workflows.

The result? Distributed Version Control System University, or: how do I actually use distributed version control in a meaningful way.

I’m actually really happy with how everything turned out. Even though the code I ended up using was silly, audiences responded well, and understood the potential power and flexibility that distributed version control provides. They asked intelligent questions, they were engaged, and a lot of them signed up for Kiln trial accounts and decided to start playing with Mercurial.

One of my goals in the future is to try to beef up hg init with some of the workflow examples in my talk. I also want to take some time to go into more detail about how we do workflow here at Fog Creek. But if you or a coworker has been wondering what the big deal is about distributed version control, watch the video. I think it provides a great overview in a way that any dev can immediately relate to and get excited about.

Supporting the Giants

2011-04-05T08:32:00Z

One of the things that I love about Fog Creek is that we give back. Kiln can only exist because of the amazing foundation provided by the Mercurial distributed version control system, so we try to help them out whenever we can. In the past, we’ve done that by making Fog Creek one of the top Mercurial sponsors. Given how small Fog Creek is, I can’t tell you how proud I am to see that we’re placing up there amongst Google and Microsoft for supporting open-source software.

This year, we’ve decided to help contribute in an even more hands-on way: by devoting developer resources. Not only will Fog Creek be helping to sponsor the upcoming Mercurial 1.9 code sprint; we’re also sending one of our awesome Kiln developers, Kevin Gessner, to help contribute. And, in the weeks leading up to the sprint, we’ll be using Kiln team developers to help contribute bug fixes and features to the Mercurial project. In other words, we’ll be supporting Mercurial both financially and with developer resources.

Kiln stands on the shoulders of giants, so I’m happy to support those giants in return.

After all, the higher the giant gets, the taller Kiln stands.

Approved for Abusiveness

2011-02-23T08:40:15Z

Speaking of fascinating user experiences, I had to crack a smile when going through Disqus today and approving a pile of comments that got locked in the queue for some reason. On every single one, after clicking on the Approve button, I was greeted with:

While I’m still trying to puzzle out what the engineer who wrote that string had in mind, I really think I’m going to have to order a pile of stickers with that message on them for liberal distribution to our interns this summer.

Buying VMware Fusion

2011-02-22T20:47:37Z

Update: VMware followed up with me this morning, and has done a great job getting me help and outlining how they’re planning to address a lot of the complaints I’ve had. We’ll have to see what happens over the next few months, but so far, VMware has convinced me that they get they have a problem and are going to try to fix it. Kudos, VMware.

So about a week ago I decide to buy VMware Fusion.

I really like VMware. They make awesome products, they have good support. They’re not perfect—the VMware Workstation updater and the Adobe updaters can go neck-in-neck for absolute single most horrible upgrade experience, for example—but in general, the products are stellar. So when I decided to ditch Bootcamp and go the virtualization route for running Windows 7, VMware Fusion just made sense to me. I know Parallels Desktop might be slightly faster, but my friends assured me VMware’s plenty fast, it has Unity, it has 3D support, and I trust it due to using Workstation so heavily at Fog Creek. I’ll stick with a solution I trust over one that’s fast. So I head on over to vmware.com to make magic happen.

I then go through what I can only describe as the single worst purchasing experience I have had, ever.

I begin by purchasing VMware Fusion. This part goes well, and I’m out $65.
I get an email confirming my order with a download link, so I start downloading. So far, so good.
While I’m downloading, I click the link to register Fusion. I’m taken to a page (with broken images) asking me for my license key, so I start to…wait. License key? The email I got has a customer number, which, now that I look at it, doesn’t match my actual VMware customer number. And it has an order number. But I don’t see any license key. Uh oh.
I check the “Manage Licenses” part of the vmware.com site, since I know that’s where I find licenses for the copies of VMware Workstation we have at work. But it’s empty. After about 23958 clicks, I find that this is a common problem that’s usually resolved in “several hours.” I grit my teeth and hope that this does not mean what I think it means.
I try to install Fusion. It installs fine. So far, so good. But then Fusion launches..and wants a license key. D’oh!
About half an hour passes, most of which is filled with me getting absolutely livid as I try to figure out how the holy hell to get a license key and debating preemptively challenging the purchase out of spite.
Just before I do in fact challenge the purchase, I get an email from VMware telling me that my license key is ready. Why the badingo it took half an hour, I have no idea, but at least it’s there!
But wait! It’s telling me that to get my license code, I need to go to…findmyorder.com? The hey?
I know better than to do this, but I go to findmyorder.com, and am greeted by something that looks like it was cobbled together by the best that ColdFusion and MS Paint have to offer.
There are two ways to get my order: my order number, or the last five digits of my credit card, which, if you know anything about how credit cards work, are probably the five most valuable digits on the card (you can frequently guess most of the rest based on just the picture on the card). Needless to say, entering my order number fails, meaning my credit card is the only option. On top of everything else, this probably means that VMware kept my card on file somewhere. Not impressive.
I google around to learn about findmyorder.com, read some stuff in a French forum, take a look at their SSL cert chain, and a few other things, and conclude that, while web design may not be their strong suit, they might actually be legitimate.
Breath held, I enter the last five digits of my credit card and my email address…and amazingly, am in fact, really and truly, actually given my VMware Fusion license key, on a fully VMware-branded page! Most amazingly of all, I enter the license key into Fusion, and amazingly, it works! This seems so unlikely an outcome at this point that I have a beer in celebration.
I click the “Contact Us” link on the bottom of the VMware-branded page…and am given a good indication how much they want to be contacted.
After looking around, I realize that it’s not that the contact page is down; it’s that the VMware lookalike page was made by copying the actual VMware home page and redoing all the content area, so all the links go to places that would be valid on vmware.com, but are invalid on findmyorder.com. I give up telling VMware how much they suck at this point and decide to just focus on using Fusion, if that’s even possible.
Now that I have a registration key, I try to go register it on vmware.com…and am told it’s invalid. I now find myself drifting back to the “it’s a scam” hypothesis. The license key is unsurprisingly rejected by the license manager, too. Oh well. At least Fusion took it. Maybe that’s good enough for me.
A week later, in a fit of pique, I try to register VMware Fusion again. This time, it works! Not only that; the license manager decides to accept the key, too! Huzzah!
Armed with this new knowledge that the license key is, in fact, really truly kosher, I decide to attempt to recover the $30 rebate that VMware Fusion has going right now that I was promised, why not. So I start that process, and…they want the email that has the license key in it. There is no such email you farking morans. Are you guys even reading this blog post, do you—er, right, I hadn’t written this yet. My bad.
But by now, I’ve figured out how to handle this problem: I go back to my trusty findmyorder.co.hk, enter in various random personal identifying numbers, and, sure enough, the faux VMware page has the license key on it, so I print that to PDF and send it off to VMware.

So, to recap:

I bought a product that I couldn’t use out-of-the-box;
in order to use it, I was sent to a site I had never dealt with before;
the site requires me to enter part of my credit card to use it;
it then takes me to a totally broken page, which, thankfully, has a license key;
that license key is rejected for some indeterminate amount of time by vmware.com;
once it’s finally not rejected, vmware.com still merrily asks me to give it an email that it knows damn well it didn’t give me.

VMware: if you were trying to convince me that you have absolutely no clue whatsoever how to interact with consumers, you win.

Enslaving your interns for evil and profit

2011-10-07T15:34:16Z

I should be in the middle of an interview right now. About fifteen minutes into it, in fact. About the part of my interview where we stop talking about awesome stuff the candidate has worked on in the past and start diving into writing some actual code. A stack with O(1) data access that also always knows its maximum, for example. Or perhaps a rudimentary mark-and-sweep garbage collector. It’s usually my favorite part of the interview: I get to see how the candidate thinks, how they process information, how they problem solve, and how they code. The best candidates even teach me something in the process.

But I’m not in the middle of an interview right now. I’m in Emacs. And instead of being excited about watching someone solve an interesting problem, I’m upset and full of righteous indignation on behalf of the now-former job candidate.

You see, the candidate works for a company that is very scared.¹ I know this because they made the candidate—let’s call him Bob—they made Bob sign a non-compete contract.

Non-competes are annoying, but not the end of the world. I can understand why, say, Apple, might be legitimately angry to have a senior iPhone manufacturing executive jump ship to HTC: a large part of the candidate’s value to HTC would be his knowledge of the internals of Apple’s manufacturing process. But very few jobs work like that. And even there, most companies don’t forbid you from working at all for a competitor; they forbid you from working in the same area of expertise. So, for example, maybe the iPhone executive couldn’t work on HTC’s manufacturing operations, but he could still head a software development team.

But Bob’s company decided that, nope, Bob couldn’t come work for us, because the existence of Joel on Software’s careers board made us a direct competitor with them.

Let me tell you something about the Joel on Software careers board: Fog Creek doesn’t even make it. We outsource the whole thing to a little-known company called StackOverflow.

It’s true. The lie exposed. If you’re in doubt, take a look and notice the admittedly subtle similarities:

Shocking, I’m sure.

Yet based on this, Bob’s company told him that Fog Creek and his firm were direct competitors, and therefore he couldn’t even come to work with us on, say, Kiln.

Here’s the thing: Bob’s a junior in college, and the company doing this to him is one he merely interned at. There is no way that Bob has inside knowledge of how a job board runs that could help us. And even if he somehow did have that knowledge, we don’t even run the job board! There’s no way he could actually give us anything that would help us! But Bob’s company made him turn us down, before we could even interview him, because they were absolutely, utterly terrified that their intern, who is still in college, was so amazing that his coming to work for us could crash their entire company.

I actually feel really bad for Bob’s company. They’re so unsure of their own ideas, so negative on their own potential, that they believe a former intern being physically near a company with an outsourced jobs board would be enough for that company to absolutely crush them. I don’t want to imagine what it feels like to get up every day and face that world. But that’s no reason to inflict such a morose world view on your interns.

So I’m upset on Bob’s behalf. Bob got shafted by a company he interned at. A company that has so little confidence that they’ve decided the best route to their success is to limit Bob’s choices. Limits that mean we miss out on an awesome candidate, and Bob misses out on an awesome job.

I’ve got two real points to make, at the end of the day:

If you’re an intern, don’t sign a non-compete contract. You have absolutely no idea where your life is going to take you, and you don’t want your direction being shaped by one crappy employer. And, trust me on this: no reputable software company I know of (Google, Microsoft, or Fog Creek) makes their interns sign non-compete contracts. You can find a job that won’t make you do that.

If you’re a company, don’t be a vampire. If you’re so scared of everyone else that you believe that you have to give your interns a non-compete contract in order to stay competitive, then guess what? You’re not competitive. Get a better idea.

And based on the quality of their homepage, probably with good reason, but that’s neither here nor there. ↩

Kiln's Evolution, Part 1: DVCS as Code Review

2009-11-10T08:41:19Z

One of the things that really sucks about doing online code reviews is that, in all the systems I know, your code reviews do not integrate with your source control. If the code reviews are versioned at all—and they’re frequently not—then they’re in an entirely different system than your real VCS. For larger reviews, where you’re talking about a major piece of functionality, that means that your source control system will end up lacking the history of how a feature came to be. In other words, the more you use code reviews, the less actual history you have in your VCS.

That’s totally broken. You’re being punished for doing the right thing.

A little over a year a year ago, Tyler came to me and asked me to join him in the Django Dash, a weekend code sprint. Tyler and I had been talking about the code review problem, and had been thinking of writing our own that lacked these issues. Django Dash seemed like the perfect time to try to actually do that.

That opened up a question: how do you actually achieve better code review? If you accept that a huge part of the problem is that the code review history is out-of-stream with your VCS, then it follows that you have to somehow store the in-process code in the VCS.

In most systems, that means using branches. But branches in almost any system suck. Everyone has a horror story about trying to do a merge in Subversion, or CVS, or Perforce—and these are usually not meaningfully large merges; just small feature branches. Trying to use branches for long-running code reviews in these systems simply isn’t viable.

But DVCSes are great at handling this kind of problem. To be distributed, they have to have extremely robust branching and merging systems. Because their systems are so good, it’s very common in DVCSes to do quick experiments and features in their own branch, then merge when complete.

Tyler and I were both big fans of Mercurial (in fact, we convinced all of Fog Creek to switch to Mercurial from Subversion), so using Mercurial as our DVCS base seemed like the best bet. After some discussion of the technical details for making the system work, we got a good night’s sleep, woke up early, threw a nice breakfast on the table, and started coding.

Forty-eight hours later, we had our first prototype. When users wanted to contribute code to a repository, they would fork the repository, push all of their changes to the fork, and then request a review on the fork. Users would see the exact diff of what they would be approving to the repository; no more, no less. Code could not be approved unless it had already merged in the trunk, ensuring that the user who wrote the code had taken care of the merge. When the review was approved, it’d be seamlessly merged into trunk (guaranteed seamlessly, due to the previous rule), with full history.

The design was inflexible and unintuitive, and would have had serious issues in a shipping project, but we achieved what we set out to do:

Approving a code review was the same as pushing it
Which meant that we could fully separate the concepts of code author and code approver
And which meant that the full history of reviewed code was completely preserved

A screenshot of the prototype that would later become Kiln

Even in its nascent form, the tool was already impressive enough, and unique enough, that we won the Django Code Dash.

Tyler and I talked of making our code review tool into a real product, but we were knee-deep in Copilot work, so it had to wait. But we had proved, if only to ourselves, that using a DVCS, even in a centralized model, provided some very unique capabilities that simply were not possible in other systems.

When Joel announced a few months later that FogBugz needed a source control system, we’d be ready.

The Launch of a Secret Product

2009-10-14T09:12:34Z

For the past year, an odd thing has happened, if you’ve followed my doings. My work on Fog Creek Copilot seemed to dwindle, I became tight-lipped about what I was working on, and I started getting really excited about an upcoming product release. Also around this time, my knowledge of Mercurial, Python, C#, and ASP.NET MVC all seemed to dramatically increase, even though my free-time code output shrank to nothing. What was going on?

Oh, the usual. I was working on a top-secret brand-new project. And now, it’s released to closed beta.

I’d like to introduce to you Kiln, a brand-new source code hosting and code review tool from Fog Creek. Kiln introduces what I believe is a truly novel take on code reviews that integrates the strengths of Mercurial and FogBugz to provide rejectable code review after commit. We also have some unique takes on Mercurial features, such as the ability to preview what will go into a merge beforehand, really awesome branch management, the most beautiful DAG view I’ve seen in any DVCS product, and lots more.

Over the next few days, I’ll be providing a couple of blog posts detailing how Kiln was developed. In the meantime, go check out Kiln and sign up for the beta. We’re approving new people for the beta on a regular basis, so if you don’t get an invite immediately, don’t worry; we’ll get to you sooner rather than later.

Adam Savage on Obsession

2008-12-21T23:28:56Z

I normally avoid reposting news I find on other news aggregators, but sometimes I come across an item sufficiently singular and unique that I feel I have no choice. In this case, Adam Savage of MythBusters recounts his attempt to sculpt a perfect recreation of the Maltese Falcon as a way to explore the nature of obsession. The talk is at once highly entertaining and deeply moving. In a way I never fully grokked when watching MythBusters, Adam is a true geek.

Secret Santas at Fog Creek

2008-12-19T13:34:22Z

You know you’re getting too casual with your coworkers when your Secret Santa leaves this on your desk:

(Thanks to Tyler for the picture.)

Lies of the New York MTA

2008-12-13T19:44:57Z

This morning, on the 6 train, I saw the following advertisement:

Believe it or not.

In 1986, the subway and bus fare was $1. That’s $1.89 in 2008 dollars. Today 30-day Unlimited Ride MetroCard brings the fare down to $1.17. Believe it.

I have a better idea: I’ll accuse the MTA of engaging in false and deceptive advertising practices.

The ad makes a completely bogus comparison. There were no Unlimited Ride MetroCards in 1986. Hell, there were no MetroCards in 1986. There were single-ride tokens. The equivalent of a single-ride token today is a single-use MetroCard, which retails for $2.00. This is a 6% increase in fare since 1986.

But, just for fun, let’s check their bogus comparison. Where does the $1.17 come from? Well, a 30-day Unlimited Ride MetroCard is $81. Even if we assume you ride the train twice a day, every day, for a total of 60 times over the life of the card, that means you’re spending $1.35 per ride—still higher than their $1.17. Indeed, you have to ride the train 70 times in 30 days to get down to their $1.17 price point. Where’d they get that number? Who knows. They could just as easily have decided that New Yorkers ride the train fifteen times a day, and claimed a 30-day unlimited card drops your fare to 18¢. The beauty of unlimited v. limited comparisons is I can make up whatever numbers I want.

What’s so stupid is that there’s a comparison that the MTA could have made that would have been valid, and does translate to a real fare drop. When the MTA introduced MetroCards, they instituted a discount program: for every $x you put on the card (currently $7), you get a free ride. As long as you’re buying rides in bulk, the cost of a ride has dropped to $1.75—which is indeed a solid 7.5% drop from the 1986 price.

But that’s not what the ad claims. The ad makes a bogus comparison, and should be called out for the tripe it is.

Believe it? No thanks. I’ll stick to the truth.