Derek Thurn's Website

Why I switched from vim to emacs

Tue, 09 Aug 2011 17:51:44 +0000

UNIX text editor mastery has long been the hallmark of top coders in our profession, and I don't anticipate that changing any time soon. Last time we did a poll at Facebook, about 50% of our engineers reported using vim, about 30% using emacs, and the remaining 20% a variety of other editors and IDEs. So the editor holy war which started in the mid 1980s doesn't show any signs of stopping soon.

Approximately a year and a half ago (Februrary 2010), I took the plunge. I switched from vim to emacs as my primary text editor for coding tasks. Since I'd been using vim for the previous 3 years, it goes without saying that this was a long and at times arduous journey. Today, though, I use both text editors almost every day, preferring emacs for coding and vim for simpler editing tasks. I think that this gives me a unique perspective on the holy war between these two camps, so I'm going to use this blog post to share the particular reasons why I switched.

The Keyboard

For the holy inquisitors of vim (who are no doubt already preparing a daring commando raid to bring me to justice as the heretic I am), this argument begins and ends with manipulating text via the keyboard. I personally find the vim model to be marginally better for text manipulation tasks, but for text creation, there are some problems. The thing is that you need to leave insert mode in vim for basically every editing or navigational operation. So if I decide I need a comma at the end of the previous line of code after all, I need to hit Esc k Shift-a. In emacs, it's just C-p C-e.

Note also that chords can be types simultaneously. My pinky hits Caps Lock (a.k.a Ctrl) at the same time as I type p, and the holds it while I type e. In vim, I hit escape, then type k, and THEN type Shift-a. Once I've typed that comma it's Esc j Shift-a to go back to where I was (of course, I could use the arrow keys, but then my cool vim friends would spit on me). Very small edits like this are the norm during the process of creation. Several times during the writing of this essay, I've deleted the previous two words and typed something else, or gone back three words to use a different one. Obviously all of this stuff is customizable (I have my own shortcut system for both vim and emacs), but I think it shows that modal editing isn't necessarily the last word in text creation.

A secondary point is that the emacs keybindings are ubiquitous. Because they don't require any complex logic to figure out what editing mode you're currently in, almost every serous text editing application has them to some extent. For Eclipse, for example, the excellent Emacs+ plugin adds considerable power to Eclipse and makes it a lot more bearable to use. Last time I checked, there was a vim plugin for Eclipse, but it was pretty feature-poor and cost money.

Integration

People who don't use emacs like to joke "emacs is a great operating system... shame it doesn't have a decent text editor". To the advanced user, however, deep integration is the greatest strength of emacs. Programs written as emacs extensions all automatically share your emacs keyboard shortcuts and custom functions, and they all use text buffers as their primary user interface, making them automatically keyboard-friendly.

A great example of this is the ability to run a shell under emacs. The shell just appears in a regular emacs buffer. One of the best things about this is that you can actually interact with shell command output. Say, for example, that you type "git status" and you see that some file you just created isn't being tracked. If you're a vim user, what do you do? You type "git add" and then you either manually enter the path to the file or you get out the mouse and select the path from the git status output. In the emacs shell, you just navigate up to the git output and copy it, then go back down to the prompt and paste it there! You can edit things in the shell *exactly* like how you can edit regular text in emacs, and this works in all interactive programs. If you run IRB or the Python Shell under emacs, your emacs keybindings work and you can paste code in from your text buffers or copy results into files. No more wishing someone would build readline support into their program! It's brilliant.

Lisp

When you get right down to it, emacs is a lisp interpreter that happens to have text editing features. As a matter of fact, emacs lisp is a pretty crummy programming language. It still doesn't have lexical scoping (allegedly coming in Emacs 24), for one thing. And it's still pretty lacking in modern features for string manipulation, like decent regexes. But none of that matters, because it's a hell of a lot more of a real programming language than VimScript. Of course, editors like TextMate and to a lesser extent vim are "scriptable in any language" because their APIs define a small number of constrained places where you might want to change their behavior in some way. The emacs "API", if you can even call it that, lets you change anything, and this includes letting you do absolutely absurd things like try to write a video editor. This means that there are emacs plugins for everything. If you're willing to get your hands dirty, you can almost certainly replicate any feature from any other editor.

Emacs is a text editor for obsessive relentless customizers. The average TextMate user might have downloaded a bundle or two at some point in their life. The average vim user probably has three plugins and a vimrc file around 50 lines long. But emacs doesn't even get good until you hit 20 plugins and several thousand lines of lisp. It's the perfect choice for a control freak power user like me who wants to be consulted on how every tiny editing command should behave.

The Programmer Salary Taboo

Tue, 12 Apr 2011 16:08:13 +0000

Salary is an interesting topic. It's certainly one everyone has an opinion on. It's also a uniquely taboo subject among members of the working public. Since I'm about a month away from being done with University and entering the programmer workforce, I've taken to asking my classmates what type of starting salaries they're getting at their first jobs. The first thing I discovered is that not everybody is very comfortable with this question, and many companies are even less so. Microsoft requests very firmly that recipients of their offers keep their salary offers confidential, for example.

This reaction is very understandable on the part of a company like Microsoft. They want to keep that information secret if at all possible. If they made you a good offer and you tell your friends, your friends might have unreasonable expectations. If they made you a bad offer and you tell your friends, then you'll find out that it's bad, and you'll be unhappy. So working to keep this taboo in effect is clearly in every company's best interests. That certainly doesn't mean that you need to go along with the taboo, though!

From the perspective of an individual, disclosing your salary is a pretty good idea. Sharing salary information with your peers is a great way to get a sense for how well you're being compensated, which in turn is an invaluable tool in future salary negotiations. Compensation is, of course, very difficult to compare accurately, since different regions have different standards, but it's certainly a start!

To put my money where my mouth is, here's some information on the starting salaries for four big technology companies based on my informal observations. I'd be pretty hypocritical if I advocated disclosure without any sharing information myself! In each of these cases, I've consulted at least three people who received full-time offers from the company in question, so these figures are approximate (and specific to undergrads).

Google: The standard for the Mountain View office and a number of other large US offices is about $100,000/year, but they do scale that figure based on where you live by some secret factor. I've heard figures between $45k and $71k (tending towards the higher end) for the value of stock they give you over a four year vesting period, and Google bonuses can count for $45,000 per year or more.
Amazon: About $87,000/year, and they're fairly well known for not being too willing to negotiate starting salary. They typically offer between $40,000 and $50,000 worth of stock over four years, with a non-uniform vesting period (you get more the longer you stay).
Microsoft: Around $90,000 per year on average, although I've heard as low as $80,000. Typically comes with about $50,000 worth of stock over four years.
Facebook: Around $85,000-$90,000 per year. The stock situation is complicated, but I believe between 8,000 and 10,000 restricted stock units over four years is typical, which potentially could be worth more than $200,000 if you believe the secondary market prices, which have been as high as $28/share.

It's 2011, and my IDE still doesn't have a REPL!

Mon, 28 Feb 2011 13:32:55 +0000

The idea of a REPL (pronounced repple) or "Read-Evaluate-Print Loop" was developed in the 1960s at MIT for early Lisp systems, as a way of easily interacting with a programming language. Today, REPLs are sometimes called "shells" or "consoles", as in the "Python shell". Almost any Python, Ruby, or JavaScript programmer would tell you they couldn't live without one. It remains a mystery to me, then, why they haven't really caught on for mainstream programming languages like Java. Implementations do exist, but they're hardly popular. In this post, I intend to argue for wider adoption of this useful tool.

I believe that people go through a series of stages of REPL mastery. The first and most obvious use of the tool is to experiment with code, particularly with APIs. It's one thing to read documentation on how something is implemented, but it's quite another to be able to see how things work in action, to know that it will work for your case. For any remotely complex task, this is essential. I have no idea, for example, how Java programmers write even moderately complex regular expressions without going insane, since the complexity of the expression demands easy experimentation. Furthermore, there's a lot of code in the world without documentation, in which case experimenting with a REPL and reading the code may be your only choice.

Once you're playing with APIs interactively, the natural next step is to start developing your own code interactively. You can basically write a line at a time, and test things out as you go to make sure everything is working well. The lines that do the right thing, you copy into a text editor. Once you're comfortable doing that, well, you can move up to the next level. Does your language let you drop a breakpoint into some code and bring up a REPL at that point? It's difficult to overstate how useful this is for debugging and experimenting. All of the state of the program is set up for you properly. You can write complex code this way, just by setting a breakpoint where you want to write it. You can test everything out interactively as you code with real program values. It's great!

One objection that comes up from time to time in the discussion of interactive development is that it's tricky to usefully interact with a complex codebase, to instantiate the objects and so on that are required. This, however, leads to another benefit: unit testing. The fixtures and mock objects that you need to build to easily try out your code manually are precisely the same ones you'll need for unit testing later on. This leads to a very natural flow of development, where you interactively write a chunk of code, you manually test it out, and then you copy the working result into the code file and you copy the manual tests into your unit tests file. This is the most effective approach to unit testing that I've experienced, since testing flows so naturally from the experimental nature of interactive programming. I highly encourage everyone to at least try this approach to development!

XML: Contrary to popular belief, it doesn't always kill babies

Wed, 16 Feb 2011 13:37:15 +0000

XML. The "eXtensible Markup Language". What happened to you, man? You used to be so cool. People loved you. They took you to meet their parents. They went to the movies with you. They walked hand-in-hand with you down a moonlit beach! Those were heady days. Good days. Now, you're nothing. XML is the bogeyman in the closet and the monster under your bed. We've woken up from the drunken night of partying that was XML's prime to a morning hangover and a heap of bloated syntax that scars the human soul.

Look. XML isn't all that bad. XML is a medium. Sure, people have done terrible things with it (SOAP), but I choose to believe that XML was an unwilling accomplice in those deeds, not a malevolent co-conspirator. And yes, it's certainly true that there are a lot of good alternatives for XML. First, if you're making a config file, is it really called for? Have you considered a nice, human-readable flat file with key/value pairs? What about YAML? These formats probably won't drive your users insane. Please do not expect humans to edit XML config files by hand on a regular basis. Computers adore XML. It's nice and easy to parse, so structured and precise. Computers love that shit. People don't. So keep the human-editing to a minimum. Second, are you trying to define a data interchange format? JSON is a great way to go. It's extremely simple, and maps almost directly to real data structures in most programming languages. For something like a web service API, only use XML once you've shown JSON to be unsuitable through trial by fire.

Ok, so XML isn't always the right tool for the job. You know what they say: "When your only tool is XML, every problem looks like it's a schema declaration and a few XSLT transformations away from a nail". People have used it in the wrong way a lot, and that makes XML sad. So when should you use XML? I'd like to make two proposals. First, XML is a good tool to use if you're creating an actual document. I'm not saying that people should necessarily edit their documents in XML, mind you (Something like Markdown might be a better fit there), but that XML is a reasonable choice for storing documents. Microsoft might not be totally crazy for basing their docx file format on XML. Honestly, I'd expect them to be uniquely qualified by years of hindsight, shame, and regret to design a document format.

The second situation where XML is ok is when you've got a very specific form for your data: a heterogenous tree. XML is a natural choice for representing trees. The example of this that I personally experienced was in a parse tree. In the course of writing a compiler for a Java-based language, I championed XML as a parse tree format, and it paid off nicely. There just isn't a really nice way to represent a tree structure in something like JSON. XML was a good format for debugging what was going on, and furthermore, XPath turned out to be a godsend. Using a nice query syntax to select elements out of the parse tree for various semantic analyses made the rest of the compiler a lot easier. XML's great entrenched tool base and natural tree structure are sometimes exactly what a problem calls for.

Ten Mac OSX Applications I couldn't live without

Sun, 30 Jan 2011 20:04:31 +0000

It's basically impossible for me to imagine living today with any operating system other than Mac OSX. It has, at this point, become an extension of my body. The way I work on OSX is probably possible to replicate under a Windows or UNIX environment, but it would be a mammoth task. I often run into people who are new to the world of OSX and who are trying to learn their way around it. I find myself showing them cool applications in a piecemeal manner. It's occurred to me, though, that I should probably just write a blog post describing how I work, and then refer people to it in the future. Not all of these things will be applicable to you, but it's definitely worth your time to consider adding them to your tool set. Without further ado, then, here are the OSX applications I could never live without.

Firefox 4 + Pentadactyl

As far as I can tell, every other person in the world has abandoned Firefox for Chrome. I can certainly understand this, Chrome's a great browser. I use it myself as my secondary browser. However, Firefox has something Chrome won't have any time soon: Pentadactyl, a fork of another extension called Vimperator. This is a Firefox extension to give the browser a keyboard navigation system that's similar to what's possible in the vim text editor. A major theme of this blog post is going to be that you absolutely need to know how to get around your computer without the mouse, and being able to browse the web using only the keyboard is invaluable. I won't claim that it's always the way I browse the internet (see the section on Better Touch Tool), but it's great in plenty of situations. It's my opinion that switching between keyboard and mouse is what really slows you down. I'm aware that there are a few efforts to bring this functionality to Chrome, and I've tried them all. None of them compare. Fundamentally, the Chrome API is just too limited to allow anything of equivalent quality to exist. Learn how to use Pentadactyl to browse the web and many parts of your life will become incomparably better. Furthermore, Firefox 4 (the current beta) is very competitive in the performance category, so you don't lose too much that the Chrome users have. You need to download a nightly build of Pentadactyl to use it with Firefox 4, but it's worth it.

Aquamacs

I'm a recent Aquamacs convert. It's only really in the last year that I've switched to it as my primary text editor. Prior to that, I was a dyed-in-the-wool vim user. Realistically, I think there's a lot of value to both editors, and I still use vim occasionally, especially for editing over an SSH connection. Emacs is my tool of choice for most things, though (and I've come to feel that syncing things locally is almost always better than editing over SSH). Emacs' limitless customizability ensures that you can replicate basically any feature of another text editor in it. I'm also a big "emerging languages" fanboy, and Emacs is where you're most likely to find good editor support for the newer programming languages. Clojure, especially, is a big part of my life right now, and of course Emacs has by far the best support for it. If you've never developed software interactively using a tool like SLIME, you have no idea what you're missing. I could probably write another twenty paragraphs about Emacs here, but frankly, you're probably already an Emacs user or are happy with your existing text editor. Either way, you've heard it all before. There's a competing OSX version of Emacs called Carbon Emacs, but Aquamacs works a lot more like a native OSX app, and I love the command-line tool it bundles.

iTerm 2

iTerm 2 is a fork of the popular iTerm terminal emulator. It's only very recently become a usable alternative, which is great. The veteran Emacs user is probably at this point thinking "Wait, you use iTerm? Why not use one of the shells built into Emacs, like *eshell*, *shell*, or *term*?" In point of fact, I've used all three of those extensively, and the issue of shell selection has been a tricky one for me. It's far from a closed issue in my mind. Right now I'm mostly using iTerm instead of one of the Emacs native shells because it performs well and can drop easily into interactive applications like bpython or even vim as needed. Aquamacs comes with an awesome command line tool that lets me type "aquamacs filename.txt" to open it in my running Aquamacs instance seamlessly... just like I used to do with vim. So far, this really is the best of both worlds. The built-in OSX terminal is pretty great, but iTerm will get you some things that it doesn't support. My original reason for seeking it out was for 256-color support, which I use for pretty vim color schemes. It also has full-screen mode which I use a lot (fullscreen is great for my small 11-inch Macbook Air screen).

Quicksilver

Quicksilver is absolutely necessary for the serious keyboard enthusiast. You can use it as an application launcher, of course, which I do often. It's also equally valuable as an application-switcher. Suppose while writing this post, I decided I needed to IM my friend about something. Obviously I'm not about to switch from keyboard mode to mousing mode for something like that, that would be slow! Instead, I hit Cmd-4. This is a Quicksilver trigger that I've set up which opens Adium. I highly suggest setting up triggers to switch to applications you commonly use. This is the best method for application switching that I've found, it usually beats hunting around for the right application with Cmd-Tab (the sole exception, I think, is for going back to the previous application). Quicksilver is also how I pull up specific folders in Finder when I want to use Finder instead of the terminal... and you can use it to do things like unzip a downloaded file or mount a .dmg, all of which speed up your life. I'm aware that Quicksilver is not actively maintained any more, and that many of its former devotees are moving to competitors like LaunchBar. I haven't personally been convinced to put up the $35 for a LaunchBar license, but it may become necessary in time.

LyX

LyX is a document editor that I'm a big fan of. It's basically a GUI for writing LaTeX, the typesetting language of choice for most papers in computer science and mathematics. I've taken notes with LyX for a long time, and it's come to the point where I typeset basically everything with it. It's usually a better choice than Word, as long you don't have very specific formatting requirements in mind. LaTeX is a good tool for making professional looking documents quickly, and it's easy to drop into mathematical equations for times when that's necessary. Now that I've got some experience with it, I can take notes with LyX in a mathematics lecture just as quickly as I could with a pencil or pen. Writing math is a bit slower, but the fact that I type about five times as fast as I write gives me a chance to catch up. LyX also integrates with the clipboard pretty well. For example, you can select a diagram in OmniGraffle and paste it straight into LyX as an image.. the OSX clipboard is unquestionably one of the best parts of the Mac experience.

SizeUp

SizeUp is not free, but it's only $13 and worth every cent. It adds keyboard shortcuts to OSX to move windows around. This is amazingly useful for configuring two windows to show up side-by-side or for maximizing a window. The impossibility of replicating the "maximize" behavior in Windows is pretty much my least favorite OSX design decision, so SizeUp is a godsend. Incidentally, the runner up for my least favorite OSX design decision is the inability to cut files with Cmd-X in Finder, and you can buy a program called PathFinder to rectify that. That annoyance doesn't even come close to the lack of maximize, though. The green "zoom" button basically always does the wrong thing.

KeyRemap4MacBook

KeyRemap4MacBook is incredible, and virtually unknown. Basically, it lets you arbitrarily remap keys in the operating system, either system-wide or at the individual application level. It's also pretty easy to customize if you're not afraid of getting your hands dirty. One thing it does that I love is remap my Caps Lock key. Basically, I have Caps Lock set to send Ctrl in System Preferences, and then I can set it up with this program so that Caps Lock when pressed alone sends Esc and when pressed with another key sends Ctrl. It's actually like having two keys in one... totally awesome. I also frequently use it to add stuff to applications that are otherwise lacking. For example, I've got it set up so I can scroll up and down in Preview using J and K. Pretty sweet. It comes with many useful default keyboard shortcuts, such as the ability to set up more Emacs-style keybindings system-wide, so you can, for example, set up C-x C-s to save a file. Excellent.

Better Touch Tool

As I've indicated, I'm a big fan of the keyboard. I also think, however, that you use a computer in either "keyboard mode" or "mouse mode". The thing that really slows you down is having to switch between these modes. When I'm in "mouse mode", I don't want to have to type if at all possible. This is where Better Touch Tool comes in. It's basically like KeyRemap4MacBook, except for the trackpad. You can use it to define arbitrary trackpad gestures, either system-wide or for specific applications. For example, when I'm in Firefox, I use a bunch of trackpad gestures all the time. 3 finger swipes up and down take me to the top or the bottom of the page. A 3 finger tap opens a link in a new tab. I've got gestures for closing tabs, opening the most recently closed tab, even a few for navigating to sites I read all the time. Don't underestimate the usefulness of trackpad gestures, they can improve your life almost as much as keyboard shortcuts.

Fluid+Gmail Notifier

Fluid and Gmail Notifier both work together in my email strategy. Gmail Notifier is kind of an easy one. I like to know about new emails right away. I tried the whole "not obsessive-compulsively checking your email" thing, and it didn't work for me. Gmail Notifier gives me Growl notifications when I get a new email. It's cool like that, and my only wish is that it would check more often. Fluid is another useful program. It allows you to make what it calls a "Site-Specific Browser". This lets you treat a web page like an application. It gets its own dock icon and everything. Fluid also does a great job with Gmail notifications, pushing a dock badge when you get a new email. I used Fluid exclusively without Gmail Notifier for more than a year, but when I started working on smaller screens, I started to realize that the OSX dock just takes up too much room for the amount of information it provides. So now I auto-hide the dock, and find myself almost never missing it. Fluid is still great, though, especially for Gmail and Google Reader. I can be in Gmail in one second with Quicksilver. Incidentally, you should learn the basic Gmail keyboard shortcuts now if you don't already know them.

Dropbox

I hesitate to even mention Dropbox, because I feel like everyone in the world already knows about it. Dropbox is the best piece of software written in the past five years, hands-down. If you don't have Dropbox, get it. If you do have it, you probably aren't using it enough. Every single thing you do needs to be in Dropbox by default. End of story. Dropbox handles backup, it handles syncing, and it handles versioning. It's saved my life so many times that they all blur together into one perpetual wave of gratitude. The idea of creating content that's not in Dropbox is terrifying to me.