Nils Jonsson, software builder

Get your Cape on

Mon, 28 Nov 2011 14:06:00 -0600

Over Thanksgiving Day weekend I released a new RubyGem called Cape. You may find it useful if you’re a user of both Capistrano and Rake.

Install Cape by typing gem install cape at the command line, or make it a dependency of your project by using Bundler.

What can Cape do?

Mirror Rake tasks as Capistrano recipes, optionally filtered by namespace or name
Embed Rake tasks in a Capistrano namespace
Pass arguments to Rake tasks by setting environment variables with the same names
Override the default executables for local and remote Rake installations (/usr/bin/env rake is the default)
Enumerate Rake tasks for your own purposes

If you’ve ever wanted to execute a Rake task remotely, Cape can help. In your Capistrano recipes, include the following:

   require 'cape'

   Cape do
     # Create Capistrano recipes for all Rake tasks.
     mirror_rake_tasks
   end

Now all your Rake tasks can be invoked as Capistrano recipes. See the readme for more Cape-abilities.

Up, up, and away!

Taking out the trash: disposing of C♯ objects that should implement IDisposable but don't

Tue, 06 Sep 2011 11:29:00 -0500

My wife speaks of two kinds of trash: “clean” trash and “dirty” trash. This is not a distinction I consciously would have made without being married to her, but I recognize how useful the categories are. “Clean” trash (non-recyclable plastic and the like) can wait till morning for me to haul it outside and the city to pick it up, whereas “dirty” trash (cantaloupe rinds, a chicken carcass, etc) will stink up the house if left inside even for an hour.

Programmers writing code for the .NET Framework deal with these categories of refuse, too. Our clean trash is pure managed code. The garbage man takes care of this for us, and we rarely have to think about it.

Our dirty trash is unmanaged resources, such as handles to Win32 API objects. We want to get that stuff out of the house as soon as we’re done with it, rather than wait for the garbage man to show up. The .NET pattern for this is that such objects must implement the IDisposable interface. C♯ provides syntactic sugar for consumers to hygienically and conveniently handle IDisposable objects: the using statement.

Other people’s garbage

Sometimes while using somebody else’s API you run across a particularly smelly kind of trash:

An object allocates unmanaged resources.
The object provides a method for deallocating same.
The object does not implement IDisposable.

An example of this software antipattern is the OpenNETCF.Phone.Sim.Sim class. Its documentation states:

It is not recommended that you have more than one SIM object open in your application at one time.

This object helpfully proves a Close method, but it does not implement IDisposable. If it did, we could tidy up by writing:

But instead we’re reduced to this:

Where’s the sink? I need to wash my hands!

I’m not being paranoid if there are actually germs everywhere

I’ve written a class called Disposer that makes cleaning up dirty trash a less distasteful experience. It makes the using block useful for such objects as OpenNETCF’s Sim:

See? Your hands didn’t even get dirty.

Under the covers Disposer uses reflection to get at the Close() method of Sim. There is certainly a performance penalty for using reflection. If you’re calling the same disposal method many times, you can hang onto it and pass it to Disposer:

I wrote this code in a test-driven fashion, so hopefully I’ve managed to keep the bugs away.

Stay healthy!

Rebasing tags in Git repositories

Thu, 07 Apr 2011 14:53:00 -0500

UPDATE: I didn’t realize that git-filter-branch(1) has the option built in: --tag-name-filter. The moral of the story is: RTFM to avoid writing unnecessary code. :-/

Perhaps you’ve noticed commits in your Git repository that you or a team member created with the wrong credentials. Perhaps you see personal email addresses instead of corporate email addresses among the commit authors. Perhaps — for reasons known only to you and the empty bottle of Jägermeister in your recycle bin — there is a commit whose author name is that ridiculous nickname you acquired in high school. These can be career-limiting maneuvers.

How to fix these commits? git-filter-branch(1) is the answer. Mr Google can tell you about how to use it.

After you’ve used git-filter-branch(1) to rewrite history, you’ll notice that all the tags you had created on that branch are gone!

Don’t panic. The tags still exist, but they are pointing to the original commits prior to your rewriting exercise. You have to manually update these tags to point to the rewritten commits. You might try the following:

git tag -l to list all your tags.
For each tag, git show TAG_NAME to see the details of the old commit. Make note of the subject line, date, and hash of the old commit.
Page through git log looking for that subject line and date. Make note of the hash of the new commit when you find it.
git tag --force TAG_NAME NEW_COMMIT_HASH to update the tag.
Hope that you have not mistaken one commit for another with a similar subject line and date.

This is a seriously mind-numbing and error-prone process, especially if you have a dozen or more tags: freebasing not rebasing.

What you really want is a --tags option for git-rebase(1) that would do this work for you: git rebase --tags NAME_OF_REWRITTEN_BRANCH.

I’ve written a script that fills the bill. You invoke it with git-rebase-tags NAME_OF_REWRITTEN_BRANCH. If there is exactly one commit on that branch whose tree hash (i.e., diff) is identical to a tag, the tag gets repointed to that commit.

A votre santé!

Vanquish ghoulish bit rot in Ruby source code with Bundler, GitHub, and Ruby Tracker

Wed, 10 Nov 2010 22:03:00 -0600

UPDATE: I used htty as an example project in this post, but its use of Bundler has since changed. Because htty is a gem, its dependencies are now defined in htty.gemspec, along with all htty’s other RubyGems metadata.

Application projects (such as Rails applications) should use the techniques described in this post; RubyGems projects should not. To understand why, read Yehuda Katz’s post on this topic, “Clarifying the Roles of the .gemspec and Gemfile.”

I’ve made some amendments (in bold type) to this post.

Continuously upgrading your code’s dependencies can be a chore. But when you neglect the chore, it doesn’t take long for bit rot to start haunting you.

Today your code may run bug-free on Ruby interpreters and libraries. Tomorrow those other projects will advance and thereby invite poltergeists of incompatibility into your code. Before long your code won’t run at all on the versions everybody else will be using. There will be “fire and brimstone coming down from the skies, rivers and seas boiling, forty years of darkness, earthquakes, volcanoes, the dead rising from the grave, human sacrifice, dogs and cats living together — mass hysteria.” All right, all right — you get the point.

Bundler was created to exorcise the demons of dependencies in Ruby projects. Here I’ll explain a Bundler best practice for defining dependencies and a tip for upgrading dependencies over time.

By the way, I highly recommend getting comfortable with the combination of Bundler and RVM. It’ll help you wrangle combinations of Ruby interpreters and libraries with as much panache as Dr Venkman wrangled spooks, specters, and ghosts. You’ll say, “We came, we saw, we kicked its [stdout].”

Specify dependency versions in Gemfile

Gemfile is the file you use to describe your code’s dependencies to Bundler. Consider, for example, the Gemfile for htty (my console application for exploring web services and web sites). [Note that htty’s Gemfile no longer looks like this.]

A dependency can be tightened using a version spec expression (for example, gem 'rake', '~> 0.8', vs the looser gem 'rake'). But htty’s Gemfile contains no version specs.

Why is this?

First, version specs are already captured in Gemfile.lock. (Bundler generates Gemfile.lock from Gemfile.) There’s no need to duplicate this information. [Gemfile.lock is no longer under version control in the htty project.]

Second, omitting or loosening the version specs in your Gemfile means you can upgrade to a newer version of a dependency using a single Bundler command: bundle update name-of-gem. This command regenerates Gemfile.lock to reflect the newest available version of that dependency.

Note that when you first add a particular dependency to a Gemfile, you may wish to specify a version temporarily. For example, perhaps you want to use RSpec v2.0.0 in your project, even though the latest available version may be 2.1.0. You would:

Cite an older version of RSpec in Gemfile: gem 'rspec', '2.0.0'.
Update the bundle with the bundle command.
Remove the RSpec version spec in Gemfile: gem 'rspec'.
Commit the changes to Gemfile and Gemfile.lock.

The end result is exactly what you want. Your code depends on the desired, old version of RSpec, and your dependency configuration retains the convenience of a one-step upgrade when the need arises.

Keep up with evolving dependencies

How do you know when new versions of your dependencies are available? Must you periodically run bundle update to see what happens?

Enter Ruby Tracker, a useful addition to the GitHub open-source ecosystem. Ruby Tracker watches your project’s Gemfile.lock in your GitHub repository. It alerts you when new versions of the gems you depend on become available.

Gregg Pollack has a great little video explaining more about Ruby Tracker.

Create a Ruby Tracker account today and set it up to help you banish bit rot from your Ruby projects.

‘htty’ v1.1 is now available

Tue, 28 Sep 2010 22:53:00 -0500

Since its release three weeks ago, lots of people have started using my console application for HTTP called htty. It’s been called the perfect tool for exploring web services.

Install it with one command.

You’ll need Ruby and RubyGems. It’s known to work well under OS X against Ruby v1.8.7 and v1.9.2.

There’s a new version out today. What’s new?

HTTP Secure support
HTTP Basic Authentication support
Support for arrow keys and Emacs key bindings (thanks, rbxbx!)
Support for Ctrl-D for terminating input (thanks, bofrede!)
A smarter userinfo-set command that:
- Permits separate entry of username and (optional) password
- Permits colon-separated “username:password” entry
- Automatically URL-escapes @ symbols to avoid invalid URLs
Various bug fixes
Corrected and expanded built-in help content

Check out the updated readme for details on how to install and use htty.

Enjoy!

Introducing ‘htty’, the HTTP TTY

Mon, 06 Sep 2010 11:46:00 -0500

I’ve been hacking instead of writing during my spare time the last couple months. The result is a streamlined console application for exploring web services and web sites. It’s something of a cross between curl and the Lynx browser.

Installation

It couldn’t be much easier.

You’ll need Ruby and RubyGems. It’s known to work well under OS X against Ruby v1.8.7 and v1.9.2.

Features

Intuitive commands and command aliases
Support for familiar HTTP methods GET, POST, PUT, and DELETE, as well as HEAD, OPTIONS and TRACE
Automatic URL-encoding of query-string parameters and URL fragments
Transcripts, both verbose and summary
Dead-simple cookie handling and redirect following
Built-in help

The things you can do with htty are:

Build a request — you can tweak the address, headers, cookies, and body at will
Send the request to the server — after the request is sent, it remains unchanged in your session history
Inspect the server’s response — you can look at the status, headers, cookies, and body in various ways
Review history — a normal and a verbose transcript of your session are available at all times (destroyed when you quit htty)
Reuse previous requests — you can refer to prior requests and copy them

Examples

Here are a few annotated htty session transcripts to get you started.

Querying a web service

This simple example shows how to explore read-only web services with htty.

You can point htty at a complete or partial web URL. If you don’t supply a URL, http://0.0.0.0/ (port 80) will be used. You can vary the protocol scheme, userinfo, host, port, path, query string, and fragment as you wish.

The htty shell prompt shows the address of the current request.

The get command is one of seven HTTP request methods supported. A concise summary of the response is shown when you issue a request.

You can follow redirects using the follow command.

You can tweak segments of the address at will. Here we are navigating the site’s path hierarchy, which you can do with relative as well as absolute pathspecs.

Here we add query-string parameters. Notice that characters that require URL encoding are automatically URL-encoded (unless they are part of a URL-encoded expression).

The headers-response and body-response commands reveal the details of a response.

There was some cruft in the web service’s response (a horizontal line, a passage reference, verse numbers, a copyright stamp, and line breaks). We eliminate it by using API options provided by the web service we’re talking to.

We do a Julia Child maneuver and use the address command to change the entire URL, rather than add individual query-string parameters one by one.

Exit your session at any time by typing quit.

Working with cookies

The next example demonstrates htty’s cookies features, as well as how to review and revisit past requests.

Notice that when cookies are offered in a response, a bold asterisk (it looks like a cookie) appears in the response summary. The same cookie symbol appears next to the Set-Cookie header when you display response headers.

The cookies-use command copies cookies out of the response into the next request. The cookie symbol appears next to the Cookie header when you display request headers.

An abbreviated history is available through the history command. Information about requests in the history includes request method, URL, number of headers (and a cookie symbol, if cookies were sent), and the size of the body. Information about responses in the history includes response code, number of headers (and a cookie symbol, if cookies were received), and the size of the body.

Note that history contains only numbered HTTP request and response pairs, not a record of all the commands you enter.

The reuse command makes a copy of the headers and body of an earlier request for you to build on.

Understanding complex HTTP conversations at a glance using history

Assume that we have the following Sinatra application listening on Sinatra’s default port, 4567.

This application expects GET and POST requests and responds in various contrived ways.

Here you can see a request body being specified. Type body-set to enter body data, and terminate it by typing Return three times consecutively.

Also note how different response codes are rendered:

Response codes between 200 and 299 appear black on green to indicate success
Response codes between 300 and 399 appear white on blue to indicate redirection
Response codes between 400 and 499 appear white on red to indicate failure
Response codes between 500 and 599 appear flashing black on yellow to indicate a server error

As with the abbreviated history demonstrated earlier, verbose history shows a numbered list of requests and the responses they elicited. All information exchanged between client and server is shown.

Getting help

You can learn how to use htty commands from within htty.

The help command takes an optional argument of the abbreviated or full name of a command.

Coming soon

Here are some features that are coming down the pike.

HTTP Secure URLs

The HTTPS code is broken right now. Pardon the dust.

Commands for streamlining web form submission

These features will make htty better at screen-scraping.

Using any of the forthcoming form commands will clear any non-form content in the body of the request. Adding at least one URL-encoded form parameter to the request will set the Content-Type header to application/x-www-form-urlencoded. Removing all URL-encoded form parameters will remove this header.

form — display all form parameters offered in the response
form-fill — prompt in turn for a value for each of the form inputs in the response
form-add name value — add a URL-encoded form parameter for the request, using the specified name and value
form-remove name — remove a URL-encoded form parameter from the request, using the specified name
form-remove-all — remove all URL-encoded form parameters from the request

You will also be able to pop open a browser window containing request and response bodies.

Shiny curses goodness

We’ll have command history using the arrow keys, command autocompletion, and Tab key navigation of forms.

Custom command aliases and shell emulation of http-console

You should be able to make your own command aliases.

http-console has a nice command-line. We should have an http-console skin for htty.

Contributing

Your patches are welcome, and you will receive attribution for good stuff.

Fork the official htty repository located at http://github.com/htty/htty and send a pull request to htty.

News and information

Stay in touch with the htty project by following the Twitter account: get_htty.

You can also get help in the #htty channel on Freenode.

Never declare another C♯ delegate again

Tue, 25 May 2010 19:32:00 -0500

I recently did some refactoring of our .NET Compact Framework codebase. The goal was to tighten up our use of delegates a bit. In case you didn’t know, there’s never a need to declare custom delegates in C♯ code.

By way of a quick review, delegates are C♯’s strongly-typed function pointers. Closures, callbacks, and events are all implemented using delegates. The delegate statement declares the signature of a function pointer and gives that signature a name. You can then assign a variable of that delegate type to a method or an anonymous function whose signature matches the delegate.

This example, obtained via 1940s corporate espionage, illustrates how the club software running the legendary Minton’s Playhouse kept out unwanted patrons and novice musicians who couldn’t hack the hot new bebop tunes.

The body of the Nightclub#Bouncer function is defined outside the Nightclub class, possibly in another program altogether. (That part of the source code was lost when an intrepid corporate spy died trying to smuggle it out of Minton’s. It has been painstakingly reconstructed here.)

An annoying thing about the delegate statement is that it’s hard to know where to put it. Do we nest it inside a class, as shown above? Do we put it in a separate .cs file? Do we group multiple delegate statements in a file, or put each one in its own file?

Another misfortune of custom delegates is that each one is yet another thing that we have to come up with a name for. Naming is often cited as one of the harder things programmers must do. I myself have uttered Hungarian obscenities such as delegate void VoidFormDoubleDelegate(Form form, double @double); when a good name escaped me or when I wanted to reuse the declaration for multiple purposes.

Wouldn’t it be nice to do away with the delegate statement altogether in our applications? We can.

All delegates of a particular arity are alike and can be unified through generics. So all we need is a generic delegate declaration for each possible arity.

Microsoft has provided those declarations in the form of the System.Action and System.Func generic delegates, introduced as long ago as .NET 2.0. These generic delegates accommodate nullary procedures, unary procedures, binary procedures, ternary procedures, and so forth through 10-ary procedures. (‘Nullary’ means zero parameters; it’s represented by either the System.Action delegate or the System.Func delegate. A 10-ary procedure can use System.Action or System.Func.)

Clear as mud, right?

Here’s how the Minton’s code can be refactored to eliminate the unnecessary delegate declaration. The client code using Nightclub remains the same.

Admittedly, the refactoring amounts to a minuscule improvement in this example. Projects that make heavy use of custom delegates will benefit more dramatically.

Perhaps you need a delegate for a function that has 11 parameters and a return value. In that case you need to declare your own 11-parameter generic delegate because the .NET Framework Class Library only declares generic delegates up to an arity of 10.

If you need to declare an 11-parameter delegate, you’ve got bigger problems to solve than the question of where to put a delegate statement.

A quick ’n’ clean jQuery plugin for measuring password complexity

Wed, 12 May 2010 22:45:00 -0500

The tasty morsel of the day is a simple jQuery plugin. It makes suggestions to the user for how to strengthen a weak password. For the purposes of this example, strong passwords contain characters in all of four categories: capital letters, lower case letters, numerals, and punctuation characters.

Several jQuery and JavaScript best practices are in evidence here.

It’s written using only JavaScript’s “Good Parts.” Run your code through JSLint to find what my grandmother from Utah would have called “an accident what’s lookin’ fer a place ta happen.”

Reusable behavior is implemented as a jQuery plugin. Adding behavior with jQuery is a snap. But one-off jQuery snippets can be made more maintainable and reusable by rewriting them as plugins.

It plays nicely with libraries that may be in use besides jQuery. Like a number of other JavaScript frameworks, jQuery defines a $ function. Be a conflict avoider. Tell jQuery to keep its hands to itself by invoking no-conflict mode.

Behavior can be applied to multiple elements on a page. An easy mistake to make in your jQuery plugin work is to assume that only one element per page will be affected by a particular behavior. If all you’re doing is altering the element itself then this may work just fine. In this example, however, we’re adding password suggestions in a next to the password input box. Each of multiple input boxes with this behavior should have its own suggestions . Note the use of each and how we create a corresponding for each input box receiving the behavior.

Bon appétit!

Throw a wrench in the works

Fri, 30 Apr 2010 22:31:00 -0500

Programmers who come to Ruby from other languages — particularly C++, C♯ or Java — tend to misuse Ruby’s throw statement. They type throw when they mean to type raise.

What’s the difference?

Like C++, C♯ and Java, Ruby has first-class support for exceptions. Exceptions can be thrown using the raise statement and caught using the rescue keyword. Rubyists often speak of “errors” rather than “exceptions.” This is because it’s accepted that the names of Ruby exception classes generally end in Error rather than Exception, excepting the base class for exceptions which is called Exception. (Still with me?)

You may say, “Okay, so I’ll start saying ‘error’ when I mean ‘exception.’ But why can’t I at least type the throw and catch keywords I’m used to typing?”

Ruby has a distinctive language feature that consists of pairs of throw and catch statements. The throw-catch paradigm works similarly to raise and rescue. When Ruby encounters a throw statement, like a good matchmaker, she walks back up the execution stack (“catch me a catch!”) looking for a suitable catch. Here’s some code to illustrate.

What is inevitable result of toiling in Anatevka? Nothing too surprising: it is being a :pauper.

But let’s change Tevye’s fortunes a bit and reward him richly for his labor. Take a look.

What comes of Tevye’s hard work if deus ex machina descends upon the village? Tevye gets to retire to the western Caribbean, and his wife, Golde, grows a proper double chin.

Just as an unrescued error causes the Ruby process to crash, so a throw without a matching catch will raise an error (which in turn you can rescue).

It’s not just semantics

You cannot simply throw and catch wherever you would otherwise raise and rescue. There are differences beyond that the latter is for representing exception conditions and the former for other kinds of stack popping.

To begin with, (as you would expect from your experiences with other languages) rescue NameError rescues errors of type NameError as well as subclasses such as NoMethodError. By analogy you might presume that catch Entertainer would handle all these throw statements:

throw Entertainer
throw Entertainer.new
throw BroadwayStar
throw BroadwayStar.new

You would be wrong: catch Entertainer handles only throw Entertainer.

Another important difference is that throw can take two arguments, as demonstrated in the Tevye class above. The second argument to throw is what Ruby returns from the matching catch statement (also demonstrated above).

A further distinction is the fact that rescue can appear more than once in a begin-end or def-end pair. The first matching rescue in a sequence wins.

By contrast, catch is a block construct. Multiple catch statements must be nested in order to appear together.

Prior to Ruby v1.9, only symbols (for example, :pauper) could be thrown and caught. This restriction was removed, so now you can throw any Ruby object and catch it successfully.

The bottom line

When should you use this language feature? Not very often. As you probably noticed, my contrived example about peasant life in Tsarist Russia is a questionable application of the throw-catch pattern. There are two plausible examples in the Pickaxe book. Fundamentally, throw-catch is a way to escape from the bowels of multiple layers of control flow without abusing raise-rescue for this purpose.

But if you have so many layers of flow control that you’re tempted to throw in order to get out of them quickly, consider trying to eliminate the layers first.