sourcematters.org

Responsible Development a la Kent Beck

2013-02-03T21:50:00+00:00

Five years ago, I had the pleasure of attending Kent Beck’s keynote address at RailsConf 2008. It was him, sitting on a stool, telling stories about his involvement in developing the ideas of developers writing tests, patterns, and Extreme Programming. Really good stuff.

About two-thirds of the way through the talk he said “I think if you start with the principles of responsibility, accountability, and transparency, you end up with practices that look very much like XP’s but with a completely different set of motivations.” He called this Responsible Development. It resonated deeply with me.

That idea and his essay “Accountability in Software Development” have shaped the way I approach software development ever since. They have become the bar by which I measure how I conduct myself as a software developer.

If you have not read that essay–or have not read it recently–I highly encourage you to do so now. It is worthy of your time. You, too, as the essay explains, may find that you can sleep when the wind blows.

ShenRuby 0.2.0: Hello Ruby Interop

2013-01-17T04:40:00+00:00

I’m pleased to announce the release of ShenRuby 0.2.0. This release brings improved performance, bug fixes, and the beginnings of Ruby/Shen interoperablilty. Please see the release notes for the full details.

The following contrived example illustrates the interop features introduced with ShenRuby 0.2.0.

Fizz Buzz in Ruby and Shen

Imagine you are being interviewed for a job and the interviewer has just asked you to implement Fizz Buzz in any language you like. You admit that Fizz Buzz was not covered in your undergraduate algorithms class and that you have no idea what it is. The interviewer smiles and explains: “it’s no big deal. You just need to make a function that takes an integer as input and returns ‘Fizz’ if the number is divisible by 3, ‘Buzz’ if it is divisible by 5, ‘Fizz Buzz’ if it is divisible by both 3 and 5, and the string representation of the number otherwise. Please implement it and show that it works on the numbers 1 through 20.”

You’ve been reading up on Shen lately and the interviewer’s description of the Fizz Buzz translates easily to Shen’s pattern matching approach to function definition. But, the standard Shen system functions don’t include a divides? function to tests whether one number evenly divides another. No problem: that’s trivial in Ruby. So, you choose a hybrid approach using ShenRuby.

First, you create a new instance ShenRuby::Shen to be your Shen environment.

require 'rubygems'
require 'shen_ruby'

shen = ShenRuby::Shen.new

You read in ShenRuby’s README that any Ruby instance method added to the Shen environment is available for use within Shen. So, you add a divides? method:

class << shen
  # Returns true if b is evenly divisible by b.
  def divides?(a, b)
    b % a == 0
  end
end

You also read that the eval_string method may be used to evaluate arbitrary Shen code and is typically used for defining new functions. Now that you have divides?, implementing Fizz Buzz in Shen is trivial:

# Implement the fizz-buzz function in Shen
shen.eval_string <<-EOS
  (define fizz-buzz
    X -> "Fizz Buzz" where (and (divides? 3 X)
                               (divides? 5 X))
    X -> "Fizz" where (divides? 3 X)
    X -> "Buzz" where (divides? 5 X)
    X -> (str X))
EOS

Finally, you write a bit of Ruby to print out the first twenty results, taking advantage of the fact that Shen functions are invokable as methods on the Shen object:

(1..20).each do |i|
  puts shen.fizz_buzz(i)
end

Intrigued, the interviewer spends the rest of the time asking questions about Shen. You get the job.

Learning More about Ruby<->Shen Interop

The above example shows most of the capabilities of the Ruby/Shen interoperablilty as it exists in ShenRuby 0.2.0. I’ve left out a few details about automatic coercsions that take place between the Ruby and Shen worlds and some limitations with the current implementation. Those are both covered in the README section of the ShenRuby README.

The next interoperablilty feature that I plan to work on is the ability for Shen code to instantiate and interaction with native Ruby objects. The syntax for this will be inspired by Clojure’s Java interoperablilty syntax.

These are still the early days of the interoperablilty story in ShenRuby. I expect the APIs to evolve as they get more mileage. Feedback, suggestions, and bug reports are appreciated.

Announcing ShenRuby 0.1.0

2012-12-31T04:15:00+00:00

I discovered Shen–a functional programming language created by Mark Tarver–a few months ago while reading a 2011 blog post by Michael Fogus, author of The Joy of Clojure. As I read more about it, I was smitten. A modern, functional Lisp with pattern matching, currying, and an optional, very powerful static type system? Sign me up!

Something I found intriguing about Shen was that it was specifically designed for portability. It is implemented in a language called K Lambda, which is a tiny Lisp consisting of only 46 functions. Port K Lambda and you have ported Shen.

A few weeks ago, while playing with ideas for implementing K Lambda, I got to thinking: wouldn’t it be nice to have Shen ported to Ruby in a way that enabled the construction of hybrid applications? Tasks well suited to Shen’s pattern matching approach to functional programming could be implemented in Shen. Tasks well suited to OOP or imperative programming could be written in Ruby. The two parts could then run together on a Ruby VM.

Inspired, and with a copy of The Book of Shen in hand, I got to work.

ShenRuby

Today I am pleased to announce ShenRuby 0.1.0, the first release of my Ruby port of Shen. The goal of the 0.1.0 release is to provide a Shen 7.1 REPL that can be easily installed and used by Rubyists to explore Shen.

Here is zero to factorial in the ShenRuby REPL:

% gem install shen-ruby
Fetching: shen-ruby-0.1.0.gem (100%)
Successfully installed shen-ruby-0.1.0
1 gem installed
Installing ri documentation for shen-ruby-0.1.0...
Installing RDoc documentation for shen-ruby-0.1.0...
% srrepl
Loading.... Completed in 18.59 seconds.

Shen 2010, copyright (C) 2010 Mark Tarver
www.shenlanguage.org, version 7.1
running under Ruby, implementation: ruby 1.9.3
port 0.1.0 ported by Greg Spurrier


(0-) (define factorial
       0 -> 1
       X -> (* X (factorial (- X 1))))
factorial

(1-) (factorial 5)
120

(2-) (quit)
% 

In case it is not obvious, the (0-), (1-), and (2-) seen above are the Shen REPL’s prompts. The number shown increase with each expression evaluated.

At this point, ShenRuby only provides a REPL, launched via the srrepl command. As ShenRuby matures towards its 1.0.0 release, my focus will be on enabling bi-directional interaction between Ruby and Shen and on improving performance to the point that hybrid applications are viable.

Porting Shen to Ruby has been tremendous fun so far and I’m excited to start work on the features that will support easy interaction between Ruby and Shen.

Learning More

If you are interested in learning more about Shen, try using the ShenRuby REPL to work through the Shen in 15 minutes tutorial on the Shen website. Afterwards, explore the other resources found on the Learn Shen section of Shen’s website.

For more information on ShenRuby, please see the ShenRuby Repository on GitHub.

To Be Continued: Async Simplified

2012-12-02T19:30:00+00:00

Asynchronous programming can greatly boost the performance of I/O-bound applications. This wins it many proponents and is a major contributor to the popularity of Node.js.

Unfortunately, asynchronous programs can be challenging to write, read, and maintain. Left unchecked, they can devolve into piles of callback spaghetti and deeply nested anonymous callback function definitions. The latter are aptly called “Pyramids of Doom” by Trevor Burnham in his book Async JavaScript.

Flow control libraries like Async and Step improve the situation in JavaScript by managing the flow of the program from one asynchronous function to the next. But, even with these libraries, the interplay between synchronous and asynchronous code is still too awkward for my taste.

In Clojure, where we can bring macros into play, we can do better. The remainder of this post introduces an asynchronous Clojure/ClojureScript library that I have been working on called To Be Continued. Its goal is to make it easier to write, read, and maintain asynchronous Clojure and ClojureScript programs.

Callback Placeholders

Callback placeholders, indicated by ... at the end of a form, are the central feature of To Be Continued. When one is encountered during the expansion of TBC’s macros, it is replaced with a generated callback function that arranges for the flow of execution to continue where it left off once the result is available.

In addition to telling the TBC macros that a given form requires a callback, the ... serves as a hint to readers that an asynchronous call is taking place behind the scenes even though TBC is allowing the flow of the code to be expressed uninterrupted.

Callback placeholders are used with TBC’s threading macros and parallel binding form, described below.

Threading Macros

To Be Continued provides two asynchronous-aware threading macros, -+-> and -+->>, analogous to clojure.core’s -> and ->> macros, respectively. They expect a value or an expression returning a value as their first argument, a callback form to invoke with the final result as their last argument, and any number of intermediate forms in between.

The intermediate forms may be synchronous function invocations, asynchronous function invocations having a callback placeholder as their final argument, or a combination of the two.

The return value of TBC threading macros is always nil. It is expected that the final form will be a callback that will do something useful with the result passed to it.

For example:

(use 'to-be-continued.core)

(defn async-square
  "Invoke the callback with the square of x after 1 second"
  [x callback]
  (.start (Thread. (fn []
                     (Thread/sleep 1000)
                     (callback (* x x))))))

(-+-> 7
      (async-square ...)
      (str " monkeys")
      println)

;; => nil
;; 49 monkeys

Once the asynchronous result of async-square has been supplied to its automatically-generated callback, processing resumes with a synchronous call to str and then to println which serves as the final callback and displays the result.

Parallel Binding

The let-par macro is the asynchronous equivalent of Clojure’s let macro. It allows the results of multiple asynchronous functions, executed in parallel, to be bound to variables that can be referenced the body expression.

For example:

(defn async-sum-of-squares
  [x y callback]
  (let-par [x-sq (async-square x ...)
            y-sq (async-square y ...)]
    (-+->> (+ x-sq y-sq)
           (str "The answer is: ")
           callback)))

(async-sum-of-squares 2 5 println)
;; => nil
;; The answer is 29

Note that the use of the -+->> macro above is not strictly necessary because there are no asynchronous forms before the final callback. Its use is encouraged, however, for error handling purposes. Once error handling support has been incorporated into TBC (see Status and Next Steps, below) any errors that occur in the chain will be properly handled and propagated.

Like -+-> and -+->>, the value of a let-par expression is always nil. It is expected to invoke a callback with its result.

Example: ClojureScript + Node.js

To see To Be Continued in action, please check out tbc-node-example. It is an example project using TBC with ClojureScript and Node.js to asynchronously fetch data from GitHub’s API.

It has example usages of the macros described above as well map-par, a parallel, asynchronous equivalent of clojure.core/map.

Status and Next Steps

To Be Continued 0.1.0 supports both Clojure and ClojureScript. The source code is available on GitHub and a distribution package is posted on Clojars.

TBC currently lacks support for handling errors that occur during asynchronous computation. Therefore, it is not yet suitable for production use. Error handling is my next focus and will be available in the 0.2.0 release.

In the mean time, the existing functionality should be sufficient to get a feel for the library. Feedback is very welcome, whether as comments here or as issues filed on its GitHub project.

xmonad on OS X Lion

2012-06-28T02:30:00+00:00

I’ve been experimenting with distraction-free development environments on OS X Lion. I had high hopes for full-screen Emacs. Unfortunately, neither Eshell nor terminal-mode felt quite right for command line work. And, even with windmove-default-bindings, navigating around the three or four windows I was using to take advantage of the full-screen space was awkward.

About the time I was abandoning full-screen Emacs, I discovered xmonad through some tweets from @Baranoksy and @SeanTAllen. It is a tiling window manager for X Windows that can be controlled completely through the keyboard. xmonad is to X what tmux is to a terminal. Trying it on an Arch Linux VM gave me exactly what I was looking for: Emacs on the left and a variable number of XTerms stacked on the right.

Intrigued and excited to incorporate it into my daily work, I set out to get xmonad working full-screen on OS X Lion. It takes some work, and the experience is not seamless–see issues to be aware of, below–but it is good enough to for serious work. I’ve been using it as my primary development setup for the past week weeks and have been very happy with it.

The following sections describe the steps I went through to get xmonad up and running in a full-screen X11 environment on OS X Lion. I assume that you already have Homebrew installed.

Installing XQuartz

X11.app, the implementation of the X Window System that ships with OS X Lion does not support full-screen usage. To get that functionality, you must install the latest version of XQuartz.

Installation is simple: download the XQuartz-2.7.2 disk image, open it, and run the XQuartz.pkg installer. After installation, log out and log in again. This ensures that XQuartz will be recognized as the default implementation of X.

Building xmonad

xmonad is implemented in Haskell, so the first step is installing the Haskell compiler and platform:

brew update
brew install ghc haskell-platform

These packages take a while to build. Now is a good time to take a break, stretch your legs, and come back in a few minutes. If they are still building, peruse the xmonad Guided Tour to get a taste for the functionality that you will soon be enjoying.

Next, install xmonad’s dependencies using cabal, which is Haskell’s package manager:

cabal update
cabal install "X11 >=1.5.0.0 && <1.6"
cabal install "utf8-string ==0.3.*"

Finally, download, build, and install xmonad as an application for use by the current user:

mkdir ~/build-xmonad
cd ~/build-xmonad
curl -O http://hackage.haskell.org/packages/archive/xmonad/0.10/xmonad-0.10.tar.gz
tar xzf xmonad-0.10.tar.gz
cd xmonad-0.10
runhaskell Setup.lhs configure --user --prefix=$HOME
runhaskell Setup.lhs build
runhaskell Setup.lhs install --user

Configuring XQuartz for xmonad

Create an ~/.xinitrc file containing:

[[ -f ~/.Xresources ]] && xrdb -load ~/.Xresources
xterm &
$HOME/bin/xmonad

Now start XQuartz (it is installed in /Applications/Utilities). You will not see anything yet. Open the XQuartz Preferences with Command-, and update the following settings:

Output
- Enable “Full-screen mode”
Input
- Enable “Emulate three button mouse”
- Disable “Follow system keyboard layout”
- Disable “Enable key equivalents under X11”
- Enable “Option keys sent Alt_L and Alt_R”
Pasteboard
- Enable all of the options

We are now ready to toggle full-screen mode with Command-Option-A. After doing so, you should be greeted by a full-screen XTerm. Hitting Option-Shift-Return should open another Xterm. Now hit Option-Shift-Q to quit xmonad (and XQuartz): we’ve got a little more work to do.

Improving the Full-screen Experience

Three-finger swipe up to enter mission control
Hold down the Option key and click the + button in the top left. This will create another desktop.
Select the new desktop and launch XQuartz. Right click on its icon in the dock and choose “This Desktop” under Options. Now XQuartz will always launch into this desktop.

Issues To Be Aware Of

If you use a three-finger left or right swipe to move from the desktop running XQuartz to another desktop or full-screen app and then return, your xmonad windows will likely be missing. Simply click on the XQuartz icon in the dock to bring them back.

I have had a few instances in which the Command-Return key sequence to launch a new terminal stops responding. If you have a terminal open, it’s not the end of the world because you can still launch them manually from the command line. But, it is annoying and I typically restart xmonad as soon as I am at a stoping point. Unfortunately, I have not found a reliable way to reporduce this in order to file a bug report. Fortunately, it is pretty rare.

Optional: Building Emacs 24 with X Support

If you’re an Emacs user, you’ll likely want a version of Emacs 24 that has been built with support for X. To avoid conflicting with the packages in /usr/local managed by Homebrew, we’ll put this build in /opt/local:

brew install fontconfig libjpeg libtiff ungif 
cd ~/build-xmonad
curl -O http://ftp.gnu.org/pub/gnu/emacs/emacs-24.1.tar.gz
tar xzf emacs-24.1.tar.gz
cd emacs-24.1
./configure --prefix=/opt/local
sudo make install

Acknowledgements

These instructions were inspired by the instructions found on the Using xmonad on Apple OSX page.

The hint about dedicating a separate desktop space to X11 came from Run X11 applications full screen in Mac OS X Lion.

Thanks to Alex Baranosky and Josh Fleming for test driving earlier versions of these instructions.

Running the Pharo by Example Image

2012-04-20T05:00:00+00:00

This week brought Version 1.4 announcements for both Clojure and Pharo. The alignment of their version numbers was a coincidence, but I took it as a subtle hint from the computer language universe that I should renew my dabbling in Smalltalk.

The first time that I took Pharo for a spin, I got through the Professor Stef tutorial and was intrigued. I was beginning to see why people find Smalltalk to be so appealing. There is a real beauty in the simplicity and consistency of its object model. I got busy and sidetracked, though, and didn’t make it much further than that.

This time around, I am planning to work my way through Pharo By Example. Unfortunately, the image used by the examples in the book does not run on the VM that comes with Pharo 1.4: it crashes on startup. This post on the Pharo Users mailing list pointed me in the right direction. It turns out that the image does not run on the Squeak/Cog-based VM distributed with Pharo 1.4.

I don’t know about other platforms, but for OS X, the answer is to use the non-Cog Squeak 5.7.x Cocoa VM available at squeakvm.org. Download it, unzip it, and drag the Pharo by Example image onto it. That will launch the image and then you can start working through the book.

Happy Smalltalking!

Full-screen Emacs 24 for OS X Lion

2012-04-11T04:00:00+00:00

I have been experimenting with Emacs in full-screen mode as a way to reduce distractions. The Homebrew Emacs formula includes a patch providing the M-x ns-toggle-fullscreen command for switching between normal and full-screen modes. It works well, but does not provide the typical OS X Lion full-screen app experience. In particular, it remains on the desktop, obscuring non-full-screen applications, rather than moving to its own space.

Searching on the web yielded a patch by Konstantinos Efstathiou and his instructions for compiling Emacs as a proper full-screen application for OS X Lion. It looked promising, but I was reluctant to install updated versions of autoconf and automake, especially given his warnings. Searching further, I found an EmacsWiki entry referencing a Homebrew formula using the same patch, but it failed to build for me. In fact, all Homebrew builds for Emacs 24 were failing for me, so perhaps the HEAD of the git Emacs mirror it uses was broken.

Frustrated, I took inspiration from David Caldwell’s build-emacs script and used the following commands to build a Lion-only, full-screen capable version of Emacs 24.0.95:

curl -O ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-24.0.95.tar.gz
tar xzf emacs-24.0.95.tar.gz 
cd emacs-24.0.95/
curl https://raw.github.com/gist/1355895/860fb678838e3cceccf896f4116b13ee5815f526/lion-fullscreen.patch | patch -p1
sed -ie "s/colorWithCalibratedRed/colorWithDeviceRed/" src/nsterm.m
./configure --host=x86_64-apple-darwin --build=i686-apple-darwin --with-ns
make clean install 

For good measure, I also included the equivalent of the Homebrew Emacs formula’s --srgb flag to fix the Emacs color rendering bug on OS X.

Once the build is complete, you can test it with open nextstep/Emacs.app. Both M-x ns-toggle-fullscreen and the standard full-screen button in the top-right corner of the window work for switching modes. If everything is OK, you can then move it to /Applications or wherever else you normally keep your Emacs installation.

The jury is still out on my distraction-free-Emacsing experiment, but at least now it seamlessly integrates with the OS X Lion full-screen experience.

In My Toolkit: Midje

2012-03-12T00:00:00+00:00

When I switched to Clojure as my primary programming language, the top item on my shopping list was a testing library. Clojure comes with clojure.test, but after 5 years of using RSpec, I wanted something more in its style. I stopped searching when I found Midje. It was exactly what I was looking for. It even had something I didn’t realize I’d been yearning for: metaconstants–more on those later.

Highlights

Below are some of the highlights of Midje that are intended to give you a taste of its style and capabilities. For the details and additional features, please see its user guide and annotated example.

Facts

The fact macro is the basic Midje building block. With it you declare the expected result of an expression. It is striking in its simplicity:

(fact (+ 1 1) => 2)

The expression before the => is the expression being tested and the one after it is the expected result. In this case the result is a specific value, but a rich set of checkers allow for more generality.

Optional document strings allow you to elaborate on your tests:

(fact "basic addition"
  (+ 1 1) => 2)

Multiple related facts may be combined into a single fact–or facts, if you prefer–expression:

(facts "about basic addition"
  "one and one are two"
  (+ 1 1) => 2
  "two and two are four"
  (+ 2 2) => 4)

Prerequisites

RSpec has mocks and stubs. Midje has prerequisites. They are specified with the provided clause within a fact:

(fact (sum-of-squares 2 3) => 13
  (provided 
    (square 2) => 4
    (square 3) => 9))

Prerequisites both provide return values for a (possibly not-yet-implemented) function and ensure that it is being called as expected.

Metaconstants

Metaconstants are my favorite feature of Midje. They are an elegant solution to what would require extensive use of mock objects in RSpec. For example:

(fact
  (sum-scores [..game1.. ..game2..]) => 10
  (provided
    (score ..game1..) => 3
    (score ..game2..) => 7))

This states that sum-scores will be passing ..game1.. and ..game2.. through to the score function and then summing the results to get its result. The beauty is that the implementation of sum-scores does not care what sort of values ..game1.. and ..game2.. are and, thanks to metaconstants, neither does the test.

Midje Mode for Emacs

If you are an Emacs user, it is worth installing Midje mode. It gives you the ability to send individual facts to the browser and see the results as comments added to the buffer above the fact. This is a big time saver compared to running the tests through, e.g., the Leiningen Midje plugin and makes the red/green/refactor cycle very quick.

As an added bonus, it also updates the Clojure mode indentation rules so that facts format as shown in the above examples.

Getting Midje

Ready to give Midje a try? The source code is available on GitHub and the library is available on Clojars.