One thing when looking at code metrics to keep in mind is that a single metric may not be as interesting. If you look at a metric trends over time it might help give you more meaningful information. Showing this trending information is one of our goals with Caliper. Metrics can be your friend watching over the project and like having a second set of eyes on how the code is progressing, alerting you to problem areas before they get out of control. Working with code over time, it can be hard to keep everything in your head (I know I can't). As the size of the code base increases it can be difficult to keep track of all the places where duplication or complexity is building up in the code. Addressing the problem areas as they are revealed by code metrics can keep them from getting out of hand, making future additions to the code easier.

I want to show how metrics can drive changes and improve the code base by working on a real project. I figured there was no better place to look than pointing metric_fu at our own devver.net website source and fixing up some of the most notable problem areas. We have had our backend code under metric_fu for awhile, but hadn't been following the metrics on our Merb code. This, along with some spiked features that ended up turning into Caliper, led to some areas getting a little out of control.

When going through metric_fu the first thing I wanted to start to work on was making the code a bit more DRY. The team and I were starting to notice a bit more duplication in the code than we liked. I brought up the Flay results for code duplication and found that four databases models shared some of the same methods.

Flay highlighted the duplication. Since we are planning on making some changes to how we handle timestamps soon, it seemed like a good place to start cleaning up. Below are the methods that existed in all four models. A third method 'update_time' existed in two of the four models.

 def self.pad_num(number, max_digits = 15)
    "%%0%di" % max_digits % number.to_i
  end

  def get_time
      Time.at(self.time.to_i)
  end

Nearly all of our DB tables store time in a way that can be sorted with SimpleDB queries. We wanted to change our time to be stored as UTC in the ISO 8601 format. Before changing to the ISO format, it was easy to pull these methods into a helper module and include it in all the database models.

module TimeHelper

  module ClassMethods
    def pad_num(number, max_digits = 15)
      "%%0%di" % max_digits % number.to_i
    end
  end

  def get_time
      Time.at(self.time.to_i)
  end

  def update_time
    self.time = self.class.pad_num(Time.now.to_i)
  end

end

Besides reducing the duplication across the DB models, it also made it much easier to include another time method update_time, which was in two of the DB models. This consolidated all the DB time logic into one file, so changing the time format to UTC ISO 8601 will be a snap. While this is a trivial example of a obvious refactoring it is easy to see how helper methods can often end up duplicated across classes. Flay can come in really handy at pointing out duplication that over time that can occur.

Flog gives a score showing how complex the measured code is. The higher the score the greater the complexity. The more complex code is the harder it is to read and it likely contains higher defect density. After removing some duplication from the DB models I found our worst database model based on Flog scores was our MetricsData model. It included an incredibly bad high flog score of 149 for a single method.

The method in question was extract_data_from_yaml, and after a little refactoring it was easy to make extract_data_from_yaml drop from a score of 149 to a series of smaller methods with the largest score being extract_flog_data! (33.6). The method was doing too much work and was frequently being changed. The method was extracting the data from 6 different metric tools and creating summary of the data.

The method went from a sprawling 42 lines of code to a cleaner and smaller method of 10 lines and a collection of helper methods that look something like the below code:

  def self.extract_data_from_yaml(yml_metrics_data)
    metrics_data = Hash.new {|hash, key| hash[key] = {}}
    extract_flog_data!(metrics_data, yml_metrics_data)
    extract_flay_data!(metrics_data, yml_metrics_data)
    extract_reek_data!(metrics_data, yml_metrics_data)
    extract_roodi_data!(metrics_data, yml_metrics_data)
    extract_saikuro_data!(metrics_data, yml_metrics_data)
    extract_churn_data!(metrics_data, yml_metrics_data)
    metrics_data
  end

  def self.extract_flog_data!(metrics_data, yml_metrics_data)
    metrics_data[:flog][:description] = 'measures code complexity'
    metrics_data[:flog]["average method score"] = Devver::Maybe(yml_metrics_data)[:flog][:average].value(N_A)
    metrics_data[:flog]["total score"]   = Devver::Maybe(yml_metrics_data)[:flog][:total].value(N_A)
    metrics_data[:flog]["worst file"] = Devver::Maybe(yml_metrics_data)[:flog][:pages].first[:path].fmap {|x| Pathname.new(x)}.value(N_A)
  end

Churn gives you an idea of files that might be in need of a refactoring. Often if a file is changing a lot it means that the code is doing too much, and would be more stable and reliable if broken up into smaller components. Looking through our churn results, it looks like we might need another layout to accommodate some of the different styles on the site. Another thing that jumps out is that both the TestStats and Caliper controller have fairly high churn. The Caliper controller has been growing fairly large as it has been doing double duty for user facing features and admin features, which should be split up. TestStats is admin controller code that also has been growing in size and should be split up into more isolated cases.

Churn gave me an idea of where might be worth focusing my effort. Diving in to the other metrics made it clear that the Caliper controller needed some attention.

The Flog, Reek, and Roodi Scores for Caliper Controller:

Roodi Report
app/controllers/caliper.rb:34 - Method name "index" has a cyclomatic complexity is 14.  It should be 8 or less.
app/controllers/caliper.rb:38 - Rescue block should not be empty.
app/controllers/caliper.rb:51 - Rescue block should not be empty.
app/controllers/caliper.rb:77 - Rescue block should not be empty.
app/controllers/caliper.rb:113 - Rescue block should not be empty.
app/controllers/caliper.rb:149 - Rescue block should not be empty.
app/controllers/caliper.rb:34 - Method name "index" has 36 lines.  It should have 20 or less.

Found 7 errors.

Roodi and Reek both tell you about design and readability problems in your code. The screenshot of our Reek 'code smells' in the Caliper controller should show how it had gotten out of hand. The code smells filled an entire browser page! Roodi similarly had many complaints about the Caliper controller. Flog was also showing the file was getting a bit more complex than it should be. After picking off some of the worst Roodi and Reek complaints and splitting up methods with high Flog scores, the code had become easily readable and understandable at a glance. In fact I nearly cut the Reek complaints in half for the controller.

Refactoring one controller, which had been quickly hacked together and growing out of control, brought it from a dizzying 203 LOC to 138 LOC. The metrics drove me to refactor long methods (52 LOC => 3 methods the largest being 23 LOC), rename unclear variable names (s => stat, p => project), move some helpers methods out of the controller into the helper class where they belong. Yes, all these refactorings and good code designs can be done without metrics, but it can be easy to overlook bad code smells when they start small, metrics can give you an early warning that a section of code is becoming unmanageable and likely prone to higher defect rates. The smaller file was a huge improvement in terms of cyclomatic complexity, LOC, code duplication, and more importantly, readability.

Obviously I think code metrics are cool, and that your projects can be improved by paying attention to them as part of the development lifecycle. I wrote about metric_fu so that anyone can try these metrics out on their projects. I think metric_fu is awesome, and my interest in Ruby tools is part of what drove us to build Caliper, which is really the easiest way try out metrics for your project. Currently, you can think of it as hosted metric_fu, but we are hoping to go even further and make the metrics clearly actionable to users.

In the end, yep, this is a bit of a plug for a product I helped build, but it is really because I think code metrics can be a great tool to help anyone with their development. So submit your repo in and give Caliper hosted Ruby metrics a shot. We are trying to make metrics more actionable and useful for all Ruby developers out, so we would love to here from you with any ideas about how to improve Caliper, please contact us.

I won't go into the details of how we discovered the issue, except to say that it manifested when using the Main and Servolux gems (both of which I highly recommend) together to start daemon processes. The problem can be boiled down to this brief snippet, distilled by Tim Pease:

ruby -e 'IO.popen("ruby -e \"STDOUT.dup; STDOUT.close; sleep\"").read

If you try to execute that line, the #read will hang indefinitely, even though STDOUT was closed in the child process. Why? Tim explains:

"...the parent script will block waiting for pipe to close in the child process. However, because there is a duplicate of the STDOUT file descriptor in the child, both must be closed or the parent will never unblock from the read. [...] ALL file descriptors have to be closed before an EOF is sent through the pipe to the parent. Hence, the read will never return."

The moral of this story: don't try to read from a subprocess which duplicates STDOUT or STDERR handles. Or if you do, take precautions by using calls which don't wait for an EOF - such as #readpartial - instead of #read.

I can take no credit for solving this one. All the honor goes to Ara T. Howard, who expended much mental energy and beer to get to the bottom of the problem; and Tim Pease, who neatly summarised the issue and explained it in terms that even I could understand. I'm documenting it here in case anyone else runs into it.

Helpers

First, though, let's recap some of our boilerplate code. Here's the preamble code which is common to all of the demonstrations in this article:

require 'rbconfig'

$stdout.sync = true

def hello(source, expect_input)
  puts "[child] Hello from #{source}"
  if expect_input
    puts "[child] Standard input contains: \"#{$stdin.readline.chomp}\""
  else
    puts "[child] No stdin, or stdin is same as parent's"
  end
  $stderr.puts "[child] Hello, standard error"
  puts "[child] DONE"
end

THIS_FILE = File.expand_path(__FILE__)

RUBY = File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])

#hello is the method which we will be calling in a Ruby subprocess. It reads some text from STDIN and writes to both STDOUT and STDERR.

THIS_FILE and RUBY contain full paths for the demo source file and the the Ruby interpreter, respectively.

Method #6: Open3

The Open3 library defines a single method, Open3#popen3(). #popen3() behaves similarly to the Kernel#popen() method we encountered in part 2. If you remember from that article, one drawback to the #popen() method was that it did not give us a way to capture the child process' STDERR stream. Open3#popen3() addresses this deficiency.

Open3#popen3() is used very similarly to Kernel#popen() (or Kernel#open() with a '|' argument). The difference is that in addition to STDIN and STDOUT handles, popen3() yields a STDERR handle as well.

puts "6. Open3"
require 'open3'
include Open3
popen3(RUBY, '-r', THIS_FILE, '-e', 'hello("Open3", true)') do
  |stdin, stdout, stderr|
  stdin.write("hello from parent")
  stdin.close_write
  stdout.read.split("\n").each do |line|
    puts "[parent] stdout: #{line}"
  end
  stderr.read.split("\n").each do |line|
    puts "[parent] stderr: #{line}"
  end
end
puts "---"

When we execute this code, the result shows that we have captured the subprocess' STDERR output:

6. Open3
[parent] stdout: [child] Hello from Open3
[parent] stdout: [child] Standard input contains: "hello from parent"
[parent] stdout: [child] DONE
[parent] stderr: [child] Hello, standard error
---

Method #7: PTY

All of the methods we have considered up to this point have shared a common limitation: they are not very well-suited to interfacing with highly interactive subprocesses. They work well for "filter"-style commands, which read some input, produce some output, and then exit. But when used with interactive subprocesses which wait for input, produce some output, and then wait for more input (etc.), their use can result in deadlocks. In a typical deadlock scenario, the expected output is never produced because input is still stuck in the input buffer, and the program hangs forever as a result. This is why, in previous examples, we have been careful to call #close_write on subprocess input handles before reading any output.

Ruby ships with a little-known and poorly-documented standard library called "pty". The pty library is an interface to BSD pty devices. What is a pty device? In BSD-influenced UNIXen, such as Linux or OS X, a pty is a "pseudoterminal". In other words, it's a terminal device that isn't attached to a physical terminal. If you've used a terminal program in Linux or OS X, you've probably used a pty without realizing it. GUI Terminal emulators, such as xterm, GNOME Terminal, and Terminal.app often use a pty device behind the scenes to communicate with the OS.

What does this mean for us? It means if we're running Ruby on UNIX, we have the ability to start our subprocesses inside a virtual terminal. We can then read from and write to that terminal as if our program were a user sitting in front of a terminal, typing in commands and reading responses.

Here's how it's used:

puts "7. PTY"
require 'pty'
PTY.spawn(RUBY, '-r', THIS_FILE, '-e', 'hello("PTY", true)') do
  |output, input, pid|
  input.write("hello from parent\n")
  buffer = ""
  output.readpartial(1024, buffer) until buffer =~ /DONE/
  buffer.split("\n").each do |line|
    puts "[parent] output: #{line}"
  end
end
puts "---"

And here is the output:

7. PTY
[parent] output: [child] Hello from PTY
[parent] output: hello from parent
[parent] output: [child] Standard input contains: "hello from parent"
[parent] output: [child] Hello, standard error
[parent] output: [child] DONE
---

There are a few of points to note about this code. First, we don't need to call #close_write or #flush on the process input handle. However, the newline at the end of "Hello from parent" is essential. By default, UNIX terminal devices buffer input until they see a newline. If we left off the newline, the subprocess would never finish waiting for input.

Second, because the subprocess is running asynchronously and independently from the parent process, we have no way of knowing exactly when it has finished reading input and producing output of its own. We deal with this by buffering output until we see a marker ("DONE").

Third, you may notice that "hello from parent" appears twice in the output - once as part of the parent process output, and once as part of the child output. That's because another default behaviour for UNIX terminals is to echo any input they receive back to the user. This is what enables you to see what you've just typed when working at the command line.

You can alter these default terminal device behaviours using the Ruby "termios" gem.

Note that both STDOUT and STDERR were captured in the subprocess output. From the perspective of the pty user, standard output and standard error streams are indistinguishable - it's all just output. That means using pty is probably the only way to run a subprocess and capture standard error and standard output interleaved in the same way we would see if we ran the process manually from a terminal window. Depending on the application, this may be a feature or a drawback.

You can execute PTY.spawn() without a block, in which case it returns an array of output, input, and PID. If you choose to experiment with this style of calling PTY.spawn(), be aware that you may need to rescue the PTY::ChildExited exception, which is thrown whenever the child process finally exits.

If you're interested in reading more code which uses the pty library, the Standard Library also includes a library called "expect.rb". expect.rb is a basic Ruby reimplementation of the classic "expect" utility written using pty.

Method #8: Shell

More obscure even than the pty library is Ruby's Shell library. Shell is, to my knowledge, totally undocumented and rarely used. Which is a shame, because it implements some interesting ideas.

Shell is an attempt to emulate a basic UNIX-style shell environment as an internal DSL within Ruby. Shell commands become Ruby methods, command-line flags become method parameters, and IO redirection is accomplished via Ruby operators.

Here's an invocation of our standard example subprocess using Shell:

puts "8. Shell"
require 'shell'
Shell.def_system_command :ruby, RUBY
shell = Shell.new
input  = 'Hello from parent'
process = shell.transact do
  echo(input) | ruby('-r', THIS_FILE, '-e', 'hello("shell.rb", true)')
end
output = process.to_s
output.split("\n").each do |line|
  puts "[parent] output: #{line}"
end
puts "---"

And here is the output:

8. Shell
[child] Hello, standard error
[parent] output: [child] Hello from shell.rb
[parent] output: [child] Standard input contains: "Hello from parent"
[parent] output: [child] DONE
---

We start by defining the Ruby executable as a shell command by calling Shell.def_system_command. Then we instantiate a new Shell object. We construct the subprocess within a Shell#transact block. To have the process read a string from the parent process, we set up a pipeline from the echo built-in command to the Ruby invocation. Finally, we ensure the process is finished and collect its output by calling #to_s on the transaction.

Note that the child process' STDERR stream is shared with the parent, not captured as part of the process output.

There is a lot going on here, and it's only a very basic example of Shell's capabilities. The Shell library contains many Ruby-friendly reimplementations of common UNIX userspace commands, and a lot of machinery for coordinating pipelines of concurrent processes. If your interest is piqued I recommend reading over the Shell source code and experimenting within IRB. A word of caution, however: the Shell library isn't maintained as far as I know, and I ran into a couple of outright bugs in the process of constructing the above example. It may not be suitable for use in production code.

Conclusion

In this article we've looked at three Ruby standard libraries for executing subprocesses. In the next and final article we'll examine some publicly available Rubygems that provide even more powerful tools for starting, stopping, and interacting with subprocesses within Ruby.

Caliper is the easy way to get code metrics - like cyclomatic complexity and code duplication - about your project. Just go to http://devver.net/caliper. Enter the public Git repository URL of a Ruby project. In a few moments, we'll give you a full set of metrics, generated by the awesome metric_fu gem.

If your project is hosted on Github, we can even re-generate metrics automatically every time someone makes a commit. Just add the URL http://api.devver.net/github to your project's Post-Receive Hooks.

Caliper is currently an Alpha product. We will be be rapidly adding features and refinements over the next few weeks. If you have a problem, a question, or a feature request, please let us know!

To further our vision of enabling Ruby developers to easily and quickly run their tests in the cloud, we've recently released the very first version of a HTTP-based Devver API. Right now, it offers a single service: a GitHub compatible web hook which will cause Devver to do a continuous integration-style build of your project, and email you the results.

The API is still in it's infancy but we already have some cool features:

We're releasing very early, so there are some rough spots (notably, the output is too verbose and the setup is too complicated). But we're also releasing often, so please let us know what you think and which features you'd like to see added.

If you have a project on GitHub, please try out the API and send us feedback. We'd really appreciate it!

Important Changes

The biggest fixes are in the default install_dependencies hook (located in .devver/hooks/install_dependencies). The old default hook failed to install Rails correctly, so you may have repeatedly seen an error like this:

Missing the Rails 2.3.3 gem. Please `gem install -v=2.3.3 rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.

In addition, the default hook now runs rake gems:install RAILS_ENV=test instead of rake gems:install to ensure that any gems declared in config/environments/test.rb get installed.

Updating your hook

If you have not altered your install_dependencies hook, you can just do the following to get the new behavior:

rm .devver/hooks/install_dependencies
devver --init

If you have altered this file, you'll need to do the following to get the new behavior:

mv .devver/hooks/install_dependencies .devver/hooks/install_dependencies.mine
devver --init

Then port your project-specific changes from install_dependencies.mine to install_dependencies.

As always, if you have any problems, please don't hesitate to let us know by emailing support@devver.net.

Dave Thomas
Dave was the first speaker for LSRC, and it was a great way to kick off the event. Dave gave a talk about Ruby not being perfect and that is why he likes it. I have heard Dave speak before, and I always enjoy his talks. It isn't like you learn anything specific about Ruby development, but you learn about the Ruby community. Actually, Dave would say we are a collection of Ruby communities, and that having a collection of communities is a good thing. It was also interesting to hear Dave speak about the entire Zed, "Rails is a Ghetto" incident. Sometimes when you are angrily ranting around online, it is easy to forget that there are real people attached to things. Feelings can get hurt, and while Dave agrees there is some valid points in the post, I think it shows that it probably isn't a good way to go about fixing them. Dave really loves Ruby and the weird things you can do with the language and it shows.

Glenn Vanderburg, Programming Intuition
Glenn talked about phyical emotions tied to code, such as a sense of touch or smell. The talk generally just evoked memories of Paul Graham's "Hackers and Painters" in my head, in fact Glenn talked about PG during his talk. The best programmers talk about code as if they can see it. The talk explored ways to feel the code and react to it. It tried to promote the idea that it is OK to just have a gut reaction that some code is a bad way to do things, because we should sense the code. Glenn also played a video showing Bobby McFerrin teaching the audience the Pentatonic scale, which I really enjoyed.

James Edward Gray II, Module Magic
James visited Japan recently and went to a Ruby conference, and he really enjoyed it. About half his talk was why Japan is awesome... He then found little ways to tie this back into his talk about Ruby and Modules. It covered some interesting topics like load order that many people just don't know enough about, but use every day. Examples of the differences between include and extend. Modules are terrific at limiting scope, limit the scope of scary magic and keep it hidden away. I enjoyed talking with James a good amount through out the conference. I had never met him before LSRC, but I used to practice Ruby working on Ruby Quiz which he ran for a long time.

James has his slides are up, Module Magic

Fernand Galiana, R-House
Fernand gave a really cool and demo heavy talk about home automation. He has a web front end that lets him interact with all his technology. His house tweets most of the events that it runs. The web interface has a iPhone front in, so he can, on the go, change the temperature or turn off lights. I have always been a real home automation geek. When I was growing up, my dad loved playing with an X-10 system that we had in the house. I am really interested in playing with some of this stuff when I have my own place, mostly looking at ways I could use it to cut waste on my energy usage.

Mike Subelsky, Ruby for Startups: Battle Scars and Lessons Learned
* You Ain't Gonna Need It (YAGNI), don't worry about being super scaling at the beginning...
* Early days focus on learning more data about what your building and what your customers want concentrate on the first 80% solution.
* Don't over build, over design, or over engineer.
* Eventually plan to move everything out of your web request, build it so that it will be easy to do in the future, but it isn't worth doing at first. (delayed job, EM, etc)
* Make careful use of concurrency, prefer processes communicating via messages (SQS etc...) If you are doing threading in your software EM is your friend.
* Avoid touching your RDBMS when you are storing not critical data:
- Storing large text blogs in S3, message across SQS, tons of logging SDB
* Don't test all the time at the beginning, it gets in the way of exploration... Things that is mission critical maybe should be BDD as it will be the most stable and least likely to change part of your code

Mike posted his slides on his blog, Ruby for Startups.

Jeremy Hinegardner, Playing nice with others. -- Tools for mixed language environments

Jeremy wanted to show how easy it is to use some code to make it easy to work with a system that uses multiple languages. He brought up that most projects in the room utilize more than one language. That it will be more common as systems grow in complexity. He looked at a lot of queues, key value stores, and cache-like layers that can be talked to by a variety of language. He then showed some code that would quickly demonstrate how easy it was to work with some of these tools. Extra points because he talked about Beanstalkd, which I personally think is really cool. I think nearly everyone is starting to look at work queues, messaging systems, and non standard databases for their project and this was a good overview of options that are out there.

Yukihiro Matsumoto (Matz), Keynote and Q&A
Matz gave a talk about why we, as a community, love Ruby. In this talk there weren't really any takeaways that were specifically about Ruby code but more about the community and why Ruby is fun. He spent a good amount of time talking about Quality Without A Name, QWAN. More interesting than the talk was the Q&A session. I thought the most interesting question was why Ruby isn't on Git yet. He said the teams doesn't have time to convert all the tools they use from SVN to git. He also mentioned that the git project tracking SVN is very close to the SVN master and is a good way to work on the Ruby code.

Evan Light, TDD: More than just "testing"
Evan first covered that the tools we as a community keep getting excited about aren't really what matters. What matters is TDD technique. After discussing why tools aren't as important for awhile, Evan began live coding with the audience. Something I thought was pretty impressive as it would be difficult to do. It made for a weird pair programming exercise with the entire audience trying to drive. Which sometimes worked well and sometimes lead to conflicting ideas / discussion (which made for interesting debate). It was over all a really interesting session, but it is hard to pull out specific tidbits of wisdom from the talk.

Jake Scruggs, What's the Right Level of Testing?
I have known of Jake for awhile from his work on the excellent Metric Fu gem. Jake explored what the right level of testing for a project is, from his experience on his last nine projects over the years. He explored what worked, what didn't and what sometimes works but only depending on the people and the project. I think it comes to this conclusion: what works for one project won't work for all projects. Having some testing and getting the team on a similar testing goal will make things much better. He also stressed the importance of metrics along with testing (really? From the metric-fu guy? Haha). If testing is old and broken, causing team backlash, low morale, and gridlock, it might be better to lessen the testing burden or throw away difficult to maintain tests. Getting rid of them and getting them out of the way, might be worth more than the value the tests were providing. In general he isn't big into view testing, he likes to avoid slow testing. He likes to have a small 'smoke screen' of integration tests, to help verify the system is all working together. In the end, what is the right level of testing for a project? The answer: what level of assurance does the given project really need? In a start-up you probably don't need a huge level of assurance, speed matters and market feedback matter more. If your building parts for a rocket or medical devices it is entirely different.

I enjoyed this talk quite a bit, and it inspired me to fix our broken metric_fu setup and start tracking on projects metrics again. Jake also wrote a good roundup of LSRC

Corey Donohoe @atmos, think simple
Corey gave interesting quick little thoughts and ideas about how to stay productive, happy, learn more, do more, fail less, and keep things simple and interesting... Honestly with something like 120+ slides, I can't even begin to summarize this talk. I checked around and couldn't find his slides online, but they couldn't really do the talk justice anyways. Keep your eyes peeled for the video as it was a fun talk, which I enjoyed. Until then here is a post he made about heading to LSRC.

Joseph Wilk, Outside-in development with Cucumber
Cucumber is something I keep hearing and reading about, but haven't really gotten a chance to play with it myself. Joseph's talk was a good split between a quick intro to Cucumber, and diving in deeper to actually show testing examples and how it worked. From the talk it sounded to me like Cucumber was mostly a DSL to talk between the customer and the developer/tester. I don't know if that is how others would describe it. I thought Cucumber was an odd mix of English and and Ruby, but it helps effectively tell a story. Since returning form LSRC, I have started working on my first Cucumber test.

Yehuda Katz, Bundler
This was just a lightening talk about Bundler, which I had read about briefly online. Seeing the work that was done for this blew me away. I can honestly say I hope this takes over the Ruby world. We have been dealing with so many problems related to gems at Devver, and if Bundler becomes a standard, it would make the Ruby community a better place. I am really excited about this so go check out the Bundler project now.

Rich Kilmer, Encoding Domains
The final keynote of the event was about encoding domains. I didn't really know what to expect going into this talk, but I was happily surprised. Rich talked about really encapsulating a domain in Ruby and then being able to make the entire programming logic much simpler. He gave compelling examples of working with knowledge workers in the field and just writing code with them to express their domain of knowledge in Ruby code. Live coding with the domain with experts he jokingly called "syntax driven development" - you write code with them until it doesn't raise syntax errors. Rich spoke energetically and keep a tiring audience paying attention to his stories about projects he has worked on through out the years. Just hearing about people who have created successful projects who have been working with Ruby in the industry this long is interesting. I thought it had great little pieces of knowledge that were shared during the talk, but again this was a talk where it was to hard to pull out tiny bits of information, so I recommend looking for the video when it is released.

Final Thoughts
LSRC was a good time besides hearing all the speakers. In fact like most conferences some of the best knowledge sharing happened during breaks, at dinner, and in the evenings. It also gave me a chance to get to know some of the community better than just faceless Twitter avatars. It was fun to talk with Ruby people about things that had nothing to do with Ruby. I also am interested in possibly living in Austin at some point in my life so it was great to check it out a bit. Friday night after the conference I went out with a large group of Rubyists to Ruby's BBQ, which was delicious. We ate outside with good food, good conversation, and live music playing next door. As we were leaving someone pointed out that the guitarist playing next door was Jimmy Vaughn, brother of the even more famous Stevie Ray Vaughan. We went over to listen to the show and have a beer, which quickly changed into political speeches and cheers. Suddenly I realized we were at a libertarian political rally. I never expected to end up at a Texan political rally with a bunch of Rubyists, but I had a good time.

Hopefully the next Ruby conference I attend with be as enjoyable as LSRC was, congrats to everyone who helped put the conference together and all those that attended the event and made it worth while.

/Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/em/connection.rb:302:in `start_tls': undefined method `set_tls_parms' for EventMachine:Module (NoMethodError)
from /Library/Ruby/Gems/1.8/gems/devver-2.4.1/lib/client/mod_client.rb:32:in `post_init'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/em/connection.rb:43:in `new'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/em/connection.rb:36:in `instance_eval'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/em/connection.rb:36:in `new'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:716:in `bind_connect'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:723:in `connect'
from /Library/Ruby/Gems/1.8/gems/devver-2.4.1/lib/client/client.rb:125:in `push_tests'
from /Library/Ruby/Gems/1.8/gems/devver-2.4.1/bin/devver:145
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:1503:in `call'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:1503:in `event_callback'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:342:in `run_timers'
from (eval):44:in `each'
from (eval):44:in `each'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:339:in `run_timers'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:322:in `run'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:318:in `loop'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:318:in `run'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/pr_eventmachine.rb:64:in `run_machine'
from /Library/Ruby/Gems/1.8/gems/eventmachine-0.12.8/lib/eventmachine.rb:242:in `run'
from /Library/Ruby/Gems/1.8/gems/devver-2.4.1/bin/devver:144
from /usr/bin/devver:19:in `load'
from /usr/bin/devver:19

... it's because EventMachine (and other compiled gems) may act a little funky after installing Apple's latest OS.

In this post, I'll go through some steps that should help you resolve this error (... assuming you use MacPorts. If you don't, please tailor these directions accordingly). But first:

• Warning: The directions below may leave your system in an unknown state. Please, please, please backup everything before going further. Time Machine is your friend!
• Disclaimer: These steps worked for me, but I can't guarantee they'll work for you. If you have any problems, please let me know in the comments or email support@devver.net.
• Note: If you already installed Devver before upgrading to Snow Leopard, things will likely continue to work fine for you for now - but these directions may be helpful when you upgrade Devver clients.

OK, now that we've got that out of the way ...

1. Upgrade MacPorts

Try running something like port list installed. If you get an error like this:

dlopen(/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib, 10): no suitable image found.  Did find:
/opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib: no matching architecture in universal wrapper
while executing
"load /opt/local/share/macports/Tcl/pextlib1.0/Pextlib.dylib"
("package ifneeded Pextlib 1.0" script)
invoked from within
"package require Pextlib 1.0"
(file "/opt/local/bin/port" line 40)

You need to install the latest version of MacPorts. Go here and download the 'Snow Leopard' dmg and install.

2. Reinstall openssl and ruby

Now that MacPorts is installed, do this:

sudo port sync
sudo port upgrade openssl
sudo port upgrade ruby

You could also try sudo port upgrade ruby186 if you want Ruby 1.8.6, but I haven't tried this myself.

3. Reinstall EventMachine

sudo gem uninstall eventmachine
sudo gem install eventmachine

And you're done!

Now you should be able to run Devver normally. If that didn't work, please let me know!

Bonus: Reinstall other gems

Matt Aimonetti has written a handy script (Update: here is my fork, with a new gem path an an important bug fix on line eight) that will find the gems you should reinstall once you've done the above steps. The only change you'll need to make is to replace the gem path:

#Dir.glob('/Library/Ruby/Gems/**/*.bundle').map do |path|
Dir.glob('/opt/local/lib/ruby/gems/**/*.bundle').map do |path|

For more info, you can find a great guide on resolving Ruby-related problems when upgrading to Snow Leopard at rubyonrails.org

For the most part, this works fine. But I've always struggled to find a good way to test interaction with the filesystem (which can often be non-trivial and therefore should be tested). On the one hand, the filesystem could be considered "external" and mocked out. But on the other hand, the filesystem is accessible when the tests run. In this way, the filesystem is sort of like a local database - it could be mocked out, but it doesn't have to be, and there are tradeoffs to both approaches.

Over the past year or so, I've tried out a few approaches for testing interactions with the filesystem, each of which I'll explain below. Since none of the approaches met my needs, Avdi and I built a new testing library, which I'll introduce below.

Mocking the file system.

Sometimes, it is simplest to just mock the interaction with the filesystem. This works well for single calls to methods like File#read or File#exist? (these examples use Mocha):

File.stubs(:read).returns("file contents")
File.stubs(:exist?).returns(true)

However, this approach breaks down when you want to test more complex code, which, of course, is the code you're more likely to want to test thoroughly. For instance, imagine trying to set up mocks/stubs for the following method (which atomically rewrites the contents of a file):

require 'tempfile'

class Rewriter

  def rewrite_file!(target_path)
    backup_path = target_path + '.bak'
    FileUtils.mv(target_path, backup_path)
    Tempfile.open(File.basename(target_path)) do |outfile|
      File.open(backup_path) do |infile|
        infile.each_line do |line|
          outfile.write(yield(line))
        end
      end
      outfile.close
      FileUtils.cp(outfile.path, target_path)
    end
  rescue Exception
    if File.exist?(backup_path)
      FileUtils.mv(backup_path, target_path)
    end
    raise
  end

end

Now imagine setting up those same mocks/stubs for each of the five or so tests you'd want to test that method. It gets messy.

Even more importantly, mocking/stubbing out methods ties your tests to a specific implementation. For instance, if you use the above stub (File.stubs(:read).returns("file contents")) in your test and then refactor your implementation to use, say, File.readlines, you'll have to update your tests. No good.

MockFS

MockFS is a library that mocks out the entire filesystem. It allows you write test code like this:

require 'test/unit'
require 'mockfs'

class TestMoveLog < Test::Unit::TestCase

  def test_move_log
    # Set MockFS to use the mock file system
    MockFS.mock = true

    # Fill certain directories
    MockFS.fill_path '/var/log/httpd/'
    MockFS.fill_path '/home/francis/logs/'

    # Create the access log
    MockFS.file.open( '/var/log/httpd/access_log', File::CREAT ) do |f|
      f.puts "line 1 of the access log"
    end

    # Run the method under test
    move_log

    # Test that it was moved, along with its contents
    assert( MockFS.file.exist?( '/home/francis/logs/access_log' ) )
    assert( !MockFS.file.exist?( '/var/log/httpd/access_log' ) )
    contents = MockFS.file.open( '/home/francis/logs/access_log' ) do |f|
      f.gets( nil )
    end
    assert_equal( "line 1 of the access log\n", contents )
  end
end

Although I suspect MockFS would be a great fit for some projects, I ended up running into issues.

First of all, it depends on a library (extensions) that can have strange monkey-patching conflicts with other libraries. For example, compare this:

require 'faker'
puts [].respond_to?(:shuffle) # true

to this:

require 'extensions/all'
require 'faker'
puts [].respond_to?(:shuffle) # false

Secondly, as you'll notice in the above example, using MockFS requires you to use methods like MockFS.file.exist? instead of just File.exist?. This works fine if you're only testing your own code. However, if your code calls any libraries that use filesystem methods, MockFS won't work.

(Note: There is a way to mock out the default filesystem methods, but it's experimental. From the MockFS documentation:

"Reading the testing example above, you may be struck by one thing: Using MockFS requires you to remember to reference it everywhere, making calls such as MockFS.file_utils.mv instead of just FileUtils.mv. As another option, you can use File, FileUtils, and Dir directly, and then in your tests, substitute them by including mockfs/override.rb. I'd recommend using these with caution; substituting these low-level classes can have unpredictable results. ")

All that said, MockFS is probably your best option if you're only testing your code and you want to mock out files that you can't actually interact with - for instance, if you need to test that a method reads/writes a file in /etc (although for the sake of testability, it's generally good to avoid hardcoding fully-qualified paths in your code).

FakeFS is another library that uses this approach. I haven't used it personally, but it looks quite nice.

Creating temp files and directories (with Construct)

Besides mocking the filesystem, another option is to have tests interact with actual files and directories on disk. The advantages are that the test code can be simpler to write and you don't have to use any special filesystem methods.

Of course, as always, you want the test itself to contain all the relevant setup and teardown - you don't want your tests to depend upon some set of files that have no explicit connection to the test itself (or create files that aren't cleaned up).

To make this easy, we created a new library called Construct. Construct makes test setup simple by providing helpers to create temporary files and directories. It takes care of the cleanup by automatically deleting the directories and files that are created within the test. And because it creates regular files and directories, you can use plain old Ruby filesystem methods in your code and tests.

To install Construct, simply run:

# gem install devver-construct --source http://gems.github.com

Using Construct, you can write code like this:

require 'construct'

class ExampleTest < Test::Unit::TestCase
  include Construct::Helpers

  def test_example
    within_construct do |construct|
      construct.directory 'alice/rabbithole' do |dir|
        dir.file 'white_rabbit.txt', "I'm late!"
        assert_equal "I'm late!", File.read('white_rabbit.txt')
      end
    end
  end

end

Let's look at each line in more detail.

    within_construct do |construct|

When you call within_construct, a temporary directory is created. All files and directories are, by default, created within that temporary directory and the temporary directory is always deleted before within_construct completes.

The block argument (construct) is a Pathname object with some additional methods (#directory and #file, which I'll explain below). You can use this object to get the path to the temporary directory created by Construct and easily create files and directories.

Note that, by default, the working directory is changed to the temp dir within the block provided to within_construct.

      construct.directory 'alice/rabbithole' do |dir|

Here we are using the construct object to create a new directory within the temp directory. As you can see, you can create nested directories like alice/rabbithole in one step. The block argument (dir) is again a Pathname object with the same added functionality noted above.

Just like before, the working directory is changed to the newly created directory (in this case, alice/rabbithole) within the block.

        dir.file 'white_rabbit.txt', "I'm late!"

Here we use the dir object to create a file. In this case, the file will be empty. However, it's easy to provide file contents using either an optional parameter or the return value of the supplied block:

within_construct do |construct|
  construct.file('foo.txt','Here is some content')
  construct.file('bar.txt') do
  <<-EOS
  The block will return this string, which will be used as the content.
  EOS
  end
end

As a more real-world example, here's how you could use Construct to start testing the #rewrite_file! method we looked at before:

require 'test/unit'
require 'construct'
require 'shoulda'

class RewriterTest < Test::Unit::TestCase
  include Construct::Helpers

  context "#rewrite_file!" do

    should "alter each line in file" do
      within_construct do |c|
        c.file('bar/foo.txt',"a\nb\nc\n")
        Rewriter.new.rewrite_file!('bar/foo.txt') do |line|
          line.upcase
        end
        assert_equal "A\nB\nC\n", File.read('bar/foo.txt')
      end
    end

    should "not alter file if exception is raised" do
      within_construct do |c|
        c.file('foo.txt', "1\n2\nX\n")
        assert_raises ArgumentError do
          Rewriter.new.rewrite_file!('foo.txt') do |line|
            Integer(line)*2
          end
        end
        assert_equal "1\n2\nX\n", File.read('foo.txt')
      end
    end

  end

end

You can learn more at the project page (both the README and the tests have more examples).

(As an aside, since Construct changes the working directory, it doesn't play nicely with ruby-debug. Specifically, if you place a breakpoint within a block, you'll see the message "No sourcefile available for test/unit/foo_test.rb" and you won't be able to view the source. If anyone knows an easy way to make Dir.chdir work with ruby-debug, I'd very much appreciate some help!)

Conclusion

We've been moving our filesystem tests over to using Construct and so far have found it to be very useful. How do you test interactions with the filesystem? Do you use one of the above approaches, or something else? Or do you skip testing the filesystem altogether?