Relevance - Relevance Weblog

Relevance Agile Bibliography

2009-07-07T14:41:00Z

You can’t learn agile development by reading a book. But reading a few books, in parallel with training and trying out the techniques, helps a lot. These references constitute Relevance’s recommended reading list for doing Agile development.

Our customers and students frequently ask us how they can learn more about how we work. We believe that Agile is not a process or method, but rather an attitude of constant reflection and improvement. Our favorite books and papers are about how to see, understand, and react to what’s happening on our projects.

Three Essentials

Almost all technical books are about particulars: particular processes, platforms, techniques, or tools. These three books, though, are about the essence: the essence of programming, projects, and people. Other books may offer more immediate, concrete recommendations for your current situation. However, these books are the foundation. If you understand them, everything else will make more sense, and you will be able to make more effective use of more specific recommendations. We don’t see these three books becoming dated anytime soon.

The Pragmatic Programmer: From Journeyman to Master, by Andrew Hunt and David Thomas. If there’s only one book that every working programmer should read, this is it. “PragProg,” as it’s known, is about the essence of programming: what programmers really do, and how to get better at it.

Agile Software Development: The Cooperative Game (Second Edition) by Alistair Cockburn. This book is an awesome exploration of how to think about process and software, and how to adapt your process to the needs of your organization and project. This book attempts to get to the bottom of how software projects work and the factors that make them succeed or fail.

Peopleware: Productive Projects and Teams (Second Edition) by Tom DeMarco and Timothy Lister. This book is aimed at software projects, but the focus is the human element: people and teams. Just as good and relevant today as when it was first published in 1987.

Software Development

The Mythical Man-Month: Essays on Software Engineering (Anniversary Edition), by Fred Brooks. This is the all-time classic about software project management. It predates Agile methods by a long time, but it’s still packed with wisdom and insight, including compelling analyses of some of the problems with traditional development processes.

Patterns of Enterprise Application Architecture by Martin Fowler. Yes, it’s a patterns book, but don’t let that scare you away. It discusses the patterns in use by Rails, Django, JEE, and the tradeoffs and reasoning behind them. Whatever architectural style you usually prefer, this book provides a valuable dose of context by explaining the situations where that style may not be appropriate, and showing you a number of useful alternatives.

Waltzing with Bears: Managing Risk on Software Projects by Tom DeMarco and Timothy Lister. Methodology failure is just one of many kinds of risk your projects face. Unfortunately, most Agile books don’t give risk management the attention it deserves. This book explains how and why to take risk seriously and manage it carefully. It’s an excellent primer on risk analysis and how to plan for failure. (It also includes one of our favorite quotes: “Risk management is project management for grownups.”)

Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans. We’ve known for years that a clean, expressive domain model that reflects the terminology of the problem domain makes every part of the development process easier. Eric Evans’ book does a great job of explaining why that’s true, and how to achieve a good domain model.

The Deadline: A Novel About Project Management by Tom DeMarco. Politics, intrigue, risk management, romance, kidnapping, schedule chicken, and spies. What more do you need to know?

I.M. Wright’s Hard Code by Eric Brechner. This book deals specifically with the agile project manager role. Most importantly, it describes how to be a better manager in the “servant/master” model; rather than dictating to the team what to do, the manager sets direction and removes obstacles. That style works well in general, and especially well on an agile project. (Also, it has valuable tips on how to interact with other teams that are working on your project.)

Agile Methods

Extreme Programming Explained: Embrace Change (Second Edition) by Kent Beck. A concise and clear explanation of traditional software methodologies and why they are broken, and why XP works and is better. The first edition is rather detailed and prescriptive, focused on developers. The second edition has a higher-level viewpoint and is a bit more relaxed.

The Buzz About Bees: Biology of a Superorganism by Jürgen Tautz. Some people are unsure about agile methods because they don’t feel comfortable with the idea of emergent behavior (i.e., simple, localized rules that result in complex, apparently organized large-scale effects). This book is an excellent explanation of that idea, with many insights that apply to human organizations.

A Simple Model of Agile Software Processes –or– Extreme Programming Annealed by Glenn Vanderburg. This paper explains how a collection of focused practices, with very little overall coordination or up-front planning, can reliably lead to good results. Although the paper deals explicitly with Extreme Programming, the core idea—that agile processes consist of nested, optimized feedback loops at various scales—applies to other agile processes as well.

Agile Estimating and Planning by Mike Cohn. Agile teams still estimate costs, make plans, and set schedules. But we do it a bit differently. This book by Mike Cohn explains how an agile team sets plans, and how to change your thinking about plans and the ways that they help you.

Reflection and Responsiveness

Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen. Kevlin Henney likes to say that some programmers with ten years of experience seem to have had the same year ten times. Some teams are like that, too. Experience isn’t very valuable if you don’t learn from it, and we’ve found that you can’t just take that for granted. Retrospectives are wonderful for making sure you learn the right lessons from both your mistakes and your successes. This book can show you how.

Secrets of Consulting: A Guide to Giving and Getting Advice Successfully by Gerald Weinberg. Even if you’re not a consultant, this book is well worth reading. It’s all about how to see problems, identify solutions, and communicate successfully within an organization. It’s about effecting change from a position of very little power. Who hasn’t needed to do that?

The Back of the Napkin: Solving Problems and Selling Ideas with Pictures by Dan Roam. This book is about thinking visually about processes and problems. It’s also a great way to learn about low-cost, high impact techniques like paper-prototyping and ad hoc data visualization.

Personal Development

The Passionate Programmer: Creating a Remarkable Career in Software Development by Chad Fowler. There’s debate about whether agile development requires exceptional developers. But it’s absolutely clear that having good developers helps a lot. Chad has written a wonderful book that can inspire you and others on your team to improve your craft and polish your skills. The entire team will benefit.

What Did You Say?: The Art of Giving and Receiving Feedback by Charles N. Seashore, Edith Whitfield Seashore, and Gerald M. Weinberg. Agile processes are all about efficient feedback, and feedback between and about people is an important part of that. But giving personal feedback is hard, and receiving it is even harder. This book can help you get better at giving feedback constructively and reacting to it positively. Unfortunately it’s out of print, but of course used copies are available on Amazon or Alibris.

Beyond This List

The resources listed here are great places to start. We also offer training on agile practices. But there’s a lot of other relevant literature. In fact, as a couple of the examples here demonstrate, there’s a lot we can learn about software development by reading about other fields, and even about other species. (Our friends the Pragmatic Programmers once gained valuable insights about software development by reading about training in the nursing profession and U.S. Marine Corps warfare doctrine. And we’ve been learning a lot lately by reading about music theory, cognition, the practice of medicine, and the history of bridge engineering.)

Furthermore, things are still changing. This bibliography will probably look very different in five years or so. (In fact, you can probably think of good resources we’ve missed. Tell us in the comments!) We learn new things about programming, and about software teams, every year. Sometimes it may seem like we have it all figured out, but more often we suspect that our whole industry is still fumbling in the dark, only gradually coming to understand the fundamentals of software development.

We intend to keep reading and discovering, and you should too. It takes a long time to get really good at software development, and there’s always room to improve. Read at least a few books on software development each year. Also, seek out some good blogs or online magazines that discuss these topics. There is a rich and fascinating dialogue happening all the time in the community of software developers. You owe it to yourself—and your team, and your employer—to be a part of it!

Focus, focus, focus

2009-06-17T17:37:00Z

We’ve been doing technical training a long time, and we’ve discovered a general rule of thumb: the fifth day of a five-day class is generally worthless. Students are burned out, concentration has waned, and that is coupled with the fact that the last day is usually the most complex topics. All of which, taken together, means that information is just not usefully transmitted on the final day.

The other major issue is the length-of-course to primariness-of-focus ratio. Some technologies are complex enough to warrant long courses on the merits of the technology alone, but are not the main job of anybody on the team. Neal Ford once described these as “condiment technologies”. Maybe Ruby, or Java, or C#, or Clojure is your hotdog; these technologies are the mustard. Even if the mustard is really really really good, nobody wants to spend a week eating just that.

We’re trying to address this through the introduction of our new one-day training classes. The idea is to harness the power of programmers to focus insanely well for a short period of time. Instead of fighting against ADD all week, let’s harness the obsessive power of the programmer’s mind and hit one topic, really hard, for one day. We think that the energy expanded will have a higher return for the students, and the information will be retained and used.

Our first offering in this space is our jQuery course. jQuery is a perfect candidate due to its nature:

it’s a library, not a language. While it has general purpose features, the programmer will be presumed to understand them before attending the class.
it has a limited application scope. It is used to build interactive web pages (DOM interaction, Ajax, general JavaScript extenstions)
it has some deeply interesting technical aspects. Is it a monad? How does it embody the ideas of functional programming? Is it different from raw JavaScript?

We think training courses like this are going to be more effective for a variety of technologies that programmers are being exposed to these days. Instead of making one massive course that programmers will have a hard time sitting through and concentrating during, pick one interesting topic and hammer it mercilessly for 8 hours. Sound familiar? That’s pretty much how we work around here. And it plays into our other belief about technology training: it should be about inspiration and getting launched, not about exhaustive minutia. One day on jQuery is plenty to get productive and be inspired.

Refactoring JavaScript, 2009 Edition

2009-06-05T11:24:00Z

I am off to Dallas this morning to give an all-new version of the Refactoring JavaScript talk. This year, we will be looking at testing and refactoring jQuery plugins, using Screw.Unit, Smoke, and blue-ridge.

You can grab the slides here, but as usual the slides tell only a little of the story. Instead, grab the project itself, and use git’s local history to start at the beginning and watch the refactorings step-by-step.

Rifle-Oriented Programming with Clojure

2009-05-27T13:53:00Z

If you come to Clojure from an object-oriented background, you may not know where to start. It is sort of like looking at a rifle for the first time and asking "But where do I put the arrows?"

Clojure solves the traditional problems of OO (and then some!) but it does it in different ways. To learn how to translate your arrows (encapsulation, polymorphism, and inheritance) into bullets, check out my new article in the May issue of NFJS, the Magazine.

Also: I'll be spending the summer on the NFJS circuit, talking about Clojure, Git, and other good things. Come see us.

Blue Ridge 1.0: JavaScript Unit Testing for Rails. Scandalous!

2009-05-12T10:30:00Z

You wouldn’t consider developing a Rails application without having a solid test suite for your Ruby code, but you’ve somehow convinced yourself to cross your fingers and look the other way when it comes to JavaScript. It doesn’t have to be that way.

Meet Blue Ridge, a Rails plugin that brings the goodness of test-driven and behavior-driven development to your unobtrusive JavaScript code in a Rails-friendly manner.

Historically, when selecting a JavaScript testing solution, you were forced to choose whether you wanted a framework that could run your tests in the browser or one that could only run your tests in a headless fashion. By providing a friendly convention-over-configuration wrapper around a collection of open source tools, Blue Ridge gives us the best of both worlds: fast, automation-friendly, and headless testing plus the ability to run your tests in whichever browser is acting up on any given day.

Last summer, Blue Ridge was just a twinkle in our eye. Last week, Blue Ridge made its public debut at RailsConf. Today, we’re pleased to announce version 1.0.

Getting Started

First, install Blue Ridge into your Rails app:

./script/plugin install git://github.com/relevance/blue-ridge.git
./script/generate blue_ridge

Blue Ridge creates a small example spec to get you started. Run your JavaScript specs to make sure that all is well so far:

rake test:javascripts

(Hint: You can also use the spec:javascripts or examples:javascripts aliases. Blue Ridge is compatible with your testing framework of choice, be it test/unit, RSpec, Micronaut, Shoulda, test-spec, etc.).

Next, try running an individual spec. When you installed Blue Ridge, it created a spec file named “application_spec.js”. You can run it like so:

rake test:javascripts TEST=application

Of course, you really want to generate a spec that’s specific to your app. Let’s say you want to write some tests for a JavaScript file called “public/javascripts/graphics.js”. Run:

./script/generate javascript_spec graphics
rake test:javascripts TEST=graphics

Cool. We can run our JavaScript specs from the command line, and they’re fast! But you just got a bug report regarding JavaScript errors in IE6 (die already!), and you want to write some tests to prove (and then resolve) the bug. To run your spec inside a web browser, simply load the HTML fixture associated with the spec (e.g., “test/javascripts/fixtures/graphics.html” for the tests in “graphics_spec.js”) and see the results of running the tests in that specific browser.

Check out the README for more information on the directory layout and a detailed description of the various Blue Ridge components.

jQuery-Opinionated

Blue Ridge wouldn’t quite fit into the Rails ecosystem if it didn’t come equipped with a few opinions. So by default, it assumes you’re using jQuery. “application_spec.js”, which was generated when you installed the plugin, includes an example of calling jQuery functions inside a test.

require("spec_helper.js");
require("../../public/javascripts/application.js");

Screw.Unit(function() {
  describe("Your application javascript", function() {
    it("provides cliché example", function() {
      expect("hello").to(equal, "hello");
    });

    it("accesses the DOM from fixtures/application.html", function() {
      expect($('.select_me').length).to(equal, 2);
    });
  });
});

And, no, we don’t actually encourage you to write tests for standard libraries like JQuery and Prototype; it just makes for an easy demo.

Prototype-Friendly

If you prefer Prototype, no problem: “Have it your way.”

jQuery.noConflict();

require("spec_helper.js");
require("../../public/javascripts/prototype.js", {onload: function() {
    require("../../public/javascripts/application.js");
}});

Screw.Unit(function() {
    describe("Your application javascript", function() {
        it("accesses the DOM from fixtures/application.html", function() {
            expect($$('.select_me').length).to(equal, 2);
        });
    });
});

Put jQuery into “no conflict” mode to give the $ function back to Prototype, require prototype.js, and chain any files that are dependent on prototype.js in the onload callback. Done. Your Prototype-based tests await you.

Act Now, and We’ll Throw in Mocking Support

Blue Ridge includes Smoke, a JavaScript mocking and stubbing toolkit somewhat similar to Mocha. Assume we’re testing a function called calculateTotalCost, and you want to ensure that it calls calculateComponentPrice for each given component. That test might look something like so:

it("calculates the total cost of a contract by adding the prices of each component", function() {
  var componentX = {}, componentY = {};
  mock(SalesContract).should_receive("calculateComponentPrice")
    .with_arguments(componentX).exactly(1, "time").and_return(42);
  mock(SalesContract).should_receive("calculateComponentPrice")
    .with_arguments(componentY).exactly(1, "time").and_return(24);
  expect(SalesContract.calculateTotalCost([componentX, componentY])).to(equal, 66);
});

Fun with TextMate

Running individual specs from the command line is an essential feature, but if you use TextMate like we do, you’d rather not have to leave your editor in order to run a spec. If this describes your development style, take the Blue Ridge TextMate Bundle for a spin.

cd ~/Library/Application Support/TextMate/Bundles/
git clone git://github.com/karnowski/blue-ridge-tmbundle.git Blue\ Ridge.tmbundle

Once you reload your bundles (or restart TextMate), just hit Command-R to run the currently-open spec directly from TextMate.

And be sure to check out the snippets as well. Type it, des, bef, or aft, and then press the tab key to expand into full it blocks, describe blocks, etc.

But Wait, There’s More

Want more examples? To see Blue Ridge in action inside a working Rails app, check out the Blue Ridge sample application. Among other things, the sample app includes examples of:

using nested describe functions
setting up per-spec HTML “fixtures”
stubbing and mocking functions
running the Blue Ridge specs as part of your default Rake task

Money Back Guarantee

Blue Ridge is guaranteed to be bug free and fulfill your every need, or your money back! Of course, in lieu of a full refund, you’re welcome to hop over to the project’s GitHub issue tracker to let us know about any issues or ideas for possible improvements. Even better, fork the repo and start hacking! If you have patches, send us pull requests.

Now, go forth and give your JavaScript code the testing love it deserves!

Some More Details on the New Space

2009-05-07T13:26:00Z

We decided to move when we realized that our dev room was too cramped, and people were being forced to work in conference rooms, the hallway or out of the office. We colocate our team precisely to avoid situations where people aren’t in the same room, and our old space was actively hindering that. So, we decided to seek out something a bit more spacious. Here is our new dev room….

Everybody loves the new space, precisely because we have enough room to spread out but stay together. And, you can’t see it very well, but if you look over Rob’s shoulder in this next picture, you can see the kitchen behind the dev room, with the conference room to the right. The conference room is big, located at the far end of the space from the dev room, and easily closed off, which means we can have meetings and still have all the chaos of our dev room going at the same time.

The other major benefit of the new space is light; sweet, sweet sunlight. Every window in the space is enormous, and they all open. To the outside. In this picture, you can also see two doors on the right, leading to the library and the phone room. The new space is a great mix of public and private space that suits us down to the bone.

On top of all this, the space is in the heart of downtown Durham, meaning we have tons of restaurants and shops in walking distance. I just found out yesterday that there’s a skateboard shop across the street. We’ve been walking to lunch every day, and all around enjoying being out of the suburbs.

Special thanks to Jess Martin for taking these photos.

Relevance Has Moved

2009-05-05T18:14:00Z

We are happy to announce that Relevance has completed its long-awaited move to downtown Durham, NC. A year after signing the lease and riding herd over the buildout, we moved in over the weekend. We are all thrilled to be in the new space: more room, more light, more fun things to do within walking distance. When the apartments are finished, we’ll even have team member sitting atop our office like hens to eggs.

While I’m writing this, we’re sitting here on a rainy spring afternoon listening to jazz quietly playing in the new dev room, windows open to the rain and sound of cars and foot traffic outside. If it wasn’t for the damn crosswalk alert about three feet from the window, it would be perfect.

Hope to see some of you here soon. The new address is:

200 North Mangum Street
Suite 204
Durham, NC 27701

JavaScript Testing at RailsConf

2009-04-30T13:23:00Z

Jason and I will be at RailsConf next week speaking about Blue-Ridge, our Rails plugin that makes test-driven JavaScript development easy and natural!

Here's our abstract:

Learn how to enjoy the benefits of test-driven development beyond just your Ruby on Rails code. JavaScript is code too, and it deserves tests! With the help of some handy plugins, Rails lets you test your unobtrusive JavaScript using tools such as Screw.Unit and Smoke. The tools and approach are library-agnostic; they work well with jQuery, Prototype, and others.

The talk is at 1:50 PM on Tuesday, May 5. Come check it out, and give your JavaScript code the testing love it deserves.

Making a Lasting Impact

2009-04-09T13:55:00Z

At Relevance, we pride ourselves on having happy customers. We are constantly striving to ensure that, from the very start of a project, we will exceed our customers' expectations. Recently one of our customers made our day by coming to us with concrete evidence of the positive impact we'd made---in the form of a video!

The project was one where the system already existed, and the customer had a small team of two developers at work on it. Three of us from Relevance joined the project part-time for about three months, helping them complete a new release.

When the project manager learned about the Codeswarm tool, he thought it would be interesting to make a Codeswarm video of that period of time. He sent a copy of the video to us as a way of thanking us for our contribution. Here's an anonymized version of the video:

The most interesting thing about that video is not the most obvious thing. Yes, there's a big, clear flurry of activity when Relevance joined the team. But some of that is artificial: right at the start we vendored Rails into the repository and added a bunch of useful plugins. Some of the big burst of activity is related to that (although less than you might think).

No, the thing that most pleased our customer (and us!) is what happened after that. Watch just the first ten seconds of the video again: just the two client developers, before we joined the team. Then start watching at about the 15-second mark, after we'd been on the project for a little over a week. Don't watch the three Relevance developers; watch the two client developers. Do you see how their pattern of activity has changed? They are each making more frequent commits, and their commits involve more files.

That's the impact that our customer wanted us to see: that our involvement made his own developers more active, confident, and productive. That effect held on as our involvement in the project diminished, and even after we were gone. In his words, "You can definitely tell the impact you had on our project!"

Needless to say, we're thrilled. That's the kind of impact we want to have: knock it out of the park ourselves, and help the customer's team grow in the process. And to have the customer be so impressed that they take this much trouble to show us what they've seen---that's just icing on the cake.

Relevance Becomes a B Corporation

2009-04-06T13:31:00Z

As of March, 2009, Relevance is officially a B Corporation. Most people reading this are probably asking something like "What, you couldn't get an A?" Or, "weren't S and C good enough?" The answers are: in this case, a B is better than an A, and no, they weren't. Read on to find out what this all means, and what we're up to.

A Bit of History

Relevance was founded in 2003 by a couple of guys. We've been steadily building the company around some core principles, some of which are obvious to anybody who knows us or surfs our site: agile is better than not-agile, sharp tools are better than dull ones, communication is better than documentation and/or confrontation, and that a small team of dedicated people with passion about their work can create more value faster than 20 bored ones.

There are other principles that aren't so obvious, though. For example, Relevance only succeeds if we challenge our team, and we can only challenge them if we support them through the challenges. So Relevance believes strongly in shared ownership and a good work/life balance. We believe that open source is a vital cog in the machinery of the modern economy; cooperation around core ideas can lead to more innovation and better competition. We donate 20% of our time to open source or pro bono work, both to benefit our team and our various communities.

Delve deeper, and there are even less obvious principles: businesses should be about the task of generating wealth, sustainably. That means that the business shouldn't exhaust its resources -- long-term success depends on cultivating them instead. Further, it shouldn't externalize its costs, but account for them. We want to be intimitely tied in to our local community, and we want to be responsible citizens.

Acceleration

Over the course of 2008 and early 2009, we began seeking out customers and projects that would allow us to express our values more directly. We built RunCodeRun not only because we love testing so much, but because we love open source, too. RunCodeRun provides free continuous integration to open source Ruby projects. That's because we think they deserve this kind of support, too, and we know they can't pay for it. We sought out customers who share our values and who like open, transparent communication as much as we do. We built a team that cares deeply about these things.

In early 2009, though, things really began to pick up. We added a focus on sustainability and, for lack of a better word, "green" technologies. As an Environmental Science and Policy major at Duke, I always thought I'd end up working at the EPA. I ended up in the private sector, though, and I've come to realize that you can achieve sustainable value working in a for-profit organization. Profit is the fuel that keeps the engine revving. I'd love to find a way to bring the way we work back to organizations like the EPA, though (anybody at the EPA listening?).

After deciding internally on that direction, stuff started happening. We met Danvers Fleury of Converdant, a consultancy helping companies achieve sustainability and community involvement goals while ensuring sound business practices. We also began to work with a customer (who shall remain anonymous for now) whose entire business is built on these principles. Both of them, simultaneously, recommended we check out B Lab and the B Corporation site. A few short weeks later, we were certified.

B Lab and Our Customers

The "B" in B Corporation stands for "beneficial" (or "benefit", depending on context). Certified B Corporations:

meet transparent and comprehensive standards of social and environmental performance;

legally expand their corporate responsibilities to include consideration of stakeholder interests; and

amplify the voice of sustainable business and for-profit social enterprise through the power of the unifying B Corporation brand.

The goal of B Lab (the non-profit behind the certification) is to one day create a nationwide legal designation to go along with the existing S and C Corporations. In the meantime, certified B Corps have to amend their articles of incorporation to "redefine the best interests of the corporation to include the consideration of employees, consumers, the community and the environment."

When I first started to look into B, I didn't think we had what it took to get certified. The survey is very broad, and the scoring is unforgiving. Without a processing or manufacturing arm, would we have enough to score well on the environmenal and sustainability scale? Since we don't have a supply chain, could we do well enough on local sourcing? How would our software company compare to a community organizer, or financial institution, or organic soap manufacturer?

As it turns out, pretty well. Our focus on the well-being of our team, coupled with initiatives and partnerships we've entered into locally, combined to get us over the survey hurdle. Then, in conversation with our customers and B Lab, it turns out that we are not only a match for the community based on our survey score, but that we also help fill an underserved niche in the B Community: software services.

We believe strongly that software professionals have the most leverage to effect change in today's economy. What industry do you know that doesn't use software? What other services organizations can touch as many varied business processes and practices as software? Knowing that technologists have such wide reach, it felt really good to see how we might begin to use that reach in conjunction with a community of like-minded organizations.

What Next?

So what does it mean going forward?

We are the same company we have always been. We chose to become a B Corporation because it represents the values we already hold. For those of you with whom we have worked in the past, nothing has changed. We build and audit applications for people and train developers; that's what we do, and if you need a kick-ass team who delivers value and believes in transparent business relationships, give us a call.

We are going to continue to make Relevance the best company we can make it. That means acting on our values and using our success to provide value back to the community. It means working with our customers to find hidden value in their data, or their processes, that might be being overlooked. It means challenging ourselves to uphold our core principles, and seeking to apply them in each of our partnerships.

And it means challenging our friends and colleagues to think about whether they should be exploring what it means to be B.

Programming Clojure Beta 9 Is Out

2009-04-04T15:05:00Z

Programming Clojure Beta 9 is now available. We are almost done, and most of the changes in this Beta are small.

What's new:

the notation conventions have changed to make console output and REPL results more distinct
a new subsection covering functions on vectors
a new subsection on sort functions
an example using map with more than one collection
an example using the :while option to a sequence comprehension
examples now use the new letfn form where appropriate
replicate is gone -- use repeat instead
an index!

I have also made the book more strict in discussions of laziness. Clojure sequences are lazy, but function evaluation is eager. In the Clojure community we often ignore this distinction, saying things like "iterate is lazy" when we really should say "iterate returns a lazy sequence." The book now uses the latter formulation.

To make sure you have the latest, greatest version of the sample code from the book, go and grab the github repo.

Thanks again to everyone who has been offering feedback. Keep the feedback coming!

Seeking Two Ruby on Rails Experts for a Client

2009-04-02T15:44:00Z

CollectiveX is Hiring, and we are Helping

One of our first Ruby on Rails clients, CollectiveX, is looking to hire two Ruby/Rails experts in the Washington DC area. CollectiveX is an established, funded startup providing a collaboration platform for companies and groups. You can read more about their platform and success at VentureBeat and TechCrunch.

We are handling the screening and interview process for these candidates. Both positions require developers who have built big apps in Rails before with established, active user communities. The full job descriptions are below. If you think you would be a good fit for either position, please email us a resume at collectivex-jobs@thinkrelevance.com. Tell us why you would be a good fit, what big apps you've worked on, and what open source you have contributed to. Expect to be interviewed as if you were you interviewing for a full time Relevance position, which includes a full day of pairing at our North Carolina office if you pass the initial screenings.

Two Rubyists Needed: Director of Software Engineering and Senior Rails Developer

Our team wants you if you are a passionate, motivated technical leader and Ruby developer ready to take things to the next level. You should have exceptional communications skills including experience working with executives, engineers and users. You must also know consumer web products and have a knack for improving existing web products.

In the Director role, we will look to you to prioritize and lead software development based on business objectives and evolving user needs. You must be an experienced lead developer or development manager with experience working in a fast paced, rapidly changing environment that demands focus and grace under pressure.

In the Developer role, you will collaborate with the technical team to develop software and applications to support the CollectiveX back-end technology.

These are hands-on roles for someone who wants a leadership role while remaining deeply technical day in and day out. The existing application is a mature Rails code with a large active user base. It needs technical leaders who can act decisively and quickly to learn the current code base, handle any critical business features or defects, all while planning and implementing future releases.

CollectiveX is a profitable startup building a platform that sits at the intersection of online groups and listservs (Yahoo! Groups), collaboration software (MS Sharepoint) and social networks (LinkedIn, Facebook). Our core application has been live and growing in users since 2006. We pay great salaries (plus stock options), are led by a CEO who has founded previous successful startups, and have an established team and proven business model.

Key Responsibilities

Maintain and monitor the production environment; troubleshoot and resolve issues
Leading software teams for key projects, product development, testing and implementation
Work with executive, marketing, design and engineering team members to define and prioritize new products and features and then lead the way in feature development
Ensure the timely launch of features
Coordinate code reviews with our development partner and team members
Develop feature enhancements internally, including analysis, design, programming, QA, deployment, and support, in an iterative, pragmatic manner
Translate product vision into technical reality

Requirements

At least 8 years progressive development experience. 4 most recent in some sort of lead developer role (i.e. tech lead, chief architect, senior developer)
A proven track record of implementing high-performance, high-traffic, high-availability, and highly-scalable systems for a consumer website
Experience in a 24x7 e-commerce environment, including payment systems
Advanced expertise in all things Ruby and Ruby on Rails - experience in releasing and/or maintaining open source code, deploying large applications, memcache, working/starling, and TDD/BDD are welcome
Solid experience in the following skill areas: HTML, CSS, JavaScript (plus related frameworks), MySQL
Experience building RESTful web services APIs for developers and integrating third-party web service APIs
Extensive experience performance tuning web applications in a Linux environment
Experience in C, Perl, Java, and other languages a plus
Experience with project management tools and/or bug tracking tools
Energized by the opportunities and challenges of a start-up

Please Do Not Apply If

Your largest web application was 5000 lines in Rails (or equivalent sizes in Java, Python, etc)
You have never needed to use memcache (or similar tools) to scale an app
Your only real world experience with Ruby or Rails includes demo apps and side projects

Logistics

We are located in the Baltimore-DC area
We want candidates who can be on-site and in our office, though we are very open to partial telecommuting once you are established
Email collectivex-jobs@thinkrelevance.com with the role you are best suited for, and why you would rock at it.

Micronaut: Innovation Under the Hood

2009-04-01T18:10:00Z

Last week, Rob announced Micronaut, our new BDD framework written by Chad Humphries. I’ve been itching to write about Micronaut as well, and my take on it is similar to Rob’s, although I start with a very different perspective.

It’s a little surprising that I’m excited about Micronaut. I’m sort of a Ruby old-timer, and I’ve never been that excited about RSpec or any of the other Ruby test frameworks that have appeared over the last few years. Yes, they offer some advantages in some cases, but I’ve always seen the improvements as incremental rather than revolutionary. I like RSpec’s facilities for structuring tests, but should never struck me as a radical improvement in expressiveness. In fact, I frequently run into situations where I think a well-crafted assertion is much clearer than a should expression.

I just never got the BDD religion, in other words.

What I like about Micronaut is that its innovation is mostly under the hood, rather than on the surface. Rather than designing some new variation on how we express our tests, Chad opted for RSpec compatibility and built a really nice engine for loading and running RSpec-style tests. Architecture matters, and Micronaut’s architecture is a joy.

First of all, as Rob noted, Micronaut is small, easy to understand, and fast. That makes it fun to hack on, and great to use on projects. It also gives me confidence; simple tools tend to be more robust, and I want that in my testing tool.

Second, Micronaut’s architecture provides a lot of flexibility. To illustrate that, while attending talks at QCon a couple weeks ago, I added test/unit compatibility to Micronaut. Here’s a short example that mirrors Rob’s example from last week, but in a test/unit style:

require 'micronaut/unit'

Micronaut.configure do |config|
  config.filter_run :focused => true
end

class MicronautMetadataTest < Micronaut::Unit::TestCase

  # 'testinfo' allows declaring metadata for a test, and works
  # like Rake's 'desc' method: it applies to the next test.

  testinfo :focused => true
  def test_this_will_definitely_run    
    assert self.running_example.metadata[:focused]
  end

  def test_this_never_runs
    flunk "shouldn't run while other specs are focused"
  end

  testinfo :pending => true
  def test_this_is_pending
  end

  testinfo :speed => 'slow'
  def test_too_slow_to_run_all_the_time
    sleep(10000)
    assert_equal 4, 2+2
  end
end

I don’t know how interesting test/unit compatibility is in its own right. I certainly wouldn’t claim that the example above is superior to the RSpec equivalent. But it was a fun exercise to really prove Micronaut’s design. It only took a few hours (of partial attention) and it clocks in at under 200 lines of code (plus assertions.rb from the test/unit codebase). I love the idea of the testing API and the test execution engine not being so tightly coupled. And it might be very useful if you have existing tests written with test/unit and you’d like to gain the benefit of Micronaut’s metadata support.

Which brings me to Micronaut’s best feature. Almost all unit-testing frameworks incorporate some support for test suites, but suites have never lived up to their promise. It would be nice to have tests organized in suites for distinct purposes, but the setup and maintenance costs associated with suites have meant that very few teams make good use of them. Micronaut’s metadata is a different—and much better—solution to the same problem. As some have noted, the idea is similar to Cucumber’s support for tags. Just attach metadata of your choice to Micronaut tests or examples, and then you can use that metadata to select what tests are run in different situations.

At the moment, test/unit compatibility is only available in my fork of Micronaut. It allows using RSpec-style describe and it blocks alongside test/unit-style test methods, and supports test blocks à la Context and ActiveSupport. I don’t think we’ll pull this completely into Micronaut; I think it would work better as a separate gem. But I’d love to get your feedback about the whole idea.

Introducing Micronaut, a Lightweight BDD Framework (Or, Reinventing the Bestest Wheel Yet)

2009-03-26T19:00:00Z

Relevance is releasing a new Behaviour Driven Development (BDD) framework, even though the Ruby community already has great tools like RSpec, Shoulda, and Context. Let me tell you why.

When I joined Relevance in 2007, I led a conversion from test/unit to test/spec. It was an easy win - we could use "expected.should == actual" syntax, which was nice, and using strings instead of "test_foo_bar" methods is huge. We wrote spec-converter to ease the conversion, went from test/unit to test/spec, and everyone was happy.

Enter 2008: RSpec gains momentum and we see its API stabilize; meanwhile we continue making little tweaks and hacks to our fork of test/spec. I added focused spec support, which basically means you change your "it" to a "fit", and that spec will run in isolation in your suite. This was a major deal for staying in flow with large and/or slow suites - if you have 15 broken specs, focus on the simplest one and make it pass, pick the next failing spec, lather, rinse, repeat. Once you are used to having focused specs, it becomes hard to go without them.

Chad Humphries joined Relevance in mid-2008, and brought with him a passion for RSpec. We had some "differences of opinion" when it came to RSpec vs. test/spec. I wanted support for focused tests and a small codebase that lent itself to extension; Chad wanted things that "just worked" from RSpec and the community of tools and knowledge that exists for it. At the time, test/spec also had some quirks that RSpec had already solved, like before blocks not inherited for nested describes, and better Rails support.

So Chad went off into a desert around Thanksgiving of last year and came back with Micronaut written on stone tablets in about a week. It was lean, mean, API-compatible with RSpec, and well under 2000 lines of implementation code. It can also be extended very easily to support focused specs, or slow specs, or ignored specs, or any other kind of metadata you would ever want in your suite. Each example (an "it" block, aka a spec) and each behavior (a describe block) accepts metadata as an options hash, which you can use to do just about anything you want -- filter what runs at runtime, tweak which modules get included/extended, or change the before/after blocks that evaluate.

A short example:

gem 'spicycode-micronaut'
require 'micronaut'

Micronaut.configure do |config|
  config.filter_run :focused => true
  config.alias_example_to :fit, :focused => true  
end

describe "Metadata support in Micronaut" do
    it "this will definitely run", :focused => true do
      self.running_example.metadata[:focused].should == true
    end

    it "this never runs" do
      raise "shouldn't run while other specs are focused"
    end

    fit "this is also focused via the fit alias" do
      true.should_not == false
    end

    it "this won't run, its pending", :pending => true do
    end

    it "this is also pending, with the no block style"

    it "this spec takes *real* long and should only run when we want to take the hit of slow specs", :speed => "slow" do
      sleep(10000)
      2+2.should == 4
    end
end

Micronaut's metadata system is the next logical step in testing tools. Focused specs in test/spec was a small, fairly ugly, pragmatic hack. Micronaut makes metadata a first class citizen. True metadata becomes really important when you get into more significant codebases. You want this sort of power at the smallest level of granularity in your suite -- the individual example - as well as at higher levels. In Micronaut, metadata at the higher level of a describe block collapse down to nested describes and examples within describes, with lower level metadata always "winning" in the merge.

Micronaut is designed to be easily hackable and extendable, while staying small enough that you can easily find the right extension points and modules you want to change. It has made my day-to-day dev experience more effective and more enjoyable, because I really want to be able to own a tool so fundamental to my daily work as my testing tool. Although Micronaut is still pretty young and the internals are still evolving, the external API should remain fairly stable. Micronaut has been in real-world, production use on at least three Relevance client applications, many of our open source tools, and on the RunCodeRun codebase.

It is important to point out that we really like RSpec and continue to use it and support it. The state of BDD tools in the Ruby world owes a lot to David Chelimsky and team for their work on RSpec, and we love seeing the RSpec codebase get leaner and meaner as components get extracted out and Ruby 1.9 support matures. We also think tools like Shoulda, Bacon, Context are all good, because choice is a "good thing" for the entire Ruby open source community.

If you deal with larger codebases, or struggle with staying in flow in autotest, or just want to try a lean and mean BDD library, give Micronaut a try. The quick and easy way to try it is via the GitHub gem:

gem install spicycode-micronaut --source http://gems.github.com

Then drop the above example into a file to start playing. You can also check out the codes on GitHub, file feature ideas and bugs on Lighthouse, verify the build passes on RunCodeRun, and above all let us know what you think!

Fork Everything!

2009-03-03T23:10:00Z

It is now reasonable for some agile teams to fork most or all of their dependencies. Here's Why:

A Brief History of Forking

Traditionally, forking a software project was a big deal. You forked when you disagreed about direction and philosophy. Disagreed a lot--so much that you have were willing to make a big effort to do things your own way. And while forking didn't have to imply bad blood between forkers and those getting forked, that was sometimes the case as well.

So what is the big forking deal? Forking causes problems at several levels:

API Compatibility: How does a prospective user of a library deal with multiple forks with differing APIs.
Quality: If there are multiple forks of a project, how do you know which one is good?
Configuration: How do you track dependencies when multiple forks of a project have the same name.
Documentation: What's the difference between the forks?

And, of course, all of these problems increase for larger code bases--possibly faster than linearly with code size.

So, for a long time, it was safe to assume that forking was a big deal, and best employed rarely.

Forking Pain

To understand what changed, it is best to go back and revisit the problems of forking and ask how they might be exacerbated or mitigated:

API compatibility is a problem that grows if API differences are large, or expensive to discover. On the other hand, API changes are much less of a problem when they are small, or cheap to respond to.
Evaluating code quality is a problem if QA is expensive, either for a library itself, or for the integration between that library and your system. On the other hand, if there was a cheap test that said "This fork of X works correctly with my project," then forking X yourself looks more reasonable.
Configuration is a problem to the extent that it is expensive to track and manage forks. On the other hand, if you can easily review your dependencies, and switch between forks of a project in a few seconds, then configuration becomes less of a problem.
Documentation is a problem. What if a fork fixes a problem, or adds a feature, but the documentation is not up to date? On the other hand, if there is a cheap way to discover what a fork does, then a buffet of forks to choose from can be a good thing.

So forking is more reasonable, to the extent that dealing with API compatibility, code quality, configuration, and documentation/discovery is cheap and fast. Different people can reasonably disagree about "how cheap" and "how fast," but everyone should be able to agree that there is a continuum, and that particular development practices could make forking easier or harder.

Reaching a Tipping Point

Over the last several years, we at Relevance have participated in several trends, each of which lowers the cost of forking.

Good unit tests help with several of the items above. A good unit test suite will document the API and demonstrate the code's quality.
Good integration tests make it cheap to discover API compatibility issues. A good integration test makes it easy to evaluate alternative configurations that use different forks of dependent libraries. Finally, a good integration test can quickly prove that fork X works with your project.
Continuous integration helps you keep on top of the code health of a large number of projects. We developed RunCodeRun in part to keep all of our forked projects building cleanly.
Low-ceremony languages make everything smaller. Code is smaller and the tests are smaller. Documentation can be done almost entirely in the code itself.
Test-driven development helps produce good APIs between subsystems, by making bad ideas painful during development. Good APIs have a smaller surface area, and need to change less.
Distributed version control systems such as Git make forking itself cheap and easy. More importantly, you can quickly update your configuration to switch between different forks. Also, it is much easier to manage merges and maintain your own personal fork that pulls needful pieces from multiple other forks.
Relentless refactoring keeps code readable, so the prospect of evaluating a fork by reading or diffing its source code is much more palatable.
Open source libraries and a culture of source-code deployment make it easy to manage all your dependencies at the source code level. Most of our Ruby projects vendor everything, so it is a simple Git operation to switch to a different commit (or a different fork!) of any dependency.
Social sites like GitHub make it easy to discover and track forks of a particular project, and for many different forks to pull from each other. Tools like the Network Graph Visualizer show how your fork differs from other forks, and can help you quickly locate commits you may be interested in.

The combination of all these factors makes forking way easier than I would have believed possible, even a few years ago.

Fork Your Dependencies!

As a result of all these trends, we now regularly fork the third-party dependencies in our projects. Imagine the following scenario: You are nearing a project deadline, and you discover a bug in a third-party library. Here are some possible reactions:

The closed-source way: Call a paid support line for your commercial software, explain your problem to a drone, and hope for a fix in the next release 18 months out.
The open-source way: Contact the project maintainers and convince them of the justice of your cause. If that is taking too long, and you are in a language that enables it, monkey patch.
The "Fork 'em!" way: Fork the project and fix it yourself. The original owners can debate your ideas in their own time (or not). Who cares? You are back up and running.

The cost of forking is now low, that it isn't even limited to fixing critical bugs. We fork to fix minor bugs, to add features, or to make usability improvements to APIs. In short, any kind of change we might make in our own code, we might also be willing to fork and make in somebody else's.

If you had asked me 18 months ago if the "fork 'em" approach would work, I would have said "no," and written you off as a crank. Luckily, nobody asked me! Rob just started doing it, and quickly showed that it worked.

Will this work for everybody? Absolutely not. You need to have a lot of other practices in place first. Otherwise, casual forking will only lead to trouble. But if you are running agile the right way, producing and consuming clean, tight, tested code, you may discover forking around is downright healthy for your code.