Derek Hammer

Binding in Swing

2013-04-23T00:00:00-05:00

Swing is pretty old. We've developed new technologies since its heyday and those new technologies have brought new best practices. One of those is the ability to bind the UI to a Model or ViewModel object. In my current project, we are using Swing as a UI framework and are attempting to move it from the old to the latest and greatest. One of those includes binding.

This is short and sweet, but you can do simple data binding in Swing pretty easily. What we have done is to create a ViewModel object that the view will render from; this ViewModel object is testable and isolated from the Swing code entirely.

For example, in this very simple application we might have three text boxes and a button. TextBox1 and TextBox2 are integer fields that will be added together when an "Add Them!" button is clicked and the result will be displayed in TextBox3. The resulting ViewModel might look something like this.

public class AdditionViewModel {
  private String first;
  private String second;
  private String total;

  public void setFirst(String value) {
    first = value;
    }
  public void setSecond(String value) {
    second = value;
  }

  public String getFirst() {
    return first;
  }

  public String getSecond() {
    return second;
  }

  public String getTotal() {
    return total;
  }

  public void addThem() {
        total = first + second;
  }
}

This is an incredibly simple class which can have tests written against it very easily. Subsequently, the view code might look something like this.

public class AdditionPanel extends JPanel {
    private final AdditionViewModel model;
    // stuff
    private void setupInnards() {
        final JTextField textBox1 = new JTextField();
    textBox1.setText(model.getFirst());
    textBox1.addActionListener((e) => {
      model.setFirst(textBox1.getText());
    });
    this.add(textBox1);

        final JTextField textBox2 = new JTextField();
    textBox2.setText(model.getFirst());
    textBox2.addActionListener((e) => {
      model.setFirst(textBox2.getText());
    });
    this.add(textBox2);

        final JButton button = new JButton();
        button.setText("Add!");
        button.addActionListener((e) => {
            model.addThem();
            redraw();
    });
    this.add(button);
  }

  private void redraw() {
        this.clear();
    this.setupInnards();
        // alternate method:
    // this.textBox1.setText(model.getFirst());
    // this.textBox2.setText(model.getSecond());
  }
  // more stuff
}

This example would get you a simple form of data binding (unfortunately you still have to manually call redraw.. but there are other ways to get to that which I may talk about in the future). The goal and the win here is to push two different things away from the views: logic (even logic we might consider 'view logic') and state management. I want my view to be as dumb as possible so that my tests involving Swing and stateful situations are reduced to a minimum.

Technical Post-Mortem for Failures

2013-03-26T00:00:00-05:00

This is just a quick template that I am using with my teams.

Hammer's Technical Post-Mortem for Failures

What was the problem? Please be as concrete and simple as possible (e.g. the build failed due to data not being cleaned up the database)
What was the root cause of the problem? Consider using the 5 Whys in order to really track down root causes.
Was the issue resolved quickly? If so, what processes or techniques made that happen. If not, what gaps in our processes or techniques exist?
Identify actionable items that would resolve the root cause of the issue or change the processes and techniques to accommodate failures of this type.
Assign owners to the action items.

Stronger Backbone - Part 3

2013-01-28T00:00:00-06:00

There is a better way to write Backbone applications and here I hope to write a quick primer. The topics I will cover in this series will follow these basics:

Everything-from-one-container
Pass-through views
Declarative rendering
Abstract away element access (TBD)
ViewModels over Models (TBD)
BONUS ROUND Coffeescript (TBD)

Declarative rendering

Rendering is one of the more complicated aspects of creating single page javascript applications. It has implications in performance, responsiveness, debug-ability, testability and can have a steep learning curve. For the most part, rendering falls into one of two categories: implicit and declarative.

Implicit rendering can be put another way: render only when I tell you to render. So, our rendering code would look something like this this.childView.render() and it might be stashed in a block of code like this:

newMessage: function(from, to, message) {
  var message = this.messages.create({
        from: from, to: to, message: message
    });
  this.messageView.render(message);
}

The characteristics of this method are:

Render is called explicitly by a line of code that the developer is writing.
Data is often passed in as an argument to the render method.
Views are passed around to services, controllers and (maybe) collections in order to have access to direct rendering.

This method is a very straight-forward, easy to read method of rendering views. The other method, declarative rendering, would look more like this:

Service Code

newMessage: function(from, to, message) {
    this.messages.create({
        from: from, to: to, message: message
    });
}

View Code

initialize: function(config) {
    this.messages = config.messages;
    this.messages.on('add', $.proxy(this.render, this));
},

render: function() {
    _.template('some-template', { messages: this.messages });
}

In this snippet, we see that the view and the service/controller are completely separated. The view gets triggered through an event that is fired by the Backbone Collection (built in behavior). The code snippet itself is a bit larger, but the amount of code is essentially the same. The characteristics of this approach are:

Use built in (and sometimes custom) events to execute render methods.
Pass in collections as part of the views constructors (so that we always have a reference).
Render typically takes zero arguments and pulls data from the encapsulated state.

This code is more disjoint than the explicit code above. That is, it is more decoupled. I argue that this is a good thing because:

Entanglement. When we have services that are explicitly calling render on views, they become entangled with the action of rendering those views. For example, imagine that the clearSelection and selectItem from Part 2 did explicit rendering calls. When we called selectItem, there would be an extra render call that we probably did not want to do. In order to separate that out, we would need to create another method (perhaps clearSelectionAndRender). This very quickly explodes the number of methods in our classes and makes following the logic difficult.
Performance. Because of entanglement, we will often miss that we are rendering a view multiple times before we finish responding to a single user action. Declarative rendering reduces (though does not eliminate) much of this mistaken rendering.
Testing. It is much easier to test both the view and the service/controllers in isolation when they are dependent on less objects.
Declarative view rendering, in a rather unintuitive sense, makes readability of our views better. For example, if we read the second example's view code above in isolation, we grasp that the view is only going to render on "add" of the messages element. This may be what we want or we may want to change the binding to "add remove", "change" or "fetch add".

Conclusion

With declarative view rendering, we have decoupled parts of our Backbone app and have really started to separate concerns of our Model layer (represented by Collections and Models), the View layer and the Controller/Service layer. We start to see that the services are not driving views but are interacting with models. Views are merely responding to and displaying changes in the models. And views are delegating user actions to the controllers.

In the next part of this series, we are going to finalize our views with a little element access massaging. Stay tuned!

Stronger Backbone - Part 1

2013-01-27T00:00:00-06:00

There is a better way to write Backbone applications and here I hope to write a quick primer. The topics I will cover in this series will follow these basics:

Everything-from-one-container
Pass-through views
Declarative rendering
Abstract away element access (TBD)
ViewModels over Models (TBD)
BONUS ROUND Coffeescript (TBD)

Everything-from-one-container

Often when evaluating other backbone applications I will see somewhere deep in the program a line of code like this:

render: function() {
  for(var i = 0; i < this.items.length; i++) {
    $('#table').append(_.template('item_template', { item: this.items[i] }));
  }
}

This code is appending to the element with id table N number of items. The biggest issue with this code is that it is not a self contained view. If there happened to be another element with id table on the page, it would add the items in two places (while also being malformed HTML). We could solve this scoping the selector to the element represented by the view. So instead of $('#table') we would use this.$el.find('#table'). This solves the immediate problem of scope, but the code here has another hidden dependency.

It depends on the fact that whatever element the view is bound to has an element with id table as a descendant. This makes the code less portable and more tightly coupled to a global resource. We can improve the code by instead appending the children to the element that the view is bound to. So the code would go from this.$el.find('#table').append to this.$el.append. This removes that external dependency and simplifies our code itself.

If we apply this principle to the extreme in our application, we will find that we are pushing more and more of our bindings "up" from the leaf nodes of the tree all the way to the root node. Eventually, we will only be left with one binding. That is the binding that the application initially goes into. This is the "one container" form. You start your application with code like this new MyApp({el: $('body')}) and, because we can inject the one HTML dependency at creation, we have successfully removed all dependencies on the static HTML that we render to the client. Our application is extremely portable and can be dropped into any HTML resource.

So, the app may look something like this

MyApp(el: some-html-element)
  append MyApp.Header
    html template 'header'
  append MyApp.Sidebar
    html template 'sidebar', { items: objects }
  append MyApp.Content
    html template 'content', { item: selectedItem }

Conclusion

With the 'one container' form/pattern, we have eliminated one of the hairiest problems in Backbone -- the disconnection between the static HTML resource and the application itself. We did this by removing our reliance on the static HTML resource altogether and to make the app truly self driven.

In next post (which I am writing now!), I'll talk about how to write views so that we can begin to build a modular, testable application that adheres to the principles of SRP and DRY.

Stronger Backbone - Part 2

2013-01-27T00:00:00-06:00

There is a better way to write Backbone applications and here I hope to write a quick primer. The topics I will cover in this series will follow these basics:

Everything-from-one-container
Pass-through views
Declarative rendering
Abstract away element access (TBD)
ViewModels over Models (TBD)
BONUS ROUND Coffeescript (TBD)

Pass-through views

In the old days (and still current for the unlucky few) of .NET web application development, there was a framework called WebForms that developers would use to create web applications. WebForms used a system called "CodeBehind" that allowed developers to interact with objects in the views.

The .NET community has since grown and the MVC/MVP stack is widely regarded as superior. The idea of separating views from handling user interactions is something that most developers are keenly aware of these days.

However, with Backbone I often see some regression. For example, it is not uncommon for me to see something like this:

events: {
  'click .item' : 'selectItem'
},

selectItem: function(e) {
  var id = $(e.currentTarget).data('id'),
      item = this.items.get(id);
  _.each(this.items, function(i) { i.selected = false; i.save(); });
  item.selected = true;
  item.save();
  detailView.setItemSelected(item);
  return false;
}

What we are doing in this view is grabbing the associated item, clearing selections of other items, modifying it, saving it and telling another view that the item is selected.

Let us assume we are okay with this code. We push it to production. It works. Everyone is happy. Then, we get a customer request to be able to clear selection with an "x" in the detailView. A common pattern that doesn't seem unreasonable. We add the following code:

events: {
  'click .clear-selection' : 'clearSelection'
},

clearSelection: function() {
  _.each(this.items, function(i) { i.selected = false; i.save(); });
}

Again, the code works well and gets the job done. However, being lazy developers, we notice that we have written the exact same code twice. How would we deduplicate this code? We extract it to a class.

var SelectionService = function(items) {
  this.items = items;
  return this;
}
_.extend(SelectionService.prototype, {
  clearSelection: function() {
    _.each(this.items, function(i) { i.selected = false; i.save(); });
  },

  selectItem: function(id) {
    var item = this.items.get(id);
    this.clearSelection();
    item.selected = true;
    item.save();
  }
});

And then we would use the class service.selectItem(id) and service.clearSelection(), respectively, in our code to reduce the amount of duplication. As your application becomes more and more complex, these components will be shared amount more views of your application. The goal to to create views that only render and delegate user interaction to the appropriate "service" classes. You can call them controllers, if you wish.

Conclusion

In this we talked about making your views smaller by having them delegate to another class. This other class can be shared among other views without having to duplicate code or logic.

There is still more that can be done to clean up this view (i.e. the setSelectedItem call to the other view). We will discuss this in the next post.

NOTE There is another way of sharing code and that is through mixins. I did not talk about them here but I definitely recognize their existance. It is my belief that they are not useful in this context but others may disagree.

A Pattern for Rails Presenters

2013-01-20T00:00:00-06:00

On my previous project, my team developed a pattern for Rails presenters that I think is worth sharing. We used this pattern to a great deal of success and I will be advocating for its use in future Rails projects.

The Presenter pattern is what I call a projection of a model. That is: given a model A which contains data Z, a presenter is a derivation of model A and data Z. We can represent this as A' (Presenter class) and Z' (Presenter data). For any given model A, there are an unlimited number of projections and, thus, presenters.

Let's do something more concrete: we have an object of class User with data {first_name: 'Derek', last_name: 'Hammer', gender: 'M'}. A presenter might be called UserSalutations and is a derivation of the User model. The data might be {greeting: 'Hello, Mr. D. Hammer'}.

The benefits of the Presenter pattern is to separate domain logic (the changing of a person's name) and the presentation logic (formatting the person's name as a salutation). I will be quick to point out that the pattern is not always applicable and should be used as a conscious choice rather than as a matter of course.

We have User and we have UserSalutation. Both of these are a part of our system. How might the code that uses this look?

class MessagesController < ApplicationController
    def new
        @message = Message.new
        @user = User.find(params[:id])
        @salutations = UserSalutations.new(@user.first_name, @user.last_name, @user.gender, @user.id)
    end
end

class UserSalutations
    attr_reader :last_name, :gender, :first_name, :user_id

    def initialize first_name, last_name, gender, user_id
        @first_name = first_name
        @last_name = last_name
        @gender = gender
        @user_id = user_id
    end

    def greeting
        "Hello, #{honorific} #{first_initial}. #{last_name}"
    end

    private

    def honorific
        case gender
        when 'M' then 'Mr.'
        when 'F' then ms_or_mrs
        else then 'Mr.'
        end
    end

    def first_initial
        first_name.first
    end

    def ms_or_mrs
        if Marriage.where(person_one_id: user_id).or.where(person_two_id: user_id).any?(&:active?)
            "Mrs."
        else
            "Ms."
        end
    end
end

This method works, but we are doing some database access in our presenter. This makes the presenter difficult to test and require outside dependencies. One solution is to push the code "up" into the controller.

class MessagesController < ApplicationController
    def new
        @message = Message.new
        @user = User.find(params[:id])
        @salutations = UserSalutations.new(@user.first_name, @user.last_name, honorific)
    end

    def honorific
        case gender
        when 'M' then 'Mr.'
        when 'F' then ms_or_mrs
        else then 'Mr.'
        end
    end

    def ms_or_mrs
        if Marriage.where(person_one_id: user_id).or.where(person_two_id: user_id).any?(&:active?)
            "Mrs."
        else
            "Ms."
        end
    end
end

This solution is actually worse, though! We are creating a fat controller and controllers are notoriously difficult to test and maintain (hence the skinny controllers mantra espoused by the Rails community). The solution that my team came up with was the following:

class MessagesController < ApplicationController
    def new
        @message = Message.new
        @user = User.find(params[:id])
        @salutations = UserSalutations.from_user(@user)
    end
end


class UserSalutations
    attr_reader :last_name, :first_name, :honorific
    
    def self.from_user user
        self.new user.first_name, user.last_name, honorific(user)
    end

    def self.honorific user
        case user.gender
        when 'M' then 'Mr.'
        when 'F' then ms_or_mrs(user)
        else then 'Mr.'
        end
    end

    def self.ms_or_mrs user
        if Marriage.where(person_one_id: user_id).or.where(person_two_id: user_id).any?(&:active?)
            "Mrs."
        else
            "Ms."
        end
    end

    def initialize first_name, last_name, honorific
        @first_name = first_name
        @last_name = last_name
        @honorific = honorific
    end

    …
end

Here, we have a testable, dependency free UserSalutations object. The dependencies are encapsulated by the factory method on the UserSalutations class object. Our Rails controller is still skinny (actually, its skinnier than before!). The downside is that the factory methods need to talk to external dependencies. This is okay, I think, because we are reducing the complexity here down to its bare minimum and testing in isolation.

So, in short, use a combination of factory methods and PROs in order to keep your presenters testable, maintainable and your controllers skinny!

Maintaining internal gems

2012-08-25T00:00:00-05:00

On my current project, we have been growing our system to a rather large size. As with any codebase that grows, this means that we have started to think about code organization that relies on creating our own libraries and gems.

At this time, we have around 5 or 6 "internal" gems that are just consumed by our project. These gems are things like third party integration classes, testing tools and generalized code that could be used outside of just this project.

One of the things that we've started doing is versioning and tagging the external gems instead of pulling from the master branch of the gems (we don't use a gem server -- just GitHub). This reason for doing this is really based around continuous delivery and the traceability of our changes.

A thought experiment: I give the QAs a build that they need to look over and verify. They find a regression. They report it back to the developers and you pick up the story to try and fix it. The first thing that you need to do is to reproduce the bug. How do you do that? Normally we would check out the commit that corresponds to the QA build and follow the steps. However, this bug is actually in the gem. When you bundle install that gem, you're pulling from the gem's master and not a specified tag. How do you get to the gem's specific version that actually created that issue? You'd have to do some time-based hackery with logs from the time that the original build was created. Ugly and error prone.

Instead, simply tag and version your gems and use those tags in your Gemfile. Then, when you checkout that specific version, all the information that you need about versioning is versioned right into your app. It'll make your life easier!

Write Ruby in Rails

2012-05-12T00:00:00-05:00

You're a Rails programmer. You've developed your site and have a good set of features that you're delivering to your users. You use RSpec to test your application and have Cucumbers pushing WebDriver tests. You may even be doing some cool process things like continuous deployment. But, do you write Ruby or do you write Rails?

Writing Rails

Rails is a great web framework that has helped many teams deliver business value in a very short amount of time. It helped to create the mantra of Opinionated Software that exists in the Ruby community and has even started to trickle down to users (for the benefit of all, in my humble opinion). Rails has also been a great leader in the area of testing and craftsmanship in the greater software practice.

That being said, Rails has become a crutch for many people (as all frameworks tend to do). It encourages a very flat world of MVC where the models can go get themselves out of a persistent store and controllers and views can access everything.

How much code do you have that is not MVC?

Name the number of classes in your code that do not inherit (directly or indirectly) from ActiveRecord::Base (or, in my case, Mongoid::Document), ActionView::Base, or ActiveController::Base. Is it more than zero? Is it a significant percentage of your application? Is it most of your application?

At a certain point your web application will become more than a CRUD web server where users manipulate objects in a database. Let us take for example GitHub. When you first log in to GitHub, you are sent to your dashboard. I'm going to write some bad code:

class DashboardController < ApplicationController
  def index
    # we're using devise and the user is logged in (checked in ApplicationController)
  end
end

.dashboard
  .news
    - current_user.alerts.limit(20).each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

Excuse the incompleteness of the example but this is enough to illustrate. This is a Bad Thing™ because we are doing many N+1 queries against our SQL database. Easy enough to fix. We should pre-fetch all of our dependencies. I am going to use a more explicit way instead of trying to let the ORM do my work for me, again for illustration.

class DashboardController < ApplicationController
  def index
    @alerts = current_user.alerts.limit(20)
    @commits = Commit.where(:_id.in => @alerts.map(&:commit_id)).to_a # using Mongoid here
    @committers = User.where(:_id.in => @commits.map(&:committer_id)).to_a # using Mongoid here
  end
end

.dashboard
  .news
    - @alerts.each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

Alright, so now we've taken care of the N+1 query issue. However, I'm loading a ton of unnecessary data from the database, building it into objects in member and then garbage collecting it later down the road. We're GitHub and we cannot afford this luxury, so we cut the fat (always test that this sort of optimization is necessary though!!).

class DashboardController < ApplicationController
  def index
    @alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    @commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    @committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
  end
end

.dashboard
  .news
    - @alerts.each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

And, we're starting to notice that our .header= HAML line is really big. We could pull it into another helper method. Or, we could pull it up into the model itself. Which is it? View logic or model logic? For sake of my example, let's put it in the model.

class Alert
  include Mongoid::Document
  
  def header
    commit? ? commit_header : pull_request_header
  end
end

And we notice that our controller method is getting big-ish. We should move that code to the model, as well, right?

class Alert
  def self.alerts_for_dashboard(current_user)
     alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
    alerts
  end
end

And this is the state in which we push the code to production and call it a day.

What about Ruby?

We forgot about using Ruby itself at some point in learning how to develop Rails applications. I would contend that the model is the place for neither of these peices of code. The header code is not part of the model but is instead presentation logic.

On helpers: helpers are the default way to encapsulate presentation logic in Rails. There is nothing wrong with helpers themselves but they start to get out of hand when you start to pull in many different objects, cascade helper calls and use helpers from many different helper files. When that starts to happen, we need to move away from helpers to a more robust method of handling presentation logic.

This dashboard view should probably have something called an AlertPresenter (for another pattern, look at Avdi's Exhibit Pattern). The AlertPresenter would look something like this

class AlertPresenter # look ma, I'm an Object
  def initialize(alert) # look dad, I'm using explicit constructor arguments!
    @alert = alert
  end

  def header
    @alert.commit? ? commit_header : pull_request_header
  end
end

This would encapsulate all of the logic needed to write the "header" of the GitHub dashboard alert in one place. If I ever needed to change what that looks like, I know exactly where to look. Another side benefit of this class is that I do not need to load Rails to run this test. That means the test can run fast. Very fast.

The other peice of code is not presentation logic, but persistance logic. Because of ActiveRecord, we have started to conflate persistance and domain logic in our codebases. This is a bad trend. It creates confusion and a dangerous amount of coupling in our code. While I could go on about the benefits of a DataMapper pattern over the ActiveRecord pattern, I will just show you what we could have done with our code to not bind ourselves so tightly with AR.

Instead of defining a class method on Alert, we could create an AlertService that accesses these alerts for us. At this point, it really is just putting the code into another file but it does have benefits. First, its not Rails and doesn't actually require Rails to test. It makes the finding of alerts for the dashboard logic all easy to find and located in a different location. And, it provides a way for us to change how we access it. If, for example, we were to implement CQRS in our application, the Service would be the place to do it. Or, if we decide that we need to implement an HTTP service on the backend that serves us the objects instead of direct access to the database, the service layer can be a point of abstraction for persistance logic. There are more benefits if we take the service/repository pattern further.

class AlertService
  def self.alerts_for_dashboard(current_user)
    alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
    alerts
  end
end

So, in this very small example, we have decided to create a couple of Ruby objects that make our application more flexible, testable and cleaner.

Cleanliness

I challenge Rails developers to develop a sense of cleanliness of their code. We have a huge commitment to tests and working software. We have a commitment to opinionated software that relies on convention over configuration and to give back to the open source community. It think us Rails developers can learn to appreciate the cleanliness of code. I would define cleanliness as follows:

Thin controllers (the same thing we have been doing for a long time)
Logicless views (use Presenters/Exhibits to wrap any presentation logic)
Domain models (avoid persistance and presentation logic)
Repositories [they can be called Services] (for persistence logic)
Services (for manipulation of Domain Models -- puppet masters if you will -- this is another area for domain logic)
Fast tests that do not rely on Rails (see Corey Haine's Fast Tests talk)

I've only talked about a couple of the things that make clean Rails code possible (Presenter and [Persistance]Service). I hope to talk about other things in the near future.

Use exceptions in Rails to create RESTful endpoints

2012-05-03T00:00:00-05:00

We have recently started to use a pattern on my project that I am currently in love with. While it is probably too early to call this a "Do This or Question Why You Do Not", I can see it rapidly becoming that way.

The typical Rails controller

The Rails controller that we typically see is something along the lines of the code below. In this code we are using some straight forward patterns of accessing objects from a database, showing them (through a template), checking to see if our new Post object is valid and handle the good and bad cases in an if statement. This is very basic Rails code that any Rails developer would be able to easily recognize and use.

def show
  @post = Post.find(params[:id])
end

def create
  post = Post.new(params[:post])
  if post.save
    flash[:success] = t(:post_created)
    redirect_to post
  else
    flash[:error] = post.errors.full_messages
    render 'new'
  end
end

Rails controllers for API endpoints

However, this is not entirely RESTful and can even hurt in the world of creating APIs. Let us assume, for the moment, that we are creating these same endpoints for a single page Javascript client. The code would look similar to the following (I'm assuming a templating language like rabl):

def show
  @post = Post.find(params[:id])
end

def create
  @post = Post.new(params[:post])
  unless @post.save
    head :unprocessable_entity
  end
end

Here we are doing much the same work and are a little more restful. We are returning 200 and 422 and 500 error responses. This is not quite restful, though. We should really handle the situation in show where an invalid id is being requested.

def show
  @post = Post.where(params[:id]).first
  head :not_found unless @post
end

And, now, we are actually going to populate these posts with images from Gravatar. We need to fetch some links.

def show
  @post = Post.where(params[:id]).first
  @gravatar = GravatarService.fetch_for(@post.user)
  head :not_found unless @post
end

But, it is possible for the Gravatar service to time out or explode in some expected way that returns a 500 error. That 500 error is actually right in its class (Server Error) but wrong in its specificity (it is probably a 503 Service Unavailable error). So, we need to be a little better about handling that error

def show
  @post = Post.where(params[:id]).first
  begin
    @gravatar = GravatarService.fetch_for(@post.user)
  rescue GravatarServiceError
    head :service_unavailable
    return
  end
  head :not_found unless @post
end

And, now we're off to the races. Our clean PostsController#show and PostsController#create has turned ugly.

Use Exceptions

So, we're going to use exceptions to clean this up correctly. Here's how I would write those in the new pattern

def show
  @post = Post.find(params[:id])
  @gravatar = GravatarService.fetch_for(@post.user)
rescue Mongoid::Errors::DocumentNotFound # Or AR equivalent
  head :not_found
rescue GravatarServiceError
  head :service_unavailable
end

def create
  post = Post.new(params[:post])
  if post.save
    head :created
  else
    head :unprocessable_entity
  end
end

And, if I really had my druthers (which I do on my project), we'd be happier to use a Service layer instead of AR.new/AR.create.

def create
  PostService.create(params[:post])
  head :created
rescue PostServiceError
  head :unprocessable_entity
end

So, we are now looking down the barrel of a very RESTful API endpoint. It is very clean and describes exactly the response that we expect based on how the server is behaving. If we cannot create a new Post (for whatever reason.. we don't know what it is here), we return a response of 422 Unprocessable Entity to the client.

Again, I am really liking how clean this is. However, I can see some issues with it: we are currently using this for an API-only area of our application. It looks to be working wonderfully for us. However, if we extend this to all parts of the application I wonder about spaghetti exceptions or introducing exceptions in one part of the code that effects multiple endpoints.

If we maintain this in only the API part of the application, we have a conflict of styles if I'm using a service in both parts of the application. One will not expect the service to throw exceptions and the other will expect it.

All in all, though, this seems to be a great start to correctly handling many different return codes from an HTTP resource. If we're trying to be more RESTful with out resources, we need a way to handle different kinds of errors at various points in the process and exceptions are looking to be a good way to solve it.

The Inaudible Annoyed Sigh

2012-04-29T00:00:00-05:00

In case you are unaware or are in need of a reminder, this is a technical blog about software. I write my thoughts and opinions on how to write software. Today, I am going to continue that by talking about Women in Technology.

"Not that dead horse." "It's been done a thousand times." "Why aren't you talking about something REAL like how to introduce a service layer peice by peice in a Rails application?" (I'll do it one day, I promise!) This, folks, is as real as it gets. If you thought this or anything along these lines then you're giving me the inaudible annoyed sigh.

Most of the technology industry (both men and women) will have stopped reading this article by now.

Last week, a reporter at Mother Jones wrote an article about Brogrammers and yet another boneheaded talk by a techie that glorified misogyny. There was an appropriate level of outrage on the internet by well meaning people.

When was the last time that the internet reacted to anything with the "appropriate level of outrage"? When was the last time that a technically oriented person let anything go with a wag of the finger on Twitter and continued to work on things. Never; in fact, the opposite happens. This is so common that there is a word for it: nerdrage.

I do not encourage nerdrage as a practice but the fact that the response to these incidents in the technology industry is so lukewarm and lackluster compared to the battles of epic proportions that span decades over things like "ViM v. Emacs" disappoints me. We can generate years of conversation, debate, slide decks, point and counterpoint, etc. into two tools but we cannot feel empathy for fellow human beings that may want to one day hang out with us and create cool things.

I am disappointed.

Technologists that approach Women in Technology as a peripheral, as an annoyance, as a dead horse that continues to get beaten just do not get it. When we accept that premise, we have already lost. We start to care more about tools and lifeless objects than we do about fellow human beings. We let people create slideshows of women in bikinis, ads with girls on stripper poles, propogate the misogynist brogrammer to unhealthy levels unless someone like Tasneem Raja calls it out and makes us feel guilty until we respond with the appropriate amount of disgust and finger wagging.

This is not exclusive to women (though it is very prevalent in the women minority of the tech world); technologists tend to shut out the "soft side" of technology. Technology is created by people that work in teams which often operate collaboratively. We create technology not necessarily to scratch an itch but ultimately serve our fellow human beings in a positive way. Race, gender, social status, nationality, etc. all suffer from this "hard side" focus in technology.

So, no, I'm not looking for a nerdrage or for a public shaming of the SXSW speaker. What I'm looking for is the absense of the annoyed sigh. I'm wanting technologists to learn and understand that being aware of antihumanism is more important than learning how to use Redis. We should not sigh and shy away from the uncomfortable discussions that have no right or wrong and that involve people and not tools; we should embrace them as being a part of a community and realizing that part of being a technologist and human being is to embrace those conversations.

This is a blog of a technologist and I am talking about how it is important to recognize and reject misogynism in our community.

P.S. I've been known to inaudibly sigh, as well. We can all learn.

Patterns to Isolate Code in Continuous Delivery

2012-03-24T00:00:00-05:00

Continuous delivery is one of those goals that is very easy to understand but very difficult to implement. The gist of continuous delivery is that the development team delivers a feature as soon as it is ready. This reduces overhead and the lag time between the process of developing a feature and seeing it to the next stage. Overall, this is a good idea because it allows for features to be developed and deployed at a more rapid pace as well as enabling users to interact with the new functionality in as little time as possible.

This is a great and simple idea, in theory. In practice, there are some inevitable complications with that process. One of those difficulties is how to develop new features while keeping your application deployable at any given time.

Imagine this situation: you're doing a continuous delivery pipeline for your product. You've told business that they can deploy at any time that they wish; they have informed the team that release is to go out tomorrow night. You've just finished a feature and pushed; the pipeline went green. Now, you have 4 hours left to work on new features. However, the next story to be played is rather large and is going to take around two days to complete. Obviously, you cannot finish the feature before the next release. What is the best practice in continuous delivery going forward?

This is a question with a very complex answer. The one sentence answer is: you isolate your new code from the users in the simplest way possible. "The simplest way possible", though, is up to the developers of the story and they must use their judgement to implement one of the follow set of practices.

Don't push until finished

In this practice, you develop your feature locally until your are finished before you push any code. In a DVCS like git, this might mean that you are committing multiple times locally for a multi-day story but do not push until you are completely done. This is essentially a short-lived feature branch that is not visible to anyone else.

You should use this practice for very small stories. If a story is going to take 1-2 hours to develop, go ahead and use this one. It is, by far, the easiest practice of any and it will work for smaller stories.
While this practice says that you should not push, it does not mean that you should not pull. Continually pull and merge your changes with what is on the master branch.
The downsides of this are the same of those of feature branches: the longer the local changes are not shared, the more divergent your code becomes to others. It becomes more and more a pain to refactor without significant merge conflicts and to understand the impact of any changes you make to code the longer the changes are not shared.
For anything of medium-to-large size, I would recommend against this practice.

Change the view last

In this practice, you are modifying the underlying system to a degree that is not permeated to the client. Then, when the underlying system is ready, you use another strategy (normally #1 Don't push until finished) to do the "view" part. This can be useful for refactoring-heavy stories.

For MVC web applications, a simple example is if you were adding a new endpoint that modified the controller and model as well as adding a new view. You can do the controller and model part first and then circle back to the view.
This can apply to non-MVC contexts, too: imagine a service that was exposing a RESTful JSON API. The JSON is the "view" in this case and if you can keep that the same you can monkey about behind it at will without effecting your consumers.
Personally, I like to work outside-in. I like to write ATDD tests that guide me through the entire story. Unfortunately, this practice is at odds with the working style. I would probably not employ this practice very often.

Short lived feature branches

Feature branches are always a controversial topic. Please see feature branches and continuous delivery are incompatible for more information on my take of that topic. However, for code isolation, short-lived ad-hoc feature branches can be useful. There are a slew of issues that make this an almost-last-resort type of thing for me.

Short-lived ad-hoc feature branches have a life cycle the same as a story in development and are one off sorts of decisions that a developer must make.
The can isolate code very neatly; the code is never seen on the master branch.
The isolation is at cost: we are wanting to isolate features from users not code from developers. Developers need the code as soon as it is written so that they have context for a variety of circumstances. Having a pile of code being "dropped" into a code base at irregular intervals can wreak havoc on a developers ability to make changes.
I would only use this strategy as a last resort for a really big, breaking change that we are consciously internalizing the cost of the Big Merge.

Short-lived ad-hoc feature toggles

Feature toggles are switches that exist inside of the production system which hide features from consumers. These switches can be implemented in a number of ways but all have the same effect: code is able to be continuously pushed without affected consumers.

These are a bit more expensive to implement than other strategies. They also introduce technical debt and complexity into the code base.
Feature toggles lifecycle should be the same as the story is "in development." They should be created at the start of the story and removed when the developer thinks the story is done.
There is a risk that business will use the feature toggles, which is a developer tool, as a system feature. Push back on this very hard. If business wants the ability to turn features on and off, it should create requirements and go through the normal product analysis. Feature toggles can become crazily complex otherwise.
There are multiple ways to implement feature toggles. I have heard or seen of the following methods: using a flat text file to set values true/false that gets read on launch, using Rails environments to set boolean configuration options, using URL parameters to turn a feature on or off, using server-level redirects to prevent new features from being accessed. Each has its strengths and weaknesses. Remember: just do what is the simplest way to isolate your code.

Build a new route

When developing an entirely new feature, it can sometimes be easier to build a new route (or endpoint) that is inaccessible in production. This is a relatively narrow use case: only when the new feature is entirely on its own is this useful. However, it is an important pattern to know because when it can be used, it is often the simplest method.

Build the new route as you would any other feature. Leave off the linking until the end (and use strategy #1). If its a sensitive feature, consider turning off the route for production.
You can use any development method for this (including outside-in TDD--yay!).
You can also use this method if you're overhauling another view entirely. Build the new view on a new route that will be swapped in at the end.
Use this whenever it makes sense.

Use existing feature turn on/turn off system

Some places have existing systems in place to do feature level system turn on and turn off in production. This is a product level system. It is, however, very close to what we want and surely we can reuse the system? Not from experience reports from friends on projects that are doing just this. It quickly becomes confusing what the job of the system is and whether a particular feature is a "development toggle" or a "product toggle".

Do not do this.

Build a cool system like Facebook's

Facebook has a system that helps them develop features and roll them out independent of the codebase. This is a really cool idea (that I unfortunately have no more insight than others about). Maybe your team should be doing this?

Maybe. Consider the engineering effort to build and maintain the system. Is it worth it?

What is the best practice to keep a system production-ready while still being able to continuous deliver? The answer is: it depends. You should probably be using a mixture of the seven patterns described above. There could even be other simple ways to isolate your code. The goal is to isolate your code from the users as simply as possible.

Testing CoffeeScript for Rails with Jasmine

2012-02-18T00:00:00-06:00

Before, I lamented about the testing story for Coffeescript in Rails. This is a description of a process that I am currently using. It is not perfect (and I'll end on why I do not think so) but it checks about 75% of the boxes on my Want list.

Quick Start

First, I just want to give a quick start guide for anyone that wants to just get shit done. I will dive into some details after this section--so if you're interested in how it works or are debugging something, that part will be of more interest.

First, we're going to create a Rails application.

rails new testing_coffeescript

It will generate output like so

      create  
      create  README.rdoc
      create  Rakefile
...
Using sqlite3 (1.3.5) 
Using uglifier (1.2.3)

Next, you'll need a couple more gems. Edit your Gemfile to include jasmine and jasminerice. Run bundle install.

group :development do
   gem 'jasmine'
   gem 'jasminerice'
end

The jasmine gem will come with some generators. I actually prefer to NOT use them and 'roll my own' scaffold because its easier to get to work with jasminerice.

mkdir -p spec/javascripts
touch spec/javascripts/spec.js
touch spec/javascripts/spec.css

In both spec.js and spec.css, you will need to use asset pipeline requires for application. You will also require the spec/javascripts directory.

//= require application
//= require_tree .

/*
 *= require application
 */

And now you should be ready to run specs. Start your rails server (rails s) and hit http://localhost:3000/jasmine.

If everything works, you should see "0 specs, 0 failures in 0.001s" as a green bar. Write a sanity spec by doing the following in spec/javascripts/sanity_spec.coffee

describe "sanity", ->
    it "1 should == 1", ->
        expect(1).toBe(1)

Save the file, run the tests by refreshing the browser and now you should see "1 spec, 0 failures". Finally, there is one last step to testing your coffeescript. Lets assume we had this class:

class Seven
    add: (anotherNumber) ->
        7 + anotherNumber

and a test

it "should add 7 to be 14", ->
    expect(new Seven().add(7)).toBe(14)

you may scratch your head at why you will get an error saying ReferenceError: Seven is not defined. Essentially, this is because Seven is not in scope. You have to add a line to your code to make it available for testing:

class Seven
    add: (anotherNumber) ->
        7 + anotherNumber

window.Seven = Seven

And now you're all set to write tests for your coffeescript!

How it Works

Most of the work is being done by the Rails asset pipeline and the jasminerice middleware. The Rails asset pipeline can give us compiled coffeescript on-demand and we do not have to compile it manually. It also does all the includes heavy lifting for us (yay! No having to manage test lists).

The jasminerice middleware uses the Rack API that Rails sits on top of in order to intercept any requests that go to /jasmine and request all the necessary bits and pieces. We actually aren't hitting Rails at all when we hit /jasmine.

/jasmine will request spec/javascripts/spec.js and spec/javascripts/spec.css as the roots for all of the javascript and css that we will use. In spec.js, we require application and the spec library. It may be necessary to include the application before the tests. Jasminerice also helpfully includes jasmine for us and supplies the necessary HTML template. If you define a new root javascript identifier, you will need to also include that in spec.js so you can get the additional coffeescripts.

Continuous Integration

We run these tests in our CI environment using jasminerice, as well. We use phantomjs to run them headlessly. Phantomjs does need a runner file which we've lifted from the guard-jasmine gem (On GitHub). We spin up a rails server (we picked port 5555) and then run the tests against it.

Issues

My issues with this approach are mainly just reproaches against Jasmine itself. I do not like having to refresh a browser to run tests. Something about it just seems wrong. It makes my skin crawl. That being said, it does make me very happy that my process is this streamlined. Just the other day I was able to truly TDD a mostly-javascript widget and it was a joy. So, with that in mind, here are the issues:

Running tests by refreshing a browser is cludgy and does not integrate well in the development process.
We run tests differently in development than we do for CI. In theory they are the same but we have already run into issues where that was not the case.
The output for the phantomjs runner we are using is ugly. This can be readily fixed but just needs some time and elbow grease.
Running tests with a DOM hides issues with code that is tightly coupled to the DOM.

And that's pretty much it. It's quite a nice setup otherwise!

Feature Branching and Continuous Delivery Are Incompatible

2012-01-29T00:00:00-06:00

When starting up our most recent project, my current project team decided on a set of development practices that included Continuous Delivery and Feature Branching (through Git Flow). There were some members of the team (myself included) that were excited by a true continuous delivery process. Others of the team were similarly excited about using Git Flow.

The first obstacle we reached when we were setting up our continuous delivery pipeline was figuring out which branch to continuously deliver. We changed our minds a few times before settling on a pipeline that would run all the code from the development branch. Except production, that is, which would deploy from master.

This was always a bit confusing and didn't make a whole lot of sense. The idea with git flow is that master always represents production while develop represents the current state of development. The idea of continuous delivery, though, is to reduce the time between what master represents and what develop represents. In the ideal case, develop and master would converge. These worldviews clash quite spectacularly when tried to use in conjunction.

We were able to ignore this uncomfortableness, though, and use git flow for a few more weeks. We found that the overhead of git flow was quite a bit unnecessary. Our team is a well disciplined team that commits several times per day, pushes as often as possible, shares knowledge and code frequently and, despite being on a continous delivery pipeline and committing straight to master, only manages to fail around two builds a week.

Another interesting result was that despite being a reasonably disciplined team, we found that our feature branches would last longer than a couple of days. This was bad because it encouraged "siloing" and discouraged emergent design and refactoring for the whole team. When a pair was working on a story that dealt with the Y Feature area, everyone steered clear of that part of the code until the pair was done and everything was checked in.

In the spirit of Kaizen, the team saw this as a problem and started to improve the process. We decided that feature branches should not last longer than a day and that we should actively pursue getting our branches in as quickly as possible. We pushed this for another week or two before stepping back and looking at the big picture:

Git flow introduced overhead (and a dirty history log) into our development process. For a team of our size and cohesion, it was definitely on the heavy side of process.
Our pipeline was somewhat compromised because our promotion scheme from staging to production was to just push a fast forward to master and deploy production. Our pipeline did not directly feed production.
Git flow encouraged long lived feature branches. With just a bit of attention, we were able to successfully make those long lived feature branches go away. At that point, though, we were creating only local branches and merging them directly to develop.
We were constantly confused about what was in production versus staging because they deployed from two different branches.

With those in mind, we realized that we needed a "develop on master" strategy. Since moving to that, the team seems to have been quite happy with the approach. The only hiccup is sharing code before it is "deployment ready" -- we have to push a new remote branch to do that. This seems to come up probably once a week and, thus, we feel okay with this solution for now.

If you are interested in learning more about continuous delivery, go get David Farley and Jez Humble's Continuous Delivery. This experience report is just an affirmation of the things that they talk about in the book and the book is a immensely important to anyone trying to implement continuous delivery at their organizations.

Getting Started With CSpec-Rails

2012-01-22T00:00:00-06:00

Yesterday I wrote a blog post about why Rails needs a better story for CoffeeScript development. Today, I'm going to describe a solution (not THE solution) for making it much nicer. We will be using CSpec and CSpec-Rails.

CSpec

CSpec is a RSpec-inspired library for CoffeeScript. The nice part about CSpec is that you are writing your tests in CoffeeScript for your CoffeeScript files. There is no need to compile either the specs or the code down to JavaScript.

CSpec runs on top of NodeJS. I realize that this is a large dependency for CSpec and is one of the things I would like to see if we can get removed. However, as of now, we're using NodeJS.

CSpec-Rails

CSpec-Rails is a simple gem that gives some Rails utilities for interating with CSpec. Currently, it only gives you a rake task that will run all of your Rails CoffeeScript specs for you. I have a few more things planned for this in the future but not much.

Getting Started

The first thing that you'll need is all the dependencies. I'm going to assume that you already have a Rails application so I will not be describing that. Next, we need to install NodeJS and NPM (Node Package Manager).

Install NodeJS from http://nodejs.org
Install NPM from http://npmjs.org

And now we've gotten our dependencies. To install CSpec, we just need to run npm install cspec.

Finally, just add gem 'cspec-rails' to your Gemfile, run bundle install and you are ready to start rocking some tests!

Simple Spec

In CSpec, we can assume that we have all of our coffeescripts compiled and ready to use. jQuery is also turned on by default. The following is a simple example of a spec:

describe "User", ->
    spec ->
        user = new User("Bob", "Saget")
        user.firstName.should be "Bob"
    spec ->
        user = new User("Bob", "Saget")
        user.lastName.should be "Saget"
    spec ->
        user = new User("Bob", "Saget")
        user.getFullName().should be "Bob Saget"

We can run these specs by running rake cspec. It would then output:

Bob should equal Bob
Saget should equal Saget
Bob Saget should equal Bob Saget

This is all pretty preliminary and incomplete but it is a very encouraging start. There is a ton of work to do to make CSpec more usable and to get CSpec-Rails to do a little more heavy lifting for us.

Rails, Coffeescript and Testing

2012-01-21T00:00:00-06:00

In Rails 3.1, CoffeeScript was introduced as the default language for client side interactions (and with vociferous discussion from all sides). I was very excited by the choice made by the Rails team and have been gnashing at the teeth to get my hands on a project with Rails and CoffeeScript working in gorgeous harmony. What I found was that for production code Rails and CoffeeScript work together swimmingly, but the development story is, at best, incomplete.

Writing a feature in CoffeeScript

A thought experiment: let's walk through the development of a feature. Let us try GitHub's "Command-T" functionality to search for files. The feature calls for the following functionality:

When a user presses 't' in her repository view, show the repository search view.
In the repository search view, the user is allowed to search for files by just typing in any part of the desired filename.
After a short delay, the search is executed and the results are displayed.
The user can then use the up and down arrow keys to browse the results.
When the user hits enter/return, she is then taken to the selected result's page.

I am leaving out many parts of this feature, but this high level overview is good enough to get use thinking about the complexity involved. From a technical perspective, we know several things: we need to respond to user events (keyboard), we need to asynchronously fetch and display results, we need to know enough about the results to act upon them and that we are pretty much stuck with client side JavaScript/CoffeeScript if we want this to be a good user experience. This is a complex peice of code that cannot be easily written. There is going to need to be spikes, research and, hopefully, tests that surround it.

How might we do this in Rails? Well, the first requirement is easy enough; it is a one liner in CoffeeScript. We could even add a Cucumber on Selenium test that verifies the hiding and showing of the correct panels.

The next part starts to get complicated, though. We need to be able to capture all of the user's keyboard interactions, decide if it is alphanumeric, save the key somewhere and then display it. Depending on how naive we are implementing this, we are talking about a significant function with many branches. We could probably get through the pain of writing Cucumber on Selenium tests for this particular part but we can already see that this is not going to be sustainable. If just one part of this breaks, then our Selenium tests are only going to be able to tell us as much as a user could: "It's broken. I don't know why but it just breaks!"

Unit Testing

The obvious part that I left out is Unit Testing. As far as CoffeeScript goes, Rails is test framework free. It has no opinions on how you should be writing these tests and no guiding light. We have to turn to external tools like Jasmine or QUnit. Both of these tools work by running tests in a browser. That is, you need to open up a browser in order to run your Javascript tests. In my opinion, that is insane. These tools barely provide an isolated environment and, by themselves, do not work in "headless" mode. In order to get them to work in CI, you actually need to fire up Selenium, open the browser to the test run page and check for a span tag with success class.

We have to dig a little deeper into the rabbit hole to get Headless Javascript testing: we need to use tools like PhantomJS or jasmine-headless-webkit. These tools are light years ahead of the Selenium method of automation but are still a bit distasteful. We are still running tests by loading an HTML page even though it is in a "headless" browser. We are even still checking for a span tag with success class.

Even with these tools, though, the development experience is bad with CoffeeScript. Most people will actually compile the coffeescript down to Javascript before running it in Jasmine. You still need to open up/reload a page in order to run tests. The experience of running tests locally and running them in CI is completely different. There is no CLI experience that matches your "normal" development experience. The tests will often run slow because of browser launching, html rendering, etc. The tests will often not be "units" because developers will write tests that interact with the DOM. When adding a test, you have to add the test to the script include blocks in your HTML file because the browser cannot access the filesystem to look for whole folders.

A Better Solution

For unit testing, I propose that we focus away from browser-based testing frameworks and focus on native Javascript interpreters that are not based in a web browser. As a proof of concept, I have gotten a spec library that I wrote for CoffeeScript on top of Nodejs to work with a Rails application. Here is a ridiculously simple example:

# app/assets/javascripts/user.coffee
root = (exports ? this)

class root.User
  setName: (value) ->
    @name = value

# spec/user_spec.coffee

describe "User", ->
    spec ->
        user = new User()
        user.setName "Bob"
        user.name.should equal "Bob"

Again, this is ridiculously simple and what I've done is a proof of concept. I plan to do some more work on this specific solution but I'm hoping that this is just an inspiration for more work in making the CoffeeScript inside of Rails story much, much better.

Not the Best Solution

My solution above has several flaws that I'd like to point out (and if someone has a resolution for it, please let me know!).

CSpec is very, very incomplete. This could rapidly change but please be aware I'm not necessarily advocating this for real use today.
CSpec requires NodeJS to be installed. This is yet another dependency that a developer must install in order to get up and running. Looking at other solutions like therubyracer is something that I would like to do.
CSpec requires that you "publish" your classes via the exports module, if it exists. It's a small but hacky fix to get at classes defined in other files.
CSpec does not have a browser. That means that JQuery is not supported. You'll have to decouple your logic from JQuery (a good thing) but you will still need to do JQuery testing through Selenium and/or Jasmine.
CSpec is not well integrated with Rails. This is something that could be quickly resolved with the appropriate generators and other railties in a cspec-rails gem.

Again, while I do hope to improve upon my solution what I really hope is that the Rails community can improve upon the development flow of CoffeeScript. I think we can and should encourage a close relationship between Rails and CoffeeScript and that we should have the same high standards for both.

Discuss this post at Hacker News

The Platform Race

2011-09-11T00:00:00-05:00

Grab a beer. I did not intend for this to be as long as it is, but I geeked out a little. Hopefully you'll find this as entertaining as I did.

There is a coffeescript implementation that runs on top of the Rubinius virtual machine. Without getting too into why that tingles my geeky insides, that's pretty cool. It started to get me thinking about the future of Rubinius, languages and the forthcoming platform race.

Virtual Machines

All code manages to get compiled down to machine instructions. These machine instructions are based on the architecture of your processor but look a little something like this:

push ax
mov ax,4
iadd ax,5
mov out,ax
pop ax

Which is a simple x86 program that adds 4 and 5 and stores the result in a variable called out. This code is pretty much activating the processor at a gate level and can become very tedious to write. In order to become more effective programmers, we created compilers that would interpret more human-readable code. We created several languages but the most popular is vanilla C.

Just like modern languages, C was created because programmers were tired of writing things they didn't care about. In the C world, we didn't need to worry (much) about stacks and registers. When we wanted to do the above code, we wrote the human-readable version:

int out = 4 + 5;

More sophisticated features like functions, control structures and data structures allowed us to reuse code and deliver software faster.

Enter Java. In the mid-nineties the team at Sun began working on a new language. One of the language's goals was to be platform agnostic. They wanted to create a language that you could write once, run anywhere. The reason that this is not possible with C is due to architectural differences between processors. The machine code above works on x86 Intel processors but not necessarily AMD processors (or x64 Intel, for that matter).

Instead, they created an intermediate compilation process that would turn the human readable Java code into closer-to-machine-code Java byte code. It looks something like this:

iconst 4
iconst 5
iadd
iload_0

The byte code is fed to the JVM which will execute the bytecode regardless of processor architecture. This is the effect of putting a virtual machine as an intermediary. We now have a common platform that we can now compile one or many languages against. For a long time, Java was the only competitor in this space. It was initially slow but feature rich. Java adoption exploded with the web and now we are in a world where virtual machines are the new hot ticket.

Development of Virtual Machines

So, after a slew of backstory, we can now talk about what is happening today and tomorrow. Java, as a programming language, is a mess. It is overly pessimistic about the skills and knowledge of those using the language. It has far too much ceremony and is only bearable through the use of heavy IDEs like Eclipse, NetBeans and IntelliJ IDEA[1].

People have been innovating. They have taken advantage of the fact that the JVM can understand any Java byte code, no matter the source. Languages like Clojure, Scala and Groovy were created to make Java development more pleasing.

Some languages, like Ruby, were relatively infant in implementation of their VM. They leveraged the years of static code and garbage collection optimization that went into Java's VM. JRuby and JPython were introduced.

As this was happening, Microsoft began to see the benefits of language abstraction from processor architecture and the ability to leverage strong frameworks and libraries. They created the Common Language Runtime and published the Common Language Specification.

This allowed people (Microsoft, mostly) to develop new languages for the CLR. C#, VB.NET and F# all run on the CLR and are interoperable. Open source communities courageously sprung up to create IronPython and IronRuby though both seem to be sputtering.

Finally, the Mono team took on the monumental task of creating a competitor to the CLR and called it Mono. It fully realizes the VM's capability of running across multiple architectures and operating systems.

It started years ago but this race is starting to heat up. More attention and opinion will be throw around soon enough but here is my take on how things currently stand:

JVM

The JVM is clearly the 'front-runner' with several programming languages across multiple paradigms that are built on top of it. It has also been around for 15 years and has been a target VM for research and experimentation that inform today's and tomorrow's virtual machine implementations. It has the most sophisticated garbage collector in the group which ultimately affects the virtual machine's speed limit. Platforms cannot afford to hold on to dead memory for long and will be completely blown out of the water if they leak. Java also has years of research and momentum in terms of static code analysis. We cannot be completely sure, but its suite of optimizations are almost definitely best of breed.

Unfortunately, it is hamstrung because even though the JVM is an open standard there is threat of legal action to any competitive VM implementation (See: Dalvik and Android). Oracle's VMs are it and the Oracle JVM team is hemorrhaging smart and talented people. There will legal battles between Oracle, Google and who-knows-what-else in the years to come about the virtual machine. This could scare people away. If the legal battle is solved quickly, though, we may see a resurgence in the JVM from the Google and Oracle powerhouses.

CLR (.NET)

The CLR/Mono/CLS is another proven platform with the full force of Microsoft behind it. It has several languages delivered by the brilliant Microsoft Research guys and gals. It has had a dabble of ground roots support and development of languages such as IronPython and IronRuby. It is a long term investment for Microsoft and seems to be doing really well.

However, it sits behind Microsoft's bureaucracy and business goals. They have already effectively cut support for IronPython and IronRuby. While there are no definite answers, I can imagine that the reason behind those cuts are because it made no business sense to support competitor languages for the .NET platform when Microsoft owns the development stack for C#, F# and the like.

Really, they need to find a way to spark the OSS community. They are making a good start by letting enough alone and not pulling an Oracle on Mono. The Mono team has really stepped up in the year that it has been since I've seriously used it. They now support all core language features of .NET 4.0. There are some gaps in frameworks (and looking at that list, nothing I would list as critical) but Mono may be another avenue to prop up the .NET platform (somebody seriously needs to change the name, though).

Javascript

Javascript is really coming of age as a first class language. It is going through some growing pains. We're finding that Javascript is powerful, gives you lots of sharp language tools but can become a huge mess in the wrong hands. Tooling is rather weak. Javascript, though, could be the most used language in the world. Everyone has some sort of interaction with Javascript. Few, though, have interaction with Javascript as a first class language.

We currently have Javascript champions and wizards like Kris Kowal, Ryan Dahl, Douglass Crawford and John Resig that are trying to advance the language with specifications, documentation, tools and libraries. The Javascript interpreter is becoming blazing fast because of the Second Browser War. The next step is to abstract away from Javascript through either libraries (jQuery, ExtJS, SproutCore) or develop new languages on top of it (CoffeeScript, Objective-J). The community in the Javascript space is picking up momentum with early adopters and the cool kids. We will have to wait and see if the web as a complete platform can scale past initial enthusiasm.

Rubinius

Rubinius is the new kid on the block. It is a virtual machine that primary interprets Ruby code. It does, however, have an IR language that other languages can compile against. It is very new and is still attempting complete conformance to the Ruby specification. What sets Rubinius apart is its goals and development philosophy.

Rubinius was created as a platform for discovery and collaboration. The creators of Rubinius want to enable programmers to try crazy language ideas, visualize great optimizations and provide reliable implementations for their virtual machines. The JVM is locked up in the Google and Oracle castles, touched by the blessed few. The CLR is behind Microsoft's business wall. Javascript interpreters are in the hands of the browser teams. Language geeks have very few options.

Not only is the development model for Rubinius very open in process, it is very easy in practice. As much code as possible has been pushed to Ruby. Only a very small (and diminishing) core of C is at the heart of Rubinius. The idea is that Ruby is far more approachable and abstract as a language. Programmers can quickly try new ideas, visualize problems and solutions and leverage the great suite of tools surrounding Ruby (like unit testing! Silly C.).

This opens the door for experimentation. Coffeescript is now a compiled language on Rubinius. It has access to all ruby gems. Lua scripts can also be compiled and use coffeescript libraries. Sometimes things make you realize we live in a wonderful world.

Also Rans

There are a couple of VMs that I want to mention.

Parrot is another virtual machine. It has some interesting research projects that are constantly being tested against it but it has not yet enthused anyone outside of academia. As of their wiki, there are currently no "Production" quality languages that are in wide use. The Perl6 implementation is likely the most complete.

LLVM stands for Low-Level Virtual Machine. The LLVM is used by Apple in Objective-C compilation and also has support for languages like Ruby. However, outside of the Mac community, the momentum seems to have slowed. Being primarily developed by Apple, it would run into the same doubts as a Microsoft VM from a business perspective. It is also a bit of a misnomer because it is more compiler than virtual machine.

[1] This is my personal feeling about the language, of course. Many others share these thoughts. Many do not. Nevertheless, this is a large part why new languages sprang up on the JVM.

TFS is destroying your development capacity

2011-09-11T00:00:00-05:00

Team Foundation Server (TFS) from Microsoft is an application lifecycle management (ALM) solution that is deployed across many development teams and organizations. After working with TFS for 3+ years, I have come to the conclusion that organizations are better off without it and should look to adopting best of breed tools (of which, I claim, TFS is not part).

This post addresses three topics: the technical flaws of TFS and its subsystems, the culture TFS encourages and strategies on how to get out.

TFS Sucks

For developers, tooling can inspire almost religious dogmatism and fanaticism. This is understandable. We work with these tools for a significant portion of our lives. Choosing the right tool for the job can decrease stress, increase productivity and make legitimate impacts on us.

Dogmatism can cut both ways and TFS is one of those topics that seems to inspire factionization. There are many people that dislike TFS[1][2][3][4]. There are many people that like TFS[5][6][7]. We'll get to the why it is so divisive in the second section, but I'm going to try to explain why TFS is not a best of breed tool. I'll start by breaking down the subsystems of TFS. However, any review of TFS would be a disservice if it did not talk about the vertical integration that TFS provides and I'll end there.

TFS Version Control

In a nutshell, this could be the worst part of TFS 2010. It is certainly the most impactful subsystem to the developer. Combine a below-par version control system with the amount of time that a developer is working with it and you have a very frustrated developer (that posts their frustations on amplicate.com!).

Frustrations with the version control system are well documented[8]. In order to be comprehensive, though, I'll give you the digest.

TFVC is a centralized-server model that requires constant and active communication between a client (read: developer) machine and the server. If, for example, the network goes down for 1 hour, development grinds to a halt. The reason here is that TFS will mark all files as read-only on the filesystem until you have asked the server to check them out for you. If you cannot communicate to the server, you cannot checkout the files and you cannot edit them [9].

TFVC wants you to do everything inside of Visual Studio [10]. When you add a file, it is best to add the file through Visual Studio's contextual menus in your solution explorer. TFS will recognize that you are adding a file and track that file for you. This is great until you need to start adding some files outside of Visual Studio. In my experience, this is usually peripheral code like batch automation scripts, build tools, XML configuration files, etc. Sometimes you use Notepad++ because its more convenient--but then you get yourself into trouble because you try to check in but TFS does not recognize the new file or that the file is edited. In that case, you need to navigate to the Source Control Explorer, add or check out the file using its contextual menus.

TFVC uses a merging solution that is heavy handed and ineffective. It can, sometimes, do automerging for a file that was edited twice. However, it prompts the developer for each file and makes them click an "Automerge" button. The default merge tool doesn't support a rather standard 4 window, 3 way merge view [11].

There are many other documented problems and errors with TFS. Some of those are very scary (e.g. losing check ins) but I have not personally run into them. Many of those are small but annoying things that should be fixed but does not seem likely to change in the near future. There can be, and has been, whole posts dedicated to just the issues surrounding the version control system.

TFS as Bug Tracking

TFS, as an ALM, has to handle bugs on software that has been released. Using a bug work item, someone will log a bug into the system. She will be required to fill in some details like Title, Description, "Classification", Assignee, etc.

In order to create this bug work item, you need to use either Team Explorer (through Visual Studio, most likely) or the TFS 2010 Web Interface. The obvious problem for the first scenario is that developers are the only ones likely to have Visual Studio and are only one of several groups of people that could log a bug.

The second option, TFS Web Access, is really a second class citizen. It is slow, painful and cluttered. Unless you work with it every day, you could easily become overwhelmed by the options when you just need to log a bug!

Who logs bugs? For the most part, bugs are discovered by customers and then distilled into actionable items by customer service representatives. Instead of creating bugs through an email, or through a simple web interface, the CSRs are asked to either open up Visual Studio (if they have it; a development tool!) or to use Web Access. On top of that, they are asked to fill in information that they do not know or care about. Both of these processes discourage bugs from being logged due to the pain that the CSRs experience.

TFS as Agile Project Management

Unfortunately, I cannot speak to TFS from the perspective of a waterfall or clean-room type of project management. I just do not have any experience in those areas (though, some of my "Agile" projects have definitely had some Waterfall-ish aspects to them!). I can speak, though, to its use in Agile.

From my experience, TFS is ignored by Agile teams. Instead, I have used a combination of Story Wall, Pivotal Tracker, Jira and Excel. Yes, Excel. In fact, my current project manager uses Excel instead of TFS. I do not know how she deals with the pain of Excel on a day to day basis but she finds it a better experience than to use TFS! She would much rather use a tool like Jira or Mingle [12] to manage the project.

This largely stems from the inflexibility of TFS as a project management tool. Agile TFS work items, by default, have only five states [13]. Agile is about self-organizing, constantly improving teams that will change in order to make themselves better. This includes process changes as a project goes on. Even if the team went to the trouble of creating custom templates, those templates are going to have to change. TFS makes changing painful and, thus, discourages instead of enables Agile teams.

Project reporting in TFS is rather primitive. It operates on high level, structured data like Work Items. One of the more sophisticated views is the "Build Quality Indicators Report". This report is a decent start to a build quality metrics report but to get additional information you're going to have to branch out and create a custom build template.

TFS as Build System

Probably one of the under-utilized features of TFS is the build system. TFS will build every checkin akin to other build systems like Jenkins or TeamCity.

You can switch this on and TFS will run your software through a default template associated with your build type. In order to do anything different that the default template (and, believe me, you will), you'll have to go through a 15 step process to create a custom build type [14].

Once you have your custom build, it will now be time to create new tasks. This means that in your new build type (which is actually a C# project) you'll need to do some PrivateAssembly referencing, some obscure subclassing and some drag and drop workflow editing (in a very slow and cumbersome drap and drop interface, I might add). All of this will need to be done against a rather odd API [15].

I once had to write a custom build pipeline for a product I was developing. The functional tests were using a Java-based test runner (proprietary) that needed to be executed on every build. We wanted a HTML dashboard of the metrics we were running and the build times. I spent a few days getting this to work. It was extremely brittle and development was a pain. In order to test the system, I would have to write code and check it in for the build to attempt to run it. There really was no local development process for build development.

On top of that, continuous deployment (which is all the rage) requires a strong pipeline that TFS just cannot provide. While TFS has agents that can execute many builds or parts of a build in parallel, I see no worthwhile way of developing a pipeline on top of TF Build.

TFS Integration

The elephant in the room is the vertical integration that TFS provides. It is, by far, the biggest thing that proponents will use as defense. It has been, in my experience, a big selling point for developers and organizations to use TFS.

TFS has the promise of being a drop in, all in one box solution to ALM within an organization. On top of that, it provides additional features such as a source control system and a build system that have some additional integration points into the ALM, as well.

The problem is that the TFS ALM is only compatible with one source control system and one build system. And that the TFS Build System is only compatible with one source control system. TFS is actually a package of highly incompatible software that happens to work together but with nothing else.

This is a problem because it prevents organizations from adopting best of breed tools. For sake of argument, let us assume that the TFS ALM and TF Build subsystems are first class. The TFS source control, however, needed to be replaced. With most tools, you're given the flexibility to make that decision and switch to a better tool. With TFS, you're locked in because the ALM and the TF Build subsystems require that you use TFS and only TFS.

The Microsoft Way

As an organization, you're probably going to use some bad tools at one point or another. There will always be a new hotness that makes one of the tools you use look like dinosaurs. That's okay. The main point is to Get Things Done. As long as you can do that, you'll be sitting pretty.

Some day, you plan to move on to better tools than TFS and are even in the process of doing so now. You're evaluating other ALMs and have installed TeamCity and have a few builds going. Great stuff.

TFS may be hurting you more than just technically. TFS may be a symptom of a more endemic problem. TFS, as a tool, wholehearted embraces the culture of cradle-to-grave hand holding and dispassionate development. Good developers and organizations all have one thing in common: they are hungry. They thirst for knowledge and delight in trying new things. While at work, they are never satisfied with "okay" but always strive for "better".

TFS promises several things for your organization. It will handle source control for you, it will handle project management (even giving you strict templates that you have to follow!) and strictly enforces development policies (you must associate a work item with your check in). TFS will be an install once, forget about it forever solution. It is easy and you do not have to think about it.

The problem with that promise is that these are the sorts of things that an organization should be questioning. Project Management is more of an art than a science and good project management evolves along with the project. Your source control set up should enable multiple teams to work simultaneously and less painfully (try working on TFS from India via VPN). Your bug tracking system should be so dead simple that CSRs are creating bugs for every tiny UI hiccup that customers experience. Your deployment pipeline should take under an hour from developer check in to production-like environment.

Instead, the teams that work with TFS will use the default templates, default build pipeline and ignore the pain of a centralized source control model. I have also seen the lack of good configuration management in an organization because someone was able to one-click install TFS on a server and then the organization can trick itself into thinking that configuration management is a solved problem. The teams gladly ignore the "not programming" part of delivering software and put their heads in the sand.

This is not only TFS's fault, though. There is a whole approach to software development that embraces the set-it-and-forget-it or cradle-to-grave software development setup. The approach discourages the use of products that are not given to development teams. Organizations with symptoms of this can be found using tools and frameworks like Entity Framework and DevExpress. Your code is likely very weakly tested. You may not have a clear path to production. Deployment can consist of manually copying DLLs or hand crafting an MSI. This is a bad state to be in and the only way to get yourself out of this hole is to start thinking wholistically about software development.

Moving Forward

In order to get yourself out of the TFS hole, you're going to have to endure a bit of pain. However, instead of doing one big switchover, there may be better ways of incrementally moving away from TFS.

The first thing that is helpful is to establish a team that can make decisions on configuration management across the organization. In ThoughtWorks, we call these kinds of roles "DevOps" because it often involves both the development part of the IT organization as well as the operations side. Appoint someone to be the leader for these kinds of decisions and as someone that can enforce a decision for the organization as a whole.

Then, set up a Jenkins [16] server and start piloting a new continuous integration approach. Jenkins has a TFS plugin that seems to work fairly well. It is likely that, if using TFS only, you are not using the build system anyway and that a switch to Jenkins will be a painless exercise.

Use Jenkins to start thinking critically about how to deliver software. You'll find that it is a non-trivial task. You'll need performance, UAT and staging environments. You'll need to be able to deploy your software automatically. You will need to understand how to get your software from the source code checked in to TFS to a deployed, usable application for your users.

After you've got the ball rolling on Jenkins, it is time to move away from TFS version control. I would highly recommend to switch to a distributed version control system like Mercurial or Git [17][18]. This is a non-trivial task and will be painful. However, you can mitigate that pain.

You will save yourself a ton of pain by using a Linux server for either Git or Mercurial. While both work on Windows, they both perform better on Linux. VMWare has appliances that you can drop in which are already set up. There is also a turn key linux distribution for these systems [19]. If you can swing it, though, the best DVCS hosting solutions are actually in the cloud [20][21].

Start by creating a pilot program that uses your new version control system and creating a small group of your developers that can be the evangelists within your organization. If you have people that are particularly passionate about the new source control system (they aren't hard to find!), use them as your experts. Train them on the new pilot program and allow them to discover the positives and negatives about the system.

After version control, TFS is now left as a bug tracker and project management tool. It isn't particularly good at either but chances are that your organization is pretty entrenched with these tools. Everyone knows how to use it and will resist any large scale changes.

I've little experience in this area and I imagine that this will be painful, as well. The best way to get your data out of TFS seems to be to export it to Excel. From there, you may have to re-enter all data into a new system. This is a slow and painful process, I imagine.

As far as alternatives to TFS, there are many. I've used Jira, Redmine and Mingle successfully [22][23][24][25]. I think this will depend largely on your organization's needs and that the transition here will need larger buy in from other departments. For example, it is likely that your CSRs are using TFS to enter in bugs.

Conclusion

Do not use TFS. It harms your organization and is extremely painful to leave. Vertical integration is another word for "lock in". Only through the hard work of some open source developers is there any lateral integration with TFS (e.g. Jenkins) and that is very likely to stay true.

One thing to note is that TFS plans to fix everything in their next release. Do not be tempted. TFS has historically been a bad system to work with and the Microsoft solution should not garner loyalty until it has proven itself to be a viable alternate system.

[1] http://www.nearinfinity.com/blogs/joe_ferner/why_i_dislike_tfs-team_found.html

[2] http://amplicate.com/hate/tfs

[3] http://goingagile.blogspot.com/2008/04/evaluating-source-control-systems.html

[4] http://blog.stevemoyer.net/2010/07/microsofts-total-failure-system-ms-tfs.html

[5] http://www.itwriting.com/blog/4744-real-world-microsoft-team-foundation-server-not-very-good-says-thoughtworks.html/comment-page-1#comment-728861

[6] http://stackoverflow.com/questions/62126/what-are-the-real-benefits-of-tfs-for-the-individual-developer/62275#62275

[7] http://teamsystemrocks.com/

[8] http://www.google.com/search?q=tfs+version+control+sucks

[9] There is a solution to temporarily remove TFS bindings, but it is an unintuitive and clunky process. It can also lose changes when re-establishing a connection to the server in some rather unpredictable ways.

[10] Some other IDEs are supported through Visual Studio Everywhere. This gets around some of the tool support problems. However, this still leaves things like copying files into the source tree very clunky.

[11] A 4-window, 3-way merge will show you 4 different files (or, rather, the same file in 4 different states). It will show you the "base" which is the common, shared file state. It will show you "mine" and "other". And, it will show you "output". There are tools that can be configured with TFS that will do this correctly.

[12] Full Disclosure: I work at ThoughtWorks. Mingle is a ThoughtWorks Studios product.

[13] http://msdn.microsoft.com/en-us/library/ff731576.aspx#ChangeState

[14] http://msdn.microsoft.com/en-us/library/dd380683(VS.100).aspx

[15] http://msdn.microsoft.com/en-US/library/aa337655(v=VS.80).aspx

[16] http://www.jenkins-ci.org

[17] http://mercurial.selenic.com/

[18] http://git-scm.com/

[19] http://www.turnkeylinux.org/revision-control

[20] http://www.github.com/

[21] https://www.bitbucket.org/

[22] http://www.atlassian.com/software/jira/

[23] http://www.redmine.org/

[24] http://www.thoughtworks-studios.com/mingle-agile-project-management

[25] Full Disclosure: Again, I work at ThoughtWorks. Mingle is a ThoughtWorks Studios product.

HamlJS with Coffeescript

2011-09-11T00:00:00-05:00

I am currently playing around with a NodeJS stack where I am writing Coffeescript for my application. To render my views, I am using Haml. HamlJS only used javascript. So, here is a quick and dirty plugin that lets you use coffeescript in those views! I apologize for the lack of tests and please let me know if there are any errors that you find.

NPM Registry

GitHub

Continuous Deployment Middle Ground

2011-03-20T00:00:00-05:00

The Hacker Chick (Abby Fichtner) wrote about continuous deployment (edit: almost 2 months ago.. my bad!) and posed the question about whether it makes sense. She presents 2 possible options: 2 week iterations and continuous deployment.

There is a middle ground[1] between the 2 week iterations and continuous deployment. The idea behind continuous deployment has two proposed benefits.

First, there is the technical benefit that consistently running through your whole deployment stack makes it less cumbersome and more reliable. Generally, continuous deployment encourages automation and 'hands off' deployments where very little human interaction is involved (usually just a push of a button on the server).

There is also the product management benefit. As Eric says in the lean startup, reducing the lead time from 'Go' to 'Customer Feedback' is a huge boon to product development. It helps startups (and large businesses trying to innovate) find the right product and/or market. This materializes as multiple deployments per day (though 50 seems pretty high to me).

The middle ground is to just focus on the technical benefit. IT should be able to do this without much involvement[2] with product management teams. Product management can then decide when to deploy new features (daily, weekly, monthly, erratically, etc.).

After that, you can reap the technical benefits. You can move to an iteration-less process (though you can do that without continuous deployment!). You can automate, automate, automate. You can stabilize your build and deployment pipeline. You could quickly deploy beta versions or A/B tests without disrupting your current production server. All in all, you can make better software.

[1] I love how 2 week iterations are the conservative point in this conversation! Amazing how certain ideas (continuous deployment) can really change the conversation.

[2] It can do something like blue-green deployment where they have a shadow production server that can be swapped into production at any point. This allows you to continuously deploy to a server but not expose that server to customers.

Root Route Testing in Rails

2011-02-19T00:00:00-06:00

This one took a little bit for me to figure out and I thought that I'd share the solution.

  it "should route root to home page" do
    opts = { :controller => 'home', :action => 'index'}
    assert_recognizes opts, '/'
  end

If you read the testing Rails guide you'll notice that they make a brief mention of the assert_routing. That code takes the hash key and composes a route based on the parameters. This doesn't necessarily work for the root, though. In order to do that, you need to compose the route hash from a relative URL. That is what assert_recognizes does for us.

The Last Mile

2011-02-18T00:00:00-06:00

The last mile of a marathon is always the most difficult. You've run 25 miles and you just need one more. You can see, smell the finish line. It's right there in front of you. It is the most difficult (and dangerous!) part of a marathon.

In software development, the last mile problem is a big one. You've decided on a product, designed the user interface, architected a great technical solution, executed on delivering business value and passed a testing suite that ensures quality. Now you just have to release. Releasing is the most difficult and dangerous part of software development.

We can mitigate this risk. Practice makes perfect and we need practice at delivery! Big software products do not release that often. If you're doing good, you may get a product out every six months. Twice a year is all your team gets for practice.

Instead of releasing twice a year, we can release far more frequently. Monthly, weekly or even daily releases cut the risks associated with the last mile. The most extreme of this risk mitigation is continuous delivery.

Releasing takes time. It is a non-trivial task that cannot be done in a couple of steps. In order to release regularly, we need to turn that manual delivery process into an automated pipeline. The goal is to have "button push" or continuous delivery so that delivery is so easy that it is mindless. When you reach that level of complexity, risks involved with delivery drop massively.

For most companies, this is not an easy process. This needs to be a conscious choice to reach a better state over a long period of time. You'll need someone that is assigned tasks associated with this goal that can spend significant chunks of time on delivery automation. You'll also need the rest of the IT organization to buy into process. Cultural change also takes time.

Release does not have to mean production, though. In fact, I am currently working for an organization that is attempting to shift to automated delivery and monthly releases to production. However, we can now deploy daily (and soon will deploy automatically). That means for a particular monthly production release, there may be 30 or more deployments. The pipeline currently looks like:

Dev -> CI -> Automated QA   --/-> Staging
        \--> Exploratory QA -/

Staging is a production environment that is not accessible. We utilize blue/green deployments so that, while we still deploy every day, we don't necessarily have to push to production. When we are ready for a production release, we just need to point the incoming connections to Staging (which becomes Production) and then start deploying to the old Production (which becomes Staging).

For truly green field organizations (mostly just start ups), I would recommend that you release every day. Whether that means manually calling "heroku push" at the end of each day or a structured deployment pipeline, it does not really matter. Getting a culture of releasing every day is important for risk management and other great things, too. Release early, release often and remove the pain with automated deployments!

On Tests and Fakes

2011-01-07T00:00:00-06:00

Testing software is one of the primary roles of software developers. In the developer's day to day, tests can use up as much time as any other activity (including writing features). Yet, so little seems to be known about the properties of tests. What qualifies as a good test? What classifications are there? What does tool X accomplish in testing? These are big questions that I may take a stab at later. This post is going to focus on one area of testing: unit tests and fake objects.

Unit Tests

Unit tests, from the perspective of mocking, fall into two categories: stateful and behavioral. Sometimes a unit test will be both (though we should avoid this) but it cannot be neither. Knowing the distinction between these two will help you with your mocks.

Stateful unit tests verify the state of the target object after internal modification. An example:

var receipt = new Receipt();
receipt.addLineItem(new LineItem('Watermelon', 5.00));
receipt.addLineItem(new LineItem('Mango', 2.00));
Assert.areEqual(7.00, receipt.subtotal());

Here we can see that the end result of subtotal (7.00) is what is being tested. We aren't verifying the contract between LineItem and Receipt.

Behavioral unit tests verify that the behavior of the object is as expected. An example:

var receipt = new Receipt();
var watermelonLineItem = new MockLineItem();
receipt.addLineItem(watermelonLineItem);

watermelonLineItem.expect('price').called(1).return(5);

receipt.subtotal();

watermelonLineItem.verify();

Here we can see that watermelonLineItem's contract is being tested. We are not verifying the state of the object after execution.

Mocking Techniques

There are a variety of mocking techniques. However, these can be relatively cleanly divided by the stateful/behavioral classification of unit tests. For stateful unit tests, stubs or fakes can be used. For behavioral unit tests, mocks or spies are used.

Stub and fakes are test stand-ins. They are not the primary engines of the test and only serve a supporting role. An example:

var lineItem = stub();
stub.when("price").return(15);
var receipt = new Receipt();
receipt.addLineItem({"price":function(){ return 5; }}); // a fake object
receipt.addLineItem(stub); 
Assert.areEqual(20, receipt.subtotal());

Mocks and spies are the primary engines of the test. They are verifying that the contracts between the objects are being met. An example:

var receipt = new Receipt();
var watermelonLineItem = new MockLineItem();
var peachLineItem = spy(new LineItem("peach", 5));
receipt.addLineItem(watermelonLineItem);
receipt.addLineItem(peachLineItem);

watermelonLineItem.expect('price').called(1).return(5); // a mock
peachLineItem.expect('price').called(1); // a spy (delegates to base object)

receipt.subtotal();

watermelonLineItem.verify();
peachLineItem.verify();

This is not completely black and white. Sometimes you want a stand in (read: stub) in your behavioral tests in order to get around some hard dependencies quickly.

Which to use?

Behavioral or stateful?

I think that both are valid and have their places in a testing suite. However, we should be disciplined about test names and do not mix behavioral and stateful methods into a single test. Example test names for the final two tests:

shouldReturn20WhenGivenLineItemsOfValue15And5
shouldCallPriceOnLineItemsWhenCalculatingSubtotal

About

2010-12-19T00:00:00-06:00

Updated the about page. It is rather lengthy. I also updated the look of the site. I actually reverted back to the Google Font API because I like the look better. Impact isn't good for readability.

On Business

2010-11-03T00:00:00-05:00

We find ourselves with companies that make it big and then come tumbling down. This is part of the capitalist cycle. Tragedies such as Enron and the Lehman Brothers do not have to be. The solution starts inside corporate culture. Business, by and large, has been locked in by the mindsets of business schools. Technology companies are no different.

ThoughtWorks has led the way in the past 10 years to get technology groups to adopt agile practices inside of their development work. While there are still holdouts, there are companies around the world pair programming and Kanbaning their way into successful products. There is still work to be done on that front but I believe we can hold a party on an aircraft tanker at this point.

What happens now?

There has been talk over beer about the future. What does a post-agile world look like? Agile has problems but the framework is really en enabler for development teams to experiment and adopt new practices continuously. In theory, we should be able to be hands off. So where do we innovate? The future cannot be to just bring the last bastions of waterfall development under our wing. That isn't sufficiently ambitious for myself or the people with whom I have shared this conversation.

We want to change how businesses work. In 10 years, we want Harvard Business School to teach the New Way of business, even if it's grudgingly. Right now, we don't have all of the answers. Right now, we have identified some of the problems. Right now, we have very few solutions.

We think that we can apply the values (but not necessarily the policies or practices) of Agile to the business world. Values such as communication and trust are almost certainly a component to the solution. We are still working this through. We are still experimenting. We are still evolving our understanding of this problem and solution.

Right now, this is a goal of a small group of people inside of ThoughtWorks. I am one of them and I have been part of several conversations. It's something that some of us feel very strongly about. We want to create a new identity and legacy for ThoughtWorks in the coming decade.

Crazy? Yeah. Crazy enough that it just might work.

OJMoq 2

2010-08-08T00:00:00-05:00

A new version of OJMoq has landed in OJTest. OJMoq has changed significantly under the hood and there are some API changes. OJMoq 2 is, however, backwards compatible and you'll be able to use your current OJMoq tests inside of OJMoq 2.

Quick History

OJMoq was started as a simple class that did one thing: verify that a selector was called on an object. It was used as a little utility that was meant to get one of my tests passing in a particular Cappuccino project. It worked out pretty well and OJMoq grew as I and others need more utility out of it. Eventually, OJMoq became a huge "God" object that was really three different things: a mock, a stub and a spy.

OJMoq started to have weird edge cases that would creep in as bugs because we did not account for that particular behavior. Then, the entangled state management (whether it was a spy or a mock or a stub) caused unforeseen side effects far away from any potential change. It was time to deal with this tangled mess. The plan was to separate the three 'modes' into three classes.

It was actually surprisingly easy. Other than the three states of the OJMoq object, we have a relatively good design for OJMoq. Much of the difficult work is done in OJMoqAssert and OJMoqSelector. Those two classes haven't changed. Separating the classes was then just a matter of test driving how we wanted them to work and picking logic out of the existing OJMoq. Only OJMoqSpy has any logic that can be considered "new."

API Changes

The best way to continue forward was to break the unified OJMoq interface into three distinct classes. Let's briefly explore the new API.

    var aMock = mock(aBaseObject);
    [aMock selector:times:]
    [aMock selector:times:arguments:]
    [aMock selector:returns:]
    [aMock selector:returns:arguments:]
    [aMock selector:callback:]
    [aMock selector:callback:arguments:]
    [aMock verifyThatAllExpectationsHaveBeenMet]

For the OJMoqMock, not much has changed from the first API. We still have the same selectors and the moq(baseObject) function became mock(baseObject). The difference, however, is in the behavior. Mocks do not accept selectors that are not owned by either itself or the base object. Mocks return null for all calls by default. None of the calls are ever made on the base object. Mock, as it turns out, is just a more strict version of Moq. Next, we have the stub.

    var aStub = stub();
    [stub canPassAnything];

A stub is a very simple concept. It is a small, benign object that has absolutely no side effects. It will eat any call that is made to it. It's very flexible. There is no API to set return values or expectations. Very simply, stubs do nothing. Finally, we have the spy.

    var baseObject = something;
    var aSpy = spy(baseObject);
    [aSpy selector:times:]
    [aSpy selector:times:arguments:]
    
    [baseObject someSpiedSelector]
    
    [aSpy verifyThatAllExpectationsHaveBeenMet];

The spy is the most changed from the old API. A spy in the new OJMoq will watch for calls on the baseObject and then capture those calls. A spy is useful for when you want the side effects to happen but still want to verify some behavior by the caller. This was certainly possible in the old API but it was much more confusing and difficult to truly grok. The new separation between spy and base object allow for us to conceptually think of the spy as an unseen watcher.

Opinionated Code

You've probably noticed that I've actually reduced features! Before, you were able to use a spy as a mock, a mock as a stub and a stub as a mock. We were able to go between them rather fluidly. However, that makes it more difficult for someone to read and understand. Instead, we focus the different classes of objects on what they do best.

I think that this is the best method going forward. The old API still exists and you are welcome to use that (it's not even deprecated.. not until I'm sure that people like the new API). However, if it all goes according to plan, developers can create a testing vocabulary that is reinforced by OJMoq that will enhance communication and understanding.

The new version of OJMoq is certainly opinionated and some may disagree with that opinion. That's both fine and great. I encourage anyone that this pisses off to make a different mocking framework for Objective-J. We need all of the frameworks we can get!

OJSpec Matchers

2010-07-25T00:00:00-05:00

After some teasing, tugging and pulling I finally got my head around what OJSpec was up to. After understanding what was going on, I stripped it bare and integrated the essential parts of the framework into OJTest. OJSpec matchers are now available for use! I am actually really enjoying writing tests using it and I will probably use OJSpec matchers in my tests from now on (not something that I expected!).

OJSpec

Before we dive into how to use the new matchers in OJTest, we should talk a little history. OJSpec was around before I even started using Cappuccino and Antonio Cardozo was the creator. It was always a framework that I was interested in but never had the time to explore. When we started the OJTest repository, I asked Antonio if we could co-opt his work into the OJTest repository. He graciously agreed and we are appreciative of his gift!

Enable OJSpec

In order to enable OJSpec matchers, you will need to add a -s argument to your ojtest command:

ojtest Test/*.j

becomes

ojtest -s Test/*.j

This is, however, only necessary as long as OJSpec is an experimental feature. When it feels like OJSpec has been used enough to rely on it working, we will enable it by default.[1] Until then, though, you'll have to add the extra parameter.

The Matchers

There are currently 10 different matchers in the repository. These matchers are

    shouldEqual:
    shouldNotEqual:
    shouldBeNil:
    shouldNotBeNil:
    shouldBeSameAs:
    shouldNotBeSameAs:
    shouldBeInstanceOf:
    shouldNotBeInstanceOf:
    should:by:
    should:

Each of these matchers are attached to every Cappuccino object (including toll free bridged objects like strings and arrays). They are built on top of the OJAssert library currently used by OJUnit and the OJTest test runner will catch everything exactly the same way as it does for OJUnit. The difference is in the syntax.

The Syntax

The best way to explain why this different approach might be better or worse is to show a side by side comparison.

The OJUnit way

    [OJAssert assert:expected equals:actual];

The OJSpec way

    [actual shouldEqual:expected];

I really like the second syntax. This is also a bit of a cherry-picked scenario, though. Let's try something a little more complicated. Let's try to verify that something loops 5 times.

The OJUnit way

    var i = 0;
    while([anObject shouldLoop])
        i++
    [OJAssert assert:5 equals:i];

The OJSpec way

    [0 should:"be 5" by:function(i) {
        while([anObject shouldLoop])
            i++
        [i shouldEqual:5];
    }];

In this case, the OJSpec way is a bit more verbose but also a bit more description. It'll also allow us to give more detailed readouts (not yet, though!) on what is happening within our code. If this is a common case, we could also build this into the library.

Matchers Spec

Each matcher operates on an object. The parameters, however, vary. Below is a rough Doxygen documentation draft.

    /**
     * Verifies that one object is equal to the other using == or isEqual
     * @param anObject the object to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldEqual:(id)anObject
    
    /**
     * Verifies that one object is not equal to the other using != or !isEqual
     * @param anObject the object to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldNotEqual:(id)anObject
    
    /**
     * Verifies that an object is nil
     * @throws AssertionFailedError
     */
    CPObject-shouldBeNil
    
    /**
     * Verifies that an object is not nil
     * @throws AssertionFailedError
     */
    CPObject-shouldNotBeNil
    
    /**
     * Verifies that one object is the same object as another using ===
     * @param anObject the object to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldBeSameAs:(id)anObject
    
    /**
     * Verifies that one object is not the same object as another using ===
     * @param anObject the object to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldNotBeSameAs:(id)anObject
    
    /**
     * Verifies that one object is an instance of a class
     * @param aClass the class to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldBeInstanceOf:(Class)aClass
    
    /**
     * Verifies that one object is not an instance of a class
     * @param aClass the class to test against
     * @throws AssertionFailedError
     */
    CPObject-shouldNotBeInstanceOf:(Class)aClass

    /**
     * Runs a test with a bounded, named context
     * @param aDescription the description of the test
     * @param singleArgClosure aClosure with the object passed into it
     * @throws AssertionFailedError
     */
    CPObject-should:(CPString)aDescription by:(Function)singleArgClosure

    /**
     * Stubs a test and marks it as a failure
     * @param aDescription the description of the test
     * @throws AssertionFailedError
     */
    CPObject-should:(CPString)aDescription

Hopefully this is just a small step and we can get into doing things like a custom OJSpec test runner and all that fun jazz. In the meantime, start using this and let me know what you think! Thanks!

[1] It isn't enabled by default currently because OJSpec modifies the CPObject class which can be very dangerous and may have unknown consequences. In order to make sure that it doesn't effect you, you should first run your tests with OJSpec enabled. If you have poor test coverage or no tests, you can do a manual end-to-end test of your application by including @import <OJSpec/OJSpec.j> in your AppController.

Personas

2010-07-16T00:00:00-05:00

Note: This post deals with personas as a tool for developers to communicate outside of the project. I am not referring to personas that are used in business analysis.

There are different software development personas that have been floating around for a while. These personas are great tools for communication when all parties involved know who they are and what they represent. In this post, I want to catalog some of the personas that I've come across and give some meaningful description to them. Of course, this catalog is not comprehensive but rather a starting point for introducing personas into your software development language.

Cryptography

Cryptography has used personas for years. As part of my introduction to cryptography, I was taught some of the following. This is both necessary and useful in the cryptographic community because so much of the discussion takes place assuming knowledge of the personas. I've recreated them here and given an explanation about how software developers, in general, may use these personas.

Alice and Bob

Person A wants to send message "Secret" to Person B

Person A became Alice and Person B became Bob. In cryptography you normally have two different users of a system interacting with each other. Both Alice and Bob are 'typical' users in the process. For software developers, we could expand this to mean typical users[1] of our system.

Charlie, Dave, etc.

Not often in Cryptography are there more than two typical parties but when they are needed, there is generally just an alphabetical use of names. Again, we can reuse this to mean typical users in our systems.

Chuck

Charlie's evil twin brother. Chuck is someone that looks like a typical user but is actually malicious. He may use the algorithm as a user but is biding his time in some grand scheme. In software development, malicious users (think people that misuse a system, not black hats) can be called Chuck.

Eve

A malicious third party. Eve represents what a typical cryptographic algorithm is protecting against. Her goals are to break the code and spy on the messages (eavesdrop). In software development, we can think of these as black hats.

Trent

A trusted third party. This is someone like a certificate provider. Both parties trust Trent to do the right thing and Trent has no aspirations to do anything malicious with the data. This one is a bit harder to bridge to software development, but if anyone has any ideas, let me know!

Comics

Believe it or not, comics have a large influence on the social interactions of people. Software developers are not exempt. Our favorite comics tend to have an impact on how we communicate.

Dilbert

Dilbert is a enterprise programmer that works for middle management and codes inside of a cubicle. He isn't necessarily passionate about his job or programming. He's the ultimate 'paycheck programmer.'

Pointy Haired Boss

Dilbert's boss. He is our view of typical middle management (whether its accurate or not). He makes ridiculous requests that can't possibly be made. He doesn't understand the software development process. He is more concerned about moving up the corporate ladder than making the business better or the employees happy.

Desmond

For NotInventedHere, Desmond is a programmer that fits the stereotype. He gets overly excited about code. He spends his time working on personal projects rather than for his company. He loves writing cool, complex code that no one else understand or uses. He's the ultrageek.

Random

And, there are some random personas that pop up and describe a particular character in the software development world. Most of these come up as the need arises.

Jimmy

Jimmy is a naive, unfortunate programmer that screws things up. He is often too sure of himself and slings code without following good practices of software development or the practices of the rest of his team. People don't like Jimmy.

Other names/Acronyms: NZPP (Net Zero Producing Programming), NNPP (Net Negative Producing Programming)

Rockstar

Rockstar is an awesome programmer than can sling code and make things work. Some rockstars follow good practices; some don't.

Other names: Guru, Ninja

Of course, there are many others that have varying degrees of use. Those are just as valid. As software developers, we need to hone our communication tools. Personas is a great tool that allow us to communicate at a very high level while creating both vivid, detailed images. If you plan to use these personas (and you should!), you should make sure that everyone on the team knows what you're talking about when you say "Pointy Haired Boss." Not everyone reads Dilbert (but they should! :).

If you think that there are personas that should be on this list, let me know. I'll probably add them. Email me at firstname.lastname@thoughtworks.com.

Node.js Patterns

2010-07-05T00:00:00-05:00

Choice selections from wtfjs!

max vs the infinite

@pbakaus points out that Number.MAX_VALUE is close to infinity, but not too close.

    Number.MAX_VALUE*1.0000000000000001 === (1/0) // false
    Number.MAX_VALUE*1.0000000000000002 === (1/0) // true

ie cursed recursion

@tlrobinson of 280 North, Cappuccino, Narwhal and CommonJS points out some lovely IE behavior in this great blog post.

    window.recurse = function(times) {
        if (times !== 0)
            recurse(times - 1);
    };
    recurse(13); // stack overflow at: 0

messing with the number prototype

@rcanine shows us this interesting this coerce behaviour within the prototype of Number. Fun!

    (1) === 1; // true

    Number.prototype.isOne = function () { return this === 1; }

    (1).isOne(); // false!

    Number.prototype.reallyIsOne = function () { return this - 1 === 0; }

    (1).reallyIsOne(); // true

All information courtesy of wtfjs.com and Brian LeRoux.

I have been writing javascript for some time now and recently have been spending some time with Node.js. The javascript as a first class language movement is still very young and we are finding new things about javascript every day. Most of them are good; some of them aren't so good.

I have been making note of patterns that I've come across, developed or read throughout the javascript world. These patterns are in the style of good javascript practice as well as pattern languages that will allow javascript developers to communicate implicitly with other developers. If you are curious about more of these patterns, I suggest reading the Gang of Four Design Patterns book and then moving on to Martin Fowler's work and writings.

Immutable Object Pattern

Alternative names	new object pattern
Description	The immutable object pattern prevents us to making changes to external objects within a function. Often this means creating a new object and returning it rather than modifying parameters.
Reasoning	Javascript is a functional-enabled language. Functions are treated as first class citizens in the language. However, the functional quality is not enforced. The immutable object pattern defines a practice that allows us to treat javascript as a functional language. This means that we can apply functions across objects without worrying about the object changing (and we can grab the new changed object from the return of those functions).

If you're writing javascript in a functional way, you will want to use good functional practices. Most functional languages will enforce immutability (to a degree). In Haskell, everything but monads are immutable. In scheme, you need to use special mutation functions like set!. In Javascript, this is not enforced.

The biggest offenders in the javascript community that I've seen is the code that I've read that was written for Node.js. For example, the following is a (sanitized) version of a popular Node.js project.

    function proxy(request, response) {
        request.headers.host = "api.com";
        request.url = "/api/" + request.url.match(/^\/myAPI\/(.*)$/)[1];
        delete request.headers.cookie;

        var proxy = HTTP.createClient(80, request.headers.host);
        var proxy_request = proxy.request(request.method, request.url, request.headers);

        request.addListener("data", function(chunk) {
            proxy_request.write(chunk);
        });
        request.addListener("end", function() {
            proxy_request.end();
        });

        proxy_request.addListener("response", function (proxy_response) {
            proxy_response.addListener("data", function(chunk) {
                response.write(chunk);
            });
            proxy_response.addListener("end", function() {
                response.end();
            });
            response.writeHead(proxy_response.statusCode, proxy_response.headers);
        });
    }

The above code takes a request and reroutes it to an external API. It works great and the project is small enough that someone could very easily understand what is going on. However, let us imagine that it was a slightly larger project with about 1,000 - 2,000 lines of code. The code that uses the proxy in the hypothetical project looks something like this:

    function handleRequest(request, response) {
        if (request.url.match(/^\/myAPI\/(.*)$/))
            proxy(request, response);
        else
            getDataFromDatabase(request, response);
    }

And now we want to store the myAPI requests in our own database for logging purposes. In a functional, immutable language we would write something like this:

    function handleRequest(request, response) {
        if (request.url.match(/^\/myAPI\/(.*)$/)) {
            proxy(request, response);
            log(request);
        } else
            getDataFromDatabase(request, response);
    }

However, this doesn't work the way that we would expect it. After we run the proxy function, the request object has been modified and is now a request to the external API. We could resolve this by storing a local variable or calling the log function first (assuming the log function is immutable!). Add another mutable function into the mix, though, and the local variable becomes the only solution.

When using the immutable object pattern, do not change the object! We can look and read the object, but we don't know and cannot change the internals of the object we are accessing.

Photo by kathrynvjones. Distributed under CC-by-nc-sa.

In order to make the proxy function operate as expected, we need to make it follow the immutable object pattern. This means that we aren't modifying the request object that is passed in. [1]

    function proxy(request, response) {
        var proxy = HTTP.createClient(80, "api.com");
        var proxy_request = proxy.request(request.method, "/api/"
            + request.url.match(/^\/myAPI\/(.*)$/)[1],
            {'post':request.headers.post});

        // below here is the same as before
    }

You can see in the modified code above that we are merely reading the information off of the request object and never modifying the request object itself. This allows us to use the request object in the handleRequest function without having to worry about the object changing from under us.

This pattern is a good practice but should not always be used. Even in 'pure' functional languages such as Haskell and Scheme, there are circumstances where mutability is required. However, the more immutable our programs and libraries are, the safer they become to use. When it makes sense, we should use this pattern where ever possible.

Module as object-oriented package

Alternative names	OO-Module
Description	The OO-Module is similar to the packages of an object oriented language such as Java and C#. The modules only contain class definitions.
Reasoning	Javascript being a prototypical language and Modules 1.1 of CommonJS both give the developer a wide range of options when organizing a package. OO-Module and Function Module (seen next) both give us a structured, predictable way of organizing our javascript modules.

Coming from an object oriented background, my first instinct on a big javascript project was to create classes and objects. However, as my projects grew I realized that dealing with a bunch of different classes at the same 'package' level was a hassle. Folders weren't a great option by themselves. So, I started writing modules something along the lines of this

    // A top-level module with the following folder structure
    //    myModule.js
    //    myModule/
    //    - myModule.js
    //    - another.js
    //    - last.js
    //    - somePrivateInternalClass.js
    exports.myModule = require("./myModule/myModule").myModule;
    exports.another = require("./myModule/another").another;
    exports.last = require("./myModule/last").last;

Where the actual code only exists inside of the folder. This allows us to control the exposure, by convention, of classes. For example, somePrivateInternalClass is never 'exposed' to the outside world (though nothing is stopping us from requiring it). This also means that we know exactly how to access a public class through any package. It is just require("path/to/package").PublicClass.

It should be noted that we should never be using the exports object to transport an entire class. For example, the following code is an anti-pattern

    var AntiPattern = function() {

    };

    AntiPattern.prototype.doSomething = function() {};

    exports = AntiPattern;

While currently safe in most instances on both Narwhal and Node.js, there is no promise that the exports object will not change in the future. Instead, you should export classes as properties on the exports object.

Photo by dramaqueennorma. Distributed under CC-by-nc-sa.

Module as a function library

Alternative names	Function-Module
Description	The function module is a collection of functions that are not tied to a particular object. This is the pattern that you'll most likely use for mix ins (see below) or utility modules.
Reasoning	Javascript being a prototypical language and Modules 1.1 of CommonJS both give the developer a wide range of options when organizing a package. OO-Module (seen previous) and Function Module both give us a structured, predictable way of organizing our javascript modules.

    exports.functionA = function() {};
    exports.functionB = function(arg) {};
    exports.functionC = function(arg, arg2) {};

And using it looks something like this

    require("myModule").functionA();
    require("myModule").functionB(1);
    require("myModule").functionC(1, 2);

While this is a simple pattern, I have gotten myself into trouble by mixing the two styles of modules in one module. Each of your modules should either by a Function module or an OO module. Function modules can be extended and organized in a way similar to the OO module by using mix ins. [2]

Mix Ins

Alternative names	Runtime Import, Extensions
Description	Mix ins allow us to extend a class or function module without bloat. It also, in some circumstances, allow us to re-use certain functions that are common among unrelated segments of code.
Reasoning	Having large, complex files makes working with code difficult. For that reason, many languages and IDEs have ways of managing packages and files. In javascript, we have nearly limitless methods of organizing our code and run time extensions are another way to manage our code.

A while back, I wrote a blog post about mix ins. Since then I have had a bit more experience with javascript and there are actually two different styles of mix ins. There is the OO mixin and the function mixin. First, the mix in class will always by a function module (see above). It may look something like this:

    exports.functionA = function() {};
    exports.functionB = function(arg) {};
    exports.functionC = function(arg, arg2) {};

In an OO mix in, we mix these functions in inside of the constructor.

    ...

    var MyClass = function() {
        for(var x in require("mixin"))
            this[x] = require("mixin")[x];
    }

    ...

In a function mix in, we mix these functions in to the exports object.

    ...

    for(var x in require("mixin"))
        exports[x] = require("mixin")[x];

    ...

The function mix ins are a great way to organize a big function module. We can use the function mixin to include a whole set of subfiles that allow logical grouping of these files for easier navigation.

Photo by Geograph.org.uk. Distributed under CC-by-sa.

    // A top-level module with the following folder structure
    //    myModule.js
    //    myModule/
    //    - myModule.js
    //    - another.js
    //    - last.js
    //    - somePrivateInternalClass.js
    for(var x in require("./myModule/myModule"))
        exports[x] = require("./myModule/myModule")[x];

    for(var x in require("./myModule/another"))
        exports[x] = require("./myModule/another")[x];

    for(var x in require("./myModule/last"))
        exports[x] = require("./myModule/last")[x];

Which, like the OO module, allows us to 'hide' certain functions from the public by convention.

Class Method

Alternative names	Creator method, Factory method
Description	The class method is a version of a factory method that exists and operates similar to that of an OO class method. These methods are usually for constructing and returning underlying objects.
Reasoning	Creating objects can be a hassle. We have to construct the object, set up the dependencies and set up the data. This can also create duplicate code all over our code base. In order to reduce this, we want to create a method close to the class that will construct a particular object for us.

Class methods are nothing new and are often factory methods. In javascript, though, there is no real concept of a "class method". We get around this by creating a function right next to the definition of the class.

    ...

    var MyClass = function() {

    };

    var createMyClass = function(someData) {
        var result = new MyClass();
        result.someData = someData;
        return result;
    };

    ...

This pattern is the one that I see most often in libraries and I love the fact that they use this. I don't want to have to know the internals of an HTTP request in order to use it; I just need to create the request. Creator methods make this a breeze and keep developers writing interesting code rather than reading through code they don't need to know about for their particular solution.

[1] I'm making a guess that they want the POST headers here. I'm not sure what else they may want from the headers. [2] There is an exception to this. 'Class methods' are acceptable to be used within an OO module.

Roman Numerals

2010-06-18T00:00:00-05:00

At the office today we did a coding dojo and the kata that we used was Roman Numerals. We actually did the kata in Java because that was what everyone was familiar with but I decided to work on doing it in Ruby.

After after 20 minutes I got a working solution. However, that's when the fun starts! I spent some time trying several refactorings on the code and didn't really get anywhere. The code that I was refactoring looked roughly like this

@value_map = {
    1 => "I",
    4 => "IV",
    5 => "V",
    9 => "IX",
    10 => "X",
    ...
}

[..., 10, 9, 5, 4, 1].each do |value|
    while(number >= value)
        number -= value
        result += @value_map[value]
    end
end

Because of the expressiveness of Ruby, this code is terse. However, we are covering our edge cases in an odd way. This worked and I couldn't necessarily refactor it to be better. However, I knew that if we added a feature that those edge cases would make it start to look ugly. So, I added a feature (and learned something about roman numerals!). Instead of just going to 3999, we would support 38,999. This meant we needed to add the roman numerals (V) = 5,000 and (X) = 10,000. I added the feature and then refactored and ended up with something like this

class Romans
  def initialize
    @value_map = {
      1 => "I",
      5 => "V",
      10 => "X",
      50 => "L",
      100 => "C",
      500 => "D",
      1000 => "M",
      5000 => "(V)",
      10000 => "(X)",
    }
  end

  def convert integer
    result = ""
    [10_000, 1000, 100, 10, 1].each do |value|
      integer, result = convert_to_highest_significant_digit integer, value, result
    end
    result
  end

  def convert_to_highest_significant_digit integer, value, result  
    if integer >= value * 9
      result += @value_map[value] + @value_map[value * 10]
      integer -= value * 9
    elsif integer >= value * 5
      result += @value_map[value * 5]
      integer -= value * 5
    elsif integer >= value * 4
      result += @value_map[value] + @value_map[value * 5]
      integer -= value * 4
    end

    while integer >= value
      result += @value_map[value]
      integer -= value
    end

    [integer, result]
  end
end

The code above is not without problems, either. We have to lean on the Ruby feature of multiple returns. We also have that ugly if/elsif/elsif statement. We may be able to handle this with another object, though. Here is a more object oriented way

class Romans
  def initialize
    @values = [
      RomanDigitGroup.new(
        RomanDigit.new(1, "I"),
        RomanDigit.new(5, "V"),
        RomanDigit.new(10, "X")
      ),
      RomanDigitGroup.new(
        RomanDigit.new(10, "X"),
        RomanDigit.new(50, "L"),
        RomanDigit.new(100, "C")
      ),
      RomanDigitGroup.new(
        RomanDigit.new(100, "C"),
        RomanDigit.new(500, "D"),
        RomanDigit.new(1000, "M")
      ),
      RomanDigitGroup.new(
        RomanDigit.new(1000, "M"),
        RomanDigit.new(5000, "(V)"),
        RomanDigit.new(10_000, "(X)")
      ),
    ]
  end

  def convert integer
    result = ""
    @values.reverse.each do |value|
      roman = value.convert_to_roman(integer)
      integer -= value.max_represented_value(integer)
      result += roman
    end
    result
  end
end

class RomanDigit
  attr_reader :value
  attr_reader :roman

  def initialize value, roman
    @value = value
    @roman = roman
  end
end

class RomanDigitGroup
  attr_reader :one
  attr_reader :five
  attr_reader :ten

  def initialize one, five, ten
    @one = one
    @five = five
    @ten = ten
    @base_value = one.value
  end

  def convert_to_roman integer
    result = ""

    while integer >= ten_x
      result += @ten.roman
      integer -= ten_x
    end

    if integer >= nine_x
      result += @one.roman + @ten.roman
      integer -= nine_x
    elsif integer >= five_x
      result += @five.roman
      integer -= five_x
    elsif integer >= four_x
      result += @one.roman + @five.roman
      integer -= four_x
    end

    while integer >= one_x
      result += @one.roman
      integer -= one_x
    end

    result
  end

  def max_represented_value integer
    integer_start = integer

    integer -= ten_x while integer >= ten_x

    if integer >= nine_x
      integer -= nine_x
    elsif integer >= five_x
      integer -= five_x
    elsif integer >= four_x
      integer -= four_x
    end

    integer -= one_x while integer >= one_x

    integer_start - integer
  end

  private

  def ten_x
    @base_value * 10
  end

  def nine_x
    @base_value * 9
  end

  def five_x
    @base_value * 5
  end

  def four_x
    @base_value * 4
  end

  def one_x
    @base_value
  end
end

Unfortunately, that doesn't look exactly right either. We still have the complexity of the while and if/elsifs. The object oriented form may be easier to read and understand but I think it is a little heavy for our purposes (the RomanDigitGroup class conveys how Roman Numerals actually work). This method also allows us to get rid of the hash data structure and the duplication of data (hash and array in the prior implementations).

So, where do I go from here? I'm not really sure. Is there a clever trick to get rid of the ugly while and if statements? Is there a better abstraction that could be applied? Maybe this algorithm has irreducible complexity? I personally like the second version (non object oriented with ugly if statements) best.

Feedback Loop

2010-05-22T00:00:00-05:00

I have just finished school. Graduation is in one week. I am really excited about that. This next week should be a flurry of hanging out with my college friends, doing one last round of stupid college stuff and relaxing. After that, my life will become a very exciting for a few weeks and then I'll settle into some sort of routine (I can only assume).

However, despite the fact that I will be leaving school, I still have a lot to learn. I will be soaking up as much knowledge as I can when I go off to Chicago to work with ThoughtWorks. But, I will be leaving behind a great group of peers, mentors and friends. I am going to be selfish and attempt to leverage them right before I leave.

I have created what Chad Fowler refers to as a 360 review in his book The Passionate Programmer (read it!). For anyone that has worked with me, mentored me or even just interacted with me while I've been at college, I am asking that you help me out and fill out the review.

Fill out the review!

I plan to leave this up indefinitely so that anyone may give me anonymous (or not anonymous.. their choice) feedback at any time. Please remember to be constructive so that I can address my own weaknesses and help improve them! Thanks!

Javascript Mix Ins

2010-05-12T00:00:00-05:00

When writing our compiler, Chandler and I had to go through several phases of development. Each of these phases were adding one or two methods to our node objects in our abstract syntax tree. When we need to add typechecking, we had to write a method for each node type that would know how to typecheck itself.

Unfortunately, this meant that we were constantly modifying the node classes. This breaks the open/closed principle and just feels bad. We didn't want to keep changing the class to extend it.

Additionally, by adding these to the existing classes, we were muddying the code. The purpose of the class would seem lost in the sea of additional code we would add. The code we added would also lose information density. Having related code in close proximity allows for greater understanding of what is going on inside of the code.

What we really wanted were Ruby-esque mix ins. And, despite not having any capability provide explicitly by the language (like it is in Ruby), the flexibility of javascript allowed for us to provide this functionality.

The class with the mix in "mixed":

var MyMixIn = require("mixin");  
  
var MyClass = function() {
  for(var key in MyMixIn) {
    this[key] = MyMixIn[key];
  }
};

MyClass.prototype.doSomething = function() {
  
};

The mix in:

exports.doSomethingElse = function() {
  
};

And, using it:

var obj = new MyClass();
  
obj.doSomething();
obj.doSomethingElse();

Good stuff. Now we can package our related code, reuse common methods through mixins and other crazy cool stuff that involves mixins.

Introducing CuCapp

2010-05-06T00:00:00-05:00

CuCapp is Cucumber for Cappuccino. Yep. And, this time, its real cucumber. It's not the weaksauce I started with Barista (I'm discontinuing Barista development--it was a great learning experience and very tough; CuCapp should be a better and more fully featured replacement now). I should mention that Daniel Parnell started the project and that I just forked it on github and put some elbow grease into it.

CuCapp from Derek Hammer on Vimeo.

Pretty, right?

OJUnit, OJMoq, OJAutotest, OJCov, OJSpec and CuCapp. I'm pretty sure we're getting somewhere with testing.

I'm going to be constantly improving all of these tools. Bringing OJSpec up to speed is next on my list, but perhaps you guys have better suggestions? What do Capp devs want to see for testing?

Collaborative Teamwork

2010-05-05T00:00:00-05:00

The following is an excerpt of an essay that I had to write. I thought it expressed how I felt about my current team and that it may be useful to people forming their own teams.

The Collaborative Team

On our project, Project OSL, we are working on a project that outstrips the level of experience each of us has attained individually. At the beginning of the project, we were faced with a daunting project but through sharing ideas, supporting, teaching, learning and communicating with each other, we were able to create a successful experience for the whole team.

Sharing Ideas

Our project is both ambitious and unspecific. We were to create a piece of software that relied on unproven technologies that would match the ambitious vision of our client and ourselves. The technical details and user needs were mostly unknown beyond the high level curiosities.

In order to overcome that obstacle, the team needed to constantly bounce ideas off of each other. We would usually work together in a shared space so that we could look up and ask a team member to head over to the whiteboard and listen to ideas.

We came up with some bad ideas; we came up with some good ideas. Each time, the feedback from our team members was essential in solving the tough problems.

Supporting

Despite the fact that we constantly asked each other for input on ideas, we were never negative toward each other. We all understood that the goal was a solid product for our client and that the criticisms were constructive rather that destructive and personal.

Teaching

We are constantly teaching each other. Whether it be a formal seminar to our teams, an informal solve-a-hard-coding-problem-as-a-team dojo, or just a "How does this work?", each member of the team became a teacher at one point or another.

Learning

With the willingness to teach, there is a necessary desire to learn. The team was always ready to learn something new from each other and this collective sharing of knowledge allowed us to be flexible and strong as a team.

Communicating

Each of the above were compounded by the fact that our team is effective at communicating ideas. We were able to stand up at white boards and create quick modeling diagrams in order to describe what we were thinking; we were able to create presentations that were quick, to the point and wasted no ones time; we were able to draft effective emails; and, most importantly, we liked interacting with each other.

Both of my teammates are guys that I can hang out with. They are certainly not my best friends but they are certainly good friends. I have had beer with both. I've had (completely unnecessary) dinner with both. None of us grate on each other.

Communicating effectively makes all of the above attributes of the team work cohesively and effectively.

HOWTO : Write Objective-J Tests

2010-04-17T00:00:00-05:00

I have recently been writing about big theoretical discussions. While I think that those are important, the other part of writing blog posts is to give back to the community. This post is about discussing the best practices of testing Cappuccino and other Objective-J projects. I have been doing this for several months now and (hopefully) can write something pretty authoritative on this subject. I hope this helps those new to Cappuccino and/or testing!

The culture of testing in the Cappuccino community has been pretty weak. However, the mindshare in the testing culture has been slowly but steadily increasing. Writing good unit tests is essential to the growth and acceptance of the testing culture. I am going to present some best practices that should help the newly minted tester to the mindset that is testing.

OJTest Suite

First, you need to know the tools. The OJTest repository contains all[1] of the tools necessary to effectively test Cappuccino applications. In this repository there are two particularly important frameworks: OJUnit and OJMoq. OJUnit is an xUnit style of framework that we will use to actually run our tests. OJMoq is a mocking framework that will allow us to break dependencies in tests.

In order to run a test, you need to issue the following command

ojtest SomeTest.j

where SomeTest.j is a test that follows the conventional test setup

@implementation SomeTest : OJTestCase

- (void)testThatSomeTestWorks
{
  [self assertTrue:YES];
}

@end

(we will get more into the conventions of OJTest in a second). In the above code, we are creating a test case called SomeTest. In OJTest, a test case is actually a group of tests. This is usually just a one-to-one mapping of the classes to tests. For example, SomeTest.j would probably be testing a class called Some that is contained in Some.j.

OJTest Conventions

There are some conventions that we normally use in the testing environment for Objective-J. These conventions make the development of additional tools (like OJAutotest) and for easier navigation and readability by other Cappuccino developers.

The first convention is where we put our tests. We should put our tests into a folder called Test in the top level directory. For example,

AppController.j
SomeClass.j
AnotherClass.j
Test/
- AppControllerTest.j
- SomeClassTest.j
- AnotherClassTest.j

You will also notice that in the above example, each of the tests correspond to a single file (e.g. AppController.j -> AppControllerTest.j) and that tests are identified by appending Test to the end of the file's name. You should also note that it is a de facto convention that SomeClass.j contains a single class called SomeClass.

The last and most important convention is that all of your tests must begin with "test". This is important because that is how the test runner identifies the tests that it needs to run. We can also leverage this for our own purposes. This allows us to create helper functions.

@implementation SomeTest : OJTestCase

- (void)testThatSomeTestWorks
...

- (Some)buildASome
{
  return [[Some alloc] init];
}

@end

In the above example, testThatSomeTestWorks will be run by the test runner but buildASome will not.

OJTest Techniques

There are a variety of techniques that can be used to effectively test your Cappuccino and/or Objective-J projects. These techniques are best practices that I have found when testing over the past few months and are not as concrete as the descriptions above. Hopefully, they will be a good starting point for many developers starting to test applications.

Assertions

Assertions are the key technique used in unit testing. Assertions are members of OJTestCase and can be used in the following manner:

[self assertTrue:YES];
[self assertFalse:NO];
[self assert:expected equals:actual];
...

For a full list of available assertions, you can look at the class documentation on OJTest. They can also be seen in the source.

Small Tests

Make your tests small. This is really an xUnit best practice but in a dynamic language such as Objective-J, this all the more important. Small tests allow a granularity that allows you to quickly find problems with your production code. A good rule of thumb (though, this is definitely not always the case!) is that you should have one assertion per test.

Mocking

We are attempting to test ClassUnderTest. In this case, we see that it has three immediate dependencies called Database, View, and MathUtility. In the 1plus technique, we would instantiate the MathUtility and View classes (they have no external dependencies). We would also instantiate the Database class but we would mock its two dependencies (LocalStorage and Connection). As with any rule, though, there are exceptions. If, for example, MathUtility cannot be instantiated in test (which can be for a variety of reasons), then we should prefer a dynamic mock over a fake.

Mocking is one of the more controversial techniques in testing communities across a variety of languages. However, I find that mocking can be useful in a pragmatic way. I usually use a technique that involves mocking only dependencies with a length greater than 1 (I refer to this as the "1plus" mocking technique). This allows us to preserve some mini-integration benefits of instantiating everything while keeping the locality benefits of mocking.

OJMoq makes mocking awesome. It is very simple and allows you to create stubs, mocks and spies in a very intuitive way. In order to create a stub, you just do the following

var stub = moq();

In order to create a mock, you just add expectations to a stub.

var mock = moq();
[mock selector:@selector(init) times:1];

In order to create a spy, you just wrap an existing object.

var dependency = [[View alloc] initWithFrame:CGRectMakeZero()];
var spy = moq(dependency);

In order to keep this post short(er), I won't go into the difference of stubs, mocks and spies. If you aren't sure what to use for your situation, just ask in the IRC channel (irc.freenode.net#cappuccino).

SetUp / TearDown

For anyone familiar with setup and teardown methods in the xUnit framework, OJTest does support these. SetUp and TearDown methods are called before and after, respectively, each test is run. This allows us to set up some information that may be repeated over and over. In order to use them, just methods called setUp and tearDown.

@implementation SomeTest : OJTestCase
{
  Some  target;
}

- (void)setUp
{
  target = [[Some alloc] init];
}

- (void)tearDown
{
  [target cleanUp];
}

- (void)testSomething
...

@end

Run Your Tests!

Part of the beautiful of unit testing is that it is also future-proofing your code. You are simultaneously testing a small segment of code and incrementally growing your regression test suite. But, the regression test suite isn't useful unless you run those tests! I suggest that you add this to your Jakefile

task("test", function()
{
    var tests = new FileList('Test/*Test.j');
    var cmd = ["ojtest"].concat(tests.items());
    var cmdString = cmd.map(OS.enquote).join(" ");

    var code = OS.system(cmdString);
    if (code !== 0)
        OS.exit(code);
});

which will allow you to run jake test and run all of your tests.

Refactor

Refactoring is considered a huge benefit in production code. Unfortunately, it is not leveraged as much in test code. We can reduce the pain of testing by refactoring. This includes using the SetUp and TearDown methods described above but also includes creating some object factories for often used objects and creating static utility functions for often executed code.

Conclusion

Hopefully this will be a good starting point for everyone interesting in testing for Cappuccino. There are definitely areas of this post that could be expanded upon and I am willing to answer those questions! Just shoot me an email ( derek r hammer at gmail dot com ) or catch me on the IRC channel.

[1] There are other testing tools that are not part of the OJTest repository. Notably there is OJSpec and Barista.

One Year Goals

2010-04-02T00:00:00-05:00

Several people state that defining long term goals is a good thing. I tend to believe them and so I'm going to create a list of long term goals. By April Fools' Day, 2011, I want to achieve the following goals. These goals may be out of reach but hopefully by reaching a little I can actually surprise myself and meet my goals!

Professional

I am going to start off with professional goals. This is the start of my professional life and I am very excited. I feel that I am starting in a great place; I am beginning life at a great company, I have a wonderful support network of colleagues, mentors, mentees and friends in the professional world and I am starting a "green field career" where I am carrying no baggage of bad experiences. However, I do not want to be satisfied with this. I need to set some goals that should help me launch into further success.

1. Find another mentor

I already have a few great mentors in Jon Fuller, Curt Clifton and Sriram Mohan. While I will continue to lean on my existing mentors, I would like to find another mentor in the coming year. The mentor/mentee relationship is one of the strongest in the professional world and having great mentors make you that much better. It would be a bonus to find a mentor in Thoughtworks (there should be plenty of viable candidates!) that would have experiences and advice relating to the company.

2. Produce for mentees

A mentor is often something that is intangible. She is there as a guide and a sounding board. However, there are also some tangibles that come from mentors. While I'm not particular about the tangible, I do want to produce something of the nature for Pete Brousalis and Eric Stokes. It could be a recommendation to a particular job offer or something else that has a direct impact on their lives.

3. Put an idea into action

I have a ton of ideas. Unfortunately, not many of them see the light of day. I want to put one idea into action. It could be as simple as writing and maintaining an open source project that I've been thinking about or launching a small business venture. I want to do something positive with my creativity.

4. Stay disciplined

I am very scatterbrained. I will do something for a couple of weeks and then lose interest. One great example is doing katas. I had done them for a few weeks but then lost track of doing them due to "time constraints." I need to stay disciplined and maintain my schedule in a professional manner.

5. Learn

This summer, I will be starting work at Thoughtworks. I will be working with some of the best minds in software development and I need to put that opportunity to use. I need to stay humble and try to soak up as much knowledge as possible. If necessary, I should keep a journal of the things that I am learning so that I do not miss the opportunity to internalize the wisdom.

6. Speak at a conference

Public speaking is a learned art. In order to get better at it, I need to practice. I should take every opportunity that I can to speak publicly--whether it is at local interest group meetings or even small presentations to project groups. Ultimately, though, I want to give a speech at a conference of some significance--it can be a regional or domain specific conference.

7. Mentor someone under 18

I think that mentoring is a key element of our responsibilities as professionals. I want to get involved with someone young (the younger the better, really) that has an interest in computing. It would be fantastic if I could find a young woman to mentor. I have no idea how I am going to accomplish this but hopefully I can pull it off!

Personal

As well as starting my professional life, I will be starting a new section of my personal life. I will be moving to Chicago (my first real city) and living in my own apartment (for the first time on my own) with almost no one around that I know. It'll be a great experience and I am looking forward to it (though I am a little terrified). My personal goals are probably a bit more humble than my professional goals.

8. Make non-work friends

I love what I do. However, I cannot put my entire life into my work. I need to make some friends outside of work that I can hang out with when I need to get away. I have been able to do this in the past so I am not worried but listing this as a goal helps me avoid the trap of just work friends.

9. Reduce clutter

Clutter is around everyone. I have a ton of gizmos, gadgets, electronics, souvenirs, clothes, etc. that I've collected. I'm not generally a packrat but I want to make my life simpler. I need to start getting rid of stuff that I once though were essential. If I can get rid of my TV (and HTPC that goes with it), that will be a huge win.

10. Cross-Train

In sports, athletes are told that they need to cross-train in order to be the most effective. Cross training is the act of training in a sport that is not your primary. This training has proven to make the athletes better at their primary sport. In that sense, I need to find something that is intellectually stimulating that is not computing. Philosophy may be something that I pick up.

11. Live healthier

This one is going to be tough. In college, we all tend to abuse our bodies. I need to put my college lifestyle behind me and live healthier. This probably means eating home cooked meals and exercising more.

Big Poster

In order to remind myself of this, I've actually taken an idea from software development. I am going to create a big poster and hang it on the wall. This poster just lists the 11 goals that I made above. Hopefully, it will be a constant reminder of my long terms goals and I can stay focused on achieving them.

Programming Practice :: Inverted Pyramid

2010-03-25T00:00:00-05:00

In the last post I described the philosophy of the inverted pyramid. At the end of the blog post, I promised that I would put this into practical terms. In this post I will give a video demonstration, tell you how all of your previous quality assurance techniques fit into the inverted pyramid, and describe the current state of the inverted pyramid.

State of Inverted Pyramid

Currently, there are several tools and techniques that exist in the world today that can help us accomplish the inverted pyramid methodology. We are not reinventing the wheel. The inverted pyramid leans heavily on the specific knowledge and best practices that exist in the quality assurance industry mixed with the practices evangelized by the test driven development community.

In this image we are presented with some "bands" of color overlaying the inverted pyramid. The red represents automated user interface testing tools such as Selenium and Celerity. The yellow represents behavior driven development frameworks such as Cucumber. The cyan represents specification test suites such as RSpec and NSpecify. Finally, the green represents xUnit frameworks such as Test::Unit and JUnit. You can see that there are no clearly defined lines that that some are more broad banded than others. For example, xUnit (green) and BDD (yellow) both touch up to three different layers of tests. Put together, they form a fairly solid testing package that can cover most of the pyramid. However, sometimes we do want redundancy in our testing (especially on very large projects!) and we will need additional testing tools and techniques.

Two Band/Pronged Approach

For most development, and I stress that this is not the case for every project, we can ensure quality of the application by attacking the code under the umbrella of two methods: behavior driven development and test driven development. Using a behavior test suite as the higher order tests and the unit test suite as the lower order tests, we can cover the quality assurance needs of small to medium-large applications. On the extremes, different methods will need to be practiced (for example, very small programs may be sufficiently covered and understood just by unit tests).

Behavior driven development is nothing new. Dan North has been putting this into practice for over 5 years! Cucumber was created to support behavior driven development and, at least in the Ruby community, it has taken off! This is great and inspiring that the developer community is embracing BDD on some level. However, we seem to be getting it wrong. Many people look at Cucumber as a replacement for unit testing. However, if we investigate the inverted pyramid you can see that just doing a BDD style framework only covers the top half. The lower half of the pyramid--the part from which developers get the most benefit--is completely absent. People just doing cucumber will find that doing pure BDD is much more difficult that doing pure TDD. You'll also find that you start making high level design decisions before they are necessary and that your regression suite doesn't cover as many bugs.

If we view BDD as supplementary instead of contradictory, though, we can create a solid process of quality assurance. This is the two band approach. There is not much redundancy (though there is a little bit) in the tests and we cover nearly the entire stack (Cucumber doesn't run against a 'real' browser environment). Depending on environments, different flavors of the two band approach may be needed (even adding more "bands" to the mix).

A Demonstration

So, maybe you're buying into the inverted pyramid and you want to get started. Where do you start? I've created a video below that demonstrates a basic use of the inverted pyramid against a Rails application. (Tools used: Cucumber, Cucumber-Rails, Test::Unit, Rails 3)

Inverted Pyramid Demonstration from Derek Hammer on Vimeo.

Programming Philosophy :: Inverted Pyramid

2010-03-24T00:00:00-05:00

I have been attempting to write this blog post for a few weeks now and have (obviously) not been successful. This is my latest iteration of the little war of thoughts that are happening inside of my head. Hopefully this blog post is better for it and that the readers can get something out of it.

We learn in a traditional quality assurance course that quality assurance is a sequence of steps to verify that the code matches the requirements. There is a rigorous, structured approach that is outlined in these courses that is modeled after how some companies in the industry perform quality assurance. Unfortunately, this is modeled after the "throw it over the wall" waterfall development methodology. How can we apply quality assurance techniques in an Agile environment?

Over the Cubicle Wall

This is how the industry does it.

I am lucky to be going to a really good school that has a great group of computer science and software engineering professors. These professors really care about teaching undergraduates (that's all they do; no research) and put a lot of time into giving us the best education possible. The result is that, in my experience, the students are taught "ahead of the curve" in academia. For example, our software engineering undergraduate degree is one of the handful in the country and teaches a solid series of core software engineering courses. This includes everything from requirements gathering to software quality assurance.

However, we are talking about academia here. To no fault of the institution or the department, academia lags behind the industry. The nature of higher education leads to more red tape and bureaucracy that cannot keep up with the bursts of acceptance that can overtake the industry. ABET accreditation along with other oversight programs hamstring the department from being agile enough to create the modern programmers that they strive to create. So, despite the efforts our my school and my great professors, there are some crufty old ideas that stay in place long passed their shelf life.

This is an image of a traditional over the wall quality assurance process. Note the dotted line that well defines where developers do and do not tread. This method of quality assurance is closely associated with the development methodologies of yesteryear. Image found using Google Image Search and is property of The Software Experts.

One of these is how to go about software quality assurance. When I took the course, there was a clearly defined line between software quality assurance and software development. We worked on a project that was feature complete but needed to be tested for quality. It was well done if we were to emulate the traditional approach: we were a team of 4 testers that needed to test the application for faults. We were given the software documentation, source code and one of our team members was a member of the development team for that project. We had no contact with any of the other team members and for sure I could not even tell you who they were! We were to report bugs to them which they would promptly fix.

Finally, let me proffer a definition of traditional software quality assurance before I go providing an alternative. In a traditional approach, there is a separate team of "testers" that test the system. They are not involved in the development of the application and they are often physically separated from the team (not always; I was on a project where this was not true). They receive their instructions via project manager or note stating that Feature XYZ has been completed and that the quality assurance team should start work right away on verifying that the requirements have been met. The testers toil away until they have tested the application fully (and the 'other team' has fixed the bugs) and all of the requirements have been met. Then they release.

Bad Thing(tm)

Yep, that's how we do it.

To me, the traditional approach sounds terrible just by its definition. However, in case you're not quite drinking the Kool Aid that I am, let me list a few reasons that this may be good and a few reasons that this may be bad.

Good Thing(tm)?

Testers need to be apart from the development team in order to test effectively. This seems to be the most sound argument for doing the above. The first problem comes from the assumption that a tester is more closely aligned to the customer. I would argue that this is marginally true but the problem is that the tester is not the customer. If we want to know how a customer uses the system, let us use the customer herself!
The second problem comes from increased risk. Any project manager will tell you that managing a project is, on some level, about managing risk. The introduction of two technical teams evaluating the requirements increases the risk that the customer does not get what she wants! The argument may be that the re-evaluation of a requirement puts more eyes on the requirement and thus increases the likelihood that the collective team gets it right. However, if a developer is unsure about a requirement, think of how solid his resolve becomes if a tester corroborates his (false) assumption.
Developers know too much about the code to test! (AKA, they cannot blackbox test). This is another relatively sound argument for the use of traditional software development. Indeed, it is more difficult to black box test code when you are the one that has written the code. However, we can mitigate this with another strategy which I'll get to soon.
Competition between the test team and development team increases quality. A bit of healthy competition is a good thing. We are, by nature, competitive. However, Us v. Them mentalities, especially when sustained for long periods of time, create animosity and not competitiveness. Animosity is definitely not what you want on your project.
Developers do not know how to test! I do not think that developers are given the opportunity to prove themselves as a tester very often. And, fortunately, it seems that companies have to sometimes improvise by putting developers on understaffed quality assurance teams. From what I have heard and seen, these developers do not struggle when they get up to speed.
"Testing is below me," says Mr. Developer. Despite the stigma of quality assurance, being a tester is just as rewarding and enjoyable as being a developer (I love doing both!). As with being a developer, it all matters on what you put into it.
... There are bound to be more; it is hard for me to get into this mindset. Please, let me know if you can think of any arguments for the traditional method of quality assurance!

Bad Thing(tm)?

Creates animosity between the test team and the development team. In each project that I've worked on with the separate team structure, the development team would hate whenever a member of the quality assurance team would contact them. Invariably, it was to point out something that they did wrong. The test team, on the other hand, would get frustrated with the development team at the amount of bugs there were (no matter how many bugs there actually was) and how the development team kept making these mistakes!
Developers lose the motivation to test. If there is a quality assurance team whose job it is to catch your bugs, then why would you spend any time catching them yourself? Perhaps you want to avoid the ridicule that comes with creating a bug, but that seems like a poor motivating factor. Eventually, the developer is just not going to care if someone ridicules him for his bug that he introduced.
Customers end up fighting both the quality assurance team and the development team. Meetings with clients involve the quality assurance and development team lead. Both of these teams have their own goals that the customer must fight against. We want the customer to be as comfortable and satisfied as possible--not frustrated and unsatisfied.
Time between development and testing is increased. This may not be apparent right now, but the time it takes to complete a feature, toss it over the wall, create a set of test cases for the feature and run them increases the time it takes to test a feature, even if its happening adjacently.

(A) Solution

This all sounds pretty bad. But just pointing fingers isn't going to solve anything! What can we do about this?

In Extreme Programming Explained, Kent Beck tells us that the "extreme" part of the practice comes from taking Good Things(tm) to the extremes. Sometimes, these practices are counter-intuitive and costly. One controversial practice that serves as a great example for the purpose of this blog post is test driven development.

In this image we are exploring the inverted pyramid approach to quality assurance. We are meant to read the pyramid from right-to-left, top-to-bottom. On the left side of the pyramid, we have a practice (such as User Acceptance Testing) and on the right we have what will concretely satisfy that practice (such as User Story). From top to bottom, we have an order of specificity. At the top, for example, we have User Acceptance Testing. At the bottom, we have Unit Testing. The practical approach of this pyramid is to write a single test, write the code to resolve that test, and repeat until you have resolved the higher order test. For example, we have an integration test that states that a business application must talk to the database in order to grab data associated with the user's age for the controller layer. We would then write unit tests, then code to those unit tests, and repeat until the integration test was satisfied. We shall never write a Unit of Functionality that is not covered by Unit Tests in order to satisfy a system (or another higher order) test.

Test driven development turns the normal development process on its head and forces developers to write unit tests before they write the production code. Without going into too much detail and analysis of this practice, it has proven to be very effective at both reducing software bugs and increasing developer productivity.

We are going to solve our problem by turning the quality assurance process on its head. In fact, I mean that a little literally. Looking at the figure of the left, we can see that the traditional pyramid has been flipped over and turned about. This figure describes a methodology that solves the riddles we outlined above. We'll get to that in a second. However, the most important thing about this approach is that it satisfies our collective values. These values, outlined in Extreme Programming, should be the driving force behind every principle and practice.

Values

We will briefly explore which values are supported by this methodology. There will be a brief discussion about how each of the values are satisfied. Each value could be explored more in detail and I would love to do so at a later date. Please ask questions if you're not sure about my reasoning!

Communication: We, as people, like to communicate and interact with other human beings. In this process, we embrace communication by creating a single coherent team. Instead of fragmenting the quality assurance team, the development team, and the customer, we bring them all together in order to foster communication and collaboration. This gives us the benefit of understanding the needs of the customer without any barriers of communication (such as a Customer v. Dev Team v. QA Team process.
Simplicity: I think that this process embraces simplicity in the most obvious of ways. Instead of having a complex system of teams that interact in a structured manner, we reduce the complexity to a single team that handles the entire process. This does, however, put increased pressure on the developer, tester and customer and we will get to that later.
Feedback: In this system, we reduce the need for the structured communication channels and increase the communication between all parties by putting them face to face every single day. The developer gets feedback from testing almost immediately and the customer can interact with a developer that is developing the story as she develops it.
Courage: Courage is a bit more abstract in this system but is still present. The courage occurs when a developer has the courage to take on the role of a tester and has the courage to put her stamp of approval on the code that she has written. Courage takes the form of the customer becoming more involved in the software development process and well as the engagement of customers from the quality perspective of the team.
Respect: Instead of breeding animosity, this system breeds trust and respect. The customer, involved in the day to day conditions of development and the care that the developers take in ensuring quality, comes to respect the developers. The developers, working closely with the customer, come to appreciate and respect the ideas and concerns that the customer has.

I am a developer, what does this mean to me?

As a developer, you are now responsible for the quality of your code and features. Instead of relying on another team to ensure quality, your team is now going to have to do this. In this aspect, you are going to have to learn. The good news is that developers that are thrown into quality assurance teams tend to survive. This isn't an impossible task and hopefully you can approach this with enthusiasm and respect. Testing your own code is going to make you a better developer and allow you to confidently release your code to customers.

I am a tester, what does this mean to me?

As a tester, you are now going to have to become a developer. A "tester" position does not exist in the process we have described. Instead, you are going to become a generalist that has a little more experience in testing than the rest of the team. You, indeed, have the hardest path ahead of you. It is your job to guide (but not 'pinch hit') your fellow developers in the process of testing their code. You also have to learn how to interact with customers as a developer and be a part of the development team. You've been under appreciated for a long time now and its your time to shine.

I am a customer, what does this mean to me?

As a customer, this is all positive for you. If you are working on an Agile project already, you are deeply embedded in the development process already. Instead of having to wrestle with two teams, you are now working closely with only one. The team, as a whole, will come to understand what you mean by quality and how to create what you need.

Conclusion

Yes, this is all just theory though! How do we do it?

I have put this article under "programming philosophy" for a reason. There is very little in the way of practice in this article and I hope to address that in my next article. It will include a screencast showing this philosophy in practice and how it may be better that the traditional approach. Unfortunately, I have no data to support my philosophical claims here but I think that it is in the spirit of Agile and Extreme Programming. Those have been proven to be effective and piggybacking on that data may serve as an effective indicator of the practical implications of this methodology.

Testing :: Acceptance Tests

2010-02-27T00:00:00-06:00

In the past couple of months, there has been a backlash in some of the industry leaders about the role of automated acceptance tests. James Shore and Michael Feathers have both expressed disillusionment with the state of automated acceptance testing. While I understand their frustration, I whole-heartedly disagree.

The arguments against automated acceptance testing boil down to three main points: a) automated acceptance tests are too costly and difficult to maintain, b) customers do not want or cannot participate in automated acceptance testing and c) the tools are too ineffective. The first argument is just a knee-jerk reaction type of response. The second argument can be resolved by tailoring the solution to the particular user. The last argument is the most legitimate but is not an unsolvable problem.

Despite not always being knee-jerk, the type of response to acceptance tests as too costly and not effective in providing business value is of the kind. A great example of this is how the industry first reacted to automated regression tests (which I think most industry leaders and most people reading this blog would agree is a Good Thing(TM)). There were (and still are) people in our industry that stated that regression test suites are an investment not worth making. They argue that when you are writing unit tests, you are wasting valuable time and effort that could be put toward implementing new features and increasing business value. However, what we've come to realize is that regression test suites give us confidence to deliver and refactor. They also catch bugs that change in far away place that may not be caught until production. These benefits (and others) increase the value of the product that we are creating.

While I cannot list all of the benefits of automated acceptance testing (we need wide-scale acceptance to really understand the true benefits of any practice), I can say that acceptance tests give developers, project managers and customers a measuring stick for progress. We can measure velocity and similar metrics. Acceptance tests also tell developers that they are done. When the test passes, the developer should not be writing any more code (refactoring / clean up excluded, of course). And, finally, it creates a medium for the idea of a negotiable contract. This is a core XP principle with satisfies the values of communication and adaptability. We can more easily give the customer what she wants.

Another argument is a lack of customer willingness. And, I certainly agree that it is not the customer's comfort zone to be thrown into a tool such as FitNesse. However, I think that this area needs to be very fluid and needs to match the needs of the customer from project to project. I'll demonstrate that fluidity with a couple of examples:

You have a customer that is very knowledgeable about the domain and serves as the project champion for the rest of his company. However, he is neither a business analyst or, in any form, a technically oriented person. In this case, attempt to get him to use Cucumber, FitNesse or Selenium is not going to work very well. However, he will be more than sufficient to help the development team draw up user stories. These user stories can then be (hopefully) quickly translated into acceptance tests in a tool by a business analyst or developer on the development team. At that point you would continue to triage and satisfy as normal.
You have a customer that is able to tackle technical responsibility and can fill the role of business analyst for the team. She sits with the team, understands the high-level technical details of the project and can communicate with developers. In this case, the customer would write the acceptance tests. She can handle the tool effectively and can write effective tests because she understands both the problem domain and the high level technical details of the project. Depending on her effectiveness, a developer or a team's business analyst may need to review the acceptance tests.

Of course, you have to have a full-time customer contact on your team. This is another XP suggestion: pay the contact to be a part of your team from your own contract. That's a tough jump for many organizations. However, if you want the best result out of your project, then having a full-time client contact is essential.

The last argument is that the tools are too slow, too brittle, too end-to-end to be effective. Let me first say that I agree with this, in general. Selenium and cousins are end-to-end web application testing tools that are very, very slow and very brittle. They require specific knowledge to run the tool. They require a full stack to even work. However, this is a problem we can solve.

The Goal: Provide a system-level test that ensures certain behaviors of the system for a user. The Current Solution: Create a user-interface full-stack end-to-end based automated user acceptance test suite. The Problem with the Current Solution: Too slow, Too brittle (End-To-End is not bad in itself, its just hard) Another Solution: Create a user-interface, subset of the full stack, end-to-middle automated user acceptance test suite. Reason Why This Works: Acceptance tests are NOT end-to-end tests. They are ensuring that a system responds to input according to specifications.

There is a solution to this problem, we just need to stop thinking of UATs as "big integration tests." The tools today attempt to solve the big integration test mindset. However, if we just want to ensure behavior of the system from the users' perspective, then we can most assuredly just test the part that interacts with the user. We can create Fakes and Mocks to enable fast, flexible tools that we can rely on to produce the UX that we want.

I'm sad to see the disillusionment caused by misconceptions and lack of imagination among our industry. Hopefully new tools will come out that will resolve these issues and automated UATs (and other forms of ATs) will become as mainstream as regression tests.

Independent Study :: El Fin

2010-02-25T00:00:00-06:00

This will be my last article on Refactoring and Refactoring to Patterns. In this article, I will reflect on what I have learned, the exercises that I performed throughout the term, the books that I read and where refactoring, patterns and design fit into the software engineer's toolkit.

What I have learned

I came into this independent study with a vague understanding what refactoring was. Before the independent study, I was haphazardly and clumsily refactoring my code already: I love the idea of clean, elegant code. This term I learned about how to turn that ad-hoc, unprofessional method of refactoring into a well measured, thought out series of steps and guidelines that will set me on the path to good code.

First, in Martin Fowler's Refactoring, I learned about code smells and how to identify them. While I had heard about code smells, I was not ever able to identify them. Fowler gives a detailed set of instructions on how to identify a set of code smells that are relevant to refactoring. I was able to identify when I was to refactor a section of code rather than doing so in almost a completely random manner.

Fowler then (among other things) gave me a catalog of refactorings that would resolve the issues of these code smells. The catalog brought concrete practices to the abstract idea of refactoring. Putting these specific refactorings to practice and being able to write in my commit message "Extract Method Refactoring" was very satisfying. This, similar to patterns, allowed me to follow best practice examples as well as communicate what I was doing in an ubiquitous language.

Extending on Fowler's book, Jay Fields' Refactoring: Ruby Edition provided insight into how to do refactorings in a dynamic language such as Ruby. He also included several nice rubyisms that increased readability of the code in that specific domain (the Ruby language).

Finally, Josh Kerievsky's book extended the idea of refactoring further into using refactoring as a tool to reach patterns. This was a huge connecting piece in my understanding of where refactoring fits in the process of writing code (I'll get to that later). Even though he was introducing me to an abstract idea, his catalog of examples provided some concreteness to the otherwise abstract idea.

Exercises I performed

In the first 6 weeks of this term, I was reading either Refactoring or Refactoring: Ruby Edition. For these books, I was able to pick up a random open source project and refactoring that project. The projects that I refactored include Android, Thingamablog, Redmine and OJMoq. This was very good practice for myself and I learned much from these exercises.

In the last 4 weeks, however, I was unable to think of a way to practice the larger refactorings seen in Refactoring to Patterns. These refactorings are much more opportunistic and you must be familiar with the code before being able to perform those large refactors. So, I posted less and created a thought experiment on JUnit.

If I were to suggest exercises to someone, I would definitely suggest practicing the basic, small refactorings that are presented in Refactoring. Any code, whether you wrote it or not, is a good candidate to apply these principles. For the large projects, I would say that we should all revel in potential opportunities. If you see an opportunity in your code base to perform one of these large refactorings, create a branch and go for it. If, in the end, it isn't more elegant then you can scrap it.

Books I read

Refactoring: Improving the Design of Existing Code by Martin Fowler
Refactoring: Ruby Edition by Jay Fields
Refactoring to Patterns by Joshua Kerievsky

I would recommend each of these books to people. Obviously, they should start with the canonical resource, Fowler's Refactoring. And, as you would expect, Jay Fields' book is best made for people working with Ruby code. Kerievsky's book is a great one for developers looking at the big picture and connecting the dots.

Where Refactoring, Patterns and Design Fit

Last summer I took a class sponsored by my employer where we studied and discussed design patterns. It was very informative and engaging and I came out in love with patterns. However, it quickly came into conflict with test driven development, continuous design and other extreme programming practices that I've come to embrace.

One of the practices of extreme programming is refactoring. I've studied refactoring extensively this term and I think that it is an essential part of development. There is the Red-Green-Refactoring methodology of TDD and applying refactoring in that manner is very useful. However, if we apply the philosophy of TDD and "Do whatever works to pass the test" style of development, is there any room for patterns and architectural design?

At first, I would have said, "No." Though I was uncomfortable with that answer, there didn't seem to be a way to fit large design and patterns into the TDD method. After studying refactoring, however, I've learned that architectural design and patterns are very much a part of the TDD method and are embedded in the Refactor step. Instead of focusing on the immediate area during the refactor step, we should be looking at the entire application. Was it painful to add this feature? Refactor. Can the interface for the object be improved? Refactor. Does this object have feature envy? Refactor.

This big leap came from Refactoring to Patterns (and a little from Refactoring). Being able to refactor to patterns is the way that developers can create flexible, maintainable, bulletproof architectures. Instead of doing BUFD (Big Up Front Design), the developers can dive right in and start satisfying features on Day 1. This, however, does not give license to developers to never think about design. Design needs to occur continuously. Good developers will know how to create good design. They will be both opportunistic and rigorous in their design practices; most of design, however, will happen during the refactoring stage.

If we apply this idea that design occurs during the Refactor step, then I would postulate that the Refactor step is by far the most important step in TDD. More time and focus should be applied in this step than the other two combined. Refactoring is the way that you're going to make sure that your system is flexible and maintainable for the future. Refactoring is the step that you make sure that other developers can read your code. Refactoring is the step that reduces pain and increases velocity. Refactoring is the step where you introduce ubiquitous pattern languages into your system.

So, where does refactoring fit into the developer's tool belt? For extreme programming practitioners, it may be the most important part of development! Of course, you'll need to be familiar with conventions of the language, how to most effectively expressed what you are doing, design pattern languages, the code you are working with, etc. Refactoring may be the most difficult and interesting part of software development.

Cappuccino :: OJAutotest

2010-02-24T00:00:00-06:00

Here is an introduction video to OJAutotest. I've been working on this the past couple of days in order to make my life easier (I'm about to go into super-testing mode for our application).

OJAutotest Demo from Derek Hammer on Vimeo.

The important part to know here is that this is very much a convention-based tool. Your tests must be in Test and you must run ojautotest from the top level. If you are interested in using this tool, it is available on GitHub.

Industry Composition :: Women

2010-02-12T00:00:00-06:00

I have been mulling this post over for a while now and, due to an inability to sleep, I finally have some time to put my thoughts down.

In the software development world, there is a serious shortage of women in the computing industry. Women only account for about ten percent of the computing industry. Contrastingly, in life sciences such as medicine and biology, women make up about half of the industry! We, as a profession, are losing out. We, as a profession, are missing a huge opportunity. We, as a profession, need to do something about it.

The Opportunity

Some people may ask: so what? Many would respond with something akin to it being a social injustice or values of a male-dominated society. And, certainly, those play a part in the overall picture. However, that isn't why I am so concerned. For the record, I believe there are some places in our industry that do participate in social injustice but that the majority of people and companies in our industry are not so afflicted.

Instead, the main reason for my concern is that women provide a diversified workplace. When I work with someone, I do not want to work with a Yes-Man. I want someone that has a different perspective about code, engineering, life, etc (to a point). I want to work with someone that challenges my ideas, that forces me to revisit and think through my assumptions. That is only achievable by putting people of different backgrounds together in the same room.

It is unfortunate that when people think about diversity, many of them think about skin color. How many African-Americans do you have in your employ? How many people from China, India and Japan do you work with? Instead, diversity is about experiences. I cannot think of a more divergent experience than that which exists between men and women. These differences in experience are universal across the world, across racial boundaries, across cultural boundaries. Women and men really are from different planets and we need both.

That is not to say, however, that cultural, racial or regional diversity is to be discounted. Instead, we should embrace them, as well. The more diversity, the better.

The Good News

There is some good news.

Several universities realize this lack of women in our industry and are implementing programs to encourage women to get degrees in computing. According to the Department of Labor, the number of women expected to graduate from computing programs is expected to increase dramatically by the end of this year. However, they do not predict that there is any increase in percentages of women in those graduating classes. These universities are pursuing the right course of action by creating program in order to increase participation of women.

Unlike other industries that require higher education such as law, medicine and life sciences, the salary disparity between men and women is almost miniscule. Women make under two percent less than men, on average, in our industry. That is truly remarkable when compared to the salary disparity between men and women in the medical industry: an astonishing 21% (in industry). It seems to me that when a woman decides to become a member of the computing industry, she is respected more than she would be in other industries. Awesome!

The Bad News

Universities are not doing nearly enough. Universities should sponsor aggressive campaigns to recruit women. This means creating local computing competitions, hiring more women in computing as speakers for events, advertising the positives about the computing industry (like the comparison to the medical industry above), etc.

Companies are not doing nearly enough. Many companies do not and never will care about the advancement of our industry as a whole and that is entirely their loss. However, responsible companies need to realize that gender diversity is a serious issue and that they, too, need to actively support it.

Conclusion

I know that my professors, especially Sriram, are going to disagree with me on the fact the universities are being passive about this. He actively participates in a group that is devoted to increasing gender diversity in our profession. The invited a professor from another university who is a specialist in this area to visit and give suggestions (She was wicked awesome, by the way. I wish I remembered her name.). However, the graduating class of 2010 at Rose-Hulman Institute of Technology has one female in the Computer Science and Software Engineering degree programs. One!

Sources

I pulled from the following sources to make this article. I didn't make these numbers up, I promise!

http://www.dol.gov/wb/factsheets/hitech02.htm
http://www.bls.gov/oco/cg/CGS033.htm
http://www.medicalwomensfederation.org.uk/Campaigns/Pay%20gap%20report.htm

Independent Study :: JUnit Analysis

2010-01-29T00:00:00-06:00

For this week, I'm going to be analyzing two things: the first part of Refactoring to Patterns by Joshua Kerievsky and the code of JUnit. This is a very interesting reflection for myself, though it may not be as interesting to readers. I would highly recommend your own such reflections, though.

Refactoring to Patterns

Oh joy! Refactoring; it is sweet music to my passionate programming ears. Patterns; they are what should be put next to "elegant (n.)" in the dictionary. Merging these two things just blows my mind. This book is very dense in insight, knowledge and wisdom. I've had to read the book very slowly, very carefully in order to make sure that I'm not missing anything. It is just that good.

"I'm delighted with the result, and I think you will be too."

Martin Fowler wrote Refactoring; and Refactoring is a really, really good book. As a test of its greatness, it still stands as extremely relevant and powerful 11 years after he wrote it. However, Martin Fowler's understanding of refactoring was still in its infancy when he wrote the book; he was still learning about refactoring and how it should fit into the developer's toolkit. He knew that he had stumbled on to something really cool, really productive, really powerful. He attempted, and somewhat succeeded in laying that out in Refactoring.

He did miss the mark a little, though. He loosely defined how refactoring would fit into a craftsman's tool belt. He didn't seem to have any strong insight or knowledge about that. His book, though awesome, failed to define the Grand Unifying Theory of Everything (in Software Development).

"Now the connection between software patterns and agile development is finally told."

What Ward Cunningham was saying was: Joshua Kerievsky has presented to us a rendition of the Grand Unifying Theory of Everything (in Software Development).

You see, as in ~~physics~~ software development, there has been these two seemingly opposing 'theories' that I knew were a Good Thing™. First was the structure, logical ~~Theory of Gravity~~ Pattern Languages. These patterns seemed to suggest that design first development was the way to go. We could use patterns like lego blocks and put together loosely coupled, high cohesion architectures that just made sense. But, then, there were the other practices that were a Good Thing™ and were at odds with design first development. Test driven development, Agile management, Extreme Programming, ~~Quantum mechanics~~ and refactoring were all crazy, bottom up development practices that declared design first development was not the way to go (Note: When I say design first, I mean architectural design). This two were incompatible.

Along comes Josh and declares that he has the solution. Pattern languages are meant to provide a means of communication, abstraction and goals to refactor towards or to. They are not, instead, meant to encourage design first development (though, under very special circumstances, Josh argues that they are useful in that capacity). Instead of opposing the principles encouraged by Agile and Extreme Programming, pattern languages now complement them.

I realize that I'm gushing here and that my enthusiasm is a little over the top. However, the reason is that I've personally struggled with the fact that I love writing good code and I love writing good architectures. Both of these loves seemed to discourage the other--as if I had to pick one over the other. Josh has shown me a way to resolve that conflict. His way may not be the Right Way but it excites me to no end because no one else has been able to so elegantly and so simply connect those two.

JUnit

Switching gears, I want to take some time to analyze JUnit. JUnit was created by Kent Beck and Erich Gamma, both of which are leaders in the industry that I follow very closely. I expected to open up JUnit and just behold the magic that occurs within its confines. Surprisingly, I immediately started seeing some things that I did not like.

Now, let me be clear; I'm a student of programming right now. I'm still very much learning. I could be, and probably am, very wrong about the following statements. I'm not presuming to know more than Kent Beck, Erich Gamma and the contributors to JUnit. So, take all of this with that statement in mind.

All code can be found on Kent Beck's Github fork.

TestCase inheritance

The first place that I looked at was the TestCase class. It is most interesting to me because, when writing JUnit tests for Java programs, this is the class that a particular TestCase inherits. This all makes sense to me. However, the first real line of interesting code is the class declaration:

public abstract class TestCase extends Assert implements Test {

That is a dense piece of declaration but the part that interested me was extends Assert. That means that each TestCase is a subclass of Assert. The class Assert is, predictably, a class full of static assertions. So, I asked myself, "How is a TestCase a particular type of Assertion?"

First, I argued, there is the fact that on a larger scale you could look at each test case, which is itself a collection of tests, as an assertion. But, this isn't really true. A common test case pattern is to create a single test case for each class in the project: Foo is tested by FooTest. FooTest doesn't come to mind as a particular type of Assert, but rather a collection of Assertions that tests Foo.

So that didn't work. Let us do a mental exploration of what ramifications it would mean to refactor TestCase to NOT subclass Assert. That would result in TestCase having to implement the methods on Assert in order to maintain the current API and that those methods would forward the calls to Assert.assertX(). This would definitely increase the lines of code in TestCase but it would also break the Assert from the highest coupling that exists--inheritance.

Assert API

This may be being too nit picky, but I think that the argument order in the Assert methods is off. For example, consider the following method signatures

static public void assertEquals(String message, Object expected, Object actual)
static public void assertEquals(Object expected, Object actual)

The first signature, I would argue, is less than optimal. I think that in cases like this, where there is a parameter that may or may not be specified, that the parameter is the last parameter. The reason for that is quite simple: if for whatever reason I wanted to specify a message when I hadn't before, I know where to start. I don't start at the beginning, or the middle.. I start at the end. Even more specifically, in many IDEs there is an autocompletion tool. This autocomplete usually filters using type information and parameter orders. In the case of prepending, it is potentially more difficult to see what options are available to you. In the case of appending, the use of a comma will give you nearly all the options for overloading that you could need.

There is another reason, too. This one is a bit more flimsy because it is a personal preference but I think its valid. Parameter ordering is very important and there are a lot of factors that can go into it. Chief among them, though, is the importance of that particular parameter. For example, if I am writing an Assert statement, the two most important pieces of information are the expected and actual values. They are what I truly case about. However, if I'm force to stop my thought process to enter in a message string instead of focusing on what I care about, I become more error prone and/or slower.

Independent Study :: Refactoring OJMoq

2010-01-26T00:00:00-06:00

I'm running about half a week behind on my posts but I hope to be able to catch up this week. For this entry, I have decided to refactor one of my own projects. I actually attempted to dig into my dark, dark repositories of code from Freshman and Sophomore year but the code that I saw horrified me so much that I decided to refactor a more recent project, instead. I may or may not visit those projects next week.

Objective-J

First, I need to give a little lip service to the language that I'll be using. It is called Objective-J and is a strict superset of Javascript. It gives javascript classical inheritance, message passing and some other nifty features like importing. Objective-J is heavily used in the Cappuccino framework and Cappuccino is the main booster for Objective-J. OJMoq is a mocking framework for Objective-J but does not rely on Cappuccino.

My Code

I quickly learned that it is much easier to refactor my own code even months after I had written it. This is because I quickly and easily realize mistakes that I made in code: I have certain habits and tendencies. I can go down a checklist in order to resolve those mistakes I commonly make. Here's that checklist (and, as I learn more, this should grow):

Are there methods that belong in another class? (Feature Envy). If so, move those methods to the other class via Move Method. I tend to be single-minded and attempt to complete the functionality in a single location rather than thinking about the implications of the location of the code.
Is the code DRY? If not, use Extract Method to remove duplication. I can sling code, sometimes. Usually it is when I'm "on a roll." Turns out, however, that whenever I'm on a roll I tend to do bad things: duplicate code, forget to write tests first, create feature envy (see above), etc.
Is there code that really belongs in another class? I'm usually reluctant to break code out into a class unless I'm refactoring or starting a new feature. When I'm refactoring, I really need to investigate the responsibility of the class. If it has more than one, break the offending code out into a new class.
Is my coupling too tight? I fall into the trap of high coupling too often when I'm slinging code (even if I'm doing TDD). The problem is that I approach code very piecewise. When I'm refactoring, I need to look at the bigger picture and the ability for the code to flex, not break.
Am I reinventing the wheel? I do this one way too often. Sometimes when I am hacking away at code I end up doing something that is already done in another part of the codebase or libraries.
Am I encapsulating the data correctly? I'm very guilty of this. Especially with collections. I should be encapsulating the internal representation of the data. This also goes for self encapsulation, though I'm not a True Believer in that.

Then, of course, I would go through the normal code smells run down. Is this method too long? Is this field named correctly? Does this constructor make use of the default constructor? Etc. This checklist, though, is a very targeted means of improving my code. I'm going to try to go through this checklist whenever I write a feature on any codebase from now on. I think that it'll improve my code greatly in a relatively short amount of time. Yay!

Refactoring OJMoq

So, with my checklist in hand and my project selected, I set out. The end result, I feel, is really good. I broke it up into two parts. The first part is my refactorings. The second part is additions of features to the code.

Part One

There are two big refactorings in this set. The first is the extracting of assertion-esque code to the OJMoqAssert class. It only has class methods but the abstraction allows for easy testing and better readability of the code. The second is the removal of feature envy from OJMoq and putting the relevant code into OJMoqSelector. It made both look better. A pivotal commit is the one with the long message. It was where I realized that having OJMoqSelectorA == OJMoqSelectorB not implying OJMoqSelectorB == OJMoqSelectorA was a bad thing.

Part Two

This part may not be interesting from a Refactoring point of view. However, as Martin Fowler points out, refactoring is often the result of wanting to add a feature to a code base. So, I wanted to see how much refactoring would be influenced by my desire to implement new features. It was definitely interesting and a more "real world" approach to refactoring. One thing that I noticed is that I had a decent set of test cases but the refactoring still introduced a few bugs that I had to fix down the road.

OJMoq :: v0.3 Released

2010-01-25T00:00:00-06:00

Version 0.3 of OJMoq was just released. I'm going to be very brief. I'll dig into some of the more technical details in a later blog post because this is part of my refactoring independent study.

API

The API has changed. The following methods have been deprecated:

- (void)expectSelector:(SEL)aSelector times:(CPNumber)times
- (void)expectSelector:(SEL)aSelector times:(CPNumber)times arguments:(CPArray)arguments    
- (void)selector:(SEL)aSelector withArguments:(CPArray)arguments returns:(id)value

and they have been replaced with

- (void)selector:(SEL)aSelector times:(CPNumber)times
- (void)selector:(SEL)aSelector times:(CPNumber)times arguments:(CPArray)arguments    
- (void)selector:(SEL)aSelector returns:(id)value arguments:(CPArray)arguments

While the old selectors still work, I'm not sure if v0.4 will have them. So, you should be migrating these to the new version. I changed these in order to be more consistent and predictable.

Base Objects

In v0.2, I basically rendered the base object (the object that OJMoq wraps) inert. It didn't really do anything. In v0.3, I gave developers the option to make the base object valuable. Before, moq() and moq(@"SomeString") did the same thing. That is not true now. If you pass moq() a non-existent selector, it will eat it (like before). If you pass moq(@"SomeString") a non-existent selector, it will throw an exception. Be sure to only pass objects that aren't dangerous as base objects!

Callbacks

Version 0.3 supports callbacks. Using an api that you would expect (selector:callback: and selector:callback:arguments:) it expects to be passed a single-argument function. The function will be passed an array of arguments that was passed directly as the arguments of the selector you are expecting. For example,

- (void)someAction:(id)someObject
{
    [someObject a:someA b:someB];
}

and the test

- (void)testThatXDoesSomeAction
{
    var target = [X newX];
    var myMoq = moq();
    
    [myMoq selector:@selector(a:b:) callback:function(args)
    { 
        console.log(args[0]); // a 
        console.log(args[1]); // b
    }]
    
    [target someAction:myMoq];
}

(Note: this is a bad test. It doesn't assert anything. Be sure that you're checking something!). Now we can do some cool behavioral expectations. Good. Now lets go test!

Project OSL :: Release 6

2010-01-21T00:00:00-06:00

This has been, by far, our biggest release and this is likely going to be a very, very long post. Sit back and get comfortable. Go get a coffee. I know that I need one for this.

Release 6 was all about adding features. We had (and have) and very solid core of an architecture. We are very comfortable with our code base. And, we feel that we can extend almost anything on top of that. So, we aggressively chose features to implement in this release cycle. Following are those features

Community features
- Implement a simple messaging system
- Implement an inbox for users to receive messages
- Implement a broadcast system for which owners of projects can broadcast messages to users
- Implement a commenting system for both commenting on line items and projects
Projects should be tied to users
- Users should not be able to localize a project that isn't theirs
- Users should be able to create their own project based on another's
- Users should not see other projects in their project list
- Users should be able to search a list of projects
The login window needed to be refactored
The menu bar needed to be expanded. This also added the requirement for context sensitive menu items
Resource bundles of a language needed to be selectable in some way, instead of showing resources for all languages at one time
- Users should be able to add a language to a localization
- Users should be able to delete a language from a localization

That list is pretty big. Surprisingly, we finished almost all of it. In order to accomplish this, the size of our code base doubled and several major refactorings were carried out. We completely ripped out the bottom end of our application and replaced it. We overhauled the controller substructure. We changed how the sidebar worked. We refactored to reduce the size of our setup (AppController). We fixed several bugs. In summary, we did a lot.

Hey, Hey Gooodbyeee

CPKeyedArchiver is a way to serialize objects in Cappuccino. It works very well and we've never found an error in the code. However, our dream when we first started to serialize objects was to be able to serialize them into JSON. The reason for this is that CouchDB, our database backend, stores data in JSON format. We couldn't get a JSONArchiver implemented in the beginning so we fell back to CPKeyedArchiver. It had served us well.

But, time to wave goodbye to our good friend CPKeyedArchiver and welcome in the latest and greatest. Chandler and I (mostly Chandler) worked for about a day and a half to get this working, but we were finally able to produce CPKeyedJSONArchiver. Gasps, awes and the look of wonder should now blanket all of your faces, right? Well, you probably don't really know how awesome it is. But it is awesome, I promise. It will take our Model layer and serialize it to JSON. We can then take that JSON, send it over the wire to our Database, and our database stores it natively. This means that we can now pick apart our Models in our database very similar to the way we could in code. It is also smaller (likely due to the fact that CouchDB stores JSON natively and handles JSON better) and will allow us to use views in our database. Then, we can get the data over the wire, unarchive it without any crazy unarchive, interpret, separate and evaluate chain. Instead, our objects come out squeaky clean in plain text format rather than an archived format.

Speaking of.. views?

This allowed us to use CouchDB as more than just a datastore. In Couch, there is a concept called views. These views are created by using a map/reduce combination on the databases inside of Couch. So, for example, I could create a listProjectNames view that would map each record to a key/value pair.

function(doc)
{
//        key          value
    emit(doc._id, {'name':doc.Name});
}

And, now, we have a view. This view is much faster than pulling the data down for all of the projects and then getting the names for them. So now we have the power to do just about anything with our data with minimal cost. This is some really cool stuff--and I have yet to see any SQL or SQL-like syntax in this project (Now if we could only find a way to rid ourselves of the PHP).

The Community

The Community featureset was a big test for our code base. If we had made a solid, loosely coupled architecture, adding a Community shouldn't be too awful difficult. It turns out that, while we had underestimated the time it would take to create the community features, our framework didn't even blink under the pressures of all this new code.

First, we implemented an Inbox / Detail View system for the mail. We followed the architecture that came before in the Project views and quickly came to realize that we did it stupidly before. Not to be discouraged, Kyle trucked ahead and got the Inbox and Detail View working. He then subsequently implemented the messaging system whose architecture is pretty rough but stable, at the moment. We hope to revisit this in RC7 (in fact, it is already assigned to Chandler).

Users, Projects, Oh my!

In the beginning, we created the idea of a project. The "Project Metaphor," as we called it, was a serious part of the workflow process. Now that we are somewhat happy with that process, we need to make it work in the larger context. This was surprisingly easy. We just needed to attach a user to a project and then create the appropriate controller logic to remap the user interactions. I was really happy with how easily our architecture handled this. There was not much that needed to change in the model layer or view layer. The model layer just needed to be able to get users associated with a project. The view layer didn't change except for features being added (such as the project name, owner name and resource bundles in the navigation bar).

REST API: Reloaded

I mentioned previously the REST API that exists for our application. This API was pretty simple because our database was a simple datastore. But, now, we can use the full power of CouchDB because we store the data in JSON. Cool. Let us see how that works

GET    /api/project/all_docs       list of all projects
GET    /api/project/ID#########    one project
POST   /api/project/ID########     update project
PUT    /api/project                create project
DELETE /api/project/ID######       delete project

the new API looks something like

GET    /api/project/_design/views/find                 list of all projects
GET    /api/project/_design/views/find_by_name/KEY     all projects that match KEY
GET    /api/project/ID#######                          one project
POST   /api/project/ID#######                          update project
PUT    /api/project                                    create project
DELETE /api/project/ID#######                          delete project

which, for the most part, is very similar. However, how to get a list of projects is very different. Also, we can create views that are keyed by name, id, whatever. This is a very powerful API. For release 7, however, we plan to simplify things and attempt to turn /api/project/_design/views/find into /api/project/find for the API (using PHP to translate from our API to Couch's API).

Menu Bar, Login, User Experience, Oh no!

We're starting to move away from "Really Cool Concept" land to "Practical Application" land. And, when we move from the first to the second, certain things become oh-so-important. Like the CEO of Twitter, Evan Williams, recently said, "User experience is everything." OSL is starting to look more and more like a complete, functional desktop application. This is a good thing, we were wanting to get here. Now, we have to focus on the tiny details that concepts can ignore but applications need to make great. Things like the menu bar, how the application reacts to users whom are not logged in trying to do things, how many clicks or actions it takes to get to a certain location, how much overhead is in it for the user, etc.

This is really exciting for us because it is a situation that we haven't had a lot of experience in. Typical school projects hardly ever run longer than 10 weeks and 10 weeks is enough to create a "Really Cool Concept" but nothing of a "Practical Application," especially considering the skillset and time constraints on students. We've been working on this since last fall and we're moving into unfamiliar, really cool territory. We hope to get it right, but learning is a really good objective here.

Metrics

I could go on and on about this release. This post is long enough already. So I'll get on to doing the metrics.

Release LOC  Classes     Files   Methods     Tests   Methods/Class  LOC/Class    Tests/Method   LOC/Method
1       1424    27          32   69             21   2.56           52.7         0.304          20.6
2       2551    35          39   147            26   4.2            72.9         0.177          17.4
3       2578    37          40   159            38   4.3            69.7         0.239          16.2
4       3436    43          47   216            46   5.03           79.9         0.213          15.9
5       3813    53          58   220            87   4.15           71.9         0.395          17.3
6       7577    84          87   450            171  5.36           90.2         0.38           16.8

As I said before, our code went WAY up. This is due to new features, not a ballooning of our core architecture. This is expected. This is the same for the Classes, Files and Methods metrics.

We added a bunch of tests but couldn't keep up with the features. I think we have a lot of very good tests, but that there are a lot of methods that aren't worth testing.

Methods per Class went up, but I think this is just because of the adding of features. We'll need to monitor this metric to make sure it isn't jumping too high. Same thing for the LOC per Class.

Finally, you can see that our methods are getting smaller (read: better) despite all of the code we've written. We still need to test more but this is a positive sign.

Independent Study :: Refactoring Redmine, Part 3

2010-01-20T00:00:00-06:00

This week (or, rather, last week) I finished Refactoring: Ruby Edition by Jay Fields and performed some more refactorings on Redmine. I discovered that the second half of the book had more value than the first and that even good code can be improved significantly.

Refactoring: Ruby Edition

After reading the first half of the book, I stated that the book didn't provide too much value beyond Fowler's original. After reading the second half, I would disagree with that assessment. The catalog continually introduced several alternative ways to do something in Ruby that you wouldn't be able to do in Java (the language of the original book).

These alternative methods are actually very powerful when put into practice. After playing with the refactorings, I believe that they give a Ruby developer better tools to refactor and could, possibly, give the Ruby developer the power to create better abstractions that allow for better code.

Some of my favorite refactorings given in the catalog include those that use the Ruby block syntax. For example, the following code

def total
    result = 0
    
    items.each do |item|
        result += item.value
    end
    
    result
end

can be refactored into

def total
    items.inject(0) { |sum, item| sum + item.value }
end

and I absolutely love that. As long as you know what inject means (and, while that's a bad name, there doesn't seem to be a better one), this code makes perfect sense, is shorter and very DRY. I like all three of those. In Java, to do this you would need to create a convoluted anonymous class that wouldn't necessarily improve the code.

I would recommend this book to any developer that is writing serious Ruby code, perhaps even experienced ruby developers. While experienced developers may be familiar with the Rubyisms, this book describes a methodical, structured way to introduce those Rubyisms during refactoring. For people new to Ruby, this is a must; and, for people that practice TDD, this book should be attached to your hip.

Refactoring Redmine, Part 3

This was my last encounter with Redmine and I came away happy with my work. While Redmine has some very good, solid code throughout, there were always areas that could be refactored. This particular area was a very small class file that likely was never revisited after it was written.

Step 1

Extract Methods

http://github.com/hammerdr/redmine/commit/08fb53b9ade8f021b24b1d5fa1f70627048c0970 http://github.com/hammerdr/redmine/commit/fb0cca25319efac87d3408bd45565a1eb2e674b5 http://github.com/hammerdr/redmine/commit/ef921d57bd4228313eeacb284f16978247eea95e http://github.com/hammerdr/redmine/commit/94a8bef751e41bc8e624a0f18cfd6f4d28b5635f http://github.com/hammerdr/redmine/commit/1cea1d0f50ffb65b07628198f7c38e114588630b

Step 2

Introduce Guard

http://github.com/hammerdr/redmine/commit/aebd019fa59d9abd23f6b3b964e45822b9ba2d70

Step 3

Extract Methods

http://github.com/hammerdr/redmine/commit/160d0a0392be269f7a2e043f9aea04470e09e68c http://github.com/hammerdr/redmine/commit/9c6a2c7c449695b9dcc64a0a6d2874c8ebc614a6

Step 4

Introduce Guard

http://github.com/hammerdr/redmine/commit/f5e8311724fa6e708622dfbf1a6250c5582f315a

Step 5

Rename methods

http://github.com/hammerdr/redmine/commit/51805b7d4356ad3ba150d5c3ca2240d4893f5fb1 http://github.com/hammerdr/redmine/commit/84d5285348343425910d9a67e3b8330207c8ae48

Step 6

Extract Methods

http://github.com/hammerdr/redmine/commit/e04e5d5eaeeb4020f02d0a3e7be844218eec7b3c http://github.com/hammerdr/redmine/commit/e04e5d5eaeeb4020f02d0a3e7be844218eec7b3c http://github.com/hammerdr/redmine/commit/7c4c6aa5487c406589675c7d6c64b5c6b4518771 http://github.com/hammerdr/redmine/commit/cbc0603eb08578093c6b941446d2a195ef8bd333

Step 7

Introduce Default Parameter

http://github.com/hammerdr/redmine/commit/31a6ab321640386d15c8c86b8ab6da7a0223a764

Step 8

http://github.com/hammerdr/redmine/commit/62ac466835c3f5b6820e7c52a7805a54717a294d http://github.com/hammerdr/redmine/commit/526d5fed1aa5b2f5319ee43d72d5054ff21d65e3 http://github.com/hammerdr/redmine/commit/b8ccc9807b9f5a9a6f17afc3351558566a6af275

Step 9

Rename Method

http://github.com/hammerdr/redmine/commit/6fe721d81fc0c1f1a07bde2bf56c8abb96d818ec

Step 10

Decomposing if block to guard clauses. I think that this is a great refactor for the arrow head anti-pattern. Quick and easy to get rid of.

http://github.com/hammerdr/redmine/commit/11b7b259c5039ee12a93dbad96d5f1b175efc19b

Step 11

Extract Conditional To Method

http://github.com/hammerdr/redmine/commit/563025817503459e98032596a9d4dcdad0d0963e

Step 12

Consolidating Conditional. I think that this is a really cool, effective refactor. It is obviously a mix of Extract(/Inline) Method and Consolidate(/Break Up) Conditional, but its effectiveness is more than the sum.

http://github.com/hammerdr/redmine/commit/5d755cee63561836a0e711b5d48540532daaa306

Step 13

Rename Method

http://github.com/hammerdr/redmine/commit/fe29e2c88b13125ccad69b968a38ce7c962d1550

Analysis of Redmine Refactor

I felt that this was by far my best refactor to date. The code that I refactored was not terrible. I could read the code without much struggle. However, because I applied the refactoring tool, I think that the code turned out orders of magnitude better. I could now quickly glance over the code and understand what was going on.

Cappuccino :: Using tusk for distribution

2010-01-14T00:00:00-06:00

You're working with Cappuccino or Atlas, and you say something like "I really wish that Cappuccino had something like X." Then, you take the initiative and create X. It's a great idea and you execute it perfectly. It works great on your machine and now it is time to share X with the world.

How do I do that?

Before package management systems existed, the only way to distribute this was to publish your source or a binary, have people stumble on to your site and download them directly. They would follow the instructions and hopefully everything would work out. Then they would have to understand how to import the files correctly and embed them inside of their application themselves.

Along came package management systems such as rubygems and all of this became abstracted away from the user. Instead of installing into the project itself, gems could be installed in a central place on the machine. Users could simply type

gem install X

and X would be installed, usable and all while never having to know how gem works.

So I can use gems?

No, not exactly. Gems, as far as I know, works for Ruby and only Ruby. Don't get too disappointed, though. There is a very good package management system for CommonJS (CommonJS is compatible with Objective-J). It is called tusk. Currently it is an extension of the Narwhal system (if you have cappuccino, you have narwhal; if you have narwhal, you have tusk). And, just like gems you can type

tusk install X

Saweet. Except, you have to do some more work to get that working.

What do I have to do?

There are a couple of things that need to happen. First, you need to make X into a CommonJS package. The package standard was just finalized a few days ago. You can go read the standard and modify your package to meet the standard, or..

tusk init

Yep. That command creates a baseline package for you to work from. Lets look at what that creates:

X/
    /-README
    /-bin/
        -/activate
        -/activate.bash
        -/sea
    /-lib/
    /-narwhal.conf
    /-package.json

Turns out that bin and lib are not needed (unless you want an external command, not a capp library. The bin should contain a shell script to do that if you want). You can remove those. You do need to look at the contents of the package.json, though.

{
    "name": "",
    "author": "",
    "dependencies": [],
    "contributors": []
}

Name is the name of your project ("X"). Author is you. Dependencies for most Capp libraries should be ["objective-j", "cappuccino"]. And contributors are others that have hacked on your code. I also suggest adding "description" and "keywords" to the package description. Now, here is the important part. You must include "objj-frameworks" which should have the following value ["Framework"]. This is telling narwhal where your code is (we'll get to putting your code there in a second). This is OJMoq's package.json for comparison:

{
    "name": "ojmoq",
    "dependencies": ["narwhal", "objective-j"],
    "author": "Derek Hammer",
    "description": "A mocking library for Objective-J that is inspired by the Moq project.",
    "keywords": ["objective-j", "unit test", "testing", "test", "mocking"],
    "objj-frameworks": ["Framework"]
}

Where's my code?

So, we've set everything up for packaging. Except for your code. This is actually quite simple, but there are a few things you need to know. First, let's visit Cappuccino Frameworks. When you import CPObject, this is how you do it:

@import <Foundation/CPObject>

This is because Cappuccino puts the following in the "Framework" folder of its narwhal package

Framework/Foundation/CPObject.j

Do you see the simple relationship? The first part of your library is the "Framework Name" and the second part is the actual file. For most Capp libraries, it'll probably be something like this

@import <OJMoq/OJMoq.j>

So, for your library called X, you should put code in "Framework/X/." Now you're done. You will soon be about to import like this:

@import <X/X.j>

Almost? There's more?

We're almost done. Two more steps need to happen. First, you need to put your code somewhere online in a zip archive. Tusk will download from that location. If you have your code on GitHub, though, you're already set. GitHub can create zip files of you code without any additional setup.

Finally, you need to have your package pulled into narwhal. You do this by editing the sources.json and asking the narwhal guys to pull your change. There are many different ways to fill this part out depending on how you make your zipfile available, but there are plenty of examples in the sources.json and you should be able to figure it out.

Good luck and hopefully we'll see some Capp packages to play with!

Independent Study :: Refactoring Redmine, Part 2

2010-01-09T00:00:00-06:00

This week I spent some time skimming through Refactoring: Ruby Edition and also put some more refactoring into Redmine. The book, unfortunately, is just a rehash of Fowler's original but it does have some nuggets of information. The continued refactoring of Redmine was both interesting and intriguing. I had a few tough decisions to make but I think that, in the end, the factoring of the code was much better than the original.

Refactoring: Ruby Edition

Jay Fields released this book back in the fall and I was really excited to grab it. It was a book specifically about one of the most interesting topics (refactoring) and in one of the most interesting languages (Ruby). I was really excited.

Unfortunately, I found that I've already read most of this book. All of the examples are, of course, in Ruby instead of Java but the examples are almost exactly the same. The catalog has, for the most part, the exact same content (I'll get to some of the differences in a second). Despite all of this, though, it is still a great reference and a great read if you haven't read Fowler's book already.

The first chapter I had already read and practiced. I did this in Chicago at 8th Light with Doug Bradbury. We went step by step through the chapter and recreated the chapter in code. It was fun.

The chapter on testing was significantly different; Jay talked about Test::Unit (and, briefly, RSpec) instead of JUnit. He gave a good overview and stuck to the spirit of Refactoring.

The catalog had a few differences that I should point out (also, I've only read half of the catalog, more next week on the other half). Specifically, Jay expands a few chapters (like Extract Method) to include several Rubyisms. This is nice because algorithms to refactor Java do not always translate completely to other languages (like Ruby).

Jay also introduced a new pattern that is very Rubyist but also very powerful.

Refactoring Redmine, Part 2

This week I'm revisiting Redmine. I really like working with this project because it

Has a lot (and I mean a lot!) of tests. This makes my life easier.
Has well factored code. This makes me really think about the refactorings. In situations such as the Android function, I don't need to think about the refactorings because the code is so bad that almost anything I do is an improvement.
Is written in Ruby and Rails. I just like these two technologies.

So, once again, I set off on a random controller and attempted to refactor. I selected the Project Controller and refactored the first method I saw (ProjectController#add).

Step 1

http://github.com/hammerdr/redmine/commit/3421454a776b838ade1049194469b39585250f19

Extract Method. Nothing special about this one.

Step 2

http://github.com/hammerdr/redmine/commit/efb6bcca1cca4a326fb859cb327f7abd020f1301

Extract method and add guard. This is my really cool Ruby refactor that I love to use. Seems to improve the code a lot.

Step 3

http://github.com/hammerdr/redmine/commit/09d75cb37601ef6b1433a179f7635f402abef6dc

Renaming to make the code more clear.

Step 4

http://github.com/hammerdr/redmine/commit/cc46fc59f81a3809e23943b782371f0dc6895444

Rename Method to follow Ruby convention.

Step 5

http://github.com/hammerdr/redmine/commit/6e348b0f9ec5c2538942f2c1083629d730504db1

Move Method. The comments already stated that they wanted to do this. I agreed. I moved the method to the Project class. I was faced with a tough decision here, though. I could have NOT passed in the parent_id because the params object is accessible in the Model, as well. However, I looked at the method and accessing the params object is not a key element to the method, but is instead just a way to access a parent_id. So, I passed it in as a parameter.

Step 6

http://github.com/hammerdr/redmine/commit/02e2490bdf9bb370d48a1225838448175e6eabc5

Move Method Block. I moved this code outside of the extracted method because it wasn't really doing what the method said it was doing.

Step 7

http://github.com/hammerdr/redmine/commit/39119d1f394009290548f5293ba24d4801a0cc1d

Extract Method. Nothing really special here.

Step 8

http://github.com/hammerdr/redmine/commit/45ace4ddcea8d826212f831ca77b9fff23279dd7

Using guards instead of an if block. If you look at this and think that this wasn't positive, just wait. I end up agreeing with you.

Step 9

http://github.com/hammerdr/redmine/commit/e22895a195947c07a3991d27979de3fdb35404fa

Extracting assignment to caller. I did this so that I could try something. I end up regressing some of this. I don't think that this was really positive or negative.

Step 10

http://github.com/hammerdr/redmine/commit/2532f6c728fb4948d2d094c6a1023bc245e7d7f7

Using a guard instead of an embedded if block.

Step 11

http://github.com/hammerdr/redmine/commit/697a49392eb7de4ea2c8a2abfc570b06b1455585

Reverting my guards and if block changes. The three statements that had the if block just looked ugly. I don't like the if block (too much nesting in this method) but we're still working.

Step 12

http://github.com/hammerdr/redmine/commit/cc6cbfa7e75fcbbe85d46d3d1af1d7bc2fab6994

Reverting the pull out here. Where I wanted to go wasn't going to happen so I reverted part of that change.

Step 13

http://github.com/hammerdr/redmine/commit/7ddcdc7facaca8fd96be78c86c9334716ba2f32a

Getting rid of that nasty if/else block. I don't like the "and return" here.

Step 14

http://github.com/hammerdr/redmine/commit/0088fb804e896f037088b2a2240280d3c51ddc89

Got rid of that nasty "and return" by extract method and add guard. Didn't know exactly what to call the method, yet.

Step 15

http://github.com/hammerdr/redmine/commit/c6a690147cb97c1c66f3321b1822c95a0144ffdd

Rename method.

Step 16

http://github.com/hammerdr/redmine/commit/48ffc423b75e8a1de46dac37dd6fbc7ba3c8a9fe

Encapsulation.

Project OSL :: RC5

2010-01-07T00:00:00-06:00

We released RC5 early last night. The first thing that you will notice is that this release is much more feature anemic than the rest of our releases and, amazingly, the interface looks exactly the same. This is because we worked on most of the features over the holiday break and, because of that, we wanted to take it easy. So, RC5 is about doing things we wanted to do in RC4 but didn't have time to do. This includes:

Default actions for forms such as the feedback form and the login form.
Asynchronous loading from the server so that the application doesn't lock up
Testing! testing! testing!

We also did some research into some other features (such as HTML5 file uploading) that never made it into this release because of road blocks.

Asynchronous loading

While writing the code to make our loading asynchronous, there were only a handful of files that I needed to change. This was amazing to me. I've worked on projects where changing the data access layer from synchronous to asynchronous would bring the entire architecture down. However, our team has been diligent about practicing good design.

Of the handful of files that needed to be changed, none of them was view code. First, I needed to change our data access class (OLActiveRecord) and then, because the API changed I had to go and fix the controllers that were calling the old API. In no time, I loaded up the app to find that it worked flawlessly. Boy did my eyes light up!

Default actions

When Kyle said that committed default actions for all known, programmatic forms, I was glad that someone was able to figure out how and had the time to implement the sure-to-be-crazy amount of things that needed to happen. He had pulled the wool over my eyes. In most cases, it was two lines of standard code and none were more than four lines of code. Kyle has shifty eyes.. never believe a word he says!

Lessons learned

While this was a short release, I still learned some nuggets of information.

First, holiday releases are hard to get through. In part, this is due to the dispersement of the team. In part, this is due to the amazing amount of things that don't get done during the holiday breaks. Our team had planned to release on Sunday but we hadn't put together enough work to make it a viable release. So we pushed it to today and did some more work. Don't do holiday releases!

Second, holiday releases keep the ball rolling. Coming off of Christmas break has been an entirely different experience for our team than Thanksgiving break. This could be due to the fact that we are still in the middle of a term but I think a lot of it is about the fact that we were thinking about our project over break. Maybe you should do holiday releases!

Experimental changes are both fun and frustrating. When all goes well, it is extremely satisfying (asynchronous loading) but when all goes wrong, it is extremely frustrating (file uploading). While the result (nailing 2 out of 3 highly volatile, unknown scope type of features) was really good for our release, I think that throwing in 2 or 3 more stable tasks would have helped the sanity of our team as a whole. This is just a project management perspective on keeping a team together and not necessarily a reflection of the team's skill, but I think that those more stable tasks would allow for developers to stay in the groove while working on those unstable tasks in parallel.

Metrics

Metrics time! For this release, our goal wasn't to add a huge chunk of functionality, as we normally do, but to instead add a couple experimental things and write tests. So, we expected to see our tests per method to go up dramatically.

Release	LOC	Classes	Files	Methods	Tests	Methods/Class	LOC/Class	Tests/Method
1	1424	27	32	137	21	5.07	52.7	0.153
2	2551	35	39	271	26	7.74	72.9	0.096
3	2578	37	40	260	41	7.02	69.7	0.158
4	3436	43	47	338	49	7.86	79.9	0.145
5	3813	53	58	351	92	6.62	71.9	0.262

Our LOC went up from RC4, which is mostly test code LOC.

Our Classes and Files increased, which is mostly test code.

Our tests increased by nearly double, but this surprises me in the fact that while testing I thought that we had covered most of the major areas in the code. The raw numbers of tests is very low compared to the raw numbers of methods. There may be a counting error in the metrics tool (I'll talk to Chandler about it).

Methods / Class went down because of the increase in Classes from tests (this also makes me think we are counting tests as methods).

LOC/Class went down for the same reason as above.

Tests/Method went up, which was a goal, but it didn't go up enough. Again, if my hypothesis is correct, we should be calculating 92/(351/92)=0.355 instead of 92/351=0.262. That is still not great but its a more reasonable number, in my opinion. It would be really nice to have a code coverage tool to see which methods we are failing to test.

Independent Study :: Refactoring Redmine

2010-01-04T00:00:00-06:00

This week I finished reading Refactoring and started to apply some more basic refactorings to Redmine. The end of Refactoring was as interesting as the beginning and Kent Beck's final chapter was really insightful. The refactorings that I applied to Redmine were more good practice at using simple steps to improve the design. Mostly, I focused on composing methods.

Refactoring, Final Chapters

The final chapters of refactoring were interesting. William Opdyke's research into refactoring and his chapter on it had several interesting pointers and thoughts on the practice of refactoring from a research perspective. One really interesting nugget of knowledge is that refactoring is a play on the mathematical term factoring.

A factoring is a way to write a mathematical equation: factoring is applied to equations in order to increase the readability of that equation in that context. For example, Newton's second law is famously known as F = ma. This is correct, but is a factoring that makes it easier to understand in the context of which it is presented (the general public). Newton actually described F = d(mv)/dt or as the rate of change of an object's linear momentum.

Kent Beck's chapter was also very good. Kent expounds the importance of refactoring as a (very powerful) tool in a developer's toolkit. However, after reading a whole book about refactoring and how it improves your code, your efficiency, the world, the universe and everything, Kent says that one of the most important aspects about being good at refactoring is knowing when to stop. Developers will often have an end goal when they refactor and that is wonderful. However, sometimes that end goal becomes too difficult, too costly, too ideal to accomplish and the developer has to make a decision: keep the refactorings or toss them. In general, the refactorings will have improved your code but sometimes the best thing is to just toss them (especially if you were regressing the code in order to make something magical happen).

As I read Beck, I wish I had the confidence to know when to stop and when to keep going. After reading that chapter and going through the refactorings to Redmine, I found myself asking "Is this too much? Or have I just barely scratched the surface?" It was difficult to judge where in the spectrum I was at any given time. Hopefully the rest of my experiences with this independent study will give me some insight into this question.

Redmine

Switching gears, I went to a Ruby on Rails project called Redmine. Redmine is an open source project management suite. I like Ruby. I really wanted to learn how to effective refactor in a dynamic language such as Ruby (hint: its not that much different, you just need *really* good tests). Redmine was a good fit because it had a significant test suite for a significant sized Rails project. What I learned was that Rails doesn't lend itself as easily to the spaghetti code monster that was the Android function, but refactoring can improve the readability of code just as much.

Step 1

http://github.com/hammerdr/redmine/commit/8ea55533023831dc456effc6ff404a47198af4dc

Extracted a Method.

Step 2

http://github.com/hammerdr/redmine/commit/85784bed9fd7009523426d2b5e7b4f2a36589f79

Extracted a Method.

Step 3

http://github.com/hammerdr/redmine/commit/6f7e8c06e96bcd78d9ff2d936ef8abafd2b0fabf

Split a Method. :find_project was doing more than "finding (a|the) project", it was finding a board associated with the project. So I split the method.

Step 4

http://github.com/hammerdr/redmine/commit/954b5a892b20c09a04fa57bd064498123dfa8d63

Extracted a Method. Moving to a different area of the code (was satisfied with the other file), I found the account_controller. There was some duplication so I extracted a method.

Step 5

http://github.com/hammerdr/redmine/commit/f0a6eb6ead70cc4a34dff770bdcadab3eaa85f82

Removed redundant code (Rubyisms).

Step 6

http://github.com/hammerdr/redmine/commit/61ce6a5867ea3182c7a590c103c05f5f7cf31da1

Then I had some trouble. So I talked myself through it and extracted a couple methods to make sure I was sane while I was doing it. My stream of consciousness thoughts are written in comments.

Step 7

http://github.com/hammerdr/redmine/commit/8c8bc970442bc22b26dc5bbb7def854a2ab377a8

Removing redundant code. else -> if can be merged to elsif

Step 8

http://github.com/hammerdr/redmine/commit/5af748768179630e2eb3f8299ddca03045cf304a

Adding guards. I could add guards, remove return statements and make the code read better. Yes, yes and yes.

Step 9

http://github.com/hammerdr/redmine/commit/b2f388aa5a54b6099547ce12e740a8e4b5e330f1

Extract method and Add Guard. This little combination was just cool. I took a long if statement, extracted it to a method with a good name, and used a guard instead of a if block. Much cleaner.

Step 10

http://github.com/hammerdr/redmine/commit/614e83eed41483df7a81380108580a3e5db85ae8

Encapsulating. Just something I forgot to do correctly the first time.

Step 11

http://github.com/hammerdr/redmine/commit/931f1717d71e976109af62edf60a50da0079c721

Add Guards. These guards are more expressive than the guards that originally existed. Cool. I like expressive.

Project OSL :: Release Cycle 4

2009-12-15T00:00:00-06:00

It's been a while. Thanksgiving break and a slow start led us to have a couple of weeks without a release. But, we released a big one. This is easily our largest release in terms of changes to both the code and the users' experience.

Here are the two features that we implemented:

Developer can upload preferred glossaries
Localizers can view glossaries

The glossaries were the first feature that we added that existed off of the main workflow. Given our architecture and practice of refactoring, this was surprisingly easy to implement.

We also fixed some bugs:

Resources context is not preserved (context refers to the location of the resource in the application)
Multiple uploads per session is not allowed
Voting isn't tied to users and users can upload multiple times.
There was no feedback whether uploading worked or not.
Context for localizable strings is lost on importing (context refers to comments on key-value pairs in .strings files)

And we also folded some user feedback into our system:

Are votes supposed to only apply to each line item, or to the file as a whole? I think that might be something to tackle in the next RC. I know that was something we said really depended on the user model being implemented, or at least started. (Caleb Allen)
The submit feedback form should remember the last used email address. Iíve had to enter it each time (even w/in the same session). (Tim Wood)
Iím already ìlogged inî, can you pre-populate the email field in the feedback window with my email? (Caleb Allen)

The big underlying problem that solved many of the bugs and user requests was a project metaphor. In RC3 we allowed users to upload a specific .strings file. We would then parse the file and display that file back to the user. The project metaphor extends and encapsulates this idea. Now, OSL has projects that represent the applications that you have uploaded. Each of these projects has a litany of resources that the users can browse. So, instead of uploading resource files, users now upload .app files and we automatically parse the entire application.

Refactoring

We refactored a lot. We probably refactored too much. Chandler and I went crazy on the refactorings for this release cycle. Much of it was necessary in order to make the project metaphor fit into our application nicely. Some of it was just Chandler and I going a little overboard. Overall, it was a great time. There are a few very ugly parts of our code and we cringe every time we look at it, but most of that exists in view code. The rest of it will just need to be looked at in the next release cycle.

Atlas and Cibs

This was the first release cycle that we made use of Atlas and Atlas' Cib Editor. This was very nice for most of the views but gave us some problems on some of the more complex UIs and options. As Atlas grows and supports more features, we'll gradually move more and more of our UI code into Cibs, thereby abstracting the team from view code entirely. That'll be nice.

BeanCounter and Barista

We also released two more open source projects that have spawned because of our project. Chandler's BeanCounter is an extendable metrics tool that the team uses in order to generate metrics for our project (we'll analyze the data in a later post). Barista is my project that enables BDD style testing of UIs through a script.

Server administrators

I have a newfound respect for them after having to troubleshoot some of the problems that we were getting with PHP and Apache. Respect, dudes. I never want your job.

Lessons Learned

I'm not sure about the rest of the team, but I feel that this was far too big of a release cycle. While I am very happy with the result (I think this is the best release cycle yet), it was hard for us to meet our own deadline because of our inability to estimate the longer release cycle. Also, our project metaphor may not be the way that the client and/or users want the application to work. If this is true, we may need to backtrack and lose 2-3 weeks of work.

Also, refactoring works. There was no better example of this in my experience than with this release cycle. Because we spent a lot of time making the code better, implementing the glossary was almost trivial.

Independent Study :: Refactoring Thingamablog

2009-12-11T00:00:00-06:00

Refactoring, Part 2

I read most of the way through the rest of Fowler's refactoring: almost all the way through the catalog. I don't have much of a response to the refactorings that Fowler enumerates, other than that they all seem to be sound practice. I'll learn more as I put the tools Fowler has provided into use.

One thing that I do know is that I couldn't have done this small set of refactorings without the catalog at my side. I've never been good at memorization (I keep a to-do list for even obvious things like "Eat," "Do Laundry," and "Sleep"). When I sat down to put some elbow grease into the open source Thingamablog (a cross-platform desktop blog editor), I was overwhelmed. The code was a mess (no offense to the authors; to their credit they did come up with a really good Swing interface). There was a mix of Java 1 arrays (String[]) and Java 2 data structures (Vector / ArrayList), there was an odd structure of all the domain logic existing in a nebulous nether-region between the UI and the Domain (and, no, not in controllers.. at least in the modern MVC sense of the word) and there were single letter parameters.

Small (baby) steps

So, I fumbled around the code for about an hour trying to understand the code and its interactions. I was making big design decisions in my head and was really quickly overwhelmed. This was going to take days. I had a deadline that wouldn't accept days. How was I going to do this?

Then, I remembered the advice back in Chapter 2 of Refactoring. Refactoring is about small steps that lead to some eventual larger goal. I didn't need to think about the big things; I didn't need to take days in order to improve the design. I just needed to take baby steps.

Baby stepping

Here's a list of the steps I took. Something that I want to note is that through every step of the way, Refactoring was an amazing resource. I leaned on the Mechanics sections in the catalog. Also, there won't be a lot of code because there is too much. I will provide some UML Diagram where appropriate.

Step 1

I was flying a little blind, so I just chose a random[1] class to start with. I picked the BlogEntry class. I didn't understand what was going on, so I started with some simple name refactorings. I did a couple field renames, quite a few parameter renames and a method rename before I felt like I was comfortable with the class.

Step 2

There was one more refactoring that I could do to the BlogEntry class. There were getters and setters on a collection (Categories). I used the Encapsulate Collection (208) refactoring to hide the implementation of the collection.

Step 3

So, then I wanted to branch out from the BlogEntry[2] and see what classes used it. I found the BlogEntryContainer had a reference to BlogEntry (no surprise there). In the method that I first was shown by IntelliJ I saw a large switch statement that was using object types. I identified this as a code smell and set out to refactor it.

The Replace Conditional with Polymorphism (255) is one of my favorite refactorings. It is a cool, practical application of polymorphism. It is also the entire basis for the Anti-If campaign. While I'm not necessarily an advocate for replacing all ifs with polymorphism, I am definitely anti-switch and would prefer to use polymorphism instead of switches.

Fortunately, the hard work was done for me in this case. The polymorphic classes already existed! All I needed to do was create a method on the abstract class and implement it in all of its children. It worked like a charm. UML Diagrams follow to illustrate.

And it collapsed a ~50 line switch statement into 1 line of code. It always makes me happy to reduce complexity and code at the same time.

Step 4

Next I noticed that there were some preferences (about 8 of them) that were being maintained as private fields in the BlogEntryContainer class. The fields were always requested data from the TBWeblog class (feature envy). I put to work the Move Field (146) refactoring. I moved the eight fields one at a time.

Step 5

Moving the fields didn't fix the problem, though. There was still feature envy in several methods. I used Move Method (142) to move 5 different methods over to the TBWeblog class.

Step 6

After moving the fields over to TBWeblog class, I noticed that there were some more refactorings that I could put to use. I renamed some methods, renamed some parameters, consolidated conditionals and decomposed one of the conditionals. These all were relatively small refactorings that changed the entire look of the application.

Final Words

After all of these steps (and about 3 hours of actual refactoring) I was surprised to learn just how effective all of this was. It reminds me of something I read/heard this summer.

Never let best get in the way of better.

I wish I remember who said that (it very well may have been Jon Fuller) but it is a great quote for programmers. It means that you should never discard making something better because you can't make it the best. For example, if there is a large refactoring that will make the code better but you don't have time to do it right now (maybe even ever!), don't let that stop you from doing it the right way in small increments. That could mean that if you realize a bad design decision (it happens) then you should follow the better design from now on at the least and incrementally improve the existing system.

This week, I almost let that small nugget be forgotten. I almost asked for an extension in order to get this "refactored properly" but instead I went with the "better" instead of "best" and I am still very satisfied with the improvements that I made to the code. There is still a lot of work to be done but that'll have to be for another day, another time.

[1] I actually tried to pick something in the domain model because that is often where class interaction gets complicated. Unfortunately the author chose to follow the "dumb" domain class model where little - to - no logic exists in the domain classes.

[2] I originally wanted to try to pull some domain logic into the BlogEntry class but I never go to that point in the refactorings. It could definitely be done, though.

Independent Study :: Refactoring Android

2009-12-03T00:00:00-06:00

For my first article on refactoring, I'm going to focus on mastering the basic refactoring techniques outlined in Martin Fowler's Refactoring. I'll provide some thoughts and comments on Fowler's book to the point that I've read (about half way) and also perform some basic refactorings on an open source project.

Refactoring, Improving the Design of Existing Code

This book is a classic; it's also pretty old. Fowler published this in 1999. What this book does is amaze me; despite it being very old for the computer industry (10 years!) and despite the fact that Fowler, at the time of publishing, didn't consider himself an expert on refactoring, every single word rings true.

I remember (re-)reading Code Complete 2 over the summer. Code Complete 2 is one of those books that some industry leaders think every developer should read; I am not inclined to disagree. However, despite all of the good parts in Code Complete, there were several out dated, misinformed or just flat out wrong sections of the book (defensive programming comes to mind). When I was reading the book, I just chalked it up to "Code Complete is an ancient book. It was originally published in 1994 and hasn't been updated in about 8 years." I have never had to say anything like that in Fowler's book. Either his book was far ahead of its time (and we are just now catching up) or the refactoring principles discussed in the book are timeless. I'm rooting for the latter.

Here are some quotes, comments and notes that I took while reading the first part of the book:

Refactoring is simple!

I think people see refactoring as a difficult process. Design is hard; so improving design of existing code must be hard, as well, right? According to Martin Fowler, refactoring can be broken down into a catalog of small, simple and "even simplistic" changes. These changes include renaming variables, or extracting methods, or moving methods to more logical locations. From these basic type of refactorings are all other refactorings composed--even a system-wide refactoring of some core part of the software.

Refactoring is continuous design!

By continually refactoring your code, you are practicing continuous design. This means that your design is continuously refined as you learn more of constraints, user needs and the problem domain. This reduces the overhead of "getting it right" up front.

"Before you start refactoring, check that you have a solid suite of tests."

In this, Fowler argues that you cannot do effective refactorings until you have a good set of unit tests. Unit tests reduce the mental overhead for software engineers. When I change one line of code, I don't want to have to think about the repercussions in every other line of code in the program. Instead, I'll let an automated test suite handle that for me.

"Any fool can write code that a computer can understand. Good programmers write code the humans can understand."

Pretty self explanatory; a large part of refactoring is making the code more approachable from a developer's standpoint--not from a compiler's or a computer's point of view.

Refactoring and Performance

Often, you are given a choice between a good factoring or good performance. According to Martin Fowler, invariably you should choose the good factoring. Often, the "good performance" benefit is actually going to hurt the performance of your system because you cannot optimize at higher level of abstractions until you refactor.

Functions belong in classes of whose data they use.

This means that if Function A is in Class Foo but is always using data from Class Bar, then Function A really belongs in Class Bar.

What about composite functions?

What about functions that use data from Class Foo and Class Bar? Fowler suggests that when facing this problem, put the function in the class that is more likely to change. That is, go with the higher volatility class

The Goals of Refactoring

Refactoring, according to Martin Fowler, has several goals and benefits. First, refactoring should improve the design of the code. Fowler alludes to what we now call the DRY (Don't Repeat Yourself) principle and the SRP (Single Responsibility Principle). Recently, Robert Martin has argued that these two principles are the core of good software design. Refactoring improves both!

Refactoring also attempts to make programs easier to understand. Whether it is by good naming conventions, smaller methods, descriptive method names, higher levels of abstraction or refactoring to common patterns, refactoring will increase the readability of code. This means that engineers spend less time figuring out what code is doing and more time adding functionality.

Martin Fowler suggests that refactoring helps you find bugs. In part, this is due to the higher readability of the code. However, the process of refactoring allows the developer to more fully understand the code and its affects.

Finally, Fowler argues that all of this is moot if you can't create functionality faster. He argues that the higher readability, better design and less bugs allow for developers to create software at a faster pace despite spending time refactoring code.<

Putting it to practice

So this is all well and good. I could talk all day about how much of a Good Thing(TM) refactorings are but I need to practice this. So I picked the coolest open source Java project I could think of, Android, and went to work.

Tools

First, I need to talk about how I decided where to even start! I've never seen the Android source code, nor even used Android in any significant fashion. So, at the point, I'm clueless. In this situation, its good to lean on some tried and true tools. I turned to JavaNCSS (NCSS stands for Non-Commenting Source Statements) to get me some metrics. In particular, I was interesting in the Cyclomatic Complexity metric.

I got back over 17,000 lines of data for just the built-in Android apps section of the code. That was too much for me to reliably parse myself. So I wrote a quick Ruby script (it was almost identical to the Weather kata [Kata4]!) to parse this data and give me back some meaningful results.

Cyclomatic Complexity

Cyclomatic complexity is important to me because I am looking for complex methods. These complex methods are usually long and have ugly conditional logic. In order to reduce complexity, I can use the Extract Method technique along with other Composing Methods techniques that are outlined in the Refactoring book.

An Example

Step 1

So, I picked the most complex method in the entire Android apps section. When I finally got to looking at the code, I was lost. The name "deleteInternal" implied that I was deleting something. So, in order to better understand what I was doing, I applied the Extract Method technique on a small, but complexity inducing piece of code. I extracted the code and my tool (IntelliJ IDEA) automatically told me that the code was duplicated in 3 other places. I replaced those areas with calls to my new extracted method (getWhereClause) and with one step I improved the design of not only my target method but of the entire class! Cool!

Step 2

In the next step, I saw the variable tableToChange2 and red flags immediately started to go off. While there was a useful comment near the declaration, tableToChange2 wasn't a good name. I used the Rename Temp Variable to rename tableToChange2->additionalTableToChange. This small change paid dividends in readability.

Step 3

Next, I noticed the huge switch statement. I quickly attempted to figure out what it was doing and decided that this should be extracted. I tried to apply Extract Method on the code but realized that there were too many inputs and outputs. So, I got out the big gun and used Extract Method Object on the code.

Step 4

After performing the Extract Method Object, there was still some code that could be extracted in order to increase the understanding of the deleteInternal method. I used the Extract Method technique on those, as well. Those methods also had implementations elsewhere in the code and I was able to reuse the methods.

Step 5

The final large chunk of code could be extracted as a method, as well. This had the unfortunate side effect of a long list of parameters, but I address this later.

Step 6

In this step, I noticed that there were still a lot of temporary variables in the code. I used Replace Temp with Query in conjunction with the Method Object that I created earlier. This made the code much more readable and reduced long parameter lists.

Step 7 and 8

In these steps I noticed that parameter assignment was happening in deleteRecordsFromAdditionalTableToChange and setIdColumnIfNull. I used the Remove Assignment from Parameters technique and pulled the assignment logic back into deleteInternal.

Result

Nr. NCSS CCN JVDC Function
  35   18   4    0 com.android.im.provider.ImpsProvider.deleteInternal(Uri,String,String[])
  36   21  14    0 com.android.im.provider.ImpsProvider.notifyOfDeletedRows(UrlMatch,int,int)
  37    5   2    0 com.android.im.provider.ImpsProvider.deleteRecordsFromAdditionTableToChange(String[],String,StringBuilder,SQLiteDatabase)
  38    3   2    0 com.android.im.provider.ImpsProvider.appendIfChangedItemIdIsNotNull(String,String,StringBuilder)
  39    5   2    0 com.android.im.provider.ImpsProvider.setIdColumnNameIfNull(String)

  40    5   2    0 com.android.im.provider.ImpsProvider.getWhereClause(String)
  47    5   1    0 com.android.im.provider.ImpsProvider.UrlMatch.UrlMatch(Uri,StringBuilder,int,SQLiteDatabase)
  48    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getTableToChange()
  49    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getAdditionalTableToChange()
  50    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getIdColumnName()
  51    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getChangedItemId()
  52    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getAccount()
  53    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getContact()
  54    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getThreadId()
  55    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isNotifyMessagesContentUri()
  56    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isNotifyMessagesByContactContentUri()
  57    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isNotifyMessagesByThreadIdContentUri()
  58    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isNotifyContactListContentUri()
  59    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isNotifyProviderAccountContentUri()
  60    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isContactDeleted()
  61    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.getDeletedContactId()

  62    2   1    0 com.android.im.provider.ImpsProvider.UrlMatch.isBackfillQuickSwitchSlots()
  63  238  65    0 com.android.im.provider.ImpsProvider.UrlMatch.invoke()

What the result above shows is that the refactoring did improve the design of the existing code. The invoke method is still pretty bad (and the way to fix that is through polymorphism) but the overall complexity has been diminished and it is now easier to update, read and change the code.

The Gist

This is the actual code. This is very long and I apologize for that.

[gist id=247327]

Cappuccino :: Automated UI Testing Concept

2009-11-21T00:00:00-06:00

For the the past couple of days I've been working on a concept for automated testing in Cappuccino. It would replace the need to test applications manually. I'm pretty excited about the concept and think that it could evolve into something really cool. Take a look for yourself!

Automated UI Testing in Cappuccino from Derek Hammer on Vimeo.

Download the .mov file

Objective-J :: Testing Cappuccino UI

2009-11-17T00:00:00-06:00

For those testers of Objective-J, there are difficulties testing Cappuccino. One particularly sticky area is Views. Unit testing these suckers is impossible because the DOM doesn't exist in OJUnit. Not true! We can actually use OJMoq to subvert (oh we're sneaky!) the Views and make them think that they are talking to the DOM.

Note/Disclaimer: I'm not saying that everyone should use this method of testing. In fact, views should be in the form of CIBs and real testing should be with the usability of the product. Put the logic in the Controllers and test those! This is just a "because we can" post and you'll note that I never use it in OSL to test views.

So, I'm going to drop a big view file on top of you. You can really just ignore that, but pay attention to the OLFeedbackWindowTest.j file. It's small but contains important information.

[gist id="237044"]

You'll notice the two lines

CPApp = moq();
CPApp._windows = moq();

at the beginning of that test file. They are what allow us to subvert the views. Two lines and previously untestable areas of your code are now testable! Good luck testing, people!

Project OSL :: Release 3

2009-11-16T00:00:00-06:00

For Caleb's final release, we had a quick, short but really good release cycle. Despite losing Chandler and Kyle for two days during the middle, we got a Release Cycle done in one week. We implemented one feature, fixed a ton of bugs, folded some user feedback (the first!) and did some plumbing. This was our most successful cycle so far.

The feature we implemented was

Allow users to upvote and downvote other user's localizations.

If you're familiar with reading features and creating Domain Models or Use Cases, you know that there are three really big parts to that. The users, the votes and the localizations. We already had the localizations; but we finally implemented the users domain model. I'm pleased that we didn't fall into the trap of focusing on users too much too early.

We also fixed ..a few.. bugs:

Issue 61 ñ Not encoding strings correctly.
Issue 54 ñ Sidebar Navigation Error
Issue 51 ñ Welcome window should have a method to close?
Issue 48 ñ Errors are not properly reported to users
Issue 55 ñ Send Feedback not forgiving
Issue 53 ñ Resource Bundles and Others on Sidebar do nothing
Issue 60 ñ Canít tell which Resource Bundle is being edited.
Issue 52 ñ New, Open, Save all do nothing
Issue 50 ñ Multi-line disclaimer is editable

And we folded in some user feedback (this early?? Yes!):

Issue 47 ñ .strings files are not parsed correctly
Feedback form editable area is not set off
Save As is difficult to get to
Some of the bugs mentioned above

So, yeah, there was a lot done in a short week. We put some time into this sucker. And, I'm proud of it. Everything came together smoothly. We are really getting a handle on this framework and our architecture is really shaping up.

Architecture

Speaking of architecture, we refactored it from a big ball of mud to a nice, structured, delegate-oriented MVC architecture. The controllers are actually layered, as well, based on function. Consult the following graph:

Here we can see our Controller architecture. The AppController also has a reference to MainView which is the window that you see at startup (and will continue to see.. we have a single window app). The BusinessLogicControllers have references to more specific views that go in the Content Area (see our definitions page on GitHub).

This architecture logically separates the controllers into a hierarchy that supports our application. Each controller has a delegate reference to the parent controller so that it can pass information up, if needed. Sometimes this goes to the AppController but not always. For example, a BusinessLogicController can tell the ContentViewController to transition using the TransitionManager.

Sometimes, though, we do need to pass it up to the AppController. For this, we create a message passing system the piggy-backs on the Objective-J message passing (less work for us! Yay!) and the AppController just passes those messages along to the appropriate locations.

Models are referenced by Controllers. We've been tossing around ideas to break this dependence but no good solution has evolved yet. We'll see where that heads.

Error Handling

For this release, we also implemented error handling (told you! Plumbing). None of us really had any good experience with this so we went with a system that handles errors in two different ways.

First, there are application-wide errors, such as database access errors, that are reported up to the AppController which will handle them. Then, there are local errors, such as invalid email addresses, that we handle locally to the specific business logic controller which displays an appropriate UI for that error.

This allows us to have a general error handling protocol and a granular user interface error handling protocol. The general error handling automatically reports to the server which allows the team to debug these unexpected errors. The granular error handling allows us to create a better experience for the users. For examples, users are presented with a pleasant UX message inline for invalid emails rather than annoying alert boxes.

UX Improvements

We did a lot to improve the user experience for our project. When I say "we," I mean Chandler and Kyle did. They both worked on our user interface files and worked to resolve those issues. Chandler created the voting mechanism, the new splash window (which is sweet) and the new editing paradigm. Kyle worked on the new user login and registration windows.

Lessons Learned

We're getting better at this. This cycle was much smoother than the others. We even finished 6 hours early! I think we're getting the hang of this. Expect even more velocity increases in the near future.

I haven't spoken to Chandler or Kyle about this yet, but I think spending more time on the use cases, as a team, helped us define what we wanted for this release. With those goals in mind, we were able to move forward relatively independently (we also pair and work in the same room, though).

The best part is that we are, by no means, slinging code. Our architecture improves every day. We spend time refactoring and reading and understand each other's code. We are, in effect, producing a product that I'm going to be glad to hand off to Omni.

Metrics

Here we go. Did we improve?

Number of Methods: 85
Number of Tests : 38
Tests per Method : .447
Number of Classes: 24
Methods per Class: 3.541

For tests per method (after removing view code--we'll not count that) we can see that it improved but it is still dangerously low. This is because I was too lazy to finish rewriting all of the tests for the Models. I'm working on it. I promise. It's getting better and we're practicing TDD so it should eventually get to >1. Hopefully next release.

For methods per class, it went up. Not a lot, but probably too much for my taste. I think this could be due to our heavy use of instance methods when they should be private methods in our controllers. If I remove that, it goes down to ~75 which is right at 3 methods/class. So, though this increase is alarming, we can easily fix that.

So, overall, we're getting better. We've got some work to do. But, let me tell ya, the architecture just feels right. So from a developer point of view, the hard part is over. Now we just have to design correctly on a business logic and domain logic scale (still difficult!).

That's all for now! Look for a screencast to show off our sweet UI.

Craftsmanship :: Own (and make) your own tools

2009-11-12T00:00:00-06:00

Recently, Jon Fuller (my mentor from last summer) wrote about owning your own tools. He also wrote about how every team needs a toolsmith. Jon's response comes from Robert Martin's assertion that a craftsman owns his own tools. I would argue that every craftsman needs to own and create their own tools.

Own your own tools

Every craftsman should own her own tools. Why?

A craftsman owns her craft. She should not be dependent upon some other entity in order to function.
Her tools are highly specialized and personalized (see next section).
When done correctly, tooling herself with the right tools for her will improve the craftsman's work in both quality and efficiency.
A craftsman should hone her skills outside of her currently project. Specifically, she should work with different material (play with new languages!).

Each of these reasons can be traced back to the craftsman manifesto. A craftsman is a professional and wants to build well crafted software. In order to do this, breaking from the standard mold is required. The craftsman is a member of a community that seek to improve software in general. She can do this by honing her skills and tooling herself with the right tools. More specifically, she can find what works and then shares that information with the craftsman community. And, most importantly, by steadily adding value through responding to change. How can a craftsman add value to the product or to the community if she is not exploring the tools of software engineering?

Tools include everything that can be specialized in some way that could improve your productivity. This includes your development machine and all of the software on it. This includes a place to work at home: a good solid desk to sit at and to hone your profession. This includes personal cloud services such as online backup. This includes books.

"My company takes care of that."

For the most part, developers are bound to their company for many of the tools that they use. Sometimes, there is nothing to be done. Sometimes, you can forego the company sponsorship and get it yourself. Sometimes, developers just want the company to take care of it. The point is that it is always better to have a copy of "Refactoring" by Martin Fowler in your personal collection rather than have it in the company's collection. You can use it across companies (let's face it, the software industry has high turnover) and you can use it for personal development.

"Visual Studio is expensive."

Yes, it is. Tools are expensive. Make sure your company compensates you in some way for your professional craftsmanship.

"I already have everything I need."

Craftsmen are always, always looking for a way to improve their work and that includes there tools.

Make your own tools

Jon argues that every team member needs a toolsmith. He lists several examples of tools that he has in some way written or used throughout a project. From my experience with Jon, I know that he was likely involved in the creation of each of those tools. This isn't because Jon is a "toolsmith." It's because Jon is a craftsman.

When I was young, I spent a lot of time on my grandfather's farm. My father thought that it was a good experience for me (he was right): it would teach me work ethic and problem solving (again, he was right). One of the things that always amazed me about my grandfather was that he was always making something. My grandfather isn't a creative guy--that is, not in the traditional sense. He wouldn't make something because he wanted to; he made something because he needed to. I can pick countless times throughout my youth where I can think of my grandfather welding, sawing, assembling something out of scrap metal in order to help him solve a problem. My grandfather wasn't a toolsmith. My grandfather was a farmer; my grandfather was a craftsman.

As software engineers, we often are so focused on the problem that we cannot see the peripherals. If, for example, we are having a difficult time testing a product, we chalk it up to "the nature of the project." Sometimes we chug along and deal with the pain. Sometimes we ignore the problem. Either way, we lose, the customers lose and the business loses.

Instead, we need to see that if we spent one day writing a little bit of "scaffolding" code, if we assemble together a tool that will solve this problem for us, then many things fall into place. The engineer can now test the product in a pleasing, efficient way. The bugs that would otherwise not have been caught would be. The customers would experience a better product. The business would have a better product to sell. Everyone wins. Just by being able to recognize development pains.

One time, we needed to move some lumber. It was a ways away and in a particularly tight spot. We couldn't drive the big tractor in there. We could drive the ATV in there. However, there was a lot of wood. How were we going to effectively transport it? I suggested to my grandfather that we make a wagon. We did. It was quick, dirty and used some pretty crappy wheels off of some cart-looking thing. We welding some metal together, bolted down a sheet of plywood and attached the wheels. Quick, dirty. It did the job. And more. Today, my grandfather still uses that tiny little cart that he and I made in a little under an hour to solve a particular problem. He found that this little wagon has been very, very useful.

My point here is that sometimes this dirty little tools turn out to be elegant solutions to a problem that often occurs. Your little ruby tool that automates user documentation or starts a build or generates code could apply to everyone one of your projects. That library could then become another tool your ever growing developer toolbox. Special little tools that are as a part of your craft as your development machine.

Objective-J :: Making Use of Autotest

2009-11-09T00:00:00-06:00

I'm a fan of Autotest when I'm doing Ruby development. It automatically runs your tests every time that you make a change to a file. This is handy because

You never, ever forget to run tests after a change.
You save those oh-so-important keystrokes.
You save those precious seconds. It doesn't seem like much, but over the course of a project those seconds add up.
It makes you feel like a cool kid.

However, I'm not working in a Ruby environment for my senior project; we are using Objective-J. The development style, on the other hand, is very similar. The tools are the same: Terminal, TextMate, and Web Browser. So, after manually running tests for a while, I thought "What if?"

So I dug into the source code for Autotest wondering if I could port it very quickly to Objective-J.

Autotest isn't a simple program and porting it wasn't going to be quick. So, I looked further and noticed that I could just use Autotest and run my Objective-J tests using a small override. So, all you'll need to do is drop the following code into your .autotest file in the directory of your source code. You'll need to modify the command to your needs, obviously. We have a RunTests.j file that can be run from Autotest, Rake or Jake.

[gist id="230517"]

Project OSL :: Release 2

2009-11-05T00:00:00-06:00

Today we finished release cycle 2 for Project OSL. The team spent much of Tuesday night and all of Wednesday morning and afternoon putting the release together. It was a particularly difficult release with a lot of dead ends and roadblocks. In the end, though, we made some incredible progress toward our goal in only 2 short weeks.

We planned to implement the following features in RC2

Localizer access to ranked list of localizations
Add support of .strings files
Create user feedback system
Move stack to Javascript**

but we were unable to complete the final feature. That "feature" was something that Chandler and I wanted to do so that we could have a complete Javascript/Objective-J stack. We ran into a slew of problems during prototyping and decided that the time tradeoff was not worth it at the time. Instead, we finished the PHP middleware so that our application could move forward. Hopefully, we will be able to revisit the full javascript stack at a later date.

We also fixed a few bugs:

Modal welcome window not centered
Design issues in resource bundle viewer
Files not filtered in dialog box
Edits to localizable resource not saved
Uploaded resources not saved

Which proved to be a bit more challenging than I had originally thought. Most of the difficulty was fixing these bugs while bringing the application up to date with the new backend architecture.

REST API Now, I get to talk about the cool stuff. We created a REST API. It's sweet. It's also ugly on the server's side. It almost looks like a hack when I read the PHP (and, knowing my PHP skills, it probably is). However, it's slick.

Our API is very simple..

GET api/:model/_all_docs --> List of 'em
GET api/:model/:id --> Give me that object!
POST api/:model/:id --> Update me some data
PUT api/:model --> Shiny new object for me to play with
DELETE api/:model/:id --> No want!

and our only :model right now is "resourcebundle." So, yes, you can actually do a

curl http://shellder.omnigroup.com/api/resourcebundle/_all_docs

and get some cool JSON back.

Okay, so that probably isn't as cool to you as it is to me, but I'm really happy about that. This was my first RESTful API and it seems to have worked out quite well so far.

Anyway, in our application, Chandler abstracted all those sexy GET, PUT, POST and DELETE headers into some ActiveRecord save, list, get, delete methods. Boo! But also very cool.

User Feedback Another addition to our application is the User Feedback submission form. You see, we want everyone to give us feedback. Lots of feedback. And we intended on developing this feature as early, as quickly and as visibly as possible from the get-go. Why?

User feedback is the only way to actually know if your application is usable. Follow as many guidelines, track as many metrics, have as many usability gurus as you want, you will Never(TM) have a good, usable product until you put it in the field and let users hack away at it for a few months. So, this means early betas with quick turn around times on feature and usability bugs. Not only are we doing it early.. we're doing it on Week 4 of development for a project with about 20 weeks of development. That's early.

Inline Editing of CPTableView If you're very familiar with Cappuccino, you know that inline table view editing isn't supported. That is, until Chandler and Kyle rocked it. Go check it out, Cappuccinoians!

Lessons Learned Caleb, Chandler and I had a good discussion about what we need to improve on. In particular, we all agreed that we need to spend more time defining use cases. The reason for this is that when I handed out tasks to the team we spent a lot of time spinning our wheels trying to understand what exactly that task or feature was trying to accomplish. In the last couple of days we found our traction and make some really good progress. But, we had almost a week of little to no traction before that.

In part, it can be blamed on our demanding schedule at Rose. In part, it can be blamed on the psychological effect of a deadline. In part, it can be blamed on this being a very difficult feature set. But, in reality, our team overcomes those with ease. Instead, the fault was with our process. We very loosely define our use cases at the beginning of the project because we want to get our hands dirty in code. Instead, we need to spend more time defining what a user would expect out of a feature so that all of the developers are on the same page.

The decision to drop the Jack server was what made this release a success. Jack would have been a constant struggle for us this entire time because of the lack of support for the javascript server. Instead, we were able to leverage PHP's maturity to our advantage and used several frameworks that help us interface with CouchDB, PLists and .strings files.

Once again, our decision to tackle the riskiest features first seems to be going great. It was an experiment that I wanted to try. Our team is clearly several steps ahead of a traditional senior project schedule. We will have 3 full releases before the end of the Fall term. There is only one other team that is really anywhere close to where we are (tip of the hat to the HiSchool team! Good job guys!) and the rest still seem stuck on wireframing or prototyping or even requirements, still! Some of that is skill level, some of that is work ethic and some of that is process. Our team would be hindered by a process that requires more documentation that what we are comfortable with.

Metrics And, as always, here are some metrics:

Number of Methods: 112
Number of Tests : 26
Tests per Method : .232
Number of Classes: 37
Methods per Class: 3.027

So, the two metrics that we care about are Tests per Method and Methods per Class. Our Tests per Method is abysmal. We need to fix that. We will. It is going to be a top priority for RC3. Our Methods per Class, however, is as close to perfect that I can imagine. This means that we are keeping a relatively good design as our project progresses. If we track the percentages from Release 1 we see:

Number of Methods: +78%
Number of Tests : 0%
Tests per Method : -43%
Number of Classes: +54%
Methods per Class: +15%

where we increased our codebase by somewhere around 60% but only increase our methods per class by 15%. Of course, I want that number to stay somewhere around 3 so I'll be looking for ~0% in Methods per Class for the future!

Project OSL :: Mocking, Cappuccino and You

2009-10-25T00:00:00-05:00

With many software developers now embracing test driven development, unit testing has become a regular practice. Unit testing, however, has a few obstacles. One of those obstacles is how to avoid dependencies and achieve isolation so that your unit tests are not actually integration testsómocking comes to the rescue!

History of Mocking

[caption id="" align="alignright" width="227" caption="Rails has 3 environments: production, development and test. The test environment is maintained in order to isolate testing changes from both production and development areas."][/caption]

In the short history of mocking, there has been quite a progression. The first mocks were real classes with fake data input. This was to avoid having to run through all of your data and save time during testing. This was accomplished by creating a testing database or by creating a general testing environment. This is still used today in integration testing and can be seen in the Rails framework.

But, that was difficult. Developers would have to create and maintain a large supporting structure for their codebase. Changes in development environments would need to be made to test environments, as well. The mental load of testing also increased: developers would need to know the entire dataflow of their application in order to perform testing. There needed to be a better way.

[caption id="" align="alignleft" width="159" caption="An example of "Mock classing" where a developer creates a class specifically for mocking behavior of a superclass and break dependencies."][/caption]

Some developers started creating ways to separate logical parts of the code. They did this by creating ìMockî classes that were subclasses of the classes in between logical sections of code. For example, Class B is the class that we are attempting to test. Class B uses Class A (and A is injected) to talk to the Database.† So, as a good unit tester, we create a test class for B called BTest. However, in order to test some functions of B, A is required. We really donít want to talk to the database for a variety of reasons[1]. These developers would create a subclass of A called ìAMockî that would break the communication between A and the database. Then, BTest could inject AMock into B for testing purposes only.

This was a Good Thing(TM). It made testing easier and now developers didnít have to maintain separate environments. As a tradeoff, the developers did need to maintain an entire mocking library that subclassed their own API. In addition, it made tests look weird. They went from this

A dependency = new A()
dependency.setDatabaseName("Test")
dependency.setDatabaseActive()
B target = new B(dependency)
B.someFunction()

to this

A dependency = new AMock()
B target = new B(dependency)
B.someFunction()

which looks like the developer is testing something entirely different. That was a small price to pay for the isolation that was provided.

But, there were developments in the dynamic language world. People were looking at the above structure and saying ìWell, I donít really need to subclass since my language is dynamic. Instead, I could create a generic mock.î Thus, the modern day mock was born. In Ruby, libraries like FlexMock and Mocha started to appear. Shortly after, mocking libraries like Moq and NMock in static languages popped up. Now, every language seems to have a mock library. These libraries are clever little tools that help the developers test their code. But how do they work?

Modern Mocking Libraries

Modern mocks operate under the assumption that there should be one mock class that mocks everything. You can literally throw anything into these mocks and theyíll break dependencies and create isolation for unit tests. Cool. How?

[caption id="" align="alignright" width="429" caption="An example of how mock libraries might work in a dynamic language. The mock libraries intercept the runtime's call to the method table and handle it in a special way."][/caption]

In most cases, a mock is just a ìwrapperî or an ìadapterî around your object. However, instead of adapting to a specific set of APIs and redirecting them as needed, a mock will intercept all requests to the underlying object. It is best explained using dynamic languages (static languages have different ways of implementing this and they are far more complication). When you make a call to an object, that object goes and finds its method and, using the method signature, marshals the call to the appropriate function. Letís call that function on all of our objects ìfindMethod.î

So, when we want to find ìBarî method on ìFoo,î we can just ask the methodTable for it with the ìfindMethodî call. Itíll return the method in question and the runtime can go about its business. But, with mocks, we want to intercept that call. So we wrap the Foo object with some Mock object that will now be the recipient of those calls from the runtime. The Mock can override the findMethod call and check to see if the Foo class has the method ìBarî with ìhasMethod.î If Foo has a method called Bar, we can return a dummy object. If Foo doesnít have a method called Bar, we can throw an exception. And, this is the case for any instance of Foo, any method on Foo, and any class in this dynamic language. Wow. With one override we seemed to have eliminated all dependency issues that could ever arise in unit testing.

Of course, mocking libraries have diversified and gone on to create new advancements. Most libraries, for example, implement expectations. Expectations allow for developers to test that dependencies are being used correctly in their class. Some libraries allow for calls to be ìpassed throughî in order to maintain some dependencies. Some allow for setting of return values or of throwing exceptions on a method call. The feature list grows every day and each feature is a great tool to creating more effective tests.

OJMoq

OJMoq is a Mock library that I wrote for Objective-J. OJMoq is inspired by Moq, a mocking library for .NET. It implements several of the features of a modern mocking library. It is also very lightweight. It is easy to use and understand.

Making use of OJMoq in Cappuccino

OJMoq was created so that I could effectively write unit tests in Cappuccino. In particular, we are going to have a complex model and controller architecture that requires some good dependency management. So, Iím going to demonstrate some good practices when coding tests in Cappuccino.

Models are the domain of the application and should always have dependencies injected. Here is an example:

 @implementation OJUser : CPObject
    ...
    - (void)initWithConnection:(OJConnection)connection
    ...
    - (void)save
    {
        [connection save:self];
    }

Now, I don't care what connection does when "save" is called. All I care about is that save is called and that I'm passing it the correct object. So, to test this, the following test would be created:

 - (void)testThatUserSendsSaveToConnectionOnSave
    {
        var connection = moq([[OJConnection alloc] init]);
        var target = [[OJUser alloc] initWithConnection:connection];

        [connection expectThatSelector:@selector(save:) isCalled:1 withArguments:[CPArray arrayWithObject:target]];

        [target save];

        [connection verifyThatAllExpectationsHaveBeenMet];
    }

And, now, we have a very effective test that would be ugly and hard to read (and, perhaps, not worth our time) otherwise. Now, in order to test controllers, we have a similar setup:

 @implementation OJUserController : CPObject
    ...
    - (void)initWithViewRepository:(OJViewRepository)repository withContentView:(CPView)contentView
    ...
    - (void)userWantsUserList
    {
        // remove currentView
        [contentView addSubview:[repository userListView]];
    }

If you are familiar with trying to test this sort of controller in ojtest, you know that is impossible. Views interact with the browser and in ojtest there doesn't exist a browser. So any normal test would fail. But, we can actually test this with mocks!

 - (void)testThatContentViewAddsUserListViewWhenUserWantsUserList
    {
        var viewRepository = moq([[OJViewRepository alloc] init]);
        var contentView = moq([[CPView alloc] initWithFrame:CGRectMakeZero()]);
        var userListView = [[CPObject alloc] init]; // I don't care what this is! As long as it is set as a subview.
        var target = [[OJUserController alloc] initWithViewRepository:viewRepository withContentView:contentView];

        [viewRepository selector:@selector(userListView) returns:userListView];
        [contentView expectThatSelector:@selector(addSubview:) isCalled:1 withArguments:[CPArray arrayWithObject:userListView]];

        [target userWantsUserList];

        [viewRepository verifyThatAllExpectationsHaveBeenMet];
        [contentView verifyThatAllExpectationsHaveBeenMet];
    }

And, now, you have a test that verifies the exact behavior of your controller. Mocking allows for developers to quickly and efficiently check the behavior of their code. It makes almost any kind of test possible, as long as dependencies are managed correctly.

Go to GitHub to find out more about OJMoq.

Project OSL :: Release 1

2009-10-22T00:00:00-05:00

We have just finished our first release of Omni Software Localization. The feature set is very basic and what the user interacts with seems very.. rudimentary. This is expected. Here are the features that are currently supported:

Upload Localizable Resources
Edit Localizable Resources (Current Supports PList only)
Select and Download Localizable Resources (Currently Implemented, but not accessible via UI)

This is a small, but high risk subset of our features. We also ran into a bug during development. We were unable to get Apache set up correctly on the Production server. Check @projectosl for updates on when we have fixed this.

In the next release, we are going to implement a user feedback system so that we can get feedback from everyone out there! That release is planned for November 3rd. So get ready to start giving us some feedback!

Lessons Learned

After the first release cycle, we learned a few things. Bugs tend to manifest themselves at the end of a cycle. Automation really does help with pushing to a test server and/or production server. Testing prevents bugs in refactoring. That our time we spent refactoring and cleaning code was not wasteful.

Bugs

We didnít find bugs during development during the first 1.5 weeks. I think it is because of the mentality that ìweíll get to it later.î So, the bugs manifest at the end of a cycle. This is fine as long as we analyze and prioritize these bugs in the next release cycle (as we have done).

Automation

Automation is a life saver. It removes all of the pain and agony that happens throughout development. Our team has scripts for pushing to test server, pushing to production server, automated continuous integration, git scripts, and documentation scripts. These automation scripts mean that we no longer have to think about any of those operations. It reduces the mental load of development and allows us to focus on producing value.

Testing

I think this should be a given, but testing has been a positive thing for our team. We do not feel as if we have ìwastedî time on tests and have ensured that refactoring / adding features do not break existing code. Also, testing helps us manage dependencies.

Refactoring

Weíve seen the effects of refactored code already. The code is easier to read and manage. It also sets up a better design that allows us to add, edit or remove features from the system. For example, in our system, we wanted to have a more Mac-like interface. We changed no code in either the Models or the Controllers and only changed the View class for that particular object. It was a good feeling that we are handling separation of concerns well.

Software Architecture

Our software architecture is starting to take shape. It follows the MVC pattern but is slightly modified. A more detailed version of the architecture will be outlined by Chandler in the near future.

Metrics

The following metrics were obtained from our metrics generating script.

Number of Methods: 63
Number of Tests† : 26
Tests per Method : .412
Number of Classes: 24
Methods per Class: 2.625

Our tests per method is way too low. Weíll hopefully rectify this in the next release. This is most likely due to the fact that a lot of our code is UI code which, as of yet, is untestable.

Our methods per class, however, is very good. It is hovering around 3 which is a good place to be. This means that classes are likely being responsible for only one thing.

Independent Study :: Proposal

2009-10-17T00:00:00-05:00

I know that a few people were interested in how I am going to go about my independent study next term, so I am posting the actual proposal. Also, itíll be a good introduction to the 10 articles that Iíll need to write next term.\

Independent Study Proposal

Adviser: Curt Clifton
Topic: Refactoring and Refactoring to Patterns

Book(s):
Refactoring: Improving the Design of Existing Code by Fowler et al.
http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672
Refactoring to Patterns by Kerievsky
http://www.amazon.com/Refactoring-Patterns-Joshua-Kerievsky/dp/0321213351

Deliverables:
10 Articles (Blog posts) that describe an action taken to refactor a piece of code from one of the selected projects. The blog posts can deal with how the code was identified, why it is bad, the process used to refactor the code, etc. Screencasts / podcasts may or may not be included with each article.

Schedule:
Weeks 1-10:
Due Friday: Article on the topic focused on during that week.

Selected projects:
    insoshi
http://github.com/insoshi/insoshi
        Chosen because it is a known application domain (social networking) and has a significant model / controller layer with 24 models and 20 controllers.
    redmine
http://github.com/edavis10/redmine
        Chosen because it is another known application domain with a larger set of models and controllers than insoshi. It has more than 50 models and has 37 controllers.
    openbravo
http://forge.openbravo.com/projects/openbravoerp
        Chosen because it is a web application, written in Java, that has a deep hierarchy of objects and dependency in the application.
    sharpdevelop
http://www.icsharpcode.net/OpenSource/SD/
        Chosen because of its large code base and because it is written in C#. The cross-project (.dll) dependencies may be interesting to look at.

Possible tools:
    RubyMine
http://www.jetbrains.com/ruby/index.html
        Ruby IDE.
    IntelliJ Idea
http://www.jetbrains.com/idea/
        Java IDE.
    Visual Studio (2008/2010)
http://www.microsoft.com/visualstudio/en-us/default.mspx
        C# IDE.
    JDepend
http://clarkware.com/software/JDepend.html
        Java code analyzer.
    NDepend
http://www.ndepend.com/
        C# code analyzer.
    roodi
http://roodi.rubyforge.org/
        Ruby code analyzer.
    Windows Live Writer
http://download.live.com/writer
        Blogging tool.
    Jing Project
http://jingproject.com
        Screencasting tool.

Independent Study Proposal

Adviser: Curt Clifton

Topic: Refactoring and Refactoring to Patterns

Book(s):

Refactoring: Improving the Design of Existing Code by Fowler et al.

http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672

Refactoring to Patterns by Kerievsky

http://www.amazon.com/Refactoring-Patterns-Joshua-Kerievsky/dp/0321213351

Deliverables:

10 Articles (Blog posts) that describe an action taken to refactor a piece of code from one of the selected projects. The blog posts can deal with how the code was identified, why it is bad, the process used to refactor the code, etc. Screencasts / podcasts may or may not be included with each article.

Schedule:

Weeks 1-10:

Due Friday: Article on the topic focused on during that week.

Selected projects:

insoshi

http://github.com/insoshi/insoshi

Chosen because it is a known application domain (social networking) and has a significant model / controller layer with 24 models and 20 controllers.

redmine

http://github.com/edavis10/redmine

Chosen because it is another known application domain with a larger set of models and controllers than insoshi. It has more than 50 models and has 37 controllers.

openbravo

http://forge.openbravo.com/projects/openbravoerp

Chosen because it is a web application, written in Java, that has a deep hierarchy of objects and dependency in the application.

sharpdevelop

http://www.icsharpcode.net/OpenSource/SD/

Chosen because of its large code base and because it is written in C#. The cross-project (.dll) dependencies may be interesting to look at.

Possible tools:

RubyMine

http://www.jetbrains.com/ruby/index.html

Ruby IDE.

IntelliJ Idea

http://www.jetbrains.com/idea/

Java IDE.

Visual Studio (2008/2010)

http://www.microsoft.com/visualstudio/en-us/default.mspx

C# IDE.

JDepend

http://clarkware.com/software/JDepend.html

Java code analyzer.

NDepend

http://www.ndepend.com/

C# code analyzer.

roodi

http://roodi.rubyforge.org/

Ruby code analyzer.

Windows Live Writer

http://download.live.com/writer

Blogging tool.

Jing Project

http://jingproject.com

Screencasting tool.

Project OSL :: Using Risk for Feature Prioritization

2009-10-09T00:00:00-05:00

Visit Project OSLís wiki here for an introduction to Project OSL.

Project OSL is my senior project. I am lucky enough to be working for a team that I feel very comfortable with and agree with on some fundamentals. However, these fundamentals are good to analyze for our own purposes and for purposes of giving back to the community. In addition, our project may become the model for Agile software development in senior projects. One of the things that we all just naturally agreed upon is that we should use risk to identify feature priority. While not radical to us, this may seem odd to some.

Feature prioritization is a huge part of project management and software development. It is important to analyze what features you should be working on and why. For our team, we have decided to work in short, manageable cycles that go through the entire cycle of software development from defining use cases, feature prioritization, development, testing and publishing. The project cycle should be covered in another post.

So, how do we do feature prioritization? Most teams will start with a basic set of features where they can create a system architecture that they can build modules from. This seems to make sense; the architecture is in place and features can be easily added as ìmodules.î However, this leads to several difficulties and problems. The architecture must be created and defined at the beginning of the project when the developers know the least about the technology, the application domain, limitations, user needs, etc. The goal of the architecture is to create one that could support all of those variables. That is a very difficult task to capture which requires the foresight and knowledge that I nor the rest of my team possess. Instead, the software architecture should be defined as late as responsibly possible.

Instead of focusing on software architecture, our team has decided to focus on features and delivery while reducing risk of project failure. We focus on features because those are the core components of our systemófeatures are what consumers care about. We also focus on delivery. Release early, release often is a philosophy championed by web application giants like Google and 37 Signals. 37 Signals even published a book that covers this philosophy. If you have an interest in web development, you must read Getting Real by 37 Signals. Its free to read online (though I prefer physical copies myself). So, the focus on features and delivery has a compelling case; it even has two industry leaders heralding the philosophy. But, why do we focus on risk?

Reducing risk of failure is a key component of project management. In fact, if project management did not reduce risk of failure for engineering projects, it could be argued that management had no role in engineering. However, it does. Project management attempts to control risk of failure by setting up safeguards such as process, metrics and risk analysis. These are all aimed at two things: reducing risk and increasing productivity. So, our team believes that it is key to identify risks and attack risk as early as possible, reducing the likelihood of failure.

Imagine a situation where there is a component in the system that is 100% likely to fail. In the standard feature development schedule, this feature may be in the 50%-60% range of features that need to be developed. So, the team chugs along until half way through the project they find that a core component to their system is going to fail. Now, they have to do some serious damage control. They could change the architecture; they could change the project scope; they could scrap the project; they could ignore it and hope it goes away. None of these are desirable and none are especially desirable after a company has invested half of a projectís budget.

Instead, imagine if the feature were developed first. It was identified within the first few weeks of development. The developers go to the project manager and the PM makes a decision. Here, his decision is easier, cheaper and less frustrating to the team. The risk is dealt with and the total risk of the project immediately drops. The project is far more likely to succeed.

Obviously, this is an extreme case and it is very rare that a feature is completely doomed to failure. However, if we apply the same principle to high impact, high risk features we should be able to reduce much of the technical risk involved in creating a system.

So, that is how our team operates. We are currently in Release Cycle 1 developing the Top 10% high impact, high risk features of our system. After we finish this cycle, we can re-evaluate the project and take on the next 10% high risk, high impact features. The best part of developing this way is that, as a team, we can feel very, very confident about the success of our project after the first cycle. That is a rare commodity and, with that confidence, I believe we can go on to create a superb product for OmniGroup.

TwitterScrubber :: Case study in refactoring

2009-09-17T00:00:00-05:00

Last week, I wrote a script that would ìscrubî twitter, match the messages based on a set of criteria and format that criteria to a specified output. We needed this script for auto-generating ìEngineering Journalsî for our senior project because our instructor has required it. I wrote the utility and did some basic refactoring until I got to a point where I found a design mistake. This design mistake was something that I needed to rectify but didnít have the time to fix, at the time. So, I accepted the technical debt and decided to revisit it later. Luckily, I am able to revisit it now.

Since bad design decisions are a part of programming and refactoring these designs are good practice for software craftsmen, I decided to chronicle the changes. Here are the tools and technologies that I will be using for this project:

TwitterScrubber, a utility for scrubbing and parsing twitter messages
Ruby, TwitterScrubber runs on ruby
Twitter4r, a popular Twitter API library for Ruby applications (RubyGem)
E Text editor, a clean, powerful text editor for Windows
Git/GitHub, source control system that will allow me to separate the changes from the previous source control.

And here are the things I want to change:

Add unit tests
Refactor Twitter::Client to be injected for testability purposes
[Feature] Add support for XhXm syntax in time parsing

Iím ashamed to say that my first go at this was devoid of unit tests (and TDD, obviously). I kicked myself in the ass for doing that as soon as I realized what I was doing. With TDD, I would have avoided that design mistake.

The design mistake is located in the following code:

require "rubygems"
require "twitter"

class TwitterScrubber
    attr_accessor :formattedValues

    def initialize
        @values = []
        @formattedValues = "Data not found"
    end

    def Scrub(config)
        client = Twitter::Client.new(:login => config["username"], :password => config["password"])
        timeline = client.timeline_for(:user) do |status|
            if( config["comparer"].Valid?(status.text)  )
                @values << { :user => config["username"], :message => status.text, :date => status.created_at }
            end
        end

        return self
    end

    def Format(formatter)
        @formattedValues = formatter.format(@values)
        return self
    end
end

Do you notice it? Twitter::Client is not injected into this class. This makes for some serious testing headaches. First, you cannot properly isolate this class. Also, each test counts as a strike against your total API calls. And, it introduces strong coupling of Twitter::Client and TwitterScrubber. How do we solve this? Well, TwitterScrubber is a business logic object. It brings together several parts of the application (and is, indeed, the focal point of the application) and delegates. Unfortunately, this design attempts to do that and also manage multiple Twitter::Clients. Instead, each TwitterScrubber should scrub what is passed to it.

So, we refactor TwitterScrubber to this:

require "rubygems"
require "twitter"

class TwitterScrubber
    attr_accessor :formattedValues

    def initialize
        @values = []
        @formattedValues = "Data not found"
    end

    def scrub(client, comparer)
        client.timeline_for(:user) do |status|
            if( comparer.Valid?(status.text)  )
                @values << { :user => status.user.screen_name, :message => status.text, :date => status.created_at }
            end
        end

        return self
    end

    def format(formatter)
        @formattedValues = formatter.format(@values)
        return self
    end
end

This allows us to write this test file:

require 'test/unit'
require 'flexmock/test_unit'
require "../TwitterScrubber"

class TwitterScrubberTests < Test::Unit::TestCase
    def test_ThatFormattedValuesIsInitializedToDataNotFound
        target = TwitterScrubber.new

        assert( target.formattedValues == "Data not found", "formattedValues not initialized properly" )
    end

    def test_ThatScrubCallsTimelineFor
        target = TwitterScrubber.new

        client = flexmock("temp")
        client.should_receive(:timeline_for).and_return([])

        target.scrub(client, nil)
    end

    def test_ThatScrubCallsValid
        target = TwitterScrubber.new

        client = flexmock("temp")
        client.should_receive(:timeline_for).and_return([flexmock("temp")])

        comparer = flexmock("temp")
        comparer.should_receive(:Valid?).and_return(false)

        target.scrub(client, comparer)
    end

    def test_ThatFormatChangesFormattedValues
        target = TwitterScrubber.new

        formatter = flexmock("temp")
        formatter.should_receive(:format).and_return("Test")

        assert( target.format(formatter).formattedValues == "Test", "Should have formatted values" )
    end
end

So, now, weíve eliminated the bad code smell and created a better, more maintainable TwitterScrubber. Also, we have tested the class and now I am confident in how my own code works. I donít need to test the Twitter::Client because the Twitter API Ruby library has tests for itself.

Now, Iím free to continue to write tests and add the new feature. The new feature will only modify one class, as was a goal of this sort of design. The new tests will affirm the new functionality and hopefully provide automated tests for any future additions to the scrubber.

Knowledge Web

2009-08-15T00:00:00-05:00

Note: This is a reposting from my previous blog. I'm just "migrating" the posts that I feel a need to keep track of.

As a student at Rose-Hulman Institute of Technology, I am lucky to work with some of the most engaging and brilliant professors in the world. These professors are a great resource for nearly any question and I am hard pressed to find a question that someone in the relatively small department doesn't know. Many of their lectures are littered with information that references colleagues and leaders in the computer science and software engineering world.

In particular, software engineering courses tend to rely on very recent research. Heuristics, models and standards that professors mention are starting to be developed later and later. For example, in my Software Maintenance and Evolution course, Dr. Bohner (Shawn) frequently mentions men and women in industry that have just been recently published on that particular topic. The body of knowledge for software engineering is evolving itself at a very rapid pace. For example, 8 years ago Agile and xPM were non-existent; but, now they are fundamental in Software Project Management.

And, it seems, the information that I am learning today will be obsolete by the end of this year. And, in 1.5 years, I will no longer be a student; I will be working in industry. This concerns me because I believe that I will no longer have the exposure to the breadth of knowledge that I have in academia. Through personal experience, I have found that members of industry are a bit lagged in adoption or creation of cutting edge methods, models and technologies. I have a great appetite for knowledge and I want to keep on the leading edge of the field. After discussing the issue with several professors, I came to the conclusion that there is no "silver bullet."

[caption id="" align="alignright" width="346" caption="An example of how a simple knowledge web might work."][/caption]

I was looking for a blog or a blog roll that would keep me up to date in a relatively short amount of time. However, I found that such blogs are very difficult to find and their value is limited. On average, the blogs that deal with software engineering in detail on a regular basis have post about once every month. In order to deal with this issue, I have come up with the "knowledge web." A knowledge web is a method and model for maintaining and evolving a software engineer's skills and knowledge. As the body of knowledge changes, software engineers should change with it. The model has a set of entities and a set of connections. Entities are people that work in the area of software engineering. Connections are methods of communication between those people. An example of a diagram representing a knowledge web is below: There are several benefits to a knowledge web. In the limited example above, one benefit can be easily seen. I (the figure at the bottom) only know two people and possibly through indirect means (email and blog).

However, I have access to the body of knowledge of 4 other people through indirect means. An example is that I could email a professor proficient in databases and ask him "What would be the best type of database to stored irregular data?" The professor may know and I would get an answer back. However, that professor may not know but knows a researcher that was answering this exact question. The professor either contacts the researcher or gives me contact information.

Another benefit of the knowledge web is that it is self serving. In the example above, if the professor gave me the contact information I would have one more direct member of my knowledge web. This is important because a knowledge web can only go so deep. It would be difficult to imagine a "chaining" of more than 2 or 3 would ever occur. However, by conversing with that researcher and making a direct contact, I have eliminated one link in that chain and giving me more access to different people.

There are some foreseen complications with the knowledge web model. For example, as a member of industry a person may know several direct contacts. However, if those direct contacts within the company only know others within the company, there are no "secondary" contacts outside of the system. This can lead to a separation of this isolated group from the rest of the body of knowledge. So, to mitigate this risk, a web should be made of several people from several different areas (academia, research, industry, etc.) for several different locations (Europe, West Coast, East Coast, etc.).

As mentioned before, chaining will only go so far. So, expectations that knowing "the right guy" that has all of the connections can get you that full body of knowledge is likely not true. Instead, software engineers should build their own knowledge webs that make them "the right guy." I am personally developing my own knowledge web. It is currently very limited but I hope to expand it quickly and thoughtfully. I want to get to know more people in European countries, big companies, west coast companies and bleeding edge researchers. I want to get to know more academics and read my industry leader blogs and attend more conferences. Hopefully, by the time that I graduate, I will be equipped to be better informed that my colleagues in industry.

Also, as an aspiring Project Lead and, eventually, Project Manager, I think that I will employ this as a simple exercise that my engineers perform every so often. The quickest method of doing this is to ask "Whom, outside of the company, do you know in the fields of Computer Science and Software Engineering? Please list name, position, company and company location for each person." With some practice, I am sure that it would be easy to estimate the effectiveness of each engineer's web.

As I have reread this post, I still believe that a "Knowledge Web" is of great importance and I feel that I have really started to build a solid foundation for that. Blogs and books are a key part of that but still are of limited use. Software engineers still need to branch out and be able to pick the mind of the entire industry--not just Scott Hanselman or Martin Fowler or Robert Martin (as good as those guys are).

Software Configuration Management :: Free Softwares Dream Team

2009-08-01T00:00:00-05:00

I have taken interest in software configuration management over the past few months and after going in knee-deep, I found that it isnít easy. I canít go to Stack Overflow and ask ìWhat is the best SCM stack for my situation?î and get a concrete answer. Instead, the ëbestí answer would likely point you to several source control systems (e.g. git, svn, mercurial), several continuous integration solutions (e.g. nant, hudson, cc.net, cc.rb) and several issue tracking systems (e.g. lighthouse, unfuddle, trac, team system). Add into that the mixture of cloud services such as GitHub and Lighthouse along side self-maintained solutions such as an SVN server and Team System, and you have a pretty confusing set of tools and practices that secure, track, maintain, monitor and store your source code. In this murky, unsexy area of software development and software engineering, what are the best tools?

First, let me note that there is no ìbest set of toolsî when it comes to SCM. There are tools that perform better in some situations and there are tools that different developers prefer over another. What Iím going to describe below is a versatile set of Free Software that could potentially solve your configuration management problems. However, I would suggest having at least one SCM expert on your team / in your organization so that you can get the best solution in your context.

Source Control

Quickly, I am going to describe the source control spectrum. There is obviously no source control. This has been called categorically Bad. I tend to agree. There is the manual versioning, manual backup method where you copy the source files to some shared or unshared storage for purposes of ìbackup.î Then, there are modern source control systems. Modern source control systems, despite being modern, have varying approaches to source control. There are systems that control all source from a central location (e.g. Team System). Each file must be ìchecked outî from the server before it can be edited. There are systems that have a central server but allow for local editing and ìmergingî of changes into the central server (e.g. ClearCase, SVN). Finally, there are systems that have a ìlocal repositoryî that sits between the central server and the ability to edit files locally (e.g. Git and Mercurial).

This can be a ìreligiousî subject. Some people love SVN. Some people love Git. Some people donít care. I am a fan of Git / Mercurial due to the fact that each has its own ìlocalî repository. These local repositories allow for personal source control without polluting the central server (read: Continuous Integration server) with your not-quite-done code. However, the nice thing about the tools that I am suggesting is that they are, for the most part, source control agnostic.

In most cases, I would choose Git as my repository of choice. Mercurial is very similar to Git except that its ìcloudî features are not as well developed. Also, when choosing Git, I highly suggest using GitHub. GitHub is a website that will manage your Git repository, for free. It will also allow for the browsing of your source code online without any additional hassle. It is a great way to manage and share your code without having the hassle of setting it up yourself.

In all cases, I would suggest using a modern source control system of some sort. I would explain the reasons, but those reasons have been highly commented on by people much smarter than I. However, it may not be prudent to try to force Git into a project. In particular, I can think of an instance where introducing Git would cause too much friction with a team because theyíve use X Source Control and like it. In this case, noticing that the modern source control system most likely offers almost all of the benefits of Git without causing that team friction would be wise.

Continuous Integration

CI is a concept that has been around for a while but has been slowly sneaking its way into software engineering practice. The term, coined by Martin Fowler, means the automation and notification of build and auxiliary processes at critical points in time. In simple terms, this means that every check-in is built, automatically, and the results are sent to the team. However, it could also mean that one build sets off another upon completion, or that a log file is written, or that a notification is sent to Twitter, or that static code analysis for checking coding standards is run.

Continuous integration is important for several reasons. First, it automates a process. Automation reduces time and error. The more times that a computer sets off the build than a person does, all the more time is saved. In addition, human error is a significant component to error in repeated processes such as running tests or building after a check in. Automating the process will reduce that risk to zero. Also, continuous integration is important for its notification features. Itís important for developers to get instant feedback about how / why they broke the build. This reduces leadup time and leads to a faster development process. Also, it lets other members of the team know why their build isnít working! We cannot discount the importance of auxiliary processes. These are processes that help improve code, transparency, visibility, project management, etc. While these are very ìunsexyî to programmers, it helps with the grunt-work so that actual work can be done.

CI can be very basic or very involved. It can be merely building after each check-in or it could be building, testing, static code analysis, push to another server, packaging, issue tracking management, etc. To this end, I suggest using Hudson. Hudson is a great CI tool that has the ease of use that makes beginners enjoy using but the flexibility and extensibility such that powerusers can mold it to do what they want. Hudson includes a web interface and a plugin for anything you would ever need. It allows you to run build scripts in any language that you could desire and provides some pretty sweet reporting.

Again, I must stress that Hudson isnít a cure-all. Instead, it is a good starting point. For some, CC.NET / CC.rb would be more appropriate. However, I would argue that any team of some size (say, greater than 3) needs a CI solution. CI allows for teams to work more cohesively and at a quicker pace than teams without CI solutions.

Issue Tracking / Project Management

Issue tracking has been a concept that has been around for a while now. Many teams and many projects make good use of issue tracking software in order to catalog, visualize, organize and prioritize various issues and tasks. Primarily, issue tracking has been used as standalone software which only connection to those issues and tasks are through due diligence of the team.

However, with more modern issue tracking software, it is possible to easily integrate with CI servers and source control. Through extensions like githooks, it allows for automatic linking and management of source-to-issue. In addition to better automation / integration, modern issue tracking software now exists in the cloud. Whereas older software platforms, such as Trac, would require the team to manage their own server, the cloud will handle all of that plus give you some additional features.

For a stack based on Git and Hudson, I would highly recommend the Lighthouse issue tracking web application. While there are several good services such as lighthouse, lighthouse provides a couple of key features that I believe set it above the rest. However, if you donít wish to use Lighthouse, Unfuddle is another great website.

The Free Software Configuration Management Stack

So, how would a team go about setting up the Dream Team stack? The beauty is that its very simple. Hardware-wise, you need one machine that sits somewhere accessible to the team. This could be a desktop computer that is hooked into the internet or this could be a VM that sits on a server somewhere. Either way, the barrier to entry is relatively none!

First, youíll need to sign up accounts with GitHub and Lighthouse. Both are free. Both are spam safe. Youíll create a project on GitHub and follow the instructions to set up git with your machine. (Note: For you windows users out there, I highly suggest TortoiseGit) At that point, youíll have a available Git repository. Youíll then want to set up your Lighthouse account and create a new project there. Follow the steps and enter the appropriate information.

Then, youíll need to download and install Hudson on the VM. Hudson does not, by default, support Git. However, if you download the Hudson Git extension, it will handle git repositories just peachy. Hook Hudson up to the public git repository. At this point, I suggest downloading and installing some ìNotificationî plugins so that emails can be sent out and perhaps even a Twitter account can be updatd. Also, I would have each team member install the Hudson Notifier. This is a small project that connects continuously to Hudson and queries the status. That way if your teammate breaks the build, youíll know ASAP.

And, thatís it! Youíre set. No fees. No trial period. Youíve got Free Software at its best! Happy coding!

.NET 4.0 :: Design By Contract

2009-07-15T00:00:00-05:00

A month or two ago, Visual Studio 2010 and .NET 4.0 betas were released to the developer community. I downloaded these betas and have been using them for a while. VS2010 is nice with a much-needed facelift that doesnít compromise the productivity of the developer. .NET 4.0 introduced F# (fsharp) as a first class language on the CLR. However, one of the most surprising, underrated and exciting features introduced in the new development stack was design by contract (or CodeContract).

Introduction to DBC

DBC is a method of writing and verifying development-level specifications. These specifications come in the form of invariants (in other words, statements that must always be true). These invariants allow for developers to strictly define class interactions, method invocations, mutations, etc. In fact, these definitions are so strict that they can be set to cause compile errors.

Imagine, if you will, a simple function that returns the name of the Person object that it is given (please ignore all bad design issues for this portion, for purposes of demonstration):

public string GetName(Person p)
{
    return p.Name;
}

This simple function, without knowing the context, is an unsafe function. The reason is that it is possible to pass in a null reference to the GetName function which will throw a run-time error of NullReferenceException.

This, being a simple function, is simple to fix. All we need to do is check for a null reference and handle it.

public string GetName(Person p)
{
    if(p == null)
    {
        return "<null>";
    }
    return p.Name;
}

However, this null checking can quickly make code hard to read and takes focus away from the actual function of the code. Another issue is that if you never expect null (e.g. have no requirements for the ìnullî case), then you should probably be throwing an exception. However, that introduces run time bugs. In this circumstance, we would normally sacrifice good exception handling and specification adherence to making the code run-time safe (which is a Good thing, overall).

However, with design by contract we can make a simple method specification that requires non null references to be passed to GetName. This requirement is enforced through static and dynamic model checking of the software:

public string GetName(Person p)
{
    Contract.Requires(p != null);
    return p.Name;
}

So, what does this actually do when you used a null reference call? The model checker will throw a warning (can be set to Error, which I would prefer) stating that the requirements for calling this method have not been met. At this point, the developer would need to look up the code itself or some sort of documentation that would guide her in solving the requirements for calling the method.

Advanced Techniques

This is all a very basic example. Following, I was playing around with some of the .NET 4.0 DBC to create a very familiar data structure. This is a simple tree / node structure with some simple insertion / removal rules. I use a wide breadth of the features provided by CodeContract in order to try to show the power of the framework.

Please note that although I have had some experience with DBC, I am by no means an expert and this is my first foray into the CodeContract framework. If any experts happen to read this and see something wrong, please be my guest to offer criticism.

Tree

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics.Contracts;

namespace DBCExample
{
    /// <summary>
    /// Basic Tree/Node data structure to show off Contracts and the
    /// new .NET Framework DBC features.
    /// </summary>
    public class Tree
    {
        public Tree(Node root)
        {
            // Just so that I could show off the invariant, I made root
            // a non-nullable object. While this invariant doesn't hold
            // until after the constructor, invarients are checked from
            // the completion of construction until the object is disposed.
            Contract.Invariant(Root != null);

            Root = root;
        }

        [Pure]
        protected Node Root
        {
            get;
            private set;
        }

        public bool AddNode(Node nodeToAdd)
        {
            // Don't allow null references to be passed to this function
            Contract.Requires(nodeToAdd != null);

            return Root.AddNode(nodeToAdd);
        }

        public void RemoveNode(Node nodeToRemove)
        {
            // For purposes of demonstration, we are requiring that a node
            // be present for it to be removed. In reality, we would want
            // less strict requirements and allow for a non-existent node
            // to be removed.
            Contract.Requires(GetNode(nodeToRemove.Id) != null);
            Contract.Requires(nodeToRemove != null);
            Contract.Ensures(GetNode(nodeToRemove.Id) == null);

            Root.RemoveNode(nodeToRemove);
        }

        // Pure is something that says "Nothing is mutated in this function."
        // This also allows us to use the GetNode statement in the Requires
        // clause of RemoveNode. Nifty, huh?
        [Pure]
        public Node GetNode(int nodeId)
        {
            return Root.GetNode(nodeId);
        }
    }
}

Node

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics.Contracts;

namespace DBCExample
{
    /// <summary>
    /// Basic Tree/Node data structure to show off Contracts and the
    /// new .NET Framework DBC features. Node is where this starts to
    /// get really interesting!
    /// </summary>
    public class Node
    {
        private List _children = new List();

        public Node()
        {
            // This shows off the "sanity check" capabilities of the
            // design by contract framework. Without any other DBC code
            // (other than the Pure tag on Descendents) this invariant
            // could run and hold. This is something that is obvious
            // in the domain and if it actually were to occur would
            // be an immediate clue to broken code. Instead, this
            // guards against it at compile time! Neat, huh?
            Contract.Invariant(_children.Count <= Descendents.Count());
            Contract.Invariant(_children != null);
        }

        [Pure]
        public int Id { get; set; }
        [Pure]
        protected Node Parent { get; set; }
        [Pure]
        protected List Descendents
        {
            get
            {
                List descendents = new List();
                descendents.AddRange(_children);
                _children.ForEach(x => descendents.AddRange(x.Descendents));
                return descendents;
            }
        }

        [Pure]
        internal Node GetNode(int nodeId)
        {
            // Essentially an implication, this states that if the Id
            // exists within the children, then the method should return
            // that child. Otherwise, return true (always true if first
            // condition is false).
            Contract.Ensures(
                _children.Exists(x => x.Id == nodeId)
                ? _children.Find(x => x.Id == nodeId) == Contract.Result()
                : true);
            // Same as above but for descendents.
            Contract.Ensures(
                Descendents.Exists(x => x.Id == nodeId)
                ? Descendents.Find(x => x.Id == nodeId) == Contract.Result()
                : true);
            // What the two statements above describe are the two non-null paths.
            // So, all we need to do is describe the null path.
            Contract.Ensures(
                !(Descendents.Exists(x => x.Id == nodeId)
                || _children.Exists(x => x.Id == nodeId))
                ? Contract.Result() == null
                : true);

            foreach (Node n in _children)
            {
                if (n.Id.Equals(nodeId))
                {
                    return n;
                }

                Node possibleNodeFound = n.GetNode(nodeId);
                if (possibleNodeFound != null)
                {
                    return possibleNodeFound;
                }
            }

            return null;
        }

        internal void RemoveNode(Node nodeToRemove)
        {
            Contract.Requires(nodeToRemove != null);

            // This condition should also propogate up to the tree and be written
            // explicitly on the tree itself. However, because _children and Descendents
            // are private and protected, respectively, there is currently no clean
            // method to access these for DBC.
            Contract.Requires(_children.Contains(nodeToRemove)
                || Descendents.Contains(nodeToRemove));

            // Another very powerful example of DBC. This states that if the children
            // contain the node to remove, then the count will be the number of original
            // children, minus one for the removal, plus the descendents of the removed
            // child up to three. Examples:
            //    1 child : 0 descendents => 0 children
            //    2 children : 1 descendent => 2 children
            //    2 children : 3 descendents => 3 children
            Contract.Ensures(Contract.OldValue<list<node>>(_children).Contains(nodeToRemove)
                ? Math.Min(_children.Count - 1 + nodeToRemove.Descendents.Count, 3)
                    == Contract.OldValue<list<node>>(_children).Count
                : false);

            foreach (Node n in _children)
            {
                if (n.Equals(nodeToRemove))
                {
                    _children.Remove(n);

                    // This is an example of a couple of tools available to the DBC programer.
                    // Assert acts as an in-line check of the current context. So, in this,
                    // we are asserting that the count of the children has been reduced by
                    // one.
                    // We also see the Contract.OldValue<t>(T value) appear here. This is a
                    // way of accessing the value of _children before the method started. So,
                    // we can now check before and after conditionals.
                    Contract.Assert(_children.Count - 1
                            == Contract.OldValue<list<node>>(_children).Count);

                    foreach (Node descendent in n.Descendents)
                    {
                        AddNode(descendent);
                    }
                }
                else
                {
                    // The really cool part about this is that we can assume that this method
                    // is pure. We know that this is not correct, but since this is a recursive
                    // method, we have already proven that the assumptions hold with the above
                    // DBC checks. Awesomeness.
                    n.RemoveNode(nodeToRemove);
                }
            }
        }

        internal bool AddNode(Node nodeToAdd)
        {
            Contract.Requires(nodeToAdd != null);

            // There are some interesting constraints that can be added here. However,
            // they really wouldn't demonstrate anything "new." This level of constraints
            // isn't always bad, either, though. Instead, this may be the only constraint
            // that you want to enforce on this method. Using the least amount of constraints
            // to ensure your functionality should be your goal (exceptions to this guideline
            // there are).

            if (_children.Count() < 3)
            {
                _children.Add(nodeToAdd);
                nodeToAdd.Parent = this;
            }
            else
            {
                foreach (Node possibleParentNode in _children)
                {
                    if (possibleParentNode.AddNode(nodeToAdd))
                    {
                        return true;
                    }
                }

                return false;
            }

            return true;
        }

        [Pure]
        public override bool Equals(object obj)
        {
            return Id.Equals(((Node)obj).Id);
        }

        [Pure]
        public override int GetHashCode()
        {
            return base.GetHashCode();
        }
    }
}

Thoughts

Design by contract is a powerful tool. However, like any tool, developers shouldnít rely on just this one. Instead, this is a tool that should be in the toolbox of developers. In fact, this tool is like a torque wrench of tools: a torque wrench is a tool that is absolutely required in some instances but in most cases isnít necessarily or is even wrong to use. I would suggest that DBC be used in very specific circumstances: critical applications, widely used frameworks, complex mathematical algorithms.

Critical applications are applications that need to be 100% correct (or near enough) such as pace makers. Even among critical applications only essential portions may need to be verified for 100% correctness. The reason for such limited usage in applications is because design by contract software could easily take 3 or more times longer to develop than software that doesnít employ the practice. One-hundred percent verified is an expensive process and should only be used in very specific circumstances.

Widely used frameworks (APIs) should use DBC because it promotes developer safety. The model checking of the framework would be of secondary concern to the verification of input requirements. This would allow for developers to know when they are doing something wrong. This could eventually be used to verify correct usage of certain objects such as a FileStream object that was opened but never closed.

Finally, complex mathematical algorithms are difficult to translate to code correctly. However, invariants provide a very mathematical approve to verifying algorithms and thus could aid in the development of these large, complex algorithms.

Dreams

Design by contract and CodeContract are very immature and underdeveloped at this point in time. Research projects have started taking extreme interest in the subject and hopefully further advancements are coming soon. However, I have a few hopes and dreams for CodeContract:

Ability to expose private instance methods / variables to an external source for the purposes of model checking (See: JML)
Ability to contract interfaces: in fact, this would make much more sense. Instead of contracting the implementations, we should contract the interfaces. This will allow us to swap implementations quickly and easily and still be able to verify that the pre- and post-conditions have been met by the new implementation of the interface.
Stand-alone contract runner. The contract runner in Visual Studio essentially runs whenever you build. In addition, the contract warnings / messages / errors appear in the error list (which is good). However, it would be nice to have a separate view that could run the checker on-demand and also aggregate the contract warnings / messages / errors into one place. Also, if that were to occur, possibly link the message and pointer-warning so that multiple errors have distinguishable pointer-warnings.

I apologize for the long post. However, this is a big area to cover in a single blog post and I didnít want to leave anything out. Conclusion: CodeContract is cool but not yet ready for prime time. Perhaps by the time VS2010 ships MSLabs will have something thatís more professional-level rather than research-level.

Ruby Test Runner

2009-06-29T00:00:00-05:00

For my Peddinghaus IDE application, I wanted to practice TDD. Ruby has a great built in unit testing framework (the unit testing framework that Rails uses extensively). Ruby also has some great plugins for testing. For Peddinghaus, I am using Cucumber and RSpec. These two plugins, along with Test::Unit are the basis for my testing stack (more information on this stack later). However, I really didnít have a way to run these tests effectively. So, I set out to create my own Ruby Test Runner.

Rails is synonymous with Ruby in most peopleís eyes. However, Ruby is a powerful language in itself that has an ever-growing library of tools and plugins that is going to make it a first-class language in the near future. My project is a desktop environment application and thus isnít a Rails app. This means that I donít get some of the built-in test runners that are in Rails.

So, you can either run the tests module by module, testing framework by testing framework, or you can create a script. Some people would prefer to make a sh script that can run in their particular environment. This is fine, but I wanted to see if I couldnít hook directly into the APIs of each framework. Getting into the Test::Unit framework was pretty easy and straightforward while hooking into the two plugins was more difficult. In the end for that, I had to settle for a work-around. Code follows:

# runtests.rb
# runtests.rb
# This file will run the Cucumber, RSpec, Integration
# and unit tests. It will also accept arguments. Examples
#
# irb> require 'runtests.rb'
# irb> RunTests::Run
#
# irb> RunTests::Run("cucumber")
#
# irb> RunTests::Run("rspec")
#
# irb> RunTests::Run("integration")
#
# irb> RunTests::Run("unit")

require 'test/unit/ui/console/testrunner.rb'

class RunTests
  def self.Run(arg)

    unit_suite = Test::Unit::TestSuite.new("Unit Tests")
    Dir.entries("unit").each { |file|
      if file.include? ".rb"
        unit_suite << file
      end
    }

    integration_suite = Test::Unit::TestSuite.new("Integration Tests")
    Dir.entries("integration").each { |file|
      if file.include? ".rb"
        integration_suite << file
      end
    }

    if(arg == "cucumber" or arg == "features")
      system("cucumber features/")
    elsif(arg == "rspec" or arg == "spec")
      system("spec specifications/")
    elsif(arg == "integration")
      Test::Unit::UI::Console::TestRunner::new(integration_suite).start
    elsif(arg == "unit")
      Test::Unit::UI::Console::TestRunner::new(unit_suite).start
    else
      print "Running Cucumber..\n"
      system("cucumber features/")
      print "Running RSpec..\n"
      system("spec specifications/")
      Test::Unit::UI::Console::TestRunner::new(integration_suite).start
      Test::Unit::UI::Console::TestRunner::new(unit_suite).start
    end
  end
end

# TestRunner.rb
# Command line test runner for runtests.rb

require 'runtests'

exit = false

while(!exit)
  print " TestRunner > "
  a = gets.gsub("\n", "")

  if(a == "exit")
    exit = true
    print "Exiting..."
    break
  end

  if(a == "help")
    print "Help for TestRunner\n"
    print "=============================================\n"
    print "run (no arguments) -- Runs all tests\n"
    print "run c(ucumber)     -- Run Cucumber tests\n"
    print "run s(pec)         -- Run RSpec tests\n"
    print "run i(ntegration)  -- Run integration tests\n"
    print "run u(nit)         -- Run unit tests\n"
    print "help               -- This\n"
    print "exit               -- Exit the runner\n"
  end

  if(a == "run")
    RunTests::Run("")
  end

  if(a == "run c")
    RunTests::Run("cucumber")
  end

  if(a == "run s")
    RunTests::Run("spec")
  end

  if(a == "run i")
    RunTests::Run("integration")
  end

  if(a == "run u")
    RunTests::Run("unit")
  end
end

I decided to break it up because I wanted to be able to separate the UI and the logic. This is a very simple case that will likely never be changed, but I may be motivated enough to write a GUI for the test runner that replaces (or supplements) the command like test runner. Or, I could change the business layer and never worry about UI layer changes. I think its more clean this way.

(Note: I realize that there are some bad variable names and inconsistencies [notably a and the long form of the arguments for the test runner] but they will be fixed before it goes into source control.)

As you may see, the workaround for Cucumber and RSpec is to just call the commands themselves. However, Iím sure that there is a way to hook into these testing frameworks and create your own runner (and, if not, that should definitely be added). I could imagine that if I were to put a real team on this project and would want some real results, Iíd have someone put some time into an integrated test runner GUI with an integration server test runner counterpart. For now, the above solution should suffice.

If anyone has information on the Cucumber and RSpec runners, please let me know!

Project #2 :: Peddinghaus IDE

2009-06-25T00:00:00-05:00

For my next summer project, Iím going to step it up a notch. The first project was a simple calculator for Windows using WPF and simple controls. Now, I am going to create an integrated development environment for Ruby. Oddly enough, Iím going to create this application using the Ruby language. I hope to get to a point where Iím going my next version of Peddinghaus IDE inside of an older version of Peddinghaus IDE.

Technologies

The first thing I needed to do, after deciding on what I was going to do, was what technologies I should use. I wanted to shy away from C#, WinForms and WPF because Iíve coded in them previously. Another language I was avoiding was Java because Iíve had experience with Swing before (and, like most people, this was not a good experience). I also wanted to stay away from lower-level languages like C; and, so, that left me with very few options. I knew that Python had the tools necessary to accomplish what I wanted, but for no good reason I like Ruby more than Python. So, I wondered, are there GUI toolkits for Ruby?

The answer was ìNot Natively.î So I did some research and found that Ruby has a bunch of GUI toolkits. There is wxRuby, which is a Ruby port of wxPython. There is Shoes, a streamlined, web-like toolkit. And, finally, there is FxRuby. FxRuby is a Ruby implementation of the Fox toolkit. The Fox toolkit is a C toolkit that is both very powerful and very popular. Another very important aspect of this is that there is a built-in wrapper for Scintilla, called FxScintilla, that will allow me to drop Scintilla into my IDE. No syntax parser, code folding headaches for me. I get to do the fun stuff.

Tools

Iím going to be developing using Ubuntu 9.04 and (at first) gedit. Even though Ruby is cross-platform and the IDE will be able to run on any operating system, development must occur on a Linux box (because the wrappers donít like Windows).

Also, Iím going to try out a beta version of Unfuddle. Unfuddle is a project management / repository service that is pretty slick. There are some user interface tweaks that could be made (in my current limited experience) but it is working really well at the moment. I really donít need the project management features but Iím putting them to use because this is sort of a trial run at this type of software for future team projects. So far the experience is positive.

Yet Another Calculator

2009-06-20T00:00:00-05:00

Iíve finished my first mini-project for the summer. Yet Another Calculator, or YAC, is a simple desktop calculator. I have created a website for the project. In this post, I want to analyze what I did and determine the benefits of my work.

What I Did

I created a WPF application in Visual Studio 2010 using .NET Framework 4.0. I have read about WPF before but I had never created an application using it. So, this was a new experience for me. I started by creating the layout of the calculator (which later changed) without adding functionality.

After creating most of the controls, I decided that I wanted to integrate this application with a F# parser. To my dismay, I found out that .NET 4.0 doesnít come paired with fslex and fsyacc. These two tools are, in my opinion, the most powerful part of the language and the crux of my deciding to use F#. So, I decided to go away from that and create and expression builder / evaluator in C#.

After creating the evaluator, I had a working calculator. After that, I had a few user interface changes that I think significantly improved the program. Additionally, I added ìtip balloonsî that appear the first time that you run the program. I also added the ability to ìshiftî and click buttons. I pulled the results display from the entry display so that they could be kept separate. I also created the About box and the Help box.

Analysis

In reality, if this were a real product, I wouldnít start out with the user interface (unless this was to be a prototype). Instead, the architecture would have needed to be laid out and major decisions about the architecture of the project would be decided. However, since this is such a small project I didnít have an architecture.

Likewise, with a MVC or Code-Behind model, I should work vertically, incrementally adding features. However, on a project I think that this entire project should be considered a sizable feature set that could be implemented vertically. So, I didnít feel too bad about that.

The expression builder / evaluator in C# is very primitive and untested. This is really an inadequate solution to even this project but I needed something quick ní dirty so that I could have a working application. Instead, I would suggest using a tried and true math parser. Iím not sure where to get one but Iím sure it exists somewhere.

The user interface changes at the end of the project really improved the experience for the user. These changes took a relatively low amount of time to implement and refactor. In terms of the application development, those huge improvements took maybe 10% of the project time. With more time, Iím sure the user interface could drastically improve.

Overall, the code behind / evaluator are terrible and not maintainable. Iím pretty happy with how the user interface turned out, even with an unskilled UI programmer at the helm. In terms of project success, Iím pretty happy. With more time, effort and skill, this could be a good model for how to create a better desktop calculator.

Benefits

I gained some experience in UI development, which was my goal for this project. In addition, I learned more about WPF (which was thankfully similar to ASP.NET). I really didnít learn much of anything about C#, though, and my evaluator wasnít anything special. However, I met (really, exceeded) my project goals and that makes this a success for me.

My next project is a Ruby IDE. Iím still trying to decide on language and how I want to proceed with that. My goals for that project include (but are not limited to) making further improvements to my desktop application UI skills / experience and to learn another UI framework other than WPF, Windows Forms and Javaís Swing.

Personal Source Control

2009-06-13T00:00:00-05:00

Today, I was finishing what going to be called Yet Another Calculator v1.0 (more on YAC in another post). I realized that I had failed to use source control during the project. This isn't disastrous. I didn't ever lose any data. My builds all work for my intended purpose. This, unlike most projects, didn't really have an intended audience but was, instead, me hacking code.

However, I still think the project should be source controlled. What if I'm inspecting a particular section of code and I wonder "Why did I do that?" Instead of guessing, the context of when and with what other changes and check in comments should help me decipher why I made a particular decision.

So, why didn't I do it? As Jeff Atwood often says, users should fall into the pit of success. What this means is that, if using source control is a general "success" criterion, then I should have to try to not use it. Instead, source control (and software configuration management, in general) is always difficult to set up and maintain. Should software development suites integrate source control automatically?

In part, this is the fault of Visual Studio, Eclipse, Xcode, etc. However, really I think this problem affects the whole configuration management community. It is pretty awkward to set up Git. It is pretty expensive to set up Team Foundation or Clearcase. SVN is the easiest of these tools to setup and use (and, really, only because of Google) and it still takes some conscious effort (it is entirely possible to have SVN set up and only commit a project once.. without having to make an effort to not commit).

Admittedly, this is usually solved by teaching programmers and engineers the importance of source control. They understand because it saves them from losing their data. However, it still takes effort that really shouldn't be necessary.

And, beyond the domain of programming, document control is nonexistent. Sure, people will regularly back up data, if they know what they are doing. However, this is a process that is both insufficient and difficult. Instead, the operating system should act as a document management system.

With virtually unlimited storage space available to computers, shouldn't document revision histories be kept? This is the sort of automated process that computers can handle without the user worrying about it. If there is a fatal system crash, documents can be automatically recovered. If there is a discrepancy in the files for some reason, change histories could be automatically display to the user at their perusing.

For editing collaborative works, such as projects or large research papers, revision history could be everything. This could be automatically handled without setup or conscious effort. In short, the user would fall into the pit of success. Obviously, more complex and in-depth systems are needed for source control, specifically, and that is a part of developing software. But, instead of making it something that developers need to fight, we should make source control easy and as transparent as possible.

Designing a Calculator

2009-05-27T00:00:00-05:00

If you follow me on Twitter (@hammerdr) or Facebook (hammerdr@rose-hulman.edu) then you may be aware that my summer goal is to become better at user interface design. In particular, I want to become better at Desktop Application GUIs. I believe that the best way to learn is through both study and application. So, Iím starting simple to get the hang of application development. Also, Iíll try different development languages, APIs and IDEs. The first application that I plan to develop is a calculator built in .NET (I think).

A few weeks ago, if someone had asked me if I should be focusing on application development and user interface design from that perspective, I would have said, ìNo.î The reason is that I believe web applications are the applications of the future. With Google Chrome able to download web applications to the desktop and web application frameworks become more and more powerful, the complex web application has become more a routine part of the computing experience. There are several benefits to web applications that I wonít explore, at the moment. So, naturally, I figured that focusing on HTML, CSS, Javascript, DOM, etc. were the future. Since Iím a forward-looking type of guy, I wanted to better myself in those areas. However, a friend and team member of mine, Chandler Kent, showed me the real future of application development recently.

In anticipation of our upcoming project next fall, Chandler showed me 280Slides.com and 280Atlas.com. These two applications are significantly progressing the design of web applications. 280 North is really breaking through the limitations of a web-based application and presenting an entirely new way to create a website. Instead, websites could now become true applications that look and act like a desktop application. This is significant and profound, especially for me. Now, Iíve realized that I need to practice UI design (Iím terrible at it) and desktop-style application development.

So, on to developing a calculator. Iím not going to try to reinvent the wheel, so to speak. This is a simple calculator application. People expect calculators to work in a certain way and the parameters for a calculator are fairly straight forward. However, there is a bit of user interface design that Iíd like to explore.

First, we have the Microsoft Calculator that comes bundled with Windows 7 RC1. This is the simple mode that the calculator starts up with, by default. This is how most people would expect a calculator to work and to look. There is the 3x4 grid of standard buttons that include the numbers 0-9 and the period. The buttons around the calculator indicate actions that manipulate the numbers. There are a few problems with this design. First, the top row of buttons do not indicate meaning. MC, MR, MS, M+, M- are all functions that appear throughout the application domain of calculators. However, despite being an engineer and using calculators extensively, Iíve never known their use. A better design with more descriptive buttons or even hover-descriptions would help tremendously. In terms of functionality, the Microsoft calculator meets the needs of 90% of applications and 90% of users. This is the functionality that I want to target. But, I want to explore some other calculator designs.

Google has a built-in calculator. This calculator is embedded into the Google search feature that the company provides worldwide. This is a very different user interface approach to calculations. It is a very expressive, command-line-esque calculator that provides more functionality than the Microsoft Calculator. I find this expression-based calculator to be easier to use. However, after asking a couple of non-computer people, Iíve found that they find the button interface to be useful. However, if they ever need to do complex calculations, they would rather pick up a Texas Instruments TI-89.

A TI-89 is a very powerful hand-held graphing calculator that can perform many functions that the Microsoft calculator and the Google calculator cannot. While the functionality is not entirely important in my design decisions, the fact that people prefer this is certain situations is interesting. People like it because of the graphing capabilities and because that it has both buttons and typing feedback (what you type shows up on the screen as you type it). Also, while the buttons are numerous, it seems to clearly indicate which buttons the user needs to press to accomplish a task.

So, I think that a calculator that combines all of these elementsóthe basic functionality of the Microsoft Calculator, the expressiveness of the Google Calculator and the typing feedback, expression-editing, clear buttons of the TI-89.

This project will, of course, be open source. I will be using Git as my source code repository. After I finish my development of the project, I will release the project itself for development (though I donít see why anyone would pick it up. If they do.. more power to them). As far as the language, I think that the built-in .NET widget library is going to be infinitely useful for this project. So, I think Iím going to use that unless I can find a sweet Ruby or Python GUI Toolkit (Pythonís looks pretty bad. Shoes for Ruby seems pretty cool, though).

Importance of Comfort

2009-04-29T00:00:00-05:00

As I was setting up Visual Studio to my normal preferences, I was considering how important it is to actually go through that process. Every time I install Visual Studio, I go directly to the WekeRoad blog and grab the Vibrant Ink theme for Visual Studio (then modify it to have 14 pt. font). This, I believe, makes coding more enjoyable. I think that the Vibrant Ink theme is easier on my eyes (I have slight light sensitivity) and just feels good. So, I was wondering how important this actually is. Does it, over time, improve my performance?

For that matter, does sitting in a comfortable desk chair make engineering more productive? Or ergonomic keyboards? Or even having nice backgrounds? I think it does, just because of the fact that happier people produce better code. However, it would be interesting to see a study of some sort on this topic, in general, and editor styling in particular.

Spring Break 09

2009-04-18T00:00:00-05:00

Woo! Party on!

This break has been really laid back and relaxing for me. Which was perfect because I needed to recharge. I really hadnít had a real break from anything since Thanksgiving. School was just as stressful and unrelenting as always (if not more) and my trips to Sweden and Maine sapped my energy.

Luckily, Iíve been just chilling with my family. I got to eat Easter dinner with both sides of my family and got to see my cousin who is shipping out to Afghanistan sometime in the near future (on that note, apparently the Army has a base in Washington state, who knew? And, temperate rainforest isnít going to prepare someone for Afghanistan either.. Iím confused as to this decision by the Army but I digress). I got to see my brother who is currently in California working to get Ed Carpenterís IndyCar series car running at top notch. I got to see my little brother who is graduating soon and my older brother, too. My sister is having a baby shower tomorrow (and I still need a place to hide for those two hours). And I got to poke fun at my step father about the Employee Free Choice Act.

I got a new truck, a 1993 Jeep Grand Cherokee. Itís a pretty sweet truck with some minor body work that needs to be done. I can get that fixed this summer. It runs well, though. And, I just put in a new stereo. Iíll post some pictures to Twitter when I get my phone charged.

Iíve been looking around at some apartments for this summer. It looks like Iíll be having to have a roommate if Iím going to be living in an apartment.

Iíve been working on an extensive set of posts about ìUser Maintenance.î I have the first part done but Iím not going to post the first part until Iíve finished the other two. I think its some pretty interesting stuff. Hopefully some fellow software engineers will think the same.

And, for school work, Iíve been getting excited about a project for CSSE376 that I call MusicMill. I realized over break, as I was organizing my music, that Windows Media Player, iTunes, Winamp and even Foobar just suck at library management. For those programs, library management just seems like an ìadditional featureî instead of the core part of the program. However, in my experience with the software, it is where I spend the bulk of my time and I think they can be done so much better. So, Iím going to propose a ìlibrary-focusedî music library manager than just happens to feature playback (e.g. playback is not priority #1, the library is; in fact, playback may not even get implemented in CSSE376). I also want to try to use BDD for the project. For the rest of my courses, I need to start figuring out what I need to do for the upcoming days. Getting back in the saddle is always a bit painful but its better to do it now than later.

All in all, Iím recharged and ready to go. Bring it on, next 6 weeks!

Exercise in TDD

2009-04-03T00:00:00-05:00

James Grenning has a good, old blog post that describes the ìPhysics of TDD.î It attempts to prove in an informal method that TDD is a good method of finding bugs. Iíve read this blog post before. It was good then; it is good now. Anyone that is on the fence about TDD (or just plain interested in TDD, in general) should read this post.

For anyone that doesnít know, James Grenning is one of the authors of the Agile Manifesto. Heís a pretty popular guy in the Agile circuit. While I agree on the very basic level with Grenningís post, I think there are some inherent fallacies with pure TDD. It needs some sort of supplement such as ATDD or BTDD.

Test Driven Development

2009-03-29T00:00:00-05:00

It's quite possibly the biggest buzzword for software developers right now. Test driven development (TDD) is being tested and employed in several places in the software engineering field. Last summer, the company that I was interning for was using it for at least one project. Due to the amazing success of that particular project, I am sure that they have since increased the presence of TDD in their company.

TDD is awesome

Unfortunately, I was not a member of the TDD team. Instead, I was on a team that was practicing more traditional methods of designing features, developing those features and "Big Bang" testing those particular features at the end of the development process. We delivered the product on time, but I feel that the delivery was at the cost of significant quality. Instead, I felt that something needed to be different.

One of my friends was on the TDD project; He told me how successful it was and how it works. He mentioned some really cool aspects of TDD that really peaked my interest. TDD, he said, helped improve the quality of the code that the developer wrote. It did this in two different ways: it forced developers to write the expected inputs and outputs†of the methods she was writing and it gives nearly instantaneous feedback to the developer about†bugs and unexpected behavior. Many bugs are introduced into software because developers misunderstand the expectations. And, the instantaneous feedback allows for the developer to quickly find and fix bugs that may otherwise take precious time for quality assurance teams to investigate.

TDD in action

I was notably excited about the prospect of TDD. I came back to school last fall with high hopes for this emerging method of developing and writing software. I got my chance to put it into action when I was assigned the task to develop a web-based Apples to Apples Online application. I convinced my partner (Kyle Rhodes) to use Ruby and Ruby on Rails as the development language and framework, respectively. Rails has aspects that make TDD very easy. It has a built-in automated testing harness that you just drop simple, easy-to-write tests in. Following this grain, several plug-ins have been developed for Rails (and Ruby) that make testing easy and even fun (e.g. flexmock). Kyle and I were chugging away, using TDD to develop the application and were moving along smoothly.

At least, everything was smooth until the part where we needed to work on the views. The program was that we worked on our features horizontally. This, I know now, is a big no-no in Rails (and MVC in general) development. Our database worked just fine; we tested that. Our application logic was solid; we tested that. The connection between the data layer and the application layer was smooth and logical; we tested that. Everything was supposed to work because we tested†it. However, as solid as our application logic was and data layer was, we did it wrong. We didn't realize this until the final part of the feature set we were developing needed to be implemented.

My confidence in TDD was shattered. My first attempt at using the development practice was a complete failure, in terms of the project outcome. Luckily, though, I was able to learn a lot out of the experience. TDD itself is a wonderful practice on the functional level. However, TDD cannot ensure that features are correctly being implemented nor can it guarantee user satisfaction. But, at this level, every thing would still need to be big-bang tested at the end of the project.

Two new practices: ATDD and BTDD

I found out about two new practices that attempt to address the problems that I found with TDD during my own experience.

Acceptance Test Driven Development takes the standard reverse pyramid method of testing and blends it with TDD. This means that, after feature prioritization, each feature is implemented incrementally. To start, the acceptance tests for each feature are written. These are broad tests that are usually based on the user stories for that particular feature. Then, the developer writes the integration tests that describe the interaction between modules, classes, etc. Finally, the developer writes the unit tests one at a time, satisfying each unit test exactly as a developer would in TDD. The TDD cycle repeats until each integration test is satisfied (solving each test sequentially). When all of the integration tests pass, all of the acceptance tests should pass. If not, then additional integration tests need to be written and we recycle until the acceptance tests pass. At this point, the feature can be considered "finished" in development terms. The acceptance, integration and unit tests will be rolled into the regression test suite (if not automatically done so).†

Behavior test driven development (BTDD) is similar to ATDD, except that it uses†the business rules (what would become user stories) of the program to test the system. This abstracts the acceptance testing material to a medium that both developers and customers can understand. The BTDD tests replace the acceptance tests and the process continues the same as ATDD does. RSpec, Rails' BTDD plug-in, has wonderful examples of BTDD on its website.

While I haven't used these methods for development, I feel excited for the promise that they bring. They eliminate both "Big Bang" testing at the end of a project and a loss of feature-focus brought on by simply writing unit tests and satisfying them over and over. There are many proponents of TDD as well as critics, but either way I think that every developer should investigate TDD in some fashion.

New blog

2009-03-29T00:00:00-05:00

I've decided to try out the Wordpress blog software. So far, setup and usage has gone really†smoothly and I think I may migrate all of my posting to here. The categories on Wordpress are really nice, as well, so I think I'm going to use that instead of multiple blogs. I've also tied it into Twitter (which is tied into Facebook) so all of my posts are now more visible to my friends. Sweet.

I'm doing a lot of work to the site so there could be some downtime / weirdness over the next few days.