Paul's techie blog

Having a CI server != Continuous Integration

Fri, 14 Oct 2011 11:04:11 GMT

Although Continuous Integration is a practice that requires no particular tooling to deploy, we’ve found that it is useful to use a Continuous Integration server. – Martin Fowler

The map is not the territory. The tool is not the practice. Having a Continuous Integration Server is not the same as continuously integrating. Continuously integrating means that your team is:-

working on the same source control branch
checking in and pushing frequently, ie several times a day
only pushing working code that keeps the test clean
avoiding long-lived feature branches

A CI server is a useful but not essential tool for continuous integration; there are teams running a CI server that are not continuously integrating.

Nordic Ruby 2011 - Summary of Day 1

Sat, 18 Jun 2011 17:08:23 GMT

Nordic Ruby is a well crafted conference. The overwhelming impression that I have is of calm attention to detail. The organisers CJ and Lilly have done a superb job. The conference has been much more than just the scheduled talks, but they are what I am focussing on here.

Git Hub Flavoured Ruby – Tom Preston-Werner

Tom Preston-Werner gave us some concrete engineering practices and tools, honed during the development of GitHub.

Readme Driven Development is a simple idea: writing the Readme before writing any code for a project forces you to clarify the goals and do just enough planning.

TomDoc provides more structure to class and method documentation than RDoc in a more readable form that YARD. A nice feature is the ability to mark a method as part of a project’s public API.

Using Sematic Versioning, particularly for Gems, helps keep your clients from dependency hell. In a nutshell, semantic versioning is Major.Minor.Patch, and only major version increments may break backwards compatibility. This way you can use Twiddle Wakka in your Gemfile with confidence.

This was all good stuff, but the next idea is dynamite: Make Believe Open Source is a superb concept for modularising projects; rather than writing monolithic applications, pretend that every area of functionality is going to be open sourced, and split it into a separate module/gem. The Path element on a Bundler can be a useful way of getting started. I am looking forward to see how this works with Rails.

API Design Matters – Anthony Eden

Good APIs should be suffient, powerful, consistent, clear, convenient, and complete. They should demonstrate the Principle of Least Surprise, and be economical with concepts. The best API is no API, meaning if you can get away with (say) implementing Enumberable then do just that.

Bridging the gap – Using Javascript in Rails to write DRY rich client applications – Thorben Schröder

How do you eliminate duplication when you have similar code implemented in JavaScript in the browser and Ruby on the server? Jon Crosby addressed this with his 2009 RubyConf talk, settling at the time with JRuby and Rhino. Thorben demonstrated that this can now be done by embedding V8 in MRI, using therubyracer and CommonJS.

I can not help the (slightly) uncomfortable feeling that this leads us to writing the server side of web apps in JavaScript (or CoffeeScript).

The Limited Red Society – Joseph Wilk

A lovely short talk by Joseph, on using feedback provided by metrics to minimise the red part of the Red-Green-Refactor cycle. Joseph demonstrated Red to be synonymous with the Work In Progress concept of Lean: while the code is Red it can not be integrated.

Must It Always Be About Sex? – Joshua Wehner

Not only did Joshua call out the Ruby community’s Sex, Race, and Age imbalance. By citing research that demonstrates that more diverse teams perform better, he showed that the imbalance is harmful. Suggestions to mitigate the imbalance included anonymising job applications, and even GitHub profiles, to prevent (research demonstrated) unconscious bias based on applicants names.

This deserves a much longer post, but in the meantime it is well worth a look at Erin O’Brien’s talk at The Scottish Ruby Conference, Where Are All The Women.

Infinte Date – Finite Solutions – Randall Thomas

Maths, whisky, Bayesian Statistics, and recreating Bach. What more could a talk need?

Taking Back Education – Joe O’Brien

Joe argues that Computer Science education is largely broken, and how we should fix it, citing the wide educational background of many effective Ruby Developers and the apprenticeship programmes of EdgeCase and other boutique Ruby development shops.

I should write

Sat, 18 Jun 2011 04:59:38 GMT

It’s over 18 months since my last blog post, about RubyConf 2009. There’s a lot happened since then, including two Scottish Ruby Conferences that I jointly organise, RubyConf 2010 that I spoke at, and the rise of EdgeCase in the UK which I am very much part of.

I am not alone in replacing my blog, with a Twitter account. I do feel poorer for not publicly expressing myself in more than 140 characters.

Writing similar to taking physical exercise: once I stop it gets very hard (and embarrassing) to start again. Here goes an attempt to start again. Wish me success.

News from RubyConf 2009

Mon, 23 Nov 2009 00:12:30 GMT

Q: What would you like to see in Ruby in 5 years? Matz: Smiles on everybody’s face at RubyConf 2014.

I avoid using the word ‘Awesome’; I leave that to the kids. RubyConf 2009 has been, well, pretty damn good. The quality of talks has been outstanding and the community is just superb. Well done Chad, Kelly, and David for putting this together. My favourite moment was when during this morning’s Matz Q&A session someone asked whether we could have a __DIR__ constant to avoid the ugly File.dirname(__FILE__) that we often need to write. There was a quick discussion in Japanese, followed by the answer ‘Ok’.

I’m not going to give a blow-by-blow account of the conference. The videos will be out soon on Confreaks. There were a few things that really excited me, though. If I were to put a theme on the conference it would be Different Ruby, Fast Ruby, Interoperability, And Stuff That’s Not Ruby. There was a lot of non-MRI and hacked MRI around.

Of course Charles Nutter and Thomas Enebo showed us just how fantastic a job that they are doing with JRuby. MacRuby is getting even more interesting. New to me was the focus on Server-Side development. Laurent talked about custom Mac servers, using Grand Central Dispatch to give massive scaleability. Rubinius is promising a 1.0rc1 next week (at last). Apart from that there were several other talks on partial or full Ruby compiling.

Scaling and performance made its way into quite a few sessions from Paul Dix’s informative “Synchronous Reads, Asynchronous Writes”, to the MongoDB session, to Greg Pollock’s Scaling Rails talk. There are some great links to tools to help improve Rails app performance from Greg’s talk here

The top talk that I regret missing was Charles Nutter’s Duby talk. The standard comment from that talk was something like “I thought it was just Charlie’s hobby, but that is really useful.” In is spare time Charlie has designed a Ruby-like language that can be easily “compiled” into a static language (ie Java). The uses are pretty interesting.

Tom Presont-Werner’s presentation on Bert and Ernie introduced us to the interoperability theme. As JSON is to JavaScript (kind-of). Ernie is a superset of Erlang’s External Term Format distribution mechanism. By extracting it from Erlang is gives us an efficient, simple and standard way to perform interprocess communication. Ernie is a RPC server for handling BERT requests in Ruby. Interop made it’s way into the MongoDB talk with the BSON (binary JSON) protocol.

Ben Scofield’s NoSQL was a great round-up of the non-relational database options that are getting popular – not just the Document Based and KV pairs, but Graph based and Column Oriented ones too. If, like me, you know little about them you should look up Ben’s talk when it is posted.

And there is still so much more to say. I haven’t even mention the Arduino controlled airships, Jim Weirich’s SOLID Ruby, and that 54 people took part in the 5k run. (I came 20th).

Update: added more including pictures.

Most webapps are nails

Fri, 31 Jul 2009 01:21:35 GMT

The child who receives a hammer for Christmas will discover that everything needs pounding – Gerald Weinberg

Fred George’s keynote at “Rails Underground” was titled “Rails is an hammer”. It was along the “grumpy old developer” line that kids today jump to use all these fancy frameworks without considering whether they are appropriate to the job in hand. It’s a fun spiel, and one I’ve levelled against a lot of Java frameworks and libraries. Is Rails really inappropriate for most web applications? No, but it’s worth asking the question.

The main charges against Rails are:

it does not have a domain layer
the models are based on relational database tables rather than object design

Does Rails not have a domain model?

This is good point that is not often made. Having MVC not the same as a layered architecture. As Jay Fields “discovered” that his Presenter Pattern was not sufficient for some applications it was Domain Driven Design his team turned to concepts such as Services and Repositories. Nowhere does Rails mandate that ActiveRecord models should be accessed directly by controllers.

Good domain models are tough. It can take a long time working with an application before the right model is discovered. Leaping to a model too early – premature abstraction – often leads to an inappropriate object architecture that it can be difficult to write your way out of and Byzantine back channels are needed for operations that do not fit the original design.

Something that Rails has taught me is how far you can go with just MVC and a good framework. I wouldn’t say this for a typical Java web app, but you ain’t gonna need a layered architecture. Until you do need one. Then won’t be a big deal to refactor towards that, as long as you keep you’ve kept your code clean, DRY, and well-tested.

Are Rails ‘models’ are based on database tables?

This came up in the Scotland on Rails CouchDB controversy, and I dispute that Rails ActiveRecord models represent rows in relational database tables. It’s the other way round: relational databases happen to be the persistence mechanism for ActiveRecord; the database is subservient to the object model. No-one normalises their Rails database; you might fret about keeping your objects DRY and separation of concerns but I doubt you look at your Rails DB worry about repeating attributes. It’s about associations, not relations and join tables.

Admittedly, right now Rails is strongly bound to being back by a RDBMS, and there are some hoops to jump through if you want to use something else. That does not stop a growing number of people using alternatives like CouchDB. The promise of Rails 3 is that it will become trivial to swap in alternative Object Persistence mechanisms, so this will become less of an issue.

Given up on Rails yet?

Fred had some valid points, and it was fun to see how far he could get with vanilla Ruby, Sinatra, and persisting straight to disk. I’m still going to be reaching for Rails to solve most of my web app problems, though. After all it’s a pretty versatile hammer, and there’s a lot of things that need pounding out there.

Jet-lag is optional

Thu, 23 Jul 2009 23:17:44 GMT

A couple of months ago I had a one week one trip from Los Angeles, which promised to be a jet-lag nightmare: 5 days to adjust to the 8 hour difference and then fly home to suffer another 5 days. Luckily a couple of days before departure I caught a TV programme in which a theory on how to overcome jetlag was tried out on a couple of racing car engineers flying back from Atlanta to the UK: extrapolating from research on mice, the theory was that by fasting for 16 hours the body’s ‘feeding clock’ takes over from the main ’circadian’clock.

By not eating while travelling, then resuming normal meal times on arrival, we can adjust immediately to local time. The experiment worked on the show and the presenter, a world-travelling journalist, claimed that it had subsequently worked for her.

On the journeys to and from LA I fasted for around 16 hours, and it worked very well for me in both directions. I slept from 22:00 to 06:00 on the first night in LA, and then later and longer on subsequent nights. In contrasts my compadres Alan and Brian both suffered quite badly from jetlag throughout the week. Kevin McDonagh tried the fasting trick on his trip to San Francisco for Google IO with similarly excellent results.

In short, if you are willing to forego eating for a 16 hours, you can avoid days of jetlag. The main points are:-

do not eat on the journey – the journalist presenting the BBC sleep programme claimed fasting on a 11 hour flight worked for her; I have only tried the whole 16 hour fast.
drink as much water as you like. I also drank a very small amount of orange juice, coffee, and Gin and Tonic on my flights without disastrous results – your mileage may vary
start eating at normal meal times when you arrive

Happy travelling, but don’t come complaining to me about your circadian rhythms: it’s all in your own hands.

Smart Tests Are Dumb Tests

Sun, 26 Jul 2009 14:50:29 GMT

A question I hear a lot is “don’t you find your test code is pretty much a copy of your production code?” Digging deeper we this kind of test.


def test_counts_the_correct_number_of_words
  sentence = 
    "The quick brown fox jumps over the lazy dog."
  expected_count = sentence.scan(/\w+/).size
  analyser = SentenceAnalyser.new(sentence)
  assert_equal expected_count, analyser.word_count
end

Here’s what I’d write.


def test_counts_the_correct_number_of_words
  assert_equal 9, 
    SentenceAnalyser.new(
        "The quick brown fox jumps over the lazy dog.").word_count
end

The more work we make our tests do, the more likely we are to introduce mistakes in the test code; this can be especially dangerous if we use the same algorithm as in the production code as we can make the same mistake in both places. And that’s apart from the second test being shorter and easier to understand.

So remember, kids, keep your tests dumb to keep them smart.

Technical Debt or When Metaphors Attack

Sun, 05 Apr 2009 19:00:38 GMT

Technical Debt is a powerful metaphor: it describes code-quality shortcomings in financial terms; until the debt is paid off we pay interest, meaning that it takes longer to implement new features. It’s a useful metaphor, but has limits. A few weeks ago I was annoyed when a whole room of Agile consultants took seriously the idea of “borrowing on code-quality” to get a start-up off the ground, and then repaying the debt later (presumably by hiring consultants). All my experience tells me that this is a disastrous strategy. Such debt becomes incredibly expensive to pay off – expensive in real cash.

Ward Cunningham coined the Technical Debt metaphor. He describes (on video) producing code without a complete understanding of the problem, as accumulating technical debt. By writing the code early we increase our understanding of the system: as long as the debt is repaid, this is prudent borrowing. It’s quite like a business borrowing to invest in capital equipment and using the increase in revenue to pay off the debt.

Metaphors are natural and useful thinking-tools. A good metaphor, though, can be dangerously seductive. We can be tempted to use it beyond its applicability. A metaphorical vehicle cannot fully describe its topic: they need to differ for it to be a metaphor rather than a description.

Recently Brian Marick suggested that metaphorical reasoning could be improved by listing ways in which the metaphor differs from its topic. Financial debt is easily quantifiable; technical debt is not. Borrowing on code quality to get your product out quickly is more like begging favours from the Mafia than taking out a business loan. The repayment terms are unknown, except that they will be entangling, larger, and more horrific than you can imagine.

New blog location

Thu, 12 Mar 2009 13:45:05 GMT

Because I can, I’ve decided to move my blog to here.

I’m hoping that it will be mostly seamless, with the Feedburner RSS feed and redirects taking away the pain. If it isn’t, sorry.

iPhone TDD using Google Toolbox and iphone_testify

Wed, 22 Apr 2009 10:27:53 GMT

Previously, we’d decided to switch from using Rbiphonetest to using Google toolbox for Mac code. It’s about time we got started.

To begin with we’ll just set up an iPhone project for testing. This will include automatically running all the tests every time – just like autotest.

Setting up the files

Of course you’ll need to be running OS X 10.5 (Leopard) and the latest iPhone SDK.

Fire up XCode and create an iPhone OS project. (For the purposes of this example it doesn’t matter what type; a Utility app is as good as any). I’ve written a Ruby Gem to help getting all the files in place, so we may as well use those. From the terminal go to your new application directory.

First we’ll install the gem. If you haven’t already you will need to to add github as a gem source. (If you can’t remember just enter gem source and see if github is listed).


sudo gem update --system
gem source -a http://gems.github.com

Either way, install the gem.


sudo gem install paulanthonywilson-iphone_testify

Then run the iphone_testify command.


iphone_testify

This adds the google unit testing code to the google_testing directory, creates a directory for the UnitTests. There’s also a Rakefile_ and an autoiphonetest.rb, and we’ll get round to those later.

Setting up XCode

There’s a few things you need to do in XCode. The gem doesn’t do this yet, but I’m working on it.

Create a new Target called Unit Test. Right click on Targets in the left pane. Choose an “iPhone OS” Application target and call it “Unit Test”.

Right click on the Unit Test target and choose “Add—> Existing Files”. Select all the files in the google_testing directory (or at least all the .h and .m files).

(If you’re feeling tidy, which I hope you are, you probably want to add these to a “google_testing” group as well.)

Right click on the Unit Test target again, to add a “new run script build phase”.

Set the script to be google_testing/RunIPhoneUnitTest.sh.

Add a group in which to keep your unit tests. Right click on the group and set the path to be UnitTests, so that the test classes will go where we expect on the file system.

That’s all a bit of faff. I am hoping to automate the xcode part: you can stay tuned by watching iphone_testify on Github.

Write a test

Let’s write a dummy test, just to get us started. Right click on your UnitTests group and add a new file (iPhone OS Cocoa Touch Classes – NSObject subclass). Let’s call it BlahTest.m. Add it only to the Unit Test target. Uncheck “Also create BlahTest.h”: you don’t need a header.

Edit the BlahTest.m to extend SenTestCase:-


#import "../google_testing/GTMSenTestCase.h" 

@interface BlahTest : SenTestCase
@end

@implementation BlahTest

@end

Note that we’ve included the interface definition in the implementation file: there’s nothing to stop you using a separate header file, but there’s not a lot of point. Let’s add a failing test.


@implementation BlahTest

-(void) testSomething{
    NSString *actual = @"hello matey";
    STAssertEqualStrings(@"hi mate", actual, nil);
}

@end

Run through XCode

The tests are ‘run’ as part of the build and failures are reported as build failures. In XCode set the target to “Unit Test” and build.

Run through Rake and autoiphonetest.rb

Running iphone_testify also provides you with a Rakefile for building/running the test target. From the command line in your project directory just run


rake

Even better, run


./autoiphonetest.rb

This will build/run the unit tests evertime a change is detected in the Classes or UnitTests directory. The automatic running of unit tests does wonders for flow.

The rake and autoiphonetest.rb builds will inform you of the state of the tests using Growl: to take advantage of this you will need to have Growl and growlnotify installed. In order to work around a bug in growlnotify, you need to enable “Listen for incoming network connections” in the Network pane of the Growl System Preferences.

Now we’re set up running with autoiphonetest.rb, let’s fix our test and see our pass notification; I’ll leave the test fixing as an exercise for the reader.

That’s it for now. In our next instalment we’ll look at building up or Twitter application using Objective-C unit tests.

Update: Corrected a couple of bugs:

The target must be called Unit Test (not Unit Tests) to work with my gem
I ought to have used STAssertEqualStrings for comparing strings (not STAssertEquals).