www.thoughtworks.com aggregator

Alister Scott: ♥ Travis CI

Wed, 08 Feb 2012 16:32:21 -0800

I’ve talked previously about how cool Travis CI is, and they’re now taking donations to implement more functionality in the future. Since watir-webdriver uses Travis, as does my watir-page-helper gem and my Minesweeper Jasmine tests, I thought I’d better contribute!

When you donate you get lots of warm and fuzzies, plus you get cool stickers, some ringtones!?!, and a neat little surprise on the thank you page.

You should donate too.

Anand Bagmar: vodQA Pune - registrations open

Wed, 08 Feb 2012 02:56:00 -0800

The registrations for attendee and speakers for vodQA Pune on 17th March 2012 is now open. Looking forward to see you there :)

Alister Scott: CukeSalad: not so yummy yummy

Mon, 06 Feb 2012 20:43:41 -0800

I’ve been playing around with CukeSalad: Cucumber specs / tests without step-definitions:

Cucumber, washed and ready to eat for Friction-free ATDD/BDD

In creating an example of my simple WatirMelonCucumber framework using CukeSalad, here’s some of the things I’ve found. Please note: these are my personal observations and are not direct criticisms of the huge effort that has gone into making this tool.

It’s not really no step definitions: they’re just called different things. Instead of writing steps, you’re writing roles and tasks, both of which are mandatory, so I found I actually wrote more code (in more files) than if I just wrote step definitions in the first place.

CukeSalad forces you to use predefined step syntax: which to me defeats the purpose of having features in business language. For example, as far as I can tell, Given statements are purely used for setting the roles, and therefore a business person can’t use it to define a precondition which I would normally use Given statements for. Another example is when you want to capture some data from the test, this must be on the end of a sentence following a semicolon (or comma)

When I search for: the phrase 'Watir'

I spent ages trying to work out why steps don’t match: as you still use Cucumber to run your features, Cucumber will only tell you that there’s no matching steps. This doesn’t help when there’s no steps whatsoever! At some point, I realized that Then steps follow a different syntax, and they can’t use placeholders like Whens, but must have an expected value on the end in single quotes.

Then I should see results greater than '30,000'

It’s very confusing what you’re actually asserting on a Then step: I couldn’t work out a way to capture the expected value from the feature (in the example above ’30,000′), and CukeSalad just asserts the task returns the actual expected result, which doesn’t help if you’re not testing for equality!

It’s very confusing trying to convert an existing Cucumber test suite: I wouldn’t bother. I found I had to completely rewrite my feature to comply to the syntactical requirements of CukeSalad, so I can’t this working on an existing feature set.

Cucumber world is not supported: so just do a normal include instead in env.rb.

Summary

Sadly, CukeSalad doesn’t offer me anything over using Cucumber. Whilst it sells itself on having no step definitions, I found there was greater effort in writing the roles and tasks. I don’t like the way it forces me to write sentences using somewhat awkward constraints, and how Given, When, & Then statements all behave differently. It certainly gave me a lot of WTF moments, which isn’t what you want when writing automated tests.

I’d be very interested to hear whether anyone has had ongoing success with CukeSalad.

Alister Scott: Turnip: trying to solve Cucumber’s problems with ruby BDD

Sun, 05 Feb 2012 04:07:23 -0800

I’ve been playing around with Turnip: a library that was designed to solve Cucumber’s problems in four main areas:

Having a separate test framework is annoying

Mapping steps to regexps is hard

Cucumber has a huge, messy codebase

Steps are always global

I don’t really care about having a Cucumber test framework as I don’t often use RSpec, I actually find regexen quite powerful, I also don’t really care about Cucumber’s code base but I can see some merit in having scoped steps.

I converted my existing demo test suite across to use Turnip and this is what I found.

Turnip follows Spinach in trying to move away from using regular expressions to define steps. Fortunately, unlike Spinach, Turnip allows you to capture values using placeholders in your steps (this was an instant NO to using Spinach):

  step "I should see at least :exp_num_results results" do |exp_num_results|
    on :results do |page|
      page.number_search_results.should >= exp_num_results.to_i
    end
  end

Unfortunately there are a number of limitations to this approach:

You can’t capture more than a single word in a placeholder without using double quotes in your feature file – making them less readable; and
You can’t capture values like “100,000″ as it doesn’t appear to support special characters in placeholders.

You can also do some magic using custom step placeholders, where you define regular expressions, but to me that kinda defeats the point of not using regexen in the first place.

The scoped steps are quite neat and ruby like, and I can see these being very useful for large projects where multiple people are writing the steps.

Running Turnip files is as simple as using the RSpec test runner, but unfortunately the output of pending steps is less useful than Cucumber in that there are no code snippets, and only the first undefined step is displayed as undefined (unlike the Cucumber which shows you every step that is undefined).

Summary

I like the approach that Turnip takes in being a simple way to write business like tests and run them using RSpec. Unfortunately that simplicity comes at a price, and losing the power of regexen in step definitions is not something I would like to give up hastily. I am hoping that the scoped steps someday make their way into the Cucumber code base.

Alister Scott: Introducing the software testing ice-cream cone

Tue, 31 Jan 2012 05:33:38 -0800

As previously explained, I like using the software testing pyramid as a visual way to represent where you should be focusing your testing effort, and often switch between using a cloud or an Eye of Providence to represent the manual session-based tests at the top of the pyramid that you should use to supplement and test your automated tests.

I often see organizations fall into the trap of creating ‘inverted’ pyramids of software testing, and only yesterday did a colleague point out to me that if you invert my pyramid with the cloud, you end up with an ice-cream cone! So, introducing the software testing ice-cream cone!

Alister Scott: Reading CSS properties using Watir-WebDriver

Sat, 28 Jan 2012 04:06:02 -0800

There was recently a question on Stack Overflow about how you read a CSS property of an element that is defined in a stylesheet using Watir-WebDriver.

It’s really quite simple:

require 'watir-webdriver'
b = Watir::Browser.start 'minesweeper.github.com'
puts b.div(:id => 'g1minesRemaining100s').style 'background-image'

Alister Scott: Writing a CoffeeScript web application using TDD

Mon, 23 Jan 2012 03:00:20 -0800

As part of the Test Automation Bazaar minesweeper challenge, a colleague and I developed a CoffeeScript implementation of minesweeper. We wanted to use a test first approach to writing our CoffeeScript, so we decided to use Jasmine.

Setting up a development environment

Even though we were developing the application in CoffeeScript, which generates JavaScript, we found it easy to use a simple ruby environment to boostrap these tools. Our environment therefore looked something like this:

Ruby 1.9.2 (specified through a RVM .rvmrc file)
CoffeeScript installed via Homebrew
guard, guard-coffeescript, guard-sass gems for automatically generating JavaScript from CoffeeScript (and CSS from Sass)
jasmine gem: for writing/running tests
Sublime Text 2 editor for writing code

Specifying a Guardfile

A Guardfile consists of a bunch of guards that perform actions whenever a file is modified in that location. Our Guardfile looked like:

guard 'coffeescript', :output => 'javascripts' do
  watch /^coffeescripts\/.*[.]coffee/
end

guard 'coffeescript', :output => 'spec/javascripts' do
  watch /^spec\/coffeescripts\/.*[.]coffee/
end

guard 'sass', :input => 'stylesheets'

Writing a Jasmine specification in CoffeeScript

A Jasmine spec in CoffeeScript looked something like this:

describe 'GameState', ->
  game_state = null
  field = Field.new mineCount: 1, rows: 1, cols: 3

  beforeEach ->
    game_state = GameState.new field

  it 'should initialise lost to false', ->
    expect(game_state.lost()).toEqual false

  it 'should initialise won to false', ->
    expect(game_state.won()).toEqual false

  it 'should initialise remaining_mines to mine_count', ->
    expect(game_state.remaining_mines()).toEqual 1

  it 'should initialise remaining_mines to mine_count', ->
    expect(game_state.remaining_cells()).toEqual 2

Running Jasmine tests

The jasmine gem makes it very easy to run your jasmine tests. There’s a rake task called ‘jasmine’ which you can run to launch a jasmine server locally on port 8888. If you browse to that page, you’ll see something like this:

Test First CoffeeScript development

Now that you have Jasmine running, and Guard generating the CoffeeScript, it’s easy to write a new spec, refresh the Jasmine browser page to run all your tests (in our case in a third of a second) and then write the code to make it pass.

Automatically running Jasmine tests on Travis CI

If you’ve got your CoffeeScript and Jasmine on github, it’s trivial to automatically run all your Jasmine tests using the Jasmine::Ci rake task on Travis CI. All you need is a .travis.yml file like:


language: ruby
rvm:
  - 1.9.2
env:
  - DISPLAY=:99.0
before_install: sh -e /etc/init.d/xvfb start

Once you add the project to travis, it’ll automatically run whenever you push. Magic.

Lessons Learned

Use Sublime Text 2 for CoffeeScript joy: TextMate 2 forces you to use tabs (4) for CoffeeScript development, we couldn’t find a way to make it stop, and had to switch to Sublime Text 2 (for the better).
You still need to know JavaScript: all CoffeeScript does is generate JavaScript for both your application code and Jasmine specs. We found ourselves often delving into the generated code to see what was actually going on, and when using Firebug to debug.
Use the jasmine_content div to test the DOM. Jasmine uses a special div with an id of jasmine_content, which we used to inject and test HTML.
CoffeeScript classes are odd: we instead used closures to encapsulate state, using a new method on an object, which has the added benefit of looking like ruby.

I hope you enjoy your CoffeeScript test driven development with Jasmine.

Anand Bagmar: vodQA Chennai starts off with a century!

Sun, 22 Jan 2012 19:49:00 -0800

I attended vodQA in Chennai on 21st Jan 2012. The event was great. Over 100 passionate testers from Chennai testing community turned up and made sure people presenting were on their toes with excellent questions and great interactions.

One of the participants already blogged about it here. She raises valid observations - and I wish I had the opportunity to speak with her directly to address some of the questions / concerns she raised.

In this session, I presented a topic - "What is WAAT?" based on my open-source project - WAAT. The slides used in this session are available here.

I am already looking forward to the next vodQA in Chennai. For now, I am preparing for vodQA Bangalore - "Agile Testing for Teams and Enterprises". on Feb 11, and then vodQA Pune on Mar 17.

Manish Kumar: vodQA Bangalore

Thu, 19 Jan 2012 01:07:00 -0800

vodQA, the Thoughtworks testing conference is happening in Bangalore on 11 Feb 2012.The format seems to be quite interactive and 11 Feb looks quite an exciting day for those who see testing as the vital cog for IT and business agility.

The key thing is - there is no entry barrier as the conference participation is free.

I am participating in a panel discussion along with some of my esteemed colleagues who along with them bring plethora of experience on the topic of agile testing.The discussion should bring out paradigms, challenges , adoption patterns ,advancement in practice and toolsets etc.

There are lightning talks, interactive agile games and showcase of agile testing.

I am looking at the opportunity to exchange thoughts and experiences with participants.

Here is the link to event-page

http://testing.thoughtworks.com/events/agile-testing-teams-and-enterprises

Cromulent Testing: Your Mum Doesn't Work Here

Wed, 18 Jan 2012 16:00:00 -0800

Dear Devs,

Have you ever had code bounce immediately out of test because it just didn’t work? It’s messy and wastes a lot time. You’re expecting the tester to do all your testing, a common mistake. We’re not asking you to do two jobs, with a little effort you can tidy as you go, catching these basic problems yourself.

The following are helpful, let our conscience be your guide.

Before you start work, check what you’re doing makes sense. Talk it over, ask who, what, where, when and why. Verify that what you’re going to do is what they want.

While coding, ensure the code actually does what you believe it does. Stray from the happy path; what could a user do by mistake? How could a user cheat the system? Ask questions like “What happens if I…” and get answers. Not all who wander are lost.

Your development skills will help you. Use your knowledge of code to bend it a little. Does any part of the code make you feel queasy? Start there.

Check white space, min and max lengths, validations, mandatory and optional fields, nulls. It’s literally (not figuratively) painful for testers to report these kinds of problems. Just make sure it works. Please.

To conclude, your forays into testing won’t replace a competent tester but it will save them time and heart ache; let them focus on more devious aspects their craft.