Jack Kinsella

Automating Popularity

2011-12-14T00:00:00-08:00

Two years ago I wrote a program that logged into my Facebook account and pretended to be me. I wanted to test the hypothesis that automatic tools could make someone more popular.

If I could build a script that caused more friends to get in touch with me, then my hypothesis would be proven true. If, instead, the script had no effect on my popularity or was detected as a fraud by the people in my network, then my hypothesis would be proven false.

Before I carried out the experiment I studied some social psychology. According to the texts, a new acquaintance becomes a friend when they appear frequently in our lives; when they share our interests, beliefs, and background; and when we know that they return our affinity for them by liking us back.

Newly informed, I modeled these concepts by creating a script that logged into my Facebook account and automatically liked the status updates, photos and comments of my friends, covering up the evidence as it went.

Five days later the results came in. Listen to my talk, recorded at Ignite the West, in Galway, Ireland for the full story:

Thanks to Ellen Dudley and Adrian Avendano, two awesome people changing the world for the better, for organising Ignite the West.

PS If you have a research project or app idea in this area, I can make it happen

Janki Method

2011-12-05T00:00:00-08:00

This is a guide to becoming a productive programmer quickly. In his book Outliers, Malcolm Gladwell told the world it takes 10,000 hours to master something. Practice three hours a day, and you will shine in ten short years. Millions of readers felt inspired since they too could become the next Bill Gates, if only they put in the hours. As the days turned to months we discovered that 10,000 hours was a lot longer than we anticipated. Limitless potential transformed into fantasy about what might have been.

Janki Method is an attempt to shorten the time needed to learn programming. It grew out of my impatient dream to build an automated web business that would free me financially, geographically and temporally. I didn’t want to wait 10,000 hours. I suspect you don’t either.

Well-intentioned learners don’t always learn quickly, despite their smarts and enthusiasm. For most, eventual ability is determined by the volume of time expended. Malcolm’s 10k sounds right.

You’ve read that learning by doing is better than passive reading; that expressing ideas in writing forces understanding; that knowledge needs constant revision to stay fresh; that creativity comes from linking disparate ideas; and that your past mistakes are your best teachers. How many of these ideas do you apply to your learning efforts?

Janki Method is built on the assumption that Malcolm’s 10k to mastery can be hastened if you take a focused approach to learning. The core of Janki Method is the use of a spaced repetition flashcard system, Anki, programmed by the brilliant Damien Elmes.

By following my approach, I believe that any intelligent and disciplined reader can achieve proficiency* in a given field of programming (e.g. web applications, iPhone applications) in less than 12 months.

I call this the Janki Challenge, and I invite you to take part.

Problems With Learning

Problem 1: We Forget Too Quickly

Have you ever spent a week studying for an exam, only to forget 90% of what you learned within 2 months and everything else within a year?

Forgetting impedes learning knowledge-intensive skills such as programming. You need to remember various languages, solutions, libraries and gotchas if you want to build large applications. Because technical material can be so abstract and dry, you forget particularly quickly.

The first rule of Janki boosts your memory:

“Every time you learn something new create a question and answer flashcard and add this card to Anki.”

Anki is a Spaced Repetition System. Most algorithms make computers efficient; Anki makes you efficient. Using the minimal number of repetitions necessary for permanent retention, Anki drills flashcards into your long-term memory.

Begin by creating a deck of flashcards called, say, computer science. A deck contains many cards, and each card consists of a question and an answer. You tag these cards with the categories that best describe their contents. For example, one card might be tagged with ‘Rails’, and another with ‘SQL’. Each card can have numerous tags, something useful given how technologies frequently overlap.

Over time you will build up a repository of knowledge on programming, cleanly categorized, easily searchable and regularly backed up in the cloud. Keeping a repository like is useful, but it doesn’t do anything to help you keep the knowledge inside your head. The key to this is doing your Ankis.

Every morning Anki calculates which cards you risk forgetting, and then prompts you to review these cards. Doing your Ankis only takes a few minutes per day, since you only need to review a fraction of your deck on any given day.

For every card you review, Anki shows you the question side with the answer side initially blocked out. Your job is to answer the question in your head, and then reveal the answer and check whether you got it right. After you answer, four buttons appear at the bottom of the screen: “again”, “good”, “easy” and “very easy”. Assess how easily you could recall that card and then press the appropriate button. Based on which button you press, Anki determines when next to show you that card, so answering honestly is crucial if you want the algorithm to work.

Do this every day and you will not forget.

Problem 2: We Give Up Too Soon

Most people fail to learn programming for the same reason they don’t get good at playing guitar: they give up too soon. Although they practice hard for the first few weeks, they soon lose motivation, and give up before they get results.

Trying to learn using a rhythm of intense activity over short periods of time followed by long pauses is problematic. Your mind needs time to absorb what it learns, so learning skills cannot be condensed into a weekend. By accepting this reality and using a learning approach that emphasizes incremental daily effort, you will be less likely to burn out and more likely to succeed.

The second rule of Janki encourages a commitment to daily learning:

“You must use Anki every single day- including weekends and holidays – and commit to doing so indefinitely.”

Doing your Ankis must hold the same force of habit as brushing your teeth, and you should feel naughty if you ever miss your Ankis.

Rule 2 isn’t as demanding as it might at first seem. After a few months of practice you will be able do your Ankis in 5-8 minutes. Finding that time shouldn’t be a problem either, since Anki is available on smart-phone, meaning you can review while you walk to work, sit in the bus or have a spare minute at the office. Anki even synchronizes the state of your decks across all your devices, so changes to your desktop deck will be reflected across all of your devices.

Problem 3: We Learn Out Of Context

Learning out of context is wasteful. Reading a textbook from cover to cover may be interesting, but if those concepts are not relevant to the technical problems you currently face, then you will lack the mental context needed to assimilate that knowledge effectively. Incomplete understanding and wasted effort ensues.

The third rule of Janki keeps you focused on what is important in the moment:

“Learn in context. Pick a project, and learn only what you need to get it done.”

When I was writing this blog post I didn’t know how to add images in the Textile language.

Step 1: I Googled it.

Step 2: I tried the code out by adding an image to the post.

Step 3: After checking that it worked, I added my new technique to my Anki Deck.

Problem 4: We Think We Can Learn Without Doing

“I learned about sex the hard way – from a book.”
Eddie Izzard

Like sex, programming is better and more enjoyably mastered through practice. Learning without practice insulates you from reality.

When you read about programming without trying out the code, you form an internal representation that is incomplete, poorly understood, and difficult to apply practically.

You will fail to notice critical details. Many things may seem trivial when you first read about them in a textbook. Maybe it’s the placing of quotations around a parameter to a function, or rules about where whitespace is allowed. These overlooked points can often be critical in practice, and anything less than complete understanding will cripple hour productivity. It is only by trying code out that you will notice the nuances of these rules, and really understanding the language, techniques, and commands in question.

The fourth rule of Janki grounds knowledge in reality:

“Only add a card to your deck after having tried to use the item of knowledge therein.”

Say you are working on an application powered by an SQL database. To build it you will need to refer to literature on database design and SQL (a query language for selecting records from database tables). As you work on the application you will be exposed to new concepts. Try out each new nugget of knowledge before adding it to your Anki deck. If, for example, you read about how to delete a row in a database table, then you should try deleting an actual row in your database before creating the Anki card detailing that SQL query.

Problem 5: We Make More Mistakes Than We Need To

/* Our past mistakes serve as excellent memory aids. This is a major reason why we learn by doing. That said, not all past mistakes are equally effective at teaching us lessons; there is a correlation between the emotional impact of an error and the length of time that lesson will remain in memory. */

Say you make a big mistake. You accidentally push incomplete code to a live server, taking your app down for 12 hours. You panic to fix it, and as the customer complaints flood in, you feel stupid and embarrassed. This emotional punishment serves as a strong reminder of your error, and you will be more careful when pushing code to a server in future.

Errors with emotional impact are, thankfully, rare, and you are unlikely to need Janki Method to learn from these. But what about the rest of the mistakes we make, where the emotional element is diminished or even absent? We may need to make these mistakes many times before eventually learning our lesson.

Mistake repetition is the number of times you need to commit a particular mistake over a lifetime before learning a permanent lesson. Because mistakes are costly, embarrassing and potentially career threatening, it is sensible to minimize their occurrence. Bosses, customers and co-workers understand a first time mistake, and all but the most unreasonable will forgive you. They will not, however, feel so forgiving the second or third time you get the same thing wrong.

The fifth rule of Janki method minimizes mistake repetition.

“Every time you make a mistake carry out a post-mortem and try to figure out the lessons in the experience. Think about what you would need to know to prevent that mistake from occurring again, and turn this insight into new flashcards.”

By applying this rule you will substitute real world mistakes with Janki reviews, and you will consequently learn from your mistakes more quickly and with less embarrassment and real world error.

Ankifying your past failures doesn’t just prevent their repetition. It also helps you understand your domain, since the act of performing a post-mortem analysis and examining the underlying causes of your error encourages deep thought. Instead of just creating the cards you need to prevent that particular error, you might immunize yourself against that entire class of error.

Every mistake becomes a learning experience. Messing up still sucks, but you feel a small sense of victory knowing that you’ve just found a way to improve, and you will feel assured by the certainty that you will never make that mistake again.

Problem 6: We Do Not Reflect On The Big Picture

By following the initial five rules of Janki Method you will quickly become proficient with the low level details of programming: then name and use cases of functions and commands, their expected inputs and expected outputs, solutions to common bugs, and techniques for keeping your code organised.

As your experience grows you will be faced with bigger questions, such as structural decisions about your application, techniques to minimize complexity, workflows for managing simultaneous versions of your code, and the tradeoff between code performance and elegance. Answering these questions requires deep knowledge and sizable experience, and for this reason you must never miss an opportunity to grow your knowledge in these overarching issues.

The sixth rule of Janki method encourages regular critical reflection:

“At the end of every project ask yourself what lessons you learned and what you would do differently if you were to repeat the project with the benefit of hindsight.”

After a large project, such as completion of a new app, take some time to reflect. Ask yourself big questions, such as:

* Did I waste time going down any dead ends? Is there anything I could do in future to help me foresee these, and prevent them from happening again?
* Did I choose the right technologies? Can I come up with a schema to help me make the right choices next time?
* Did I accurately estimate the time required?
* Could I have built simpler architecture? Could I have figured this out earlier?
* Can I create a checklist to ease development of that kind of feature in future?

Some of these questions will be difficult to answer because you have to run through alternatives in your head, thinking about the benefits and tradeoffs of each. This effort is worthwhile since sensible architectural decisions prevent wasted effort. Without active reflection you will be less likely to notice these project level patterns, and so your wisdom will grow more slowly than it otherwise would.

Problem 7: We Let Our Knowledge Representations Grow Messy

Like a front lawn left idle, your Anki decks can grow ugly. Perhaps one of the programming languages you previously learned has changed dramatically in version 2.0 and now a percentage of your cards are invalid. Perhaps some of your cards are no longer worthy of keeping in periodic review– their usefulness doesn’t justify the cumulative revision time needed.

As you learn, you will discover simpler ways to understand concepts; without updating your cards to reflect these improved paradigms you risk losing these insights forever. Often a group of cards can be replaced with a single card on a higher order concept.

Sometimes you will see links between cards in your deck. If you note the connection between these cards you create multiple neural paths to the same item of knowledge. If one path were to fail, you could still arrive at and trigger that knowledge from another route. The ability of mind maps to help you remember works using a similar idea. By looking for and noting these connections in your decks this you add a layer of redundancy to your knowledge representation that further insures against future forgetting,

The seventh rule of Janki keeps your decks in shape.

“Delete or modify any incorrect, outdated, difficult to remember, incomplete or unnecessary cards. Update existing cards as improved understanding dawns.”

The following shows a poor understanding of Git, later updated with a more nuanced understanding.

Problem 8: We Do Not Reference Primary Sources

Graphic designers, artists, architects and other creatives keep scrapbooks of designs they like. Later, when working on a project, they draw inspiration from these same scrapbooks.

As a programmer your primary source of inspiration will be other people’s code. Download open source projects (e.g. on Github) and read through the code. Look up parts you don’t understand, and create new cards as necessary. Get a pen and paper, and sketch out the paths through the program. What are the inputs, and what are the expected outputs? What happens to the data as it moves through the program? Why was it designed in this way? What techniques were used?

By reading code you will be exposed to various styles and of programming and solutions to programming problems. You will see things you like, and lots of things you don’t like. With time you will find your own style.

The eighth rule of Janki encourages you to use your readings of other people’s code as a source of learning:

“Read code regularly. If you come across something interesting – be that an algorithm, a hack, or an architectural decision – create a card detailing the technique and showing the code.”

Janki Method: More Than Just A Memory Aid

My initial goal with using Anki to learn was to stop myself from forgetting, and Anki delivered on this promise. What I wasn’t expecting was the secondary advantages that emerged through long term daily use.

Advantage 1: Program Faster

Knowing thousands of commands saves time otherwise spent looking up reference materials. You instantly recall previous solutions when faced with a problem, and dozen of possibilities spring to mind when architecting a system. You will read other people’s code rapidly, confident in your understanding. The closest analogy is fluency in a natural language. You will speak code.

Most of the time I spend programming is spent debugging, that is fixing errors in your code or in the code of other programmers. This is where a deep knowledge of the platform will speed you up the most. Debugging consists chiefly of gathering information. What was the system supposed to do? What data was in the system when we saw the error? What could have went wrong? Frequently a single delinquent line of code can be the cause of a bug that takes down an entire site. The hard part is figuring out which of the thousands of lines of logic caused the issue.

Let’s take an example. Say your program uses 100 functions and 70 of these are built into the language. That means that there were 30 custom defined functions. If you know with confidence the inputs and outputs of the 70 built-in functions, then debugging is simplified, since you will only need to figure out how the remaining 30 custom functions work. Considering that custom functions are themselves defined in terms of built-ins, this task should not take too long either. Total certainty about the syntax, inputs, outputs, and uses of the built-in functions lets you quickly isolate unknowns and bring the bug to surface.

Brainstorming ability is improved through Janki Method. Deep knowledge in mind means you can try out alternatives on paper rapidly and so increase your chances of coming across a cheeky non-obvious solution to your problem.

You might argue that you will get the same speed boost with knowledge earned through experience. That is true, but this argument misses the point. Janki enables you to gain that same knowledge faster, and retain it for longer. If you’ve ever forgotten how to do something after leaving it aside for a few months, then Anki is for you.

Advantage 2: Chunking

Skill in a technical field is the product of your intelligence and your knowledge. Weakness in one can be overshadowed by strength in another. Bright yet inexperienced can perform on par with dull yet experienced.

This point is best illustrated by example. Let’s multiply 130 by 30 using two approaches.

Approach 1: You know that multiplication is repeated addition, and so the problem becomes 130 + 130 + 130… and so on. Applying the rules of addition you arrive at 3900 – eventually. Let’s call intelligence your ability to carry out these individual calculations quickly.

Approach 2: Perhaps you remember from your multiplication tables in school that 13 X 3 is 39. You might also remember that the commutative and associative properties of multiplication let you simplify 130 X 30 to 10 X 13 X 3 X 10 and finally to 39 X 100, which can be calculated trivially as 3900. Your intelligence, or speed of addition here, is irrelevant since domain knowledge let you simplify the problem dramatically.

In approach 1, the problem was labour intensive because you did not use knowledge to ease your calculation. This wasn’t the case in approach 2, where you reasoned using higher order chunks of knowledge, so solving an otherwise lengthy problem with ease. The application of remembered chunks of knowledge to problems at hand is called chunking.

If you remember the answer to a question, you have no need to deduce it in the moment, and so your mind is free to deduce more abstract concepts. This is the value of chunking.

Janki Method encourages chunking. This was an unexpected benefit, yet ultimately the most useful to me. After six months of daily application you will reason about programming concepts using mental shortcuts equivalent to those used to simplify the multiplication problem in approach 2. Maximize chunking by adding increasingly abstract rules and concepts to your decks, and making an effort to draw on these ideas when solving problems.

Advantage 3: Searchable Archive of All Your Programming Knowledge

Has your computer ever spat out an error message and, although you remember seeing it before, you don’t remember how to fix it? Before I started Janki Method this would happened to me lots.

The first time I saw the issue I would spend half a day solving the problem. Six months later the problem would happen again, perhaps in a slightly different form. Even though I was vaguely aware of having seen it before, I’d forgotten how to fix it.

This felt wasteful and I didn’t like it. I don’t like having to solve the same problem twice. Janki Method helps prevent this from happening..

Anki has a search feature that finds all matching cards in your deck. You can even filter by date or tag. For example you might search through cards containing the word “ssh” created more than 6 months ago and tagged with ‘deployment’.

After solving a bug you should always add some cards to your deck containing the knowledge needed to prevent that bug from occurring again. Better yet, abstract one level and add cards containing the knowledge needed to prevent that class of bugs. Now, whenever you are faced with a bug the second time, all you need to do is search your archives.

Advantage 4: Not Reinventing The Wheel

Modern programming languages have many algorithms built in, or easily included as parts of a readily available core library. This is especially true of high-level languages such as Ruby, and Python. You want to sort an array? Array#sort does the trick.

If you didn’t know the sort algorithm was built into the language, then you might write your own in its place. When you do so you risk making mistakes and missing out on low level optimizations. Furthermore you alienating future users of your code, who might think there was a reason other than ignorance for why you wrote a custom sort algorithm instead of using the one built into the language.

By filling your deck with built-ins from your chosen languages you will be less likely to accidentally implement existing features in a confusing, half-assed, and bug-prone way.

Appendix

Should I Download Other People’s Decks?

Anki has an online searchable database of other people’s shared decks. Although there are not many decks for computer science, you might be able to find some and piece these together as the basis for your deck.

Be careful with this. You might add facts to your deck that are of no use to you, and you may end up reviewing facts that you do not fully understand, thus building a poor foundation for future knowledge.

Ideally you should build your own decks from scratch, drawing from your programming experiences; your readings of other people’s code, textbooks; viewing educational screen casts; and reviewing past mistakes and previous projects. By creating your decks in this way, you keep them filled with well-understood real world contextual knowledge, phrased and categorized in a way that makes sense to you.

If you are going to use other people’s decks make sure you understand every card you learn. Be sure to try the code out when you first see it; this takes more discipline that you might have. Make sure you trust the author of the deck – you don’t want to learn incorrect or dated knowledge, and sadly most of the decks freely available are abysmal.

I have a deck with close to 6000 cards which focuses on the Ruby on Rails technology stack, including cards on Ruby and Rails 3.1, the HTTP protocol, the Unix command line, Git, CSS, JavaScript, VIM, Rspec, Capybara, TDD, Coffeescript, jQuery, coffeescript, sass, deployment, design patterns, regular expressions, SQL, refactoring, and algorithms. The knowledge is cleanly tagged, nicely formatted and up to date. It’s everything I’ve needed to build, manage and deploy a variety of web apps. I’m considering cleaning it up for public consumption and releasing it. Since this would be a lot of work on my behalf I’d want to be assured of public demand first. If you are interested in downloading my deck then tell me by sending the message “i-want-your-decks” to (@jackkinsella) on Twitter. If I get enough interest I’ll release them and tell you when they are available.

What Knowledge Belongs in my Decks and What Doesn’t?

When I first began Janki Method I was over-zealous in my addition of facts to my decks. In particular, I added stacks of cards for third party code libraries. This turned out to be wasteful, seeing as I rarely used this knowledge. Technology moves rapidly, and today’s in-vogue libraries become tomorrow’s baggage from the past. I ended up deleting many of these cards after having wastefully committed them to long-term memory.

It’s not always easy to distinguish between knowledge worth keeping long term and knowledge only needed short term To help me with the decision I came up with this schema:

Only enter a card into your decks if one of the following conditions is met:

a) The fact contains general wisdom about computer science (refactoring, regular expressions, object orientated programming, memory issues and so on).

b) The fact relates to a major technology that has been around for a long time and is probably here to stay (Linux, Ruby, Javascript, VIM, git, HTML, CSS, SQL)

c) The fact, while part of a library that is likely to change, is something I use everyday and something I foresee using for some time. Bonus points if the technology is in high demand, since this equates to higher consulting rates. (Rails, jQuery).

What are the main keyboard shortcuts I need to know?

You will be reviewing your cards everyday so it helps to small time savings here quickly compound. Keyboard shortcuts are crucial.

Press Enter to reveal answers and then evaluate your response with the keys 0,1,2 or 3.

Command + Backspace deletes a card. Use this for troublesome/incorrect/out of date cards.

Command + E lets you edit a card and then Tab moves the cursor between the question and answer panes.

The Janki challenge

Want to learn to program and start building powerful apps? Then take the Janki challenge. Download Anki (link below), pick a programming project, and publicly announce to your friends that you’ve committed to learn to program.

Why public commitment? Public commitment creates social pressure that pushes you to honour your promises to the world. This pressure will see you through the difficult first stages where you will be most tempted to give up.

I believe that if you follow the system exactly as described above, you will learn to program significantly more quickly than you would using any other approach other than one-to-one tutoring.

In San Francisco? Let’s hang out.

I’m visiting San Francisco during December 2011, so get in touch if you’d like to meet for lunch/coffee/beer. I’m looking to take on new consulting projects, or just to have a friendly chat with local techies about programming, entrepreneurship, making money online, and all the usual suspects. Get in touch; I want to meet you.

Why Bother With Cucumber Testing?

2011-09-26T00:00:00-07:00

I recently heard an account of a web development project from both the point of view of the consultancy doing the work and the client. A member of the consultancy told me how they educated a technologically backward client about agile processes. Although this might be tedious for the client initially, with time the client would appreciate their wisdom. Glowing business referrals would follow forever more.

A few days later I bumped into the client:

“[X] were a nightmare to deal with. They wouldn’t let me have what I wanted, and they wasted my time arguing over petty details. Although they did the job well, they had no business sense. I can’t see them lasting much longer.”

The consultancy screwed up: Amongst other things, they pushed the client to use a process that wasn’t appropriate. In particular, they gave the client Cucumber feature files to read and approve, even though the client didn’t give a damn. In the client’s words “all we wanted was a website”. (It was actually a web app but most non-technical people don’t draw a distinction). The client just wanted it “to work”.

Imagine You Are The Client

Step out of your programmer skin for a moment and pretend you are a busy business owner while you read the following:

Feature: In order to let customers organise their information across themes on various pages. 
         As an administrator of a micro-site
         I want to be able to add subpages

Scenario: Adding a subpage
  Given I am logged in
  Given[sic] a micro-site with a home page
  When I press "Add subpage"
  And I fill in "Title" with "Gallery"
  And I press "Ok"
  Then I should see a document called "Gallery"

Scenario, but not feature description taken from You’re Cuking It Wrong. (Note that Jonas doesn’t give his clients Cukes to read.)

According to Jonas Nicklas at E-labs, the above is an example of acceptable Cucumber style, written in the language of stake-holders. As a past consumer of development services, I have to disagree with him. The appropriate level of detail here is this:

  I can add subpages to my micro-site.

Any more is superfluous. If it takes you ten lines to communicate the idea of adding subpages, then you’ve wasted my time. I’m not alone in thinking this. BDD expert Elizabeth Keogh tells us:

“If your scenario starts with ‘When the user enters ‘Smurf’ into ‘Search’ text box…’ then that’s far too low-level. However, even “When the user adds ‘Smurf’ to his basket, then goes to the checkout, then pays for the goods” is also too low-level. You’re looking for something like, ‘When the user buys a Smurf.’”

If you take this view, then the overwhelming majority of the Rails community have been using Cucumber incorrectly. Despite believing otherwise, most programmers never wrote a single acceptance test; instead they wrote integration tests using the Cucumber syntax.

This is a damning claim and so I offer evidence. In my experiences both as a Ruby contractor and as an employer of programmers, most feature files I’ve seen are composed of the web steps included with Cucumber by default. These web steps are integration tests in disguise. Cucumber steps such as “When I fill in ‘search’ with ‘Smurf’” are rampant in the Rails community, despite their position at the most fault-worthy level in Elizabeth Keogh’s schema.

My own anecotal experience isn’t enough so I did some research. I browsed through the feature files in six major open source Rails projects, including Spree, Radiant and Diaspora. All bar one, Tracks, wrote their feature files near exclusively in web steps. In effect they wrote integration tests using the Cucumber syntax.

The Make-Believe Analyst

Let’s consider the above feature file again, imagining that you are the client. You run a business and so are conscious of costs. You might reasonably ask what’s with all this ‘in order to’ stuff? Why is this developer playing make-believe analyst? Doesn’t he know that you’ve already communicated that you’ve determined the business value when you approved development? And, most importantly of all, is the developer charging you for this work?

The Cucumber way says you sit with your client and determine, feature by feature, the business value that each piece of functionality serves. That’s cool, but it isn’t a realistic job description for many programmers. High level analysts and consultants might sometimes do this, but the likelihood is that you aren’t acting as a high level consultant or analyst on your current project, and so Cucumber is inappropriate. Not only this, but using Cucumber is downright wasteful considering its cost.

The Cost of Using Cucumber

1. Cucumber breaks text editors

Text editors, like VIM, multiply productivity. Auto-completion eases the use and reuse of descriptive method names. Compilation checks prior to saving catch syntax errors before they cause harm. The editor’s ability to understand the signature of a method makes tasks such as finding method definitions and uses simple.

Cucumber’s steps are method names written using the Gherkin syntax, and this unique syntax breaks text editors. Steps contain white space, include their parameters at non-standard locations (When “john@gmail.com” has “4” unsent messages), and use regular expressions for pattern dispatch (“and|or”). Text editors are not adapted to deal with this, and so auto-complete, search and many other features break, damaging productivity.

2. Cucumber Requires Maintenance of a Second Testing Environment

Anyone thinking of using Cucumber for acceptance tests most likely already unit tests using something like Test::Unit or Rspec. When a project’s complexity grows, we organise test suites by placing shared test code into helper methods and eventually into modules shared across many test files. We also categorise tests using tagging systems, use tools like Spork to speed up test startup time, and use watchr or autotest to run tests automatically. We add gems to the test environment to make use of advanced helper methods that freeze time, open up emails, or fake web requests.

Cucumber doesn’t like to share. It does not pick up existing test configurations or helper methods. Our taxonomy of tags carries no weight and so we must mirror our existing setup in our new Cucumber world. If we are undisciplined about refactoring we might duplicate code, denormalizing our code base. Even if we are disciplined, we will, at the very least, increase the complexity of our project and thus the scope for error. The parallel worlds of Cucumber and our unit tests then need to be maintained, and the creators of testing gems must now care for two masters. Cucumber takes another toll on the community’s productivity.

3. Cucumber’s Routing Causes Cognitive Strain

Cognitive strain refers to the total weight of facts and rules we must hold in our minds to be productive using a technology. Rails imposes a high cognitive strain. When I began using it I faced a period of difficulty as I untangled the labyrinthine naming patterns of routes, the pluralisation conventions, and the subtle distinctions between various ActiveRecord::Base persistence methods. I’m not saying that cognitive strain is a bad thing; it is often a necessity in powerful tools. I am saying that cognitive strain must be justified.

In Cucumber steps, the mention of a specific web page, for example “the login page”, requires us to map this stake-holder description to a url helper method our app understands.

#features/support/paths.rb
    when /the login page/  
       login_path

The Rails routing system is complex: in The Rails Way Obie Fernandez suggested that you could squeeze everyone who understands it into one taxi. Cucumber adds an abstraction layer over this already complex system, and this leads to slower development and increased error on larger projects.

A problem I faced when adding new features was that I would unintentionally describe paths using differing natural language descriptions. Perhaps I’d call “the login page” the “the sign in page” one day, and “the log in page” another. When I did this Cucumber complained that it could not find the permuted path name, and so I had to look inside the paths.rb file to remind myself of my previous phrasing. That’s a lot of remembering, and a lot of room for error and slowdown.

I’ve experimented with removing the paths.rb file and writing logic that automatically determines the route based on humanized url helpers. This leads to unacceptable step definitions such as “And then I am on the new user session page”. This is far too close to implementation language to be understandable to the average stakeholder, and so this option must be avoided if we’d like to use Cucumber in the way it was intended.

4. Cucumber’s Organisation Defaults Are Impractical

My favorite feature of Rails is one of its simplest: everything has a place. Mailers go in one folder, models go in another and configurations go somewhere else. Presuming that a project sticks to convention, you can find the source code for any function effortlessly. This brings enormous productivity advantages, especially on projects you inherit from other programmers.

Cucumber asks you to create step definitions for any custom steps you use. The convention is to keep these steps in a file {feature_name}_steps.rb. This convention makes sense on tiny projects, and step definitions are easily found. Problems start once you need to reuse step definitions across many features and you want to keep code DRY. Two things typically happen:

1) If you’ve got a good memory you find the old step definition and use that method again in your new feature. This can be a terrible move. The shared helper’s presence in promote_post_feature_steps.rb file suggests exclusive connection with that feature. If you later remove the promote_posts feature file you will probably remove its step definition file too, having forgotten that it contained global step definitions. Ideally you should have extracted these shared step definitions to a global step definition file, but realistically this doesn’t always happen. We’ve all got deadlines.

2) If you don’t have a good memory, or it has been a long time since you last visited the project, you might have forgotten that you previously wrote a step definition for promoting a post. Alternatively you might never have created a step definition but another programmer had done so without you knowing. You write a fresh step definition in taxon_steps.rb, but now your code is no longer DRY. At a later date you might be refactoring the promotion logic and make the changes in promote_post_feature_steps.rb, and then assume you had finished refactoring. The taxon_steps.rb now contains an out of date version of the logic for this step, and this will lead to confusing test failures, since you have no good reason to suspect an issue with the recently refactored promotion logic. Cucumber’s default way of organising features sets you up for these difficulties, and it takes of discipine to avoid them.

5. Increased Wordiness

Do you consider one of Ruby’s advantages its brevity compared to other languages such as Java? Do you prefer Sass, Coffeescript and Slim to CSS, Javascript and ERB for the same reason? Then why do you persist in using Cucumber?

Method names in Cucumber need twice as many characters as their plainer Capybara equivalents. Compare:

Given I am on the home page #27 characters

visit root_url #14 characters

Increased wordiness reduces expressiveness and power, and introduces error since more characters means more places a typo can appear.

6. Cucumber Syntactically Discourages Code Reuse

Cucumber users tend to write their step definition in terms of other step definitions. The library creators have implicitly encouraged this by creating simple means for doing so. Here’s an example:

When /^I toggle the full sample on the “(^”.*)” product/ do |product_name|
  steps %Q{
    When I am on the last upload page for “#{product_name}”
    And I follow "full_samples"
    And I press "toggle_full_sample"
    When I refresh the page
  }
end

This is an abomination, albeit one that is understandable. A developer wants his code to remain DRY but he doesn’t want the added hassle of encapsulating the step definitions in Ruby methods. So, he reuses his step definitions.

But consider this: When you are inside a step definition you are exclusively in the domain of the programmer. The stake-holder never reads this. With this in mind, wouldn’t it be advantageous to use the brevity, precision, composability, editor support and abstraction tools of the full Ruby programming language? Using Ruby, you will be able to do things like this:

When /^I toggle the full sample on the “(^”.*)” product/ do |product_name|
   admin do 
        feature_latest_upload 
        refresh_page
        should_see(sample_path, "features")
    end
end

The reuse of step definitions within other step definitions, and the composition of higher level steps using the web steps included with Cucumber, are, in my opinion, the main reasons why so many companies attempt integration tests only to abandon the effort within a few months. The tests become too difficult to maintain and get of sync. The solution to this pain point isn’t simple, but the most sensible course involves use of more powerful tools of abstraction – such as the full Ruby programming language over awkward step definitions.

Conclusions

Cucumber has its uses, principally as a high level analysis tool on large, polyglot projects. That said, few programmers work in this kind of position, and acceptance tests beyond a list of the method names of regular integration tests seems wasteful. Cucumber, as used by the majority of Rails programmers, is no more than a clumsy wrapper over basic integration tests. The differences are not just cosmetic: Cucumber’s syntax is costly, both to the programmer and to the client, whose time and money are wasted. Furthermore, the use of Cucumber in open source software intended for technical users and its use in solopreneur efforts is downright ridiculous. Nevertheless, programmers continue to use Cucumber inappropriately.

Why not admit to yourself that you don’t do acceptance testing and that you do not need it in your projects? Swap Cucumber for pure integration tests using Capybara, and you’ll be surprised by how much more productive you can be.