Solving usability problems

In a previous post I explained why you should do usability testing. Something that I neglected to explain is that doing usability testing shouldn’t trigger mass rewrites of code or entire redesigns of functionality. Once you’ve got your results and have them prioritised you should: fix the biggest problems, with the least effort, that can be solved for most people. After these issues have been fixed, try testing the system once more and you’ll be able to gain better feedback on what you might be missing.

Tweak or Redesign

The most obvious benefits of tweaking include: reduced cost, less work and faster completion time. However, from a project management perspective, tweaks are much more likely to happen as they don’t require as much planning, client meetings or questioning to get signed off compared to big overhauls. In addition to this, most people aren’t too fond of entire redesigns, so a tweak might mean less of the user’s time needs to be spent on understanding what’s changed.

On the other hand, too many tweaks can make for a confusing system. By adding too many tweaks at the same time you may sacrifice the navigation or continuity of the system, leading to long-term issues. Sometimes a well thought out implementation can produce better results than a bundle of tweaks that are poorly executed.

Tweak and then refactor

You’re never going to end up with perfect solution on the first try (maybe you do, I don’t know), but it’s always best to iterate and then clean up. Personally, after every five tweaks that I implement I tend to try and clean them up where possible. My thinking behind this is that five tweaks aren’t so many to cause issues down the line, but  any more may give you problems when adding functionality at a later date.

In conclusion, while tweaks tend to be frowned upon they are a necessary evil. As long as you clean up what you’ve done on a regular basis, you won’t cause yourself too many major issues.

My challenge was trying to keep things fun, while still learning how to program. Over the years I’ve read numerous technical books, but none compare to the accessibility of Why’s (poignant) Guide to Ruby when it comes to understanding the mindset of a language. The bonus of this would be that a lot of the knowledge learnt would be transferable to Python as well.

Learning to program was a secondary aim

When I learn a new programming language I tend to read a book or a set of tutorials end-to-end, completing every exercise and example until I understand it. In comparison, the poignant guide reads like a story rather than sticking to technical examples. You’re taken through different landscapes with bacon-craving foxes, granny bombers and floating whales. While you’re laughing at chunky bacon jokes you’ve already learnt the basics of instance variables, class methods and method arguments. The beauty of the poignant guide starts to shine through when you see the beautiful comics. At first they seem messy, but the style adds to the stories making for a much more compelling learning experience.

As a side not, while I find the comics endearing I can understand why many programmers would be put off. If you’d like to get started with Ruby quickly and are confident in your core computer science knowledge this might not be the book for you, but if you’re a beginner or are about to teach someone the basics this might be a great choice. As an alternative, you might be better off trying the pick-axe book for Ruby.

Hello World!

We both breezed through the poignant guide within a day, allowing enough time (another day) for me to try and apply some of the knowledge to Ruby and gaming. We both sat down and started to build a basic text adventure game and began by typing out some of the common features of the game. It’s a puzzle game where a player moves through different rooms fighting demons by farting on them. I initially spent an hour building out the classes for each of the objects in the game explaining why this is useful, after which we switched seats and I helped with any unknowns. Some of the programmers out there might call this pair programming.

Marcus was a great student and was able to get the game finished in less time than I expected. Probably the most enjoyable part of using the poignant guide to teach programming was that I didn’t have to state “use an array” and get a confused look back. Instead I would refer to the different components as they were laid out in the stories, such as caterpillars (arrays) and loud words (destructive methods).

Where do we go from here?

It looks like I’ll be heading back next weekend to add to the core Ruby knowledge. My aim is for Marcus to be using Python and PyGame by March and I’ll start teaching some simple habits to make the switch as smooth as possible — whitespace is probably going to be the first of these.

If you’re ever asked to teach someone how to program I can’t emphasise enough how useful the poignant guide will prove to be. In addition, using Ruby meant that the usual traps that most new programmers fall into — forgetting semi-colons, curly braces and brackets — never see the light of the day.

If possible, I’ll try to follow this blog post up and track Marcus’s progress. Should he also consent I might put some of the source code up should you wish to get a better idea of how a text adventure game could be laid out in Ruby. Chunky Bacon!

I feel the process of usability testing can be summed up in these four five steps:

1. Watch someone use your product
2. Take note of any issues they have
3. Reward/de-brief the participant for their time
4. Fix the usability problems encountered
5. Go to step 1

While usability testing has been defined and re-defined numerous times, the basic principle is simple: you watch people use your product. Coming from a development background I found this hard to swallow as I was of the opinion that people should use (and test) the product when it’s finished. I was regularly doing app-based unit testing to make sure that the core logic was always working as expected, but I didn’t think that giving an incomplete experience would at all benefit the final outcome.

Testing on a budget

Having recently read Steve Krug’s Don’t Make Me Think, I opted for a DIY approach to testing. The project was around mid-way through design and implementation and due to the overall complexity involved in the product I felt that a usability test would be wise before moving forwards. Being a member of the development team I found it easy enough to take notes and explain them to the relevant team members for improvement.

The test format was fairly simple. Three individuals were gathered who had no previous history or knowledge of the product. They were each (separately) sat down in a room with myself infront of a computer. They would use the product (an application) and would be asked to complete a series of tasks and give a play-by-play commentary on what they’re doing and thinking. For the purposes of this test, a set of ten common tasks were set as the base-line to make testing fair amongst the different participants. In case you were wondering, Silverback was used as the recorder.

Why was it useful?

After voicing my concerns about possible difficulties in using the UI, my opinions were shot down as I wasn’t the target audience. Having completed the usability test one thing was made blindingly clear to the design team: their core concepts weren’t easy to understand. Despite some features being unexpectedly beneficial to the test participants the core of the design had basic problems, which luckily could be dealt with this early on.

Initial reaction was frustration and anger, given the shear amount of time that was spent on this app, however within a short while it became a competition to see who could spot further issues to do with speed and sight when going back over the test videos.

I can happily confirm that the team I worked with in creating this application have been more than happy with my thoughts and findings on usability testing. So much so in fact, that they’ve made it a weekly occurrence! Participants are dragged in from all areas and rewarded with vouchers and free food. The money spent on them clearly outweighs the lost sales and man hours that would occur when attempting to do changes last minute or after release.

If you haven’t done any usability test yet, what are you waiting for? The cost incurred for the above was around £30 total (Silverback was used while still on a trial license) and the benefits to the business were a great deal more than that. Happy testing.

As a side note: It’s interesting to consider how many people’s professional lives have been shaped by the software framework that DHH released to the world. While many argue that 37signals needs to be more open about their business figures and research methods, it’s impossible to negate how they’ve created jobs (growing the industry?) without ever dilluting their revenue.

I recently graduated (June 2009) from University and luckily it hasn’t been very difficult to find work – either full-time or freelance. The tech industry seems to be one of the few that saw a boost during the recession. Whilst many were being laid off from work, those same companies sought expertise in building systems to automate and streamline as many tasks as possible. As a result, there are much more tech jobs than a few years ago and many of my friends who work in recruitment can attest to this. Gone (hopefully) are the days where candidates have to outright lie on their job applications and instead are chosen on their ability to create. Whether it be open-source projects, personal websites or even public repositories, there are a number of ways that a new computer science graduate can show their worth.

DHH takes this one step further and claims that it’s an opportune time for any new start-up (business) to gain traction. The reason given is that when the economy is steady, consumers are more likely to go with the tried and tested (read: expensive) option, but when they’re in a recession there’s a greater chance of trying out new products and services. With this in mind, should any graduate have difficulty finding a job, perhaps it’s time to start on that business idea they’ve had for some time? Rather than plan out how they’re going to run this new business, they should just do it.

Planning out the next five years of a small business is clearly just harmful guessing. A significant change will happen when the business starts to get things done, not by planning for them. Spending time procrastinating and pretending to work is never going to help anyone and is going to be especially detrimental to any employees of the small business. Small business owners need to stop relying on fancy projections to save them from their bank managers. Instead an emphasis on completing tasks and helping employees needs to be made. </rant>

It’s very rare that a company is an overnight success and in a few cases, the business may have just been working it’s way up to release for some time. However, unlimited time and money never lead to great products. By limiting the amount of time you have to go live, it’s likely that you can strip out any extra noise from the product/service and produce early iterations that will be indicative to it’s core features and/or functionality. In short: great, simple products!

Personally, I’ve found limiting the amount of time I spend learning new languages has forced me to concentrate more in that short period of time and has accelerated my learning to no end. In addition to this, I recommend anyone learning a new programming language to combine their progress with the problems at Project Euler. I’ve found it interesting learning how different languages treat the same set of problems and am indebted to that community for their comprehensive break-downs of elegant solutions.

A prime example of this is my current phone — Nokia N73 — which comes with 32MB of built-in storage but is expandable using the provided miniSD slot. I thought this would be brilliant! I could store applications, images, messages and contacts all on the miniSD card and never have to worry about transferring any of this data, phone to phone, ever again. As always, there’s a catch. I’m limited to storing just media files on the miniSD card and all other data is stored on the phone’s paltry memory. The reasoning provided is for security, but surely I — as the user — should be able to choose where my own personal files are stored?

In much a similar way, Google is failing to understand the same issues that are plaguing my three year old phone. While the Nexus One has an expandable Micro SD card, the on-board memory is just 512MB. This wouldn’t be too bad until you take into account that the Nexus One is a smart phone targeted towards users who will take advantage of it’s many applications. Sadly, a limit has been placed on how much storage can be used for applications to 190mb:

The Nexus can accommodate memory cards up to 32 gigabytes (a 4 gigabyte card comes with it) — and yet, inexplicably, the Nexus allots only the tiniest sliver of that (190 megabytes) for downloaded apps. Source.

Why are Google so insistent on messing up the storage situation on Android phones? The N1 had 512mB memory built-in, which Android will only use for storing the OS, applications and any relevant application data. Similarly, the earlier G1 went through the same issues having just 256MB built-in memory, which wasn’t even enough to install any official OS updates! Instead of learning their lesson, Google has just taken it up one notch. Rather than learn from these mistakes and become a real competitor to Apple, they’ve failed miserably.

Considering the cost of physical memory these days, it doesn’t make sense when companies aren’t willing to increase built-in storage sizes. These days 4GB storage should be seen as a minimum, while allowing room for premium SKUs with 32GB+ to pile on markup without anyone complaining. Apple’s abhorrence towards removable storage isn’t just because Jobs’ saying is final, or compromise on form factor, but simply because it’s a usability nightmare.

Modded firmware has been released for most phones, allowing users to bend the will of their mobile device’s storage. However, allowing all photos, music and applications on the removable media will mean that it has to be unmounted on the device, removing the use of any applications in the process. I understand that an easy way around this may be to expose a samba share, e.g. Archos, rather than give direct access to the storage media, but what’s to prevent a user from just ejecting the media and doing so anyway?

As a result of this fatal flaw, I have no plans on buying the new Nexus One in the immediate future. If I were to purchase a new phone based purely on usability of storage, it would probably be an iPhone. Alternatively, I could just stick with my current N73 and save myself a lot of wasted money on applications.

When the field of HCI was established in the early 1980s, learnability was the research focus. By the late 1980s we had a handle on how to design for the novice user, demonstrating that even the best commercially available personal computer was much harder to learn than claimed by it’s manufacturer.

We created computers because our ability to think using rigorous logic is limited in terms of speed, capacity and accuracy. Given these limitations, shouldn’t any interfaces we create play to our strengths without demanding too much from our weaknesses?

Programming languages go through rapid development to fit formal (hardware) specifications and have meant that usability best practices have fallen by the way-side. However, usability is holistic and something that has to be intentionally designed in from the beginning rather than slapped on at the end.

Usability studies in this area are lacking, but there are some that have managed to have an effect on the final product. More recently though, full on task forces have been springing up and providing good information on designing the interface that creates the interface.

Testing products with a steep learning curve

As stated by Jakob Nielson in his Iterative User Interface Design paper:

For some user interfaces, full expertise may take several years to acquire, and even for simpler interfaces, one would normally need to train users for several weeks or months before their performance plateaued out at the expert level

Sadly, by the time you’ve learnt how to use the product effectively, you’re too biased to comment on it’s usability fairly. This is just one of the fundamental limits on usability testing of complicated products. Despite this, there is still knowledge to be gained from testing programming languages at a novice or intermediate level.

A simple programming test could be held, with people who have the same amount of training — university students? — in different languages and then measure factors ranging from the number of errors along the way, to overall performance. A simple task may involve scraping data from a set of web pages such as last.fm and organising them into a searchable data structure. The user could attempt to use any programming language but a time-limit would need to be set to keep the test fair.

Libraries vs. Languages

Being a programmer, I tend to look for solutions by dissecting the problem into smaller chunks and in many cases that makes me lean towards libraries. It’s likely that any seasoned developer would do the same, opting for the best tools for the job rather than any one single language. Does this fulfil the efficient use category of usability? No.

Most of these libraries are built by software developers to facilitate their work, increasing the need for usability in this particular area. If something is too time-consuming, a developer will automate it. However it does then raise the question of whether the flaws lie in the language, where the expert knowledge is needed to understand how to automate a given task in the first place.

Can we have a yardstick?

There is ample evidence to support the notion that experts behave differently from novices, but that should never mean we pick one of these two groups and design specifically for them. Program designers should understand the context of use, set appropriate goals, and measure that we are achieving those goals. The myriad of different programming paradigms mean that we can’t effectively compare the usability of different languages, but that shouldn’t hinder the process of increasing usability for the lowest common denominator.

The lack of cross-over between Funtional, Procedural or even Event-based programming languages makes formalising a test harness extremely difficult. Attempting to test all of the different kinds of programming languages may produce a clear winner in terms of usability, but are likely to fail large proportions of the development community.

Disclaimer: I would never pretend to be an expert in this area, so I’m sure there is a lot of good material that I’m not aware of yet. Please let me know if you have any further useful information.