Falkayn's Nest

My geek origin story

2011-08-30T14:30:00.001+10:00

Delicate Genius (aka Microsoft’s Michael Kordahi) tweeted me a while ago about putting this up, and in the interest of historical accuracy, here it is! Other geek origin stories can be found from his blog post.

I wasn’t terribly geeky as a young lad, except I loved Lego and got into roleplaying games in a big way. At university I fell into a crowd that were much geekier than I (hat tip to Justin McLean), and along the way got pulled into doing an Information Systems major for my Bachelors of Commerce degree.

Something must have happened because by the time I met and married my lovely wife I looked like a true geek! (gotta love the Doc Martens)

Looking back I think the same qualities that caused me to fail Accounting Financial Management 1B led to my success as a geek. Things like:

Always wanting to improve the system,
Searching for truth,
Wanting my work to matter.

None of those things applied to rote learning T accounts and double entry accounting, but they sure do matter when you are building fantastic tools for web developers to build awesome, award-winning sites with.

So that’s how I ended up becoming the geek t-shirt wearer I am today – although I no longer have those DMs ...

Context may be key to blended learning

2011-06-25T14:51:00.004+10:00

I have been reading two excellent books recently, John Medina’s Brain Rules and Dan Coyle’s The Talent Code (see other Personal Transformation Books), both deal with how our brain works, and there are interesting things they tell us about learning skills that we can apply to how we structure and run blended learning programs.

I have mentioned before David Maister’s article Why (Most) Training Is Useless in the context of saying that behaviour is what we should be aiming to change, and measure, not simple skill identification. The point was that it is not good saying I want to train your consulting skills if I don’t know what behaviour indicates that you are have learnt and are applying them. Maister also wants incentives made to match those behaviours, based on seeing his training undermined by the structure of the organisation’s incentives.

Coyle breaks down two ways we can affect skills development, ignition, which is a combination of incentives/encouragement/vision and deep practice, which is the focused practicing of technique at the edge of our ability, with the aim of achieving perfection in the small components of a skill, and then building into perfection in the larger components. Maister’s complaint about incentives not matching behaviours is talking to the ignition side of skill development.

Medina’s book is interesting because it gives us lots of clues about how memory relates to instructional methods, with the main points being that repetition is important, but so is learning in the context in which we want to be able to remember what we learn. In this sense it relates well to Coyle’s idea of deep practice.

This last point is the one I want to focus on. If Medina is correct when he says that we recall things more easily when in the physical/emotional context we learned them in, then most teaching is done in the wrong context!

This suggests that eLearning is best used when the skills being taught are to be expressed online, with use of much the same tools that the learning occurred on. It is one reason learner drivers are encouraged to use their own cars when learning and when going to do their driver’s test.

Classroom learning may suit well skills that need to be exhibited in front of other people, in social or meeting situations, where the ability to perform before your peers may be required (it may also help people perform better in classrooms - any professional students listening?). Even then the layout of the room may benefit from being closer to their likely environment. When Anthony Milner and I did the excellent NIDA Corporate Performance Course the format was that of a class, but the frequent practice sessions were done in front of the group with us facing them as a very real audience, an experience that made it easier to recall the skills learnt when actually presenting in front of audiences later.

One on one mentoring may be the best approach when the skill must be exhibited in the context of a personal relationship, and even then it is best done in the workplace. This is how we handle important parts of facilitator training in the Careforce Lifekeys’ courses – the small group facilitator is given the opportunity to practice their skills in the context of a one on one conversation with a trainer playing the role of a small group member.

There is one caveat to this, both Medina and Coyle point out that the ability to comprehend the big picture is a vital component of skill competency – one of the factors that differentiates experts form beginners in a particular field is that the experts have a mental abstraction that allows them to deal with the complexities of the problem domain more easily. Getting that big picture view across need not necessarily take place in the environment the skill is exercised in – in fact it may benefit from the distance provided by being away from that environment.

Why web will win the mobile app race

2011-05-15T21:19:00.001+10:00

Everyone wants a mobile app, banks, health funds, airlines, pubs and all sorts of marketers want us interacting, playing with and using their mobile apps. This is fine and dandy, and is a Good Thing for the future of humanity, but … which OS do you want to target?

Is your app going to target iPhone, Android, Blackberry, Windows Phone 7 or (heavens!) Symbian?

The right answer will, of course, be all of the above. It really isn’t hard to figure out, and there is a precedent you see.

What do you do if you want to target users on MacOS, Windows 7, Linux (a bazillion flavours itself), Google OS or (heavens!) Unix?

The answer to that question is you created a web app, because the browser environment was designed to be (reasonably) neutral between vendors. The same paradigm exists in the mobile world. Creating a specific OS version just limits your app to that OS, and who wants that?

However there is one big difference, the mobile app needs to deal with being occasionally (dis)connected, right? Solving that problem has been hard enough that mobile apps are still sprouting up that basically show content offline.

Riding to the rescue of the beleaguered user comes HTML5 with its cache manifest offering to give web apps a completely sane way of specifying what content should be held offline and what resides online. The only problem is the memory limits most browsers place on the cache – except for Opera they all only let you have 5MB (or 10MB if you are on IE) – and in these days of fast connections and rich media that simply isn’t enough to get the job done.

There are various ways around this, with Google Gears, Microsoft Sync Framework and Flash also offering ways of getting offline storage to work, and there are some jQuery plugins that hint at the promise of getting this working in a framework that leaves the browser sensitivities to someone else (although I’m always leary of potentially leaky abstraction layers).

Personally I don’t care how we solve the problem of sufficiently large offline storage, but I think the future of web development demands that solve it we must.

In the meantime we will continue to see niche agencies offering native applications for various phone OSs, but not necessarily delivering the value the business needs because the cost is so high to develop cross-platform apps – and some other applications that target specific OS flavours, most notably Apple iOS or Android, and get away with that because the user base can be targeted that way (for now).

EDIT – 23rd Jun 2011

GigaOM recently weighed in, telling us that native mobile apps were beating HTML5 ones. One commenter, Roshan Shrestha, mentioned that:

“I see that many of the apps are just a wrapper against an HTML browser component. Most of them do not store much data locally and require internet connection, so these are basically web apps.”

I agree that most apps could be web apps, and I think it will end up there, but not yet. In the meantime everyone needs an Android app, wants an iPhone app and should have a Windows Phone 7 app.

Arthur Douglas Burgess

2011-03-10T14:44:00.001+11:00

I just got delivered a very nice present. The National Library of Australia maintains an online archive of Australian newspapers for the last hundred years or so called Trove (http://trove.nla.gov.au/). A couple of weeks go I found a reference to my grandfather on my mother’s side, and a few days ago I ordered a copy of it via PDF. Thanks to the miracles of scanning + the internet, here it is:

My other grandfather was also involved with Fairfax and the Sydney Morning Herald, but in the capacity of a typesetter on the printing presses.

Is knowledge all there is?

2011-03-09T14:02:00.002+11:00

I spent a fair bit of time this week helping a client with their information strategy and information management policy. Their definition of information included “Emails, Databases, Documentation and Knowledge”, which is a mixed bag if ever I saw one!

Clearly they needed to get a better handle on what they were dealing with, so I introduced them to a little pyramid that I had worked on several years ago (back when I was thinking about pursuing a career in knowledge management). I call this the Wisdom Pyramid, and use it to help differentiate between raw data, meaningful information, contextualised knowledge and applied wisdom.

Wisdom Pyramid

The example I usually use to bring it to life is that of a traffic light changing colour from amber to red.

At a data level there is a single bit of information, isolated from context and basically without meaning, unless one is familiar with that particular data type.

That data becomes information when meaning is given to it so that a human can more easily understand it.

The information becomes knowledge when context is considered, in this case that the traffic light is one I am heading towards.

Wisdom is exhibited when that knowledge is applied to my situation, so that I stop the car at the red light.

The pyramid is pretty useful, although there is a catch with wisdom as we only call an action wise when knowledge is applied correctly to a situation. Incorrect application is at worst foolish, and at best thoughtless.

Much ado has been made about the management of corporate knowledge, especially the attempt to capture explicit knowledge, although tacit knowledge is also sometimes acknowledged as something that must be transferred. The real issue however is how do we inculcate wisdom into our staff so they make wise decisions and not foolish ones?

“Take hold of my words with all your heart; keep my commands, and you will live. Get wisdom, get understanding; do not forget my words or turn away from them. Do not forsake wisdom, and she will protect you; love her, and she will watch over you. The beginning of wisdom is this: Get wisdom. Though it cost all you have, get understanding.”
Proverbs 4:4-7 (NIV translation)

You grow wisdom by growing people, and that’s where most knowledge management should start – tools are useful (and Elcom has some good KM tools) but mentoring, teaching and encouraging wisdom in our people is where the real benefits come from.

Fresh design!

2011-02-11T09:19:00.002+11:00

In case you’re reading an RSS feed, I have updated my blog design to use a core Blogger template, and done some (slightly) artistic tweaking to get a look that I’m happy with.

Occasionally (Dis-)Connected is the Future

2011-02-10T10:32:00.002+11:00

As web application developers we tend to build for always connected scenarios. They simplify the problem of web development and allow us to keep the complexity of our solution hidden on web and application servers back at corporate headquarters.

Pity the poor rich GUI developer who needed to handle both connected and disconnected modes – mainly for corporate management with laptops. This requirement increased the complexity of their applications and forced sharing of business logic with the client GUI layer. Recently mobile developers have realised the need to provide the same service, and with HTML5 web developers are now being pushed to as well.

At the same time, the ability of any client machine to get connected is increasing dramatically. Modern cities provide plentiful sources of free, commercial public and private wifi services, and most modern telecommunications devices allow for secondary use as internet modems.

It is now clear that most web applications/websites are likely to be accessed via mobile devices, and there are far more people with internet access via their mobile devices than via landlines/roaming broadband.

One might wonder whether disconnected clients still need to be supported, or whether the vast majority of clients in the vast majority of locations are best handled by assuming connected, always-on access?

The problem with this thinking is that it ignores a fundamental truth:

“Space is big. Really big.” – The Hitchhikers Guide to the Galaxy

OK, so these are not all apps that will be used in space (has the first iPhone gone to space yet?), but they can’t all assume that the user is always connected.

There are two fallacies with the premise of always-on access, the first is assuming that most people’s situations mimic the developers’ own ultra-connected, hyper-geek, lives, the second is limiting “everywhere” to the points on the daily commute. For most people in the world, in most places, there are going to be frequent losses of signal – even in Australia most telcos can’t get reliable mobile signal to all urban locations, let alone cover the vastness of our Outback.

Of course mobile developers know this is a problem, and might decry that their app can handle the phone losing and resuming signal flawlessly. But is there still value in the app when it is disconnected?

The most interesting opportunities lie with providing access to applications that can still provide value during the occasional disconnection – these are the applications that will be truly useful all the time and everywhere. For example the mapping application on my Nokia N85 can operate with three levels of connection:

Connected to the internet and GPS.
In this mode I get the latest map updates and can tell where I am on the map.
Connected to GPS only.
In this mode the maps used are the ones stored locally but my current location is derived from GPS.
Disconnected.
In this mode the maps are available to be read, queried or manually moved around on, but my current location is not available.

Even in the most disconnected mode the maps are useful, I can lookup addresses and perform most of what I would have with an old-school street directory. When they connect further I get increasing amounts of value. Because of this sort of disconnected operation, the map and email applications are the most useful aspects of the phone to me (the other is the clock). They are more useful than a static application because they can occasionally connect and update themselves.

There is a term for this sort of behaviour that web developers have coined with websites that offer substantial value for all browser clients, but increase their value for some special browser clients.

Progressive enhancement

The point is to view the extra potential as something great to have, but not necessary for the application to fulfil its value proposition.

I really think this is the killer feature for a mobile app – provided of course that there can still be a value proposition in the disconnected mode. I also think that the best mobile apps will be web apps that respond and adapt to the restrictions of the mobile space rather than custom built ghetto-apps that can only prosper in one particular mobile OS. But that can be another post …

Dependency injection our way

2010-11-17T14:12:00.002+11:00

Like most everyone else practicing automated unit testing the developers at Elcom Technology have the issue to deal with of how to handle dependency injection – that is the exposure of a given object’s dependencies on other objects outside of the object so that the calling object can be made responsible for acquiring those dependent objects. In general this is most easily done by exposing said dependencies in the object’s constructor as required parameters – thus ensuring that the calling objects knows it needs to supply them when new'ing the object. So far, so good.

Recently (as of CM v7.2) we have introduced the Elcom.API library in an attempt to use the adapter pattern to present a more cogent, safer and easier to use interface to our CommunityManager.NET web CMS product. This is all well and good, but it means that to ensure testability we now have chains of dependencies throughout our application code – and declaring something a dependency of object A, when it is required by that object only to call object B, which then only needs it to call object C, creates some very smelly code.

Clearly we needed a dependency injection framework of some sort, but what should we use? Like Uncle Bob we didn’t want our DI framework to leave code smell throughout our projects. Well, we already have an object with all static/shared methods called Current. We use this to give an easy way of accessing some current state items, such as:

Dim a = Current.Configuration.ConnectionString
Dim b = Current.Configuration.EncryptionSettings
Dim c = Current.Configuration.ShortDateFormat
Dim d = Current.HttpContext
Dim e = Current.UrlId
Dim f = Current.User
Dim g = Current.UserId

Now we want to expand that to offer this sort of syntax to give easy access to services/repositories:

Dim myContSvc As Content.IContentService = Current.Service.ContentServiceGet()

That would give a simple way for object B to instantiate an IContentService without needing to declare it as a dependency to object A (but object C should still declare this in its constructor). For this to be truly useful, we have to be able to wire up the Content.Service container at runtime from either a configuration file (the default behaviour) or from code (during testing).

Sidebar: On DI Containers

To digress briefly, you will notice that the Current.Service container is a custom-built one, rather than a Unity or other DI framework’s container. The reason for this is twofold:

We want to minimise our product’s dependencies on other products or open-source projects.
We want developers using our product to be able to know in their IDE whether a given service can expect to be found wired up in the container, hence our use of methods like ContentServiceGet() and SecurityServiceGet().

That second reason is very important given that we are still actively developing the Elcom.API and pushing it out across all of our product’s many modules. One version of CommunityManager.NET may have very different abilities to the next one, and our partners’ developers may be dealing in multiple versions at once – hence the need for good IDE support.

Saved by the Static Property

So we like the syntax afforded us by static/shared methods, but they present problems when it comes to setting the values during testing. We looked at lots of options, including using delegates or even events, but they all made production code smelly in order to aid testing, and that didn’t sit well with us.

What we ended up doing was creating an IServiceContainer interface which declared all the types of services that could be returned, and using a factory within the Current.Service method to build a DefaultServiceContainer from a configuration file. We then provided a Current.ServiceContainer object which would contain the concrete version of IServiceContainer being used and which could be overwritten when testing.

''' <summary>
''' This holds the concrete implementation of IServiceContainer that is used. This is writable in order to help testability.
''' </summary>
''' <value>The concrete implementation of IServiceContainer that is used by Current.Service (or that is created by that function).</value>
Public Shared Property ServiceContainer As IServiceContainer = Nothing

Public Shared Function Service() As IServiceContainer
Dim currentContainer As IServiceContainer

If ServiceContainer IsNot Nothing Then
' retrieving the previously created one (or the testing one if its been written by our tests)
currentContainer = ServiceContainer
Else
Dim factory As New ContainerFactory

currentContainer = factory.ServiceContainer

' obviously we need to store/cache this and not just instantiate each time, so put it back into Current.ServiceContainer
ServiceContainer = currentContainer
End If

Return currentContainer
End Function

This approach works really well, provided you don’t mix production and testing code in the same application instance, which we never do. It doesn’t offer a generic container for developers to use for custom projects built on CommunityManager.NET – but there is nothing stopping them using Unity or other generic containers for their own purposes.

In case you’re wondering this has become a pattern we’re using across the Current object’s static/shared methods, including of course Current.HttpContext. Our caching at the moment is still fairly primitive (stick the object in the static property), but we are looking at making improvements in that area in this release too. If nothing else we have a proxy for Session objects that can be swapped at test time for something not dependent on web projects (although ASP.NET cache is available outside web projects anyway, See Scott Hanselman’s reference to using HttpRuntime.Cache outside ASP.NET).

Is blogrolling dead?

2010-09-20T23:27:00.000+10:00

I just removed my Blogrolling links as Google warns me that the site was being used to distribute malware. It's a real shame, but it opens up the door to me looking at freshening the design for the first time in a few years.

Using Enterprise 2.0

2010-04-15T13:25:00.001+10:00

Continuing from my previous post, this one is more in line with the actual talk I gave on the 13th April at The Internet Show in Melbourne. We will be putting the actual transcript and slides of that talk (and video if the quality is good enough) up for people to download in the next week or so.

If you are interested in Web 2.0 and social software then you should have a look at Enterprise 2.0, which could be thought of as just Web 2.0 within the enterprise, although the originator of the idea, Andrew McAfee has a more precise definition for it:

“Enterprise 2.0 is the use of emergent social software platforms within companies, or between companies and their partners or customers.”

He places particular emphasis on the idea that it should be emergent, and for this reason excludes traditional intranets as part of the Enterprise 2.0 toolset.

“Emergent means that the software is freeform, and that it contains mechanisms to let the patterns and structure inherent in people’s interactions become visible over time.”

The whole idea of emergence is a tricky one, but a good example is the way Google assesses the relative merit of web pages based on the number (and quality) of inbound links, something which emerges over time as people find a page useful and link to it, rather than something pre-planned and orchestrated. A more concrete example is the idea of not placing paths in a garden up front, but allowing people’s actual movement to determine where paths should be placed – hence having them emerge as real patterns of use are known.

A point worth noting is that this means that the early structure or format of an Enterprise 2.0 tool can be expected to change over time. Initially it may seem somewhat useless, and not the sort of quality resource the business hopes it will be. This is a good thing as this quote from Art & Fear illustrates:

“The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the ‘quantity’ group: fifty pound of pots rated an ‘A’, forty pounds a ‘B’, and so on. Those being graded on ‘quality’, however, needed to produce only one pot -albeit a perfect one - to get an "A". Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the ‘quantity’ group was busily churning out piles of work - and learning from their mistakes – the ‘quality’ group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.”

The point about emergence is that it usually leads to a more appropriate and higher quality result than something meticulously planned from “theorizing about perfection”.

It is important to realise that the interactions that will take place, and help the emergence of patterns and structure, will be around ideas and especially the communication of ideas. Any Enterprise 2.0 initiative must have as its basis the ability for people to express, challenge, endorse and modify their and other’s ideas.

It can seem that some of this is a waste of people’s time, creating yet more ideas (some rubbish) for them to have to trawl through and interact with. We should remember that what ultimately matters is how it helps to optimise overall organisational effectiveness, not necessarily the impact on individual efficiency. Efficiency is one of the items that gets addressed as complexity emerges in the unstructured content that Enterprise 2.0 deals with – good tools will help you deal with this complexity.

What Can Enterprise 2.0 Do?

One of the biggest differences in Enterprise 2.0 tools is that they bring broadcasting to the average employee. This means that everyone can take advantage of the virtues of pull versus push publishing. Instead of identifying who should see something, and then pushing it into their email inboxes, or onto their IM clients, pull publishing allows the idea to be pulled into someone’s attention when they want it to be. Key enablers for pull publishing are RSS feeds and feed-readers, easily browsable content and micro-blogging enterprise tools like Yammer.

Another key difference is that most Enterprise 2.0 tools have come out of the consumer marketplace. That means they are highly usable and are designed from the ground up to be easy to learn. It turns out that’s a really important point, because it makes adoption across an enterprise much easier than most enterprise software. One key point is that the tools should address security and legal concerns, unlike most Web 2.0 tools, the content being dealt with is commercially sensitive.

Here are some business problems that Enterprise 2.0 helps address better than many other tools:

Bring new employees up to speed
Wikis allow corporate knowledge to be easily captured, reviewed, edited and re-published as necessary so that new employees can quickly see the latest information. Internal blogs can operate the same way, but have the additional bonus of making it obvious who knows what, which gives the new employee someone to connect with.
Accurately forecast something
Some cutting edge applications of Enterprise 2.0 include crowd sourcing from within the organisation, using internal prediction markets to identify probable issues or forecasting delivery dates or prices.
Help customers
One of the oldest Enterprise 2.0 ideas, and something pre-dating Web 2.0, self-help community forums enable customers to be helped by other customers acting as a volunteer helpdesk.
Make things findable
Blogs (and intranets) that implement folksonomy tagging well allow users to easily tag things to find later and make content more browsable. Good enterprise search solutions that cover blogs, wikis, intranets and other Enterprise 2.0 tools also help make things more findable.
Broadcast ideas (pull not push)
As mentioned above, RSS and Yammer allow users to publish content in a way that broadcasts it outside their normal circle of influence, without creating disruptive interruptions as push publishing tends to.
Transparent project progress
Using a wiki, blog or a simple project management tool like Basecamp to track project progress both allows the emergence of patterns and can make those visible to people outside the immediate project team.
Communicate to the rest of the organisation
Department blogs can act as venues for communications between components of an organisation, and capture the discussions around those communications for future users to review and understand.
Helping upcoming leaders network
It is well established that upcoming leaders do better when their networks are more diverse and cross organisational boundaries. Internal blogs, the comments around them and a good flexible corporate directory tool can help upcoming leaders get noticed outside their existing teams and make the connections they will require to succeed.
Enhance employee self-learning
I’ve mentioned before David Maister’s opinion that most training is useless and the current focus of much etraining is actually on how to teach the individual to become a better learner, rather than pushing particular content at them. Wikis, blogs and social bookmarking all place a part in enabling better self-learning.
Improve innovation
Innovation is done by creative people, usually passionate creative people (see my previous post on passion) and Enterprise 2.0 helps them do this by making it easy for their ideas to be shared with internal audiences to inspire and call forth more ideas from them. Any Enterprise 2.0 tool can help innovation, but the best ones are often internal blogs.

Below is a great slidedeck that illustrates how someone might use these tools:

Meet Charlie - what is Enterprise2.0?

View more presentations from Scott Gavin.

What Do I Need in an Enterprise 2.0 Tool?

What should you look for in an Enterprise 2.0 tool?

You want to start with one that is more of a toolbox than a single tool, a good start is to look for something that offers blogs, wikis and a well-integrated corporate directory. You want something that is expandable, so you have the option to add more tools later.

Any tool you choose should play nice with other enterprise software, so you should find something that offers single sign-on (SSO) with your particular network, for example Active Directory/LDAP integration and that satisfies your IT department’s security needs.

Playing nicely with other Enterprise 2.0 tools is also mandatory. In this case that is mainly done by providing (and consuming) RSS feeds and integrating with your chosen enterprise search tool, but integration with a corporate directory offering will become necessary.

Basic functionality to allow emergent behaviours is also necessary, at a minimum this requires tagging and the ability to comment on anything, but micro-blogging and social bookmarking are also good candidates for this.

If you want to try something without involving IT, then you can look at various hosted solutions (such as Yammer), but eventually you should make sure that IT is happy with your choice of tools – especially of it ends up being something that you need to host yourselves, or if it will contain content vital to the continuity of the organisation in the case of a disaster.

Do I Need It?

Small companies, or larger ones faced with large amount of routines tasks and few opportunities for passionate creative work, might find that Enterprise 2.0 offers less robust ROI than most IT purchases, but even they may benefit from using Yammer or similar low-cost, hosted tools.

Companies in highly competitive environments, where the ability to be agile and quick to react to change is important, or organisations seeking to ignite the passion of their employees and find more effective ways to work will find that these tools are indispensable in helping flatten the organisation and efficiently distribute ideas.

Untapping your people’s passion

2010-04-06T11:44:00.001+10:00

I’m in the midst of preparing for The Internet Show in Melbourne, 13-14 April 2010. As a sponsor my employer, Elcom Technology, has several slots for presentations. We’re doing a case study on a social networking startup and a CMS clinic, but most importantly for me I am also doing a session on Web 2.0 and Social Media within the enterprise. These are the key ideas behind that talk, although this is more detailed than I will be on the day.

Passion is one of those wonderful ideas, it is hard to quantify or measure, but unmistakable when you meet it. In the business world we tend to think of it as messy, disorganised, hard to manage, hard to inspire and hard to control.

However, passion is one of the greatest competitive advantages, with something they are passionate about, people will spend more time, think more creatively and fight harder than they will with other things in their lives. It is also well-recognised that passionate employees create passionate customers, and that’s something any business should be interested in.

Deloitte’s 2009 Shift Index says that “Passion is when people discover the work that they love and when their job becomes more than a mode of income.” They describe the need for passion in terms of competing for employees:

“Why does passion matter? Because staying competitive in the newly globalized labor market requires all of us to constantly renew and update our professional skills and capabilities. The effort required to increase our rate of professional development is difficult to muster unless we are passionately engaged with our professional activities.”

A History Lesson

Around 100 years ago the Ford Motor Company implemented an assembly line in order to boost the production of the Model T. Initially, Ford raised the wages of assembly line workers to keep them interested in the innovations they were rolling out, but now we think of assembly lines as places where cost is minimised by reducing the complexity of the work across many simple steps, and therefore minimising the training (and wages) required for employees.

By simultaneously innovating to a lower price point and creating a middle class that could afford their product, Ford managed to reach a dominant early lead in the production of motor cars. This precipitated a movement from craft production to mass production and in large part was based on scientific management, a term coined by Frederick Taylor.

The motor car industry might have continued in this vein forever, except for the advent of the Toyota Production System (TPS). The founders of Toyota were unimpressed with Ford’s assembly line, but found a way to improve it greatly. Their ideas were wildly successful and a host of studies tried to understand the core ideas in the TPS in order to apply it in the western world. Thus we received just-in-time manufacturing, total quality management, continuous improvement and lean manufacturing.

Most of these ideas have failed to find as fertile soil in the west as they did in Japan – however the evidence is that the difference is not cultural (Toyota runs plants in America run as well as they do in Japan), predicated upon Japanese corporate solidarity, or based on some secret sauce hidden from researchers. In fact the key items were in plain view all the time, but hidden by the biases the researchers brought with them.

There are two key differences they ignored:

1. People Management

The first key difference was the way people were treated. In the TPS people at all levels are valued and given control of their area of work. For example, an assembly line worker is able to shut down the entire line if they find a defect, and will then be part of the team tasked with finding the root cause of that defect and solving it.

2. Practice Makes Perfect

Another key difference is that the Japanese understood W. Edward Deming’s statistical process control methodology, and saw their assembly lines as being imperfect and in need of continuous improvement, rather than the perfectly defined processes that scientific management assumed.

An illustration of this is the pottery class that was split 50/50 between students who were told they would be judged on the quality of a single pot at the end of the course, and ones that were told they would be judged on the quantity of the pots they produced. The ones who focussed on quantity ended up producing better quality pots because they learnt from each one they created, whilst the others got stuck in trying to create the single perfect pot first time.

Eventually in the mid-80’s, Ford asked Deming to help them turn around their quality problems. They were surprised when his focus was on their management methods, but his influence was quickly seen in the profitable Taurus-Sable line.

Today these principles are working their way into other areas of work, through Agile software development and the general concept of Lean. An important key concept is the idea of waste – whether it be spare inventory, transportation, defects, unnecessary motion, over-production or over-processing. The well-known Scrum methodology lists three important pillars for empirical process control:

Transparency
Know what is happening and be able to measure everything.
Inspection
Regular and frequent inspection of the process to ensure it is functioning without waste.
Adaptation
Adapting the process to remove any waste.

So What?

A big part of Deming’s management principles was helping people re-discover “pride of workmanship” (or passion!) in their work, and removing barriers to them doing this (e.g. work standards, quotas, annual ratings, management by objective, etc.), substituting leadership for those barriers.

Whilst Deming’s methods have transformed the manufacturing industry, with companies like Dell and Boeing taking lessons from the TPS and applying them well, there are many industries where Taylor’s scientific management methods still rule, mostly because of habit and management (mis)training.

I know that I learnt how to manage very much in line with scientific management techniques (mainly by osmosis from poor managers). Plan, control, punish, reward, etc. The reality is that there is a shift in our working lives, we need more passionate creatives than ever, more artisans or craftspeople than unthinking drones. There is scarcely any industry or business that does not need at least some of their people to be passionately involved, engaged and thinking outside the box.

That last point is key to applying this. For some organisations the amount of passionate creative work that overlaps their routine work is minimal, for others there is almost a total overlap. The trend is towards more passionate creative work, and we will see that increasingly the leaders in their field are organisations that tap into this and harness it properly. Ford’s initial success in keeping their assembly workers by doubling their pay was short-term, soon the rest of the employment market matched their wages and the middle class expanded rapidly thereafter.

Your efforts need to match the amount of passionate creative work that overlaps routine work in your industry – but don’t shy away from finding ways to move away from the industry average and get more creative work in there.

How Can We Do It?

The question is how do we help people tap their passionate creativity in the workplace so they want to stay?

We need to go back to the basics of good management again to find out what really motivates employees. Daniel Pink talked about this at TED in August last year.

“There is a mismatch between what science knows and what business does. And here is what science knows. One: Those 20th century rewards,those motivators we think are a natural part of business, do work, but only in a surprisingly narrow band of circumstances. Two: Those if-then rewards often destroy creativity. Three: The secret to high performance isn't rewards and punishments, but that unseen intrinsic drive. The drive to do things for their own sake. The drive to do things because they matter.”

In summary Daniel points to three key ideas for managing people in the 21st century:

Autonomy
The urge to direct our work and have some say in what we do.
Mastery
The desire to get better and better at something, to make progress and overcome challenges.
Purpose
The yearning we have to work in the service of something greater than ourselves.

There was some interesting research reported in the Jan-Feb 2010 Harvard Business Review that supports the power of mastery in motivating employees:

“On days when workers have the sense they’re making headway in their jobs, or when they receive support that helps them overcome obstacles, their emotions are most positive and their drive to succeed is at its peak. On days when they feel they are spinning their wheels or encountering roadblocks to meaningful accomplishment, their moods and motivation are lowest.”

Deloitte’s 2009 Shift Index had similar findings about autonomy and mastery:

“24 percent responded that flexibility, freedom, and autonomy were the reasons they “loved their job.” Similarly, 23 percent of the respondents cited challenges and opportunities for problem solving and creativity as the reasons they loved their job.”

These keys can all be found within the TPS, and are the foundation of any great 21st century business. If they can work for assembly line workers in one of the world’s largest car manufacturers, then surely they can work in our office environments?

More to the point, if we don’t offer work environments that have these keys, then we can expect to see more of our best and brightest leaving to go freelance, or joining that new startup, or even finding a big company that gets this idea.

Getting a Leg Up from Enterprise 2.0

I’m a Technical Director, so of course at some point I want to see just how technology can help us, and in this area as so many others there are some interesting possibilities thanks to Enterprise 2.0.

Learning

Modern creative work places a huge demand on people to be constantly learning, diversifying their experience and increasing their speed of uptake of new ideas and skills. More then ever before success demands that people self-educate themselves, and with information overload and the skyrocketing complexity of our global business environment (where your competitors may just as easily be half a world away as down the road) that is harder than ever.

Enterprise 2.0 offers lots of help in the area of learning and knowledge transfer.

Wikis
These are great for helping people record their knowledge, and then allowing that knowledge to be reviewed, improved, updated and generally kept alive.
Blogs
Fantastic for putting ideas out there and getting feedback on them. They can provide employees with acceptable avenues for bridging great gaps in relative seniority (e.g. letting a new hire question the corporate vision statement) as they can allow people other than the original author to respond and correct mistaken ideas. The ability to browse posts by chronology and/or tags means that content is more usefully categorised than it would be in an email inbox.
Social Bookmarking
Whilst bookmarks can be overrated, if properly categorised and rated, they allow people to make new information available to their colleagues and can act as scent as to who is most interested or interesting.
Enterprise search
Having an effective and useful enterprise search tool is a must-have Enterprise 2.0 tool. Most Enterprise 2.0 content is unstructured and a good search tool is essential to ensuring that people find the right content when they need it.

Networking

There is a need for us to connect with ever greater numbers of people, and being increasingly time-poor to find the right people to connect with. Research has shown that high performing individuals maintain better networks than others, and in particular that their networks are more diverse/disconnected. This allows them to “see the big picture better, generate innovative solutions by integrating the expertise of those with unique backgrounds, position their efforts well, bypass bureaucratic gridlock and obtain necessary resources and support.”

Enterprise 2.0 makes it possible to find the right people in your network by reviewing their ideas and following the commentary and discussion about those ideas. It also helps people develop connections beyond their immediate team or management hierarchy.

Corporate directory
Having an active and useful corporate directory, one that tracks projects worked on, positions held and teams an individual has belonged to can help identify key people.
Employee pages
Giving employees their own page to talk about themselves can be risky, but for those passionate creatives it can offer a way to get their ideas out there, attracting others that have similar interests and passions.
Blogs
The commentary and discussion facilitated by blogs, along with their attachment to individuals (and hopefully the corporate directory), makes them an ideal place to connect with others and find out who else you should connect to.
Yammer
Internal tweeting tools like Yammer offer interesting possibilities as people can follow who they like within the organisation and find out who is following them.

Managing Autonomy

One of the main reasons people fear worker autonomy is that they will lose oversight of what is going on. Obviously implementing transparency properly means this is not an issue, but there are problems with managing transparency when collaboration is limited to email inboxes and offline meetings.

Enterprise 2.0 offers ways for collaborative work to be both done at a distance (of either time or place), and to be recorded in ways that allow for relatively easy monitoring by managers who want to help ensure that responsibility is taken seriously and known problems are not repeated.

Wikis
Tracking progress on a wiki gives managers insight into what is happening and an avenue for querying ideas and outcomes.
Online document collaboration
Collaborating on a document online makes the draft version available to people beyond the immediate team, often in a read-only version.
Basecamp
online project management tools like Basecamp are highly regarded for their ease of use, simplicity and ability to facilitate complex project discussions and task workflow.
Google Wave
Still in Beta, and with real performance and usability problems still, Google Wave offers interesting possibilities with low-overhead asynchronous conversations organically evolving into rich document collaboration opportunities and even real-time conversation.

Making Progress

We have seen that a key motivator is the sense of making progress, achieving real results and conquering challenges. One way our current systems can rob us of this is by interrupting our state of flow and causing us to drop out to check something that turns out to be unnecessary – it is estimated that a 1 minute interruption kills 15 minutes of productive time. Two deadly forms of this are unimportant emails and face to face social desk visits.

Enterprise 2.0 tools can help reduce the amount of email flow by moving items to more useful forms of communication that rely on a pull method of distribution (where users choose to use it), rather than the invasive push method employed by email.

RSS feeds
By eliminating the push nature of methods like email, RSS feeds give a generic method for tools like blogs and wikis to make their data available for the user when they want it.
Yammer
By bringing micro-blogging to the enterprise tools like Yammer remove the instant interruption hassle of instant messaging (IM) and provide an outlet for broadcasts of ideas that may be useful, but don’t need to be considered immediately.

Enterprise 2.0 tools can help move social interaction away from invasive face to face time in the office to more asynchronous means, or at least ones that include less wasteful interruption time.

Yammer
Social engagement via micro-blogs like Yammer is ideal because it is asynchronous, based on the push/broadcast mode of communication and carries an implicit understanding that it is not important content.
Intranets/Wikis
Placing social event information online, and providing interactive photo galleries can help reduce the amount of social interaction that must occur face to face.

In Summary

Deciding to manage work empirically, through transparency, inspection and adaptation can help give your employees the sense of autonomy, mastery and purpose that is necessary to untap their passionate creativity.

There are key ideas I have not covered here, such as the willingness to accept failure as learning experiences, the need to align authority with responsibility, and responsibility with ability that must be addressed for this to work well.

However untapping passion is not only possible, but well within our reach. It fundamentally depends on the willingness of senior management to set the direction, free their people and unleash their own passion.

Finally, Enterprise 2.0 offers some interesting tools to help manage and enable this change in our organisations. But as ever, technology plays an enabling and supporting role rather than being the driver of true change.

Thanks

There is a lot packed into this particular post, but a few people were key in helping me understand these issues.

Jason Yip, from ThoughtWorks Australia, whose passion for Lean got me interested in looking at Agile software development and whose ideas and blog have helped me work out my own thinking.

Lachlan Heasman, also from ThoughtWorks Australia, who runs the Sydney Scrum Meetup, and is a good friend and great Scrum trainer.

Craig Bailey, who in a year as my boss ignited hope in me for great managers once again, and who has gone on to embody following your passionate creativity wherever it may lead.

Chris Scoggins, my first great manager and a great example to me of how to mentor and build leadership into your staff.

Daniel Pink for bringing the hidden truths about motivation into plain view, and pointing out the Emperor’s New Clothes of management.

My Dad, for always making me question what is true, and helping me understand my own passionate creativity.

Goals, Behaviours, ROI and Testing Ideas

2010-03-09T00:09:00.001+11:00

You're developing a great web application, training course or social software strategy. You cost it and it looks like a substantial piece of work, so you need to get executive approval. Putting together a list of the benefits you find to your dismay that the list is full of terms like "easier", "more efficient", "more usable" and "collaborative". You know your executive will ask questions this list doesn't answer, "How will you know you're successful?" and "What's it worth to us?".

There are two common (and wrong) approaches to this question:

It's Web 2.0/training/social, it's intangible and can't be measured.
It should add to the bottom line, so just measure revenue changes.

The first one incorrectly assumes that intangibles can't be assessed, but market analysts know that they certainly can, and indeed regularly are, just ask what Google's stock valuation is based on.

The second one jumps to the über-goal and ignores the fact that as everything the organisation does can potentially affect revenue, it will be nigh impossible to really know whether any change in the bottom-line is the result of this particular web application, training or social software strategy (you can easily have a great success in one area cancelled out by a horrible mess in another).

Fortunately this problem is well known. Unfortunately the answer is hard. That is, hard as in deciding what you want for your birthday, not hard as in needing a PhD in Applied Rocket Science.

Find your desired user behaviours

The bottom line is that you need to find what change in user/trainee behaviours will support your business goals and then use that to derive metrics that can be measured (and eventually valued).

David Maister talked about the importance of identifying desired behaviours in his article Why (Most) Training Is Useless.

There is no point putting on skills training if there is no incentive for the behavior; the people don’t believe in it and they don’t yet know exactly what it is they are supposed to be good at! ...

What behaviors by top management need to change to convince people that the new behaviors are really required, not just encouraged? If the behavior is going to be optional, then so should the training be.

In other words, if we don't know what behaviours we want training to encourage, or if we don't actually support the display of those behaviours (e.g. through measurement and feedback) then we can't expect our training to actually help our business goals.

In exactly the same way software applications (not just web or social ones!) are often built without thinking what change in user behaviours is really desired or how the change in user behaviours will be measured - thus guaranteeing that even if success is achieved there is no way of valuing it.

ROI for difficult stuff

Adaptive Path introduced a way of valuing user experience by reflecting on the change in behaviours we desire to solve our business problem. They gave us the graphic below as a way of envisaging how business problems map to user behaviours and how they in turn can be mapped to valuable financial metrics.

Source: Adaptive Path

via Marina Chiovetti at ThoughtWorks

Business Problem = a specific problem you want to affect
User Behaviour = a change in user behaviour that would introduce the effect desired
Behaviour Metric = a way you can measure the change in user behaviour
Value Metric = the dollar value we can apply to the behaviour metric
Financial Metric = the expected amount of user behaviour change to come from the project, multiplied by the value metric and compared to the expected cost of the project

A specific example might help make this clearer. Imagine you want to increase leads and you decide to do this by improving your website's ability to elicit customer responses via the contact us form:

Source: Adaptive Path

If we add some case studies to the website, and then place the contact us form at the bottom of each one as a call to action then we might expect some increase (assuming this was a new initiative for our website).

The behaviour metric for this is the number of leads received from the website contact form per month. Our sales team has estimated that 1 in 10 leads becomes a sale, worth on average $1,000 to us (whether revenue or profit depends on what you are most interested in - the smart money is on profit though).

Based on our website's traffic patterns we expect to increase our leads from 7 per month to 20 per month by implementing this measure. If we are happy to get a breakeven return on investment (ROI) over the next 3 months then we could invest $1,000 * (20 - 7) * 3 = $39,000 in the project.

Before someone tries to tell me I don't know how to value project returns, I know that you could use Net Present Value (NPV), Internal Rate of Return (IRR) or other methods to ensure this is comparing apples with apples across projects, time and investment opportunities - but that's not the point of this post.

Now we might well spend a fraction of that on case studies and a contact form, but if we wanted we could now justify getting a professional copywriter to help shape up our case studies and spend a bit on usability testing to ensure our contact forms get out of the way and give us the leads as easily as possible.

Okay, okay, this is a fairly simple example, with straightforward metrics. After all, we might well not know how much a lead is worth, and often we're asked to solve business problems far more complex than this one.

What if I’ve got no history (i.e. I’m a startup)?

We have used this process with startups to help them prioritise wildly diverse feature lists when they do not have the budget to afford to build it all in their launch timeframe.

(Aside: There is a lesson here about launching a simple, compelling product and then iteratively adding value to it based on user feedback, the problem is that in many startup ideas there is usually no simple, compelling offering. It's the fault of funding requirements, but more about that in another post.)

Startups do not have financial history to fall back on, and even with good financial modelling they might only have a guess to give you for their value metric. In this case we take the prioritisation this gives us and keep the rest of the information for analysis of the success of the startup down the track and (as importantly) for helping us identify which changes we should look at implementing later.

What if you don’t know what to change?

The methodology is still useful even when you have a complex situation with incomplete information. If nothing else it focuses your attention where it belongs, on the problem you are solving and the user/trainee behaviours that you think will help solve it rather than on the application technology or the subject matter of your training courses.

Of course you often don’t know what to do to create a particular desired behaviour. In this case a bit of prototyping and A/B testing can go a long way. Google Website Optimizer does a great job of helping you do this with content or separate functional pages, but it does require two separate pages exist, and sometimes you just want to tweak the way a particular feature works (e.g. adding a couple of extra fields to a form).

Assaf Arkin has created a plugin for Rails called Vanity that supports A/B testing in a rather unique way, by creating an easy way for developers to embed the tests into their code and then run them for a set number of iterations and/or exceed a set probability for one option over the other.

By using a simple API, and elegant admin functionality, Vanity provides a very viable way of testing one idea against another. While the time to make such a change is greater than not doing the A/B test – the marginal extra cost is small enough that you do it in order to find out which option really works better.

Assaf has a great post explaining how this is really Experimental Driven Development in action. He explains the cost/benefit tradeoff this way in the comments section:

“You start with an idea for a change that will improve your software. Your baseline cost of development is having both alternatives — before and after the change. Without EDD these alternatives will be separate in time, with EDD you’re going to have some overlap (the duration of the experiment).

For the experiment, you only need a skeletal implementation, you’re not committed to fully developing the feature until after it proves itself. For small changes it makes little difference, the cost is the same.

For complex changes, you can save a lot by not fully developing features that don’t matter. You’re going to know whether a feature matters or not quickly enough, and with data to back up, that you can make the decision to *not* develop it further.

You can also kill unused features early. So these are two ways to reduce development costs using EDD.”

Summary

Get behind whatever you are doing and try to understand the underlying business goals, the user behaviours that would support those and derive the ROI from that. If you are not sure what would best support the change in user behaviours, then try A/B testing to establish which way you should jump. Whatever you do, don’t allow yourself to be sucked into doing something just because your competitors did, or to be “simpler”, or more “usable” in some undefined, unaccountable, way.

Sydney Scrum Meetup (March)

2010-03-04T11:45:00.001+11:00

Jeff Sutherland and Jens Østergaard visited the Sydney Scrum Meetup last Thursday. We had some great pizza and then they got everyone doing the Nokia test!

My Test Results

Iteration length, 2 weeks – 10

Testing, customer acceptance testing – 8

Agile specifications, poor user stories – 4 (good enabling specifications might be 3-5 pages long – before sprint planning)

(for venture companies Jeff finds that 2 sprints planned is necessary)

Product owner, product owner with backlog – 5 (or 2 with last project!)

Product backlog, single product backlog - 3

Estimates, backlog estimated by BA – 0

(Jeff said that usually as teams get better their stories get smaller and eventually are about the size of tasks, and then estimating changes to use points vs hours – surprising what can be untangled and done separately)

Sprint burndown chart, no chart, but team knows velocity – 0 + 3 + 2

(Jeff mentioned that partial completion of tasks creates a high-risk environment)

Team disruption, project leaders telling people what to do – 3

(self-organise to maximise velocity)

Team, no emergent leadership - 1

Total = 4.33

Most of the room were between 2 and 5, and apparently, most of OpenView’s venture companies start out around 4, but when they work on it they end up around 6. It looks like TIDC might have a great Scrum team as someone from there scored 8 for their team! (they were the only one higher than 5)

If your reference stories change then so do the story points. But Jeff said it is really the delta in velocity that is interesting. One of the symptoms of hyper-productive teams is that you get asked to slow down!

Sustainable pace
Quality
Velocity
“Balanced Scorecard” (not right term, but Jeff couldn’t remember what it was)

Jens recommends the XP game for teams to learn about velocity.

How long does it take to fix a bug from CI/automated testing. If longer than 2 hours then create an impediment list and get the Scrum master to remove them. Simple metric, track the day you start a story and the day you finish a story and then compare to your standard process efficiency (usually around 20% – measures the quality of your backlog) {calc elapsed days versus theoretical ideal days based on velocity}. Raising the efficiency meant getting the backlog well enough detailed that you spend less time waiting to do a particular item. Measuring the time to Done.

Speed of testing is usually the bottleneck, and is more important than the speed of coding. So interrupt devs immediately with a bug, rather than leaving them to finish what they’re doing. The availability of testable code is another way of thinking of it.

Just evaluate the code in production is one way of incentivising people.

I asked about how to handle client projects using Scrum, Jeff mentioned that Systematic is doing big fixed price contracts using Scrum. They provide 2 bids on every project, with the Scrum bid at around half the price.

In response to someone’s question about Scrum, Jeff mentioned he’d written about the answer in an article titled Future of Scrum from the Agile 2005 conference.

Metrics to Live By

Jeff got me hooked on metrics as it is obvious when talking to him that empirical process control requires metrics to be successful. Some of these metrics assume that you use story points for estimating, which we have done with some success on one project. To do this well you need a range of reference stories of various sizes and rough sizing (e.g. Fibonacci numbers) that the team can agree on the size of.

Here are some of the metrics Jeff mentioned:

Backlog Velocity = story points/sprint

Sprint Velocity = story points ‘done’/day

Efficiency = story’s ideal days/actual elapsed days from ‘start’ to ‘done’
(if an item should have taken 2 ideal days, but it actually took 10 days from the ‘start’ to the ‘done’ date, then you have 20% efficiency)

Churn = % of ‘done’ items that testers send back to developers for fixing
(if all items churns once then you have 100% churn, if half of those churn again then you have 150% churn! Ignore size of item when calculating)

Jeff has an upcoming conference session on this very topic and he discusses it further in his Excel Spreadsheet for Hyperproductive Scrum Teams post. The spreadsheet seems a little hard to get into, but I’ve given it to our Solutions team to review.

Do I Need User Stories?

2010-01-29T10:06:00.001+11:00

How do you create a worthwhile specification of the requirements for a system?

As an Agile Scrum practitioner I’m supposed to answer this by pointing to user stories, but as others have found before me, these are a slippery beastie to define. Cat Schwamm takes a stab at a definition, but finds Scott Bellware telling her she is following Scrum too blindly and should go change what she’s doing.

Scott’s point of view is interesting, but like a lot of his comments he is coming at the problem from the frame of his lean methodology, which is not fully explained and therefore extremely hard to argue with/understand. This is not Scott’s fault, he is after all busy consulting not writing Agile methodologies, but it does make his commentary a tad alarming when one first reads it.

The classic format of a user story is:

As a [user] I need/want [feature] so that [benefit].

Or as Cat puts it:

As a [role] I need [feature] so that [business value].

Cat points out that this is fairly barebones, and leaves lots of scope for you to choose the wrong level of granularity for your user story. She points out that her team also follows the INVEST acronym and use the concept of minimum marketable feature (MMF) as defined by James Shore.

INVEST = Independent, Negotiable, Valuable, Estimable, Small, Testable

MMF = “Smallest possible set of functionality that, by itself, has value in the marketplace”

This is obviously working for her team, but I suspect they benefit from having a single Business Analyst who works everything through for them and ensures some consistency across user stories. This is actually very Agile, as the manifesto points out we value “individuals and interactions over processes and tools”. However, we still need some processes/tools for those individuals.

Scott Bellware’s suggestion was to change the format to:

As a [role] I want to [goal] so that [motivation].

He wants us to avoid mentioning the software altogether and have the user story simply capture what the users actually do (or will do once they have a system to help them). He then divorces the user story from work items altogether, and simply relegates them to the role of being information about the goals of the system. This may work for him, but it then begs the question of how he defines what the work item is all about, which is where we started talking about the need for user stories …

I have been reading a great book called Designing for Interaction by Dan Saffer that gave me another viewpoint on this issue. One of the things I learnt from this book was that user-centred design (UCD) is not the best way to design interactions, it is simply one of the possible ways you could design them and its relevance depends upon the goals of the thing you are designing.

Dan Saffer lists 4 different approaches to interaction design:

User-Centred Design
Activity-Centred Design (aka Use-Centred Design ???)
Systems Design (aka Service Design???)
Genius Design (aka Rapid Expert Design???)

Cat’s user stories actually sounded like activity-centred design, and Scott sounds like he is more interested in user-centred design. Quite possibly they are both right in the contexts they are working in. Neither one may be better than the other (Scott at this point would probably call BS on me, he has previously talked about the need for user stories to communicate expectation).

Coming back to my own issues with user stories, I need to ask what sort of design approach am I taking, otherwise I will find my user stories being all over the place. Is the user story meant to encapsulate the MMF in a single story, or is it supposed to be used to specify all ways of looking at the functionality.

As an example, when looking at the search function for administrators of an organisation I have specified:

As an admin I want to be able to search for a member by their Member Id so that I can find a specific member quickly.
As an admin I want to be able to search for one or more members by their Member Type so that I can target an action against all members of a specific type.
As an admin I want to be able to search for one or more members by a wildcard search of first and last names so that I can find a member just by knowing their name.
As an admin I want to be able to search for one or more members by region or industry so that I can find members from specific industries or regions.
As an admin I want to be able to search for one or more members by a wildcard search of job title so that I can find members with specific job titles.
As an admin I want to be able to search for one or more members by member status for my organisation so that I can target an action against all members of a specific member status.
As an admin I want to be able to search for one or more members by activation status so that I can target an action against all members of a specific activation status.
As an admin I want to be able to conduct a search against multiple facets of a member at once (e.g. by member status, industry, member type and job title) so that I can get very specific search results.
As the system I want to give admins a common page which allows them to search for members for all sorts of reasons and then target actions against the members found.
As the system I want to ensure that admins can only ever see their organisation’s own members in search results.

You can see the format slipping towards the end of that list, with the “so that [benefit]” element disappearing. There is also no regard to the INVEST principles, particularly the independent one. If I adopted another (MMF?) approach I might reduce all of that to:

As an admin I need to be able to find the members I’m interested in from my organisation so that I can work with those members’ details.

Scott’s format might leave me with:

As an admin I want to work with my members so that I can help them use our organisation’s online tools.

I think this demonstrates that there is a lot of variance possible within this simple definition of user story. In the past I think I have over-specified functionality with user stories, leaving my developers with a weird tension between overly specific (and hence out of date) user stories and little or no supporting documentation around other design issues.

This actually speaks to the N in INVEST, as Cat explains it:

“Negotiable: User stories are not a contract; they are reminders for the team to discuss and collaborate to clarify the details near time of development

What a user story really is is a placeholder for a conversation. We’ve gotten away from writing strict, rigid use cases and into these user stories, which are much better for the way the business world really works. By writing a user story this way, I am opening up a higher bandwidth conversation. Instead of a developer just reading what I have and doing it word for word, now we can talk about things and make sure everyone is on the same page and fully understands the business value and the needs of the user. I like that Kelly specified “near time of development” though; that’s one thing you’ll always find: CHANGE IS A CONSTANT. Friggin annoying, but it is inevitable. The closer to development time you talk about something the more likely it is that you’ll have the best information and the most clarity on the item.”

I especially like the idea of a user story as a placeholder for a conversation. Of course this depends on your developers actually being willing to talk to anyone about the requirements (sometimes they still seem to think that it is faster to code away on assumptions and correct later than to have a 30 min conversation up front and ensure you know what you are doing!).

Scott Reynolds works with Cat, and he points out that not everything can be captured in user stories:

“Here’s the thing, stories and tasks are only part of the equation. Stories alone aren’t enough to define a system, and trying to define everything in text is a fool’s task. (I’ve been that fool). You need a full arsenal of specification tools to do the best job possible.”

I think I’ve been the fool Scott mentioned myself (once or twice). Scott (and Cat) mentions 3 other tools that they use as well as user stories:

Mockups
Basically a low-fidelity prototype. They are great for eliciting feedback about visual design and interaction issues without leaving anyone feeling like these ideas are final. They are poor for actually designing off of as they frequently ignore or skimp on the design of areas that were not pertinent to the original design issues being explored in them.
High Fidelity Design
High-fidelity screenshots showing exactly how the screen should look. These are great when you need signoff on something from outside the development team as they show everything that needs to be taken into account. They are poor when the design needs to change and you don’t have time to re-do those rich initial screenshots for the change request. Actually they are poor for Agile, full-stop, as they reek of BUFD.
Conversation and Whiteboarding
This is the higher bandwidth conversation that Cat referred to. This is a chance to meet, collaboratively engage each other and work out what the business need for this feature is. The caveat is that it is only as good as the conversation and whiteboarding ability of your developers, and it requires that the customer be available immediately when the developers need them (this is why Agile doesn’t work for many projects).

There are other good ideas for artifacts to capture interaction design ideas, such as:

Acceptance Tests
Here I mean a light version of the full test scripts that are possible. Ones that match user stories come in the form:
Given that [context], when [event/action], then [expected outcome].
Like user stories they have the same problems with being at the right scale.
Key Scenarios
Penny Hagen describes these as combinations of user stories that help define the strategic direction of the website. Whilst she uses them as a tool to help define design strategy early on, they have a useful purpose in helping developers understand the context of the user stories they are working on.
User Pathways
Again Penny Hagen describes these as a detailed view of the pathway a user needs to follow for a key scenario. They don’t explicitly restrict themselves to pages, or data, but they do help identify the user’s view of what is happening when they interact with the system – and give insight into the user’s experience over time.

Personally I’ve used mockups, high fidelity design, conversation/whiteboarding and acceptance tests to varying levels of success. One of the problems you face in Agile development is that the more artifacts you ask for, the longer it takes to go from designing to building – and the greater the chance that your artifacts are out of date and, frankly, wrong.

I’m still not sure about how to set my user stories, I sure appreciate the discipline they grant even quick feature specifications, but I’m loathe now to simply bang them out when the reality is that I might need to use either less, or more, detail for a given feature. I’m going to keep experimenting because that’s what Agile is all about. In the meantime let me know if you find something better to use!

When Agile Doesn’t Work

2009-11-19T13:44:00.001+11:00

I am a huge fan of Agile software development and since becoming Elcom’s Technical Director I have made it my business to push us to become more Agile at every step. But a few months ago I realised I was pushing a square block into a round hole.

It started with a realisation that most proponents of Agile software development had a common single clause that modified their promises of great results.

Customers must get involved for Agile to work.

In Scrum it’s the Product Owner, and it must be the single wringable neck – the person accountable to the business for the project’s delivery of real value. It is the person who not only understands, but can make decisions about functionality, business needs and the user interface.

Oops.

While this sounds great, and you would think that people paying you tens (if not hundreds) of thousands of dollars would want to get involved, we had found all too often that big promises of ongoing involvement disappeared as soon as the project kicked off. Clients wanted to tell us what they wanted, approve our quote, pay us the money, and then pretend we didn’t exist until we had finished their project. Or they had a committee of thousands that wanted to give us their pent up IT requirements, regardless of whether it was in scope or not, and often in conflict with each other.

To be fair to our clients, we often are simply implementing our software and the amount of customer involvement is quite low – however that didn’t fit the Agile methodology we wanted to work within.

Things became clearer for me when I attended a ThoughtWorks Quarterly Technology Briefing in Sydney, and got the chance to hear Martin Fowler speak about how to fail in Agile adoption. During the Q&A I asked him how he suggested we handle customers who were reluctant to get involved.

Martin pointed out that there are in general two sorts of projects:

Strategic: Key to business success, and as such important.
Infrastructure: Necessary to do, but not seen as particularly vital for business success.

Immediately I saw that for many of our clients their projects were internally seen as necessary (new intranet, re-skinned website) but were not particularly risky or impactful on the organisation’s success. I mulled that answer over for a while and wondered what it might mean for our use of Agile.

A few weekends later I was re-reading David Maister’s classic book Managing the Professional Service Firm and came across his definition of the three basic types of professional service delivery. He writes for any type of professional service and uses examples from disciplines as far apart as architecture, accounting, engineering and management consulting to make his point.

Brains/Expertise

Client problem is at forefront of professional or technical knowledge
“Hire us because we’re smart”
One-off projects
Highly skilled and highly paid professionals

Grey Hair/Experience

Clients require highly customised output, more familiar problems
“Hire us because we’ve done it before”
Specialised knowledge in project area
We sell our knowledge, experience and judgement

Procedure/Efficiency

Client problems are well-known and familiar, solutions also well-known
“Hire us because we can do it for you”
Repeat projects, well understood, standard solutions
Well trained and supervised junior staff

Maister points out that you can see a continuum from Brains to Grey Hair to Procedure for a range of different service features.

Using this idea, I could see that most of the “infrastructure” projects we had involved procedure or efficiency services. A simple deployment of CommunityManager.NET with implementation of a specified basic website design and some custom styling of forms and other modules. Agile simply slowed these projects down and was generally ignored by the team doing most of this work.

On the other hand some of our custom development work fell into the realm of grey hair or experience services – enhancing or extending CommunityManager.NET for a client. These usually did not need to be Agile projects, but were being forced into our product sprint cycle, even when they were a day or two of custom work not dependent upon enhancements to our core product.

Our product work deserves a mention here. Whilst not for any specific client, this is in fact the work that provides the greatest return on investment for the business. However a need for cash flow meant there was a limit to how much could be done at any given time. We had a settled sprint duration of 6 weeks (5 weeks dev + 1 week regression testing and release preparation), wit 1 week in between sprints, that the team had a good rhythm with. Agile was a great fit for this work.

After this analysis there was a small, but important part of our business. Small in terms of number of projects, but important in terms of revenue. This was the projects that involved significant custom development work and had little dependency upon CommunityManager.NET. Sometimes these jobs could have been delivered without our product at all, but using it gave a convenient head start to the project. These projects were strategic to the client, but as a result often had loose, or non-existent requirements because it was acknowledged that the requirements would change during the course of the project. This type of work suits Agile methodologies, but the business needed to recognise that this work shouldn’t be sold the same way our other services were.

Clearly Agile suited some parts of our business, but was a poor fit for others. Our sales material made a big deal of our Agile methodology, but it was not always relevant or appropriate for our clients. Worse, working in an Agile manner was causing many projects to have a greater chance of failure, and to delay client solutions.

Agile Wasn’t Working

I now knew why Agile wasn’t a great fit for much of our business, but it still had value in some areas. Maister also made the point that when for a professional service firms to mix Brains, Grey Hair and Procedure work successfully it is necessary to create divisions within the firm to allow each to have a management style and culture that fit their style of work.

I really like the way that Jurgen Appelo organises his development teams, with a project manager assigned 3-5 other staff as their permanent team, with them needing to complete each of their projects using that team. By pushing resource constraints to the lowest level (that of the project team) they ensure that other projects’ resource variances do not adversely affect them. However I know that Elcom is much smaller than Jurgen’s company, and such a profound shift would create more friction than it would generate in gains.

I discussed these ideas with my boss and some of my colleagues, but felt that we quickly needed to make a change as a large custom development project was about to kickoff its second stage.

The team structure we are now heading towards is this:

Delivery Team

This is a team made up of web designers, front-end web developers, and .NET analyst/programmers. Their role will be to implement new customers’ versions of CommunityManager.NET as painlessly and efficiently as possible. They will handle custom development work that is tightly integrated to, or built around, CommunityManager.NET. They work in traditional BUFD (big up-front design) projects with well-scoped requirements. Project duration would usually be anything from 1-8 weeks.

Solutions Team

This is a team made up of user experience (UX) consultants, designers, and web developers. It is the renaissance team that is responsible for finding new ways to deliver innovative solutions for strategic client problems using design thinking and Agile software development methodologies. They work in a more flexible manner, preferably with a pseudo time and materials approach, with the client purchasing entire sprint’s worth of work. Project duration would usually be more than 12 weeks, with usually two week sprints.

Product Team

This is a team made with a web designer, front-end web developer and .NET developers. It is responsible for enhancements to, and strategic development of our products, particularly CommunityManager.NET. They follow an Agile software development methodology and are very much the traditional product-centred Agile team. Sprint duration would depend upon the ability to regression test in a timely manner – but for the moment this would be around the same 6 week duration we had used previously.

Right Now

When we first made this change, there wasn’t enough strategic product work for us to justify having a dedicated Product Team. Instead members of the Delivery Team are made available as and when required to work on the product. We still have 6 week product sprints.

The Delivery Team has started transitioning away from Agile, and their time is now more easily shifted from one piece of client work to another (they previously were committed up to 6 weeks in advance for a particular sprint). Our clients have started to notice that we have become more flexible and responsive to sudden requests. Our project managers have been learning a new resource planning tool which is slightly clunky (built in Excel, requiring double data entry) but that we soon hope to replace with a more effective system.

The Solutions Team has had some success working in a more Agile manner, but they need to learn to do it without the moral, emotional or practical support of the rest of the technical teams. By removing many of the restrictions that product work placed on them, they are now free to explore new technologies and test them in practice before we roll them into our products. For instance they have used Domain-Driven Design (DDD) and LINQ2SQL.

The teams are differentiated by more than just name. They have different KPIs, different management styles and different things to get good at.

The Delivery Team needs to become more process-driven. Too many things are left to individual choice when the best solution is already known and has been tested in practice. Also documenting requirements up front is a skill set that needs to be re-discovered and applied. After all if you are going to do BUFD projects then you need to be good at it.

The Solutions Team needs to become more Agile, and look at ideas like TDD/BDD and more comprehensive automated testing so that short sprints do not lead to a reduction in quality. Selling a job as an Agile development project is proving harder than we would have thought. It seems lip service to Agile is easy to sell, it sounds good and requires few changes in the way clients work. Real Agile development requires an altogether different mindset, and this is much harder to sell up front. We may well need to lead these projects with an initial fixed price project that uses BDUF, and then transition to a more Agile approach once we have some credits in the trust account with our client.

The Future

The Solutions Team will be undertaking a major strategic product initiative for Elcom over the Christmas period and will bring their Agile approach and technical expertise to bear in order to give us a major new element to our product for CommunityManager.NET version 7.0. After this we hope to have another major project lined up, but if not then they may become our Product Team.

If another Solutions Team project does come up, then the next step will be to create a full-time Product Team. We believe that version 7.0 will create enough demand that a full-time team will be justified. An important part of this strategy is to find partners that are comfortable enough implementing CommunityManager.NET that our Delivery Team is not a bottleneck on our sales. One of the keys to version 7’s success will be the way that it makes it far easier for digital agencies and systems integrators to use our product for their own web projects.

After that fame, fortune and fun might consume our days – or more likely we will find ourselves looking beyond enterprise content management to create a new market niche in the manner of good Blue Ocean strategists. Whatever we do, Agile will remain a part of it, but only when it deserves to be.

Keep Blogging, Don’t Stop!

2009-11-13T09:01:00.001+11:00

This is a reminder for myself as much as anything else. With my meeting schedule and responsibilities it can be easy to avoid blogging for weeks at a time. However, I think that is a mistake – and I want to record why. In fact, blogging about not blogging is a meme all bloggers seem to fall into at one stage or another, so this might seem a little trite. My excuse is that I’m encouraging my team to consider blogging more themselves (and I need to kick my own backside from time to time too!).

In summary I think the Reasons to Keep Blogging are:

Career
Mental health/sharpness
Community
Memory
Discipline

Career

I have had the unfortunate experience of working somewhere for a relatively long time (5 years) and then being retrenched through no fault of my own. What I found was that I had been ignoring my career in general in favour of my career in the company that employed me.

Let me unwrap that a little. It is easy when working for someone to find yourself learning just what you need to for the tasks at hand. To restrict your professional relationships to colleagues and clients. To ignore new ideas that are not acceptable where you work. To stunt your growth in other words.

Just as it’s important to have a life outside work, it is important to have a professional life outside of work. If you get to know people outside your normal range of professional contacts, explore ideas outside what is needed for work and generally get interested in your industry then it is surprising how much more effective it makes your job search chances later.

For information works, especially creative types like developers, blogging can be a very effective way of building a portfolio of work and ideas that future employers can be impressed by. As an employer, I know that it also helps potential employers validate their face to face opinion of you.

Mental Health

As a creative type you need to keep your mental faculties well exercised and challenged. Work is not always the best place where you can get a chance to push yourself. Sometimes it helps to try something in public, but not to have too much hanging on the result.

Use your blog as a place to explore new ideas, techniques and ways of doing things. It is not just the content that needs to be fresh, so does the design and layout of the site. (reminder to self: take your own advice!)

Community

Don’t just blog on your own. Leave comments on other people’s site and link your name back to your blog so people can come see more of what you think if they liked it. Get into conversations, write articles criticising other’s ideas, commenting on them, stretching them, validating them, taking them further, or writing them off.

In other words, get involved with people.

Memory

Sometimes I wish I’d record how I fixed something, or where I found an idea out, because often I need it again later and can’t find it (easily). I discussed this with Neil Houghton the other day, sometimes you use a nifty technique in apiece of code, but then requirements change and it gets deleted, and you;re left hunting through sourced code control (Subversion in our case) to find it. Far better to make the effort to blog about it, and then some other poor schmuck might get help too!

Discipline

Aaaah the “D” word. Need anything else be said?

Self-discipline is something to be much admired, the enemy within is always vastly more difficult to conquer than the one without. But it seems to have a bad name.

Discipline is not about some rigid, heavily controlled lifestyle that simply denies and restricts our actions to the ones most acceptable, or least harmful to others.

Discipline is useful because it builds up the strength of will, and willpower is often the only difference between success and failure. Can you keep going even when you fail? Does one loss make you a loser? I think discipline is the habit of trying to succeed just one more time. It is denial, and restriction, but the purpose behind it is to do what we really want to do and not get distracted by the unnecessary, the mundane or the superficial.

If you really want to blog, if it fulfils one of the purposes I’ve outlined above, then you will find that continuing to do it helps build in you the discipline needed to do other things you really want to do. And that is the best reason I can think of to keep blogging.

Software projects are like sailing voyages

2009-08-11T13:09:00.001+10:00

I have been reading Greg Smith's new Becoming Agile book and was struck by the false analogy that is often applied to software projects, which compares them to construction projects, where Smith points out you typically "gather requirements, create a design, excavate, lay the foundation, and then build the walls."

Smith points out two problems with applying this analogy:

"1. Additional requirements will be discovered during design, coding, demonstration, and testing.
2. Software development does not require completion of each step before the following step can be initiated. You can start design and coding based on a few initial requirements. You can iteratively build out the system, revisiting it and going deeper on requirements, design, and code as you make discoveries and refine the vision for the project."

Analogies are dangerous because they build in these sorts of inaccuracies, and that can lead to dangerous assumptions and attitudes later on. However they are excellent for helping non-experts in a field understand complex topics. I dearly want a better analogy for software projects and I have never heard a really satisfactory one, so I got the mental juices going and looked for one that makes sense to me. This is what I found ...

Software Projects are Sailing Voyages

Software projects, at least the ones I have been involved in, feel more like ocean voyages in small yachts than anything else. Like any sailing voyage you will have an idea of where you want to get to, and if you are smart you will have checked the weather forecast and planned your route - even calculating what you average speed should be and when you will reach your destination.

from: http://www.flickr.com/photos/14132971@N05/3100906233/

This is reasonably similar to the planning and preparation for a software project where a goal will be set, the estimated time to reach it mapped out and the current environment and conditions researched.

However, in sailing you can expect to find a number of things changing your plans. Perhaps the wind changes in unexpected ways, or your plan failed to take into account the wind shadow of a particularly large island, or the strength of local currents or tides. The helmsman may have neglected to keep his eye on where you were headed, being distracted by a pretty sight, or other crew may not have kept the sails properly trimmed or closed the bilge valve.

from: http://www.flickr.com/photos/flasporty/2791384256/

Mishaps can occur, such as equipment failure, collisions with other vessels or even crew sickness or injury. Your destination port might be closed, and you find that you need to make landfall either sooner or later than planned. Even the captain's intent may change as it becomes apparent that the passengers would prefer a leisurely cruise around the coast to a deep ocean adventure.

from: http://www.flickr.com/photos/wili/2691809503/

Very similar changes can be expected in software projects. Your tools vendor may bring out a new release that opens up new development options, business goals can change or be re-assessed, leading to changes in requirements. Staff changes can cause a change in vision, or highlight mistakes or negligence in the work previously performed. Individuals' performance may fall short or exceed expectations for their role, and unexpected leave or changes in priorities may derail timelines.

These problems occur on construction projects too, but they have a real, physical object they have been developing with building approvals already acquired so it is far more likely that they will simply continue to the end (even then, you might end up with a project like Sydney's World Square, where it was just a huge hole in the ground with some cement in it for years).

from: City of Sydney

With software it always seems easier to throw it all away and start again (and indeed this is sometimes the best course to take). Simply having a shared analogy with the customer that makes more sense than a building will help them understand that while we can give up the voyage now, we can't avoid the fact that we have sailed this far, and perhaps we should take that learning and make use of it.

Agile software development in this case can be seen as a series of small hops around the coast, rather than the usual grand exploratory voyage undertaken by many traditional software projects. Each hop allows us to review where we have gotten to, and lets passengers get off if they so desire.

Of course analogies have their limitations, and using them to model a complex system by comparing it to another simpler, or more familiar, system does not guarantee success. It just makes client meetings shorter (which by some definitions is success!).

Half of TEDx Sydney 2009

2009-05-29T09:55:00.001+10:00

Thanks to Josh Anstey's generosity I was able to attend TEDx Sydney last night. Whilst it was enjoyable, I ended up leaving at the half-time break and spending the evening with my family instead. Nevertheless if you're interested in TED then I would recommend giving an event like this a go, if nothing else there were some great people there like Mark Cohen, Jodie Miner and Brian Maguire.

Some background: TED is an event that focuses on thought leadership in the Technology, Entertainment and Design fields every year and has as its catch-cry "Ideas worth spreading". The TED conference sells out quickly every year (despite some unusual conditions) and attracts all sorts of luminaries and leaders. A TED event is the place to meet movers and shakers and has been the birthplace of some wonderful ideas over the years. They make the videos of each event available online during the year, and part of the focus of a TEDx event is to create that same buzz in a local venue by mixing TED videos with local speakers.

So without further ado here is my thoughts on TEDx Sydney 2009:

Venue

The Australian Technology Park hosted the event, and as that is where Elcom is based that proved convenient for me. The theatre used was a great venue for watching TED videos and live speakers alike and I think it did a good job of providing some atmosphere - although I wonder whether other venues could have made a bigger impression? (e.g. Darling Harbour Convention Centre)

MC/Host (Nils Vesk)

I don't think a Gen-Y host was the right choice for an event like this - I think less entertainment and more credibility was needed (although it was Mark Cohen who blew Nils' credibility for me). However, Nils got the job done, and managed to do the more cheesy sales aspects of his role without making it seem too cheesy (more about that later).

Craig Rispin (Futurist)

I have heard of Craig Rispin, and indeed knew someone who he'd invited to the event. Unfortunately his recursive telling of the history of futurists who tell the history of the future left me bored and dual-tweeting with Mark Cohen.

I'm not sure Craig had an idea worth spreading, perhaps he intended it to be that we are all futurists, but it felt more like he was telling us that we were smart and forward-thinking people - which was nice of him, but didn't give me a reason to care about his talk. He also pointed out the prophetic nature of speculative/science-fiction in a way that might have impacted those who didn't yet care about it - but I've known about that for most of my life so it was wasted on me.

Patti Maes (Video, MIT Media Lab)

Patti Maes is from the MIT Media Lab and demonstrated a form of augmented reality where her students had cobbled together a wearable apparatus that linked a camera, phone and some software so that you could make any surface around you into a computer work area, and interact with your environment in ways that could both instruct the computer/camera to do something (e.g. take a photo by holding your hands up in a rectangle) or could bring digital information to you based on your context (e.g. talking to a person, reading a book, looking at a product label).

She pointed out that the technology could be cobbled together for a few hundred dollars easily enough, but their point was that they were investigating what were reasonable ways to augment reality. Some of this has been done before with for example the head-up displays (HUD) that Boeing engineers use when wiring plane bodies.

This was the most exciting talk, but hardly surprising to anyone who has been following technology for the last 15 years or so.

Peter Baines (Hands Across the Water)

Peter Baines was one of the forensic specialists sent to Thailand in the aftermath of the recent Boxing Day Tsunami. He gave us a fairly somber account of that, especially dealing with the human side when giving bodies back to relatives. He went on to show how this created a desire in him and his colleagues to do more than they were for the Thais - out of that desire was birthed Hands Across the Water.

I think that Peter's idea worth spreading was that if you just focus on the results, and not the obstacles, then you can achieve your dreams - whatever they are, whoever you are. I think this was a great message, and Peter is obviously a good example of this in action. Focus, and the single-minded pursuit of an idea, is too often given lip-service as a reason for success, when I believe it might just be the major factor in it.

A large part of Peter's story was a dig at the international aid agencies that quickly went into Thailand and then just as quickly pulled out around a year later. One of his ideas is that charities should be forced to put some stats on every piece of marketing literature telling the reader how much money they spent on admin versus aid in the last financial year. The idea (much like the MIT Media guys) being to give people information to make informed decisions about where to give their money. I think this ignores part of the bigger picture, which is a question of how much admin is required to run an international charity that coordinates volunteers from many nations into situations that are often not as attractively packaged as the Thai one was. However that discussion can't happen without making the information available, and so I would welcome it form that point of view.

Bill Gates (Video, Curing Malaria and Making Teachers Better)

I had seen this one before, but hung around to watch it on the big screen. Bill and Melinda Gates' foundation is really trying to focus money and attention onto big issues that the market and governments are ignoring.

The first issue he covered was curing malaria, which seems both an achievable aim and one that has been largely ignored now that wealthy nations are free of it. His idea here was that if we focus on eradicating it then a relatively modest effort will see it gone, whilst if we tackle it half-heartedly we only make things worse.

The second issue was about how do you make teachers great, and he gave a long analysis of why this mattered, which seemed obvious, but I guess the hard numbers he pulled out showed people that it was an issue worth focusing on. I think his idea here was that we should care about the quality of teaching - mainly because in the US at least, the quality of a teacher is not easily discovered or rewarded.

Event

Overall the event had a good vibe, seemed interesting, and was a good way to share the watching of a TED video with people you knew. I think I missed too much of the schmoozing part of it to know whether that worked - but I was happy catching up with who I did see there.

There was a cheesy side to the event which was the blatant pushing of wares for the main sponsors, especially the silent auction of the Thought Leaders' services (with some fairly suspicious dollar figures on their market value). I understand that some of this is necessary to get the event paid for - but I do wonder if this could have been handled in a more subtle and positive way.

Top 5 classic programmer folly projects

2009-05-14T17:41:00.000+10:00

Back in the day when the British Empire was still new, wealthy landowners who had heard about great places elsewhere (usually ones where it didn't rain all the time) started a trend of re-creating these great places on their own land - often right down to the current decrepit state of said places.

Sometimes they didn't have a specific place in mind, but wanted a "ruined Greek temple" or similar stereotypical "interesting" structure (sometimes they were just plain strange). The point seemed to be to prove that a) you were interesting, b) you could pay for a spurious building for the sake of it, and c) foreign travel was all so unnecessary. These were known, rather aptly, as follies (presumably because they represented the folly of spending your children's inheritance on an expensive monument when investing it might have proven more prudent).

From: Xerones

Programmers it turns out are no less susceptible to follies, although they might not realise it, and they are not deliberate fakeries. There are several classic programmer follies which tend to get repeated by each generation, with ever-increasing zeal and lack of awareness of their futility.

Now, don't get me wrong, these are all fine examples of throwaway projects that you might play around with and practice on, but for some reason programmers can't seem to leave them stay as just throwaway projects - they always want to upgrade them to the status of "next big thing".

Without any further ado, here they are.

Top 5 classic programmer folly projects

1. Bug tracking

Every software program has bugs, and most programmers find they need a good system to track said bugs. Some companies, Atlassian and Fog Creek Software have made their living providing great, simple yet powerful solutions for exactly this problem.

However, there is always a temptation to take shortcuts, customise the bug tracking, or get a cheaper solution, and so inevitably every programmer gets tempted to build their own bug tracking application.

It's a folly because ...
This is actually a fairly complicated domain, especially once you add in the bug resolution workflow, web interface and email integration that even free, open-source applications have. Of course there is also the problem of what to do when your bugs are in your bug tracking software ... evil recursion in the real world!

2. To do list

I just want to track what I need to do ... and what I want is to improve this to do list. Remember the Milk has the best free implementation I've seen of this type of application, but BaseCamp, your mobile phone and even GMail are worth mentioning (not to mention trusty old pen and paper).

It's a folly because ...
You really don't need to waste brain cycles re-inventing something this simple, and when you look at the sophistication of Remember the Milk (web, offline Google Gears, Twitter, email and mobile integration) anything you do yourself looks kinda ... sad.

3. Website/blog engine

I want to use something I built myself, and it's a good chance to try my hand at content management ... except it isn't. If you are blogging then do your readers and yourself a favour and use an existing blogging application. You will have a more consistent user interface and will never find yourself not posting because the build is broken after that last refactor and you can't be bothered fixing it yet.

It's a folly because ...
The free alternatives are great, the cheap ones are even better and frankly it's not that great a domain for learning how to code, especially web stuff.

4. Hobby management

My hobby (roleplaying games, model robots, high performance cars, salsa dancing, etc) is out of control and I need a way of managing it. Actually no you don't, you probably just need to get out and live more.

It's a folly because ...

You don't really want to mix up your hobby and your work, it just makes one less fun and the other less professional. it is also an area where you will never really be satisfied with the result because it's your hobby. Finally, when you show it to other people they are likely to go "Ummm ... very nice. What is it?" Hugh MacLeod defines a geek as "Somebody who socializes via objects" and congratulations, that's exactly what you've just become!

5. Application Building Applications

This is the one where you build an application that uses meta-data to build an application. It seems this is the purvey of the very smart and the very dumb. The really smart guys believe they can be the ones to break through and nail this problem, the really dumb ones just don't realise how hard it is.

It's a folly because ...

It's possible to do this in theory, but in practice you only ever get around 80% of the way there, and it turns out the other 20% is the bit that an application really needs to live and breathe. Really, really smart guys manage to push that 80% up to the giddy heights of 81.63%, but hey, it still doesn't do what you really wanted it to.

Bonus! Installer software

OK, here is number six. What do you do when you want to distribute your application to clients? Use the MSI installer for Visual Studio? Heck, no! You go ahead and create your own installer software, complete with license number check, built-in unzipper and some cross-platform nastiness.

It's a folly because ...

You already know you shouldn't be doing this. Go use something someone else created and love the fact that you don't need to do it by hand!

OK, time for confession time, I've been guilty of each of these (at least through to the design stage). What about you, what were your follies?

Loading objects' fields/properties automagically

2009-02-20T15:17:00.001+11:00

There is a very nice function in the new ASP.NET MVC code that allows a Controller to load an object from values passed in from a HttpRequest (UPDATE: called UpdateFrom, see ScottGu's post). It did a best guess on matching values to object properties, but most (99%?) of the time that is all you need. Well, when I started using Domain-Driven Design (DDD) and Test-Driven Development (TDD) on the latest additions to our Community Manager.NET product I wanted to find something similar - but I couldn't find anything as terse and useful.

So I did what any good dev would, I read the sourcecode and derived my own functions to do something similar, but with the SqlDataReaders that my repositories would be getting out of SQL Server (it almost feels like the ActiveRecord pattern ...). Below are the result, two LoadToObject() functions (and a PopulateTypeException class) in my BaseRepository class that give me the functionality I need. I'm putting these out there in case someone else needs this, and in the hope that any glaringly obvious errors might get picked up by others!

One issue I had with the Microsoft version of this function was they only load public properties of an object, not public fields. Now I know that in C# you can setup properties as easily as fields, but in VB it's still a hassle. I personally like using fields for simple value storing because it is clearer to see what they do in the code and leave more space for real method calls.

The final problem I ran into was that my custom Html data type needed to implement it's own TypeConverter so that the HTML in the SQL Server ntext field could be converted to a strongly typed Html type. But that can go into another blog post if people are interested.

    ''' <summary>
    ''' Populates an object's public fields and properties with values from a SqlDataReader.
    ''' </summary>
    ''' <param name="readerToLoad">The SqlDataReader that needs to be loaded into the object.</param>
    ''' <param name="obj">The object we want to populate.</param>
    ''' <param name="objectPrefix">A prefix to the object field/property names. Tries to match using "." and "_".</param>
    ''' <returns>The object passed in with fields/properties loaded.</returns>
    ''' <remarks>
    ''' Does a best guess as to how to match names from the SqlDataReader to the object.
    ''' </remarks>
    Protected Function LoadToObject(ByVal readerToLoad As SqlDataReader, ByVal obj As Object, ByVal objectPrefix As String) As Object
        Dim newColl As New System.Collections.Specialized.NameValueCollection()
        Dim x As Integer

        If obj Is Nothing Then
            Throw New ArgumentNullException("obj", "Object must have a value in order to be loaded to.")
        End If

        For x = 0 To (readerToLoad.FieldCount - 1)
            newColl.Add(readerToLoad.GetName(x), readerToLoad.GetValue(x))
        Next

        Return LoadToObject(newColl, obj, objectPrefix)
    End Function

    ''' <summary>
    ''' Populates an object's public fields and properties with values from a NameValueCollection.
    ''' </summary>
    ''' <param name="valueCollectionToLoad">The NameValueCollection that needs to be loaded into the object.</param>
    ''' <param name="obj">The object we want to populate.</param>
    ''' <param name="objectPrefix">A prefix to the object field/property names. Tries to match using "." and "_".</param>
    ''' <returns>The object passed in with fields/properties loaded.</returns>
    ''' <remarks>
    ''' Does a best guess as to how to match names from the NameValueCollection to the object.
    ''' </remarks>
    Protected Function LoadToObject(ByVal valueCollectionToLoad As System.Collections.Specialized.NameValueCollection, ByVal obj As Object, ByVal objectPrefix As String) As Object
        Dim objType As Type = obj.GetType()
        Dim objName As String = objType.Name
        Dim exceptionList As New StringBuilder()
        Dim props As PropertyInfo() = objType.GetProperties()
        Dim fields As FieldInfo() = objType.GetFields()
        Dim ex As PopulateTypeException = Nothing
        Dim prop As PropertyInfo
        Dim field As FieldInfo

        ' try writing to the object's properties
        For Each prop In props
            'check the key, going to be forgiving here, allowing for full declaration or just propname
            Dim key As String = prop.Name

            If objectPrefix <> String.Empty Then
                key = objectPrefix + key
            End If

            If valueCollectionToLoad(key) = Nothing Then
                key = objName + "." + prop.Name
            End If

            If valueCollectionToLoad(key) = Nothing Then
                key = objName + "_" + prop.Name
            End If

            If valueCollectionToLoad(key) <> Nothing Then
                Dim conv As TypeConverter = TypeDescriptor.GetConverter(prop.PropertyType)
                Dim value As Object = valueCollectionToLoad(key)

                If conv.CanConvertFrom(System.Type.GetType("System.String", True, True)) Then
                    Try
                        value = conv.ConvertFrom(valueCollectionToLoad(key))
                        prop.SetValue(obj, value, Nothing)
                    Catch e As Exception
                        Dim message As String = prop.Name + " is not a valid " + prop.PropertyType.Name + "; " + e.Message

                        If ex Is Nothing Then
                            ex = New PopulateTypeException("Errors occurred during object binding - review the LoadExceptions property of this exception for more details")
                        End If

                        Dim info As New PopulateTypeException.ExceptionInfo()

                        info.AttemptedValue = value
                        info.PropertyName = prop.Name
                        info.ErrorMessage = message

                        ex.LoadExceptions.Add(info)
                    End Try
                Else
                    Throw New Exception(String.Format("No type converter available for type: {0}", prop.PropertyType))
                End If
            End If
        Next

        ' now try writing to the object's public fields
        For Each field In fields
            'check the key, going to be forgiving here, allowing for full declaration or just fieldname
            Dim key As String = field.Name

            If objectPrefix <> String.Empty Then
                key = objectPrefix + key
            End If

            If valueCollectionToLoad(key) = Nothing Then
                key = objName + "." + field.Name
            End If

            If valueCollectionToLoad(key) = Nothing Then
                key = objName + "_" + field.Name
            End If

            If valueCollectionToLoad(key) <> Nothing Then
                Dim conv As TypeConverter = TypeDescriptor.GetConverter(field.FieldType)
                Dim value As Object = valueCollectionToLoad(key)

                If conv.CanConvertFrom(System.Type.GetType("System.String", True, True)) Then
                    Try
                        value = conv.ConvertFrom(valueCollectionToLoad(key))
                        field.SetValue(obj, value)
                    Catch e As Exception
                        Dim message As String = field.Name + " is not a valid " + field.FieldType.Name + "; " + e.Message

                        If ex Is Nothing Then
                            ex = New PopulateTypeException("Errors occurred during object binding - review the LoadExceptions property of this exception for more details")
                        End If

                        Dim info As New PopulateTypeException.ExceptionInfo()

                        info.AttemptedValue = value
                        info.PropertyName = field.Name
                        info.ErrorMessage = message

                        ex.LoadExceptions.Add(info)
                    End Try
                Else
                    Throw New Exception(String.Format("No type converter available for type: {0}", field.FieldType))
                End If
            End If
        Next

        If Not (ex Is Nothing) Then
            Throw ex
        Else
            Return obj
        End If
    End Function

    ''' <summary>
    ''' Contains exceptions that arise during the populating of an object by the LoadToObject() method.
    ''' </summary>
    <Global.System.Serializable()> Public Class PopulateTypeException
        Inherits Exception

        Public Class ExceptionInfo
            Public PropertyName As String
            Public AttemptedValue As Object
            Public ErrorMessage As String
        End Class

        Public LoadExceptions As New List(Of ExceptionInfo)
        Private PopulateTypeException()

        Public Sub New(ByVal message As String)
            MyBase.new(message)
        End Sub

        Public Sub New(ByVal message As String, ByVal inner As Exception)
            MyBase.new(message, inner)
        End Sub

        Protected Sub New(ByVal info As System.Runtime.Serialization.SerializationInfo, ByVal context As System.Runtime.Serialization.StreamingContext)
            MyBase.New(info, context)
        End Sub

    End Class

Challenging software development blogs

2009-02-10T15:30:00.001+11:00

I am not hugely keen on top 100 lists or similar trite listings of people, blogs or posts, however sometimes I notice that I have too many tabs open in Chrome (hello, 58?!) and need to itemise some of them. Many of the ones I keep open are blogs that challenge me enough that I want to read them frequently (I know ... I should be using an RSS reader, so call me old-school! ;).

I think these are some of the more challenging blogs* out there with regards to software development - read these and you will find your brain, assumptions and dealer-held beliefs under high-voltage attack.

* OK, so I am going to miss a couple of fantastic ones - what is important is that you check out the ones I do mention!

Joel on Software (visit)

Joel Spolsky of Fog Creek Software and Stack Overflow fame has had one of the most well-read and discussed software development blogs on the planet for years now. He is an original thinker in a market filled with duplicates, he also has a deep commitment to giving his developers great office space.

The Software Simplist (visit)

Udi Dahan's blog specialises in Service Oriented Architecture (SOA), but I read him for his refreshing view of domain-driven design (DDD). I got a new insight into DDD from his DDD & Many to Many Object Relational Mapping post.

Scott Bellware's Blog (visit)

Scott Bellware is probably the most hated/respected .NET rebel on the planet. His abrasive manner and obvious chip on his shoulder with Microsoft and the ALT.NET mainstream put some people off, but he has a deep insight into what Agile development is meant to be all about and a straightforward manner in sharing what he knows. He also seems pretty sure that most developers are learning impaired. A good example of his (more positive) thinking is Productivity: It Comes from Software Design Rather than Software Tools.

Noop.nl (visit)

Jurgen Appelo's software development management blog has some of the most pragmatic and interesting advice available for functional managers of developers. His company's situation is similar to Elcom's, so I appreciate his honest and smart insight into how to make Scrum work with large teams working on many disparate projects. He has useful posts like How to Handle Many Simultaneous Projects which apply Lean/Scrum principles to the real world of custom software development.

You'd think with all my video game experience that I'd be more prepared for this (visit)

Jason Yip from ThoughtWorks focuses his blog on lean manufacturing principles and how they apply to the reality of software development. A challenging thinker and opinionated professional, he manages to stay as radical as someone like Scott Bellware with a more pleasant demeanour. A typical (longer) post from him would be No matter how many times you say it, we still don't need a QA on the team.

Greg Young's Blog (visit)

Greg Young is a Microsoft MVP who has some very interesting opinions on domain-driven design, namely that the domain model is only there for write operations. Check out his DDDD: Master-Detail Question post for more details on this and how he uses a thin DTO layer to power his read-only UI pages.

Chad Myers' Blog (visit)

I first found out about Chad from a talk he and Jeremy Miller gave on Opinionated MVC (see videos here and here). Jeremy Miller developed StructureMap, which is a Dependency Injection/Inversion of Control framework. Chad is also the driving force behind the Virtual ALT.NET (VAN) group which has helped developers connect from places as far afield as Texas and Australia!

James Bach's Blog (visit)

Interested in software testing? Then you need to quickly head over to James Bach's blog and check out his controversial, but smart and honest posts on the topic. To understand where he is coming from, check out his Schools of Testing ... Here to Stay blog post.

So ...

Who did I miss? Leave a comment and let me know!

The Fat Controller must die!

2009-02-05T09:06:00.001+11:00

As most parents with little boys know the Fat Controller is a key figure in the Thomas the Tank Engine TV series (and books, toys, clothing, linen, bags, crockery, snacks, etc.). he is a cheery fellow, unfortunately prone to angrily shouting at train engines, but equally kindly and caring about his beloved vehicles (especially steam powered ones like Elizabeth the lorry). For the sake of my children I must point out that I have no problem with this jolly fellow.

There is another domain where fat controllers exist, and that is the world of the MVC (Model-View-Controller) pattern. The pattern has experienced a renaissance recently in web applications, particularly because of Ruby on Rails, and now Microsoft's ASP.NET MVC offering.

Now these Ruby on Rails guys have been doing this for a little while longer than the ASP.NET guys, and one key paradigm became clear fairly early on:

"Try to keep your controller actions and views as slim as possible."

This was most clearly explained by Jamis Buck in his excellent Skinny Controller, Fat Model post back in 2006, which is still worth reading even if you don't use Ruby on Rails. More recently Ian Cooper has pointed out this is the same problem webforms had:

"This is the good old problem of domain-logic in code-behind that we had in ASP.NET webforms. Indeed Webforms are just another type of controller, a page controller, and switching to an application controller model does not remove the need for us to watch for domain logic creeping into the controller. There can be a temptation to believe that just because the controller is easier to test it is now safe to put domain logic in there. Do not fall into that trap."

In his book, Domain-Driven Design, Eric Evans identifies this as the Smart UI Anti-Pattern.

“Put all the business logic into the user interface. Chop the application into small functions and implement them as separate user interfaces, embedding the business rules into them. Use a relational database as a shared repository of the data. Use the most automated UI building and visual programming tools available.”

Image from David Hayden's blog

Eric is kind (and pragmatic) enough to point out when it is useful.

“A project needs to deliver simple functionality, dominated by data entry and display, with few business rules. Staff is not composed of advanced object modelers.”

Clearly ASP.NET's webforms model encouraged this sort of practice (it is just far too easy to do), but surely a controller is a safe place for this code? After all, a controller can use many views, so it is more clearly separated than code behind. The problem is that whilst application logic ("Which view do I show next?", "What domain object is handling this request?") makes perfect sense in a controller, there are too many examples of them being used as stores for business/domain logic - which is the province of the domain model.

Hang on, we probably have a (ubiquitous) language problem here. You see, the "Model" in MVC != the "domain model" in Domain-Driven Design (DDD). In fact we need to map the DDD concept of layered architecture to MVC in order to see what we truly have.

Instead of giving the Model layer a fully fleshed out domain and supporting infrastructure (repository and services), many ASP.NET developers seem to want to treat the Model as pure database infrastructure and DAL (like the one in the Smart UI graphic above). This leads to the problem of where to put the business logic, and the natural assumption is that it belongs in the controller ... which is like making the navigator the captain of the ship.

Ian and Jamis have much more (technical) stuff to say on this subject so I'd advise you to check their posts out if you're into .NET or Rails, respectively, but do take this message with you, the Fat Controller must die!

Useful URLs

2008-12-10T10:39:00.002+11:00

Whilst at the Sydney ALT.NET meeting ~~this week~~ last month (this post took far too long to write) I had two divergent lines of thought crossover and mashup inside my brain ...

I have been working for some months on a large corporate extranet portal and have been steadily fighting to keep the client focused on organising the site around, not my ideas of their organisation's needs, not their ideas of their organisation chart, but rather on the information needs of their clients, the people the portal is really there for.

This can be subtle, after all we all have good reasons for liking our ideas of how to organise the site. In fact we needed to make sure that the content authors from our client in fact could publish their content in a way that made sense to them (and thus organised around their needs).

Back to the Sydney ALT.NET group, as Ali went about explaining the usual routing options with ASP.NET MVC it occurred to me that ASP/ASP.NET developers have been doing the same thing with the organisation of their application URLs. For years they have been organised around the structure of the site as it is developed, with folders in the original web projects visible on the final website URLs. Even when we have introduced user-friendly URLs, and used ASP.NET MVC's routing we still manage to come up with ideas that fit our view of the business model and not our users' needs.

Aside: Do users even care about URLs?

This is a great question because it is at the heart of what I'm talking about here. My subjective view is that they do, I know that I do and I've seen people refer to the URL when trying to work out where they are within a site. However, many sites still have complex, unfriendly URLs and I think the average user (quite rightly) doesn't trust them.

Let's look at it another way, do you care about your users? If you do then you will want to help them every way you can, including making your URLs friendly and useful.

I suspect that the real problem is that we lack a clear design language, or paradigm for helping us work these things out. After all when you can do anything with a URL, who is to say what is right? Yet, if there is one thing the growth of the web has taught us, it is that UI standards grow out of users liking websites that follow common practices (e.g. shopping carts instead of bags, boxes or orders; horizontal main navigation menus as opposed to vertical, circular, drop down boxes, etc.). See Jakob's Law of the Internet User Experience:

"Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know."

Similarly we may find that websites that follow RESTful URL paradigms with particularly clear syntax are seen as much more useful than ones where you have to rely on the menu structure or search. Here is a (contrived) online shopping example:

Old School (ASP.NET Webforms):

http://domain/shop/cart/default.aspx?Method=AddCart&ProductId=123

New School (ASP.NET MVC):

http://domain/cart/add/123

Most Useful (or not):

http://domain/buy/Sony/Walkman?quantity=1

Now I'm unhappy with that last example (perhaps going against the "shopping cart" paradigm muddles things), but my point is that we should phrase things the way the user sees them and not what suits our architecture/business model. Give the URL back to the user as another useful tool and see what happens!

My new role at Elcom Technology

2008-11-30T00:32:00.001+11:00

Some of you might be aware that Elcom's Technology Director and my friend Craig Bailey is leaving Elcom Technology - in fact his last day was last Friday. As our Technical Director he was inspiring, motivating and a great encourager - he let us know we had his confidence and trust, and backed that attitude up with his words and actions when under stress.

I will be stepping into Craig's shoes, and taking on the role of Technical Director for Elcom and helping guide our development, sysadmin and helpdesk teams. Whilst we have some similarities, I have more of a focus on Agile development and software craftsmanship (as Craig has mentioned). I will probably be taking a more hands-on approach to the role - simply because I have contributed to the dev work I will now be managing.

Craig and I have been discussing this role over the last month or so, and he got me interested in the hedgehog concept. That last link explains it pretty well, but basically it is all about helping you focus on what you could be best at in the world, how you can add (money) value and what you are most passionate about. The point where you will be most effective, rewarded and fulfilled, lies in the region where those three overlap.

After thinking about these, and discussing them with people that know me, my (current) take on these three is below:

I had followed a similar process many times in the past, but without the emphasis on coming up with a "simple, crystalline concept that flows from deep understanding about the intersection" of these three circles. Or as my Dad put it to me, something actionable and concrete.

Being Technical Director at Elcom offers me a chance to move closer to that intersection point, and I hope that in my first few months I can work on those things that might currently distract me from that focus in order to generate some very tangible results for myself and Elcom's owners.

I am a firm believer in servant leadership and am honoured that I have some of the best technical people it's been my pleasure to work with in our development, sysadmin and helpdesk teams. With great people in the team around you, it certainly makes my job easier, and more pleasurable. Like a good scrum master I am aiming to remove obstacles in the way of our technical teams and act as force multiplier for the whole team (the OODA Loop has some interesting parallels with the Agile principles).

It should be an interesting journey ahead!

My first Agile retrospective

2008-11-07T09:52:00.003+11:00

I facilitated my first Agile retrospective today, going over a six month project with 12 people, mostly from Elcom's development team, along with one or two others from the systems admin, project management and implementation teams. We followed the format suggested by the Retrospective Goddesses in their Agile Retrospectives book. It was an interesting and rewarding experience, so I thought I'd document it here.

Alan Lee, Rajesh Stephen and Chae Wan Kim do some group work.

This project had been a large and challenging one for us, whilst still a success for the ultimate client. It had sucked in most of the development team at one point and I was one of the few people not directly involved. When our management decided to run the retrospective they had envisaged nothing more than a brainstorming session and opportunity to vent some emotions.

We had purchased the Agile Retrospectives book some months ago and I suggested that our Technical Director run it using the book - except it had somehow gotten stuck in my bookshelf at home. Whilst bringing it in I flicked through it and noticed that the main facilitator role sounded exactly like the facilitator role I've been fulfilling in the Valiant Man courses I've been involved in this year. So I volunteered myself as the facilitator, negotiated to increase the time from one to two hours and promptly got stuck with all the preparation involved!

Working Agreement

As the person with the most experience of this sort of group work I suggested most of the agreements, but the group did come up with "Don't interrupt others" on their own. Going "topless" was particularly hard for the managers in the group - but I think it really helped them engage with the process.

Goal

This was an interesting one, I had selected a reasonably neutral, positive (and useful) goal by focusing on one of the unusual characteristics of the project, our work with a partner organisation. But the team thought that added to this should be a focus on big projects as this was another defining characteristic of this particular project.

I then got everyone to give us a quick one or two word description of their feeling or state at the end of this project. This was interesting as we got flat out positive and negative responses as well as mixed ones like, "stressed, happy." The range of emotions described was a reminder that not everyone sees things the same way.

Information Gathering

I chose the timeline method to draw out details from the group as that would allow us to vary the amount of time we spent on it, and for a six month long project would help people remember their experiences.

Tracey Schneider, Gavin Silver, Balaji Sridharan and Chae Wan Kim place their cards

The group broke out into three groups of 4 and worked on identifying important events in the project timeline, categorised into client (blue), Elcom (green), people/team (pink) and technical (yellow). I had previously prepared the whiteboard by separating it into months, and including the number of days of developer time used in each one.

We ended up with a very interesting overview of the project that included many events that people had forgotten, or not even known about, as well as some that everyone was affected by (we stacked duplicate cards on top of each other). I mostly stood back and watched (that's me below, closest to the camera), and occasionally stepped in to help people understand how to handle duplicates (stacking), confusions with long-running events (put multiple cards across months at same level) and disagreements about what was an event (pretty much anything could be).

Craig Bailey, Tracey Schneider and Gavin Silver check out the cards, watched by me.

Once each group had their cards up, I asked people to review the board, one group at a time, and discuss what others had identified, as well as looking for duplicates or events still missing. That only got us one or two extra events, but gave everyone a chance to see the total timeline up close.

I then told each participant that they had five positive (green) dots and five negative (red) dots to apply to cards on the timeline - but they could only put one dot on each card. They came up in pairs to dot the cards. I wanted to use colour coded dots, but we made do with red and green pens. You can see the result below:

People wanted to know how to identify what was positive, or negative, but I pointed out it wasn't supposed to be scientific, just a subjective review of their feelings about the events listed. That worried people less than I thought it might.

Once they had all finished I took over and went through the events that seemed the most interesting (lots of green dots, red dots, or a mix). I also pointed out events that were usually regarded as important, but that ended up having little significance to the team.

Some of the event cards turned out to be confusing, so I also queried what they meant and made sure I added text where necessary to help clarify their meaning. One of the events that had a real mix of dots, turned out to be interpreted two completely different ways, so we drew a line between the dots and added text to each side to explain further.

Identify Experiments and Actions

Unfortunately time was short for this retrospective, so I had planned a quick 5 Whys activity to help the team quickly find useful insights. They broke up into small groups/pairs (two groups of 3 and three pairs) and started asking why.

5 Whys has one participant ask the others in their group why an event was important, positive or negative. It was left pretty open for them to choose their favourite item from the timeline. Once they got the answer, then then asked why that was important/relevant. They would do this a total of 5 times, and only record the answers to the fourth and fifth questions. Every member of each group got a chance to do this at least once and we got some funny and insightful responses as a result (one one of the cards below the fifth answer was that the person was "speechless", so they drew an empty word bubble!).

Next was the only awkward part of the retrospective (from my perspective). I had planned for people to do a vote on each item with one stroke per card, as many strokes as they wanted (hat-tip to ThoughtWorkers Lachlan Heasman/Jason Yip for that one!), but the criteria I wanted to use was "which item would you most want to work on". Indeed this is what I announced to the group, but I noticed when people actually came up to vote that the question/answer cards did not identify the experiment/action that could be taken to answer them.

We were over-time by this stage, so I made sure everyone knew what the problem was, but let them keep voting anyway. I then had to step in to organise the cards by stroke-votes (as shown above) and then run a quick discussion of what steps might be taken for each one.

The result was a set of 5 actions that were not really owned by the team. Coincidentally this was also when the more vocal and confident management types really shined, limiting input from others. To be fair (to myself and the team) this project highlighted some really basic issues in our project management, sales and quoting procedures - and these caused so much grief that it was a bit hard to see past them to other problems that might be owned by more of the team.

Summary

I'm really proud of how enthusiastic the development team were in approaching this. I did a Return On Time Invested poll at the end of the event and on a rating of 0-4 (0=No return, 2=Break even, 4=High return) most of the guys thought it was a 4, some thought it was 3 and a quarter of the group rated it as a 2 (one of those turned out to misunderstand what we were measuring and later said she would have voted 4 too!).

We should be making this a regular part of our Agile process and I will certainly be agitating for it occur more often. I'm sure that having more retrospectives will help identify some of the more fundamental issues that affect the development team and get us continually improving.