Being Extreme

Software Factories Skepticism

2005-12-22T10:47:00.000+00:00

I'm no expert on Software Factories but frankly I am a skeptic. I have spent a day or so reading about them on the InterWeb at the behest of one of my colleagues. There are several very well written articles on Software Factories out there. I particularly like the Jack Greenfield articles on the Microsoft site (Parts 1, 2 , 3 (where's 4 BTW?)). There's no doubt the Software Factory proponents are very articulate and maybe that's one of the things that makes me skeptical?

There is no conflict between Software Factories as a basic idea and Agile. Indeed Jack Greenfield acknowledges, "modest gains in productivity have been made over the last decade, the most important perhaps being byte-coded languages, patterns, and agile methods". I'm not sure about the first two, but I'd agree with the 3rd.

Where my skepticism starts building is around the following points:

The Software Factory arguments sound a lot like the old Object Oriented/Reuse Library ideas. I used to be part of the Reuse Library effort at Morgan Stanley in the "early days" of adoption of objects in finance, so I am am familiar with the pitch. The issue which arose was that as much as you ended up with tools, frameworks and configurable components the productivity only came in when the users of the libraries and tools really understood the whole kit and could apply them. This required a lot of learning, which meant a lot of time before becoming productive. Forget "big design up front", this is "big learn up front". Clearly this can be solved by having "domain" experts learning on somebody else's penny before having to pay for them to build your system. But isn't this just reducing the accounting cost of a single project by ignoring the cost of the learning hump?
On a similar vein, I find it curious that a lot of the people from the early days of Objects and Reuse Libraries are the same people involved in DSLs and Software Factories (e.g. Steve Cook, Alan Cameron-Wills, etc). Same arguments and ideas, different century? Are better tools and larger distribution the answer? Maybe there is a flaw in the model that made it not live up to its promise the first time around?
Part of the argument for Software Factories is "the progress of the level of abstraction" that developers use to create systems. In my experience increasing abstraction often leads to reduction in productivity. The quote Jack Greenfield uses is from Smith and Stott's OOPSLA paper: "The history of programming is an exercise in hierarchical abstraction. In each generation, language designers produce constructs for lessons learned in the previous generation, and then architects use them to build more complex and powerful abstractions.". My problem is that the more abstract you become the more strongly defined (and smaller) your context must become in order to maintain a productive level of relevance. Without narrowing the context, the more abstract you become the more meaningless the artefacts become. My experiences with abstract frameworks and developers trying to use them led me to question abstraction. The XP approach of "create the value then refactor" was a better approach IMHO which still recognised the value of abstraction but maintained a practical grip on the scale of the context. I was also influenced by a quote from Joseph Joubert:

"How many people make themselves abstract to appear profound. The most useful part of abstract terms are the shadows they create to hide a vacuum."

The value in Industrialization in manufacturing came mainly from the change in the practices associated with creating goods. New tools were created to match the new practices and definitely increased the productivity too, but ask the question in reverse. If the specialized machinery for creating components had been created and applied with a craftsman process, would the productivity have increased as much as happened through purely changing the practices to production-line?
I believe these tools are emerging from the Agile community at the correct level of abstraction required for necessary productivity. IDEs with refactoring tools, Open Source frameworks for persistence, messaging, etc. Are these not the same elements of a Software Factory? There is a way of looking at APIs and scripting as DSLs.
An Agile team I believe they create their own Software Factory for there own domain through disciplined practice. This may appear to support the Software Factory argument, but there's a big difference: the people. Tools and documentation by themselves are not sufficient without the people who share and build the context the abstractions exist within. I would suggest that a manufacturing factory without people is pretty useless - indeed a manufacturing plant without people who have some idea how to use the tools is not much better. Possibly the term 'software factory' would be more appropriate for refering to a team of people who can produce systems using tools?
Where is the demonstration of Software Factories improving productivity? I am struggling to find evidence. If someone has good data please show me! I find plenty of opinion. I find plenty of confidence. But I find no evidence?

There is a business model that is sometimes refered to as "selling shovels to gold prospectors", that is to say that there is better money in selling the tools rather than taking the risk of striking it lucky. This definitely has a hint of that to it, but it feels more like they're selling divining rods, magic maps, voodoo incantations, lucky rabbits feet as well as shovels in a whole kit - and if you "use them properly" everything will be perfect!

I have no doubt if they achieve the essence of Software Factories in a coherent, understandable way then combining Agile Development with Software Factories will produce productive software delivery. But I currently believe its the Agile rather than the Factory that is the productivity boost.

I recognise my own skepticism is not necessarily useful. I am reminded of George Bernard Shaw:

"Every person who has mastered a profession is a skeptic concerning it. "

There is a strong possibility that Software Factories may be a better way of looking at IDEs, APIs and Frameworks. And maybe I'm the wrong person to measure the effort having been in software so long - but I don't see the evidence that this will help software development productivity or quality.

Winning an Award for Being Agile

2005-12-16T12:49:00.000+00:00

I came across this article describing how a game company won a industry prize for using Scrum. I can't decide if this is good or not? Maybe I'm just jealous ?

Agile and Organizational Change Management

2005-12-15T11:16:00.000+00:00

Agile System Delivery in itself is not the main problem when dealing with clients. They want the benefits: regular delivery every 30 days, high quality, ability to change their minds, etc. They understand it, they understand to an extent why it might work but there is always "resistance" to the adoption of Agile practices. This is because you are trying to get the people to change the way they behave and Agile by itself does not address this process.

The field of Organizational Change Management (OCM) addresses the issues arising when trying to change the behaviours in an organization, but on the whole whenever I talk to Agile coaches or consultants they seem to ignore the fundamentals of OCM and are often ignorant of them! Next time you meet an Agile consultant ask them who John P. Kotter is?

Agile consultants, or even those who wish to have Agile adopted in their teams, need to become more aware of OCM techniques and tools. Of course there is the issue of what is the useful stuff in OCM? There is such a huge volume of stuff out there and on the whole it is fluffy, touchy, feely and frankly useless.

There are one or two classic pieces on why Organizational Change fails, such as John P. Kotter's "Leading Change: Why Transformation Efforts Fail" in the March-April 1995 Harvard Business Review (a summary here). These are well worth reading as they contain real tools you can use to start thinking and structuring an approach to the changes required in your context for Agile to be adopted and used successfully.

Let me give an example:

Recently, the Harvard Business Review published the Boston Consulting Group's change assessment model called the DICE framework. The DICE framework has been around since the late 90s and is interesting because it is an easy metric that can assist you in understanding the likelihood of success in a change effort. It is based on "hard factors" as the authors describe them rather than the fluffier stuff you normally see in change models. More importantly the metric is derived from a statistically significant study of 225 change programmes and has been shown to be useful in 1000 or more since.

So why am I waffling on about the DICE model? I think it is useful for looking at the adoption of Agile and the change process associated with it - particularly because one of the hard-factors used in the metric is the "Duration" or rate of review of progress. Since a major behaviour in all Agile approaches is Small Iterations and regular reviews associated with them, you may start to see my point?

The DICE model consists of a score generated from a metric in 5 variables:

Duration - the amount of time between reviews (or duration of change program)
Integrity - the ability of the team to deliver what is required
Commitment - two variables here, C1=commitment to change of top management, and C2=commitment to change of employees effected by change
Effort = the perceived effort over and above normal workload that the change initiative demands

These variables are scored in a 1 to 4 point range and then plugged into a formula:

DICE = D + 2I + 2C1 + C2 + E

The score is then correlated on a distribution graph which shows you if the change effort is in a Win, Worry or Woe state (management consultants do like their alliteration!)

So what does this tell us about Agile Adoption? Given that the best score of 1pt in Duration is given for review points that are at least 2 months apart, you may see that Agile is already on a winning roll! In fact, I would suggest 30-day Scrum Sprints and XP iterations might be allowed a cheeky 0.5 pt score, but its not my model and that may be cheating.

So what of the other factors?

I asked about 20 people I know who are Agile people about what they thought teams perceived the effort of beginning to adopt Agile was. After getting through the "it depends" qualifications, I found that there was a average coming out of about 35-45% extra perceived effort! (generally all 20 said the reality was that Agile was actually less effort!). In the DICE model this is a big factor and would fall into the poor 4pt range.

The Integrity and Commitment factors are more complex, as they are so context dependent. To score low figures in the Integrity category you need good team leads and at least 50% of the team's time being spent on the project, so to me Agile projects would never score the poor 4 but would always exist between 1 and 3 pts. With commitment, neutrality will score about 2, 3 if people are being divisive.

So what does this mean? The DICE model with these figures means that Adopting Agile processes are hovering in the Win-Worry zone, but not the Woe zone. This implies if you can get a team to start adopting Agile you have a an above average chance of it succeeding compared to other approaches to improving software delivery.

But you're not out of the woods. This DICE model points at working on several factors to improve the success of Agile adoption:

Senior Management Commitment - the more clear, consistent and definite the senior management can be about why adopting Agile is important the better
Team commitment - ensure the team understands why it is worthwhile them changing their behaviour. Help them construct a "what's in it for me?" answer
Integrity - there is no substitute for a strong team lead who gets why the change is needed. Make sure there is a leader. And make sure the team members are committing more than 50% of there time to behaving in an Agile manner on the project
Effort - do not under-estimate the impact of the "perceived effort". A slow adoption approach may pay off better than an all-or-nothing. Break the perceived 40% overhead into 10% chunks - for example, adopt the Scrum practices and slowly feed in the XP practices as they are palatable to the team (but keep the vision of doing them all finally). According to the model and my numbers doing this alone would move you into the Win zone by itself

MSF for Agile Software Development - You've Got To Be Kidding, Right?

2005-12-13T11:00:00.000+00:00

Matt Davey pointed me at the MSF for Agile Software Development Beta. Damian Guy and I read through it - its a very good laugh! I would like to thank Microsoft for producing such a magnificent pre-Christmas spoof for our entertainment. I recommend it for your geek office party.

Hold on - maybe they're serious?? If so this is a real problem.

This is a terrible whitewash of the MSF product with an Agile brush for the purpose of marketing. The phrase which really rubs me the wrong way is in the Principles section of the Overview:

"This approach helps create an adaptive process that overcomes the boundary conditions of most agile software development processes while achieving the objectives set out in the vision of the project"

What the hell are you talking about? What boundary conditions? They don't define them anywhere. Is the boundary the need not to use the MSF product?

I cannot see much in here that would help you get any more Agile than you would be using normal development practices. Further a lot of the statements in here are non-sensical to an Agile ear, for example:

Under the Role>Developer>Fix A Bug>Step-By-Step it describes the process as fixing the bug THEN write the test to prove it works. Clearly the Agile practice of Test Driven didn't quite get through?
There is a section in the role>Developer>Implement A Development Task>Step-By-Step that describes doing a code review and says "Stay in the zero defect state". What? When do you not operate in a zero defect state?
The Role of Architect clearly describes a "big design up front" approach
It is very light on any Practice description - although a practice described in the "Pride of Workmanship" Mindset is "giving projects code names to clearly identify the project". Wow.
One of Damian's favourites is the diagram at the bottom of Overview>Team Model under the Scaling Down subtitle. This recommends which roles not to try to share in individuals if you scale the project down. It recommends not confusing people by combining the roles of Developer, Tester and User Experience. Errr...that's, like, completely opposite to most Agile thinking?
The Project Manager appears to be very busy estimating Scenarios, Development Tasks and Testing Tasks. Thankfully the MSF for Agile helpfully co-ordinates this into Microsoft Project so he can print out a Gannt chart for developers
The list goes on....

I thought Ken Schwaber's comments on this were very interesting, too.

This is an abuse of the Agile ideas for vending the same old tools and maintaining the same behaviorial practices. There is nothing in here about the practices you use to achieve valuing your people over your tools. There is nothing in here that would create a self-organizing complex adaptive Agile team.

If you're looking get the benefits of Agile Delivery, don't start with MSF for Agile.

They're using the words but not singing the tune.

Breakfast in NY with Ken Schwaber

2005-12-12T12:39:00.000+00:00

Finetix hosted a breakfast for about 30 senior people from NY banks who were interested in hearing about Agile in Capital Markets. Ken Schwaber came along to add further weight to the Scrum argument.

We took a reasonably Agile approach to the session: we did a basic introduction to Agile, Scrum and XP including how they dovetail and what the benefits are when building Banking systems. We then opened the floor to very interactive Q&A.

Most of the questions were not a surprise:

Can you do this if your QA is offshore? [yes, and...but...]
Do you mean do the architecture in 30days? Then the design in 30days? etc [no...]
Can you do Agile without XP? [yes, Scrum...but...]
Do you mean get the QA involved early to sort the tests out? [kind of but more...]
Can you do this with junior people, you've all done it with good people? [no, done it with all sorts of people...]

And most of the statements were challenges that were straightforward to handle:

Developers won't want to Pair Program [not 100% true, and...]
You need documentation for when the system gets taken over in 3 years time [you ain't gonna need it if you don't deliver a system PDQ]
You need to design an architecture first! [er, no you don't if...]
40-Hour Week, hah! I work that a day! [empirical data 40 useful hours...and you can do other things...]

It was great having Ken there to lend examples and data from other large, successful organisations outside Banking. Thanks, Ken.

We did something right as I ended up visiting several institutions as a result of this breakfast to further the Agile discussion in their particular contexts.

And in each place I ended up going through the traditional conversation along the lines of:

They: "That's what we did when [things went wrong]. We had [horrible deadline X] and we worked as a small team, met every day, did what it took and delivered"
Me: "Yes. Did it work?"
They: "Yes - but it nearly killed us working at that rate"
Me: "So what happened afterwards?"
They: "We went back and took time to rearchitect the system properly over [n>10] months"
Me: "Did that succeed?"
They: "Sort of - it was late. And it wasn't as good. And the users weren't happy"
Me: "So why did you change your successful way of working?"
They: "Errr"
Furrowed brows
They: "We couldn't keep working at that pace!"
Me: "I agree. You clearly know what it takes to succeed you just keep going back to old habits when the pressure is off. Why not work that way all the time, but at a reasonable pace, with reasonable deadlines?"

(Its becoming unamusing to repeat this conversation with people)

The Banking IT world knows this model of delivery probably better than any other! The hard, high pressure deadlines are often externally imposed by regulations and law. It is amazing that Financial institutions haven't embraced Agile development more.

I found there was a lot of misunderstanding about what Agile Delivery entails. Everything from fear of XP and believing its just about programming, to the "it's a license to hack" attitude. Agile seems to be a dirty word? Why I wonder? Is it the fear of change and not understanding the impacts, or is it something more?

Agile Letter to The Guardian

2005-11-24T11:52:00.000+00:00

I never really believed it was worth writing to newspapers, however, I was irritated by an article by Ed Hasted in the Technology Guardian sufficiently to respond. I got a call from Bernard Horan to tell me that they'd published an excerpt from it on the letters page!

So now I believe in writing letters to newspapers - look out Mr. Murdoch!

Pity they didn't put my contact info in, that would have been interesting...

XPDay and Buying a Beer

2005-11-21T12:05:00.000+00:00

So XPDay is nearly upon us. Well worth going to if only for the social events on Monday and Tuesday! As per my previous blog entries I am a fan of the social event as a useful practice. If you study the programme closely you will notice that Finetix is sponsoring the drinks on Monday night - clearly I've managed to convince others of my view :-)

If you're going to XPDay please do come along, have a drink and start a discussion or argument of your choice. Myself and several of my colleagues including Matt Davey and Damian Guy will hopefully be coming along. So you've got time to prepare your attack....

Light Bulb Jokes

2005-11-17T14:52:00.000+00:00

Now that Agile has hit "the big time" with a Dilbert cartoon we (some old OT/SPA cronies at lunch today) were trying to remember if we'd heard an Agile or rather XP Light Bulb Jokes. Failing to do this we decided to make one up:

How many XP developers does it take to change a light bulb?

What's the test for your use of the room? We cannot possibly estimate the amount of work or the resources required without understanding the "what". Why do you want light in the room? We can simplify the room by deleting the roof, walls and lightbulb so that you can get light at least 8 hours a day - of course you should only be doing 40 hours in there anyway. And you'll never need a new lightbulb! Which has tangible business value over the life of the system. We estimate it will take 2 developers 4 ideal days provided the roof doesn't cause injury if we refactor the floor first.

Maybe you had to be there. Any better ones?

Team Building Exercises? Bah! Humbug!

2005-11-15T17:53:00.000+00:00

As the season is soon to be upon us I feel comfortable issuing a "Bah! Humbug!" opinion on the notion of Team Building Exercises.

I find the notion of Team Building Exercises such as Go-Karting, Karaoke, Party Games, Chicken Stuffing, whatever! as pointless. I have always had this sneaking suspicion that they are merely excuses for people to have fun at the company's expense. Bah! Humbug!

Now don't get me wrong! I believe in companies investing in social events for staff - but not disguising them as Team Building Exercises. The success of The Highgate Bar underneath Connextra will atest to this attitude.

I believe these Team Building Events do nothing to build the team when its outside of the context where the team forms to solve a problem. Teams are fluid and task oriented in my Extreme view of the world. They form and dissolve as necessary. Investing time, effort and money in making the environment and place that you actaully want the teams to form and execute is a much better investment. Don't spend your money on Go Karting, get rid of the cubicles and buy a decent kitchen table so people can sit at a table to share lunch with each other instead of sitting at their desks!

Do or Do Not Do. There is no Try.

2005-11-04T11:04:00.000+00:00

Master Yoda may well be an Agile master. His sage advise came back to me again in a recent discussion with some colleagues about adopting a stand-up practice for the whole company.

Quite often I find that people try to adopt Agile practices a little at a time - in this case do it with a few people to see if it works rather than the whole team. The lets-try-it-and-see-if-it-works approach. The problem is you're not actually doing the real thing, but a prototype or a simulation. And the commitment is not there in this attitude. Furthermore the actual value is often found in doing-the-whole-thing-for-real and there is no middle ground that is informative.

Its a bit like saying your going to learn to dance but only committing your left leg to trying it.

I can appreciate the fears associated with adopting a new practice especially when transitioning to an Agile approach. However there is usually a fear of "doing the right thing" at the base of it. Is it right for us? How do you do it? What if its the wrong thing? Lets try it in the small and then if its wrong the damage will be less.

So by the dancing analogy this would amount to only committing your left leg because you're afraid that you might look silly, you don't really know how to do the dance, and you might hurt yourself if you tried the whole thing - but you're willing to risk a leg.

There is no right or wrong in practice based approaches, only more valuable or not - and you can't figure that out unless you really do the practice.

I believe the lets-do-it-and-restrospect-it-later approach is best. Do the thing which believe will give you the highest value. Commit to doing it 100%, unquestioning, non-analyzing, applying yourself, by-the-book, this-and-this-only. But agree at the start how long to do it for and when to retrospect upon the practice. So for example, agree to try the stand-up practice for a given team for four weeks and the retrospect on whether it is as valuable as you thought it might be and how you might change things to improve it. During the 4 weeks just do it. Don't discuss it, don't argue about it, don't try to optimize it. Then do a brief retrospective on it.

So this is like committing yourself to learning the dance, whole heartedly and with body and soul. But giving yourself a time limit when you decide whether your getting what you want from it.

This is one of the reasons why I often bang on about Agile being about discipline. It takes discipline to commit to something and not analyze whilst doing it, but then set aside time to reflect. This attitude in itself is a valuable part of being Agile, and especially being eXtreme.

Finetix Breakfast in NY

2005-10-28T17:24:00.000+01:00

Finetix is hosting a breakfast in New York for those of you who may be interested in the application of Agile in Capital Markets. Ken Schwaber will be talking about SCRUM and I'll be presenting on XP in Capital Markets. There will also be some brief case study's from David Lattimore-Gay's experiences.

My First Real Pairing Experience

2005-10-27T11:19:00.000+01:00

This recent blogging of "how to do good pairing" caused my first real pairing buddy to remind me about how different we were but how well it worked.

Back in the mists of 1992, I was a Smalltalk developer at Morgan Stanley where I got to work with Lynn Riehl. She was a more senior IT person than I and understood the business, I was a young techy geek who could make Smalltalk dance. Every morning for two months, I'd turn up at her desk and we would commence to develop the system sitting side-by-side in front of the machine. I think all the good pairing behaviour was there:

neither of us ever used our "authority" to push a point, we would discuss things and try to find agreement
we would talk as we did things to externalise what was going on - Lynn could hear how I was thinking about the business, I could hear how she was thinking about this new object-oriented stuff
We didn't have huge design documents or a framework in mind, we would implement specific features one at a time all the way through
the On-Site customer was the FX desk on the same floor. Lynn had great relationships with all of the traders and the head of the desk. Lynn would do regular demos for them and generate feedback and prioritised to-do lists
we would agree on an approach and stick to it until one of us would say, "hold on..."
both of our language patterns became, "I think/I feel...." rather than "generally..", "the right way...", or other such terms (actually I think this was Lynn's natural speaking pattern which I learnt to adopt)
we had periods of intense work with frequent breaks for ice-cream

I think a lot of this behaviour emerged because we were under a tight deadline (probably unreasonable deadline), neither of us had all the information or ability to make things happen, it wasn't entirely clear what the business needed or wanted but we were both determined to make things work (stubborn is the word that comes to mind).

I don't even remember how we started to get into this behavior model? I think it was probably something along the lines of Lynn getting me to show her how to write a piece of Smalltalk to do X using the frameworks Morgan Stanley had at that time. Then she tried to modify it and had some questions so I'd come back, and pretty rapidly it was just easier and more effective to sit there all the time.

We did get complaints about the noise of us pairing from one of the other developers - which is of course one of my measures of good pairing!

We did do some unusual behavior though which I'm not sure is appropriate for pairing but is worth mentioning:

Lynn would stay later than I did and be in earlier, and would go through and review what we'd done and clean a lot of it up. Sort of refactoring and tidying things up so it was ready for us to have a clean start in the morning when I finally turned up (there was an interesting incident where I got shouted at across the Morgan Stanley staff dining room, but thats another story....). Not exactly a fair balance of work
Also Lynn would deploy the latest version every night so the users always had the latest and greatest (the single person responsibility for deployment is the unusual behavior, no the regular delivery) [Lynn's memory is we didn't do it every day - maybe I just thought we did!]
although there were other resources we tended to stick together as a pair. I think Lynn did work with Bernard Horan occasionally, but I've never really asked them about if this was a pair experience or just two people working together
I discovered only recently that occasionally when I thought we were both thinking hard about how to solve a tricky technical problem Lynn was actually trying to decide what she was going to do at the weekend! Now interestingly this did not effect our good pairing experience - so I can assure you that ignorance can be a good thing. But it does make me realize I can could be duped if I wanted to be duped (or is that "can be duped"?)

Clearly, I had the easy time of it not having to deploy or do the clean-up.

But was it ultimately effective and valuable ? I believe it was, but the following facts stand:

we delivered the system incrementally, delivering every day. Bugs could be fixed within minutes, features rolled out as they became available
Lynn rolled it out to 2 currency traders as a trial, and discovered the next day that the other traders had started using it without telling us! Not only that it was working fine for them and they were very happy
2 people of very different skill sets, who didn't really know each other managed to figure out and deliver what the FX desk needed within the bounds of an impossible deadline
I learnt about the FX business inside and out - and it still stands me in good stead today
Lynn became a pretty decent Smalltalk programmer - but she jokes that working with me on that project taught her that she wasn't really geek enough to be a serious hands-on developer, and her talents lay in project management

Lynn and I are still great friends 13 years later - so contrary to the fear about falling out that my colleague raised, I think you should be more afraid of the number of close friends you'll end up with after pairing with them.

Pair Programming: Issue 4 : You're in my way!

2005-10-25T17:29:00.000+01:00

[The fourth installment of my response to a colleague's queries about Pair Programming]

Issue 4: The more senior coder can often feel impatience with the other coder, resulting in a loss of respect that the senior person might feel for the junior one

Similar response to my previous blog entry: you're not really Pairing. The most senior person can learn something from a junior, even if it is only patience. I would go as far to say that in such circumstances this "senior" person is actually an intermediate who has not yet learned the behaviours that define a senior developer (to me anyhow). These include patience, welcoming questions and admitting when you don't know.

Often mistakes are made in rushing to implement "the obvious" or "the cleverest" solutions and it is a mistake to confuse speed of development with effective development. Efficiency of code generation does not equal effective development. Having juniors who require a slower pace due to either their questioning or their need to understand the "obvious" is an advantage. This challenge to the intermediate developers helter-skelter, I-know-what-I'm-doing approach and is often the main value of Agile!!

Think of it this way: if all us senior development guys and gals are so good and know what we're doing why was it necessary for Agile to come into existence? What is the point of the discipline? Is it maybe because the development processes we grew up doing (the non-agile ones) were failing? Maybe we have bad habits that need to be trained out of? By having the juniors around we (a) can show them the better way we know is better (but might not practice yet), but maybe more importantly (b) they can keep us true to our intentions of Agility by questioning us and pacing us.

It's a way around the hypocritical "Do as I say, not as I do" maxim. Its more, "Do as I say, and help me do as I say".

Sometimes juniors get criticized for asking "stupid questions". There is no such thing as a stupid question: just ones you do and do not know the answer to. Often the frustration intermediate developers have in this context is having to explain something they've taken for granted. This comes back to the fear/embarrassment point again. Sadly many intermediate developers who think they're senior haven't yet learnt a valuable lesson: admit when you don't know something and do something about it. People will respect you for this behaviour.

Pair Programming: Issue 3 : I like you so much I won't Pair with you

2005-10-24T12:40:00.000+01:00

[The third installment of my response to a colleague's queries about Pair Programming]

Issue 3: The ill feelings that develop during PP exercises can ruin a team dynamic. Good friends who do PP can often end up as not-so-good friends

Nonsense. I have only ever seen Pairing improve the team dynamics - the issue here I think may be different: Two people sitting at the same desk is not necessarily Pair Programming.

As per my previous post the practice of Pairing is not doer-criticiser. It is a real collaboration. You can hear a real Pairing experience. However, I have seen poor Pairing leading to falling out.

People who attempt Pairing without starting with someone who's really done it before, or who do not commit to it as a different way of behaving will not gain the benefits. Like any practice you have to really do it: externalize your dialogue, concentrate on the system not the mechanism of its construction, seek to agree, state who you feel and think using "I" and don't speak for others (e.g. "this is not the right way","generally...","the rest of the team...")

Pairing is a different behaviour model to normal working patterns of two people. The "norm" in projects is either person-with-problem-and-helper or senior-guiding-junior. Hence why a lot of questions of fear-about-criticism and seniority come up in talking about Pairing. They are generally the only models people have experienced.

So who is responsible for ensuring good Pairing? Everyone! The Coach, the Manager, other Pairs, the Customer....everyone. If you hear a Pair not working you must say something. If Pairing is broken, then Agile is broken IMHO.

Pairing is different. In my experience the interactions help build teams as everyone starts to appreciate the way others work and the personal benefits of Pairing. It is challenging but it pays off.

Pair Programming: Issue 2 : Stop looking at me!

2005-10-20T15:34:00.000+01:00

[The second installment of my response to a colleague's queries about Pair Programming]

Issue 2: Many coders are nervous when someone is watching them code over their shoulder. People get embarrassed when they code something that the compiler will catch as a syntax error. People can't concentrate to their fullest.

Nervous, embarrassed, can't concentrate - these are all outcomes of the most basic of things: fear. And the fear in pair programming is exposure to another person's judgment about your performance. "What are they going to think of me?" , "Will they discover I don't know what I'm doing?" , "If I get something simple wrong they'll think I'm an idiot", blah, blah. But this is based on a model of pairing which is false, and usually held by those who have not really experienced true Pairing.

This fear comes from a model where you are programming and your partner is watching and critiquing you. In this interaction the only sounds are the tapping of the keyboard and the occasional, "that's not right", from the critic. In good pairing there are the additional sounds of commentary from the coder and uh-huh-ing from the partner. Think of it like the internal dialogue that often goes on inside your own head when you're doing something, except you're externalising it and the "other voice" really is someone else.

I go back to this "my-our" difference in the Agile attitude: its not my development of X but our development of X. Its not about you, its about us creating a system.

In good pairing the concentration is on the screen on not on the performance of the individual who happens to be in control of the keyboard. In fact because of the nature of the externalization of the dialogue there is often a merging of the way a pair approaches developing code. There is a discussion about the system not about how an individual is doing things.

In my experience the externalization of your inner dialogue coupled with concentration on the instantaneous state of the system as the subject of discussion produces a much higher level and intensity of concentration. Especially if you are practicing Test Driven and Refactoring. In fact, this is one of the reasons the 40-hour week practice exists as the intellectual burn on a pairing situation is so much higher.

Pair Programming: Issue 1: Space Violation!

2005-10-18T17:32:00.000+01:00

A colleague of mine suggested I blog something on why I am an advocate of Pair Programming (PP) - he is somewhat of a skeptic. He mentioned some unfortunate incident with an insect and an orifice.

I'm going to try and avoid reiterating the papers or data you can find on the web like http://www.pairprogramming.com/ or any of the wiki sites. Rather I'm going to make a statement, and address four concerns he raised in four separate blogs over the next few days.

Statement: I have seen Pair Programming work over and over in many circumstances and that is why I believe in it. To me it is the primary mechanism upon which successful Agile depends. It is the only execution practice which is first and foremost about people and their interactions, all the rest are technical (Stand-Ups are an organization practice to me)

Issue 1: People generally do not like to have their personal space violated, whether it be their physical space or intellectual space.
I find this a worrying view point, but not an unusual one. Many developers consider themselves to be owners of a physical or intellectual place that they need to defend against violation.

One of the big shifts in adopting an Agile stance is the attitude change from "my" to "our". Traditional: My code, my space, my goals, my tasks. Agile: Our code, our space, our goals, our tasks. And not just in words, but in actions. Pair Programming is the practice that helps most with this my-to-our change. I suppose initially it is the very violation of the "my" attitude that PP demands that causes the change in itself.

I advocate teams setting a side a set of shared machines and space to start pairing. In this way you can get around the "my" space issue and the space becomes "our" place to develop. Equally the machine used for development should be a consistent development environment that is not personalized to an individual. If you stick to discipline of PP with pairs regularly swapping partners and machines a lot of the "my way" behaviour fades away (coding styles, dev environment set up, my IMs, my email, etc).

Still having some personal space is important! 10 years ago the "cave and common" architecture was a hot topic, and I believe this is appropriate for Agile teams. This "special space" is often criticized as needing more space. However, it is possible to fit into the same space by halving the personal desk area and using the space for pairing stations.

So what about the "intellectual space" violation? Well being extreme about it, I believe it is irresponsible and unprofessional for a developer to defend an intellectual corner and expose the client to the attendant risks. I often describe the problem as "getting run over by a bus" - in the traditional defensive model a project can fail if one developer gets run over by a bus. In a Pairing project feasibly all but one of the developers could die in a bus crash and the project could still continue. Fatuous? It does happen: I was told only the other day of a real incident of a "key" developer dying during a project and it causing terrible consequences (beyond the obvious trauma).

There is a fear associated with PP about exposing your knowledge and losing control of your position through sharing. I have never seen this happen. If anything it can help strengthen your place as exposing your skills helps others appreciate them more. Further, by needing to explain your thoughts I find it deepens your understanding and cuts through the fluffiness of thought to get things over to your partner. Also, pairing is a two way street: think of it as an opportunity to gain new skills !

One way of looking at the traditional reaction to own part of the system or own part of the project intellect is that it is a way for developers to try and demonstrate their personal worth to the project and team. I saw the detriment that long-term PP had at Connextra where the value of the individual faded into the background of "the collective". Consequently we created the Gold Cards practice as way for individuals to have time to show their individual prowess. Just as there needs to be personal physical space, there needs to be personal project task space.

On a slightly different note: PP is the way in which collaborative behavior becomes visible to stakeholders. How else do you show that collaboration is really happening?

I find it curious that everyone is bothered by Pair Programming but rarely picks on Shared Code Ownership -with respect to the my/our change. I wonder if this is because they interpret Shared Code Ownership as "access to your code so I can change it" rather than it being "our code"?

What does you Recruitment Process say about you?

2005-10-17T17:52:00.000+01:00

Having recently moved jobs, I have been generally unimpressed with the recruitment processes I have encountered. My particular bug-bear is the "technical assessment" questions and my second favorite "the personality profile".

From an eXtreme point of view I fully support the "test first" approach but as ever you must be careful what you are testing for. A lot of the technical tests take a long time and are extremely detailed and demand deep knowledge of the syntax and foibles of particular systems. But is that what you want to know about a candidate? Especially if they are going to be working as a consultant ? This is testing knowledge not application, and it is application which counts. I suppose an analogy is technical tests are like coverage tests, not functional tests: and to the eXtreme mind functional rule!

The "personality profile" supposedly addresses these short comings but again I fear they are testing the wrong thing. These are analogous to architectural reviews of a system -they inform you about the general shape of things, how you believe things hang together and you decide whether you believe its fit-for-purpose. But at no time is there any kind of measure (and I use that accurately, i.e. not a metric) of fit-for-purpose.

So what do I believe is the right way of doing this interview process? What do I see as the recruitment equivalent of functional testing?

Well as a test I might write it as, "I want a candidate who can understand a users needs, use development tools, discuss implementations, do test first, etc. so that they can deliver working systems in the context of our usual working environment".

And how do you implement this test? Pair Programming.

We used this technique very effectively for years at Connextra. The interview process goes like this:

Sift through CVs for reasonable candidates (I should blog on how to advertise too...)
Brief telephone interview to test whether they are genuine and have the basic skills necessary for the role. This could include a basic technical and syntax test, but should be brief and take no more than 10 minutes and able to be asked by anyone (i.e. not necessarily requiring technical knowledge)
If they pass that (and its amazing how many don't) invite them in for a Pair Programming session of about 30-60 minutes. If they freak and say no then you probably didn't really want them anyhow (never had this happen BTW)
If they pass the Pair Programming, then get into the company fit and position discussions, which is probably only 2 or 3 more phone calls or interviews

So how do they pass the functional test of Pair Programming? Well its definitely an instinct thing, but in the eXtreme tradition the Pairing process will expose a little of everything all the way down. You will within 10 minutes figure out if the candidate knows the technology, knows how to program, knows how to interact with other stakeholders in the system, and whether they have what it takes to work in your business and deliver value.

I recommend the 30-60 minute timing so you can get 2 people to do 30 minutes each with a candidate. The second one tries to understand what happened in the first 30 minutes through the candidate reiterating and demonstrating. This also helps avoid the single chooser problem.

There is an anxiety about Pair Programming in the interview process about it being a waste of resource. I would argue that it is in fact a valuable investment for two reasons. Firstly, it is the only truly effective way of testing if a candidate is entirely suitable for what you wish them to do. By testing the right thing early it reduces the costs of fixing mistakes later - which might include your company's reputation! Secondly, it involves your current employees directly in helping to choose their future colleagues - which can in itself be a motivational exercise.

Also, remember the interviews are a two-way street, and your company will be judged on the process of interview. In fact it is the only true measure a candidate gets of you. Think: What does your recruitment process say to the candidates?

Blogging Virgin

2005-10-17T11:03:00.000+01:00

So this is the traditional, "I've never blogged before" first entry. Being an eXtreme Programming type, I of course Wiki but have never blogged. I have recently joined Finetix where blogging is encouraged, so I have been encouraged so....