eXtreme Pill

A Lean journey in Software Development - Seeing problems when and where they occur

2011-11-23T08:18:00.001-08:00

This article is the fourth of a short series, describing few aspects of one concrete attempt to follow a Lean Approach in a Agile Software Development project. Each post of the series follows the same structure: I first explain my current understanding of one concept, and then present how we applied it in the last project I was involved in.

Like in most other complex working fields, unforeseen difficulties sooner or later show up in a Software Development project, that hinder the effectiveness of your daily activities.

In the first of his QSM volumes, Jerry Weinberg has written:

It's not the event that counts, it's your reaction to the event.

Indeed, we are free to choose among a large set of options for reacting to those prickly events. Here are what I believe to be 2 extremes of the spectrum:

Do nothing: accept them as "necessary inconveniences". If absolutely necessary, design a workaround to the problem, and resume immediately to "real, productive work".
Make Problem Solving a daily activity for everyone, at every level, starting by detecting problems as early as possible in the process (when they make no or little harm, and the environment still provides contextual data for investigation).

The first option seems to represent the path of least resistance, which is why we so often find it in companies. In isolation, most problems cause a seemingly negligible additional complexity to achieve the same outcome, so people quickly get used to them, and stop paying attention. Do you see the deathly spiral? In the sacred name of 'responsiveness to business needs', we build sure foundations for mediocre performance: over time this mindset leads to clumsy, fragile environments and permanent firefighting.

At the opposite of the spectrum, the exploration of the second mindset aims at generating solid organisations, that don't fear "crisis situations". Steve Spear, respected author of "The High Velocity Edge", explains:

When you are seeing and solving problems everyday, there are no crisis per se; there are just problems that are bigger and more demanding than others.

"Everyone at every level adopts a focused, almost obsessive, attention to discovering, solving and capitalizing on problems." I think this mental model alone guarantees long term success: the organization gets results faster than its competitors because behind in each problem lies a true opportunity for improvement.

Seeing problems "where and when they occur" requires the habit of "Genshi Gembutsu" (go and see for yourself):

interview end users and see them in action as often as you can, for knowing (not assuming) what are the problems you need to work on first, and fixing early your false assumptions,
understand the actual working place, where value-adding activities take place, for grasping the working conditions and spotting (not imagining) the true obtacles to attaining your goals.

This article focuses on the second item. I highly recomment reading "The Lean Startup", of Eric Ries, for those who need guidance on the first.

Look at this picture:

Perhaps this team and their boss believe the project is on track. This may be true... or not. Perhaps the situation deteriorates a little everyday, but nobody notices the trend because the physical environment simply gives no clue. This is a typical problem in software development: one key challenge is to design the team environment to help them realize problems without effort as they arise.

In his book "The High Velocity Edge", Steven Spear emphasizes the crucial importance of seeing problems early:

(...) detect abnormality in things that one cannot see by default (...) if they go wrong and the problem is not detected, they cause trouble. If they are seen as they start to go wrong, their effects may be mitigated quickly and the organisation may learn from their occurrence.

Real World Experiments : Environments that sustain Problem Awareness

Value Stream Map from initial idea to software in production

We felt, but couldn't prove, that our backlog was exclusively filled with "urgent stories", at the expense of important subjects for the middle and long term client satisfaction.
The outcome (value) of Software development consists in solutions to actual needs and problems, faced by the target users. With that definition in mind, we decided to map on the wall the flow of value from raw idea in the head of the client to features available and used in production:

Flow from raw Needs to Development

Flow from Development to Production

The goals:

develop a shared, constant awareness of where we stand and where we go,
avoid discrepancies between client wishes and actual developments,
see immediately when an idea gets lost or poorly addressed,
notice and address bottlenecks early, before they cause any harm,
limit WIP and slowly develop a pull system

Standardized daily meetings

Informative and short daily meetings bring many concrete benefits to projects. Make them too long or disorganized, and they become a huge waste of time for everyone.

Besides timeboxing the activity to 15 minutes, we decided to describe and publish on the wall the fixed sequence of questions that would define, for us, a "good" daily meeting.

Standard Daily Meetings

This simple technique proved very useful for bringing difficulties early to everyone attention, and organize our working day accordingly: who's taking care of the broken build? Are we late? why? Let's agree on a critical working path. Is there a need for a demo or a quick design session before starting the day?

Continuous Integration : Unit, Acceptance, Performance and Platform Tests

We considered very important to be warned as soon as something that used to work suddenly stopped working. Our Continuous Integration System played perfectly that role:

- each VCS commit triggered the complete build, Unit and Acceptance Tests,

- every night the system was deployed on a Bench platform and we tested a few performance metrics,

- every hour, the different platforms were compared to a predefined standard (symbolic links, file system sructure, file permissions...)

Platform Tests

Standardized Deployment process

Several applications composed the solution under development, and deploying a release was far from a simple task. We were working hard to slowly automate the full deployment process, heading towards a push-button deployment.
Howebver, as long as deploying the application required more than 1 step, we used to print our best known procedure before each deployment, and log any anomaly as we were progressing through the operation:

Deployment Standard

After each deployment, we systematically explored the conditions of each encountered difficulty, and applied a rigorous problem-solving process to define and test a countermeasure for the next round:

Tracking Anomalies after Production deployments

Pair Programming

I've experienced pair-programming for more than 10 years in many very different projects. I am a strong proponent of this XP practice for many reasons.
Among other benefits, pair-programming helps detect and fix problems in the code as early as possible: the co-pilot reads and analyses the code as it is written, warns his mate about potential risks, raises questions and brings his perspective to produce a solid emerging design.
From my experience, this practice is much more effective at producing bug-free programs than a formal code review, executed days (or weeks) after the code has been checked in.

Conclusion
The examples provided above represent only a subset of what we have done in our context. Listing more would serve no purpose, as I believe every professional development project should customize its own system for detecting problems as they occur, for their specific value stream.
Good news: Problem Solving begins with Problem Awareness. Once your working environment triggers the attention of the right people on the right problems, the most trivial disappear by themselves, and the situation gets a little better.
I now think the next question should be: what is the thinking process of the best performing organisations for solving their problems? How can we apply it to our situation, in software development?

Waiting for the next article, I wish you a happy new year. See you in 2012!

A Lean journey in Software Development - Visualize your performance

2011-11-15T05:50:00.001-08:00

I think very highly of my mentor and friend Régis Médina. However, I couldn't help feeling a little suspicious when he enjoined me to set up indicators and measure our performance everyday.
Tedious experience with a former high level manager came back to my mind: he used to patronize the development teams, bluntly exhorting people to improve his metrics of the project. I can't remember the exact nature of those metrics and to be honest, very few of my development team cared to look at them. Why? Because we felt no connection between these measurements and our daily activity.

I'm now convinced we were wrong in our attitude. True enough, this manager failed to involve the team when designing his key metrics. But more importantly, WE failed to understand that a good scoreboard, updated daily, could help us relate our work to measurable effects, hence, direct our efforts more effectively.

Completed with a clear and shared direction, few, simple operational indicators represent the best (if not only) way to know for sure whether or not we are progressing, and by how much.

Michael Ballé (french Lean Sensei) states :

"Lean starts at the gemba, there’s no way around it."

Gemba is the actual place where the value gets created: for software projects, you typically find it at the team workplace where people interactions occur, and on the computer screens of developers, system engineers, and support functions.
When looking closely at your gemba, tons of problems should call for your attention, provided the environment is informative enough (more on this later): few, good performance indicators help direct our attention to the right, most important problems.

What characterizes a GOOD performance indicator?

Performance as seen from the client perspective (Is his/her problem solved? Quality? Delay?),
Displayed on the wall, for everyone to see,
Simple to understand,
With a visible target (the gap with actual measurement makes the problem explicit),
Updated everyday, in less than 5 minutes,
Graphed over time (to see progress and trends),
With annotations explaining peaks and valleys,
Reviewed frequently: key questions should be "where do we stand? what have we improved since the last time we met?" instead of "how much  did we make last week?"

A Lean Sensei might argue that this list of key points isn't accurate nor exhaustive... well, this is a reflection of my current knowledge, and I have to say we unfortunately have not even managed to satisfy these criteria in our past attempts.

Performance in the Outcome or in the Process?

It is crucial to clearly distinguish outcome metrics (client, or end user centered) from process metrics ("internal", meaningless to the client).

Here are a few examples of outcome metrics applicable to software development:

Lead Time from idea to working software in production
Number of Production Incidents
Right First Time newly introduced features

Let's now illustrate the "process metrics" concept to see the difference:

Volume of Work In Progress
Success of automated tests in the Continous Integration System
Code coverage with automated tests
Amount of time in Rework
Mean time from code ready to working software in production

The outcome metrics only matter in the end: properly defined, they form the true performance indicators of the project. The process metrics focus on a particular aspect in the value stream, and may or may not help to see a problem early in the value creation process (before it deteriorates into a problem for the client).

A reference point for Leading Improvement

Performance indicators should support the Plan and Check phases of high level PDCA improvement cycles. We'll explore PDCA in another article to come, but let's illustrate the last point with two imaginary problems (notice the pattern in the description, clear evidence of the underlying scientific approach).

1- Performance indicator: Lead time for solving a user request => objective : less than a day

We want to address every customer request within the day it was issued,
Our Mean Time addressing a customer demand is currently 10 days,
We have observed waste X in 12 occurences out of 15 from initial demand to final answer,
We think that waste X mainly stems from ...,
We are now experimenting action Z,
And expect the average Lead Time to drop to 4 days in 2 weeks,
Because A and B

2- Performance indicator: Number of incidents associated to production deployments => objective : 0

We want ZERO defects discovered after production deployments,
Our average number of quality defects per deployment is 2 over the last 2 months,
We have observed ...,
We are experimenting action Z for a month,
And expect the mean number of defects to drop by 1,
Because A and B

In both cases, the indicators reveal the problem (a gap between a desired performance and an actual outcome) AND are used to make predictions about the effect of the countermeasure in the future (objective: test the accuracy of our current understanding, and grow our knowledge in case we were wrong).

Real World Experiment : Performance Indicators for a Software Project

In the context of our large legacy project, we had to refocus on producing quality, and stop the devastating pattern of regressions and rollbacks following production deployments. We thus set up 2 indicators, targeted at these 2 aspects (I unfortunately have lost the photo for the second):

Ratio of Rollbacks in Production

Number of Production Incidents

We also set up an indicator on the accuracy of our estimates: at the end of each sprint (2 week iteration), we graphed the percentage of finished work compared to what we engaged at the beginning of the sprint.

Sprint reliability in terms of stories engaged

Retrospectively, I can say we largely missed the point with those indicators: we looked at them as a report of our past performance, not for testing PDCA experiment and driving improvement.
I believe the lack of momentum around them was partly caused by the fact that they got updated only once every 2 weeks, and have too rarely been the primary focus of attention in our team retrospectives (mea culpa).

On top of that, I am only now slowly realizing that we may have missed the most important indicator of all, directly inspired from my current reading, the excellent book of Eric Ries, "The Lean Startup": our speed (or cyle time) for making experiments and testing predictions toward total client satisfaction. As Eric states in his book:

Success is not delivering features; success is learning how to solve the customer's problem.

Well... albeit incomplete, our measurements looked like a random suite of numbers. Instead of questioning again the pertinence of those performance indicators, we decided to understand the current working conditions and stabilize the process outcome by tackling concrete problems at the gemba.

A Lean journey in Software Development: first, be clear about the direction

2011-10-22T16:53:00.000-07:00

This article is the second of a short series on the definition of Lean and its application to the Software Development field.

The next 3 or 4 posts will follow the same structure: I first explain my current understanding of the concept, and then present what we did in the last project I was involved in.

Short disclaimer: I may be making a huge mistake trying to catch the Lean Approach with bullet-points, but I can't resist the temptation of such a convenient way for me to articulate what I learned in distinct articles.

As far as I understand Lean Thinking today, if you (individual, team, organization) want to consistently outperform the competition in the long term for any complex field, you should address 4 key aspects as rigorously as you can (if possible with the support of a coach):

Define your 'True North', the long-term direction you will commit to pursue for years,
Visualize everyday your actual Performance and its evolution over time,
Identify clearly what problem(s) you need to work on now,
Progress by applying the Scientific Method for Problem Solving.

Define your 'True North': what for? How?

I attended last week to the 1st European Lean IT Summit in Paris. Gee! My little brain really enjoyed those 2 days!

Dan Jones (co-author of 'The Machine that changed the World' with James Womack, and top-notch thought Leader of the Lean community) shared with us his definition of 'Lean':

"Lean is the Practice of using the Scientific Method to solve Business Problems in order to create Value"

Wow! There is absolutely no waste in this sentence. Almost every word deserves a careful attention.

In particular, Dan emphasized that the 'Value' had to be considered along 3 axis:

for the customers: help them solve problems in their lives,
for the people experimenting the process: they become experts in their fields and grow everyday their problem solving skills,
for the organization to grow and prosper.

This may at first look obvious, but have you met any business organization truly value-oriented, considering this entire definition? I have not.

Anyway, we certainly need to find words more tangible than "Value Creation" for making a good Vision statement for a project, a product or a company.

What about following these guidelines?

The 'True North' must state in very few words the ideal success of your project in the middle or long term, as you see it today,
It will serve as the direction for ALL your efforts,
As a direction it does NOT have to be precise, so avoid delving into much details, or spending months to tailor it,
Your activity ultimately must bring value to someone : start by identifying your client(s), and (if possible) ask him to articulate your conditions of success,
The statement needs to combine two qualities : it has to be CLEAR to everyone, and SHARED by everyone involved in the value chain.

Aha! Interesting, isn't it? Let's now examine how we attempted to apply these guidelines to a concrete case in the Software Development field.

Real World Experiment : defining a True North for a Software product

For you to understand certain of our choices, get a look to this quick overview of our context when I joined the company 2 years ago:

The product: public video portal (live and on-demand) for mobile devices,
1 unique customer: the historical and largest french Telecom Company (our customer is not the end user of the product),
over 1 million unique visitors each month, growing, with peaks of traffic during sport events,
heavy-weight project generating revenue, and critical for the short and middle-term cashflow of the company,
6-year old project, with loads of features dumped on top of a 3-month prototype,
ecosystem of 15 applications (front apps, Web interfaces and batches, taking care of provisioning, content management, customer support, statistics...),
450 thousands lines of code, mostly in Java,
many layers of technical debt (unused features, complicated code, different technologies for the same needs),
70 production servers,
about 15 people directly involved: 4 system engineers, 1 functional expert, 8 developers, a network expert and the project manager,
several critical incidents in the past months, resulting in Low Quality of Service (slow response time) and potential loss of statistics

This of course depicts a very incomplete presentation of the situation. I could add more to it, but prefer to stop here to focus on the core of our current subject.

The development team created a first version of our True North, and asked a feedback from 2 client representatives, that led to a few minor changes.

We identified our client to be the 2 distinct departments we were dealing with on a regular basis:

one was affiliated to the Marketing department, and expressed the needs for the next-generation capabilities of the product,
the other focused exclusively at the quality of the overall system running in production (our part formed a portion of a more global solution)

Although we found many reasons to object to this bicephalous mode of organisation, we had no choice but to adapt.

We then attempted to state what problem we had to solve, from the client perspective: "Do what I want, when and where I want it, be reliable and do not make me waste my time".

We published the result on the wall, for everyone to see and remember:

The team forged a few key performance metrics out of it, and accepted it for a while.

After about 8 months, we noticed however the low level of energy this vision statement created in the organisation. We formulated the hypothesis this was due to the poor clarity of the message (post-its scattered all over the place), and the fact that it was not formulated by the client himself.

We want our client to be totally satisfied. We thus directly questioned the client:

"What are our conditions of success? What will make you say, one year from now, 'I am happy at what you have accomplished'?"

We then tried to capture the core vision in very few key points to make them stick in our memories (we published underneath goals more specific to each client division).

We found this last version more satisfying (even though less pretty):

We could proudly say to anyone getting into our room:

"Here is our direction, the Graal we are after"

I remember situations where this post-it directly helped deciding between several choices for action:

"This idea may cost a little more than this other one... but is it the right question, really? Look at our direction (pointing at the wall): which option do you think will provide the best user experience?"

More pratically, we used this clear vision statement to determine how to measure our performance in a less ambiguous manner... but this story is for the next post!

A Lean mindset for software development: why does it matter?

2011-10-04T06:01:00.000-07:00

This article is the first in a short series on the definition of Lean and its application to the Software Development field.

I often hear people say: "Software Development is a craft"... well sure, I agree with that.

Each good developer requires many years of experience to develop his/her craftsmanship in forging and maintaining high-quality, long-lasting code. She/he needs to learn from training, books, seminars, conferences, but mostly by doing day after day, advised by more experienced mentors... (and of course making mistakes)

Yet, projects in Software Development share common characteristics with other types of projects:

1- structurally, they are defined by the presence of stakeholders, a team, limited financial means and goals to be attained in a finite amount of time,

2- dynamically, they all follow (consciously or not) some kind of working process, and encounter unforeseen obstacles along their way.

The Software field is still very young compared to other businesses.

This is I think why any serious software organisation looking for more-than-average results must look at what top performers in other fields do to stay well ahead of all their competitors.

Some recent studies demonstrate that there concretely exists a common mindset and type of behaviour in the face of difficulties encountered by all those organisations: read "The High Velocity Edge" (Steve Spear) for thorough examples applying to several, very different domains.

Let's review a few common false assumptions before moving on.

Lean IS NOT :

one more trick or control tool invented by managers to exploit workers,
about cost reduction and layoffs (as some want to make believe),
the rigid application of standards defined by some Quality Department,
Kanban (but, Kanban may sometimes be used as a tool, following Lean Thinking under a certain set of circumstances)

Where does the term "Lean" come from?

The expression "Lean Manufacturing" was first used in 1990 by James Womack, in a book called "The machine that changed the world", that describes the history of the automobile industry and in particular the very successful Toyota TPS (Toyota Production System). According to Wikipedia, Lean Production is "a practice that considers the expenditure of resources for any goal other than the creation of value for the end customer to be wasteful, and thus a target for elimination".

How do I define Lean today?

I've been thinking about this for the 2 past years, reading and experimenting in the context of a heavy project, with a mix of both very interesting and poor results.

In the meantime, I applied last year (2010) to a 6 day training course called "Lean IT Project Management", broken down into 3 chunks of 2 days for 3 consecutive months. The trainer was my former XP mentor, Régis Médina. This course offered me great insight on the core of "Lean Thinking", and ways to apply it to the IT world.

I'm definitely not a Lean expert, but actively want to undestand its nature. I personally prefer to talk about "Lean Thinking", or the "Lean Mindset".

First, let's be clear about one thing: true Lean Thinking is about developing the true asset of the organisation: its people. If you can't feel this quality in a company, then it is definitely NOT a Lean organisation (in spite of anything they claim or say).

From that assumption, I would define "Lean Thinking" as a rigorous, generic, adaptable and repeatable thinking process for targeting operational excellence and maximizing the value (thus, the satisfaction) of the end customer in any complex organisation.

What does it mean for a software development project?

From the former section, I hope you understand why the "Lean mindset" does matter.

In the next article, I'll be more specific about what it meant for my past experience in a large development and maintenance project for the Telecom industry, starting by what should be the very first step : defining the "True North" of the project team.

--------------

References :

I recommend you the reading of those books on the subject, as they have profoundly affected the way I think and work, all for the better :

The Gold Mine, by Mickael Ballé
The Lean Manager, by Mickael Ballé
Toyota Kata, by Mike Rothers
The High Velocity Edge, by Steve Spears

You can also find many interesting information online. I'll just mention 2 resources I find just great:

10 years of Agile... and?

2011-01-22T17:12:00.001-08:00

This year, the Agile 2011 conference will be held at Salt Lake City. A very symbolic choice for a symbolic year: the Agile manifesto got signed exactly 10 years ago very close by.

For this occasion, I’m thrilled and honored to having accepted Esther Derby’s invitation, and become member of the review team for the stage "Collaboration, Culture and Teams".

2011 means much to me too, as I discovered XP at the beginning of 2001 (exactly 10 years ago, what a coincidence!) in a large telecom manufacturer... The project was one of the very first Agile projects in France (second or third, I don’t know for sure).

Let’s try to quickly recap my last 10 years as an Agile practitioner:

2001, year 1 : a “short” rewrite project in C++ with predefined, fixed requirements (no user interface, but an old and mature communication protocol with other subsystems). I joined a set of 3 talented freelance developers, now friends for a long time. A frank success: we released the ultimate version of this pilot project on time, and with extremely few defects.
Years 2 to 6 : a much bigger project for the same client (telecom company) with plenty of uncertainty along the way. Our incremental and adaptive approach allowed for an organic growth from scratch to a solid (“state-of-the-art”, would even write a customer, important telecom carrier) configuration tool for large network carriers. We all agree, none of us would have been able to predict and design ahead of time what we ended with. After 1 year, a natural split emerged in two subprojects (a kernel & GUI team that I was part of, and a plugin team for domain checks and wizards). After 3 years, the product met such success that the company assembled several other plugin development teams around the globe (US, Russia, China) for other domains, and a multi-user, collaborative version got released few months before I left for new adventures.
Years 7 to 9 : an Agile offshore experience (team of 3 in France and 4 in India) as an XP coach, with a high turnover in the first year. Aside from showing the way of the engineering practices and driving the XP rituals (stand-ups, iteration planning games and retrospectives), my assignment consisted in giving rise to a genuine team spirit in a team divided by over 7000 kms of lands and seas. Concretely, we pair-coached for 3 years with the project manager to make it real (I often wore the technical lead hat), and we all traveled several times to physically get to know and trust each other.
Year 10 : management through a mix of Agile and Lean approaches of a large project flooded with quality problems inherited from history. The development team was familiar with Scrum and XP engineering practices, which allowed us to experiment a (limited) Lean journey for Problem Solving on top of Agile, and progress steadily on the path of excellence.

This has been a truly exciting human adventure so far, thanks to the numerous great people I had the chance to work with... I am learning very much every passing year and realizing with a mix of fear and pleasure that the journey will never end.

What about the software community as a whole? Where does stand the Agile movement after 10 years?

Some Agile proponents publicly say, “Agile has been widely accepted as a mainstream way for building software ; the success of any project is only a matter of implementing correctly its tried and tested techniques!”

Paradoxically, I think that the stakes have never been so high for the Agile movement.

First, the vast majority of so-called agilists deliberately ignore the roots and motivations under the practices: any fellow having built a product backlog, worked iteratively and attended a retrospective will consider himself an expert, and ruin the image of Agile by his blathering. Not true? Just look and listen around you... “Agile” has become a revered buzzword, the temptation is just too high for people seeking quick money.

To XP core values of Communication, Simplicity, Feedback, Courage, Respect... we need to add Humility today: the good practitioner needs to keep the beginner’s mind, in the sense of “Shoshin” (http://en.wikipedia.org/wiki/Shoshin).

Motivated persons might devour many books on any subject, but that will not make them proficient or experts... Practice only does. Getting involved in real projects for years, testing, coding, refactoring, pairing... and the most important, making mistakes.

I don’t know for the rest of the world, but I believe there are scarcely anybody (a handful at most) that I would call an Agile expert in France. I’ve never claimed to be one, notwithstanding my double-digit experience.

The second source of fragility comes from inside the movement.

For the past three years, a dangerous rift have been threatening the Agile movement unity: in response to the neat predominance of Scrum over the other Agile flavours, Bob Martin has launched and popularized the “Software Craftmanship” movement.

The two approaches basically differ in what they consider should be the primary focus of attention for a software organisation:

Organisation and communication (soft skills) for Scrum
Coding as a craft (technical skills) for Software Craftmanship

This creeping internal conflict is now discussed openly on many top-notch blogs (Dan North, Robert Martin, and Martin Fowler among others). See Martin’s article for a summary at http://martinfowler.com/bliki/CraftmanshipAndTheCrevasse.html.

How sad Scrum has prevailed over XP. Aside from this certification folly, I think the system advocated by XP more powerful than Scrum or Software Craftmanship, as it devises a synergetic work on both the two aspects.

Finally, Agile, Scrum, XP, Software Craftmanship, Lean... who cares? These are mere labels. We professional software developers are employed to help our clients and users by delivering the best possible tool for their specific, real-life needs, on time. That only matters. The rest is crap.

And happy birthday, sir Agile!

Increase the odds of a lasting, healthy software project

2010-05-16T08:39:00.000-07:00

Standard project life today: same as 15 years ago... with iterations

I have too often witnessed (and heard about) the same dramatic pattern of behavior over the course of software development projects:

Startup: Zeal, pleasure, exploratory thinking, excitement and optimism in the first few months, typically lead by a very small team of focused people with a pioneer mentality. They get acquainted with the domain, form a shared vision of the project with stakeholders, produce a product backlog of user stories to attain that goal, and happily engage in the battle (after optionally having spiked few technical solutions),
Expansion: the team expands (often too much and/or too quickly), in order to increase its “story production capacity”. The shared vision and momentum from the first months begin to falter. Yet, most team members still feel motivated and involved in what appears to be a great adventure. Code base grows at light speed.
Cruise control: The team less and less considers the long term aspects, and enters in a routine mode, continuously stacking up features, patching here and there the old design to make it fit, without questioning it in depth. In this context, project stakeholders consider the project mature and focus on team velocity. The project’s no longer appealing to innovators, that flee towards more juicy, creative horizons. Technical and functional debts become the norm, but there is very little time and money to address them: let’s be serious, clients pay for getting features now, or they will leave for competition, right?
Great Depression: The project has become a huge, fat mammoth, deadly wounded with numerous patches, obsolete code, and technical debts. Adding features turns difficult as heck, and stakeholders just don’t understand why. The team, almost exclusively composed of recent members now, suffers the consequences of past lousy project management. They ask the pace of demands to slow down for some time in order to cut the fat off the code. However, who are these guys anyway? They get slower and slower every month, and would like to slow down even more? They must be either lazy, incompetent, or, more likely, dumb idealists striving for perfection everywhere... Let’s push for quick and dirty for the moment ; we’ll invest time in beauty later, promised!
Collapse: Adding a very basic feature takes a month or two. The management decides we need to redesign from scratch, or the client stop to pay altogether and go for competition. The project slowly dies, a ghost team manages the transition to the new system with a mixed feeling of bitterness, rage and lack of interest.

Does it look even vaguely familiar to your experience? Of course, this “tale” depicts a very simplistic and tragic view of the reality. Yet, some projects succeed! How come? With courage and support from management, there is room for action in steps 2, 3 and even 4.

My three cents of advice

If I had only 3 meta-advices to give for software projects, these would be today:

1/ Alignment first :

Teamates strive to be aligned at all times on standards and processes to move forward and respond to change.
Very small teams (from 4 to 6 members) achieve this with very little effort. By my experience, the difficulty is simply exponential above 8 members.
However remember that alignment applies outside the team boundaries as well :

build a ubiquitous language from the code to the user stories
manage stakeholders expectations, to build trust (read the instructive book from Naomi Karten on the subject, or at least this introductory article)
maintain and update a shared vision, a common understanding of where you stand and where you would like to be a year or two from now (this kind of product ownership develops a genuine motivation for everybody as a side effect).

2/ Expertise within:

Nobody should be indispensable, I agree with that. Yet, if the project’s technically complex, a healthy team needs at least 2 proficient or expert members full time (see Dreyfus model on Wikipedia for more details on this skill distribution model).
On living (features under development) projects in Cruise Control phase, the day-to-day activities may require less innovation. However, this certainly does not mean the coding activity has become trivial ; you always need technical leaders on your team, attached to high quality built-into the code, and that other members will trust. Their resignation could represent the visible sign of the future collapse, unless you make sure the best elements in your team are more than just “competent”.

3/ Beginner’s mind and Continuous Exploration:

Extend (or revive) the high energy momentum of step 1, by encouraging focused exploration activities at every iterations (functional, technical or organisational, decided in iteration retrospectives).

Let me share with you this piece of Zen wisdom from a blog (http://zenhabits.net/how-to-live-life-to-the-max-with-beginners-mind/): “Zen philosophy encourages us to keep our beginner’s mind. What does this mean? Beginner's Mind is about using the spirit of enquiry – without getting stuck in preconceived ideas.”

Invest part-time on learning, broaden and stretch your mind beyond the mainstream thinking, listen to your intuition, experiment new techniques and share with your peers. Reject those silver-bullet dogmas said to be applicable everywhere.

Hey, what about the current mainstream Agile dogmas then?
I reckon Agile may not be applicable everywhere. Values and principles advocated by Agile methodologies will help only if applied matter-of-factly by people who deeply understand their underlying reasons and motivations. Same thing for Lean thinking process: start your journey with a Lean coach that also happens to know intimately what software development is all about (this is a rare gem today).

Exploration requires a large enough Time Horizon

Investing on middle and long-term future in a context of strong market pressure... how many organisations dare to do that? Aren’t they aware of the risks?

In his 4th volume of the QSM series, “Anticipating Changes”, Jerry Weinberg introduces the concept of Time Horizon: “Any change seems more expensive than the status quo, and it is, if you look on a short enough time horizon”.

Striving to convince managers to invest in costly activities that (you think) will guarantee a high ROI potential in the long run makes little sense, if you don’t first make sure they value this Time Horizon... the bad news is that they often don’t so much, or giving in because of too much market pressure.

I read from a blog post (http://tinyurl.com/252r4wx) : “At the operational level one has a time horizon of days and weeks, while at the strategic level one has a time horizon of several years”

Well, I believe any responsible, competent, developer should work at both levels. Stakeholders seldom understand this, which makes our job so terribly interesting and difficult at the same time!

Indeed, it is our responsibility, as developers, to develop a strategic attention for the project inner quality. The growing, insidious decay of code over time is completely invisible to stakeholders (unlike in manufacturing, where any external attentive eyes can notice the inventory stacks, the mess on the floor impeding movements or the bottleneck machine on the production line...).

A team without this level of attention can develop relatively fast for months, in some cases even a couple years, before bad effects start to show up. However, then it is often too late, or at least very expensive, to turn the system into a healthy, productive, state again. (See the story at the beginning of this post) I like to often recall my coworkers that code is both the product we ship to end users and our working environment. Are we ready to mess it up? (unless I know my project will be killed in less than a month, I’m not)

What can we do at our level?

There are several scales of actions :

- At a fine-grained level, the Refactoring automatism before every commit represents the only sustainable way to clean code, and allows best designs to emerge naturally over time through the coding activity flow itself.

- Pair-Programming and Retrospectives (after every iteration and quaterly) are also strong allies, as they offer room for tactical and strategic exchanges between peers, to build up alignment and shared vision. Retrospectives also make it easy to share problems and develop corresponding action plans as a team.

- Slack time ensures the long term success of a project. The key is to put some distance between you and your project from time to time and:

explore design ideas
take control of yet “enemy” territories (harness legacy with tests, and redesign parts)
perfect the testing DSL (invest in your production tools)
share your knowledge (dojos, presentations...)
expand your mind with new techniques, new paradigms (invest in yourself)

- Finally, develop Awareness for you and stakeholders, with visible data and charts:
Gather data evidence before irreparable damages surface, in order to effectively communicate the problem with stakeholders.
Be factual, and treat stakeholders with much respect to gain their trust (they don’t know, or don’t remember, what software development is... and what? This is our area of expertise, not theirs, right?).

Of course, those measures will not guarantee the success of your project, but I feel ready to bet they will significantly increase the odds!

Dynamics of project success and failures

2010-02-28T10:49:00.001-08:00

Let’s examine one aspect the complex nature of software projects, that lately has given me a hard time.

How do well-intentioned stakeholders sabotage technical projects?

So many large technical projects suffer from stakeholders misunderstanding of complex, underground dynamics. In period of stress, or when the project releases turns out to be late, they sometimes hastily take decisions that dramatically lowers the odds of success, although they genuinely believe they act for the good.

Imagine this common case-scenario:

A 5 year-old, pretty big, software project, with very few automated tests, and having slowly accumulated heavy technical and functional debts over the years,
Early team members have nearly all quit (current members have all less than 2 year-experience on the project),
The team is the only true asset of the project: technically experienced, they’re used to work extremely well together for several years with Agile methods,
The required investments to clean up the project are huge,
The client, (rightly) focused on the roadmap of the next functional features, does not understand the poor team velocity, and every sprint trust less and less the team abilities to respond to demand,
A vicious circle installs : the client requires to monitor and pilot the detailed team activities, with no understanding of the software development complexity... precipitating the death of the project.

I have attempted to draw a diagram of effects of this project, following roughly the notations of Jerry Weinberg (see next chapter for some theory).

Disclaimer: As many people before me have thought and written about this, it is very possible that other authors (or Jerry himself) already contributed similar diagrams.

Theory: Seeing the larger system through “Diagram of effects”

Some years ago, my friend and mentor suggested me to read a book from Peter M Senge, “The fifth discipline”, in order to learn “System Thinking”. Later on, I read further on the topic in Jerry Weinberg’s first volume of the “Quality Software Management” series. (I highly recommend these 2 books for anyone interested in explaining the dynamics behind a tough situation)

Don Gray provides an interesting first explanation for people unfamiliar with diagram of effects.

Jerry Weinberg also defines the following notations :

arrows with squares stand for open choice for action from the management
arrows with double pipes ( ----| |----> ) represent a time delay between a change and its consequence.

The concepts of delay and non-linearity may at first seem secondary when we look at or design a diagram of effects: balancing and re-inforcing systems indeed provides an interesting level of understanding, that may sometimes proves sufficient for acting wisely.

However, the combination of delays and non-linearities often explain the inability of some people to manage even simple systems :

delays hide the effects of decisions : the consequences (positive or negative) of any action occur weeks (or months) later,
non-linearities take people by surprise, by the dramatic magnitude of change (again, positive or negative) a minor change can produce

If your environment seems to run out of control, I highly recommend you take a sheet and a pen, and brainstorm for producing a diagram of effects, reflecting your system dynamics (read Senge and Weinberg excellent books for guidance). This analysis alone certainly won’t alone solve your problem, but it stands as a solid starting point before choosing your next actions.

My perspective from 2009 AYE conference

2009-12-22T15:21:00.000-08:00

Wow... what a trip! I enjoyed a truly delicious time at this year AYE conference, and offered afterward myself and my beloved wife a great (way too short) trip through the Arizona state... gorgeous!

I did attend to the full 5-day version: the “pre-conference WarmUp tutorial” on november 8, followed by the standard 3-day conference, and finally ending with the ‘post-conference full-day workshop” on november 12.

In one short sentence, I remember a friendly and fun atmosphere with plenty of focused and brain-intensive conversations (in small groups, triads, or pairs).

I finally got so intellectually exhausted that I unfortunately couldn’t take full advantage of the last day workshop... I guess the continuous english-to-french translation effort partly contributed to this result.

Everyday from 9am to 4:30pm, information was continuously flowing among hosts and attendees. The evenings then were certainly more easy-going and relaxed, but the flow never really stopped... and I definitely liked that!

Extraordinary People

This aspect had already been “advertised” on the conference wiki, and I confirm that the conference acted as a networking booster for me. I definitely thought it would be a positive side effect of this kind of event, but I quickly discovered it was actually one its core properties!

Immediately on my flight from London to Phoenix, I was very lucky to get seated next to the only other conference attendee on the plane, Leonardo Almeida. Naturally I didn’t know, until I decided to continue my progress reading “Secrets Of Consulting” from Jerry Weinberg ; my neighbour suddenly laughed, staring at my book: ”Hey! I bet we’re both heading to the same final destination! You know what? I was precisely thinking about getting my own copy of this book from my bag...”. This coincidence was just crazy! From this moment, we had long and interesting conversations about our respective experiences, concerns, hopes and ultimately our core motivations to come to this conference... This made the flight appear considerably shorter in time, and I think I have won a good brazilian friend.

At the conference itself, I then got acquainted to several other great persons, like Moto San (Japan), Udo (Germany), Tom (Netherlands), Jon (GB), Henrik and Rhaza (Sweden), Gerry (Canada), Eric, Chris, Kurt, Fiona and Marije (US)... I apologize for the persons I didn’t mention. I am very confident some of these relationships will survive the coming years.

Of course, I have to thank very much our hosts: Johanna, Steve, Don, Esther, and Jerry!We (attendees) were honored and thrilled to chat privately with you guys... I am looking forward to meeting you again soon.
Another unexpected and awesome extra was this dinner with Jerry on monday: he expressed at his first session the desire to share an evening with up to 5 conference attendees he didn’t already know, in a nearby restaurant. I eagerly jumped on this unique occasion! The subjects of conversation were varied (for a good part related to Software Engineering Management of course) ; we all knew the extraordinary author, and happily discovered Jerry, the lively person. Check out this page for more details on this dinner.

What did I get from the Sessions?

I will try to summarize hereafter what I have learned from the sessions I attended. For the most part, I won’t narrate the sessions themselves, but rather present a snippet of the information I extracted from them. It is important though to remember I didn’t suffer any long and boring presentations, but participated in activities for illustrating theories, and interactive discussions. I found it essential for all the core wisdom to sink in. Consider the following lines as a vague foretaste of the true conference content, and don’t hesitate to apply if you get the opportunity.

I already had read about a large part of the presented concepts. For these, I revised, but more importantly, the conference made me reflect upon my basics, and develop some new perspectives.

Disclaimer: this kind of conference can’t be objectively summarized in an article. On top of that, I have already forgotten plenty of details, so I will share only those that I remember at the time I am writing this post.

Day 1 - WarmUp Tutorial

The purpose of this first day was to learn or revise a vocabulary to be shared for the rest of the conference (like MBTI types, Congruence, and a few of Virgina Satir’s concepts and techniques)

The small audience (about 25 participants) helped me getting introduced gently with the event, in a very relaxed atmosphere. Steve an Don did a great job at leading the day!

Workshop n°1 - we formed triads, interview each others and each drew important aspects of his/her life (only what we felt was interesting sharing of course) on flipchart pages, and presented it to the rest of the group in a 1 minute time box

What did I notice?
- Truly International, worldwide conference
- Different backgrounds
- Mostly Agile-Oriented software people (but not all!)
- Jerry Weinberg, regularly presented as the initial motivation for attending
- Common genuine curiosity for teh science of people interactions

What surprised me?
I realized that many came not only for practical matters in their professional assignments, but more generally to improve their lifes, help reframe their day-to-day issues in a larger picture and get a deeper understanding on how to handle them.

Workshop n°2 - The Satir Interaction Model

Human communication is not reduced to the audible and visible exchanges (words, body language), far from it... although this explicit part shouldn’t be ignored, it represents only a fraction of a complex and invisible process. This lack of understanding explains why we all have experienced or witnessed people interactions going inexplicably awry. Improving the quality of our communications requires to understand better these inner dynamics.

Virginia Satir, a renowned family therapist, has developed a very powerful, yet simple, theory involving a 7 steps process : Sensory Input, Interpretation, Feelings, Feelings about the Feelings, Defenses, Rules for Commenting, Observable Outcome.

For helping non specialists like us at least remember a few most remarkable phases, Jerry Weinberg simplified the model into a 4 steps process: Intake, Meaning, Significance, and Response.

Of course, both the reality is still more complex ; 2 or more steps can overlap in time, thoughts can loop several times before finally issuing a response... The value of these theories is the same as with all thories: to help us getting a better grasp on an infinite complexity.

If you’re new to the theory or want to revise, I suggest you take a look at this nice article from Joe Rainsburger for an introduction.

Jerry has presented those concepts in many forms in several of his books (for instance in Chapter 10 of “Becoming A Technical Leader”).

Virginia Satir has also written a book dedicated to the topic: “The Satir Model: Family Therapy and Beyond”. (I haven’t read it, but it is said to be worth reading)

Workshop n°3 - MBTI Personality Type Preferences

The Myers-Briggs Type Indicator also improves our ability to communicate and understand each others. This stands as one among several models attempting to explain why people don’t have the same reactions in similar situations, and the great value this variety can represent. It is crucial to remember this tool should never be used as a means for classifying or appraising people: this would not only be unethical, but also completely senseless.

Basically, the MBTI distinguishes 4 axes for describing personality types:

1- Where do you get your Energy from? Extravert (External) / Introvert (Internal)
2- How do you extract Information from the world? Sensitive (Facts) / iNtuitive (Possibilities)
3- How do you take Decision? Feeling (Warmth) / Thinking (Logic)
4- How do you prefer to act (Lifestyle)? Judging (Organisation) / Perceiving (Adaptability)

There are numerous articles, books and online tests on the subject, so I won’t delve into details. This is really worth the discovery though, both for personal introspection and as a team building tool.

Workshop n°4 - MBTI Temperaments

The notion of ‘Temperament’ looks at MBTI personality types from a different, reduced scope, perspective:

- SJ types are called Organizers, or Duty Seekers
- SP types are called Troubleshooters, or Action Seekers
- NF types are called Catalysts, or Ideal Seekers
- NT types are called Visionaries, or Knowledge Seekers

When interacting with your manager or a difficult colleague, you should attempt to guess their temperament and then address accordingly what you deduce could be their true concerns.
More details can for instance be found on the Web, or in Weinberg’s 3rd QSM volume: “Congruent Action”.

Workshop n°5 - Temperature Reading

Temperature reading is a technique for uncovering the state of group. A facilitator leads the discovery, and asks 5 questions in the following sequence:

1- Appreciations (positive aspect of past experiences between members)
2- New Information: any news someone might want to share with the rest of the group?
3- Puzzles: anything that puzzles one or more members of the group?
4- Complaint With Recommendation: any complaint to formulate, that you have a suggestion for? (“I don’t like this. I recommend that for fixing it.”)
5- Hopes and Wishes : what would you like to have happen in the future?

If the facilitator makes sure everyone has a chance to speak up for each question, this technique provides an explicit framework for the team to openly share information and concerns and cool down emotionally, before delving in the meeting agenda itself.

I strongly believe this activity can deliver great value when applied at the beginning of every team iteration meeting (providing that the team members buy in with the idea of this kind of ritual).

Workshop n°6 - Congruence

I like to look at congruence as both a source and a consequence of self-esteem. Congruence is a fundamental asset to effectively deal with any situation. It fosters genuine and healthy human relationships.

The first challenge is to be congruent with yourself: make your thoughts, your words, your feelings and your acts all aligned. For instance if you feel angry, appear so and say so! For all sorts of reasons, we can often be very prompt at disguising or distorting our genuine feelings. This attitude invariably acts as an energy drainer... I guess as a first step out we could all try to recognize when we fall in such situations (more difficult than it looks!).

The second challenge is to be congruent with the situation. This roughly means:
- be aware of the variety of potential actions (always try hard to find 3) in front of a given situation,
- don’t get blinded by the system,
- think ‘out of the box’!

Yet another perspective of behaving congruently is to consider and balance the requirements of 3 distinct components: Self, Other and Context.

Most typical incongruent behaviors found in nature are:
- Blaming (ignoring Other)
- Placating (ignoring Self)
- Loving / Hating (ignoring Context)
- Super Reasonable (Context only)
- Irrelevant (ignoring everything!)

This is a wide subject, explored in depth in Jerry’s book “Congruent Action” (vol 3 of the QSM series).

Day 2 Morning - Managing Change (Steve Smith)

Every person, every organization has to deal with change. Virginia Satir has developed a model for helping people better handle changes in their lives. This session consisted in a game, illustrating the Satir Change Model.

The change process can consistently be decomposed into 5 steps:
- Late Status Quo (how things were before)
- Resistance to a new, foreign element (sudden external pressure, new competitor...)
- Chaos (people have accepted the need for change, but can’t organize effectively)
- Integration (people have discovered a “Transforming Idea”, that will help organize)
- New Status Quo (the transforming idea has been turned into a new working standard)

You can find more material in Steve’s excellent article on this topic.

Day 2 Afternoon - Saying No that really means No (Jerry Weinberg)

How difficult it is (at least for me) to turn down an offer, reject a proposal, or refuse to take one more “very urgent” task in charge!

Jerry alloted the afternoon for that thorny, though universal, subject. Here are some interesting quotes I extracted from that session:
When asked a question, never begin your answer by arguing. Choose between
- just tell your point,
- don’t close the door and ask for more time.
Know your bottomline BEFORE your negotiation meetings.
Don’t necessarily engage, but take it as a joke “ (grinning) Why not? All is virtually possible!”
If you’re the field expert and you know something’s not feasible, make this clear and stick to your point!
Make a rule of refusing any deal if someone pushes you (like “You have to decide NOW!”, “There’s no choice!”)
Deal directly and only with the person in charge (that is in a sufficient position of power)
Don’t set a trap for yourself (you want to say No, but to be kind you say you’re OK if the other is ready to change something...)
Check people around you to see if they can say No (ask unreasonable questions) => good test for trustworthiness

Day 3 Morning - Make the Changes you want: Finding Organizational Levers (Esther Derby)

When you begin to notice that your team or your organization develop bad habits, what strategy can you develop to produce effective countermeasures?

According to Esther, 3 main factors drive patterns* in organizations: Containers, Differences and Exchanges. (definition of pattern in this context is “repeated behavior that has meaning over time”)

Containers refer to anything that hold focus. There are 3 main categories of containers
=> physical containers, like buildings, floors, rooms, clusters of desks, cubicles, but also VCS repository, bug tracking system...
=> organizational, like functional groups (developers/DBAs/testers/marketing/support...)
=> psychological, like social groups, cultural affiliation

Differences in gender, age, proximity, inclination, sense of urgency... can all trigger behaviours and distort working relationships in human systems.

Exchanges occur between containers AND within containers, of material and immaterial goods (resources, communication, energy, knowledge, ideas...)

The technique begins with listing those 3 kinds of factors in your current environment. Then, you’re ready for identifying what actions you should take for stopping the development of the bad pattern, and replace it with a better one:
- think about which factors or combination of factors in your list is likey to reinforce the unproductive pattern, and look for missing ones,
- could you add a new container or remove another to help?
- look at the timing of the exchanges: is there something you should change here?
- ...

Day 3 Afternoon - Beyond the Org Chart Illusion (Jerry Weinberg)

Clear disctinction between “Process Description” (diagrams, that you happen from time to time to follow - if ever) and Process (actual actions performed)

This session was a time for picturing our perspective of our working organization on flipchart pages, following a Virginia Satir technique for family therapy. Weinberg guided us step by step in the process, and finally drove a group conversation on two pictures from 2 persons from the same organisation (the drawings were surprisingly very different!).

This technique pictures on a workable map how you perceive your working place. From there, you can think more comfortably about the nature and origins of your problems, notice important interactions and systems, and look for what to do next. This was fun and insightful!

Day 4 Morning - Structuring your Conversations (Johanna Rothman)

Unless you’re an ermit, you hold conversations many times in a day, with your spouse, your children, your colleagues, your managers, your clients, your friends, and even strangers on the street...
When it comes to professional life, we may have to convince, to negotiate, to brainstorm or just to gather data.

The quality of the gathered data largely depends on our ability to build a relationship through the conversation. Johanna has shared with us her strategies for consistently achieving that purpose. We explored her framework through little workshops in pairs. I won’t expand much on the explaining the full framework here.

Here are a few quotes of wisdom extracted here and there from the session:
- Physical connections (like handshakes) allows the emergence of emotional connection
- Always give your undivided attention for at least one minute, and smile (be engaging)
- Find techniques to stay present, practise active listening
- Be aware of personal boundaries
- Don’t ‘confront’ each other face to face around a table
- Look for energizing common grounds (builds up a conversation)

Day 4 Afternoon - Don’t Let a Four-Year-Old Run Your Life (Jerry Weinberg)

Jerry explored with us the intricacies of the human nature, by running a live interview with a volunteer from the assistance. I had read several times about the theory in some of his books, but I just wasn’t able to make it practical. This live session, I think, has allowed me to go past my doubts and give it a serious try on me...

Lets review quickly the theory (you’ll find lots more in his books of course):

Detect when you notice you resist something and you don’t know why. Some survival rule might be at work!
4 year-old is an age where kids begin to try to make sense out of the world.
Survival rules are old programs from this age that “haven’t been updated”: it feels almost like you’re going to die if you break them.

Here are a few examples of survival rules:
- I must always do a perfect job
- I must never ask for help
- I must be liked by everyone
- I must never be angry
- ...

Unfortunateley, such “programs” can’t simply be erased and replaced by others. Instead, Jerry (and Virginia Satir, behind the scene) suggests to just try to transform them gradually (starting by introducing some exceptions to the rule), and let time to the idea to really sink in between each step.

Let’s take one of my own strong survival rules, and apply Jerry’s transformation steps:

1/ State the rule precisely: “I must always solve problems of people around me”

2/ Change ‘must’ to ‘can’: “I can always solve problems of people around me”
Is this true? NO

3/ Change ‘always’ to ‘sometimes’: “I can sometimes solve problems of people around me”

4/ Select three or more circumstances when you can follow that guide:
“I can sometimes solve problems of people around me when:
the person has explicitly asked me for help,
I have time to do so,
I am motivated to help,
I am sufficiently qualified on the topic.”

At first sight, this progressive transformation might not look as interesting as it really is... just give it a serious try, and let me know how it worked (at worst, it should open a small breach in your rule... which should then appears a little less oppressive).

Day 5 - Collaboration Tools For Leaders (Esther Derby & Johanna Rothman)

We shared this full last day with Esther and Johanna.

Provide a focus in your meetings: hang a big sheet on the wall with the following informations:
- Outcome / Objective: What do you want to get out of this meeting?
- Agenda: Which steps/activities will we go through to attain that goal?
- Roles: Who has a special role in the meeting? (facilitator, scribe, domain expert...)
- Rules / Working Agreements: What behaviour do you want, or don’t you want, from participants during the meeting?

The Kaner’s Participatory Decision Making Diamond Model provides a very useful practical support for guiding a successful meeting progress: you can find more information here (Kaner’s book review). I think I vaguely was aware about this multi-step approach, but I began to really understand and appreciate it through this practical workshop.

We ran multiple activities from the book on a few real-life examples, from the divergent to the convergent zone, and ended up with a concrete action plan. The simulations were accompanied with numerous precious feedbacks from Esther, Johanna and the whole group.

Conclusion

Again, please don’t believe the conference can be summarized only by what you have just read in this article. I have highlighted a few theories, quotes and events particularly interesting to me, and I realize it might give a false, over-simplistic view of the conference...
Participate to the next and check by yourself!

AYE Conference 2009

2009-11-02T15:18:00.001-08:00

Why have I subscribed to the AYE conference this year?

Like Jerry Weinberg's "Becoming a Technical Leader" book describes (inspired from Virginia's Satir Change Model), I think I have attained a strong plateau in my professional abilities... I mean, I certainly continue to progress, refining some of my mental models, principles and techniques ; technically, I also try to learn other frameworks from home, but... I somehow feel the unpleasant and growing belief of a stagnation. I'm going there to be intellectually jiggled. I'm now looking for ways to aim at my next "great leap forward".

Listening to experienced people, opening my mind to different perspectives will constitute one of the keys to unlock the barriers impeding my personal 'Continuous Improvement'.

On top of it, the conference format and the idea of meeting prestigious hosts have finished to convince me to join the adventure this year.

"Amplify Your Effectiveness" - what an appealing tagline... OK, let's play the game! Here's a list of some key areas I want to be more effective at:

exchanging views with archetype Routine managers (those who look for stacks of documentation / who tend to favor blaming or ultrareasonable style of management),
pinpointing and eliminating sources of impediments for a team progress,
fostering a energetic, positive team dynamics (and making it last),
quickly recognizing system patterns in foreign environments (and jiggling judiciously),
managing projects with unmeetable schedules and/or technical debts,
reflecting on myself and my actions (to learn about me, my strengths and weaknesses).

What a program... daunting isn't it? May the conference help me to grow!

I promise to post another message in a few weeks, to describe what I have learned / liked / disliked.

Stay tuned!

On the Path to Excellence

2009-03-24T15:39:00.000-07:00

How to improve in the software field? What is the best path to excellence?

Cynics might say that our future as software engineers is doomed to an endless enslaving race, lost in advance. As a matter of fact, the wide, increasing variety of development languages, with their frameworks and libraries, can’t be mastered by a single individual. For 50 years, the hardware industry has obeyed the Moore’s law prophecy for processing speed and memory capacity... and the software technologies followed cheerfully! Will this evolution slow down one day soon? I doubt it, for the 50 years to come.

On top of it, the technical expertise (languages, frameworks, system) constitute only one portion of the software field. Numerous other disciplines have developed in parallel: system architecture, code design, ergonomics, security, testing, organisational methodologies...

State goals

The obvious response to that diversity of complex disciplines could consist in considering each discipline in total isolation, and creating silos of activities... here’s coming the good old Waterfall! Hum... I won’t delve here into the limitations of this strategy.

In XP, anyone can ideally participate and bring fresh ideas to any aspect of a project. I personally welcome field specialists only if they understand the basics of other fields, and willingly participate in other problematics with the rest of the team. A truly shared vision in a team only comes at this price.

Now, I totally feel specific field knowledge is a strong asset to a team. If a teamate feel a natural inclination for a subject related to the project, investing in her knowledge is the same as investing in a financial, secure asset.

Let’s try to explore some ways an individual can accelerate her learning pace in a discipline.

Practice

Nothing’s better than actual practice, with its necessary bunch of failures. This requires a lot of personnal commitment, patience and tenacity: a genuine experience to gain decent authority in programming can sorely be acquired in 10 years minimum, I think.

One can understand why many people may find this reality too frustrating and disheartening. A few (too many, if you ask me) hide their ignorance behind a complicated vocabulary, that sound very clever and knowledgeable. Others give up hope to get one day a leadership or expertise position in the field; they stop to learn and their skills start to devaluate slowly, but inexorably until they decide to improve again.

Please, hold on, don’t lose heart! I can attest from experience that the life of software professionals can (and should) be a lot of fun. How come? Precisely thanks to the nearly infinite depth of the field, along with its exciting youthfulness and vitality.

Training

A training program certainly provides an overview, along with some tips and tricks. Furthermore, a living expert in person (if the training is serious) can answer directly your questions.

Unfortunately, people tend to give too much credit to training courses, secretly hoping that they will be able to become specialists in a week. The sad truth is, the same people often forget 75% of the course content less than a month later, and, if they don’t practice, 95% is gone 3 months later.

Communities

Get in touch with communities through mailing lists and conferences. Networks of professionals provide a means to share opinions, or occasionally ask for advices from field experts... and it is so fortifying not to feel alone! However, don’t expect to ever become an expert this way.

Books & online tutorials

Try to learn at home from books, online resources (websites, screencasts, podcasts and so on).

The Pragmatic Programmers Andy Hunt and Dave Thomas suggest reading one book a month on average. I’m a slow reader, I read about a book every 2 months... but, even at that pace, I am sure I have been learning much more from this source than from school.

My advice: if you don’t really enjoy a book you’ve just started, put it immediately back on the shelf (or get ready for a very long, boring and useless journey!).

Mentoring

Look around in your circle of coworkers ; you may find someone more experienced than you and willing to share his knowledge. This person could even stand as your temporary mentor. In the long run, I believe mentoring to be a very powerful path to excellence, but it requires a win-win relationship to last: show your curiosity, share your thoughts, challenge ideas in a constructive manner! Be more than a basic apprentice.

In my (still short) career, I have met many interesting, smart people, and I learned a lot to their contact. These were not my mentors. I have been deeply impressed by no more than a handful out of them all. These were not my mentors either. One out of this small group has greatly influenced my thinking for almost the 10 past years. This is what I call a mentor.

Finally, I find mentoring great, but... there is a but! It can become unreasonably addictive (I know what I speak about ;) Be ready to “disengage”, gain your intellectual freedom.

Conclusion

We do not learn equally from the same sources: some prefer to experiment from examples, while others assimilate more from the theory of books. Whatever your path, hold on steadily to your goal, long enough to feel the genuine pleasure of a true improvement.

Agile going mainstream

2009-02-22T14:22:00.000-08:00

Let’s try to map the technology adoption lifecycle bell curve along a timeline for Agile methodologies:

1995-2000 - Innovators for a coherent set of values, principles and practices,
2001-2005 - Early adopters, spreading the word and crossing the chasm,
2006 up to now - Early Majority.

As Martin Fowler suggests in his recent post on Scrum, from now on, the highest risk of decay for the movement stems from the lack of understanding of the huge mass of new adopters. They tend to focus exclusively on the practices, while forgetting about what truly matters and represents a competitive advantage, namely values and principles.

The worldwide proliferation of Agile Conferences for the 10 last years constitute a sure sign that the approach has become more accessible. I highly encourage those who want to gain an accurate insight of what it is all about, to attend to one (or more) of them.

Here is, in chronological order, a list of Agile conferences I know about for this year (2009):

XP Day Swiss (Geneva, March 30, french) - If you speak french, please welcome the first edition of this event in Swiss ; I know a few organizers of this one and believe they will make it a great, practical and useful experience for the participants!
XPDay France (Paris, May 25 and 26, french) - every year since its first occurence 3 years ago, I have dutifully attended to the Paris conference ; the french community is more active than ever, and I am very happy to meet new talents from my country. In addition, the place where it will take place is truly gorgious ; once again, I just can’t miss this event!
XP2009 (Sardinia Italy, May 26-30) - The tenth International Conference on Agile Processes and eXtreme Programming in Software Engineering
Agile 2009 (Chicago USA, August 24 to 28) - This event gathers the international VIPs and some founders of the Agile movement. The far geographical distance makes it a bit expensive for us europeans, but I am considering maybe giving it a shot this year.
XPDay Benelux (likely November)
XPDay London - (likely December)

I address this post to Agile newbies that want to succeed in their projects: “Please, pay a visit to one (or two) of these conferences, to get in touch with more experienced folk, and try to understand what the essence of Agile is truly about”.

Work Smarter, not Harder

2009-02-14T01:47:00.000-08:00

Isn't it a nice tagline?

Both Agile methodologies and Lean push in that same direction : use the combined brainpower of the team to build and maintain the most efficient system in terms of value to the customer and return on investment.

In our daily work, we try as a team to work that way... and the more we try, the more we feel strong and confident (but also the more we measure how incredibly far we stand from true efficiency!).

Around us, a basic “routine” organisation (pattern 1 in the set of cultural patterns of software organisations, as described by Gerald Weinberg in his first volume of the QSM series) interact with us. Most project managers seem to value this single equation:

"people count * average individual productivity = added-value per day"

They hardly allow a hint of self-criticism to reflect on their habits. From time to time, they buy expensive tools, ask for a quality audit, or add people to the workforce, confident that it will bring the great leap forward they need in their ability to deliver fast and with high quality.

Everybody knows (or should know) there are tons of insightful litterature about the pitfalls of that approach (the oldest I know being “The Mythical Man-Month” from Fred Brooks), so I don’t feel like repeating these brillant analysis.

In this starting year, I’d rather like here to list what I believe, from my experience so far, to be fundamental rules for a “small” team for succeeding in any complex software project, independently of the adopted methodology:

Reduce the number of “simultaneous threads of activity” for a same person: our conscious brain is mono-threaded, and the cost of switching between contexts is daunting,
Allow some slack time for your team, priceless for the long run success (promotes innovation and a spirit of continuous improvement). Encourage individual studies and organize team workshops on various aspects (product vision, software design, work environment, organisation...),
Prefer developping the competence of your team before hiring external resources; then consider the latter action to help achieving the former,
Stop the dumb rat race for features - numerous studies have clearly demonstrated that most softwares are overloaded with features, that are seldom, if ever, used and a pain to maintain: rather, slow down a bit and choose judiciously the new functionalities by focusing on their alignment with a coherent product vision,
No mercy for defects: pause the current activity as soon as a defect has been detected. Fix it immediately if it should take less than a few minutes, or plan for a better (still near) time otherwise, but never forget to hold on the task flow, to analyse and look for the root cause of the occurence of this defect,

I may certainly have forgotten other important points, but I believe the respect of these define a necessary backbone of good habits for a software team.

To end this post, I will quote Robert Martin, describing what I believe to be the core discipline of all members of a “smart working” team:

“[The professional craftsman] adopts an attitude of calm, focuses on the problem to be solved, and then step by step solves that problem without rushing, without yielding to the need for speed, without surrendering to the desire for a quick conquest”.

The Art Of Conflict - phase 3: Redirection

2008-11-02T12:59:00.000-08:00

Finally, we have managed to talk! The alignment phase has brought some level of reciprocal trust back in the game.

The third and final phase of the technique consists in presenting our perspective in a way that does not threaten the quality of the dialogue, and allows the true magic to develop: the creative synergy of two minds from a positive, rational confrontation of opinions.

This quality of dialog certainly represents the only property worth to monitor. A golden rule is: be ready to come back to steps 1 and 2 as often as needed (whenever you feel the balance of the relationship starts to deteriorate).

The following list presents a few effective recipes we’ve been able to apply so far with some degree of success:

Share and understand all opinions, feeling, theories, and experiences, before bargaining
Differentiate facts from opinions: both have value, but only the latter is questionable,
Set a timeslot for a brainstorming session where you note every innovative ideas (especially rough, incomplete ones),
Stay focused: don’t get sidetracked by new issues that could arise in the middle of the conversation, but consciously choose what you need to address now,
Finish clearly: Who does What by When, set a follow-up time if needed, hold people accountable for any planned action (from the book "Crucial Conversations")

I do aknowledge that this article is very simplistic on the matter, but instead of writing a long article, that (seriously) nobody would read to the end, I instead recommend those of you who are interested to get a look at the following books on the subject, that I found truly inspiring:

Leadership and Self deception
The Anatomy of Peace
Crucial Conversations
Crucial Confrontations

And now, some quotes:

Virginia Satir

Integrity, honesty, responsibility, compassion, love-all flows easily from the person whose self-esteem is high. He feels that he matters, that the world is a better place because he is here. He has faith in his own competence. He is able to ask others for help, but he believes that he can make his own decisions and is his own best resource. Appreciating his own worth, he is ready to see and respect the worth of others. He radiates trust and hope.

Gozo Shioda (a very famous aikido sensei) sayings:

It is said that Aikido is the Way of Harmony.
I think it is simple to explain this saying.
If you face someone, and you can make that person's animosity
disappear, by your own true character,
This is the Harmony of Becoming One. This is not a compromise.
Harmony is a matter of having strength yourself, and then making the
other your ally.
He becomes your partner. This is "making harmony in opposition".
But, unless you accumulate virtue, it is impossible.
To sum up, the foundation is your inner strength.

The Art Of Conflict - phase 2: Alignment

2008-09-12T10:02:00.000-07:00

If my Centering phase has succeeded, I should now be able to show genuine curiosity and seek alignment. The tricky part is learning to lose one’s ego without losing one’s balance.

The bottom line of the alignment phases is to openly look for a balanced, win-win partnership in the dialog, and continuous improvement for both: it usually pays off thousand times the investment!

A graceful win/win resolution first requires establishing the connection to restore sanity (or harmony). While Aikido looks for a physical contact, we actually look for verbal alignment techniques.

The first level of Alignment consists in clarifying facts, and agreeing on a mutual purpose. Is this an argument on goals, or the means to achieve those goals?

“Please, could we stop arguing and think for a minute? Can we clearly state the aim of this discussion? Are we speaking about the same thing? Do we share the same goal?”

Even when this stage brings no fresh data to any party, it at least diverts the focus from a dumb opposition to a constructive thinking, which is a considerable gain.

I haven’t directly read any essays from Virginia Satir, a famous family therapist, but have learned about her work through Weinberg’s books and friends of mine. She’s described for instance a precise, and widely accepted model for a communication between two persons, in which the several transformations an idea goes through often alter the accuracy of the message, and sometimes distort its meaning. The visible part of the communication represents only a fraction of the whole message journey from the source brain to the destination brain! (I unfortunately couldn't find a diagram on the Internet, but will post it as a comment to this post as soon as I can get such a link)

Instead of making assumptions about how to interpret what the other said, keep asking questions: “Did I well understood your point? Do we put the same meaning behind this word?...” Remember this: to be a learner, you’ve got to be willing to be a fool.
To reduce the risk of a misunderstanding when I feel there could be one, I also try to repeat my point with different words.

Once certain the wording is clear and the goals are the same, we can gradually move to the sensitive subject of the conflict, starting with the facts, and exploring all possible means to attain the goal as objectively as possible.
“Let’s gather all the raw information we have got so far on this subject... maybe either you or I is missing some critical data to agree on a decision...”

Make clear that you are willing to listen to and understand the other’s perspective (of course be honest about it!).

The second level of Alignment is emotional: make it safe for your interlocutor to talk freely.

A conversation would be too simple if we only had to deal with facts, logic and interpretation problems. Under the smooth surface of rationality, each sentence often echoes as emotions on the human interlocutor, that sometimes attain her/his self-esteem.

Forgetting to take this irrational dimension into account invariably jeopardizes the success of the alignment phase.

Dialog with empathy: learn to recognize safety problems in the other’s attitude (look and listen for silence/violence, frown, signs of annoyance...).

Whenever you notice a safety problem, underline the importance of his opinion, and the sincerity of your intentions. There is a high chance that your own past behavior has been fueling the problem.
We can follow some tools from “Crucial Conversations” book to restore mutual respect. Here are the ones I particularly liked:
- Apologize when appropriate (when your words may have been inappropriately hurtful)
- Contrast to fix misunderstanding: “I didn’t mean that..., but rather...”

Whatever what happens next, I have to remember the conclusions from the centering phase, and stick to a respectful attitude of curiosity and patience. The contact has been made, and I am looking toward the same direction as him.

As soon as the partner feels respected and listened to, he should be ready to consider my opinion. This defines the last and final step for restoring an healthy dialog: Redirection.

The Art Of Conflict - phase 1: Centering

2008-06-07T11:43:00.000-07:00

Managing conflict, a science or an art?

At the 2008 XP Days conference in Paris about a month ago, Régis Médina, Antoine Contal and I co-presented an approach to conflict management inspired from the Aikido martial art and several books we read on conflict handling.

A short disclaimer first: we don’t claim to be experts in the field of conflict management, but rather apprentices, actively trying to define consistent ways of building great teams. We have attempted to gather some of the wisdom of some authors, and mixed it with our own experience. We are convinced that the capacity to dialog and resolve conflicts is precisely what will differentiate a tedious, second-rate XP team from an ultra performant XP team.

Conflict is in human nature

Deliberately ignoring this evidence would be insane, wouldn’t it? Unfortunately there are still people that tend to blissfully believe that a team works like a charm when teamates don’t argument. Actually, silences are sure signs of an artificial harmony, potentially much more harmful than a burning debate!

Conflict is generative

Accepting each others' ideas is certainly not bad by itself, but in a creation work two perspectives are seldom exactly the same right from the start ; what a shame people so often loose this wonderful opportunity for a true creativity potential to develop.
The emergence of a third, smarter solution is not a myth. I have personally witnessed it many times, especially with complex, controversial problematics. It arises from a sort of stereo, 3D vision of the problem at hand, where each participant enlightens the weaknesses of the other's suggestions and asks for direct and honest feedback on his owns.
Solutions produced by 2 brains have far fewer defects and this alone lowers the probability to revisit the code months later, trying to find the faulty instruction that caused the system to crash... who never spent hours striving to understand a context, only to finally find out that a variable was badly initialized or an overriden function did not respect its contract?

Conflict can nurture or destroy individual commitment

Some authors call it the psychological distance people have with each other and with the project. If somebody feels he has “lost a battle” with a peer, he will soon lose the initial commitment he may have had to the success of the project and the team.

Conflicting does not equal fighting

If we do not pay attention, when faced to criticism, a common, unconscious belief almost invariably guides our immediate reactions : there must be a winner and a loser at the end of this discussion. Why? Cannot we control ourselves and provide a more constructive answer to a person losing his temper?

The martial art by excellence for that is Aikido, or the Art of the Harmony of Energy. I’m not an expert, so please forgive me if the analogy sometimes turns a little rough.

I will immediately start by quoting a definition of the japanese word ‘aiki’, that introduces the analogy: "United spirit. The spiritual principle of destroying an adversary's will to fight, or the physical act of dominating an adversary by harmonizing with his force and redirecting it."
Another very important aspect: Mastery of Aiki is evolutionary. It requires several years (maybe decades) of regular training, during which the practitioner slowly evolves and passes through different degrees.

As with its physical version, we believe the verbal conflict is consistently best handled when following three phases, that we will call Centering, Alignment and Redirection.

There are many ways a conflict can be generated ; the one steming from an open, direct and straightforward attack is the focus of this article.

Centering: Start with yourself!

Control your physical instinct to fight back

Take a step back, breathe deeply, sit up on your chair...
Why not even preparing a ritual? Look for a pen and a piece of paper, start to scribble something as if you were thinking hard, while breathing as calmly as you can.

I hardly manage to be consistent on this first stage ; most of the time, the attack comes unexpectedly, in an inappropriate context (bad place, bad time...). In spite of my good will, I still often happen to fail here; this failure is commonly expressed in two possible ways : violently fight back or go to silence (and burn in the inside to say how stupid, inept and arrogant the other is). As for me, my introverted nature generally guides me to the second type of behavior.

A common explanation to this very human reaction relates to the 3 layers of the brain: when recognizing a threat, the compulsive reptilian brain immediately takes the lead, quickly followed by the lymbic brain, center of our emotions; both contribute to blur our consciousness and judgment faculties.

OK, we fail, again and again, against this same barrier... well, the wise persons we are now should remember this famous adage: failures represent our best opportunities to learn and grow.

Let’s see... what kind of knowledge could I capitalize from these? The purpose is to improve, to better handle my reaction the next time such an agression occurs. What about recognizing the way my brain and my body are changing in front of a clash?
One technic we found efficient is the personal diary: in the evening, a few hours after the clash, we describe the facts, and how my body reacts under stress (tensions in the neck, heavy breathing, lower lip biting, having butterflies in the stomach...).
This effort of analysis will help a lot in recognizing when we start losing control; the conscious mind can finally take over from the animal instinct of defense, I can breath without restrain, accomplish my ritual and so on.

Now, let’s take advantage of these precious seconds to quickly promote the intellect to captain of the ship! We can identify here the second level of centering.

Analyse the situation positively

What do I really want? To demonstrate my superiority and win this battle whatever the consequences? (Wounded relationship, ground prepared for an infinite succession of other stormy battles... how depressing, isn’t it?)

We also noticed very helpful to question our honesty and objectivity in those moments. Some filter in my mind has certainly distorted reality and obscured my vision of the situation?

Have I fallen in this condition that Arbinger Institute terms as "being in a box": the condition of being the problem and not knowing that I am the problem?In this box, we create problems and cannot see that we are often creating them; we think that others are responsible for that situation; therefore we feel justified in resisting.

I think better to quote this excellent definition: “The traits of being in a box include focusing on others' faults, being defensive, blaming, exaggerating values, feeling victimised, and exaggerating differences.”

Am I respecting the other in my thoughts, though my external behaviour? Am I considering his/her ideas, feelings, and needs as important as mine? If not, I am seeing this guy in front of me as a thing rather than as a person, and the conditions aren’t met for any constructive outcome.

“Wait a second! He/she certainly is in the box towards me as well; why should I be the one to make the first step?”, could be replied to this logic. It makes sense, but let’s look at it straight out: I do not have any other immediate way of resolving that conflict, but to start working on myself. The other’s thoughts and feeling are out of touch. I need here to remember Stephen Covey’s circle of concern and circle of influence.

Don't we say, doubt is the best quality of any scientist... none of us owns the truth!
Even if I was 100% certain my position was true, showing patience, genuine curiosity and some humility is the surest path to nurture the quality of my next interactions, my relationship as a whole and to unite my team toward the success of the project.

Let's stop here for this time; I will devote the next posts to alignment and redirection.

The difficulties of grasping complexity

2008-03-24T10:47:00.000-07:00

We humans are almost never satisfied for very long by an accomplishment, be it great or not. As soon as we manage to succeed somewhere, we quickly define new goals that challenge our imagination. Degrees certainly differ from one person to the next, but I am intimately convinced it is in our genes.

Gerald Weinberg has suggested in his writings that we face a natural law that he calls the Square Law of Computation: "Human brain capacity is more or less fixed, but software complexity grows at least as fast as the square of the size of the program".

The first response of companies is often to try and hire the smartest, most talented people they can... and I won't claim it is a mistake, but the magic soon evaporates when the ambition rises again. The law is the law!

They thus strive to find innovative ways to overcome this complexity.

All efforts we make to perfectly understand the present, guess and control the future are doomed, but it is not forbidden to build approximations models... As Weinberg (again) states, "we'll never have complete control, but neither are we victims"

We easily distinguish two families of effort:

- The scientific approaches try to build a complete theory describing a coherent system.
Some examples I think about are:

MBTI for human psychology
Newton gravity, and then Einstein relativity for astrophysics
Taylorism for car industry
Cascade for software engineering

- The pragmatic approaches keep repeatable and simple rules, that happen to give better results.

Modularity and low coupling for software design (to divide and conquer)
XP principles and practices as an approach to software projects
Lean Thinking for continuous improvement in any construction process

In the manufacturing industry, the 20th century was largely dominated by the Taylorist (and all its derivatives) thoughts. The world has changed, and a new model is proving its superiority for the 21st century: the car manufacturer Toyota is crushing the competition with its TPS and the Lean thinking... and the competitors have initiated the paradigm change to be back in the course.

The much younger software industry confusingly tries to imitate the elder-sister, with a mitigated success: the same recipes are disappointing, as the properties and constraints of software development are not exactly the same! Well, maybe we are not so far to attain our next plateau thanks to a subtle combination of XP and Lean, the second permitting to scientifically measure, control and improve the first. Unfortunately, I do not have enough hindsight and experience to go further, but hope to be able to experiment in my current project.

Special thanks to Regis, for our very interesting discussion on the topic, that inspired this post.

Personal and social boundaries, an idea of what a great team player is

2008-01-13T13:52:00.000-08:00

I really like this concept of boundary for people: this is a powerful metaphor for the individual maturity (see Stephen Covey).

When I left school and started to work as a software developer, I strongly believed that the true and only model for improving my skills was to learn from experts: "These people hold the knowledge and the wisdom of my field ; I need to listen carefully to them and record their skills into my genes as much as I can." My wishes turned into reality: I was soon very lucky to meet and work with truly smart, experienced and motivated people, whose purpose was nothing less than excellence in software development.

Noticing my interest, my new friends included me in their frequent debates and discussions. I felt there was a technical gap to fill, but it took me a few more years to realize there was a pattern of behavior common to these people I was admiring: they were never satisfied by their accomplishments, and therefore were tirelessly pushing their limits.

These guys didn't fear failures as much as the others, but instead worked often at their limits to seek a certain level of risk (technicallly and organisationally).
As Kent Beck states, "when you're afraid you'll fail, you're likely to be right", so I also tried to overcome some of my fears and move forward... this was not easy but very rewarding: I was learning much more than before out of my experiences, and was demonstrating to my colleagues a certain level of motivation.

Today, I think of it as a discipline: every year, after having reassessed my goals, I try to find a few new challenges to work on my weaknesses. I believe I am not yet very good at this game, as the risks I have taken so far are pretty low... and I expect the potential rewards to be directly proportional to the level of risk.

In most scenarios, significant improvements require a solid motivation, as it comes after experimenting a new technique or strategy that may impede our work at first, but proves to be efficient when mastered. For those interested in this subject, Jerry Weinberg exposes in his book "Becoming a Technical Leader" his theory about a "Common development cycle" defined by a succession of plateaus, separated with ravines and "great leaps forward" (look to Martin Fowler post for a little more on the subject).

This willingness to take risk in order to improve seems to be a necessary condition for achieving great results in any field... however I don't think it is always sufficient.

A strong and disciplined approach to self-improvement does not change the fact that each individual brings unique assets to a team ; the complexity of most projects has grown so much that very few people are able to tackle optimally all the aspects of a problem.

That's why I like to try and foster synergetic approaches (6th habit of Stephen Covey): exchanging complementary perspectives for building a common, deep understanding about a problem, and then bringing complementary skills for targeting efficiently all the aspects of the solution. As Kent Beck states, "two ideas about a design present an opportunity, not a problem". One risk is that it can take forever to come to decide about anything, but I think the risk is low as far as the team shares a common vision and is result-oriented.

At a personal level, this gives a psychological boundary to work on... instead of trying to force people to embrace our ideas, let's just encourage a free flow of ideas, and genuinely give others a true opportunity to influence my thoughts. This pattern of behavior often engage healthy and constructive dialogs: the other person generally feels safe and soon naturally tends to do the same with you.

Weinberg's chief assumption is that everyone wants to feel useful, to make a contribution.
Why not using that positive energy to its best? I personally witnessed many times the emergence of innovative ideas when the team coach was encouraging variety and differentiation.

Now, back to reality.
I would be really naive to claim that we should force everybody in a team to embrace these 2 patterns of behavior (self improvement, and synergetic work adept). As Regis Medina told me one week ago, "you cannot change anybody, but anybody can decide and (to some extent) manage to change".

I also had recently an interesting discussion with Conan Dalton, about hiring people (see his post on the subject). His focus is rather on looking at concrete achievements in former projects... and I agree it is essential for wisely matching a position in a project.
However, the more I experience teamwork, the more I prefer to exclusively interact and work with people that demonstrate some evidence of these qualities, even at the expense of a lack of experience, they bring a great potential and are so much more interesting!

Managing technical debts

2007-11-11T08:51:00.000-08:00

I have recently read a bunch of articles and discussions about a common pattern in software projects development: the technical debts.

Controlling the chaos: in all living structures the entropy invariably tends to develop ; one key role of the team in charge of a software project is to lower this entropy before the code becomes unmaintainable.
Indeed, a too common outcome of an overwhelming code entropy is what can easily be assimilated to a "death by suffocation": the projects abort because the teams end up spending 100% of its time and efforts fixing bugs, whereas one or two years before everybody was feeling in control and confident for a great and happy future.

Why is this so often happening? How should we fight it?
I will just try and gather here the few ideas I have selected from my readings and experience.

First key - becoming aware of the debt

The technical debts come too often unnoticed. They form sneaky underground diseases, relatively harmless at their beginning, and yet possibly mortal in the long run.
To keep the system healthy, a responsible team must learn to realize when they authorize a technical debt to get in.

Second key - measuring the usefulness of the debt

The "debt" metaphor hasnt' been chosen by accident. By contracting a technical debt, you borrow Quality in order to buy Time: for delivering faster, you decide to weaken the structure of your shiny new building. You will eventually pay it back later, using time you assume you will have in the future.

And as with financial debts, there are interests! The first, fixed-rate, interest is the maintenance work ; another one, more vicious, is the"magnet effect" provoked by this quality hole: with time, its existence serves as a implicit justification for using it elsewhere (as deeply explained by the "broken window theory"), making a future fix much more difficult to perform.

However, the technical debt is not necessarily always a bad thing. For business strategic reasons, a team can decide to allow it:
- it may be wise to be ready to pay more in the future if time-to-market is critical
- if the software is close to the end of its life in production, it makes sense to forget about technical debts that you will never have to pay back.

Third Key - reassessing regularly the debt

Technical debts are part of most projects lifetimes ; since they bring with them a cost that often grows over time, I think that a responsible team should ideally keep track of all existing debts, and build a system that calls their existence back to mind when appropriate (for instance when the next developped features impact the code closed to the debt)

Agile methodologies advocate this continous assessment, as well as the usage of a very powerful arm against the emergence of technical debts, i.e. merciless refactorings.

Fourth key - consider the debt as a risk factor for the future team velocity

Though best suited to fight this software development disease, I have learned the hard way that XP can also be endangered by it. Indeed, it is sometimes necessary to run into large refactorings for the health of the application, and this can badly impact the team velocity. The rate for adding new features varies a lot from one iteration to the next, and the team loses confidence in its ability to reliably estimate stories.
If we don't take this reality into account and communicate badly about it to the management or the stakeholders, it also becomes very awkward to continue to tell them that Agile is really that agile...
A core response is certainly the "merciless refactoring" of XP, but from experience, I find it very hard, even for an experienced team, to avoid the emergence of all technical debts that will ask for large refactorings.
That's why another, more pragmatic response, could be to lower the forecast team velocity sufficiently in advance before an actual large refactoring becomes mandatory to the developement of a feature... and define and plan a smooth and incremental strategy to conduct the refactoring.

Do you think of better/complementary ways of dealing with technical debts?

If you want to read more about it, I suggest to start with Steve Mc Connell blog post, that explores further the analogy with financial debts and categorizes the technical debts into different types.

Lean and Agile - Heads and tails of the winning coin?

2007-10-26T14:12:00.000-07:00

These last few months helped me in thinking about complementary ways for improvement in my job.

Of course, I still strongly feel the system advocated by XP is a worthwhile objective for most software teams, but it may not cover alone all the aspects of what makes a brilliant software organisation. Very smart minds, from spheres outside of the Agile community, have also thought about how to drastically reduce the many inefficiencies that we all witness in software organisations. I try not to forget that the key to continous improvement first lies in our continous open-mindedness!

For example, I don't know much about Lean yet, but the little I heard and read lately about it has convinced me to explore it further.

Let's try to define it in one short sentence (sorry for Lean purists!) : the Lean approach is fully directed toward tracking and eliminating the deep causes of the inefficiencies in a given process. First developped by Toyota decades ago in the car industry, it has since brought substantial improvements in many other fields, and now begins to spread in the software world (agile or not).

This makes me believe that the combination with XP could be awesome : genuine XP teams are constantly ready (and willing) to revise and adapt the day-to-day process to the changing reality, and Lean brings the objective tools for analysing processes and measuring the performance... isn't there a match there?

Well, let's stop this post here ; I obviously need to learn more about it before continuing on the subject. Mary Poppendieck and her husband Tom, have wrote many wonderful articles on the implementation of Lean in software, as well as several books. I wonder if this is enough to get a full picture, comprising the roots of the Lean philosophy... let's see and discover!

The Agile Alliance in the Agile movement

2007-07-30T14:03:00.000-07:00

The Agile movement is well alive today: its values have crossed the chasm and are finally reaching the "early majority".

However our work is not over. Now that the agile concepts are getting popular, the risks are not anymore of the same nature, and the health and consistency of the Agile Alliance are strong guarantees for a true success.

Please get a look to Laurent Bossavit's post, where he explains his candidacy for the Agile Alliance Board. I know him a bit, and I believe his firm intent to bring more agility into the Agile Alliance organisation itself.

Inefficiencies and large teams

2007-07-22T12:02:00.000-07:00

When is it profitable for a team to grow? Do we really need big teams to handle big projects?

I deplore the fact that so many people measure the importance of a software project by the sole size of the team working on it.

Many members certainly imply more brainpower potential, but also a greater difficulty to realize this potential. Actually, I notice two common pitfalls for crowded teams, namely miscommunication and lack of motivation.

I witnessed 7 years ago a big telecommunication company hiring hundreds people worldwide for developing the OAM (operations, administration and maintenance) system of the 3G cellular network. The project lasted for more than 7 years, during which the 3G cost center was largely off-balance. The teams communicated poorly, and the development was overall running slow (often behind schedule). Finally, the 3G business was sold to a competitor... what a waste of energy, isn't it?

At the time I had the incredible chance (though I prefer to believe this wasn't only due to luck) to integrate a small team directed by talented people, who were advocating the adoption of XP methodology ; thanks to their competence and a well-conducted presentation, they managed to convince directors to experience XP on a sub-project. We achieved it in one year and 4 team members, and the quality was clearly above expectations.

We then wrote another success story with a second sub-project, much bigger than the first one. It took us 2 years to have a useful product, and 3 more for incrementally making it really great to customers. The sense of product ownership and the freedom to adapt and refine our process allowed these years to become an exceptional experience, in spite of the heaviness of the environment (the organization reflected a huge waterfall model for 90% of the project).

I remember though that this wasn't that easy to make it happen: on the second year, the management decided that the team wasn't moving fast enough, and the team grew directly from 3 to 12 people in a matter of 3 months. The project barely survived this electroshock: the newcomers mostly didn't understand the XP values and principles (some were even opposed to it at first), and we were honestly too few and unprepared to integrate them all in such a short time.

As you can guess, the disorganized and inexperienced team was very quick to mess up the existing code design, and our self-confidence was beginning to drop dramatically... this is a lesson I will never forget: do never abruptly impose new members to a team, whatever your reasons. More generally, forget about setting the pace of your team from outside, and focus rather on helping them find and take the initiative that will optimally elevate their capacity.

The early awareness of the crisis seeds and the quick reaction of the coach fortunately saved the project ; the application was big enough to define two distinct working areas, and every member, depending on his aspirations, had to decide to either work on the application kernel (entailing HMI, core concepts and transversal functions) or on business plugins (wizards, checks and operations reflecting all the 3G business logic). Mmmhh... isn't specialization an anti-pattern of Agile and XP, might you say... well, in our case, the split resulted into two XP teams working much more efficiently and collaborating very well with each other. I strongly believe that their small sizes were crucial factors in this trend reversal.

There are other concrete examples of how people hiring can badly harm even healthy projects... and sometimes there is no such happy end to the story. Usually, the same logic applies less brutally: the project blindly welcomes new members in a continuous flow and nobody notices the trap.

For knowledge work, managers should always require the active help of the established team to determine if hiring is a good or a bad thing for the sake of the project. If you were responsible for a project, you wouldn't want to end up with an desynchronized, irresponsible and poorly motivated team, would you?

Sadly, the team size is sometime a pride for carreer-oriented managers, that are tempted to use it as a lever for granting them more power and budget. Every company should fight this tendency with no mercy, even if I feel it represnts a tough task to big corporations...

Maybe a good start could be to read the excellent book "Maverick" by Ricardo Semler, in which he is advocating the involvement of every actors in the enterprise success. To overcome the inefficiencies inherent to big corporations and inspire "cathedral builders", he suggests the "amoeba approach", frequent job rotations, rounding the pyramid... there are plenty of ideas in there. I highly recommend reading it.

Here are a few quotes from his book, that I especially liked :
- "Economies of scale exists, of course, but it is overtaken by the diseconomies of scale much sooner than people realize"
- "Bureaucracies are built by and for people who busy themselves proving they are necessary, especially when they suspect they aren't"
- "Fairness for employees is like quality for customers: it takes years to build but collapses over a single incident"

Cultivating creativity

2007-05-13T10:38:00.000-07:00

What made the greatest civilisations in human history, as well as the most successful organisations nowadays? How funny, for nearly all of them, the failure to cultivate this charateristic also caused their subsequent slow agonies and disappearances.

Creativity is what concretely distinguishes humans from other species on earth: farming, clothes, transportation, technology, litterature, arts... our imagination has been extraordinary. And we know there is still plenty of room for innovation!

Paradoxically, today culture promotes via advertising and other media the homogenization of mind and body into one acceptable norm.

It already starts at school, where poorattention is given for originality and novelty. One single response is valid to any problem, and failures are harshly penalized. Sadly, the school often miss to reveal and develop the talents of children. On this matter, I greatly recommend the talk of Sir Ken Robinson, at the excellent TED conference.

Once a young student then gets his first job, he will be expected to comply with standard procedures for working. The same logic takes place. Very few people are asked for some creativity, though it would be in the interest of both the employer and the employee.

I dream that the Agile movement will break this suicidal tendency, by focusing on collaboration between individuals and incremental iterative development. There is finally hope for a disciplined and creative approach. In the 21st century, the 'soft' nature of software makes the field an ideal candidate for experimenting a societal evolution at work.

XP in big organisations: why is it so difficult?

2007-03-27T23:38:00.000-07:00

At first glance, this problem could look straightforward: by definition, big organisation means big inertia. Can you honestly expect a mammoth to change its course as fastly as a gazelle when the surrounding environment has brutally changed?

I admit this is simple common sense, but I would like here to think a little further than this apparent simplicity. I don't want Agility to become a buzzword with no sense, but I believe there is room for big companies to improve the way they handle lightweight methodologies.

Over the last 6 years, I have been involved in 3 XP-driven projects in big organisations, and each turned out to be frank successes. Don't misread my words though: I am still totally convinced there is no silver bullet. XP adoption is not a guarantee to succeed, but merely a means to concentrate more efficiently on creating customer value.

For two of the projects, I was very lucky to witness their births (their father was Régis Médina). Conan Dalton (another agilist and good friend of mine) has told me how he initiated the third project, which I am currently working on as a coach.

Lately, I suspect the organisation I work for to act against its own interests on a few other projects: middle managers have been striving with all their might to reject the initiatives from Conan and Régis to make their projects healthier! Why that?

What makes an organisation decide to replace its traditional project management recipes, or at least give a try to something else? I have noticed that a genuine support from some part of the middle management was incredibly helpful to make the agile way a success. I now think it is a mandatory precondition.

While a (sometime harsh) resistance to novelty is expectable from people at first, there are situations in which things go only worse with time. I consider it should be our responsability to try to understand the system behind this apparent blind resistance. Can we describe a reinforcing process somewhere? Can we find a leverage for inversing the tendency and create a virtuous circle instead?

I certainly don't claim to have the whole picture in mind and give a miraculous process for getting out of this kind of trouble. However I believe there must be some generic way to handle resistant organisations. Let's try and list a first set of concrete principles:

Take the heat out of debates: it is OK to disagree, but let's be objective and focused
Chase individual fears away (power loss, layoff...)
Align personal interests with the project interests
Understand the structural and cultural barriers to change of the organisation (system thinking)
Do not reject everything: acknowledge on what is working well (strengths of the current approach)
Identify and anwer only to concrete problems (a dogmatic presentation won't make Agile convincing)

The bottomline here is that everything must be set up to establish constructive collaborations with the management and the staff ; this is the basis for any collective success.

This list of principles is of course only a first draft ; there are plenty of litterature on the subject (The Covey "7 habits", the Arbinger "Leadership & Self-Deception", or DeMarco "Peopleware" are very good references on this matter, but I am sure they are not the only ones). Go and check them out!

The Slack Power

2007-02-25T10:24:00.000-08:00

It's now been a month I have been hired for coaching a team that develops a Web Application in a bank.

Apart from the XP methodology and the Java language, most things are new to me: a new domain (I come from the telecom field), a new kind of application (I am used to heavy client Java/Swing apps) and a team scattered geographically (2 members out of 5 live in India).

Surprisingly enough, I very soon noticed that these little shortcomings would not be the most important difficulty to deal with ; the team was invariably overwhelmed with urgent matters or interruptions, and thus had no way to get on top of the project for the upcoming year.

After 1 week of observation, I gathered the team to have a wide discussion about organisational stuff, XP and the Agile movement in general.

I first told them theoretically what the Agile movement is all about, the recurrent problems it attempts to resolve, then we reviewed the Agile Manifesto and some of the methodologies that inspires it. I then explained in more details XP (values, principles and practices), and what had proved to work well in my previous assignments. Afterwards, we identified together the weaknesses and bottlenecks of the team organisation, and what were the concrete actions that could address directly these.

This meeting took a complete afternoon, meaning of course that no task and almost no support had been accomplished by that time. However, everybody in the room perceived the powerful relevance of this activity: they were eager to participate in a debate around what team organisation could produce more value for our concrete project.

This kind of activity is really what makes a team alive: it enables adaptation, innovation and responsiveness. (I strongly suggest on this subject the excellent book "Slack" from Tom DeMarco)

We will certainly spread over the next iterations a few other subjects, like building a shared vision for the product and the internal design, optimising the relation with users to better understand their needs, readapting again the process for effectiveness, organising an open 360-degree feedback session for all team members, and so on.

Of course, the team should obviously not spend days and days in endless debates! The aim is to take advantage of the brain diversity and the mutual feedbacks to improve our performance and have more fun at it.

These slack times are more generally for brainstorming (think more thoroughly, share ideas, ...), organising (logistics...) and investing (learn about promising tools or languages, refactor the code to reflect the shared design vision, ...).

Trust me, this is worth the regular investment. Enjoy!

Growing team efficiency

2007-01-26T14:49:00.002-08:00

In my previous post I was advocating the power of the "Collective Kaisen" to gradually improve a team performance.

I tend to consider a team as a kind of multi-brained living organism, because it demonstrates patterns of behavior when pursuing a goal and dealing with unforeseen difficulties. Even more, this organism is intelligent: it can learn from past experience, reflect on its behaviors, project itself in the future... as a human being.

As a matter of fact, these capabilities can be demonstrated only by strong, heathy teams. How could we strengthen a team to that level? Is there a kind of recipe?

I will try here to list some basic ingredients that I think could allow a group of average workers to change into an organized, efficient and thinking organism. I am certainly missing many others, but these are the first coming to my mind.

1/ Mission

First and foremost, we have to agree on a mission; what are our roles in the organisation? Can we decide on a set of responsabilities that will bring a maximal value with the same amount of effort? Shouldn't we ask for a separate department for basic user support? Why not brainstorming 1 or 2 hours when one feels responsabilities aren't very clear or too much scattered?
Answers to these matters will sooner or later provide some guidance in planning the daily activities. For instance: "Shouldn't we focus on delivering a more intuitive HMI instead of struggling like a mad man for documenting in depth the steps through that lousy interface?"

2/ Open Kimono

A team is almost a kind of tribe: anyone should feel secure when sharing her doubts or fears about a given feature or issue.
Everytime I discuss with teamates about a technical subject that I simply have no experience with, I immediately make it clear to them. I don't loose time hidding my weaknesses, but prefer demonstrate trust in "experts" opinions. I expect my teamates to be mature enough to then let me know when I should bring my own experience and help them on other matters.
The Open Kimono principle has first been presented in the excellent book Peopleware from Tom DeMarco and Timothy Lister.

3/ Discipline

Sadly enough, fully auto-organised teams whose members truly share a common vision and always work in synergy are often a myth, but it is undoubtely a worthy objective.
The XP coach and the manager have here a great challenge: adapting the practices to fit some immutable eccentricities of the surrounding organisation, so that the team can grow in maturity and approach that goal.
I have recently listened to an interview with David Allen (author of the best-seller GTD), where he explained that a strong discipline is first necessary for any desired behavior to become a standard habit. For example, a child does not usually want to brush his teeth every evening; after months (sometimes years) of obligations, he becomes so used to it that he doesn't even imagine letting two days without brushing his teeth! It's become a standard!
Kaisen and discipline are the keys. It is up to the manager and the coach to gradually enforce a discipline and convince people it helps being better at work!

4/ Enthusiasm

The level of commitment and enthusiasm of members determines the consistency of the team spirit.
From my past experience, I would say that the presence (or emergence) of a charismatic leader is very helpful to drive a vision, resolve never-ending conflicts, and call a lazy member back to work.
However I think this leadership has to be implicitly approved by the team : an "appointed leader" set by the organisation and not recognized by the team will certainly meet much more difficulties to change the team mindset.
Among many others, I think about two feelings every good leader should communicate to the team: optimism and enthusiasm about the work to be done and the outcome of the project. They work for a great deal in the confidence and the commitment of every single member.

Be positive! Think Big! ;o)