Agile Thinkers

The Art of Lean Software Development Interview with Curt Hibbs

2009-04-10T09:34:20+01:00

1. Hi Curt. Your new book The Art of Lean Software Development, co-written with Stephen C. Jewett and Mike Sullivan, has just been published. Can you tell us why you wrote it?

The story, at least in part, is told in the preface of the book. I was having a discussion with a colleague about Lean software development. She knew next to nothing about it and was asking a lot of questions. Finally, she asked "If I could only do one thing, what should that be?"

The answer I gave was "Automated Testing", but I couldn't stop thinking about the motivations for asking such a question in the first place. I finally realized that she really didn't want to spend a lot of time learning and understanding Lean software development, she just wanted to be told what to do.

This attitude fit perfectly with what I knew about the Dreyfus model of skill acquisition. This model states when someone acquires a skill, they pass through five levels of proficiency: Novice, Advanced Beginner, Proficient, and Expert. At each proficiency level the way that they work with that skill changes. Novices want rigid rules and prescriptive advice, while experts discard rules in favor of intuition and tacit knowledge.

Most Lean-Agile books are aimed above the level of Novice, and this presents a entry barrier for many (or at least makes it more difficult).

This book is aimed squarely at the the novice and doesn't require the reader to make a bunch of decisions for which they don't yet have the experience to handle. Instead, we lay out five concrete practices that we present roughly ordered the value they provide for the effort expended (this ordering is our subjective opinion).

I think that this target audience was not previously being served (or at least poorly served).

2. Can you tell us a little about you and your co-authors backgrounds? [Feel free to pass this on to your coauthor if you like]

All three of us work for Boeing in St. Louis.

I have been working professionally in software for 36 years. Most of that time I've been an independent consultant or part of a startup. I'm a tech nerd who has always been attracted to the cutting edge, and in 2003 I joined Boeing and started applying my entrepreneurial skills to improving the state of software development in Boeing.

Steve Jewett has is a Boeing lifer who has been with the company for 23 years. He has always been a forward thinking maverick who has pushed for change when change was needed (including being an early adopter of Agile within Boeing). That's why I asked Steve to co-author the book with me.

Mike Sullivan has a university teaching background and joined Boeing in 2005 (I think) where he immediately began introducing Agile practices into his team.

3. Who is the book's primary audience? What will they be able to do differently after reading the book?

As I said before, this book is aim squarely at the complete novice. The software developer who has been hearing the buzz about Lean and Agile software development, but know nothing about it. Its for the developer who finally decides its time to do something, but they don't know where to begin.

This book will introduce them to the major concepts, topics, and practices surrounding Lean and Agile development, but without too much depth. With this working knowledge, they should be able to dive more deeply on selected topics of interest (we provide references to allow them to do this).

The will also learn what we consider to be the five most important practices they can used to improve their software development. These practices are ordered roughly be return on effort, and can be implemented one at a time and still realize benefits (of course they are synergistic, so the benefits multiply). Once they have implemented all five practices, they will be doing full on Agile development.

The book finishes off by introducing a variety of common Lean analysis practices and some alternative approaches. This includes things like kaizen, value stream maps, root cause analysis, kanban, ToC, etc.

4. Why is the book so short?

This is my third book, and they are all short. I guess I write the kind of book that I like to read: short, to the point, without any fluff. I'm too busy for anything else!

[Click to read a sizeable google preview of the book: http://oreilly.com/catalog/9780596517311/preview.html]

Israel Gat - podcast

2009-03-23T16:47:40+00:00

I've just spent a fascinating hour talking with Agile Executive Israel Gat. Click here to download the mp3 podcast where you'll hear about what Israel is up to now, the hugely impressive agile transformation at BMC (we're talking >1000 engineers), what's going on in the agile market place, amongst other equally interesting things. Israel knows his stuff and - most importantly - he knows how to translate Agile into the language of the executive. Great stuff, if you ignore the occassional splutter where I was holding the microphone too close to my mouth ...

Israel's blog: http://theagileexecutive.com/

Israel's Agile2006 BMC presentation: http://www.slideshare.net/IsraelGat/how-bmc-is-scaling-agile-development

Here's the paper (but unless you've got membership you'll probably have to pay for it): http://www2.computer.org/portal/web/csdl/doi/10.1109/AGILE.2006.33

Israel's Cutter page http://www.cutter.com/meet-our-experts/gati.html

Clarke Ching: Rocks Into Gold - a business parable

2009-01-21T20:58:00+00:00

If you read this blog using RSS then you may not know that I, Clarke Ching, am the interviewer.
I thought you might like to know that I've just published a book called "Rocks Into Gold" which some of you might find very helpful in "selling" agile. You can buy a copy or you can read it for FREE at www.rocksintogold.com. It's a parable and it only takes about 20 minutes to read, it doesn't mention Agile at all, but it does a pretty good job of explaining the core economic/commercial benefits of agile. I wrote it when I realised that one of the biggest benefits of agile - it's cash flow advanatages - suddenly became very, very important during a credit crunch.
In the last week It's been downloaded, for free, over 2,000 times and it's got some very positive reviews too. I'm pleased with it. I wrote it because I want to give something back to the Agile community. Read it and it might save your job.

New Blog!

2009-01-19T08:01:23+00:00

Michael Cote and I joined forces to start a new blog - The Agile Execxutive: http://theagileexecutive.com/. Michael is an analyst with RedMonk - the first analyst firm built on open source. He was a key member of the team that started Scrum at BMC software in January 2005. His passion for Agile is nicely captured in the name he had chosen for his own blog - People Over Process.
Both Michael and I are of the opinion Agile methods can make a big difference during the current macro-economic crisis. We cater in this blog to the needs of the executive who realizes you can't sustainably cut costs by cutting costs. We focus on the business benefits of Agile. For example, mitigating business risk through Agile is a thread we plan to emphasize.
You don't need to be an expert in software engineering in order to enjoy this blog. Our intent is to overtime help refine your pragmatic knowledge of the benefits to be had by applying and using Agile methods. The blog will be relevant to you whether you are in Sales, Revenue Recognition, Finance, HR, Marketing, R&D, General Management or any other discipline.
We hope you will be a regular reader of The Agile Executive. And, we trust you will give us feedback that will enable us to tune the blog contents to your needs.
The best,
Israel

Cutter Articles

2008-12-23T12:37:57+00:00

Yet More from our good friend Israel:
Download your complimentary copies of these Cutter ConsortiumExecutive Updates by Israel Gat, below!
In Part I, the author explains why he considers the whole release concept a myth. Like a set of nested Russian dolls, the duality of the term "release" often masks a deeper duality. You can view the end user as the primary target of a release or you can view the release as a vehicle for market development. Philosophical as this differentiation of the release might seem, it actually brings to the forefront essential questions about the nature of software and the nature of the market.
Part II of this series suggests that software that is alive and always evolving poses unique opportunities for custom-tailoring solutions directly from Research and Development. It also describes an effective way to distribute custom-tailored solutions, and it highlights partial disintermediation that transforms the traditional software value chain.

Israel Gat - The Social Contract Revisited

2008-11-19T17:15:51+00:00

Macro-economics were very much on the mind of everyone I spoke with during last week’s Agile Development Practices conference and APLN Summit in Orlando, FL. While the specific financial concerns pertaining to Agile varied from one conference participant to another, the overarching theme was the inevitability of change. For some, it was the way in which Agile would be impacted. For others, the way Agile could impact the new economics, micro- and macro-, we all face.
In response to these concerns, I tweaked my “Leading the Agile Disruption” presentation. Specifically, I proposed the following variant of the Social Contract I had originally posted in Agile Thinkers:

“Team, my overarching organizational objective is to preserve our team and its institutional knowledge for our corporation and its customers for years to come

We will achieve this goal by enhancing our software engineering prowess to the level that the resultant benefits will outweigh the repercussions of the current financial crisis

The state of the Agile art should enable us to attain hyper-productivity

In the event that we fail to accomplish hyper-productivity and our assignments fade away, you will find the Agile skills you developed much in demand in the market

Whether you will or will not be with the company in the future, I acknowledge your need to develop professionally as an Agile practitioner and commit to invest in your education/training”

Later in the day I discussed this subject with an Agilist working for a Fortune 500 company. She was kind enough to say the proposed Social Contract went straight into the bottom of her heart. Yet she was emphatic that HR and Legal would never ever allow her (or anyone else) to propose such a “contract” to the teams with which she worked.
We discussed in some length and depth why the proposed “contract” - which is basically a gentlemen’s agreement between an Agile executive and his/her teams - would be a non-starter in her company. Both of us agreed that the five points listed in the contract need to be stated. To ignore the subject would be like denying the existence of the flu.
While I can’t say I really know this Agilist well, she undoubtedly was a smart and knowledgeable lady. I was surprised that an Agilist like she would feel hesitant about addressing the subject in her company. It was obvious she considered doing so a career limiting step.
As I felt a fairly strong cognitive dissonance during and after speaking with her,
I e-discussed the subject with a few knowledgeable colleagues and asked for a second opinion on the Social Contract. Here is a summary of the reflections I got:

Jim Highsmith, Information Architects, thinks the Social Contract actually goes beyond the current economic situation. Tenure, in Jim’s opinion, is variable. The Social Contract is for the period of time in which an employee and employer are together.

Debra Germaine, CTPartners, agrees with Jim’s observations. She views mutuality of interests as the “cement” in the employer/employee relationship. To quote Debra: “contribution to company in parallel and in sync with increasing the dollar value of the employee.”

Tim Miller, Rally Software, indicates Rally already took the proposed path. The key, according to Tim, is to create the culture within which employees are pursuing the goal of “creating your own reality.” By helping folks create their own reality Tom creates loyalty to Rally. The temporal nature of the Social Contract is very clear to Tom: “Sometimes the only progress is by transferring or moving on.”

Evangelos Simoudis, Trident Capital, suggests reinforcing the principles articulated in the Social Contract by highlighting the power of Agile as a “differentiating weapon to augment a software solution in ways that will satisfy a customer or prospect and facilitate the closing of a sale.”

Steve Greene, Salesforce.com, believes the Agile Social Contract should be framed in optimism with a nod to the reality of the current macro-economic environment. He suggests the message “now is the time for us to invest in the company's future and your future (increased skills, Agile in particular)”.

Having received the second opinions cited above I am even more perplexed by what I heard from this Fortune 500 Agilist. Call me naïve if you want, but for the life of me I do not know how to instill trust, collaboration and empowerment in Agile teams without having authentic conversations on the subject of layoffs and doing something meaningful to counter the corrosive effects of layoffs. The Social Contract proposed above is an informal antidote anyone of us could easily apply at the team/project/business unit level. I trust many of us will do so.

Israel Gat - Cutter article

2008-11-16T19:19:07+00:00

If you enjoyed Israel's articles then pop over to the Cutter website where you can download:
To Release No More or To "Release" Always: Part I -- The Myth
In this Executive Update, the author explains why he considers the whole release concept a myth. Like a set of nested Russian dolls, the duality of the term "release" often masks a deeper duality. You can view the end user as the primary target of a release or you can view the release as a vehicle for market development. Philosophical as this differentiation of the release might seem, it actually brings to the forefront essential questions about the nature of software and the nature of the market.

http://www.cutter.com/offers/releasemyth.html - use promocode RELEASEMYTH

Israel Gat - The Equipoise of Agile

2008-09-02T17:01:57+01:00

August 31, 2008

Over the past couple of years I have tried very hard to distill the BMC success in Agile to a set of best practices that I could share with anyone who is thinking of starting an Agile project. Unfortunately, after I examined the various “secret sauces” that I concocted, presented, and discussed since 2006, I had to conclude that all of them – including my recent Post, A Social Contract for Agile, in Agile Thinkers – somewhat missed the mark. Having said that, everything stated to date in my secret sauces is true, relevant, and hopefully useful to other Agile executives. What has been missing, however, is the deeper truth that binds together the many ingredients– the equipoise of Agile.

Equipoise is the equilibrium formed by offsetting conflicting forces. The equipoise of Agile is the skill of an Agile leader to orchestrate conflicting forces and pressures in and around Agile in a manner that utilizes Agile principles, without denying the realities of the non-Agile world. It is the ability to function and get teams to function amid contrast and ambivalence that are systemic.

The systemic conflicts to which I refer are the inevitable outcome of Agile being a disruptive methodology. When you apply Agile, you are trying to align two worlds that exist in parallel. In one world is the set of core competencies, practices, procedures, policies and cultural norms that made your company successful. In the other world are the origins, life, and momentum of the Agile movement. Both worlds evolve, but at a different pace and at a distance.

A good way to think about how the two worlds affect each other is to study the chapter in Japanese history that started with Commodore Perry steaming into Tokyo Bay in 1853. Bringing gunship technology in touch with the Samurai sword technology (and culture) started the difficult process of the modernization of Japan. The modernization process has been traumatic to the degree that made (Commodore Perry’s) Black Ships a term of art in the Japanese culture--expressing the surprise and confusion induced by such two dissimilar worlds, as the Japanese world and the US world touching each other.

In the way that the people of Japan maneuvered through the duality and contrasts that followed Commodore Perry’s visits to Japan, Agilists must often strike a balance between two parallel worlds. The measurement of Agile maturity at BMC is a splendid example of striking this kind of balance. After about a year of practicing Agile, I became increasingly anxious to know how the teams were performing. At my request, Dean Leffingwell and Becky Strauss devised a self-assessment methodology for the BMC Agile teams. Effective that it was, there was nothing revolutionary about the methodology itself. What was revolutionary was our decision to treat the self-assessment as privileged data that was owned by the respective teams -- separate from the overall set of measurements that we used at BMC to evaluate projects, teams and individuals.

From a traditional management perspective, the decision to exclude these self-assessments from the official measurement system was quite problematic. We were investing a fair amount of money to convert to Agile; many forthcoming products were critically dependent on the progress we would make assimilating Agile; certain commitments to customers could be met only through good progress on the Agile learning curve. Clearly, Agile maturity was an important metric by traditional Balanced Score Card standards.

Our decision to exclude Agile self-assessments from our official measurement system stemmed from our belief the data was too important to be used as part of a system that associates accomplishments with monetary rewards. Because we were betting the business unit on Agile, we could not afford team maturity to be gamed in the usual manner in which CPM metrics tend to get gamed by everyone and his grandmother. The Agile assessment data had to be pristine in the sense that there was nothing “good” or “bad” about the attained level of maturity -- it was what it was. The teams needed to be able to use their assessments of maturity as baselines for self-induced improvement. We empowered them, in collaboration with their Rally Software consultants, to look in the “mirror” of their self-assessments and determine what to do about this Agile “wrinkle” or that Scrum “wart.” We trusted the Scrum teams to use the data to drive their Agile maturity and to own their Agility.

Managing the friction between the official corporate hierarchy and Agile structural constructs is another good example for the equipoise that an Agile leader must orchestrate. For Agile to scale at the enterprise level, the Scrum team needs to cascade--like fractal building blocks spanning multiple levels. Consequently, responsibility and accountability get vested in the network of Scrum Masters and Scrum of Scrum Masters, obsolescing in many ways some traditional job definitions, like QA Director or Senior Engineering Manager. Due to these dynamics, the Agile world at BMC created its own de-facto Scrum hierarchy that existed parallel to the formal corporate hierarchy. Corporate inertia being what it is we did not believe we could successfully consolidate the two hierarchies into one in a short period. Hence, we depended on the equipoise that we struck between the two worlds, enabling the Scrum hierarchy to be focused on the delivery of value to customers, and the traditional corporate hierarchy to be focused on the administrative aspects of “life.”

You too will need to strike the kind of balance that we at BMC struck with our measurement systems and with the corporate hierarchies. To succeed, you must teach your teams, as well as yourself, to live with duality and ambivalence without feeling torn apart inside. Do keep in mind that you are dealing with two levels of conflict that accentuate each other – the visible conflict between the two worlds, and the invisible one that is likely to evolve in your heart as to where your allegiance lies.

The Equipoise of Agile as a critical skill, which enables the Agile methodology to be effective in corporate setting in spite of conflicting belief systems, might seem a little intangible. Yet, whatever you do, maintain the equipoise. It will keep your projects going even if the multi-faceted answers that you provide to various tough questions are considered lacking by some purists.

Acknowledgement: I am indebted to Dean Leffingwell and Becky Straus who created the Agile self-assessment methodology at BMC and provided very useful comments and insights to this post. I am also most thankful to Melody Locke who helped me immensely clarifying and articulating my musings.

Israel Gat - Cover to Cover

2008-08-28T10:25:03+01:00

I have been reading James Anderson Winn’s masterpiece, “The Poetry of War,” during my recent vacation. Regardless of whether or not you like poetry or war, this book is a must-read. Why the strong recommendation? I have never read anything that can be applied to the roots of programmer behavior in Agile context as well as this book does. Winn might not know what Agile or Scrum or XP means, but standing on the shoulders of giants like Siegfried Sassoon, Wilfred Owen and Robert Graves, he penetrates the psychology of motivation in an exceptionally deep manner. If you are asking yourself “What makes them tick?” – a question that Paul Beavers of BMC addressed in his blog entry – you owe it to yourself to pick up Winn’s book and read it.

In analyzing the poem, “To Lucasta, On Going to the Wars – For the Fourth Time” by Robert Graves, Winn makes the following observation:

“Graves dismisses courage, fear, love, anger, and hate as motives, claiming that the soldier fights only because of his pride”.

Readers of this blog who read my post, One of Us, might remember that I emphasized how essential the pride of the craftsman is to great Agile leadership. Borrowing the eloquence of Winn, I would dare say something like the following:

“IMHO economical rewards and fear of failure are secondary factors in Agile leadership. A great Agilist leads primarily through taking great pride in his/her work”.

Fascinatingly enough, Winn illuminates numerous motives (in addition to those cited above) that are quite applicable to debates currently taking place in the Agile community. For example, his insightful discussion of Honour as an inexhaustible treasury versus honour being finite, applies nicely to a major theme presented and discussed in the Agile 2008 conference - individual rewards versus team rewards.

I don’t doubt that various methodical aspects of Agile – e.g. Planning Poker and/or Continuous Integration – are critical for success. There exists, however, a deeper, human truth to Agile that is both simple and elusive. It is the simultaneous application of rigorous methodical practices together with the touch of the pride that makes the Agile leader One of Us.

If you are interested in Agile, I encourage you to read “The Poetry of War,” cover to cover.

Q4 - Glen Alleman

2008-08-21T15:51:12+01:00

Q4 - You suggest Q4!

When I think of “being agile” it’s not about writing code. I did write lots of code at one time. FORTRAN, ADA. Realtime control systems, flight control systems, radar and sonar systems. Lots of enterprise system integration systems. But I moved on to managing groups and departments that did the writing. They were better at it than me. What I learned over those years, starting back in the late 70’s was that writing code and managing projects that write code are not the same thing, once you move beyond a room with your 4 best friends and the customer. There were projects in the defense business in those days, where the customer and I (two people) were in the same room. I wrote FORTRAN and he looked at the results and said, “nope that’s not it, try again.” But those kind of projects are rare now.

What became obvious after awhile was that the design-code-test process was flawed. Not because of the steps, but because of the granularity between doing something and getting feedback. I designed and built control systems early in my career. These systems control some piece of machinery that flies our swims. They use feedback to decide what to do next. The sample rate for the control loop needs to be designed to not under control or over control the device. Too much control and the device “oscillates.” Too little control and it gets lost. The design of the sampling rate is an engineering problem.

The design of the sampling rate for feedback in software development is an engineering problem. Two weeks in XP or 4 weeks in Scrum appears to be a sweet spot. But it is highly dependent on the problem being solved.

Our approach is based on getting feedback at the right intervals to “control the loop” within the error bands defined by the problem domain. But the critical aspect “being agile” is that the control loop and the data must have high fidelity. In order to have high fidelity, you must have accurate data. When someone says “oh I’m making good progress on that Oracle integration section of the project,” what does that mean? It’s meaningless. “Good” and “progress” have no units of measure. So the way out of this is to define what “done” looks like before you start working towards done. That’s what project managers do. They build a description of “done.” One that can be meaningful to all the participants. This doesn’t have to be a details plan. In fact it should not be detailed beyond your ability to define “done.” But “docking at Hubble, changing the wide field camera and connecting the auxiliary battery cable” is a description of DONE.

When the “agile” term “emergent” is used it is usually an excuse to not trying to define “done” in some analytical term. This is the key to success in deploying agile for project management. Never to confuse effort with results. In order to do that, “done” needs to be defined in some way that is measureable. The path to “done” can emerge, but if the state of “done” itself is emerging, then you’re on the “death march” project we’re so familiar with.

The unit of measure of “done” is the deliverables from the effort. This is why Scrum is a natural fit as a framework for developing software and managing the efforts of developing software.

The final concept of agile project management goes back to the core process of the Project Management Institutes Knowledge Areas. It’s a “test question.” If someone says “we’re doing agile project management,” then ask how are you managing: integration, scope control, time, cost, quality, the people resources, communications to and from the customer, technical and programmatic risk, and any procurement activities?” If the answer is something like “oh we don’t do that, we do Scrum.” Then they’re not managing projects they are developing software. They are not the same.

Q3 - Glen Alleman

2008-08-21T15:47:56+01:00

Q3. Tom Gilb says a lot about "Systems Engineering". So do you. Can you tell us more about Systems Engineering and why it's been so influential?

Systems Engineering as defined by the International Council on Systems Engineering - an interdisciplinary approach and means to enable the realization of successful systems. It focuses on defining customer needs and required functionality early in the development cycle, documenting requirements, then proceeding with design synthesis and system validation while considering the complete problem of operations, Cost & Schedule, Performance, Training and Support, Manufacturing, Test, and Disposal.

This paradigm integrates the management of the product or service with the development of the product or service. Product architecture and Program architecture are interconnected. Systems Engineering looks at the interfaces between the components first. This is a natural paradigm for discussing Coupling and Cohesion.

For Project Management and especially the management of agile software development projects, systems engineering provides the language and tools for making connections between the elements of a product in ways not found in the convention approaches.

The “tradeoffs” between system elements is the language of systems engineers. This implies the collective ownership, the shared code base, the “emerging” requirements now exist inside a larger “systems architecture.” Making it better – refactoring code – must consider the systems impact. The impact on the components that live behind the interfaces. So managing the interfaces – be they software or process – is critical to keeping the system moving toward its goal of being “done.”

Q2 - Glen Alleman

2008-08-21T15:46:07+01:00

Q2 - Our worlds collide in the Agile PM space where you work on BIG systems. A lot of people reading this will, I suspect, struggle to understand the scale of the projects you've worked on and how they've influenced your thinking. Can you tell us about some of the more interesting projects?

BIG likely has no real definition. What is BIG for a 300 person energy company here in Denver is small for NASA in Houston. What is small for that same energy company may be large of the 4 person wireless antenna company our neighbor owns.

BIG usually starts with multiple customers and multiple development streams. This usually means some type of Enterprise projects. Something used by everyone – or most everyone – in the firm. A Claims Processing system, an ERP system, a Document Management system. This also can mean that the “customer” is not really a person, but a process. The Land Management System in a mining company requires that all documents be managed according to the governance policies of the firm. “Customers” of Land Management have some, but not much say in how the system will work. Enterprise system are not Personal productivity systems. They belong to and are defined by Governance frameworks.

So when BIG is used, it may not just be the size of the code base, or the number of developers. But rather the impact on the firm’s ability to conduct its business.

For product development, Big usually means some kind of “system,” with multiple moving parts. A product where managing the interfaces between these parts is more important than the contents of the individual parts. The individual parts are important as well – of course. But if they don’t “play nice” together the product doesn’t work. This is the basis of the Systems Engineering view of software development and the management of that development.

Q1 - Glen Alleman

2008-08-21T15:43:46+01:00

Q1 - Hi Glen. I think I first met you either via a comment you put on my blog or on the newgrange project management email list. Anyway, it's been a long time but since we live worlds apart, we've never met. Here's what I think I know about you - you're a big hitting PM, with a strong history in aerospace; you live somewhere sunny; you like cycling; and, you're a fast typer. How close is that and what did I miss? Can you tell us a bit more about yourself?

I do work in the aerospace and defense business, but those domains have large IT projects associated with them as well. Our firm – Lewis & Fowler - provide delivery services to commercial as well as defense clients. Our approach to managing projects is called Deliverables Based Planning ® and is based on the principles of Integrated Master Plan / Integrated Master Schedule found in the US Department of Defense. This approach defines and measures progress through Accomplishments and their Criteria measures as 100% or 0%. Either you did what you said you had planned to done during these period or you didn’t. No partial completion. No almost done, or “I’ll catch up in system test.”

While I personally work in the defense side of software development, ERP, CAD, application software for ground systems, flight systems, and on orbit operations are common. Software development is still a large percentage of most defense and aerospace programs. But I also work in commercial domains. Mostly with ERP and Enterprise applications. These two domain have similar attributes – mission critical development, distributed teams, rapidly changing requirements, emerging technologies, and inconsistent funding sources. Agile as a principle provides a framework for “managing in the presence of change,” rather than the more conventional approach of “trying to manage change.”

One critical aspect of my work is the separation of “managing projects” from “developing code.” Many might say these are the same, and I actually held that view at one point. But it is not the case. The management of projects involves aspects that are not found in agile software development activities. The primary missing attribute is the discovery of the business value. Agile development assumes the “customer” can articulate this business value and use it to prioritize the work for the next iteration. This is usually not the case for larger projects. Therefore some form of Business Capabilities and the associated Requirements Management must take place. The next attribute is resource management. If there is truly a single team in the same room with the customer, then resource management is self managed. In the absence of this “ideal” world, someone needs to plan for the allocation of resources.

Israel Gat - Over the Top

2008-08-17T15:18:45+01:00

Like every young man in Israel, when I reached the age of 18, I was drafted into the army, and started adjusting to the army way of everyday life. While army life posed many challenges, it soon became very clear and that I faced a single all-consuming question: how do you go over the top at the face of enemy fire? The question actually was two-tiered:

How do I get myself to go over the top?

How do I get other soldiers to go over the top?

Of course, physical fitness, small arms handling, and tactical skills were critical. But, they were of little value unless you could break out of the shock and paralysis caused by incoming enemy fire.

While the answer to the question of how one gets over the top might seem complex, the key ingredient turned out to be camaraderie. We were comrades-in-arms. Our bonds enabled us to rise above the instinctive self-centric and self-preservation view of a battle situation and elevated our concerns to the team (squadron, platoon, company) level, enabling us to concentrate on the tactical task at hand. The looming presence of death, particularly during the 1967 and 1973 wars, was omnipotent, but camaraderie was our antidote.

I recently received a lovely email from a former employee of mine, which connected the dots for me from the 1973 war in the Middle East to 2008 Agile in the US, as follows:

“…I want to thank you for pushing DSM into the Agile waters "feet first". Working with the agile teams at BMC those last two years were some of the most rewarding in my career because of the sense of purpose, direction and camaraderie that the process instilled in everyone.”

The most fascinating point to me in this email was the process -> camaraderie directionality. I had tried hard, perhaps too hard at times, to instill camaraderie as the means to improve our Agile process. As the email attests, the process actually fostered camaraderie among the team members.

I can imagine that camaraderie was instilling process that was instilling camaraderie that was instilling process, and so on, in a virtuous cycle that over time became extraordinarily effective for us. If you accept this premise, you must then ask yourself, “Where does one start in order to trigger a virtuous cycle in Agile?” My hunch is people over process is a good place to start.

Israel Gat - One of Us

2008-08-08T13:40:29+01:00

Years ago, during Bill Clinton’s campaign for the presidency, I watched a town hall meeting that Bill and Hilary held somewhere in the Midwest. Question from the participants indicated that they were concerned about the state of the economy and its impact on the standard of living in rural America. After some references to various real estate investments and stock transactions from which the Clintons had benefited, a participant in the meeting asked, “But are you really one of us [as you claim in your campaign messages]?!”
While I don’t remember the Clintons’ response, I remember the question as the defining moment of this town hall meeting. Instead of arguing whether GDP growth could be expected to be 2.1% or 2.2%, the person who asked the question went straight to the fundamental question of who the Clintons were in their heart of hearts. The piercing power of the question reminded me of Richard Nixon’s vicious quip about his opponent, Helen Graham Douglas, during his bid for the Senate in the early 1950s: “Pink right down to her underwear.” Competent and accomplished though Congresswoman Douglas might have been, during the Cold War with Russia, most US citizens were reluctant to cast a vote in favor of a candidate who might harbor pro-Socialist or pro-Communist leanings. During the weeks following Nixon’s accusation, California voters did not come to terms with the real Congresswoman Douglas. The rest, as they say, is history: Nixon won the campaign and became California’s Senator-elect.
I mention these two political anecdotes because of my strongly-held conviction that Agile leadership must be absolutely genuine. As an Agile leader, you must be of the same DNA (so to speak) as the developers, testers, writers, and product managers in your Agile/Scrum team, and identify with them completely in terms of both the talk and the walk. While they might not explicitly ask, “Are you really one of us?!” the question can linger like the proverbial elephant in the family room. For example, I sincerely doubt that you can achieve great success as an Agile leader if your passion is revenue recognition. You might be able to make various hard decisions in a dispassionate manner, but can you “touch” your Agile buffs (and those who are still sitting on the fence about adopting Agile) in a deep manner? Can you galvanize them around Agile as a very cool methodology?
You might be asking, “What on earth is in the DNA of software developers?” I believe the simple answer has been given in the Cluetrain Manifesto: the passion and pride of the craftsman in his/her work. If you consider your software to be an extension of yourself--worrying about each bug as if it were a potentially malignant wart on your nose--you probably possess the right DNA for Agile leadership. On the other hand, if you do software en passant, and do not take insane pride in the software that you produce, you are unlikely--IMHO--to make a great Agile leader. Because of your skills, experience, professionalism, and dedication, you might do a credible job, but I doubt that you will be able to electrify the Agile team to the level of hyper-productivity.
I do not pretend that I can measure your software passion and pride – these are for you to assess. Rather, my simple suggestion is as follows: before you start leading an Agile project, take a good introspective look in the mirror and come to terms with the real you – craftsman or no craftsman?! Agile can easily put you in a situation in which you will curse yourself for ever embarking on an Agile “adventure.” Do your self-test before you start, not during the course of a disastrous iteration when you start suspecting that Agile had been conceived by people who were “inhaling”
I am not sure how one administers a self-test for passion and pride. However, I will share a self test that I experienced after the time-to-market, productivity, and quality results for the BMC Performance Manager releases were published by QSM Associates. In preparing for an all-employees meeting of my business unit I wondered whether I should conclude the meeting with the following words:
Between the quarterly results and the QSM Associates study reside the 386 employees of this business unit. To each of you, I say, “this great quarter is yours alone!”

After wrestling with this question for quite some time, I ultimately used these words in the meeting (and never regretted it) for a very simple reason: they precisely expressed what I felt in my heart. I had no doubt that I really was one of us and would be viewed in this light by the business unit employees.
Other values, skills, and virtues are vital to great Agile leadership – see for example Jim Highsmith’s excellent presentation on the subject in the Agile 2006 conference. However, I would contend that the “One of Us” requirement in the sense defined above is absolutely necessary. Please remember – you can’t fake authenticity.

A Social Contract for Agile

2008-07-29T10:37:00+01:00

Here is an added bonus from Israel:

Perhaps the most difficult aspect of my leading Agile projects over the past five years has been dealing with the broken social contract. With globalization, off-shoring, outsourcing and reduction in the workforce being so pervasive, the social contract between employees and corporations in the software industry has for most practical purposes broken. The implied agreements by which employees form productive teams and maintain cohesive organizational order have at the very least been shaken, and very possibly voided altogether.

The voiding of the social contract typically manifests itself in the refusal – overt or covert - by employees to collaborate in their own demise. Knowledge transfer to an “assassin” in a remote country – i.e., to a person who will displace me after I train him – is the way employees often describe their reaction. Collaboration and empowerment go down the drain in the absence of an effective antidote to the “assassin” syndrome. Needless to say, trying to do Agile without collaboration and empowerment is like trying to drive your car without gas and wheels.

Conversely, an employee could and should expect an employer to pay close attention to the economical realities of business, including the cost of labor. This consideration is particularly true for the software industry where labor cost is often the #1 component in the cost structure. With software becoming more and more pervasive – in car safety bags, avionics, cell phones, etc. – software labor cost is fast becoming an issue for any industry that embeds software in its products.

The pity of it all is that nobody benefits from a broken social contract. Employers often stare at disappointing results from off-shoring and outsourcing as well as unsatisfactory outcomes from software initiatives that span multiple countries. Employees are frequently uprooted, either in terms of their sense of security/belonging/commitment; or, in terms of the actual dislocation that often accompanies a new job; or both. The simple fact that both employers and employees are driven by very similar Darwinian rules of economics is of little consolation to either party in the face of mutual failure(s).

While Agile most certainly is not a panacea, it actually provides the opportunity to resurrect the social contact in the corporate setting in the following two interrelated levels:

Enhance productivity big time. Results reported by Sutherland, Mah, and me show that productivity can be increased to the point that it compensates for the differential in labor cost from one country/continent to another.

Provide safety net for employees through the development of marketable Agile skills.

If I were to start a major new Agile project today, I would deliver something similar to the following message:
“Team, my overarching organizational objective is to preserve our team and its institutional knowledge for our corporation and its customers for years to come. We will achieve this goal by enhancing our productivity to the level that it offsets disparities in the cost of labor across geographies. I wholeheartedly believe the current state of the Agile art would enable us to reach this level of excellence. In the event that we fail to reach such a high bar and our assignments fade away, you will find yourselves in demand in the market, as in the course of doing the project you will become competent Agile practitioners. I am committed to your professional development, regardless of the outcome. A significant amount of your time will be spent in Agile training, coaching, and retrospectives.”

Saying so and walking the talk might not constitute a full-fledge social contract in the classical sense of the term. However, I believe it could be quite effective as “contracting in the small,” i.e., compensating for mega issues in the software industry by way of redefining the contract at the Agile project/business unit level. I tend to think about it as “mini social contract” – i.e., a contract between the team(s) and me.

This “mini social contract” provides the critical third ingredient in the “secret sauce” that I plan to henceforth use to drive Agile projects, as follows:

Empower them – everyone loves to have the opportunity to make an impact.

Excite them – genuinely executed Agile is a great antidote to the weariness, bitterness, and cynicism that a broken social contract generates.

Invest in them – hone their Agile skills in a way which (should the need arise) will transcend organizational/corporate boundaries.

While the approach I advocate might fly in the teeth of some prevailing wisdom, practices and policies, I believe it is mutually beneficial for both employers and employees. Neither employer nor employee can single-handedly change mega trends in the industry; both can, however, learn to live and evolve with these trends in a way that leaves both as whole as is realistically feasible. The simple sentence “Whether you will or will not be with the company in the future, I acknowledge your need to develop professionally as an Agile practitioner” is all it takes to start reconstituting the social contract.

Q6 - Israel Gat

2008-07-09T12:33:26+01:00

Q6: What should question 6 be? And what's it's answer?

I would suggest the following question: Can one apply Agile beyond software?

While I have not really done any systemic primary research on the subject, my hunch is that the answer is a resounding “Yes!” I offer three data points to support my hypothesis, as follows:

Jim Highsmith has been doing Agile with an architecture firm in Italy.

Paul Beavers and Becky Strauss are currently working on introducing Agile in BMC’s Customer Support organization.

My own experience with my staff.

Of these three data points, I will elaborate on the deep transformation we went through as the staff of BMC’s Distributed System Management business unit for the simple reason that it offers special insight on the nature of Agile principles.

As mentioned in previous postings, we started doing Scrum in 2004. By 2006 we had mastered many of the aspects of Scrum. Furthermore, the business unit was “Scrummed” in entirety at this point in time.

During this year (2006) a product line was moved into my business unit. While the leaders of this product line were generally competent, their assimilation in the business unit was not working. It was clear they were struggling. We, as staff, were struggling to figure out why they were struggling…

The answer to the question why they were struggling eluded us for quite sometime. Examination of the “usual suspects” – strategy, market, execution, structure and time (for integration/assimilation) yielded no insight. There existed, of course, all kind of very legitimate differences in thinking and approach between the old hands and the new product line folks, but nothing to write home about, nothing that could shed light why the assimilation was so problematic.

Sean Duclaux, who was the director of product management for the business unit, came up with the insight we needed. Sean grasped that doing Agile transformed me and my staff with respect to our mode of operation. Unbeknown to us we became so-very-agile in the way we functioned. Actions were taken quickly; decisions were postponed to the latest responsible time; something we were working on in a certain staff meeting would be dropped in favor of a more important item in the next meeting; etc. None of these was conscious – we did not set ourselves to run staff using Agile principles. These things were just happening.

Checking with the struggling product line team, Sean’s hypothesis was validated. From their perspective we were an unbelievably chaotic bunch. We were not even managing an “organized chaos” – in their eyes we were increasing chaos. They found it next to impossible to operate within the fluidly dynamic environment that de facto prevailed in my staff.

The problem actually proved very hard to solve. While we had any number of staff members and staff members of staff members who could immediately help with just about any implementation aspect of Scrum, we really had no expertise on the transformation in our own functioning as decision makers. I am out of my depth here, but I tend to think we had gone through a prolonged evolution period in which Agile principles, in the most general sense, sank into us and became part of our standard operating procedures in any domain we focused on. This evolution period was missing altogether for the new folks on my staff. Expecting them to adapt quickly, we were sort of implicitly asking them to skip a developmental phase that one has to go through whether one is doing Agile in the R&D lab or performing in an agile manner in staff.

I find it fascinating that we had not realized Agile was transforming us as staff while we were transforming software engineering practices at BMC through Agile. There is an intriguing symmetry here that speaks straight to the heart of the software craftsman that I am.

[Clarke: My thanks to Israel. If you have a question for Israel then leave a comment and I'll pass it on.]

Q5 - Israel Gat

2008-07-09T12:30:36+01:00

q5) Has the conversion stuck and what are you doing to keep it sticky?

The conversion has more than stuck. In terms of scope of the practice, it flourished to the level of close to 1,000 Scrum users/seats at BMC nowadays. In terms of quantifiable accomplishments, the joint SQMA/Cutter Consortium/ BMC study has concluded:
"Nearly three times faster time to market than industry average; 20-50% improvement in individual team productivity; and, one quarter the expected number of defects based on team sizes and schedules...”
Furthermore:
“In their 20 years of benchmarking software development projects, the QSM Associates team has not seen such a high performing team [BMC’s] distributed across such a broad geography."
Suffice it to say I believe transforming BMC to achieve this level of excellence in Agile/Scum is probably the greatest accomplishment of my whole career.

To reach even higher levels of effectiveness and efficiency in software development, I believe we need to focus on four elements: methodical change, system development,operational innovation and organizational adaptation.

The initiatives in these four areas I am promoting/pushing these days are as follows:

Methodical change: We are more and more moving these towards Lean Agile.

System development: Agile, to me, is not “just” a R&D “thing”. Rather, one reaps much greater benefits by end-to-end Agile, instilling it all the way from the R&D lab to the customer shop. For example, Innovation Game® from Enthiosys is a tool I am considering using to refine our Agile/Scrum system and process.

Operational Innovation: My recent work on Agile-Based-Market-Of-One (ABMOO) is a good example (even if I have to say so…) of innovative exploitation of the methodology. The power of ABMOO is in transforming the business design to fully capitalize on the methodical capabilities.

Organizational adaptation: With so much programming work carried out through outsourcing, the boundary between where the corporation ends and the outsourcer begins becomes very fuzzy. My contention is that the next big frontier for Agile is in developing joint Agile infrastructure which will be used by both the corporation and its strategic outsourcers.

It is a little premature to assess which of the initiatives listed above will really take off big time. Until we know the answer, I am so very happy to be part of the Agile drama. This is not a drama we passively watch in the theatre. Rather, you, me, the developer in the cubicle adjacent to yours, the tester next to him, etc. are active players in this drama. What else could one ask for?!

Q4 - Israel Gat

2008-07-05T18:12:00+01:00

q4) When did you start seeing results?

I will have to give a complex answer. Some results were discernible very quickly. Others took quite a long time to materialize.

Three discernible results became very clear within three months:

Our testers became much more knowledgeable on the product(s) almost overnight. Working “real time” with product management and development from the very early stages of product conception, design and implementation immensely enriched their understanding, experience and knowledge.

It soon felt “As if a new sun had arisen”^¹. Various things were not working well for the business unit before we started Scrum, and numerous team members were quite worried. The introduction of Scrum shifted the mindset from “The world hates us” to “Gee, this stuff [Scrum] is cool!”

In three months we shipped the first Scrum release. It was not a great release, but it was working code at the hands of our customers. Walter Bodwell, the director in charge of the BMC Performance Manager (BPM) summarized the effect very astutely:

“If we used Waterfall on BPM, we would still be in development. We would likely be cutting features right and left to try to bring the date back in. Changes requested along the way by the solutions teams would have been pushed back on rather than embraced.”

In contrast, it really took us one to two years to master various important aspects of Agile. Examples are uniformity in story cards, accurate assessment of velocity and effective Scrum of Scrums. All in all, I would characterize the phases of our methodical progress as follows:

First year – we were learning how to do Scrum on a fairly large scale basis.

Second year – we were starting to be proficient in Scrum.

Third year – we rocked.

Various practitioners with whom I discussed our Scrum evolution felt that the three phase learning curve described above is too slow for executive management to buy into Agile. As this observation indeed is quite true sometimes, whenever I discuss Agile with executives who seem to have a very short time focus, I add a stern warning:

“Don’t agile the Agile!”

1 This phrase had originally been coined to describe the reign of Edward III (1327-1377) in England.

Jerry Weinberg - Q2

2008-07-05T18:05:55+01:00

Q2. You've probably written more books than I've read in my lifetime ... and, as far as I know, none of them have been specifically about testing. Why this book, now? Does the software development world really need another book on testing?

The software development world definitely needs THIS book on testing. I've written it now because I've grown almost fatally tired of dealing with clients who have all sorts of cuckoo ideas about testing. Reviews have already ordered multiple copies of the book to give to their managers, customers, developers, and testers. It's short and hopefully easy to read by non-techies. It's time we cleared up these recurrent fallacies if we want to our profession to progress. That's why I wrote the book now.

And, BTW, I've written about testing in every software book I've published, but people have been asking me to bring lots of material together in one place. This book is not a compendium of what I've already written, but more of a deeper analysis behind all of those things written by me and a bunch of wise commentators. I intend it as a must-read for everyone connected with computing in any way.

Jerry Weinberg - Q1

2008-07-05T18:04:38+01:00

Q1. Hi Jerry, I'm intrigued by the title of your new book "Perfect Software and other illusions about testing", which will be published later this year. Are you saying that "perfect software" is impossible but a lot of people believe it is? Can you elaborate on this?

Perfect software is indeed impossible, and even if we had it, we'd have no way of knowing. The Laws of Thermodynamics tell us why it's impossible, but that doesn't prevent lots of people from believing it's possible--just like many people believe perpetual motion machines are possible.

The book elaborates on this and many other illusions about testing--why people continue to believe them, what kinds of trouble they cause, and what can be done to mitigate that trouble.

(BTW, the book will be available in July 2008, any day now.)

[Clarke: I'll post an entry here when it's out]

Q3 - Israel Gat

2008-07-04T18:08:00+01:00

q3) What were the (say) 3 biggest obstacles to conversion and how did you overcome them?

Is Agile another good-for-nothing management consulting fad? We did not start from square zero trying to implement Agile. Rather, having been exposed in the past to various methodical and management consulting experiments which were carried out half-heartedly, people in the trenches wondered at the beginning whether Agile was another one of “this too shall pass” initiatives. Will Israel and his staff walk the talk was an open question when we started rolling.

The defining moment for our sincerity and commitment happened at the conclusion of one of the release planning days. Paul Beaver, the senior director in charge of PATROL and the BMC Performance Manager, asked for a thumb-of-five vote^¹. One of the teams came back with an average of 2 as their confidence level. Upon enquiring, the team members sort of said “Well, we do not really believe we can deliver, but we thought this is the plan management wanted…”

Paul, to his great credit, sent the team back to the drawing board, held another release planning day at a later date, and ultimately accepted a revised plan for which the team confidence level was 3.5. Since this episode, nobody ever doubted our total commitment to Agile.

Always-releasable: I can’t convey in words how difficult this intuitively simple principle was to implement. There is nothing extraordinary that I can point out to as the reason for always-releasable being so difficult to attain. The grand total of “infinite” number of trivial deltas/incompatibilities between components of the product suite, their containers (i.e. the SCM’s), and the engineering practices across five development sites proved to be almost insurmountable. We overcame the challenge by simply putting in an incredible amount of hard and tedious work, eliminating the obstacles one by one over a period of time that felt like ages.

Once we reached the always- releasable state, the improvement in productivity, velocity, quality and predictability was dramatic. The code became reasonably stable on an on-going basis rather than rarely stable till the very end of the release. The effects of various bugs in our code were not accumulated nor compounded. Rather, most of the bugs got knocked out “real time.”

While in general I believe Agile practices should be reasonably flexible – i.e. tailor the practices to the situation at hand – always-releasable is for me a non-negotiable.

I feel so strongly about it I would go as far as saying one does not really practice Agile until the always-releasable state has been achieved.

When the magic fails to work: Towards the delivery of a major release I had a layover in NYC on the way back from Tel Aviv, Israel to Austin. TX. I called my good friend Roy Ritthaler who was in charge of product management for the business unit. To my friendly question “How are you doing, Roy?” I got the authentic, maybe too authentic, answer “Better than you, Israel”…. Turned out the release was in fairly deep sneakers. The number one reason for the difficult situation we were facing was me being overly aggressive on the release, betting that our velocity would rise to the level required to produce a very complicated suite of products. The fact of the matter was our velocity at this point in time was not yet high enough (thought it became incredibly fast later). I lost in the critical race between Scrum methodical progress and the need to introduce a “big bang” product suite in our market.

The crisis around the release being in trouble revealed I had not done a good enough job educating Marketing, Sales, Software Consultants and Customer Support on the nature of Agile. Things that are standard operating procedure for Agile in R&D - e.g. ship the features you can and catch up on the features you missed in the next release – became highly controversial. In particular, the spirit of Agile, as distinct from the mechanics, was extremely difficult to explain amidst the crisis.

Bill Miller, who was my boss at the time, saved the day. He grasped that the fundamental problem had nothing to do with Agile – we would have faced the very same crisis had we been playing Tin-ker-toy instead of Scrumming. The furor we were facing was about our go-to-market machine being thrown out of balance as a result of the release problems. The challenge was to align R&D with the business, not about Agile.

Painful that the crisis was at the time, it was the trigger for our transition from Waterfall optimized organization to Agile organization. This transition eventually led to our pioneering work in the area of Agile-Based-Market-Of-One. While it certainly did not feel so at the time, the release crisis actually was a blessing in disguise – it forced us to thoroughly revisi our approach to Agile. We ultimately gained much deeper mastery of the subtle issues of end-to-end Agile within the corporate fabric.

1 A 1-5 confidence voting scale based on the number of fingers one displays: 1=this project is heading towards total disaster and I am updating my resume as soon as the meeting ends; 5=we will deliver even if a hydrogen bomb explodes in 10431 Morado Circle, Austin, TX (one of the BMC site where we were Scrumming).

Q2 - Israel Gat

2008-07-03T18:06:33+01:00

q2) Can you describe the scale of your Agile conversion, the time frame and how you went about it?

By the end of the first year (2005) of Agile operations we had 350 product managers, testers and developers furiously “Scrumming”. Today (summer 2008) BMC has close to 1000 Scrum users/seats. Details of growth as function of time during the first year are given in the chart at the bottom of this answer. [clarke: click the image below to get the full size view]

Three “ingredients” were at the heart of our Agile/Scrum implementation:

1. Intentionality: We were crystal clear about converting to Agile no matter what. Becky Strauss, who managed the Scrum assimilation program, summarized it so very eloquently, as follows:
“There has never been a thought towards returning to Waterfall – we only think about how to be more agile – how to do this better. No one wants to go back!”
2. Trust: We fully trusted the teams to do the right thing(s) within the boundaries of the methodical rigor prescribed by the Rally Software consultants. The fundamental tenet we held was very simple: we hired the team members for their creativity and motivation; both go down the drain without trust.

Our trust was amply rewarded in multiple ways: A) it was reciprocated - we, as the management team - were trusted by employees in the trenches; B) the trust instilled confidence in the teams; and, C) the trust, compounded by the first few visible accomplishments, built enormous pride.

3. Risk taking: Doing Agile is like learning to ride a bike – you had better be ready to fall, fall and fall again until you master the art. The risks of falling are nicely compensated by the velocity – on the bike and in doing Agile - one ultimately attains. The important thing is to be able to learn the lessons from calculated risks that were not well calculated.

My philosophy of risk-taking in the Agile context is deeply rooted in my view of the art of programming being craftsmanship that one best learns through apprenticeship. An apprentice does not become a good craftsman, let alone a great craftsman, by minimizing risks. Rather. It is the willingness to accept failure as the sometimes inevitable twin brother of aspiring to always do better that makes a craftsman.

To my way of thinking, the three elements listed above form a cohesive philosophy of Agile transformation:

Trust is a derivative of intentionality. One cannot transform an organization single handedly. Trust “recruits” partners for carrying out the transformation.

Unless one is willing to takes risks, many of the benefits of trust are lost. Trust leads to empowerment; and, empowerment and risk-taking are inextricably linked. One needs to manage empowerment and risk-taking in tandem as the foundation on which successful Agile projects evolve.

I will discuss various details of BMC’s Agile transformation in my forthcoming answers. However, if you are starting an Agile program today, you do not need to burden yourself with the myriads of details that need to be figured out along the way. If you genuinely adhere to them, the three principles discussed here – intentionality, trust and risk-taking – will get your Agile program going.

Q1 - Israel Gat - BMC

2008-06-29T14:33:25+01:00

Q1. Hi Israel, we first chatted about 3 years ago (I think) when I helped shepherd your agile2006 experience report. I remember being very impressed at the time with the approach you took to converting to agile. Could you tell us a little about your team, about your team's pre-agile life, and what lead up to your decision to embrace agile?

When I joined BMC Software in summer 2004, I was not too certain about the “Other duties as assigned” line item in my job description. Within a few short weeks it became very clear: “other duties” meant doing quick psychotherapy for the endless stream of strangers who came to my office to complain about the software I was responsible for. I did not need to ask the classic question “How did you feel about the software bug?!” – I was proactively advised how the person calling upon me – every person! - felt about it... Some actually reverted to Hebrew (my native tongue) in order to make doubly certain I did not miss any nuance of their disappointment, dismay, despair, anger and anguish. A distinguished industry analyst actually described my challenge as “Israel, you need to cross a chasm of Biblical proportions”…

The view of my role as a resident psychoanalyst was soon dwarfed by an acute feeling of cognitive dissonance. On the one hand, everyone and his grandmother were complaining about software problems. On the other hand, my staff members, and their staffs, and many other individuals whom I met in the business unit seemed quite competent and dedicated. I could not reconcile the gap between good developers, testers and product management professionals on the one hand and unsatisfactory results on the other hand. More and more I felt like one foot in cold water, the other foot in hot water.

After scratching my head for a period of time, I reached a very simple conclusion: good people who do not achieve satisfactory results must be using the wrong methodology and/or lacking adequate tools. A quick investigation revealed the business unit was choking in rigid Waterfall practices. I found the diagnostic clue I needed – severe case of chronic Waterfall addiction punctuated by acute outbursts around release dates. My true mission, I concluded, was to turn the guys into recovered Waterfallics. My initial impression of my role as a resident psychoanalyst turned out after all to be quite accurate…

Vetting the radical idea of Agile/Scrum turned out - much to my surprise - to be a piece of cake. I think there were four main reasons for the ease with which Scrum was wholeheartedly adopted by the business unit, as follows:

1.    I was reporting to an exceptional executive and human being – Mary Smars. While Mary was not an expert in Scrum, she possessed this rare combination of pragmatic common sense, wisdom of life, patience and ability to trust people in a deep manner. To this very day I believe these are the four requisite virtues to look for in an Agile executive.

2.    I was blessed to have numerous competent colleagues who rallied around Agile. Walter Bodwell, Becky Strauss, Paul Beavers, Michael Cote, Roy Ritthaler, Glenn Jones, Chet Henry, Igor Bergman, Mike Lunt, Melody Locke and many others enabled effective diffusion of Scrum through the organization. They ensured the business unit was entirety galvanized around Scrum, had the required know-how at each and every level and operated cohesively across multiple sites.

3.    We invested a lot in top notch consulting. The Rally Software A team - Dean Leffingwell, Ryan Martens, Jean Tabaka, Hubert Smith and Michele Sliger - spent a ton of time with our teams, ensuring we do Scrum right from the very beginning.

4.    We were willing and able to learn and improve every step of the way. I am still not 100% certain how we so naturally evolved to be a learning organization. My hunch is that unbeknown to us we all shared the philosophy so eloquently expressed by Alexander Pope:
“Be sure yourself and your own reach to know,
How far your genius, taste and learning go;
Launch not beyond your depth; but be discreet,
And mark that point where sense and dullness meet.”
A couple of months after we started Scrumming, I met with one of the teams to brainstorm about their Scrum experience. I was most pleasantly surprised how well things were working for them and expressed my deep appreciation to the team. One of the developers said “Israel, you do not understand – we have been dreaming about doing Agile!” To my naïve question “What stood in your way of so doing?” I got the real life answer “Your predecessor would not let us…”

Tom Gilb - an example from Ryan Shriver

2008-06-08T10:26:50+01:00

I'm not sure what you call people who follow Tom Gilb's work ... Gilbettes, perhpaps? Gilbittes? Gilbees? Who knows?

Anyway, I'm one and so is Ryan Shriver. Here's Ryan's stunning story of how he enhanced his already impressive Agile work by following Tom's advice.

Q: My first question: is your story the same as Jens'?

No, not really. I'm envious of Jens' story. My implementations of Evo + Scrum haven't been as clear-cut and clean.

I got introduced to Tom and Evo in Sept 2005 when I presented Scaling Agility at the Agile Business Conference in London. My talk was on how you do XP on teams of up to 100 for product development. It contained lots of "tricks of the trade" and "lessons learned". I got good feedback from people in the field struggling to apply these concepts on their projects.

As background, I learned Agile (well, XP) by reading Kent Beck's white book and putting pieces into practice on projects I was working on. This was around 2000 or 2001. Our team was floundering and as a senior developer I was seeking out better ways to work together when I came upon XP. We implemented some of the development practices (like unit tests, pair programming, etc.) but it wasn't a whole-team adoption.

As I moved onto newer projects, I'd do a little more XP each time. In December 2003 I was brought in as the lead architect for a product company that needed to get a new product to market (large loan system, J2EE, relational databases, rules engine, web based, etc.). I convinced the COO to use XP for development and she agreed (I learned later that prior to my involvement, her top analysts had spent 9 months trying to identify requirements and design and had gotten essentially no-where. That, and they had a signed contract and needed to show something soon, helped convince her to try XP).

From Dec 2003 until Sep 2005 the team did XP and we grew from 6 developers / 12 persons to 40 developers / 100 persons on the project. I was both the technical architect and agile coach. We had some successes, but also struggled with aspects as we got bigger that bothered me. I gave the Scaling Agility (slides are on my site) about this time that talked about the positives and negatives we experienced. Some of the issues I struggled with included:

1. Thousands of user stories with no clear sense of what was REALLY important to the stakeholders. The Customer Team wrote lots of stories but there was no higher-level sense of the larger objectives for the system.

2. "Non-Functional" requirements were practically non-existent, even when I asked I got vague answers. The product company typically relied on their customers to provide these. We'd design to handle one customer's volume, only to have a new customer come along with 10X the volume and we spent too much time redesigning and refactoring working code to handle the higher volumes. This was doable with agile, but very expensive and time consuming. As an example, one large customer had 4,400 functional requirements in their RFP for the system and less than 40 non functional requirements. I'm not kidding. This company was going to put about $60 Billion USD on this system. Both parties severely underestimated the system qualities and were very far apart in what their expectations were. Agile didn't have much to help here, aside from "create user stories".

3. Overall frustration that while XP worked well at the development level, there was no way to report "progress" aside from what stories got done. And that doesn't mean anything to powerful people who make important decisions :-) We could only report 'velocity' based on done stories and this didn't make sense for the COO and other senior executives. I had no way to tell them how we were delivering value, if we were at all. No idea the business value of some stories over others.

So I present at the conference and hear Tom for the first time. He was doing a keynote and he said something like, "Unless agile methods are quantifying critical success criteria, they're incrementally better than waterfall methods". I was floored. In a good way. At an agile conference full of agile cheerleaders all drinking the agile kool-aid, this guy I've never heard of was speaking some truth. And some of what he was talking about seemed to address my needs and concerns using agile and my complaints about it.

So I tracked him down in the hallway, we found a table and Tom starts showing me impact estimation tables, the concepts around objectives, targets, failure and parts of Evo. I picked up his copy of Competitive Engineering at this conference and started reading it on the flight home. It was a dense read, and it would take me a few readings to get through a chapter, but I kept at it for the next 4-6 months. I'd read a little, apply some of his concepts to my project, and figure out what worked, what didn't and how to fit the concepts to my team.

Because I was the lead architect, I first put Tom's principles into use on the performance testing and non-functional requirements for a major client of the software vendor. These were areas I was working on and tuning system response times, transactional throughputs, job run times and other things like them benefited from impact estimation tables. Once the performance objectives were defined with target and failure level, we used application profiling to identify the bottlenecks and to estimate the time savings we'd achieve if these bottlenecks were removed. Once we'd get an estimate for the potential savings, we'd estimate the complexity to make the change in the system (these were often cross-cutting changes that had to be thoroughly tested and thought-through). Using these, we'd identify the "biggest bang for the buck" change, make it, and re-test performance. We kept doing this until we got performance in line.

Oftentimes, our "gut" about which changes to make next were the same as the IE tables indicated, but I always felt better about having numbers to back-up my decisions about which changes we'd do and why (performance to cost benefit). I found that some of the other architects liked these methods, some didn't see the purpose and others couldn't care less...it didn't change their world. So adoption on the team was a mixed bag.

Our performance turning resulted in two high-profile benchmarking sessions at Sun Microsystems for an important customer. There was lots of fancy expensive hardware and the system performed well. It was stressful for me and our group of engineers with the client's team in the room while we dialed in the software and hardware

I left this project in Feb 2007 and started doing two new projects, in an agile coaching and requirements analyst capacity. Around this time I switched from XP to Scrum, mostly because Scrum was being better marketed here in the US and it was becoming the "corporate agile" method of choice. I actually like some aspects of Scrum better, such as the Product Owner vs. Customer Team, but even as I switched to Scrum I found out that it had the same shortcomings as XP with respect to issues I mentioned earlier.

I kept reading Tom's book and looking for ways to apply the concepts I'd used to performance test the system to cover some of the shortcomings with XP (and now Scrum). When I started the two new projects, I had some new opportunities to try out some of Evo concepts (for planning and requirements) with Scrum (for delivery).

The first project was for a local start-up in the travel industry. It was a subsidiary of a global company. The were / are building a patent-pending product for cross-selling products and services to people who travel. We pitched the CEO in the proposal to do the project agile with measurable business goals, and our other three competitors all pitched waterfall. We won the work :-) I did the inception phase almost solo, documenting the high-level business objectives and system quality objectives. I used Tom's planguage to do this. Also did the high-level functional and quality (performance) requirements along with a high-level architecture of the solution.

I found the CEO liked the ideas of the objectives, but didn't embrace them like I thought he might. The director of development, who I worked closest with, loved them though and continues to use them. The CEO agreed to 13 high-level business objectives that I defined, developed target and failure levels for and said we'd use to prioritize our product backlog. I spent a good bit of time getting these done, but as I transitioned to a larger team from our company who came in to deliver the system in the next phase, this work really helped set the project up for success.

We ended working with them for another 12 months delivering newer versions of the system. We'd re-evaluate the progress towards goals each quarter, make adjustments, and continue. The team was doing eight-week releases and two-week sprints so about every 1.5 releases we'd revisit the goals (should have been every release). We're working with this client right now to produce a case study of this project, but unfortunately they haven't been as diligent about running down actual results and sticking to it as I'd like. I think we will get some numbers that we can share, but it's not as perfect of a case study as I'd like.

But the system has been a big success, launched with 4 major airlines and travel companies. They're doing about 1 million daily web-service transactions and volume is growing as they add new partners. One of the things that I really focused on when I was doing requirements is the response time and throughput transaction performance. I set aggressive target and failure levels for each that would allow them to grow the business without major re-designs for performance. Target response time was < 500ms with failure > 2500ms. Target throughput was 50 TPS if I recall, more than enough for the to grow the business. From day 1, the architects, developers and testers knew these numbers so they designed to hit them and early on we put in load testing and monitoring. They'd test, I'd help them tune the system, repeat and continue. To this day they've had near zero issues with performance even there volumes are growing rapidly. I attribute this directly to the clarity that planguage brought to the problem and the teams focus on a few performance metrics that were important.

The second project was for a Fortune 500 financial services company (all these are in the USA). They had an IT department that looked at new tools and methods and wanted to learn agile. The found us and we agreed to help teach them agile. But the one thing that differentiated us from others is we emphasized using agile to deliver measurable business value with Scrum. We won the work and I had to get on a plane! The team didn't know how to select the best project to do Agile with, so I helped them use Evo to figure this out. We found one of the lines of business as a willing partner, did some interviews with people at all levels (associate to CEO of a line of business) to understand better they're challenges, frustrations and get a sense of where we could focus. We also looked at the organization's objectives to see what they said. We did a workshop with the entire team (executive sponsors, sales, technologists, analysts) and validated the top 4 business objectives and zeroed in on "Increase Revenue" which was the most important. We asked the sales team, "what could we do to help increase revenue?" and they said, "Give us more time to sell. We've got lots of issues with our systems, but if we just had more time to sell we'd generated more revenue".

We started identifying that the sales people did a lot of their own administrative work when not selling and they struggled to find out what forms to use to open new accounts. Some of them also had labor-intensive processes for creating new letters. We identified 13 possible opportunities out of this session that could help us increase revenue.

Then the technologists were provided the constraints of the project: Six weeks, small development team. That's it. We asked them which of the opportunities could even be impacted in that time. They said it had to be established technology and involve next to no system integration. So that ruled out some options and eventually we can back with three options to the business, any of which could make some improvement towards in six weeks.

We collectively agreed to focus on the system objective "reduce the time it takes to open a new account". We used impact estimation and zeroed in on the design that asks some simple questions (between 2 and 7) and then returns the right account form based on configured business rules. In six weeks the team designed and implemented the three-screen interface and got 24 new account type rules configured in the system. A few other minor features but that was it. It was a big success. Small, simple and was done with minimal costs. Now it's rolled out to about 100 people in the field. That was last year and this year I'm following up to get results, but it's been difficult to get more than qualitative success.

So I wish I had more case-study data to back these up, and I'm working on each, but I don't have as much as I'd like. There is a third project I coached early this year that went from measurable objectives to backlog. This project has the best success because the company really bought in. Release 1 just rolled out and the goal is that by July we have our first feedback numbers. I'm seeking permission to publish it or at least share it at Agile 2008.

So that's my story ;-)

Oh, your questions:

Q: Tom said that you've been using Scrum combined with EVO. Can you tell me a little background about the project and what problem EVO solved that Scrum didn't?

See above!

Q: How did you learn enough about EVO to get started? Was that easy?

Met Tom, bought Competitive Engineering. Taught self, with emails and reading papers from Tom. Also attended his seminar last year and plan to this year. Tom's very giving with information and his time and this has helped quite a bit.

Q: Did you have to change Scrum much?

You don't really "change" scrum, you just augment it with Evo. You change your thinking about how you prioritize your Scrum backlog, but the mechanics of Scrum still work the same (stories, sprints, retrospectives, stand-ups, backlogs, etc.). Essentially, the parts of Evo I adapt don't really have a counterpart in Scrum, and while there is some overlap between the two (Evo has Implementation Cycle, Scrum has Sprint - same concept), both methods are also complimentary if the best of each are used together.

Tom Gilb Q7

2008-05-11T14:17:41+01:00

Q: Can you give us a simple example (or two) of how to quantify requirements? I recall one interesting example you gave when you did training for us at AgileScotland that involved (I think) making online survey forms easier to use ...

Well, the longer answer is available:
http://www.gilb.com/community/tiki-download_file.php?fileId=124 <- Quantifying Quality Paper.

But the main idea is that for a critical improvement (usually 'Quality') attribute - we need to articulate the idea with Numbers.

This involves two steps:

1. Define scale of measure
2. Define some useful numeric points on that scale.

So instead of 'better user friendliness"

we might write:

Usability:
Scale: The average time needed for an average user to correctly perform representative tasks.
Past 10 hours.
Goal [2009] 1 hour.

Most people can quickly pick up on doing this.
When you get stuck there are lots of available Scale patterns to use, and to tailor to your specific purposes.

My favorite teaching example is quantifying 'Love' !
A friend pointed out the Bible, Corintians 1, had the solution! - as I pointed out earlier.

Google them! For example: "Intuitiveness software metric" , if all else fails try Intuitiveness software metric gilb :)

Tom Gilb Q6

2008-05-11T14:16:35+01:00

Q: Where and when do you think it all went wrong in our industry? I'm particularly interested in the historic misunderstanding that caused the waterfall model to spread. What made it so "sticky?" compared to incremental / evolutionary development?

(I have asked Craig if i can put this on my website).

Well, It is a long story! Once Upon A Time, we had small hardware memories, and small programs, and smart people. So projects were short, simple and worked pretty well.
Then memory started getting much bigger. Software got much bigger, and complexity of the software got horrendously much bigger. Things got late and buggy.

The obvious solution seemed to use good 'engineering methods'. So, based on a misunderstanding of Winston Royce, who pointed out that a Waterfall Process would only work for small systems, and that iteration/feedback/learning was necessary for complex systems, the US DoD, encouraged by Barry Boehm (a TRW Colleague of Royce), adopted the Waterfall Standard (DoD 2167 and 2167a). Well, it was not intended to be a strict Waterfall Process, the author personally told me, but it got misinterpreted that way. A little bit of requirements and design before coding is not such a bad idea. But the DoD, whose project size was exploding during this time, discovered that half of their $40 Billion annual software projects were total failures anyway. So in 1994 thy officially cancelled the 2167a Standard, and officially instituted the Mil Standard 498 Evolutionary Standard. It turns out that smart rocket scientists, who were not forced to do Waterfall, had always used Iterative/Feedback methods (http://clublet.com/examples/docs/IIDHistoryByLarman.pdf). I campaigned for Evolutionary methods, mostly for deaf ears practicing in Sixties, and publishing in Seventies. But the real awakening came when the Agile Boys read Principles of Software Engineering management (1988) and made iteration a fundamental practice. Too bad they did not read the entire book and realize that to control the iteration and learn, you also need to quantify the top level critical improvement objectives of the system. Some of my Agile friends, like Ryan Shriver, have recognized this failing and are actively practicing iterative control as I would advise. With focus on delivering measurable value to stakeholders, and not just stories to customers.

My underlying theory is that the Waterfall Method allowed Software Companies to earn a lot o money without actually delivering any or much value. So the real culprit is not unenlightened programmers, they are just foot soldiers. The real problem is the political leaders (top managers) who persist in paying so much for so little to so few.

http://www.gilb.com/community/tiki-download_file.php?fileId=38

So there was a historic misunderstanding (of Royce's message) - but the real problem, as I see it was lack of responsible management - to pay for value , not pay for useless programming work.

Tom Gilb Q5

2008-05-11T14:14:23+01:00

Q: Is it true that you are related to Santa?

I love giving gifts, especially knowledge, and I have many helper elves who help spread my gifts. I live nearer the North Pole than you do.

And my Summer cabin is right next to Drøbak - home of Santa Claus

http://www.emb-norway.ca/facts/Traditions/Christmas/julenissen.htm

But, I am not allowed to admit any more.

Tom Gilb - an example from Jens ...

2008-05-11T14:07:21+01:00

Jens Egil Evensen has been using Tom and Kai Gilb's EVO approach with Scrum. Here's a few questions and some very interesting answers with Jens. I strong recommend you read on ...

A very

Q: Hi Jens, Tom said that you've been using Scrum combined with EVO. Can you tell me a little background about the project and what problem EVO solved that Scrum didn't?

I’ve been studying and using Evo for the past four years. First as an employee in the Norwegian Product Register (governmental) after attending a course with Tom Gilb,, and later as a consultant in one of Norways largest IT corporations (EDB ASA (Avenir)). Avenir has a rather complex and “heavy” framework for managing large waterfall like projects called AvePro (“homemade” PMI inspired) that they have been working on and used for 20 years.

One of my first assignments in Avenir (in December 2006) was when Microsoft asked us to be an implementation partner for a Norwegian telecommunications company that had decided to implement Microsoft CRM 3.0 as their standard CRM platform. The requirement specifications that where presented to us where purely functional, and there where no references to what goals or business values they wanted to achieve by implement the system. I quickly concluded that our traditional methods of project management would not cope with such a project. One of the reasons where that I suspected that if we just developed a system based on the specification that we had received, the changes in requirements during the course of the project would turn the project into one giant contract nightmare, and that delivery of the system would be infinitely delayed. It would simply join the majority of all other software development projects in the basket of failure.

We agreed to do a one month test period with one consultant (me), kind of no cure no pay, to qualify us and the system for their needs. I told the customer that with the method I intended to use, we were quite sure to give at least some stakeholder some value, and that this value would be far beyond the cost of the initial implementation of the project.

To do this I had two simple requests for the customer. I wanted one workshop with the sales managers ad the director of sales, and one workshop with the employees from their internal IT department that where suppose to participate in the project.

I started the workshop with the management by asking two simple questions:
1: What is this company's vision?
2: If you don’t think about IT or software at all, what are your greatest problem today, that prevents you from fulfilling this vision or goal?

(BTW, I’m not of the very formal types, so I attended this meeting with the management in my usual work outfit; T-shirt and jeans – in Norway that is almost perfectly okay)

At question #1, they answered a bit staggering, and after a short while, they said with a little laugh that is was their vision to be a leading telecommunications company in Norway. Laughing as if they where embarrassed by this vision (I call this Norwegian modesty). I thought that was an excellent vision, and a good starting point for our project. No point in setting the goal to low. If you’re aiming for the moon you are more likely to reach the sky than if you only aim for the roof of the nearest building.

At question #2, the director of sales had a clearer view. He said, and I quote: “We are loosing xx million Norwegian kroner each year on contracts that are ending and lost, without even trying to win them back, or prolonging them. Nobody knows they even exist”.

I answered: Okay, so if I implement a function in your system that alerts the account manager of a customer, let’s say, 2 month’s in advance that a contract is ending, you will potentially earn xx million kroner more?”. He answered yes.

This was actually a function that they could have implemented in their existing system at any time, but the management of sales had never discussed this business related problem with the IT department, and the IT department where probably to busy surviving bugs in the existing systems to listen to the real stakeholders everyday problems.

I told them that I could implement such a function for them in only 14 days and where met with a short silence... Then the director of marketing said: I never thought I would hear such a thing a IT consultant”.

In the workshop with the resources from the IT department, we discussed all the practical tasks necessary to install and configure the CRM database and server, and some other technical stuff. Just enough for me to gain their respect, and to make them know that I really knew what I was talking about when it came to programming and technical issues.

I also told them that to be real heroes to the management, they had to deliver solutions that really made some increased business value for important stakeholders in the management. Cool features might be impressive, but they are short lived and soon forgotten. Business value is sustainable. They where very skilled resources, and had many years of experience, but they had never thought of it in that way.

I told them: When your boss ask you to do something for him, you must ask why. Only then will you know what his real goal is, and only then will you be able to really help him. The answer to his question is never yes or no, it is rather one or more design ideas with an explanation, a cost estimate, and an impact estimate on his goal and other relevant impacts (good or bad). This information will make your boss able to make a smart decision on whether the solution should be implemented or not, and it will make the IT guys real heroes in the eyes of the management.

After working in this way for a month, creating business value they really needed, and showing the potential of the method, they prolonged the contract, and wanted more resources from our company. The total number of hours for this project became about 5,300. In Norway, that is considered a mid size to large project. We kept delivering business value to the customer in small weekly (!) increments for 15 months.

When the number of people working on the project increased, organizing the work became challenging. Mostly because of the rapid increments in the project, and that not all of the developers working on the project where experienced, and needed more follow up. Evo is a great method to break down top level goals into business value and end state requirements, but contains no mechanism to organize the developers, and how they work. Although design specifications where made in detail, we had trouble with developers that where developing outside the scope of the project, and struggling for days with problems that weren't important to solve at all, without telling anybody (also called thrashing?).

To get the necessary control of the production environment Scrum where partly implemented with one team, daily stand-up and a product backlog. That product backlog where populated with design specifications from the Evo cycles. This, I found, was an ideal combination. Now I had total control of the most important requirements, expressed as the business value they wanted to achieve. A breakdown into end state stakeholder requirements. Mapping between business value and designs that where implemented. The stakeholders received business value for their money (bang for bucks), and the programmers no longer wrote code; they created business value!! Their work had suddenly got meaning, and they loved it.

As far as my knowledge goes, no single method widely used today covers all aspects of a project, from Vision to values, values to requirements, requirements to design, and design to working systems.

All methods seems to cover the “project layer” where the author of the method works, and maybe one layer up and one layer down. To cover all layers you will have to use more than one method, and select methods that have compatible interfaces, such as Evo and Scrum (Evo design specifications into the Scrum product backlog).

Avenir has now added this methodology (in a modified form) as one of our preferred agile project delivery methods (Avegility). It’s built up as three parallel cycles named “Goals and requirements”, “Production (ie Scrum)” and “Delivery and measurement”.

Q: How did you learn enough about EVO to get started? Was that easy?

As mentioned earlier, I first learned about Evo when I attended a course held by Tom Gilb, and I must admit that the first two days of the course, I did not understand where he was going with this at all. After the course I tried it in a project at work, and I was truly amazed with the result. Lifting my eyes from the code and the software, and focusing on the required results opened up a whole new world of possible solution for me. As a consultant for a software company I must admit that this sometimes is a problem; when looking at clients requirements and goals today, I far too often sees that the best solution seldom is buying or developing new software, but to begin with the existing software, or the way they work.

If you got that business gene, and a true passion for the results you create, living the Evo is easy. If code, technicalities and gadgets are all you live for, you will probably not get the point at all.
Some people never realize that the best system is the one delivering the best results for the business, and not the fastest, prettiest and the one with the perfect architecture (although we have to care about those things as well, but then as expressed quality and performance requirements related to business values and goals)

Q: Did you have to change Scrum much?

I suppose I get a lot of people angry now, but in this particular project, because of the short cycles, we removed the Sprint term, and worked continuously with developing and delivering items from the back log, meeting every day in the daily stand-up. In the formal method Avegility Scrum is unchanged but for one thing: It’s not the product owner that is responsible for handling and prioritizing the product backlog, it’s the Evo cycle’s output in the form of design specifications. Cost/benefit ratio decides the priorities.

Tom Gilb - Q4

2008-05-06T10:37:44+01:00

Q4. I first met you at XPDay2004. You were one of the keynote speakers. I loved your talk because it resonated with my own concerns about "Agile". Can you elaborate on those concerns? Do you still have them?

You bet!

For readers who were not there, the slides are at

Paper: http://www.gilb.com/community/tiki-download_file.php?fileId=50

Slides http://www.gilb.com/community/tiki-download_file.php?fileId=170

The good news is that some people who have practiced Agile thoroughly, now see my point - and realize that if we are to deliver value to our customers, we need to add to the conventional Agile vocabulary. We need to be far more explicit about what value improvements our stakeholder need, and how we are going to deliver them.

'Stories' and 'Use Cases', Features and Functions - are nowhere near what we need to deal with. Quantification of Values (Qualities) is essential. The Agile boys got the message about small iterations OK, but they completely ignored the message about quantified quality requirements. I put that down to immaturity. We are all immature at some stage. It has to be put right in time. Rapid delivery of the wrong values is still wrong.

A great example of a Scrum Master who 'gets it' is a friend I met at the Agile Business Conference in London, Ryan Shriver:

Ryan's home page www.dominiondigital.com

Slides… http://www.gilb.com/community/tiki-download_file.php?fileId=148

Goalkeeper Tool http://goalkeeper.dominiondigital.com

I have a very simple message to all those failed projects for which we are so famous, and will be more infamous:

1. Quantify the critical stakeholder values (current Agile culture does not understand 4 of 5 of those words)

2. deliver those values early and frequently. (delivering value to stakeholders is NOT the same as delivering code or a story)

The survivors will get it :)

My favourite theory of how to change our world: "no cure no pay"

No Cure Paper http://www.gilb.com/community/tiki-download_file.php?fileId=38

Slides No Cure http://www.gilb.com/community/tiki-download_file.php?fileId=85

I cannot believe there are so many managers, 99.999% of them, who prefer to pay for effort and failed results, when common sense says they should only pay for good results.

People called 'programmers' sometimes crowning themselves 'software engineers' , have learned to fleece these rich managers thoroughly. A fool and their money will soon be parted. Hang in there guys, it may soon be illegal!

Well, I'd better quit before I insult too many people, after all they have a right to use their company's money any way they want to?

Warm Wishes to You All

Tom

Tom Gilb - Q3

2008-05-06T10:36:47+01:00

Q3. Can you tell us a little about your personal life?

Yes.

Ohhh you mean will I detail it here now?

Well I can tell you it has been fun - and to protect innocent people I can't tell you all the details!

The essence starts when I was 17 and met this wonderful Norwegian girl of 18 in London. She is sitting next to me in my living room as I write this. True Love = Endureth Long! (Corinthians 1 again). I was in my first year of 6th Form (boring). So of course I got engaged, moved to Norway and got a job at IBM just by asking! All really great decisions. :)

I mean who knew that Norway was going to be the richest country with the best quality of life in the world?

I continued my studies at University of Oslo for about 10 years, on the side of full time job, and a family from 1964. Sociology, Economics, Political Science, Philosophy. What fun - and I did it for fun, not for a job. Yes it is all useful for me today. I realized that I would not be a good bureaucrat at a University - and also realized that 95% of University people dislike me or my ideas. That is probably a good sign. The key is who the other 5% who do!

Thank goodness I never had to do a Thesis approved by the 95% who dislike my ideas! So I decided my books would be my 'Thesis". My ideas are evaluated by the readers - and I have full respect for that - even from those who feel they do not like my ideas. At least they had a look at them!

We have 4 boys and 5 grandchildren. We live just outside Oslo in a new apartment, half hour away is our amazing beach-front cabin on the Oslo fjord, and we have a "town cabin" in Covent Garden - which is useful as we love to go out in London, and visit my London Family.

In January 2008 the Norwegian Government decided to pay me a nice sum (which I have apparently been forced to save for 50 working years) every month for the rest of my life. I still continue working with my son Kai - but I am even less focussed on earning money, and more focussed on fun, travel for me and my wife, playing grandfather, and writing creating and spreading the word. I like the model of my friend W. Edwards Deming (Statistical Process Control) who held his last lecture at 93. I like to tease my listeners that I will teach their grandchildren when they are retired.

I enjoy perfect health, which I know is a gift at my age. No, I never smoked.

I am Norwegian Chairman of the Board for The Art of Living Norway (artofliving.org) which has made available to me a lot of wisdom, and contact with a younger generation.

I think I believe, but cannot measure or prove, that we had past lives and will have future lives. This explains some things to me - but I might be wrong - so I am doing the best I can in this life. If I can get back to you afterwards and confirm my hypothesis, I will. Watch this website. Failing that I hope we can discuss this 'afterwards'.

One consequence of this hypothesis is that maybe my ideas are not as 'original' as I might like to believe: maybe I either 'remember' ideas, or are fed them by spirit guides? A least I get to keep the income and royalties. And spirits don't sue for plagiarism! Hey with a bit of luck I will either remember, or find an old copy of, my books, when I am young in my next life, and can follow Newton's idea of standing on the shoulders of giants. Well, even if I can't - you can!

My philosophy, from 17 years old: do what you love,       do it well,    don't let money dictate what you do - you will survive.

Tom Gilb - Q2

2008-05-06T10:35:34+01:00

Q2: How did your life change after you published the "Principles" book?

Not much. My working life has been similar for almost 50 years, teaching, lecturing, consulting, writing, reading, networking, inventing, travelling - running my own company. So much fun - who would want to change it!

Tom Gilb - Q1

2008-05-06T10:34:48+01:00

Tom Gilb is one of my Agile heroes. He's been doing and teach agile for a looooong time, but many Agile newbies may not recognise his way as the agile way.

Q1: Hi Tom. I started University in 1988 ... which is the same year that your classic book "Prinicples of Software Engineering Management" was published. According to my calculations that means your book has now been around for 20 years. I only came across it a few years ago but it has influenced my thinking more than any other "software development" book. I know of several of the better known Agile guru's who say similar things. Can you tell us a little (or a lot) about what lead you to write the book and where your ideas came from?

Yes PoSEM is now in 20th Printing - I was asked by my publisher to make an 'evergreen' - and apparently it is!

I like to think my 2005 book, Competitive Engineering (CE), is a worthy successor. It treats the same subjects, but more deeply. It is not such an easy read, but we have other books to serve that purpose including Kai Gilb' s "Evo" book       http://www.gilb.com/community/tiki-download_file.php?fileId=27

It was important to me that the CE, which officially defines Planguage, was reasonably rigorous. Time will tell, but it is in Second Printing and is well reviewed, so there is hope for the future!

What led me to write the book?

Well, let me remind you of the book's predecessors:

1976 Software Metrics (coining the term, and the official foundation for IBMs CMM Level 4 [I don't take all blame for CMM!]). In USA 1977.]}

1974-5 Reliable EDP Application Design, and Data Engineering (neither in USA, but both in Sweden and UK).

These were the real beginnings - PoSEM was second generation!

You might say the centre of my technical universe is 'quality quantification'.

The driver there is - the Lord.

OK!    Lord ..... Kelvin.

About 1965 the following appeared in a Norwegian newspaper: (I joined IBM Norway 1958, at 17)

"I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it;

but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind;..."

http://zapatopi.net/kelvin/quotes.html,

That got me thinking about whether we could quantify critical quality properties like portability and security. I had no internet to search, and my international personal network from conferences was a good Who's Who (Dijkstra, Langefors, Naur, Nygård, Dahl) - so I tried common sense and invented and published some ideas on how to quantify them. I also quickly found out that they worked in practice.

As I worked as a consultant, I got challenged with extending the vocabulary, for example extensive work with ICL UK in the early 1980's led to defining Adaptability, and from 1980 with IBM led to defining Usability. As late at 1995 at Ericsson (Erieye airplane) we extended Usability with Intuitiveness and Intelligibility. And these and others became stable, and became patterns for tailoring local variations. More recently my friend Lawrence Day of Boeing pointed out to me that my method is in the Bible, 1st Corinthians, defining Love by decomposition into several quantifiable factors ('endureth long'). And I realized that Rene Descartes was on to 'my' method centuries ago.

Another early inspiration was a UK Professor Christopher Strachey, Oxford University Computing Labs, in the 1969 NATO Science Council "Software Engineering Techniques" Proceedings page 65) http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF

Christopher Strachey (Oxford) : “Computing science has been under some attack on the grounds that it isn’t software engineering. I propose to attack it on different grounds. I think we should seriously ask ourselves the question: is computing science?

Recently I did a small survey as to whether computing is suitable as an undergraduate subject in an English university. I did this by grading all the topics I could think of under the headings of relevance and state of development. The premise is that it is clearly wrong to teach undergraduates the state of the art; one should teach them things which will still be valid in 20 years time: the fundamental concepts and underlying principles. Anything else is dishonest.

The gradings for relevance ran from “clearly relevant and essential” to “part of another subject” (like numerical analysis) and those for state of development from “well developed with theorems, laws and text books” to “a fruitful field for research”. Note, incidentally, the importance of text books. They are designed to be taught from; they are quite different from treatises and even further from research papers. Now, it turned out that all those subjects which score highly for relevance score very low on state of development and vice versa.

Until we have a sufficient body of topics which are important, relevant and well developed we cannot call the subject a “science”. I am quite convinced that in fact computing will become a very important science. But at the moment we are in a very primitive state of development; we don’t know the basic principles yet and we

must learn them first. If universities spend their time teaching the state of the art, they will not discover these principles and that, surely, is what academics should be doing.

I do not for a moment underestimate the importance of the state of the art in engineering. Clearly it is essential and furthermore from engineering practice we must get our experience and material from which we develop theory. But, before teaching students we must get our basic principles right." [Strachey 1969]

Well, I found he was right. There were not many principles published or available (Langefors (Theoretical Analysis of Information Systems) and Naur had a handful).

So, I drafted a number of principles, based on my 'engineering experience'. About 124 principles were published in PoSEM (1988) and at least 100 Principles in CE (2005).

I like to think they "still be valid in 20 years time". In fact we can now test the 1988 Principles - would anyone like to argue they are not valid today?

My own arguments for subjects that will "still be valid in 20 years time". is in my paper Undergraduate basics http://www.gilb.com/community/tiki-download_file.php?fileId=98

The paper argues that principles, concepts and measures are primary tools to learn about.

A few academics have understood this, but not many, as far as I can see. I also see the consequent helplessness and waste in software projects that seem to me to be the consequence of the lack of fundamentals. We cannot seem to get decent quantified requirements at the top level for large projects - ever. We never seem to get past step One.

Q1 - The Poppendiecks

2008-04-11T09:07:35+01:00

Q1. Hi Tom and Mary, I suppose I should have started this blog with you two since it was your book (actually, an early draft of your book) that got me into this business, but I didn't, so please forgive me :) It must be about 5 years now since you started writing the book, but can you step back even further than that and tell me about your lives before that? Several people have emailed me to complain that there's too much techie stuff in this blog and not enough romance, so how'd you two meet? Was it love at first sight? I'd also like to hear more about your "pre-LSD" lives - correct me if I'm wrong, but your resumes don't hint strongly that you were both headed towards gurudom ...

Mary replied:

Life before Lean Software Development

When the professor was calling role the first day of our college physics class, he asked some guy in the front: “Are you the one who won the high school science fair a couple of years ago?” At the end of the class he asked us to choose a lab partner, and since I didn’t know anyone in the class, I made a bee line toward the guy in front who was smart enough to enter and win a science fair. Meanwhile, that guy was looking around for a female lab partner, and so we met. It was certainly not love at first sight, it was all about getting through physics class with a decent lab partner.

But by the end of the semester, we were serious about one another, and I had finally learned to spell “Poppendieck”. The professor had graded all of our lab reports and put them in a pile for us to claim. As Tom and I dug through the pile to retrieve our lab reports, time and again Tom would laugh and say – look at how you spelled Poppendieck this time! Yes, there are many ways to spell the name, and when our kids were young, they discovered that there are also a great number of nicknames that can be derived from Poppendieck. When they complained, I would tell them “Talk to your father. He grew up with the name – ask him how he handled it.”

I proceeded to get a masters degree in mathematics (before computer science programs were widely available), while Tom supported us by teaching physics in high school. Then he went on to get a PhD in physics, while I supported us by working as a programmer in the physics department at the University of Wisconsin. We lived in married student housing, where the apartments were so small that we refused to share our meager space with a TV. Parents traded babysitting script for evenings out, and Tom took our daughter to a day care center on the back of his bike.

My job was to teach one of those new things called mini-computers to digitize bubble chamber film and send a preliminary analysis to tape for later number crunching on the big Univac computer on campus. Tom got his PhD by analyzing the (100) surface layer of silicon in ultra-high vacuum. Eventually Tom ended up getting a job as assistant professor at Hamline University in St. Paul. I asked my local DEC salesman in Madison: “Who’s buying DEC mini-computers in the Minneapolis area?” and I quickly found a job in the engineering research department of 3M, exploring options for mini-computer control of processes.

After a few years as a physics professor, Tom found himself bringing the first computers into Hamline University. He remembers crawling through the heating tunnels stringing cables, and eventually helping the school establish the policy that all students should have access to a word processor. I was glad that Tom was a professor, because I was traveling quite a bit starting up process control systems in 3M plants. Professors have free time to take kids to the doctor and dentist, and they can even stay home most of the day when kids are sick.

But then our jobs changed. Tom’s university decided to formalize the job he was doing and call it “Director of Academic Computing”. He was asked to apply for the job. I suggested that if they were going to have him apply for his own job, he might as well send out some resumes and see if anyone else would want to hire him. Within two weeks, Tom had a job working for Honeywell designing a product data management system for a division which developed inertial laser gyroscope based navigation systems for commercial airplanes.

At about the same time, I was offered a job as IT Manager in a nearby 3M plant where I had just overseen the installed a process monitoring system. This was my first management job, and my first encounter with the quality movement and with just-in-time. Tom encountered the total quality movement, activity based management, and just-in-time at Honeywell at about the same time.

After a few years, I returned to 3M headquarters and worked in new product development, while Tom went on to become a software consultant. When 3M was spinning off Imation, I was offered an early retirement package I couldn’t refuse. So in the late 90’s I abandoned Minneapolis for a year and moved to California, working for a start-up company specializing in high purity polymer that I had taken an equity position in while I was at 3M. Meanwhile, Tom traveled extensively, trying to help companies as far away as Brazil use IBM’s “SanFrancisco” framework. We ended up in the same city every weekend – sometimes home, sometimes California, sometimes Las Vegas, or wherever. We had two tandem bikes – one in California and one home in Minneapolis – and we usually went on a long bike ride together every weekend. Our vacations were almost always week-long bike rides across some state or other.

Back home in Minneapolis, I was looking for some way to make a bit of money for a few years, after which my pension would kick in and pay the bills. Tom said there was a great need for project managers, and I figured that I could get back into software development easily enough, having plenty of background in programming and IT management. I started up our business and got a job gathering requirements (using JAD) for a state government project. JAD I understood, I had used it before. This thing called “Waterfall,” however, was new to me; I couldn’t figure out how that could possibly work – and actually it didn’t. I ended up as the third project manager of that project, trying to rescue it when it got into trouble, with mixed results. When it was over I decided to write a book about how ideas from the quality movement and lean manufacturing offer a better way to develop software.

Part way through the book, an economic downturn caused Tom’s consulting firm to falter and thus Tom joined me as author of the book and as a partner in the business. At first we worked separately, but one November I was invited to give a talk at XP Days in London. Airfares to London were so cheap that I thought it would be nice if Tom joined me. That trip turned into a whirlwind tour of Edinburgh and Malmo, and as we were struggling with the luggage and the logistics I said to myself: “I was going to come here alone? How dumb was THAT?” And we have never traveled separately overseas again.

We have found that we love to travel and see new places, we love to meet people and talk about lean software development, and we are very fortunate that people want us to share our insights with them and give us more insights to share with others.

Q7 - Jared Richardson

2008-03-17T16:25:06+00:00

Q7 - Tell me - or really, the folk who've not read your book - a little more about the ins and outs of Ship It! What's in it for them for reading it? Who will enjoy reading it? Who won’t enjoy reading it?

Ship It is our take on the entire software ecosystem. Often a shop that does a great job in many different areas will have huge blind spots. Maybe a team has done a great job at following a process but ignores the infrastructure. Ship It! covers three major areas. Techniques, infrastructure, and process. You can see this in our poster of key practices http://media.pragprog.com/titles/prj/color_poster.pdf. The book also has a chapter of Frequently Asked Questions that talk about common scenarios you'll run into and how to use the key practices to solve those problems.

But it's not a religious diatribe on "The Right Way" to work. We tried very hard to introduce you to the reasons and ideas around following a process, then we suggest one you can use. I'm more about getting teams to use the ideas than to work like Will or me. We also covered the "what happens when you ignore this area" too.

Who should buy Ship It? The book was written as a guide for developers who wants to make a difference on their team, but it's been very popular with project managers and tech leads as well. In fact, it's often categorized as a project management book. I that's because there aren't a lot of ecosystem books out their and no one knows how to categorize it. :) It's not Java or C#. Must be.... project management!

Clarke: That's my last question for Jared for now. I've found Ship It! extremely useful and I thank Jared and William (his co-author) for their wonderful contribution.

Q6 - Jared Richardson

2008-03-16T10:34:48+00:00

Q6. What surprised you most about other peoples reaction to Ship It!

Two things have surprised me the most. The first is the broad range of reactions. Early in the review process, I got three emails on the same day. The first said "I can't believe you suggest these practices. Everyone already does everything you talk about." The second mail said "The book is too theoretical. The practices are great ideas but no-one actually does these things." The third mail said "Wow! I knew most of these practices, but I've never seen them put together so well. And I needed to be reminded of the ones I knew anyway. Thanks!"

And that sums up people's reactions. Some love it. Some don't. And that's okay. No single book will reach every person. We've reached a lot of people with Ship It! and that feels great. It's now in German, Japanese, Korean, and there's even an India sub-continent version. I get email fairly regularly from people who've retooled some part of their shop's environment around Ship It! It's a great feeling to know you've had an impact on people you've never met.

The second thing that's surprised me is it's sustainability. It's been almost three years and we continue to sell more books every quarter. It's a slow climb, but it's still moving up. We worked hard to keep Ship It! language and technology neutral, and I think it's worked.

Q5 - Jared Richardson

2008-03-16T10:33:57+00:00

Q5. I give your book away a lot. Tell me about the books you give away or recommend.

Thanks for that! I can't tell you how many times I talk to someone at a conference who came to hear my talk because they were given a copy of Ship It! by a friend or co-worker.

I interact with a broad range of companies and technologies, so my book suggestions vary widely. Here's a quick sampling.

If you've got someone trying to learn the Java stack, point them at Scott Davis' JBoss at Work.

Learning Ruby? Dave Thomas and Andy Hunt's Ruby Pickaxe book and Hal Fulton's The Ruby Way.

Rails - Dave Thomas's Agile Web Development with Rails

Ajax? Justin Gehtland's Pragmatic Ajax. And the Prag Press has new Prototype and Dojo books out as well.

Groovy? Check out Venkat Subramaniam's Programming Groovy

Grails? Jason Rudolph's Getting Started with Rails.

Andy Hunt just released his "Refactoring Your Wetware" book. It's all about your soft skills... the stuff in your head. I think it's something everyone should read.

Q5 - Chris Fry

2008-03-16T10:23:56+00:00

Q5: So, what approach did you take to the rollout? Did you get help? And how long did it take?

Initially we planned on doing a few teams, one at a time, and learning from our mistakes. After talking it over with our R&D leaders we decided that the commitment to the change and the need were great enough to justify an "all in" approach and decided to move the entire organization to one methodology. Mike Cohn has written about the "all in" pattern here http://www.agilejournal.com/articles/articles/patterns-of-agile-adoption.html

Some of the advantages of the all in pattern are: 1) Management demonstrates commitment to the new model; 2) The transition is quick; 3) There is no time where you are supporting two styles of work, everyone is committed to the change; 4) You deliver value early and are able to demonstrate it to the organization and the customer base; 5) It reduces the risk of one team working in a waterfall approach; 6) You are able compare teams going through the transition at the same time. There are some drawbacks but it’s a great model when it works, as it did for us.

We did get help. Mike Cohn gave us great advice when he said the "all in" approach would cost us more. We got a lot of help from Pete Behrens who organized agile coaches for us and gave us great help throughout the transition (see http://trailridgeconsulting.com/blog/?p=96 ). We sent Scrum Masters to certification and had Mike Cohn run in-house training for our Product Owners. All the training we invested in really helped smooth the bumps for the organization.

It took us 3 months to transition the entire team to agile releases. Change is never done, I would say we are still constantly improving, learning from our mistakes, and introducing new agile techniques each month.

Q4 - Chris Fry

2008-03-16T10:22:52+00:00

q4 - How big was the rollout?

We moved thirty teams in three months. It was basically the entire product R&D organization inside salesforce.com. You can read our agile 2007 experience report here http://www.cfry.net/docs/cfry-agile-2007-final.pdf

Q3 - Chris Fry

2008-03-16T10:21:53+00:00

Q3: Can you tell us about what led up to the decision to "go agile" at salesforce.com?

The decision to go agile at salesforce.com grew out of a desire to create shorter more predictable releases. We had gone a year without a major release and wanted to get onto a more predictable release schedule that would deliver value at a consistent rate to our customers. Salesforce.com is a young internet company and an agile methodology was a great fit our culture. A big driver for us was to move away from a process that was predictive and move to an empirical model. We also like to do big things quickly so we decided to go "all in" and change the entire product development team at the same time which was very aggressive. Steve Greene was the Scrum Master of the rollout team and I was the Product Owner. The rollout team was a motivated, volunteer, cross-functional group of people that were aligned on the move to the agile model and principles. The results have been extraordinary and we will be presenting them at the annual Scrum Gathering in Chicago this April.

Q2 - Chris Fry

2008-03-16T10:20:17+00:00

Q2: Did you have an agile "lightbulb" moment? Can you describe what led to that momement?

The agile lightbulb went off for me when I was managing my first team at Weblogic (BEA Systems). There was always a strong automation culture and engineering practices inside Weblogic. I picked up Agile as a way to run my first team when a friend recommended a couple books on the topic. Looking back I had a lot to learn but I remember thinking the Agile principles matched my personal philosophy about running a good software team based on automated testing and cross-functional collaboration. The flexibility and responsiveness of the agile model provided a great framework for building great software.

Q5 - Alistair Cockburn

2008-03-16T10:03:09+00:00

Q5 - Tell me about your consulting work. What do you like most about it? Least?

This one should be short :).

What I like is that I get control my own time. I can say 'No' once my travel schedule hits a certain threhhold (50%). That was actually the reason I quit IBM – they put me permanently in the field, and I started traveling 5 1/2 days a week, coming home Friday night and leaving Sunday afternoon. I lasted 3 weeks and then turned in my resignation. The primary immediate advantage I got was the ability to say No.

The second advantage, related, is that it's my own business, so when I work overtime, I know who's benefitting from it – and it's me! (Not some corporate stockholder or exec shooting for a bonus). Then, when I'm not actively working, I know what I'm dong and why – I'm changing baby diapers (OK, that was back in 1994), helping kids wth homework, going shopping, or going swimming. When I was an employee, that was all "the other me, the one outside work". Nowadays, it's all "just me", whether I'm filing papers, doing bills, shopping, consulting, teaching, writing, hiking, inventing flip-flops, whatever. There is "work me" and "home me", there's only Me. That's a really freeing feeling.

On the downside, my brain can never shut off. I call it "riding the fences" (rancher duty – making sure cattle can't go through a hole in the fence). It's a little process that runs all the time. I can't stop it. Invoices, clients, the work pipeline, the bills. I've met other small company owners who tell me the same story.

The other, and from my perspective, biggest downside, is that I don't have professional colleagues to trade notes with and learn from. It turns out that keeping up with all the new technology is done much better in an office situation – there's all this osmotic communication about the new browser APIs or Google's new products or something about the Eclipse interface or ... or ... or ...

Taking it all together, I'm much happier being on my own.

Q4 - Alistair Cockburn - The Books

2008-03-16T09:58:40+00:00

Q4 - Tell us about your books. Which order should I read them in and what would I get out of each of them? Which one are you most proud of?

Well, there's something like 8 books (depending on what the definition of 'is' is, and 'book' and 'your'). Here are some unpublished notes about them ...

Surviving Object-Oriented Projects was a summary of my investigations and project debriefings in the early 1990s. It contains snapshots of the projects I debriefed that led me to to recommend the kinds of things I do. Very little of it is specific to object technology, most of it is about what makes projects succeed and fail, how to staff for experimental versus production projects, project management strategies, and of course, how object oriented programming affects things. I still meet people today who tell me this is their favorite book of mine.

I succeeded quite well with it, with one exception. I wrote an appendix at the back called "Project Management Patterns". One of the reviewers (in 1996) complained bitterly about me getting on "the patterns bandwagon" and suggested strongly I drop the term "pattern".

Seeing that "pattern" is synonymous with "strategy", I did a global search-and-replace of "pattern" to "strategy" (and "patterns" to "strategies"). The surprise was that there were literally no further editing changes to clean up the wording change! When I submitted the appendix to PLOP 96, I changed the words back to "patterns", and after incorporating their recommendations, changed them back to "strategies." Try it – you'll find that most patterns are nothing more than strategies!

However, it turned out that my reviewer was completely wrong! and I was a fool to listen to him. Had I stuck with the word "patterns", I could have kicked off a project-management patterns movement, whereas as it now stands, few people have any idea that there are a dozen project management strategies sitting in the back of that book. Foo.

Nonetheless, I got permission from my wife to put all the proceeds from the book into a separate savings account for a little sports car. In 1999 I had $20,000 in the account, so I went to the local Mazda rep, showed her my screen saver image of a silver Miata, told her I had exactly $20,000 in the bank from my royalties, and lo and behold! She turned up with a silver 1999 Miata that, after taxes and registration came to $19,986. My son and I picked it up and bought ourselves (a small) dinner on the way home with the remains of the saving account.

– Note on creating a macro-structure –

I spent weeks drawing pie charts and mind maps of topics to make sure that it contained the full range of topics without getting stuck in any job speciality. I'm not really a mind-map person, but I tried it. What worked for me in the end was a three-layer pie chart, the inner pie showing % of space allocated for the major topics (chapters), surrounding circle showing % of space for major headings, some third-level breakdowns in a third surrounding circle. From these percentages I could calculate how many pages to spend on each topic (I knew I wanted a 300-page book, so I just did the math, then wrote that many pages on each topic.)

I can recommend this as one way to structure your book. Keeps it to size (and I know _many_ authors who could benefit from this discipline).

– Note on writing style –

I used to write really complicated sentences (still do, given half the chance). I ran across the book Revising Business Prose, a short, small easy-to-read, 80-page paperback, which taught me to write short, active sentences. I took one entire pass through the book splitting compound sentences, choosing shorter words and eliminating adjectives and adverbs. The book was much the better for the change. Try the book, I believe you'll like it. (p.s. I only used the advice in the first half of the book.)

– Book 2 –

Crystal Clear was the second book that I wrote (1999). However, XP was just on it's ascendancy at that time. I decided I couldn't compete with it's popularity, so I stuck the book draft on my web site and let it sit for five years, until friends pestered me to get it published just so some people would see that there is an alternative to XP.

Amazingly, a professor Chris Jones, then at Weber State University in Utah, found the draft on my web site, downloaded it and made his students use it for their semester projects - And he never even wrote to me! I found out years later when talking with his son Nate during an agile roundtable. Chris Jones later used my brand-new undergraduate software engineering text – while I was writing it, in real time! Talk about someone willing to experiment.

– Book 3 –

At that point I signed a contract to write a book series! Which meant I needed to write the zeroth book in the series, the anchor book. This was originally entitled Software Development as a Cooperative Game. I started drafting it in 1999.

– Book 4 –

But while I was drafting that, I was being deluged with catastrophic book proposals about use cases. I decided if I wasn't careful, I'd find myself having to teach out of them. Out of desperation, I scraped my hard drive for whatever I had on use cases and overnight submitted a 100-page book proposal for Writing Effective Use Cases. That was put on the front burner and came out late in 2000 – and has been paying my mortgage ever since.

– Note on macro-structure –

I'm famous for not finishing books. I generally run out of energy by half-way through the book (as, for example, Revising Business Prose, which is really short, but I still only read half of).

So for each of my books, I've worked hard to arrange that all the really important stuff happens in the first half of the book, preferably before page 100. The hard part, of course, is what to fill the second half with. I toyed with rubbish generators, but concluded that some readers would actually read the back half of the book and complain. So I fill the second half of the book with helpful-but-not-critical information. Dedicated readers can then mine the back half for extra goodies.

– Book 3, again –

I got back to The Cooperative Game, whose writing was redirected by the writing of the Agile Manifesto. The publishers renamed it Agile Software Development, even though the book is much more general than just agile.

What was fun about this book was that I got to design the page size, 2-column layout, typography, and even ran its production. We had a tight deadline to get it out by OOPSLA 2001, so I used all the advice in the book in doing the copyediting, illustrations, layout, etc. We shrank the time-to-production from 5 months to 5 weeks!

For example, at one point the copyeditor was resisting meeting me for a discussion. I started to explain why it would be useful to meet. She said, "I know all that – I'm editing that chapter!" :-)

The first edition of Agile Software Development was and still is my favorite book, because of the Introduction and first two chapters. It was the book I had long wanted permission to write, because in it I got to talk about the nature of communication and humans. Without the first two books behind me, I'm not sure I would have felt brave enough to write it.

– Note on macro-structure –

I had trouble getting the macro-structure of this book. The pie charts didn't work. What solved it was Writing For Story. Where Revising Business Prose teaches how to edit single sentences, this book teaches how to construct the macro-structure. Using it, I constructed the Agile book as a sort of mystery story, where every chapter resolves certain tensions but introduces new tensions to resolve. Take a look at Writing For Story and see.

I failed on my strategy of making the second half of the book "goodies", although I did get the most important information in before page 100 (my publisher wrote me just as the book was gong to press, "I'm just readng the book now – I'm up to page 80, and you still haven't gotten to the meat!" I wrote back – "That IS the meat!").

So this book is really two books in one – the first halves of two books, glued together.

– Book 5 –

All this time, Steve Adolph, Paul Bramble, Andy Pols and I had been trying to write a patterns version of Writing Effective Use Cases. I was also working on my doctoral thesis at the University of Oslo.

It was all too much, and I withdrew from the Patterns for Effective Use Cases after we worked out the pattern language. Steve and Paul invited my back onto the author list just before publication, but I thought that would take too much attention away from the work they had done, so Andy and I put ourselves as "contributors".

– Book 6 –

My doctoral thesis, which I completed late in 2002, was entitled People and Methodologies in Software Development. It was intended as the academic version of Agile Software Development, just as readable, but with the necessary attention to research method and full of researh citations. I gave my Agile book to the examining professors, and my thesis to Addison-Wesley. AW never got around to publishing it :-(.

Part of the point of writing the thesis was to let people who had read Surviving OO Projects and who had heard me rumble on for years about my project debriefings, get a closer look at what those debriefings and interviews looked like. It was also to answer the question, "Can methodologies ever converge?" (The answer is No, we'll need an ever-increasing number of them, which means they should be really really easy to construct).

– Note on macro-structure –

I rewrote this thing 3, 4, or 5 times. What cracked the problem was Lars Mathiassen cornering me and saying – "A thesis should answer three questions. You know those answers, don't you?" I said, "Yes." He said, "What are the three questions those answers answer?" I told him. He said, "Good – Start by saying what the three questions are, then answer them."

Egad! So easy! (after the fact). That's exactly what I did, and I still like the structure of the book: No windup, it just starts running at full speed. I wish more dissertations did that.

– Book not! –

My next book was supposed to be the project management patterns book, but the Agile Development Conference got in the way – I spent two years creating and babysitting that before the AgileAlliance took it over, and by that time I was already past the PM patterns topic – the book had "written itself" in my mind and my body didn't want me to take the time to write it onto paper. Shucks. I really miss that book.

– Book 2, finally! –

Finally it was time to dust off the Crystal Clear book and blow the cobwebs out of the words. It had been intended back in 1999 for people who didn't have any notion about small-team, incremental-iterative development. What a change in the intervening five years! By 2004, thanks to XP and agile, those topics didn't need any defending.

Crystal Clear: A Human-Powered Methodology for Small Team is currently my second-favorite book, because it covers pretty much the full set of topics needed by a team, with concrete examples of each artifact needed.

While writing it I discovered that project properties are much more powerful than methodology rules. Instead of saying, "Follow these rules, and good things will show up," I found it stronger to say, "Here are some good things to have show up – Ensure that they do!" And then let the people on the team find ways to get them to show up.

– Book 7 = Book 3, revised –

By 2006 it was time to update Agile Software Development, because five years had passed since the writing of the agile manifesto, and there were five years of discussion and misunderstanding to deal with.

After this revision, Agile Software Development: The Cooperative Game had become three books – a 100-page book about people, a 100-page book about methodologies, and a 100-page book about agile development. If you read it, you might like to approach it in this way.

– Book 8, unpublished –

In fall 2006 I started on my next book, tentatively titled Software Engineering in the 21st Century, an undergraduate textbook for third-year undergraduates in a semester-long course. Neil Harrison and Chris Jones agreed to teach it in two separate courses while I wrote it in real time underneath them! Brave or foolish, we did it that fall term. I turned in 20 pages of book, powerpoint lecture slides, homeworks and class activities, twice each week, Monday and Wednesday. That was a wild ride (and it took me months to recover from the exertion, because I was consulting, teaching and traveling at the same time).

That book isn't out in paper form, because I don't have a publisher. I may publish it in pieces on my new web site.

I like this book, because I got to incorporate everything I knew and a bunch of stuff I didn't, all the latest and greatest ideas I could find about modern software development, cut down to the size suited for undergraduated. You can read about this book and course on my web site.

That leaves me with two books outstanding (PM strategies and the undergraduate SE book) plus a zillion more trying to get out. I'm literally limited by typing speed.

Which do I like best? Still the first edition of Agile Software Development. It's thinner than the second edition (I like the heft of the thinner book), and it contains that Introduction I enjoyed writing so much.

Which should you read first? Dunno – what's your reading style? Some people like Crystal Clear because it's the most concrete, some like Agile Software Development because it's the most theoretical. Some like Surviving OO Projects because it's contains the most project pragmatics. These three actually form a sort of trilogy, and I wish they could be bought as such.

Only read Writing Effective Use Cases or Patterns for Effective Use Cases if you're in the use case business, because they are really sharply focused to that specific topic.

Read People and Methodologies from my website if you're curious about my project debriefings or the argument that methodologies are transient and should be picked up, used and thrown away like tissues.

Go and get lost in the (currently) 800 content pages on my web site if you want more of anything. http://Alistair.Cockburn.us.

Q3 Alistair Cockburn

2008-03-16T09:11:27+00:00

Q3. Have you ever worn a cowboy hat or flares?

Yes.

I had lime green bell-bottoms and blue platform shoes with about an inch of toe platform and two inches of heel platform. Really. Hopefully all photos have been destroyed.

Back in the late 1970s, I was one of those disco people. I pretty much stopped dancing disco after "Saturday Night Fever" came out in 1977 and John Travolta stole my moves :-). The movie did fire up disco popularity for a while, and I had a short gig as a professional disco dancer at a place where we were supposed to be the seed couple to get other couples up to dance. Really.

I got into trouble with the mountaineering ski crowd because I'd wear my black shiny plastic disco-style windbreaker to go back-country skiing, instead of the "just-rolled-out-of-the-garbage-can" polypropylene and wool outfits that constitute official uniform in that crowd.

When I moved to Utah (1975) I got myself a good cowboy hat, cowboy shirt, and cowboy boots (still have them, and they still fit). I scared the dickens out of my girlfriend when she came back from a summer vacation and I picked her up wearing all that. She nearly climbed out of my beat-up 1963 VW beetle when she turned on the radio and it came on with cowboy music. "Just kidding" plus a fancy dinner and lots more cleaned that up.

You can tell I'm culturally sensitive ;-)

Alistair Cockburn - Q2

2008-03-07T19:46:10+00:00

Q2. I know a lot of folks who glaze over when I mention use cases. They are kinda dull and they don't seem all that agile. I've never seen the point of drawing stick figures, for instance. But then I discovered your book and your particular way of writing them and I was wowed! A couple of weeks ago I took a client's management team (along with a few of their senior techies) through an example and they were also wowed. Can you tell us a little bit about the format and how you developed it to the point where you realized you had to write a book about it? Can you tell us about a real live agile team that's using them in iterative/incremental development?

You're right – by today's agile standards, use cases are long and dull. I probably wouldn't learn them today if I didn't already know how useful they are. Actually, this is a bit troublesome, because I am having difficulty finding new people to teach them – all the use case experts I know learned them in the 1990s.

What makes them hard is they require thinking and communicating – both tough sells at any time, and especially into the agile space, where the reflex is type-and-show. (It's a good thing I helped write the agile manifesto, or they'd have ripped up my agile membership card long ago for saying stuff like that!)

I got to use cases through my assignment at the IBM Consulting Group in 1991. They asked me to construct a methodology for their consultants to use on OO projects, with no constraints on what the answer was.

After some interviews, reading, and research, I settled pretty quickly on the Ward Cunningham / Kent Beck CRC cards technique - Rebecca Wirfs-Brock' "Responsibility Driven Design", and incremental development.

These left out something critical. As I wrote in "In Search of Methodology",
"Responsibilities reveal which object or subsystem will carry out a part of the required function, interaction diagrams show how the function is carried, but neither say why it is required in the first place. That is what scenarios and use cases provide."

It is similar to the "warp and weft" in weaving: one direction lays down the nouns or objects (or components) needed, and the other direction lays down the verbs or functions needed.

Trying domain modelling without knowing which subset of the domain is important is just too big and too hard. We saw in the 1990s companies spending /hundreds/ of work-years (I am not kidding) on domain analysis and domain modelling because they didn't know incremental development and how to bound their inquiry.

Anyway, in 1992 I read Ivar Jacobson's 1987 OOPSLA paper on use cases, and saw that they were just the right size and shape to fit into this hole that was left by all the OO techniques at the time.

I read the pre-pub draft of Jacobson's OOSE book, and didn't understand how to write them from that, so I flew over to Stockholm and took a class from his company on how to write them. After that I still didn't understand how to write them. Indeed, at that time, nobody knew how to write them, no matter how many words Ivar wrote on the subject. I think maybe they sit too close to his spinal cord for him to be able to see what he himself is doing.

So at that point, I had to work out how to break use case writing down into small enough pieces that I could write an instructional booklet for the IBM consultants to use. I sat at my favorite coffee shop for two days trying to break it down. It turned out to be a 3-Aha! problem.

* The first Aha! that hadn't been articulated to that point was that each use case collected scenarios based on a /goal/ - that's what held the various scenarios together.

* The second Aha! was that essential to the notion of goal is /goal-failure/. Specifically, every use case has not one but /two/ exits, success and failure. This goes against everything we had learned from Edsgar Dijkstra about single-entrance / single-exit subroutines, so I had to be quite careful that it really worked (David Parnas came up with an interesting semantics for multiple-exit functions back in the 1980s, so I knew there was some precedent). Anyway, the two exits turns out to be essential to understanding how they all fit together.

* The third Aha! was that goals are recursive – we have big goals, which we break into smaller sub-goals, which ... etc. We tackle each sub-goal in turn until we achieve the main goal. If we get blocked, then we look for an alternate sub-goal. All this is common sense (but not commonly articulated), and also a well known artificial intelligence technique. Therefore, there was good, safe theoretical and practical precedent for believing it would be sound.

From these three, I was able to construct a semi-formal theory that allows us to do what is easy and natural in writing and expressing ourselves, and which at the same time is mathematically sound ... and also, just by the way, produces use cases that look just like what Ivar Jacobson wrote! I think that was one of the happiest little problem-solving moments of my life, to be honest.

I wasn't allowed to publish this at the time, because the IBM Consulting Group thought all this methodology stuff was top-secret, mission-critical, revenue-critical information. Not true, of course, processes and techniques rarely are, but it meant I couldn't speak about it for some years.

A side note – I couldn't publish my first methodology, written in 1992-3, for the same reason. In fact that methodology never even got a name, more's the shame. It would be interesting to look at now, because it was one of the first agile methodologies contructed (after Tom Gilb's Evo). All it contained was use cases, incremental development, responsibility-driven design, and programming. I wrote a little article about it in Object Magazine just before leaving IBM (see http://alistair.cockburn.us/index.php/In_search_of_methodology).

Use cases turned out to be a 4-Aha! problem, because there was a little hole left over, which people would often get around to asking about: Why do we write down system internal actions (such as "check that the account has a high enough balance before giving out money"). The goal-subgoal business doesn't explain that. We used to answer, "Because these are requirements, that's why. If you don't write them, you'll get the wrong system."

The 4th Aha! came (no doubt at the coffee shop) a few years later, when I realized that a system's design enforces a contract across competing interests of various stakeholders, and that the use case describes the contract to be enforced. This introduced the stakeholders and interests model, which closed the gap. We now can say that every sentence in a use case describes a stakeholder's interest that must be satisfied or protected ... and now it is clear why we write all the validation checks in the use case.

I still wasn't going to write a book on the subject, because I thought my original article was sufficient, but around 1999, I got so scared by all the terrible book proposals I was having to review that I decided I had to publish a book just I wouldn't get stuck having to teach out of someone else's. That, combined with the construction of my class on the subject, and my just having coached a team at Fireman's Fund Insurance Company, indicated that I had enough information to write a "Shu"-level technique book.

You ask for examples of real use.

I'll give you two, my first and my most recent, which happen also to be my biggest and my smallest projects, the least agile of my "agile" projects, and possibly the most agile of my "agile" projects. The usage of use cases is quite different between the two. I'll also say how I recommend using use cases on modern agile projects.

The first project I call Winifred. It is written up in detail in "Surviving Object-Oriented Projects", and I reference it quite a lot because it was a very interesting and educational project, and I got to see it from initial bid until delivery and payment.

It was interesting for this discussion because I had just gotten done with my use case theory, and I got a call from the IBM Consulting Group field team to hurry up and go out to a project with 180 freshly written use cases (written in "casual" style, per my book). We used what I had plus what they had written to construct finally about 240 use cases, casual style, captured in Lotus Notes, to be used as the formal contract between the IBM Consulting Group and it's client.

It was a fixed-price contract, so we spent two months writing, examining and estimating those use cases, the interfaces, the estimated domain complexity, the people we could hire, etc. All of us were itchy-fingered programmers, and we were nearly gnawing on our fingertips from the frustration of not being allowed to start typing. However, since the contract was fixed-price, at about 15M$, we weren't allowed to start programming until the contract was signed on all sides.

Our incremental delivery cycle was 3 months; we did incremental and iterative development within each cycle, for about 9 weeks, followed by a 4-week debug-and-deliver cycle. More about this is written up in Using both incremental and iterative development, so I won't repeat it all here.

My current project is the exact opposite. I'm trying to replace the Media Wiki engine running my website, because I think Media Wiki is crippled when it comes to any media besides text and images (but never mind that for now). What's interesting is that this is not a fixed-anything project, it consists entirely of one programmer and me. I pay for it out of my pocket, we pair-program whenever we're both available, and he/we work on it about one day a week.

So according to modern agile theory, we'd never write use cases for it, right? Wrong. I still need the basic advantages of use cases (see http://alistair.cockburn.us/index.php/Why_I_still_use_use_cases), like having some idea of where we're going, how big the project is, where we are, contextualizing our work.

Therefore, I wrote really really light use cases, and replaced the extensions with "work tokens" (I don't like either of the terms User Stories or Features for this level of granularity). My programmer friend and I peel off a work token to implement, then check how we're doing against the big picture given by the use cases. We're currently going through the use cases to see what's the mimimum we have to do before we can replace my Media Wiki engine with the new engine. Cross your fingers! (If you are reading this some months after I wrote it, with luck you'll see the new site. Compare it to Wikipedia).

These use cases are shown at http://alistair.cockburn.us/index.php/Examples_of_ultra-light_use_cases

What I recommend to a project team doing hard-core agile and using use cases is a little different from either of these. It is described in http://alistair.cockburn.us/index.php/Agile_Use_Cases.

Write the use cases as in "Writing Effective Use Cases." You probably won't have more than 30 of them.

Print out the one(s) you are working during this month or 3-month period, in 16 point font. Tape them to the wall of your work room (you are in a team room, aren't you?). Take a yellow highlighter and highlight in yellow the particular sentences you are working on this iteration or sprint. Color them green when they are done.

Now you can show what you are working on, what is complete, and what has been left out, at any moment in time. You'll get the benefits of use cases and user stories and agile and incremental and information radiators, etc.

P.s. typically, the first "story" or work token to implement in a use case is the first and last sentence of the main success scenario.

One last thing, since this is supposed to be an interview and not a tutorial on use cases (never mind that you asked for a bit of a tutorial).

Most researchers and writers write whatever's on their minds at the time, not really caring if it is right or wrong. I recall a professor telling me that this (some) particular author was famous for his first book – which turned out to be all wrong, but never mind that, because he got famous for it, and then got famous again for his later book which refuted his first book.

I recall at the time thinking, roughly, "Yuck! That's just bad research!"

Then I found that almost everyone does this. Booch wrote his methodology back in the 1980s, then spent half the 1990s recanting and cutting stuff out. I used to tell people to stick with him, he'd get to the right answer sooner or later ... but they could join me and I'd just tell them the right answer now ;-)

When I was given the assignment in 1991 to write a methodology, I made two rules for myself:

1. Always honor the programmers in their job activity, because if they leave, the company ships nothing, no matter who else is around; on the other hand, if everyone else leaves, the programmers can still ship useful software.

2. Try not to have to recant. This meant, for me, to be correct and err on the side of leaving good things out, rather than sticking bunches of stuff in and having to do like Grady and take stuff out.

I've been fairly successful in those two rules, particularly the second one.

My first methodology in 1992-3 only had incremental development, use cases, responsibilities, and code. I added design patterns in 1993-4. I added more stuff for the Project Winifred project, but only temporarily, for that project. Crystal Clear has some more stuff, but nothing I think I'll need to recant about. In general, my error, if any has been to underspecify than overspecify, and I'm still happy with that decision. We'll see how long it goes before I have to recant on something.

Well done Johanna Rothman

2008-03-07T19:33:49+00:00

... and to all the other winners in the 2008 Jolt Awards

Johanna won a productivity award for Manage It! Your Guide to Modern Pragmatic Project Management

Feedback! Please ... :)

2008-02-28T12:39:55+00:00

Hi Everyone,

Let me quickly introduce myself. I'm Clarke Ching, I live in Scotland, I'm the one doing the interviewing.

Can you spare a moment to send me a quick email to Clarke.Ching@gmail.com?

I've a few questions:

What do you think of the interviews so far?

Is there anyone else who you'd like to see interviewed?

Are there any question's you'd like to ask the interviewees?

Is there anything I could do differently that you think would improve the blog?

Would you like to do some interviewing yourself?

Clarke Ching
www.ClarkeChing.com

Q4 - Jared Richardson

2008-02-28T11:40:47+00:00

Q4. Tell me about your current venture? A start-up ... that's a big change from SAS.

The startup I'm at has a very interesting idea. It's about creating an extreme level of transparency and then using that information to measure progress. We look at information from the team's source code management, issue tracker, feature tracker and even the IDE.

At the end of the day I can look at a team and see that they're spending too much time in the debugger, so I can talk about continuous integration and test automation as a way to keep people out of the debugger and in code. Another team might be reading code all day, trying to understand it. Then we talk about the peer code review or pair programming to get training levels up.

We automatically create burn up charts and then use them to teach managers why feature creep is bad. They can instantly see how the team can't catch up with an ever growing scope. Or how the team committed to this feature set but only delivered half of that. Then we pull up a "difference from estimation" report to see which features had bad estimates.

And at every step, because we automatically harvest real data in the background, we've got the actual time spent on each feature. Instead of guessing how much development time was spent over three days, we can say it took 12.74 hours to create that feature.

But this level of transparency scares a lot of people. Our industry has been very closed traditionally. Our managers generally don't understand what we do or how we do it. And as a result, most projects fail. Some from bad estimation, others from bad management. Sometimes it's because no-one realizes there's a problem until too late to help. The last numbers I recall seeing were a 75% failure rate for the industry. We've got to step back and see if we can't find a better way to build software. Many of the agile practices do just that. 6th Sense Analytics makes an amazing amount of information available and we're starting to see some pretty amazing insights come out of that data.

A tool like 6th Sense can be abused by a bad manager and it's a fundamental culture shift for many shops. But on the teams with the courage to make the leap, we've found both development and managers come away with a better understanding of the other does.

Why am I at a startup? If you saw my first question and it's answers, you know I like to tinker. And I get to do a lot of that here. We have working software, but there's still a wide variety of work to be done.

Q1 - Alistair Cockburn

2008-02-28T11:23:50+00:00

I really need to introduce Alistair Cockburn do I. You think you know him? Wait 'til question 3 ...

Q1. Hi Alistair, Tell me about yourself, especially, how did you get to be so damned brilliant? I don't say that lightly but some people I know go all gooey eyed when talking about your work and I reckon your approach to writing Use Cases gives sliced bread a run for its money. What's your story? What got you here? And, what does your wife think of all this agile stuff?

That's one question? Can't wait to see the rest!

Many thanks for the kind words ... There are enough people around me who think I'm incredibly stupid that maybe they sort of offset each other – kind of like the man with his feet in the freezer and his head in the over but who is, on average, at a comfortable temperature.

One of the things fairly unique about my background is we moved around a lot – My parents were English and wanted to get out of England after WW II, their goal was (a) to get far away from England, and (b) to travel the world. They dropped kids, in succession, in London (in the London Zoo to be exact), Kansas, Colorado, and Bangladesh. I was one of the ones dropped in Colorado. We lived there for almost a year after that before moving somewhere else. (Some people ask me where I was born; when I say "Just outside Denver", they say, "Oh!" as though they understand something, as though that fact might actually have any meaning.)

My father was an outside-the-box thinker and pretty headstrong. He nearly didn't get his MD degree because he kept disagreeing with one of the professors about some theory about the origin of diseases. He was an early epidemiologist. One of his earliest theories in the late 1950s was that diseases and parasites evolved along with homo whichever, all the way from pre-apes to modern humans. He couldn't get people to write about that in the US in the late 1950s because of the Christian evolutionist lock on science (so recently!). His book Evolution and Eradication of Ancient Diseases was published in 1963. He started a paleopathology movement, arranging multi-disciplinary autopsies on mummies. That was while I was in college, so I'd come home to see him discussing these trial autopsies (see http://ambassadors.net/archives/issue13/selected_studies.htm for more on him).

Between my mother's arts-literary nature, my father's outside-the-box research and our constant moving (Sri Lanka, Massachusetts, Bangladesh, Ohio, Sweden, Detroit, Scotland, Cleveland, Salt Lake City), I had to adjust to many cultures and never got to be believing that one could be right ... quite different from home-folk in most parts of the world. I suspect this shows in the way the Crystal family of methodologies is constructed, and I believe you'll see it leak out on almost every page if you're looking for it.

I graduated from a city high school in Detroit at age 15, and went to be an exchange student to Sweden (1969-70) to kill a year before college. That got me hooked on travel even more. I arranged my own junior year abroad at St. Andrews. When I got married, I checked first with my wife-to-be that it would be OK to move to Europe for a few years. A few turned to seven, at the IBM Research Lab just outside Zurich, where I worked on formal specification techniques for a while (an interesting precursor to methodologies, and one I dip back into every now and then) ... and that's where our two oldest children were born. We moved back to the US for a while, but I got itchy feet and sent myself plus family to Norway in 1997, where I worked with the Central Bank of Norway. Along the way, I discovered I like learning languages, so now I speak some extent of French, German, Swiss German dialect, Swedish and Norwegian. I tried to learn Farsi back in the late 1970s, but then the Shah got overthrown and I've never been there to tamp it down (sigh).

My wife is an artist. While I used to work in computer graphics at Evans & Sutherland, we had a lot to talk about. We still argue over additive versus subtractive colors. She's an outside-the-box thinker herself, tolerant, smart and self-motivated --- she would have to be or we'd never have been able to put up with each other for 28 years! Thanks to her, I have sculptures of skeleton faces and bones in my living room. Thanks to me (and my father), we have a smallpox mask from India – a two-foot high scary face with tongue sticking out and snakes coming out of the head – hanging in our front entry way.

Agile comes easy to her – all she has to do is change her mind at any moment!

Anyway, you get a picture of our home life ;-).

Our kids grew up thinking this was normal :-). They're teenagers by now, so they don't any more, of course.

And regarding the word "normal", when my middle son Sean was in calculus last year, we realized that the "normal to a curve" is perpendicular to the tangent at that point, i.e., the vector pointing farthest away from the rest of the curve at that point. So now my kids have fun calling me "normal."

That was a freaking long question (answer). Sorry about that.

[If you feel comfortable here Alistair, I suspect people would love to
hear a little about your home life]

Not too much to say. Since leaving IBM in 1994, I live in Salt Lake City and do most of my writing in coffee shops (for example, right now). I wrote Surviving Object-Oriented Projects and Writing Effective Use Cases at the Beans & Brew coffee shop just below the ski resorts; I wrote Agile Software Development largely at the Salt Lake Roasting Company, Crystal Clear and my doctoral thesis at the Salt Lake Coffee Break (open till 2 a.m. every day!), and these days sit mostly at the Two Creeks coffee (good for articles, but not for books – for that I still need the Salt Lake Coffee Break :-).

I keep travel down to 50% so that I have enough home life for our family to survive; and I write books to have a good reason to stay home. Lately, I haven't been writing a book, so my travel time's been going up again. Ugh.

I swim competitively, mostly every lunch time when I'm home to keep fit and to keep motivated. My goal over the last 4 years has been to break every Utah swim record (short course yards) in the 50-54 age group. That obliges me to change events every year. I have 14 of the 18, but only 5 meets left – I don't think I'll get the last two, but I hope to get two more before time runs out.

Swimming and traveling don't go well together, mostly because I don't have enough discipline to train on my own while on the road.

Q1 - Chris Fry Salesforce.com

2008-02-28T11:14:51+00:00

Chris Fry is the instigator of one of the agile worlds must credible and newsworthy (in my opinion) Agile Success Stories ...

Q1: Hi Chris, We first met a couple of years ago when I reviewed/shepherded your submission to the Agile2007 conference. To be honest I don't think I added a lot to your submission - you had an excellent story and you told it well. Before I ask about your story can you tell us a little about yourself - your personal and professional background?

I am a Senior Director at salesforce.com where I lead the platform development team. I've worked at a startup and in enterprise software in various roles (architect, engineer, manager). I have a Ph. D. in Cognitive Science from UCSD and have always enjoyed cross functional work in academics and industry. I live in the Bay Area and surf on the weekends.