Agile Montage

Agile and Beyond 2012

noreply@blogger.com (Rajeev Singh) — Mon, 12 Mar 2012 04:17:00 +0000

Agile and Beyond 2012 was a sold out event with about 650 attendees. Great Keynote by Steve Denning.My session on 'Agile Pathologies: Backyards of Agile Shops' was well received. The slides are available on Slideshare here or you can see it embedded below.

I was surprised to see the vibrant Agile community in Detroit. Had no idea, it is so big. Thanks to the organizers for a fabulous event. Well done!

Agile Pathologies: Backyards of Agile Shops

View more presentations from udairaj

Practices or Mindset: An Observation for Agile Adopters

noreply@blogger.com (Rajeev Singh) — Wed, 29 Feb 2012 22:01:00 +0000

"These practices are not suitable for our environment". "Let's shoot for as well formed backlog as we can". The difference between the two individuals is that the former is thinking of practices and the latter has the mindset.

What's the big deal, you ask? The subtle difference results in an order of magnitude difference in morale during execution. Your metrics, very likely, are a reflection of this nuance. And, in case you have ever wondered, these metrics then become the perfect medium of spreading the culture around; both good and bad. The "practices" crowd also worries about if they are executing as per the "textbook" or not. The differences are analogous to 'one size fits all' versus 'one size fits none'. Dogma and pragmatism are the other polar pair that come to my mind.

I intend to keep a log of how the practice-mindset subtlety manifests. Expect an update coming. Please don't confuse this topic with my earlier blog on Lean and Agile.

[Image taken from: www.smallbiztrends.com]

Signatures of Distrust in Agile Teams

noreply@blogger.com (Rajeev Singh) — Thu, 19 Jan 2012 14:00:00 +0000

Trust is one of the key attributes of any successful agile team. Its lack hurts the velocity and increases the stress level in members. This, in most cases, is unsustainable. I would recommend a thesis titled, 'The role of trust in strategic alliances' (Michaela Weinhofer, June 2007. Baltic Business School, University of Kalmar, Sweden). Two simple definitions of trust from the references in the thesis are:

Combination of credibility, predictability, and straightforwardness, and
Decision to rely on another party under the condition of risk.

The cooperative relationship required to achieve a common goal can hardly ever be built with distrust. The reason I say "hardly" is because experts (referenced in the above thesis) think cooperation and trust are independent entities. The relationship may be cooperative but not productive. I would also highly recommend The SPEED of Trust: The one Thing That Changes Everything by Stephen M. R. Covey.

For an agile team, how will you know that you are in a distrustful relationship? Here are some symptoms:

Perpetually prioritizing new functionality over defect fixes.
Continuous scrutiny of velocity, deep interest in individual developer velocity, pressure to sign-up more points than velocity.
Questioning story estimates beyond a reasonable limit.
Avoiding story splits fearing 2 plus 2 won't add up to 4.
Comparing actual effort to estimated effort.
Key leadership (technical and non-technical) entrusted to non-team members
Attempt to pile-on stories mid iteration in the interest of getting more done.

There are many reasons contributing to above situations. It could be lack of awareness or lack of experience with Agile. But don't discount the possibility of distrust as a driver.

[ Image taken from: http://www.rockwerxclimbing.com/3292.xml ]

Guilty of a Process Diagram - Don't be, just be Smart

noreply@blogger.com (Rajeev Singh) — Thu, 19 Jan 2012 14:00:00 +0000

"Let's not get this process too heavy", "We'd prefer lightweight over the traditional heavy". Everybody has got process diagrams, even in the Agile sphere. You don't believe me? Ask any Agile manager and you'll get at least two; a defect triage process diagram and an iteration lifecycle diagram.

Replacement for Common Sense
You probably have a process diagram that you loathe and scoff at. Usually processes are proposed and adopted to replace common sense and then, even worse, compliance is monitored. If you have to have them, processes should be guidelines to remind folks of the steps and sequences in case there are doubts. The idea should be to provide support to those who need guidance; beginners or occasional troubleshooters.

Analogous to an Elevator Pitch
Based on observation (haven't come across a research on it), I have concluded that most people prefer simple processes but fall in the trap of creating complicated ones. That calls for a couple of rules of thumb:

Start so simple that you void the decision box. If you are using decision boxes, chances are you are in the realm of questioning people's common sense.
Expand a process area only when team members are missing the boat. If team members repeatedly fail to execute a certain step, question its need first.

In gist, a process diagram should be a very high level happy day path free of clutter designed to handle unhappy day scenarios. An analogy for a good process diagram is to an effective elevator pitch; crisp, clear, and simple.

It is important to understand the 'why', specially if you have to innovate. And if the 'why' is clear, there's no reason you'd ever stand complicated process diagrams.

[ Image taken from: http://www.basement.org/2005/12/weird_naked_white_collar_guys.html ]

A Case for Enhancing the Agile Triangle

noreply@blogger.com (Rajeev Singh) — Tue, 17 Jan 2012 14:00:00 +0000

The Iron Triangle (Project Management Triangle) is an antithesis of Agile mindset. Jim Highsmith recognized it and proposed the Agile Triangle. It seems that the value vertex, releasable product, is still narrow in its focus. It is talking just about product/service value. It is restrictive to only customers and shareholders. How about Employee/Team Value? Or, even intermediary value focused on enhancing the core environment and sharpening the saws. For better software, we ought to think of empowerment, leadership and mentorship.

I want to make a slight modification to Jim’s triangle and add Intermediary/Internal value to the Value vertex. We just can’t ignore that anymore. That’s how continuous improvement happens. That’s when you take time out and act on retrospective tasks. That’s when functional product ownership balances itself out with technical product ownership.

An Agile approach to product delivery encompasses enabling the people and infrastructure. We just can't think at the project level anymore. We ought to start focusing on the whole organization. It is untenable to continue delivering external value without investing in internal value. The risk of not doing so is to put continuous improvement on the back burner.

A new normal, but a bad normal

noreply@blogger.com (Rajeev Singh) — Fri, 13 Jan 2012 00:44:00 +0000

In the last few years, I have been focusing on Agile Pathologies. My thesis is that as individuals we exhibit certain patterns of thinking, reacting, and politicking that usually impede Agile adoption. At times it is outright contrary to what Agility is all about. We can blame the tools, spaghetti code base, team distribution or whatever else you have, but Agile adoption is a people challenge.

A few in the industry discounted my views. Some examples:

"People behave in fairly predictable ways."
"That's within the realm of normal human response."
"80% of projects fail because of lack of attention upfront."

We are considered knowledge workers and supposedly can intelligently deep-dive to fix our problems. Under this misconception, we have built a culture of band-aiding the broken bones when it comes to people issues. This is the new normal, but it's a bad normal.

With Agile manifesto, the software development community for the first time officially declared people to be more valuable. If we really believe that, we have to try hard to identify a pattern in people problems, and address their root causes. Or else the debate will not cease, “Is Agile the best way to write software, or is it the way best people choose to write software?”

Just as any good constitution or a body of law isn't of much consequence without understanding the basic principles and right execution, so is Agile. By execution, here, I mean responses to emerging problems and situations. Good Agile environments need relentless focus on improving people issues. More importantly, they require that we are hawk-eyed in spotting pathologies.

[Image taken from: http://www.zimbio.com/pictures/UXs56SKql2q/Italian+Team+Training+Session/3EfoOAs1Rqt/Marcello+Lippi ]

Speaking at Agile Development Practices East

noreply@blogger.com (Rajeev Singh) — Tue, 04 Oct 2011 23:20:00 +0000

On Thu, Nov 10, I will be in Orlando to share my views on some recurring anti-patterns I have encountered during Agile adoptions. If you are an Agile manager or sponsor, or want to be one, and have ever wondered if Agile is a fit for you or your organization, this session is meant for you. I will share some war stories and my secret sauce of success on ThoughtWorks' engagements.

I will delve into these pathologies at four levels; individual, team, management, and organization. We'll have a look at the 'Agile Iceberg' and try to define what an Agile Transformation means after evaluating notions such as 'Preconventional' and 'Postconventional' agility.

As a speaker I get to share with you $200 discount. Just use the discount code, SPAS, to register. With an additional early bird discount of $200, this should save you $400 for conference registration.

Here are some links to the program:

http://www.sqe.com/BetterSoftwareEast/file/BSCE%202011%20Broch_FINAL.pdf

http://www.sqe.com/AgileDevPracticesEast/file/ADPE%202011%20Broch_FINAL.pdf

Agile Frontier: Where's Agile not Working Yet

noreply@blogger.com (Rajeev Singh) — Tue, 09 Nov 2010 05:00:00 +0000

"What kind of projects are suitable for Agile?"
"We've got to be careful in choosing projects that we want to be Agile."

At the onset I must make it clear that a big component of Agile is continuous improvement. That said, generally speaking improvement can usually be made most of the times; especially if Agile is not yet an established approach within an organization. The reason is simple, while most people focus on the structure of Agile and appreciate its aesthetics and lightweight nature, good Agilists focus more on its soffits; rigor, discipline, Socratic approach to problem solving, etc.

Relatively small and/or web-based projects have been experimenting with Agile for quite a while. The knowledge base in the Agile community on what works and what doesn't with such projects is rich. The landscape turns dismal on some other fronts, primarily due to lack of experimentation. In some instances a whole section of software community has been insulated from Agile. A prominent case study is Siebel (more on that in a later post). The idea in this post is to discuss some of the environments that are still a frontier for Agile. Without further adieu here's where Agile is struggling:

Big Opaque Software Frameworks: Ownership, access, and control over the code being written is vital for software teams. If developers can't modify a code base to suit their needs and hook other 3rd party tools into it, they can't really derive a fundamentally sound and quality product out of it. Basic practices that contribute to stability and quality of software like Unit Testing are hampered due to inability to touch this code base and, again, lead to a poor quality product. Lack of unit testing in turn, usually, lowers the confidence level of developers who want to refactor the code. Thus they avoid it , which leads to inefficient designs.

Shared and Aggregated Environments: Rolling software changes into environments and databases that affect organization wide operations are not too well suited for Agile development on their guts. Some changes are impossible to make and other very expensive. An example would be a common environment shared between inventory management, inventory ordering, and inventory finance and accounting systems. A spaghetti like this with myriad of inbuilt interdependencies usually warrants heavy frontloaded analysis. Regulatory oversight only makes the matter worse. e.g. a change made to an Oracle module once made might be difficult to roll back due to financial and regulatory restrictions. Often the Big Opaque Software Frameworks (above) almost mandate a heavy analysis and design phase. Big ERP and CRM applications fall into this category.

Software Hardware Boundary: This is the most exciting and promising challenge that Agile will have for a few years to come. Traditional heavy equipment and machinery manufacturers that utilize large scale control systems are getting comfortable with Agile in their software shops. Some of them are trying to take Agile out to their manufacturing plants and experimenting with how to make the mechanical design teams work with software design teams. Agile has not yet addressed the complementarity in software and hardware design. How can hardware design keep up with the software as it is evolving and vice versa? They inherently seem to have not just different characteristics of work cycles but also skill sets and personality traits in their ranks. Agile approach to software is largely unknown in the manufacturing world. It's a challenge of lag, lack of experience; hence the frontier.

Untamed Team Distribution: Lack of interaction between software development teams and business stakeholders adversely impacts the feedback cycle and affects quality of the product. Development velocity takes a hit and ultimately feedback cycles lengthen to a point where rework increases and impacts velocity. Agile leaders in the industry have figured out approaches to make Agile work in distributed environments but only when the teams distributions are purposefully designed, i.e. they have the luxury to choose the people on the teams and their physical location or have a say on the rotation schedule of people between locations and teams. The persuasion to form teams with individuals that are dispersed between multiple locations across different timezones and are "staffed" on projects because they happen to be available might backfire and lead to woefully suboptimal value. Designing for Agility is a resource management challenge, and therefore a frontier.

Systematizing Agile: A natural propensity once Agile has delivered a few success stories is to systematize it in an organization. This urge for "systematization" is detrimental to Agile's success. Agile is not Coca Cola; each implementation is different and should be different to suite the needs of an individual project's needs. Systematization and templatization curbs the "inspect and adapt" nature of Agile. A challenge, therefore, emerges. How can Agile be institutionalized such that the necessary functions within corporate like budgeting and accounting can still run efficiently, smoothly, and cost effectively? How can financial planners ensure optimal use of available resources and timing of investments? This is not to say that in the traditional world financial planners are blessed to have projects executed as they had planned. Projects are extended, teams underdeliver, and expenses are all over the place. Not to mention that the OpEx CapEx ratios are much different than initially planned. The challenge therefore is to build accounting communication and data-churning tools that can react to the realities of projects on the ground. Agile, I think, catapults this necessity to forefront. However, Agilists haven't yet built the tools or reporting mechanism, to the best of my knowledge, that addresses these specific needs of financial planners.

Agile Scalability: 60-70% capitalization rates in software development industry are not unheard of. They are rare, though. On an average this rate is around 20%. Since software capitalization rate is far less (or even none due to short spans between technical feasibility and actual development) in Agile it essentially becomes unattractive for corporation to even think of agile transformation across the portfolio without getting creative and aggressive about it. That's a risk, I don't think anybody is willing to take. The outdated (in context of Agile software development) FASB edicts pose a huge barrier to scaling Agility in large software development environments.

Talent: With the growing adoption of Agile comes a sad realization. It's still the people that make the difference, as they always have. A common misunderstanding in the community is that it's the lightweight process and use of innovative tools and techniques (e.g. TDD) that make Agile successful. If Agile is as much about the people as it is about tools, then we have a significant problem at hand. Scaling Agile within an organization suddenly seems to be a herculean task. Who are these people that "get" Agile and where can we find them?

Now some would argue that regulation intensive environments are not conducive for Agile. True. But that's all there is to it, it's not conducive but Agile is works in many such cases.

This list is a work in progress. As I encounter other challenges you'll definitely see them reflected here. Change is an eternal process and I hope that the next time someone is thinking of making a change in any of these frontier environments (s)he considers Agile as a creative process that teaches us to confront novelty and improvise as opposed to a new "process" or a "methodology" that comes with its own peculiarities of Do's and Don'ts.

Image take from: http://antarcticsun.usap.gov/aroundTheContinent/images/pole_plane.jpg

Agile Release Planning

noreply@blogger.com (Rajeev Singh) — Thu, 28 Oct 2010 13:02:00 +0000

"Almost every project has it; there's nothing new about Agile Release Planning." Not quite! What's different about Agile Release Planning, like many other good Agile practice is that it is lightweight, inclusive, and a living plan that is well socialized. Every release plan is different. The anathema of the traditional release plans, though, are their elitist origins, seclusionary existence, and staleness. The realities of project kick in and then an eerily familiar yet enigmatic (to me) behavioral (anti)patterns start to fortify; the unrealistic to begin with plans morph into contracts between IT and Business and nothing ever changes - budget is set, schedule does not move, and scope of course only increases.

Someone once said to me, "You Agilists are lucky". I think, he meant to say that Agilists are pragmatic and call out the fashionable insanity of planning. The notion that neither scope nor schedule and budget move is untenable. Release Plans are and should be evolutionary by nature. This evolution stems from other Agile practices like prioritization and iterative design practices.

The slide deck below is an aide I use to convey the above message.

Release planning

View more presentations from udairaj.

Agile User Stories

noreply@blogger.com (Rajeev Singh) — Thu, 28 Oct 2010 04:00:00 +0000

A pattern that helps me during Agile enablements and rapid project inceptions is to conduct brown bag sessions on the topic that I am about to introduce in the following few days. One such presentation is on Agile User Stories. I will try to keep the content updated with every new nugget of knowledge that I gain. Until then, please do not hesitate to drop me a line if you have comments/suggestions/questions.

User stories

View more presentations from udairaj.

Agile Pathologies: Anomalies in Agile Practices

noreply@blogger.com (Rajeev Singh) — Fri, 13 Aug 2010 14:30:00 +0000

Pathology is a strong word and a bold one to use to deliver the idea I intend to. An interview candidate at ThoughtWorks at the time pointed it out to me during our one on one. He said, "Do you really use that word in front of your clients?" The idea behind the use of this word is to focus on the emotional and behavioral challenges that are frequently observable during Agile enablements and transformations. For lack of a better word, it is 'pathology'. Although pathology indicates existence of a science behind its eradication, I consider it an art. A presentation on the topic is available at the end of this post..

Exhibited by individuals:

ScrumMasters don't speak in Daily Scrum or Standup: In a few of the coaching engagements I encountered ScrumMasters who didn't give any insights into what they were doing for almost a week at a stretch. In one instance the ScrumMaster said to me, "I am a ScrumMaster, I am not supposed to speak in a Standup; it's a Team's meeting".
Lone Wolf Syndrome (anti-pairing): Pairing comes across as an invading concept to many. It seems to threaten some and slow down others (even though only for a short while). Egos are hurt and disagreements on implementations arise. In a short duration experiments with pair programming evokes all the reasons why most software developers are lone wolves; there's too much talking, difficult questions are asked about the design decisions, peer pressure to delivery something everyday is sustained and flexibility during the work hours to indulge in non-programming work is substantially reduced. Most importantly, the glory of having delivered a piece of code single-handed and a chance to be a hero doesn't exist any more. All these are good reasons that further fuel the lone wolves.
Ineffective neuro-linguistic tendencies: How many times have you encountered someone in a meeting was asked an open ended question that triggered a rambling from the other side? How frustrated have you been when a key decision maker asked a closed question and did not assess the root cause of a problem? Have you witnessed a meeting which dragged on just because no one asked reflective questions? Have you ever read user stories or requirements that are sprinkled with nominalizations or generalizations or unspecified nouns and verbs? I have, over and over again.
Emotional block to learning new concepts (e.g. metrics, estimation scale): Some external influence has set an expectation that there is a test or a quiz following the introduction of this new concept. To some it is disappointing that there is usually none. To others it's the anxiety of using a new approach in their routine work and an expectation to succeed with it. There's also a third category where some reject the notion of a new ideas because they don't see a problem with whatever they currently use or simply latch on to old ideas a little too tightly.
Fear of Analytics and Usage Statistics: This is easily to spot in the Product Owner (PO) group. It is not uncommon to find requests for features that are a 'pie in the sky' with few indicators that justify the demand. Quite often it is not difficult to access these statistics, but they are deliberately hidden by POs. There's a fear of embarrassment, "How would I look in the face of poor usage statistics of the features that I fought for tooth and nail a few releases ago?" or, "I don't want IT to question my PO skills with software usage statistics".

Exhibited by Teams:

Broken Windows: I use this term metaphorically for an imagery. Broken windows theory suggests that if problems are not fixed as they occur or corners are cut and not corrected they will become examples for others and further deterioration will ensue. In software development this may mean writing poor quality of unit tests or avoiding them or bypassing them during build development. It may mean poor quality of narratives or constant rework for developers if critical feedback loops are short circuited. Unfortunately, all of these examples are real-life that plagued their teams.
Acceptance of Technical Debt: Technical Debt is an acceptable notion on Agile teams. The slope, however, gets slippery when technical debt is ignored. In the race to get releases out on a predetermined schedule or to now lower the throughput of software to production teams seldom take a break to get rid of this debt or even refactor.
Release planning without Analysts: The ivory tower thinking of most program managers and project managers come to visibility on topics related to project finances and resource management. I always wondered where it all started and zeroed in on release planning. Who contributed to the software release map? It doesn't surprise me anymore to know that quality and business analysts were not involved. PM ran the show there with a few high-level stakeholders and agreed to a fictitious timeline with some events sprinkled over it under pressure from authorities up top.

Exhibited by Organizations:

Squirrel Agile: Imagine a squirrel crossing a street. Imagine undergoing an Agile adoption. There are, in some instances, similar patterns - let's do this new thing, keep doing it, don't do it, OK do it, NO get back to what we did earlier. Halfhearted, uncertain, knee-jerk driven, fear instigated decisions that lead to a hodge-podge environment are too common. The stink over the mess is a proud declaration by some managers that they have their own flavor of Agile. That's a problem. Command and control tendencies, assignment of resources to multiple teams, setting up teams that are function specific (analysis, development, and testing), tracking capacity utilized are often the the first signs of squirrel agile.
ScrumMaster Certification: The newest emerging trend is to hire certified scrum masters and professionals. There's nothing wrong with it except that most of the Agilists that I have worked with and talked to don't care for these certifications. On the flip side, a lot of certified scrum masters haven't even once been on Agile projects. These certifications, in my opinion, are dangerous; they create an illusion of experiential knowledge and arm the attendees (that's what most so called trainees are) with enough vocabulary to make them a risk. Yes, risky, because the traditional mindset is prevalent and practices are rampant under the guise of Agile.
Seeking an Agile Playbook/Cookbook/Rulebook: As an Agile Coach one of the requests often made to me is, "Can you leave us with an Agile Cookbook?" A steadfast belief that Agility will flourish if there's an Agile operator's manual warrants a discussion. But believing that Agile is esoteric and a rulebook as opposed to actual practice will help adopt and sustain Agile is an untenable notion.
Agile without being Agile: I once consulted to a client that had a regimented release calendar - two releases a year. Their entire technical, political, and bureaucratic infrastructure was designed and tuned to serve these two releases. The discouragement and hindrance in this environment was that all but two of the stakeholders perceived a great threat in exploring ways to redesign or tweak the processes. Even the optics of trying to do so, to some, was a risk too big to take in the institutionalized environment where adherence to these processes was rewarded.
Strong Identity (e.g. "We are a metrics company."): Who doesn't love metrics and who doesn't have a strong identity? Some enterprises have a strong folklore culture that propagates such fanciful identities. Management gurus may argue that these are must have characteristics to build a successful enterprise. I agree! Let them, however, not become roadblocks to change. The problem occurs when strong identities become roadblocks to pragmatic and rational change of mindset that Agile predominantly is. Metrics is a good subject. Some organizations and project managers are obsessed with them. Too much time is spent on tracking indicators that really don't indicate much. If someone is trying to plot a polynomial regression graph for a burn-up trend projection when a linear regression usually suffices, you know you have a problem at hand.
Nominal Product Owner: Recently I was working with a company that is designing a new product and has an aggressive schedule. Some of there people are new to Agile, including the President of the company. As we were presenting our findings and plans to the team the President interjected and asked us a lot of questions about the level of details we came up with, our confidence in what we found out and understanding the domain and the intent of the product, roadmap and our ability to deliver on time. Those were all excellent questions and we are glad that he asked them. He then switched his attention to the product owner and asked her many questions. Those included questions around her comfort and confidence in working with us, product boundaries, depth of product exploration exercise that we did, etc. Gist of the story; the PO was the anchor in the situation. She was entrusted with a market leading product and the responsibility to get it out of the door in time. Period. That incidence made me realize how underpowered and untrusted some of the other POs were in their organizations when I worked with them. They were titular Product Owners.

There are more pathologies. The above mentioned are harbingers. The intent of this post is not much different from the last few - spread an awareness of what plagues agile adoptions and to prepare people to spot these anti-patterns as they develop.

Agile pathologies

View more presentations from udairaj.

Image taken from: http://farm1.static.flickr.com/191/480869012_f1493aceb2.jpg

Agile in Three Words: An Attempt at an Epigram

noreply@blogger.com (Rajeev Singh) — Tue, 13 Jul 2010 00:00:00 +0000

As I spend more time with leadership at some of my client organizations a theme has started to emerge in our conversations. An aspect of that theme is the curiosity about the preconditions of Agile; "What is the sine qua non of Agile?". A variant is the line of questioning on what I look for in an Agile environment. My summary of Agile in three words: Creative Empirical Mindset. Here's what I mean:

Creative: Regardless of the current stance, people leading agile adoptions are, usually, after a new approach to deliver business value. The only realistic and sensible path forward, quite convincingly to me, is to break the routine. This may include, for instance, suspending the division of labor while approaching problem solving within teams. In the interest of getting rid of the historical hangovers of the past, highlighting the unproductive and unfriendly practices, new ideas, and objective feedback are often the best place to start. Unfortunately, the pressure-cooker environment that has come to be in businesses, the approach to "manage" people, and the sophistication of services provided seem to hint at specialization in skills, maximizing their utilization, and staying focused on the "best practices". This has left little room for tolerance of mistakes, missteps, or adventures, which are expected byproducts of creativity. A valuable observation in creativity I have come to admire, watching a few good leaders in the field,is to stop being antagonistic to those who are.

Empirical: This is one of the more subtle concepts to tackle in the Agile sphere. Agile or not Agile, empirical approach is utilized in a lot of environments. Agile empiricism is contextual to creativity and to the practices being followed by team(s) on a daily basis. Empiricism in an agile environment is the lever to direct and better utilize creative aspects of Agile and for the team to have its own self-awareness of sorts. Is the team able to, motivated by a desire to continuously improve, reach empirical conclusions?

Mindset: In contrast to a process, which tends to codify a sequence of steps in order to reach an end goal, Agile teams are driven to confronting novelty and improvise. In effective Agile teams that craft software, I have not seen a propensity for processes. There almost seems to be no need for it. Often, as I have noticed, the people engaged in these teams don't need one. They are motivated to deliver a good product and seem to know what it takes to get one built and running. They abhor "process" because it restrains them and injects delays in their work. Most importantly, it takes the very soul out of work, because it leaves little space for the thoughtfulness to prosper.

Now, as a few of my esteemed colleagues have pointed out, everyone has their definition of Agile. I'd be curious to know what is yours.

Agile Abstinence: Forbearance in Leadership

noreply@blogger.com (Rajeev Singh) — Wed, 07 Jul 2010 23:57:00 +0000

"When should we avoid Agile?" At my last client the President of the company asked me, "When does Agile not work?" It seems that a blog post is on the topic may be useful. Given that Agile is a continuum of an effort to foster communication (technical and social), increase collaboration, drive efficiencies and increase quality, I understand and translate these question as, "What are some of the significant challenges and barrier to Agile adoptions and sustainability?" I want to call out those leaders who understand and acknowledge these challenges, set right expectations with their sponsors, and take a cautious approach forward. And hats off to those who decide not to do go down the route until they get the right technical, social, and political infrastructure in place. Back to the question; factors that pose significant challenges to agile adoption are:

Dedicated teams(s): Agile teams thrive in communication rich environments. As communication suffers chaos starts to creep in. A lack of affiliation to a team or perceived split to many teams makes it worse. Team members do not bond well with each other and commitments remain shallow that result in poor quality of outputs. Though many factors contribute, this drop in quality is, in most cases, correlated to lack of communication. Agile adoption becomes an uphill battle and ultimately becomes counterproductive. Utilization metrics are a political heavyweight and sway many decisions in favor or "resource" split between different teams. Poor communication follows not long after.

Big Opaque Software Frameworks: In context of teams that produce software, ownership, access and control over the code being written is vital. If developers can't modify this code base to suite their needs and hook various tools into it they can't really do much. Basic practices that contribute to stability and quality of software like Unit Testing are hampered. Extensive upfront design becomes mandatory in rigid software platforms. Big ERP and CRM applications fall into this category.

Systematize Agile: Systematization of Agile adoption within an organization are laden with a high risk of being detrimental to its success. It curbs the "inspect and adapt" nature of Agile. Agile adoption will become painful and disappointing if it's template-ized along the lines of some prevalent industry certification models, specially the ones that have over emphasized/specified processes and documents.

Untamed Team Distribution: Lack of interaction between development team and business stakeholders adversely impacts the feedback cycle and affects quality of the product. Development velocity takes a hit and ultimately feedback cycles lengthen to a point where either rework increases or a team's progress is adversely impacted. Agile leaders in the industry have figured out approaches to make Agile work in distributed environments but only when a team's distribution is purposefully designed, i.e. they have the luxury to choose the people on the teams, their physical location or have a say on the rotation schedule of people between locations and teams. The persuasion to form teams with individuals that are dispersed between multiple locations and "staffed" to be available might backfire and lead to woefully suboptimal value.

Right sponsors: Agile practitioners who typically focus on consulting in Agile enablements or organization transformations often debate about a strategy to spread agility within the organization. Top-down and bottom-up camps exist. I think, regardless of the approach, the common denominator has to be the change of the traditional value models. Adherence to best practices and business processes is still rewarded. Agile enablements are usually not served well by these practices and processes and call for a change. If the leadership of an organization is not yet personally ready to sponsor that risk taking the outcome is fairly straightforward to guess; bad news and a bad aftertaste.

Right people: Some of my previous posts talk about the people side of Agile adoptions. It's the people that matter; more so in Agile than any other transformation that I have seen or read about. Agile practices are extremely people focused. The orientation of systems and decision making is designed to make it an extremely fast paced learning environment that is heavy on communication and busy with collaboration. It almost warrants frontier individualism of teams.

It is likely that this post may cast a doubt about Agile and its fit for certain kinds and sizes of organizations. That's not my intent. It is merely to highlight that pioneers of Agile are now starting to experiment on the above stated fronts. Some leaders in the industry are aware of this trend and choose to wait and let the Agile community learn valuable lessons before they adopt it. They don't fear being cast as late majority or even laggards. I have quite a bit of admiration for such hardheaded and matter-of-fact decisions. Image taken from: http://wizard.webquests.ch/pics/upload/98/frontier_400.jpg

Building an Agile Ship - A Taoistic Approach

noreply@blogger.com (Rajeev Singh) — Tue, 26 Jan 2010 14:00:00 +0000

Big challenges in building agile teams, in my recent experiences, have come not from lack of Agile knowledge on team's part. They have come from a lack of co-operation that the team members have demonstrated within the team. Seemingly unrelated frustrations like team member's tuning out of critical meetings, not being able to learn new concepts, and even bigotry can be attributed to lack of co-operation. Strategies that I have successfully utilized to increase cooperation, which comes at premium, are:

Focus for Agile education and practices should be on all the roles including managers. By design, most focus of education and coaching should be on hubs that have the most nodes connecting to them. Those are typically managers in teams. In addition to setting the right precedent of leading by examples, managers then set the reciprocation cycle in motion.
Select an Iteration Manager (IM) who sees the role as a challenge, views it as a progression, and is an early adopter of new concepts in general. This will keep the energy level high in the team, and catalysis will keep the team going.
Choose a person that is closest to the team and is liked the most in the team as the IM. In cases where outsiders are the IM, the role should be transferred to one of the team members as soon as possible.
Keep the team size as small as possible. Small teams are most likely to exhibit voluntary cooperation, which increases as communication increases.
Consider a person's personal threshold to adopting a new mindset. It is usually difficult to enable a team of 15 people for Agility with one Agile coach. A few coaches interacting with the team as opposed to just one or two is likely to be compelling enough for potential adopters to give Agile a try. Teams should be seeded with enough critical mass of Agilists who can weigh in on Agile topics, demonstrate the benefits, and convince the skeptics.
It is critical to recognize that learning is a co-operative activity. I have discussed the importance of Socratic methods to enable Agile teams in one of my previous posts. If there's not enough bandwidth of coaches, satisfying Agile adoption in a team is unlikely.
Assemble a good team to start on the Agile journey and learn the right lessons; good or bad. It is extremely important to "design" the Agile team as opposed to starting with just any. This is in line with Maslow's "growing-tip statistics". Maslow asserted that it is the growing tip of the plant where the greatest genetic action takes place. He leaned towards studying and focusing on the best. I paraphrase:

"If we want to answer the question how tall can the human species grow, then obviously it is well to pick out the ones who are already tallest and study them. If we want to know how fast a human being can run, then it is no use to average out the speed of a "good sample" of the population; it is far better to collect Olympic gold medal winners and see how well they can do. If we want to know the possibilities for spiritual growth, value growth, or moral development in human beings, then I maintain that we can learn most by studying our most moral, ethical, or saintly people."

These strategies will solve a few key problems that we observe in environments starting to practice Agile. These are environments where the propensity for Squirrel Agile is high. Bad and selfish proclivities like wanting to be Agile without being Agile are common. Other such common regressive behaviors often include:

Free Riding - Some team members get by without participating or working to make teams agile
Lack of commitment - Dismissing Agile from the get go or losing faith too early
Lack of discipline - Teams cut corners and don't follow the agreed upon practices
Lack of initiative - No one steps forward to carry out tasks, e.g. daily standups not held because Iteration Manager was out
Contingent co-operation - I yield this much of my "comfort zone" to you, you cede some of your Agile practices to get my co-operation

Some design principles addressing the above issues that I have followed for Agile teams are:

Define clear boundaries. On one of my projects at a large corporation we drew a virtual garden wall around the project teams. It was clear to every stakeholder which people were in an Agile project and which were not. The folks within were to a large extent insulated from the business-as-usual outside of the garden wall. Ideas still permeated through the wall and people from outside could peek over to see what the Agilists were doing. But it was evident to everyone the teams withing the garden were different and they were experiment with news approaches of operating.
Rules should match the team's needs and conditions. Agile teams should be able to adapt the rules of the game to match their needs and organizational environment. On a project that had unsustainable meeting schedules, we reduced the frequency of some meetings, recommended increasing the details of some documents so that distributed teams can get more of their questions answered. The latter goes against the conventional wisdom of Agile, but it suited the needs of the team.
Individuals affected should be able to change the rules. Agile teams should be able to form the rules for themselves. Changing rules should be a mutually co-operative endeavor; not a decree from senior managers or external consultants. External authorities should respect rights of the team members to devise their own operating principles.
Leverage reputations. Reputations are often the most powerful tool to utilize. On one of the teams that I worked with, I went out of the way to find out who's respected and valued for what in the teams. The next obvious step was to connect their passions with project tasks that needed to be accomplished and had them sign-up for these task in the team's and stakeholder's presence. This practice was then transitioned to the retrospectives and sustained there for iteration level tasks.

The temptation of changing a team and radically altering how it works is too great for the Agile coaches to resist. It gets complex as the purview of a coach increases and the size of the organization gets bigger. There has to be a reason, almost always, why seemingly bureaucratic processes exist and are followed in client organizations. Often such elaborate "processes" are the communication mechanisms and the eyes and ears for the leadership into what's going on in the company. Such processes are central to governance. It is wise to acknowledge the fact and evaluate how it can be adapted (not rooted out) to accommodate Agile projects. An approach to introducing and experimenting on Agility with this mindset is better suited to generate co-operation from the traditionalists in large environments. Image taken from: http://commons.wikimedia.org/wiki/File:Tao_character.svg

Humanics - The Next Frontier of Agile Adoption

noreply@blogger.com (Rajeev Singh) — Mon, 31 Aug 2009 16:55:00 +0000

Agile is a reusable and, for most part, a transplantable strategy. It's easy to replicate, simple to understand and well socialized. It is comforting to those who have successfully adopted it. Yet, to others it is elusive, at best. The discrepancy between adoption rate and success rate is stark.

Pockets of Agility Numerous Agile adoptions are identical but few bear success. Reasons are simple; change agents ignore the "humanics" and are often reluctant to make Agile the central philosophy of the enterprise. Pockets of experimental runs produce some compelling results that can be attributed to the purposeful rigor that Agile instills. Some observations about Agile:

Its rigor is almost an asperity to some,
The "just do enough" philosophy seems counter intuitive to software "engineering", and
Its lightweight nature makes the optics of Agile ad-hoc.

For the above mentioned reasons, it is sometimes difficult to scale. To make matters worse, several Agile adopters still lack the will to formulate a strategy to scale. Organizational momentum of multi-year product development plans and cultural inertia are formidable challenges. (Interestingly enough, it is these very environments where I have also seen Centers of Excellence and Communities of Practice on Lean. This goes back to the problem of disconnect that still exists between Agile and Lean.)

What's in it for me?

The root of the problem, I think, is that Agile proponents haven't clearly answered the question "What's in it for me? Why should I conform?". The business case for Agility attempts to leverage the "better quality working software faster and frequent to market" and "adaptability" propositions. So What? What's in it for me if I belong to a testing organization in a ten thousand people organization that is regulated and has little competition? Why should I care about Agile? This is purely a "humanics" issue. And, it shouldn't be a surprise that even the C-suite has the same question. They didn't "grow-up" with Agile, afterall.

Success requires Heterogeneous Audience

A critical insight that Agilists, including me, get into and grasp a little late is the desired composition of stakeholders to succeed. This vulnerability has single-handedly and more definitively caused more Agile adoption failures than anything else. The pithiness of Agile value proposition hasn't reached the board rooms yet, as Lean in manufacturing has. It'll be a while before Agile recommendation are dusted off and looked at seriously. In the interim, the Agile community is focused and busy honing the mechanics of Agile. It is steadfastly employing the technical infrastructure of Agile in project teams and training the minds of future. Projects are still aborted and hopes are still dashed. The onus on the Agilists now is to come clean on the extent to which Agile can provide benefits and what to realistically expect in absence of or inadequate "air-support".

The Psychological Art of Agile

As mentioned in one of my previous posts, Agile has ceased to be a technical challenge. Growing aspects of its social challenge pose a bigger uncertainty and pale the technical complexities. The current frustrations aren't as much around the capability development and mechanics of Agile as they are about the value system. Discussions on scaling are heavy on the people side of issues. We are slowly seeing a trend where sustaining Agile is becoming a psychological art. There are some people who are cut out for Agile, others simply are not. "Agile is not a skillset, it's a mindset", as one ThoughtWorker says it. With that lay of the land the question becomes even more important - how do we scale? What's the people strategy for those who don't have the Agile disposition? A good deal of intelligent and responsible attention has to be given to why some hold an unfavorable disposition to Agile and how can they be favorably persuaded. The answer, it seems, lies in self-awareness of the teams. 'Self-directed Team' has been a long standing notion in the Agile community, albeit somewhat confusing and misunderstood. Before being self-directed we need the teams to be self-aware of what's unfolding around them; their impact on project(s), where's their stock with the management, and their contribution to long-term technical capability-development. Without answers to these questions teams cannot be self-directed.

Connect team's vision with the corporate vision and tie team members' aspirations to the vision. This would answer the question, "What's in it for me?". Managers should complement the Scrum Masters in ensuring that they clearly communicate to team members what they stand to gain from the transition and the skills they'd build.
Next up, I'd recommend an old trick that works most of the times. Stop selling Agile. Instead, start asking why Agile would not work. Ask it often and induce people to think about it. Focus their attention on Agile as much as possible.
Teach people something about Agile at every opportunity you can find. Remember, focus is power, expectation shapes reality, and attention density shapes identity. Use the powerful conduit of education (conferences, guest speakers, etc.) as a means to persuade. A word of caution here though. There's a stark difference between education and training. Don't undercut your returns by mere training people; it won't last long. To create an environment of learning and bring a lasting change engage industry experts who see themselves as instruments of cultural indoctrination as opposed to just trainers.
There always are some who don't absolutely embrace new concepts and systems. Then there are also a few who just don't get it; like I don't get Quantum Physics. Find these people something else to do. Get the ones who understand and are passionate about Agile on your teams. Yes, teams are formed; they don't happen by chance. Don't be timid about team formation. This is a controversial thought employed too often and creates havoc for so many careers. More on it in later post(s).
Don't go piecemeal with Agile Adoption on a project. There's no such thing as setting up the right Agile management structure (Scrum) first and then introducing the engineering practices (if at all on a project). This will frustrate people and signal a half-hearted and fearful leadership style.

This list is far from all-inclusive. A cursory study of the partial list is sufficient to support the argument that planning for success should prepare the participants to deal with variety of mental states, outlooks, and behaviors and not just tools, technologies and processes.

Conclusion

Agile has come a long way in the last decade. It now has a foothold in the mind of leaders, innovators, and strategists. It's popularity is growing, but so is the list of failures with Agile adoptions. The long trial-and-error phase is over and we are entering into a phase of trial-by-masses. Soon (5 years) the polarity of Agile acceptance will increase manifold - a quasi-official product development approach at some places and banished from others. Illusions and myths will follow. Humanics will, very likely, emerge as a pivotal factor. Agile adoptions with a pure-play technical focus will languish unless the change agents demonstrate good knowledge of the accumulated humanics lessons and leverage them. Image taken from: http://designbivouac.typepad.com/designbivouac/images/dresden_8.jpg

Mindful Change - Beyond Behaviorism and Humanism

noreply@blogger.com (Rajeev Singh) — Thu, 08 Jan 2009 19:10:00 +0000

My last post discussed steps to sustain the goodness introduced by a transformation. As an abstraction, introducing Agility is an organization transformation (OT) process - an important one in software industry. The famous trifecta of any OT initiative is People, Process, and Technology. People first!

Leading people to change in order to transform an organization into a more efficient, productive, profitable, pragmatic, and agile business is tricky and often left for the very last. At best, I have seen managers read and recommend literature that deals with behavioral science - psychology, organization theory, and sociology. I always felt that this is not enough, although far better than not being cognizant of or sensitive to people issues or using brute force for business transformation or in OT.

Recently, I read The Neuroscience of Leadership , which is an excellent primer on the next frontier in organizational transformation - cognitive science. This writeup concludes that, and I quote:

Change is pain. Organization change is unexpectedly difficult because it provokes sensations of physiological discomfort.
Behaviorism doesn't work. Change efforts based on incentive and threat (the carrot and the stick) rarely succeed in the long run.
Humanism is overrated. In practice, the conventional empathic approach of connection and persuasion doesn't sufficiently engage people.

The next three points, which can be treated as prescription are significant. Again, I quote:

Focus is power. The act of paying attention creates chemical and physical changes in the brain.
Expectation shapes reality. People's preconceptions have a significant impact on what they perceive.
Attention density shapes identity. Repeated, purposeful, and focused attention can lead to long-lasting personal evolution.

What does this mean for OT practitioners?

The first message is that behaviorism and humanism can be and should be leveraged only so much. Second, if change is pain then it should be done as fast and as quickly as possible. I prefer an orderly big-bang; a carefully spreaded out step-by-step rollout will not only delay the reap but agonize the people too.

Onboard OT teams with 'right' people. There's those who can see themselves operating in a new and better environment, and then there's those who are extremely comfortable with status-quo. The latter are a detriment to change initiatives.

How could change be lead by people who don't know why they are changing and for who? OT leaders and participants should also nurture a vision to make their services/products more competitive. The expectation to make services/products more appealing for consumers and the business more efficient and productive is essential for OT success. Also, include people from different functions of the business - marketing, accounting, project management, customer service, to name a few.

It takes multiple sessions/workshops for people to understand and remember concepts. The root cause is not participant's IQ but their lack of focus and sometimes the learning material not being insightful enough. 'Induce others to focus their attention on specific ideas, closely enough, often enough, and for a long enough time.'

I don't know of any successful transformation that was a one man show, even if it was glorified as such by media. Transformations and turn-around are executed by teams that have a critical mass, the appeal within the organization, and the know-how of the business. Hence, OT initiatives will not be successful unless change is perceived to be necessary by others. Therefore, embarking on a OT journey can be a decision that a CXO can take, but the demand has to be generated from the bottom first. Invest in creating an environment where this demand gets generated. A good consultant would know how to get there.

Image taken from: http://watarts.uwaterloo.ca/~celiasmi/images/brainclimb.jpg

Recipe for Agile Sustainability - Don't Forget Process Irreversibility

noreply@blogger.com (Rajeev Singh) — Fri, 12 Dec 2008 01:00:00 +0000

So, you adopted Agile in the past. It was also a success by your yardstick. Sustaining Agile transformation, however, has proven out to be a challenge. There's also challenges with scalability.

It's not an unusual challenge. As with any change process, Agile transformation requires checks and balances with an eye on building process irreversibility. Only then would I consider an Agile transformation/adoption to be really successful.

Balanced Teams

Amongst the building blocks of Agility are Teams. An often overlooked component of teams are people and their inclinations. There's often members in Agile teams that are of traditionalist persuasion (some of my colleagues call them Skeptics, although I disagree with the term). They can't be convinced and changed despite overwhelming evidence (which distinguishes them from Skeptics); that's what makes them traditionalist in the first place. The idea is to balance the Agile teams out with Generative Thinkers - those who can model their business and processes with innovative practices.

Balancing teams out is the first step towards process irreversibility. Some other steps are:

Individual Retrospection: In one form or the other a question I often encounter is, "What'd you focus on in a team that is already Agile?" Individual Retrospection is always top on my list and is an offshoot of sustainable pace. It is not the same as team retrospectives. If a team has a sustainable pace, its members will have time think about what they are doing and how they can improve it. Individual retrospection is highly undervalued in the industry and often has a negative connotation. Traditionalists interpret it as underutilization of resources. It is considered elitist and a privilege of managers.
Meeting Facilitation Training: Sliding back into old ways of doing things is common. Teams often loose steam which results in bad behavior creeping back in. Meetings are the first to suffer. Lack of attendance, lack of focus, too many meetings, and a general confusion about the objectives of meetings are some of the first signs of old ways creeping back in. Team members trained in meeting facilitation skills and techniques prevent this backsliding.
Metrics and Reports: As in any change management system, there's got to be reason why Agile was considered. Those reasons should be tracked using some metrics by the management. Agile also recommends some metrics that should always be tracked, even if they were not on management's radar. Success of Agile adoption should be measured by these metrics and nothing more and these should form the Agile Dashboard by which business should be run.
Agile Evaluations: A good management practice associated with Agile rollouts is periodic Agile evaluations. The focus of the such an evaluation is on different aspects of Agility, e.g. Responsiveness, Simplicity, Configuration Management, and many more. .
From Flavor of the Month to Business As Usual: The more seasoned a team member is the more compelling it is for them to continue being who they are and to follow practices that have made them successful. There's quite a few other team members who want to be like these successful Einsteins. Emulating their stances is only natural for the less experienced. That's the DNA of a corporate culture. Convincing the Einsteins that Agile is not a flavor of the month but here to stay and soon to become a norm, is paramount. This message should be loud and clear.

Agile allows team members to participate and control their course. At the same time it requires discipline. There's no magic in Agile that makes it ceaseless, unless process irreversibility is intentionally designed for its adoption.

Consulting Tools - Thinking, Modeling, and Inquiry

noreply@blogger.com (Rajeev Singh) — Mon, 24 Nov 2008 17:43:00 +0000

Ever wonder what tools make a consultant effective? It may be a long list. If you read The Opposable Mind - How Successful Leaders Win Through Integrative Thinking, it is obvious that consultant would benefit from most of the process techniques that Roger Martin talks about in this book. The three that stood out for me are:

Generative Thinking - Pondering along the lines of 'what might be'
Causal Modeling - What causes Something, and what's the purpose of that Something
Assertive Inquiry - Asking questions that encourages dialog, rather than shut it down

The Process of Thinking and Deciding is a good revelation of how a lot of us reach decisions. Recollecting my interactions on past engagements, it is quite obvious that these patterns have serviced me well. I was, until now, unaware of their interdependence and collective power.

Going forward, it'll likely serve me as a good framework for interviewing candidates who aspire to be consultants. I have never been subjected to an interview myself where someone tried to assess these skills. That clearly demonstrate how much little we understand and value the utility of these tools in our professional lives, or so it seems.

Image from: http://www.childcareaware.org/images/category_icons/tools.gif

Got Agile?

noreply@blogger.com (Rajeev Singh) — Tue, 04 Nov 2008 12:24:00 +0000

VersionOne released The State of Agile Development report for 2008. A few days ago I wrote about barriers to Lean implementations, which is available here. It is interesting to see that the barriers to further adoption of Agile are not very different from those of Lean implementations. 44% attribute it to "General Resistance to Change", 45% to "Ability to Change Organizational Culture", and 32% to "Management Support".

To avoid stalling, as I read it, it is important to concentrate on the soft issues as much, if not

more. Experience and know-how of Agile technicalities is not sufficient to ensure success.

It seems that Agile has reached a point where it has ceased to be a technical challenge. Agile is now a cultural challenge. Either address the fundamental issue of resistance to change, be ready to get mediocre returns for your buck.

Lean Implementation - Skills Needed?

noreply@blogger.com (Rajeev Singh) — Wed, 22 Oct 2008 19:43:00 +0000

A Lean Enterprise survey in 2007 reflected that about 36% of lean implementations face middle management resistance as an obstacle. Approximately 28% face employee resistance. Lack of implementation know-how came second at 31%.

This is, of course, a survey conducted amongst organizations engaged in lean production. Humans being humans, I would guess that these numbers won't be very different in Software and Services industry. Basically, any change program involving humans will face challenges that are similar in nature and magnitude.

So, what skills do you think a change agent would need? Seems to me that technical knowledge is only a third of the solution.

The Episodic Sprint

noreply@blogger.com (Rajeev Singh) — Sat, 06 Sep 2008 01:00:00 +0000

One of the first thoughts on hearing the word Scrum - 4 weeks to a Sprint. To some this is music to ears, but to others not so much. And who are these "others"? Usually, it's the Product Owners (PO). While developers want a sprint to be locked out, POs want to be able to add/remove/swap stories in it. Scrum's answer to that is simple - avoid it. If you can't avoid it, flush the sprint and restart. The inspiration behind that recommendation might have been to discourage changing the sprint in-flight.

Where's the Agility? Isn't a locked sprint bit contrary to the whole concept of agility? What's the point in locking out if the underlying philosophy is to be able to adaptive? Clearly, it's a conflict of interest between the developers and POs. One way to avoid it is to reduce the sprint length, to let's say 1 week. Now, this may be a mighty fine deal for the POs, but it not always the case for developers. One week may come across as a bit rushed to them to produce anything meaningful in some contexts. So, what's the best way out.

Multi Episode Sprint A good compromise can be a half-locked sprint. In a four week Sprint plan and lock the first two, plan but keep the last two open to changes. One of my colleagues at ThoughtWorks calls it an Episodic Sprint. If there are changes mid sprint, have a sprint planning meeting after the first two weeks to re-plan the remaining two. Episodic sprints, in my opinion, are a step towards creating a software development environment that prevents itself from being fed half-thought through and low priority stories but still accommodates for genuine changes. For teams that are just starting Scrum, I would highly recommend Episodic Sprints.

Opportunity for Increased Agility

noreply@blogger.com (Rajeev Singh) — Mon, 09 Jun 2008 15:14:00 +0000

Being a big fan of Agile, I didn't miss the opportunity to present my thoughts at the Microsoft Community Summit - 2008 in Orlando, on June 7. Thank you all for your eager participation. I throughly enjoyed the questions. Based on your feedback and the questions you asked, I have modified the presentation a bit. It is available at http://www.slideshare.net/udairaj/rapid-project-inception-456123 Please do not hesistate to contact me at rsingh@thoughtworks.com with any questions, comments, or suggestions.

To Re-estimate or not to Re-estimate

noreply@blogger.com (Rajeev Singh) — Thu, 15 May 2008 05:00:00 +0000

A project is incepted with a story list that has all the stories estimated. The team has completed 4 iterations and there are 8 more to go. The team also has usual churn of people. Are you in favor of or against re-estimating stories during next sprint planning? Practioners are on both sides of the fence on this one. Some recommend re-estimation and some want to stick to the original estimates. A colleague of mine recently explained why he is not in favor of re-estimating. He said, "I don't want to have to explain to the project sponsors why the project estimate just changed from 133 points to 97. Do you think the scope has changed?" That was a very insightful discussion. He then said that if the team is not finding the big stories as big as they initially thought, it won't matter. That simply means the team is now burning more points of a larger scope number. This will proportionally be the same as burning less points of a smaller re-estimated scope. Re-estimation is a tough thing to convince people on. I still see a huge benefit in it.

The idea of re-estimation is not about lowering or increasing the numbers; it is about estimating the relative size of the stories.
The stakeholders/sponsors, ideally, should not be concerned about total point scope. This is an internal number for the Dev team. Sponsors should just be concerned about the rate of progress towards delivering a product.

As a sponsor, I'd be more interested in the secondary estimate, i.e. time. When will my product be ready for release? It really doesn't matter so to me if this scope is estimated at 150 points or 800 points. Re-estimating the stories should be encouraged because it:

Keeps the relative size of stories up to date,
Makes it almost mandatory to revisit the release plan, and
Promotes conversation.

Relative size is a big thing to watch out for in situations where new stories are added or existing ones are split. The reality of agile greenfield software development is that the initial brainstorm hardly ever reveals all the stories. Total effort, as estimated, at the beginning of the project will most likely change (increase). An initial estimate that is not changed with the lessons that have been learnt in the last few iterations is not acknowledging that:

Work done in the last few iterations can be leveraged,
Assumptions with initial estimates may have changed.

Typically, teams start with developing the simplest functionality first. Let's assume that Story A is the simplest functionality and is 1 point. And, the initial estimates had story B estimated at 3 points thinking that this is three times the size of A. Why would I not want to re-estimate B as a 2 if I now know that is twice the effort of A and not three times the effort?

One of the biggest benefits of re-estimation is that it reevaluates a team's progress towards the end goal and indicates if you are on track or not. Not re-estimating will increase the probability that one of the stories a few iterations down with an initial estimate of 2 points will now take 5 points worth of effort. There may be many more such sitting in your pile of stories. Example Problem: Let's say that you have a 1000 sq. ft. backyard that needs to be landscaped and upgraded with a water fountain. You hire a contractor and (s)he gives you the following story list:

Tilling out existing grass - 2 pt
Discard old grass - 3 pt
Transporting grass to backyard - 1 pt
Laying out and setting the grass - 4 pt
Edging the grass - 1 pt
Post installation cleanup - 2 pt
Fountain Installation - 2 pt

That's a total of 15 pts. Your contractor also tells you that last time they did a similar backyard it took them a day to till and clean (stories 1 and 2). Based on that the plan will be:

Day 1 - Story 1, 2
Day 2 - Story 3 , 4, 5
Day 3 - Story 6, 7

You as the sponsor/customer now have a plan and expect the backyard to be ready in 3 days (iterations) assuming a 5 pt velocity based on the contractor's past experience.

Work starts, and you now request the contractor for two new things. You want the lawn to be leveled in a different gradient and you also want the soil to be fertilized. These are two new stories that get added to the story list.

While tilling, the contractor also finds out that the soil has lot of gravel and stones in it. Scenario 1: No Re-estimation When there's no re-estimation, the story list will likely be as under. The team will estimate the new stories.

Tilling out existing grass - 2 pt
Discard old grass - 3 pt
Transporting grass (sod) to backyard - 1 pt
Laying out and setting the grass - 4 pt
Edging the grass - 1 pt
Post installation cleanup - 2 pt
Fountain Installation - 2 pt
Re-level lawn - 1 pt
Fertilize Lawn - 1 pt

Total scope is now worth 17 points. Based on a velocity of 5 pts. the team will sign-up for stories in the following order:

Day 1 - Story 1, 2
Day 2 - Story 3 , 8, 9, 4A
Day 3 - Story 4B, 5, 6
Day 4 - Story 7

I split story 4 into 4A and 4B, both of which are 2 pts and they encompass laying out and setting grass in half of the lawn.

From the plan it seems that the team will take 3.5 days to finish the job. However, story 3 & 6 have not taken into account the fact that now the scale of the problem has changed. It is just not cleaning and hauling grass, but also gravel and stones. Scenario 2: When we Re-estimate When we re-estimate, the story list will likely be as under. The team will estimate the new stories.

Tilling out existing grass - 2 pt
Discard old grass - 3 pt
Transporting grass (sod) to backyard - 3 pt
Laying out and setting the grass - 4 pt
Edging the grass - 1 pt
Post installation cleanup - 4 pt
Fountain Installation - 1 pt
Re-level lawn - 1 pt
Fertilize Lawn - 1 pt

Total scope is now 20 points.

Story#3 is now 3 points because the team just figured out that transporting sod to the backyard in absence of a wheel barrow is not 1 point but 3 points. Story#6 is 4 points because post installation cleanup is not just grass and dirt but also gravel and stone cleanup, without a wheel barrow. Story#7 is now 1 point because we just gained plumbing skills and found a water pipe running in the yard, closer to the fountain.

The new plan is:

Day 1: Story 1, 2
Day 2: Story 3, 8, 9
Day 3: Story 4, 5
Day 4: Story 6, 7

Traps of Not Re-estimating

When we don't re-estimate it seems like we can finish the job in about 3.5 days. If the Product Owners request additional scope of 2-3 points, I would have no problems accepting it. My plan tells me that I have a little over half the day available.
A new story estimated at 1 pt should roughly be the same as an existing story of 1 pt in effort. But it won’t be the case here; at least it won’t be reflected in the metrics. The idea behind agile project metrics is to make knowledge very explicit. Not re-estimating fails to achieve that goal. Estimate and velocity metrics become very tacit.
As a team member my level of trust in the estimates goes down. There are two 1 point stories, but they are not the same effort. One is a new story and, hence, has a new estimate, whereas the other is an old story and is not re-estimated.
There may be stories that have initial estimates that are much higher than the effort expended on them now. These may not be included in Sprints because they will increase the Sprint sign-up points to go beyond the target velocity. The tendency is to avoid signing up for these stories.
Agile Software Development is also about adapting. Not re-estimating will lead us to become non-adaptive.

On big projects where number of stories for each release easily run into 3 digits, these traps amplify themselves. The benefits dervied from re-estimation may not always be so obvious, but they for sure increase the level of comfort and confidence in the team.

Picture taken from: http://www.stellman-greene.com/blog/wp-content/uploads/2007/05/schedulereview13.jpg

Analyze on Software Not on Paper

noreply@blogger.com (Rajeev Singh) — Thu, 01 May 2008 03:00:00 +0000

Let's visit a very common conversation that I have often been involved in or heard of: Q: What's the difference between Waterfall and Agile? A: Agile doesn't involve the upfront analysis that Waterfall plans and invests in. This is probably the first thing you hear in terms of differences between Agile and Waterfall. The follow-up question usually is, "How can you do software development without any analysis?" It starts the classic dialog that a lot of agilists repeatedly have. The Luck Factor How can we really deliver business value without 'proper' analysis? We may be able to deliver some, but it'd just be a matter of luck. That's exactly the inspiration to invest in upfront analysis; eliminate the factor of luck. The temptation to know it all and plan everything is just too much. Sometimes, before you even realize someone reminds you that your planning and analysis is starting to look like Waterfall. Not planning, I think, is wrong. Not analyzing is a blasphemy. You are wasting money and time if you are not planning and analyzing. But if its all on paper, powerpoint or visio and planning your software to the Nth possibility you are most likely about to run into a lot of potential waste. Release Zero The solution to this paradox is to have a release of whatever you are tyring to build as soon as possible. Get that extra-lean yet functional piece of software out to your small test-user base ASAP. A colleague of mine termed it 'Release Zero'. In addition to a lot of usability feedback, this would provide your analysts a working system that they can analyze. Scenarios, situations, dead ends, etc. that I as an analyst have come across in actual working software could never ever have been thought of if I were to design theses flows in detailed functional requirement documents and system design documents. The stretch of imagination needed to design a good system on paper becomes so thin and flaky that it starts to consume extraordinary amounts of energy and time to make them useful. So, do just enough, think only so much ahead in time. Before you proceed any further on that theoretical trajectory, get it built. Do your analysis on this release and iterate further. Example Imagine designing a content management and publishing system where a user should be able to:

Create article, etc.
Publish, revise, republish articles.
Rate and comment on articles.
Search on articles
Provide sneak peak into some work-in-progress articles to readers.
Workflow between writers and editors.

This can prove out to be a very complex system to design if we introduce the concept of user groups. i.e. a group of Writers, Admins, etc. in addition to the Readers. Now let's say there's another user group in it called Classified. And let's say that this is a group of users that writes very sensitive information that shouldn't be leaked to the rest of the publishing house and certainly not to the readers before the due date. Imagine how complex the analysis can get. Content should be available to a certain group for editing before the publishing date, but to others after it has been published. Search should not return classified material or should return only parts of it to some readers. Internal editorial comments should not be visible to the readers and/or vice-versa. As an analyst this product would be a big challenge. As I said earlier, my analysis will start becoming flaky beyond a certain functional depth. Decision on functional aspects will start taking long time because product owners, designers, and developers will have very little common understanding. Hypothetical and imaginary usage scenarios will start to consume time. In this situation I'd recommend to start development with a simple model in place. Two user groups - editors and readers, simple search, and basic data privacy and security in place. Once I have this system in place in 3-4 weeks, I can start analyzing on this system and have users test it. In this paradigm, there will be frequent releases and the analysis will be based on the last release. It may or may not be available to the end users yet for feedback. It won't be a surprise if a lot of the current requirements may not be requirements anymore 3 months down the line. Having a working system around to play with will likely make some requirements unattractive or not worth the investment. Image taken from: http://www.time.com/time/magazine/article/0,9171,1630556,00.html

Excellence in In-House Software Development

noreply@blogger.com (Rajeev Singh) — Mon, 21 Apr 2008 13:00:00 +0000

One of my client groups recently had a meeting in which they briefly discussed what it means to be excellent in software development. The client is an IT arm of a company whose staple is not software product or services.

It was interesting because this group's definition of 'excellence' did not include a few key points. I feel bad for these key points being overlooked. It's probably because my business is software development whereas software is a means to a different end for this group.

Here under are some attributes that would constitute 'excellence' in my opinion. In a software company or a software support arm of a non-IT business, I would strive to achieve an environment of:

Transparency - This could be anything from who's busy doing what to how the past efforts have been impacted the business. It never hurts to update your team on why certain decisions have been made. Transparency is the foundation of excellence.
Problem Solving - Often we find people say that they are problem solvers or that their's is a group of people that solves problems. I am not surprised anymore when on further investigation it turns out that most of them are patching problems up. Solving a problem is a bigger initiative. It, usually, requires bold steps and long-term vision without a greed for short term results.
Innovation - Continuing to solve problem the way you have been for the last 5 years, if at all, may not the best return. How have you, your team, your systems, and your processes become better and more efficient?
Entrepreneurship - Innovation and problem solving often remain in silos. Is there drive, energy, and enthusiasm in your innovators and problem solvers to take on the challenge to weed out inefficiencies from the system? Entrepreneurship usually is a response to an incentive of some kind; tangible or intangible. Create a few to foster it, if you haven't already inherited some.
Solution Focus - Enthusiastic technologists, specially in IT, are very eager to try new products, services, languages, and platforms and want to build their skills in them. Sometimes this is the reason why we find hundreds of applications written in 10 different languages with 4 different database systems. This also happens to be a reason why we also find features in these applications that took months to build but are rarely used by very few in the user community. That happened because it was a cool thing to build. Gist is that, after all, it is not about software. In the end, it's still about a solution to a business problem. It's a tough job to balance the big picture with the innovation.
Investment in Knowledge Sharing - One of the big reasons that high-end IT consulting is such a big industry is because consultants seem to know the latest and the greatest. On a few occasions I have been surprised how much one of the client associates already knew about a certain topic. It was even more surprising to know that in some of those cases the managers knew that a certain employee is very knowledgeable but did nothing about it. Instead, they waited for the consultants to show up and charge high dollars for it. Give your people room, time, and incentive to share what they know with the rest of the community.
Discipline in Action - Innovation, Solution Focus, etc. become the culture only if there's discipline around it. Do it every time and at every given opportunity. Very few people realize how demoralizing and demotivating it is to find out that there's really no rigor around in the culture.

What would you add or take away from the list?

Picture taken from: http://www.gapingvoid.com/excellence224.jpg