The behavior that somewhat disturbs me is when every possible activity within a team is estimated. These estimates are subsequently utilized to provide a sense of comfort that value is being generated, even though very little working software is being crafted and deployed.

Let’s revisit some fundamentals of estimation.

Why do we estimate? What’s the motivation behind estimation?

A set of questions consumers often ask, “When will I get a piece of software? What capabilities am I likely to get in this time-period?” are age-old and not likely to go away anytime soon. We answer those two questions by either measuring something or estimating it. Since software isn’t like a manufacturing line of similar objects, we have to estimate for the dissimilar pieces of value-generating pieces (features) of software. Hence, we estimate to be able to answer those two questions.

Which activities do we Estimate?

We estimate only those activities that:

• Have constraints on them.
• Form the critical path, and
• (Where the path is the longest. More on that later.)

In software teams, the activity of writing and assuring quality of software is often the constraint and the critical path. We try to time-box the rest of the non-constraint, non critical path activities. We also don’t estimate the routine tasks, since we already know a whole lot about them.

Why don’t we estimate activities like analysis, documentation (user manual, etc.), UX, writing, etc.?

It’s a myth that we don’t estimate activities that don’t get estimated. We do.

The activity of writing good quality software happens to be the critical path to perceived valued. Software development is also the activity where more is routinely desired than what is often available (constraint). Documentation, analysis, UX, etc. are critical path, but often don’t encounter those demands for three reasons:

• We either agree to not start until we are ready with them, or
• We are good at time-boxing them.
• Some of these activities can also be done in parallel to software development.

So, it is not true that we don’t estimate activities like documentation. It is just that in our rough estimation, those activities are either significantly less voluminous then software development and quality assurance, or are not the constraints, or can be done in parallel, or are expected to be time-boxed, or we agree not to start development until we are ready with the outputs of those, e.g. analysis.

Ask youself why you want to estimate. If you fear that you may not be done by the time you want to deploy your product, then you should estimate. If you fear that you may be undertaking more work than the time available to you, then you should estimate. If none of these are concerns, don’t bother estimating.

Why don’t we estimate defects?

Estimates assume that the code will work. Defects are unfinished code. If software doesn’t behave as expected and someone discovers it later (defect), we assume that this defect wasn’t supposed to be there; that’s what you estimated for. When teams estimate a story, they don’t estimate it with a certain number of defects in mind. If you start estimating defects, be warned that this is a slippery slope.

Why don’t we estimate housekeeping/retrospective/improvement activities?

For the same reason we don’t estimate travel time to work, time to eat lunch, bathroom breaks, etc. We don’t estimate what’s routine and is expected to remain a routine (a good team is expected to be continually addressing issues that pop up). These activities are better being acknowledged as the cost of doing business.

Units of estimation?

Time (hours) and Points are two prevalent scales to estimate. Time is a deceiving unit to measure in. We are not good at tracking time (do you stop the clock everytime you take a coffee break or respond to a text/email?). We also don’t agree about how long a day is. Some think that a day is 8 hours, others like me think that a typical day at work is about 3.5 hour long. Since we don’t agree on the length of a day, we plan different. Time is also deceptive is because we associate different value to time. In other words an hour of meeting for you may not be work but for me it may contribute to my next two hours of work and hence valuable. In gist, time gets measured inconsistently and team members lack the discipline of measuring it well. Time also gives an illusion of accuracy that is difficult to achieve. Point scale acknowledges these inefficiencies and deception, and has given us an abstract unit of measurement that solves these problems. The concept of relative sizing brings about yet another simplicity to it.

Why time-box, if we have a simplified point scale?

We strive to remove as much variance from the system (especially the critical path) as possible. Even point estimates have variance that add noise to forecasts. Time-box on the other hand is a cleaner way to remove that variance. For activities that are difficult to estimate because they are unbounded, pick time-boxing over estimation.

Conclusion

Estimation is not a report card to show how busy a team is. Rather it is a tool to make pragmatic and wise decisions. To that end, it doesn’t matter if you use it to measure everything or a part . Teams that utilize the concept of estimation effectively report another benefit. Knowledge creation and discovery. The benefit of estimation is not only the awareness of the size of a story/feature; it is the discovery of another 10 points that were hidden in there and a follow-up discussion on if we want to implement those or not, now or later.

Organizations continue to have a defect backlog. What’s mind boggling, though, is the lack of awareness of the cost of that backlog. Allow me to break it down:

• Data entry in the backlog system.
• Data maintenance (e.g. priority/severity), if the manager is good about it. (Paradox! You are a vigilant manager, but spending time on wrong things.)
• Deduplication of data as the backlog grows.
• Struggle to reproduce defects, especially the old ones.

At this point you have already incurred debt and suffering inflation at the same time:

• The effort to fix a defect is growing with the time it is spending sitting in a backlog. Compare and contrast the effort to debug/fix one line of code vs. hundred lines of code.
• New code is being added on top of buggy code. That’s a Taj Mahal on quicksand. You believe you can do that, so you continue to build. The magic of compounding kicks in. Before you know, you have a shaky structure on top of a weak foundation.
• All-hands on deck are likely at some point in the near future to contain the crisis.
• A process will then be proposed and lobbied for to maintain the defect backlog.
• And, you also incurred an opportunity cost. This was the moment to build a culture of a quality conscious organization. Instead, that is now deferred.

The lack of basic understanding on how defect backlogs spoil the culture of an organization is quite disheartening.

Defect backlog indicates debt and inflation.

You have probably been advised to eliminate your defect backlog. In case you still have doubts about the ‘state of the union’ without a defect backlog, imagine your team fixing defects as they find them. Now, ask yourself these questions:

• What difference did it bring to your release readiness?
• How did it impact the effort of backlog management?
• How did it impact the cycle time of your product?

The answers are self-evident.

Then came the housing and stock crash. Revenues were hit and the pension fund was squeezed. A bit of simple mathematics forecasted desperate times ahead. Salaries were shrunk by 10 percent. Housing market still kept falling. Now the choice was between closing fire stations and tapping the tax payers of Contra Costa County for increased taxes. To be precise, an additional $75/year in taxes; 20 cents a day. At less than a quarter a day to make sure your house stayed safe made it a straightforward and easy choice for the taxpayers.

The goodwill for firefighters alone was enough for this measure on the ballot to pass. Who doesn’t love a firefighter! “From the time a little kid can reach up and grab something, they want to play with firetrucks. They dress up like firemen. Firefighters are the stars of local parades. They ride on their big red engines, and people cheer.” And let’s not forget, 9/11 had just happened. Americans adored their firefighters. That ensured a vote of approval.

The measure didn’t pass. The unbelievable happened.

Isn’t this all too familiar a story?  Was it hubris? Sabotage? Or, was it a disconnect with the sentiment on the street? Now, it could also have been an ambitious and manipulative leader who mislead his union.

This is a great case study to appreciate the value of studying your market.

Revenues and markets are seldom the glue between organizations and employees. Once we acknowledge this operating principle, the obvious question becomes, “What brings people to work everyday?”. The answer lies in purpose.

Purpose (or meaning) builds a sense of legacy. If those smart glasses that you are busy building are an aid to the visually impaired, you have a purpose and you find meaning in it. It won’t matter that Luxotica’s revenue is higher than your smart glass startup’s.

Purpose yields motivation. Motivation also comes from empathy. An executive’s storytelling has to go beyond markets and revenue. It has to reflect connectedness to what’s happening inside the company as well. Imagine a story about a team’s struggles and its success. Genuine storytelling is triggered by immersive experiences. Narratives that originate in immersive experiences demonstrate connection and control and build respect and influence. The resulting meaningful conversations give teams the stamina to manipulate around difficulties on their own. Stories anchored in immersive experiences give the teams a compass to navigate with. Such immersive stories require listening.

Listening motivates, but has diminishing returns unless it is followed up by action. Conceptually, this seems straightforward. But the pervasive inaction, especially in large enterprises, is a puzzle in itself. An environment that lacks action leads to broken windows. When left unfixed, these broken windows promote all sorts of undesirable behavior just like a decline in the habitability of a neighborhood. Emotionally the pain caused by inaction is the same as the pain caused due to poor work/living conditions. Inaction ultimately leads to employee flight. The best certainly prefer to move out.

Sadly, this mirage of motivation, purpose, and listening is quite real in most large companies. Motivation/purpose/happiness is a complicated web. It can be untangled. Organizational untangling should start with an acknowledgement of the reality as-is. But first, senior leadership should be vigilant about the mirage, intended or unintended.

Stop talking about money. Listen, empathize, and focus on purpose.

Then I sat through the report-out. Everything was just peachy. Big boss failed to recognize that he was being mislead. Assumptions and beliefs were not questioned. Credulous would be the right word to describe the leadership in the room.

That project took three times as long to deliver. Team members got burnt-out, some quit, work culture got destroyed and trust eroded (whatever was left of it to begin with). The big boss, in his inattentiveness left the organization incompetent to deliver software.

Now imagine the outcome with one small change of stance in that fateful meeting.

• Weighing the evidence,
• Exercising impartial enquiry,
• Seeing things truly as they were.

The foundation for that stance is constructive doubt. And, that is the role of a senior leader. Asking questions that enquire, bring clarity, build confidence, and suggest direction (next steps).

There are many battles fought between the guardians of the old and the proponents of the new. Those go beyond just the labels often used; traditional and modern, old-school and new-age, mature and immature, Waterfall and Agile, rigid and adaptive. Contempt and factionalism as deep and sacred as in religion and sports are fairly common. Traditionalists thinks that serious/big/real businesses can’t be run using new-age (lean-agile) practices and mindset. Adaptives believe that serious/big/real businesses are destined to collapse in the hands of the traditionalists.

The zone where the two, often opposing, mindsets meet is quite frictional. The biggest difference I see is that the adaptives/modernists/new-age folks like to get things done. They are doers who roll up their sleeves and jump right in to collaborate. The idea of collaboration, as it is practiced by highly effective teams, is foreign to the traditionalists. They prefer assigning tasks and review meetings to monitor progress. Another characteristic trait of the traditionalists is their inability to disassociate size with success. Large projects, large teams, large revenues dominate the narrative. Hence, they don’t limit work in progress. This compounds, in their heads, the need for more team members to track the work. The undesirable outcome of the large mentality is that teams quickly get bigger and add more trackers than doers. But they still find themselves behind the eight ball, initiative after initiative.

Imagine a team where everybody is interested in getting things done and is also a doer. Team members are comfortable with small initiatives and they finish them. They understand that the real pace of work is the pace at which work is being finished. They don’t succumb to the temptation of starting too much work. The illusion of a lot getting done, because a lot has been started, doesn’t quite trap them. They acknowledge and respect the cost of finishing work; collaboration, doing things yourself and working as a group.

Effective meetings, in a collaborator’s world, are hands-on workshops.

What is it about collaboration that the traditionalists don’t like, you may ask? In my experience, it is not that the traditionalists don’t agree on the value of collaboration. Rather, collaboration is uncomfortable for them. Seeing work done in all its messiness is distressful for them. It is not too different from how you may react to construction or renovation in your home, while you continue to live there. It’s loud and it’s messy. Heavy tools get used on the delicate things you took utmost care of, and occasionally things also get damaged. You’d much rather stay in a hotel or with a friend, while your home gets an uplift. You trick your brain into believing that if you don’t see it, it is not happening. That’s the emotion of a traditionalist. They too can’t bear to witness opinions and counter-opinions, and unfamiliar tools and approaches being used. They’d much rather see the finished product.

A desire to avoid the pain of collaboration, usually, renders the traditionalists incapable of accepting anything less than perfect. Gradually, they lose the art of imagination; the ability to see or envision the final/finished state. They just can’t extrapolate from work-in-progress to the final form. This is the reason, that they don’t often appreciate the incremental and iterative mindset, be it in building a product or putting together a slide deck.

The ability to listen and be aware would be my biggest advice to the community. Improv would be my second advice. That reminds me of what Greg Hohn said:

Improv is about realizing that everything you need is in the moment. If you are aware of it, you can act on it. 

Ask the following questions:

• What is the chaos I see?
• Why doesn’t that make sense to me?
• Why does it make sense to them?
• Why are my ideas not sticking?
• What experiments can I run?
• What other details can I find out?

Why is this a big deal and what has it got to do with improv? Almost every agilist I see brings a certain bend to the trade. Some prefer people approach, others are process folk. Trying to jam your ideas in can lead to frustration. The path to global/system optima lies in suspending your own preferences and promoting solutions that the situation dictates. That’s the zen of coaching and facilitation.

In a learning organization, which a high-tech organization has an imperative to be, it is important to constantly improve. Improvements aren’t an outcome of magic. They are a result of new ideas put in place. These ideas have to demonstrate a more context sensitive approach to improving the condition of a certain process or approach. This required context sensitivity comes from being able to understand the context, which in-turn requires indulgence. It shouldn’t be a surprise that there is time involved in this indulgence. Hence, if you want to improve you have to dedicate time for it. In other words, improvements require capacity. And just as you have a reason to have people, you have people to be able to reason. If you don’t want your people to reason, you are better off with automation of some sort.

As a leader you have to believe in your people’s ability to find choices instinctively and spontaneously. History has enough evidence of this human ability already. The only trick is to let the people have some time to think. Slack is therefore necessary. Build it in your enterprise.

Startups provide a keen insight into what makes folks aggressive. The can-do attitude is essential for survival of a typical startup. The same trait is also visible in team members at large corporations. However, during some segment of their journey the can-do attitude becomes an opiate of the aggressive type and adventurism sets in. The stakes aren’t as high as start-ups, scrutiny may not be as granular, and maybe the coffers aren’t as metered.

Adventurism is good. It is rather essential for a learning enterprise. But it is also the zone that ought to be closely monitored. If the adventurers aren’t reigned in or coached, not only will they hurt an enterprise, but they will also indoctrinate the following generation. This new generation won’t be nuanced between can-do and adventurism and instead become reckless. This is all about building a culture.

This is the generational divide I observe in almost all the teams. Recklessness and stupidity have to be called out, when spotted.

There has always been a fear that a populace left to itself wills the good but doesn’t quite know or see what that good is. That may explain societies with backward general will (customs, laws, etc.) despite reasonable and learned people. “Individuals see the good that they reject; the public wills the good that it doesn’t see.” A lack of enquiry ferments a lifeless brew and nothing ever changes. Co-incidentally, this intellectual sloth also results in social stigma, which further draws the society towards inaction.

As an Agilist, who has the privilege of observing many teams, I see a parallel between the social conditions of societies that are unjust, abusive, backward, and tyrannical and organizations that are stagnating. For progress, reason has to prevail. In a society the path to reason comes from an awareness of history and respect for philosophy. Similarly, in organizations that want to be nimble, they have to know the problems that the industry faces, the reasons behind those specific problems, the utility of an Agile mindset to fix those problems, the utility of practices and the reasons they work, their limits, the different flavors of practices, a situational awareness of the current environment, etc. Agile Literacy for each and every person involved in the value stream is a must. Do you hear that Mr. Manager? It includes you as well.