XPloring around

Mere uniformity does not make for consistency

2013-03-31T17:56:00.000+05:30

We often play the consistency card in the support of an argument.

I want this UI (form) to be laid out this way for the sake of consistency.

Let's be consistent and use the corporate presentation template.

I can't make an exception for you in case of this policy. We'll have to be consistent.

Used as above, consistency is an inappropriate substitute for uniformity. To check, ask "consistency with what?"

I want this UI (form) to be laid out this way for the sake of consistency.
    Consistency with what?
Consistency with the rest of the UI.
    You mean uniformity with the rest of the UI?
Isn't it the same thing?
    Well, laying it out this other way is consistent with the aim of keeping
    the user from making an incorrect assumption about the feature.

Let us all be consistent and use the corporate presentation template.
    Consistent with what?
With corporate guidelines of course.
    Guidelines don't demand uniformity. We could use our own templates as long as they are consistent with the image we want to portray as a company.

I can't make an exception for you in case of this policy. We'll have to be consistent.
    Consistent with what?
Consistent with the application of this policy.
    You mean you want to uniformly enforce the letter of the policy on everyone?
Well we certainly don't want to encourage exceptions.
    It seems you have already concluded that my case is an exception and one not worth granting.

Consistency is a higher order goal than mere uniformity. It requires some deliberation to decide if a certain course of action is consistent with broader objectives. On the other hand, uniformity is easily achieved by mechanical application of a rule book. This is what call center agents are trained to do. We know what the resulting user experience feels like. As Emerson said,

"A foolish consistency is the hobgoblin of little minds".

Why then, do we fall for this? Often, it is just laziness to listen and think. Sometimes (as in the case of call centers), it is side-effect of organisation design. Decision makers and policy makers sit on top of corporate hierarchies. They don't devolve discretionary powers to subordinates. The subordinates learn to function uniformly, without regard to context. They unconsciously cultivate a habit of using better sounding words like consistency to defend their actions. The makings of a classic bureaucracy!

Why continuous delivery needs devops, and why devops needs infrastructure-as-code

2012-10-12T09:33:00.000+05:30

Update 02-Apr-13
Just came across this detailed relevant talk by Reinertsen - the author I reference for silo reduction and batch size reduction.

http://www.infoq.com/presentations/lean-product-dev
===============================================

Update 26-Oct-12

Slides, with notes
=========================

I'll present on this topic on 25th October 2012 as part of a ThoughtWorks Studios webinar series on continuous delivery. Please register from the webinar page.

Abstract
Continuous delivery and devops have gone mainstream, at least in terms of mindshare. As a result, a lot of vendors have jumped onto the bandwagon. Most products that have anything to do with deployment now try to associate themselves with devops and continuous delivery. In this webinar, Sriram will try to clear the air in a product independent manner. He will also cover common devops anti-patterns and explain the idea of infrastructure as code.

The economics of aesthetics

2012-07-27T19:13:00.001+05:30

Many will argue that it is just plain wrong to subject aesthetics to cost and benefit, at least until a certain threshold. Someone said, “An economist is someone who knows the price of everything and the value of nothing.”

But this is the reality of business. A product or service can be made just useful or it can be made useful and beautiful. As consumers, we frequently choose from varying degrees of form and function. However, things get more interesting in a B2B situation.

Clients agree to foot the bill for aesthetics to the extent it promises to help their topline. Funds are made available for aesthetics on a public facing website. The belt is tightened for an internal facing application. Vendors use the same logic. They spend money to make sure that their presentations are not only substantial but also aesthetic enough to impress clients.

What about the people who write code? They also have a sense of aesthetics. Their bosses don’t grudge them the inexpensive effects – standard tools that can ensure uniform indentation, nice fonts and syntax highlighting. But higher order aesthetics are a different matter altogether.

I mean lucid, performant code, fluent interfaces, pithy names, succinct tests and coherent classes residing in cohesive modules loosely coupled with their neighbours.

“Don’t just call these aesthetics”, I hear the indignant coders say.

“They confer vitality, suppleness, and longevity to the codebase.”

There is truth and perhaps some irony to this claim. Unfortunately, even the truth is appreciated largely only by the coding intelligentsia. As a result, efforts to secure these higher order aesthetics go unchallenged only as long as the development team stays on track for budget and schedule. As Stewart Brand said, form follows funding.

But estimates being estimates and competitive bids being competitive bids, there is many a slip between the cup and the lip. Screws get tightened,

“Enough gold plating already.”

Some will contend that effective visualization of internal software quality will bring about the necessary loosening of purse strings. Well, only occasionally. For one, a common (and sometimes legitimate) reaction is,

“Why did you incur technical debt in the first place? That’s not why I hired expensive consultants like you.”

But first, there is the question what makes for effective visualization. Should we show a tangled ball of wool versus a neatly wound ball in order to contrast the current cyclomatic complexity with desired state? Should we progressively untangle the ball in progress reports? Is it not enough to state in text that the industry standard is about 20 per 100 hundred lines of code and that we are currently at 45?

Should we depict a slummy urban sprawl for a monolithic codebase and gradually introduce suburbia with zoning as we modularize? Is it not enough to show a dependency structure matrix?

How accessible does a visual need to be before a non-technical/post-technical decision maker gets it? One kind of visual targets the recipient’s analytical faculty. Another kind targets their aesthetic faculty. Aesthetics are in vogue among consumer devices. I suspect this has spilt over to the kind of visuals favoured by IT decision makers.

They very people who scrutinize spend on aesthetics through the lens of the topline lean towards aesthetics when they ask their sub-ordinates and vendors to send them reports and presentations.

tl;dr and visualization abuse

2012-07-27T19:09:00.003+05:30

A picture (visual) is worth a thousand words when it is a photograph of the real world. Even in this case, an accompanying narrative can provide valuable context. It is sloppy to try to make sense of something without its context. When the picture (visual) in question is a mere illustration or graphic, the accompanying narrative becomes even more important. And yet, I see a trend of trying to make sense of the visual without providing, asking for or reading the narrative:

Archiving presentation slide decks in document repositories without a supporting narrative.
Demanding all sorts of reports in the form of presentations rather than the more traditional form of a document. Tradition makes sense sometimes.
Graphing all manner of metrics without any narrative that provides a context for the measurements.

Well argued supporting narratives are passé. I suspect this is an example of the Shallows effect. It has become fashionable to say tl;dr. Instead of introspecting if I have lost the ability to focus, I lull myself into believing that I have just run into a wall of text - a newly minted pejorative for what used to be known more honourably until recent times as a page of text. As a result, content creators fear that if their text is any longer than a tweet, it won’t be read. They have to make it interesting with visual effects.

Instead of admitting my newly gained inability to parse a carefully constructed paragraph or argument full of nuances, I smugly proclaim myself a visual thinker. This is not to say visual thinkers don’t exist, just that the rate at which they seem to be proliferating is a little suspicious.

One kind of visual targets the recipient’s analytical faculty. Another kind targets their aesthetic faculty. Aesthetics are in vogue among consumer devices. I suspect this has spilt over to the kind of visuals favoured by us. So we encounter graphs where a table would do, a 3-D visual where 2-D would do and only a visual where a paragraph is called for. The visual is no longer just a means to tap into the pattern recognizing, parallel processing prowess of the analytical brain, rather it is meant to catch the eye and increasingly, only the latter.

The irony of maintainable code

2012-07-27T19:03:00.000+05:30

Suppose there exists a lucid, performant, appropriately documented codebase with fluent interfaces, pithy names, succinct tests and coherent classes residing in cohesive modules loosely coupled with their neighbours. Is it maintainable? Very likely in theory, less so in practice.

Most often, this is simply because the maintainers are a different set of people than the creators. Also because of the way business finance works, maintenance is often the job of a less skilled set of people. Less skilled in the sense that they often have a problem with reading object oriented code. They are used to linear reading of procedural code and get disoriented by all the polymorphic jumping around. Ha! Speak of unintended consequences – object orientation leads to disorientation.

Less skilled also in the sense that they don’t appreciate unit tests as documentation, even when they attempt to read tests, they are put off by all the mocking. They don’t understand what the right granularity is for dependency injection and all the fuss over small classes and avoiding primitive obsession.

So they don’t add/fix tests along with bug fixes and enhancements. The builds turns red and remains red. Continuous integration is forgotten. Soon there is no automated way to check against regression. No one attempts even small scale refactoring. The codebase is now unmaintainable by any definition.

How email shapes us... and how to get back into shape

2012-04-09T00:41:00.002+05:30

In pre-email days, the inbox was physically separate from the file cabinet. ‘Read’ items were carefully filed away or trashed. The inbox was only meant for fresh arrivals. One didn’t attend to the inbox constantly. Just once a day, maybe twice. People didn’t expect a two hour turnaround for written correspondence. They just used the telephone for urgent matters. Corporate announcements, bulletins, circulars were often stuck on notice boards. Every employee didn’t have a copy of the notice thrust on his face in the middle of the working day. Communication, although a part of work, was still considered distinct from actual work.

Email has altered the balance of communication and actual work. It was McLuhan who observed that technology shapes us even as we shape it. How has email shaped us?

The inbox is the file cabinet

Say we decide to wrest control from the ever brimming inbox. Check email only twice a day at fixed times, keep it closed otherwise, let’s say. But no, closing our email clients (whether on the laptop, tablet or mobile) also means locking our file cabinet. Thanks to multi-gigabyte inboxes and terrific search algorithms, our email clients are also our file cabinets. And we often need access to our file cabinets during the course of actual work. There we go, looking at our inboxes again. Filters, rules, labels or folders may help us avoid looking at what’s new but now we begin cutting against the grain of the technology. To those who say “don’t blame the technology for your lack of restraint”, think again - “technology isn’t value neutral”.

Email clients need to evolve, whether browser based or otherwise. They need to provide a ‘go offline’ mode that gives us access to our file cabinets but keeps the inbox away. It should also be possible to specify our email checking times similar to how we already specify our working hours in our calendars. Those eager for a response will be able to lookup these times.

By the truckload, by the minute

Because it is so cheap, we send and get loads of it. Never mind spam, why have corporate notice boards disappeared? All sorts of corporate announcements flood our inboxes through the working day. At the very least, corporates should make it a rule to send announcements near close of business. But then there is no suitable time for a global corporation. So it should be possible to tag an email as a not-so-urgent announcement and have the email servers deliver it near close of business as per the time zone of the receiver. Configurable “delayed delivery” may work better than filters and rules.

A new addiction

Many managers and others spend a big part of their day reading and writing email. Email notifications provide endless distraction for those to choose to be notified of new email. New email provides a psychological kick to many. Some get bored when there is no new email for half an hour. They take a break if they see no new email. Less actual work, more communication, most of it not addressed directly to the reader.

Frivolous Documentation

Email has bred a culture of excessive documentation. The ability to easily get things on record has increased the tendency to do so. This is another reason why people don’t use the telephone for not so important but urgent matters. They write an email to put in on record and wait for a quick reply, even if the recipient is only meters away. This needs to be discouraged. Get them to use the phone. Cost need not be a hindrance. A global corporation requires people to collaborate long distance in near real time. Long distance telephony isn’t cheap but VOIP is.

Privacy

Vocal conversations can be overheard or may disturb others, especially so in open plan layouts. Email affords privacy and doesn’t disturb. Fair enough.

Of luddites and of having access to the latest information

Am I a luddite for refusing access to real time updates, for wanting to go offline? Communications technology is evolving rapidly. Embracing all of it as good has unexpected and undesirable side effects. On the other hand, is it just filter failure?. After all, would we want to miss a relevant email in the midst of our work and then rework? To me, this sounds like an insurance salesman. If we expect new email to have an immediate impact on our work, wouldn’t we just choose to stay online for that piece of work? How often does unexpected new email from the past couple of hours have an impact on our work? Can some of these just-in-time scenarios be avoided by encouraging a change of behaviour at the other end?

Summary

As a reader of email,

1. Let your colleagues, manager know that you only check email twice a day. Publish your timings.

2. Disable email notifications.

3. Ask your email vendor to support an offline mode that screens new email from you (but lets you access old email) when you are working.

As a writer of email,

1. Don’t expect a two hour turnaround. Call if you need a quick response.

2. Include just as many people as necessary in your email. Don’t cc the whole department/team.

3. Don’t send meeting invites without sufficient notice (4 hours at least?)

As a leader,

1. Encourage styles of working that don’t require people to check email constantly

2. Encourage people to use VOIP and call their colleagues for things that need a quick response rather than write email.

3. Regulate the volume and timing of corporate announcements into people’s inboxes.

As a vendor of email software,

1. Allow people to specify their email checking hours that others can look up.

2. Provide a ‘go offline’ button that screens new email from the user until she is ready for it.

3. Provide options for delayed, time zone sensitive delivery that lets corporates send announcements etc. without interrupting the working day

4. Make notifications opt-in rather than opt-out

Measurements and targets: The use and abuse of metrics

2011-10-29T13:25:00.002+05:30

Software development is a social activity. As such, it does not lend itself very well to measurements. Sure, we can measure a whole lot of things about software development but we can never contend that a given set of metrics is exhaustive. Plus, it is costly to regularly measure and track too many things. So, we restrict ourselves to a subset of feasible measurements.

For instance, although return on investment is a useful metric, it is often not feasible to measure and track it for a piece of software. Thus, in practice, we settle for things that only paint part of the picture. It is important to acknowledge this. Often, enterprise IT loses sight of this truth. It deludes itself that the reported metrics constitute the big picture (From the school of “If you can’t measure it, it doesn’t exist.”).

What is worse, in the name of continuous improvement, these metrics are converted into targets. Teams now have an incentive to work towards local optima. No wonder large swathes of big enterprise IT are in a desperate mess. New causes of failure are discovered all the time. Heads roll and the new heads track a slightly different subset of metrics.

Automatically converting metrics to targets also has adverse psychological effects. Here is an example from day trading. What is more important? Making more winning trades than losing trades or making money? Obviously it is the second. But the moment you start tracking win-loss ratio it may become a target it itself. Emotional conditioning will also come into play and we will try to have less losing trades.

A software team can get severely constrained when a velocity target is imposed on it. At a certain threshold of constraints, team members lose a sense of empowerment. They play it completely safe and lose initiative. We experience this phenomenon sometimes when we deal with call centres. Government sponsored healthcare and education in western economies often exhibits similar characteristics. It is not just bureaucracy. In an effort to scale and centrally control efficient delivery of services, a raft of metrics is imposed on the practitioners. Performance of schools/hospitals is then tracked against these metrics (they now become targets). Teachers and doctors get frustrated, lose initiative and just play by the book much to the exasperation of parents and patients.

This effect is also known as Goodhart's law, after Professor Charles Goodhart who was Chief Adviser to the Bank of England. Restated more succinctly and more generally:

When a measure becomes a target, it ceases to be a good measure.

Measurements should not automatically become targets. When measurements indicate something is wrong, it calls for a conversation in context, not for a rating downgrade. Conversations in context may reveal that things are still ok in the larger scheme of things, the measurements only point to a local sub-optima.

Trouble is, it is easy to scale management by numbers. It is very difficult to scale management by context. Perhaps it is worth asking if scaling is more important than delivering a quality experience. Besides, too much scale creates “too big to fail” monsters that have to bailed out with the money of the innocent in times of crisis. This is not just true of macroeconomics. It is true of macro IT as well.

Dashboards promote ignorance

2011-04-11T21:30:00.000+05:30

It is possible to collect a wide range of metrics on a project. Metrics about the codebase, unit tests, performance tests, build, delivery progress etc. It is also possible to define red, amber, green thresholds for each metric. Some go for standardized organization wide thresholds while others are comfortable with project specific ones. The obvious next step is to create a dashboard that rolls up all these metrics into a single red, amber, green project status indicator. Middle managers spend a good part of their life managing inputs to these dashboards. Some "programme managers" assiduously refer to their dashboards as balanced scorecards.

Trouble is, most of the time, an overall green status indicator doesn't mean anything. All it says is that the things under measurment seem ok. But there always will be many more things not under measurement. To celebrate 'green' indicators is to ignore the unknowns. The value of a metric is not when it is green. Lets take unit test coverage for example. Is 100% test coverage a cause for celebration? What if most tests don't have any assertions? When measurements become targets, they encourage gaming. On the other hand, a coverage of 50% at least tells you unambiguously that half the code is not covered (or there is an error in measurement).

One may argue that it is only a matter of measuring even more so that the overall green becomes truly indicative of overall project health. This is somewhat unrealistic in a fast paced knowledge work environment. Tools and technologies keep changing. Measurement tools don't keep pace. It takes significant effort to maintain the measurement infrastructure on a project.

Avoiding subjectivity in assessments is also cited as a reason for resorting to metrics. But that's like saying that a person can be certified healthy without the judgement of a physician by a dashboard that rolls up a hundred diagnostic tests.

In my experience, metrics are much more useful when they report bad news than when they are green. In the words of the Way of Testivus, "Good tests fail". Unfortunately, the tendency to roll up metrics into dashboards promotes ignorance. We forget that we are only see what is under measurement. We only act when something is not green.

Then comes the final argument. Don't blame the tool for human laxity. Dashboards cannot promote ignorance or even wisdom. Unfortunately, tools aren't value neutral.

Tools aren't value neutral

2011-03-16T17:16:00.002+05:30

It is accepted that language influences the way we think1. Language is one of the earliest technologies of communication. We don't have to make a big leap to see that all tools and technologies influence the way we think and act2.

Therefore, it is erroneous to say things like the following:

“The internet isn’t a pro or anti-democratic technology. It is value neutral”
or
“Powerpoint isn’t good or bad. It is how you use it.”

McLuhan understood this very well. He famously said:

“We shape our tools, and thereafter our tools shape us...Our conventional response to all media, namely that it is how they are used that counts, is the numb stance of the technological idiot.”

Why am I writing this in the middle of posts on software excellence? Because I need this for my forthcoming posts :)

1. Language influences the way we think
http://www.edge.org/3rd_culture/boroditsky09/boroditsky09_index.html
http://www.amazon.com/Language-Thought-Action-S-I-Hayakawa/dp/0156482401

2. Tools (and technologies) influence the way we act
http://www.amazon.com/Understanding-Media-Extensions-Marshall-McLuhan/dp/0262631598
http://www.amazon.com/Shallows-What-Internet-Doing-Brains/dp/0393072223

Build time impacts team performance

2011-03-14T18:22:00.002+05:30

Velocity often grabs a lot of managerial attention on a project. But while velocity is the effect, build time is a common cause. A slow build has several insidious side effects. Say a build takes 20 minutes:

10:00am	My pair and I work for the day by doing a fresh checkout and initiating a local build. 20 minutes to kill? Maybe check email etc, talk to analyst about my story, take a coffee break. OK, build passes.
10:20am	Work begins.
10:50am	We are ready for checkin. Oh no, we can't wait for a 20 minute build every half hour can we? There goes the motivation for frequent checkins. We decide to develop locally for another hour at least before trying a checkin. Signs of discontinuity in continuous integration!
12:00pm	Ready for checkin again. Run local build.
12:20pm	oops. Build broke. Our code broke some old test. Good catch by the test suite!
12:30pm	Attempt another build after fixing the code.
12:50pm	Build passed! Now we just need to check once again after merging with latest code from the repository. Is the build green? Yes. Safe to checkout from repository. We checkout, couple of auto-merges, no conflicts. One more local build and we are good to go.
1:10pm	Local build passed. We are about to checkin when we notice that the build is yellow. Darn, someone just checked in. Now we have to wait for the build to go green, then checkout and build locally again before we checkin.

Anything short of this rigour leaves a window open for build failure on the CI server. But hey, surely you can't expect us to play the waiting game indefinitely. What if someone checks in again while we are diligently verifying locally? Besides its time for lunch. So we gamble and checkout. No conflicts or even auto-merges. Good sign. It isn't worth building locally again. We checkin and go for lunch. Halfway through lunch, I get a call.

"@#$!, why did you checkin while the build was still running?"
"Why what happened? Don't tell me your build didn't go through"
"Yes it didn't and now I am having trouble fixing it on top of your changes"

As build time increases, it tests my patience as a developer. I am tempted to take shortcuts that occasionally backfire. When this becomes standard team behaviour, the occasional turns into regular. Secondly, it increases the syncing window (12:50pm to 1:10pm above). This window is at least equal to build time. Greater this window, greater the chance that someone else might check in while I am still getting ready.

Now what if the build time were five minutes instead? There is much less waiting time associated with frequent bite sized checkins. So it encourages frequent checkins. This in turn reduces the likelihood of merge conflicts when I sync with trunk. Finally, if someone else does check in during my syncing window, I only have to wait five more minutes to see if it goes green.

In summary, everything else constant, team performance correlates well with build time.

So what to do?

Keeping a build fast is a topic by itself. Basically try to keep the unit tests fast. At an average of 20 millisecs per unit tests, we can run 5000 tests in 100 secs (just under 2 minutes). Avoid all I/O in unit tests. If you must have some unit tests that do I/O, keep them in the next stage of your build pipeline. Don’t write unit tests that exercise a lot of code. They aren’t unit tests if they do. As a manager, be very concerned if you see build time going up steadily.

The point of Object Orientation

2010-08-17T16:59:00.003+05:30

Transaction script cost escalation

A common characteristic of large codebases is that it takes disproportionate effort to add or extend functionality. Couple of common reasons:

Hamstrung team - The original developers may have moved on, development beyond the first release may have been transferred to a less capable team or handover may have been ineffective.
Poor codebase - Test quality and coverage may be lacking, encapsulation may be broken, or the build may be inordinately long.

Bad encapsulation is a common culprit. All too often, OO-capable languages like Java and C# are used to author transaction script1 applications. The business logic is use case driven rather than domain driven. Use case driven code is characterized by global code acting on global data. Well encapsulated codebases, on the other hand, consist of collaborating objects, each object encapsulating its local data and behaviour.

For a typical business app, everything else remaining constant, good encapsulation will lead to constant or decreasing delta effort for delta functionality. The point of object orientation is that it is meant to help manage complexity of the business domain. This can only happen when we encapsulate along the grain2 of the domain.

1:http://martinfowler.com/eaaCatalog/transactionScript.html
2:Cutting with the grain: the rhythms of design - Kent Beck

Agile anti-pattern summary

2010-05-02T19:13:00.000+05:30

Here is a summary of anti-patterns I have written about so far:

How to do XYZ in Agile?

2010-03-24T14:18:00.003+05:30

Clients sometimes ask, "How do you do estimation in Agile?" or, in general "How do you do XYZ in Agile?". Typically, they are people making a transition from non-Agile methods. They often (understandably) want to hold on to some existing ways of functioning. They would like to blend Agile into their existing processes. There is a problem here. To be agile is not about following a different set of prescribed processes or practices. The only things that matter are:

Continuous delivery of valuable functionality
Happy team (team includes client)

The agile manifesto starts off by saying:

We are uncovering better ways of developing
software by doing it and helping others do it.

The practices codified under XP (or Scrum) is just documentation of how a bunch of practitioners were able to achieve continuous delivery and happy team. The question, "How do you do XYZ in Agile?" misses the point. It is a relic of a process conformance mentality. What's more, I was once asked, "Is it ok to ask for a number of tailorings or deviations from the master process template for agile?" I was speechless. Turned out that the organization still retained the services of a group called SEPG (software engineering process group, a relic of CMM) to define a master process template for agile. Every project was supposed to conform to the template and ask approval for tailorings (tweaking a process/practice) or deviations (omitting a process/practice)!

If you are achieving continuous delivery and happy team, you are obviously doing something right. It doesn't matter how Agile it is. If you aren't achieving continuous delivery and happy team then again it doesn't matter how Agile your processes are. One might argue that this is watered-down agile. Big deal. Granted, it is definitely wise to go by the book first. It is arrogant/foolish to assume that we are smarter than the book before we begin. After all, the book represents distilled wisdom of practitioners. But it is important to keep an eye on the outcomes. All advice is contextual. It is no use wailing that you have done everything by the book and arem't getting results. It is dogma to stick to the book in the face of contrary results.

Agile maintenance in a devops world

2010-03-15T13:13:00.003+05:30

"What is an agile way of doing maintenance projects?", is a common question. An answer like the following is par for the course:
"It isn't very different from development projects. Write tests to reproduce bugs before you fix them. Treat enhancements as one or more stories. Keep the CI server running. Make sure that changes are occasionally merged from the maintenance branch into trunk. Pairing is probably not as important."

In the context of this post, a 'maintenance' project is one where a client outsources ongoing bug-fixes and enhancements for a live application to an IT vendor (often offshore). Conventional wisdom has it that development is investment (capex) and maintenance is cost (opex). Hence, clients look for much lower billing rates when it comes to maintenance. This results in separate contracts/teams/vendors for development and maintenance. Some clients go a step further and outsource even IT operations to a different team/vendor under a separate contract. The rationale is to stick to core 'business competency' and outsource everything else (let them vendors compete with each other for our slice of IT).

Depending on how critical the said application is for revenue generation, this strategy of 'divide-and-conquer-IT' can be frustrating at best and suicidal at worst. The best of the businesses that make most of their money off their websites outsource little to none of their IT. This is simply because you cannot go to market fast enough if you have to orchestrate between three teams/vendors for every new feature. Equally importantly, the feedback loop gets badly constricted at contractual boundaries. Designing formal, SLA driven protocols of communication between business, development, operations and maintenance is a recipe for bureaucracy and indifference.

Not everyone can afford not to outsource. It is very difficult to put a good IT team in place. A lot of outsourcing has been a reaction to uncooperative in-house IT. But the solution has gone overboard in trying to compartmentalize development, operations and maintenance. You don't want to be overly reliant on a single vendor? Fair enough, consider outsourcing product A to vendor X, B to vendor Y and C to vendor Z. This is better than outsourcing the development of A, B, C to vendor X, operations to vendor Y and maintenance to vendor Z. The latter model requires well defined, stable interfaces between different constituencies. Not suitable where business is changing every week. It is a repeat of the layered-team anti-pattern at a higher level.

What if we want to deliver a grand SOA? Once the team of architects have specified every service, surely it should be possible to outsource a bunch of services to different vendors and the consuming applications to yet others. Splitting up integration work is extremely risky, in my opinion. Service contracts are rarely cast in stone. On the contrary, if you like the idea of evolutionary, guerrilla SOA, you no longer plan to have stable interfaces - the service evolves in tandem with the needs of its consumers. So, try to give all integration work and important applications to your primary vendor. Better yet, don't do big bang top down integration. Evolve bottom up.

On the other hand, it may be that your business doesn't change that often or your application doesn't generate revenue. If so, it is useful to ask "Why build? Why not buy?" every once in a while. SaaS is growing fast. It is likely that someone is offering your bespoke application as a service. At the cost of some tweaks to your business process, you might end up with a better application at lower cost. The SaaS vendor in turn is likely running a full in-house IT.

The world has changed yet again. Devops is gaining currency. These days, a common way of testing the uptake of new features is to divert, say 5% of your traffic to a new deployment and see how it goes. A separate maintenance team is a dinosaur in an age of frequent deployment (blue-green or otherwise) and continuous delivery.

Q. What is an agile way of doing maintenance projects?
A. Don't do maintenance as a separate project.

There is one situation where a maintenance project might make sense. End-of-life support. Basically, you pay for a team to keep an old app running while a new replacement gets built. Other than that, it is all about breaking down silos.

Of course, you will never hear Indian IT majors singing this tune. Their recruitment and business model is based on economies of scale. 'Changing businesses' are anathema to their ossified processes. However, in response to changing realities of the marketplace, some of them have begun to sing an agile tune. Some of them even claim to have proprietary, hybrid, high-performance methodologies that "synergize best of breed practices from CMMi, ISO, Six Sigma and Scrum". I'll explore a certain aspect of this 'synergy' in my next post.

Pair Programming Payoff

2010-03-04T12:49:00.004+05:30

For a project that runs for, say six months or more, there should no extra development cost on account of pair programming. Period. If there is extra cost, it means pairing is not paying off for you. Pairing should pay off in the following difficult-to-measure ways:

Lesser rework during development on account of misunderstood/ill-defined requirements. These things surface quickly when pairs talk to each other.
Slight reduction in maintenance cost on account of clearer code. Because every line is now considered readable by at least two people. (more in case of pair rotation)
A better detail level design results when each pair acts as a sounding board for the other. Good design reduces cost of future change.
Faster/better process of bringing new team members up to speed.
Lesser impact of people taking vacations.
Lesser bugs in QA, UAT
More hours of focused work per day - however professional someone may be it is natural for concentration to wax and wane. Pairing almost always helps here.
Unlike the case of no-pairing, there is no separate code-review activity required

How do you figure if there is extra cost? Estimate both ways. The total 'release-lifecycle effort' for the case of pairing should not exceed that of no-pairing. Individual stories may indicate greater effort but that is okay. It is very difficult to do controlled experiments to nail this down. There are some studies but your mileage may vary.

A word of caution when estimating for such comparisons. Remember that on all real projects, actuals mostly exceed estimates. Some of the difference is borne by the client (change control etc) and the rest by the vendor (unpaid overtime etc). You need to have a realistic idea of effort overrun for the case of no-pairing to be able to compare it with the overrun for the case of pairing.

If pairing is resulting in net higher development cost on a long running project, then it simply means (to paraphrase Kent Beck) that the client is getting XP'd on :)

Label activities, not people

2010-02-25T12:19:00.003+05:30

Ideally, an XP team consists of all talented and experienced people. Every person is assumed to have good communication skills, technical skills, ability to analyze and synthesize, understanding of business drivers, ability to make trade-offs based on business constraints and the ability to work as part of a team. Indeed, there probably are thousands of people who qualify on all these counts scattered around the world. But IT is a pervasive beast. It touches almost every aspect of modern business. IT is also labor intensive. The demand for really good people outstrips supply by a factor of hundred if not more. IT vendors who practice XP are therefore forced to move away from the ideal of more or less homogeneous poly-skilled teams.

Instead, we hire people who have a subset of the above skills and assign them to specific roles in a team. So we have roles like developer, analyst, QA, manager, UX dude/dudette etc. It is essential to recognize that this arrangement is a departure from the ideal - a compromise driven by market reality. How we look at roles depends on our model of the ideal world. If you agree with the XP ideal, then roles are labels attached to the activities on a project. e.g. performance tuning activity needs developer role, activity of scope negotiation needs manager role.

But all too often, we lose sight of the ideal and think of roles as labels attached to the people on a project. e.g. person A is a manager, anything she says outside her labeled area of competency should be taken with a pinch of salt, person B is a developer, anything she says outside her labeled area of competency should be taken with a pinch of salt, and so on. We forget that the labeled area of competency is merely a mutual agreement at the time of recruitment. We end up discounting the credibility of people outside their labeled domains.

This kind of thinking also magnifies capacity constraints in a team. Say we have 4 people with label QA and 12 people with label developer on a team. What happens when stories start piling up in QA? We are less likely to see if some developers can add temporary capacity to QA. Oh no, that would never do. Developers can't test. They are not wired to test. Mass stereotyping sets in.

Soon people start buying into their labels. Developers refuse to test. If you can communicate well, you are suspect as a developer. Managers shy away from writing code, or (shudder) gaining even basic user level technical competence. They go about saying "I can't handle technology", as if it were a feather in their cap. We become more and more entrenched in the silos created by our labels. Truly ploy-skilled people get disheartened in the process. A person with a label of analyst may have a great aptitude for domain modeling. But her inputs may be sidelined by developers. After a while she either starts believing in her artificial limitations or she starts looking for another job where her skills are more broadly appreciated.

It is no good to call everyone consultant and then use a different labeling scheme when it comes to staffing projects. To the extent of their skills, people should get to play multiple roles within a single project. Oh, that complicates staffing. So? Should the tail wag the dog?

Organized religion came about partly as a result of our need for identity and self-worth. It is probably reassuring to say "I belong to this group" and then move on to thinking "My group is better than the other group". We may have outgrown traditional organized religion but we seem to be falling for newer versions.

Fail fast

2010-02-12T16:20:00.005+05:30

When failure is a possibility, design to fail fast rather than slowly. Doing so reduces the cost/impact of failure. What is equally important, failing fast makes further attempts feasible. Learning from previous failures makes future attempts more likely to succeed. This principle is widely applicable in software development:

Methodologies that have fail fast mechanisms baked in are more likely to generate greater ROI. More on this later.
Guerilla SOA is arguably a fail fast take on big up-front SOA.
Code that is written to fail fast is likely to be more reliable in production.
Small, frequent check-ins are likely to cause less overall rework than big, infrequent ones.

Verification: When and How?

But how do you decide at any given checkpoint if we have a success or failure at hand? The quality of verification is crucial. Verification by peer review, while valuable, is prone to oversight. The proof of the pudding is in the eating. The more times you get to eat the better. The analog of eating here is testing functionality. A truly iterative process of software development where functionality gets tested iteratively is likely to achieve better ROI (everything else remaining constant).

Okay, so no one uses waterfall anymore. But we still have projects where big up-front analysis and design are the norm and continuous integration means weekly build. In such cases, we only have limited verification (peer reviews of requirements, design and code) till the very end. Failures (if any) are slow and horrible.

Incremental agile is what almost all XP and Scrum teams follow. They run through the stories for a release doing just enough/just in time analysis, design, coding per story. The boundaries between design and code are often blurred but that is not material to this illustration. Truer verification now becomes possible at the end of every story (QA/customer testing/sign-off). However, each story still gets only attempt. Any changes (learnings?) after that go back into the backlog to be prioritized and taken up with everything else.

But as Jeff Patton points out, it is possible to view each story as a series of progressive enhancements:

Necessity - core functionality (e.g. user registration)
Safety - validations etc. (e.g. confirm via email, add a captcha)
Flexibility (e.g. support openID)
Luxury (e.g. add ajaxy feedback on available userids, password strength)

Yeah the example is a bit contrived (openID would mostly be another story even in incremental mode) but you get my drift. We can now design a release plan that allows the team to iterate on a story, progressively enhancing it. The story sponsor reviews (tests) each story multiple times. Failures (if any) are faster and cheaper. The team learns better. I like this line from a Werner Vogels interview:

With a new radical service, you try to go into prototype mode pretty quickly, and then you start iterating on that until you feel that you understand your business problem.

Stories in regular business applications may not qualify as radically new but very often the team is new to the application in question. "You only understand it when you do it" is a much under-appreciated truth of all knowledge activity (if not all activity).

Expected cost/time

It may be argued that for a given chunk of functionality, iterating increases overall cost/time as compared to doing it in one go. This is only true when the risk of failure is zero. For situations of non-zero risk, the expected cost/time can often be lower with an iterative approach. The table below shows this for a hypothetical but realistic risk profile where risk decreases as learning increases. Your mileage will vary depending on the risk profile of your team and functionality.

Failure is not an option

Sometimes you get to hear sponsors saying they don't care about downside risk because failure is not an option. Some of these projects run into a death march followed by a blow out followed by movement of key people. Then a new team and a new IT partner get to do it all over again.

When Agile Estimation becomes pointless

2010-02-05T08:52:00.002+05:30

If a team has to deliver a certain amount of functionality by a certain date, then the act of estimation becomes an act of commitment. The team has essentially committed to cost, schedule, quality and scope. There are no levers left. In such a situation, velocity becomes a target rather than just a measurement. Targeting velocity makes points based estimation a charade. The team is better off estimating in real days in this case. It helps the cause of commitment. Points based estimation is useful when you agree to the following:

Plan the story list for a release with only about 70% must-have functionality and the rest as negotiable. This implies it is potentially okay to release (go into production) with just that 70% functionality.
By definition, estimates are approximations. Software estimates are more so.
It is counterproductive to insist that your IT vendor/team deliver x points of functionality by date y. Scope negotiation does not mean x remains x. Yes, sometimes only the contents of x need change. At other times, x may become x ± delta. This is not a rip-off. It doesn't make business sense for your IT vendor to rip you off because you won't give repeat business if you are ripped off. Without repeat business, your IT vendor is likely to become uncompetitive because the cost of new business development is very high.

"Embrace change" is a two way street. IT delivery team should embrace changes to requirements and priorities. Clients should also embrace changes to scope when delivery work proceeds at a different pace than originally expected.

The Tragedy of Commons based Peer Production

2010-01-29T08:55:00.003+05:30

In my previous post, I mentioned how authors of small but useful open source projects often struggle for compensation. Using a FOSS/commercial license is one option but it could go either way. They wouldn't really have to resort to compulsory payment if donations were forthcoming from users/organizations who commercially benefit from it. Therein lies the tragedy of 'Commons based Peer Production'. Only a tiny minority donate. This is detrimental to the growth of all forms of free digitial distribution (indy music, books, software etc). It also makes for an unhealthy society. Big leap? Bear with me.

The bulk of financial transactions in G20 economies happen between:

Corporations (B2B)
Corporation and citizen (retail - citizens pays corporation, wages - corporation pays citizen)
Corporation and state (taxes - corporation pays state, state funded projects - state pays corporation)
Citizen and state (taxes - citizen pays state, pension/welfare - state pays citizen)

But what about transactions between citizens? When this goes towards zero, we get a unhealthy society. Centralized services, mostly helpless consumers (citizens). Donating money to the creators of 'digital stuff' that we enjoy is a great way of changing the status quo. The internet has provided a great platform for disintermediation but it will only work if we choose to participate in the process.

We could also try to build momentum within our organizations towards this. Companies that commercially benefit (however indirectly) from free software could set aside some money annually for donations. The beneficiaries could be decided by a poll within the company. After all, this is also part of CSR. Plus it will make for good PR copy.

Open Source authors' struggle for compensation

2010-01-25T16:37:00.006+05:30

I guess we all know the difference between free as in speech (freedom) and free as in lunch (gratis). All open source software confer certain freedoms of use, modification and redistribution. None of them are required to be made available to users at zero cost. Yet, I don't know of a single significant project that is open source and fully paid, i.e. no version available at zero cost. Reasons typically offered are:

It is the freemium business model - Give away the basic software and charge for enhanced functionality or support.
It is not enforceable - Users will simply build the software from the source code and there is no way you can get them to pay.

#1 is okay for projects run by companies. It is often not feasible for projects run by individuals. This a big category of small useful pieces of software (think libaries, plugins, utilities). These people try (in vain?) to make some money via advertisements or by appealing for donations. It is a sad reality that they get no compensation from commercial software/organizations using their work. These people should relook at #2.

I suspect payment is very enforcable against commercial enterprises. Just add a line to your EULA saying "It is illegal to use this software for commercial purposes without paying for it". Most companies would pay a small fee (say $50 for a full version but without support) for a useful piece of software. Either that or their lawyers would blacklist the software. At least you wouldn't have legions of freeloaders just using the software without contributing anything (money, bug reports/fixes, documentation) back. The argument that even freeloaders help spread the word to someone who might eventually buy something is a trifle bleak. Of course, all power to you if are doing this altruistically or if you are happy with the other rewards (fun, learning, reputation).

Giving software away at zero cost means you have to depend on services for income. You either offer your services as an employee (day job) or as support (adding value to what is given away). Either way, it is a model of pricing input instead of output. Scales only with people. Or you have to fall into a category where you can be adopted by a big foundation like Eclipse or Apache or Mozilla. Good luck with that.

No software patents versus patent reform

2009-11-17T00:08:00.005+05:30

We often cry ourselves hoarse trying to justify non-waterfall methods of software development saying that software is fundamentally different from manufacturing or civil construction. There is substance to this. Code is design. Compilation (build) is zero cost for all practical purposes. If we agree that compilation is the equivalent of a manufacturing assembly line or brick-laying, then the economics of software development are very different indeed. The economics of software make it particularly vulnerable to the ill-effects ot the current patent regime.

I used to find it surprising when highly credible people said there was nothing fundamentally different about software patents - "If you're against software patents, you're against patents in general." Until I realized that software patents aren't inherently evil - the devil, as usual, lurks in the details.

Patents exist to balance competing objectives:

Encourage innovation
Give innovators a chance to profit from their success
Improve the body of knowledge in the public domain (eventually)

In practice, the second objective is generally met by conferring a twenty year period of exclusive usage rights. Thus, patents are economic devices. Won't the efficacy of an economic device depend on the underlying economics of the industry it is meant for? Do all industries require twenty years to recoup investment and monetize an innovation? It certainly is ridiculous for software. Technology cycles keep getting shorter - twenty years is several generations in most industries today. A two year patent might be more reasonable for software. Of course, that still leaves jokes like the one-click-checkout patent untouched.

When patents were first conceived, they were meant for flesh and blood patent holders, not for mere legal persons (Corporations). Corporations gained personhood much later. However, the majority of patents today are held by corporations rather than individuals. This has lead to barely legal behaviors that end up stifling innovation in the industry as a whole.

What's really needed is a nimble international body that can tweak the terms of patenting for different industries. But then, international co-operation is hard to come by even for life threatening issues such as climate change. No wonder people push for elimination of software patents instead of push for a better patent regime.

Being explict is better than being implicit - only when everything else is constant

2009-09-30T08:00:00.003+05:30

The benefits of dependency injection are:

Separating configuration of dependencies from their use.
Improving unit testability.
Making dependencies explicit.

As I finished explaining these points to some new hires, I followed it up with a generalisation, "And in general, being explicit is better than being implicit - it reduces ambiguity and improves communication, always welcome in software development." Immediately I was struck by a few incongruities. After all, we have come to appreciate:

The absence of Java-esque checked exceptions in C#.
The flexibility afforded by dynamically typed languages.
The evolution friendliness and interoperability of loosely typed SOAP web service contracts (documents, coarse-grained interfaces, restricting ourselves to the simplest of XSD types).

In the above cases, being explicit comes with a stiff cost. We trade it off for flexibility. So the generalisation still holds. In general, being explicit is better than being implicit. But there is a caveat implicit in there - "In general, being explicit is better than being implicit provided everything else remains constant." If there are additional benefits to be gained from going implicit, then being explicit is questionable.

It helps to make this caveat explicit. A case of making the generalisation practise what it preached.

SOA in India

2009-08-21T22:27:00.003+05:30

businesstechnology.in run by S&S Media (people behind JAX conference) interviewed me on the state of SOA in general and in India in particular.

How applications learn. What happens after they are built?

2009-06-18T00:47:00.002+05:30

I have been reading Stewart Brand's much celebrated book over some time now. Took some inspiration from it for a local KM-community talk on innovation. In particular, I was tickled by his characterization of how ideas spread:

Perhaps culture is driven by just such flea-market ideas in a vast network of uncredited influence.

Replace culture by 'innovation in software' and it still makes perfect sense. Our aggregated blog page is a good example of a flea-market of ideas.

Stewart argues for evolutionary design over visionary design - now where have I heard that before? It seems he once suggested to an architect that he go back to his building to see how the users were finding it. The architect said, "Oh no. You never go back. It's too discouraging." Just reinforces that we need to be on our guard when we choose technologies to build applications. Will the eventual maintainers of the application be comfortable with them? Is the app designed for change? Does the roof (read abstractions) leak?

It's true that construction industry (or any other industry for that matter) isn't a good analogy for software development. In particular, coding is not analogous to construction (compilation probably is). But the life of a building seems to have striking similarities with that of an application. Though buildings are expected to have much longer lives than applications, the total churn that they are subjected to is probably comparable. Buildings are subjected to form-over-function pressures of the marketplace - software apps are less so. Both suffer every now and then from architect hubris.

Wired covered the book long ago. The BBC documentaries based on the book are still available.

Part 1: Flow
Part 2: The Low Road
Part 3: Built for change
Part 4: Unreal estate
Part 5: The romance of maintenance
Part 6: Shearing Layers

There is also a good summary up at: http://www.gyford.com/phil/writing/2004/10/24/how_buildings_le.php

ThoughtWorks on App Engine for Java: Google I/O Enterprise track

2009-06-10T19:06:00.006+05:30

ThoughtWorks' CTO, Rebecca Parsons and Martin Fowler presented (with usual panache) this overview of ThoughtWorks' findings on Google App Engine (and the cloud in general) at Google I/O. Being part of the enterprise track, their talk focussed on enterprise readiness/applicability.

A summary for the busy technologist:

Google App Engine

A reasonable option for a number of applications.
Sets the bar in terms of low barrier to entry
Technical things to consider before taking the plunge:

Testing - development environment not (yet) quite the same as production.
Persistence - Different idioms needed than when persisting to relational databases.
Concurrency - The deployment sandbox is effectively single threaded. A whole bunch of Java libraries that spawn their own threads will need to be ported over before they can be used on the app engine. Moving away from shared memory concurrency is probably good anyway.

Advice for the enterprise

There is a spectrum from infrastructure (Amazon) to applications (Salesforce) with platform (app engine) being in the middle but closer to infrastructure. Domain specific play (statistical analysis cloud) is possible in the space between platform and applications. Of course, considering the venue, the speakers did not emit competing names like Amazon or Microsoft.
Standard potential advantages - dynamic scaling, low entry barrier (therefore low downside risk), pay as you go (op-ex over cap-ex)
Economies of scale (in terms of skilled people and infrastructure) mean that it is much harder to do-it-yourself (private clouds).
Security, privacy and IP - SLAs are ok but not nearly as good as the ability to fire your CIO when things go horribly wrong.
Is it really cheaper? - Unambiguous yes only if you move everything and close down your own data center.
Need new IT org? - probably yes - hopefully one that has better relations with the business.
Vendor lock-in? - probably not much worse than Oracle lock-in.
Other use cases - experiments, spiky one-time computational needs.
This is another of those things that lets you focus on core competencies.
Has the potential to solve the last-mile problem in agile software development - frequent and incremental deployment to production.