www.thoughtworks.com aggregator

Sean Doran: What are the Pennies in Your Process?

Fri, 01 Jun 2012 00:46:53 +0000

I was intrigued and pleased to see that Canada has recently decided to eliminate pennies from the coinage system. Although not the first country to do so, Canada certainly will not be the last.

The simple reason to eliminate the lowest denominated coins is that they cost significantly more to produce and manage then they deliver in benefits to the economy. The raw cost of the metal added to the significant manufacturing and transportation costs means that “it costs the Government of Canada 1.6 cents to produce each new penny.” These tangible costs to the Mint are substantial but the total costs to the economy are estimated to be even higher, perhaps as much as $150 million. All for a coin that most Canadians simply toss in a jar when they get home.

Eliminating the penny got me thinking about the nature of continuous improvement on a software team and the need to eliminate practices that were formerly useful but are no longer worth the effort. Said another way, what are our pennies?

One practice that comes to mind is high ceremony estimation activities. Even Agile teams that are doing effective relative estimation of effort can have time consuming estimation practices. Symptoms include long estimation meetings with elaborate planning poker techniques to evaluate each individual story. Like a penny, each question of “how big is this story?” may be small, but across a large user story list it adds up to a lot of extra time invested.

Is the investment in doing high ceremony estimation worth the value achieved? I would argue that most teams do get some value out of accurate estimations in the form of predictability, but many people challenge this. For new teams (or teams new to Agile) I would argue that value also comes from the conversation about the story during estimation as it helps increase understanding of the project backlog, helps establish a common domain language, and helps explore differences in implementation approaches between different developers. All in line with the old cliche that the plan is nothing, the planning is everything.

However, as a project team gels, it may find that high ceremony estimation delivers less and less value. A high performing development team should be able to provide consistent estimates quickly because it has a well understood relative effort scale. And the additional benefits are less important to the team as it already knows the user story list well, does lots pairing, and so on. So these teams should be able to greatly reduce their investment in estimation ceremony and estimate much quicker. Or even stop estimating altogether and just manage a flow of stories.

I see lots of teams approach continuous improvement from the perspective of fixing broken processes. The metaphor of eliminating pennies highlights a perspective that focuses on the value returned from the small, little activities that a team does over and over again. These practices are not necessary broken. The penny works perfectly well in the Canadian economy to deliver a very small amount of change. And some high ceremony team practices do provide some value. But is the level of effort worth the value received? A worthwhile question for more than pennies.

Mark Needham: Google Maps without any labels/country names

Thu, 31 May 2012 21:52:29 +0000

I wanted to get a blank version of Google Maps without any of the country names on for a visualisation I’m working on but I’d been led to believe that this wasn’t actually possible.

In actual fact we do have control over whether the labels are shown via the ‘styles’ option which we can call on the map.

In my case the code looks like this:

var map = new google.maps.Map(document.getElementById("map_canvas"), {
  zoom: 3,
  center: new google.maps.LatLng(31.492121, 14.919434),
  mapTypeId: google.maps.MapTypeId.ROADMAP
});
 
var emptyStyles = [
  {
    featureType: "all",
    elementType: "labels",
    stylers: [ { visibility: "off" } ]
  }
];
 
map.setOptions({styles: emptyStyles});

And this is the result:

Jim Highsmith: Reducing Cycle Time

Thu, 31 May 2012 18:05:14 +0000

An increasing number of organizations are moving towards radical reductions in cycle time as they move towards rapid business responsiveness and Continuous Delivery. (I’m trying to reduce my personal cycle time, but that’s another issue.)

One mantra that seems to help teams and organizations in this quest is, “If it’s hard to do, do it more often!” Keep this mantra in mind. If something appears too hard, or too costly, or too slow—figure out a way to do it more often. I once worked with a company whose product took six months of Q/A prior to release. The Q/A manager couldn’t imagine how to reduce the time to two-week iterations, so I asked him if he could figure out how to do it every two months. After several subsequent iterations, his group was able to support two-week iterations. From lean manufacturing examples, we often see that 80% or more of the time taken to accomplish a process usually works out to be wait time, not work time, so pushing for significant reductions is often much easier than anticipated at first.

In trying to answer the question “How can we do something frequently?” the answer may come from a combination of simplification, eliminating constraints, and automation. Every time we push to do something more often, be it a software build and integration or design, we learn. Learning comes from repetition.

From Goldratt’s theory of constraints we have learned to look for process bottlenecks. The bottleneck could be the lack of a particular skill or the throughput of a machine or a computer, but the key aspect of a bottleneck is that eliminating it can create significant improvements in overall throughput and cycle time reduction. Conversely, if we don’t think about bottlenecks, we can add significant resources without impacting throughput at all.

Teams should always ask the question, “How could we simplify this process or activity?” Or, the question may be more like, “What can we eliminate or streamline in order to reduce the time to do this from 6 weeks to 1?” Again, the time to accomplish some overall process can often be traced to delays between groups and excessive control processes or steps. For example, if there is a sequential process that takes eight weeks and involves four groups, each group may have a logging, prioritization, and reviewing process for work items. Reducing the process to a week by eliminating communication delays may also eliminate the need for these control processes.

Since we are in the business of IT, automation always comes to mind as an enabler to doing things more often, but we shouldn’t jump to automation until some of these other ideas have been tried.

The mantra “If it’s hard to do, do it more often!” espouses the agile value of responding to change. In today’s high-change world responsiveness to change is tied to the cost of change—reducing its cost increases our responsiveness. The high cost of some change should not be viewed as a barrier to responsiveness, but as an opportunity to increase our responsiveness by overcoming that barrier.

Anand Bagmar: Test Driven Development via Agile Testing

Thu, 31 May 2012 16:48:00 +0000

I will be giving a talk in the "Next Generation Testing Conference" held in Bangalore on 21st June 2012.

The topic and abstract is as mentioned below. See you at the conference!

Title:

Test Driven Development via Agile Testing

Abstract covering main features of the talk:

In this talk, I will cover the following topics and answer questions related to:

· Overview of Agile Testing

· The Test Pyramid

· Different flavors of TDD

o BDD – Behavior Driven Development?

o ATDD – Acceptance Driven Development?

o BDT – Behavior Driven Testing?

§ Difference between BDD and BDT

§ Tools that support BDT

§ The value proposition

Ross Pettit: Rules versus Results

Thu, 31 May 2012 15:20:00 +0000

Assessments are based, not on whether the decisions made are any good, but on whether they were made in accordance with what is deemed to be an appropriate process. We assume, not only that good procedure will give rise to good outcome, but also that the ability to articulate the procedure is the key to good outcomes.

-- John Kay, writing in the Financial Times

A common error in both the management and governance of IT is an over-reliance on rules, process and "best practices" that portend to give us a means for modeling and controlling delivery. We see this in different ways.

Project managers construct elaborately detailed project plans that show business needs decomposed into tasks which are to be performed by specialists in a highly orchestrated effort. And detailed they are, often down to the specific task that will be performed by the specific "resource" (never "person", it's always "resource") on a specific day. This forms a script for delivery. The tech investor without a tech background cannot effectively challenge a model like this, and might even take great comfort in the level of detail and specificity. They have this broken down into precise detail; they must really know what they're doing.
Companies invest in "process improvement" initiatives, with a focus on establishing a common methodology. The belief is that if we have a methodology - if we write unit tests and have our requirements written up as Stories and if we have a continuous build - we'll get better results. Methodology becomes an implicit guarantor of success. If we become Agile, we'll get more done with smaller teams.
"Best practices" are held out as the basis of IT governance. Libraries of explicit "best practices" prescriptively define how we should organize and separate responsibilities, manage data centers, contract with vendors and suppliers, and construct solutions. If there is a widely accepted codification of "best practices", and we can show that we're in compliance with those practices, then there's nothing else to be done: you can't get better then "best". We'll get people certified in ITIL - then we know we'll be compliant with best practices.

* * *

To see how stultifying such behaviour can be, imagine the application of this emphasis on process over outcome in fields other than politics or business. Suppose we were to insist that Wayne Rooney explain his movements on the field; ask Mozart to justify his choice of key, or Van Gogh to explain his selection of colours. We would end up with very articulate footballers, composers and painters, and very bad football, music and art.

-- John Kay

It is tempting to try to derive universal cause-and-effect truths from the performance of specific teams. Because this team writes unit tests, they have higher code quality. Or, Because we ask users to review security policies and procedures at the time their network credentials expire, we have fewer security issues. Does unit testing lead to higher quality? Does our insistence on policy result in tighter security? It may be that we have highly skilled developers who are collaborative by nature, writing relatively simple code that is not subject to much change. It may be that our network has never been the target of a cyber attack. A "rule" that dictates that we write unit tests or that we flog people with security policy will be of variable impact depending on the context.

There are no such things as best practices. There are such things as practices that teams have found useful within their own contexts.

@mikeroberts

Suppose a team has low automated unit test coverage but high quality, while another team has high automated unit test coverage but low quality. A rule for high unit test coverage is no longer indicative of outcome. The good idea of writing unit tests is compromised by examples that contradict lofty expectations for going to the trouble for writing them.

It isn't just the rules that are compromised: rule makers and rule enforcers are as well. When a team compliant with rules posts a disappointing result, or a team ignorant of rules outperforms expectation, rule makers and rule enforcers are left to make ceteris peribus arguments of an unprovable counterfactual: had we not been following the rule that we always write unit tests, quality would have been much lower. Maybe it would. Maybe it would not.

Rules are not a solid foundation for management or governance, particularly in technology. For one thing, they lag innovation, rather than lead it: prescriptive IT security guidelines circa 2007 were ignorant of the risks posed by social media. Since technology is a business of invention and innovation, rules will always be out of date.

For another, rules create the appearance of control while actually subverting it. Rules appear to be explicit done-or-not-done statements. But rules shift the burden of proof of compliance to the regulator (who must show the regulated isn't following the rule), and away from the regulated (who gets to choose the most favourable interpretation of the rules which apply). Tax law which explicitly defines how income or sales transactions are to be taxed encourage the individual to seek out the most favourable tax treatment: e.g., people within easy driving distance of a lower cost sales tax jurisdiction will go there to make large purchases. The individual's tax obligation is minimized, but at a cost to society which is starved for public revenues. In IT terms, the regulated (that is, the individual responsible for getting something done) holds the power to determine whether a task is completed or a governance box can be ticked. I was told to complete this technical task; I have completed it to a point where nobody can tell me I am not done with it. The individual effort is minimized, but at a cost to the greater good of the software under development which is starved for attention. It is up to the regulator (e.g., the business as the consumer of a deliverable, or a steering committee member as a regulator) to prove that it is not.

When the totality of the completed technical tasks does not produce the functionality we wanted, or the checboxes are ticked but governance fails to recognize an existential problem, our results fall short of our expectations despite the fact that we are in full compliance with the rules.

* * *

Instead, we claim to believe that there is an objective method by which all right thinking people would, with sufficient diligence and intelligence, arrive at a good answer to any complex problem. But there is no such method.

-- John Kay

Just because rules-based mechanisms are futile does not mean that all our decisions - project plans, delivery process, governance - are best made ad hoc.

Clearly, not every IT situation is unique, and we can learn and apply across contexts. But we do ourselves a disservice when we hold out those lessons to be dogma. Better that we recognize that rigorous execution and good discipline are good lifestyle decisions, which lead to good hygiene. Good hygiene prevents bad outcomes more than it enables good ones. Not smoking is one way to prevent lung cancer. Not smoking is not, however, the sole determinant of good respiratory health.

Principles give us an important intermediary between prescriptive rules and flying by the seat of our pants. Principles tend to be less verbose than rules, which makes them more accessible to the people subject to them. They reinforce behaviours which reinforce good lifestyle decisions. And although there is more leeway in interpretation of principles versus rules, it is the regulator, not the regulated, who has the power to interpret results. Compliance with a principle is a question of intent, which is determined by the regulator. If our tax law is a principle that "everybody must pay their fair share", it is the regulator who decides what constitutes "fair share" and does so in a broader, societal context. Similarly, a business person determines whether or not software satisfies acceptance criteria, while a steering committee member assesses competency of people in delivery roles. Management and governance are not wantonly misled by a developer deciding that they have completed a task, or an auditor confirming that VMO staffing criteria are met.

"We always write unit tests" is a rule. "We have appropriate test coverage" is a principle. "Appropriate" is in the eye of the beholder. Coming to some conclusion of what is "appropriate" requires context of the problem at hand. Do we run appreciable risks by not having unit test coverage? Are we gilding the lily by piling on unit test after unit test?

We are better served if we manage and govern by result, not rule. Compliance with a rule is at best an indicator; it is not a determinant. Ultimately, we want people producing outstanding results on the field, not to be dogmatic in how they go about it. Rules, processes, and "best practices" - whether in the form of an explicit task order that tells people what they are to do day in and day out or a collection of habits that people follow - do not compensate for a fundamental lack of capability.

Adjudication of principles requires a higher degree of capability and situational awareness by everybody in a team. But then we would expect the team with the best players to triumph over a team with adequate players in an adequate system.

Martin Fowler: NoSQL Distilled Rough Cut Now Visible

Thu, 31 May 2012 13:45:00 +0000

The rough cut of NoSQL Distilled is now online. If you have a subscription that supports rough cuts you will see the whole book, if not you’ll see a preview of the book with the table of contents and the first paragraph or two from each section. Remember this is a rough cut and has not been copy edited.

I was a bit hesitant to announce this, after my false sighting on Tuesday, but I’ve had a few tweets now confirming that it’s online.

Daniel Aragão: Simple things to develop public speaking and presenting confidence

Thu, 31 May 2012 01:24:00 +0000

It's quite common to fear public speaking. Getting in front of everybody and defending your point of view can be daunting. If you aren't good at it and would like, or need to, get better, here some tips.

Stand up. Literally. When in meetings or at a restaurant, get out of the chair and stand up. You can use whatever excuse you want. Say you’re stretching. The point here is to get used to you standing while others are seated. Stay up as long as you can.
This is a corollary of standing up. You need to enjoy the silence while standing up. A good presentation has good pauses. They are powerful. Don't be a chatterbox.
Present it! Present to your dog, children or stranger at the bus stop. Get used to talking about the things you do to people that don’t necessarily care or understand them. They will give you feedback on how to dumb things down.
Watch as many presentations as you can on the subject you’re trying to get better at. Observe how presenters and consultants behave, the things they say and how they say them.
Present other people’s ideas and presentations. Get a random slide deck from slideshare.com and present it. Don’t forget to give proper credits, of course. Then see what you’d have done differently if you were to create the slides yourself.
Learn to keep track of time. It’s crucial to gain peoples trust that you will guide them through the meeting/presentation on time. Let people know you’re tracking time and what’s left of it. You can do this by going to have coffee with your work mates. Establish a time to do the task and track it.

Mark Needham: Haskell: Using type classes to generify Project Euler #31

Wed, 30 May 2012 12:08:25 +0000

As I mentioned in my previous post I’ve been working on Project Euler #31 and initially wasn’t sure how to write the algorithm.

I came across a post on StackOverflow which explained it in more detail but unfortunately the example used US coins rather than UK ones like in the Project Euler problem.

To start with I created two versions of the function – one for US coins and one for UK coins:

combinations :: Int -> [USCoin] -> [USCoin] -> [[USCoin]]
combinations 0 current _ = [current]
combinations _  current [] = []
combinations total p (c:cs) = concatMap (\ times -> combinations (total - (times * fromEnum c)) (p ++ replicate times c) cs) 
                                        [0,1..(total `div` fromEnum c)]

combinationsUK :: Int -> [UKCoin] -> [UKCoin] -> [[UKCoin]]
combinationsUK 0 current _ = [current]
combinationsUK _  current [] = []
combinationsUK total p (c:cs) = concatMap (\ times -> combinationsUK (total - (times * fromEnum c)) (p ++ replicate times c) cs) 
                                          [0,1..(total `div` fromEnum c)]

As we can see the only difference between the two functions is the type being passed around and they are almost identical as well:

data USCoin = Quarter | Dime | Nickel | Penny deriving (Eq, Show)

instance Enum USCoin where
    fromEnum = fromJust . flip lookup usTable
    toEnum = fromJust . flip lookup (map swap usTable)
usTable = [(Quarter, 25), (Dime, 10), (Nickel, 5), (Penny, 1)]

data UKCoin = OnePence | TwoPence | FivePence | TenPence | TwentyPence | FiftyPence | OnePound | TwoPounds deriving (Eq, Show)

instance Enum UKCoin where
    fromEnum = fromJust . flip lookup ukTable
    toEnum = fromJust . flip lookup (map swap ukTable)
ukTable = [(OnePence, 1), (TwoPence, 2), (FivePence, 5), (TenPence, 10), 
           (TwentyPence, 20), (FiftyPence, 50), (OnePound, 100), (TwoPounds, 200)]

We can run those two functions like this:

> length $ combinations 200 [] (reverse $ map fst usTable)
1463
 
> length $ combinationsUK 200 [] (reverse $ map fst ukTable)
73682

After spending a lot of the past week reading about type classes I figured we could probably make use of one here so I created a ‘Coin’ type class:

class Coin a where 
  value :: a -> Int

And then implemented that with ‘USCoin’ and ‘UKCoin’:

instance Coin USCoin where value coin = fromEnum coin
instance Coin UKCoin where value coin = fromEnum coin

Then we need to make some small adjustments to ‘combinations’ to make it work with ‘Coin’ instead:

combinations :: (Coin a) => Int -> [a] -> [a] -> [[a]]
combinations 0 current _ = [current]
combinations _  current [] = []
combinations total p (c:cs) = 
  concatMap (\ times -> combinations (total - (times * value c)) (p ++ replicate times c) cs) 
            [0,1..(total `div` value c)]

Instead of calling ‘fromEnum’ we call ‘value’ and the function can now be used with any types which implement the Coin type class should we wish to do that.

We can actually go even further and get rid of our Enum type class instances and just put that code onto the Coin type class:

class Eq a => Coin a where 
  table :: [(a, Int)]
 
  value :: a -> Int
  value = fromJust . flip lookup table

instance Coin USCoin where 
  table = [(Quarter, 25), (Dime, 10), (Nickel, 5), (Penny, 1)]	
 
instance Coin UKCoin where 
  table = [(OnePence, 1), (TwoPence, 2), (FivePence, 5), (TenPence, 10), (TwentyPence, 20), (FiftyPence, 50), (OnePound, 100), (TwoPounds, 200)]

We can then call the function with either of those coins:

> length $ combinations 200 [] (reverse $ map fst (table :: [(USCoin, Int)]))
1463
 
> length $ combinations 200 [] (reverse $ map fst (table :: [(UKCoin, Int)]))
73682

Mark Needham: Haskell: Java Style Enums

Wed, 30 May 2012 11:10:15 +0000

I’ve been playing around with problem 31 of Project Euler which is defined as follows:

In England the currency is made up of pound, £, and pence, p, and there are eight coins in general circulation:

1p, 2p, 5p, 10p, 20p, 50p, £1 (100p) and £2 (200p).

It is possible to make £2 in the following way:

1 £1 + 150p + 220p + 15p + 12p + 31p

How many different ways can £2 be made using any number of coins?

Having coded way too much in Java my first thought was that the coins could be represented as an Enum but I wasn’t sure how to do that in Haskell.

As it turns out the question has been asked on Stack Overflow and there’s an Enum type class we can implement to achieve the same thing.

In this case:

data UKCoin = OnePence | TwoPence | FivePence | TenPence | TwentyPence | FiftyPence | OnePound | TwoPounds 
              deriving (Eq, Show)

instance Enum UKCoin where
    fromEnum = fromJust . flip lookup table
    toEnum = fromJust . flip lookup (map swap table)
table = [(OnePence, 1), (TwoPence, 2), (FivePence, 5), (TenPence, 10), (TwentyPence, 20), 
         (FiftyPence, 50), (OnePound, 100), (TwoPounds, 200)]

If we want to get the monetary value of a coin we use ‘fromEnum’:

> fromEnum TwoPounds
200

And if we wanted to go back to the enum value we’d do this:

> (toEnum 200) :: UKCoin
TwoPounds

If we want to get a list of all the instances of ‘UKCoin’ we can just do a map over ‘table’:

> let ukCoins = map fst table
> ukCoins
[OnePence,TwoPence,FivePence,TenPence,TwentyPence,FiftyPence,OnePound,TwoPounds]

Aaron Erickson: On (not) Being Post-Technical

Wed, 30 May 2012 00:20:25 +0000

In ThoughtWorks, one of the most poignant insults one can throw at you is “so-and-so has gone post-technical”. This usually means one has entered the land of management, that place where you give back your brain in exchange for money (or prestige.. or nothing, as it turns out).

Lately, as I have taken on additional roles at ThoughtWorks, the temptation to go “post technical” has put itself forward. Imagine not having to think anymore. Imagine being able to just focus on “strategy” and “people issues”, without all that hard technology stuff.

I could take that path, but I’d rather not…

I remain technical. My day-to-day job may involve things like Statements of Work and such, and I do not write code 100% of the time, but I am making a decision to at least remain somewhat involved in the technical communities in which I have interest.

Of course, my time is more limited, given I have taken on some management responsibility lately. So I can’t pursue everything. And I do have to admit that with my reduced time, I am likely never to be the most “technical person” in any given group of TWers. In fact, given that TWers are all generally really good at their jobs, it would be utter arrogance for me to assume I could keep up with them when I only act in a technical role for 40% of my time. So my bar isn’t TWers – they will most likely all be more technical than I am. But it is remaining competent enough to understand at least 40% of what they talk about, and more importantly, remain excited about technology.

So what am I excited about these days, here is my own, personal, “tech radar” (no fancy graphics required):

Functional Languages and Big Data

Someday, the world will catch up to where the F#/Clojure people are now. I think the tie between FP and Big Data is strong, and given that is where much of the value will be created in the next 5 years with corporate IT spend, the use of FP will only increase.

IaaS (note, I don’t say cloud… too overloaded)

I got the cloud bug this year, and I will admit, I am a terminal case. I saw an app where someone deployed a load balancer with an http request. And not an http request that converts into an email that goes to a tech that puts a physical one on a rack. Infrastructure as code is changing this business. Imagine the ability to specify, in human readable code, an organization’s entire server configuration. And realize that configuration by executing the DSL it is written in. That day is approaching, fast. Imagine the possibilities – from everything to devops to disaster recovery.

Why don’t I say cloud? The term is so overloaded that it is meaningless. Anything that runs on the internet is putting “cloud” in front of it’s name, which is why I have stopped using the term and do everything I can to use more descriptive terms.

Lean Startup

Though not a technology, it is the first thing from a project management standpoint I have been truly excited about in years. At least since Agile, maybe more so. Why? It finally closes the loop, bringing in the entire scientific method into the business of software development. It seems so obvious now, it never ceases to surprise me that it took our industry 50 years to adopt it. Lean Startup, to me, is the application of the scientific method to how you run business. I recently did a webinar that nicely encapsulates how I feel about it in much more detail, but my general sense is that this builds on CD and gives organizations that embrace it a much more repeatable, sustainable path to successfully delivering business results with software.

The three things above are the tools I believe will create most of the value in corporate software over the next 5 years. And I fully intend to remain competent enough to discuss these topics with both technical and business audiences, even if I don’t write code every single day anymore. The trick, not being a full time software developer anymore, will be to:

Make sure I don’t stop doing some technical work – I still intend to pair program with the team when I can, as well as keep up with a selection of OSS projects I am involved with.
Continue to read new technical material – I am on planes a lot, should not be too hard. For example, I am learning Python now, despite that I don’t have a good reason to personally code in python.
Be aware – ultra-aware – that I am not the most technical person in the room. My team-mates will likely know more than me, and I will defer to them on technical arguments more often than not, especially if they have decent data on why a given technical decision needs to be one way or another.

Will this work? We will see. I am only in the starting stages of this journey – will be interesting to see the degree to which my technical skills atrophy from not being a full time developer anymore!