chunkysalsa

When switching on the oxygen pumps, there was an explosion on board of the Apollo 13 space craft. A short circuit in a small module led to an explosion rendering most of the space craft useless.

For days, the crew frantically worked in cramped quarters trying to return to Earth. Tensions ran high, but, instead of blaming each other for the mistakes they made, they concentrated on the system of the space craft and how to work with what was left of it. Only that broad, holistic focus on the system let them survive.

The same is true for your development process. Focus on the whole system instead of blaming each other for slow releases of new features. Value-Stream-Mapping can help you shift your focus from fixing individual shortcomings to the optimizing the entire system.

Value-Stream-Mapping is a lean technique

It visualizes the whole process from idea to customer release as a series of connected tasks. Both value and non-value adding tasks should be defined and tracked from customer requirement to delivery.

By looking at the holistic representation of your complete development cycle, and whether or not the individual steps are adding value, you can create a visual model of your process and easily identify waste in the system.

How do you make waste visible?

To draw a Value-Stream-Map, everyone involved in the process comes together. Choose one feature you recently worked on. Try to remember the exact steps which resulted in the finished feature:

• Who had the idea and when?
• How long did it take to decide to actually do it?
• What exactly was the first action taken to start design?
• How long did the design take?
• When did the developer get assigned the new feature?
• …

And so on – all the way to release and successful customer adoption.

You’ll come up with a long, chronological list of actions and waiting times between those actions. Now draw that sequence as a step-diagram. Put the actions as lines above the ground line. And put the waiting times as lines below the ground line. Mark actions in green and waiting times in red for better visibility.

Now, you’ve got a visual representation of how that one feature came into existence. The longest waiting times (waste) should quickly become obvious.

Just seeing those long waiting times starts the discussion

Suddenly, operations folks will talk with QA and developers. How can we shorten that big gap? Can we skip it entirely? Everyone is engaged, because you’re trying to tackle an obvious problem instead of finger pointing.

Value-Stream-Mapping makes it visible that the biggest problems aren’t individual people, but the system they work in. It’s obvious that finger pointing does not address the biggest issue. Instead, everyone focuses on how to improve the system.

We had a whopping 18 months of accumulated waiting time

It was a typical feature we mapped out. While the actual time we worked on the feature was measured in hours and days, the waiting times between the steps were weeks and months. The biggest gap was created by our 6-month release cycle. It was adding at least three months of waiting time to every feature. And if a feature request came in just before the previous release, it waited for over six months.

It was immediately obvious that it wasn’t the fault of the consultants or the developers. It was the fault of our system having only two releases a year. Immediately, we stopped blaming each other and focused on the problem at hand: how could we reduce those long release cycles?

It didn’t take long and everyone agreed: Let’s try two month release cycles instead of six months. The risk seemed great, but the reward greater. Everyone was committed to the goal and we took that leap of faith.

It was a hard and bumpy ride – nearly breaking our necks – but it forced us to improve our work so that we survived those two month release cycles. A few releases later, surviving became thriving and we ditched the cycles in favor of continuous releases.

But continuous releases must have brought down your quality

Yes, quality was a major issue in the early days. Initially, we stuck to manual regression testing cycles – each one a couple of weeks long. But not very helpful. We found out that by releasing every two or three days we had fewer bugs which were much easier to locate and fix. So from the customers’ view our quality went up.

Our customers didn’t have to fear those huge releases with hundreds of bugs we pushed down their throats periodically. Instead, we released a small feature and a few bug fixes every other day. Major customer complaints all but vanished. On the contrary, they were thrilled to get their features and bug fixes within weeks instead of months or years.

Cut out waste by doing Value-Stream-Mapping

Gather everyone involved in bringing a new feature to your clients. Time all product development actions and the waiting time between those actions. Draw them on a step-graph and identify the biggest waiting time. Discuss how to change the system rather than blaming individuals for the delay. Only then can you reach your destination in one piece as the Apollo 13 astronauts did.

VMOS

Imagine a seven year old playing the piano. She hits every note like it’s the only one, taking long breaks between each note. The play drags and listening to the singular notes is a pain. Instead of music, all you hear is a bunch of individual sounds, each one rivaling with the others to be the loudest one.
Now imagine the same play performed by the same kid five years later. The notes flow like a river, the emphasis not on individual sounds but on the whole sequence at once. Listening to the piece is pure joy, because every note works together with the others to create a beautiful experience.
Distributing your software development through separate departments is like a seven year old playing the piano. Every department works on its own, rivaling with the others to be the most important. The output is a pain for your customers and the quality is really poor. But how can we create an organization where the individual parts play nicely together? One way of making departments play together are feature teams. Let’s see how this could work out.

Feature teams are cross-functional teams

They consist of team members from every department involved in feature development: business analysts, developers, quality assurance, operations, and so on. Ideally, they’re co-located – working in a big room together or at least in the next room. This shortens communication paths and speeds up necessary information exchange.

Feature teams are responsible for a certain set of features or a specific service. At amazon.com every part of the website like recommendations or reviews are separate services developed by individual feature teams. Each team is fully responsible for the feature, from idea to operations. Any challenges that arise within their specific feature must be solved by the team.

Solving challenges is simpler in smaller teams

If you form feature teams you should keep them small. A famous quote from Amazon: “If a project team can eat more than two pizzas, it’s too large”. Keep your team between five to nine people. This will keep the complexity of inner-team communication as low as possible. You scale your projects by combining multiple small teams – each one fully responsible for certain features.

Embed every required skill within your teams to create a successful feature. This isn’t possible with all skill sets, but you should limit external party support as much as possible. External dependencies always slow down a team and increase complexity.

It’s all about reducing complexity

Trying to drive a feature through multiple departments can be quite a challenge. Every department has their own goals which usually conflict with other departments. Even if they want to help, they have a hard time understanding exactly how.

Building feature teams tears down those departmental barriers and joins people from every department into one team. This helps to create empathy and, more importantly, trust – the number one factor to speed things up in complex environments. Without trust, blame games and CYA tactics are rampant. Only if people know each other, have the same goals, and work together are they more likely to trust one another.

That trust thing sounds great, but won’t work for us

Let’s say you already have feature teams. This is often the case in so called “matrix-organizations” where every employee is part of a certain department but works in multiple projects. And with multiple projects, the problems start. The employees are supposed to serve multiple bosses: the project leads as well as their department head. Because they’re assigned to multiple projects, they usually find home in their department and give it the highest priority.

But giving the highest priority to the department is not what you want. Therefore it’s important to assign people to one and only one project. Make that project (and the feature team driving it) the home for your employees. The department should mainly serve as interest group.

Feature teams take responsibility

They’re fully responsible – end-to-end – for a certain feature or service. They’re small, co-located and they have all the needed skills to be successful. Feature teams act as the home base for employees as departments fade into the background providing mentorship for honing one’s skills. But the main, business contribution happens in the feature teams.

If you setup your organization like this, you’ll begin to see a nice flow of great feature releases – like a well played musical piece.

Tom Check

“But our customers don’t want 10 new versions a year. The last release alone had over 600 bugs!” retorts the hotline manager.
“How about a small update with just a handful of bugs?”

Your big-bang release is scary. It’s full of issues and weird, new features that nobody understands. It requires documentation and training and who the hell has time for all that?

Monthly, bite-size updates will have fewer features requiring less support (pro-tip: less code == less bugs). Speeding up your release cycle also allows quicker response to customers’ feedback. You’ll finally feel your company moving in the right direction again.

Of course, it’s easy to say. But how can you actually achieve this positive flow? Follow these key points and you’ll be well on your way.

Keep it Simple Stupid

• don’t spend three months gathering specs
• create immediate user value (MVP) and prepare to iterate based on customer feedback

Cross-functional Team Empowered to Release

• Product owner, QA and devs write acceptance test together up-front
• devs code until test is green
• sysadmin deploys live

Automatic Distribution Channel

• easy way to get updates out quickly and reliably
• painless for the customer to install

Switch on the Feedback Loop

• know about problems before the customers (and have fixes ready to deploy)
• listen to what customers are saying about the new features and adjust for the next update

Bonus: Business Shifts to Monthly Subscription Revenues

• align the business with the development schedule
• inflexible departments don’t work well with motivated project teams
• make sales with a carrot not a stick ($19.99/mon vs $3000/yr)

GrahamPics1

Out with the Lion and in with the Lamb

Which of these two beasts would you rather handle? Thinking about opening up a petting zoo and charging money? I’m pretty sure not too many folks would pay to pet an enraged lion, yet that’s exactly what enterprises expect.

It’s time to tame our releases. Make them something our users actually look forward to.
“Would you look at that? What an adorable little thing!”

Burning Image

Monday morning on the highway. Your speed: 0 mph. You’re stuck in the usual rush hour traffic jam because the capacity of this road is exceeded. And it’s now obvious you’ll reach your destination much later than if the road were empty.

But what happens if you exceed the capacity of your development team? What happens when you cram in more features than the team can develop? Your features will get stuck in development. Let’s see how this can happen.

What happens if you push too many features into your development pipeline

Let’s discuss three negative outcomes of overloading your pipeline:

1. You lose focus by increased task switching
2. Managing the waiting queue costs a lot of time
3. Big releases create big headaches

Having too much upcoming work leads to task switching

When an avalanche of requirements hits, you have the urge to do everything in parallel because everything is important. You dutifully start with the most urgent task, only to have the second most important thing blow up in your face. Switching focus to the new number one in your list lets you watch the next three things go off like an ill-timed firework display.

When you’re constantly switching from one task to the next, you’ll hardly finish any one of them. Every single task stays on your todo list longer than it should. And I’m not even talking about the time you lose adjusting to every new task. Just the fact that you’re hopping from task to task increases the lead time of each dramatically. Instead of getting your overloaded pipeline cleaned up, more and more stuff piles up in front of you.

The second problem is managing the waiting queue.

If more and more stuff piles up, you waste time managing that queue

Again and again, you revisit and re-prioritize your ever growing list of tasks. Every time you go through that list you need to remember the details of each item, it’s current status, and so on. This takes a lot of mental energy and a considerable amount of time. That time and energy are lost instead of used to finish things.

And while you’re not getting anything done, guess what? That todo list grows longer and longer resulting in even more queue management work. A vicious cycle which is only made worse with today’s tools. We’ve got a host of bug tracking tools which let you manage hundreds or even thousands of open issues. I was once very proud of having over 1400 open issues sitting in our bug tracker. But it was an ordeal to go through even the upper part of that list again and again. We finally scrapped it and never looked back.

And that brings us to the third ill effect of overloading your development pipeline: big releases.

Dealing with too many features in parallel means preparing big releases

And assembling big releases means a lot of complexity. This complexity increases management overhead and reduces quality because no single person has a complete overview anymore.

Instead of knowing exactly what went live (and what might have caused a certain bug), you’ve got a big batch of stuff going live all at once and no chance of ever finding out why something broke. This is a very bad situation.

But it gets even worse than that: with growing complexity, I’ve found the number of bugs increase more than just proportionally. That means if you’re releasing double the amount of features, you’ll have to deal with way more than double the problems. Remember that nice task list – your “development pipeline”? Feel free to cram these release issues right on top.

“Yes, quality issues are clogging my pipeline, but there’s nothing I can do about it”

I know, it looks like that. But I’ve seen teams move from big releases with weeks of stabilization phases to continuous deployments. By creating a pull system and focusing on flow it is possible to break the vicious cycle.

These things get you into a vicious cycle

If you overload your pipeline, things will get worse because of more task switching, increased management overhead to manage your todo lists, and the unfortunate dynamics of big releases.

Create a pull system and focus on flow to break that vicious cycle and you’ll never feel like being stuck on the highway during rush hour.

What are your experiences with having too much stuff to do? Please share it with us in the comments.

Wikipedia

A rope. Eight people on either side. “Pull!” And then it begins: both parties are pulling in their own direction. A tug-of-war has started.

Imagine your developers and sysadmins as those two parties starting that tug-of-war

Each group has different goals. And having different goals leads to each party pulling in another direction. How can this happen and what to do about it?

In most organizations every employee has a set of personal goals

They usually bubble down all the way from the CEO (who get’s his goals from the board) via the middle management down to each and every developer and sysadmin. In most organizations development and system operations are separate departments and have separate managers. Both might report to the same C-level executive but both will get a different set of goals.

Those goals seem to make sense from the point of view of the C-level executive

But things get out of control when goals are passed down the hierarchy. What began as a nicely orchestrated company roadmap, entangles into a distorted mess of misaligned, even conflicting goals for the teams responsible for execution.

The mess thickens faster than you’d imagine

Like when playing Telephone (aka Chinese whispers), the original intentions are changed after filtering down through a few levels in the hierarchy. Just one or two levels are enough to end up with conflicting goals.

Conflicting goals lead to a tug-of-war

To avoid such a tug-of-war you need to set coherent goals. The most important step towards coherent goals is the harmonization of goals for all team members involved in a business process. With everyone finally pulling in the same direction, you can create an aligned DevOps organization.

You can harmonize goals by discussing them openly across the teams

Letting teams play an important role in defining goals makes sure that everyone understands and embraces the strategic business goals defined by top management.

Understanding the strategic business objectives helps to create harmonized goals

Once everyone understands and embraces the strategic business objectives, you have a harmonized set of coherent and realistic team goals. You’ll avoid that tug-of-war and your departments will collaborate naturally. You’ll experience the dawn of a real DevOps culture.

Alasdair Middleton

There are DevOps tools and DevOps job ads. People talk about culture and sharing and being nice to each other. Sounds pretty fishy, right? The only thing missing is a DevOps certification and we’re done with the DevOps hype. Is DevOps really just a fad? Let’s take a closer look…

You’re frustrated with another department – again

They don’t seem to understand the simplest things. They just throw stuff at you which keeps exploding in your face. The only way to deal with the continual onslaught of their crap is to become cynical. You rant about those guys and try to blame them whenever you can. You are sick of working on their garbage again when you could be doing real work instead. The things that really matter to your department.

But this is why you get a paycheck, right?

Actually, it’s exactly where DevOps comes into play. Why should your day be filled with anger? DevOps tries to overcome the aforementioned obstacles. By encouraging collaboration earlier in the process, DevOps tries to smooth out the boundaries between departments. By demanding automation, DevOps tries to minimize chores and create more room for valuable work. And by encouraging sharing of problems and solutions openly, DevOps tries to dissolve the combative cultures of blame.

It’s up to you to decide whether this is a fad or not. What do you think? Please let us know in the comments!

AMagill

Investing into code improvement is a dual edged sword: on the one hand you know that if you don’t improve your code you’ll get slower over time. On the other hand improving your code does not deliver tangible value to your users. So how do you know whether you’re on track?

Track the time a user story needs from idea to production

If that time is getting longer you’re off track: your code is slowing you down. It’s time to invest more into improving your code.

Measure the number of bugs per code push

If that number increases, you’re investing too less into code quality. Build quality in by writing automated tests upfront and by streamlining your code base. Rip out over-complicated stuff which people fear to touch because it breaks every time they do.

Are you spending too much time gold plating your code?

If the above measures do not move in the right direction even though you’re trying to keep your code as clean as possible you’re optimizing the wrong stuff. Try to find those areas in your code which are worth improving and focus your improvement efforts there.

Don’t try to improve stuff which is never touched again

It’s only worth changing even the worst piece of code if you’re going to touch it again. While it sounds like a good idea to ensure that all your code adheres to your coding standards, that’s not feasible most of the time.

Make sure you put the right measures in place to find out whether you’re investing too much or too less into improving your code. What is your strategy to balance new features against code improvements? Please leave a comment below.

State Library of South Australia

London Stansted Airport, early afternoon – a huge crowd at the railway station. No trains – only people, and a lot of confusion and anger. Having just flown an hour from Munich, Germany, I was supposed to meet a client two hours after touch down. But here I stood, waiting with the crowds: the railway just broke down. After several hours of waiting, I was able to catch a bus and reached London city late in the evening – but the client meeting was long over. The whole trip (not to mention an entire day of my life) was wasted just because one leg in my journey failed. Do your users waste their time because you have failing legs in your product development journey?

Your customers suffer because of delays!

If you accept delays within your product development pipeline, your customers lose out on opportunities and waste a lot of time. No matter whether a feature comes late because you ignored it, wasted time gold-plating the code, or just let it sit in your code repository for weeks, your clients derive zero value.

Your customers only get immediate value from developed functionality when you avoid delays

You need to avoid delays throughout the entire product development cycle. Optimizing just one part (i.e. developers should do “agile”) is like flying a faster plane to London just to get stuck there earlier (and longer). If you want to ship features as fast as possible, look at every stage, not just your area of influence.

You can’t influence the way features are designed

But you can try to talk to your colleagues from product management and try to find ways to avoid delays. Can you do with less upfront specification? Would it be possible to start spiking complex technologies long before a feature is completely specified? Is there a way to break down features into smaller independent parts? Show them you’re ready to adapt the way you’re working to help the overall goal of shipping sooner.

Still playing the blame game?

If you are, take one step back and look at the overall process. How does blaming others help in avoiding delays? Blaming means you’re just looking at your part in the product development chain. You’re working optimally but everyone else sucks.

That attitude is not very helpful

In contrast: it will deepen the divides between departments and even increase delays due to less communication and trust. Accept that everyone is trying their best and assume that everyone is willing to contribute to the company’s joint success. With this basic mindset, you can approach other parties and work on increasing collaboration and trust while removing the delays.

Are you repeating yourself?

You know what I’m talking about: pushing new code to staging is a lot of manual work. And, if your testers need a fresh database dump, they wait for you to manually anonymize and import it. You’re sick of them requesting fresh code and data all the time because this manual labor keeps you from doing your real job. Manual steps in any process slow things down, create dependencies on the people who are able to do it, and are error prone.

You need to automate as much as you can

Setting up a new server should only be one command away. Loading and anonymizing a database dump? There’s a script for that. Rolling back a bogus change? Just another command and we’re green again. Continuous deployment and configuration management tools help you avoid those unnecessary delays.

This won’t work in my company

You might think that your environment is too complex to try any of the above. While it is true that a long history of silos and strong departmental boundaries make things harder, you’d be surprised to know everyone inside the other silos are striving towards the same goal: making your customers happy.

Approach others with respect and a positive attitude

By stopping the blame-game you can build trust. And by building trust you can increase collaboration and eventually speed things up dramatically.

You can start today

Changing your attitude is solely up to you. Approach your colleagues and discuss the process delays with a positive attitude. Together, try to find ways how to avoid delays by working on smaller items, doing less preparation work, and automating recurring, intensive manual labor. You’ll see that looking at a shared goal of avoiding delays gives everyone focus and energy.

By working on the whole product development process, you avoid getting stranded by a broken railway after rocketing over the longest leg of your journey at over 800 km/h.

We’ve teamed up with Packt Publishing to organize a giveaway of their new book “Continuous Delivery and DevOps: A Quickstart Guide” as a holiday gift to you our readers!

Four lucky winners stand a chance to win copies of this new book. Keep reading to find out how you can be one of them.

Continuous Delivery and DevOps

As the author puts it, the book is a “collection of suggestions based upon experience and observations.” It’s all about:

• Real world examples of how to go about implementing continuous delivery and DevOps
• Learning how continuous delivery and DevOps work together with agile tools
• An honest and open guide to consistently and quickly shipping quality software

Read more about this book and download the free Sample Chapter

How to Enter?

All you need to do is head over to the Continuous Delivery and DevOps book page and look through the product description of the book. Drop a line via the comments below this post to let us know what interests you the most about this book. It’s that simple.

Winners from the U.S. and Europe can either choose a physical copy of the book or the eBook. Users from other locales are limited to the eBook only.

Deadline

The contest will close December 31, 2012 at midnight Pacific Time (PST). Winners will be contacted by email, so be sure to use your real email address when you comment!

Don’t lose any time and head over to the Continuous Delivery and DevOps book page right now. We’re looking forward to your comments here on our site. Happy hunting and good luck!