Agile Web Operations

Simulating a Scrum And a Lean Project In The Classroom

Matthias Marschall — Tue, 10 Nov 2009 06:06:59 +0000

I will continue my course about agile methodologies at the University of Augsburg with both a Scrum and a Lean project simulation. The Scrum simulation will introduce the students to concepts like User Stories, Backlog, Iteration, etc.
After doing lots of Gantt Charts, Use Case Diagrams etc. in the waterfall simulation, it’s time now to break down the requirements into small and independent User Stories. The User Stories shall then be sorted by priority: The Backlog comes into existence.

Iterations

Now the teams will have to do a planning game: Estimate the most important stories (enough of them to be able to fill a first iteration) and commit to an iteration goal. The main target here is to come up with a releasable piece of (simulated) software. We’ll use the same technique as in the waterfall simulation: Implementation and testing of user stories will happen with pseudo code on cards and paper prototypes of UI screens. After a fixed time, I’ll ask the teams to demo their working functionality.

Retrospectives

After each iteration I’ll run a short retrospective with the teams. They should be able to learn quickly to improve the coming iterations. I’ll ask them to address at least one impediment they found in the previous round within the current iteration. Let’s see how this goes

End Of The Scrum Simulation

While I won’t introduce every Scrum concept in depth (I’ll skip the Product Owner for example), the students should get a good feeling about what it means to get something workable out of every iteration and how retrospectives help for quick learning. I’m curious how the results will differ from the previous waterfall simulation. Will the students really come up with more working features? Will they feel more in control and more productive? Let’s see…

Simulating a Lean Development Project

The third, and final, simulation will be Lean. Instead of working in iterations, the team will use a Kanban Board and try to get stories done as fast as possible. In the beginning, I’ll let the team start without any WIP (work in progress) limits. I can imagine, that the teams will stall – not being able to finish a single story in a given time. Then I’ll introduce the WIP limits and the flow should become smoother.

Wrap Up

After running all three simulations: Waterfall, Scrum and Lean, I’ll do a final retrospective including all three approaches. What were the experiences of the team? Where did they got done most stories? Of course, we’ll take the learning curve for the basic techniques (like writing and interpreting pseudo code and UI paper prototypes, formulating user stories, etc) into account.

What do you think? Will I be able to show the various effects we know from the real world in a classroom setting? Let me know in the comments!

Back to the roots: Bridging the Deployment Gap

Dan Ackerson — Tue, 03 Nov 2009 19:45:16 +0000

Matthias and I started this blog over a year ago because we had first-hand experiences with the rift between developers and sysadmins. We knew this was a lose-lose situation not only for those directly involved, but the companies they were working for as well. We’ve described many real-life examples of how to overcome this rift, but were never sure how these ideas were resonating out there with our fellow colleagues. How many developers had moved into the operations realm? How many sysadmins knuckled down and wrote end-user code in a pinch?

Enter Patrick Debois and devopsdays. What surprised me the most was the overwhelming number of sysadmins here who really understood the importance of agile development in beating the competition. Instead of blocking new features from being deployed, they actively work with the development teams to ensure timely, high-quality releases. Chris Read’s presentation of Continuous Integration, Build Pipelines and Continuous Deployment demonstrated a boatload of processes to help bridge the deployent gap.

As I am slowly running out of Pingdom checks, Lindsey Holmwood’s live demo of Cucumber-Nagios was especially interesting. As a developer, I would like to know immediately if the application changes I’ve made are somehow wiping out any existing functionality. And, of course, as a sysadmin I don’t want to be woken up at 2am, figure out exactly what’s wrong and rollback to the last known working revision. Cucumber adds a ton more flexibility to your acceptance tests and the Nagios plugin integrates perfectly with our existing monitoring solution. Additionally, Stephen Nelson-Smith has already done a post-conference, in-depth blog post on the above topic under the title Intelligent Monitoring.

How in the world can operations folks take advantage of agile processes in their day-to-day business? Scrum is too time consuming and XP is to resource intensive. Mattias Skarin gave a real-world example of how he helped an operations team move to Kanban – an endeavor that was so successful, the Ops team is now officially integrated into the Dev team proper. There were some excellent tidbits in his presentation Introducing Kanban in Operations that I can’t wait to try out myself.

The Open Spaces were a first for me, but I had fun excercising the “Law of Two Feet”. Too bad this can’t be applied more often to everyday meetings. Adrian Silva’s idea for Admin Coding Dojos is long overdue and I hope he is able to successfully get this ball rolling down in Spain – lemme know how it works out for you, Adrian! Finally, I’d like to thank Marcel Wegermann for a bit of BDD advice which has hopefully earned yet another convert to this tried-and-true coding discipline. He managed to convince my colleague that with some simple refactoring, you can break out of Zend – PHPUnit tests dependency hell and wrap those sticky areas with some testing goodness!

Like Bart Vanbrabant notes in his blog post, I also found many of the discussions to be very low level and missed more abstract discussion about how to bridge the gap between development and operations. I hope that next year we see a more balanced group in order to foster this important topic and call on every developer reading this blog to seriously consider making the trip next year to devopsdays!

Simulating a Waterfall Project In The Classroom

Matthias Marschall — Sat, 24 Oct 2009 15:10:28 +0000

The first simulation in my course about agile methodologies will be waterfall style. Here’s how I plan to do it.

Before we go into the details of the waterfall simulation, I want the whole group (around 20-30 people) to come up with requirements for the product to build: an online office suite (maybe the most boring but also the most well known thing in the world). I plan to gather the requirements upfront because I do not want to burden the first simulation with this additional task – all simulations should start mostly from the same starting point. The team will be free to document the requirements as they like, no special format required (those special formats of recording and managing requirements will be part of the individual simulations). When we have a good set of requirements, we’ll start with the first project waterfall style.

Splitting The Team

To simulate real world scenarios, I’ll split the team into four groups. Every group shall work on a sub-module of the online office suite. To simulate typical hand-offs between analysts and designers, etc. each group will first do the analysis of one module. Then the results of the first phase will be handed over to the next team for doing design. In the second phase, every team will do design another module as before. After the design stage, again the same story: Every team will hand over the design documents to the next group for development. And the same will happen for testing. That way every team will do every phase – but with a different module each time.

The Analysis Phase

In the first phase, the team will have two tasks:

Write use cases based on the initial requirements
Plan the project using a Gantt-Chart

I’ll hand out templates for the use case documents. The team will have to identify actors, come up with reasonable use cases and make sure they structure them by including and extending use cases and sub-use cases. The outcome will be a set of documents and a use case diagram.
After coming up with all the use cases, the team has to estimate the use cases and create a project plan with all dependencies and resource allocations. Of course, there will be “bottleneck” resources: a specialist team member, who is the only one to be able to do a certain type of tasks . The output of this exercise will be a big, hairy Gantt-chart. I’m already looking forward to tell the teams to do a tiny, little change shortly before the end of the phase

The Design Phase

Now every team has to hand over their use cases and project plan to the next team for designing a solution. In the design phase I’ll ask the teams to come up with class and sequence diagrams and a database model (in 3rd normal form, of course). The team will have to create a multi-tier SOA with a MVC based front-end. Just the common stuff

The Implementation Phase

After yet another hand-off, every team will start implementing the design. They will write pseudo code on cards and draw screen scribbles. Later, a “user” can interact with the UI by announcing where he would click or what he would type. Then the “CPU” (just another team member) has to try to interpret the pseudo code.

The Test Phase

In the last phase, the development team and the test team have to interact to ensure, that the written pseudo code and the scribbled screens are correct.
A “user” and a “CPU” will try to run the pseudo code. The user will announce how he interacts with the screen and the CPU will try to do what is written in pseudo code and come up with the required changes in the UI and data. If the CPU cannot execute a piece of pseudo code or the result is not what the user expects, there will be an “exception”. For every exception there will be a trouble ticket and the “developers” of the feature have to fix it.

End of The Waterfall Simulation

When every module has ventured through all four phases, we will do a “post-mortem” analysis of the problems and the achieved goals and record some key learnings. Documenting the experiences of every simulation is essential to be able to compare all three approaches.

What do you think about my approach on simulating a waterfall project in a class setting? Do you have any questions or ideas on how to improve it or what could go wrong? I would be very grateful for your feedback. Just leave a comment below.

Kanban vs. Iterative Development

Matthias Marschall — Sat, 17 Oct 2009 06:49:34 +0000

Agile methodology builds on the concept of iterations – time boxes – in which you create a piece of working software. Each iteration starts with a planning meeting where the team takes stories from the backlog and commits to the sprint goal. If you use a tool like Pivotal Tracker, you even get emergent iterations – the tool automatically cuts your backlog into iterations based on your team’s velocity.

In my post Agile on steroids I described how our team slowly dropped most of the agile estimation and planning practices in favor of optimizing cycle time of stories. Now we’re running a true hybrid of iterative development and kanban. So, what makes the difference?

Iterations Tell You When a Story Will be Finished

With time boxes and a known Velocity all stakeholders can easily find out, when a given story at a certain position in the backlog will be delivered. There will be variation in the velocity, but overall the estimation will be quite good.

Kanban Tells You, How Long a Story Will Take

In Kanban, you don’t measure how much you can do within a certain period of time, but how long a story needs from idea to roll out. You can’t really plan big feature packages, but you can optimize the time to market of any story. This is usually a good thing.

Iterations Provide Integration Points

As every iteration should result in working software, all stories need to be integrated by the end of the iteration. The iteration end acts as a reminder for the developers to integrate regularly (at least once an iteration, which isn’t often enough in my opinion).

Kanban Creates Continuous Flow

When you use a Kanban style, lean development process, there are no artificial iteration borders. Each and every story leads to working software and, optimally, to a release. This requires a higher degree of self discipline within the team.

Decision Criteria

If you need big synchronization points, e.g. for big marketing campaigns, you might be better off using an iterative development approach. If you want to ensure a continuous flow of features, which are optimized for time to market (cycle time), Kanban might work better for you. What are your experiences with the one or the other? Let us know in the comments or send me a tweet.

Waterfall, SCRUM and Lean Software Development simulation as teaching platform

Matthias Marschall — Thu, 08 Oct 2009 04:39:03 +0000

Currently, I’m preparing for teaching my next course on Agile Methodology. Again and again, I wonder what is the single most important thing my students should be able to take with them after four full days. One of my core messages is definitely that agile is more about principles than about practices. If you absorb the principles you have a good chance to get it right no matter which practices you decide to use or not. But, even teaching the underlying principles seems not enough to me. Tacit knowledge is far too important to be completely ignored in any course. So how can I transfer tacit knowledge in a course setting?

Hands on Simulation

This year I want to go for hands-on simulation of a software development project. The students shall run a simulated project multiple times, using different processes. I’m sure they’ll learn more from doing rather than me talking for four days.

Waterfall

I’m planning to start with simulating the project using a waterfall methodology. I’ll split my students into four teams. They’ll work on four modules in parallel. In the first phase all of them will work as analysts trying to define the requirements for their module and planning the complete project using Gantt-Charts.
In the second phase, all teams will work as designers specifying the features. But as it happens in the real world, none of the teams will work on the same module as before. Every team has to hand over the requirements they came up with to the next team, which has to do the detailed specs.
Same will be true for the development and testing phases.
Let’s see how much they’ll get done in that setup, and how they’ll react to changes in requirements during the later phases.

Scrum

The next round shall simulate a typical agile project with timeboxes and cross functional teams. Every module will be assigned to exactly one team and that team will try to develop the feature in iterations based on a backlog. The main tools for the teams will be a backlog of requirements (in the form of user stories), iterations and velocity for planning and retrospectives to adapt. Of course requirements will change during this simulation, too.

Lean Software Development

The final round shall simulate a typical lean software development project where the teams do not work in iterations anymore but use a Kanban system to let requirements flow through. The main tools to be used will be a story board, story cards, and cycle time for planning. The team shall learn to see waste and eliminate it.

Having run smaller simulations embedded in former lectures I’m curious to see how such a full blown sim will work out. Do you have any experiences in project management simulation games? How did it go? Let us know in the comments!

Giving up – or not?

Matthias Marschall — Sat, 03 Oct 2009 06:21:22 +0000

“Never give up” is an advice we hear far too often. We’re taught that giving up is a failure. But nothing could be further from the truth if we’re caught in a dead end. If there is nothing left to reach, why should we bother to go on? It’s a waste of time and energy trying to move mountains. Better give up and go for something worthwhile. It’s the only reasonable thing to do. Not giving up is a problem. Wasting your energy where nothing can be gained is a bad decision. Giving up and focusing your power to reach worthwhile goals is the smarter strategy.

But even if we go for a big, valuable goal, we might reach a point where it looks like we should quit. Everything looks bright and easy at the start. But sooner or later we realize how difficult the way to our goal really is. We might lose all our hope or even panic as we think we cannot make it. But as Seth Godin says in his little book “The Dip,” it is never a good idea to quit something because you panic. Difficulties on the way to achieve something great are natural and even necessary. Only those difficulties create the scarcity, which makes the goal worthwhile in the end. If it wouldn’t be hard to reach, everyone would do it and there would be no value in it anymore.

So, quit everything where nothing can be gained and be prepared for “the dip” when you are heading out to do something great. Don’t panic if things get rough and ugly. You might have to change your tactics if they don’t lead anywhere, but never stop striving to attain the big goal.

No related posts.

Pair Programming: Staying within “the zone”

Matthias Marschall — Fri, 25 Sep 2009 04:44:29 +0000

Today I spent the whole day debugging an elusive concurrency problem in ruby on rails running on JRuby. We start some threads during the web request and, usually sooner than later, all our database connections are blocked.

Getting deep into the details of multithreading, connection pooling and the like is nothing I enjoy doing. Especially not alone. I was really glad to have a buddy for the job.

Talking about a Problem helps sort your thoughts

Even thinking about all the twists and turns of our business logic made my head hurt. Having someone to actually talk about them made things better. If you force yourself to articulate the mess in your brain, it usually starts sorting out much easier than if you would try to only think it through. I know that this is not everyone’s thing to do, but for me it is very helpful.

When you run out of ideas, your pair might come up with a new one

Especially in chasing bugs, I usually start with a host of ideas where I could look for the issue. But, after hitting some dead ends, it becomes harder and harder to think of new approaches to address the issue at hand. Working together in such a bug hunt helps because as soon as the one runs out of ideas, the other one might come up with the guess, where to find the solution. Together, you’re able to avoid unproductive gaps throughout your workday.

Staying in “the zone” through laser sharp focus

Sitting together at one desk helps to completely ignore all those permanent distractions like incoming e-mail, twitter or IRC. Talking to each other and working on a problem together enabled us to keep focused on the issue at hand for over 8 hours today (short breaks included). After so many productive hours I feel completely drained, but it was worth it. Even though we didn’t fix the issue today, we sure learned a lot… Hope to squash it tomorrow.

Waterfall, SCRUM and Lean Software Development simulation as teaching platform

Open Communication Stops De-Motivating your Team

Matthias Marschall — Thu, 17 Sep 2009 04:46:54 +0000

Instead of motivating our teams, we should simply stop de-motivating them. Everyone you work with is highly motivated by default. But, bad information policies, countermanding orders or simply ignoring ideas will turn a highly motivated team member into a disgruntled road block.

Open Communication

Highly skilled team members are able to deal with the truth. Especially in a start-up environment, it is crucial to constantly let everyone know how the company is doing. There are two reasons for being totally open:

Informed team members take more risk.
Missing information will be guessed.

The Higher the Risks, the More Open the Communication has to be

If you start a new business and are able to get a great team, everyone in the team is usually very aware of the risks involved. If you don’t treat them as adults when things get rough, you will lose their trust and your team mates are more likely to look for a new job. If this sounds counter intuitive to you, you’re not alone. But I’ve seen it to be true various times. Remember, a great team is usually a smart team and smart people tend to know when they’re being left in the dark. Seeing upcoming risks makes you feel more secure than complete darkness.

If You Don’t Know, You Guess

I’ve seen this to be true for every size of organization I worked for: If team members don’t know something, they make something up. It might be gossip about a manager’s private life or, worse, about the future of one’s job. If you try to tell your team everything going on there’s less room for guessing. You trust your team to run your business, so please trust them to be able to deal with whatever situation comes up! If a bad rumor is born, it is incredibly hard to kill it again. By being totally open, you can avoid most rumors getting started in the first place.

What to do with Really Confidential Information?

Ok, if you’ve been in a management position for some time you definitely have come across information you shouldn’t disseminate. Sometimes things are extremely complicated and can be easily misunderstood. And once misunderstood, again, you’ve started a hard to kill rumor which might severely harm your business. But, more often than you think it would be better to take the time to explain rather than creating a secret. People sense secrecy and get nervous very easily. Trust your team, educate them to understand a situation fully and get through it together!

Deploying with Capistrano

Dan Ackerson — Sun, 13 Sep 2009 07:38:07 +0000

At the end of April, I wrote about how automatic rsyncs were making my operations life a living hell. Enter Summer, vacation, new developer and here we are mid-September before I finally get around to permanently fixing this problem.

But, I can’t really blame all of it on life – after Matthias gave me a crash course on Capistrano basics, I was a bit daunted by recreating the application directory environment for every deploy. Symlinks, local configuration details, file permissions…oh my!

K.I.S.S.

Last weekend brought some cold rainy weather and I got around to thinking about Cap again. Could I write a simplified Cap file which would just run svn info & svn update?

Here’s my info task:

task :info, :roles => 
  defer { eval(Capistrano::CLI.ui.ask("Server roles ([:role1,:role2,...]) : ")) } do 
    run "svnversion #{deploy_dir}" do |channel, stream, data|
      data = data.chomp     # remove line separators
      result = "r" + data + " " + channel[:host]
      #check for trailing SVN status characters 'M','C',etc.
      # if (data.to_i.to_s == data)
      #  result += " => clean"
      # else
      #   result += " => contaminated"
      # end
      puts result
    end
  end

The CLI.ui.ask allows me to input the desired role group(s) at runtime. I’ve also figured out how to identify if a remote repository’s not 100% clean (which I’ve commented out for now). Here’s the sample output of a run:

danackerson ~/dev$ sudo cap info
  * executing `info'
  * executing "svnversion /data/www/svn/"
Server roles ([:role1,:role2,...]) : :all
  servers: ["db.netdoktor.de", "slave1.netdoktor.de", "slave2.netdoktor.de", "test.netdoktor.de"]
  [db.netdoktor.de] executing command
  [slave1.netdoktor.de] executing command
  [slave2.netdoktor.de] executing command
  [test.netdoktor.de] executing command
r3357 db.netdoktor.de
r3357 slave1.netdoktor.de
r3357 test.netdoktor.de
r3357 slave2.netdoktor.de
  command finished

Nice. Why sudo you may ask? Just because I’ve made deployment drop-dead easy, doesn’t mean I should be able to cold-cock my application. With great power comes great responsibility, and I reassert this by making my cap file executable only by root.

Where the Rubber Meets the Road

Ok, I’d cut my teeth and gained confidence that I could write custom cap tasks to accomplish important goals. Now time for the main event: svn update:

task :update, :roles => 
  defer { eval(Capistrano::CLI.ui.ask("Server roles ([:role1,:role2,...]) : ")) } do 
    rev = ENV['REV'];
    if rev == nil 
      rev = "HEAD"
    end 
    run "svn update -r#{rev} #{deploy_dir}"
end

The REV env variable gives me the ability to specify a specific revision (useful for a rollback as well):

danackerson ~/dev$ sudo cap update REV=3356
  * executing `update'
  * executing "svn update -r3356 /data/www/svn/"
Server roles ([:role1,:role2,...]) : :test
    servers: ["test.netdoktor.de"]
    [test.netdoktor.de] executing command
  ** [out :: test.netdoktor.de] U    /data/www/svn/do_news.gif
  ** [out :: test.netdoktor.de] U    /data/www/svn/do_arzt.gif
  ** [out :: test.netdoktor.de] U    /data/www/svn/do_forum.gif
  ** [out :: test.netdoktor.de] Updated to revision 3356.
    command finished

Leaps and Bounds

After putting the final touches on this Monday morning, I gave it a thorough workout. Not only did I notice a signficant time savings in using cap, but I suddenly had extremely fine deployment possibilities. By the time the day was through, I was running 4 different versions of our codebase among the various servers. This probably sounds horrifying, but it really allowed us to insulate the productive slaves from untested (and unnecessary) code changes. The sudo cap info script gave me a quick overview of exactly what was running where.

Ultimately, it only cost me a few hours to get this setup and I certainly wish I would have done it like this four months ago. How are you using capistrano in your environment? Share with us in the comments!

“Done” is the Wrong Measure of Success

Matthias Marschall — Fri, 11 Sep 2009 04:37:29 +0000

It’s a very important thing for any agile team to find a definition of Done, which fits the expectations and the environment of the current development. For User Stories, I definitely prefer Done = Released as the most helpful metric. Only if a story is really out there serving users can you truly forget about it. Or can you?

The Lifecycle of a Feature

In my introduction to a Kanban Board for Features, I emphasized that a Feature is the unit of work for strategic product management and can be cut into User Stories for a smooth development flow. If one, or even all, of the User Stories which comprise a feature are Done is the feature Done, too?

Unfortunately not. On the contrary: As soon as the feature is finally out there the most important phase starts. Now, it’s time to measure the (hopefully) positive effect of the feature and compare it with the expectations. Does it live up to them? Or not yet?

A Feature is Done, if it is Successful

More often than not you’ll find out, that your feature is nice, but not as good as expected. Now it’s time to take a critical look and do some root cause analysis. Why does it fall short of expectations? Ask five times “why?” to identify things to change. Then, move the feature from Released to In Progress. It has not reached it’s final state: Successful.

Only thinking in User Stories is too short sighted, even if you have the already quite good definition of Done = Released (which too many teams are far from using). To ensure long term success of the software you build, you need to make sure that the features you build are successful. This usually needs a couple of iterations after releasing a feature. Only if you accept that and measure your progress, will your software become truly successful.