Code Monkeyism

Why the Toyota Product Development System is a thing of the past

2009-06-29T18:29:28Z

This post tries to explain why website/app development is Production and should use the Toyota Production System (TPS) and why classic software application development is product development and can use the Toyota Product Development System (TPDS).

photo credit: drewgstephens

Recently there has been a conundrum in parts of the agile and lean community: Obviously the Toyota Product Development System (TPDS) and software development uses a waterfall (or similar spiral) process:

What might surprise some, was that they were using a waterfall model (in Ishii-san’s own words - in reality I think it was more like the spiral model). In spite of that, I had a feeling afterwards that I had just talked to perhaps the most skilled software development managers I have ever met! Does that sound like a paradox? I do not think so.

Toyota today looks so much like a religion that people are willing to suddenly proclaim “waterfall” is a good thing if it’s done by Toyota:

I once said to myself that I did not want to waste my time as a developer on non-agile projects. In the Toyota case, I would certainly make an exception “Toyota is using waterfall!”

Or even, from Mary Poppendieck,

When you are dealing with embedded software in production hardware, a 3 month waterfall is really fast.

And as Sean tries to explain in his comment on another post:

If the projects that Toyota is working on span only a couple of weeks, then waterfall is probably going to work fine. They’ll only develop requirements for a few weeks worth of work, and their in-process code inventory will still be pretty small.

So why should we use TPS and lean in software development, when Toyota does Waterfall and TPDS? We’re confused. But remain calm. There is no conundrum. If you have clear requirements, a limited project, a fixed feature set and a fixed deadline, use waterfall. This is what Toyota probably experiences. Most people in the agile community will tell you it’s only best to use agile and go lean when you have unclear or changing requirements, changing feature priorities and work in a flow model instead of a project.

After reading the Toyota Way book, and twittering about TPS, I got the following reply:

@flowchainsensei: “Might like to take a look at TPDS too (more direct relevance to software development) Kennedy, Ward, etc.”

So TPS/lean isn’t for software development? After some more thinking, whether software development is more like Toyota Product Development (TPDS) or more like production (TPS), I had an insight. For your web app development there are 2 phases:

Phase 1: Version 1.0 done with product development (PD)
Phase 2: After that companies shift to a production (P) model

Note: You should keep the PD phase as short as possible.

Websites and web applications are different than your usual software application development. Website/app development has many more direct stakeholders: Reporting, Controlling, Marketing, Sales, Product Development, Customer Support, Backend Services and more. Contrary to classic software apps, they all have direct involvement into features. In webSite/app development, those stakeholders write their own (!) stories and directly contribute features. On the other hand in software app development, features are mainly developed by a product development department with indirect input from other deparments.

Therefore software app development is much more product development (PD) heavy, while website/app development is much more production (P) heavy. With more direct involvement, smaller stories and changing requirements, more and more development will move into production style in the future.

The same will happen inside Toyota. My prediction: Over time at Toyota TPDS and TPS will merge, when every car is build-to-order, with it’s own design, with custom colors, each with a one-in-a-kind motor etc.

This is what Scrum tries to accomplish, merging production and product development. An enviroment in which web companies are already. Each story to be developed is the same - consisting of web forms, services, database code, reporting etc., but at the same time every story is different. Think of stories as being all the same, e.g. web-service-db stories, but highly customized and build-to-order for each customer (marketing, product development, sales). Then it becomes clear that development in web companies is production, not product development.

This merging is also the area where Scrum struggles: How to put design, architecture work, technical debt etc into development, and Scrum still hasn’t settled for the one best practice.

Hope this helps clear up the “Conundrum”, which isn’t one.

New Version of my Simple Kanban Board Application

2009-06-15T10:09:15Z

Over the weekend I’ve worked on my Simple-Kanban application. Simple Kanban is a small Kanban board application in one Html file. New features are a data mode that displays the data in raw format for easier cut & paste and drag & drop support for moving stories around.

There is a website now! I’ve added a small website at http://www.simple-kanban.com, where you can find new versions. I’ve also created a GitHub repository for Simple Kanban, where I’m planning to post the code (funny, the code is already open source as part of the Html file :-)

Much fun with using Simple-Kanban in your company, think lean!

Micro Book Review: Agile Retrospectives, making good teams great

2009-06-11T08:30:43Z

Title: Agile Retrospectives, making good teams great
Author: Esther Derby / Diana Larsen
Pages: 165

What the book is about
The book is about leading retrospectives. Retrospectives came into fashion with agile software development, especially Scrum has retrospectives every sprint. The beginning of the book motivates retrospectives, explains them, shows how to lead them and establishes the concept of phases: set the stage, gather data, generate insights, decide what to do, close the retrospective. The second part explains detailed several activities you can use in each phase. A description is structured like the famous design pattern book: Purpose, Time needed, Description, Steps, Materials and Preperation and Examples.

What I’ve learned from the book

Retrospectives have several phases: set the stage, gather data, generate insights, decide what to do, close the retrospective
There are many activities for retrospectives, you should change activities from time to time
Use an activity like”appraisals” for closing a retrospective

Should you buy this book?
Yes, highly recommended

Who should buy the book

Every ScrumMaster or Iteration Manager
Everyone who leads retrospectives

Notes
Book bought by myself due to a mentioning on Twitter.

I’ve chosen the micro review format because it lends itself to be used as a future micro format and I like short reviews myself. You can read the table of contents elsewhere, I don’t like it when reviews iterate the content.

Kanban Board Application in One Html File

2009-06-09T10:29:20Z

For some years I’ve been interested in lean software development and how to reduce waste. While introducing lean practices, I’ve needed a small, simple Kanban Board application. Thought I’d write one.

You can download a very first alpha here or see it in action here.

Suprised? The application is just one HTML file, no installation needed, no Java, Ruby or PHP. One HTML file, no other dependencies.

How to use the application?

Edit the source of the HTML file with an editor of your choice, preferably one which knows HTML. You will find a list of stories. The example contains those:

T_Q,S18,Checkout optimize
DE,S2,Build old shop
DE,S4,Rebuild with SOAP
DE_Q,S10,Rebuild with REST
P,S17,Do something with OpenID
D,S3,Make application faster
D_Q,S7,Credit Card Payment
DE,S13,Build something astonishing
P_Q,S17,Fix YSlow
R,S39,Google Page Speed fix

There are three columns for stories. The first column contains the state the story is in, the second contains an identifier for your story and the last column the name of the story. You can edit the columns, change states, change names, remove and add stories. You can also export form Excel to csv, then cut&paste into the application source.

The available states are described next in the file:

D,Design
DE,Development
T,Test
R,Release

They need to be available for the stories and match the states of the stories. You can add, remove or change states. Every state has a “sub-state” as a ready queue. For example the ready queue state for Design “D” is “D_Q”, for Test “T” it is “T_Q”. You do not need to describe the ready states, they are automatically created and a ready queue is shown in front of every state. For example “Test Ready” is shown left to “Test”, if there are stories in that particular state.

Customize the colors

The colors of each state are defined in a CSS block.

.box_P { background-color: #FFFF00  ; color: #000000;}
.box_P_Q { background-color: #F0F0F0; color: #606060;}

Feel free to change them to your taste. “.box_P” is for the “P” state box, “.box_P_Q” for the corresponding ready queue.

The main use case for this application is to inform a company about the stories which are in development and in which state they are. It’s an ideal information radiator. I plan to use it on a huge screen.

Future
I’m thinking about a CouchDB storage implementation for storing data and application logic. Or storing data in the file, with drag and drop, inlining Jquery, editing and storing like TiddlyWiki does.

Future features? Add WIP limits, add “From here” signs to display cycle time until live.

Have fun with this One-File-HTML-application. Tip: you can easily mail it around, no install needed.

Top 5 Things to Know About Constructors in Scala

2009-04-23T04:46:16Z

I’ve been toying with Scala for some months now, one thing I’ve struggled with coming from Java are constructors in Scala. They are comparable to Java, but the syntax is different.

credit: .Paolo.

To help get you going faster in Scala, the top 5 things to know about constructors. Here we go:

How to do constructors with a parameter
```
public class Foo() {
   public Bar bar;

   public Foo(Bar bar) {
       this.bar = bar;
   }
}
```
Looks in Scala like this:
```
class Foo(val bar:Bar)
```
In this case val creates an immutable final public field, using var would create a mutable public field.
How to have private fields
```
public class Foo() {
   private final Bar bar;

   public Foo(Bar bar) {
       this.bar = bar;
   }
}
```
Looks in Scala like this:
```
class Foo(private val bar: Bar)
```
Update: Changed due to comments. Thanks for the commentors to point this out

Private fields are not as necessary as in Java, you can have public fields for attributes and change them to a method (def) later - without changing your clients.

How to use super() ?

public class Foo() extends SuperFoo {
   public Foo(Bar bar) {
      super(bar);
   }
}

Looks in Scala like this:

class Foo(bar:Bar) extends SuperFoo(bar)

How to have more than one constructor?

public class Foo {
    public Bar bar;

    public Foo() {
       this(new Bar());
    }

    public Foo(Bar bar) {
	   this. bar = bar;
    }
}

Looks in Scala like this:

class Foo(val bar:Bar) {
  def this() = this(new Bar)
}

Secondary constructors like this() need to delegate to another constructor to work (Thanks @Synesso).

How to get bean style setters and getters?
```
 public class Foo() {
   private Bar bar;

   public Foo(Bar bar) {
       this.bar = bar;
   }

   public Bar getBar() {
      return bar;
   }
   public void setBar(Bar bar) {
      this.bar = bar;
   }
}
```
Looks in Scala like this:
```
class Foo(@BeanProperty var bar:Bar)
```
The attribute bar will still be public, which is not a big issue (see above) in Scala. But @BeanProperty helps when working with Java libraries and you need Bean conventions for the libraries to work.

To add getBar and setBar but not a public field you need to:
```
class Foo(aBar:Bar) {
    @BeanProperty
    private var bar = aBar
}
```

Update: Changed to var, thanks to @eivindw.

Hope this helps you, if you have something to add, leave a comment. Should you struggle with the limited this() syntax (only one expression), then perhaps your constructors are doing too much. Consider the factory or better builder pattern instead.

Additional tip: use @Serializable to make your Scala classes serializable.

Nice coding in #Scala.

7 More Good Tips on Logging

2009-02-24T06:06:39Z

Logging in web applications is important - to know what’s going on, for performance tuning and incident analyis. This is my second post about logging. The first post “7 Good Rules to Log Exceptions” was specific to logging exceptions, ths is about logging in general. What makes your logs more useful to you?

credit: schoschie

1. No debugging logs in production

I have seen time and again that debug logging is enabled in production. This can be intentional or happening by some developers who accidently checked in a debugging logging configuration. Enabled debugging slows down your application remarkedly and makes it impossible to read production logs due to noise. Make sure during deployments - best with some scripts - that debugging level logging is disabled during production.

2. Look through your logs

Some companies have good logging in their production system, but do not look into their logs. Look into your logs, discover issues (bugs, performance, memory) with your application and fix them. Essentially your logs should be without known errors.

3. Log to the correct log level

Developers who write logging code often don’t know which log level to use. Have a document ready which explains which log level developers should use. For example SEVERE should only be used for technical problems which need immediate action. ERROR should be used for errors that someone needs to look into and fix, like not getting a databasde connection, low resources or failing integration points. This is specific to your company and application.

4. Do not log locally

If your server has major problems like resource troubles, it’s often impossible to log in. Therefor you can’t get to your logs finding the problem. Logs should be written to a network drive, copied over to another host or written to the network e.g. with Syslogd. A nice solution is to use the Spread Toolkit to write to a network group with multicasting. This also enables easy monitoring (see “Scalable Internet Architectures”).

5. Monitor your logs

Similar to “Look into you logs”, you should setup a monitoring solution which looks at SEVERE entries, ERROR entries, exceptions and other conditions in you logs. With Spread it’s easy to add monitors. A good idea is also to classify and count exceptions, then do something about the severe and most frequent ones.

6. Use a human readable format

Developers often don’t think about the output they produce. This leads to hard to read log files. “Release It!” has an example for readable output:

[8/14/06 8:22:14:653 CDT] 0000a SSLComponent I CWPKI00001I: SSL service not available
[8/14/06 8:22:14:813 CDT] 0000a WSKeyStore   W CWPKI0041W: One or more key

This row oriented format makes it easier to fast scan logs. Compare this to the your logs.

7. Use error codes in logging

Each cause which leads to log output should have a unique error code. Without a unique error code it’s hard to find the cause in your source code. Error codes make it also much easier to count and classify log statements and enables communications between development and operations.

Want to know more? Books with good sections on web site logging are “Release It!” by Michael T. Nygard (really excellent book!) and “Scalable Internet Architectures” by Theo Schlossnagle.

Better Null Handling Strategies for Java

2009-02-04T10:25:32Z

Uploaded a presentation on “Better Null Handling Strategies for Java” to SlideShare. Enjoy.

View more presentations from Stephan Schmidt. (tags: java null)

Scrum is not about engineering practices

2009-02-03T05:59:16Z

Scrum is not about engineering practices - which is a good thing. Martin Fowler writes:

They adopt the Scrum practices, and maybe even the principles
After a while progress is slow because the code base is a mess

and connects Scrum failure to missing engineering practices. This completely misses the point. Scrum is not about engineering practices, it’s about management.

credit: elliottcable

Engineering practices are a responsibility of the team. Scrum creates self organized teams. They organize themselves, they organize their quality. They organize their tools. They organize their engineering practices (TDD, pair programming). Why? Because they are responsible for delivering. No one else is.

Craftsmanship and level of done

Often quality and craftsmanship are organized by a level of done agreement in the team which describes what a team considers when code is done. This can include

Documentation / JavaDoc
Functional tests
Unit tests
Bug free
Refactored, maintainable code
Reviewed code

What about technical dept? Technical dept is not professional. Seeing an ice berg and keeping a crash course is not professional. With the right level of done there will be no technical dept.

Scrum helps with good engineering practices

Concerning Marting Fowlers comments Dean Wampler twitters:

His comments match my experience at client sites. Teams using Scrum w/out the XP prog. practices don’t work long-term.

Not very insightful it says: developers who do not use professional practices will fail. Of course they fail, but they fail independently of the process you use.

The main difference to classic project management is that developers have the freedom to define their level of done and the amount of work they do. Developers define what stories they work on in each sprint, management doesn’t set a (unrealistic) finishing date as often happens in classical software projects.

Is there too much quality? Doesn’t the product owner care if the team takes “too much” time for quality? The product owner is entitled to two things:

Story estimates
Shippable products
Professional developers

The product owner is not entitled to speed. Scrum sets resources and time in the Iron Triangle and let’s developers decide about scope. Speed is scope / time and is an output variable of the team, not an input variable. If she thinks the velocity of the team is too low, talk to the ScrumMaster and remove impediments. But engineering practices should never be dropped. Other crafts will never drop theirs. Ask a doctor to drop sterilizing to gain speed. Ask a banker to drop double-entry bookkeeping. They won’t. Neither should developers drop theirs [1].

[1] This could mean developers need to be trained to be professionals. But a company needs to teach professionalism to developers independently of what process it uses. If you do not have skilled, professional developers you’re doomed and no process will help you.

Micro Book Review: The Definitive Guide to Terracotta

2009-01-28T09:57:09Z

Title: The Definitive Guide to Terracotta: Cluster the JVM for Spring, Hibernate and POJO Scalability
Author: Ari Zilka (Terracotta CTO) and his team
Pages: 368

What the book is about
The books is an introduction about Terracotta which helps you distribute -transparently- the Java Virtual Machine memory over several JVMs. The main part of “The Definitive Guide to Terracotta” focuses on use cases. Those are quite good motivated, explained and described with many examples and working code.

What I’ve learned from the book

What Terracotta and virtual heaps are
How to use TC with ehCache, Hibernate and for session clustering
Dropping in ready-to use functionality with TC integration modules (TIM) is easy

What I didn’t like

Chapter about optimizations but not extensive enough and not enough information about deployments and deployment scenarios.

Should you buy this book?
Yes, highly recommended, it’s written by the Terracotta guys, you can’t get better and more accurate information.

Who should buy the book

Every developer or architect who wants to use or evaluate Terracotta

Notes
Book kindly supplied by the publisher. This is a short version of my former review

I’ve chosen the micro review format because it lends itself to be used as a future micor format and I like short reviews myself. You can read the table of contents elsewhere, I don’t like it when reviews iterate the content.

What do you think about this short review style?

ScrumMaster and ZenMaster: The joke of certification

2009-01-23T10:24:32Z

Many people object to ScrumMaster certifications:

It’s a money making machine
Scrum Masters do not learn anything during classes
The certification is nothing worth - because nothing is certified

I have been a Certified ScrumMaster (CSM) and a Scrum practioner for some years. People who object to the certification do see it from the wrong angle. You need to understand Zen to understand the goodness in CSMs.

photo credit: darkpatator

Certification is a Zen joke, because the role of a ScrumMaster cannot be certified. It’s not about knowing some technical questions. What should a trainer certify in such a class? That you can lead an agile Scrum team as a ScrumMaster? No one can certify the fact that you’re a leader, catalyst and enabler. You either are or you aren’t. Zen masters (ha, another master without a master!) would laugh at the fun in the ScrumMaster certification. They laugh about the idea of certifying enlightenment.

Scrum without ScrumMasters

As another parallel, both in Scrum and in Zen, masters are only enablers. They are not needed after the act of enabling Zen/Scrum. My Scrum trainer told me, the goal of a ScrumMaster is to make himself obsolete. There is a Zen koan which goes like this:

If you meet the Buddha, kill him.
— Linji

If you see a ScrumMaster, kill him. Zen tells you:

If you are thinking about Buddha, this is thinking and delusion, not awakening. One must destroy preconceptions of the Buddha. Zen master Shunryu Suzuki wrote in Zen Mind, Beginner’s Mind during an introduction to Zazen, “Kill the Buddha if the Buddha exists somewhere else. Kill the Buddha, because you should resume your own Buddha nature.”

If you think the ScrumMaster is Scrum, you’re delusioning yourself. In Scrum the product owner and the scrum team can, and should from my view, act by themselves, without the need of a ScrumMaster. The ScrumMaster helps them achieve their Scrum. Helps them overcoming initial obstacles in their productivity.

Kick your ScrumMaster
If the ScrumMaster is not good enough for them, certification and coaching inside the company hasn’t helped, the Scrum team has always the right to kick their SM if he isn’t good enough for them. And they should do so. If in Zen a master isn’t good, pupils will just leave him. This might lead to problems within the organization, especially if the ScrumMaster is their boss, but that should be the problem of the organization, not a team problem.

Practitioner certification

There are many more certifications from the Scrum alliance. If you dig deeper, the real fun part is that CSM doesn’t mean anything, practitioner means much more:

The practitioner level of certification (CSP) is only offered to those CSMs who have hands-on experience using Scrum. Applicants must complete an extensive questionnaire with probing questions that focus on applicants’ real-world experience using Scrum on software development projects. Their application is reviewed for answers demonstrating competence and comprehension of principles that can only result from hands-on work. The applicant may be questioned to determine eligibility. To maintain CSP status, you must submit a new application every two years.

Is the certification any use?

Yes. The Certified Scrum Master training has several merits:

Calling the Scrum training “Certified” guaranties the quality of the trainer
It motivates the Scrum master to think in Scrum
If managers take part, it helps the organisation adopt a “we can do it” view about Scrum
Certification (CSM) seems to be one of the main reasons for Scrum success in the enterprise. The certification makes Scrum compatible for managment.

The view about acceptance is shared by Peter Stevens:

It is also about branding, and has been quite successful. The acceptance of the CSM program is high (especially from corporate customers, and this is where the money is). I believe the CSM program is an important reason why Scrum is better accepted than say, XP, in corporate management circles.

Scrum is successful. I’ve seen it help development departments gain productivity. If you do not scrum yet, go for it.