From the Agile Transformation Trenches

How agility drives product development - an inside view

2009-07-20T22:15:03+01:00

Greetings. In my last post - part two of a three-part series on Agile Testing - I talked about the importance of test automation in Agile (Agile Testing: what does it take? Pt. 2). Before finishing that thread, I would like to expand on this topic a bit more. You see, today is a milestone for me. We are releasing a significant update to my product line - a suite of test automation products - and I think it is a great moment to share with you what makes me so excited about the work I do. First of all, I consider myself fortunate to work with a great team. Collectively, we are responsible for defining the strategy of Borland's Lifecycle Quality Management (LQM) products. In the last couple of years the general direction of these products has been greatly influenced by Borland's own transition from a traditional, waterfall-based organization to one that lives and breathes Agile.

During our transition we not only realized that we needed to change many of the long- established processes and cultural norms, we also realized that we needed to enhance our products in order to better support Agile development scenarios in addition to continuing to improve our support for the core needs of all QA teams. Also, as an ISV we obviously have the same need for quality as any of our customers and, consequently, we not only use our own products to test our software but leverage our many years of experience in the testing domain to continuously improve our products.

As you may have seen, Borland today announced the latest versions of the Silk products which represent a significant step in supporting 21st century testing. The new releases not only provide true test automation for state-of-the-art applications, but they now offer some really exciting enhancements to better support Agile teams. Now I can already hear many of you moan about how I am trying to sell the Borland products. But that part of the post is done. The link above will let you take a closer look at the products if you are interested. My real intention for this post is to share the experiences that were the genesis for these products and through that sharing, hopefully, show you why am so excited about them.

There is no hiding in Agile

As we adopted more and more Agile practices in the Linz office, we noticed some issues surfacing and then becoming more and more pressing. Amongst these was our need for better support of Agile workflows in our Silk products. For example, we quickly learned that Agile teams work in a pull model as opposed to the traditional method of assigning work items. This was especially relevant from a test management perspective. As a consequence of our own teams' experiences, we realized we needed to provide much more collaborative views that allowed the team members to see what was going on so that they could select testing-related tasks from a list of available work items that had not yet been picked up by another team member, or have easy and quick access to related information. These requirements led to a number of enhancements in SilkCentral Test Manager such as Agile project templates, a new Web-based manual testing interface and an integration to Version One to support additional Agile test planning tools. These enhancements quickly proved to be an enormous help in our teams' daily work process by supporting self-organization, providing the necessary visibility into the many activities, and making collaboration much easier and efficient.

Another important need we quickly identified was the ability to leverage existing tests as well as the ability to easily integrate new tests, e.g. unit tests. It quickly became clear that it was a major hurdle for the Agile teams to have to maintain test plans both externally (for example as part of a unit testing framework) as well as in the test management tool. This gave us the idea for external test package support, which now allows users to just point to the external test plan and execute all the tests that are part of it. Gone are the days where we needed to manually create test stubs in the test management tool and add the necessary details in a subsequent manual step which, of course, is cumbersome when dealing with a large number of tests. Now it is as simple as pointing to a suite of existing unit or FitNesse test plans and executing them directly from SilkCentral Test Manager. And voila! All your results get automatically pulled in!

Finally, probably the most important requirement we uncovered, was the need to automate increasingly more tests. Anyone who has built test automation frameworks in the past knows that this is a very challenging task. The brittleness of test scripts combined with the potentially fast- changing GUI of an application are just two of the factors that have made the creation of a robust test automation framework extremely difficult. Our teams quickly realized they needed to do something about it.

Addressing the challenges

We immediately identified two closely related areas of improvement that we would have to address. First of all, from the perspective of a tester we looked for ways to improve the testability of our applications. Introducing unique identifiers for each of the objects in the products we develop was a significant step to help improve testability of the applications as well as the stability of test scripts. This also quickly lead to an improved understanding from the product development point of view about how we needed to enhance our test tools' capabilities to best leverage these kinds of testability enhancements of an application. In order to be able to build solid test scripts we needed to have stable and consistent object recognition. So the first obvious improvement we set out to make to the Silk suite was to ensure that the products provided a way to consistently find simple locators which were unique for each control - the same for each call, the same for each build, and the same for each instantiation of the application. Ideally, they would be readable and follow standard naming conventions well. With that in mind we enhanced all of our products so that they would intelligently optimize their object recognition strategies accordingly for testability.

Another major challenge our teams faced was around testing AJAX and other Web 2.0 applications. Due to their asynchronous nature they cannot be tested reliably with a traditional serialized test logic approach. It became obvious that synchronization needed to be done in the test tool rather than at the script level. Consequently, we enhanced SilkTest with everything that was needed to remove the necessity for the tester to try and mimic the (nondeterministic) nature of asynchronous applications allowing for much leaner test script code.

Finally, with all of our teams moving to two week iterations, speed became of paramount importance. The teams needed to be able to run their ever-increasing regression sets in much smaller time frames. There was an increased need to test more - and more often. We quickly realized that we needed to directly leverage RIA technologies to speed up replay of test scripts instead of leveraging an on-the-surface approach like using Javascript engine-injection for browsers. This approach, used by many other testing tools, just didn't work for our teams. The security restrictions, incredibly slow replay speed, and inability to handle actions that happen outside the browser's DOM (like Windows message boxes that come up) are just some of the problems we ran into with other tools. So what do you do if you are a test tool vendor? You build what you need!

These are just a couple of the challenges that our teams experienced that influenced the direction of the product line. One of the key advantages you have when you get immediate feedback through internal usage, and then have the opportunity to and refine and improve products on a sprint-by-sprint basis is that customers (and we are one of our customers) reap the benefits very quickly.

Bye for now.

What Good Are Pigs And Chickens?

2009-07-07T20:13:51+01:00

Recently I received an email commenting on David Anderson's "Where Everyone's a Pig" article. The author agreed to let me share part of their message along with my reply. They said, in part:

To a significant amount Scrum is intended to put a protective shell around a group of persons to ensure that the vulnerable process of self organization is not disrupted from outside forces. The metaphor of chickens and pigs is intended to underline that notion. I don’t see chickens and pigs as opponents at all.

To which I replied:

As with many agile practices, the pig/chicken distinction must be adapted to fit the organizational context and maturity of the team. In early stages of agile adoption this "protective shell", as you call it, is a defense mechanism intended to help the team counteract the larger organization's resistance to, or even hostility toward, agility. This is often based on fear and misunderstanding, but it can still be a real threat.

David's point is that a mature agile organization does not have that problem. He suggests that one way to remove the opposition to agile methods is to include the rest of the organization in your agile process. Viewing participants as pigs/chickens maintains an us/them attitude and therefore can undermine full organization-wide collaboration. Although it may be useful to provide some degree of safety to new agile teams, the need to "keep management at arm's length" is a symptom of a larger organizational dysfunction.

Ultimately, if anyone, inside the team or outside the team, is an impediment to timely delivery of value to the customer, we need to ask why they are doing that and what goal (presumably more important to them) is being served by their actions. Problem-solving based on whole-system thinking is not served by the pig/chicken distinction.

What do you think? Is there value in the pig/chicken metaphor? Does it encourage dysfunction?

Agile Testing: what does it take? Pt. 2

2009-06-19T16:35:12+01:00

Hello again. This is the second in a three-part series on the most critical issues in Agile testing. In my first post, I talked about how people are affected by a change to the Agile paradigm focusing on team integration and the required skill sets (Agile Testing: what does it take? Pt. 1). Now let's look at the role of testing tools, and more specifically the importance of test automation.

Striving for a high percentage of test automation is indeed another important aspect of Agile testing. In order to keep up - or accelerate -- the velocity of Agile teams, test automation becomes an absolute must. I'd even say it's impossible to keep up with quality in an Agile environment without test automation.

In Agile environments there is a serious need for speed! The best way to accelerate the Code-and-Test Process is with fast, automated test scripts. Automated testing enables you to achieve "Simplicity - the art of maximizing the amount of work not done," which is considered essential in an Agile environment.

Once tests no longer have to be performed manually, teams can test much more rapidly and achieve more in less time. Automation also enables testers to create simple, reusable scripts, which they can deploy to save time and increase the consistency of testing across similar user stories, or requirements within and across projects. And, finally, automated testing allows "sponsors, developers, and users … to maintain a constant pace, indefinitely." It significantly lightens the workload of testers and eliminates the need for late-night and weekend testing marathons that can that burn teams out.

In addition to automating the more common areas of unit testing and functional testing, automated performance testing gives teams the ability to address fundamental issues in the architecture of their application early on when it is easy to make changes.

So, I've covered two key concepts: integrating and automating. In the last post in this series, I will expand on this in the context of the "test early and often principle," which not only works well in Agile environments but is a key contributing factor to keeping quality levels high.

Bye for now

Culture Change with Pigs, not Chickens

2009-06-16T18:24:44+01:00

Last month David Anderson wrote a two-part article on why agile transformation initiatives fail. David’s suggestion to concentrate on cultural change reminded me of one of my favorite bits on process initiatives. From Peter Coad’s book, Java Modeling in Color with UML, it starts with ‘We think most process initiatives are silly’. I particularly like the paragraph that says, “No amount of process specification will make up for bad people. Far better: Staff your project with good people, do whatever it takes to keep them happy, and use simple, well-bounded processes to guide them. ” In fact, I would go so far as to say process over-specification gives non-performing people something to hide behind. Agile processes strip away that cover exposing weaknesses in the ways individuals and teams work and communicate.

However, exposing dysfunctional teams and working practices is not the same as fixing them. In part one of his article, David talks about the problems with management-imposed change that does not win the hearts and minds of those on which it is being imposed. I certainly recognize these problems and not just with agile transformations, but with any major process improvement initiative including standardizing on a new toolset.

In fact, there are some facets of such programs that I have come to recognize as patterns:

The ‘Sheep-dip training’ pattern:

Most process improvement teams realize that staff will need training in the new process or tool set. However, with one eye on the budget, they opt for a training program that introduces everyone to basic the mechanics of the process or toolset.

Unfortunately, this is a little like giving soldiers basic training in a new type of warfare or equipment and sending them into battle without trained NCO’s and field officers.
The ‘If we build it, they will come’ pattern:

Some people seem to believe that if the process improvement team create the perfect description of the new way of working on a intranet site (or worse, in a single large MS Word or PDF document), everyone will read it and follow it. I have even known one client that, on deciding to adopt agile, disappeared for a couple of months to write a large document to describe how to do agile in his department!!!

This is like HQ addressing the lack of trained NCO’s and field officers by sending more detailed written orders to soldiers already under fire.
The ‘Centralized coaching team’ pattern:

A team of experts, possibly external consultants, is set up to visit, assess and advise teams on issues they are encountering with the new way of working.

To continue the military metaphor, at regular points in the battle, officers from HQ appear to advise briefly on the new tactics and then hurry back to the safety of the HQ.
The ‘The disappearing external consultant/coach’pattern

End users of a new tool or process need expert guidance to get over the initial learning curve and to apply it within their particular project context. The easy solution is to parachute external expert consultants and coaches into these teams. Unfortunately, again the budget often limits this to part-time or a few weeks only.

Having your process documented on a website is not wrong. Giving users basic training is not wrong. Providing access to experts and coaching is not wrong … but it is not enough. The fundamental problem with the patterns above that turns them into anti-patterns, is one of pigs and chickens.

Note: In the UK, we talk about ‘having skin in the game’. In other words, having grazes and bruises from being an actual player rather than supporting or coaching from the sidelines. It seems a more appropriate metaphor for a process called Scrum but the pigs and chickens thing has stuck. Pigs have 'skin in the game'. Chickens are well-intentioned, interested spectators willing the team on but not able to contribute in any concrete way to the final result.

In the patterns above, chickens, not pigs, are providing the expertise whether written or face-to-face.

Personally, I believe the key to effectively achieving a change in culture and a sustainable transformation is to:

Identify the technical leaders within projects; those that are “self-driven to produce quality results on time … combine technical ability with enough people skills …are trusted and respected by both their managers and fellow developers, are determined to make the team succeed, and usually live the work.” (Chief programmers, Chapter 2: A Practical Guide to FDD).
Sell them the vision: if you cannot sell these people on the benefits to them, their colleagues and the organization of the new way of working then something is wrong. Either these people are not the real technical leads, you are not communicating the advantages of the change well enough, or the new way of working is actually a pile of ‘emperors new clothes’.
Provide in-depth training and on-going coaching. It is better to have a single lead person trained in-depth who can coach his teammates through the basics than to have the whole team trained on the basics with no-one on the team to turn to when the basics are not enough. Providing initial training is simply not enough. When the pressure is on, the temptation to return to previous ways of doing things is often too strong to resist. If technical leads are to work for you, they need on-going support and coaching, and a means by which they can support each other. Good external coaches (expert chickens?) can help here.
Let the technical leads continue working on their projects. Some fail at the final hurdle by doing 1-3 above and then assigning or scheduling the technical leads to coach other projects, effectively turning them from pigs into chickens.

To summarize, if you can produce a change in the behavior of the lead pigs, the other pigs will, by definition, follow. However, pigs will not follow chickens for long because chickens are simply not pigs.

Out for summer, out 'til fall; We might not go back at all

2009-06-16T15:50:28+01:00

Photo by sunchild_dd Watching my kids rejoice in their summer vacation I'm struck by the irony of our situations: as children we all disliked school; as adults we crave new learning opportunities. In the workforce, this usually means either conventions or training classes, but not always. Books remain a low-cost (and my favorite!) way of obtaining information, especially when coupled with a discussion group. Blogs, listservs, and podcasts are all excellent methods of finding new viewpoints and new voices in the community. A more social way of getting exposed to new ideas is to join a local organization such as a user group or technology incubator. Attending a BarCamp gathering is another great community learning opportunity. After all, meeting new people can be just as rewarding as understanding a new concept!

While my boys are excited they don't have any more homework for a few months, I look forward to my next opportunity to learn. What do you do to acquire new knowledge?

Post title is a lyric from Schools Out by Alice Cooper.

Agile is Fragile

2009-06-05T16:42:28+01:00

I recently had the pleasure of seeing David Douglas and Raj Vaidyanathan presenting "Integrating into the whole...embedding agile processes into all aspects of S1's business model" at Agile Austin. During that presentation, David was lamenting the fact that agile is fragile. He related several experiences where, despite visible indicators of success, agile adoptions had ultimately failed to be sustainable. This lack-of-sustainability pattern is certainly a familiar one, and we should look hard to find root causes so we can address them.

I think the lack of sustainable agility can be traced—in large part—to a loss of trust. "The first thing to build is TRUST!", as Brad Appleton says. Trust is critical to the proper functioning of an agile organization. Where there is less trust, we see more dysfunction. On the surface it can appear that everything is going well. We all have a lot of real-world experience with putting on a façade of trust. Ultimately the actions of the team will reveal how much, or how little, trust actually exists.

Trust is slow to build, and quick to destroy. Since trust is a foundational element of agility, the fragile nature of trust makes agile fragile. However, there is hope here as well. Trust relationships exhibit a plateau behavior. Once established, they achieve a degree of stability that allows trust to survive sort-term attacks. A slow erosion of trust, or a catastrophic trust-breaking event, will still lead to dysfunction. If you are feeling a loss of agility, it may be a good time to check on the level of trust in your organization.

Agile Scope: A picture and a few words

2009-06-02T17:04:46+01:00

A couple of weeks ago, I was peer reviewing some work by a colleague. I was struggling to communicate what was and what was not covered by the popular agile processes - what I call the scope of the process. For example, Scrum and eXtreme Programming essentially focus on a single small, cross-functional, development team. At a fundamental level, they do not cover larger teams or projects or organizations consisting of many small dependent teams. They say little about how customers or product owners decide on the business value and priority of the user stories or backlog items they create. Neither do they say much about working with a formal testing organization.

While there are plenty of suggestions and some popular approaches to extending Scrum and eXtreme Programming to cover multiple dependent teams, they are just that; extensions to the original core set of rules or practices.

Anyway rather than try and explain this by waving my hands around I resorted to drawing a picture to communicate what was in my head. Although very simplistic, it seemed to help so I thought I'd share it. If I have done it correctly, you should be able to click on the image below to see it full-size (legibly in other words) in a separate, pop-up window.

The circles represent perpetual activities all running concurrently taking input off lists (or queues, backlogs, or whatever) and generating results on subsequent lists/queues/backlogs.

The green dashed box represents the scope of Scrum/XP and supporting tools like Borland TeamFocus today; sets of small independent teams working from their own backlogs.

The red dashed line represents the space where more and more organizations that have been using agile successfully for a while seem to be focusing. In these organizations individual development teams are working well. The result is that inefficiencies in working practices around the individual teams are starting to show up. This enlarged scope is closer to that covered by an approach like Feature-Driven Development (FDD) with broader demand management aspects informing the individual team product backlogs and collaboration across teams around an overall domain and technical architecture where appropriate. It is the space I see the combination of tools like Borland Team Demand and Team Focus being applicable, for example. Of course, we can't forget Borland Togetherfor recording and communicating the overall architecture as it emerges from collaborative design sessions.

Outside of the red dashed line are the activities that trigger new projects or changes to existing systems, and my colleague's interesting contention that this should all be inspired from a business process improvement perspective. He asserts that all IT development work should be about improving one or more steps in a process somewhere and if we understand our processes or our customer's business processes well enough we can use that knowledge to identify and prioritize development work. For example, Borland as a software tools vendor needs to understand the end-to-end processes involved with developing and delivering software. Identifying pain points, bottlenecks, opportunities for increased efficiency, etc, in those processes enables us to deliver genuinely useful and easy-to-use products.

What the picture does not show explicitly is the current interest in more sophisticated agile testing with tools like the Borland Silk suite. The idea is included implicitly in the Design, Build and VERIFY activities.

Also not shown are the measurement and analytic aspects that I view as another dimension to the picture entirely.

All suggestions for a better picture would be gratefully received.

He planed and worked that wood until he made silver from the yellow pine

2009-05-26T12:29:46+01:00

Craftsmanship has always been widely admired and appreciated. Anyone can pick up a paintbrush, but few compare to Monet; a wooden chair from Walmart will provide a place to sit, but the beauty of handcrafted custom furniture is undeniable. There is a movement in software to try and capture some of this magic, but with the large amount of truly terrible programs and outright project failures this crusade clearly hasn't yet taken hold. Given that, is there anything more satisfying that completing a project? Whether it is woodworking or software, that feeling of a job well done is beyond compare.

One of my teams recently got to experience this euphoria: Borland TeamDefine 2009 was just released to the market for the first time. TeamDefine is simulation software that helps people flesh out requirements early in the cycle, eliminating time-wasting rework. Agile is all about short feedback loops, so a with tool like this which can deliver both rapid prototypes and gather comments from a wide variety of sources, those feedback loops can get even shorter. Most agile stories are a collection of words with an occasional screen shot; if a picture is worth a thousand words, a simulation is worth a thousand pictures! The earlier a team knows it is building what the customer really wants, the more likely they are to build well-crafted software.

Our team is now using our own product to explore stories, which is very cool. I've been on a lot of different software troupes, but the number that actually got to use what they built as a part of their job is very small. We have had a lot of fun building TeamDefine, and it has developed into a product of which we are unabashedly proud (and are having a lot of fun using). Check it out for yourself by watching the (funny) video below and downloading a free trial. Play with it and let me know what you think!

Post title is a lyric from Walnut Street by Walt Wilkins.

Agile Testing: what does it take? Pt. 1

2009-04-30T19:53:03+01:00

Greetings. My name is Joachim Herschmann, and I am a Director of Product Management here at Borland located in Borland’s Linz office. I have over 15 years of experience in the software development and testing disciplines, and I am a certified ScrumMaster. I joined Borland in April 2006 through the acquisition of Segue, a leading testing solutions provider, and I am now responsible for Borland’s Lifecycle Quality Management (LQM) (aka testing) products.

When you are developing testing products you are in an interesting position. Not only are you building and evolving the products, but you use them internally on a daily basis as part of your development work! With Borland being in the midst of an Agile transformation, adapting testing to the Agile paradigm has become very important to us. So in my first few posts, I’d love to talk about some of the issues I find interesting and critical to evolving testing and QA for an Agile environment. After all it is quite a change coming from a traditional testing background.

This is the first post in a 3 part series on this topic, and I would like to kick it off by sharing a few observations about how people are affected by Agile's paradigm of team integration (QA and dev team members collaborating as one team) and the implications this has for skill sets. In future posts I’ll touch on testing tools, the increasing importance of test automation in Agile, and the process side of things -- highlighting the necessity for testing early and often.

So let’s talk about team integration.By that I mean the tight interlocking of testers and developers that the Agile approach requires –- e.g. pairing of developer/testing focused team members. I feel like this is quite possibly the most important issue related to testing in Agile, because it is a fundamental change in the way things have always been done. It requires a new, shared responsibility for quality by all team members and not just a dedicated (and separate) QA team.

Without a doubt there are a number of benefits that come with the integration of testers into a development team. Here in Linz, since we have integrated the members of what used to be a centralized “QA team” into our Scrum teams we have seen several of these benefits first-hand. Improved communication is an obvious one, since now our functional test scripts and code are developed in parallel based on the same requirements and changes.

Also, a combined team has an improved ability to meet quality objectives as everyone is now proactively responsible for quality. Our own teams have seen improved test coverage through test pairing and by sharing libraries of unit tests (test scripts created by developers) and functional tests (test scripts created by testers). It also takes us less effort to develop automated scripts and guarantee code-and-test coverage. Because our combined teams are able to isolate defects and quality issues earlier, we have also seen improved velocity since we integrated them.

But all of this doesn’t come for free. To make it work, it is necessary to expand the tester skill set. While the core responsibilities of QA -- identifying the most appropriate implementation approach, implementing, setting up and executing individual tests, logging outcomes, verifying test execution, analyzing and recovering from execution errors –- remain, there is also a need to adapt the traditional testing role to accommodate Agile development practices

Among other things, this means everyone needs to become a member of “the team” and the Dev/Test barrier gets removed. And, removing this barrier is much more than just changing organizational structure and seating arrangements. It’s most definitely a cultural change –- which means an investment in time and training.

To make it all work, it also becomes necessary for testers to develop programming skills –- that is part of the evolution of the tester’s role in an Agile environment. Testers need to get involved in test conception from the beginning. Many roles in software development have evolved over the years and Agile will certainly require testers to become more skilled as well. There are a number of ways this can be achieved and not all involve formal training or classes. For example, allowing the team members to invest a certain percentage of their time for self education and leveraging Agile best practices like pairing will provide team members who don't have a strong development background an opportunity to ramp up their skills.

In my next post, I will explore a more tool-focused topic and talk about the increased importance of test automation in Agile environments. Bye for now.

You're my most beautiful regret; I wish I never met you and you were easy to forget

2009-04-30T16:18:55+01:00

Today was the demo and retrospective for the last sprint of our current release. Normally I'd next have the team roll into release planning for the upcoming three months (we are on a quarterly schedule), but instead we skipped release planning entirely and went straight into our normal sprint planning.
Photo by Chris Campbell I think it will work well for our team, but as I'm normally a fan of release planning I have some trepidation.

Mike Cohn wrote recently, "A release plan helps a team avoid finishing a series of sprints and feeling that, while they always worked on the highest priority items, the collection of work completed does not add up to a satisfying whole." Even though I skipped this with our group, I strongly agree with the sentiment. My reasoning is simple: I think that a solid, prioritized roadmap and a commitment to releasing on a particular date (as opposed to including some minimum feature set) can largely eliminate the "lack of a satisfying whole" problem.

To be fair, we did have a talk about the next quarter: discussing in more detail the top items on the backlog, coming to an agreement that staying releasable and automated testing remain paramount, and prioritizing significant architecture work that needs to come earlier in the release cycle. We also got the PM to agree publicly once again that the roadmap was not a commitment and as we learned more priorities might change over time. The team seems excited about the results, and we had an energetic sprint planning session afterward.

I think this is going to work well for us, but it isn't a direction I've used with other teams. What do you think; will I regret this decision?

Post title is a lyric from Beautiful Regret by Morrison-Williams.