Merriam-Webster defines agile as:

1: marked by ready ability to move with quick easy grace <an agile dancer>

2: having a quick resourceful and adaptable character <an agile mind>

The Cambridge Dictionary defines it – in British (not American) English – as:

able to move your body quickly and easily <Monkeys are very agile climbers.> <You need to have agile fingers to do this kind of work.>

Notice that the key word here isn’t ‘fast’, but ‘quick’.  ‘Fast’ is a fraught word, and seems to be what most think will be the new nature of an otherwise traditional project – exhaustive, ill-informed, up-front specifications and all – once we just get agile.  But do we really want the whole train wreck to simply increase speed and arrive sooner?

Agile’s primary benefit isn’t speed but, rather, feedback – between development team members, the development team and the business, and the business and the market.  The most noted effect is greater understanding and insight into what’s working and desired (and what isn’t).  This allows course correction that wouldn’t be possible if you were only hurtling toward a mistaken destination ever faster.  Admittedly, over time and with experience, this difference does lead to a speed-related effect:  more frequent delivery of only the highest-value items.

But what’s delivered to the business and the market isn’t faster in the same sense as what’s expected; You won’t get a whole year’s worth of pre-agile production of code in only three months post-agile. And you wouldn’t want to, would you?  What you might get – with learning, practice, and experience – is a traditional year’s delivery of value in half or a third or even a quarter of the time.  But first you’ll have to get really good at listening to the feedback you elicit, involving your customers, and changing your mind and your plan.  But will you be faster? Which is more important?

While it’s understandable to want to allocate credit for the effort spent to the sprint to which it ‘belongs’, it’s important to remember why there’s no partial credit in Scrum: Because working software is the primary measure of progress (#7 of the principles behind the Agile Manifesto: http://agilemanifesto.org/principles.html). The metric of record is therefore how much a team can complete to potentially-shippable quality each sprint, not how much they can complete to 90% finished status. This is a common occurrence with new Scrum teams. By the end of Sprint, they have a whole bunch of backlog items “90% complete” (whatever that means), which means they’ve got nothing done to potentially-shippable standard, which means they’ve delivered zero business value. In other words, they’re 0% done.

Attempting to ‘give credit where it’s due’ by tracking how much work was performed in which sprint skews the emphasis toward measuring how hard people tried to reach a goal, which isn’t what we want. We want to measure the goals reached: backlog item story points competed per sprint, or business value delivered if you’re using that metric.

The most common recommendation is that an unfinished backlog item gets returned to the product backlog, with its original estimate intact. When it gets committed to a new sprint and completed, the backlog item’s full effort estimate gets credited to the Velocity of the new sprint. While this may appear to skew the numbers, the numbers average out over time – and it’s this average velocity that a PO uses for forecasting and planning. Moreover, this practice adds an incentive for the team to do their best to discuss, negotiate, estimate, and commit to finishing backlog items to full completion in a sprint.

If management and finance want to track costs for purposes outside of managing a team (one that doesn’t need managing because it’s self-organizing) – say, for hourly client billing – then a task-level metric might be used. A carried-over backlog item’s constituent tasks done in the first sprint will remain counted as tasks completed (or task hours completed and remaining if you’re using those metrics) as part of graphing the sprint burndown in any good agile project management software such as ScrumWorks Pro.

This is a not-uncommon business need, although the practice is easily abused to “manage” agile teams’ “productivity”. The danger here is that it leads to a mentality (to use a relay-race analogy) of tracking the runner rather than tracking the baton. In Scrum it’s the baton that’s important – that is, story points of backlog items done to potentially-shippable completion each sprint (Velocity). Since ultimately that’s what we care about, then that’s what we should spend our expensive, limited time encouraging and focusing on.

Q: I’ve found that when asked, people come up with a very ‘full’ definition of done, but in the end they don’t follow it. How do we walk our own walk?
A: I could answer this in a number of ways, but I’ll take a more hard-edged approach here. It is about accountability. An important aspect of Scrum is the idea of the “single wringable neck”. At the end of the day, it’s the product owner who determines whether a story has met the agreed upon definition of done. If he or she chooses to ignore aspects of that definition for whatever reason (expediency, effort to be a liked, business pressure), ultimately he is the one who bears the responsibility for the impact.

Q: Isn’t it likely that someone writing poor code might also write poor and ineffective tests?
A: Absolutely. Agile practices are designed to uncover and display issues like this so that they can be addressed. Practices such as Pair Programming, code review, daily stand-ups, retrospectives, and sprint review meetings are so important. There is no magical fix for poor test writing, but when we know the problem exists we can work to fix it.

Q: How was “”easyness”" vs “”difficulty”" of changing the code measured? What metrics exist for what seems like a semi-subjective value?
A: There are many variables to estimating the difficulty of a code changes. We could talk about things like the complexity of the code (Cyclomatic Complexity), the experience of the programmers, their familiarity with the topic and/or the specific module, the quality of documentation, etc. It ends up being a fairly subjective estimate. In Scrum teams, story sizes are estimated in relative terms in terms of story points.

The primary benefit to using a technique involving Relative Estimates is that you are asking the team to give you an estimate of difficulty relative to other work that has already been completed. This means that a team can easily give judgments like “This will be twice as hard as that” and come up with functional estimations for predictions without spending a great deal of time coming up with them. Estimates are just subjective guesses anyway, understanding that can be a valuable way to put more time into building something and less time into trying to guess how much time it will be to build it.

Planning Poker, also called Scrum poker, is one technique for building relative estimates and for coming to consensus on the effort or relative size of the stories.

Q: Technical debt always meant: those things that you postpone doing that you know must be done, that become more expensive to do as time goes on — e.g. the technical “”debt”" has compounding “”interest”" applied.
A. I like the extension of the metaphor to include the interest on the “debt” because it is quite valid. That said, I would also point out that there are often valid and justifiable reasons for incurring technical debt. As long as teams incurring that debt know what they are doing, have justified it, and have the means to pay it off in the future then that’s fine. (Perhaps there should be some kind of Consumer Protection Agency for developers? Never mind…)

Q: How often does pair programming fail as the result of pairing developers where one has a dominant and one has a passive personality — and how can this be detected and treated?
A: In pair programming you have a two programmers work together at one workstation – a driver and an observer. The driver is the person who types in code while the observer reviews each line of code as it is typed in. The idea is that the driver is focused on completing the tactical aspects of the task at hand. The observer takes a strategic view, looking for ways to improve the code and at the how it fits into the overall system framework.
Generally, you know there is a problem when you see lack of engagement between the two. Simply put, they are not talking. A good way to try to encourage communication is to swap their roles frequently – at least once per day.

Q: If the automated tests exist – but are as old as the code that it’s testing – how would that help?
A: They help to prevent regressions, where changes in other portions of the code may break something that was untouched.

Q: Should refactoring be included as a user story in the sprint?
A: It works to have refactoring stories in sprints, or even to devote an entire sprint to refactoring. Whichever way you choose, the stories must be compliant with INVEST – independent, Negotiable, Valuable, Estimable, Small, Testable. (Bill Wake).

Q: what happened to defensive programming?
A: I think it is an issue of semantics. The principles of defensive coding are well represented and addressed in Agile development techniques. These include reducing source code, complexity, engaging in formal source code reviews, software testing (especially in the context of Continuous Integration practices), re-use, and so on. I would posit that an experienced and mature Agile team is practicing defensive programing.

Q: In the case of a legacy system with existing technical debt, is it advisable to dedicate one or more sprints to reducing the technical debt?
A: This isn’t a technical decision, of course. The decision to devote a sprint to paying down the technical debt of a legacy application should be made based on the business merits of doing so. Look at how much that technical debt is costing you in terms of delivering new functionality and addressing defects. Look also at the impact of moving your developers off enhancements and defects, and onto a project whose impact will not been seen for months. You strategy will be informed by the current maturity and projected lifespan of your application.

You might be interested in a recent webinar we did that looked at integrating Agile Project management and Project Portfolio management. See the link here.

Judging by the number of questions, the audience was clearly heavily engaged.  We had a 15 minute Q&A and did not come close to answering all of the questions, so I’ve listed most of them here and provided a response. The ones pertaining to HP PPM will be in a separate blog post written by Russ King of Result Positive.

If you missed the session live, you can watch the recording and download my slides. I also encourage you to try SrumWorks Pro for free and find out why ScrumWorks Pro is the best Agile project management solution on the market today. Also, don’t forget to register for our upcoming webinars. CollabNet offers multiple webinars a month that cover a variety of topics related to software development.

As promised, here’s the follow up to the live audience questions.

Q: How feasible is Agile on Projects & Programs?
A: Agile is typically thought of in the context of individual projects. Companies sometimes fail to scale that paradigm to a program level, where the program is a superset of multiple projects, each running its own lifecycle and release plan. The trick is to weave those separate lines of development (projects) into a coherent and seamless deliverable (program). The complexity comes in gathering meaningful metrics and planning releases that thread the elements together. This is exceedingly difficult to do manually.   CollabNet’s ScrumWorks Pro is a tool that can make this manageable. It supports the planning of complex releases that weave in multiple development threads.

Q: Will this process will be feasible for maintenance related projects (Incident handling, less than 8 hours development works, etc.,)?
A: From the PPM perspective, an individual defect is not in and of itself a project and as such, would not be tracked.  What might be tracked is a larger group of maintenance items in the form of an Epic. From an Agile perspective, a bug report or defect is just another piece of deliverable business value, like a User Story or any other Product Backlog Item. From a bug report, the product owner and team would create a Product Backlog Item (PBI), along with success criteria (definition of done). It is prioritized against all of the other Product Backlog Item by the product owner. Again, multiple bugs/defects are often grouped in an Epic.

Q: It seems the PPM is geared toward a waterfall process.  It appears there is only visibility into the Development phase, but with agile, you could potentially address all phases within a single sprint.  Is that just because of the way this implementation was set up or is it there isn’t a true marriage of the agile within PPM?
A: PPM in this scenario is focused on evaluating the ROI of different projects and deciding where to make investments. Agile is focused on execution of the projects that are chosen. That said, the scenario we propose makes the entire organization more Agile, in that the feedback loop is instantaneous.  This allows those that are making the investment decisions to adapt and make course corrections that are indicated by that feedback loop. The integration gives all team members the ability to work in a more Agile fashion, and gives Stakeholders and Project managers the ability to benefit from the faster feedback and data generated by the team working this way.

Q: Can the tasks in Scrum WorksPro be connected to tasks, timelines in Source forge?
A: Not with Sourceforge. However this is possible with CollabNet TeamForge, the current commercial version of Sourceforge.

Q: Can you clarify what part of Agile PPM can be done in scrum works pro without need for HP PPM?
A: ScrumWorks Pro is focused on project execution and project management. As such, ScrumWorks does a number of things not accomplished in HP PPM. These include PBI tracking and prioritization, Task management sprint planning, release planning, team velocity, forecasting, and many other functions related to the management of an Agile project.

Q: So are you proposing (in the demo) to combine a phase/waterfall planning and design phase, but then execute in an agile framework?
A: Combining HP PPM and ScrumWorks Pro adds to the agility of the entire organization. Feedback loops between the development team and the PMO are enhanced allowing the PMO to make course corrections required. I would not say that as a result the entire enterprise has become agile – only that they’ve become more agile.  Generally, we do not see many organizations that practice a pure version of ANY methodology –be it Agile or otherwise. The reality is that organizations have a mix of methodologies, like Scrum, Kanban, Waterfall, hybrids, etc. Different teams in large organizations will often build software differently, so the challenge is to roll up the data from those disparate teams. Despite their differences, there are a number of common metrics you can track regardless of project type.  These include actual cost versus budgeted cost, scope change, personnel/resource change, delivery dates, and others. Tools like ScrumWorks and HP PPM do a good job in tracking these kinds of numbers.

Q: Continuing from the first question, from a portfolio perspective, having “”open-ended”" project budgets within the Agile/SDLC process is not in the best interest of my customer.  How does budget planning and Agile development work together while still having some control over costs?
A: Project prioritization and the associated budgeting/funding are is under the purview of the PPM tools.  The agile project management tool tracks the amount of time individuals spend on the project. The integration between the HP PPM tool and the Agile Project Management tool, allows you to easily compare budgets against actuals.

Q: For the Forecast report in SWP, if new backlog items are added during the sprint, does that add to the top or bottom of the bar? Also, how does the Project Portfolio Management tool fit into the larger Enterprise Architecture discipline?
A: It depends on what report you are looking at. In the forecast report, added backlog item appears on the bottom of the report and impacts the forecasted delivery date.

The “Burn-Up” Report shows the relationship between work completed per iteration (sprint velocity) and project scope change. The forecast feature extrapolates the rate work gets done against the rate of scope change to provide an empirical release completion forecast for more accurate release planning.

Q: In agile, what are the differences between being adaptive to late changes in requirements within a sprint and scope change?
A: Scope change refers to any added or subtracted scope, typically measured in some form of relative effort unit like Story Points. As such, scope may be added as a team discovers more about an existing requirement. In other words, if the team finds out that a requirement is more complex than was originally envisioned, they may re-estimate the number of story points and this might add scope to a sprint. The opposite could also be true. Whether this occurs because of a discovery inside a sprint or outside of it doesn’t change the nature of how it is tracked or reported upon.

Q: When a committed backlog item could not be completed in a sprint, naturally it holds the top most priority in the following sprint. How does ScrumWorks helps in tracking this item from the beginning to end?
A: An unfinished PBI may or may not be a high enough priority in a future sprint. The determination is made by the product owners. In any event, any activity against that PBI is tracked. Tasks completed that relate to that PBI are tracked, as are those that were uncompleted.

Q: What certification do CollabNet-trained scrum masters receive?
A: Those who attend one of our Certified ScrumMaster or Certified Product Owner training are eligible to take the exam deliver by the Scrum Alliance. It should be noted that CollabNet is one of the leaders in ScrumMaster product Owner training.  We have more  Certified Scrum Trainers on staff than any other vendor, and we’ve trained more than 12,000 ScrumMasters.

Q: If an organization wants to be able to report a metric of time to resolution for individual PBIs, what settings are available in this integration to include/exclude a PBI from the current active lists so that a countdown starts appropriately?
A: Forecast reports in ScrumWorks can be filtered on any number of aspects, allowing a user to deliver estimates on individual tasks, Stories, Epics or Themes. By the way, you can try out ScrumWorks Pro either in a hosted environment or as a free download.

Q: We are HP PPM users. Who owns the IP for the ScrumWorks integration technology? Cost?
A: Results Positive and CollabNet worked closely together to build the integration. Results Positive however owns the integration IP and can provide any required assistance you might need in getting the integration set up.

Article summary:

Technical debt describes the cumulative consequences of cutting corners in software development, but it escapes the attention of many project managers as they focus on scope and schedule. That’s a mistake because it impacts both. Here are questions to help you ascertain the real state of technical affairs.

I once had this put to me bluntly by a would-be product owner when we were discussing how much information and detail should be given the team when defining requirements. I was trying to impart the value of the more is-better approach, when he stated he wanted to provide only “Just enough detail to hold others accountable [in the event of failure], but not so much that I can be held responsible.” He appeared to be joking, but there was no denying the element of morbid truth.

If you look closely, however, it’s obvious this arrangement is an attempt to come to grips with the intrinsic unpredictability of software development, by reserving the ability to change requirements or direction mid-course. A dysfunctional and chaotically wasteful method, for sure, but such things happen when laboring under false assumptions like those embedded in waterfall.

Ironically, this ability to change course is the very raison d’être of agile development – fortified with clearly defined roles and transparency to prevent the perverse situation above. The team has the authority to say how much work it can do in a given time, and the responsibility to meet the commitments it makes. The product owner has the authority to determine what does or doesn’t get into the product (and set the priority order of what does), and the responsibility for the success or failure of those decisions. With power comes responsibility. And there’s nothing like your neck on the line to provide the motivation to perform at your best.

Much of the resistance to agile stems from our habits of over-specialization and strict serialization of tasks, both symptoms of mistaking efficiency for effectiveness. Workers not spending as many work hours as humanly possible in their specialty? Waste! Two people working on the same task? Redundant! It’s all too easy to view these ‘inefficiencies’ as a reason to modify or reject tenets of a new framework such as Scrum, whose instructions encourage such behaviors; The more adaptable will see the requirements as cause to reevaluate old assumptions about what’s best to develop product. For what’s the goal here: shaving person hours and other micro-optimizations, or supporting group creative problem solving to deliver business value?  As Victor Szalvay (our Director of Engineering for ScrumWorks) says about Scrum: “We take waterfall and put it in the blender”. One of the best ways to do that is to make sure you have a team of flexible individuals who will do whatever they need – including ‘each others’ job’ – in order to meet the Sprint goal.

I wonder what this team will come up with. Systems architects who write code? Programmers who hand-sketch UI designs? UI designers who code the front end? QA engineers who write end-user documentation? Technical writers who do business analysis? Business analysts who write tests? I’ve seen many of these ‘job-swaps’ happen on various other teams. A team that could do them all might do amazing things.

With the benefits so clearly delineated, why is it still “gaining” mainstream acceptance? Why isn’t it already the de facto standard?

I would posit two related reasons for this:

1. Agile education (including the sessions at Agile 2011) is too theoretical;
2. There is an overabundance of focus on the people and project management aspects of Agile and little on the engineering disciplines that must be implemented.

I made it a point to engage with as many attendees at Agile 2011 as possible, and in so doing I gained a broad (if unscientific) consensus that as smart as the presenters were, and as compelling as their ideas were, many were lacking in practicality. It’s not that the ideas themselves were invalid or not useful. It is just that the ability to put those ideas into practice was not there. Or as one participant said to me, “When I get to the office on Monday morning, how exactly, do I take advantage of what I learned?”

Stepping back from Agile 2011 and looking at Agile education in general, you might see the same criticism about the ScrumMaster, Product Owner and other certification classes. They teach you the theory, but not (in most cases) the practical application of that theory.

I understand that Agile is a framework that can be applied to many disciplines – it is not just software engineering.  I’ve seen marketing and sales teams that benefit from practicing Agile, but for this blog, I’ll just concentrate on the core use-case of Agile – Software Engineering. I would argue that software engineering teams doing iterative development of any kind must employ underlying disciplines and technologies to make it successful. The human and project management aspects must have the underpinning of Agile software engineering crafts like continuous integration, pair programming, test driven development and others.

One of the primary criticisms I heard at Agile 2011 was the lack of such sessions. The handful that were offered were standing-room only, attesting to the demand from the delegates.

In future conferences (Agile 2012 amongst others) I am hopeful that we will see sessions that directly address these two concerns. For the former, I hope to see case studies, presented by the users themselves rather than the vendor or consultant who assisted in the implementation. The difficulties and problems are just as important to understand as the solutions, and it isn’t necessarily in the interest of vendors and consultants to highlight these. For the latter, I am hopeful that we will see more “hard-core” technical sessions which will help delegates adopt the underlying disciplines more effectively.

As far as I understand it, every restaurant in LA county is randomly assessed by the health inspector and every restaurant must post their results in a highly visible location.

I like this system a lot even though I generally dislike big government hand-holding.  I get a modicum of confidence that by eating at an “A” rated restaurant I likely won’t get sick.

I wonder what it would be like if this system were imposed on commercial and open source software products.  I envision this would translate to something like a thorough code review, engineering practices audit, quality practices audit, etc.  If we were being honest in defining a rating scale, I think most products on the market wouldn’t get above a “C”.

Why? Because I think the technical debt situation at most shops is bordering on overwhelming.  CollabNet gets a unique look across the industry by virtue of our project management and engineering practice consulting services.  Sure, a lot of companies are trying to be “Agile”, but most are forgetting a key component which is holding the quality bar high and immovable.  Successful agile teams are doing this by adding strict “done” or “acceptance” criteria to their user stories and adding quality oriented practices like TDD and refactoring to the engineering environment.  I think the situation is equally out of hand (or worse) for companies doing waterfall.

Even without seeing the code you probably have a sense that this may be the case at your organization based on interaction with your own products.  Would you bet your health on your software’s quality?  Yet we risk so much on software these days: transportation systems, medical equipment and records, financial systems, business workflow, etc.

It would be nice to see a big letter grade reflecting the technical health of software before I rely on it.  Then again, it may expose a depressing state of affairs.

• Feature Walk-through video
• “What’s New” release notes

The features are described in the links above, but here’s a little more depth on this release:

The ScrumWorks Pro 5 release brings a lot of new stuff to the web client. In fact, we focused most of our energy on revitalizing and expanding the web client. Why? Because most users are team members making updates to tasks, editing stories, and getting status.  The web client is a light, easy way to do just that.

We updated the look and feel. After much discussion, we settled on a light/white theme with blue links and buttons and orange highlights throughout. I think it looks a lot better than the old grey/black interface. Everything was re-written in the web client; the editors, layout, fonts, colors are all new.

Performance is greatly improved. When I say everything was re-written, I mean it. Under the covers we worked hard to support huge sprints with hundreds of tasks and large backlogs. I can’t think of another application that attempts javascript drag and drop with hundreds/thousands of drop targets. This apparently does a number on browser performance.

Browser support is expanded. We now support Chrome on top of Safari, Firefox, and Internet Explorer. Let’s be honest, some browsers are faster than others, and we’ve officially dropped IE6. For a detailed list of reqs, take a look at our docs.

Where do I get it?

For both new downloads and upgrades go here.  Remember that 10 users are now free.  There’s also an upcoming webinar and web-based training. Give it a whirl and let us know how we’re doing. We’re listening.