by Anupam Kundu & Tiffany Lentz

Agile Journal editor Russell Pannone asked us to write an article on the agile and lean software development movement 10 years from now. Tiffany and I took a novel approach of what we believe agile and lean development will look like in 2020.

We started thinking as a future CIO and why it will make sense for me to adopt and grow agile. Result of that effort is an article which is in the form of an acceptance speech – Best CIO of 2020 Award Acceptance Speech.

Its in first person so hopefully reading will be enjoyable.

“When it comes to the future, there are three kinds of people: those who let it happen, those who make it happen, and those who wonder what happened.” -  John M. Richardson, Jr.

Good Morning!

For people who do not know me, it must be difficult to comprehend how I became the CIO of this mega multinational conglomerate. It has been an uphill task to move from the server rooms to this boardroom; believe me that technology skills alone were not good enough to make this gradual but sure leap. Looking back, I give credit to a heady concoction of Portfolio Management experience, stead fast communication, and adoption of Agile principles and practices across all the IT divisions that made this happen. And also a lot of good old luck.

When I joined this organization as the VP of Engineering, 20 years back, Agile was still in its teens. Many of my fellow managers had heard of rapid release capabilities, but had never seen a company really do it.  I knew from my experience in media start-ups that Agile would be mainstream in a few short years; it was just a matter of time. And there are obvious benefits in doing iterative development to incrementally build software products. Besides, Agile with its ‘inspect and adapt’ philosophy chimed well with the PDCA (plan-do-check-act) paradigm of lean operations. However it was hard for me to sell this apparently obvious concept of evolutionary software product development to the bosses.

The idea of “iterating” over a concept was something that only startups were doing.  Those ideas somehow did not seem ‘stable’ for an organization as large as ours.  In addition, we were tied to licensing large technology stacks and rigorous deployment checklist gates, making it difficult to accommodate frequent releases; we were limited to only two enterprise wide releases in a year keeping in mind the cost of integrating with the legacy systems for each such release. While my bosses understood the proposed benefits of evolutionary software product development, they were not ready or prepared to revolutionize our IT divisions. So I was up against massive organizational dissent when I started advocating change: adoption of Agile’s evolutionary concepts and using open source platforms and tools for product development.

Change is always an alien, still you see it happen! Since I believed so strongly in this vision of evolutionary software product development, I wanted to prove it to those who could fund my vision and allow me to effect change.  I started small, with one project in my department.  I was able to illustrate moderate success with the pilot; the business sponsor for the project was impressed with the cost control and the discipline involved in building the product iteratively. I got the go-ahead to introduce the change virus in my department, one project at a time. This took me a couple of years, but at the end of the second year I was asked to run another division and made the same changes.  These divisions were releasing so frequently and their business counterparts were so involved that other divisions started referring to us as the “Business Technology” group within the next few years. Finally, I tackled the data warehousing and mainframe divisions.  A Value Stream Mapping exercise pointed out the areas needing the most attention and we started the multi-year effort of reworking our mainframe.

Back in 2010, the Gartner group wrote an article that strongly impacted the way I thought about team dynamics.  I was not the CIO, but thinking like one helped me further transform my teams.  This article, “Leading in Times of Transition: The 2010 CIO Agenda”, called out a few key differentiators that CIOs (and managers like myself) should consider as we were moving from a time of recession to recovery. By focusing on the business, not on the needs of IT as a separate entity and by iterating through business solutions, we were positioning our teams and our company to move forward faster than our peers. Over the years, our mindset changed.  We began to not just SAY that the needs of our business was the heartbeat of our company, but we began to align our technology solutions around the needs of our business so our actions matched our words.   We, IT, began to really value the aspects that businesses value: output, productivity over efficiency, priority driven schedules, demand driving supply, ROI, etc.

Gradually, one success at a time, I found strong support from the upper echelons of business that didn’t well-understand agile but were seeing the output and hearing the excitement from their own peers.  These business leaders comprehended the obvious cost benefits of the following concepts that became foundational for our company:

• using open source software to develop and deploy iteratively
• doing more frequent discovery-delivery-discovery cycles to keep in touch with the market
• waiting till the last responsible moment to commit to key decisions
• adopt metrics to foster collaborative team dynamics rather than focusing on individual brilliance

Now, 20 years later, Agile is truly mainstream. No development organization, internal or external, speaks of also-doing-Agile or Agile-in-stealth-mode. Agile is the default software development practice across most organizations.   In the same way that no one specifically speaks about “Object Oriented Development” or Cloud Computing as separate practices, Agile today is not considered alien but an integral part of the software development teams.  Over the past 20 years, I’ve seen larger organizations who could not envision a roadmap of change stagnate and often close their doors to their smaller, more “Agile” competitors, so I believe that there are invaluable lessons to be learned here.

Go Team!

Original Agile Journal post is here.

by Anupam Kundu & Tiffany Lentz

Abstract

As agile practices become more prevalent across organizations, Product Management divisions face increasing challenges to adapt to these agile techniques and respond to their partners in IT. Sure, both groups seek the values of agile, in terms of higher productivity, improved product quality and predictable business values. We must ask, however, are agile techniques inherent to the way Product Managers create and manage their product portfolio? The quick answer is No. A change in mindset and technique are needed. Given that strategic objectives are frequently set at an executive level, Agile product/portfolio managers struggle to address the dual requirements of defining a product roadmap aligned to those strategic objectives while simultaneously addressing the constraints of resources and budgets consumed at the release and sprint levels.  This dual challenge is testing the bandwidth of current practitioners of the Product Management discipline. More often than not, most Agile project teams look forward to direct collaboration with the strategy makers for quick and effective decision making over reporting extensive metrics for each and every project/program; the reality is that only a few product/portfolio managers are actually capable of making that shift to accommodate this drift. What is needed to make that shift?

This paper outlines a modest agile-enabled framework adopted by the product wing of the digital division of a publishing house to charter their product roadmap and simultaneously enable their project team with the “big picture”. We adopted highly collaborative, feedback dependent, iterative and time boxed activities geared to developing and maintaining a rapidly evolving product roadmap. This report reinforces the necessity of embracing agile product management principles as being central to successful agile projects and teams. Without appropriate product management that integrates continuous feedback loops, agile teams might end up delivering the wrong products faster. This article provides the tools to enable medium sized practicing agile teams and their product owner(s) to steer their product portfolio in the right direction.

Introduction

In his book “Agile Estimating and Planning” [i] Mike Cohn stresses that the accuracy of a plan decreases rapidly the further we attempt to plan beyond what we can see. Hence agile teams usually get involved in the elaboration of planning at three distinct horizons (if you cannot see past the horizon, you need to look up and adjust your plan) – daily, iteration (sprint), and release. By planning across these time horizons, agile teams focus only on what’s visible and important to the plan they are creating. Though most agile teams are usually concerned with only these three levels of planning, an Agile Product Manager doesn’t stop at release planning; product managers need to plan with equal ease at multiple levels. They have to scale from release planning to creating a full product roadmap, portfolio management, and finally to executive strategy making. The scale of activities for the Product Manager now spans across all the following horizons: daily, iteration (sprint), release, product, portfolio, and strategy.

Figure 1 highlights the different levels at which an Agile Product Manager is expected to work; the outer levels (product/ portfolio/ strategy) usually have a different audience and different rates of progression than the inner circles. This makes it more important for the Product Manager to be agile in their communication and deliverable. Unfortunately, it is more often the case that enterprises have traditional product owners or managers who are unable to catch this fast drift from release planning to product strategy and vice versa. Agile Product/Portfolio Management is geared to address a number of these concerns of product managers by embedding key agile principles of iterative feedback, constant collaboration and prioritization into product/portfolio management.

Figure 1: Planning Onion shows possible multiple planning horizons

In an article in the Agile Journal[ii], Joe Krebs explains how Agile project management and Agile Portfolio Management practices enable organizations to define their corporate strategy by using a pyramid like the one shown in Figure 2. This pyramid is dysfunctional without close and continuous bi-directional collaboration between the portfolio managers and the actual project members so that the latter are able to execute projects that achieve strategic corporate objectives while providing detailed insight into the state of the projects. The direction of the two arrows in the pyramid shows the flow of information and feedback across the different units.

Figure 2: Agile Portfolio Management is build on agile software engineering and drives corporate strategy

We can derive a formal definition for agile portfolio planning from this pyramid:

Product/Portfolio planning is a key activity for the Agile Product Manager, which usually consists of planning and management of existing product sets, and defining new products for the portfolio.

Now, in order to define the portfolio, the product manager has to develop a product roadmap in collaboration with her stakeholders that consists of new upcoming products and existing product plan updates based on the their current status. The product roadmap thus enables identifying future release windows and drives planning for tactical development.  The company referenced in this article is the 4^th largest publishing house in U.S. based in New York whom we will refer as the client company.  At the client company, the team – working alongside the product owner and other business stakeholders – adopted an agile roadmapping model for building and sharing the digital strategy.

How did it start?

It all started with re-engineering of the existing consumer facing website of the client company. The primary goals of the new site were to simultaneously give a voice to the authors, outside of their books, as well as to provide a richer and more compelling user experience to both readers and authors. This was achieved through the use of multimedia, author- and user-generated content, social marketing, and content syndication. Conceptualized in-house, the project was outsourced for both design direction and development implementation.

When the site went live and was considered a success in the media industry, the business sponsors were eager to execute new projects while riding high on the waves of success. Within a short period of time, the project backlog grew longer than expected.  Eventually, the development team was confronted with multiple backlogs prioritized by multiple stakeholders with little or no consolidated prioritization.

The main backlog was a long list of specialized new projects with multiple degrees of business impact. A second – and steadily expanding – backlog consisted of enhancements to the existing site. Finally, a third backlog consisted of ad-hoc multiple small projects with various goals and objectives. With the desire to satisfy all the various stakeholders, the teams started delivering products from all the backlogs with no overall product strategy in mind. All the stakeholders were equally involved in prioritization exercises, and soon realized that although the project teams involved were delivering releases in a timely manner, the business impact of those releases was hard to realize. This is when the team, the product owners, and the stakeholders decided to put agile product/portfolio management principles into practice to enable the definition (and subsequent execution) of a product roadmap.

Building a Product Roadmap

Building a product roadmap for the digital division of a large publishing firm is a strenuous process that is fraught with dangers. There are work items that need immediate attention, new marketing outreach initiatives, and then strategic projects to organize and develop the internal infrastructure. Correct prioritization of feature set, and planning a proper release that will address the executive strategy, are serious challenges for the product manager and the agile project teams.

Lack of fast feedback, inability to change course direction based on new priorities, and reluctance to gather inputs from multiple stakeholders can throw the team off track quite easily. To deal with this, the agile team introduced a tiered approach to develop and execute the roadmap. The tiered approach made sure that everyone’s voice was heard. To support this approach, the project moved from its initial one-week sprint to a two-week sprint to ensure the availability of sufficient lag to alter priorities for the teams without overburdening the release cycles.

At the client company, each product request in the roadmap was judged on multiple parameters to make sure that the roadmap consisted of feature sets that delivered maximum business value and remained aligned with the corporate strategy. A few of the key questions that were considered for building and prioritizing the roadmap are:

• What is the business value for the product?
• Is the new feature considered a legal obligation for the market?
• Does the new product provide a distinct competitive advantage in the marketplace?
• How much can the proposed product leverage the newly created infrastructure?
• Which product can help launch or promote new or emerging lines of business?
• Will the new product allow the stakeholders to reach and exploit new marketing geographies?
• How much will it cost to launch the new product?
• Is there a need to build follow-up modules to the product?
• Is this a new product a catch-up with rest of the players in the market?
• Is there a partner obligation for the product launch schedule?
• Are all necessary resources available for the product to be implemented?
• Which product addresses the most demanding stakeholder group in the company?

The idea was to initially create and maintain two backlogs for the product roadmap: one for all the bugs and enhancement requests; the second for all high business value products and new feature requests. Unless the bugs and enhancements were deemed to be critical, or if there was not enough work for the whole team, the sprint focus of the project team was always dedicated to new features and high value business products based on prioritization from the product manager.

The team adopted a quick feedback model to ensure that the project teams, distributed across the world, had visibility into the prioritized product roadmap, enabling them to them to stay focused on delivering the right projects. This provided the product owner and the stakeholders bandwidth to prioritize the project backlog based on the business realities of cost and implementation timeline.

What to work on from the product roadmap?

Once an initial draft of the roadmap was created, loaded with multiple new products and features, the challenge facing the organization was to make sure that the agile project teams were dedicated to working on the right products selected from that roadmap. This is where we had to extend the bandwidth of the product manager by introducing full time business analysts to the project.  Figure 3 outlines the different phases, and the key groups involved in active collaboration in each of these phases. The process of selection of projects from the roadmap for implementation consisted of four logical and overlapping phases, to which each product is subjected. We divided our effort to make a project Go/No-Go decision into 4 different, yet cohesively connected buckets:

• Identification
• Prioritization
• Exploration
• Confirmation.

Identification: This is the phase where business stakeholders brainstorm and define the business goals for a new product.  The initial tensions between different stakeholders about getting their projects in the priority list are overcome during this identification process, as the business and technology stakeholders come into close contact with the product owner (portfolio/product manager). Based on the initial round of discussions and evaluations every project idea is assigned a priority ranking.

Key Outputs: A ranked product roadmap with high level business visions and goals outlined for the highest priority projects and features. Also included are initial definitions of targeted user roles, initial workflow for prioritized product(s).

At the client company, the business stakeholders during this phase conceptualize the need of a product to enhance their digital presence and drive corporate strategy. Primary business goals are defined and initial user flows are identified.

Prioritization: During this phase the whole team is normally brought into the roadmapping process. A quick kickoff is arranged to make the team aware of the roadmapping process. Based on priority inputs from the product owner, the team evaluates the project ideas and generated epic backlogs to provide initial “order of magnitude” estimates (estimating in T-Shirt sizes). All the risks are identified and assumptions are laid out. It is at this stage the product manager reassesses the risks, estimates, and potential business values. This reassessment results in revised priorities for one or more projects, through collaboration with the other stakeholders.

Key Outputs: An initial story backlog that has been prioritized by the business owners in collaboration with the team and the product owner. The backlog is supported by initial coarse-grain estimates, and lists of risks and assumptions to make sure that everyone understands the work scope.

At the client company, all of these (re)prioritization activities are incorporated as part of the regular sprint work, so that the entire project team has more visibility into the potential pipeline of work and can provide quick feedback to the product owners on the ‘current’ state of the backlog and team capacity. Also first draft of user workflows and wireframes are created to aid in estimation.

Exploration: At this stage, the risks get well defined as the team performs early technical spikes for integration touch points. Refined estimates are available as user attributes and user interface workflows are defined to the next level of detail. This results in a tentative release plan based on the current sprint backlog and team capacity.

Key Outputs: A new version of the story backlog with refined granular level estimates and risk lists and a draft of the release plan.

At the client company, this phase is used to share the initial release plan with the teams for feedback. Also the outputs of the technical spikes are shared with the product owner to make them aware of the potential technology choices.

Confirmation: This is the phase when the business stakeholders review all the available information (business value, risks, estimates, product definition, and suggested release plan) to reach a Go or No-Go decision. It is the responsibility of the Product Owner along with the team to refine the release timelines and resource scheduling based on the decisions taken.

Key Outputs: The approved project backlog is the main output of this phase, which drives a formalized release plan owned by the team and the product/portfolio owner.

At the client company, if the project is a Go, the portfolio owner refines the release timelines and resource scheduling. If the project is a No-Go, the portfolio owner puts the project back into hibernation for reconsideration at a later time.

All four phases are interrelated and interdependent, each drawing input from the one before and providing output back to refine the decision making process at this stage. Constant feedback, collaboration, free exchange of information and artifacts, and a centralized dashboard are among the key factors that make these phases work seamlessly across multiple sprints and release cycles.

Figure 3: A diagrammatic representation of the steps followed to adapt agile product/portfolio management[iii]

Advantages of collaborative roadmapping

This feedback oriented, collaborative roadmapping process was a learning experience for all of us involved in the projects at the client company. We were quickly able to adapt the implementation direction based on learning from constant feedback sessions during roadmap meetings. All the team members felt empowered and considered themselves to be in the ‘thick of the mix’ rather than just doing vanilla implementation projects. Following are a number of the positive effects we witnessed.

• Rapid portfolio management: Usually portfolio planning is a slow process as portfolio decisions are slower and more deliberate – involving a broad range of inputs from multiple stakeholders – and more impactful than product level decisions. However, agile product roadmaps are created and maintained iteratively with close collaboration between the stakeholders and project teams, hence portfolio planning progresses rapidly.

• Ability to change roadmap direction: Agile portfolio management offers the flexibility to frequently update the roadmap based on close feedback and strong collaboration from the agile teams and the stakeholders. The product roadmap can undergo updates at any time during the release cycles; current project status, change in future financial projections, change in prioritization by the stakeholders, or a new legal requirement can be factored into the roadmap. Each of the phases provides enough information to the stakeholders to change their priorities based on reality checks.

• Knowledge sharing: The goal of each of the phases overlaps with the next one; they are not water tight containers of activities guarded by gatekeepers, but rather collaborative, feedback-based time boxes geared to moving forward in the project selection process. At this client company, these logical phases not only provided our team with the perspective of the overall road map and the proverbial ‘big’ picture from a business value view point, they also enabled our understanding of the nature of the potential work, and the risks attached, during early analysis and estimation processes.

Overcoming Challenges

All these great advantages brought with them multiple challenges, both for the agile team and for the product managers and stakeholders at the client company.  Most of these challenges were related to a lack of timely and effective communication between the different participants, and to overstretching of the available bandwidth of the team and the product owners. We could reach quick solutions for some of them, while others took a little bit of time to earn buy-in from stakeholders.

Challenge: An agile product/portfolio manager has to constantly balance the demands of both the project teams and the stakeholders; she has to switch between specifics of stories and sprints and more grueling new product definitions and prioritization of product roadmap with ease. Suddenly she is not just communicating with sprint schedules and reporting on tactical development activities, but is also involved in strategic planning with other stakeholders using the quintessential product roadmap.

Solution adopted: This was effectively handled by creating product owner proxies, which included full time business analysts in product teams to scale the product manager to achieve her goals. Expanding this role was a relatively small price to pay compared to the increased effectiveness of the project team and faster delivery cycles, ensuring the delivery of priority business values. So business analysts became these key resources who could move with equal ease between the development teams and the product management team.

Challenge: Stakeholders, in traditional mode, will be involved only once or twice to define product roadmaps. In this agile roadmapping process, on the other hand, it is expected that the roadmap will continuously evolve and change based on feedback from agile teams and on changes in market realities. The product roadmap becomes a ‘live’ artifact to which the stakeholders were held accountable along with the product manager.

Solution adopted: Weekly meetings were introduced to discuss and dissect the product roadmap and reprioritize the backlog based on inputs from different departments within the organization. This top-down input to the product roadmap process increased the overall trust and accountability of the stakeholders. Few or all of these meetings happened within the scope of the sprint activities depending on the duration of the sprint and current sprint backlog. Soon, we adopted a two-week sprint schedule to accommodate the product roadmap prioritizations into the sprint backlog.

Challenge: Agile project teams – used to working in daily, sprint, and release modes – usually have low visibility into the product roadmap and hence corporate strategy.  With Agile roadmapping they are involved in early estimation and release planning which affect the overall velocity of delivery.

Solution adopted: We successfully introduced this phased approach so that the team could get involved in the roadmapping process without significant delivery impact. Introduction of dedicated business analysts also enabled the project team to understand not only project backlog, sprint backlogs and release plans, but also business plans that provided overview of product vision, goals and how they fit into the overall corporate strategy, and the product roadmap.  This new insight increased the confidence of the team as they were now sure that in every sprint their effort was directed towards building the prioritized product roadmap to deliver the most optimum business value.

Conclusion

We experimented with this approach by moving over to two week sprints (instead of the usual one week sprint cycles) and involving the business analysts more closely in the roadmapping process. The business analysts enabled the product owners to significantly extend their bandwidth. We also modified the sprint planning meetings to accommodate discussion on the roadmap projects. These changes yielded positive results for our whole team and the stakeholders at the client company. As a whole, we saw marked improvement in productivity by adopting this phased model; the count of the number of new products released in a quarter went up, the highest priority products got quicker release timelines compared to the less important ones, and the team morale got a big boost.  However, we recognize that this model is not a silver bullet for other projects or other business situations.  It is important to remain aware that they are not guaranteed to give the same results every time for all teams. So product owners and project team members, feel free to adopt this model for your portfolio planning and then tweak it, if need be, as per the local sensitivities to achieve your goals.

___________________________________________________________________________________________

[i] Agile Estimating and Planning, Mike Cohn

[ii] Agile Portfolio Management / Product Strategy Pyramid: http://www.agilejournal.com/articles/columns/articles/415-the-agile-pyramid-aligning-the-corporate-strategy-with-agility

[iii] Diagrams and review courtesy of Steven “Doc” List