Agile Booknote

CMMI desde una Perspectiva Agil

2011-10-11T12:47:00.000-07:00

Mi presentación en #agiles2011

CMMI = Control & Agile = {Collaboration/Cultivation}

2011-09-03T09:59:00.000-07:00

Pete Behrens gave a talk on The Culture of Agility @Agile2011 that left me thinking on the topic of my latest posts, the relationship between CMMI and Agile.

In his presentation, he shows a quadrant which contains 4 different types of cultures:

And he defined certain characteristics for each culture. Look the characteristics for a control culture:

This really looks like CMMI, right?. "Policy and procedures are extremely important", "The system is more important than people" (these phrases really sound as the explanation that CMMI gives as to why processes are so important "What holds everything together? It is the processes used in your organization. Processes allow you to align the way you do business. They allow you to address scalability and provide a way to incorporate knowledge of how to do things better. Processes allow you to leverage your resources and to examine business trends.")

Look now the characteristics of a Collaboration Culture:

And the Cultivation Culture:

These really look like Agile, right?

Look now the characteristics of personal and impersonal cultures:

These one's REALLY looked like Agile on the left side and CMMI on the right side. It is clear that CMMI would fit a control culture much better than a collaboration/cultivation culture and , inversely, Agile fits perfectly on collaboration/cultivation cultures. It's clear how Agile is "Personal", while CMMI is "Impersonal".

So what about compatibility. From the perspective of culture, what is the result of combining them?

I can imagine a couple of scenarios. In a collaboration/cultivation culture, applying CMMI would push the organization culture towards a more Control/Impersonal culture. From the other side, a control organization (which may be CMMI certified, as it's totally compliant) and wants to become more Agile may encounter a CMMI an obstacle.

So in the culture context, I see the 2 forces going in opposite directions.

Could this be synergetic or complementary or the effect would be the total opposite? Could this lead to an undefined culture or to an organization where employees are confused on their culture and values?

Software Craftsmen Movement

2011-07-24T15:27:00.000-07:00

When software development started, perhaps I guess about 50 years ago (probably more), I believe programming was the most important task in all software development life-cycle. Other tasks or phases probably didn't exist. They were embedded in what they called programming (right, the old ones?). After a few years, we started learning how to develop software better. This learning derived in methodologies that were created. In these methodologies, new phases such as analysis and designed appeared and in many of them, those phases became more important than programming itself. This was weird because the actual task of building the software lost importance. In heavyweight waterfall methodologies, analysis and design took most of the work. Programming was just a translation work that could be performed (probably by anyone) when all the important work had already been done.

Changing the subject, I usually asked myself: How do we, programmers, know that we do our work correctly? Programming is one of those jobs where anyone, with google's help, can combine a few frameworks, throw a few lines of code and create a basic software system. But does that make you a good programmer? Another circumstance which perhaps is different than in other areas, is that our job market has been in need of professionals for many years, absorbing most or all the people in the field. So for many of use, getting a job as a programmer as been an easy thing. We just got off from University and they were waiting for us. But does that make us good programmers? I believe I can speak on behalf of lots of people when I say that we go from receiving very theoretical classes of Computer Science at University to try to resolve down to earth problems using a programming language. And this happens, a lot of times, without guidance (because supossedly we learned this at University?). Or isn't it true that lots of us have throw our first few thousands of lines of code to production without anyone checking them? I wonder: who teaches us how we should do our job as programmers, the day to day tasks. I'm not talking about learning a programming language. I'm talking about how to use it to resolve a problem, which is the best process to do it, what are the techniques. This lack of guidance, of someone that verifies our first steps, of someone that shows us the way, makes it difficult for us to understand how we progress in our career. The leap from finishing University to becoming a good programmer is huge. And how do we fill it? How are we supposed to know what it takes to be a good programmer? Is there any curricula we could follow? Is there a career path somewhere? For sure, I wasn't aware of one.

I believe those are the 2 main reasons that fueled the creation of the software craftsmanship movement. Agile was in part a revolution against the essence of heavyweighted methodologies. A revolution that put the programming task as the most important task in all software development life-cycle again. The software craftsmanship movement took this revolution a step further. The actual task of coding the software is the most important task and it is not by any means a simple task. It is a task that requires understanding the business problem to solve, model it in the best possible way, program it using a programming language and by many programmers who all need to agree and implement the same model. To add more complexity, many times the business scenarios change and software needs to change, to adapt to those changes. University doesn't prepare us for this job and companies need developers developing as soon as possible.

The Software Crafstmanship Manifesto states the following:

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

Not only working software, but also well-crafted software

Not only responding to change, but also steadily adding value

Not only individuals and interactions, but also a community of professionals

Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

The first important thing to note in this manifesto is that it treats software programming as a craft. A craft is by the dictionary "an art, trade, or occupation requiring special skill, especially manual skill". A clear example of a craft is a carpenter. And how do you learn the carpenter's craft? First you start besides a carpenter as an apprentice and follow him, for a lot of time. Then you start doing small things on your own. And after a long time you master that craft and are able to perform it on your own. Look how different it looks from the path that many of us have followed. Also note that a craft is something that needs to be practiced. Software craftsmen practice their profession not only daily at work, but also through code katas and getting together to improve their craft in code retreats.

Look at the values now. Not just software, but well crafted software. Although some people may say that well crafted software is not the goal (producing value is), software craftsmen know that the best way of producing value is to produce good software,software that is robust, adaptable and can keep growing and producing value for a long period of time. I believe the third value indicates what should be the way to achieve this. A community of professionals. Through the software craftmanship community, I believe that now it's possible to follow a career path that takes you from apprentice to good developer. Professionals on this community are very open to teach what they know and help others become good professionals.

The software craftsmanship movement provided a new perspective to our profession of software programmers. It stressed the importance of programming and also of becoming a good programmer. And it also indicated the way to achieve this. A programmer should start as an apprentice who learns from someone that knows the craft. With practice, and more practice, this apprentice will learn the craft and become a good programmer.

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa

2011-07-04T06:02:00.000-07:00

Code & Beyond: Video: ¿Qué esperas de Agile? con Federico Zuppa: "Este video es de una de las reuniones mensuales de Agiles @ Buenos Aires , pero me había quedado en el tintero procesarlo y publicarlo, así..."

The Lean Tribe

2011-06-15T15:13:00.000-07:00

The Agile Manifesto triggered the growth of "the so called until that moment lightweight methodologies". Scrum and XP became very popular and the tribes associated to them grew with them. However, not long after the Agile Manifesto was signed a new actor entered the scene with outreagous strength. His friends use to call him "Lean" :-)

So what is Lean? Lean has its roots in the Toyota Production System developed by Taichi Ohno. Its core idea is to maximize customer value while minimizing waste.

Lean was brought into the Agile world by Mary and Tom Poppendieck who in their book "Lean Software Development: An Agile toolkit" translated 7 lean principles used in manufacturing to software development language.

Although Lean and Agile share many of the same values and principles as they both are people centric and empower people and they both try to adapt/improve the process continually, they also have some differences as well. The main one in my opinion is its main objective: Agile was developed mostly by software developers and therefore its main objetive is "to uncover better ways of developing software". Lean has in its essence a broader objective (and a broader target) as it seeks to improve the flow of value and this could only happen at an organizational level.

The Agile world was greatly modified by the entrance of Lean. So much that probably now it is an Agile/Lean world.

So who are the referents of this new Lean tribe. Where do they get together. Do they breed with Agile folks? :D I know I may be missing many of the referents of the manufacture industries but these are the ones I know:

- Mary and Tom Poppendieck
- Eli Goldratt (RIP)
- David Anderson
- Karl Scotland
- Alan Shalloway

These folks get together in conferences like "Lean Software and Systems" (in its website, you'll be able to find all the speakers that attended in 2011)

In the last few years, the Lean tribe grew incredibly fast. Particularly, a lean tool called Kanban has gained a great deal of attention and is becoming the new "Scrum" of the moment (or the new star of the moment).

How is the relationship between Agile and Lean? Well, I believe that although there have been some fights (or discussions should I say) because sometimes tribes defend their interests, the Agile community has a lot to learn from the Lean community. Although in the Agile world, Lean is new, it really is much older than Agile.

Agile Tribes

2011-05-31T08:19:00.000-07:00

The people that got together 10 years ago in Salt Lake City had different backgrounds, specialties, positions, etc.There were some folks already working on their own methodologies, such as Beck with XP, Schwaber and Sutherland with Scrum and Cockburn with Crystal. Some of them were more technical, specializing in OO, programming and testing while some others were managers or people more interested in the methodology aspects of software development. This is more or less what I believe were the backgrounds that depict the folks that were there:

Now I wonder how these 'tribes' (or communities or subfields of interests - I wonder what is the best name) develop over the years. How did they grow or mutated? Which are the new tribes that make up the Agile world?. Who are the new referents of these tribes? Will you help me create this map?
How do I define what a tribe is? Let's list some items that define a tribe:

They share the same values and principles (which should enter the big Agile umbrella)
They work with a certain methodology or method
They have some referents
They get together in one of more congresses (this could be a consequence of the previous)

Am I missing something to define a tribe? Hummm, yeah for sure. Will see it as we go (iteratively). For the moment, I will research for each tribe the following items:

Reason to be (values/principles they share, objectives)
History
Referents
Present/Future?

Now I just need to select the most important tribes in today's Agile map. I will start with these ones:

Kanbaners
Software Craftsmen
Lean-startupers
Scrumers
XPs

I will focus on the first 3 groups, as the last 2 exist since the beggining of Agile.
I'd love to receive suggestions/corrections, etc.

log4jdbc: Una herramienta muy útil en desarrollo

2011-05-23T13:40:00.000-07:00

Me estaba preguntando si existía una herramienta que permita ver los queries que se ejecutan contra la DB cuando se usa SimpleJdbcTemplate en Spring. Y.. existe. Se llama log4jdbc y la instalación es sencillisima.
Una vez instalado, en el log de la aplicación se podrán ver los queries y también el tiempo de ejecución.

(       45728) [http-8400-3         ] INFO select DISTINCT location_code from PURCHASE_LINES where number = 'XXX'
                              | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:223) [2011-05-23 21:32:21,341]
(       46995) [http-8400-3         ] INFO select DISTINCT codes from PURCHASE_LINES where po_number = 'XXX'
{executed in 1267 msec}      | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:322) [2011-05-23 21:32:22,608]
(       46997) [http-8400-3         ] INFO 5. ResultSet.new ResultSet returned                                                                                       | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,610]
(       46998) [http-8400-3         ] INFO 5. PreparedStatement.executeQuery() returned net.sf.log4jdbc.ResultSetSpy@159d796                                         | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,611]
(       46999) [http-8400-3         ] INFO 5. ResultSet.next() returned true                                                                                         | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,612]
(       47000) [http-8400-3         ] INFO 5. ResultSet.getMetaData() returned oracle.jdbc.driver.OracleResultSetMetaData@19e80b1                                    | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,613]
(       47001) [http-8400-3         ] INFO 5. ResultSet.getString(1) returned 10176 KKKKKK                                                                      | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,614]
(       47002) [http-8400-3         ] INFO 5. ResultSet.next() returned true                                                                                         | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,615]
(       47003) [http-8400-3         ] INFO 5. ResultSet.getMetaData() returned oracle.jdbc.driver.OracleResultSetMetaData@113126e                                    | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,616]
(       47004) [http-8400-3         ] INFO 5. ResultSet.getString(1) returned SSSSSSS                                                                           | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,617]
(       47005) [http-8400-3         ] INFO 5. ResultSet.next() returned true                                                                                         | at net.sf.log4jdbc.Slf4jSpyLogDelegator(Slf4jSpyLogDelegator.java:158) [2011-05-23 21:32:22,618]

Pretty cool, ah?

Inyectando un Spring Service con PowerMock + Mockito

2011-04-08T08:57:00.000-07:00

Ayer a la tarde estaba intentando hacer un test para nuestra aplicación que usa Spring en el backend. Una de las cosas que me di cuenta que cambiaron desde la ultima vez que use Spring es que todos los beans son inyectados automagicamente con la anotación @Autowire, por lo que no es necesario ni tener un constructor ni un setter para inyectar una dependencia. La pregunta ahora es como hago para testear estos beans? (para inyectar mocks en estos beans). Dps. de investigar un rato encuentro la libreria PowerMock que permite hacer cosas locas, como stubbear un metodo estático u obtener/setear el estado interno de un bean (rompiendo asi la encapsulación). Por ejemplo, la sintaxis para setear una variable privada es la siguiente:

UserRepository userRepositoryMock = mock(UserRepository.class); // declaro el mock, usando mockito

when(userRepositoryMock.safeGetLoggedUser()).thenReturn(adminUser); // seteo las expectativas

Whitebox.setInternalState(invoiceRepository, "userRepository", userRepositoryMock); // inyecto el mock, usando Powermock

Cual es el problema con el que me tope? Spring utiliza proxies para manejar las transacciones via AOP y estos proxies implementan la interfase del servicio, que no contiene los setters para sus campos. Osea que si quisieramos hacer lo mismo para un servicio (en vez de un repositorio como en el ejemplo),

Whitebox.setInternalState(scannedFileService, "userRepository", userRepositoryMock); // inyecto el mock, usando Powermock

PowerMock se quejaría (porque estamos queriendo llamar un método en una clase que no tiene ese metodo). La exception que arroja es la siguiente:

Caused by: java.lang.RuntimeException: You want me to set value to this field: 'userRepository' on this class: 'Object' but this field is not declared withing hierarchy of this class!
    at org.mockito.internal.util.reflection.Whitebox.getFieldFromHierarchy(Whitebox.java:40)
    at org.mockito.internal.util.reflection.Whitebox.setInternalState(Whitebox.java:25)
    ... 31 more

Esto se podría arreglar agregando el setter a la interfase del servicio, pero la verdad es q no me parecia limpio tener q modificar la interfase publica solamente a efectos de testear. Por suerte, existe una solución. Investigando un poco mas, encontre este foro que indica como hacer con Spring para obtener el objeto 'aconsejado' (advised) desde el proxy.

Advised advised = (Advised) scannedFileService;

ScannedFileService target = (ScannedFileService)advised.getTargetSource().getTarget();

Whitebox.setInternalState(target, "userRepository", userRepositoryMock);

et voila! Una vez obtenido el objeto aconsejado, powermock se encarga del resto. Espero sus comentarios (les parece un overkill? Agregarian los setters directamente? etc.)

Agrego dependencias de powermock en maven para aquellos que son lentos como yo y les cuesta encontrar las dependencias :-).


                org.powermock.modules
                powermock-module-junit4
                1.3.9
                test


                org.powermock.api
                powermock-api-mockito
                1.3.9
                test

Are Level 5 CMMI Process Areas compatible with Agile?

2011-02-22T06:04:00.001-08:00

Introduction

After a quantitatively manage organization, the next objective is an optimizing quantitatively managed organization. An organization that is constantly improving. CMMI level 5 process areas deal with this. Improvement, change, should be done (otherwise, the organization is left behind by others) but it should be managed with care, not to overwhelm the organization.

The first process area is Organizational Innovation and Deployment (OID). Its goals and practices represent a measured way of selecting and deploying improvements. The second process area is called Causal Analysis and Resolution (CAR) and aims at resolving the root causes of defects for ever.

The specific goals and practices for OID are:

SG 1 Select Improvements

SP 1.1 Collect and Analyze Improvement Proposals

SP 1.2 Identify and Analyze Innovations

SP 1.3 Pilot Improvements

SP 1.4 Select Improvements for Deployment

SG 2 Deploy Improvements

SP 2.1 Manage the Deployment

SP 2.3 Plan the Deployment

SP 2.2 Measure Improvement Effects

and for CAR are:

SG 1 Determine Causes of Defects

SP 1.1 Select Defect Data for Analysis

SP 1.2 Analyze Causes

SG 2 Address Causes of Defects

SP 2.1 Implement the Action Proposals

SP 2.2 Evaluate the Effect of Changes

SP 2.3 Record Data

Observations

Both of these processes are designed to work in a quantitatively managed organization. CMMI states that if it isn’t quantitatively managed, it would work, but it wouldn’t be as effective. I wondered when I started reading the book if optimization wasn’t left for too late in CMMI. In one side, how would you optimize if you can’t measure? On the other, my experience with Agile is that optimization comes from the very first moments, at the standup and in the retrospective meetings where opportunities for improvement are detected. Of course all these optimizations are really subjective, as the improvement they provide can’t really be measured.

Another point that I wanted to make is that CMMI mentions that everyone should be empowered to propose optimizations. This goes very much in line with the lean approach to optimization.

Conclusion

I didn’t see any point of incompatibility in these areas. Having a framework in the organization to do Kaizen is very important for the organization as it allows to optimize with the organization’s objectives (i.e. with everyone aligned) , it allows to do it in an orderly fashion and it allows to really know what is been optimized and how much.

Are Level 4 CMMI Process Areas compatible with Agile?

2011-02-19T05:11:00.000-08:00

Introduction

I merged the process areas in level 4, because they are really inter twinned and probably don’t make sense to do one and not the other. The first one, the Organizational Process Performance aims at “establish and maintain a quantitative understanding of the performance of the organization’s set of standard processes”. Based on these baselines, the Quantitative Project Management process area sets the objectives and “quantitatively manage the project’s defined process to achieve the project’s established quality and process-performance objectives.

The Organizational Process Performance goals and practices are listed below:

SG 1 Establish Performance Baselines and Models

SP 1.1 Select Processes

SP 1.2 Establish Process-Performance Measures

SP 1.3 Establish Process-Performance Baselines

SP 1.5 Establish Quality and Process-Performance Objectives

SP 1.4 Establish Process-Performance Models

The Quantitative Project Management goals and practices are listed below:

SG 1 Quantitatively Manage the Project

SP 1.1Establish the Project’s Objectives

SP 1.2 Compose the Defined Process

SP 1.3 Select the Subprocesses that Will Be Statistically Managed

SP 1.4 Manage Project Performance

SG 2 Statistically Manage Subprocess Performance

SP 2.1 Select Measures and Analytic Techniques

SP 2.2 Apply Statistical Methods to Understand Variation

SP 2.3 Monitor Performance of the Selected Subprocesses

SP 2.4 Record Statistical Management Data

Metrics? What metrics?

In CMMI Level 4, the organization reaches a level where the existing processes can be quantitatively measured. Objective objectives can be set based on these measures. Being able to measure is vital to being able to manage and optimize. Jurgen published a post recently that states “You cannot manage what you don’t measure”.

A very important point to keep in mind when selecting what to measure is that these metrics need to be compatible with the Agile values and practices. Wrongly selected metrics may create dysfunctional behaviours, detrimental to the organization’s objectives. Dave Nicolette has a very nice presentation regarding Agile metrics. Some of the metrics he discusses are “Running Tested Features”, “Hard Financial Value” and “Earned Business Value”. I wonder, with these metrics, how to handle at the organizational level the context differences among the projects. I mean, each project is different, runs under different circumstances, will deliver business value in a different way, etc. All these factors need to be considered to draw conclusions from the metrics.

These metrics need to provide the picture needed by upper management to take decisions and manage risk appropriately. Being able to visualize the processes used, detect and resolve problems is of utmost importance in mature organizations.

Can everything be measured?

There’s a nice quote by Albert Einstein in Dave’s presentation that says “Not everything that counts can be counted, and not everything that can be counted counts.”. Is there place for subjectivity in a mature organization? How and where would it work? Agile methods stimulate optimization since the very beginning and this is performed, in most cases, subjectively. Teams introspect on the previous iteration, try to determine how they feel and propose ideas that would optimize the process. At upper levels, it would be impossible to optimize this way. Managers need to see the broad view and the only way seems to be through measures.

Keep it light!

I must confess that after reading both process areas in level 4, I ended up a bit scared. It really sounds difficult and requirements seem pretty heavy. Agile processes are characterized by their minimum overhead. The machinery required to measure and optimize processes shouldn’t increase this overhead significantly. For this, measures should be taken in the most automatic way, only what is needed should be measured and when it’s not needed anymore it should be dropped. There should be constant improvement in the improvement processes.

Conclusion

Being able to measure, to understanding where it is standing allows an organization to set improvement objectives. These measures need to be align with the Agile philosophy. The improvement machinery should be kept light.

Is the Decision Analysis and Resolution Process Area compatible with Agile?

2011-02-16T05:19:00.000-08:00

Introduction

The Decision Analysis and Resolution process area, a process area at level 3, aims at “formal evaluation process that evaluates identified alternatives against established criteria.”

Below are the specific goals and practices:

SG 1 Evaluate Alternatives

SP 1.1 Establish Guidelines for Decision Analysis

SP 1.2 Establish Evaluation Criteria

SP 1.3 Identify Alternative Solutions

SP 1.4 Select Evaluation Methods

SP 1.5 Evaluate Alternatives

SP 1.6 Select Solutions

Inclusive and Fair

The process should be inclusive and fair.

Inclusive: All stakeholders should be involved in the decision. With this, I am not saying that all stakeholders need to agree on the solution. Probably this is impossible or too costly. I am just saying that everyone should be able to participate in the decision process, express their opinions, being heard. A good facilitator will increase the chances of everyone being heard and of taking a good decision. Being included in the decision process provides great encouragement to follow through the selected paths. Collaborative decision making is of utmost importance in Agile.

Fair: the process should be fair. Every person in the team should feel that it was not an arbitrary decision imposed. The decision was taken using a process that they understand and consider fair. Of course, not everyone will agree, even if a very fair process is used. However, even not agreeing, if they consider that the process was fair, they will accept. On the contrary, if the feeling is the decision was imposed through a not so transparent process, the decision taken will be very difficult to accept. An excellent paper regarding this topic can be found here.

Conclusion

I believe it would be very beneficial to have a defined formal process that can be used to take difficult decisions. To be more Agile, the process should be inclusive and fair (which are really very inter twinned)

Is the Risk Management Process Area compatible with Agile?

2011-02-15T05:54:00.000-08:00

The Risk Management process area, a process area at level 3, aims at foreseen risks so that they can be mitigated in case they occur.

Below are the specific goals and practices:

SG 1 Prepare for Risk Management

SP 1.1 Determine Risk Sources and Categories

SP 1.2 Define Risk Parameters

SP 1.3 Establish a Risk Management Strategy

SG 2 Identify and Analyze Risks

SP 2.1 Identify Risks

SP 2.2 Evaluate, Categorize, and Prioritize Risks

SG 3 Mitigate Risks

SP 3.1 Develop Risk Mitigation Plans

SP 3.2 Implement Risk Mitigation Plans

Is there the need of a risk management process area in an Agile organization?

I have heard this question a lot in organizations that transition to Agile: How does Agile handle risk management? This question usually comes from project managers used to Waterfall methodologies where maintaining a list of potential risks and how to mitigate them was of utmost importance. Why? Because these risks appeared late in the life of the project, when a good percentage of the budgeted money has been expended and there wasn’t much space to take decisions. Been able to deal with these risks at that moment decided the fate of the project. Agile approaches risk management differently. Agile tries to deal with risks as early as possible or as soon as risks appear. The risk that a user story entails is a decisive factor in the prioritization of this user story. If a new risk is detected, constant reprioritization assures that it will be dealt with soon. With this approach, the fate of the project is decided when there is still room to maneuver and money to do it. Instead of trying to foresee risks in the future and imagine solutions, risks are aggressively attacked. A side effect of this approach is the infrastructure needed to deal with risks is lighter than the one needed in Waterfall. Would you need a list of risks if you know that the top priority user stories are attacking the same set of risks? Not too sure. Would you try to imagine how to deal with them if they are going to be tackled by the team in the next iteration? No need to draw in the air, right? The processes and practices in Agile make up the risks management infrastructure.

Conclusion

Risk Management is embedded into the process in Agile. Therefore, I am not sure this area would be too valuable.

Is the Integrated Project Management Process Area compatible with Agile?

2011-02-14T07:53:00.000-08:00

Introduction

The Integrated Project Management process area, a process area at maturity level 3, aims at establishing and managing projects “according to an integrated and defined process that is tailored from the organization’s set of standard processes.”

Below are the specific goals and practices:

SG 1 Use the Project’s Defined Process

SP 1.1 Establish the Project’s Defined Process

SP 1.2 Use Organizational Process Assets for Planning Project Activities

SP 1.3 Establish the Project's Work Environment

SP 1.4 Integrate Plans

SP 1.5 Manage the Project Using the Integrated Plans

SP 1.6 Contribute to the Organizational Process Assets

SG 2 Coordinate and Collaborate with Relevant Stakeholders

SP 2.1 Manage Stakeholder Involvement

SP 2.2 Manage Dependencies

SP 2.3 Resolve Coordination Issues

Can the Organization knowledge model be more people oriented?

CMMI defines a level 2 organization as one that can plan and manage their projects, but there isn’t a unified view (in the organization) on how to do it. In the next level, there’s a set of defined processes that individual projects use, tailoring them to suit their specific needs. Therefore, the company has a centralized, unified view on how to plan and manage projects. The flow repository -> project isn’t unidirectional. As the project ‘learns’, this knowledge is pushed back to the central repository. This model not only allows to have a unified view, but also allows reusing and learning inside the organization. I published in previous posts that I wasn’t sure that all knowledge needed to develop software could be contained in a central repository. I still recognized that maintaining a central repository that allows learning at the organizational level could be very beneficial. It’s hard for me put up in words what I think it would be missing in an Agile environment to complement and improve this learning model. I believe it may be this culture where more senior people seat with junior people to program or to resolve a difficult issue. Perhaps is just that the CMMI is a model centered around processes while Agile in centered around people. Should this learning process be shifted towards people in an Agile organization? How should it be done?

Conclusion

Having worked in many companies, I always have the impression that each project has its own life (use its own processes, tools, etc.). This shouldn’t be the case in a CMMI level 3 organization. The organization’s knowledge should be leverage inside the project, and the projects learnings should leverage the organization’s one. Can this model be shifted towards a more humanistic, not so process oriented one, to make it more compatible with Agile?

Other Areas

- Is the Organizational Process Definition Process Area compatible with Agile?

- Is the Organizational Process Focus Process Area compatible with Agile?- Is the Validation Process Area compatible with Agile?
- Is the Verification Process Area compatible with Agile?- Is the Product Integration Process Area compatible with Agile?
- Is the Technical Solution Process Area compatible with Agile?
- Is the Requirements Development Process Area compatible with Agile?
- Is the Configuration Management Process Area compatible with Agile?

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

- Is the Requirements Management Process Area compatible with Agile?
- Is the Project Monitoring and Control Process Area compatible with Agile?
- Is the Project Planning Process Area compatible with Agile?

The 7 Duties of Great Software Professionals

2011-01-30T18:19:00.001-08:00

Check out this SlideShare Presentation:

The 7 Duties of Great Software Professionals

View more presentations from Jurgen Appelo.

Is the Organizational Process Definition Process Area compatible with Agile?

2011-01-30T14:38:00.001-08:00

Introduction

The Organization Process Definition process area, a process area at maturity level 3, aims at establishing and maintaining a usable set of organizational process assets and work environment standards.

Below are the specific goals and practices:

SG 1 Establish Organizational Process Assets

SP 1.1 Establish Standard Processes

SP 1.2 Establish Lifecycle Model Descriptions

SP 1.3 Establish Tailoring Criteria and Guidelines

SP 1.4 Establish the Organization’s Measurement Repository

SP 1.5 Establish the Organization’s Process Asset Library

SP 1.6 Establish Work Environment Standards

Would the process asset repository be effective in an Agile environment?

No doubt a process asset repository is vital in a framework like CMMI that gives processes so much importance (see section ‘About Capability Maturity Models’ in the CMMI). But would it be the same in an Agile organization? I guess my question is: Can a process repository contain all the necessary knowledge needed to build software successfully? I believe software development cannot be performed just by following a set of processes. Building software requires other things as well, such as creativity, trade-off balancing and software craftsmanship. But then.... how can a software company build and maintain such a knowledge repository that lives and dies in the employee’s minds? Well, I don’t know. What I do know and saw in Agile companies is a culture of sharing the knowledge and learning together. Pair programming and code review practices allow more senior members to spread the knowledge on how things are done in the company. Are there any other ways? Particularly, ways that allow the organization to stick with the knowledge? (after all, the CMMI is right when it states that “people typically work for many companies throughout their careers”). I don’t know again. After all, we are departing from a point where we think that knowledge workers are irreplaceable, right? Probably there isn’t.

Flexibility versus Consistency

CMMI tackled an issue that I always had in my mind, but never knew how to express correctly. Flexibility goes against consistency. CMMI deals with it by defining a set of tailoring guidelines. But, can we predict all future scenarios to be able to create the tailoring guidelines? I believe that breaking rules comes with maturity. In the ‘Ri’ level of the Shu Ha Ri, very mature teams don’t go by the rules. They understand the rationale of each of them and are able to break them, based on the current needs. But to get there, all the previous paths need to have been traversed. The rationale of the rules need to be understood and the consequences of breaking them too.

Conclusion

I am not entirely sure that a process repository would be as effective in software engineering as it is in other harder engineerings. Also, I believe that tailoring a process can be performed not with tailoring guides, but with the understanding of the particular context and a thorough undestanding of the rationale of the existing processes.

Is the Organizational Process Focus Process Area compatible with Agile?

2011-01-25T05:32:00.000-08:00

Introduction

“The purpose of Organizational Process Focus (OPF) is to plan, implement, and deploy organizational process improvements based on a thorough understanding of the current strengths and weaknesses of the organization’s processes and process assets.”

Below are the specific goals and practices:

SG 1 Determine Process Improvement Opportunities

SP 1.1 Establish Organizational Process Needs

SP 1.2 Appraise the Organization’s Processes

SP 1.3 Identify the Organization's Process Improvements

SG 2 Plan and Implement Process Improvements

SP 2.1 Establish Process Action Plans

SP 2.2 Implement Process Action Plans

SG 3 Deploy Organizational Process Assets and Incorporate Lessons Learned

SP 3.1Deploy Organizational Process Assets

SP 3.2 Deploy Standard Processes

SP 3.3 Monitor Implementation

SP 3.4 Incorporate Process-Related Experiences into the Organizational Process Assets

How would an Agile organization do process improvement?

My experience in the Agile world has always been in particular projects. This means we were doing Agile in those projects, but the organization as a whole was not Agile (Probably this is the case for the majority of the Agile projects …?) Why am I saying this? Well, because I was thinking how should process improvement at the organizational level should be performed in an Agile organization, but I am a complete ignorant in this regard. Anyway, If I think how we do process improvement in a particular project with Agile and I scale it to the organization, probably the result would be something very similar to what the CMMI proposes. First, determining the process improvement opportunities (retrospection), then make a plan, deploy it and monitor how it is going.

What would change at the organizational level?
1. Organizational level goals may not coincide %100 with project level goals: Cockburn says they are 2 different games, the project one and the organization one (and the one that the employees are playing). I believe that having to fulfill a process area like this one may really help an Agile organization align its projects.
2. Implementation is much harder: Depending on the organization, teams are different and have different contexts. The problem is much more difficult to resolve.
3. Need for a specific team to handle this: this may be a consequence of the previous point. The CMMI says and I agree, that a specific team that handle process improvement in the organization is needed.

There’s something I like to add to the previous points. The CMMI does never mention the employees as a source for process improvement initiatives and underestimate the importance of them in the implementation in my opinion. I believe that team empowerment is vital to any process improvement initiative. As the Poppendieck’s stated in their book “The people on the front line combine the knowledge of the minute details with the power of many minds”. I believe this is also related with the mentality needed for Kaizen. If all employees are constantly thinking how to improve the current processes, the results may be amazing.

Conclusion

I believe the CMMI goals and practices are completely compatible in this area. Knowledge from the lean methods of process improvement should be incorporated to make the process area even better.

Other Areas

- Is the Validation Process Area compatible with Agile?
- Is the Verification Process Area compatible with Agile?- Is the Product Integration Process Area compatible with Agile?
- Is the Technical Solution Process Area compatible with Agile?
- Is the Requirements Development Process Area compatible with Agile?
- Is the Configuration Management Process Area compatible with Agile?

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Validation Process Area compatible with Agile?

2011-01-19T07:10:00.000-08:00

Introduction

The Validation process area, a process area at maturity level 3, aims at “demonstrating that a product or product component fulfills its intended use when placed in its intended environment.”

Below are the specific goals and practices:

SG 1 Prepare for Validation

SP 1.1 Select Products for Validation

SP 1.2 Establish the Validation Environment

SP 1.3 Establish Validation Procedures and Criteria

SG 2 Validate Product or Product Components

SP 2.1 Perform Validation

SP 2.2 Analyze Validation Results

There are a couple of points to address in this area:

What is Validated?
The CMMI lists a number or work products where validation can be performed. However, using an incremental approach for software development, the only work product that I conceive for validation is the final product, the increment that has been delivered. Being able to perform validation early on the final product is one of the distinctive characteristics of Agile methodologies.

What is the difference between Validation and Verification?
The CMMI states that Verification is the “you built it right” and Validation is the “you built the right thing”. In Agile, both process areas appear very intertwined, probably because both are performed at the same time or because requirements/user stories are not carved in stone and communication is so important to build it right. Thinking a bit about it, I concluded that Validation goes a bit further than Verification. A customer can validate an increment in functionality, beyond what the requirement for that functionality specified. In other words, a product owner (a good one) can judge the result of building a user story, learn from this result and make changes to improve it. This means Validation. Verification is the task of the Quality Assurance members that try to refine a user story, finding errors or discrepancies, but their only point of reference is what the customer expressed in the user stories or conversations (he’s not in the customer’s mind). In any case, I am not sure that both process areas are necessary in Agile.

Self-Verification or Self-Validation?
See, it’s hard for me to see the difference. Probably, Verification is related with the technology facing testing quadrants in Marick’s testing quadrants while Validation is related with the business facing quadrants. If this is the case, probably self-verification is more important than self-validation. With Validation, as I pointed previously, there’s a component of human intervention that won’t be automatable (exploratory and usability testing always have a big human component). Anyway, there is a big chunk of Validation work that can and should be automatable . In BDD, the customer specifies what he needs, the validation criteria as executable specifications. Doing this specification probably diminishes a lot of the subjectivity in the customer’s mind.

Conclusion

It is hard for me to see the difference between Validation and Verification. If a Waterfall development process is followed, it would be much easier because there is a big period of time where only Verification is performed (the team members trying to detect if what is being built matches the requirements). Validation is performed only in the end by the customers. In Agile, both tasks are performed concurrently and immediately. This diminishes the value of the Verification and increases the value of the Validation. This is what I say that I would probably merge both areas in Agile, performing only Validation. What do you think?

Is the Verification Process Area compatible with Agile?

2011-01-16T07:39:00.000-08:00

Introduction

The Verification Process Area, a process area at maturity level 3, aims at “ensuring that the select work products meet their specified requirements”. The specific practices on this process area build on each other in the following way:

1. Select Work Products for Verification
2. Establish the Verification Environment
3. Establish Verification Procedures and Criteria
4. Perform Verification

Below are the specific goals and practices:

SG 1 Prepare for Verification

SP 1.1 Select Work Products for Verification

SP 1.2 Establish the Verification Environment

SP 1.3 Establish Verification Procedures and Criteria

SG 2 Perform Peer Reviews

SP 2.1 Prepare for Peer Reviews

SP 2.2 Conduct Peer Reviews

SP 2.3 Analyze Peer Review Data

SG 3 Verify Selected Work Products

SP 3.1 Perform Verification

SP 3.2 Analyze Verification Results

There are 4 points I would like to discuss on this process area:

How is Verification performed in Agile?
Verification is performed in small chunks, continually and needs constant readjustment due to the iterative nature of the development process. The CMMI states that Verification is an “incremental process because it occurs throughout the development of the product and work products, beginning with verification of the requirements, progressing through the verification of the evolving work products, and culminating in the verification of the completed product.”. However, it doesn’t mention the need of constant readjustment, which is perhaps due to the fact that Validation is performed concurrently with Verification, turning Verification into a very dynamic task.

What’s the meaning of building the “thing right” in Agile?
CMMI states that the Verification is the “you built it right”, whereas the Validation ensures that “you built the right thing.”. I must say that having spent the majority of my professional life in Agile environments, I never heard the “you built it right” applied to how well it “fulfills the requirements”. I heard that concept applied to the internal quality of the system. How well designed, tested, maintainable and robust is the system. I guess it makes sense to say “you built it right” when the implementation of a user story correctly addressed the written user story. The thing is that this whole cycle of starting with the user story , building it and immediately verifying it is so fast, and there may be so many changes in between, that the whole concept of you build it right looses semantic.

What is an Agile Peer Review Process?
The second goal of this process area talks about peer review, as one of the verification mechanisms available. How should a peer review process be implemented in Agile. I believe that every commit should be reviewed. Reviewing %100 of the code assures that every programmer codes with attention, and that always at least 4 eyes look at the code before being committed to the mainline. Code should be committed at least once a day, to perform continual small integrations. This requires that the review process is very light, fast and effective. How can this be achieved? I believe in informality, mixed with discipline and trust to perform the peer review. First, the criteria should be very clear to anyone. Then, there should be trust and people should be able to discuss and change the code based on the review, without getting mad or getting hard feelings. In an ideal situation, I believe it should be performed face to face, at the moment the developer finish and wants to commit (of course, after having verified that all tests are running, the are no style errors and whatever else is on the review criteria).

Self-Verifying Code
The iterative and incremental nature of Agile turns self-verifiable code or self-tested code into a necessity. If only manual testing is performed, the project is not sustainable in the long run as it becomes too expensive to re-verify all features every iteration.

Conclusion

I see the verification part of a user story as another step towards refining a requirement and delivering what the customer is expecting. This whole process needs to be very light and dynamic. It needs to be very Agile, adapting continually to increase the probabilities of delivering a product that satisfies the user. Automation is vital to sustainably verify all the code every iteration.

Other Areas

- Is the Product Integration Process Area compatible with Agile?
- Is the Technical Solution Process Area compatible with Agile?
- Is the Requirements Development Process Area compatible with Agile?
- Is the Configuration Management Process Area compatible with Agile?

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Product Integration Process Area compatible with Agile?

2011-01-14T04:58:00.001-08:00

Introduction

The Product Integration Process Area, a process area at maturity level 3, aims at “assemble the product from the product components, ensure that the product, as integrated, functions properly, and deliver the product.”

Below are the specific goals and practices:

SG 1 Prepare for Product Integration

SP 1.1 Determine Integration Sequence

SP 1.2 Establish the Product Integration Environment

SP 1.3 Establish Product Integration Procedures and Criteria

SG 2 Ensure Interface Compatibility

SP 2.1 Review Interface Descriptions for Completeness

SP 2.2 Manage Interfaces

SG 3 Assemble Product Components and Deliver the Product

SP 3.1 Assemble Product Components

SP 3.3 Evaluate Assembled Product Components

SP 3.4 Package and Deliver the Product or Product Component

What should be the integration procedure suggested by Agile?

If the project is small and the contractual possibilities allow it, the recommended approach should be without any doubt to do Continuous Integration with all the practices suggested by Fowler in his great article (everyone committing to the same mainline, every day). What if the project is too big or a 3rd party team develops certain components of the system? Probably the different components are developed separately and then integrated. The question is how and how often.
One option is to study carefully the integration points before starting developing, then develop the different components (based on the documents created during the BUFD) and then integrating everything in the end. This approach has been proved too risky. No matter how well agreed and documented the integration points are, if there is a problem, it happens when the project has consumed most of its budget and changes are too expensive to be applied. The opposite option would be that all teams (belonging to all components) try to use the same mainline. Sometimes, this can be troublesome as it can happen that the different teams create problems between each other. So some separation, in these types of projects, could be valuable. However, I believe that the separation needs to be made following the Continuous Integration values. So if in a small project, the rule is that every developer integrates and commit every day to the mainline, the development of a big system could be done slicing the products in a way that many integration points are planned. Perhaps once a week or once a month, depending on the project. Of course this creates overhead, specially I can imagine if a lot of mocks and doubles needs to be constructed. However, the risk is also sliced and minimized throughout the project.

Does CMMI recommends one approach?

It is not clear to me, to be honest. In the description of the process area, it states that “Product integration can be conducted incrementally, using an iterative process of assembling product components, evaluating them, and then assembling more product components.”. However, again the sequence of goals would seem to suggest that a big analysis should be conducted at the beginning, then the interfaces should be managed and at the end the final product should be assembled, using the sub-components. This looks to me as goals that are more related with building a car than building software.

Conclusion

The Agile approach towards integration should be to it continually. How continually? As continually as it possibly can be performed. Remember that what needs to guide the decisions are the values and the principles. What is the rationale for doing Continuous Integration? How can it be applied to this particular project? These are the types of questions that should guide decisions.

Is the Technical Solution Process Area compatible with Agile?

2011-01-12T05:04:00.000-08:00

Introduction

The Technical Solution, a process area at maturity level 3, aims at designing, developing and implementing solutions to requirements.

Below are the specific goals and practices:

SG 1 Select Product Component Solutions

SP 1.1 Develop Alternative Solutions and Selection Criteria

SP 1.2 Select Product Component Solutions

SG 2 Develop the Design

SP 2.1 Design the Product or Product Component

SP 2.2 Establish a Technical Data Package

SP 2.3 Design Interfaces Using Criteria

SP 2.4 Perform Make, Buy, or Reuse Analyses

SG 3 Implement the Product Design

SP 3.1 Implement the Design

SP 3.2 Develop Product Support Documentation

There are a couple of points I’d like to discuss about this area:

1. The 3 specific goals distills a Waterfall process life cycle. After reading carefully the whole process area, I would say that the process suggested would look like this:

It selects the solution (SG1)
It creates lower lever requirements (SG1 - SP1.2)
It creates a broad design (SG2 - SP2.1)
And a detail design (SG2 - SP2.1)
It implements the design (SG3)

The ways it is written enforces this approach by, for example, specifying the role of the architect who “postulate and develop a model of the product, making judgements about allocation of requirements to product components”

A more Agile friendly maturity model should suggest the following steps in the process development life cycle:

It selects a solution
It creates a broad design: The architecture. As I said before, there’s a discussion on how much architecture is enough. For sure, in any Agile approach the architecture remains very broad, just with the most important structural components.
Design and implement top priority requirements: Detailed design is performed at the same time than implementation of the features.
Retrofit project: Designing and implementing a set of requirements increases the knowledge of the product. Requirements and existing design should be refined. This represents the iterative nature of Agile development.

2. The proposed method to the technical solution treats software development as another hard engineering. Let’s not forget that CMMI is used not only in software development process improvement efforts, but also in other harder engineerings. It was the belief a while ago (at the moment of the CMMI writting) that software development could be treated as other engineering fields. There is a very interesting discussion in Alistair Cockburn’s book about treating software development as other engineerings. Anyway, for now let’s just say that software uses a different material than what other engineerings use. Software can be changed more easily. Therefore, it isn’t necessary (or profitable) to have the whole design of the product made up front. The design can be evolved inexpensively (and cheaper if the team uses the practices suggested by Beck). This is completely different than, for example, the case where a bridge needs to be built. In the later, the whole design needs to be specified upfront and changes during the life of the project are very expensive, if not impossible.

Conclusion

Although not said explicitly, the nature of the technical solution process area goes against the nature of the technical aspects in Agile. I believe that trying to fullfill the objectives would lead to too big upfront designs, which in Agile are not recommended. The separation of roles between Architects (the people who take the big decisions up front) and the developers who just need to follow the detailed design provided to them goes in a complete different path than what Agile recommends. Treating Software Engineering as another hard engineer is also not appropriate.
Another thing that caught my attention is that the technical process area doesn’t say anything about good engineering practices. Of course, CMMI is too general to go into these details. However, I wonder if a useful software improvement model shouldn’t provide goals related to this (e.g. one that I would add is having a automated testing strategy, which defines what kind of automated tests are going to be developed during the project and how the product is going to be tested). As a final sentence, I feel this area too far from the Agile technical area.

Other Areas

- Is the Requirements Development Process Area compatible with Agile?
- Is the Configuration Management Process Area compatible with Agile?

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Requirements Development Process Area compatible with Agile?

2011-01-07T06:20:00.000-08:00

Introduction

The requirements development process area, a process area at maturity level 2, aims at producing and analyzing customer, product and product components requirements. CMMI states that requirements start as customer requirements (stakeholders expectations are expressed here) and are later refined into product and product components requirements (expressed in a more technical fashion) with the input of the technical members after analysis and design is performed.

Below are the specific goals and practices:

SG 1 Develop Customer Requirements

SP 1.1 Elicit Needs

SP 1.2 Develop the Customer Requirements

SG 2 Develop Product Requirements

SP 2.1 Establish Product and Product Component Requirements

SP 2.2 Allocate Product Component Requirements

SP 2.3 Identify Interface Requirements

SG 3 Analyze and Validate Requirements

SP 3.1 Establish Operational Concepts and Scenarios

SP 3.2 Establish a Definition of Required Functionality

SP 3.3 Analyze Requirements

SP 3.4 Analyze Requirements to Achieve Balance

SP 3.5 Validate Requirements

Is it Compatible?

CMMI states that stakeholders needs, expectations, constraints and limitations may be difficult to obtain and therefore it suggests an iterative process (very much like Agile). It also states the importance of having a customer surrogate, role that very much resembles that of the Product Owner. However, it also talks about refinement and allocation. I have 2 concerns about those.

Is there really the need to express requirements in a technical fashion? Is it advantageous?

Requirements in a software development project represent the common area of understanding between technical and business people. Refinement of Customer Requirements into Product Requirements implies having 2 sets of requirements and therefore the integrity of both have to be assured (i.e. assure that all customer requirements have been address in the product requirements, which if they are written technically will be inaccessible to business folks). This is not to say that requirements should not be refined. I believe they should, but the end result should be one set of requirements that all members of the project can understand. Some people could say that Customer Requirements may be expressed in a business language too difficult to understand for developers. However, is it possible to build a software system without the developers understanding the business logic of what they are building. I don’t think so. There is another issue that worries me about this refinement. All Agile requirements (User Stories) have some business value associated with them. Although sometimes it can get difficult to slice functionality small enough to enter into an iteration, the advantages of building tangible, visible functionality are many. Although CMMI doesn’t state anything about the business value of any type of requirements, I am afraid that this type of requirements may result in technical requirements without visible value. In any case, stating that all requirements introduce some new business value to the system is really important.

Is there really the need to allocate requirements to technical components?

Who is this useful for? When is it done? Requirements should express what needs to be done. They shouldn’t express how it is supposed to do it. Otherwise, the requirement is biasing the implementation. Traceability is one of the reasons that CMMI states for performing this allocation. However, as I said before I am not sure about the benefits of having this traceability that assures that requirements have been addressed, but not that they have been full filled. Besides, I reckon that this traceability may be too heavy to keep up to date. Instead, I believe that automated tests expressed in the business language provide the certainty that the requirement have been full filled while being always up to date.

Conclusion

Although requirements in Agile should be refined (user stories start as a point of conversation and grow afterwards), the common shared place of understanding is always the product backlog, which is unique set of requirements. Translating customer requirements brings more disadvantages than advantages in my opinion. Same with the allocation of requirements to technical components. Something that really caught my attention is that the CMMI doesn’t say anything about ordering requirements by priority. In Agile, this is a top of the shelf matter, vital to the success of the project.

Other Areas

- Is the Configuration Management Process Area compatible with Agile?

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Configuration Management Process Area compatible with Agile?

2011-01-04T04:02:00.000-08:00

Introduction

The Configuration Management Process Area, a process area at maturity level 2, aims at establishing and maintaining the integrity of work products. The process area deals with selecting the work products that compose the baselines, controlling changes and maintaining the integrity over these items and providing accurate status on this data.

Below are the specific goals and practices:

SG 1 Establish Baselines

SP 1.1 Identify Configuration Items

SP 1.2 Establish a Configuration Management System

SP 1.3 Create or Release Baselines

SG 2 Track and Control Changes

SP 2.1 Track Change Requests

SP 2.2 Control Configuration Items

SG 3 Establish Integrity

SP 3.1 Establish Configuration Management Records

SP 3.2 Perform Configuration Audits

Is it too Formal?

CMMI states that the first thing is to create a baseline, which is a “a set of specifications or work products that has been formally reviewed and agreed on, that thereafter serves as the basis for further development or delivery, and that can be changed only through change control procedures.” and then modify this baseline through change requests, which should be approved by all stakeholders. It also states that this process should be audited.

Is this really suggesting that ALL changes should go through a change request system? When the developers are working in the project, they should create a change request ticket to commit their code? When the Product Owner is completing a User Story that was missing some information, s/he should do it via a change request ticket? I believe this formality would take too much time to the activities that add value, hurting the agility of the project. Of course, I believe it is important to track everything under a source control system, to keep the history of all configuration items and to analyze the impact of all changes (specially to requirements). Nonetheless, this should be done in the lighter, more non-obtrusive way. For example, the comments written in the commit to the source control system are (or should be) more than enough information that explains this change. Changes (the history) to the product backlog should be kept automatically, without the Product Owner needing to do anything. Formality is replaced with efficiency and trust. The Configuration System method chosen for an Agile project should be light and efficient. It should keep all the necessary information without the users spending too much time maintaining it. Automation (again in this area) is vital. The easiest way to maintain the integrity of a product is to avoid human intervention in the creation of the product. If the product is created automatically (running a command) there is no room for human mistakes.

Conclusion

When I first read the title of the process area, I thought that this process area was going to be totally compatible. However, after I read it, I began having my doubts. Having to create and maintain a baseline could lead to an overhead that reduces agility. Managing the backlog through change requests is practically impossible. Nonetheless, the need to have all items under a configuration management system is unavoidable. Having all or most of the steps of the configuration management process automated is vital.

- Is the Process and Product Quality Assurance Area compatible with Agile?

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Process and Product Quality Assurance Process Area compatible with Agile?

2010-12-29T06:52:00.000-08:00

This is a special post number 50!! So....

Enough, let's go back to Agile and CMMI compatibility..

Introduction

The Process and Product Quality Assurance Process Area, a process area at maturity 2, aims at objectively evaluating projects for adherence to the standards processes and procedures. Objectivity is key. It can be achieved through an external group or through demonstration of objectivity.

Below are the specific goals and practices:

SG 1 Objectively Evaluate Processes and Work Products

SP 1.1 Objectively Evaluate Processes

SP 1.2 Objectively Evaluate Work Products and Services

SG 2 Provide Objective Insight

SP 2.1 Communicate and Ensure Resolution of Noncompliance Issues

SP 2.2 Establish Records

Is it compatible?

This process area aims at ensuring that each of the projects in the company follows the company’s defined processes and practices. Something that can make this process area difficult to apply in an Agile company is the empiricist nature of Agile methodologies. In one side, there’s a company defined methodology (and the willing to follow it represented in this process area) and on the other there’s the Agile encouragement to introspect on the methodology and modify things that are not working. Probably this goes back to the advantages and disadvantages that I mentioned in the introduction to this series of posts and also to the fact that Agile is a value system. Therefore (and again), probably it’s more important to ensure that the company’s values (which in an Agile company should be compliant with the Agile values) are followed than to ensure that a specific defined process is followed step by step.

Another thing I also wonder is what is the best mechanism to provide this insurance. The CMMI reckons that “Traditionally, a quality assurance group that is independent of the project provides this objectivity.”. However, it also states that “It may be appropriate in some organizations, however, to implement the process and product quality assurance role without that kind of independence. For example, in an organization with an open, quality-oriented culture, the process and product quality assurance role may be performed, partially or completely, by peers; and the quality assurance function may be embedded in the process.”. I believe that the second mechanism should be the one recommended in Agile environments. A mature Agile team is one that doesn’t of any external group to follow the processes and practices and that realizes when it’s incurring in technical debt and can recover and pay it. How does it get there? Through a leader embedded in the project. A leader is a software craftsman that teaches the rest of the group the practices and ensures that they are always followed, even in tempestuous times (when the only thing that matters is handling some software by the door!)

Conclusion

Exact adherence to processes shouldn’t be encouraged in Agile environments, but exact adherence to values should. The best mechanism to ensure quality are through leaders embedded in the projects that act as quality and process keepers.

Other Process Areas

- Is the Measurement and Analysis Process Area compatible with Agile?

Is the Measurement and Analysis Process Area Compatible with Agile?

2010-12-23T05:06:00.000-08:00

Introduction

The measurement and Analysis process area, a process area at maturity 2, aims at developing a measurement capability that is used to support management information needs. Measurement objectives based on identified information needs are defined and using these objectives, objective and quantifiable measures are defined. Mechanisms to collect and store the data should also be defined.

Below are the specific goals and practices:

SG 1 Align Measurement and Analysis Activities

SP 1.1 Establish Measurement Objectives

SP 1.2 Specify Measures

SP 1.3 Specify Data Collection and Storage Procedures

SP 1.4 Specify Analysis Procedures

SG 2 Provide Measurement Results

SP 2.1 Collect Measurement Data

SP 2.2 Analyze Measurement Data

SP 2.3 Store Data and Results

SP 2.4 Communicate Results

Are they compatible?

Agile possess a culture of continuous improvement, through continual inspection and adaptation. The concept of Kaizen, that comes from the Lean world is also very popular. The question is ‘how is this improvement attained?’ and ‘is it always aligned with the wider company objectives?’.

How is this improvement attained? How should be the process to improve? The mechanism suggested by CMMI is the only one I know. The objective is identified, measurements that tell us if the objective has been reached as well as a plan to get there are defined and then successive measures are taken through the path. Having a company wide process improvement framework like CMMI can help identify these objectives in a project more easily and also assures that the objectives selected are aligned with the wider company objectives. It can also help define the measures, which should be “precise, quantifiable measures”.

However, I have a couple of concerns related with setting objectives and trying to reach these objectives:

1. Not weighting properly the importance of the objective within the whole context of the project

Let me picture this with an example, for which I will use Highsmith’s Agile triangle.

We may set up our objective as “Improve prior levels of quality” (taken from the CMMI examples). What is a quantifiable precise measure for this objective? Probably most will say counting the number of raised bugs. Having no bugs is really wonderful, but having no bugs on a software that doesn’t do what the customer needs is really valuable?

2. Creating dysfunctional behaviours

The human beings find ways to reach the objective that don’t match exactly what the objective was created for. Having raised less bugs assures us that the software is more robust? Are we really doing things better? The human/social aspects could be tricky.

Another thing to keep in mind when devising measurements and processes to be used inside a project is that all these processes and measures are “waste” (things that don’t add value to the product). Of course, this could all be good waste, because it will allow the company to improve, to do things better in the future. Nonetheless, they should be treated as such and designed to be as light as possible. Probably, automation (again) is the key in this regard.

Conclusion

I believe that creating more formal mechanisms to improve, that are aligned with company objectives, could be beneficial to Agile projects which in general focus their improvements efforts informally and with a project scope. The real objectives and the human aspects should be taken into consideration to avoid optimizing in the wrong side. The improvements processes should be light and agile.

Other Process Areas

- Is the Requirements Management Process Area compatible with Agile?
- Is the Project Monitoring and Control Process Area compatible with Agile?
- Is the Project Planning Process Area compatible with Agile?

Is the Requirements Management Process Area compatible with Agile

2010-12-19T06:52:00.000-08:00

Introduction

The requirements management process area, a process area at maturity 2, deals with managing all requirements related to the project throughout its life.

Below are the specific goals and practices:

SG 1 Manage Requirements

SP 1.1 Obtain an Understanding of Requirements

SP 1.2 Obtain Commitment to Requirements

SP 1.3 Manage Requirements Changes

SP 1.4 Maintain Bidirectional Traceability of Requirements

SP 1.5 Identify Inconsistencies Between Project Work and Requirements

SG 1 Manage Requirements

The only objective in this process area states that requirements should be managed by doing the following:

Managing all changes to the requirements
Maintaining the relationships among the requirements, the project plans, and the work products
Identifying inconsistencies among the requirements, the project plans, and the work products
Taking corrective action

All of this is completely true in an agile environment. However, the practices recommended to get there is very different. Let’s analyze each of the practices.

CMMI lists the following as the recommended practices to manage requirements:

1. Obtain an understanding of the requirements. When and how complete? Well, the objective states that “The project maintains a current and approved set of requirements” so my assumption is this understanding is complete.
2. Obtaining commitment to Requirements. From those “who have to carry out the activities necessary to implement the requirements.”
3. Manage Requirements Changes. Analyze impact and document rationale.
4. Maintain Bidirectional Traceability of Requirements. From requirements to source code and vice-versa. This traceability helps determine requirements have been completely addressed.
5. Identify Inconsistencies Between Project Work and Requirements.

Requirements are managed differently in Agile. Let’s see how the practices would change:

1. The project backlog doesn’t represent a current approved set of requirements. Requirements in a backlog have different levels of understanding and approval. Requirements live their life in the backlog with different levels of understanding and approval. They are born as ideas, “topics of conversation” and grow their understanding and approval, even after they are implemented. The driver in this understanding is their priority. More important requirements need to be understood/approved first. There could be phases in the form of releases in the backlog that will determine the understanding and approval of a set of requirements.
2. It is important the Agile recommendation of having cross-functional teams. A jelled team which contains all the necessary expertise represents the easiest path for full commitment.
3. Changes in the backlog should be managed lightly, but continually. In fact, I am not even sure that they should be named changes. As previously mentioned in point 1, a requirement grows in the backlog, perhaps changing its status, form and priority. This doesn’t mean that Agile embraces high volatility or churn. All stakeholders should be aware of the impact of the changes. A change may be important, but is never free. The Product Owner, responsible for the ROI is the one to make the trade-off.
4. Maintaining traceability could be very time consuming and only helps determine that requirements have been addressed. Acceptance Tests are less time consuming are help determine that requirements have been fulfilled.
5. Again, automated acceptance tests are the recommended practice to detect and fix inconsistencies in an Agile project. Doing this manually would be too time and mind consuming. Computers should maintain all this knowledge gained and detect the inconsistencies for us. The second recommended practice is to involve the customer through all the project. Customers are the best persons to detect inconsistencies (should we call them inconsistencies?)

What is missing in the recommended practices?

Prioritization: One of the most important aspects in managing requirements in an agile environment is managing priorities. Prioritization works starts at project inception and follows throughout the project. Of course, this is a very difficult task as it involves not only understanding the business value that different requirements add but also making trade-offs based on technical feedback and measures of effort.

Requirements live and Requirements light: One of the most difficult and time consuming tasks during a project is to maintain requirements up to date. If doing this is too time consuming, agility suffers. Many agilists suggest that the best way of maintaining them is through maintaining them as executable requirements (or executable specifications, examples, etc.). An executable requirement must be maintained and it always up to date. Being able to execute/debug this requirements allows to easily trace requirements with source code.

Conclusion

I believe the only goal of this process area is entirely applicable in an agile environment, but the path to get there goes in a complete opposite direction. The CMMI recommendation is to get a complete approved set of requirements, which is maintained and traced to the source code (and viceversa) throughout the life of the project. The agile recommendation is to maintain a prioritized executable specification at all times. The later mechanism is lighter, assures is up to date and automated.

Other Process Areas

- Is the Project Monitoring and Control Process Area compatible with Agile
- Is the Project Planning Process Area compatible with Agile?