The humane assessment blog

Free Moose apprentice course (June 2-3)

Tudor Girba — Wed, 16 May 2012 00:08:30 +0200

I am happy to announce that on June 2-3 I will give a public Moose apprentice course. The course is organized at the University of Bern, Schützenmattstrasse 14, Room 107, and it is free of charge.

This is an introductory course on how to use the Moose analysis platform for putting humane assessment into practice. It is dedicated to software engineers, and the concepts covered include model navigation, querying techniques, code metrics, and visualizations. The course sessions are accompanied by hands-on exercises.

If you are interested in registering, please contact me directly.

How much do you know about your system?

Tudor Girba — Wed, 09 May 2012 23:24:32 +0200

I while ago I met a company of some dozen developers that were working on a new part of a system. The team was quite motivated, and it included both senior and junior developers.

This new part was implemented in Java EE, and they were working on it for a bit more than half a year. Given that the development was quite at an early stage, the system barely totaled 1500 classes (both client and server side), which for Java is not that much. During an assessment workshop, I gave them the following task: List from on top of your head all the service classes that you provide on the server side. This basically boiled down to enumerating the session beans they had.

To make it easier I also told them that it is not important to get the names precisely correct, and that in fact I will be doing a union of all the item they list. After a couple of minutes, we got to 12 items on our list. And they seemed content. Then I asked Moose to tell us how many classes are isAnnotatedWith: 'Stateless' and the result was 20.

Of course, when they saw the complete list, they did recognize all the other 8 services. And they also found reasons for why they did not list them in the first place: some were not used for most use cases, and some were simply implemented early in development.

These were competent developers that cared about their system. Still they only got 12 out 20. And this was a coarse-grained question about a medium-sized system.

The problem is simply the amount of data. As soon as we are talking about more than a handful of details we should remember, the fancy mechanisms in our brain will kick in and help us focus on what seems more important. Essentially, we scale by not remembering everything.

Why is this an important issue? Well, most modern engineering practices rely on putting the right people in one room and expect them to find the best solution. But, when the facts are not in that room, how are we to expect to get to the best solution?

The size of the systems and data you have to deal with is increasing every day. This changes the game rather profoundly. To keep this game under control, you have to change the way you approach it.

When testing does not cut it

Tudor Girba — Mon, 07 May 2012 21:30:43 +0200

Testing works great, especially when it drives the work of transforming specs into code. But, testing is just about functionality. And functionality is just part of the story that makes a software system.

Let's take an example. Suppose you are writing a system that has a server side based on JBoss. And suppose you want your system to rely on the clustering possibility of JBoss. How would you test that the system works in a cluster as expected? For a complete test, you would need to do something like this:

setup a cluster with at least two instances of JBoss running,
create multiple clients and lookup all the session beans that you want tested from the cluster,
make sure you get a connection to the first node,
take that one node down, and
check to see that the requests to the already acquired session beans is not failing.

This test would work nicely for the current set of beans. When a new one is created we would have to extend the test with that particular session bean. However, even if we would develop test-first, we might could easily forget to test the clustering requirement simply because it is not a functional one: from a purely functional point of view, a use case could be satisfied even without the clustering in place.

It is not impossible to setup a test for our problem, but it is expensive from multiple perspectives: it is difficult to setup, it is not straightforward to extend it, it takes long to run the test. But, the worst part is that when the test fails, it will be unclear what exactly went wrong.

What is the alternative? First, let's see what does it take to specify clustering in JBoss. It turns out that all you need to do is to annotate the interesting session bean classes with @Clustered. If you accept that JBoss does work as expected, the only thing you need to test is that you have the right annotations in place. This can be checked structurally. With Moose, the check would look like:

model allClasses select: [ :each | 
   (each isAnnotatedWith: 'Stateless') and: [
      (each isAnnotatedWith: 'Clustered') not ]]

Using this rule on a system results in a list with all (stateless in this case) session beans that do not have the required annotation. As long as the list is empty, you are fine. If the result is not empty, you know you have a problem.

The direct cost of building this rule is significantly lower than the one associated with the functional test. But, the best thing is that when the structural rule fails, it provides an immediate pointer to what needs to be fixed.

Testing works great, but only for problems that are best described functionally. For the other problems, you are better served by different analysis tools.

Assessment talk at Jazoon/SET 2012

Tudor Girba — Mon, 30 Apr 2012 05:48:53 +0200

On June 26, I will be giving a talk at SET/Jazoon 2012 (in Zurich) about Why you should care about software assessment.

Here is the abstract:

What is assessment? The process of understanding a given situation to support decision making. During software development, engineers spend as much as 50% of the overall effort on doing precisely that: they try to understand the current status of the system to know what to do next. In other words, assessing the current system accounts for half of the development budget. These are just the direct costs. The indirect costs can be seen in the quality of the decisions made as a result. That is why you should care about it.

What can you do about it?

Make it explicit. Ignoring it won't make it go away. By acknowledging its existence you have a chance of learning from past experiences and of optimizing your approach.

Tailor it. Currently, developers try to assess the system by reading the source code. This is highly ineffective and it simply does not scale to the size of the modern systems. You need tools, but not any tools. Your system is special and your most important problems will be special as well. That is why generic tools that produce nice looking reports won't make a difference. You need smart tools that are tailored to your needs.

Educate it. The ability to assess is a skill. Like any skill, it needs to be educated. Enterprises need to understand that they need to allocate the budget for those custom tools, and engineers need to understand that it is within their reach to build them. It's not rocket science. It just requires a different focus.

All in all, assessment is a discipline. More information about our view on assessment can be found at: http://humane-assessment.com

The talk is targeted equally to software engineers and managers, and it exemplifies the above points through several case studies.

Q&A: The tool or the method?

Tudor Girba — Sun, 29 Apr 2012 09:01:52 +0200

Moose is a tool. Humane assessment is a method. A smart engineer asked me the following question:

You seem to have two crusades: on the one hand, you want developers to make assessment explicit, and on the other hand, you promote Moose as a tool. Which one is more important for you?

Moose is a novel tool. Its novelty stems from its focus: it's actually not a tool, it's a platform. Instead of providing ready-made generic analyses, it aims to make crafting custom ones cheap. Still, Moose is just a tool. And like any tool, it requires a method to produce actual value.

There are plenty of analysis tools out there, but they all tend to fall short. That is because to drive them, you need knowledge. And, as long as you do not invest in building up this knowledge, nothing major will change. The industry focused on the technological aspect, and forgot about the human aspect of the equation.

Which one is more important? The method.

Ask more questions via @humaneA, @girba, or by email.

Q&A: Can developers be data analysts?

Tudor Girba — Thu, 19 Apr 2012 18:22:37 +0200

This is the first post in the q&a series.

During a recent presentation at SAP Research, one the researchers from the audience threw a friendly challenge that sounded like this:

I like the argumentation around humane assessment. But, it relies on developers having to create their own tools and to use them for decisions, and I believe most developers are not able to do proper data analysis.

My answer has two parts.

First, it is not true that developers cannot be data analysts. Most of them already are, even if unknowingly. Let's take SAP. Most of what SAP developers do is to build software systems that store data, transform it, and then present it in a form suitable for decision making. It's true that the data is not about software, but it's analysis nevertheless. It's also true that these developers are not the ones making the decisions based on this data either, but they do know how to support it. The capability is there. What is missing is the know-how for how to apply the very same techniques to software problems.

In the end, we deal with an educational issue, and of course, in practice, it does take a while to switch the state of mind. This brings me to the second part of the answer. Not every developer needs to be a data digger from the beginning. It is precisely for this reason that humane assessment introduces the role of the assessment facilitator. In most contexts I saw, one facilitator can support some 8 developers. It is similar to how a system administrator can support multiple developers.

Let's look at the system administrator case more closely. Even if developers are not specialists in system administration they are still able both to formulate requirements and to utilize the feedback they receive from the specialists. Similar dynamics are at play in between developers and facilitators. Even if a developer does not understand all the details of how certain parts of software analysis are realized, he will still be able to interpret and act on the results of the analysis.

All in all:

We should give developers more credit.
Not everyone needs to know everything. Facilitators are the data specialists in the team. Everyone else just ask questions and transform the results into actions. This is achievable with a small initial investment.

Ask more questions via @humaneA, or @girba, or by email.

Assessment Q&A: what would you like to know?

Tudor Girba — Fri, 13 Apr 2012 23:15:40 +0200

Assessment is not difficult once you jump in the boat. But, this jumping can be perceived to be difficult. To dispel the magic, let's open the curtain and play a game.

It's simple. You ask questions, and I will answer them on this blog.

How to ask questions:

send me an email at tudor@tudorgirba.com
post a comment on this blog
post a tweet mentioning @humaneA, or @girba
ask me in person when you see me (I will be at PharoConf and Jazoon in the following months)

What questions? Any questions.

What's in it for me? You will help me shape my ideas.

Your turn now.

Moose demo: browsing class naming cohesion

Tudor Girba — Mon, 26 Mar 2012 00:48:11 +0200

A previous post demonstrated how to build a simple visualization to explore the overall class name cohesion. In that case, the visualization showed a picture of how classes that have similar names are also placed in the same package.

The result was a global view that was applicable to all classes. But, what if we want to get the naming cohesion of classes on a per package basis? Simple, we need a browser that enables us to explore the packages in the system, and for each of these to show the visualization. And, as we do not have it yet, we have to build one.

Take a look.

The code for the browser can be seen below.

composer tabulator with: [:t |
  t column: #namespaces; column: #classes.
  t transmit to: #namespaces; andShow: [:a |
    a tree
    display: [:mooseModel | mooseModel allNamespaces select: #isRoot ];
    format: #name;
    children: #childScopes ].
  t transmit from: #namespaces; to: #classes; andShow: [:a |
    a mondrian
      painting: [:view :namespace |
        view shape rectangle
          size: #nameLength.
        view nodes: namespace classes.
        view edgesToAll: [ :class |
          namespace classes select: [ :eachTarget |
            (eachTarget name pairsDistanceFrom: class name) > 0.5]].
        view graphvizLayout neato ]]].
composer startOn: model

The quality vs your quality

Tudor Girba — Mon, 19 Mar 2012 17:17:42 +0200

Thoughtworks released a new version of their Technology Radar. Essentially this is a map meant to help people understand and rate software engineering trends and technologies.

An interesting development consists in the increased profile for metrics and visualizations. I am happy that finally practitioners start to recognize code analysis as being important. There is indeed a great body of research in this field that can be of great use for practice.

But, even if metrics and visualizations appear at the very center of the radar (hence very important), they are not yet well understood. Take a look at the following statement from the July version of the radar:

Measuring software internal quality is still a mystery.
Technology Radar, July 2011

The authors feel code analysis is important, but they just do not know how exactly to use it in practice. This pretty much reflects the opinions of most practitioners I encounter.

Let's take a closer look at the statement. I do not know if they meant it like that, but the word "still" denotes a slight exasperation that seems to say "after all these years of research effort and promises we still cannot utilize the results."

And it is not because of lack of effort. There is a large body of literature on this subject, and there are significant advances, too. The art and science of measuring software aspects has come a long way over the past two decades. Still, even if the technical aspects of how to attach numbers to software facts is well understood, capturing quality in the large still evades us. This is frustrating.

I believe the root of the problem comes from the underlying meaning of quality. When people talk about software quality, they usually talk about the quality. The quality is the globally unified understanding of what is good and what is bad about software systems. This is indeed a mystery, and will most likely remain so for the foreseeable future. From this point of view, the quality shares commonalities with the Holy Grail: they are both supposed to be real, they both come with great promises, and they both remain elusive.

Software is too complex, and it lives in too many contexts for us to be able to have a global and quantified view of its quality that is still meaningful. But, what if measuring the quality is not that relevant for daily business?

To measure something, you first have to be able to say what that something is. And, we just cannot say what the quality is. Or, to put it in the words of Christopher Alexander, this quality has no name. Better still, it has no single name.

Quality is highly contextual. If you ask an engineer about quality in general, you will likely get yourself into a philosophical debate that can last for hours. If you ask the same engineer about comparing the quality of two hands-on solutions within the context of your system, you will likely receive a decisive answer within minutes.

Your context makes all the difference. Once you know the context, you can start to instantiate general patterns and make them operational. This is what leads to action and to engineering solutions. This is not to say that concerning ourselves with the quality is not important. It is. It is this effort that helps us identify general patterns. However, you will not be able to measure it directly.

Instead focus on your quality. Learn to embrace it. Learn to measure it. Learn to manage it. It is your quality. Make it yours. It might look like a compromise, but it brings value.

The budget is already allocated

Tudor Girba — Sun, 18 Mar 2012 20:54:22 +0200

What does it cost to adopt humane assessment?

Before we answer that, let's step back and see what humane assessment is. It is a method for making decisions regarding a software system. It's actually quite simple, and it boils down to understanding that proper decisions require information that is both accurate and specific. Considering that software systems are large and accommodate complex contexts, it follows that to retrieve the proper information you need tools that are customized to match the problem at hand.

Here is the thing. You already spend a significant amount of energy on making decisions that concern the fate of your system. And because the stake is so high, you already try to obtain the best information you can out of the system to reach the best decisions.

It's just that you might not do it explicitly. It happens somehow. Because of that, you probably spend the time doing manual work that both does not scale and is inaccurate. And because of that your decisions suffer.

How much does it cost to do it differently? It requires some initial investment to change the mindset and make assessment explicitly part of your daily life. But, in the end, this cost pales in comparison to the overall efforts you already spend.

The budget is already allocated. And the good news is that if you decide to spend a part of it differently, you can probably not go worse than you are doing right now. It can only get better.