We Are From Mars

All agile methodologies stress the need of co-locating development with the customer's representative - the Product Owner - or at least, having them in close proximity - both in time and in space.

Our team seems to be the exact opposite of what "the agile way" requires - we are located on the other side of the planet (Gdansk, Poland) from the company headquarters (Sydney, Australia). This means we could just as well be located on Mars. Actually, one could argue that outside of the narrow communication windows (early morning and late at night), communication latency between Sydney and Mars is smaller than the one between us and our Product Owner. Yet - we still manage to work in an agile way.

Talking To Each Other

How do we do it? First of all, we utilise high-bandwidth communication opportunities as much as possible. We have a local Product Owner proxy (that would be me), who talks to the Product Owner regularly, at predefined times and at least once a week. Also, if the out-of-band need arises, ad-hoc meetings are scheduled. Skype (or some equivalent technology) is your friend here, as well as Instant Messaging systems, such as Jabber. I cannot stress this enough - talking to each other in real-time is the most important and most effective way to communicate. I do not think it would be possible for us to work without it.

Technology Helps

But that is not all. For us, the second most important communication method (and tool) is JIRA - and it does the job just fine in its plain, out-of-the-box form. We use pretty much a stock JIRA setup, with just two custom fields added (both reflecting standard Scrum practices) - one called "backlog order", used for prioritizing user stories (a fancy word for "new feature or improvement"), the other called "story points" - for estimating the "size" of the user story. That suffices all our needs for making sure both the PO and us know what has to be done and how much delivering the requested feature is going to cost (after all, the effort estimates translates almost linearly into monetary costs). We also use the GreenHopper JIRA plugin to provide "virtual story cards", instead of maintaining our user stories on pieces of paper, to make our life easier when it comes to copying the outcomes of our planning sessions to JIRA. But for day-to-day ticket management, we just use JIRA. Or more properly: one of us (that would be me again) is using JIRA, everybody else is just moving around the actual paper story cards on the cork board, which is a central part of our workspace.

What about the communication in the other direction? What if a Product Owner needs to describe some story to us, and words are not enough? Sometimes the Product Owner will want to draw a mockup of the user interface he has in mind. Here is where the Balsamiq Mockups tool is very helpful. It is a Confluence plugin, which lets you draw diagrams right there in the Confluence page, and share it with whoever you want to communicate with. We would typically edit the diagrams in-place and simply comment on them until everybody is satisfied with the resulting sketch of the user interface to be delivered.

Delivery

How does the Product Owner know we are doing what he wanted us to do?

First of all, just as the Scrum methodology requires, we have a proper (internal) release every two weeks. The official builds are published in an externally accessible place for everybody to download and try it out - this gives us a way to regularly demo our progress and provides a way for the Product Owner to correct our course (e.g. whenever we stray from what has been planned or whenever the plan needs to be changed due to external circumstances).

Second - our Bamboo-based automated build system lets the Product Owner download and use the snapshot of our work every day - the snapshot may be barely usable, but typically it is in a good enough shape to let the Product Owner try new features as they are delivered.

Remote Dogfooding

We regularly practice dogfooding, using our product regularly on a daily basis. Almost every nightly snapshot is taken and used by each team member every morning. So if we happen to have a bug, we are quite likely to intercept it before the official version of the product is released. Also, if some user interface elements are awkward to use, or somebody has a bright idea for improving it, we are able to try multiple solutions and pick the best one - simply because we use our own software for our day-to-day work.

But in order to get even better feedback, developers in the Sydney office should also dogfood our software and give us feedback. Achieving sufficient levels of internal adoption requires a bit of advertising from the Product Owner, and regular presentations of cool new features that the product provides, so that folks in Sydney have a reason to bother installing our stuff.

Separation Of Responsibilities

The last thing that is important for a remote team and the Product Owner working with it is a separation of responsibilities. In our case, the Product Owner is only responsible for prioritizing features and controlling if they are delivered as promised. Planning and implementation are left to us. Also - all bug reports are handled locally by the team, as a part of the Quality Assurance process - we are solely responsible for keeping the product in good shape and the Product Owner does not interfere with that.

Another important rule we follow, is that the team's effort estimations are final. The Product Owner (or anybody else) does not even attempt to question them, trusting our judgement - in true agile spirit.

First, my definition of refactoring:

Refactoring is improving code without changing the features it implements.

That's all.

If you're refactoring, you're not fixing bugs, you're not improving performance and you not increasing robustness. Refactoring is simply improving the design of the code, while ensuring that it still works the same, warts and all.

Pointy haired bosses the world over froth at the mouth to hear such things. Veins pop out in their temples. You mean the business value of the software stays the same but the cost to the business goes up? I didn't say that.

If you measure business value only in terms of features you have today, then you can end up deep in technical debt; you can add features today in such a way that features tomorrow cost more and more. The value of refactoring is wholly contained in the future ability of programmers to comprehend and modify the code. It's called maintainability, but that's a boring word, so let's call it Agility. Mmmm, sexy. Well-factored code is agile code because it's better able to change.

In economic terms, refactoring is an investment, or the repayment of a debt. It's only worth doing over a time frame when the interest payments or repayments (in the form of ongoing productivity gains) compound to exceed the time invested. Fingers crossed the business or project sponsor is also planning over such time frames.

The term refactoring comes from mathematics. You may remember your high school algebra:

2x² + 10x

Stay with me! No glazing over! If you refactor the expression, extracting the common factor, 2x, you get:

2x . (x+5)

Sometimes it's hard to spot the common factors, in both mathematics and programming, and it can certainly be done poorly. More on that later.

Like most powerful techniques in software development, the purpose of refactoring is controlling complexity.

Complexity is bad, mmmkay. Complexity is evil.

I was fortunate enough to be chatting about project complexity with the legendary Dave Thomas (OTI, Eclipse) at JAOO Sydney in May. He nailed it: "kLOC kills". Complexity and scale in codebases is a major contributor to schedule blowouts, poor velocity and excessive development cost. Complexity is a kitten killer from way back.

Fred Brooks discusses two categories of complexity. Essential complexity and accidental complexity.

Essential complexity is the complexity of the domain. In NASA software, there's no escaping rocket science. You can isolate and divide essential complexity but you can never remove it. Essential complexity belongs to the problem. By contrast, accidental complexity is an artefact of the systems, languages, frameworks you're using. In principle it can be reduced by changing the system. Accidental complexity belongs to the solution. Refactoring reduces accidental complexity.

If you don't have much experience with it and you're looking for some concrete tutorials on refactoring, I suggest you start with Martin Fowler's seminal book Refactoring. Fowler also maintains a catalog of refactoring recipes with an Object Oriented flavour.

One of the most basic techniques is Extract Method which all decent IDEs can do automatically. You know you need Extract Method if you have a multi-page method with a sequence of of comment blocks which look like this: Now that we have the InductionActuator, look up the FluxCapacitor.... Doing it manually means snipping out a logical sequence of code and pasting it into a new, small, well-named method, stitching the local variables used from the originating context into parameters to the method. If this is hard due to sloppy scoping or too many variables, you may consider Introducing a Field from a local variable.

The inner loop of agile development should go like this: Red, Green, Refactor. Red means you have a test which is not passing. Getting the test to pass is the next step. Green means you are passing all tests. Refactor means... refactor.

Even if you're a good agile developer, doing things as simply as possible, complexity and duplication of common factors creeps in while you're trying to pass tests. Everybody hacks. Everyone copies and pastes. This is fine as long as you go back and refactor when you've got the green bar. Sometimes you may need to avoid mentioning this to PHBs, for their own good. Shhh!

Unit tests are really important for refactoring. If you're not doing unit testing you've got a long way to go. A good unit test suite is a necessary precondition for confident, aggressive refactoring. And IMHO a good test system is a necessary precondition for confident, aggressive, automated refactoring. These preconditions can present a quandary for some developers. Legacy systems often have no effective automated tests. And since they're often composed entirely of spaghetti, they need to be refactored. It's a chicken and egg situation, where do you start? All I can say here is you start small.

Refactoritis

Can you have too much refactoring? Absolutely. If you're somewhere around middle-stage zealotry for this refactoring stuff, you may not be in danger of copy+pasting your way to a big ball of mud, but you may fall prone to exceed the safe working abstraction load of your language or go too far beyond the idioms of your team's codebase or comfort.

Every language has limits imposed by its design and implementation. In Java and C#, for example, the limits are seen by many in the dynamic languages camps to be too much to bear. For example, say you're refactoring some Java or C# code. You might create a new interface with a few alternate concrete implementations and, whereas before you had two methods on a concrete class and a few big if-else blocks, after refactoring you might have three files and more actual lines of source code. It can be somewhat subjective but sometimes you may have more complexity even though you've removed duplication!

If this happens you have fallen asleep on the refactoring train and missed your station. Often you should just roll back the code and go write a feature. Some duplication is easy to see and cope with, especially if it can fit on one screen and any reader can see the pattern. In other languages, Lisp comes to mind, there are constructs (like macros) which allow you to encapsulate expressions that cannot be elegantly factored in, say, Java. Disclaimer: IANALN; I Am Not A Lisp Nerd.

So the expressiveness of the language can constrain refactorability. Another way of saying this is that the language contains accidental complexity and only factoring out the language can remove that complexity. I should say here that I have recently found Groovy to be a great candidate for doing this on Java projects.

As a more concrete example, lexical closures are a great way of implementing things like the new for loop (for each) introduced in Java 1.5 and functor frameworks that employ anonymous inner classes in Java for similar purposes (e.g. composable transformers, ad hoc iterator delegators instead of explicit looping) often feel too cumbersome compared to most closure implementations. So you just have to suffer the duplication and code bulk.

So in summary, Red, Green, Refactor, don't go overboard and be aware when your language makes capturing factors you see in your system worse. Kill complexity before it kills you.

If you're interested I'll be telling war stories and going into some side issues over on my personal blog.

One of the best things about this integration is that it speaks to both the power of Alfresco's open-source code and Atlassian's open APIs: the integration was started without any formal communication between the three companies. All the more reason to add another party to the mix: YOU!

While the integration is still a bit rough around the edges, we're inviting you to get involved and take the integration in the directions that you'd like to see. We're basing the initial integration on the new CMIS (Content Management Interoperability Services) standard, which means that not only can we build a range of new functionalities on top of the baseline integration, but we can also extend the integration to a diverse range of different technologies.


We've Accomplished So Much Already

The current Alfresco Plugin for Confluence provides a set of custom macros that enable Confluence wiki pages to display or embed Alfresco documents or document metadata through ID reference, Path reference or CMIS Search Query. The expectation is that we can provide a seamless experience to users of both platforms:

• Search integration within Alfresco: By adding OpenSearch capabilities to Confluence, Alfresco is now able to aggregate search results from Confluence wiki pages and the Alfresco repository. So, for example, a user logged into Alfresco will be able to retrieve data from documents hosted in an Alfresco repository and any Confluence page she's got access to. This is a good example of what open standards and extensible systems can offer.


• New macros for Confluence, providing wiki users with the ability to interact with an Alfresco repository in a number of ways, such as:
  
  
  
  • Browse an Alfresco repository
  
  
  • Reference (link) and embed (display directly) documents stored in Alfresco
  
  
  • Build custom reports (such as listing documents that match specified criteria) by running queries against the Alfresco repositories
  
  
  


• Use Web Scripts or CMIS: As an added bonus, macros are implemented using either Alfresco-specific technologies such as Web Scripts or a pure standard-based approach (CMIS). The Web Scripts technology is very, very cool and opens the door to do some interesting new capabilities for Confluence.

This is only the beginning

Even as I type this, an email has hit my inbox noting that new features, like the ability to pull Alfresco Share (Alfresco's collaboration application) site activities into Confluence via an Atom feed, have been added. We'd love to have you help us figure out where to take this integration, and to fill in pieces you think are missing. That's the benefit of open standards and open source: Confluence can be even more than Atlassian envisions.

We're hosting the project on Google Code: http://code.google.com/p/confluence-alfresco/. Please stop by and get involved.


UPDATE
You can also get SSO w/ Crowd for your Alfresco install:
http://code.google.com/p/alfresco-crowd-security/

With that thought I set out to write the deeplyIsEqual matcher. The result is the DeepIsEqual Matcher. Using it is pretty easy. In that previous post I described the Matcher for comparing lightsabers, LightsaberIsEqual. We no longer need that. Now we can simply do

    assertThat(anakinsLightsaber, is(deeplyEqualTo(lukesFirstLightsaber));

No extra boilerplate is needed and we still get all the benefits of only seeing the fields that didn't match in the error messages. Oh, and the reported expected value is built using reflection too, so you don't need to worry about whether the type of objects you're comparing implement toString or how they implement it.

After going through the initial implementation I started switching over the OAuth tests to use this new matcher. Everything was going great until I hit a case where PublicKey or PrivateKey objects were being compared. The problem was, one was a generated key and the other was converted from an encoded string value. Apparently, depending on which way you create Key objects, the internal fields can be slightly different. So, comparing them recursively was failing. I struggled with what to do - I could add the Key to the internal, hardcoded types that are compared by simply using equals(), or I could add the ability for testers to specify how they wanted certain types to be compared using a MatcherFactory. This being 20% time and me seeing other places where specifying custom matchers for certain types would be very useful, I went with the second option.

So, if we wanted to match Hilts in a very specific way we could that pretty simply.

    public static Matcher<? super Lightsaber> equalTo(Lightsaber lightsaber)
    {
        return deeplyEqualTo(lightsaber, extraTypeMatchers);
    }

I wrapped the deeplyEqualTo call in a convience method, because the creation of the extraTypeMatchers can be a bit gnarly and makes the tests a bit harder to read. As a simple example, let's say we want to compare {{Hilt}}s using their equals() and don't care if the subtypes are different. To accomplish that we could use the following extraTypeMatchers.

import static com.atlassian.hamcrest.ClassMatchers.isAssignableTo;
import static com.atlassian.hamcrest.DeepIsEqual.deeplyEqualTo;
import static com.atlassian.hamcrest.MatcherFactories.isEqual;
    Map<Matcher<Class<?>>, MatcherFactory> extraTypeMatchers = new HashMap<Matcher<Class<?>>, MatcherFactory>()
    {{
        put(isAssignableTo(Hilt.class), isEqual());
    }};

That's not very pretty, but by wrapping it up our test can go back to simply being

    assertThat(anakinsLightsaber, is(equalTo(lukesFirstLightsaber));

and we get all the benefits we had before, plus we are matching Hilts in the desired way.

The biggest thing left to tackle is how to handle object graphs that have cycles in them. We have some ideas about how to do it, such as tracking the objects visited and if we find one that we've already visited just stop recursion and assume it will be true, relying on all the other matching to prove otherwise. I'm not 100% sure this will work. The main problem being, what happens if a field in expected value has a reference to, for example, the first object in the graph. But, the actual value doesn't have a cyclic reference but does have a reference to an object that should be considered "equal" to the same object as the expected value (I know, it's probably not entirely clear what I mean but I'm having a problem figuring out how to phrase it more better as my brain keeps breaking when I think too hard about it). I'm thinking for now, being able to specify custom matchers for types should be enough. If there is some part of the object graph that we expect to have a cycle, then just specify a custom matcher for it and move on.

To play with this you can checkout the code from the labs project, or you can get from our public maven repository. If you find any problems with it report them in JIRA.

Thanks, and enjoy testing again!

We thought about how to get around this new problem, and decided the best approach would be to make it possible for macros to find out if they are being rendered in an email or a feed, so they can display themselves appropriately. It was already possible for macros to find out if they are being rendered in a PDF document or several other contexts. In Confluence 2.10, we made it possible for macros to find out that they were being displayed in an email or feed, an addition to the previously defined contexts. Previously, macros being viewed in an email or feed reader would have just had the render type "display", which is the default.

Now the Jira Issues Macro is able to render itself differently in display versus feed modes:

Okay, so how can you find out the current render context from within your macro? When creating a plugin that includes a macro module, you return the HTML that the macro will display from the execute() method of the macro class. One of the parameters to the execute() method, the one with type RenderContext, can be used to determine how the macro is being rendered.

Here's a sample execute method from a macro that prints out the current render context type:

public String execute(Map parameters, String body, RenderContext renderContext)
{
 if(RenderContext.FEED.equals(renderContext.getOutputType()))
 return "FEED render type";
 else if(RenderContext.EMAIL.equals(renderContext.getOutputType()))
 return "EMAIL render type";
 else if(RenderContext.HTML_EXPORT.equals(renderContext.getOutputType()))
 return "HTML_EXPORT render type";
 else if(RenderContext.PREVIEW.equals(renderContext.getOutputType()))
 return "PREVIEW render type";
 else if(RenderContext.DISPLAY.equals(renderContext.getOutputType()))
 return "DISPLAY render type";
 else if(RenderContext.PDF.equals(renderContext.getOutputType()))
 return "PDF render type";
 else if(RenderContext.WORD.equals(renderContext.getOutputType()))
 return "WORD render type";
 else
 return "some other render type";
}

If you used this sample macro on a page you were editing (by first installing the plugin that contains it), you could visit the preview tab to see it output "PREVIEW render type". In the case of a more complex macro, you could, say, disable some UI elements when the RenderContext.PREVIEW.equals(renderContext.getOutputType()) check is true. Using these checks is exactly how the Jira Issues Macro decides whether to render itself using JavaScript or just stick with a basic HTML version.

Pair programming is easily the most controversial agile development practice and in my experience the most frequently avoided. It makes some people pretty uncomfortable.

Just so we're all on the same page, pair programming means two programmers working together on one computer. Yes really. Normally one codes while the other thinks ahead and around. Or, one drives and the other navigates, swapping regularly. The motivation is higher quality output from the outset.

Pair programming requires working much closer with team mates than most developers are accustomed to. This is both its power and it's greatest weakness.

Working so closely with others can bring otherwise hidden personal issues to the surface. My experience in previous jobs has shown me that pair programming can be so personally challenging for some people that the practice is infeasible for that team. Doing it takes courage and maturity, it requires a healthy environment where mutual respect, humour and humility are all expected. Not to mention minimum standards of personal hygiene! This is quite different from the stand-offish formalism characteristic of many alternatives and this humanistic style sets the bar too high for some. Cowboys, incompetent introverts, control freaks and superhero programmers all resist pair programming. It's like their kryptonite.

Like culture shocked westerners first visiting Tokyo, unused to the dramatic reduction in personal space one can expect, say, on a commuter train, for some, getting used to the personal invasion that is pair programming is a self-imposed barrier they just need to break through.

Pairing up with different people makes this personal contact a dynamic challenge. There are several axes of individual difference which demand different behaviour. For example there's the novice-expert axis and the introvert-extrovert axis and people have different natural working styles and favoured problem solving strategies. For example I can often be willing to explore unconventional solution paths which make methodical people a little unsettled. Being aware of that helps me communicate appropriately and change my behaviour where necessary.

Pairing requires bilateral adaptability and the art of compromise.

There are many common anti-patterns that lead to ineffective pair programming. These things are fairly well documented, for example see Pair Programming Illuminated by Laurie Williams and Robert Kessler. It's worth tackling these problems actively because ineffective pair programming is worse than no pair programming and it's obvious even to would-be starry-eyed XP fans.

One of the reasons I've been thinking a lot about pair programming recently is because I haven't been doing it for a change. (gasp!)

Apart from some time on support and fixing a pile of trivial bugs (not pairing), I've been working solo on a front-end feature for JIRA 4.0. Solo because our team has an odd number of people right now (wanna job?). It has actually put me in a great position to evaluate the practice of pair programming somewhat objectively.

First the good stuff about not pairing. I have some great new headphones and with the right music I can get into the zone and stay there for hours without needing to coordinate with anyone. I do not have to continually exert effort listening or thinking of ways to express my questions or ideas. I just think and do. That's refreshing.

But I'm not ashamed to admit it's a little isolating. Hey, I'm a people person.

The negatives of not pair programming are especially visible when you are accustomed to doing it. While not pairing recently I have become curiously suspicious of the quality of my work.

I think it's good, Is it really good? I'm not gold-plating am I? Maybe this is absolutely the wrong approach. Is there something obvious that I'm missing? I'd better get someone to review this.

All those thoughts are plaguing me in my "step back moments" when the tests pass and I get to wonder about the next move. This wouldn't happen if I was pairing. I would be in dialogue with my partner; we would have common mind (just imagine Yoda saying that last bit).

Pair programming can really help in making a team gel. Spending time together - not just in pair programming but in other activities like sharing a meal or social event during work time help teams to gel.

So while there are many reasons to practice pair programming, it's not the only way to get these benefits. Code review is an obvious alternative for many of the same benefits. Marketing may want to break my arms if I don't mention Crucible at this point.

So if you're not improving the quality of your code with pair programming, then I'd hope you are doing code reviews because there really isn't much disagreement in the industry that this is a very well known and effective method of ensuring quality software development.

Atlassian development teams do either one or the other, or sometimes, like in JIRA some of both.

Using a tool like Crucible for code reviews has some advantages over pair programming, such as record-keeping and working across distributed teams and time zones. Pair programming has some advantages over code review such as swapping keyboard shortcuts and shell one-liners and helping a team to gel.

So if you have a business case for better quality software then remember: two heads are better than one!

Learn about agile software development inside Atlassian

Index Cards have become an indispensable tool for a software development team practicing agile. There are a host of good sites with information on strategies for and benefits of utilizing index cards.

Agile teams use Index Cards for a variety of reasons, including:

• helping with iteration planning
• assigning tasks and then tracking developer assignments
• recording task estimates
• as one component of the team's Information Radiator.
• reminders of the conversations between customers and developers around the definition of user stories.

The cards are an excellent visual representation of the health of the iteration. A majority of cards still in the "to do" with the end of an iteration pending usually means the team is behind schedule. A scarcity of cards means the developers will soon run out of tasks and so the backlog had better be up to date. Cards also work well as a great tool for facilitating the assigning of work (more specifically, allowing developers to assign themselves individual tasks). On the JIRA Studio team at Atlassian, back in "Ye Olden Days" (circa 2008) we used to use actual physical index cards, stuck to the team white board with blu tack. The use of physical index cards does have drawbacks:

• Duplication of effort
• Data from cards and issue tracker being out of synch
• Bad handwriting leading to incorrect implementations
• Tons of old index cards lying around

We now use virtual index cards, thanks to the magic of Greenhopper, an agile planning tool that is now bundled free with JIRA Studio: 

Greenhopper is key to being able to do agile project management. We use the virtual cards for the traditional tasks of iteration planning, capturing requirements, and capturing acceptance criteria. The use of virtual cards allows for all of the functionality and advantages of physical cards, with the added advantage of better communication, and instant updates. Developers still "pick" their cards off the task board, track work on the cards, and place them in the "done" column when complete. 

The JIRA Studio team was not a distributed team when we made the change to using the Greenhopper plugin exclusively. But now that one member is on another continent, the benefit of virtual cards to help with communication has grown even greater. Our Information Radiator is created with the virtual card wall displayed on a large monitor which also shows the current build status. Everyone on the local team can glance at the monitor for a visual representation of the health of the build and of the iteration, while the remote team member can view the exact same information in his web browser.

The synchronization of data is by far and away the biggest advantage of virtual cards, as an instant update of all information occurs whenever a card is updated. No more does a task get ignored due to a stray mark on the card. No longer does a developer waste time entering the same data onto the card and into JIRA. And the Team Lead no longer spends 30 minutes transcribing cards into JIRA issues or copying JIRA tasks onto cards in his chicken-scratch handwriting.

Greenhopper is integrated with JIRA Studio (and available as a plug-in for JIRA), so changes to a card are reflected in the JIRA issue, and vice versa. Moving the card to a new iteration, or new column on the board is reflected in the JIRA Issue. Changing the JIRA issue status to Assigned or Resolved immediately moves the card into the correct column on the Information Radiator. Almost as valuable is the ability to have multiple views of the information, without having to continually move physical cards. The cards can be viewed as a task board or a planning board, and sorted by priority, assignee, or a host of metrics.

Last, but definitely not least, the use of virtual cards mitigates a few of the dangers of agile.

Watch Ted talk about Virtual Index Cards with GreenHopper


Watch more videos on agile software development inside Atlassian

There are many ways to become agile. And many ways to fail horribly while trying! Purists say that you are not really agile until you do everything in a truly agile fashion. That might be true. But beware - don't try to start everything at the same time. That's a recipe for disaster. To make the transition to agile go smoothly, this blog post focuses on a simple way to get started with agile: perform daily standup-meetings.

Why start with stand-ups? Because they work perfectly in any environment, including traditional waterfall environments. So you can get started without even telling your boss about your agile intentions, and achieve good results even if you ultimately decide to remain non-agile. Daily stand-ups will still save you money and keep you focussed.

How does it work?

Daily standup-meetings are as simple as they sound.

You gather the team in the same spot each day, stand in a circle, and then everyone takes turns in answering three simple questions: What did I work on yesterday, what will I work on today, and is there is anything that is blocking me? Each person gets one minute on average (sometimes more, sometimes less), and overall it shouldn't take more than 10 minutes. That's it. In a way this is so old-fashioned that I wouldn't be surprised if ancient Greeks did this...

But beware! Daily standup-meetings are also a bit harder than they sound. After the "new" effect wears off, and before the long-term gains start to be fully understood, people might find it tempting to slack. So let's have a look at the long term benefits outside of having a nice team chat...

How does it help?

Introducing a quick daily standup-meeting is the cheapest most astonishing improvement I can think of for any project and one of the fundamental aspects of communication with an agile team.

• It helps your team avoid duplication: if Steve plans to work on a problem that was already solved two months ago by Sam, then you can bet that Sam (or someone else) will tell Steve before he even starts wasting time;
• Clear roadblocks: if someone is blocked by something particular, this is the perfect time to check who could help: "I have been working on bug X, with little progress. Has anyone seen X happening? Let's chat after the standup meeting";
• Detect roadblocks: if John tells the same story for the fifth time in a row, everyone else will realise that he is severely stuck and needs a hand. Catch him right after the meeting and get him some help;
• Boost team coherence and commitment: team coherence and commitment is increased when everyone speaks to everyone else on a regular basis. Especially when you don't all work in the same room, or when different members of the team work on different aspects of the same project;
• Increase project manager accountability: daily standup-meetings even make the project manager more accountable. When your manager comes to ask your status, would you dare ask him his? Probably not. But in a daily standup-meeting the manager is reporting his status to everyone else;
• Increase team member accountability: daily standup-meetings make each team member more accountable as well. A manager might not notice that Sal is over-engineering her solution just for the fun of it - but the rest of the team will;
• Spread transparency: if you are the project manager, then the project is probably transparent to you already. But there is immense value in team-wide transparency too. People are not machines, and love knowing what's happening. This is the place\!

What are the costs and risks?

Admittedly, the daily standup-meeting does take time. And some people don't like speaking in front of others. And at times the meeting can distract people from doing important work. These are all valid points. But you need to compare them to the costs that arise by not having this meeting. How much work would be lost by double work, or by people not realising they are stuck? How important is staff retention to you - and what better ways of open communication can you get at a price tag this small? As long as you apply common sense and accept that some people occasionally may miss a meeting, and that others may not be great at speaking (but do attend and listen), there is little to worry about.

A bigger risk are people who like to hear themselves talking. The stand-up meeting is not for story-telling. You need to be strict about the time-limit. If someone exceeds the standard 60 seconds, usually a smile combined with the "Time-out sign" will help. A common problem is that the boss feels inclined to turn his 60 seconds of fame into a 5 minute speech, to ensure he looks more important than everyone else. Someone needs to have the guts and speak up - in a friendly way, and depending on the kind of boss it may make sense to do this after the meeting, to come across less confronting. The same applies to discussions. Stand-up meetings are not for discussions. They should encourage discussions, but after the standup-meeting, and with just the people who really care. If someone frequently starts discussing what someone else said, take them aside and explain that the discussion is great, but it needs to happen after the meeting.

It will help to nominate one person to become the 'Standup-Champion' who will make sure that the meeting remains focused and finishes quickly and on-time. This person also makes sure the meeting happens in the first place, so the person should not be too shy to shout out "It is STANDUP time now, guys and girls!")

So what's agile about it?

You can start stand-ups without turning agile. But the impact is bigger than "just" efficiency, transparency and team building. The standup-meeting quickly turns into the catalyst meeting of the day. It regularly spawns discussions about things that were not considered in the "big plan". It helps you address problems earlier. It helps you reshuffle your road-map because you learn about good and bad news early. It encourages people to wrap up tasks more frequently, and to tackle tasks in smaller increments, because no one likes to tell (or hear) the same report every day. Standup-meetings don't make you agile, but it would be harder to become agile if you didn't have them. So they are a great first step on your long winding road to agility.

Q&A

When is a good time to get started?

As with any change, you need to be careful. People are often hesitant about change, especially if they have been subjected to frequent random change before that didn't help. So don't introduce this when the tension is high already, or when you are new and don't have any credibility yet. In software development terms, how about waiting until the current version of your software has been shipped, and everyone had a great time at the post-launch party. As with every change, first make the proposal, discuss it with the team and with relevant stakeholders, and take on their feedback before you actually get started. You may even want to try it out with a subset of people who are more keen on trying new things out. Just make sure to keep everyone in the informed.

Is it necessary all the time? Can we just have standup-meetings in times of crisis?

Standup-meetings are awesome during times of stress and crisis. They keep you focused and help avoid unnecessary work. But ask yourself: How can you avoid stress and crisis in the first place? How about avoiding straying, and how about avoiding unnecessary work in the first place? Standup-meetings are no silver bullets, but they can help avert crisis by catching problems early.

How about just doing it once a week then? That should save us some time?

The opposite will be true: People will talk for longer. People will prepare more (if they only get to talk once a week, they will want to shine\!). There will be less focus on the week ahead. If someone did double work, it will be too late to save time. If someone was stuck, she will have been stuck for a while now. And so on. Weekly meetings also have their place, but they should be dealing with larger issues, like planning and discussing the work ahead, and they should only have the people involved who are actually needed for that discussion.

Can we sit down? Standing is uncomfortable...

You are missing the point. It should be uncomfortable! You have to keep the meeting short. If everyone sits in a comfy chair and sips on great coffee, what are the chances of the meeting being short? Don't even dare to lean on that table - HTFU mate!

Is my team big enough to justify this?

At my previous job, I thought stand-ups would not be necessary for a team of just five. After all, we were sitting in the same room already, had plenty of discussions, and I talked to the individual developers quite frequently already. We started only when we ramped up to 8 and then 10 developers. Only when we did so, I realised how much I had missed what the original 5 developers had been doing (and what they had not been doing..). And of course the transparency improvement for my colleagues was even bigger than for me. Stand-up meetings even with just three people make sense, after all double work needs to be avoided all the time, and with just three people on the team you should be done in 3 to 5 minutes anyway.

Is my team too big to justify the costs?

If 16 people spend 15 minutes a day, that adds up to 4h per day, and a whopping 20h per week. This is of course still just 3% of each person's time, and a team will need to communicate anyway, so 3% is not too bad. But indeed, if your team is so large that people start complaining about the duration, or you see people dozing off, then you need to reconsider. On the Confluence team, we decided to split the big standup into two meetings when we reached 18 people. Our team consists of four sub-teams, and we do a rotation now. During one fortnight the sub-teams A and B meet at 9:45, and sub-teams C&D meet at 10:00. The next fortnight sub-teams A and C meet at 9:45, and subteams B and D meet at 10:00. And so on. Obviously you will have a little less knowledge-sharing because some people will only meet every two weeks. But of course there are so many other ways to share information too. Wiki-pages, wiki-blog posts, wiki-discussions and (sometimes) good old email don't get replaced by stand-ups after all.

When is a good meeting-time?

Learn about other agile practices

That was an interesting introduction - where can I learn more?

Try Martin Fowler for plenty of stand-up patterns and anti-patterns. I also recommend seeing how Atlassian practices agile development.

Programmers, the authors proposed, are at their most efficient when they reach a state of 'flow' in which they are able to concentrate fully on a single problem. Any developer will tell you how satisfying it is to be in this state, how productive it is... and how easy it is to be pulled out of it by the smallest interruption. What's worse, since it takes at least fifteen minutes to reach this level of concentration, any interruption no matter how small can cost you a quarter hour's productivity. A constant stream of small interruptions, say a phone near your desk that rings just twice an hour, can make a developer unproductive (and miserable) for most of the day.¹

Still, distractions are inevitable, and problems that need a developer to help solve them will come up every day. A production system may run into trouble, another developer may need help with a hairy problem, a member of another team may want to know how your product solves a particular problem, and so on. At Atlassian, we try to manage these interruptions in two ways:

1. Asynchronous Communication

Atlassian runs a Jabber server for staff, and during the day all developers are logged on to instant messaging. The advantage of instant messaging over other forms of intra-office communication is that it is so easily ignored. A developer who doesn't mind being interrupted can monitor their messages and respond as they come in. A developer who is busy working on some tricky problem can minimise the window and respond to messages at his or her leisure.

For this reason, developers will often use IM to talk to a colleague, even when the other person is sitting only a few feet away.

The other form of asynchronous communication popular in Atlassian is blogging: writing a blog post on our internal wiki is a great way to get feedback from the rest of the company about an idea without walking around interrupting people to ask them their opinion.

2. The Disturbed

Asynchronous communication introduces its own inefficiencies. It turns out there are still enough problems that need to be solved now, not at the convenience of the developer, to constitute a steady stream of interruptions to the development teams. Our solution, pioneered by the Bamboo team, was to designate a single developer to be "The Disturbed"; taking a two week stint to act as a magnet for all the questions, requests and distractions that would otherwise be distributed across the whole team.

For Confluence, the name of the current Disturbed is prominently displayed on a whiteboard just outside the team's office space. The Bamboo Disturbed has a flag on his or her monitor.

The downside of this approach, of course, is that for two weeks the Disturbed gets very little work done on whatever feature track they are nominally committed to. On the other hand, because the Disturbed isn't expecting (or expected) to get much of that work done while being disturbed, it is far less frustrating for the developer and far more predictable in terms of scheduling and estimation for the team as a whole.

To learn more about how we develop software at, visit the Agile at Atlassian site.

¹ DeMarco and Lister use this to argue that each developer should be given a private office with a door, a practice enthusiastically embraced by Microsoft in the 1980s and more recently by New York software firm Fog Creek. Atlassian remains defiantly open plan: an office layout that encourages collaboration and communication at the expense of pure efficiency.

Today in the Engineering division, we still have an agile process in each of the development teams defined for and by that team, that is as true now as it was in 2007.

The things that have changed are mostly in the category of second order effects.

Proliferation

The biggest difference in Atlassian's agile process in the last two years is that now it's not just a process for engineering work. Every department is getting agility. Top books on Agile practices are handed around, stacked available on common bookshelves and regularly seen on the tops of busy desks, scruffy yellow sticky notes protruding.

And now I'm even beginning to think that people are reading them.

Another sign of mass agile process adoption was about a year ago when middle management (which was at that time brand-new at Atlassian) started doing stand-ups. After that it seemed as though a critical mass had been reached and the practice of stand-up meetings became ubiquitous. It seems there is a culture of minimalism in meetings, not just at Atlassian, but amongst all smart people who want to get things done.

It's not just Stand-up Meetings either. User stories, You Aren't Going to Need It, Do The Simplest Thing That Could Possibly Work, Don't Repeat Yourself as well as some of the foundation thinking such as kaizen are all now part of the Atlassian lexicon.

I attribute this adoption to three things:

1. People have seen it work.
2. Leaders like it.
3. This stuff isn't new-fangled, it fits in with much established wisdom.

Scaling

Atlassian has grown quickly over its six year history, both in staff and in the number of offices. Processes have had to scale and over long distances and time zones. A good example of the growing pains is JIRA's automated test suite. It's an awesome beast. There are a growing number of unit tests (junit), functional tests (jwebunit) , web browser driver tests (selenium) and there is now a QA department who reviews and improves all aspects of our quality assurance regime which includes testing.

JIRA supports multiple operating systems, JDKs, application servers, databases, and versions thereof. And it comes in three editions (Standard, Professional and Enterprise) with different feature sets. Each edition comes in different distributions including EAR/WAR, Standalone which includes Tomcat as an application server and there is a Windows installer executable. As if that wasn't enough, it works in multiple versions of different web browsers.

So that's a lot of combinations to test! We really wish we could automatically test every combination! Currently our full functional test suite takes more than two hours to run. Multiply that by every combination and it would probably take several days of CPU time to complete. Of course Unit Tests run comparatively quickly and their green bar gives fairly good confidence, but if we want to be sure we haven't broken something in the JIRA user interface when we refactor the code, we have to wait at least two hours.

Running many thousands of functional tests is what is known as an embarrasingly parallel workload and accordingly, I am embarrassed to say we have not completed the work to properly "parallelify" it.

But now that Bamboo supports agents running in the EC2 cloud, we can crank up the compute resources available for continuous integration and we should be able to bring our functional test latency down under 20 mins without too much trouble. Fingers crossed.

Code Review

"Pair programming is the ultimate code review" goes the "agilist" line. I can see this. I can see how it works and I have felt code-review benefits from pairing. I'll be blogging about pair programming specifically in the coming weeks, so I won't exhaust this side of the argument here, however I have to say that code review is the single most effective quality assurance measure on top of a traditional agile process that I know.

There are many advantages of code review over pairing as well. Of course, these are not reasons to abandon pairing, because pairing provides much more than code review. If you are using a web-based code review tool like, ahem, oh look here, it seems we have a product I can pimp here... Crucible... the main advantages are the most obvious ones. You can scale up the number of reviewers and they can act from different locations and time zones. You also get a permanent record.

The JIRA team is probably an average adopter of code review as a practice where it remains a discretionary thing. This suits us because it lets us always use the Agile advised dosage: Just Enough.

Virtual Index Cards with GreenHopper

While we really like the simplicity of paper cards for tracking tasks in the small, the electronic version can be better in some respects. Not only for distributed teams. Of course we use JIRA for tracking large numbers of bugs and feature requests, but it was not until the GreenHopper plug-in that we have felt that JIRA was fully capable of giving us the immediate feedback and visibility that cards can give a co-located team. It also makes burndown charts which are increasingly to be found on big LCD screens around the offices.

So that's it. We like our agile process. It suits Atlassian.

Automated web testing generally has problems testing sites using Windows Integrated Authentication. This is because the nature of integrated auth is to supply the credentials of the logged-on user - which in the general case is going to be the user running the tests. This may work in some cases, but at the very least it's likely you want to test using both an admin and non-admin user. This can be done with Selenium, but requires some work to get it set up (I'm assuming we're using Selenium Server here).

The goal here is to get the browser running as a specific user (I'm not aware of a way to make the browser authenticate using other credentials). The problem is that your browser is spawned by the Selenium Server, so naturally inherits the logon credentials which started the server. Therefore there are two possible paths to take here - fool Selenium into starting the browser under another user or start Selenium itself under another user. I picked the latter option, although I expect it would be possible to do the former (possibly by replacing the browser executable with a custom script before each test?).

Selenium is fairly quick to start up - certainly in comparison to the speed at which it executes tests, so can be started at the beginning of each test without too much impact on overall testing time. (It would also be possible to maintain state and only respawn if the user changes - but this may not be simple to close on the final test - is there a hook for this?) For our testing (which tests using integrated auth), all Selenium tests create their own Selenium Server instance in a SetUp() method from a base test class (note that the tests are C#/NUnit), and close it in the TearDown() method. The C# code is below which uses the .NET API to start the process under another user - this would be marginally more complex in Java, but could be achieved by invoking the runas command to spawn Selenium.

public class SeleniumContainer
{
 private readonly Process process = new Process();
 private bool started;
 
 public SeleniumContainer(bool asAdmin)
 {
  process.StartInfo.FileName = "java";
  process.StartInfo.Arguments = "-jar lib/selenium-server.jar";
  process.StartInfo.CreateNoWindow = false;
  process.StartInfo.WindowStyle = ProcessWindowStyle.Normal;
  process.StartInfo.UseShellExecute = false;
  string[] domainAndUser = Config[asAdmin ? ConfigProps.AdminUser : ConfigProps.NonAdminUser].Split('\\');
  string password = Config[asAdmin ? ConfigProps.AdminPassword : ConfigProps.NonAdminPassword];
  
  process.StartInfo.UserName = domainAndUser[1];
  
  SecureString pw = new SecureString();
  
  foreach (char c in password)
  pw.AppendChar(c);
  
  process.StartInfo.Password = pw;
  
  DirectoryInfo currentDir = new DirectoryInfo(Environment.CurrentDirectory);
  
  process.StartInfo.WorkingDirectory = currentDir.Parent.Parent.Parent.Parent.FullName;
  
  if (Config.Exists(ConfigProps.SeleniumFirefoxProfile))
  {
   process.StartInfo.Arguments += (" -firefoxProfileTemplate " + Config[ConfigProps.SeleniumFirefoxProfile]);
  }
 }
 
 public void Start()
 {
  lock (this)
  {
   if (started)
   throw new InvalidOperationException("Selenium already started");
   if (!process.Start())
   throw new Exception("Failed to start Selenium process");
   
   started = true;
  }
  
  //give it half a second to start up
  Thread.Sleep(500);
 }
 
 public void Stop()
 {
  lock (this)
  {
   if (!started)
   throw new InvalidOperationException("Selenium not running");
   
   try
   {
    process.CloseMainWindow();
   }
   catch
   {
    //Ignore
   }
   
   //give Selenium 5sec to close before we kill it
   if (!process.WaitForExit(5000))
   process.Kill();
   
   started = false;
  }
 }
}

Testing using Firefox

Firefox in general does not send integrated auth credentials, but can be told to. To do this go to the firefox about:config screen and search for ntlm - there should be three options, two of which need changing.
network.ntlm.send-lm-response: enable this (set to true)
network.automatic-ntlm-auth.trusted-uris: set this to a string representing your host (it's a URL pattern)
Once this is working for firefox you will need to save the profile directory and pass it to Selenium when it starts so that it can use your modified firefox profile (by default it uses it's own). This is done using the -firefoxProfileTemplate parameter, as you can see in the above code. Oh, and you'll need at least version 1.0-beta-2 of Selenium if you want to use firefox 3.

The purpose of the discussion was not to compare teams; it was a subjective self-assessment of the different products' development processes, to help identify areas for improvement within teams. We also discussed some key characteristics of each team, like team size, codebase age, and something we politely refer to as "Founder involvement", to see if there was any correlation with Acid Test results (there wasn't, so I've left these out).

The results

The discussion

• The Confluence team has monthly process meetings which provide a means of "empowerment" and allows them to "reflect & adapt". Issues from the retrospectives are voted on by the team, and high-ranking issues are added to the project backlog. Per (team lead) "smoke tests" the user stories from Adnan (product manager) for completeness and clarity, before passing them on to the developers.
• The JIRA team conducts laptop demos for founders at monthly reviews. Feedback is not as continuous as other teams - milestones are less frequent but release cycles are longer. Process champions and tech leads are in place.
• The Studio team has not been tested on "accountable for results", as they have not stuffed up a release (yet). The team hasn't asked for many process changes after "reflecting" (apart from requesting the use of real hours for estimating, rather than story points).
• Generally the team leads felt frustrated at their ability to remove all impediments, particularly IT infrastructure issues. Luckily we now have an unsuspecting new IT Manager on board to help out with these.
• The FishEye/Crucible and Clover teams have had slippages. They are starting to reflect and adapt.
• It's difficult to update our internal Crowd instance for dogfooding without disrupting IT. Need to work on a better system for this.
• The Poland team holds structured retrospectives and follows up. They work well around the remote location issues.
• UI Design - need to feed this into our agile processes more effectively, and in parallel with the Product Managers' conceptual phase for the next release.
• Need to include the San Francisco team next time (they were busy actually working when we did this test).

Dean's definitions

Variable scope. Fixed quality.
1. Can the teams vary scope? - Does the team have the authority to vary scope even as the release deadline draws closer?
2. Is quality acceptable and fixed? - You can't go fast building on crappy code. Agile accomplishes little without the requisite code quality which must be built into the process through TTD, continuous integration, test automation, coding standards and the like. If you see your teams iterating or sprinting with a primary objective of working down code-level defects, then you are not truly agile.

Incremental Value Delivery
3. Is software delivered incrementally? - If your teams are sprinting but there is no feedback until the final delivery (one form of "waterscrumming") then you are not achieving agility as there is no meaningful feedback to drive the solution to an acceptable outcome
4. Is it valued and evaluated? - Demos are great, but you need real value delivery to assure fitness for intended use and early and continuous ROI. If the incremental code is not being actually used, you are not very likely to get the results you seek.
5. Is feedback continuous? - In addition to customer feedback, product owners, product managers and other stakeholders have a responsibility to continually assess the current result. This is achieved through story-level acceptance and iteration-level demos.

Empowerment and Accountability
6. Are the teams empowered? - Are the teams truly empowered and able to modify and improve their own software processes? Do they self organize and self-manage? Are resources routinely moved to the most critical bottlenecks?
7. Are they accountable for results? Empowerment and accountability are two sides of the same agile coin. Are the teams delivering reliable quality code in incremental fashion? Do you they commit to iteration and release objectives, subject to responsible scope management?
8. Do they regularly reflect and adapt? - Do the teams adhere to Agile Manifesto Principle #12? - At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
9. Does management eliminate impediments? - Is management also engaged in continuous improvement? Are impediments routinely surfaced, addressed and eliminated.

The follow-up

First off, let's start with some assumptions:

1. We want our tests to be clear to the reader and convey what we are trying to test without a lot of extra baggage.
2. We want our test failures to be clear about what the problem was and not have to spend copious amounts of time figuring out what the problem in the result is.
3. We want to follow the "One Assert Per Test" TDD rule - or get as close as we can to it.
4. Implementing the equals method purely for comparisons in testing is a dubious, potentially wrong and harmful, practice.

That last one probably merits a blog all of its own. The problem is that for some objects, domain entity objects as an example, implementing equals and hashCode in terms of the complete state of the object is not usually what you want. Often, for these types of objects, there is an identifier member such that, if 2 objects had the same value for the identifier member, you would consider them to represent the same entity. So you need to think long and hard about why you're implementing equals and what 2 objects of that type being "equal" really means.

If all that seems a bit abstract for the end of the week after long week of Atlassian Summit and JavaOne, we can consider another case that I ran into recently - the objects that you want to compare are of types from a 3rd party library and the developers didn't implement equals, hashCode or toString! So, you can't use assertEquals directly and even if you could any output you get is completely useless!

The pain of assertEquals

So, lets say we've got a type called Lightsaber that doesn't implement equals, hashCode, or toString. How can we compare 2 instances to be sure they're the same? The plain old JUnit with assertEquals way would be

assertEquals(thisLightsaber.isSingleBladed(), thatLightsaber.isSingleBladed());
assertEquals(thisLightsaber.getColor(), thatLightsaber.getColor());
assertEquals(thisLightsaber.getHilt(), thatLightsaber.getHilt());

That's all well and good, but there are a few problems. If there is a failure, it's hard to tell exactly which statement failed. In this trivial example it will be easier because each of the things we're comparing is of a different type. But if we had multiple assertions on Color or boolean this wouldn't be so readily apparent. Another problem is that the Hilt object returned from Lightsaber.getHilt is another object, and if it doesn't implement equals then our test is suddenly going to blow up as we need separate asserts for each of its attributes. So we can do that, and let's say we even pull these asserts out of our test into a separate assertEquals(Lightsaber, Lightsaber) utility method so we can reuse it all. We're still left with a problem: the tests only run until the first mismatched field. Instead of showing us all the failures, we only see the first one that is detected which leads to a long test-fix-test-fix-test cycle.

Enter Hamcrest

So, what's a guy to do? We've been using Hamcrest for a while on the SF team. What first attracted me to Hamcrest is its really nice, fluent language for making assertions

assertThat(anakin, is(equalTo(vader));
assertThat("May the asserts be with you", containsString("assert"));
assertThat(weaponsOfGalacticDestruction, hasItems(deathStar, sunCrusher));

but what kept me coming back for more is the way you can define custom Matchers to do the comparisons. Each of those statements above uses the same assertThat method, but a different Matcher is used for the actual comparison. There are all sorts of Matchers built into the core library, but it is trivially easy to implement our own as well.

// factory methods for fluent language
static Matcher<? super Lightsaber> equalTo(Lightsaber lightsaber) {
    return new LightsaberIsEqual(lightsaber);
}


static Matcher<? super Hilt> equalTo(Hilt hilt) {
    return new HiltIsEqual(hilt);
}

private class LightsaberIsEqual extends BaseMatcher<Lightsaber> {
    private final Lightsaber lightsaber;
    private final Matcher<? super Hilt> hilt;

    LightsaberIsEqual(Lightsaber lightsaber) {
        this.lightsaber = lightsaber;
        hilt = is(equalTo(lightsaber.getHilt()));
    }

    public boolean matches(Object o) {
        Lightsaber l = (Lightsaber) o;
        return lightsaber.isSingleBladed() == l.isSingleBladed()
            && lightsaber.getColor().equals(l.getColor())
            && hilt.matches(l.getHilt());
    }

    public void describeTo(Description desc) {
        desc.appendText("{singleBladed is ")
            .appendValue(lightsaber.isSingleBladed())
            .appendText(", color is ")
            .appendValue(lightsaber.getColor())
            .appendText(", hilt ")
            .appendDescriptionOf(hilt);
            .appendText("}");
    }
}


// HiltIsEqual would look much the same, comparing the attributes in the matches() method and 

Now our assertions in our test look like

assertThat(anakinsLightsaber, is(equalTo(lukesFirstLightsaber));

If this were to fail the output would look like

  Expected: is {singleBladed is true, color is GREEN, hilt is {...}}
  but: Lightsaber@35324

That's better, at least it shows us what we should have gotten. But we're still not seeing what failures there were.

Describing mismatches

If that's all there was to the story, it wouldn't be much of a story and wouldn't be that much more compelling than having an assertEquals(Lightsaber, Lightsaber) method. But the story doesn't end there, it continues with the release of Hamcrest 1.2. The most important new bit for us is "added mismatch reporting". Using this we can take care of that second part of the output, with an added benefit.

private class LightsaberIsEqual extends TypeSafeDiagnosingMatcher<Lightsaber> {
    private final Matcher<? super Boolean> singleBladed;
    private final Matcher<? super Color> color;
    private final Matcher<? super Hilt> hilt;

    LightsaberIsEqual(Lightsaber lightsaber) {
        singleBladed = is(equalTo(lightsaber.isSingleBladed()));
        color = is(equalTo(lightsaber.getColor()));
        hilt = is(equalTo(lightsaber.getHilt()));
    }

    public boolean matchesSafely(Lightsaber lightsaber, Description mismatchDescription)
        boolean matches = true;
        mismatchDescription.appendText("{");
        if (!singleBladed.matches(lightsaber.isSingleBladed())) {
            reportMismatch("singleBladed", singleBladed, lightsaber.isSingleBladed(), mismatchDescription, matches);
            matches = false;
        }
        if (!color.matches(lightsaber.getColor())) {
            reportMismatch("color", color, lightsaber.getColor(), mismatchDescription, matches);
            matches = false;
        }
        if (!hilt.matches(lightsaber.getHilt())) {
            reportMismatch("hilt", hilt, lightsaber.getHilt(), mismatchDescription, matches);
            matches = false;
        }
        mismatchDescription.appendText("}");
        return matches;
    }

    public void describeTo(Description desc) {
        desc.appendText("{singleBladed ")
            .appendDescriptionOf(singleBladed)
            .appendText(", color ")
            .appendDescriptionOf(color)
            .appendText(", hilt ")
            .appendDescriptionOf(hilt);
            .appendText("}");
    }
}
    

static void reportMismatch(String name, Matcher<?> matcher, Object item, Description mismatchDescription, boolean firstMismatch)
{
    if (!firstMismatch)
    {
        mismatchDescription.appendText(", ");
    }
    mismatchDescription.appendText(name).appendText(" ");
    matcher.describeMismatch(item, mismatchDescription);
}

Now, let's say I try and do this assertion

assertThat(lukesFirstLightsaber, is(equalTo(maceWindusLightsaber)));

That should clearly fail, and when it does we get the following output (assuming the hilts are the same)

  Expected: is {singleBladed is true, color is PURPLE, hilt is {...}}
  but: is {color is GREEN}

But wait, what's this?!? In the output it only shows us the color of Lukes first lightsaber. Is that a bug? Lets see, they're both single-bladed, we assumed the hilts were the same, but we know Mace Windu had a purple blade and Lukes had a green one. Why, it's only showing us the field that didn't match! How cool is that?!? For this example it would've been pretty trivial to determine where the mismatch was, but suppose we didn't make the assumption that the hilts were the same. Let's say they're the same, except for the size. Then the output would have been

  Expected: is {singleBladed is true, color is PURPLE, hilt is {size is LARGE, ...}}
  but: is {color is GREEN, hilt is {size is MEDIUM}}

Even with composite objects we can see at a glance what the differences are!

A question you are probably asking is whether this extra effort is really worth it. After all, this is Java and there is a bunch of boiler plate that wouldn't be necessary if we went the simple route of the assertEquals(Lightsaber, Lightsaber). I can only speak from my personal experience, but doing it this way has vastly improved my ability to write new test and functionality for the OAuth plugins that I have been working on. It was a major pain to go through and write all the matchers for our own OAuth objects and the OAuth.net objects, but it was a write them once and enjoy the benefits for a long time to come. Every new test I write where I see the abbreviated output that tells me the exact fields that were wrong makes me thankful I put the extra effort in because it saves at least 5-10 minutes everytime there is a test failure. And if it helps me, right now, as I'm writing the tests, I feel confident that it will be a tremendous help to whoever might pick up the code after me and be expected to make changes and add new stuff. Because I know from experience what a pain it is to make a change, run the tests and see

  Expected: <BIG GLOB OF OBJECT AND FIELD ATTRIBUTES GENERATED WITH ToStringBuilder>
  but was: <OTHER BIG GLOB OF OBJECT AND FIELD ATTRIBUTES WITH MINOR DIFFERENCES GENERATED WITH ToStringBuilder>

What I would typically wind up doing was copying and pasting to a text editor and lining everything up and trying to scan for what changed. But no more! Now my unit tests actually tell me what is wrong!

There's another huge benefit of creating these equal matchers, you can use these them with Mockito using argThat, try that with assertEquals(Lightsaber, Lightsaber)!

Additionally, after showing this to some of my colleagues here at Atlassian, it was suggested that I should be able to write a Matcher that does a deep comparison of two objects of the same type using reflection. It's a good idea and is something that I am working on in my 20% time. I don't expect it to be something that will take away the need to write some custom equal Matchers, but if I can get it to a point where it works in 80% of the cases, then I'll be very happy as much of the above boiler plate can be eliminated. More on that to come in the future

The Plugin Exchange was built out of a need to consolidate and scale our burgeoning plugin libraries. Over the last couple of years, interest in developing and using plugins with Atlassian's products, particularly Confluence and JIRA, has skyrocketed. But we've started to become a victim of our own success: between our plugin spaces at confluence.atlassian.com, our public subversion, partner sites, etc., its become difficult to reliably find and evaluate new plugins.

Our goals then with the Plugin Exchange have been two-fold: one, make it easier for users to find and install plugins, and two, make it easier for developers to add new plugins. Towards the first goal, we have built a new application from scratch (using Grails) to warehouse all the plugins and make them discoverable by product, by category, by search, by popularity, or any combination of these. Towards the second goal, we have made the process of adding a plugin as simple as filling in a form with information about your plugin, including links to download jar, issue tracker, etc., adding version info, screenshots, etc. and submitting to Atlassian for approval. No more maintaining Confluence pages, editing XML files in your source, etc., we've made the process of adding and maintaining a plugin as easy as possible.

But it doesn't end with the Plugin Exchange. Having a site that makes it easy for you to find and add new plugins is great, but what we really wanted was a way to get at this information from the product itself. To that end, we've built a new plugin manager, dubbed the Plugin Exchange Manager, to handle installation and management of plugins in our products. We've built this using Atlassian's Plugins 2.0, meaning that it will work in any product that supports Plugins 2, such as Confluence and JIRA. For a sneak preview, check out the video below. The Plugin Exchange Manager isn't ready for release quite yet, but we'll get an alpha release out there as soon as we can.

Please head over to the Plugin Exchange and have a look. Better yet, find a new plugin, check it out, and add a review. For more about the Exchange, as well as the plugin manager, please check out the video below.

If you're a plugin developer and have any questions, please contact developer-relations@atlassian.com.

If your build times are blowing out because of long test runs (greater than ten minutes), then you are most likely suffering from CI (Continuous Integration) latency and the above problems are real problems for you and your team.

Clover can help alleviate theses problems, by optimizing both unit and acceptance tests, drastically reducing the feedback time for each commit. Below is a case study of how Clover's Test Optimization is run on the Confluence project.

Serious CI

With about 55 different CI Plans setup in Bamboo, the Confluence team are very serious about CI. So serious in fact, that if each build were to run end-to-end, a single commit would take over two days to be tested by each of those plans. Fortunately, Bamboo provides a pretty impressive CI-Cloud, that entails 20 different agents that run build plans in parallel. This makes it possible to get feedback in a couple of hours, as opposed to days.

Often, those few hours can mean the difference between one changeset being included in the build, or many. The main build can run for up to 40 minutes before a failure is detected. In that time, possibly multiple other commits have been made, making it more difficult to track down the root cause of the failure. 40 minutes is also long enough for a developer to be tempted by the ultimate SCM sin; commit-and-run.

For the past month, the Clover team have run a shadow plan of the Confluence trunk build which only runs the acceptance tests that cover code which was modified since the previous build. This is made possible by Clover's per-test coverage data, that reports which tests hit which lines of code during a test run.

Results

The optimized build (charted on the left) is configured to do a complete test run every 10 builds to refresh the per-test coverage data.

The optimized build provides faster feedback on average compared with the main build, and because it completes on average a lot faster than the main build, more builds get run in the same amount of time.

Faster Feedback

A specific case of where the Clover Optimized build failed before the full Confluence build can be seen in CCD-CONFDF-164, where it took 7 minutes to detect the Acceptance test failure, as opposed to 38 minutes in CONFFUNC-MAIN-5247. This was one case where the changeset that triggered the Optimized build was identical to the changeset that triggered the main build. Quite often, the Optimized build was started on another agent while the main build was still churning through each and every JWebUnit acceptance test.

Clover Optimized build failing in 7 minutes

The full build took 38 minutes to fail

On average the Clover Optimized Build takes just 7 minutes. This currently runs all unit and integration tests, and optimizes the long running Acceptance Tests. The main Confluence build takes 40 minutes on average to complete.

The Clover optimized build is a 'gateway' build for the Confluence CI-pipeline. It is the canary down the CI-mineshaft, if you will. If the optimized build smells danger, then it fails; preventing other builds from being triggered and hogging valuable CI cycles.

Greater CI Throughput == Clearer CI Results

The faster a build can run, the greater its throughput will be. What does this mean for a build that typically takes 40 minutes to run? This means that you get a much clearer picture of exactly which changeset has caused a build failure.

These next two screenshots show the Bamboo build history page for the full build plan and the Optimized build plan:

Full build results for the past two hours

You can see that build 5285 failed fairly spectacularly. However, who do we blame for this failure? Three developers made changes that triggered the build which causes a possible disturbance for all three devs as each tries to clear their name.

The Clover Optimized build paints a clearer picture of the situation:

Optimized build results for the past two hours

Over the past seven days there were 74 Full Confluence builds triggered. For the same time period there were 116 optimized builds, which represents an approximate 56% increase in build throughput.

Where's the catch?

Of course, Test Optimization of acceptance tests is not a silver bullet.

A full build, that runs all tests regularly should still play an important role in the CI stack. This is because there are still cases where an optimized build may pass, but a full build will fail. Since Clover tracks per-test coverage for Java source files only, it will not detect which tests to run when a non-Java source file is modified. This means that if the only modification for a changeset is for a non-Java file, such as web.xml; velocity macro; pom.xml; build.xml; .jsp (and so on) then possibly no tests will be run, causing the build to pass — when it should have failed!

The aim of an optimized build plan is for it to fail faster than a full build, and also to run more often than a full build does. An optimized build should be about quantity, whereas the main build is there for quality.

What is per-test Coverage?

Per-test coverage, is a mapping of each line of code that was covered by one or more tests, back to the tests that covered the line. As an example here is a screenshot of the Clover report for a test run of Confluence's Acceptance Tests showing all the tests that covered the "AllQuery" class:

Per-test Coverage for the AllQuery class

This shows us that "AllQuery" was covered by 16 test methods and 6 TestCases. If the class containing this code is modified in any way Clover will ensure only those 6 TestCases get run. This means the Confluence Acceptance tests can complete in just a few minutes as opposed to 40.

Per-test coverage data is also excellent for answering the question: "Is there already a test for this Class and if so, which one?".

The Benefits of Test Optimization

The Confluence team benefit by having a Clover optimized in the following ways:

• On average, they are alerted earlier to build breakages.


• When a build does break, fewer committers are involved with the breakage, making it easier to discern who broke the build.


• Fewer CI resources on our build server are consumed by Confluence's long running MAIN build, thereby reducing latency of other builds.