I’ve been developing using Java for around ten years now and during that time it’s been a pretty happy relationship. I’ve got used to the fact that some changes I make require a server restart whilst others can be seamlessly hot-swapped leaving me free to carry on with my work. I’ve accepted this for a long time, but for some reason I’ve really started to get sick of restarting my server. As discussed previously:

The longer a build-deploy cycle takes, the more compelling testing becomes. On many occasions I’ve seen people finally give in and start to write unit tests not when shouted at and told to by the lead developer, but instead, when they are sick of waiting for their application to load for the 100th time and just want to get their work finished.

These frustrating server restarts have led many people, to rely on testing much more than ever before and to shift the focus away from debugging and logging. By moving the focus away from compile-build-deploy-test and forcing testing earlier into development, productivity can be greatly improved helping the developer to say in the zone. Although testing helps, it’s still not enough and sometimes it feels like there needs to be more support.

After searching for more I stumbled across JavaRebel which boasted some very impressive features in comparison to the standard hot swap technology. After using their ROI calculator it claims the cost of JavaRebel could be repaid within six days which would seems like a pretty convincing argument to even the most frugal of managers. This all sounds great and confirms exactly how I’ve been feeling lately. There has to be a better way to do things, it has to be easier and simpler to be more productive. Maybe JavaRebel is the answer, but after recently spending some time using Grails, I wonder if I’ve found another potential solution to my problem.

In one evening of coding with Grails, I recently managed to clock up several hours without a single server restart (this was only cut short by the need to sleep). This is made possible by a feature within Grails known as auto reloading. As the documentation states this isn’t without it’s quirks, but when using this with Grails I really do feel productive. By removing the necessity to restart my server when I make changes, I’ve removed that frustrating minute between stopping and starting which not only seem to take an age but also distract you from your train of thought and send you over to have a quick browse on DZone. Grails has so far provided a really impressive and effortless development experience and most of all it’s confirmed something to me, “I’m Sick of Restarting My Server!”.

How can we get teams to write tests as they write code?

For many developers the biggest challenge to writing tests as they write the code is a behavioral one. They are not use to writing tests alongside the code and usually test-last once all the code has been written. Some developers do exactly the same thing when writing JavaDoc, with very similar results. If developers don’t like writing tests, they really don’t like writing lots of tests right at the end of a project. Typically when jobs like this are left right until the end of a project, they either don’t get done, or don’t get done very well.

The main problem with this approach is that the developer doesn’t really benefit from the tests during development. When the code is developed, there is a reliance on logging, debugging and manual testing to ensure the code works as expected. When changes are made to the code, the whole cycle must be repeated again. This feedback cycle can very quickly decrease the velocity of development.

By writing the tests at the same time as the functionality, it not only saves a marathon session of writing tests at the end of a project, it also means the tests are used to help develop the code. There is confidence in the changes being made, and if tests are continually run it’s easy to know which code change broke the tests. This leads to a much shorter feedback cycle when developing and subsequently over time, developers are more productive.

@Test(expected=IllegalArgumentException.class)
public void nameCannotBeNull() {
    // now we've finished coding
}

@Test(expected=IllegalArgumentException.class)
public void nameMustNotBeEmpty() {
    // it's time to write some tests
}

@Test(expected=IllegalArgumentException.class)
public void nameIsTooLong() {
    // I really wish I'd started sooner
}

Developers really don’t like writing lots of tests for code they’ve already written.

What about all the existing untested code?

If developers don’t like writing lots of tests at the end of a project, they really really don’t like writing tests for legacy code. This single task is one of the most hated and time consuming to complete. The typical problem is that the code wasn’t designed for testing, hence the code must be refactored before tests can be written. The real problem with this approach is that tests are actually supposed to catch changes that might break the code. It really is a catch-22 situation, as performing the refactoring to allow tests, may actually introduce changes which break the code.

Writing tests for legacy code can be extremely time consuming. One of the biggest problems (if it is actually possible to write tests), is that it isn’t always obvious what the code was supposed to do in the first place. Even if the original author is still working on the project, the code isn’t fresh in their mind, and as such, there is going to be an initial period of familiarisation. Even if this is successful, it can be extremely time consuming and sometimes very haphazard.

Can the project afford to not refactor the code and have limited test coverage? Can the developers risk making changes and potentially introduce bugs in the short-term thus allowing long-term stability? Ulimately there is a tough decision to make here, but the situation isn’t going to get any better. Retrospective testing costs more time and money as development progresses, can this be left for the time being or is it wiser to tackle this problem now? Fixing defects later in the process will inevitibly cost more time and money. Is manual testing alone sufficient enough to find enough of the defects?

When modifying or adding to legacy code I personally prefer either writing tests first, or performing the minimum amount of refactoring required to add tests. The sooner the code has tests, and the less code changes I have to make to allow testing, the happier I am.

public class TestsForReallyComplicatedCode {
    @Test
    public void todo() {
        // ok now I'm scared!
    }
}

Is it a really a good idea to let this problem get any worse?

Conclusion

Test-driven and test first development have become very fashionable over the past few years. Ensuring code has tests is one of the core Extreme Programming rules, but it can also be one of the most difficult to introduce to a team. Although the principles are very straight forward, many teams have great difficulty adapting to an automated testing approach. Some of these teams don’t have any existing testing strategies in place, meaning that incorporating automated testing into development seems like an unattainable goal. In this series of articles we’ve covered some of the common problems encountered when trying to incorporate testing into development including:

• We DON’T do testing
• I don’t write tests
• I don’t know what I’m supposed to be testing
• Nobody knows anything about testing
• We aren’t allowed to write tests
• How can we get teams to write tests as they write code?
• What about all the existing untested code?

Developers are willing to write tests, they just need some guidance. Anyone trying to encourage the adoption of automated testing needs to make sure they are approachable and on hand throughout the process to answer any questions and try to quell any frustrations before they occur. Once developers understand how to write tests effectively, it soon becomes second nature to them. These are indeed testing times.

References

• Testing times Part I: How do team introduce automated testing?
• Testing times Part II: Why aren’t people writing tests?
• Testing Times Part III: Are developers able to write tests?

Nobody knows anything about testing

So many teams I’ve encountered in the past don’t automate testing because neither they nor any of their team have used this approach in the past. In the previous article one of the comments from PhiLho was about this very subject.

Beside the classical problem of time, there is even some base questions. For example, I write a private method in Java doing some algorithm, pattern matching, date computation, etc. How do I test it since it is private?

A typical response to a question like this is, What do you think about it? How would you test the method? Have a discussion about it in the team, have a debate about it. Try something out and see how well it works out. Throw it away and try something else completely different.

Well we could use reflection but is that really such a good idea, it seems a little hacky and a complete sledge hammer for a simple problem. We could just lower the visibility rules, how about package private? If the test hierarchy mirrors the original source are what issues are we introducing here? Do we really need to test the method directly, won’t it be tested implicitly by executing the calling code?

All of these questions have many answers and I’m sure a quick search will yield results to prove these debates have and still rage on. The interesting part in all of this is having a go and seeing what works for your team. “I didn’t want to do it because I didn’t want to do it wrong”, is such a common attitude amongst some teams. I would actively try to convince people that you have to fall down and make some mistakes to learn, most of the time without any previous experience the only way to find out is to try! If you want to jump start this process there are many excellent blogs, discussion groups, mailing lists, forums and books you just need to search for them, ask for recommendations and see what works for other people. But if there is nobody in your team that knows anything about testing, somebody is always going to have to take the first step to encourage the learning process.

We aren’t allowed to write tests

There are still some teams out there that are not allowed to write unit tests. There seems to be a couple of popular variations on this story; people that are not allowed extra time to add unit tests to their project and managers that don’t want developers wasting their time writing tests. In either of these situations, people have to understand the positives that come from testing and how this can improve both the teams development and it’s overall velocity.

One of the biggest preconceptions about testing, is that developers who write tests deliver code slower than those who don’t. Although some developers can release code more quickly without tests, this trend doesn’t typically last. When new code is developed it requires testing, when it’s integrated with the rest of the code base it has to be tested again, and each an every time the code is modified it requires further testing. If this testing has to be performed as a manual activity, it either doesn’t get done, doesn’t get done very well or doesn’t get done at all.

It is simply too time consuming to undertake this type of testing on a regular basis. The untested code can be a constant source of recurring bugs or in the best case it just takes longer to maintain than it should. In this situation, automated testing has to be an essential part of the development process. Manually testing can start to become a bottle neck in the process very quickly. Once automated tests are written however, a development team can prevent then wasted effort of excessive manual testing.

Some developers may initially take longer to develop code with tests, but this is a common symptom of any new approach. Once developers understand how to write tests effectively, it becomes second nature to them. Developers then tend to write test-backed code at least as fast as using a logger and debugger to develop the code. The longer a build-deploy cycle takes, the more compelling testing becomes and also the more productivity gains can be achieved.

@Test
public void codeDeliveredOnTime() {
    assertTrue(true);
}
 
@Test
public void confidenceInCode() {
    assertTrue(true);
}

If the code is still delivered as expected, does it matter if automated testing is helping this process along?

I’ve since recommended it several times to other people who were interested in building applications new applications. Most of the this feedback has been extremely positive so I’ve managed to get around to having another look at it recently. After @snaglepus suggested I check out Build twitter in Grails in 40 minutes (registration is required) I have to say I was very impressed. Since watching that I’ve gone from knowing absolutely nothing about Grails (three years it too long to keep information in memory) to building a sample application very quickly. After finding plug-ins for many of the tasks I wanted, I’ve implemented the logic I need and let the tool do the rest of the heavy lifting work which led to a very productive evening of coding.

I really love it when you find technology which just seems to work and does exactly what you expect it to do. There is still much I want to learn about Grails and I’m sure there are many other tips and tricks I can pick up along the way. At the minute I’m just looking through the many Grails books to find a winner and I think I have in The Definitive Guide to Grails, Second Edition. If you haven’t looked at Grails before, I really recommend you check it out and see what it can do for you. And if you aren’t following me on Twitter, I’m @karldmoore and I’m sure there will be a few more tweets about Grails ;-).

I don’t write tests!

As discussed in a previous post, the best approach to improving a teams testing practices, is to work closely with the developers to understand the real problems they are facing. Why aren’t they writing tests? Is anything preventing them from writing tests? What are the problems they are facing when writing tests? Are they receiving enough help to understand approaches to testing?

For the developers who simply refuse to write tests, a coverage tool can sometimes be useful. By simply viewing the coverage reports on a regular basis, the teams testing efforts can be monitored. It can initially help developers to know that their testing efforts are being measured as an encouragement to write tests. If a continuous integration tool is in use, it is also possible to take the manual labour out of this task, and simply fail the build if code is added without the required tests hence lowering the coverage.

Developers aren’t stupid and so they can find ways to fake this information by simply writing tests that exercise the code without actually testing anything. Code coverage alone does not guarantee that the right tests are being written, it is simply another tool that can be used. Another approach is to ensure that a member of the test team works with the developer(s) throughout the process. Their job is to make sure they are satisfied that the deliverables passed over to the test team contain the required automated tests. Although the automated tests aren’t a replacement for manual testing, by ensuring both teams are working closely together there can be a mutual understanding and appreciation for the required testing effort. This kind of peer pressure can be a valuable last resort when all else fails.

By combining code coverage with code reviews, education and targeted reading, developers testing efforts typically improve. This is the real goal that teams should be aiming for, writing good quality tests, which overtime naturally lead to better code coverage. There are some tools that take this approach a step further, but these really are not for the faint hearted. If developers are expected to write tests, their problems have to be recognised, addressed and resolved!

BUILD FAILED: Test coverage too low…… please right some tests!

I don’t know what I’m supposed to be testing!

One of the biggest hurdles to testing adoption isn’t that developers don’t want to write tests, it’s that they are not sure what they are supposed to be testing. They are not sure what they should be testing or how they should be testing it. They are not sure if they are writing the right tests or are testing the right areas. They aren’t sure what a good unit test should look like and therefore don’t write good tests.

You can force developers to write tests, but you can’t force them to write good tests. In these circumstances, it’s fairly easy to put teams back on track (if they are willing to learn). With good example tests, lots of guidance and related reading, it’s possible to work with the developers to address the problems they are having. Pair programming and code reviews can also prove extremely useful when developers are learning to write tests. Encourage developers to ask questions and to try different approaches to their testing. It’s very easy to see what is working and what isn’t, and this feedback is essential to improving the process. Many developers are willing to write tests, they just need some guidance. Anyone trying to encourage the adoption of automated testing needs to make sure they are approachable and on hand throughout the process to answer the questions and try to quell any frustrations before they occur.

@Test
public void setFirstName() {
    User user = new User("pramatr", "pramatr@domain.com");
    user.setFirstName("Pramatr");
    user.activate();
}

Yes, you are calling methods, but are you actually testing anything?