Axel Fontaine

Maven Release Plugin: Dead and Buried

2016-04-15T00:00:00+00:00

This is the second major update to a very popular article series titled 'Maven Releases on Steroids' which I originally published in 2011. The goal of the series was to provide a simple and efficient way to produce releases on a Maven project. Not only that, but it did so in a way that is compatible with Continuous Delivery.

The original series was written for Subversion. By 2013 Git had taken over the market and so I published an update called Maven Release Plugin: The Final Nail in the Coffin that integrated both Git itself and the improvements it provided into the workflow.

With its 3.2.1 release, Maven finally introduced support for Continuous Delivery friendly versions. While not perfect, even this minimal implementation warranted this long overdue update to my previous articles.

The improvements so far

Maven has traditionally required the version number to be present in the project descriptor (pom.xml). This concept permeates through all aspects of the Maven ecosystem, culminating with the monstrosity commonly referred to as the Release Plugin.

So how big exactly was the improvement of Releases On Steroids and Final Nail over the Release Plugin? See for yourself:

	Releases on Steroids	Release Plugin
Clean/Compile/Test cycle	1	3
POM transformations	0	2
Commits	0	2
SCM revisons	1	3

The reduction in the number of Clean/Compile/Test cycles alone results in a 3x build time improvement! And this doesn't even take into consideration the other adavantages like increased reliability and recoverability. Quite the win indeed!

Turning it up to 11

Both original article series and the Final Nail update required the use of the Versions Maven Plugin to dynamically update the generic version contained in the POM to the new version assigned dynamically for the current build. This resulted in the need to run Maven twice: once for updating the version and the again with the updated version.

With the Maven 3.2.1 release and the newly introduced support for Continuous Delivery friendly versions we can now finally eliminate this extra step and get down to a single execution.

At its core the purpose of producing a release is nothing more than being able to link a version of the software as deployed onto a machine back to the matching revision of the source code in SCM. This is essential to reproduce bugs and revert to previous versions deterministically if required.

To accomplish this we have to go through a number of steps. At a minimum these include:

checking out the software as it is
giving it a version so it can be uniquely identified
building, testing and packaging it
deploying it to an artifact repository where it can then be picked for actual roll out on target machines
tagging this state in SCM so it can be associated with the matching artifact

Surprisingly simple, isn't it?

Eliminating the Versions plugin

Now let's revisit the ideas presented in Final Nail and see how they can be improved with Maven's new Continuous Delivery friendly versions support.

For this we need to ensure our POM accepts versions set externally. As this will usually only be the case on the CI machine (the one and only machine you should ever build releases on), we also need a sane default for local development to avoid having to pass a dummy version for every single local build.

Taking that into account our pom.xml now looks like this:

<project ...>
    ...
    <version>${revision}</version>

    <properties>
        <!-- Sane default when no revision property is passed in from the commandline -->
        <revision>0-SNAPSHOT</revision>
    </properties>

    <scm>
        <connection>scm:git:<<your-git-repo-url>></connection>
    </scm>

    <distributionManagement>
        <repository>
            <id>artifact-repository</id>
            <url><<your-artifact-repo-url>></url>
        </repository>
    </distributionManagement>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-scm-plugin</artifactId>
                <version>1.9.4</version>
                <configuration>
                    <tag>${project.artifactId}-${project.version}</tag>
                </configuration>
            </plugin>
        </plugins>
    </build>
    ...
</project>

And that's it! You can now produce releases on your CI server very easily by invoking:

mvn deploy scm:tag -Drevision=$BUILD_NUMBER

Where BUILD_NUMBER is the environment variable provided by your CI server to identify the current build number for the project.

For services and deliverables consumed by other teams and external parties you can also easily combine this technique with sematic versioning by prefixing the version tag in your POM with the correct semantic version. You can then automatically produce releases internally and manually updated this semantic version before each external delivery.

Example:

<version>1.2.34.${revision}</version>

Conclusion

In this article we have taken what was already a gigantic improvement over the Maven Release Plugin and simplified it even further by finally getting it down to a single Maven execution:

	This article	Releases on Steroids	Release Plugin
Clean/Compile/Test cycle	1	1	3
POM transformations	0	0	2
Commits	0	0	2
SCM revisons	1	1	3
Maven Executions	1	2	2

This is by far the simplest and most effective way to produce releases on projects using Maven and Continuous Delivery.

Along the way, we strongly focused on keeping as much knowledge as possible inside the POM. This keeps the release process concise, provides us with sane defaults and minimizes dependencies on specific environments or CI servers.

So do yourself a favor and join me in finally burying the Release Plugin for good. It's been dead for a while now anyway.

Immutable Infrastructure and Cloud Architecture Talks

2015-10-17T00:00:00+00:00

2015 has proven to be another very busy year so far. I toured quite a bit, giving lots conference talks and teaching my Architecting for Continuous Delivery and Zero Downtime course.

Here are the slides and the videos of my two latest ones:

Immutable infrastructure

App deployment and server setup are complex, error-prone and time-consuming. They require OS installers, package managers, configuration recipes, install and deployment scripts, server tuning, hardening and more. But... Is this really necessary? Are we trapped in a mindset of doing things this way just because that's how they've always done?

What if we could start over and radically simplify all this? What if, within seconds, and with a single command, we could wrap our application into the bare minimal machine required to run it? What if this machine could then be transported and run unchanged on our laptop and in the cloud? How do the various platforms and tools like AWS, Docker, Heroku and Boxfuse fit into this picture? What are their strengths and weaknesses? When should you use them?

This talk is for developers and architects wishing to radically improve and simplify how they deploy their applications. It takes Continuous Delivery to a level far beyond what you've seen today. Welcome to Immutable Infrastructure generation. This is the new black.

Immutable Server Generation: the new App Deployment - Axel Fontaine from JavaZone on Vimeo.

Architecture and Deployment of Microservices for the Cloud

The case for running an own datacenter is vanishing rapidly. The cloud seduces with a low barrier of entry and full flexibility. Infrastructure can be spun up almost instantly in a fully automated way through an API. All this with no upfront costs and the ability to decommission it just as fast. But what does this mean for our applications and their architecture? Can we just lift and shift them to the cloud?

Everything comes at a price. To be able to fully leverage the potential of the cloud, new challenges must be mastered: from data privacy and security to cost-based architectures, dynamic provisioning, service discovery and efficient deployment models.

This talk provides architects and developers clear answers and battle-tested solutions for a successful journey to infrastructure heaven.

Up in the sky: Architecture and Deployment of Microservices for the Cloud - Axel Fontaine from JavaZone on Vimeo.

Enjoy!

Flyway Java EE Hangout and Voxxed Interview

2015-02-19T00:00:00+00:00

I've been quite active lately both doing conference talks and teaching my Architecting for Continuous Delivery and Zero Downtime course.

In the midst of all this, I also had the pleasure to take time out to give some interviews. These are the two latest ones.

Flyway Java EE Hangout

Yesterday Arun Gupta, of Java EE fame, and myself hosted a joint Google Hangout about Database Migrations in Java EE using Flyway.

In this hangout you'll learn how Flyway simplifies database migrations and seamlessly integrates with your Java EE application.

Topics covered:

Need for database migration tool in a Java EE application
Seamless integration with Java EE application lifecycle
SQL scripts and Java-based migrations
Getting Started guides
Comparison with Liquibase
And much more!

Voxxed interview on the Mainstreaming of Continuous Deployment

Back at Devoxx in Antwerp, I sat down with Lucy Carey, from Voxxed, where we discussed the mainstreaming of Continuous Deployment and my work at Boxfuse.

Enjoy!

Jfokus 2014 and Nighthacking interview

2014-02-11T00:00:00+00:00

Last week I had a full week of fun at Jfokus in Stockholm. I had a very busy agenda with:

a tutorial on monday
a talk on wednesday
a sold out post-conference training on thursday and friday

Here are the slides of my talk:

Nighthacking Interview

On wednesday I also had the pleasure to sit down with Stephen Chin of Nighthacking.com and the Java Youtube channel for an interview about Continuous Delivery. I gave a quick demo of things to come in 2014. Check it out and stay tuned!

Additional Continuous Delivery training in Stockholm

The Jfokus course sold out very quickly, so we decided to run an additional Architecting for Continuous Delivery: from Zero to Hero training in Stockholm on 12-13 March 2014. Seats are limited and most are gone already, but there are still a few remaining places. So don't wait if you want to participate!

Looking forward to see you there!

E-book and JAXenter Interview (in German)

2013-11-15T00:00:00+00:00

This fall I have spoken at a large number of conferences. Last week at W-JAX I sat down with Claudia Fröhling from Java Magazin for a brief interview about Continuous Delivery. You can enjoy it in here (in German):

Java Magazin article as E-book

This week my Architecting for Continuous Delivery article (in German) for Java Magazin has also been published as an E-book, as part of the Shortcuts series from entwickler.press:

If you haven't read it already, you can now enjoy it both on Kindle and on your iPad or iPhone.

Happy reading!

Java Magazin article and Fall 2013 speaking tour

2013-10-10T00:00:00+00:00

I have been giving talks about Continuous Delivery over the last three years. During these talks, I have had the chance to meet lots of interesting people with whom I had many great conversations.

While talks are great as you can interact and really get the message out, they do have one problem: reach. You simply can't pack 20,000 people in one room. I was therefore delighted when Claudia Fröhling, the editor-in-chief of Java Magazin, asked me to write the leading article for their upcoming Continuous Delivery issue. Java Magazin is Europe's largest print magazine about Java and therefore a great platform to spread the word!

The issue finally got published this week:

It contains both my article titled "Architecting for Continuous Delivery" and an interview with me as author of the month. The iPad edition also contains a video of my Continuous Delivery talk at JAX 2013 in Mainz.

So if you speak German, are interested in Continuous Delivery and haven't had a chance to attend one of my talks yet, make sure to check it out!

Conference speaking tour Fall 2013

I started this year with a spring speaking tour which took me to 33rd Degree in Warsaw, Devoxx FR in Paris, Confess in Vienna and finally JAX in Mainz. It was great fun and a good chance to flex my language muscles, as I gave talks in French, German and English.

After the summer break, the conference season has kicked in again. My speaking tour for the fall 2013 already took me to Oslo for a Flyway talk at JavaZone, and to Göttingen for a sold-out Continuous Delivery workshop.

I'm in Kiev in the Ukraine this week, as I will giving the keynote at the XP Days conference, as well as giving a Flyway talk later that day. After that in November, I will in Munich for W-JAX, before heading to Antwerp for Devoxx and finally to Gothenburg, where I will return to jDays.

I will then be giving a final talk for this year at the Lightweight Java User Group in Munich to round things off.

If you are at any of these events, come and say hello! I'd love to have a chat with you!

Maven Release Plugin: The Final Nail in the Coffin

2013-08-15T00:00:00+00:00

Update: I updated this article in 2016 as Maven Release Plugin: Dead and Buried

Two and half years ago, I published a very popular article series titled 'Maven Releases on Steroids'. The goal of the series was to provide a simple and efficient way to produce releases on a Maven Project. Not only that, but it did so in a way that is compatible with Continuous Delivery.

So how big exactly was the improvement of Releases On Steroids over the Release Plugin? See for yourself:

	Releases on Steroids	Release Plugin
Clean/Compile/Test cycle	1	3
POM transformations	0	2
Commits	0	2
SCM revisons	1	3

Fast forward to 2013

The original article was written in January 2011. Two and a half years have passed, and boy did thing change in the SCM space! In 2011, CVS was dead and burried, Subversion was well-established and both Git and Mercurial were on the rise and battling it out to be the Next Big Thing™.

And then one thing happened: GitHub's popularity exploded. It triggered a mass-exodus from Sourceforge, Java.net and Google Code. The SCM war had been decided. Git was the new King of the Hill.

Git then trickled down from its open-source roots into the deep dark SCM alleys of the enterprise, effectively relegating Subversion to legacy street.

This change brings a number of advantages, of which one is of particular interest to us today: massively improved initial checkout performance. git clone is orders of magnitude faster than svn checkout!

The Releases on Steroids process comes from a time where an initial checkout was slow. To keep performance in check the practice was to update the working copy with the latest changes instead of checking it out fresh every time. This meant that for the update to succeed the working copy could not contain local changes, as these would cause conflicts.

The Releases on Steroids process went through great lengths (and worked around many Maven bugs) to ensure this.

But now with the performance of Git, things have changed. And in change lies opportunity. So let's revisit Releases on Steroids and bring it into 2013's brave new world!

Revisiting the Release Process

Even though technology has evolved, the purpose of a release hasn't. What we want is being able to link a version of the software as deployed onto a machine back to the matching revision of the source code in SCM. This is essential to reproduce bugs and revert to previous versions deterministically if required.

To accomplish this we have to go through a number of steps. At a minimum these include:

checking out the software as it is
giving it a version so it can be uniquely identified
building, testing and packaging it
deploying it to an artifact repository where it can then be picked for actual roll out on target machines
tagging this state in SCM so it can be associated with the matching artifact

Pretty simple, isn't it?

Releases with Maven and Git

Let's see what this translates to in a Git and Maven world:

Checking the software out: git clone
Giving it a version: mvn versions:set
Building, testing, packaging and deploying it to an artifact repository: mvn deploy
Tagging this state in SCM: mvn scm:tag

But first, the POM needs some preparation.

Having the project version come from outside means we can use a simple generic snapshot version for the code checking into SCM. Our pom.xml then looks like this:

<project ...>
    ...
    <version>0-SNAPSHOT</version>
    ...
</project>

We can now add our project's Git and artifact repository urls, as well as some basic plugin configuration:

<scm>
    <connection>scm:git:<<your-git-repo-url>></connection>
</scm>

<distributionManagement>
    <repository>
        <id>artifact-repository</id>
        <url><<your-artifact-repo-url>></url>
    </repository>
</distributionManagement>

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>versions-maven-plugin</artifactId>
            <version>2.1</version>
        </plugin>
        <plugin>
            <artifactId>maven-scm-plugin</artifactId>
            <version>1.8.1</version>
            <configuration>
                <tag>${project.artifactId}-${project.version}</tag>
            </configuration>
        </plugin>
    </plugins>
</build>

And that's it! Nothing more is required!

Now let's take this 11 and integrate it in a CI environment. Jenkins is the absolute market leader, so that's what we'll use.

Jenkins integration

Prerequisites: Jenkins with a JDK and Maven configured, and both the Git and the Workspace Cleanup plugins installed.

We're going to start by creating a new Maven job and making sure we have a fresh workspace for every build:

We can now point the job to our Git repository:

The next step is to set the version upon checkout. There are two main strategies we can apply:

Create a parametrized build and pass the version in
Let Jenkins assign the version

In this example we're going to use the latter. A good version number is both unique and chronological. We're going to use the Jenkins BUILD_NUMBER as it fulfills both these criteria wonderfully.

So let's set the version in a pre-build step:

We are now ready to build, test, package, deploy (to the artifact repository) and tag (the Git repository):

All that is left now is deciding between

Triggering the release build manually (the classic way)
Polling SCM to release after every commit (the Continuous Delivery way)

Happy releasing with Jenkins!

Conclusion

In this article we've followed the footsteps of the Releases on Steroids articles and taken the concepts to their natural conclusion: the simplest and most effective way to create releases on projects using Maven and Git.

Along the way, we've kept a strong focus on keeping as much knowledge as possible inside the POM. This keeps the release process concise and minimizes dependencies on specific environments or CI servers.

So come with me and wave the Maven Release Plugin goodbye. This was the final nail in its coffin. Good riddance.

P.S.: I've published a companion repository on GitHub to get you started easily: http://github.com/axelfontaine/final-nail

Relational Database Popularity Ranking

2013-03-23T00:00:00+00:00

While planing and developing features for Flyway, I often wonder how the different relational databases supported by Flyway stack up in terms of popularity.

These are the results based on Google Analytics page view counts of the database-specific pages in the docs.

This is a very imprecise measure at best, so take it with a grain of salt.

Ranking based on the analytics data from 2013-01-01 until 2013-03-23:

Oracle
MySQL
PostgreSQL
SQL Server
H2
Google Cloud SQL
DB2
Hsql
Derby

Three big eye-openers for me so far:

The Oracle dominance is even stronger than I thought
H2 is more popular than Hsql and Derby, combined
Google Cloud SQL is surprisingly well represented

What do you think? Does this match your experience?

Flyway 2.0 and Methods & Tools article

2012-11-20T00:00:00+00:00

It was during the last days of 2010 that Flyway 1.0 saw the light of day.

Since then almost 2 years and 11 months have passed. During this time Flyway improved considerably. Next to the Maven plugin and API, there is now a Command-line tool and Ant tasks. Database support has vastly improved with the addition of SQL Server, DB2, Derby and Google Cloud SQL. Meanwhile, all of Flyway's dependencies have been made optional. Flyway now has zero required dependencies! And if that wasn't enough, many smaller features have been implemented and countless bugs have been fixed!

It's been quite a ride indeed! And now the next chapter can begin! Flyway 2.0 was released a few days ago. The number of download has quadrupled in the last year, and there doesn't seem to be any stopping in sight!

To celebrate Flyway's new release, I've also launched a completely redesigned project home page and moved it to its own domain: flywaydb.org

For announcements or referencing the project, there is now also an own twitter account: @flywaydb.

And to make getting started even easier, I also wrote an introductory article for Methods & Tools.

The future looks bright! Enjoy!

Optional Dependency Strategies for Java Libraries

2012-08-06T00:00:00+00:00

In software, contrary to common belief, lines of code are a liability not an asset. As you gradually accumulate code, little by little the pain sets in. The complexity increases. It gets harder to understand. And eventually, quality suffers.

Dependencies only exacerbate this problem. With each dependency you add, you pull in more code, more APIs to learn and more potential bugs.

Transitive dependencies take this problem to 11. You now have the possible issue of conflicting dependencies! And before you know it ... you have landed in JAR Hell! So much for laughing at our Windows friends stuck in DLL Hell...

And yet, dependencies are a necessary evil. If you want to avoid reinventing the wheel or if you want to integrate with the outside world, there very often is no way to avoid them. In the spirit of doing the simplest thing that could possibly work, strive to reduce the number of dependencies to the absolute minimum required. Every single dependency you add should be weighed carefully: does the functional benefit outweigh the complexity cost?

Special case: Libraries and Frameworks

A library or framework in broad general use (like Flyway) usually supports a wide range of scenarios for a very diverse set of users. In Flyway's case this currently means dependencies on:

10+ Jdbc Drivers
Spring Jdbc
OSGi (Equinox)
JBoss VFS v2
JBoss VFS v3
Ant
Maven

Nobody in their right mind would be keen on pulling all these dependencies in just to use the subset they need. And in fact, it wouldn't even be possible as some of them are conflicting! Yes, JBoss VFS v2 and v3, it's you I'm looking at!

How do we solve this? How do we support all users without forcing all these dependencies onto each and everyone of them?

The answer lies in optional dependencies.

There are 5 basic strategies for dealing with optional dependencies:

Create a separate module which depends on the base module and the optional dependency
Create a module per optional dependency which the base module depends on.
Mark the dependency as non-transitive and activate the functionality at runtime if present
Use reflection and activate the functionality at runtime if present
Use a Service Provider Interface and call the correct implementation at runtime

Each of these strategies has its pros and cons. Let's look at them in turn.

Separate Module

Typical use case:

Well-isolated dependencies that do not need to be combined together. In Flyway's case, this applies to the Ant and Maven dependencies. You will not need the Ant dependency when executing the Maven plugin and vice versa.

Advantage for the End-User:

Smallest amount of code in KB as no code to support other optional dependencies is pulled in.

All dependencies are transitive.

Disadvantage for the End-User:

Must know which specific module from the library should be imported.
Can only make use of one of the optional dependencies at a time.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

Disadvantage for the Library Developer:

Two modules to maintain and release instead of one.

Module per Dependency

Typical use case:

Dependencies that require a large amount of custom code to deal with. This becomes especially relevant in environments where application size has a direct impact on user experience. (Installation size vs download times on mobile, ...)

Advantage for the End-User:

Smallest amount of code in KB as no code to support other optional dependencies is pulled in.

Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

No transitive dependency support. Must manually reference the correct library module for the optional dependency.
Largest amount of modules to manage overall.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

Disadvantage for the Library Developer:

Two modules to maintain and release instead of one.
All access to the code of the optional dependency support module must be carefully guarded to avoid NoClassDefFoundError.

Non-transitive Dependency

Typical use case:

Dependencies that can be combined together. In Flyway's case, this applies to JBoss VFS, OSGi and Spring Jdbc. It is very well possible for a JBoss or an OSGi user to have Spring on board. Being able to use both at the same time is critical.

Advantage for the End-User:

A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

Must pull in some unused code to support the other missing optional dependencies.

No transitive dependency support. Must manually reference optional dependency in own project.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

A single module to maintain and release.

Disadvantage for the Library Developer:

All access to the code relying on the optional dependency must be carefully guarded to avoid NoClassDefFoundError.

Reflection

Typical use case:

Support multiple, mutually incompatible versions of the same dependency. In Flyway's case, this applies to JBoss VFS v2 and v3. Even though they have different package structures, they share the same artifact id. Maven will therefore always pull in the higher version. One of them can be declared and used as regular optional dependency, while the other must then be used via reflection (and not appear in the POM).

Advantage for the End-User:

A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

Must pull in some unused code to support the other missing optional dependencies.
No transitive dependency support. Must manually reference optional dependency in own project.

Advantage for the Library Developer:

Possibility to support multiple, mutually incompatible versions of the same dependency.

A single module to maintain and release.

Disadvantage for the Library Developer:

The code relying on the optional dependency must use reflection, sacrificing readability and compiler safety.

All access to the code relying on the optional dependency must be carefully guarded to avoid NoClassDefFoundError.

Service Provider Interface

Typical use case:
Support various implementations of the same interface. In Flyway's case, this applies to the Jdbc drivers. Flyway supports many of them, and yet they are all accessed through the same API (Jdbc).

Advantage for the End-User:

A single library module to depend upon.
Smallest amount of code in KB as only code to support the common SPI is pulled in.

Disadvantage for the End-User:

No transitive dependency support. Must manually reference optional dependency in own project.
May have to configure which SPI implementation to use.

Advantage for the Library Developer:

Common SPI to program against, with compile-time checking and IDE completion.

A single module to maintain and release.

Disadvantage for the Library Developer:

Must match the SPI with its implementation at runtime. Failure to properly do so will result in Exceptions at runtime.
Should test the library with the different SPI implementations to ensure they behave as expected.

Checking if a dependency is present at runtime

A number of these strategies depend on being able to check whether a certain dependency is available at runtime and guard against using its related features if it isn't.

This sounds complicated, but it turns out to be relatively easy on the Java platform:

public static boolean isPresent(String className) {
    try {
        Class.forName(className);
        return true;
    } catch (Throwable ex) {
        // Class or one of its dependencies is not present...
        return false;
    }
}

...

if (isPresent("com.optionaldependency.DependencyClass")) {
    // This block will never execute when the dependency is not present
    // There is therefore no more risk of code throwing NoClassDefFoundException.
    executeCodeLinkingToDependency();
}

Conclusion

The 5 different strategies each deal with different scenarios and different forces that must be balanced. Even in a relatively small library like Flyway, it wasn't possible to simply rely on a single one. Know them well and know when to use them. But if there has to be a single most important guideline to remember, let it be this one:

Always favor the convenience of the end-user over your own when designing your library or framework.

Thanks for reading this far! Find out more and stay in touch by following me on Twitter

Testing private methods: easier than you think!

2012-07-10T00:00:00+00:00

Testing private methods is something that has traditionally drawn heated debates in the Java world.

The solutions usually fall into 4 categories:

Don't test private methods
Use reflection
Use a nested class
Change the visibility

Let's look at them in turn:

Don't test private methods

This really leaves us with three choices:

refactor to make the method public in some helper class
test through a calling method with a higher visibility
give up

A delightful choice between increased bloat, higher test complexity and resignation!

No, thank you!

Use reflection

Why make things simple when you can also make them hard and long-winded? Haven't you always been dreaming of needing 10 lines of code to make a method call? (Yes, there are now even entire tools dedicated to this big operation!) Extra bonus points because you can now also prevent your IDE's compiler and refactoring tools from helping you!

No, thank you!

Use a nested class

No too bad, but with 3 significant drawbacks:

no separate sources / test sources folders
larger classes
unit test code in production binaries

We can do better.

No thank you!

Change the visibility

This leaves us with the last option. Not perfect either, but by far the most pragmatic!

It trades a slight increase in visibility (to package-protected) for greatly simplified calling (a regular method call, no less!), while still preserving the valuable sources / test sources separation.

And with a simple documentation habit, it becomes clear to everyone why this design trade-off was made:

/* private -> testing */ void myMethodUnderTest() {
    ...
}

So the next time you face this problem, choose the pragmatic route. And enjoy a coffee while everyone else is still trying to complicate things. :-)

One Minute Book Review: Getting To Yes

2011-05-30T00:00:00+00:00

Today I take a look at a big classic from the field of negotiation: Roger Fisher's and William Ury's
Getting to Yes: Negotiating an agreement without giving in.

Negotiation is really only about one thing: getting what you want. The prize can be anything from the choice of dinner for this evening, to the amount of your next raise or the details of a new strategic partnership. Private or professional life, it doesn't matter. We negotiate all the time and the outcome of our negotiations greatly influences our lives.

And yet, this is a skill most of us in software engineering didn't receive any formal training in. Professionally the matter is even worse. Most of our "opponents", ranging from managers to sales people, have both more practice and better training. This leaves us fighting an uphill battle!

We must stand up and level the playing field!

I found Roger Fisher and William Ury to be powerful allies for this quest.

Getting To Yes makes the case for principled negotiation. The main focus is reaching a mutually satisfying agreement. This implies meeting the needs of both parties, and resisting the urge of trying to "win" against your opponent.

The reason they designed this method, is because most people get stuck bargaining over positions, effectively creating a win-lose situation.

The alternative is to take a step back and understand what your opponent really wants. Once his interests have been uncovered, you can then look whether the pie can be divided or whether new options should be invented for mutual gain. A good way to achieve this is by looking if the pie can not be expanded instead. Adding new elements to the negotiation opens up new scenarios and increases the chances of reaching an agreement.

If the negotiation gets difficult, make sure you focus on the problem and not on the people. You should always insist on objective criterias. And if all that doesn't work, you should have another plan ready: your BATNA (Best Alternative To Negotiated Agreement).

I found this book to be immensely useful. And I am not alone thinking this: they have sold millions of copies!

Highly recommended.

One Minute Book Review: Getting Past No

2011-05-29T00:00:00+00:00

Last month I finally finished reading a book I've been wanting to read for a long time: William Ury's
Getting Past No: Negotiating with Difficult People.

Continuous Delivery Talk on Parleys

2011-05-27T00:00:00+00:00

Earlier this month I was at the BeJUG in Leuven to present my Continuous Delivery talk.

It was great fun and I really enjoyed meeting lots of interesting people. But what is extra nice about the BeJUG, is that it is organized by Stephan Janssen of Devoxx and Parleys fame. And so, with the precious help of Daniel De Luca they made the talk available online at Parleys.com as well!

Here are some pictures of the event (more on the BeJUG page):

And here is the presentation on Parleys:

Thank you very much to all the ones that came. It was an absolute pleasure for me. And for all the ones who didn't have the chance to come, enjoy watching it online!

Environment Detection

2011-05-26T00:00:00+00:00

One of the key aspects of my Continuous Delivery talks is applications being aware of where they are.

Environment Detection stands for the ability to sense in which environment an application is currently deployed using the simplest means at its disposal.

Once this information is available you can act on it in a number of meaningful ways, such as loading the appropriate configuration or enabling/disabling features for the current environment.

This is powerful and simpler than it sounds. I'll show you a number of ways to implement the detection and then, we'll look at how to integrate it in an application. But always remember: use the simplest technique that works for you.

The most common detection strategies are:

IP Address/Range
HostName
File contents
Operating System
System Property

Let's have a look at how to implement them in Java.

IP Address/Range

// Assuming 1.2.3.4 is the IP Address of your DEV machine
// and 5.6.7.x is the IP Range of your PROD environment
String ipAddress = InetAddress.getLocalHost().getHostAddress();
if ("1.2.3.4".equals(ipAddress)) {
    // DEV
} else if (ipAddress.startsWith("5.6.7.")) {
    // PROD
}

HostName

// Assuming abc and def are the hostNames of your DEV and PROD servers
String hostName = InetAddress.getLocalHost().getHostName();
if ("abc".equals(hostName)) {
    // DEV
} else if ("def".equals(hostName)) {
    // PROD
}

File contents

import org.springframework.util.FileCopyUtils;

// Assuming /etc/my.env exists
// and contains the name of the environment such as DEV, TEST, PROD, ...
String environment = FileCopyUtils.copyToString(new FileReader("/etc/my.env"));

Operating System

// Assuming DEV is on Windows and PROD on Linux
String operatingSystem = System.getProperty("os.name");
if (operatingSystem.startsWith("Windows")) {
    // DEV
} else {
    // PROD
}

System Property

// Assuming my.env is set
// and contains the name of the environment such as DEV, TEST, PROD, ...
String environment = System.getProperty("my.env");

And that's all there is to it !

Here is a quick example of how to leverage this newly acquired piece of information:

Say we use a framework like Spring and want to easily load the appropriate configuration for the current environment.

We use an EnvironmentDetectionService to detect the environment and store the result in a System Property called my.env. The configuration for each environment is stored in a separate folder.

We can now load the appropriate configuration like this:

<bean id="environmentDetectionService" class="my.pkg.EnvironmentDetectionService"
    init-method="detectEnvironment"/>

<bean
    class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"
    depends-on="environmentDetectionService">
    <property name="location" value="/config/${my.env}/my.properties"/>
</bean>

Environment Detection is a simple, yet very powerful capability. Start using it today and you'll never look back!

Continuous Delivery Talk at the Rheinjug (in German)

2011-04-05T00:00:00+00:00

On Thursday I had the pleasure to go and present my Continuous Delivery talk (in German) at the rheinjug in Düsseldorf.

I really enjoyed my visit. The organisation was great, and it drew a large and interesting crowd. We had a great Q&A session, and the conversation continued for a few more hours around some fine beers at the social event.

I've made the slides of the talk available on slideshare:

The video is now online as well:

Thanks to everyone who came. I hope you enjoyed it. I sure did :-)

One Minute Book Review: Inbound Marketing

2011-02-20T00:00:00+00:00

Over the Christmas holiday, I finally got around to something I had planned for a while: reading Brian Halligan's and Dharmesh Shah's very popular
Inbound Marketing: Get Found using Google, Social Media, and Blogs

The business world is changing. And it's changing fast. One of the most interesting statistics supporting this does not appear until the last chapter of the book. Here it is:

Between 1955 and 1995, on average, every year there were 20 companies leaving the Fortune 500 and 20 new ones joining. Since 1995 this number doubled to 40. Of the 500 companies in the Fortune 500 in 1995, only 250 remained in 2009!

So why did companies like Amazon, eBay and Google replace other ones likes Toys 'R' Us and Polaroid? The answer to this question is very much related to why Obama defeated McCain in the 2008 US presidential elections.

So what did the Obama campaign have in common with Amazon, eBay and Google?

They realized that throwing the most money at traditional media (television, print) to repeat the same message to as many people as possible as often as possible may not be the most effective anymore.

The point Brian and Dharmesh make is that this 50 year era of outbound marketing is coming to a close. People are getting increasingly better at blocking out these messages.

We are now at the dawn of a new era in the field of marketing: inbound marketing. Instead of focusing on bombarding as many people as possible with a message they don't want to hear, there are new rules that must be mastered on the road to success. Producing remarkable content that people want to read about is the way forward. Turning customers into fans.

And it turns out, we live in an age where it has never been easier. And yet it has never been harder. Google, Facebook, Blogs and Twitter enable anyone to produce great content and get found. Quality trumps dollars. And yet, all of a sudden the competition became global.

How can you and your company stand out and become better at this game?

This book is filled with practical advice on how to get there. Putting Google, Twitter, Facebook , LinkedIn and your blog to use, to achieve a loyal following who will help you spreading the word. And the work they will spread is one about how it is worth their time to be involved with the remarkable things you do!

Highly recommended.

Maven Releases on Steroids (3): Rounding it up with Jenkins

2011-02-10T00:00:00+00:00

Update: I updated this article in 2013 as Maven Release Plugin: The Final Nail in the Coffin

In Part 2, we looked at what changes where necessary to the Maven POMs to make it all work.

In this final part, we'll put the finishing touch by choosing a version number strategy and integrating what we have so far with Jenkins.

Version Number

A good version number has a number of properties:

Natural order: it should be possible to determine at a glance between two versions which one is newer
Maven support: Maven should be able to deal with the format of the version number to enfore the natural order
Machine incrementable: so you don't have to specify it explicitely every time

Typical candidates can be:

build number
timestamp (in a suitable format such as yyyy.MM.dd.HH.mm)
revision number (SVN only)

We must now find one we can use. Luckily for us, it turns out Jenkins has a very useful feature for this. During each build Jenkins exposes a number of environments variables. A few of these are particularly interesting:

Environment Variable Description

BUILD_NUMBER The current build number, such as "153"

BUILD_ID The current build id, such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss)

SVN_REVISION For Subversion-based projects, this variable contains the revision number of the module. If you have more than one module specified, this won't be set.

Environment Variable	Description
BUILD_NUMBER	The current build number, such as "153"
BUILD_ID	The current build id, such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss)
SVN_REVISION	For Subversion-based projects, this variable contains the revision number of the module. If you have more than one module specified, this won't be set.

In this article, we''ll use the BUILD_NUMBER variable as a version number for our releases. We could have used one of the others as well, but this one fits our need perfectly: a fine machine-incremented number with a natural order supported by Maven.

Jenkins

The first thing to do is to set up a new Maven 2/3 project.

In this project you must

configure the SCM URL
set the Maven Goals and options to
clean deploy scm:tag -DVERSION_NUMBER=${BUILD_NUMBER}

And that's it! Every time this job is run, a new release is produced, the artifacts will be deployed and the source code will be tagged. The version of the release will be the BUILD_NUMBER of the Jenkins project. Nice and simple.

A small note for Nexus users:

In order to allow Maven to upload the corrected POMs from Part 2 of this article, you need to set the Deployment Policy of the Releases repository to Allow Redeploy.

Next step

You're now ready to add a build trigger on SCM changes in Jenkins. Once you have this,every commit causing a green build will produce a new release. It's the next step up.

Download

Last, but not least, here is a sample multi-module Maven Project with all the POMs correctly set up available for download.

Conclusion

Looking back at the workflow of the Release Plugin we discussed in Part 1, we have come a long way!

If you're willing to live with 3 simple things:

you do NOT depend on SNAPSHOTs, ever.
the POM does not contain the version number
you ALWAYS build releases from a CI server, and NEVER locally on a developer's machine

you can now enjoy very fast (3x) and robust releases, without the headaches of the Release Plugin.

For reference:

Releases on Steroids Release Plugin

clean/compile/test cycle 1 3

POM transformations 0 2

commits 0 2

SCM revisons 1 3

	Releases on Steroids	Release Plugin
clean/compile/test cycle	1	3
POM transformations	0	2
commits	0	2
SCM revisons	1	3

Enjoy ! (You'll never look back :-) )

Maven Releases on Steroids (2): Preparing the POMs

2011-01-29T00:00:00+00:00

Update: I updated this article in 2013 as Maven Release Plugin: The Final Nail in the Coffin

In Part 1, I wrote about the problems with the Maven Release Plugin and what could be a better way to perform releases with Maven.

In this part, we will make the necessary adjustments to the POMs to make it all work.

Our goal is to have a single command to:

Build and Test the SCM revision currently checked out
Deploy the binary artifacts to the Artifact Repository
Tag the SCM revision with the version number of the artifacts

As we only want one commit instead of 3 per release, the version number can not be contained inside the POM anymore, as this would make it impossible to update it without updating the POM itself.
We have to balance this against the need for a default version number when performing local, non-release builds, as we don't want to pass in a version number for every build.

This means we have the following requirements:

Version number settable from outside for release builds
Default version number for local, non-release builds

This can be solved by introducing a VERSION_NUMBER property:

Parent POM:

<project ...>
  ...
  <version>${VERSION_NUMBER}</version>
  ...
  <properties>
    ...
    <VERSION_NUMBER>1.0-SNAPSHOT</VERSION_NUMBER>
    ...
  </properties>
  ...

Child POMs:

<project ...>
  ...
  <parent>
    ...
    <version>${VERSION_NUMBER}</version>
    ...
  </parent>
  ...
  <version>${VERSION_NUMBER}</version>
  ...

The VERSION_NUMBER property now determines the version of our project.

By default it is set to 1.0-SNAPSHOT for local, non-release builds.
It can also be set externally using -DVERSION_NUMBER=... for release builds.

And normally, this should be it!

Unfortunately, Maven has a minefield of bugs we need to work around. What it basically boils down to, is that Maven neglects to replace variables in installed (local repo) and deployed (remote repo) POMs. This means our POMs get deployed with <version>${VERSION_NUMBER}</version>, which causes problems at runtime.

We will need to overwrite the broken POMs with a new version that has its variables replaced with their values.

The first thing we need to add (only to our parent POM) is a way to distinguisch between snapshot and release builds:

<project ...>
  ...
  <properties>
    ...
    <releaseRepoUrl>http://my.release.repo</releaseRepoUrl>
    <snapshotRepoUrl>http://my.snapshot.repo</snapshotRepoUrl>
    <deployRepoUrl>${releaseRepoUrl}</deployRepoUrl>
    <isRelease>true</isRelease>
    ...
  </properties>
  ...
  <profiles>
    <profile>
      <id>snapshot-deploy-url-override</id>
      <activation>
        <property>
          <name>!VERSION_NUMBER</name>
        </property>
      </activation>
      <properties>
        <deployRepoUrl>${snapshotRepoUrl}</deployRepoUrl>
        <isRelease>false</isRelease>
      </properties>
    </profile>
  </profiles>
  ...
  <distributionManagement>
    <snapshotRepository>
      <id>snapshots-repo</id>
      <name>Snapshots Repo</name>
      <url>${snapshotRepoUrl}</url>
    </snapshotRepository>
    <repository>
      <id>releases-repo</id>
      <name>Releases Repo</name>
      <url>${releaseRepoUrl}</url>
    </repository>
  </distributionManagement>
  ...

And now comes the real juicy part (only necessary in the parent POM):

Fixing the POM by substituting the variables
Overwriting the existing POM in the local repo
Overwriting the existing POM in the remote repo

<project ...>
  ...
  <build>
    ...
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>2.4.3</version>
        <executions>
          <execution>
            <id>replace-pom-placeholder</id>
            <phase>package</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <resources>
                <resource>
                  <directory>${basedir}</directory>
                  <includes>
                    <include>pom.xml</include>
                  </includes>
                  <filtering>true</filtering>
                </resource>
              </resources>
              <outputDirectory>${project.build.directory}/pom-install-deploy-fix</outputDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <artifactId>maven-install-plugin</artifactId>
        <version>2.3.1</version>
        <executions>
          <execution>
            <id>overwrite-pom</id>
            <phase>install</phase>
            <goals>
              <goal>install-file</goal>
            </goals>
            <configuration>
              <packaging>pom</packaging>
              <file>target/pom-install-deploy-fix/pom.xml</file>
              <pomFile>target/pom-install-deploy-fix/pom.xml</pomFile>
              <version>${project.version}</version>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <artifactId>maven-deploy-plugin</artifactId>
        <version>2.5</version>
        <executions>
          <execution>
            <id>overwrite-pom</id>
            <phase>deploy</phase>
            <goals>
              <goal>deploy-file</goal>
            </goals>
            <configuration>
              <packaging>pom</packaging>
              <file>target/pom-install-deploy-fix/pom.xml</file>
              <pomFile>target/pom-install-deploy-fix/pom.xml</pomFile>
              <url>${deployRepoUrl}</url>
              <version>${project.version}</version>
              <updateReleaseInfo>${isRelease}</updateReleaseInfo>
              <uniqueVersion>false</uniqueVersion>
            </configuration>
          </execution>
        </executions>
      </plugin>
      ...
    </plugins>
    ...
  </build>
  ...

If you also want to deploy source artifacts, check out the Maven Source Plugin.

So far, so good: we now have taken care of the deployment in the Artifact Repository!

Let's add the final bit to enable SCM tagging...

First add the scm section to your parent POM:

<project ...>
  ...
  <scm>
    <connection>scm:my-provider:my-read-url</connection>
    <developerConnection>scm:my-provider:my-read-write-url</developerConnection>
  </scm>
  ...

And now add the scm plugin to the plugins section of the parent POM:

<plugin>
    <artifactId>maven-scm-plugin</artifactId>
    <version>1.4</version>
    <configuration>
      <tag>${project.artifactId}-${VERSION_NUMBER}</tag>
    </configuration>
  </plugin>

Done!

We can now

publish new snapshots using
mvn clean deploy
release and tag new versions using
mvn clean deploy scm:tag -DVERSION_NUMBER=1.2.3

In Part 3, we conclude our adventure by choosing a version number strategy and looking at Jenkins integration.

Maven Releases on Steroids: Adios Release Plugin!

2011-01-23T00:00:00+00:00

Update: I updated this article in 2013 as Maven Release Plugin: The Final Nail in the Coffin

One of the central themes of my current series of talks about Continuous Delivery is releasing software.

Releasing software encompasses a number of activities such as

Picking a version number
Checking out the latest revision from SCM
Building and Testing binaries
Tagging the SCM revison with the version number
Publishing the binary artifacts in an Artifact Repository
Writing release notes, announcement e-mails, etc...

In order to avoid the Works On My Machine syndrome, releases are typically built centrally on a continuous integration server such as ~~Hudson~~ Jenkins.

The Continuous Integration server serves as a choreographer between the following components:

SCM (Subversion, Mercurial, Git, ...)
Build Tool (Maven, Ant, Gradle, ...)
Artifact Repository (Nexus, Artifactory, ...)

On Maven projects, the Building, Testing, Tagging and Publishing steps have traditionally been handled by the Maven Release Plugin. It works, to a reasonable degree, and it has a decent Jenkins plugin.

Using Jenkins and the Release Plugin, here are the typical steps performed by the various components:

(Safety Notice: buckle up, it's going to be a wild ride!)

Jenkins checks out the latest revision from SCM
Maven compiles the sources and runs the tests
Release Plugin transforms the POMs with the new version number
Maven compiles the sources and runs the tests
Release Plugin commits the new POMs into SCM
Release Plugin tags the new SCM revision with the version number
Release Plugin transforms the POMs to version n+1 -SNAPSHOT
Release Plugin commits the new new POMs into SCM
Release Plugin checks out the new tag from SCM
Maven compiles the sources and runs the tests
Maven publishes the binaries into the Artifact Repository

Phew! Did you count it?

That's 3 full clean/compile/test cycles, 2 POM transformation rounds and 3 different SCM revisions!

No wonder this thing has proved to be error prone and frustrating to many developers for years now. And let's not even get me started on how well this thing blows up when another team member unsuspectingly checks in a change to the POM in the middle of this multiple transform and commit carousel...

So is there another way?

What if we could deal with only a single SCM revision (the one we want to release) going through a single clean/compile/test run before tagging the source and publishing the binaries?

Let me say this loud and clear: yes, we can!

The solution I present in part 2 does just that. This is what it looks like:

Jenkins checks out the latest revision from SCM
Maven compiles the sources and runs the tests
Maven tags the SCM revision with the version number
Maven publishes the binaries into the Artifact Repository

Simple and to the point.

Continue to Part 2 where we go into the adjustments to make to the POMs.

Ubuntu Server 10.10 on Virtual PC under Win7

2011-01-16T00:00:00+00:00

A while ago, I described how to get Ubuntu Server 9.10 to run Windows Virtual PC.

As it turned out though, the beasts were only tamed for a release or two. For Ubuntu Server 10.10 this guide needed an update, so here it is!

These are the steps required to get Ubuntu Server 10.10 to run on Windows Virtual PC under Windows 7.

For this to work you will need:

Windows 7 with Virtual PC installed
The Ubuntu Server 10.10 32-bit ISO (Virtual PC still does not support 64-bit guests)

Ready? Let's go !

1. In the Virtual Machines folder, create a new VM. Before launching it, edit the settings to mount the Ubuntu ISO

2. Launch the VM and select your language.

3. Press F4 and select Install a minimum virtual machine

4. Perform the installation all the way through.

5. After the reboot, in Grub, select the first entry and press "e" to edit it.

6. Add "noreplace-paravirt vga=771" (no quotes) before "quiet" on the second last line and press Ctrl+x to boot

7. Log in and edit the file called "/etc/default/grub" as root. Change the line with GRUB_CMDLINE_LINUX_DEFAULT in:

GRUB_CMDLINE_LINUX_DEFAULT="noreplace-paravirt vga=771 quiet"

8. Execute this command:

sudo update-grub

9. Reboot

10. Welcome to Ubuntu Server 10.10 under Virtual PC on Windows 7!

The 600 Euro upgrade that cut build times in half

2009-11-24T00:00:00+00:00

In the past, I have talked here and here about my test environment for Veturanto. Today I'll talk about my development environment instead.

My main workstation used to have the following hardware configuration:

Intel Core 2 Duo (dual core) @ 1.86 Ghz
4 GB DDR2 RAM
Western Digital VelociRaptor 300 GB @ 10000 RPM

Decent, but not the absolute latest.

About two and a half months ago, a worthy successor of the Core 2 Duo arrived on the market: the Intel Core i5. By that time, Intel had also released the second generation of its high-performance SSDs. It was time for an upgrade.

I decided to take the plunge and invested about 600 Euro in the following hardware:

Intel Core i5 750 (quad core) @ 3.01 Ghz (overclocked from 2.67 Ghz)
Scythe Ninja 2 Cooler (yes, I do share Jeff Atwood's giant heatsink fettish)
Asus P7P55D Pro motherboard
8 GB DDR3 RAM
Intel Postville 34nm SSD 80 GB

But what kind of performance do you get out of this?

Let's compare two day to day activities: the full build and the application startup times.

Here is how they fared:

	Core 2 Duo + Raptor	Core i5 + SSD
Full build	1 minute 50 seconds	45 seconds
Application startup	17 seconds	5 seconds

The difference is nothing short of phenomenal !

Go out, get a modern processor and a high-performance SSD and you'll never look back !

One Minute Book Review: Crossing the Chasm

2009-11-18T00:00:00+00:00

Last week, I finished reading Geoffrey Moore's excellent
Crossing the Chasm: Marketing and Selling high-tech products to mainstream customers.

(image courtesy of wikipedia)

Moore sets out by describing the technology adoption lifecycle, and the different demographics that compose it:

Innovators

The Technology Enthusiasts live on the bleeding edge and take pride to being the first try out new things. They generate the first buzz, but lack loyalty.

Early Adopters

The Visionaries see great new strategic opportunities in new products, and are willing the invest and participate heavily in the product development to get a game changing advantage over the competition.

Early Majority

The Pragmatists stand for continuity. They want a proven solution helping them make an incremental, predictable improvement to their business.

Late Majority

The Conservatives believe more in tradition than progress. They will finally adopt new technologies when I becomes inconvenient not do so. By that time, the products have become a commodity, that is well tested and sells with low margins.

Laggards

The Skeptics pride themselves in intentionally being behind the mainstream. They do not participate in the high-tech marketplace, except to block purchases. Their contribution is feedback about the discrepancies between sales claims and delivered products.

But then comes the key observation: the technology adaption lifecycle does not look like a nicely shaped bell-curve, but more like this:

(image courtesy of wikipedia)

Ans this will be the central theme of the rest of the book: what separates early and mainstream markets, the gap between early adopters and early majority, the Chasm, and how to cross it.

Moore makes a compelling argument around what he calls Beachhead Strategy and the Whole Product Model. He describes what it takes for a company to get a foothold in the mainstream market and expand from there.

A great read. Highly recommended.

The perils of the missing javascript block scope

2009-11-12T00:00:00+00:00

Javascript variable scope is something of a strange thing. You have a global and a function scope. But unlike most other languages, there is no block scope! This is often a cause for hard to find bugs.

Let me show you the consequences of this with a small puzzler:

<html>
<body>
<a id="link1" href="">Link 1</a> <span id="part1"></span><br/>
<a id="link2" href="">Link 2</a> <span id="part2"></span><br/>
<a id="link3" href="">Link 3</a> <span id="part3"></span><br/>
<a id="link4" href="">Link 4</a> <span id="part4"></span><br/>
<a id="link5" href="">Link 5</a> <span id="part5"></span><br/>
No Link 6 <span id="part6"></span><br/>

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script type="text/javascript">
  $(document).ready(function() {
    for (i = 1; i <= 5; i++) {
      $("#link"+i).click(function() {
        $("#part"+i).html("Clicked !");
        return false;
      });
    }
  });
</script>
</body>
</html>

Now what does this do? (you need a bit of jquery knowledge to guess it)

At first glance, it walks over the 5 links present on this page, and registers a handler for mouse clicks that adds the text "Clicked !" to the span element with the same number as the link.

Or does it?

Well, this is what the output looks like when you click on "Link 3":

It is totally wrong ! Now why is that? Well, as we have no block scope, by the time we are done with the for loop, the variable i has been incremented up to 6. It is however still in scope, and therefore this last value gets used when the closure for the click event gets called. And so it is part 6 that receives the new text !

So how can we fix this?

We have to separate the scopes, and to do that we need a new function. This is what the correct code looks like:

<html>
<body>
<a id="link1" href="">Link 1</a> <span id="part1"></span><br/>
<a id="link2" href="">Link 2</a> <span id="part2"></span><br/>
<a id="link3" href="">Link 3</a> <span id="part3"></span><br/>
<a id="link4" href="">Link 4</a> <span id="part4"></span><br/>
<a id="link5" href="">Link 5</a> <span id="part5"></span><br/>
No Link 6 <span id="part6"></span><br/>

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script type="text/javascript">
  $(document).ready(function() {
    for (i = 1; i <= 5; i++) {
      addClickEvent(i);
    }
  });
  
  function addClickEvent(i) {
    $("#link"+i).click(function() {
      $("#part"+i).html("Clicked !");
      return false;
    });
  }
</script>
</body>
</html>

With the registration of the click event taking place in a separate function (and therefore a separate scope for the variable i), this is what the output looks like:

Which is much more like what we were expecting !

So beware of the lack of block scope in Javascript! It can come and bite more easily than you think if you're not extra careful about it!

Taming the beasts: Getting Ubuntu Server 9.10 to run in Virtual PC on Windows 7

2009-11-04T00:00:00+00:00

As I wrote, I am a big fan of Windows 7 and virtualization. While my development environment for Veturanto runs on Windows, the production server runs on Ubuntu Server. Ubuntu Server is great and by far the best Linux distribution available at the moment. One piece of the Veturanto infrastructure puzzle I haven't talked about yet is the test server. As I want it to be as close as possible to the production environment, it also naturally runs the same OS. And being a virtualization fan, I want it to run in a window to boot.

However Virtual PC and Ubuntu Server have a history of making life difficult for each other and not playing together nicely. There is however a way to get them to cooperate.

So let's go and tame these beasts!

For this to work you will need:

Windows 7 with Virtual PC installed (see my previous blog post)
The Ubuntu Server 9.10 32-bit ISO (Virtual PC does not yet support 64-bit guests)

Ok, let's get started!

1. In the Virtual Machines folder, create a new VM. Before launching it, edit the settings to mount the Ubuntu ISO

2. Launch the VM and run the installation normally.

3. In Grub, select the Linux 2.6.31-14-generic-pae entry and press "e" to edit it.

4. Add "noreplace-paravirt vga=771" (no quotes) before "quiet splash" on the second last line and press Ctrl+x to boot

5. Log in and create a file called "/etc/default/grub" as root with this content:

GRUB_DEFAULT=0
GRUB_CMDLINE_LINUX_DEFAULT="noreplace-paravirt vga=771 quiet splash"

6. Execute this command:

sudo update-grub

7. Reboot

8. Welcome to Ubuntu Server 9.10 under Virtual PC on Windows 7 !

Note: Credits go to this post and this post as they were essential inspiration to arrive at this solution!

Update: Removed the bit about changing the kernel. It wasn't necessary.

Browser testing on steroids: Running IE 6, 7 and 8 simultaneously on Windows 7

2009-11-04T00:00:00+00:00

I use Windows 7 on my development machine. It works great, and I've especially come to like the deep integration features provided by Windows Virtual PC, as I'll show you in this blog post. They have been a tremendous time-saver when it comes to testing Veturanto, the website I am currently working on. I usually test the website using Firefox 3.5, Chrome 3, IE 6, IE7 and IE8.

Firefox, Chrome and IE8 all run natively on Windows 7, no problem there. But what about IE6 and IE7? These guys only run on XP (IE 6 & 7) and Vista (IE 7). Are we out of luck? Not quite. First we have virtualization to the rescue and second we don't even need our own licenses as Microsoft freely provides VHDs of both these browsers both on XP SP3 and Vista to run tests on. (Note that they do expire every few months, but Microsoft always provides new ones in time.)

This post shows how to use them most effectively to deeply integrate IE 6 and IE 7 into Windows 7.

So let's get to work!

For this to work we will need the following things:

Windows 7 with Windows Virtual PC installed. It is a free optional component of Windows 7. (On the download page, select Ultimate to reveal the download buttons. If you have Windows 7 Home Premium you can still use Virtual PC, but not XP Mode)
The XP SP3 IE6 and XP SP3 IE7 application compatibility VHDs, extracted in your Virtual Machines folder.
A Windows XP Installation CD in your CD drive (this will be needed for the Integration Components installation)

Now that we have all that ready, let's get started. The following steps describe the IE 6 installation, simply repeat them with the other VHD to install IE 7.

1. Select the IE6 on XP SP3.vhd file and click Create virtual machine to create a new VM

2. For simplicity's sake, name it to match the VHD

3. Leave the memory to its default setting of 512 MB

4. Associate the VHD with the VM and click Create

5. Select the VM (IE 6 on XP SP3.vmcx) and click Settings to mount the XP CD

6. Launch the VM by clicking Open and watch XP boot

7. By default a user IETest with Administrator privileges gets created. Assign it a password (Start -> Control Panel -> User Accounts -> IETest -> Create a password) Pick IETest again to keep it simple.

8. Update the integration components, and let the installer reboot the VM when it needs to.

9. Make sure that at the end of the installation, the checkbox "Download updates required for running virtual applications from your PC" is checked before clicking Finish.

10. Download and run the RemoteApp installer. The VM will restart at the end after you click Finish.

11. Log in and click Enable Integration features

12. When prompted for credentials, click "Use another account" and enter IETest/IETest and don't forget to tick the checkbox

13. Right click Start and select "Open all users", go into Programs and drag the IE icon from the desktop into this folder.

14. If you now go back to the Windows 7 Start menu, you will magically find the IE icon again under

All Programs -> Windows Virtual PC -> IE 6 on XP SP3 Applications

15. Repeat for IE 7 :-)

16. (optional) Pin the icons to the taskbar next to the IE 8 icon, so that they are just only click away.

And finally there you have it, IE 6, IE 7 and IE 8 all running at the same time on Windows 7:

Hope you enjoyed it and that it improved your life as much as mine when it comes to testing in different browsers !

Transparently supporting HTTP HEAD requests in Java and Spring MVC

2009-09-03T00:00:00+00:00

The http 1.1 specification (RFC 2616) defines a number of methods: OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE and CONNECT.

Of these, the most familiar are GET and POST.
Web browsers rely on these two methods to send and receive data from web servers. (In compliance with the W3C Html 4 recommendation)

GET is meant for retrieving content from a web server. The requests should be idempotent. No critical state should change. Successive requests should return the same content. (This makes them ideal candidates for caching!)

POST on the other hand is typically used for operations that manipulate content on the server, such as adding, editing or removing content.

So, what about the other methods, you might ask?

Well it turns out browsers are not the only clients talking to our web servers. The web of 2009 has two more essential infrastructure components that our servers frequently have to deal with: proxy servers and web crawlers.
And it turns out these two types of clients are very fond of a third http method: HEAD.

HEAD is identical to GET except that only the http headers are returned. The body is discarded. This is primarily used for checking the validity of URLs. The load on the server will most likely remain the same as the content-length header must be returned (and thus potentially calculated based on the generated response body). Only the bandwidth is saved.

How do Java servlets deal with this?

Not too bad it turns out ! Deep inside the HttpServlet class (part of the Servlet API 2.5), we find the following code:

protected void doHead(HttpServletRequest req, HttpServletResponse resp)
 throws ServletException, IOException {
    NoBodyResponse response = new NoBodyResponse(resp);
 
    doGet(req, response);
    response.setContentLength();
}

Here is what the NoBodyResponse wrapper does (from the source code documentation):

A response that includes no body, for use in (dumb) "HEAD" support.
This just swallows that body, counting the bytes in order to set the content length appropriately. All other methods delegate directly to the HttpServletResponse object used to construct this one.

So this means the standard way of the servlet api to deal with with HEAD requests consists of:

Wrapping the response using the NoBodyResponse in order to suppress the body, but preserve the headers.
Execute the GET functionnality of the application (with the wrapped response object)
Set the content-length header of the response
Return the response headers to the client (without the body)

Sounds exactly like what we need, so where is the problem?

Comes in Spring MVC...

A typical Spring MVC 2.5 controller looks like this (from the Spring 2.5.6 reference documentation):

@Controller
@RequestMapping("/editPet.do")
@SessionAttributes("pet")
public class EditPetForm {
    private final Clinic clinic;

    @Autowired
    public EditPetForm(Clinic clinic) {
        this.clinic = clinic;
    }

    @ModelAttribute("types")
    public Collection<pettype> populatePetTypes() {
        return this.clinic.getPetTypes();
    }

    @RequestMapping(method = RequestMethod.GET)
    public String setupForm(@RequestParam("petId") int petId, ModelMap model) {
        Pet pet = this.clinic.loadPet(petId);
        model.addAttribute("pet", pet);
        return "petForm";
    }

    @RequestMapping(method = RequestMethod.POST)
    public String processSubmit(
            @ModelAttribute("pet") Pet pet, BindingResult result, SessionStatus status) {

        new PetValidator().validate(pet, result);
        if (result.hasErrors()) {
            return "petForm";
        }
        else {
            this.clinic.storePet(pet);
            status.setComplete();
            return "redirect:owner.do?ownerId=" + pet.getOwner().getId();
        }
    }
}

The editPetForm controller will respond to the /editPet.do URL.

GET /editPet.do executes the setupForm method.
POST /editPet.do executes the processSubmit method.

But what about HEAD /editPet.do?

It generates an error! Spring MVC cannot find a method annotated with

@RequestMapping(method = RequestMethod.HEAD)

and throws an exception.

There are two solutions to this problem.

Solution #1
Add the missing RequestMapping annotation to all controllers handling GET requests. This solves the problem, but not without significant drawbacks:

It is verbose (this annotation must be added to all relevant controllers)
It is tedious (every controller must be reviewed)
It is error-prone (the burden lies on the developer not to forget this)

But luckily there is an alternative...

Solution #2
Add a servlet filter (in web.xml) in front of the Spring MVC servlet to lie about the http method and present all HEAD requests as GET.

Here is the code:

//Imports and documentation have been omitted...

public class HttpHeadFilter implements Filter {
    public void init(FilterConfig filterConfig) throws ServletException {
    }

    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;

        if (isHttpHead(httpServletRequest)) {
            chain.doFilter(new ForceGetRequestWrapper(httpServletRequest), response);
        } else {
            chain.doFilter(request, response);
        }
    }

    public void destroy() {
    }

    private boolean isHttpHead(HttpServletRequest request) {
        return "HEAD".equals(request.getMethod());
    }

    private class ForceGetRequestWrapper extends HttpServletRequestWrapper {
        public ForceGetRequestWrapper(HttpServletRequest request) {
            super(request);
        }

        public String getMethod() {
            return "GET";
        }
    }
}

With this filter, our previous example would work as follows:

The HEAD request will be seen as a GET by Spring MVC and therefore HEAD /editPet.do executes the setupForm method without the need for the extra annotation!

But what about the response?

The Spring MVC DispatcherServlet overrides the doService method from HttpServlet. This means that the nice NoBodyResponse logic is overridden, and thus never called.

There are once again two solutions to this problem:

Solution #1
Rely on your web container to suppress the response body for HEAD requests. Some containers, like Apache Tomcat, provide this functionality out of the box. Relying on container-specific behavior will increase your dependency on this particular server. This may or may not be a problem.

Solution #2
Integrate the NoBodyResponse wrapper with the HttpHeadFilter.

This is how the final solution looks like:

//Imports and documentation have been omitted...

public class HttpHeadFilter implements Filter {
    public void init(FilterConfig filterConfig) throws ServletException {
    }

    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;

        if (isHttpHead(httpServletRequest)) {
            HttpServletResponse httpServletResponse = (HttpServletResponse) response;
            NoBodyResponseWrapper noBodyResponseWrapper = new NoBodyResponseWrapper(httpServletResponse);

            chain.doFilter(new ForceGetRequestWrapper(httpServletRequest), noBodyResponseWrapper);
            noBodyResponseWrapper.setContentLength();
        } else {
            chain.doFilter(request, response);
        }
    }

    public void destroy() {
    }

    private boolean isHttpHead(HttpServletRequest request) {
        return "HEAD".equals(request.getMethod());
    }

    private class ForceGetRequestWrapper extends HttpServletRequestWrapper {
        public ForceGetRequestWrapper(HttpServletRequest request) {
            super(request);
        }

        public String getMethod() {
            return "GET";
        }
    }

    private class NoBodyResponseWrapper extends HttpServletResponseWrapper {
        private final NoBodyOutputStream noBodyOutputStream = new NoBodyOutputStream();
        private PrintWriter writer;

        public NoBodyResponseWrapper(HttpServletResponse response) {
            super(response);
        }

        public ServletOutputStream getOutputStream() throws IOException {
            return noBodyOutputStream;
        }

        public PrintWriter getWriter() throws UnsupportedEncodingException {
            if (writer == null) {
                writer = new PrintWriter(new OutputStreamWriter(noBodyOutputStream, getCharacterEncoding()));
            }

            return writer;
        }

        void setContentLength() {
            super.setContentLength(noBodyOutputStream.getContentLength());
        }
    }

    private class NoBodyOutputStream extends ServletOutputStream {
        private int contentLength = 0;

        int getContentLength() {
            return contentLength;
        }

        public void write(int b) {
            contentLength++;
        }

        public void write(byte buf[], int offset, int len) throws IOException {
            contentLength += len;
        }
    }
}

Conclusion

We now have a drop-in solution, compatible with any web framework and any container. It allows us to transparently support http HEAD requests in our applications and finally treat web crawlers and proxy servers as first class citizens.

The source code for HttpHeadFilter is available here. Feel free to use it as you wish.

Feedback is always welcome.