This release includes some important bugfixes and other improvements, including:

Removed feature from 2.1.0 that resolved expressions in version elements within the POM on installation and deployment.

This code was causing inconsistencies between the POM that landed in the repository and artifacts that are derived from that POM. The most prominent example deals with the GPG plugin: since the POM was modified on deployment, the GPG signature generated during the build was useless. This by itself effectively made Maven 2.1.0 unusable for releases.

Switched to a HttpClient-based wagon implementation for reaching HTTP repositories.

This is crucial for people sending long passwords, since the HttpUrlConnection-driven wagon did not-nice things with BASIC authentication headers when the password was very long…it line-wrapped the Base-64 header value, rendering the HTTP request invalid. In addition, HttpClient offers a wide range of options for configuration over HttpUrlConnection.

New default execution IDs for goals bound from the default lifecycle mapping and those invoked from the command line.

Previously, the only way to configure plugins that were used from the command line was to put the configuration options into the plugin-level configuration. This meant that it was impossible to separate CLI-oriented configuration settings from those used in goals that were bound to the lifecycle. Now, you can simply use an execution block with an id of default-cli to compartmentalize CLI-specific options.

Likewise, Maven 2.1.0 forces users to add configuration at the plugin level for any goals bound to the build via default lifecycle mapping. This could be particularly troublesome when you had multiple goals from a single plugin bound in via lifecycle mapping, and needed to respecify the same configuration option differently for the different goals. For instance, using different includes or excludes between the compiler:compile and compiler:testCompile goal executions was basically impossible. In Maven 2.2.0, you can use executions with the new per-goal default execution-Ids. For the compiler example, you can use default-compile and default-testCompile, respectively, to separate configurations for these two default goal executions.

Java 1.5+ is now a requirement.

Maven 2.2.0 upgrades the Java requirement to 1.5 or later. This allows us to finally start making the migration onto the Java5 generics syntax, and other nice little perks that come with a less archaic version of Java.

…and more…

For the full list of updates, see the Release Notes.

We will be using Emma to measure and report the coverage. EMMA is an open-source toolkit for measuring and reporting Java code coverage.

Unit test coverage

Let’s start by generating a coverage report for unit tests only. This will require 2 new plugins to be added to our pom. The first one:

37
38
39
40
41
42
43
44
45
46
47
48
49

<plugin>
  <groupId>org.sonatype.maven.plugin</groupId>
  <artifactId>emma-maven-plugin</artifactId>
  <version>1.1</version>
  <executions>
    <execution>
      <phase>process-classes</phase>
      <goals>
        <goal>instrument</goal>
      </goals>
    </execution>
  </executions>
</plugin>

This plugin will instrument classes at target/generated-class/emma/classes, this won’t change the original classes at target/classes. Then, we must tell surefire plugin to look for classes at new location:

51
52
53
54
55
56
57
58

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.4.3</version>
  <configuration>
    <classesDirectory>${project.build.directory}/generated-classes/emma/classes</classesDirectory>
  </configuration>
</plugin>

At this point, we do 50% of code coverage. Mainly because the psvm ( public static void main ) is never executed.

Here is the EMMA text report. Emma does produce xml and html reports in addition to this simple text report:

[EMMA v2.0.5312 report, generated Fri Jun 12 15:43:28 BRT 2009]
-------------------------------------------------------------------------------
OVERALL COVERAGE SUMMARY:
 
[class, %]	[method, %]	[block, %]	[line, %]	[name]
100% (1/1)	67%  (2/3)!	56%  (10/18)!	50%  (4/8)!	all classes

Note: at this point no report is generated, we will configure report generation after integration tests coverage reports are in place. Once you run Surefire, you may notice the coverage.ec in the project root directory. This file contains the coverage result in a binary format used by Emma.

Integration test coverage

So far, we have the unit tests results. As I demonstrated in the previous article in this series, the integration tests need the project artifact to run, so, we need to instrument this jar in order to get coverage result on ITs. Let’s see the pom:

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

<plugin>
  <groupId>org.sonatype.maven.plugin</groupId>
  <artifactId>emma4it-maven-plugin</artifactId>
  <version>1.3</version>
  <executions>
    <execution>
      <id>instrument</id>
      <goals>
        <goal>instrument-project-artifact</goal>
      </goals>
      <configuration>
        <appendEmma>true</appendEmma>
      </configuration>
    </execution>
  </executions>
</plugin>

Note that is using a different plugin: emma4it-maven-plugin. This plugin provides some additional goals for instrumenting integrations tests. For this project we use instrument-project-artifact goal. This goal will add Emma instrumentation into the project jar. This is specific to tests running on integration-test phase, because it does requires that the packaged artifact be instrumented.

There is also the configuration appendEmma. When true, this goal will shade the Emma jar into the instrumented jar. This is very useful to test jars that will be launched from command line, like this project.

[EMMA v2.0.5312 report, generated Fri Jun 12 16:05:45 BRT 2009]
-------------------------------------------------------------------------------
OVERALL COVERAGE SUMMARY:
 
[class, %]	[method, %]	[block, %]	[line, %]	[name]
100% (1/1)	100% (3/3)	89%  (16/18)	88%  (7/8)	all classes

At this point we have a very decent code coverage, almost 90%, 7 of 8 lines of code, and on this project it is not possible to get 8/8. And why is that? System.exit(). when this line is invoked nothing else is executed, that includes Emma instrumented code, so Emma won’t be able to know if System.exit() was executed, our integration tests catch that, because we check the exit code of the process, but Emma can’t do the same.

Test Coverage Report

At this point we have all coverage data stored at coverage.ec, in Emma binary format. Let’s now convert that into something that is more readable.

Add the following execution on emma4it-maven-plugin:

74
75
76
77
78
79
80
81
82
83
84
85
86
87

<execution>
  <id>report</id>
  <phase>post-integration-test</phase>
  <goals>
    <goal>report</goal>
  </goals>
  <configuration>
    <sourceSets>
      <sourceSet>
        <directory>${project.build.sourceDirectory}</directory>
      </sourceSet>
    </sourceSets>
  </configuration>
</execution>

Now the text, HTML, and XML reports are generated at target/site/emma.

Emma HTML coverage report

That is it for part 2. On the next part, I will show some ideas about testing more complex projects like flexmojos and nexus.

Repositories per Project/Team

The first and most obvious way to support multiple teams is to configure a pair of repositories per team (one release, one snapshot). The team is then given the appropriate C.R.U.D. permissions and they are able to use the system for their artifacts.

Our http://oss.sonatype.org instance is for the most part configured in this manner, where each project like Jetty has their own repositories separate from everyone else.

Partition Shared Repositories

Another option is to have a single (or a few) pair of Release/Snapshot repositories for your entire organization. In this case, the access is controlled by a mechanism we call “Repository Targets.”

Simply put, a Repository Target is a way to manage a set of artifacts based on their paths. A Repository Target is simply a list of regular expressions and a Name. For example, a Repo Target for Maven would be “./org/apache/maven/. and Nexus OSS would be “./org/sonatype/nexus/.“

Note: while it is most common to manage artifacts based on the path of their groupId, the Regular Expression is matched against the entire path, and so it is also possible, for example, to define “Sources” as “.*-sources.jar” … it’s also worth noting that Repository Targets are not mutually exclusive. It is perfectly valid for a given path to be contained by multiple targets.

In this model, you would create a Repo Target for each project in your system. You are then able to take the Repo Target and associate it with one or more Repositories or Groups in your system. When you do this, new, specific, C.R.U.D. privileges are created. For example, I could take the Maven Repo target, associate it with my Release and Snapshot repository, and now I get privileges I can assign to Create, Read, Update, Delete “Maven” (./org/apache/maven/.) artifacts in my Release and Snapshot repositories.

This method is used to manage the http://repository.apache.org instance, where we have just one Release and Snapshot repository and each project team gets permissions to their artifacts based on the path.

Which Method is right for me?

First of all, these choices aren’t mutually exclusive. In fact, the first option builds upon the default Repository Target of “.*” which simply gives you access to all artifacts regardless of the path. You still associate the default Repo Target with specific repositories to create the assignable privileges

In general, it’s my opinion that fewer repositories will scale better and are easier to manage. It’s also easier to start off with a single pair of repos, with the default “All M2″ (.*) target and simply refine the permissions as you scale. Most things that are configured per repository (Cache, Storage location, Snapshot purging, etc) will generally be applicable for all projects, so this mode avoids the duplication of these tasks. Since everything will be stored together in a single folder on disk, it makes backups easier as well.

The reasons why you would want multiple sets of repositories is essentially the opposite of above: If you need different expiration, Snapshot purging or storage folders, then a single shared repo won’t work. Replication and failover strategies may also make this method easier to support. If you absolutely must maintain total separation between Project teams, ie they can’t read each other’s artifacts, then this solution might be more applicable as well. (but is still possible with Repo Targets…just grant Read to only the appropriate targets)

In Summary, Nexus allows you to control the security of your artifacts based on the repository and/or the path of the artifact, meaning it is possible to slice and dice the system any way you see fit. My default position is to use a few Hosted Repositories as possible and control the permissions by the Repository Target.

It’s also worth mentioning that the Staging functionality in Pro also uses the Repo Targets to decide how given artifacts are staged, so the definitions are reusable across the system…and if you are using Staging you will define the

At Sonatype, we want to make synchronizing and publishing your artifacts to central easier both for you and for us, and we want to improve the quality of repository metadata for everyone at the same time. We have setup a dedicated instance of Nexus Professional at http://oss.sonatype.org specifically to host the artifacts of other Open Source projects. In this post, I talk about the process of creating a repository for your open source and publishing artifacts that can easily be integrated into the Maven Central repository once they are published to this freely available resource.

This instance has actually been running and in production for months by invitation only for projects such as Plexus, Jetty and Ehcache (Greg wroteabout his experience with migrating to oss.sonatype.org). We now have tooling in place to make it relatively easy for us to process a larger set of requests, so we’re just now starting to publicize the availability.

To get the process started, go here. We’ll setup a release and snapshot repository for your project, along with the appropriate configuration to allow you to use the Staging features for your releases. If you have an existing repository somewhere, we can migrate that for you too.

Since the Sonatype machines sit in the same rack as the Maven Central repository, we are able to synchronize the repositories every hour instead of twice a day.

In the coming weeks we will have a pre-release of Nexus 1.4 Pro running on this instance that allows customizable rules to be run during the staging process. This will allow the system to automatically check things like valid pgp signatures and correct POM parsing. This will ensure that your users have the best experience possible when using your artifacts, and relieve some of the manual validation on your side, a win for everyone.

On the technical details, this instance gets its network connection via Contegix’s high availability network, the same one running Maven Central (and Codehaus.org and Atlassian.com), and the data is backed up offsite daily.

1. From Nexus, click on the ‘Repositories’ link in left navigation.
2. Click on the repository you want to use, or create one (If you need help with that take a look at the Nexus book, here)
3. Click on the ‘Configuration’ tab.
4. Set the ‘Allow Deployment’ field to false.
5. Click the Save button.

That is it! Users will not be able to deploy or upload artifacts directly to the repository.  All artifacts must be staged and promoted to this repository.  For more information on staging and promoting take a look at this,  or with the maven plugin, here.

Create a Staging repository

To create a staging repository:

1. From Nexus, click on the ‘Staging’ link in left navigation.
2. Click the ‘Add’ button then the ‘Staging Profile’ item.
3. Enter the following information:
   • Profile Name: Staging Demo Profile
   • Profile Repository Target: All (Maven2)
   • Staging Repository ID Template: staging-demo
   • Staging Repository Name Template: Staging Demo
   • Staging Repository Template: Default Release Hosted Repository Template
   • Target Groups: Public Releases
4. Click the ‘Save’ button

Assign permissions to users

A new role is created for each staging profile that is created ( in this example the role is ‘Staging Deployer: (Staging Demo Profile)’).  Assign the new role to your users.  You can find more details on user management, here.

• Manifest: sonatype-images/nexus-os-1.3.4_hudson-20090608.manifest.xml
• AMI ID: ami-7314f21a
• Operating System: Fedora Core 8
• Platform: i386

Launch it. In minutes, you will have a capable build server. Read more to find out about some suggested next steps to secure your new build server.

What’s Installed? This instance is simply the latest installation of Hudson 1.309, Nexus 1.3.4 Open Source, OpenJDK 1.6.0, and Maven 2.1.0. Nexus is installed in /usr/local/nexus and Hudson is installed on Tomcat 6.0.20 in /usr/local/hudson. Both Nexus and Hudson are configured to startup in /etc/init.d/nexus and /etc/init.d/hudson.

Post Launch Instructions: Since Nexus and Hudson are both running with the default authentication configuration, it is imperative that you change the default administrative password of Nexus after installation (the default is “admin123″). To change the Nexus password:

1. Go to http://<host>/nexus
2. Log in as “admin” using the default password “admin123″
3. Click on “Change Password” in the Security Menu

Also, it is imperative that you turn on security for your Hudson instance. To do this:

1. Go to http://<host>/hudson
2. Click on “Manage Hudson”
3. Click on “Configure System”
4. Make sure that “Enable Security” is checked, and configure Hudson security. If you don’t have an LDAP server, just use Hudson’s on Security Database

While you are at it, you will need to configure Hudson’s JDK and Maven installation. To do this:

1. Go to http://<host>/hudson
2. Click on “Manage Hudson”
3. Click on “Configure System”
4. Click on ADD JDK. At this point, you can download a JDK directly from Sun, or you can point your Hudson instance at the OpenJDK 1.6.0 installed on this EC2 instance. The JAVA_HOME directory for OpenJDK on this box is: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/
5. Click on Add Maven to add a Maven installation. At this point, you can install Maven directly from Apache, or you can point Hudson at the Maven 2.1.0 instance installed on this machine. M2_HOME on this machine is /usr/local/maven.

Long-lived Build Machines on EC2 (Get an EBS Volume)

If you are planning on running this instance as a long-running build server, you will likely want to configure the sonatype-work directory to run on an EBS volume. If you do end up running Hudson and Nexus on this instance for a long time, and you are planning on depending on it: invest in an EBS volume and move /usr/local/nexus and /usr/local/hudson to run on that persistent volume. This way, you’ll have the peace of mind that your build and repository data will persistent in the event of unforeseen downtime.

Happy Nexusing

• Indexing (Part 1)
• Searching
• Packing
• Updating

This post will focus on Searching and provide you with some concrete code examples that show you how to use the indexer search capabilities in your application.

Searching

The Nexus Indexer uses Apache Lucene to handle storage and searching of the index data, and provides a very simple method to search against fields in the index.  Below I have extended my plexus component from the Part 1 of this blog series, to include search functionality.

package org.damian;
...
/**
 * Sample app to show how to integrate with the nexus indexer.  Note that this is a simple plexus
 * component extending the SampleApp interface
 * 
 * public interface SampleApp
 * {
 *    void index() 
 *        throws IOException;
 *    
 *    Set<ArtifactInfo> searchIndexFlat( String field, String value ) 
 *        throws IOException;
 *    
 *    Set<ArtifactInfo> searchIndexFlat( Query query )
 *        throws IOException;
 *    
 *    Map<String, ArtifactInfoGroup> searchIndexGrouped( String field, String value )
 *        throws IOException;
 *    
 *    Map<String, ArtifactInfoGroup> searchIndexGrouped( String field, String value, Grouping grouping )
 *        throws IOException;
 *    
 *    Map<String, ArtifactInfoGroup> searchIndexGrouped( Query q, Grouping grouping )
 *        throws IOException;
 * }
 * 
 * @author Damian
 *
 */
@Component( role = SampleApp.class )
public class DefaultSampleApp
    implements SampleApp,
        Initializable,
        Disposable
{
    // The nexus indexer
    @Requirement
    private NexusIndexer indexer;
 
    ...
 
    // search for artifacts
    public Set<ArtifactInfo> searchIndexFlat( String field, String value ) 
        throws IOException
    {
        // Build a query that will search the documents for the field set to the supplied value
        // This uses predefined logic to define the query
        // See http://svn.sonatype.org/nexus/trunk/nexus-indexer/src/main/java/org/sonatype/nexus/index/DefaultQueryCreator.java
        // for details
        Query query = indexer.constructQuery( field, value );
 
        return searchIndexFlat( query );
    }
 
    // search for artifacts using pre-built query
    public Set<ArtifactInfo> searchIndexFlat( Query query )
        throws IOException
    {
        // Build the request
        FlatSearchRequest request = new FlatSearchRequest( query );
 
        // Perform the search
        FlatSearchResponse response = indexer.searchFlat( request );
 
        // Return the artifact info objects
        return response.getResults();
    }
 
    public Map<String, ArtifactInfoGroup> searchIndexGrouped( String field, String value )
        throws IOException
    {
        // We will simply use the GAV grouping, meaning that each groupId/artifactId/version/classifier
        // will have its own entry in the returned map        
        return searchIndexGrouped( field, value, new GAVGrouping() );
    }
 
    public Map<String, ArtifactInfoGroup> searchIndexGrouped( String field, String value, Grouping grouping )
        throws IOException
    {
        // Build a query that will search the documents for the field set to the supplied value
        // This uses predefined logic to define the query
        // See http://svn.sonatype.org/nexus/trunk/nexus-indexer/src/main/java/org/sonatype/nexus/index/DefaultQueryCreator.java
        // for details
        Query query = indexer.constructQuery( field, value );
 
        return searchIndexGrouped( query, grouping );
    }
 
    public Map<String, ArtifactInfoGroup> searchIndexGrouped( Query q, Grouping grouping )
        throws IOException
    {
        GroupedSearchRequest request = new GroupedSearchRequest( q, grouping );
 
        GroupedSearchResponse response = indexer.searchGrouped( request );
 
        return response.getResults();
    }
}

As you can see, there are 2 ways to handle how the search is performed. The first is to use the nexus indexer to decide what type of lucene Query to create, based upon the input string. The second is to pass in your own lucene Query object, you can use any object that extends the org.apache.lucene.search.Query abstract class. If you choose to let the nexus indexer build your query, the following rules will apply to the search term:

• string converted to lowercase
• if first character is ‘^’ drop it
• if first character anything other than ‘^’ prepend string with ‘*’
• if last character is ‘ ‘, ‘<' or '$' drop it
• if last character anything other than ‘ ‘, ‘<' or '$' append '*' to end of string
• if modified string does not contain ‘*’ then a TermQuery is used (exact match)
• if modified string contains ‘*’ in last position, then a PrefixQuery is used (and ‘*’ is dropped from string)
• if modified string contains ‘*’ in any other position, then a WildcardQuery is used

There will probably be occassions where this list of rules doesn’t apply to your use case, so you instead will want to give the Indexer your own Query object, that leaves you to choose from the many Query objects that lucene provides, or even to make your own if necessary.

You will also notice that there are 2 different types of result sets you can get back, flat or grouped. Flat results are simply a Set of ArtifactInfo objects for each match in the index. Grouped results will combine results based upon the Grouping you request (in my sample, the default is GAVGrouping which will group items together that share the same groupId/artifactId/version/classifier).

Here are some samples from a unit test in the project source code, that do the different kinds of searching.

    public void testSampleSearch()
        throws Exception
    {
        app.index();
 
        Set<ArtifactInfo> artifacts = app.searchIndexFlat( SampleIndexCreator.MY_FIELD, "value" );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
    }
 
    public void testSampleSearchWithTermQuery()
        throws Exception
    {
        app.index();
 
        // This type of query will be totally built outside of nexus indexer, and will not
        // be tied to constraints defined in 
        // http://svn.sonatype.org/nexus/trunk/nexus-indexer/src/main/java/org/sonatype/nexus/index/DefaultQueryCreator.java
 
        // A TermQuery matches equal strings
        Query q = new TermQuery( new Term( SampleIndexCreator.MY_FIELD, "value" ) );
 
        Set<ArtifactInfo> artifacts = app.searchIndexFlat( q );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
    }
 
    public void testSampleSearchWithPrefixQuery()
        throws Exception
    {
        app.index();
 
        // This type of query will be totally built outside of nexus indexer, and will not
        // be tied to constraints defined in 
        // http://svn.sonatype.org/nexus/trunk/nexus-indexer/src/main/java/org/sonatype/nexus/index/DefaultQueryCreator.java
 
        // A PrefixQuery will look for any documents containing the MY_FIELD term that starts with val
        Query q = new PrefixQuery( new Term( SampleIndexCreator.MY_FIELD, "val" ) );
 
        Set<ArtifactInfo> artifacts = app.searchIndexFlat( q );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
    }
 
    public void testSampleSearchWithWildcardQuery()
        throws Exception
    {
        app.index();
 
        // This type of query will be totally built outside of nexus indexer, and will not
        // be tied to constraints defined in 
        // http://svn.sonatype.org/nexus/trunk/nexus-indexer/src/main/java/org/sonatype/nexus/index/DefaultQueryCreator.java
 
        // A WildcardQuery supports the * and ? wildcard characters
        Query q = new WildcardQuery( new Term( SampleIndexCreator.MY_FIELD, "*alue" ) );
 
        Set<ArtifactInfo> artifacts = app.searchIndexFlat( q );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
 
        // A WildcardQuery supports the * and ? wildcard characters
        q = new WildcardQuery( new Term( SampleIndexCreator.MY_FIELD, "v?lue" ) );
 
        artifacts = app.searchIndexFlat( q );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
 
        // A WildcardQuery supports the * and ? wildcard characters
        q = new WildcardQuery( new Term( SampleIndexCreator.MY_FIELD, "val*" ) );
 
        artifacts = app.searchIndexFlat( q );
 
        assertNotNull( "returned artifacts is null", artifacts );
        assertFalse( "returned artifacts is empty", artifacts.isEmpty() );
 
        for ( ArtifactInfo ai : artifacts )
        {
            assertEquals( "returned artifact has invalid data", "value", ai.getAttributes().get( SampleIndexCreator.MY_FIELD ) );    
        }
    }
 
    public void testSampleSearchGroup()
        throws Exception
    {
        app.index();
 
        Map<String,ArtifactInfoGroup> groupedArtifacts = app.searchIndexGrouped( SampleIndexCreator.MY_FIELD, "value" );
 
        assertNotNull( "returned groupedArtifacts is null", groupedArtifacts );
        assertFalse( "returned groupedArtifacts should not be empty", groupedArtifacts.isEmpty() );
 
        for ( ArtifactInfoGroup artifactGroup : groupedArtifacts.values() )
        {
            String[] parts = artifactGroup.getGroupKey().split( ":" );
            //1st part groupId
            //2nd part artifactId
            //3rd part version
            //4th part classifier
            assertEquals( "should be 4 parts to the group key", 4, parts.length );
            assertFalse( "each group should contain at least 1 artifact", artifactGroup.getArtifactInfos().isEmpty() );
        }
    }
 
    public void testSampleSearchGroupNewGrouping()
        throws Exception
    {
        app.index();
 
        // Search using my own grouping, which will group based upon the MY_FIELD parameter
        Map<String, ArtifactInfoGroup> groupedArtifacts = app.searchIndexGrouped(
            SampleIndexCreator.MY_FIELD,
            "value",
            new AbstractGrouping()
            {
                @Override
                protected String getGroupKey( ArtifactInfo artifactInfo )
                {
                    return artifactInfo.getAttributes().get( SampleIndexCreator.MY_FIELD );
                }
            } );
 
        assertNotNull( "returned groupedArtifacts is null", groupedArtifacts );
        assertEquals( "returned groupedArtifacts should have 1 entry", 1, groupedArtifacts.size() );
        assertEquals( "group key should be value", "value", groupedArtifacts.values().iterator().next().getGroupKey() );
    }
}

As you can see from the last test method, you can create your own Grouping object, to group things anyway you see fit, in this case, the results will be grouped using the MY_FIELD parameter.

That’s all for now, next time we will get into packing up indexes for publishing.

Sample Maven project can be found here http://svn.sonatype.org/nexus/trunk/sandbox/nexus-indexer-sample and will be updated periodically as I put together more details for the blog posts.

The application

Let’s define a very simple application to be tested. The application will return an exit code equal to the number of command-line parameters. Running the program with no parameters will produce an exit code equal to zero, two parameters exit code two, and so on.

The application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

package org.sonatype.simpleclientapp;
 
public class Main
{
    public static void main( String[] args )
    {
        System.exit( execute( args ) );
    }
 
    public static int execute( String[] args )
    {
        if ( args == null )
        {
            return 0;
        }
 
        return args.length;
    }
}

This class is very simple to test, in the next section you will see how you can write a simple integration test using JUnit.

Creating and running integration tests

Let’s create a simple integration test using JUnit:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

package org.sonatype.simpleclientapp;
 
import java.io.File;
 
import junit.framework.TestCase;
 
public class MainIT
    extends TestCase
{
 
    public void testExecute()
        throws Exception
    {
        assertEquals( 0, execute( new String[] {} ) );
        assertEquals( 1, execute( new String[] { "one" } ) );
        assertEquals( 6, execute( new String[] { "one", "two", "three", "four", "five", "six" } ) );
    }
 
    private int execute( String[] args )
        throws Exception
    {
        File jar = new File( "target/simple-client-app-1.0-SNAPSHOT.jar" );
 
        String[] execArgs = new String[args.length + 3];
        System.arraycopy( args, 0, execArgs, 3, args.length );
        execArgs[0] = "java";
        execArgs[1] = "-jar";
        execArgs[2] = jar.getCanonicalPath();
        Process p = Runtime.getRuntime().exec( execArgs );
        p.waitFor();
        return p.exitValue();
    }
 
}

Note that this test needs the packaged jar (simple-client-app-1.0-SNAPSHOT.jar), which isn’t created when surefire runs the unit tests.

To run this we need a different approach. By different approach I mean a different plugin: the Failsafe Maven Plugin. The Failsafe Plugin is a fork of the Surefire plugin designed to run integration tests.

The Failsafe plugin goals are designed to run after the package phase, on the integration-test phase.

The Maven lifecycle has four phases for running integration tests:

• pre-integration-test: on this phase we can start any required service or do any action, like starting a database, or starting a webserver, anything… you can think on this as Junit.setUp()
• integration-test: failsafe will run the test on this phase, so after all required services are started.
• post-integration-test: time to shutdown all services… you can think of this as Junit.tearDown()
• verify: failsafe runs another goal that interprets the results of tests here, if any tests didn’t pass failsafe will display the results and exit the build.

Configuring Failsafe in the POM:

23
24
25
26
27
28
29
30
31
32
33
34
35

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>failsafe-maven-plugin</artifactId>
        <version>2.4.3-alpha-1</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

By default, the Surefire plugin executes **/Test*.java, **/*Test.java, and **/*TestCase.java test classes. The Failsafe plugin will look for **/IT*.java, **/*IT.java, and **/*ITCase.java. If you are using both the Surefire and Failsafe plugins, make sure that you use this naming convention to make it easier to identify which tests are being executed by which plugin.

In the next part of this series, I will talk about test coverage and demonstrate a method of including coverage of both Unit Tests and Integration Tests in a report that can be generated for a Maven project.

This post covers some of the initial steps, that are required to write a plugin for Nexus. Although Nexus is a Plexus application, we want to give 3rd party developers ability to extend Nexus, without forcing them to know Plexus. We also want to give 3rd party developers the ability to extend Nexus, without burying themselves into Nexus internals. Clearly if someone wants to add complex, highly custom behavior to Nexus they will need to dive into the internals, but it should be easy to add a simple extension to Nexus without having a PhD in Nexus Internals. We are also committed to adopting 3rd party “specs/suggestions/APIs” that look promising, even if it comes from another IoC provider.

Now, let’s replace the “Nexus” in above sentences with “Sonatype Application” in the sentences above. It’s not so different, right? At Sonatype, we’re convinced that providing an intuitive plugin and extension mechanism is critical for adoption and we want to make it as straightforward as possible. To start this discussion, we need first a look at how Plexus works.

Plexus in 3 minutes

Plexus “managed beans” are called “components”. In contrast to the way most people use Spring, where each bean has a unique name or “ID”, Plexus components are addressed with two coordinates: role and hint.

Plexus encourages the use of interfaces when defining components. Generally spoken, when implementing one component in Plexus, you will almost always end up with two compilation units: a Java interface, that sets the contract that your component fulfills, and an implementation (or multiple implementations) that implements the contract interface. Again, this is not enforced at all, but is considered as “best practice”.

While these two coordinates are internally represented as plain strings, the role is usually a FQN of the interface that component “provides” (or implements). For example, the role might be something like “com.sonatype.component.TestComponent” which is the FQN of an interface used to define the contract of the component. This is not enforced at all in Plexus and it’s tooling (for example, the plexus-component-metadata Maven plugin), but again, is considered a “best practice”. The “hint” is actually a free string qualifier, to be able to differentiate amongst multiple implementors (components) with same role, if needed. There is one special hint used internally by Plexus: “default”, which is used when no hint is supplied.

Why is this important? Well, usually you have two major situations with your components (managed beans). For one component contract interface you either have:

• one “default” implementation across your application (singular case),
• or you may have multiple different implementations of same component contract (plural case)

Do not confuse the singular case with “singleton” pattern: in both cases the components (managed beans) are singletons. We are just providing one or multiple implementations for same interface. But in both cases, the actual classes being created by container are singletons (unless you say different in your Plexus annotations, but that’s another story, and I don’t want to complicate existing examples).

Another case is how the “consumer” of that component — which is probably some other component in your application — decides which implementation it wants to use. Or maybe it doesn’t care at all?

The simplest example of “singular” case, the component contract interface and it’s implementation:

public interface MyComponent
{
  String getHello();
}
 
@Component( role=MyComponent.class )
public class DefaultMyComponent
  implements MyComponent
{
  public String getHello() {
    return "hello";
  }
}

That’s all you need. We just created a MyComponent component contract (interface), and provided one implementation. The implementation is even annotated using Plexus annotation, that states “this class is a component with role MyComponent”. Since we say nothing about the hint of component, Plexus implicitly manages it as “default” implementation (one-and-only).

The consumer of this component does something like this:

@Component( role=SomeOtherComponent.class )
public class AComponentConsumer
  implements SomeOtherComponent
{
  @Requirement
  private MyComponent myComponent;
 
  ...
}

Here, with @Requirement annotation we declared a “requirement” (we need it injected) to the MyComponent interface. The Plexus will try to fulfill this requirement by looking up a component with role MyComponent, and hint “default” (implicit hint), since we did not say anything about needed hint.

For the second case (one contract, multiple implementations), the most simpler example is this:

public interface Archiver
{
    void archive( File source, File archiveFile ) 
      throws IOException;
 
    void unarchive( File archiveFile, File destination ) 
      throws IOException;
}

and having component implementations like these:

@Component( role=Archiver.class, hint-"zip" )
public class ZipArchiver
  implements Archiver
{
  ...
}
 
@Component( role=Archiver.class, hint-"tgz" )
public class TgzArchiver
  implements Archiver
{
  ...
}
 
@Component( role=Archiver.class, hint-"7z" )
public class SevenZArchiver
  implements Archiver
{
  ...
}

The consumer of Archiver does something like this:

@Component( role=SomeOtherComponent.class )
public class AComponentConsumer
  implements SomeOtherComponent
{
  @Requirement( hint="zip" )
  private Archiver zipArchiver;
 
  ...
}

In case above we “wired” the AComponentConsumer to support “zip” archives only. But we may do something like this:

@Component( role=SomeOtherComponent.class )
public class AComponentConsumer
  implements SomeOtherComponent
{
  @Requirement( role=Archiver.class )
  private List<Archiver> archivers;
 
  ...
}

or

@Component( role=SomeOtherComponent.class )
public class AComponentConsumer
  implements SomeOtherComponent
{
  @Requirement( role=Archiver.class )
  private Map<String, Archiver> archivers;
 
  public void archive( File src, File dest ) 
    throws IOException
  {
    String destExt = getFileExtension( dest );
 
    if ( archivers.containsKey( destExt ) )
    {
      Archiver archiver = archivers.get( destExt );
 
      archiver.archive( src, dest );
      ...
    } 
    else
    {
      throw new IOException( "Archive format unsupported!" );
    }
  }
}

It is obvious, that in this example, Plexus will inject a Map<String, Archiver> into AComponentConsumer, where the String keys will be hints of components in the Map, and values will be Archiver instances with given key.

So?

So, why is this important? — you ask. Well, in Nexus, the “extension points” are simply taken, “marked” interfaces (roles), that will be automatically pulled by Plexus collection lookups similar to this last example. Marked by Nexus developers, and pointed out as “extension point” to the 3rd party plugin developers.

Actually, up to now, this was how Nexus supported “plugins”. “Plugins” and not plugins, up to now, the job of the developer was to code Plexus components, build them, provide Plexus descriptors for the plugin (usually by annotating the classes as above and using the plexus-component-metadata plugin, which embeds the descriptor into resulting JAR), and “drop it” into Nexus lib.

Since the developers implemented and marked their classes as @Component with roles that were looked up, after booting up Nexus with new JAR in it’s lib folder, Nexus was aware of these and would start using them as if they were a part of the “core”. Simple as that.

The downside of this approach is twofold:

• 3rd party developers are forced to use Plexus, plexus tooling, etc. Also, they must be aware of Nexus internals, for example that Repository components should be **prototypes**, and not **singletons**.
• the plugins are more like “extensions”, since they total and uncontrolled access to all of the Nexus internals, not just extension points.

The Idea

In short:

• let the “host application developers” mark and parametrize the extension points (@ExtensionPoint)
• let the “3rd party plugin developers” create extensions by implementing interfaces marked with @ExtensionPoint.
• provide IoC abilities for “3rd party pluging developers”
• let the “3rd party plugin developers” create their own managed beans, components with all the IoC benefits
• load the plugin in separate classloader

Okay, let’s draft the idea: having “extension points”, which are all simple Java interfaces (component contracts) and are provided by “host application” developers (in this case Nexus developers). This is simple to understand: the host application developers are the one who know the what, where, and how about what is extensible within the application. Thus, we need to provide some means for developers, to mark their interfaces as @ExtensionPoint. Almost always, the case with components here is the “plural” case (one component interface, with multiple implementations, 3rd party developers just adding new ones to the pool of existing). But again, usually the host application developers are the ones who knos, and should provide information on extension points about instantiation details of the component implementations (singletons versus prototypes for example).

When the host application API has decorated interfaces (API), the **3rd party plugin developers** may kick in, by implementing @ExtensionPoints. All they have to do is pick a component interface marked as @ExtensionPoint, and implement it. That’s all. And what about IoC?

We decided to accept the @Inject project proposal at code.google.com, and use “meta-metadata” to decorate the components. Since @Inject already proposes a meta-metadata to make possible to annotate classes and use them across Guice and Spring, we thought making it work with Plexus would be nice too.

Thus, in short, 3rd party Nexus plugin developers should use @Inject annotations only. Naturally, when deployed to Nexus, their components will be “powered by Plexus” under the hud, but it’s not their problem and they should not care about it.

The Nexus Plugin Manager will have another great feature: **class loader separation**. Right now, since as I said, the plugin JAR was just dropped into Nexus lib folder, that implicitly means that plugins were doomed to use the same versions of dependencies that were already present in Nexus (newly introduced dependencies are fine, though). This led to the proliferation and over user of the Maven Shade plugin. Don’t get me wrong, I think this plugin is very-very useful, but it must be used with care (not to mention it’s flaw, that it “shades” stuff always under hidden package, instead to somewhere module specific place. Two jars with same shaded dependency may again clash).

One more thing: the @Inject proposal does not mention how to create new managed beans, they have no means to annotate a new component (well, in case of Guice it happens implicitly with @Inject, making the interface needed in binding). This is where another new annotation comes in: @Managed. The @Managed annotation will be used on Java **interfaces only**, to mark them as component contracts.

So, in short:

• Host application developers should point out the “extension point” interfaces in the apps. Also, they should provide as much “metadata” as possible about the instances of those components.
• plugin developers should only care about implementing those marked interfaces, and not fiddle with annotating them with some container-specific stuff
• use @Inject project annotations
• add ability to plugin developers to enjoy the benefits of IoC containers, and let them defined their own components (again, no Plexus needed).

Teaser

Here is an example of a theoretical implementation of artifact virus scanner for Nexus. In short: we want to make sure no virus infected artifact will come in our Nexus instance. Internal deployments are considered “safe”, thus proxy requests are checked only.

These is what we need to implement this plugin:

• A Virus scanner implementation (not shown here for example’s brevity’s sake)
• A Nexus RequestProcessor (extension point) implementation. RequstProcessor components are able to interact with every Nexus request, and stop their execution at different stages
• A Nexus RepositoryCustomizer (extension point) implementation, to inject the RequestProcessor from above to repositories we want

We start with creating a managed component contract for Virus Scanner:

package org.sample.plugin;
 
import java.io.InputStream;
 
import org.sonatype.nexus.plugins.Managed;
 
@Managed
public interface VirusScanner
{
    boolean hasVirus( InputStream is );
}

Then we implement it. This is a “singular” case, the newly introduced component contract interface has only one default implementation:

package org.sample.plugin;
 
import java.io.InputStream;
 
import javax.inject.Named;
 
public class XYVirusScanner
    implements VirusScanner
{
 
    public boolean hasVirus( InputStream is )
    {
        // DO THE JOB HERE
        return resultOfTheScanning;
    }
 
}

Next, a RequestProcessor. It requires VirusScanner to do the work. We are not making a decision until we get to the content in shouldCache(). We are using the @Inject annotations to inject our newly created component. Since VirusScannerRequestProcessor implements a Nexus extension point (they are “plural” components), we need to reference it later (see below in VirusScannerRepositoryCustomizer), we are “naming it” with the @Named annotation.

package org.sample.plugin;
 
import java.io.IOException;
 
import javax.inject.Inject;
import javax.inject.Named;
 
import org.sonatype.nexus.proxy.ResourceStoreRequest;
import org.sonatype.nexus.proxy.access.Action;
import org.sonatype.nexus.proxy.item.AbstractStorageItem;
import org.sonatype.nexus.proxy.item.StorageFileItem;
import org.sonatype.nexus.proxy.repository.ProxyRepository;
import org.sonatype.nexus.proxy.repository.Repository;
import org.sonatype.nexus.proxy.repository.RequestProcessor;
 
@Named( "virusScanner" )
public class VirusScannerRequestProcessor
    implements RequestProcessor
{
    @Inject
    private VirusScanner virusScanner;
 
    public boolean process( Repository repository, ResourceStoreRequest request, Action action )
    {
        // don't decide until have content
        return true;
    }
 
    public boolean shouldProxy( ProxyRepository repository, ResourceStoreRequest request )
    {
        // don't decide until have content
        return true;
    }
 
    public boolean shouldCache( ProxyRepository repository, AbstractStorageItem item )
    {
        if ( item instanceof StorageFileItem )
        {
            StorageFileItem file = (StorageFileItem) item;
 
            // do a virus scan
            try
            {
                return virusScanner.hasVirus( file.getInputStream() );
            }
            catch ( IOException e )
            {
                // handle it
                return false;
            }
        }
        else
        {
            return true;
        }
    }
 
}

Finally, we are creating a RepositoryCustomizer (also a “plural” component, but we don’t care about naming it), that will inject our RequestProcessor into repository instances we need. In this case, those are proxy repositories (the only ones able to fetch artifacts from remote repositories).

package org.sample.plugin;
 
import javax.inject.Inject;
import javax.inject.Named;
 
import org.sonatype.nexus.configuration.ConfigurationException;
import org.sonatype.nexus.plugins.RepositoryCustomizer;
import org.sonatype.nexus.proxy.repository.ProxyRepository;
import org.sonatype.nexus.proxy.repository.Repository;
import org.sonatype.nexus.proxy.repository.RequestProcessor;
 
public class VirusScannerRepositoryCustomizer
    implements RepositoryCustomizer
{
    @Inject
    private @Named( "virusScanner" )
    RequestProcessor virusScannerRequestProcessor;
 
    public boolean isHandledRepository( Repository repository )
    {
        // handle proxy reposes only
        return repository.getRepositoryKind().isFacetAvailable( ProxyRepository.class );
    }
 
    public void configureRepository( Repository repository )
        throws ConfigurationException
    {
        repository.getRequestProcessors().put( "virusScanner", virusScannerRequestProcessor );
    }
 
}

That’s all for now. In the next part, we will make more in-depth explanations of the plugin API, and explain what is happening “behind the curtains”. Have fun!