Java Evangelist John Yeary's Blog

NetBeans 7.1 IDE: Remote Database Connections

noreply@blogger.com (John Yeary) — Sat, 21 Jan 2012 23:43:00 +0000

The NetBeans 7.1 IDE has a feature that has been around for a while, but does not get as much attention as it should. NetBeans allows you to take advantage of using remote databases for doing Java development. There are a number of wizards which can take advantage of connections created in the Services → Database tab.

In the demonstration video, I connect to a remote Apache Derby Database, but the same principals apply to any database as long as you have JDBC drivers available.

Tips & Tricks

To verify that a remote database is Apache Derby (Java DB) which is listening on port 1527 (default Derby port). I connect to the remote application and append XXX;create=true; to the end of the connection string. When I test the database connection, if it is Apache Derby, it will create the new database which confirms our suspicion.

This 3 minute video demonstrates this valuable feature which developers should use for all their database needs.

NetBeans 7.1 IDE: Shelve and Un-Shelve Changes

noreply@blogger.com (John Yeary) — Sat, 21 Jan 2012 22:54:00 +0000

The NetBeans 7.1 IDE introduces a really cool new feature called shelving. This allows a developer to make changes to a project without committing them to a source control system.

How often have you wanted to try out some new idea on your code without having to check it into source control? This allows you to do just that.

What is shelving?

Shelving creates a patch based on whether an individual file, files, or project is selected. This is a standard patch file and can be applied from the command line, or sent via email to others.

A nice feature is that you can provide meaningful names to the shelved changes. This makes it easier to go back later and apply them to your project.

Note: This functionality is only available for Subversion and Mercurial based projects.

I created a five minute video to demonstrate this really cool new feature.

NetBeans 7.1 IDE: Inspect and Transform to JDK 7

noreply@blogger.com (John Yeary) — Sat, 21 Jan 2012 22:36:00 +0000

I gave a talk this month at the Greenville Java Users Group (GreenJUG) on the new features, and tips & tricks in the new NetBeans 7.1 IDE.

One of the most popular demonstrations was using NetBeans to download Apache Commons IO from the Apache Subversion repository, open the Apache Maven project natively, and upgrade it from JDK 5 to JDK 7.

This is a real world demonstration of the incredible and powerful new capabilities in the IDE. What is particularly interesting is that the Apache Commons IO project does a great job of providing unit tests to validate our changes.

I created a 20 minute video which demonstrates this really vital new functionality to help you migrate your projects to JDK 7.

Jersey Tip of the Day: Use GZIP compression

noreply@blogger.com (John Yeary) — Fri, 20 Jan 2012 21:02:00 +0000

JAX-RS (Jersey) offers a GZIP filter to compress data for responses, and to handle GZIP compressed requests. This functionality is very easy to enable, and is configurable for both requests, and responses. That does not get much easier.

You can prove that this works by querying your resource with Firebug, or Developer Tools (depending on browser). You can also confirm that it is working by performing a query like:

curl -HAccept-Encoding:gzip -HAccept:application/json http://localhost:8080/content-coding-gzip/webresources/widget > json.gz
gzip -v -l json.gz

method  crc     date  time           compressed        uncompressed  ratio uncompressed_name
defla 3a79ae18 Jan 20 15:25                1370                3389  60.3% json

The only issue that I have with the currently implemented version is that it does not use configurable compression level.

JAX-RS Tip of the Day: Use OPTIONS Method to Determine Resource Capabilities

noreply@blogger.com (John Yeary) — Tue, 17 Jan 2012 16:52:00 +0000

Did you know that you can make an OPTIONS method call to a JAX-RS resource to determine its capabilites? JAX-RS supports a complete set of HTTP methods including OPTIONS.

If you make a specific URI request to a resource, it will return the supported methods like GET, PUT, HEAD, and OPTIONS. In addition, Jersey supports returning the WADL file by default. This allows tools like NetBeans to take advantage of the resource.

So I guess some examples are in order. The first example is a request to the server URI which in this case is GlassFish 3.1.1. This is followed by a request to a specific resource on the server instance.

curl -X OPTIONS -v http://localhost:8080

* About to connect() to localhost port 8080 (#0)
*   Trying ::1... Operation not permitted
*   Trying 127.0.0.1... connected
> OPTIONS / HTTP/1.1
> User-Agent: curl/7.22.0 (i686-pc-cygwin) libcurl/7.22.0 OpenSSL/0.9.8r zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.1 Java/Sun Microsystems Inc./1.6)
< Server: GlassFish Server Open Source Edition 3.1.1
< Allow: GET, HEAD, POST, PUT, DELETE, TRACE, OPTIONS
< Content-Length: 0
< Date: Tue, 17 Jan 2012 14:31:17 GMT
<
* Connection #0 to host localhost left intact
* Closing connection #0

As you can see the command reports that GlassFish supports all standard HTTP 1.1 methods as denoted by the Allow: header. This example calls a specific resource on the server which only supports a subset of the HTTP methods based on the annotations in the resource. The methods always supported by a resource are HEAD and OPTIONS, but this has two methods annotated as @GET and @POST. Additionally, not that the Content-Type: application/vnd.sun.wadl+xml is returned which includes the WADL for this particular resource.

curl -X OPTIONS -v http://localhost:8080/RESTApproachFormParameter/resources/example

* About to connect() to localhost port 8080 (#0)
*   Trying ::1... Operation not permitted
*   Trying 127.0.0.1... connected
> OPTIONS /RESTApproachFormParameter/resources/example HTTP/1.1
> User-Agent: curl/7.22.0 (i686-pc-cygwin) libcurl/7.22.0 OpenSSL/0.9.8r zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.1 Java/Sun Microsystems Inc./1.6)
< Server: GlassFish Server Open Source Edition 3.1.1
< Allow: OPTIONS,POST,GET,HEAD
< Content-Type: application/vnd.sun.wadl+xml
< Content-Length: 1085
< Date: Tue, 17 Jan 2012 15:01:49 GMT
<
* Connection #0 to host localhost left intact
* Closing connection #0

In summary, you can take advantage of the OPTIONS method to determine what a particular resource will provide for you without any knowledge of the subject domain.

GlassFish Default Encoding ISO-8859-1

noreply@blogger.com (John Yeary) — Fri, 13 Jan 2012 16:12:00 +0000

GlassFish default encoding returns text encoded as ISO-8859-1. This is based on the Hypertext Transfer Protocol -- HTTP/1.1 RFC 3.7.1 Canonicalization and Text Defaults which requires the default encoding for text to be returned in this manner.

Web application developers often use UTF-8 as the encoding in their applications, but fail to change the server defaults. This applies to anything which uses Servlets (including JSP, and JSF). You can change the defaults in the sun-web.xml, or glassfish-web.xml as shown below by setting the parameter-encoding element. This remove a number of default encoding warnings like this:

WARNING: WEB0500: default-locale attribute of locale-charset-info element is being ignored

WARNING: PWC4011: Unable to set request character encoding to UTF-8 from context /, because request parameters have already been read, or ServletRequest.getReader() has already been called.

JAX-RS Tip of the Day: Client Content Negotiation

noreply@blogger.com (John Yeary) — Thu, 12 Jan 2012 19:33:00 +0000

One of the less discussed aspects of JAX-RS based on the HTTP 1.1 specification is client negotiation. We often write our applications from the server side and expect the client to send us an Accept: */* header which details which media types the client can accept. The server simply returns whatever we have coded for example @Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML}). However I bet a number of you have seen something like this before:

http://www.example.com/documents/mydoc?format=doc
http://www.example.com/documents/mydoc.xml

The first example is not very subtle. It tells you the client wants a mydoc.doc by requesting a specific format. The second example is more subtle, and the end user (client) may not notice. I made it a "little" more obvious by changing it to .xml. However, most folks would have missed it if I used mydoc.doc. You would have assumed it was a MS Word document to start with. In actuality, the media type returned here is controlled by the file extension.

Note: Updated the source code to include header based negotiation, but this is just mimicking server content negotiation.

In the attached Apache Maven project developed on NetBeans 7.1: jersey-content-negotiation.zip. I demonstrate how to do it.

The first example returns JSON by default, and other media formats as requested. If it does not understand the media type requested, it simply returns plain text.

On the second example which uses the media type extension, I return a 400 - Bad Request instead of plain text like the first example. This subtlety is based on the fact that the client is requesting a specific type based on extension. If you ask for .pdf and get .txt, the result may cause an unanticipated consequence so it is better to throw an exception.

The mixed media type returns are possible because of the Response being wrapped as demonstrated in a previous post: JAX-RS Tip of the Day: Use Response to wrap... Responses.

ContentNegotiationResource.java

Client Negotiation Examples

JAX-RS Tip of the Day: Use Response to wrap... Responses

noreply@blogger.com (John Yeary) — Wed, 11 Jan 2012 21:15:00 +0000

One of the really nice features of JAX-RS is the ability to implement fine grained control over the response returned from the server based on the client request. The Response.ResponseBuilder gives us incredible flexibility to generate the response. Since it uses the builder pattern, it is easy to daisy-chain the information we want into a custom response.

I would highly recommend that you consider returning a custom response for all requests. I would like to explain with an example. I would like to perform a @POST request where I am creating a new object on the server. Since it is a HTTP 1.1 request, I look at Hypertext Transfer Protocol -- HTTP/1.1 (RFC-2616) to see what I must, should, and may return. The difference in the word choices has specific meaning in the specification. In my case, I would like to be completely compliant.

10.2.2 201 Created
The request has been fulfilled and resulted in a new resource being created. The newly created resource can be referenced by the URI(s) returned in the entity of the response, with the most specific URI for the resource given by a Location header field. The response SHOULD include an entity containing a list of resource characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. The origin server MUST create the resource before returning the 201 status code. If the action cannot be carried out immediately, the server SHOULD respond with 202 (Accepted) response instead.
A 201 response MAY contain an ETag response header field indicating the current value of the entity tag for the requested variant just created, see section 14.19.

OK, the basic response must return 201 - Created status, and a Location header. It should contain an entity (which may only be entity-headers) and may contain an ETag. This is all very easy to do with Jersey. As you can see from the code snippet, we read the @FormParam information from a form, and create an object which we add to a List<Widget%gt; objects signified by the ws.add(widget) we get the index value for use in our URI. Next we create a URI from our request, and add a path to it for our index.

Finally we use Apache Commons Codec Hex to generate a hex string for our ETag. Finally this is all combined in a Response sent back to the client. This can be seen below.

Interesting Articles on JAX-RS

noreply@blogger.com (John Yeary) — Tue, 10 Jan 2012 14:20:00 +0000

What do I Read?

I was asked what articles I really like on Jersey, and how to learn more about it. Here is an incomplete list.

If you find more really good ones, please post comments, and I will add them to the list if I like them.

What Makes Jersey Interesting: Parameter Classes - This article includes examples which explain parametrized classes being used in Jersey.
What Makes Jersey Interesting: Injection Providers - This article demonstrates a method of doing dependency injection in Jersey. Specifically, it addresses using the @Context annotation to inject a UriInfo object.
RESTful services with jQuery and Java using JAX-RS and Jersey - This is a great article on using jQuery with JAX-RS. It also demonstrates using curl to test your services and includes code and sample syntax. This article is very professionally done.
Building RESTful Services with Java Using JAX-RS and Jersey — Sample Application - A demo application including code which demonstrates JAX-RS (Jersey). It is very well done, and it is easy to follow.
JAX-RS @FormParam example - This simple demo application uses an HTML based form to submit to a Jersey based REST service.
@FormParam - This is a simple to the point version of using @FormParam. It also explains how to test it using Jersey client.
REST web services with JAXB, JAX-RS and Sun Jersey - A well written article on using JAXB along with Jersey to create applications. The article includes examples of using curl to test the results.
JAX-RS @MatrixParam example - An example application that demonstrates @MatrixParam usage.
How to access parameters in a RESTful POST method - Multiple examples of how to get parameters from a POST.
REST with Java (JAX-RS) using Jersey - Tutorial - This is a superb article on JAX-RS and is very professionally done.
JAX-RS as the one Java web framework to rule them all? - This is just a general discussion article with links to other related technologies.

Using Apache HTTPD (Web Server) mod_proxy with GlassFish and Mercurial

noreply@blogger.com (John Yeary) — Sun, 08 Jan 2012 01:29:00 +0000

I wrote an article on how to use Secure Mercurial in GlassFish using SSL a few weeks ago. After the article was published, I was asked about URL rewriting. I had not really messed with it in the past except with PrettyFaces (which works like a champ for reference).

I tried a framework called Url Rewrite Filter which did a decent job of rewriting URLs. It is a simple library added to your project which uses a servlet filter to handle the URL re-writing. Add the filter to the web.xml, and a urlrewrite.xml which handles the rewrites. It works much in the same manner as PrettyFaces, but is targeted at JSP/Servlets.

In my case, I have an Apple G5 PPC which is the main server for my source repository for Mercurial. This presents some challenges since the Java version is limited to 1.5, and therefore GlassFish can't be upgraded to v3 from v2.1.1. I did manage to get GlassFish v3 to run with OpenJDK 1.7 (BSD Port), but the Zero VM is too slow to handle the load. Kurt Miller has a couple of the builds for Mac PPC There are a number of reasons for my interest, the primary one for URL rewriting is that GlassFish v3 supports AJP out of the box.

Enabling JK Listener (mod_jk)

Using GlassFish v3 would provide a more optimal solution as you can see from the administration console. I decided to try using the mod_jk articles I found for GlassFish v2 from Amy Roh: GlassFish supports configurable AJP Connector and Jean-Francois Arcand's Running GlassFish with Apache httpd. I tried to combine these with mod_proxy and mod_proxy_ajp. The articles are focused on a specific build, and I was unsuccessful in using them. I am sure it has to framework versions, but I did not want spend too much time troubleshooting.

Finally, I tried using a simple mod_proxy arrangement along with mod_rewrite. This arrangement was surprisingly easy to configure, and worked the first time I tried it.

Here is the configuration I used:

httpd.conf

httpd-vhosts.conf

Those simple changes allowed me to rewrite the URL, and open only two ports on my firewall 80 and 443 which are a very good arrangement. As noted in Secure Mercurial in GlassFish using SSL, I am using GlassFish SSL and security to handle my authentication so this is truly a very good solution.

Mercurial Repositories

MD5 Checksum and Cryptographic Signature Checks on Code

noreply@blogger.com (John Yeary) — Thu, 05 Jan 2012 16:04:00 +0000

I just wanted to post a generally good idea on downloading code. I recently downloaded some code from an Apache mirror site which I checked against its MD5 checksum and it failed. At first I thought that the file was corrupt and re-downloaded it. Again it failed the MD5 check, so I checked its cryptographic (GPG) signature and it failed.

I downloaded the code from another mirror and everything worked correctly. I notified the mirror site of the inconsistency, and carried on. However, I often wonder how much we shortcut our work, and fail to check that vital information.

Here is a gentle reminder. If the code has an MD5, SHA, and cryptographic signature, please take the extra 5 minutes to check all three. It will verify your downloads, are safe.

Also keep in mind that if you don't, and make a war file that contains these potentially infected sources, you are propagating the problem.

JAX-RS Tip of the Day: Using @CookieParam

noreply@blogger.com (John Yeary) — Thu, 05 Jan 2012 13:08:00 +0000

I am not a fan of using cookies in REST based services. I think that it makes the service less usable since the client must be able to store and retrieve cookies. This limits usage of the service, and introduces state outside of the service. Even the RFC for cookies is called HTTP State Management Mechanism. curl can be used to send cookies to test so not all is lost.

Alright, if I have not talked you out of using them yet, you must have a need for them. (Please rethink the idea though).

Use of cookies is accomplished by using the @CookieParam annotation. Also you can return responses as shown in the code below which include cookies.

Cookies have a limitations on the use of some characters, and names so please look at the RFC noted above for additional limitations. I demonstrate a couple of limitations which include the use of semi-colons (;), and commas(,). The nice thing about this example is you can see the output in a real browser (Internet Explorer not included) since most can handle application/json objects which are returned.

Here is the example code developed using NetBeans 7.1 RC2 and GlassFish 3.1.1: cookie-parameters.zip

CookieParameterResource.java

OpenJDK 1.7 with NetBeans 7.1 RC on Mac OS X 10.7

noreply@blogger.com (John Yeary) — Wed, 04 Jan 2012 20:04:00 +0000

A question was posed to me the other day on how to use OpenJDK 1.7 on NetBeans on a Mac since there is only a Developer Preview available. I decided that a video does a better explanation. This applies to NetBeans 6+ as well when using JDK 7 on a default platform of JDK 6.

JAX-RS Tip of the Day: Using YUI2 and YUI3 DataTables with Jersey and jQuery

noreply@blogger.com (John Yeary) — Sat, 31 Dec 2011 17:00:00 +0000

This is a great example (if I do say so myself) that demonstrates some really cool capabilities of Jersey combined with Yahoo User Interface versions 2 and 3 combined with jQuery to demonstrate interoperability of multiple technologies.

The example code that I have included uses GlassFish 3.1.1 with Jersey 1.11 and was developed using NetBeans 7.1 RC2. The data is from the default sample database included in NetBeans. Here is the code: jsonp-database.zip

The YUI2, YUI3, and jQuery libraries are provided by Yahoo and Google CDN. This is a further example of a truly RESTful application where the application is provided as a mashup of multiple web resources.

Yahoo User Interface (YUI) is an incredibly well developed UI framework for the web. The DataSource implementation is easy to use and is extremely adaptable. It is also very stable, and Yahoo eats its own dogfood like Google so it is battle tested. YUI3 is the next generation code, and the DataTable implementation includes a number of new features.

jQuery is an extremely popular Javascript framework with great AJAX functionality. I have included two examples (YUI2/YUI3) in which jQuery provides JSON data to the Yahoo DataSource.

Jersey provides the heavy lifting. In all of the examples I provide the data in a variety of formats, but I focus on providing both JSON and JSONP. JSONP allows us to use have cross-domain capabilities. The code also demonstrates how to use Jettison to encapsulate POJOs. The best part is that the data can be tested not only using your browser, but can also be tested using curl. This provides a very simple visual means of verifying data.

So what does it look like... look at the images below. The results speak for themselves.

YUI2 DataTable using JSP

YUI3 DataTable with JSONP and JAX-RS (Jersey)

Now for the most important part, how do you get here.

Technical Details

The CustomerFacadeREST class is the most interesting. The JSP and HTML pages have the details of how to use the various YUI DataSources along with jQuery. I will not print the pages out, and suggest that you simply download the code and examine it.

CustomerFacadeREST.java

CODE

Here is the code: jsonp-database.zip

JAX-RS Tip of the Day: Changing JSON-JAXB Default Mapping

noreply@blogger.com (John Yeary) — Sat, 31 Dec 2011 14:00:00 +0000

JSON-JAXB marshalling and un-marshalling by default uses a MAPPED configuration by default. However this can be changed by creating a @Provider which can change the default JSONConfiguration.

The example below changes the configuration to use BadgerFish which provides a better representation of an XML document. This representation does not provide the visual clarity that JSON generally provides, and is not generally human readable beyond basic notation. However, as noted that is not the point.

JSONJAXBContextResolver.java

This class specifically handles the Customer object, and maps it to use BadgerFish. Output:

JAX-RS Tip of the Day: Automatic Mapping of Jettison JSON Objects

noreply@blogger.com (John Yeary) — Sat, 31 Dec 2011 01:19:00 +0000

One of the more interesting features in JAX-RS (Jersey) is the ability to automatically "wrap" POJOs. This combined with automatic wrapping of JAXB annotated beans, and using Jackson to create JSON objects is a powerful mix. However, sometimes we want to wrap an object, and add some additional metadata.

This tip is a very simple tip which allows you to create Jettison JSONObject, and JSONArray objects to create new objects, "wrap" existing objects, or add additional metadata. I mention "wrap" the objects, please don't confuse this with JSONP.

To enable automatic POJO mapping, and Jettison objects add the following to the web.xml
This simple configuration will give you considerable flexibility in how you handle your objects. This will now allow you to produce JSONObjects like the code example below:

Multiple File Upload Examples

noreply@blogger.com (John Yeary) — Wed, 28 Dec 2011 15:05:00 +0000

I received an email today from a developer who was looking for examples of how to do multiple file uploads. I had written a post previously Multiple File Upload Options which detailed some of the options available. The option I recommended was Andrew Valums ajax-upload. The developer had indicated that the servlet I contributed was not working with Internet Explorer correctly. He was correct.

The code that I provided to Andrew Vallums was expecting application/octet-stream. However Internet Explorer and Opera were sending multipart/form-data. This would not work with the servlet example I posted since the handling mechanism is considerably different.

Alan O'Driscoll and I discovered the differences using Wireshark to sniff the packets and see what was being sent over the wire. As a result, I have written a new servlet to handle the differences between browsers.

The example application I wrote has a number of different examples included with minor variations on the theme. The version most folks are interested in is example #6 which uses the ajax-upload, and the new servlet. Here is the Apache Maven project which was developed on NetBeans and tested on GlassFish v 2.1.1. Here is a link to the project: ApacheFileUploadEE5.zip.

Here is another resource for file upload plugins: 9 Powerful jQuery File Upload Plugins. I discovered it while I was looking for something else.

MultiContentServlet.java

RichFaces 4.1.0 Final on NetBeans 7.1 IDE

noreply@blogger.com (John Yeary) — Sun, 18 Dec 2011 15:53:00 +0000

I created a video which demonstrates how to add RichFaces 4.1.0 Final as a component library in NetBeans 7.1. The video details how to add the required jars to create the library, demonstrates code completion, and finally deployment to GlassFish 3.1.1 for display.

In the video I take advantage of using my local Apache Maven repository which includes all of the dependencies required.

The list below includes references to the required libraries and versions. There is a link to the video as well.

sac-1.3.jar
guava-r08.jar
cssparser-0.9.5.jar
richfaces-core-api-4.1.0.Final.jar
richfaces-core-impl-4.1.0.Final.jar
richfaces-components-api-4.1.0.Final.jar
richfaces-components-ui-4.1.0.Final.jar

NetBeans 7.1 Library Manager

JAX-RS Tip of the Day: NetBeans Project Layout Recommendations

noreply@blogger.com (John Yeary) — Sat, 17 Dec 2011 03:10:00 +0000

When developing a JAX-RS project using NetBeans, I have a couple of simple recommendations. This will hopefully save you some time up front.

First create a Java → Java Class Library project.
This project will be used to contain classes we expect to share between the service and the client.
Create a second project Java Web → Web Application.
This project will serve as our REST service application.
Add the first project to the second project.
(Optional) Create a Java → Java Application
This final project will be used for as our client application.
(Optional) Add the first project to the client project.

The most important part of the recommendation is to make sure that you have a separate library project. The shared code can later be used in multiple projects without refactoring later.

Here is an example project that meets the recommendations above. It was created with NetBeans 7.1 RC2.

JAX-RS Tip of the Day: Use GenericEntity for Collections

noreply@blogger.com (John Yeary) — Sat, 17 Dec 2011 03:10:00 +0000

A common requirement is to generate List<T> for a REST service. The JAX-RS client must retrieve this, but the question is how to use it. The easiest method to use a GenericType<T>.

As an example, we have a service which returns a List<Widget>;. The client application will need to wrap the returned List<Widget>. We accomplish this as shown below using GenericType<List<Widget>>(){}.

The GenericType<List<Widget>>(){} allows us to wrap the List<Widget>, and return the <code>Collection</code> object we are really interested in. So remember, if you need to wrap a <code>Collection</code> use GenericType<T>.

Java Tip of the Day: Using ProcessBuilder to Make System Calls

noreply@blogger.com (John Yeary) — Thu, 15 Dec 2011 13:42:00 +0000

A few years ago I posted an article on how to execute OS level system calls using the Runtime class. When I wrote that article I was using a methodology which has been around since the 1.0 days of Java. In Java 5 Standard Edition, and subsequently improved in Java 6 and Java 7, is a class called ProcessBuilder. This class is a significant improvement over using the Runtime class. It provides complete flexibilty around the system call in terms of environment including directories, input and output streams, and execution.

Here is the NetBeans 7.1 Apache Maven project: process-builder-example.zip

When I execute it from NetBeans I get something like this:

[exec:exec]
total 24
drwxr-xr-x  12 jyeary  staff   408B Dec 15 08:19 .hg
-rw-r--r--   1 jyeary  staff    52B Dec 15 08:08 .hgignore
-rw-r--r--   1 jyeary  staff   1.7K Dec 15 07:59 nbactions.xml
-rw-r--r--   1 jyeary  staff   2.0K Dec 15 08:19 pom.xml
drwxr-xr-x   4 jyeary  staff   136B Dec 14 23:31 src
drwxr-xr-x   4 jyeary  staff   136B Dec 15 08:35 target
Exit Status : 0

Subversion to Mercurial Migration: Using --splicemap to Map Branches and Merges

noreply@blogger.com (John Yeary) — Wed, 14 Dec 2011 05:07:00 +0000

I am in the process of migrating all of my existing code projects to Mercurial from Subversion. However, the process has some snags associated with it. The biggest snag is that Mercurial does not understand the branching and merging process which is common in most Subversion repositories. If you convert a repository from Subversion to Mercurial the timeline indicates the branch, but does not associate it with the parent (default head). This results in "floating" heads (branches). One blogger I found called it a multi-headed Hydra.

Floating Heads (Branches)

What we really want is to show consistency, and progression. We want to show the branches and merges that are stored Subversion in Mercurial. We don't want to lose that vital historical information.

So we now know what the problem is, but how do we fix it. It turns out that the convert extension method has a very poorly (read not useful at all) documented switch called splicemap. This allows us to map specific revisions in parent-child relationships.

In the Rock Hymas blog, they give an example of the format for our splicemap for Subversion. It looks like the following:

svn:<uuid>/path/to/module@revnum

However, he ends short of what to do with this information, or how to obtain it. The article is good on explanation of the issue, but not on how to solve it. I will detail a solution and show you how I did it. I will also show how to use a splicemap "so that others may live"... oh wait that is the US Air Force Pararescue motto. I will show a working example to make a point.

Process

The first thing we will need is the UUID from Subversion. This can be obtained using the svnlook command.

apple1:~ root# svnlook uuid /usr/local/svnhome/
dbc3d074-2bd0-48a6-aa6f-e69526dc1c92

The UUID is absolutely vital. It will be different for every repository.

Next, we need a log of the activity for the project we are interested in mapping. We will use the svn log -v command to get the details. The results will look something like this. I have shortened my output to just a couple of relevant versions which we will need.

------------------------------------------------------------------------
r417 | jyeary | 2011-12-11 06:28:21 -0500 (Sun, 11 Dec 2011) | 1 line
Changed paths:
   M /MailService/trunk/MailService/build.xml
   M /MailService/trunk/MailService/nbproject/build-impl.xml
   M /MailService/trunk/MailService/nbproject/genfiles.properties
   A /MailService/trunk/MailService/nbproject/jaxws-build.xml
   M /MailService/trunk/MailService/nbproject/project.properties

Merged 3.0 tag to trunk, and updated NetBeans project files.

------------------------------------------------------------------------
r137 | jyeary | 2008-09-05 19:15:25 -0400 (Fri, 05 Sep 2008) | 1 line
Changed paths:
   M /MailService/branches/3.0-DEV/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/MailServiceBean.java

Updated properties.
------------------------------------------------------------------------
r128 | jyeary | 2008-09-02 17:51:49 -0400 (Tue, 02 Sep 2008) | 1 line
Changed paths:
   M /MailService/trunk/MailService
   M /MailService/trunk/MailService/nbproject/project.properties
   M /MailService/trunk/MailService/src/conf/sun-ejb-jar.xml
   D /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/MailService.java
   M /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/MailServiceBean.java
   D /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/MailServiceRemote.java
   A /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/local (from /MailService/branches/2.0-DEV/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/local:124)
   R /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/local/MailServiceLocal.java (from /MailService/branches/2.0-DEV/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/local/MailServiceLocal.java:124)
   A /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/remote (from /MailService/branches/2.0-DEV/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/remote:124)
   R /MailService/trunk/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/remote/MailServiceRemote.java (from /MailService/branches/2.0-DEV/MailService/src/java/com/bluelotussoftware/mail/ee/ssb/remote/MailServiceRemote.java:124)

merged 2.0 changes into trunk.
------------------------------------------------------------------------

Next we determine how to splice it together. We want to ensure that the parents, and children match up. The file format is: child parent with a space between the child and parent. We may also provide an additional parent which is separated by a comma such as child parent1,parent2

So the first splice in the splicemap for the 2.0-DEV branch looks like this:

svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@95 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57

Note: I am splicing the branch to the trunk (parent).

Using the hg convert --datesort --splicemap splicemap1 http://10.0.1.2/svn/MailService resulted in the following output:

assuming destination MailService-hg
initializing destination MailService-hg repository
svn: cannot probe remote repository, assume it could be a subversion repository. Use --source-type if you know better.
scanning source...
sorting...
converting...
21 creating structure
20 Moving project out of MailServiceApplication
19 Fixed javadoc
18 [Netbeans SVN client generated message: create a new folder for the copy]: '
spliced in ['svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57'] as parents of svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@95
17 2.0 development branch
16 1. Updated version numbers.
15 Added new sendMessage() method that allows for different MIME types.
14 1.Added new MimeMessage format to @Remote
13 1.Added new MimeMessage format to @Local
12 1. Added new MimeMessage format
11 Updated project properties
10 1. Created a new local package
9 1. Updated Javadocs
8 Updated project properties
7 merged 2.0 changes into trunk.
6 Adding new development branch
5 Adding trunk to 3.0 Development branch
4 Removed @Local interface implementation from bean.
3 Removed @Local interface.
2 Removed directory.
1 Updated properties.
0 Merged 3.0 tag to trunk, and updated NetBeans project files.
updating tags

More importantly, the result is what we are looking for. See the image below.

Branch

Alright, that solved the branching problem, but the more difficult issue is the merge. I was banging my head on my desk when I came across this Nabble post by Bruce Frederiksen which gave me the clue I needed to merge the branch back to the default head.

We need to tell Mercurial what the parent branches are for the merge from the perspective of the repository. In this case we want the revision at 57 as one parent from the branch above, and the other to be the revision we merge at on the default head. In this case revision 123. We also need to reverse the order (remember what I said about perspective... thanks Bruce). Our new splice looks like:

svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@95 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@127 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57,\
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@123

Note: I am splicing the trunk@127 to the trunk@57 and branch@123. This is the trick.

Note: The splices are wrapped to fit on the page, the actual file requires each complete splice to be on a new line.

The result when we run the hg convert command again results in merging the branch back to the default head.

Branch-Merge

Finally, I make the remaining splices to complete the mapping which look like:

svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@95 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@127 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@57,\
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/2.0-DEV@123
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/3.0-DEV@129 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@128
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@417 svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/trunk@128,\
svn:dbc3d074-2bd0-48a6-aa6f-e69526dc1c92/MailService/branches/3.0-DEV@137

Note: The splices are wrapped to fit on the page, the actual file requires each complete splice to be on a new line.

This results in the final completed completed product.

Completed

Conclusion

I hope that this explains how to effectively use a splicemap to branch and merge your Subversion repository into Mercurial. I hope that this saves some other unfortunate developers from trying to figure out a really cool, but poorly documented feature of the convert extension.

Secure Mercurial in GlassFish using SSL

noreply@blogger.com (John Yeary) — Wed, 14 Dec 2011 03:00:00 +0000

A friend of mine, Wade Chandler, contacted me the other day to ask about implementing security (SSL) for Mercurial on GlassFish. Wade is NetBeans Dream Team member, and all-around great guy. I told him I would figure it out, and tell him how to do it. I also decided that I would explain to everyone how to take advantage of this cool functionality using GlassFish 3.1.1 and NetBeans 7.1 RC 2.

In a previous blog entry, Mercurial on GlassFish 2.1 Using Multiple Repositories, I successfully deployed Mercurial repositories on GlassFish. This time I have given everything a Java EE 6 upgrade.

One of the great aspects of using GlassFish to provide SSL is that we don't require messy configuration in Python and Mercurial to implement SSL. GlassFish handles the security for us. The only thing that the end client users will want to do is add the server fingerprint to the .hgrc file in their home directory. This will remove any warnings about a missing server fingerprint, or cacerts file.

If you have used Mercurial on Apache HTTPD Server, along with cacerts, and client certificates... this will make you make the switch immediately.

Requirements:

You can create your own web project in NetBeans, but the Mercurial.zip is a self-contained Mercurial repository, and NetBeans project which can serve as the basis for your own server with very little modification. I would simply use my project as the basis for your own deployment.

Note: The requirements include repos.zip which is optional. The repos.zip contains sample mercurial repos which are required for the demo.

Procedures

Configure GlassFish

Start the GlassFish administration console.
Go to Configuration → server-config → Security → Realms → file.
Note: You may create your own realm, this is just for demonstration purposes.
Click on Manage Users.

GlassFish Security Configuration
Click on New. Add a user called mercurial in a group called Mercurial Users with the password hg.
Note: Please change the username, group, and password for production.
The server configuration is complete.

NetBeans

Using NetBeans open the Mercurial project from the requirements link above.
There are four files in the project (simple huh?) which control the behavior of the Mercurial repository on GlassFish: hgweb.cgi, hgweb.config, web.xml, and glassfish-web.xml.
The hgweb.cgi should not need to be modified under normal circumstances.
Open the hgweb.config file. Make any changes that you would like to customize the application. Pay particular attention to the location of the repositories.
There are two examples: (1) using the "/" along with a * on the end of the file path will list all repositories in the specified directory, and (2) a specific listing.

Note: Please note that allow_push = * and push_ssl = false are set. This will allow push to the server and let GlassFish take care of the SSL.
The web.xml file has been modified to add security constraints for the alpha, and beta example repositories.
The alpha repository has security only on PUT, POST, and DELETE.
The beta repository will not allow you to perform any actions without logging into the server.
The gamma repository has no security constraints.

Note: You will need to modify the security constraints to meet your requirements.

Note: You secure the entire repository by adding:
<url-pattern>/cgi-bin/hgweb.cgi/*</url-pattern>.
The glassfish-web.xml is used to map the security role(s) to the file realm on the GlassFish server.
Clean, build, and Run the application.
When the application successfully deploys, go to the secure connection on https://localhost:8181/Mercurial. It may prompt you to accept the server certificate. Accept it.
You should see something like the image below:

Mercurial Repositories
Click on the beta repository. It should prompt you to login.
Congratulations you are running Mercurial on GlassFish securely.

Note: To secure the application completely, you should disable the non-secure http port listener on port 8080, or redirect requests to the secure port.

Verification

It would be unwise to accept that the application is secure without verification.

I made this demonstration video to prove the connection is secure. The demonstration shows both insecure and secure connections to the repository. I use Wireshark to sniff the connection, and pull the repository from the command line using hg clone.

Conclusion

Mercurial is a great distributed source control system. Its ease of use is further enhanced by using GlassFish 3.1.1 to provide the front end to your repository. It also simplifies the deployment, and use models.

Java Tip of the Day: Generic JAXB Map XmlAdapter

noreply@blogger.com (John Yeary) — Sat, 03 Dec 2011 04:27:00 +0000

There are a couple of well-known issues with using Map<K,V> with JAXB. First, JAXB does not like interfaces. Second, it considers a Map<K,V> to be a bean so it does not handle the internal implementation. The JAXB JIRA Issue is here JAXB-223

There are a number of articles published about how to adapt a Map<K,V> to work in JAXB. However, all of them I have seen are very specific mappings like Map<String,String>, or Map<Integer,String>, etc. None of them address the more specific generic case Map<K,V>. This example handles the more general case using generics. All the other examples I looked at online generally follow the example code from the XmlAdapter javadoc. The code snippet does not encapsulate the values, and generally does not adhere to good coding practices.

The process is to create an adapter is as follows:

Create an mapping entry for Map<K,V>. In our case, we will create one called: MapEntryType<K, V> which will map our generic types.
Next we create a value holder for our Map<K,V> which contains a List<MapEntryType<K, V>> of our entries. The value holder looks like MapType<K, V>
Finally, we create an adapter which extends XmlAdapter. In our case, it is very long since I am using Java 6. In Java 7, we can use the diamond <> operator to shorten it. The adapter is XmlGenericMapAdapter<K, V> extends XmlAdapter<MapType<K, V>, Map<K, V>>.

Note: Please note that the MapEntryType<K, V> has the @XmlAccessorType(XmlAccessType.PROPERTY) set to property and the getters are annotated as @XmlElement.

The NetBeans 7.1 RC1 Apache Maven project can be downloaded here: generic-jaxb-map-adapter.zip

MapEntryType<K, V>

MapType<K, V>

XmlGenericMapAdapter<K, V> extends XmlAdapter<MapType<K, V>, Map<K, V>>

Here is the output from our adapter.

We take advantage of JAXB to provide automatic XML Schema definitions for our keys and values. The only problem with this approach is that it is very "chatty". It provides a schema definition for each key and value.

If you know that you are going to be using simple types, it is more appropriate to use a specific type rather than using generics. We would replace all of the <K, V> with <String, String>. This will remove all of the schema definitions and produce an output which is much cleaner. The String adapters are included in the source code for further review.

Coding Tip of the Day: Using Map Collections

noreply@blogger.com (John Yeary) — Fri, 02 Dec 2011 20:12:00 +0000

I was working on some code in the last few days where I came across a protected Map map instance variable. I examined the code and javadocs which indicated that the intent was to provide a mapping of keys and values. However the actual implementation was setting the keys and values to the same value. This again is not really an issue. However, the usage of the Map was incorrect.

See if you can determine the issue from the code below.
The MapValueHolder<K,V> looks like the following. Here is the implementation.
As a general rule, you should use the key to fetch the value. Even if you expect that the values are the same. Since the instance variable is protected it can be modified in a sub-class, or any class in the same package. Since we code to interfaces, or abstractions (super classes). We can not be sure that the implementation class we are provided does what we expect. In this case, it does what we expect.

MapValueHolder Key --> A
MapValueHolder Key --> B
MapValueHolder Key --> C
A --> A
B --> B
C --> C

Here we continue using another implementation. This time the result looks different.

MapValueHolder Key --> A
MapValueHolder Key --> B
MapValueHolder Key --> C
A --> null
B --> null
C --> null

This is because our MapValueHolder<K,V> is different. Admittedly, the implementation above is a bit contrived. We really are getting is a Set<K>. In the next example, we get another result. The result is considerably different.

MapValueHolder Key --> A
A --> A

Here is the implementation. Again, this is a contrived result where we are just returning the first key and value in a Map<K,V>. Here is another example which demonstrates unexpected results. This results in this very strange result.

Map --> {}

Here I use reflection to modify the expected result. If you have followed me to this point, you will have learned a little bit about the Java Type system, reflection, abstraction, and Collections. Yet, you may be asking what is the tip? Here are some rules:

Use the keys to fetch values.
Check for null values. In my examples, I did not check for null values.
Use the correct Java Collection.

The Java Collection Framework covers most common cases. Take the time to read about them. The code above was based on a couple of issues I discovered: (1) Using keys for values, and (2) not using the correct collection. In this case, they should have used a List<V>. If they wanted to use an ordered Map<K,V>, they should have used a LinkedHashmap<K,V>.

The code for the examples are here: bad-mapping.implementation.zip

The code was developed using NetBeans 7.1 RC1.