Scratching my programming itch

Book Review: Gradle Effective Implementation Guide

2013-04-30T22:02:00.000-05:00

Overview
The choice of build tools in the Java space has basically been Ant or Maven since the early days of Java. Ant was the first tool to arrive and then Maven was created to help provide standardization, assuming you drank the Maven Kool-Aid, meaning that you could live with the standards set out by Maven.

Now a new build tool has arrived, Gradle. Gradle adoption started slowly at first, but has definitely been picking up momentum as more and more projects migrate from either Ant or Maven over to Gradle. Gradle can't be ignored!

Gradle is built on top of the Groovy. The tool provides a rich Domain Specific Language (DSL) based on Groovy. The DSL is the key component of Gradle.

Contents

Starting with Gradle, creating and working with build scripts
Gradle for Java projects
Dependency management
Testing building and publishing artifacts
Multi-project builds
Gradle with Groovy and Scala
Maintaining code quality with tools like Checkstyle, PMD, FindBugs, JDepend, CodeNarc and the Sonar plugin - one of my favorite chapters since I recommend using code static analysis tools for improving code quality
Using Gradle with Continuous Integration - covers Jenkins, TeamCity and Bamboo
IDE support - provides excellent coverage of Eclipse and IntelliJ IDEA support

Pros

Very good reference guide
Sample source filled with a lot of complete examples
It's by Mr. Haki, creator of Groovy Goodness, Grails Goodness and of course, Gradle Goodness blog postings!

Cons

I started out running and verifying every example, but after submitting 6-7 errata for just the first chapter, I stopped running all of the examples and dropped back to just executing examples that peaked my interest.
The book contains only a single reference to Ant and no discussion on Gradle's integration with Ant. In all fairness, Ant integration is not listed in the bullet points on the back of the book that shows "What you will learn", but I think that is a mistake. Readers are going to be coming from either Ant or Maven and will be looking for help in migrating to Gradle or for help on how to do things in Gradle that they are already doing in Ant or Maven.
The book is 356 pages but a lot of pages are 'eaten up' by showing the output of all the example build scripts, and screen shots from Chapter 11, Using Gradle with Continuous Integration and Chapter 12 IDE support.

Summary
The book is a very good reference for using Gradle and I would definitely recommend it for anyone interested in learning more about Gradle.

Gradle thoughts (not specific to the book)
This the second book I have read on Gradle. I have read the User's Guide a couple times over the last two years in an attempt to get a firm grip on Gradle. One of my challenges is that I don't get to use Gradle in my day job, just at night and on weekends. This limited use seems to extend the learning curve, which I think for Gradle is larger/longer than other tools.

"Convention over configuration" is great for reducing the amount of code in projects, BUT somewhere the user needs to know where the conventions are defined. I think one of the keys to learning Gradle is to get a firm grasp on the Gradle DSL. Without a grasp on the DSL, a new Gradle user just keeps wondering "Where are these properties/methods coming from?"

Minor Gotchas from migration to Java 7

2013-02-18T19:43:00.003-06:00

After several years of pushing, we are finally starting to pull our application out of the dark ages in terms of software levels and one of those steps is upgrading our Java version to Java 7. For the most part, this has been fairly painless, but there have been a couple surprises:

Test cases that had been running successfully for years started failing when we switched to Java 7. It turns out that we had some test cases that depended on previous test cases within the same class. This is bad on two fronts, 1) the test cases should each be independent and 2) the execution order of test cases within a test class is not guaranteed, although, up to now(before Java 7), the order was the order found in the source code. I found this link that helped explain the inconsistent ordering I was seeing within Eclipse. This is not a problem with Java 7 but rather with our test cases. Just something that cropped up when we started running with Java 7

Our Swing clients began throwing ClassCastExceptions, and this is code that has been in place for over 10 years, dating back to Java 1.4 Reading the javadocs helped explain the problem, but I decided to research a bit more to find out exactly why this was happening now. We were adding entries to a TreeSet to hold forward and backward traversal keys for the FocusManager, putting KeyStroke objects into the TreeSet. The javadoc clearly states that the objects put in the TreeSet (or TreeMap underneath) must be Comparable or you need to create the TreeSet with a Comparator for ordering the objects. We were doing neither, and not getting exceptions. Digging into the java source code, I found that starting in Java 7, the TreeMap class has starting enforcing this 'requirement', where the earlier versions of the class had that code commented out, thus not really requiring the comparator. The fix was easy, I created a KeyStrokeComparator and passed that to the TreeSet constructor.

I also noticed that our java jar signing had started logging a warning about the certificate not being valid. Wanting to keep the build clean and not have these warnings, I did some searches and found several postings related to Java 7 and the removal of the MD2withRSA algorithm from the code signing certificates. I worked with support from Verisign to help with the removal of that algorithm from our certificate chain and the warning went away! Unfortunately, that was not the end of the code signing problems.

Next, our web start clients would no longer start successfully, instead throwing an exception with the all-too-familiar 'not all jars signed with the same certificate' error message frequently encountered by web start programs, at least during development! I've written another blog post about how to help debug through this and figured it was just 'one of the usual suspect jars' that we have problems with, but nope, that wasn't it. Knowing that all this happened since the certificate was updated caused me to review all of our signed jars to look for clues. All the jars had been signed with the same certificate, but the 3rd party jars were signed BEFORE the chain entry was remove, whereas our application code was signed as part of the build process, AFTER the certificate was changed. This led me to believe that even though the jars were signed with the same certificate, that there must be some difference between the jars signed before and after the change to the certificate.

SHA1-Digest

SHA-256-Digest.

All in a days work! Just Happy to be current on the Java level. Next up, migrating to EJB3 and Hibernate 3 or 4.

Hope this helps!

Grails & EJB2, my stumbling points

2012-11-29T23:19:00.000-06:00

I have been 'playing' with Grails for a couple years now. I say 'playing' because I don't get to use Grails for my day job. My day job revolves around a Point-of-Sale (POS) and Back Office system with Web Start clients written using Swing and the server back-end uses EJB2.1 with Hibernate handling the persistence-layer. Yeah, I know, nobody still uses EJB 2.1! I have been pushing for some technical upgrades and hopefully that will happen this year, but that not want I want to talk about.

In the past, when I wanted to create a small work-related Grails project, I took what I saw as the easier way out. I used the static mappings in the domain object to map tables and columns to fields in the domain object, and this works fine. Recently, I decided to re-visit one of my small projects with the intention of using Grails as the front-end and using the existing EJBs to get access to server data. Below is a list of my stumbling points along the way to getting this project working. Only one of the points is EJB2 related but the others are all valid for a lot of current projects.

My stumbling points:

resources.groovy bean definition: JndiObjectFactoryBean vs SimpleRemoteStatelessProxyFactoryBean
LinkageError - referencing javax.management.MBeanServer
Compile errors after removing Hibernate

BuildConfig.groovy

resources.groovy bean definitions:
When I decided to use the existing EJBs within Grails, I did some google searches to find the correct format for defining these beans within the resources.groovy file. I found some examples that worked great if you were using EJB 3.x. If you are using EJB 3.x, you reference the JndiObjectFactoryBean and this returns the remote service interface. If your beans are actually EJB 2.1 and you use the JndiObjectFactory, you get back the 'home' interface, which has no business interface methods included. Again, the problem is that I am using EJB 2.1. Another couple google searches turned up Dave Klein's "Grails Integration Strategies" presentation and the answer was there on page 15. For EJB 2.x, you need to use the SimpleRemoteStatelessProxyFactoryBean in order to get the remote service interface. Spring's SimpleRemoteStatelessProxyFactoryBean handles retrieving the home interface, calling PortableRemoteObject.narrow() and then calling create() in order to return a remote service interface for EJB 2.1.

resources.groovy for EJB 2.x

beans = {
    ejbJndi(org.springframework.jndi.JndiTemplate) {
        environment = [
            "java.naming.factory.initial" : "org.jnp.interfaces.NamingInitialContextFactory",
            "java.naming.provider.url" : "jnp://localhost:1099",
            "java.naming.security.principal" : "root",
            "java.naming.security.credentials" : "root"
        ]
    }
    environmentSvc(org.springframework.ejb.access.SimpleRemoteStatelessSessionProxyFactoryBean) {
        jndiName = "ejb/EnvironmentService"
        businessInterface = "com.xxx.server.service.environment.EnvironmentService"
        jndiTemplate = ref("ejbJndi")
    }
    
}

resources.groovy for EJB 3.x

beans = {
    ejbJndi(org.springframework.jndi.JndiTemplate) {
        environment = [
            "java.naming.factory.initial" : "org.jnp.interfaces.NamingInitialContextFactory",
            "java.naming.provider.url" : "jnp://localhost:1099",
            "java.naming.security.principal" : "root",
            "java.naming.security.credentials" : "root"
        ]
    }
    processService(org.springframework.jndi.JndiObjectFactoryBean) {
    jndiName = "ejb/ProcessService"
    jndiTemplate = ref("ejbJndi")
    
}

Linkage Error:
The next step was to compile and run the application, but I kept getting these 'Linkage Errors' shown below. Again, after sufficient google searches, I found a couple different postings with the same issue. I am using JBoss as my application server and using the JBoss jars within Grails. This problem appears to be a duplication of classes on the classpath between Tomcat and JBoss in the JMX area. According to the postings, there are (at least) two options:

delete Tomcat and install Jetty, which I did and it worked great!
deploy a war to Tomcat. Apparently this problem only happens in the development environment and deploying your application as a war side steps this problem.

Below is an example of the LinkageError

c:\dev\groovy\hv>grails run-app
| Compiling 113 source files
Note: C:\Users\mmiller\.grails\2.1.0\projects\hv\plugins\cache-1.0.0\src\java\grails\plugin\cache\web\GenericResponseWrapper.java uses or oerrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
| Running Grails application
| Error Failed to initialize component [StandardServer[-1]] (NOTE: Stack trace has been filtered. Use --verbose to see entire trace.)
org.apache.catalina.LifecycleException: Failed to initialize component [StandardServer[-1]]
        at org.grails.plugins.tomcat.InlineExplodedTomcatServer.doStart(InlineExplodedTomcatServer.groovy:137)
        at org.grails.plugins.tomcat.TomcatServer.start(TomcatServer.groovy:102)
        at _GrailsRun_groovy$_run_closure5_closure12.doCall(_GrailsRun_groovy:176)
        at _GrailsRun_groovy$_run_closure5.doCall(_GrailsRun_groovy:153)
        at _GrailsRun_groovy.runInline(_GrailsRun_groovy:121)
        at _GrailsRun_groovy$_run_closure1.doCall(_GrailsRun_groovy:67)
        at RunApp$_run_closure1.doCall(RunApp.groovy:33)
Caused by: java.lang.LinkageError: loader constraint violation: loader (instance of ) previously initiated loading for a differnt type with name "javax/management/MBeanServer"
        ... 7 more
| Error Server failed to start: org.apache.catalina.LifecycleException: Failed to initialize component [StandardServer[-1]]

Compile errors after removing Hibernate:
After getting Jetty installed, I was ready to run the app and get things going. Nope, more errors - now I can't compile because something can not find org.hibernate.cfg.Configuration! I am working on Grails 2.1.0 and starting in that release, the database migration plugin depends on Hibernate, so if you are going to remove Hibernate, you also need to remove the database migration plugin. Easy fix - edit BuildConfig.groovy to remove the database migration plugin and re-run. Below is a copy of both the short error version and the longer version, which used --stacktrace to get a full stacktrace.

c:\dev\groovy\hv>grails run-app
| Uninstalled plugin [tomcat]
| Uninstalled plugin [hibernate]
| Compiling 113 source files.
| Error Fatal error during compilation org.apache.tools.ant.BuildException: java.lang.NoClassDefFoundError: org/hibernate/cfg/Configuration
(Use --stacktrace to see the full trace)


c:\dev\groovy\hv>grails --stacktrace run-app
| Compiling 113 source files.
| Error Fatal error during compilation org.apache.tools.ant.BuildException: java.lang.NoClassDefFoundError: org/hibernate/cfg/Configuration
(NOTE: Stack trace has been filtered. Use --verbose to see entire trace.)
java.lang.NoClassDefFoundError: org/hibernate/cfg/Configuration
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.codehaus.gant.GantBuilder.invokeMethod(GantBuilder.java:99)
        at _GrailsCompile_groovy$_run_closure3_closure8_closure9.doCall(_GrailsCompile_groovy:62)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at _GrailsCompile_groovy$_run_closure3_closure8_closure9.doCall(_GrailsCompile_groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at _GrailsCompile_groovy.withCompilationErrorHandling(_GrailsCompile_groovy:69)
        at _GrailsCompile_groovy.this$4$withCompilationErrorHandling(_GrailsCompile_groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:168)
        at _GrailsCompile_groovy$_run_closure3_closure8.doCall(_GrailsCompile_groovy:61)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at _GrailsCompile_groovy$_run_closure3_closure8.doCall(_GrailsCompile_groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at _GrailsCompile_groovy$_run_closure3.doCall(_GrailsCompile_groovy:58)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at org.codehaus.gant.GantBinding.withTargetEvent(GantBinding.groovy:90)
        at org.codehaus.gant.GantBinding.this$4$withTargetEvent(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.processClosure(GantMetaClass.java:81)
        at org.codehaus.gant.GantMetaClass.processArgument(GantMetaClass.java:95)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:128)
        at _GrailsCompile_groovy$_run_closure2.doCall(_GrailsCompile_groovy:45)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at org.codehaus.gant.GantBinding.withTargetEvent(GantBinding.groovy:90)
        at org.codehaus.gant.GantBinding.this$4$withTargetEvent(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:168)
        at _GrailsPackage_groovy$_run_closure2_closure9.doCall(_GrailsPackage_groovy:45)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at _GrailsPackage_groovy$_run_closure2_closure9.doCall(_GrailsPackage_groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at _GrailsPackage_groovy$_run_closure2.doCall(_GrailsPackage_groovy:44)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at org.codehaus.gant.GantBinding.withTargetEvent(GantBinding.groovy:90)
        at org.codehaus.gant.GantBinding.this$4$withTargetEvent(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.processClosure(GantMetaClass.java:81)
        at org.codehaus.gant.GantMetaClass.processArgument(GantMetaClass.java:95)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:128)
        at RunApp$_run_closure1.doCall(RunApp.groovy:28)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantMetaClass.invokeMethod(GantMetaClass.java:133)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16_closure18.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at org.codehaus.gant.GantBinding.withTargetEvent(GantBinding.groovy:90)
        at org.codehaus.gant.GantBinding.this$4$withTargetEvent(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy:185)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at org.codehaus.gant.GantBinding$_initializeGantBinding_closure5_closure16.doCall(GantBinding.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at gant.Gant$_dispatch_closure5.doCall(Gant.groovy:381)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at gant.Gant$_dispatch_closure7.doCall(Gant.groovy:415)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at gant.Gant$_dispatch_closure7.doCall(Gant.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at java_util_concurrent_Callable$call.call(Unknown Source)
        at gant.Gant.withBuildListeners(Gant.groovy:427)
        at gant.Gant.this$2$withBuildListeners(Gant.groovy)
        at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
        at gant.Gant.dispatch(Gant.groovy:415)
        at gant.Gant.this$2$dispatch(Gant.groovy)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at gant.Gant.invokeMethod(Gant.groovy)
        at gant.Gant.executeTargets(Gant.groovy:591)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at gant.Gant.executeTargets(Gant.groovy:590)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
        at com.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1231)
Caused by: java.lang.ClassNotFoundException: org.hibernate.cfg.Configuration
        ... 123 more
| Error Fatal error during compilation org.apache.tools.ant.BuildException: java.lang.NoClassDefFoundError: org/hibernate/cfg/Configuration

Minor mention:
Starting in Grails 2.x, BuildConfig.groovy plays a bigger role in dependency resolution. I read in some forum postings that modifying BuildConfig.groovy was now the preferred way of handling plugins rather than using the command line 'grails install-plugin ' or 'grails uninstall-plugin '. My problem was that I was starting with a Grails project that was originally created with Grails 1.3.7 and BuildConfig.groovy had not been updated with all the plugins. I had to use a combination of the command line (grails uninstall-plugin) plus edits to BuildConfig.groovy to resolve my problems.

Final Thoughts:
The problems documented above were mainly my fault and not that of Grails! I am just documenting them to possibly help anyone trying the same type of integration. Maybe this will save them some time and frustration!

Hope this helps!

Resolving the "Not all jars signed with same certificate" issue

2012-08-28T22:32:00.001-05:00

Almost a year ago I wrote posting on how to resolve the "Not all jars signed with the same certificate" problem encountered from time to time with signed Web Start applications. My solution at that time was to use a short Groovy script to show each jar and a list of the certificates the jar was signed with. For those familiar with Groovy, that's still my suggestion.

Recently one of our customer service folks was fighting this same problem at a customer site. To make matters worse, they were not familiar with Groovy, so using my previous script was not an option. I decided to bite the bullet and write the Java equivalent.

Below is my Java-equivalent of the Groovy script. The code is saved as the JarSigners project at GitHub.

package util;

import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;

/**
 *  Show which jars are signed with which code signing certificates
 */
public class JarSigners {

 private String folderName;
 private Map> map;
 private Set certs;
 
 public JarSigners(String folderName) {
  this.folderName = folderName;
  map = new HashMap>();
  certs = new HashSet();
 }
 
 public void listJarSigners() throws ZipException, IOException {
  File dir = new File(folderName);
  FilenameFilter filter = new FilenameFilter() {
      public boolean accept(File dir, String name) {
          return name.endsWith(".jar");
      }
  };
  String[] files = dir.list(filter);
  System.out.println("Found "+files.length+" files in folder: "+folderName);
  for (String fileName : files) {
   checkJarForCertificate(fileName);
  }
  System.out.println("Certificates used to sign the jars: "+certs);
  if (certs.size() > 1) {
   System.out.println("WARNING: More that one certificate shows up in the jars!");
  }
  System.out.println("Details jar->certificate: "+map);
 }
 
 private void checkJarForCertificate(String fileName) throws ZipException, IOException {
  File jar = new File(folderName, fileName);
  ZipFile zip = new ZipFile(jar);
  //System.out.println("Processing "+fileName);
  for (Enumeration entries = zip.entries(); entries.hasMoreElements();) {
         // Get the entry name
         String zipEntryName = ((ZipEntry)entries.nextElement()).getName();
         if (zipEntryName.endsWith(".RSA")) {
          if (!certs.contains(zipEntryName)) {
           certs.add(zipEntryName);
          }
          if (map.containsKey(fileName)) {
           List list = map.get(zipEntryName);
           list.add(zipEntryName);
          } else {
           List list = new ArrayList();
           list.add(zipEntryName);
           map.put(fileName, list);
          }
         }
     }
  zip.close();
  
 }

 /**
  * @param args
  * @throws IOException 
  * @throws ZipException 
  */
 public static void main(String[] args) throws ZipException, IOException {
  if (args.length != 1) {
   System.out.println("Invalid invocation: you must pass a folder name");
   return;
  }
  JarSigners js = new JarSigners(args[0]);
  js.listJarSigners();
  System.out.println("Done!");
 }

}

Hope this helps!

Auto-completion for email addresses in Swing

2012-08-23T22:56:00.001-05:00

Recently we added the ability to send Electronic Receipts from our Point-of-Sale product at work. One of the requirements was to enhance the user interface around the capture of email addresses. We were just using a standard JTextField to capture the value. Specifically, we were looking to make the entry of the email address easier by way of auto-completion, much like a lot of ajax-enabled fields in web applications.

I didn't want to re-invent the wheel and felt certain that I could something to get me started with the help of Google! After a couple searches, I found a great starting point for the new code. Samuel Sjöberg's auto-completion was extremely close to what I wanted. Rather than auto-completing a person's name, I wanted to provide auto-completion for the part of the email address after the "@". I also modified the code a bit to read the auto-completion candidates from a properties file. This allows you to provide a properties file containing some of the most common email addresses. Below is a copy of a sample properties file.

#################################################################################
# Commonly used Email Addresses                            
#                                                          
# This file will be read when the component starts up      
# and used to create an ArrayList of names to be used 
# for auto-completion of email addresses entered into 
# fields that assign the EmailAddressAutoCompletionDocument                      # to the field.  
#
# For better performance, put the most commonly used/encountered
# email addresses at the top of the list.
#
#################################################################################

 hotmail=hotmail.com
 gmail=gmail.com
 yahoo=yahoo.com
 verizon=verizon.net
 comcast=comcast.net
 aol=aol.com
 outlook=outlook.com
 mail=mail.com
 bigstring=bigstring.com
 shortmail=shortmail.com
 lycos=lycos.com
 excite=excite.com
 zoho=zohomail.com
 zzn=zzn.com

Below is the interesting part of the auto-completion of the email address field. First, I included a check for a max length of 255 to prevent too much data for our database columns. Next, we check for an "@" sign entered. If no @ sign, add the text. There is also a check to prevent a second @ from being added to the field. Once we get past that, its time to call the completion service looking for a match on the email address. If a match is found, we add the rest of the string to the text box and select the part of the address that wasn't actually entered by the user.

 public void insertString(int offs, String str, AttributeSet a) throws BadLocationException {
        if (str == null || str.length() == 0) {
            return;
        }

        // check for max length first
        if (str == null || ((str.length() + getLength() > maxLength) && maxLength > 0))
   return;
        
        String text = getText(0, offs);        // Current text.
        int atSignIdx = text.indexOf(AT_SIGN); // 1st @ sign
        if (atSignIdx != -1 && offs > atSignIdx) {
         String fullText = text+str;
         //prevent entry of 2nd @ sign
         if (fullText.indexOf(AT_SIGN, atSignIdx+1) != -1) {
          return;
         }
         String address = fullText.substring(fullText.indexOf(AT_SIGN)+1);
         String completion = complete(address);
         int length = offs + str.length();
         if (completion != null && text.length() > 0) {
                super.insertString(offs, completion.substring(address.length()-1), a);
                documentOwner.select(length, getLength());
         } else {
          super.insertString(offs, str, a);
         }
        } else {
            super.insertString(offs, str, a);
        }
    }

I also added automatic validation of the full address by providing a regular expression to match against and then assigned a KeyListener to help implement the validation with color coding. As the user types an email address, the foreground color is red and then when the email address is complete, either through auto-completion of the address portion or just typing, the foreground color is changed to green for a valid address.

Thanks to Samuel for posting his code. My updated version can be found at GitHub.

Hope this helps!

Book Review: MongoDB in Action

2012-08-13T21:18:00.000-05:00

Overview
I am a big fan of Manning Publications books and so when it came time to do some personal research on NoSQL, I started with the Manning website to what books they had that covered the topic. I didn't really have any specific requirements, like document model vs key value pairs vs graph database. I just wanted something fairly current and covering one of the main players in the area.

I found this book on the website and the publication date (2012) helped ensure that I was getting was something current and not a couple years old. The other nice surprise was the size of the book. For anyone that's read an 'In Action' book from Manning, you know they generally tend to be fairly large, on the scale of 300-400 page or more. This book weighs in at 278 pages and that includes 5 appendices!

Contents
The book is broken down into 3 parts:

Getting Started, which covers about 50 pages talking about MongoDB at a high-level, the core server, tools, replication, the JavaScript shell and writing programs and using drivers to interact with the database.
Application Development in MongoDB covers principles of schema design, MongoDB databases, collections, Mongo's query language, updates atomic operations and deletes.
MongoDB Mastery covers index and query optimization, replication, sharding and deployment.

Summary
An excellent book by Kyle Banker and yet another excellent book from the folks at Manning Publications. I blew through the first two parts of the book, eager to learn more about NoSQL and MongoDB. The author does an excellent job explaining concepts within MongoDB and backs up his statements with the reasons why things are done the way they are.

It took a bit longer for me to get through part 3 of the book, no fault of the author or book. Since this was really my own personal research and there is no way I will be implementing MongoDB or NoSQL at my day job, other than maybe some small personal tools/utilities, the topics of replication and sharding just didn't have the same appeal as learning about the core server, the query language and schema design!

Overall, I would recommend this book to anyone interested in learning about MongoDB.

GDK vs JDK

2012-07-16T21:22:00.002-05:00

A lot of folks new to Groovy quickly ask the question, "What's is the GDK?" or "What's the difference between the GDK and the JDK?"

If you take a look thru the GDK documentation, you will see that the GDK acts like an extension to the Java JDK. New methods are added to the GDK help remove the ceremony and allow Groovy code to be short and concise. Think of it as a big decorator pattern! Slight clarification: the JDK is used to build the GDK and referenced by the GDK but it is NOT part of the GDK and not shipped with the GDK.

Maybe a picture will help. You can see some of the new GDK methods added to the Java classes below.

Hope this helps!

JBoss AS7 JNDI & EJB 3.1 Naming changes

2012-07-14T20:48:00.000-05:00

As a result of the "feature train" continuing to march on and us not keeping our software stack up-to-date, our team finds ourselves in the un-enviable position of having to migrate:

JBoss 4.2.3 to AS 7.1.x (currently looking at 7.1.1)
EJB 2.1 to EJB 3.1
Hibernate 2 to Hibernate 3 or 4

in quick fashion. I mean, who wants to ship a new release with 8-10 year old software, not me!

The following is the result of some research I did while looking at upgrading from JBoss 4.2.3 with EJB 2.1 to AS 7.1.x with EJB 3.1. I am sure there will be more posts related to this migration in the near future, but this one is related to changes in the JNDI naming area.

In the past/current

In our current code, the JNDI naming has been very simple:

We concatenated "ejb/" with the name of the remote Session Bean interface in the deployment descriptor to indicate the name that the service should be bound to.
In the code, we use the following code to handle the JNDI lookup. The really nice part was that the same code could be used by remote clients AND on the server, within the container.

Current Code

Hashtable properties = new Hashtable();
properties.put("java.naming.factory.initial","org.jnp.interfaces.NamingContextFactory");
properties.put("java.naming.factory.url.pkgs","org.jboss.naming:org.jnp.interfaces");
properties.put("java.naming.provider.url", "jnp://localhost:1099");
Context ctx = new InitialContext(properties);
Object ref = ctx.lookup(jndiName);

AS7 and EJB 3.1
The EJB 3.1 spec has made some changes to mandate portable JNDI names for EJBs and you inherit this in AS 7.1.1. The other tricky thing I found was that I could no longer use the exact same lookup code from our remote clients and the server.

AS 7 now has two options for remote EJB invocation. The information you need can be found in the JBoss docs, it just didn't hit me over the head! After struggling with this issue for a couple days, I decided to create a small program to help make the differences (hopefully) very clear. Below is the program that I created to invoke a stateless session bean from a remote client. I deployed the "ejb-remote" sample from the 7.1 Quick Start samples. The code attempts to load the remote service using both remote methods. The expectation is that for the first set of lookups, the first lookup is successful and the second, using the "ejb:/" naming format fails. Then I add the Context.URL_PKG_PREFIXES property with a value of "org.jboss.ejb.client.naming" to the jndi properties passed to the InitialContext constructor and repeat the lookups. Now, both lookups should be successful. I have included all the JNDI properties in the code rather than relying on a copy of "jboss-ejb-client.properties" or "jndi.properties" being picked up from the classpath.

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

import java.util.Hashtable;

public class EJBClient {

     private static String[] JNDINAME = {
          "jboss-as-ejb-remote-app/CalculatorBean!org.jboss.as.quickstarts.ejb.remote.stateless.RemoteCalculator", 
          "ejb:/jboss-as-ejb-remote-app/CalculatorBean!org.jboss.as.quickstarts.ejb.remote.stateless.RemoteCalculator" 
     };
     
     private Hashtable jndiProps;
     
     public EJBClient() {
       // setup 'base' jndi properties - no jboss-ejb-client.properties being picked up from classpath!
       jndiProps = new Hashtable();
          jndiProps.put("java.naming.factory.initial","org.jboss.naming.remote.client.InitialContextFactory");
          jndiProps.put(InitialContext.PROVIDER_URL, "remote://localhost:4447");
          jndiProps.put("jboss.naming.client.ejb.context", true);
        
          // needed for remote access - remember to run add-user.bat
          jndiProps.put(Context.SECURITY_PRINCIPAL, "client");
          jndiProps.put(Context.SECURITY_CREDENTIALS, "password");
     }
     
    public void doLookups() {
         // the 'exported' namespace
         for (int i = 0; i < JNDINAME.length; i++) {
               lookup(JNDINAME[i]);
          }
         
        
         // This is an important property to set if you want to do EJB invocations via the remote-naming project
        jndiProps.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
        
        // now with the ejb
        for (int i = 0; i < JNDINAME.length; i++) {
               lookup(JNDINAME[i]);
          }
    }
    
    private void lookup(String name) {
         System.out.println("Lookup name="+name);
         
         Context ctx = null;
        try {
               ctx = new InitialContext(jndiProps);
               Object ref = ctx.lookup(name);
               System.out.println("...Successful");
          } catch (NamingException e) {
               System.out.println("...Failed");
               //System.out.println(e.getMessage());
               e.printStackTrace();
          } finally {
               if (ctx != null) {
                    try {
                         ctx.close();
                    } catch (NamingException e) {}
               }
          }
    }
    
    public static void main(String[] args) throws Exception {
         EJBClient client = new EJBClient();
         client.doLookups();
         
         System.out.println("Done!");
    }
    
}

AS7 on the server-side
Now the easy part, doing the JNDI lookups on the server are very similar to the old way, except that you still need to format the JNDI name according to the new specs and the naming factory is no longer the jnp version!

Hashtable jndiProps = new Hashtable();
    jndiProps.put("java.naming.factory.initial", "org.jboss.as.naming.InitialContextFactory");
    ctx = new InitialContext(jndiProps);
    Object ref = ctx.lookup(jndiName);

Conclusion
As I said earlier, the information was/is in the JBoss documentation, but I probably read past it several times. What I was looking for was an example showing loading the EJBs from both the client and the server.

Hope this helps!

Ajax Examples upgraded to Grails 2.0.0

2012-03-25T21:59:00.000-05:00

Overview
A little over a year ago, I wrote a blog posting providing a complete set of working code showing ajax examples for each of the Grails tags that supports Ajax. Now, I finally found time to upgrade that code to Grails 2.0.0 (from Grails 1.3.7). There are plenty of article out there helping point users in the right direction when upgrading to Grails 2.0.0 and most of them are helpful. Seems like no one wants to RTFM, we just want to get at it! My recommendation is RTFM, and checkout some of the helpful postings out there, especially when the upgrade includes as many changes and new features as Grails 2.0.0.

I have to confess, I tried to just "get at it" with my ajax application and Grails 2.0.0 and it got me part of the way there. To finish the job, I needed to read the excellent documentation produced by the Grails team and also the documentation accompanying some of the plugins.

Steps for my upgrade
In order to get my application running under Grails 2.0, I needed 5 basic steps:

Run the 'grails upgrade' against my project.
Upgrade database to use H2 - instructions under Persistence here
Install plugins: prototype and resources plugin.
Modify the main.gsp layout to accommodate the plugins (resource, prototype).
Modify Config.groovy to set the java script library to prototype.

Step 1

This step is required for any application being upgraded from an earlier version of Grails. I am not going to say anything else about this step.

Step 2
Follow Peter Ledbrook's instructions on upgrading the persistence layer to use H2 database instead of the HSQLDB. If you want to continue to use HSQLDB, there are instructions for that too!

Step 3

I consider myself 'javascript-challenged', so I decided to stick with the prototype library for javascript for this upgrade rather than attempting to use jQuery, which is the new default library. If all goes well and I get some free time, maybe I will upgrade the application to use jQuery too, but for now, I am sticking with prototype - less changes too!

Make sure the plugins are installed, and then verify that they all got installed successfully (last statement).

grails install-plugin resources
grails install-plugin prototype
grails list-plugins -installed

Step 4
One of the tricky parts here was including plugin="prototype" on the javascript tag in the header section of this page. Below is a copy of my layout\main.gsp. (Be careful using cut/paste on the page below, I had problems with the syntax highligher or blogger (not sure which) adjusting my html tags as it sees fit). Best option, grab the code from GitHub. Also be sure to understand the changes needed for the Resources Plugin.

  
 
        <g:layoutTitle default="Grails" />
        
        
        
        
        
           function showSpinner(visible) {
              $('spinner').style.display = visible ? "inline" : "none";
           }
           Ajax.Responders.register({
           onLoading: function() {
                 showSpinner(true);
           },
           onComplete: function() {
             if(!Ajax.activeRequestCount) showSpinner(false);
           }
           });

Step 5
Update Config.groovy to define prototype as the javascript library to use.

... 
grails.views.javascript.library = "prototype"
...

The project has been updated and now available on GitHub.

Hope this helps!

Book Review: Programming Concurrency on the JVM

2012-01-30T21:02:00.002-06:00

Overview
Programming concurrency is a tough task to get right, just ask Venkat, he'll tell you so! I attended a presentation by Dr. Venkat Subramaniam on this very subject, Concurrency on the JVM, at my local Java user group a couple months ago and he really caught my attention. The presentation was a condensed version of the book, and needless to say, I bought the book to help fill in all the details.

Not only is Venkat a 'subject matter expert', but his presentation kept everyone engaged and laughing!

Contents:
The book is broken down in 5 parts, with 10 chapters total, at just over 250 pages. The main sections of the book are:

Strategies for Concurrency - this discusses the impact of multi-core machines and the three design approaches that are discussed throughout the rest of the book, Shared Mutability, Isolated Mutability and Pure Immutability.

Modern Java/JDK Concurrency - covers the Java 1.5+ concurrency features and talks about how to use the newer concurrency classes, like ExecutorService, CountDownLatch, Locks and Fork/Join to solve concurrency problems.

Software Transactional Memory - covers Software Transactional Memory (STM), popularized by Clojure, which places access to memory within transactions. STM is one option for getting away from the 'Synchronize and Suffer' model as Venkat call it. The intention is to make threaded code more deterministic. Using the JDK concurrency tools, there is no way to be sure you code is correct because it doesn't always fail when/where you need it to. By putting the memory access inside transactions, the transaction manager helps resolve the conflicts without explicit locking.

Actor-based Concurrency - covers actor design approaches as another option for concurrency design without using synchronization. This section exercises actors from the Akka library from Scala, Actors from the Groovy library GPars, and mixing Actors and STM as a possible solution.

Epilogue - is a short recap of the topics presented previously and points out the scenarios when each of the solutions would be appropriate.

The book lays out a couple example problems and then develops solutions for these problems across a range of JVM languages (Java, Groovy, Scala, Clojure and JRuby). The examples are short and concise, enough to help you to get started prototyping against your own concurrency issues.

Summary
This is a good book to get you started on some of these newer concurrency concepts and libraries. It was an easy read, and I completed the reading fairly quickly, as opposed to the "Java Concurrency in Practice" book, which I started and stopped twice and still have not completed yet!

I would recommend this book to anyone looking to improve their programming skills in the concurrency area. The book provides you with some alternatives to the old standby of ' Synchronize and Suffer' model.
My last recommendation, if you get the opportunity, attend one of Venkat's presentations, you won't be sorry!

Passing parameters into Groovy script using Binding class

2012-01-19T22:03:00.001-06:00

I recently saw a question posted on the Groovy/Grails group on LinkedIn asking about ways to pass in parameters to a Groovy script. There were several responses pointing to the CliBuilder class which is certainly one way to handle the problem. I had just finished reading an article by Ken Kousen in the November issue of GroovyMag where Ken mentioned another option: using the Binding class.

The example below shows a couple ways of setting variables in the binding, getting the variable values from the binding and how to capture standard output from the script. The one tricky part is the question "When is something in the Binding and when not?" The answer is: when it's not defined, it is in the binding! In the example below, variable c is placed in the binding, but since it is def'd in the script, the value for that variable comes from the local variable rather than the binding.

// setup binding
def binding = new Binding()
binding.a = 1
binding.setVariable('b', 2)
binding.c = 3
println binding.variables

// setup to capture standard out
def content = new StringWriter()
binding.out = new PrintWriter(content)

// evaluate the script
def ret = new GroovyShell(binding).evaluate('''
def c = 9
println 'a='+a
println 'b='+b
println 'c='+c 
retVal = a+b+c
a=3
b=2
c=1
''')

// validate the values
assert binding.a == 3
assert binding.getVariable('b') == 2
assert binding.c == 3 // binding does NOT apply to def'd variable
assert binding.retVal == 12 // local def of c applied NOT the binding!

println 'retVal='+binding.retVal
println binding.variables
println content.toString()

Output

[a:1, b:2, c:3]
retVal=12
[a:3, b:2, c:3, out:java.io.PrintWriter@1e0799a, retVal:12]
a=1
b=2
c=9

Hope this helps!

Book Review: SQL Antipatterns Avoiding the Pitfalls of Database Programming

2012-01-17T22:30:00.001-06:00

Overview
This is book for software developers that are either new to database programming (SQL) or those that may not have had any formal database education and may have learned 'on-the-job'.

The book is bit longer, approximately 300 pages, than some of the other titles I've read from The Pragmatic Programmers shelf, but it is a very easy read. The chapters all follow a similar format: Object, Antipattern, How to Recognize the Antipattern, Legitimate Uses of the Antipattern and Solution. Each chapter the author leads you through a problem that needs correcting within the scope of the database. Next the antipattern is laid out for you and the author shows the disadvantages of using the antipattern. Tips are provided to help identify the antipattern and he also suggests some possible legitimate uses of the antipattern. Finally, the author describes better solutions to the problem presented; ones without the disadvantages discussed previously and solutions that are considered 'best practices' and that will provide better database performance.

Contents
The book is broken down into 4 main sections:

Logical Database Design Antipatterns - planning database tables, columns and relationships
Physical Database Design Antipatterns - defining tables, indexes and choosing datatypes
Query Antipatterns - SQL command usage, for example, SELECT, UPDATE and DELETE
Application Development Antipatterns - correct usage within the scope of a language

Some of that antipatterns described may make you laugh and wonder, "Who the heck would do something like that?" while others you may have run into previously and some might even hit very close to home from your work or personal projects.

There is also an appendix that covers the Rules of Normalization. I had heard of terms like Normalization, First Normal Form, and Second Normal Form before, but never actually read any explanations. The appendix includes discussions on normalization and first normal form through fifth normal form with examples to help clarify the topics.

Summary
This is a very good book for software developers just starting to 'cut their teeth' with databases and SQL. It also serves as a good refresher for those with a bit more database experience. In my case, I fall into the later scenario. You can view it as a book of short stories, showing what not to do in these cases.

Jars signed with mutliple code signing certificates?

2011-10-26T13:27:00.001-05:00

For anyone that has built and maintained Web Start applications, you have probably been through this issue before. Your QA group or worse yet, a customer, calls to tell you that they cannot start your application because the download failed because "jar resources in jnlp are not signed by the same certificate".

If want your Web Start application to have full access permissions, then you need to sign all the jars that get downloaded. This generally presents the opportunity to encounter one of the following errors:

None of the jars are signed. Most likely a build issue and generally only happens once, (I hope!)
Single jar is not signed. Again, most likely either a build or process error.
Not all jars signed with the same certificate.

The first two are easy to resolve. The last is a bit of a pain because Web Start doesn't bother to tell you exactly which jars where signed with different code signing certificates. Now, it's time to make some educated guesses as to what changed recently and look at the most likely/usual suspects. Another option is to get the list of download jars from the JNLP document and figure out which certificates(s) each jar has been signed with.

We have been through this at work once or twice and so I finally decided to make it easier for the next time this happens. The approach I took was to read all the jars in a deployment folder, open them and look for the signature files inside the jar file. For the results, I created a map keyed by the jar name and the value being a list of the signature file names (*.RSA, where the * represents our code signing certificate alias). Keeping a list in the map allows for cases where a jar may have multiple signature files, which was encountered when we switched over code signing certificates. Writing this in Java would be possible, but why write so much code? Why not write a script in Groovy, it would be some much shorter and concise.

Groovy Script

def dir = new File("C:\\MyDeployment\\jboss\\myserver\\deploy\\myserver.ear")
def jars = dir.list( { d, f-> f ==~ /.*.jar/ } as FilenameFilter)
def map = [:]
jars.each() {
   def zipFile = new java.util.zip.ZipFile(new File(dir, it)) 
   zipFile.entries().each { zipEntry ->
      if (zipEntry.name.endsWith('.RSA')) {
          if (map.containsKey(it)) {
             def list = map[it]
             list << zipEntry.name
          } else {
             map[it] = [zipEntry.name]
          }
      }
   }  
}
map.each{ println it}

Hope this helps!

Lessons learned from recent PA/DSS audit

2011-09-29T16:11:00.000-05:00

The following are some tips/hints and lessons learned during our recent PA/DSS audit. Everyone's mileage will vary.

Our application runs on both Windows and Linux. We support JBoss and Weblogic for the application server. We support Oracle, SQL Server and MySQL for the database. We have three client applications which are all WebStart clients, so this is NOT a web application. Our EJBs are EJB 2.1 Session Beans.

Hints

Determine exact software stacks needed including all software levels, build levels and whatever else you need. Collect all of this in one safe place.
Get the hardware ready and any scripts that may be needed to use those devices. Our application is a POS, so we had a collection of pin pads that we were required to test with during the audit.
Get any utility software needed

Ghost for Win* images (or Win 7 provides a Backup/Restore feature)

We ran into some problems with Windows Server 2008 and had to use one of the Acronis tools to create the backup image.

PartImage for Linux images
Darik's Boot and Nuke
anything else you might need

Wipe the hard drives before loading any software, including the OS, using Darik's Boot and Nuke to be absolutely sure that the drive does contain anything that might match a credit/debit card number.
Install the OS and any 'base' software, which isn't the software to be verified by the audit

Keep the partition sizes as small as possible without causing disk output problems. The larger the disk, the longer the forensic scans will take to run.
On Windows - you can have a page file, just don't allow it to grow and shrink. Set the min and max file size to the same value.

Problem here is that if card holder data is written to the page file then later the page file size is shrunk, some of that card holder data MAY be inside freespace, which is NOT cleaned up by the OS. This can cause some instances of card holder data to be found during a forensic scan and NO ONE wants card holder data found during the scans.

On Window - you should also set the Windows Update setting to be Notification because you don't want Windows Updates being applied to a machine during the audit.
Our auditor also recommended we turn off the Windows Restore points. Again, you don't want Windows doing any extra work that could impact the audit results.

Take images of all boxes. These may be needed to reset a machine back to a 'clean' state for ad additional set of tests.
Talk to each of your authorization providers

let them know when your audit is scheduled and ensure you will have connectivity to their test servers
Verify that each provider can supply 'magic authorization values' that will help your trigger the following responses:

approvals
declines
referrals
timeouts
void
split tenders
partial auths

If any of the authorization providers encrypt their communications, then you may need to ensure you have whatever is needed for that. We needed to download and install the JCE .
If you support multiple authorization providers, you will have to run tests against each provider so be ready to do whatever it takes to switch your application from one provider to the next.

Practice taking images and restoring images on all boxes

The Linux tools are not as user friendly as the Windows tools, so you need to be ready and able to make and restore your images in a timely manner.

Now you can install your software and you should be ready for the audit.
If you do any testing before the auditor arrives, be ready to restore the machines back to a 'clean' state because you don't want to take a chance that your early testing wrote any card holder data somewhere on the hard drive.
Our auditor previously used WinHex to do the forensic scans. We used the evaluation version to try and double-check ourselves. The problem with that version is that it is slow and can only scan for one card number at a time. This time the auditor had a new set of software for the scans and unfortunately I didn't get the name of the package, although I know it was a purchased product.

Good luck - hope this helps!

Book Review: Building and Testing with Gradle

2011-07-20T12:56:00.002-05:00

Overview

Gradle is one of the new "cool kids on the block". It is quickly gaining momentum as projects like Hibernate and Spring migrate their build processes over to Gradle. Gradle can be considered the 3rd generation of Java build tools, with Ant and Maven representing the earlier generations.

Gradle is built on top of Groovy, which provides the ability to define a Domain Specific Language (DSL). In this case, the Gradle DSL provides a language tailored to the task of building code. If the DSL does not provide exactly what you need, then you can extend the DSL using plug-ins. It is possible to learn Gradle without knowing Groovy, but some initial knowledge of Groovy is beneficial.

This book is short (89 pages) . It brings back memories of the O'Reilly "A Developer's Notebook" series.

Contents

Chapter 1, Hello Gradle This chapter leads the reader thru the installation and configuration of Gradle and demonstrates the ubiquitous Hello World (build file) example.

Chapter 2, Gradle Tasks This is the best chapter in the book. It drives home the concepts associated with a 'task' and how the tasks can be configured and customized. The most helpful suggestion in the book is found in this chapter. The recommendation is that you read the Gradle DSL documentation. I had previously read the Gradle Users Guide but was still a bit fuzzy on what the properties and methods were and how they were used. . Reading the DSL documentation provides the full picture. It also helps drive home that you are working with a programming language/DSL and not a set of XML tags like Ant and Maven.

Chapter 3, Ant and Gradle This chapter compares and contrasts Ant and Gradle. It also demonstrates that Gradle can run Ant tasks by importing an Ant build.xml or by using the Ant Builder that is part of the Groovy distribution.

Chapter 4, Maven and Gradle This is the longest chapter in the book comprising about 25% of the book. Like all of the Gradle documentation, this book re-assures the reader that any previous investment in Maven or Ivy will not be lost in a move to Gradle. Gradle provides support for Ivy, Maven's Central repository and a local repository.

Chapter 5, Testing with Gradle This chapter is a good overview of using JUnit , TestNG, Spock, Geb and EasyB. Coverage of testing is a bit light given the title of this book.

Chapter 6, Multiproject Builds This chapter does a nice job of taking a sample project and showing three different approaches to creating the project build structure. The approaches shown are one master build file, project-specific build files and a hybrid approach. While the sample projects are very small, they provide the reader a starting point to converting their own multiproject builds.

Summary

This is an excellent book for getting started with Gradle. There are plenty of working examples provided in the source code download package. The authors do a very good job of explaining the concepts and presenting alternative ways of expressing the same options within the DSL.

One minor issue is that matching up the downloaded source examples to the examples in the printed chapters is a bit of an exercise because they are not organized in a folder structure by chapter like many other books.

Initially the slim size of the book worried me as to it's ability to provide a depth of coverage of Gradle, but the authors stated in the Preface that "Future volumes will cover the Gradle plug-in ecosystem, how to extend Gradle with your own business logic and even more advanced topics." Sounds good to me, bring it on Tim and Matthew!

Gradle dependencies with jars that have no version number

2011-07-13T23:44:00.000-05:00

I have seen presentations and read blogs and articles on Gradle. It sounds pretty good. Gradle is gathering momentum as some of the major players in the open source community (Hibernate and Spring Security to name a few) have switched their builds over to using Gradle. I figured it is time to give it a try.

The project I work on is fairly old, pre-Maven I suspect. The project is a multi-layer, multi-component Java project built using Ant. The project does not use Maven or Ivy. The project includes a folder that contains all of our dependent jars. These jars are committed to SVN and included as part of the project when it is checked out. This way we have all the dependencies we need to build the project. We have quite a few dependent jars that do not have version numbers in the jar name. Examples include a copy of junit.jar and log4j.jar. This all existed before I started working on the project, so I have no idea how it got that way - it just is. And since it's not really broken, we haven't attempted to fix it!

Not having version numbers in the jars is a minor inconvienence. If you read the Gradle User Guide chapter on Dependency Management, they seems to push dependency management for jars with version numbers. Most of the examples shown in the documentation and elsewhere declare the dependencies as seen in Example 34.1 in Dependency Management chapter. Great, but what do I do if I don't have version numbered jars in my dependencies? I did not believe the Gradle guys would leave me hanging... And they didn't!

I really wanted to figure out how to setup my classpath to include these jars without version numbers. I figured out how to do this and started writing this blog. Then as I re-read the Dependency Management chapter, I saw a clue that I must have blown right by the first time I read it. The clue is in Section 34.3.5 File dependencies. You can use the FileTree to help define the classpath to include these non-versioned jars.

I created a small project called GradleTest that did not use the standard Java plugin conventions for the source and resource files and has dependencies in the "libs" folder, none of which have version numbers in the jar names.

I created two different options, which both seem to work. The first is shown below. This option handles the dependencies more specifically, assigning just the jars that are needed for either compile or runtime.

apply plugin: 'java'

sourceCompatibility = 1.6
version = '1.0'

sourceSets {
    main {
        java {
            srcDir 'src'
        }
        resources {
            srcDir 'resources'
        }
        compileClasspath += fileTree(dir: './libs', includes: ['**/log4j.jar'])
        runtimeClasspath = classes + sourceSets.main.classes + fileTree(dir: './libs', includes: ['**/log4j.jar'])
        
    }
    test {
        java {
            srcDir 'test'
        }
        resources {
            srcDir 'testresources'
        }
        compileClasspath += fileTree(dir: './libs', includes: ['**/junit.jar'])
        runtimeClasspath += classes + fileTree(dir: './libs', includes: ['**/junit.jar', '**/log4j.jar']) 
        
    }
}

sourceSets.main.classesDir = new File("./build/classes")
sourceSets.test.classesDir = new File("./build/classes")

The second option looks very similiar to the first option, but uses the dependencies definition and it cheats by assigning all the jars in the "libs" folder to both the compile and runtime classpath, rather than just exactly what is needed.

apply plugin: 'java'

sourceCompatibility = 1.6
version = '1.0'

sourceSets {
    main {
        java {
            srcDir 'src'
        }
        resources {
            srcDir 'resources'
        }
        
    }
    test {
        java {
            srcDir 'test'
        }
        resources {
            srcDir 'testresources'
        }
        
    }
}

sourceSets.main.classesDir = new File("./build/classes")
sourceSets.test.classesDir = new File("./build/classes")

configurations {
    compile {
        description = 'compile classpath'
        transitive = true
    }
    runtime {
        extendsFrom compile
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: '*.jar')
    runtime fileTree(dir: 'libs', include: '*.jar')

}

I am sure there are other, possibly better or cleaner alternatives. Please feel free to share improvements or other alternatives. Hope this helps....

Groovy vs Java difference provides a WTF moment

2011-05-19T22:06:00.001-05:00

We all (should) know that not ALL Java is valid Groovy, but if you don't spend you day working in Groovy, then sometimes you forget these differences. These 'gotchas' are documented here but who reads this stuff, right? :-)

I wanted to get DatabaseMetadata for a database, specifically a list of table names, so I started with a copy of Java code that already did the same thing.

Class.forName(jdbcDriver);
Connection con = DriverManager.getConnection(jdbcURL, "root", "root");
DatabaseMetaData dbmd = con.getMetaData();
String[] tableTypes = { "TABLE" };
ResultSet allTables = dbmd.getTables(null, null, null, tableTypes);
System.out.println("processing tables...");
while (allTables.next()) {
    String table_name = allTables.getString("TABLE_NAME");
    System.out.println("processing table"+table_name);
}

Should be simple, right - drop the semi-colons and look for other opportunities to shorten the code using Groovy.

def db = Sql.newInstance('jdbc:mysql://127.0.0.1:3306/mydb', 'root', 'root', 'com.mysql.jdbc.Driver')
def metaData = db.getConnection().getMetaData()
println("Driver Name: " + metaData.getDriverName())
println("Database Product: " + metaData.getDatabaseProductName())
println("Driver Name: "+ metaData.getDriverName())
println("Driver Version: "+ metaData.getDriverVersion())
String[] tableTypes = { "TABLE" }
ResultSet allTables = metaData.getTables(null, null, null, tableTypes)
while (allTables.next()) {
    String table_name = allTables.getString("TABLE_NAME")
    println "processing table "+table_name
}

And it almost worked! The thing that threw me was that not only does the code compile, but it also returns correct values for some of the methods in the DatabaseMetadata, but when it came time to iterate across the table names, there were none. Come on, I just ran almost the exact code in Java and it spit out all the table names in my database. What the heck did I mess up this time?

My error was the 3rd bullet on the list of gotchas from above - how NOT to initialize an array in Groovy...

String[] tableTypesJava = { "TABLE" }
String[] tableTypesGroovy = [ "TABLE" ]
println tableTypesJava.class
println tableTypesGroovy.class
assertEquals(tableTypesGroovy, tableTypesJava)  // this will fail!

Both variables say they are String[] but tableTypesJava is really closure when compiled in Groovy! The fix to the Groovy code is to use the brackets when initializing the array and not the braces - basically the tableTypesGroovy from above. Guess it's time to RTFM...

Hope this helps!

Grails AJAX Examples

2011-01-31T21:02:00.001-06:00

Early on, newcomers to Grails encounter a lack of working samples accompanying the framework. Good places for examples and documentation include the Grails documentation and bloggers.

Ajax is one of the features with built-in support by Grails. Once people get their "head around" the Grails basics, they generally want to move onto more interesting features like Ajax. This usually requires scouring the web, looking for examples and/or reading the documentation fairly closely. I went through this process and figured that others might benefit from a set of short working ajax examples in Grails. I have created a small application that demonstrates some of the Ajax features and provide the application for download to anyone interested. (see bottom for download information)

The example application uses each of the following:

FormRemote
RemoteLink
SubmitToRemote
RemoteFunction
RemoteField

General Steps

Pick your javascript library - prototype used in my example. Also shown is some Ajax event registration.



     function showSpinner(visible) {
         $('spinner').style.display = visible ? "inline" : "none";
     }
     Ajax.Responders.register({
     onLoading: function() {
           showSpinner(true);
     },
     onComplete: function() {
     if(!Ajax.activeRequestCount) showSpinner(false);
     }
   });

Update your controller with any code needed - below is my entire BookController

package ajax

class BookController {

     def scaffold = true
  
  def showEditAjax = {
   render (view:'editAjax', model:[bookInstance:Book.get(params.id)])
  }
  
  def showListSelect = {
   render (view:'listSelect', model:[bookInstanceList:Book.list()])
  }
  
  def showTitleSearch = {
   render (view:'titleSearchAjax')
  }
  
  def showRemoteLink = {
   render (view:'remoteLink', model:[bookInstanceList:Book.list()])
  }
  
  def showDetails = {
   render(template:'bookDetails', model:[bookInstance:Book.get(params.id)])
  }
  
  def showSearch = {
   render (view:'searchAjax')
  }
  
  def String wrapSearchParm(value) {
   '%'+value+'%'
  }
  
  def searchTitle = {
   def list = Book.findAllByTitleIlike(wrapSearchParm(params.searchvalue))
   render(template:'searchResults', model:[searchresults:list])
  }
  
  def search = {
   def list
   if (params.publisher && params.title)
    list = Book.findAllByTitleIlikeAndPublisherIlike(wrapSearchParm(params.title), wrapSearchParm(params.publisher))
   else if (params.publisher)
    list = Book.findAllByPublisherIlike(wrapSearchParm(params.publisher))
      else if (params.title)
       list = Book.findAllByTitleIlike(wrapSearchParm(params.title))
   
   render(template:'searchResults', model:[searchresults:list])
  }
  
  def listByPublisher = {
   def list
   if (params.filter.equals("All"))
    list = Book.list()
    else
      list = Book.findAllByPublisher(params.filter)
   
   render(template:'searchResults', model:[searchresults:list])
  }
  
 def update = {
  def bookInstance = Book.get( params.id )
  if(bookInstance) {
   bookInstance.properties = params
   if(!bookInstance.hasErrors() && bookInstance.save()) {
    render "
Book ${params.title} updated with Ajax using FormRemote"
   }
   else {
    render(view:'edit',model:[bookInstance:bookInstance])
   }
  }
  else {
   flash.message = "Book not found with id ${params.id}"
   redirect(action:edit,id:params.id)
  }
   
  }
  
}

Create/update view(s) - See screen shots below and/or download the application

Most of the examples are fairly straight forward and should not need a lot of explanation. They may not be great 'real world' examples but the intention is to provide a working example for someone to start from.

FormRemote
The formRemote taglib creates a form tag that uses a remote uri to execute an ajax call. In this case, I have slightly modified the default generated edit.gsp to change the form tag to a formRemote tag and added a div to show the edit results. The screen shot below shows the result of first clicking the Form Remote navigation menu button, changing the number of pages in the 'Grails in Action' book and pressing the Update button at the bottom of the screen.

RemoteLink
The RemoteLink tag creates a link that calls a remote function when clicked. In the example application, I display a list of book titles and the title is a link. Clicking on the title shows the book details at the bottom of the screen.


    ${fieldValue(bean: bookInstance, field: "title")}

SubmitToRemote
The submitToRemote tag creates a button that submits the surrounding form as a remote Ajax call serializing the fields into parameters. In the example application, I created search fields for Title and Publisher. The search button was created using the SubmitToRemote taglib to submit the search parameters using Ajax.


             Title:                        
             Publisher:

RemoteFunction
The example application uses the RemoteFunction taglib to create a remote javascript function that is assigned to the onChange DOM event. When the user selects or changes the value in the Publisher dropdown list, the remote function is called and the bottom half of the screen displays books published by the selected Publisher.

RemoteField
The example application uses the RemoteField taglib to create a dynamic search for Book titles, similar to Google Instant. Search results are returned the screen as you type values in the Title Search field. The controller code in this case, wraps the search parameter with the wild card character (%) to search the database and returns all matches to be displayed.

Title Search:

Problems Encountered

I ran into problems trying to get the spinner to show correctly until I found this entry on StackOverflow that showed how to register the Ajax events. When running locally on the same box, this happens very fast so you really need to keep and eye on the upper right corner of the screen if you want to see the spinner.
I also ran into problems with the RemoteFunction due to syntax problems. Being used to the taglibs, I was setting the update parameter using update="mydiv" when the syntax should have been update:"mydiv". This did not flag as an error. The controller was called, it just never updated the target div!

Downloads

Download a zip from here
Get a copy from Github - this is my first crack at creating a repo at Github, so if I did it wrong, please let me know.

It was fun putting this together. Hope it helps some of you!

Shutterfly Holiday Collection

2010-11-20T22:18:00.000-06:00

We have used Shutterfly to create family albums and mouse pads in the past. I have found the quality of the albums from Shutterfly is great. The album pictures are clear and sharp. Based on our past experiences and quality of products that we’ve received, we’re excited about using Shutterfly for our Christmas cards this year.

I have found the Shutterfly Christmas card selection is tremendous. In particular, my family was pleased at the number of religious photo cards that are available. We particularly like the following photo cards: here and here . The cards in the links above allow us to include four to five pictures of our family so that we don’t have to agonize of picking the one best picture! Our plan is to use one of these photo cards for the dual purpose of spreading the message of the season, as we believe it, and to share pictures of our family with loved ones and friends.

Enjoy the selection from Shutterfly and Happy Holidays!!!

Be careful using GStrings for SQL

2010-10-20T21:52:00.000-05:00

If you are like me, you love the ease with which you can create Groovy scripts to manipulate your databases. Gone are the days of writing utility classes to make these changes, thanks to Groovy and the SQL package. But if you aren't careful, you can run into a couple of strange errors if you use the GStrings.

For the below example, I have a table named LOCATION in my 'test' database that has two columns: location_id and location_name. The example below works fine and you notice that I am not using any variables in the SQL statement

import groovy.sql.Sql

db = Sql.newInstance('jdbc:mysql://127.0.0.1:3306/test', 'root', 'root', 'com.mysql.jdbc.Driver')

def sqlStatement = """select * from location where location_id > 3"""

db.eachRow (sqlStatement){ 
   println it
}

println "Done!"

Next, try making the SQL a bit more dynamic by specifying the table name, column name and key value as variables within the GString and watch what happens.

import groovy.sql.Sql

db = Sql.newInstance('jdbc:mysql://127.0.0.1:3306/test', 'root', 'root', 'com.mysql.jdbc.Driver')

def tableName = 'location'
def columnName = 'location_id'
def key = 3
def sqlStatement = """select * from $tableName where $columnName > $key"""

db.eachRow (sqlStatement){ 
   println it
}

println "Done!"

WARNING: Failed to execute: select * from ? where ? > ? because: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''location' where 'location_id' > 3' at line 1

The quick fix I found was to call the toString() method on the GString and this resolves the problem. I found a line in "Groovy In Action" which helps explain the problem:

The special use of GStrings as SQL statements limits the use of placeholders to places where a question mark would otherwise be allowed in a prepared statement.

import groovy.sql.Sql

db = Sql.newInstance('jdbc:mysql://127.0.0.1:3306/test', 'root', 'root', 'com.mysql.jdbc.Driver')

def tableName = 'location'
def columnName = 'location_id'
def key = 3
def sqlStatement = """select * from $tableName where $columnName > $key""".toString()

db.eachRow (sqlStatement){ 
   println it
}

println "Done!"

Keep this in mind the next time you use the Groovy SQL class!

Get value from SQL into an Ant property

2010-07-30T16:13:00.000-05:00

Recently I had to adjust some of our JMS configuration and needed make sure we set the ServerPeerID configuration value to a unique value among our servers. We have a unique server number in our database so I figured that I would just use that value. In order to do this, I needed to get the database value into an Ant property, which is then used to generate the configuration file, which includes the ServerPeerID. Below is the Ant task and steps I used to do this.

Steps:

Define where the SQL results will be written
Make sure the file does not already exist
SQL task to get the result
Load the file back into a property

Warning: The syntax highlighter appears to be reformatting my XML and not allowing the self closing tags. Probably my problem but you should be able to get the idea from the


   
   
   
      
      SELECT PREF_VALUE FROM PREFERENCE WHERE PREFERENCE_ID = 71;

Groovy Sockets example

2010-04-28T23:31:00.002-05:00

I found several sites and blogs with Groovy socket examples but none that showed a matched pair of the server and the client sending data back and forth. Collecting bits from different sources, I put together the following server and client scripts using Groovy.

The sample works by sending a line of data as a request; thus the server calls readLine() and the client sends a new line character at the end of the input. The server echos the request back to the client along with a timestamp. For those that skip reading the documentation, ServerSocket's accept method "Accepts a connection and passes the resulting Socket to the closure which runs in a new Thread." so this server creates a new thread for each request.

Groovy Socket server

import java.net.ServerSocket
def server = new ServerSocket(4444)

while(true) {
    server.accept { socket ->
        println "processing new connection..." 
        socket.withStreams { input, output ->
            def reader = input.newReader()
            def buffer = reader.readLine()
            println "server received: $buffer"
            now = new Date()
            output << "echo-response($now): " + buffer + "\n"
        }
        println "processing/thread complete."
    }
}

Groovy Socket client

s = new Socket("localhost", 4444);
s.withStreams { input, output ->
  output << "echo testing ...\n"
  buffer = input.newReader().readLine()
  println "response = $buffer"
}

Give it try! Hope this helps...

Trouble connecting to Web Start client?

2010-04-24T14:44:00.000-05:00

Do you have problems getting either the debugger or JConsole attached or connected to your Web Start client? We worked around this for way too long until we finally found the trick: environment variable JAVAWS_VM_ARGS.

JNLP allows for vm args to be passed to the Java runtime but there is a limited set of arguments that are allowed to be passed on. Anything not in that list of secure properties is ignored. You can find the list of secure properties in the Developers Guide.

As I said ealier, if you want to attach a debugger or JConsole to you Web Start application, then you need to use the JAVAWS_VM_ARGS parameter. For example, if I want to attached to my web start client for debugging purposes then I can do the following:

set JAVAWS_VM_ARGS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

in my environment, start the client and then attach to the process using port 5005. The same goes for attaching to JConsole, set the properties you need in JAVAWS_VM_ARGS, so com.sun.management.jmxremote.* should all be set inside JAVAWS_VM_ARGS.

Hope this help!!!

Ant task ported to Groovy - Part 3

2010-01-03T22:28:00.000-06:00

Welcome to the final posting of this series where I have been porting one of my Ant tasks from Java to Groovy. If you haven't read part 1 or part 2, then you might want to take a quick pass through those before reading this posting. In this posting, I will show the following:

FileValidator class ported to Groovy
Using HTML Builder to generate the HTML report
Full listings of the scripts & classes created

FileValidator ported to Groovy
One of the last tasks left to do is to port the FileValidator class from Java over to Groovy. The FileValidator class is created in the process() method of the Validator class and called for every file to be validated. This is another example of where the Groovy code is a lot shorter and more concise than the Java code. The FileValidator class handles the iteration thru a file, line by line, checking each regular expression checking for a potential error condition.

import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Matcher;

/**
 * File validator runs a list of regular expressions against the file
 * attempting to validate the file.  Errors are kept in a list and
 * then the caller can get the list of errors and log them - right
 * now the ant task will log them. 
 *
 */
public class FileValidator {

    private List regExDescriptors;
    private File file;
    private List errors;
    
    /**
     * Default constructor
     * @param list List of RegExDescriptors used to validate a file
     * @param file File to be validated
     */
    public FileValidator(List list, File file) {
        regExDescriptors = new ArrayList(list);
        this.file = file;
        errors = new ArrayList();
    }
    
    public List getErrors() {
        return errors;
    }
    
    public void validate() {
      // see if we should exclude this file  
      processExcludes();
      
      // Run the regEx(s) against the file to check for problems
      BufferedReader in = null;
      try {
          in = new BufferedReader(new FileReader(file));
          String str;
          int lineNumber = 1;
          while ((str = in.readLine()) != null) {
              for (Iterator iter = regExDescriptors.iterator(); iter.hasNext();) {
                  RegExDescriptor validator = (RegExDescriptor) iter.next();
                  Matcher matcher = validator.getMatcher(str);
                  if (matcher.find()) {
                   errors.add( validator.getDescription() + " on line " + lineNumber);
                  }
              }
              lineNumber++;
          }
      } catch (IOException e) {
          e.printStackTrace();
      } finally {
          if (in != null)
              try {
                  in.close();
              } catch (IOException e) {
              }
      }  
    }

    private void processExcludes()
    {
        String fullName;
        try
        {
            fullName = file.getCanonicalPath();
        }
        catch (IOException e)
        {
            fullName = file.getName();
            e.printStackTrace();
        }
        for (Iterator iter = regExDescriptors.iterator(); iter.hasNext();)
        {
            RegExDescriptor descriptor = (RegExDescriptor) iter.next();
            if (descriptor.excludeFile(fullName))
            {
                iter.remove();
            }
        }
    }    
}

Groovy code for file validation

//**************************************
// process each file, line by line
//**************************************
files.each {fileName ->
  def errors = []
  def lines = new File(fileName).readLines()
  def lineNumber = 1
  lines.each {line ->
    // for each line - iterate thru descriptors looking for matches if file not excluded
    descriptorList.each {
      if (!(fileName =~ /$it.exclude/)) {
        if ((line =~ /$it.regex/).find()) {
          errors.add(it.description + " on line " + lineNumber)
          errCnt++
        }
      }
    }
    lineNumber++
  }
  if (errors)
    results.put((fileName), errors)
}

Generating the results report
The last part of the port was to create an html output report similar to a JUnit report. The Java code could have been better but it was simply a large set of write statements. I decided to use the HTML markup builder to generate the output report.

//*******************************************************
// generate the report
//*******************************************************
def writer = new FileWriter("$opt.r")
def html = new groovy.xml.MarkupBuilder(writer)
def rptDate = new Date()

html.html {
  head{
    title 'Validator Report'
  }
  body {
    h1 "File Validation - as of $rptDate"
    h2 "Validation Descriptors for file extension $opt.e"
    table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'80%'){
      tr ('bgcolor':'#a6caf0') {
        th ("Description")
        th ("Validating Regular Expression")
      }
      //iterate thru descriptors - listing the regular expressions used in validation
      descriptorList.each { descriptor ->
        tr ('bgcolor':'#eeeee0') {
          td (descriptor.description)
          td (descriptor.regex)
        }
      }
    }
    // Summary table - just a file & error count
    h2 'Validation Summary'
    table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'80%'){
      tr ('bgcolor':'#a6caf0') {
        th ("Files Processed")
        th ("Error Count")
      }
      tr ('bgcolor':'#eeeee0')  {
        td ("$files.size")
        td ("$errCnt")
      }
    }
    if (errCnt)
    {
      // Error details table
      h2 "Validation Details for extension $opt.e "
      table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'100%'){
        tr ('bgcolor':'#a6caf0') {
          th ("File")
          th ("Errors")
          th ("Details")
        }
        //iterate thru results map key is filename and value is list of error text)
        results.each { k, v ->
            tr ('bgcolor':'#eeeee0')  {
               td (k)
               td (v.size())
               td {
                 ul {
                   //iterate thru the list of errors
                   v.each {
                     li (it)
                   }
                 }
               }
            }
        }
      }
    }
  }
}
writer.toString()

Porting Results
Initial Java Ant task: 3 classes and 326 lines of code
Groovy port: 1 script of 102 lines of code and 1 class with 9 lines of code. There are about 40 lines of processing code and the rest of the code/lines are from the html markup builder for the report.

Shorter and more concise, you decide!

Groovy script and class source
Here's the full Validator.groovy script

/**
 *  Validator script ported from Java Ant task
 */

//********************************************
// handle command line parms - all required
//********************************************
def cli = new CliBuilder(usage: 'groovy Validator -e extension -p propertyFile -d directory -r reportFile')

cli.h(longOpt: 'help', 'usage information')
cli.e(longOpt: 'extension', args: 1, required: true, 'file extension to be validated')
cli.p(longOpt: 'prop', args: 1, required: true, 'property file containing regular expresssion')
cli.d(longOpt: 'directory', args: 1, required: true, 'base directory to start file search')
cli.r(longOpt: 'reportFile', args: 1, required: true, 'output file for validation report')

def opt = cli.parse(args)
if (!opt) return
if (opt.h) cli.usage()

println "Processing $opt.e files \n\tusing $opt.p \n\tfrom directory $opt.d \n\tand generate report output to $opt.r"
println 'extracting files...'

//*******************************************
// get all the files for provided extension
//*******************************************
def files = new FileNameFinder().getFileNames(opt.d, "**/*.$opt.e")
println "processing $files.size files..."

//**************************************************************
// load properties into list of ValidatorDescriptor objects
//**************************************************************
Properties properties = new Properties();
try {
  properties.load(new FileInputStream(opt.p));
} catch (IOException e) {}

def config = new ConfigSlurper().parse(properties)
def descriptorList = []
config."$opt.e".each {
  descriptorList << new ValidatorDescriptor(it.value)
}

def errCnt = 0
def results = [:]

//**************************************
// process each file, line by line
//**************************************
files.each {fileName ->
  def errors = []
  def lines = new File(fileName).readLines()
  def lineNumber = 1
  lines.each {line ->
    // for each line - iterate thru descriptors looking for matches if file not excluded
    descriptorList.each {
      if (!(fileName =~ /$it.exclude/)) {
        if ((line =~ /$it.regex/).find()) {
          errors.add(it.description + " on line " + lineNumber)
          errCnt++
        }
      }
    }
    lineNumber++
  }
  if (errors)
    results.put((fileName), errors)
}

//*******************************************************
// generate the report
//*******************************************************
def writer = new FileWriter("$opt.r")
def html = new groovy.xml.MarkupBuilder(writer)
def rptDate = new Date()

html.html {
  head{
    title 'Validator Report'
  }
  body {
    h1 "File Validation - as of $rptDate"
    h2 "Validation Descriptors for file extension $opt.e"
    table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'80%'){
      tr ('bgcolor':'#a6caf0') {
        th ("Description")
        th ("Validating Regular Expression")
      }
      //iterate thru descriptors - listing the regular expressions used in validation
      descriptorList.each { descriptor ->
        tr ('bgcolor':'#eeeee0') {
          td (descriptor.description)
          td (descriptor.regex)
        }
      }
    }
    // Summary table - just a file & error count
    h2 'Validation Summary'
    table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'80%'){
      tr ('bgcolor':'#a6caf0') {
        th ("Files Processed")
        th ("Error Count")
      }
      tr ('bgcolor':'#eeeee0')  {
        td ("$files.size")
        td ("$errCnt")
      }
    }
    if (errCnt)
    {
      // Error details table
      h2 "Validation Details for extension $opt.e "
      table ('border':0, 'cellpadding':5, 'cellspacing':2, 'width':'100%'){
        tr ('bgcolor':'#a6caf0') {
          th ("File")
          th ("Errors")
          th ("Details")
        }
        //iterate thru results map key is filename and value is list of error text)
        results.each { k, v ->
            tr ('bgcolor':'#eeeee0')  {
               td (k)
               td (v.size())
               td {
                 ul {
                   //iterate thru the list of errors
                   v.each {
                     li (it)
                   }
                 }
               }
            }
        }
      }
    }
  }
}
writer.toString()

println "validation complete - possible error count = $errCnt"

//********************************
// for testing launch browser
//********************************
//"c:/program files/Internet Explorer/iexplore.exe $opt.r".execute()

Here's the one short class - ValidatorDescriptor.groovy

class ValidatorDescriptor {
    String description
    String exclude
    String regex

    String toString() {
       "regEx="+regex + " description=" + description + (exclude ? " exclude("+exclude+")" : "")
    }
}

And finally, my set of properties used for validation:

#################################################################
#
# SQL file regular expressions
#
#################################################################

#
# Find instances of trailing spaces after the semicolon.
# This causes MySQL problems.
#
#
sql.1.regex=;\\s+$
sql.1.description=Trailing spaces after semicolon
sql.1.exclude=SQLServer|Oracle

#
# Instances of lines starting with GO
#
sql.2.regex=^GO
sql.2.description=GO statement


#
# Find spaces before ending semicolon (but not DELIMITER ; which is used for MySQL Stored Procedures)
#
sql.4.regex=^(?!DELIMITER)\\s+;$
sql.4.description=Spaces before ending semicolon
sql.4.exclude=SQLServer|Oracle


#
# Oracle scripts using nvarchar
#
#
sql.5.regex=nvarchar
sql.5.description=Oracle scripts using nvarchar
sql.5.exclude=SQLServer|MySQL|Store Update

#
# MySQL scripts using ntext
#
#
sql.6.regex=\\sntext
sql.6.description=MySQL scripts using ntext data type
sql.6.exclude=SQLServer|Oracle

Summary
There you go - another itch successfully scratched! Maybe this task/script will be of help to others. The design was to make this a generic file validation utility and the SQL validation is the only use we have found so far. Enjoy!

Ant task ported to Groovy - Part 2

2009-12-21T23:44:00.004-06:00

Welcome back if you read part 1 of this posting. If you haven't read part 1, you may want to take a quick look at that before reading this part of the posting. This posting (part 2) will show the following changes or ports from Java to Groovy for the validator ant task.

CliBuilder for command line interface
ValidatorDescriptor class
Consuming the properties file

Switch from Ant to a Groovy task
As I mentioned in part 1, this port will not be an exact port because I am creating a groovy script rather than an Ant/Gant task - partly because I haven't played with Gant yet! Some of the existing Ant task code is shown below. For those familiar with creating Ant tasks, you will notice the class extends Task which is an abstract class. The code overrides the execute() method. The validator task checks the input and output parameters and then calls the validate() method to validate the files.

import java.io.File;
import java.io.FileFilter;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.regex.PatternSyntaxException;

import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Task;

public class Validator extends Task {

    private static final String DEFAULT_EXCLUDE = "";
    
    // constants used to create the property file keys 
    // --> extension..regex or extension..description 
    private static final String REGEX       = ".regex";
    private static final String DESCRIPTION = ".description";
    private static final String EXCLUDE     = ".exclude";
    
    private String extension;
    private String directory;
    private String propertyFile;
    private String reportFile;
    
    private int fileCnt;
    private int errorCnt;
    private List descriptors;
    private Properties props;
    private Map errorMap;

    /**
     * Default constructor
     */
    public Validator() {
        props = new Properties();
        descriptors = new ArrayList();
        errorMap = new HashMap();
        errorCnt = 0;
        fileCnt = 0;
    }

    /**
     * Sets the extension for the task
     * @param extension extension of the files to be validated
     */
    public void setExtension(String extension) {
        this.extension = extension;
    }
    
    /**
     * Sets the root directory to be processed.  This directory
     * and all of its subdirectories will be processed.
     * @param directory directory to be processed.  
     */
    public void setDirectory(String directory) {
        this.directory = directory;
    }
    
    /**
     * Sets the root directory to be processed.  This directory
     * and all of its subdirectories will be processed.
     * @param directory directory to be processed.  
     */
    public void setPropertyFile(String fileName) {
        this.propertyFile = fileName;
    }

    /**
     * Set the report file to be created as output
     * @param reportFile name of the html file to be
     *   generated to show any errors
     */    
    public void setReportFile(String reportFile) {
  this.reportFile = reportFile;
 }
    
 /**
     * Creates an xml file from the bugs file
     */
    public void execute() throws BuildException {
        checkParameters();
        try {
            validate();
        } catch (BuildException e) {
            throw e;
        }
    }

    /**
     * Check that all required attributes have been set
     * @throws BuildException if any of the required attributes
     *      have not been set.
     */
    private void checkParameters() {
           if (extension == null || extension.length() == 0  
                ||   directory == null || directory.length() == 0 
                ||   reportFile == null || reportFile.length() == 0 
                || propertyFile == null || propertyFile.length() == 0) 
               throw new BuildException("Extension, directory, reportFile and property file attributes "
                      + "must be defined for task <" + getTaskName() + "/>" );
    }

    private void validate() {
        File dir = new File(directory);
        log("Processing file extension : "+extension+"\n");
        
        try {
            props.load(new FileInputStream(propertyFile));
                      
            setDescriptors();
            visitAllDirsAndFiles(dir);
            
            writeReport();
            
            log("Validator processed "+fileCnt+ " files and found "+errorCnt+" possible errors!");
            
        } catch (Exception e) {
            log(e.toString());
        }
    }
   private void writeReport() {
     try {
  FileWriter writer = new FileWriter(reportFile);
  writeHeaders(writer);
  writeContents(writer);
  writeTrailer(writer);
  writer.close();
 } catch (IOException e) {
  throw new BuildException(e);
 }
   }

   // some code omitted here - but shown later!!!
}

Groovy replacement using CliBuilder
While not a whole lot shorter than the Ant equivalent, it is short, concise and a good example of CliBuilder usage.

def cli = new CliBuilder (usage: 'groovy Validator -e extension -p propertyFile -d directory -r reportFile')

cli.h(longOpt: 'help', 'usage information')
cli.e(longOpt:'extension',  args:1, required:true, 'file extension to be validated')
cli.p(longOpt:'prop',       args:1, required:true, 'property file containing regular expresssion')
cli.d(longOpt:'directory',  args:1, required:true, 'base directory to start file search')
cli.r(longOpt:'reportFile', args:1, required:true, 'output file for validation report')

def opt = cli.parse(args)
if (!opt) return
if (opt.h) cli.usage()

println "start processing $opt.e files using $opt.p from $opt.d and output to $opt.r"

RegExDescriptor becomes ValidatorDescriptor
First the java...

import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.regex.PatternSyntaxException;

/**
 * Regular Expression Descriptor used to help
 * validate source files.  
 *
 */
public class RegExDescriptor {

    private String description;
    private Pattern excludePattern;
    private Pattern pattern;

    public RegExDescriptor(String regExString, String description, String exclude) throws PatternSyntaxException {        this.description = description;
        pattern = Pattern.compile(regExString, Pattern.CASE_INSENSITIVE);
        if (exclude != null && exclude.length() > 0)
            excludePattern = Pattern.compile(exclude);
        else
            excludePattern = null;
    }

    public Matcher getMatcher(String line) {
        return pattern.matcher(line);
    }

    public String getDescription() {
        return description;
    }
    
    public boolean excludeFile(String name) {
        if (excludePattern == null) 
            return false;
        Matcher matcher = excludePattern.matcher(name);
        return matcher.find();
    }

    public String getPattern()  {
     return pattern.pattern();
    }
    
    public String toString() {
        StringBuffer sb = new StringBuffer();
        sb.append("Validator description=").append(description);
        sb.append(" regEx=").append(pattern.pattern());
        if (excludePattern != null)
            sb.append(" exclude regEx=").append(excludePattern.pattern());
        return sb.toString();
    }
}

Now the Groovy equivalent, which I renamed to ValidatorDescriptor. Like a lot of Java classes ported to Groovy, it is significantly shorter since we don't need to specify the getters and setters. The toString() method is really needed but I used it in testing to display the ValidatorDescriptor classes created from the properties file.

class ValidatorDescriptor {
    String description
    String exclude
    String regex

    String toString() {
       "regEx="+regex + " description=" + description + (exclude ? " exclude("+exclude+")" : "")
    }
}

Consuming the properties file
The next task is to read the properties file and create a list of objects representing each of the errors I am attempting to capture. If you remember from part 1 of the posting, there can be three properties: regex, description and exclude. The java processing to read the file and create the list of RegExDescriptors was located in two steps: the first was to read the file into a Properties object and then call the setDescriptors method in the Validator class as part of the main method called by Ant.

The idea is to group the properties together by the extension and the number that is part of the key. Each matching set is used to construct the RegExDescriptor. The RegExDescriptor creates a Pattern for the regular expression checking for the error and another for the file name exclusion check. The patterns are compiled at construction time.

props.load(new FileInputStream(propertyFile));
setDescriptors();

// later in Validator
/*
     * Setup the regular expression descriptors 
     * using the extension to determine which properties
     * are to be used when validating the files.
     */
    private void setDescriptors()
    {
        // strip the leading . and add one to end
        String prefix = extension;
        char c = extension.charAt(0);
        if (c == '.')
            prefix = prefix.substring(1);
        prefix = prefix + ".";
        
        int idx = 1;
        boolean done = false;
        
        // build list of validators using the extension to create the property keys
        while(!done) {
            String regexKey = prefix + idx + REGEX;
            if (props.containsKey(regexKey)) {
                String descriptionKey = prefix + idx + DESCRIPTION;
                String excludeKey = prefix + idx + EXCLUDE;
                try {
                    idx++;
                    String exclude = props.getProperty(excludeKey, DEFAULT_EXCLUDE);
                    descriptors.add( new RegExDescriptor(props.getProperty(regexKey),
                                                       props.getProperty(descriptionKey),
                                                       exclude) ); 
                } catch (PatternSyntaxException e) {
                    StringBuffer sb = new StringBuffer(100);
                    sb.append("Exception compiling regular expression key(").append(regexKey);
                    sb.append(") :").append(props.getProperty(regexKey)).append("\n").append(e);
                    sb.append("\nSkipping this expression...\n");
                    log(sb.toString());
                }
            } else {
                done = true;
            }
        }
    }

Now for the Groovy implementation. It starts out the same, loading a Properties object. Next, we use the ConfigSlurper class to group and parse the properties. An empty list to hold the descriptors is defined and then we iterate across the extension (sql in my case) keys and build the new ValidatorDescriptor objects. Notice the use of the GString aware string as part of the key for calling the each closure.

Properties properties = new Properties();
  try {
     properties.load(new FileInputStream('validator.properties'));
  } catch (IOException e) {}

  def config = new ConfigSlurper().parse(properties)
  def descriptorList = []
  config."$extension".each {
     descriptorList << new ValidatorDescriptor(it.value)
  }

In the next (and last) posting in this series, we will cover the following:

Replace the FileValidator class
Using the HTML Builder to generate a report
Complete listing of the script and classes