<![CDATA[Christian Oestreich]]> 2013-12-13T20:45:33+00:00 http://www.christianoestreich.com/ Octopress <![CDATA[Knockout Validation Rules Engine]]> 2013-12-13T10:00:00+00:00 http://www.christianoestreich.com/2013/12/knockout-validation-rules-engine We have recently begun working with Knockout.js and needed a way to run validation on the models. We found the amazing plugin Knockout.Validation. This plugin did an awesome job and we were able to port over all our jQuery Validation rules fairly easy. As I was working with the plugin, my model grew and additional models were being created it became tedious having to append every rule to every model property and not remember which rules applied to which model properties across model contexts. Thus the concept of the Knockout Validation Rule Engine was born. A working example can be found via github pages

Getting Started

Download the latest knockout-rule-engine file.

Define a rule set that uses the parent key as the name of the model property you want to map to. If you wanted to set an email rule for a model with a property of userEmail, you would provide the following rule set.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

define(['knockout', 'knockout-rule-engine'], function (ko, RuleEngine) {
    var ruleSet = {
        userEmail: { email: true, required: true }
    };

    var ruleEngine = new RuleEngine(ruleSet);

    var model = {
        userEmail: ko.observable('')
    };

    ruleEngine.apply(model);

    ko.applyBindings(model, $('html')[0]);
});

This would be equivalent to the following code.

1
2
3
4
5
6
7

define(['knockout'], function (ko) {
    var model = {
        userEmail: ko.observable('').extend({email: true, required: true});
    };

    ko.applyBindings(model, $('html')[0]);
});

Override Knockout Validation Options

You can pass in the options you want to use for the knockout.validation library as the optional second param in the constructor. For example if you wanted to disable the validation plugin from auto inserting messages you would use the following.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

define(['knockout', 'knockout-rule-engine'], function (ko, RuleEngine) {
    var ruleSet = {
        userEmail: { email: true, required: true }
    };

    var ruleEngine = new RuleEngine(ruleSet, {insertMessages: false});

    var model = {
        userEmail: ko.observable('')
    };

    ruleEngine.apply(model);

    ko.applyBindings(model, $('html')[0]);
});

See Configuration Options for details on all the Knockout.Validation options.

Deep Mapping

By default the plugin will attempt to recurse your model tree and look at all properties and try and match rules against them. If you only want to apply rules to the first level object simply pass a flag with deep set to false in the options param.

1
2
3
4

define(["knockout", "knockout-rule-engine", "rules/address/rules"], function (ko, RuleEngine, personRules) {
    var ruleEngine = new RuleEngine(personRules, {deep: false});
    ... do work ...
});

Reusing rules

If you store your rules in a common directory and include them via require into your models you will ensure you have a common experience across your site. See main.js for more detailed examples.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

define(['filters/filters'], function (filters) {
    return {
        address1: {
            required: true,
            noSpecialChars: true,
            filter: [filters.noSpecialChars, filters.ltrim]
        },
        address2: {
            noSpecialChars: true,
            filter: [filters.noSpecialChars, filters.ltrim]
        },
        city: {
            required: true,
            noSpecialChars: true,
            filter: [filters.noSpecialChars, filters.ltrim]
        },
        state: {
            validSelectValue: {
                message: 'Please select a state.'
            }
        },
        zipCode: {
            required: true,
            validDigitLength: {
                params: 5,
                message: 'Please enter a valid zip code (XXXXX).'
            },
            filter: filters.onlyDigits
        },
        phone: {
            required: true,
            pattern: {
                message: 'Invalid phone number. (XXX-XXX-XXXX)',
                params: /^\D?(\d{3})\D?\D?(\d{3})\D?(\d{4})$/
            }
        }
    };
});

Then you can include this module named rules/address/rules.js into any model that has address or nested address properties that match the keys above (address1, address2, etc).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

define(["knockout", "knockout-rule-engine", "rules/address/rules"], function (ko, RuleEngine, personRules) {

    // set deep to false if you do not want to traverse child properties on the model
    // var ruleEngine = new RuleEngine(personRules, {deep: false});
    var ruleEngine = new RuleEngine(personRules);

    var PhoneModel = function () {
        return {
            phone: ko.observable('')
        };
    };

    var AddressModel = function () {
        return {
            address1: ko.observable(''),
            address2: ko.observable(''),
            city: ko.observable(''),
            state: ko.observable(''),
            zipCode: ko.observable(''),
            phone: new PhoneModel()
        };
    };

    var personModel = {
        firstName: ko.observable(''),
        lastName: ko.observable(''),
        middleName: ko.observable(''),
        address: new AddressModel()
    };

    // example of wiring a field at apply time
    ruleEngine.apply(personModel);

    ko.applyBindings(personModel, $('html')[0]);
});

Adding Validation Rules At Runtime

If you have already instantiated the RuleEngine and need to add a rule later at runtime you can do so via the addRule method.

1
2
3
4
5
6

ruleEngine.addRule('nameNotTom', {
    validator: function (val) {
        return val !== 'Tom';
    },
    message: 'Your name can not be Tom!'
});

Adding Rule Sets At Runtime

You can add additional rule sets to your model via the following code.

1

ruleEngine.addRuleSet('firstName', { nameNotTom: true });

This is extremely handy if you make use of the onlyIf clause in knockout.validation that depends on other model data. You can add these rules later and not have to inject your model into your rule definitions and keep the them clean.

1
2
3
4
5
6
7
8
9
10
11
12
13

var model = {
    firstName: ko.observable('');
    foo: ko.observable('');
}

//do other work

ruleEngine.addRuleSet('firstName', {
    nameNotTom: true,
    onlyIf: function(){
        return model.foo() === 'bar';
    }
});

Using The Filter Extender

It is pretty common that you must also filter the input of data on the knockout model via a form. This is an example filter extender that can be used in conjunction with the rules definitions as in the above example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

ko.extenders.filter = function (target, filter) {
    var writeFilter = function (newValue) {
        var newValueAdjusted = (typeof filter === 'function') ? filter(newValue) : newValue;
        if ($.isArray(filter)) {
            $.each(filter, function (o) {
                if (typeof o === 'function') {
                    newValueAdjusted = o(newValueAdjusted);
                }
            });
        }
        var currentValue = target();
        if (newValueAdjusted !== currentValue) {
            target(newValueAdjusted);
        } else {
            if (newValue !== currentValue) {
                target.notifySubscribers(newValueAdjusted);
            }
        }
    };

    var result = ko.computed({
        read: target,
        write: writeFilter
    }).extend({ notify: 'always', throttle: 1 });

    result(target());

    target.subscribe(writeFilter);

    return target;
};

Global filters can be setup to be reused via something similar to the following. See Filters for more information.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

define(function () {
    return {
        ltrim: function (value) {
            return (typeof value === 'string') ? value.replace(/^\s+/, "") : value;
        },

        onlyDigits: function (value) {
            return (typeof value === 'string') ? value.replace(/[^0-9]/g, '') : value;
        },

        onlyAlpha: function (value) {
            return (typeof value === 'string') ? value.replace(/[^A-Za-z _\-']/g, '') : value;
        },

        noSpecialChars: function (value) {
            return (typeof value === 'string') ? value.replace(/[^\/A-Za-z0-9 '\.,#\-]*$/g, '') : value;
        }
    };
});

Without RequireJS

You can still include the plugin without require js. The plugin adds a global ko.RuleEngine singleton that you can instantiate. This is done in the Inline Tests.

1
2
3
4
5
6

<script src="../app/js/knockout-rule-engine.js"></script>
<script>
    var ruleSet = {firstName: {required: true, validName: true, filter: function(){}}};
    var ruleEngine = new ko.RuleEngine(ruleSet);
    ...
</script>
]]> <![CDATA[Grails Plugin Template Override]]> 2013-07-02T10:00:00+00:00 http://www.christianoestreich.com/2013/07/grails-plugin-template-override Recently in the Grails Filterpane Plugin I had to figure out a way to allow users to selectively override the bundled template files that were rendered in the filterpane taglib. I was able to achieve this through a bit of simple code that may be helpful to plugin authors in the future.

The Code

The bit of magic in the code below is the line groovyPageLocator.findTemplateInBinding(path, pageScope). This equates to is asking grails to find a template for the path in the project scope. I then check if the result of the call is null and append the filterpane plugin to the resulting map causing the render to use the plugin vs the local template. This works well because apparently you can specify plugin: null in the render block which is the same as saying “use the current project scope”.

FilterPaneTagLib.groovy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

 def filterButton = { attrs, body ->
    //do some work here
    //...
    //...

    Map template = getTemplatePath('filterButton');
    out << render(template: template.path, plugin: template.plugin, model: renderModel)
}

public LinkedHashMap<String, String> getTemplatePath(String templateName) {
    def path = appendPiecesForUri("/_filterpane", templateName) //create the url path
    def template = [path: path] //add it to the map
    def override = groovyPageLocator.findTemplateInBinding(path, pageScope) //check if template exists in project scope
    if(!override) {
        template.plugin = 'filterpane' //looks like no, so use default plugin version instead
    }
    template  //return map
}
]]> <![CDATA[Travis CI & Grails GVM]]> 2013-05-09T10:00:00+00:00 http://www.christianoestreich.com/2013/05/travis-ci-grails-gvm For quite a while I have been stuck using grails 2.2.0 on a Travis CI build server due to any later version not being added to the groovy ppa on ubuntu. This was done because of the move to the GVM tool. I was able to cobble together a travis script and GVM hack to get this working. This is great because now I can use Grails 2.2.1, 2.2.2, etc.

The Hack

When setting up the script I noticed that it was failing due to a return code of 1. I forked and inspected the GVM code and identified a fix. It needed to return a 0 after it does the install.

I have submitted a pull request to fix the return issue in the gvm-install.sh script, but in the meantime you will need to keep the bit in the before-install block where it grabs my file and replaces the current one as seen in the script below.

The Setup

Travis appears to come with a package called GVM already on the box for Go language management. We have to remove that first via rm -rf ~/.gvm. We are then free to install GVM as it will not fail the check for the ~/.gvm directory.

Travis doesn’t like you to pipe commands back to bash as suggested in the GVM docs via curl -s get.gvmtool.net | bash so we have to get the install script and stick it into local file to execute. We do this via this code block.

1
2
3

curl -s get.gvmtool.net > ~/install_gvm.sh
chmod 775 ~/install_gvm.sh
~/install_gvm.sh

We then have to tell GVM to not prompt us defaults during install by appending to the config echo "gvm_auto_answer=true" > ~/.gvm/etc/config.

Then we hack the install with my pull (which can hopefully be removed later) via curl -s https://raw.github.com/ctoestreich/gvm/master/src/main/bash/gvm-install.sh > ~/.gvm/src/gvm-install.sh

After that we are golden to set up the source and install any version of grails we need.

1
2
3

source ~/.gvm/bin/gvm-init.sh
gvm install grails 2.2.1
gvm use grails 2.2.1

The rest of the build file is pretty vanilla.

The Script

Here is the .travis.yml script that you will need for your project and a link to a working example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

language: groovy

jdk:
- oraclejdk6

before_install:
- rm -rf ~/.gvm
- curl -s get.gvmtool.net > ~/install_gvm.sh
- chmod 775 ~/install_gvm.sh
- ~/install_gvm.sh
- echo "gvm_auto_answer=true" > ~/.gvm/etc/config
- curl -s https://raw.github.com/ctoestreich/gvm/master/src/main/bash/gvm-install.sh > ~/.gvm/src/gvm-install.sh
- source ~/.gvm/bin/gvm-init.sh
- gvm install grails 2.2.1
- gvm use grails 2.2.1

branches:
  only:
    - master

script: grails test-app --non-interactive

Once the pull request is complete you will be able to remove the line:

1

- curl -s https://raw.github.com/ctoestreich/gvm/master/src/main/bash/gvm-install.sh > ~/.gvm/src/gvm-install.sh
]]> <![CDATA[WSS4J Simple Security Using Groovy WSLite]]> 2013-01-17T10:00:00+00:00 http://www.christianoestreich.com/2013/01/wss4j-simple-security-using-wslite Recently I ran into an issue where using the builtin HTTPBasicAuthorization provided by wslite did not meet the needs of using a simple plain text username and password in the soap header of a soap service request.

I tried at first to use the basic authorization through HTTPBasicAuthorization realizing that I didn’t need http security, but a soap header. I was able to manually add this header to wslite in my test case by adding a header closure with the Security and UsernameToken attributes and children.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

def username = "wsuser"
def password = "secret"

SOAPResponse response = client.send {
    envelopeAttributes "xmlns:test": 'http://test.cxf.grails.org/', "xmlns:soapenv":"soapenv"
    version SOAPVersion.V1_1
    header {
        'wsse:Security'('soapenv:mustUnderstand': "1", 'xmlns:wsse': 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd', 'xmlns:wsu': 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd') {
            'wsse:UsernameToken'('wsu:Id':"UsernameToken-13") {
                'wsse:Username'(username)
                'wsse:Password'('Type':'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText',password)
                'wsse:Nonce'('EncodingType':'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary',new String(password.bytes.encodeBase64().toString()))
                'wsu:Created'('2013-01-18T16:19:17.950Z')
            }
        }
    }
    body {
        'test:simpleMethod' {
            param('HELLO WORLD')
        }
    }
}

For more details you can see my usage in the documentation for the cxf plugin or in the spock test directly.

]]> <![CDATA[CoffeeScript Ant Task Updated]]> 2012-12-07T10:00:00+00:00 http://www.christianoestreich.com/2012/12/coffeescript-ant-task-updated In one of our projects at work I really wanted to use CoffeeScript as a tool to create better scoped and JSLint friendly JavaScript. The problem I ran into was that we are using Apache Ant and getting the .coffee to .js was a bit of a chore.

I played around with using the coffee.exe from github. This approach worked, but as our conversion of files and lines grew so did the compile time. We needed something smarter that could inspect files and skip unmodified files.

I found the CoffeeScript compiling ant task project out on github. This worked well, but was on an older version of CoffeeScript 0.9.0 and would output all the .js files under a single destination directory. Meaning all the cs/*/.coffee files would output to one single directory. We really wanted to inherit the directory structure and keep the compiled JavaScripts neatly tucked into their respective directory structures.

I decided to fork and update the project. In doing so I also learned a bit about creating custom ant tasks. Hopefully anyone else out there who is using ant and wants to use CoffeeScript finds this helpful. A huge thanks to Patrick Mueller for creating this project. With a few small tweaks it has become essential in our JS->CS migration.

Here is the 0.1.6 Change Log:

0.1.6 – 2012/11/26

  • Updated to CoffeeScript 1.4.0
  • Changed Task for CoffeeScriptC to inherit Task instead of MatchingTask so we can add a boolean flag to optionally inherit directory nesting. The new flag is called nesting.
  • Changed noWrap to the new bare param for coffee-script compiler.
  • Added download directory containing versioned jars.

You can get more details at https://github.com/ctoestreich/CoffeeScriptAntTasks. The latest 0.1.6 jar can be found in the downloads directory of the project or the downloads section of github.

]]> <![CDATA[Grails CXF Plugin Version 1.x Released]]> 2012-11-10T10:52:02+00:00 http://www.christianoestreich.com/2012/11/grails-cxf-plugin-version-one-released The Grails Cxf Plugin makes exposing classes (services and endpoints) as SOAP web services easy and painless. Since version 1.0.0, it has been rewritten and enhanced to support more features including the migration to grails 2.x.

The current cxf version is 2.6.2

Some new things as of version 1.x are as follows:

  • The plugin will autowire configured classes in the grails-app\endpoints* AND the grails-app\services* directories
  • Endpoint creation scripts create-endpoint and create-endpoint-simple will create cxf artefacts in grails-app\endpoints
  • Service creation scripts create-cxf-service and create-cxf-service-simple will create cxf artefacts in grails-app\services
  • The suggested pattern to isolate cxf endpoints is to have endpoints live in grails-app/endpoints directory (or you can use grails-app/services for overlapping and shared services)
  • Built in support for simple Map response type handling via @XmlJavaTypeAdapter(GrailsCxfMapAdapter.class) method annotation has been included to use or to kick start your own map adapter creation
  • Many new examples to help with configuration can be found in the source via functional specs and test classes at https://github.com/Grails-Plugin-Consortium/grails-cxf
  • Default plugin configuration is provided via DefaultCxfConfig.groovy. Although usually not necessary, you can override in your project’s Config.groovy
  • The default url for wsdl viewing remains http://.../[app name if not root]/services as it was in previous versions. Multiple cxf servlet endpoints can be configured or the default changed via Config.goovy
  • Wsdl First services are now available to use
  • Plugin should be mostly backwards compatible and work in grails 1.3.x

You can get more details at https://github.com/Grails-Plugin-Consortium/grails-cxf

There is also a demo project if you do not want to run the cxf plugin code inline at https://github.com/Grails-Plugin-Consortium/grails-cxf-demo

]]> <![CDATA[Testing Domain Constraints Using Grails 2.x & Spock 0.7]]> 2012-11-10T10:00:00+00:00 http://www.christianoestreich.com/2012/11/domain-constraints-grails-spock-updated We were migrating some existing Java code to Grails 2.0 and we were looking for a good solution to test domain constraints as we were migrating from an existing schema. We already use and love Spock for most of our testing needs. We came up with a relatively easy and reusable solution for testing constraints that I wanted to share. As we make improvements or changes I will update the post.

Update I have updated the project and samples to work with Grails 2.1.1 and Spock 0.7.

All files for this demonstration can be found at my grails-spock-constraints GitHub repository.

Setting Up A Domain Object

I set up an arbitrary Person object with some constraints defined on it that will help demonstrate how to test a wide variety of constraints using spock. The latest Person class can be found on github, but here the file at the time I published this post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

class Person {
    static hasMany = [children: Person]
    String firstName
    String middleName
    String lastName
    String email
    Integer age
    String ssn
    String amex
    String gender
    String login
    Date birthDate
    Float wage
    Integer fingers
    String homePage
    String username
    static constraints = {
        firstName size: 1..50
        middleName size: 0..50
        lastName size: 1..50
        email email: true, notEqual: "bill@microsoft.com"
        age nullable: false, range: 0..150
        ssn unique: true, blank: false
        amex creditCard: true
        gender inList: ["Male", "Female"]
        login matches: "[a-zA-Z]+"
        birthDate max: new Date()
        wage min: 0F, scale: 2
        children maxSize: 10, minSize: 2
        fingers nullable: true
        homePage url: true
        username validator: {
            if(!it.startsWith('boba')) return ['invalid.bountyhunter']
        }
    }
}

As you can see there are variety of constraints defined for this object. Some of these might be nonsensical, but I wanted to demonstrate testing a wide variety of scenarios. To get the invalid.bountyhunter message working I simply added the following to the bottom of my message.properties file:

1

person.username.invalid.bountyhunter=Invalid bounty hunter ({2}) tried to

log in.

Grails Constraints

The following are a list of available constraints to define on your domain classes.

I will let the reader check the documentation for the specifics of each constraint, but I do want to mention a few Gotchas that I ran into:

Gottchas

  • Setting an inList constraint will not throw an error when passing a blank ‘’ string. I had assumed that using inList: [‘one’,‘two’] would require the value to be not null and a valid value in list. Not True
  • Setting a field as url type will allow blank and null values are valid urls unless you explicitly define blank or nullable as a constraint. (This seems to be fixed to not allow null by default in final 2.0.0)
  • Setting column length for String type should be done using field: size 0..50 for a string that can be empty to 50 length. And would be set to field: size: 1..100 for a field that can not be empty and has a max length of 100. MaxSize and minSize are NOT valid for this.

I will add to that list as I run across items that are unexpected.

Spock Basics

If you aren’t familiar with Spock and it’s feature set, read up on it and start using it. It brings very rich and powerful tooling to testing your Grails application. The important pieces to be familiar with for these tests are the use of the where clause, parameterizations and the @Unroll features.

It is due to some Spock magic that these tests are able to do so much with such little code. You will see the line in the tests like the one below of:

1

def obj = new Person("$field": val)

Spock will replace both the field and val with data from the configured where table. I had to put the field in quotes and treat it as a gstring so Spock would replace that correctly as simply using the new Person(field: val) would correctly replace the val, but treat the field as an actual object property named field instead of being replaced with a valid field name from the where table.

Other Possible Techniques

We also really like the build test data plugin and started off writing these as integration tests with buildWithoutSave giving us fully hydrated live domain objects and simply changing the fields we wanted to invalidate. But with Grails’ ability to mock for constraints in unit tests we can run these tests even faster and earlier in the test cycle if we use unit tests instead. This is import to us as we can run the test-app -unit in a few seconds during development. Our machines are not the fastest and running -integration takes several tens of seconds to run.

Having the objects built with the build test data plugin is another good option that some people may opt for since you can mix a richer set of tests together since you are in a headless app during integration. I will leave that decision up the readers which method they prefer.

Testing Constraints

I first created a simple abstract helper class that can build some of our data for us with reusable methods. This class also holds the function that checks for the error message to exist on a field after the validate is called. It is called ConstraintUnitSpec.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import spock.lang.Specification

abstract class ConstraintUnitSpec extends Specification {
    String getLongString(Integer length) {
       'a' * length
   }

   String getEmail(Boolean valid) {
       valid ? "dexter@miamipd.gov" : "dexterm@m"
   }

   String getUrl(Boolean valid) {
       valid ? "http://www.google.com" : "http:/ww.helloworld.com"
   }

   String getCreditCard(Boolean valid) {
       valid ? "4111111111111111" : "41014"
   }

   void validateConstraints(obj, field, error) {
       def validated = obj.validate()
       if (error && error != 'valid') {
           assert !validated
           assert obj.errors[field]
           assert error == obj.errors[field]
       } else {
           assert !obj.errors[field]
       }
   }
}

We then create our constraint tests and extend this class. We also use the new Grails 2.0 @TestFor annotation to inject some test helper methods such as mockForConstraintsTests since we are in the unit test phase.

1
2
3
4

@TestFor(Person)
class PersonSpec extends ConstraintUnitSpec {

}

We want to set up the test and tell Grails that we are mocking the person object so it will add the validate method and we can also add an existing person to test unique constraints against. We do this by adding the following.

1
2
3
4
5
6
7

@TestFor(Person)
class PersonSpec extends ConstraintUnitSpec {
    def setup() {
        //mock a person with some data (put unique violations in here so they can be tested, the others aren't needed)
        mockForConstraintsTests(Person, [new Person(ssn: '123456789')])
    }
}

After this is done we can start adding some test blocks. The first test I always add is the test for the standard constraint bounds. This will help with any refactoring that you you do later to change field definitions such as changing the size/length of a string. If you had size: 1..50 and your test checks for 0 and 51 and then change the field constraint to size: 0..50, this will cause your test to now fail and hopefully save you some headaches later so you can double check your new change against the domain model and data. Here is the comprehensive test for the Person class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

@Unroll("test person all constraints #field is #error")
def "test person all constraints"() {
    when:
    def obj = new Person("$field": val)

    then:
    validateConstraints(obj, field, error)

    where:
    error                  | field        | val
    'size'                 | 'firstName'  | getLongString(51)
    'nullable'             | 'firstName'  | null
    'size'                 | 'middleName' | getLongString(51)
    'nullable'             | 'middleName' | null
    'size'                 | 'lastName'   | getLongString(51)
    'nullable'             | 'lastName'   | null
    'notEqual'             | 'email'      | 'bill@microsoft.com'
    'email'                | 'email'      | getEmail(false)
    'range'                | 'age'        | 151
    'range'                | 'age'        | -1
    'nullable'             | 'age'        | null
    'blank'                | 'ssn'        | ''
    'unique'               | 'ssn'        | '123456789'
    'creditCard'           | 'amex'       | getCreditCard(false)
    'inList'               | 'gender'     | 'Unknown'
    'matches'              | 'login'      | 'ABC123'
    'max'                  | 'birthDate'  | new Date() + 1
    'min'                  | 'wage'       | -1F
    'maxSize'              | 'children'   | createPerson(11)
    'minSize'              | 'children'   | createPerson(1)
    'url'                  | 'homePage'   | getUrl(false)
    'invalid.bountyhunter' | 'username'   | 'buba'
}

Using Spock, these tests become very concise and easy to read in the where clause. I set it up so that the field when using val will cause the error constraint to be violated.

One of the reasons we like this method to test constraints is other constraint tests simply test for errors to exist on a field, but not the specific type of constraint violation. While the test for general field errors is still valid, it isn’t quite as fine grained as checking the actual type of constraint violation expected. An example of a generic constraint testing can be found in a couple places, but OPI published an article here on this style of constraint testing. This is a good starting point if you don’t need as fine grained control as I offer here.

As a side note, every time the test checks for a constraint violation there will be many fields violated, but we only care about and check for one specific field and constraint to be violated for each row in the where table as a time. It might be possible to mix multiple checks together, but we like testing each scenario and constraint individually.

Adding Valid Tests

We can also add additional tests that check for valid values. I added some logic in the validateConstraints method that will expect the field to pass validation if you use the value in the error column of ‘valid’ or just a null. Using the actual word ‘valid’ instead of null will help the test names be more concise when Spock unrolls them. In the following age tests we are checking for both failure and valid criteria. Since age is defined as:

1

age nullable: false, range: 0..150

We will be checking for values that fall at the limits, outside and inside the range as well as passing a null value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

@Unroll("person #field is #error using #val")
def "test person age constraints"() {
    when:
    def obj = new Person("$field": val)

    then:
    validateConstraints(obj, field, error)

    where:
    error      | field | val
    'range'    | 'age' | 151
    'range'    | 'age' | -1
    'nullable' | 'age' | null
    'valid'    | 'age' | 100
    'valid'    | 'age' | 150
    'valid'    | 'age' | 0
}

Here is the full PersonSpec

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188

   import grails.test.mixin.TestFor
   import spock.lang.Unroll

   @TestFor(Person)
   class PersonSpec extends ConstraintUnitSpec {

       def setup() {
           //mock a person with some data (put unique violations in here so they can be tested, the others aren't needed)
           mockForConstraintsTests(Person, [new Person(ssn: '123456789')])
       }

       @Unroll("test person all constraints #field is #error")
       def "test person all constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error                  | field        | val
           'size'                 | 'firstName'  | getLongString(51)
           'nullable'             | 'firstName'  | null
           'size'                 | 'middleName' | getLongString(51)
           'nullable'             | 'middleName' | null
           'size'                 | 'lastName'   | getLongString(51)
           'nullable'             | 'lastName'   | null
           'notEqual'             | 'email'      | 'bill@microsoft.com'
           'email'                | 'email'      | getEmail(false)
           'range'                | 'age'        | 151
           'range'                | 'age'        | -1
           'nullable'             | 'age'        | null
           'blank'                | 'ssn'        | ''
           'unique'               | 'ssn'        | '123456789'
           'creditCard'           | 'amex'       | getCreditCard(false)
           'inList'               | 'gender'     | 'Unknown'
           'matches'              | 'login'      | 'ABC123'
           'max'                  | 'birthDate'  | new Date() + 1
           'min'                  | 'wage'       | -1F
           'maxSize'              | 'children'   | createPerson(11)
           'minSize'              | 'children'   | createPerson(1)
           'url'                  | 'homePage'   | getUrl(false)
           'invalid.bountyhunter' | 'username'   | 'buba'
       }

       @Unroll("person #field is #error using #val")
       def "test person age constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error      | field | val
           'range'    | 'age' | 151
           'range'    | 'age' | -1
           'nullable' | 'age' | null
           'valid'    | 'age' | 100
           'valid'    | 'age' | 150
           'valid'    | 'age' | 0
       }

       @Unroll("person #field is #error using #val")
       def "test person ssn constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error      | field | val
           'blank'    | 'ssn' | ''
           'nullable' | 'ssn' | null
           'unique'   | 'ssn' | '123456789'
           'valid'    | 'ssn' | '123456788'
           'valid'    | 'ssn' | '123-45-6787'
       }

       @Unroll("person #field is #error using #val")
       def "test person username constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error                  | field      | val
           'invalid.bountyhunter' | 'username' | ''
           'nullable'             | 'username' | null
           'invalid.bountyhunter' | 'username' | 'bubua'
           'valid'                | 'username' | 'bobafet'
           'valid'                | 'username' | 'bobajunior'
       }

       @Unroll("person #field is #error using #val")
       def "test person homepage constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error      | field      | val
           'url'      | 'homePage' | getUrl(false)
           'valid'    | 'homePage' | '' //blanks work for url
           'nullable' | 'homePage' | null //null works for url (2.0.0 not anymore)
           'valid'    | 'homePage' | getUrl(true) + '/page.gsp'
           'valid'    | 'homePage' | getUrl(true)
       }

       @Unroll("person #field is #error using #val")
       def "test person gender constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error      | field    | val
           'inList'   | 'gender' | 'Unknown'
           'nullable' | 'gender' | null
           'valid'    | 'gender' | '' //blanks work for inList
           'valid'    | 'gender' | 'Male'
           'valid'    | 'gender' | 'Female'
       }

       @Unroll("person #field is #error using #val")
       def "test person credit card constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error        | field  | val
           'creditCard' | 'amex' | getCreditCard(false)
           'nullable'   | 'amex' | null
           'valid'      | 'amex' | ''
           'valid'      | 'amex' | getCreditCard(true)
       }

       @Unroll("person #field is #error using #val")
       def "test person birth date constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error      | field       | val
           'max'      | 'birthDate' | new Date() + 1
           'nullable' | 'birthDate' | null
           'valid'    | 'birthDate' | new Date() - 1
           'valid'    | 'birthDate' | new Date()
       }

       @Unroll("person #field testing #error")
       def "test person children constraints"() {
           when:
           def obj = new Person("$field": val)

           then:
           validateConstraints(obj, field, error)

           where:
           error     | field      | val
           'maxSize' | 'children' | createPerson(11)
           'minSize' | 'children' | createPerson(1)
           'valid'   | 'children' | null
           'valid'   | 'children' | createPerson(10)
           'valid'   | 'children' | createPerson(2)
       }

       private createPerson(Integer count) {
           def persons = []
           count.times {
               persons << new Person()
           }
           persons
       }
   }

Conclusion

I hope that this can be a useful tool or guideline for you when you are creating constraint tests for your domain objects. These tests are a bit tedious to write, but using some of the techniques here makes writing them pretty quick to create and test.

I would probably never test ALL the constraints like this, but in case you wanted to test anything specific and guard against certain non-allowed data then you might want to consider something like this. This would certainly be to your benefit to do for any custom constraints you define on your objects.

Sample Project

All files for this demonstration can be found at my grails-spock- constraints GitHub repository.

Getting Grails 2.0 and Spock 0.6 Working (Legacy)

We had a little bit of trouble getting grails 2.0.0.RC1 and Spock 0.6 to play well together, but found some useful information in this jira. Add the following to the repositories section of your BuildConfig.groovy

1

mavenRepo "http://m2repo.spockframework.org/snapshots"

and the following to the plugins section of your BuildConfig.groovy

1

test ":spock:0.6-SNAPSHOT"

These will no doubt change as 2.0 becomes final and the official spock plugin is updated.

References

]]> <![CDATA[Grails 2.x Deployed To Websphere 7ish]]> 2012-10-20T12:00:00+00:00 http://www.christianoestreich.com/2012/10/deloying-grails-2-to-websphere I have struggled a lot with getting our corporate-standard WebSphere container to play nicely with Grails in the past and when we moved to Grails 2.1.1 and WebSphere 7.0 it was no different.

EAR Script

The first thing you will want to do is add the Ear script to your project under scripts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

includeTargets << grailsScript("_GrailsWar")

target(ear: "Creates an EAR file from a Grails WAR") {
  war()
  event("StatusUpdate", ["Building EAR file"])
  generateApplicationXml()
  def warDest = new File(warName).parentFile
  def appVersion = metadata.getApplicationVersion()
  def earFile = "${projectTargetDir}/${contextRoot}-${appVersion}.ear"
  ant.ear(destfile: earFile, appxml: appXml, update: true) {
    fileset(dir: warDest, includes: "*.war")
  }
  event("StatusFinal", ["Done creating EAR $earFile"])
}
target(defineContextRoot: "defines the context root") {
  contextRoot = "${grailsAppName}"
}
target(generateApplicationXml: "Generates an application.xml file") {
  depends(defineContextRoot)
  def warDest = new File(warName)
  appXml = "${projectTargetDir}/application.xml"
  new File(appXml).write """<?xml version="1.0" encoding="UTF-8"?>
<application xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="5" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/application_5.xsd">
<display-name>${grailsAppName}</display-name>
<module>
    <web>
        <web-uri>${warDest.name}</web-uri>
        <context-root>${contextRoot}</context-root>
    </web>
 </module>
</application>"""
}

setDefaultTarget(ear)

Install Web.xml and JNDI

If, like us, you use JNDI for your database connection you will probably want to add it to the web.xml. Just run the grails install-templates command and modify your web.xml with your jndi node similar to below.

1
2
3
4
5
6
7
8
9

<web-app>
...
    <resource-ref id="ResourceRef_12345">
        <res-ref-name>jdbc/MyName</res-ref-name>
        <res-type>javax.sql.DataSource</res-type>
        <res-auth>Application</res-auth>
        <res-sharing-scope>Shareable</res-sharing-scope>
    </resource-ref>
</web-app>

You should have your DataSource.groovy set up similar to this for each environment you are using.

1
2
3
4
5
6
7
8
9

environments {
    ...
    production {
        dataSource {
            jndiName = "java:comp/env/jdbc/MyName"
            dialect = "org.hibernate.dialect.OracleDialect"
        }
    }
}

If you need this jndi available in dev (run-app local mode) you will need to add it to your Config.groovy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

environments {
    development {
        // this will create the JNDI entry in the tomcat plugin for local execution
        grails.naming.entries = [
                "jdbc/MyName": [
                        type: "javax.sql.DataSource", //required
                        auth: "Container", // optional
                        description: "Data source for Database", //optional
                        driverClassName: "com.mysql.jdbc.Driver",
                        url: "jdbc:mysql://server:3306/db?useOldAliasMetadataBehavior=true",
                        username: "blah",
                        password: "secret",
                        maxActive: "8",
                        maxIdle: "8",
                        poolPreparedStatements: "true"
                ]
        ]
    }
}

Additional WebSphere Web Xml Settings

There are two additional files you will probably need to create and put into your web-app\WEB_INF directory.

ibm-web-bnd.xml

1
2
3
4
5
6

<?xml version="1.0" encoding="UTF-8"?>
<web-bnd xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://websphere.ibm.com/xml/ns/javaee"
         xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-bnd_1_0.xsd" version="1.0">
    <virtual-host name="default_host"/>
    <resource-ref name="jdbc/MyName" binding-name="jdbc/MyName"/>
</web-bnd>

ibm-web-ext.xml

1
2
3
4
5
6
7
8
9
10

<?xml version="1.0" encoding="UTF-8"?>
<web-ext xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://websphere.ibm.com/xml/ns/javaee"
         xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-ext_1_0.xsd" version="1.0">
    <webApp href="WEB-INF/web_merged.xml#WebApp_ID"/>
    <jsp-attribute name="reloadEnabled" value="true"/>
    <jsp-attribute name="reloadInterval" value="10"/>
    <jsp-attribute name="jdkSourceLevel" value="16"/>
    <jsp-attribute name="trackDependencies" value="true"/>
    <jsp-attribute name="keepgenerated" value="true"/>
</web-ext>

WebSphere Settings

I am not even sure if these are needed, but I did them anyway. (from <http://grails.org/Deployment#Websphere 6.1>)

Server JVM settings:

  • In “Application servers > server > Process Definition > Java Virtual Machine” set “Generic JVM arguments” to “-Xverify:none”
  • In “Application servers > server > Process Definition > Java Virtual Machine > Custom Properties”, add a custom property name: com.ibm.ws.classloader.getInputStream.enableIOException value: true
  • In “Application servers > server > Web container > Custom Properties”, add a custom property name: com.ibm.ws.webcontainer.invokeFiltersCompatibility value: true

Also you will need to make sure that your Enterprise Application > Class loading and update detection

  • Classes loaded with local class loader first (parent last)
  • Single class loader for application

War Settings

In the end you will probably have jar/lib hell and your application may not start correctly. Make sure all your plugins and dependencies have appropriate excludes. For example, we are using spring security (not the plugin) and here is one dependency exclude example:

1
2
3
4
5
6
7

compile('org.springframework.security:spring-security-core:3.0.7.RELEASE') {
    excludes 'spring-expression', 'spring-core', 'spring-context', 'spring-tx',
             'spring-aop', 'spring-jdbc', 'spring-web', 'spring-test', 'aspectjrt',
             'aspectjweaver', 'cglib-nodep', 'ehcache', 'commons-collections',
             'hsqldb', 'jsr250-api', 'log4j', 'junit', 'mockito-core', 'jmock-junit4',
             'spring-aop', 'spring-beans', 'spring-context', 'spring-core', 'spring-tx'
}

I currently am also telling grails to remove certain jars from the war in BuildConfig.groovy via

1
2
3
4
5
6
7
8
9

grails.war.resources = { stagingDir, args ->
    delete file: "${stagingDir}/WEB-INF/lib/geronimo-servlet_2.5_spec-1.2.jar"
    delete file: "${stagingDir}/WEB-INF/lib/geronimo-jms_1.1_spec-1.1.1.jar"
    delete file: "${stagingDir}/WEB-INF/lib/geronimo-commonj_1.1_spec-1.0.jar"
    delete file: "${stagingDir}/WEB-INF/lib/jta-1.1.jar"
    delete file: "${stagingDir}/WEB-INF/lib/ojdbc.jar"
    delete file: "${stagingDir}/WEB-INF/lib/antlr-2.7.6.jar"
    delete file: "${stagingDir}/WEB-INF/lib/geronimo-javamail_1.4_spec-1.7.1.jar"
}

Conclusion

Good Luck…

]]> <![CDATA[Grails Cxf Interceptor Injection]]> 2012-04-06T12:00:00+00:00 http://www.christianoestreich.com/2012/04/grails-cxf-interceptor-injection.markdown #### Apache Cxf Security #### All I have to say is good luck with using the sample code from the [apache cxf security docs][4]. Their code for 2.5.x is lacking and the code provided didn’t work and caused me a lot of ???? moments. The [cxf plugin][3] wires up service factories that will match the name of your exposed service such as `secureService` with bean named `secureServiceFactory`. You will want to inject any interceptors onto that factory during application boot. For the following examples I use the BootStrap.groovy to inject these. #### Grails Cxf Interceptor Injection 2.3.x #### This is the “old” way of adding simple token authentication with 2.3.x. “` groovy import org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor import org.apache.ws.security.WSConstants import org.apache.ws.security.WSPasswordCallback import org.apache.ws.security.handler.WSHandlerConstants import javax.security.auth.callback.Callback import javax.security.auth.callback.CallbackHandler import javax.security.auth.callback.UnsupportedCallbackException class BootStrap { def secureServiceFactory def init = { servletContext -> Map inProps = [:] inProps.put(WSHandlerConstants.ACTION, WSHandlerConstants.USERNAME_TOKEN); inProps.put(WSHandlerConstants.PASSWORD_TYPE, WSConstants.PW_TEXT); inProps.put(WSHandlerConstants.PW_CALLBACK_REF, new CallbackHandler() { void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException { WSPasswordCallback pc = (WSPasswordCallback) callbacks[0] println pc.identifier println pc.password if(pc.identifier == “wsuser” && pc.password != “secret”) { println “error :: wrong password” throw new IOException(“wrong password”) } } }) secureServiceFactory.getInInterceptors().add(new WSS4JInInterceptor(inProps)) } } “` #### Grails Cxf Interceptor Injection 2.4.x-2.5.x #### Since the new apache cxf 2.4.x and 2.5.x use a Validator instead of the callback, here is how you would now inject that same check into the same service factory. I am using an anonymous UsernameTokenValidator class just like the anonymous callback in the previous example. “` groovy import org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor import org.apache.ws.security.WSConstants import org.apache.ws.security.WSSecurityEngine import org.apache.ws.security.WSSecurityException import org.apache.ws.security.handler.WSHandlerConstants import org.apache.ws.security.validate.UsernameTokenValidator import org.apache.ws.security.validate.Validator import javax.xml.namespace.QName class BootStrap { def secureServiceFactory def init = { servletContext -> Map inProps = [:] inProps.put(WSHandlerConstants.ACTION, WSHandlerConstants.USERNAME_TOKEN); inProps.put(WSHandlerConstants.PASSWORD_TYPE, WSConstants.PW_TEXT); Map validatorMap = new HashMap(); validatorMap.put(WSSecurityEngine.USERNAME_TOKEN, new UsernameTokenValidator() { @Override protected void verifyPlaintextPassword(org.apache.ws.security.message.token.UsernameToken usernameToken, org.apache.ws.security.handler.RequestData data) throws org.apache.ws.security.WSSecurityException { if(data.username == “wsuser” && usernameToken.password == “secret”) { println “username and password are correct!” } else { println “username and password are NOT correct…” throw new WSSecurityException(“user and/or password mismatch”) } } }); inProps.put(WSS4JInInterceptor.VALIDATOR_MAP, validatorMap); secureServiceFactory.getInInterceptors().add(new WSS4JInInterceptor(inProps)) } def destroy = { } } “` #### Conclusion #### My frustrations really only apply to the username token authentication issue, but I haven’t tried getting the certificate or ldap authentication working on the server side in 2.5.2 yet. In general if you need in or out logging or security interceptors on your [cxf plugin][3] exposed services, the above mentioned is a pretty good way to get them on there. What you put into the properties of those interceptors to make them work is on you. Good Luck! [1]: https://www.github.com/Grails-Plugin-Consortium/cxf-client-demo (Grails Cxf Client Demo) [2]: https://www.github.com/Grails-Plugin-Consortium/cxf-client (Grails Cxf Client Plugin) [3]: http://www.grails.org/plugin/cxf (Grails Cxf Plugin) [4]: http://cxf.apache.org/docs/ws-security.html (Apache Cxf Security Docs) ]]> <![CDATA[Grails Cxf Interceptor Injection]]> 2012-04-06T12:00:00+00:00 http://www.christianoestreich.com/2012/04/grails-cxf-interceptor-injection In my grails cxf-client-demo project I needed a way to test if the simple username token authentication wss4j interceptors were working. I looked around the web for help on how to inject an inbound security interceptor into the grails cxf plugin. I found some example on how to do this with cxf 2.3.x versions, but when I was recently helping update the grails cxf plugin to use cxf 2.5.2, I found out they changed the way that username token was done.

Apache Cxf Security

All I have to say is good luck with using the sample code from the apache cxf security docs. Their code for 2.5.x is lacking and the code provided didn’t work and caused me a lot of ???? moments.

The cxf plugin wires up service factories that will match the name of your exposed service such as secureService with bean named secureServiceFactory. You will want to inject any interceptors onto that factory during application boot. For the following examples I use the BootStrap.groovy to inject these.

Grails Cxf Interceptor Injection 2.3.x

This is the “old” way of adding simple token authentication with 2.3.x.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor
import org.apache.ws.security.WSConstants
import org.apache.ws.security.WSPasswordCallback
import org.apache.ws.security.handler.WSHandlerConstants

import javax.security.auth.callback.Callback
import javax.security.auth.callback.CallbackHandler
import javax.security.auth.callback.UnsupportedCallbackException

class BootStrap {

    def secureServiceFactory

    def init = { servletContext ->

        Map<String, Object> inProps = [:]
        inProps.put(WSHandlerConstants.ACTION, WSHandlerConstants.USERNAME_TOKEN);
        inProps.put(WSHandlerConstants.PASSWORD_TYPE, WSConstants.PW_TEXT);
        inProps.put(WSHandlerConstants.PW_CALLBACK_REF, new CallbackHandler() {

            void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
                WSPasswordCallback pc = (WSPasswordCallback) callbacks[0]
                println pc.identifier
                println pc.password
                if(pc.identifier == "wsuser" && pc.password != "secret") {
                    println "error :: wrong password"
                    throw new IOException("wrong password")
                }
            }
        })
        secureServiceFactory.getInInterceptors().add(new WSS4JInInterceptor(inProps))
    }
}

Grails Cxf Interceptor Injection 2.4.x-2.5.x

Since the new apache cxf 2.4.x and 2.5.x use a Validator instead of the callback, here is how you would now inject that same check into the same service factory. I am using an anonymous UsernameTokenValidator class just like the anonymous callback in the previous example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

import org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor
import org.apache.ws.security.WSConstants
import org.apache.ws.security.WSSecurityEngine
import org.apache.ws.security.WSSecurityException
import org.apache.ws.security.handler.WSHandlerConstants
import org.apache.ws.security.validate.UsernameTokenValidator
import org.apache.ws.security.validate.Validator

import javax.xml.namespace.QName

class BootStrap {

    def secureServiceFactory

    def init = { servletContext ->

        Map<String, Object> inProps = [:]
        inProps.put(WSHandlerConstants.ACTION, WSHandlerConstants.USERNAME_TOKEN);
        inProps.put(WSHandlerConstants.PASSWORD_TYPE, WSConstants.PW_TEXT);
        Map<QName, Validator> validatorMap = new HashMap<QName, Validator>();
        validatorMap.put(WSSecurityEngine.USERNAME_TOKEN, new UsernameTokenValidator() {

            @Override
            protected void verifyPlaintextPassword(org.apache.ws.security.message.token.UsernameToken usernameToken, org.apache.ws.security.handler.RequestData data)
                throws org.apache.ws.security.WSSecurityException {
                if(data.username == "wsuser" && usernameToken.password == "secret") {
          println "username and password are correct!"
      } else {
          println "username and password are NOT correct..."
                    throw new WSSecurityException("user and/or password mismatch")
                }
            }
        });
        inProps.put(WSS4JInInterceptor.VALIDATOR_MAP, validatorMap);
        secureServiceFactory.getInInterceptors().add(new WSS4JInInterceptor(inProps))
    }

    def destroy = {
    }
}

Conclusion

My frustrations really only apply to the username token authentication issue, but I haven’t tried getting the certificate or ldap authentication working on the server side in 2.5.2 yet. In general if you need in or out logging or security interceptors on your cxf plugin exposed services, the above mentioned is a pretty good way to get them on there. What you put into the properties of those interceptors to make them work is on you. Good Luck!

]]> <![CDATA[Grails Cxf Client 1.2.9 Released]]> 2012-03-01T12:00:00+00:00 http://www.christianoestreich.com/2012/03/grails-cxf-client-soap-exceptions.markdown <![CDATA[Grails Cxf Client 1.2.9 Released]]> 2012-03-01T12:00:00+00:00 http://www.christianoestreich.com/2012/03/grails-cxf-client-soap-exceptions The Grails cxf-client plugin has been updated to version 1.2.9. When I first deployed the plugin I overlooked allowing checked exceptions and soap faults to bubble out of the client proxy appropriately. I modified the plugin to allow this now. In addition I added a configuration parameter for proxyFactoryBindingId to allow soap 1.2 (set to “http://schemas.xmlsoap.org/wsdl/soap12/”).

Updated Parameter Documentation

The following partial documentation is available in full at https://github.com/Grails-Plugin-Consortium/grails-cxf-client

Property Description Required
proxyFactoryBindingId The URI, or ID, of the message binding for the endpoint to use. For SOAP the binding URI(ID) is specified by the JAX-WS specification. For other message bindings the URI is the namespace of the WSDL extensions used to specify the binding. If you would like to change the binding (to use soap12 for example) set this value to “http://schemas.xmlsoap.org/wsdl/soap12/”. (default: “”) No
]]> <![CDATA[Grails Docs Helper Script]]> 2012-02-24T15:00:00+00:00 http://www.christianoestreich.com/2012/02/grails-script-docs-helper I am creating a hello world grails project for internal use here at our company. The project encapsulates both the source code and the documentation in gdoc format. When a new user comes onboard, they get the project from the source repository and then run the grails doc target to generate the guide locally with a tutorial on building a simple application. In the project I have already included a full test suite that should pass when the tutorial is completed.

I needed a way to have my version control ignored domain, service, and controllers that I was using be added in a way that kept them out of the base grails-app directory so the user could generate them. At the same time I really wanted to put a link to what the finished classes would look like in the docs somewhere to help users along during the process if needed. During this process I only wanted to have one master file and not have to maintain the code blocks in the gdoc files separately from actual code.

Background

As I wrote the hello world tutorial I was in effect writting a simple application at the same time. As I want the user to create all their own artifacts, I added them to my ignore list in version control. This worked great and allowed me to build the docs and grab snippets of code for the gdoc and still maintain a shell of a project for users to be able to add these artifacts on their own when they followed the guide.

Alas, I really didn’t want to just throw away these objects I was creating or leave them on my machine only. I came up with a way for me to have the best of both worlds by using a Grails script that executes a few simple ant tasks. I figured I would backup the files into another directory that was checked in as well as generate gdocs out of the code directly at the same time.

Creating A Backup Script

I created a script in the scripts directory of the project named GenerateDocs.groovy which would allow me to copy the sources (in my case a directory named answers at the project root), generate the gdoc, and run the docs task in one command by invoking the target:

grails generate-docs

First I added the following block to the code to do the backup and run the grails doc target.

GenerateDocs.groovy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

includeTargets << grailsScript("_GrailsDocs")

target(main: "Copy controllers, services, domain and views from svn excluded grails-app dirs to the answers dir and create gdoc from them in ref sidebar") {
    //make sure they all compile
    depends(compile)

    //closure to include or exclude certain file types
    def filesetToInclude = {directory ->
        fileset(dir: directory) {
            include(name: '**/*.groovy')
            include(name: '**/*.gsp')
            include(name: '**/*.xml')
            include(name: '**/*.properties')
        }
    }

    def initCopyDirs = ['grails-app/controllers', 'grails-app/domain', 'grails-app/services', 'grails-app/views']

    ant.sequential {
        //add the conf dir for now
        (initCopyDirs + ['grails-app/conf']).each { directory ->
            println "copying $directory to answers/$directory"
            copy(todir: "answers/${directory}") {
                filesetToInclude(directory)
            }
        }
        println "completed copying files"
    }

    depends(docs)
}

setDefaultTarget(main)

This would take all the directories in my app that matched the path in the list initCopyDirs and copy then into the answers directory with identical pathing relative to grails-app. I also only wanted to copy a subset of the files so I used the fileset include parameter to limit the files I copy. I then added all the files under the answers directory into version control for safe keeping knowing if and when they changed in the original grails-app directory my script would overwrite these backups and they would be added for update in my next commit. If I created any new files they would have to be manually added to version control. Not really all that exciting but at least now I would know that my source code wasn’t going to disappear since it was being ignored.

Creating Answers From Code

I wanted to take all the same files we were backing up and create a viewable code item on the Quick Reference sidebar. To accomplish this I had to transform the .groovy files into .gdoc files and put them into the src/docs/ref/Answers directory. I added the following ant build task to my script above to accomplish that:

GenerateDocs.groovy 
1
2
3
4
5
6
7
8
9
10
11
12

 initCopyDirs.each { directory ->
    ant.fileScanner {
        filesetToInclude(directory)
    }.each {File file ->
        println "wrapping file ${file.name}"
        echo(file: "src/docs/ref/Answers/${file.name}.gdoc", """
            {code}
                ${file.getText()}
            {code}
        """)
    }
}

What that code block accomplishes is to create a gdoc file for all matched files with the file’s contents wrapped in {code} blocks. For example it would take a Person.groovy file in the grails-app/domain directory and copy it answers/grails-app/domain an then create a file at src/docs/ref/Ansers/Person.groovy.gdoc with the following code snippet:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{code}
package grails.hello.world

class Person {

    String name
    Integer age
    Date createdDate

    static constraints = {
        name(nullable: false, blank: false)
        age(nullable: false, range: 0..150)
    }
}
{code}

That would show up in the generated doc pages as the following:

The Whole Script

Conclusion

Maybe this will be useful for you in some or all of it’s entirety in the future!?

]]> <![CDATA[Grails Cxf Client 1.2.7 Released]]> 2012-02-15T15:00:00+00:00 http://www.christianoestreich.com/2012/02/grails-cxf-client-http-client-policy.markdown #### Updated Parameter Documentation #### The following partial documentation is available in full at [https://github.com/Grails-Plugin-Consortium/cxf-client][2] Property|Description|Required :———–|:————|:———— connectionTimeout|Specifies the amount of time, in milliseconds, that the client will attempt to establish a connection before it times out. The default is 30000 (30 seconds). 0 specifies that the client will continue to attempt to open a connection indefinitely. (default: 30000)|No receiveTimeout|Specifies the amount of time, in milliseconds, that the client will wait for a response before it times out. The default is 60000. 0 specifies that the client will wait indefinitely. (default: 60000)|No allowChunking|If true will set the HTTPClientPolicy allowChunking for the clients proxy to true. (default: false)|No httpClientPolicy|Instead of using the seperate timeout, chunking, etc values you can create your own HTTPClientPolicy bean in resources.groovy and pass the name of the bean here. This will override the connectionTimeout, receiveTimeout and allowChunking values. (default: null)|No #### Timeout And Chunking Parameters #### Here is an example cxf client configuration block using the connectionTimeout, receiveTimeout and allowChunking parameters:
Config.groovy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

cxf {
    installDir = "C:/apps/apache-cxf-2.4.2" //only used for wsdl2java script target
    client {
        simpleServiceClient {
            wsdl = "docs/SimpleService.wsdl" //only used for wsdl2java script target
            wsdlArgs = "-autoNameResolution"
            clientInterface = cxf.client.demo.simple.SimpleServicePortType
            serviceEndpointAddress = "${service.simple.url}"
            namespace = "cxf.client.demo.simple"
            receiveTimeout = 0 //no timeout
            connectionTimeout = 0 //no timeout
            allowChunking = false
        }
    }
}
#### HTTPClientPolicy Parameter #### Here is an example cxf client configuration block using the httpClientPolicy config parameter:
resources.groovy 
1
2
3
4
5
6
7
8

beans = {
    customHttpClientPolicy(HTTPClientPolicy){
        connectionTimeout = 30000
        receiveTimeout = 60000
        allowChunking = false
        autoRedirect = false
    }
}
Config.groovy 
1
2
3
4
5
6
7
8
9

cxf {
    installDir = "C:/apps/apache-cxf-2.4.2" //only used for wsdl2java script target
    client {
        simpleServiceClient {
            clientInterface = cxf.client.demo.simple.SimpleServicePortType
            serviceEndpointAddress = "${service.simple.url}"
            httpClientPolicy = 'customHttpClientPolicy'
        }
}
[1]: http://www.grails.org/plugin/cxf-client (Grails Cxf Client Plugin) [2]: https://github.com/Grails-Plugin-Consortium/cxf-client]]> <![CDATA[Grails Cxf Client 1.2.7 Released]]> 2012-02-15T15:00:00+00:00 http://www.christianoestreich.com/2012/02/grails-cxf-client-http-client-policy The Grails cxf-client plugin has been updated to version 1.2.7. I have added the ability to specify timeouts (connection and receive) and allow chunking separately through params and also provided another parameter to pass the client name of an HTTPClientPolicy bean to use instead of setting the params seperately.

Updated Parameter Documentation

The following partial documentation is available in full at https://github.com/Grails-Plugin-Consortium/grails-cxf-client

Property Description Required
connectionTimeout Specifies the amount of time, in milliseconds, that the client will attempt to establish a connection before it times out. The default is 30000 (30 seconds). 0 specifies that the client will continue to attempt to open a connection indefinitely. (default: 30000) No
receiveTimeout Specifies the amount of time, in milliseconds, that the client will wait for a response before it times out. The default is 60000. 0 specifies that the client will wait indefinitely. (default: 60000) No
allowChunking If true will set the HTTPClientPolicy allowChunking for the clients proxy to true. (default: false) No
httpClientPolicy Instead of using the seperate timeout, chunking, etc values you can create your own HTTPClientPolicy bean in resources.groovy and pass the name of the bean here. This will override the connectionTimeout, receiveTimeout and allowChunking values. (default: null) No

Timeout And Chunking Parameters

Here is an example cxf client configuration block using the connectionTimeout, receiveTimeout and allowChunking parameters:

Config.groovy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

cxf {
    installDir = "C:/apps/apache-cxf-2.4.2" //only used for wsdl2java script target
    client {
        simpleServiceClient {
            wsdl = "docs/SimpleService.wsdl" //only used for wsdl2java script target
            wsdlArgs = "-autoNameResolution"
            clientInterface = cxf.client.demo.simple.SimpleServicePortType
            serviceEndpointAddress = "${service.simple.url}"
            namespace = "cxf.client.demo.simple"
            receiveTimeout = 0 //no timeout
            connectionTimeout = 0 //no timeout
            allowChunking = false
        }
    }
}

HTTPClientPolicy Parameter

Here is an example cxf client configuration block using the httpClientPolicy config parameter:

resources.groovy 
1
2
3
4
5
6
7
8

beans = {
    customHttpClientPolicy(HTTPClientPolicy){
        connectionTimeout = 30000
        receiveTimeout = 60000
        allowChunking = false
        autoRedirect = false
    }
}
Config.groovy 
1
2
3
4
5
6
7
8
9

cxf {
    installDir = "C:/apps/apache-cxf-2.4.2" //only used for wsdl2java script target
    client {
        simpleServiceClient {
            clientInterface = cxf.client.demo.simple.SimpleServicePortType
            serviceEndpointAddress = "${service.simple.url}"
            httpClientPolicy = 'customHttpClientPolicy'
        }
}
]]> <![CDATA[Grails Redis Plugin Memoization Annotation Transformation (A Retrospective on Groovy AST)]]> 2012-02-03T17:00:00+00:00 http://www.christianoestreich.com/2012/02/groovy-ast-transformations-part-1 Recently while developing a prototype application for performance testing using redis and jesque (see post) I got to thinking “why doesn’t the grails redis plugin currently have support for annotation based memoization like spring cache?” I figured I would set out to add that support and learn to write Groovy AST Transformations at the same time. The process was long and arduous and I learned a lot during my several weeks of coding.

First Steps

I started off by reading articles on AST and found many great examples of existing AST transformations, blogs, and tests already written that I could leverage. Here are some if you are looking for additional resources:

There are a lot more out there in the world and you can find plenty here.

The End Goal

The end goal was pretty simple; to transform an annotated method like the first below during compile time into something that looked like the second method below at runtime. Using an annotation with a key, expire, etc. and injecting code that would wrap all the method contents into a call to an appropriate redisService.memoize method. I would then create memoize annotations for each type of memoize to be performed (domain, list, hash, set, etc).

Turn this:

1
2
3
4

@Memoize(key = '#{text}')
def method(String text, Date date) {
    return "$text $date"
}

Into this:

1
2
3
4
5

def method(String text, Date date) {
    return redisService.memoize(text) {
        return "$text $date"
    }
}

Start Coding – Issue 1

Getting started was easy. I created the annotation class Memoize.groovy and the actual transformation MemoizeASTTransformation.groovy as follows:

1
2
3
4
5
6
7
8

@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(['grails.plugin.redis.ast.MemoizeASTTransformation'])
@interface Memoize {
    Class value() default {true};
    String key() default '';
    String expire() default '';
}

1
2
3
4
5
6

@GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION)
class MemoizeASTTransformation implements ASTTransformation {
    void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {
        println 'in transformation'
    }
}

Easy enough right?! When trying to decorate a service method in the project with the @Memoize method I ran into my first big hurdle… it didn’t work. I scratched my head for a long time. Errors during compile? No. Printing ‘in transformation’ during compile? No.

I will cut to the chase and give you the solution to this issue. Having another project using the AST annotations that pointed to the grails-redis plugin (AST annotation source) inline fixed this issue. Why? Well it appears that the AST Transformation classes themselves are compiled along with everything else so during the actual CompilePhase.CANONICALIZATION there are no ASTs available to apply yet since they aren’t compiled. When a project was consuming the source as a plugin, it would compile the plugin code first and then compile the new project and the correct statement would print out. I am not sure if I was doing something wrong or if there is another way around this, but for me this solution worked like a charm.

Write More Code – Issue 2

I was able to move beyond those issues and start adding a bunch of code to the MemoizeASTTransformation.groovy class. Everything was humming along nicely. On a side note, I did have a bit of a hard time deciding between using the AstBuilder or the more verbose statements and expressions to build up the code and opted for the more verbose usage of the direct Statements and Expressions.

I had built up a fair amount of AST code that seemed to be working as expected until it came time to create the method closure to hand to the redisService. If we look at the redis service method definition we can see that it takes a closure as the code to execute if the key isn’t found.

1

def memoize(String key, Map options = [:], Closure closure)

There is a very convenient ClosureExpression available in the code to create a closure block of code and it’s usage is rather simple. The following two blocks of code attempts to create the arguments for the memoize method, including the closure, and wrap the existing method code of the annocated method in the closure. There is a lot to the code here that I am not covering, but I don’t want to dumb down the code too much in explaining the issue so I can try and help you avoid it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

protected void addRedisServiceMemoizeInvocation(BlockStatement body, MethodNode methodNode, Map memoizeProperties) {
    ArgumentListExpression argumentListExpression = makeRedisServiceArgumentListExpression(memoizeProperties)
    argumentListExpression.addExpression(makeClosureExpression(methodNode))

    body.addStatement(
            new ReturnStatement(
                    new MethodCallExpression(
                            new VariableExpression('redisService'),
                            new ConstantExpression('memoize'),
                            argumentListExpression
                    )
            )
    )
}

protected ClosureExpression makeClosureExpression(MethodNode methodNode) {
    ClosureExpression closureExpression = new ClosureExpression(
            [] as Parameter[],
            new BlockStatement(methodNode.code.statements as Statement[], new VariableScope())
    )
    closureExpression.variableScope = new VariableScope()
    closureExpression
}

This code failed and failed and failed to run (but did compile) in many different rewrites. It usually errored at runtime telling me that some variables I referenced in the parent method like $text was not in scope in the closure. I figured from some earlier troubles with VariableScopes that It was something to do with how the variable scopes were inherited. I could also see a drastic difference in the decompiled code using JD-GUI in that the closure created didn’t pass into themselves the variables used inside the actual method call.

Again to save headache I discovered, through a miracle I think, the following code:

1
2
3
4

VariableScopeVisitor scopeVisitor = new VariableScopeVisitor(sourceUnit);
sourceUnit.AST.classes.each {
    scopeVisitor.visitClass(it)
}

Running this at the end of the main visit method caused all the variable scopes to be correct propagated down to the newly created objects I was using, including the ClosureExpression.

Having inspected the source, I can best describe what it does as aligning the correct variable scope inheritance in your class. There is an explicit visitClosureExpression method that does the magic and injects the scoped variables into the closure. Following code that uses the visitor pattern is a bit tedious at times, but that is the best I can surmise from digging through that code. Here is a snippet from the VariableScopeVisitor.java class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

public void visitClosureExpression(ClosureExpression expression) {
    pushState();

    expression.setVariableScope(currentScope);

    if (expression.isParameterSpecified()) {
        Parameter[] parameters = expression.getParameters();
        for (Parameter parameter : parameters) {
            parameter.setInStaticContext(currentScope.isInStaticContext());
            if (parameter.hasInitialExpression()) {
                parameter.getInitialExpression().visit(this);
            }
            declare(parameter, expression);
        }
    } else if (expression.getParameters() != null) {
        Parameter var = new Parameter(ClassHelper.OBJECT_TYPE, "it");
        var.setInStaticContext(currentScope.isInStaticContext());
        currentScope.putDeclaredVariable(var);
    }

    super.visitClosureExpression(expression);
    markClosureSharedVariables();

    popState();
}

I believe it is the line:

declare(parameter, expression);

which appears to do a lot of the magic in defining and adding the variables to the scope. The declare method ultimately calls

currentScope.putDeclaredVariable(var);

for each variable in the n – 1 scope, in our case the parent of the closure.

This relatively short issue and code solution took a big chunk of my free time in December and early January. Time I can only hope to save someone else in the future when doing this type of closure wrapping transformation.

It Works… Almost – Issue 3

When defining a key property string in the annotation the use of the $ character is not allowed as that will reference an ACTUAL GString at compile time. We want the value to represent a GString to get interpreted at runtime. I opted to have the users provide variables in the format #{var} leveraging the # sign as a replacement for the $ character.

With that issue resolved it was time to tackle passing the GString variable into the annotation closure as the memoization key. That went awry very quickly as using a GString gets a little tricky when a user is passing in compound GStrings. Representing the key as an expression $variable might be easy, but using something like ${key}:string${key2.prop}:value2 was proving to be hard as you would have to split and iterate all the GStrings vs non-GStrings in that statement.

The following is some sample code from the codehaus AST builder tests illustrating the typical usage of a GStringExpression in both builder and regular code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public void testGStringExpression() {
    // "$foo"
    def result = new AstBuilder().buildFromSpec {
        gString '$foo astring $bar', {
            strings {
                constant ''
                constant ' astring '
                constant ''
            }
            values {
                variable 'foo'
                variable 'bar'
            }
        }
    }

    def expected = new GStringExpression('$foo astring $bar',
            [new ConstantExpression(''), new ConstantExpression(' astring '), new ConstantExpression('')],
            [new VariableExpression('foo'), new VariableExpression('bar')])


    AstAssert.assertSyntaxTree([expected], result)
}

I went down the path of trying to parse the user provided string into a meaningful GStringExpression with little to no luck using complex/compound strings.

I figured if I could combine the AstBuilder.buildFromString{} method I might be able to use the statement(s) it generated and inject them into the code while letting the AstBuilder do the work in creating the GStringExpression. After some tweaking, this is what I came up with, and it worked!

1
2
3
4
5
6
7
8
9
10

protected void addRedisServiceMemoizeKeyExpression(Map memoizeProperties, ArgumentListExpression argumentListExpression) {
    if(memoizeProperties.get('key').toString().contains('#')) {
        def ast = new AstBuilder().buildFromString("""
            "${memoizeProperties.get('key').toString().replace('#', '$').toString()}"
       """)
        argumentListExpression.addExpression(ast[0].statements[0].expression)
    } else {
        argumentListExpression.addExpression(new ConstantExpression(memoizeProperties.get('key').toString()))
    }
}

The real guts of this is in the following statements.

1
2
3
4

def ast = new AstBuilder().buildFromString("""
    "${memoizeProperties.get('key').toString().replace('#', '$').toString()}"
""")
argumentListExpression.addExpression(ast[0].statements[0].expression)

Combining the expression built from the AstBuilder with the ArgumentListExpression married well together and I was happy that the solution ended up being so simple. At one point I think the parsing, looping, etc. logic was nearly 100 lines of frustrating and non-working code.

Up Next

Next week I will post a more concise list of AST-isms that I feel are important when working with the code and things to be mindful of when coding. It may end up looking similar to what Graeme posted on lessons learned. I think the pain I experienced at least warrants a list of things to do/avoid next time around. Stay Tuned!

Code

The memoization AST transformation code I am referencing here is available at github.

]]> <![CDATA[Grails Performance Framework]]> 2012-02-02T14:12:00+00:00 http://www.christianoestreich.com/2012/02/grails-performance-framework I built a relatively simple framework to do performance testing of code using redis, jesque and executor. It uses some jQuery and ajax on the front end and provides a relatively simple administrative UI. The code is currently availabe on github. The overhead of the perf runner is relatively light and extremely fast as it is using redis as its storage mechanism for queues, jobs, statistics, and status. If you would like to change where or how the results are saved you could certainly hack the code in the ResultsService.

Update 2/7/12

I have made some slight changes to the framework and have updated this post and docs to illustrate the changes. More specifically I have moved the Result interface and Result inheritors into the src/groovy directory as domain object support isn’t currently required and was causing a bit of an issue during jsonification.

Detailed Description

Using a web admin console, configured jobs can be submitted via ajax to a queue in jesque with a type (workerClass) and number of threads to run. A jesque worker will then pick up the queued jobs and spawn off a number of worker threads using executor. These threads will continue to run until the user then stops the job via the admin console. Stopping the job is essentially flipping an active flag for the job to false in the datastore (redis). Ther results will be updated near-time on the screen using a custom ajax enabled jQuery widget that polls and aggregates the data from redis. These operations are extremely fast due to the speed of redis.

The admin console provides the following functionality:

  • View paged resultset of successful & error results per job
  • List all keys in Redis
  • Clear all results
  • Clear all data (Flush Redis Database)

See the sequence diagram below for a UML view of how the system operates.

There may be a slight delay in the updating of statistics both when starting and stopping the tests as the jQuery widget only updates stats every 5s.

Create A Job

To create a new performance job simply create a new service in the services/com/perf/runners directory. I have 3 sample jobs created in the base project. It is important that you extend the AbstractPerformanceService and implement the performTest method.

The abstract service contains a benchmark method that accepts a closure which you should use to encapsulate the code you wish to time. The benchmark method will return the time in ms the closure took to execute.

1
2
3
4
5
6
7
8
9
10
11

class LargeNumberPerformanceService extends AbstractPerformanceService {
    Result performTest() {
        Long result = 1
        def executionTime = benchmark {
            100000.times {
                result += it
            }
        }
        new SimpleResult(testName: 'Long Number Performance Service', executionTime: executionTime)
    }
}

This method performTest must return a Result interface as this will be used to log into the results data cache in redis. I have two types of results, SimpleResult and ComparisonResult, that both inherit from Result and are demonstrated in the project. Users are free to add their own types of result objects if they need more fields.

1
2
3
4
5
6
7

 interface Result {
     String testName
     String details
     Integer executionTime
     Date createDate
     Boolean isError
 }

1
2
3
4
5
6
7
8
9

@ToString(includeNames = true, includeFields = true, excludes = "class, id")
class SimpleResult implements Serializable, Result {

    String testName
    String details
    Integer executionTime = 0
    Date createDate = new Date()
    Boolean isError = false
}

1
2
3
4
5

@ToString(includeNames = true, includeFields = true, excludes = "class, id")
class ComparisonResult extends SimpleResult {
    String result1
    String result2
}

Result Interface Fields:

testName - This can be an abitrary value that you would like, simply used when displaying detailed results view.
details - If you would like to put any specific output from the test like facts, figures, etc. you can do so in the details field.
executionTime - This should be the value of the result from the benchmark method.  You could roll your own timing schema and put that value here.
createDate - The date, defaulted to now, simply used when displaying detailed results view.
isError - This will cause the result to be logged into the error queue.  You should set this if an error condition occurs perhaps in a try catch or when unexpected results are reached.

Here is an example of a test the may set the error flag if the results are empty or an exception occurs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Result performTest() {
    String quote = ''
    Boolean isError = false

    def duration = benchmark {
        try {
            quote = stockQuoteClient.getQuote(randomStock)
        } catch (Exception e) {
            isError = true
        }
    }

    new SimpleResult(details: quote, isError: (isError || !quote), executionTime: duration, testName: 'Stock Quote Performance Service')
}

The last and most important step for activating a performance job is adding the job config to the Config.groovy perf runners block.

1
2
3
4
5
6
7
8
9

perf {
    runners {
        jobName {
            description
            maxWorkers
            workerClass
        }
    }
}

The values are represented via the following:

jobName - A unique name for the job node (no spaces)
description - A name that will be used to display on the jobs screen
maxWorkers - Max number of workers available to choose on job screen.  Will be between 1 and maxWorkers in drop down.
workerClass - Points to the class of the worker to wire up to the job.

The jobName MUST be unique or you will get overlapping and/or inaccurate results.

To wire up the LargeNumberPerformanceService job above to show up in the admin console as an available job you need to add the following:

1
2
3
4
5
6
7
8
9

perf {
    runners {
        largeNumberPerformanceRunner {
            description = 'Large Number Performance Test'
            maxWorkers = 20
            workerClass = com.perf.runners.math.LargeNumberPerformanceService
        }
    }
}

Note: I have not tested a large number of jobs, but having several available to run would be okay. If you tried to run several jobs with large thread pools, you will probably experience inaccurate results as your machine struggles to keep up. Leaving the job service classes in place but simply commenting out the jobName block in the config will cause the admin console to not list the job for running and should take up no overhead while essentially disabling the job.

Simplified Sequence Diagram

Running the Application

You will need to start a redis server and make sure the Config.groovy points to it before running the application.

Once one or more jobs are configured and redis is running you can run-app and navigate to localhost.

Admin Console

The main admin console view:

Clicking the start button and using ajax to collect some results:

A view of the list all keys feature. Some keys are clickable if they contain more details of data to view:

A detailed view of a success queue:

PerformanceRunnerJob Class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

class PerformanceRunnerJob {

    GrailsApplication grailsApplication
    RedisService redisService
    ResultsService resultsService
    def executorService

    def perform(jobName, workers) {
        println "jesque queueing up job ${jobName} with ${workers} threads"
        Class clazz = grailsApplication.config?.perf?.runners[jobName]?.workerClass
        if(!clazz) {
            log.error "Can not start a performance worker without a workerClass defined in the config attribute"
        }
        PerformanceService service = (PerformanceService) grailsApplication.mainContext.getBean(clazz)
        Integer.parseInt(workers).times {
            runAsync {
                println "running ${jobName} on thread :: ${Thread.currentThread().id}"
                while(redisService.get(jobName) == PerformanceConstants.RUNNING) {
                    saveResults(jobName, service.performTest())
                }
            }
        }
    }

    private void saveResults(String jobName, Result result) {
        log.debug result
        resultsService.saveResults(jobName, result)
    }
}
]]> <![CDATA[Grails Cxf Client 1.2.5 Changes]]> 2012-01-10T10:17:00+00:00 http://www.christianoestreich.com/2012/01/grails-cxf-client-wsdl2java-changes Based on a user request, I have added the ability to pass the wsdl2java script a wsdlArgs param in which you can use to pass custom args into the client creation process. I have also modified the name of the bindingFile parameter as the name binding was in conflict with groovy reserved keywords.

PropertyDescriptionRequired
wsdlLocation of the wsdl either locally relative to project home dir or a url. (default: “”)No
wsdlArgsA custom list of args to pass in seperated by space such as [“-autoNameResolution”,”-validate”]. This can also be a single string value such as “-autoNameResolution”, but when using multiple custom params you must specify each in a list [“-one val”,”-two”,”-three val”] due to limitations with ant. (default: “”)No
namespaceSpecifies package names to use for the generated code. (default: “use wsdl provided schema”)No
clientUsed to tell wsdl2java to output sample clients, usually not needed. (default: false)No
bindingFilePath of binding file to pass to wsdl2java. (default: “”)No
outputDirPassword to pass along with request in wss4j interceptor when secured is true. (default: “src/java”)No

Here is an example cxf client configuration block using the new wsdlArgs param.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

cxf {
    installDir = "C:/apps/apache-cxf-2.4.2" //only used for wsdl2java script target
    client {
        simpleServiceClient {
            //used in wsdl2java
            wsdl = "docs/SimpleService.wsdl" //only used for wsdl2java script target
            wsdlArgs = ['-autoNameResolution', '-validate']
            //wsdlArgs = '-autoNameResolution' //single param style
            namespace = "cxf.client.demo.simple"
            client = false //defaults to false
            bindingFile = "grails-app/conf/bindings.xml"
            outputDir = "src/java"

            //used for invoking service
            clientInterface = cxf.client.demo.simple.SimpleServicePortType
            serviceEndpointAddress = "${service.simple.url}"
        }
    }
}
]]> <![CDATA[Moving To Octopress]]> 2012-01-03T16:30:00+00:00 http://www.christianoestreich.com/2012/01/moving-to-octopress I am currently in the process of porting all the blog posts to Octopress. This is a WIP and I will try and get all the content updated ASAP.

Thanks!

]]> <![CDATA[Grails Cxf Client 1.2.3 Released]]> 2011-12-13T13:58:55+00:00 http://www.christianoestreich.com/2011/12/grails-cxf-client-custom-interceptors.markdown The following partial documentation is available in full at [https://github.com/Grails-Plugin-Consortium/cxf-client][2] To begin create the interceptor bean.
CustomLoggingInInterceptor.groovy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

package com.cxf.demo.logging
import org.apache.cxf.common.injection.NoJSR250Annotations
import org.apache.cxf.interceptor.AbstractLoggingInterceptor
import org.apache.cxf.interceptor.Fault
import org.apache.cxf.interceptor.LoggingInInterceptor
import org.apache.cxf.message.Message
import org.apache.cxf.phase.Phase

@NoJSR250Annotations
public class CustomLoggingInInterceptor extends AbstractLoggingInterceptor
{
    def name

    public CustomLoggingInInterceptor() {
        super(Phase.RECEIVE);
        log "Creating the custom interceptor bean"
    }

    public void handleMessage(Message message) throws Fault {
        log "$name :: I AM IN CUSTOM IN LOGGER!!!!!!!"
    }
}
Next you will want to wire up the interceptor bean in the resources.groovy.
resources.groovy 
1
2
3
4
5
6
7
8
9
10
11

customLoggingInInterceptor(CustomLoggingInInterceptor) {
    name = "customLoggingInInterceptor"
}

verboseLoggingInInterceptor(VerboseCustomLoggingInInterceptor) {
    name = "verboseLoggingInInterceptor"
}

customLoggingOutInterceptor(CustomLoggingOutInterceptor) {
    name = "customLoggingOutInterceptor"
}
Last you will need to define a list of interceptors in your config.groovy cxf {client {…}} block for either in, out or out fault. The following defines two custom inInterceptors named customLoggingInInterceptor and verboseLoggingInInterceptor. 
1
2
3
4
5
6
7
8

simpleServiceInterceptorClient {
    wsdl = "docs/SimpleService.wsdl" //only used for wsdl2java script target
    clientInterface = cxf.client.demo.simple.SimpleServicePortType
    serviceEndpointAddress = "${service.simple.url}"
    inInterceptors = ['customLoggingInInterceptor', 'verboseLoggingInInterceptor'] //can use comma separated list or groovy list
    enableDefaultLoggingInterceptors = false
    namespace = "cxf.client.demo.simple"
}
The flag enableDefaultLoggingInterceptors will turn on (true) the default logging interceptors. You do not need to provide this property if you wish to use the default logging as the default value is true. If you wish to turn them off simply provide the property for your client bean config and set the value to false similar to the above block. See the docs at [https://github.com/Grails-Plugin-Consortium/cxf-client][2] for more details on further use of the cxf-client plugin and additional tips for wiring up interceptors. [1]: http://www.grails.org/plugin/cxf-client (Grails Cxf Client Plugin) [2]: https://github.com/Grails-Plugin-Consortium/cxf-client ]]>