Morning musings on Grails and Cloud Foundry deployment

2009-12-08T09:02:00.000-08:00

One pain point I have with doing agile cloud development with Java is the overhead of re-deploying the app to the cloud. The default model is to re-deploy the entire WAR (or EAR) to the cloud after every change. This isn't that bad when the app server you are deploying to is nearby although even there you often end up doing a more efficient in-place update using an exploded WAR. It is pretty miserable though when you have a humongous lib directory making up the lion's share of your app footprint and it can take time and money (in bandwidth costs) to upload the entire app after a change which may involve only a few classes that are 10s of K in size rather than 10s of MB.

I am using Cloud Foundry as my default cloud deployment platform and while its currently free, the underlying platform, EC2 is not. The overhead of re-deploying a 40+ MB app usually runs to around a sec/MB for the upload and then 10-20 seconds for the app restart. This means at least 1 to 2 minutes per re-deploy. Cloud Foundry (and the underlying SpringSource tool chain) have some optimizations (existing or planned) that can cut this down such as:

CloudFoundry shared Libs: You can upload a single archive file (tar, zip) as part of your application specification and this will be unpacked in the app server shared libs directory. I tried to use this initially but gave up after running into many classpath and shared state errors in the logging and GSP libraries. I may revisit this when I start using tc server rather than stock tomcat in my local environment. The reason being that the shared lib errors may be specific to tc server since I don't have a problem with them when using stock tomcat in my local environment.
shared-libraries-war: This feature is available for apps that are deployed to SpringSource's DM server (their packaging of an OSGi server). AFAIK, this is not yet available and its not clear how you would indicate what parts of your app should be included in the shared vs app-specific modules that make up the PAR.
Standard JAR stripping: There are multiple mentions of the ability of SpringSource tools (see spring-3-with-maven for an example) to strip standard JAR from the WAR on the client-side and then "re-constitute" them on the server-side. I haven't seen this work in practice as the entire WAR is uploaded AFAICT based on the fact that there is no noticeable reduction in the upload time. Even if it does work, the standard JAR make up only a subset (often less than a quarter) of the app footprint and the ideal solution would strip all the JAR leaving the equivalent of the "nojars" option of the Grails war command. I'm also not clear on whether standard JAR equate to libraries that are part of the Enterprise Bundle Repository (EBR) or whether they equate the to broader set of libraries that are part of the Grails distribution.

I'm wondering what, if any issues there would be with extending Cloud Foundry to have the concept of a "nojars" redeploy. The idea would be that if the app signalled that it wanted to do a "nojars" redeploy, Cloud Foundry would run the Grails war command with the "nojars" option, upload the WAR to S3 and then to the instance(s) and update the app on the server in a way that keeps all of the JAR from the initial deploy. I don't see any obvious gotchas and this would avoid all of the sticky issues with shared libs. Ideally, you could even have the option of doing a lighter weight redploy which is currently really a restart of the app server.

Another more general approach could be extending Cloud Foundry to fire Grails events at relevant points. To make this really work, there would have to be both client-side and server-side events and more work on defining the server side event model. In this model, you might have events like DeployOnClientStart, DeployOnServerStart, RedployOnClientStart, RedployOnServerStart and these would be defined similar to CreateWarStart and CreateWarEnd. On the server-side, Cloud Foundry would need to be extended to set up the execution environment and fire the events. There is likely an existing alternative way to do server-side deployment scripting that I am not aware of. If not or if a more Graily way makes sense, then maybe this could be considered?

Types of Plugins based on level of integration in Grails

2009-12-06T21:49:00.000-08:00

Some quick notes on distinctions between Grails plugins. One useful cut is by how integrated the plugin is into Grails. There are the following levels of integration that I see from most to least integrated:

Internal Core plugin: this type is part of the Grails core build. Loaded statically at startup by CorePluginFinder
External Core plugin: maintained as part of the Grails distribution but installed using scripts and opt-in. See Grails plugins for list.
Supported plugins: Superset of the external core plugins that is maintained by SpringSource.
Central repository plugins: grab-bag of plugins that are maintained according to a loose set of conventions and accessed via a list at Grails.org
User repository plugins: plugins that are accessed via a user repository specified in BuildConfig.
Local plugins: plugins that are accessed using "grails.plugin.location"

Gabe's Groove

Morning musings on Grails and Cloud Foundry deployment

Types of Plugins based on level of integration in Grails