Ivar's rambling

Rusting, a quick dabble with the Rust language

noreply@blogger.com (flurdy) — Sat, 15 Apr 2017 00:12:00 +0000

I have spent a few spare moments in the last week looking at Rust, a relative new language.

With the kids on Easter break and naturally requiring frequent attention, and a few brief actual paid client work, I have not had as long uninterrupted time focused on Rust as I would have liked. But I have managed to knock together some basic code.

Rustic Pizza

I created github.com/flurdy/rustic-pizza as my playground with rust and various web frameworks. It is very basic code and I expect I will be quite embarrassed by it in just a few months but it is a start. Rustic pizza will eventually contain several very basic web applications each in a different Rust web frameworks, all for ordering pizza. The pizzeria is a concept that I have created many times over in Java and Scala as an example for piece of framework or similar, some of which I have documented on my flurdy.com website.

Experience so far

So far I have found Rust interesting.

It perhaps use a bit too much of lazy abbreviated names for its keywords, primitives and core methods than I feel is reasonable, but that is me being pedantic. The ownership transfers, lifetimes and everything by reference is a bit of headfuck, but I think I got the hang of it. Rust does also have some of the monadic traits I am comfortable with from the Scala world, though its Futures seems to be somewhat in its infancy at the moment.

Install Rust

Rust can be installed in numerous ways, including via Brew for both Rust and Cargo, its build tool. But I recommend using Rustup, rustup.rs. Rustup lets you toggle between release and nightly version of Rust, and some frameworks require nightly builds.

Learning Rust

The main Rust website, rust-lang.org, is a good place to start to learn about Rust. Especially the "Rust book", https://doc.rust-lang.org/book/README.html, is a great source to learn about basic Rust concepts. Further reading at rustbyexample.com/index.html and aturon.github.io was very useful. As well as various blogs on Rust such as hoverbear.org and hermanradtke.com.

Once you start coding I also recommend the standard library documentation, doc.rust-lang.org/std/, to find out what methods are available, e.g. does option have map, and what is getOrElse called in rust (unwrap) etc.

Web frameworks

As I most of the time create web services or full webapps an important part of my interest is which web frameworks are available. Fortunately the Rust community also recognise the importance of this, so www.arewewebyet.org and github.com/flosse/rust-web-framework-comparison are great information on which frameworks and libraries are available with Rust.

This lead me to take a look at the newest kid on the block Rocket, rocket.rs. So far Rocket have worked very well for me.

For my Rustic Pizza I plan to also take a look at Iron, http://ironframework.io, as it is the most popular Rust web framework on github. Though it has not been updated as much lately compared to Rocket.

Hopefully if time allows I may get to look more in depth into Nickel, Conduit and Rustful as well.

Keep rusting?

Whether I will keep using Rust time will tell. Core Rust concepts such as ownership, references, memory management etc are not aspects I care to manually control and think about too much as I hope a compiler and garbage collector optimises and handles those for me. But I will keep at it so maybe Rust will accompany if not replace Scala as the main hammer in my toolbox some day.

ROC theorem: Readable, Optimised and Correct code. Pick two.

noreply@blogger.com (flurdy) — Tue, 22 Nov 2016 14:35:00 +0000

ROC theorem

With database you have the well known CAP theorem. Consistency, Availability and Partition Tolerance. You can only have two. Databases have to make compromises between these pillars. You can not fully have all three.

With code you also have to decide on compromises between readable code, optimised code and correct code (ROC). And you can not have all three.

This often creates arguments between people on soap boxes from the various camps.

Correct code

Correct code, clever, terse, generic code that avoids handling a lot of edge cases. Often functional code that can be very elegant with little to no theoretic side effects. And easily composed as part of other code.

Correct code can be readable and fast, but also sometimes horrible to understand and very costly to train, write and maintain.

Optimised code

Optimised code, fast and scalable. No unnecessary cruft and takes short cuts to achieve the end results so performant from day one.

Scalable code, code you optimise to support horizontal scalable solutions, with little state and restartable.

May discourage typed system e.g. a message based Actor system, multiple layers of caching, or overuse of parallelised futures to avoid bottlenecks.

Optimised code can be "correct" code but often full of unclear and undocumented short cuts and frustratingly slow/buggy to develop.

Readable code

Readable code, simple to read and quick to understand, by people of different levels of skill. Easy to spot bugs and is maintainable by anyone. It is pragmatic in its approach and quick to develop. Can be terse if it is the most readable but often more verbose.

Readable can be "correct" as flow is easier to understand, but often not particularly performant and can be at more at risk of bugs due to more exposed code.

Not mutually exclusive

You can have all 3 pillars for some smaller sets of code. But not for whole code bases and at a cost for how much.

This is more about the priority and focus of the code you write.

Will others work on the code base, today, next week, next year? Then readable is important.
Is multiple people working on the same part of the code base? From different teams? With varied experience, or even just different locations? Then readable is important.
Is the solution used by millions? Does big O make any difference? Then consider optimised code. Note: very, very few companies/products actually need this.
Is a bottleneck been proven in production? Then optimised code is valid. But not necessarily across the code base.
Is the product business critical? With heavy integration dependants? Then correct code may be a priority.
Is the team highly skilled? Not that large, and low churn? Then correct code is an option.
Do you pair program 100%, from day 1 of onboarding? Then correct code is an option.

With unlimited time to implement and continuously heavy training, and therefore a lot money, you may achieve higher levels of 2 of them or even all 3. But that is not realistic. Pick your priorities. These are not mutually exclusive but they are at a cost of each other.

All the ROCs

You may detect my preferences. I prefer very simple and readable code, that is functional, and scalable. In that order.

I like that anyone can pick up and work on a task for most parts of a code base. I subscribe to the idea of frequent pair rotation across tasks and systems to make sure multiple people is aware of and had an input into any part of an architecture. That leads towards readable code so the overhead of swapping is low.

That a new member of our team or from another team in 6 months time can easily contribute to "our" code base for a small pull request without learning "our take" on category theory is valuable.

I prefer functional programming, with proper type checking, using functors and monads for composition. I like terse code that I can trust, but it must still be readable and maintainable by others than the original author(s).

So some overuse of higher kinded types, free monads, etc adds too much cruft for me, and risky recruitment demands. (At the moment, I am prone to evolve and may have completely changed my mind by the time you read this...)

Horizontally scalable, concurrent code is in the back of my mind of most of solutions. No state, using futures, REST principles etc are core to all my code.

But I detest premature optimisation. Only occasionally in my career have I had to modify any code to support some optimised flow. I do not work for Facebook/Twitter/Google (yet) but I have worked for financial, telcos, and games companies with enormous traffic, and still this was rarely a problem at code level.

I often spot potential optimisations and consciously say no, not yet, if it is not also the most readable and correct alternative. I even avoid parallelisation of futures if there is not yet any obvious need especially if it makes the code less readable.

Readable, Optimised and Correct code. ROC. Pick two. Or just rank them.

Evolve or wither slowly

noreply@blogger.com (flurdy) — Tue, 25 Nov 2014 13:59:00 +0000

3 years ago (wow that went fast!) I wrote a blog post called Do not rewrite. The main points in it was to never rewrite whole applications. Never. Instead gradually rewrite one feature at the time whilst still delivering new business value.

I also wrote a more recent post about Paper cuts and broken windows. Which emphasises the importance of fixing problems straight away and not let them fester.

I want to extend these posts to include you should rewrite all the time, even when not immediately necessary.

Recap

Extracting the main points in the previous blogs why big rewrites are bad and continual rewrites are good, and to also sum the obvious why no writes at all are very bad.

Big rewrite disadvantages

Cost

No business feature - no value

Risk

Big deployments
Forgotten features

Likely to never finish
Not fully replacing old system

Support yet another system

Continual rewrite benefits

Reduced risk

Smaller delta

Quicker feedback
Modularising

Faster - less bottlenecks

Leaner

Remove features and bloat

Actually finishing

No rewrite disadvantages

Death by thousand paper cuts

Even the smallest change becomes slow and painful

Broken windows

People do not care if they introduce bugs or break other things

Staff exodus

Few wants to work with painful systems

Less obvious benefits

There are other perhaps unexpected but important benefits not raised in the previous post.

Business domain knowledge

Every time a refactor or minor rewrite takes place the knowledge of that part is refreshed and stay current in the company. If a system is not touched for a long while the knowledge of it may be forgotten especially if staff involved has moved on.

If a critical bug or urgent feature needs to be added to that system then the turn around is exponentially different between a recently updated system and one forgotten.

Technical knowledge

This is also true for the technical part of a system, the how, not just the what and why it does something. An old COBOL or complicated legacy custom build Java application will take a long time to work out compared to a contemporary tech stack application. You might even have to hire new staff or expensive contractors to fix it.

With up to date knowledge it will be a lot less risk that they might not do it properly.

Quick turnaround

If a new feature needs to be added then to any system then getting people ramped up and delivering it will be much quicker if it is on a contemporary and well known tech stack. If a new system needs to be integrated with existing systems then the API integration will be smoother and a lot quicker.

Technical migration

If a mass migration of a technology stack is needed, perhaps migrating from in house data warehouse to a public cloud based provider, or moving from monolithic to horizontally auto scaled instances, then having most systems on a contemporary and probably quite similar technology stack will speed any urgent migrations. This will avoid/reduce the need to for many deep cave explorations of old customised mystery legacy applications.

Staff retention

A very important reason to rewrite applications and features and in general keep technology up to date is staff retention. (Obviously a million other reasons exists for staff churn as well).

If you avoid death by a thousand paper cuts and broken windows staff will not mind working on the products. A negative culture will be less likely.

If you migrate to newer technologies and let people frequently learn new stuff they will be much more interested in the work and less likely to want to move elsewhere. (refer to my blog post Peak interest - the learning and sharing curve).

Recruitment

If word spreads that you keep an up to date and interesting technology stack then hiring new members of staff will be a lot easier, and you will hopefully attract more qualified candidates. On boarding will also be quick and people will be up to speed quicker with newer well known technology.

Lean technology - lean organisation

If the applications are continually refactored and evolved (as long as it is not done in a hacky ninja rock star development fashion) the architectures will become modularised, scalable and leaner in general.

Not necessarily a micro service architecture but a lean architecture that are more likely to be adaptable for the future.

Hopefully a lean architecture and highly skilled retained staff will also lead to / need a lean organisation so the business is more likely to succeed.

Rewrite when not needed

I would also suggest people and organisations refactor and rewrite when they do not see the obvious reasons for it.

I do not entirely mean rewriting applications on a whim when there is no features to be added or bugs to be fixed, after all I do emphasise the need for delivering business value along with all rewrites.

But I do think even when the obvious pain is not there that you should try to refactor and update applications. For example if the application is just one or two version behind the latest but still seems fine then still update it. It will be less painful than when it next time is 3-4 version behind with a bigger delta of change.

Rewrite encapsulation

One type of no business value rewrite I would suggest is a good idea is to encapsulate legacy systems API as soon as you come across it.

Even if you are not changing a legacy system but merely reading data from for example, then adding a contemporary facade to it straight away will be of value in the future when you do need to update it.

Experiment and rewrite experiments

Another important staff retention technique on top generally keep up to date with technology is to let them experiment with new technology.

I would avoid core systems, especially customer facing systems, but internal tools is prime candidates for field testing technology.

This will also lead to discovering technology that you can use in other applications if proven useful that otherwise would have been missed or delayed.

Naturally these experimental applications should also not be abandonware and be rewritten as often as other systems. Though they will probably always be good candidates for next set of technology experimentations.

Rewrite exploration and euthanasia

Certain system never comes involved in new features so will not be included in a normal rewrite. These will consciously have to be found and rewritten without business value. Though probably just as a contemporary facade initially.

Leaving these as abandonware is not a good idea. Either kill them or update them.

Summary

The main points was already covered in my previous blogs, but I hope people see the value of rewriting frequently to especially keep staff, organisational architecture and company in general up to date for whatever happens in the future.

A utopian expectation of all applications being up to date is too much to hope for. At the other end where companies that rarely and/or minimally update their applications, they are doomed to fail. The grey scale in between decide whether companies will wither and eventually die or survive.

Don't download the internet. Share Maven and Ivy repositories with Docker containers

noreply@blogger.com (flurdy) — Fri, 21 Nov 2014 14:09:00 +0000

Downloading the internet. A term commonly used when building Maven projects. It is due to downloading your applications dependencies and their transitive dependencies. With a normal application those dependencies can add up to quite a few jars and a lot of megabytes to download. Especially on a 'clean' machine without cached dependencies in your local repository.

Running a new container in Docker sometimes feels the same as every image layer it is built on top of also has to be downloaded. Eventually these are also cached in the Docker cache.

If you then have a Maven based application in Docker you have the perfect storm of bandwidth hogging. Especially as your Docker image's Maven configuration will by default not reuse any cached dependencies and instead always download everything from Maven Central. And for each new build or launch it will re-download the internet as it knows of no local cached dependencies.

Work around, not solution

My work around is to mount my local host repositories for Maven, and for its derivative Ivy, as data volumes for the Docker container. This then avoids re-downloading the internet on every further build and launch and will also reuse dependencies I most likely have already downloaded on the host.

It is not the best solution as your images should ideally not be influenced by what you have on your host machine, but it saves a lot of time, and a lot of grief.

Vagrant

When I run my docker containers inside a Vagrant instance I first mount my host repositories by adding these 2 lines to the Vagrantfile:

config.vm.synced_folder "/Users/myusername/.m2", "/home/vagrant/.m2"

config.vm.synced_folder "/Users/myusername/.ivy2", "/home/vagrant/.ivy2"

Boot2docker

Since Docker 1.3 the OSX /Users path has been by default shared with the boot2docker VM on the same path. So to share the maven repositories you need to link the /Users path to your docker user home folder.

boot2docker ssh;

ln -s /Users/mysername/.m2 /home/docker/.m2;

ln -s /Users/mysername/.ivy2 /home/docker/.ivy2

Data volume container

For easy sharing these folders between many docker containers I mount them as a data volume container and naming it ‘maven’.

docker run -d -P --name maven \

-v ~/.m2:/root/.m2:rw \
-v ~/.ivy2:/root/.ivy2:rw ubuntu

I am basing it on the basic Ubuntu image, and it will stop immediately as no process is running. That is fine, the mounted volumes will still work.

Alternatively instead of mounting the host folders you can have a persistent container with these folders exposed as VOLUME in the Dockerfile so it is shared in a similar manner.

Launch container

docker run -d --volumes-from maven \
yourapplicationimage

Obviously you probably have other options on your docker launch, but the important bit here is the ‘--volumes-from maven’ which maps all the volumes from the data volume container called maven into this container.

Now in theory all containers with Maven and Ivy based build tools such as SBT should first look at the mounted Maven and Ivy repositories for their dependencies.

Build problem

Unfortunately if you build new images frequently you will have the same problem still as the above solution is only for running containers not building. During the build stage Docker does not allow you mount any volumes. This is so it is reproducible anywhere.

In your Dockerfile you can ADD or COPY whole repositories into your image, but that will make it very bloated with a lot of irrelevant dependencies unless you somehow construct and maintain a perfect repository. (Ps. don’t do this).

Maven repository manager

One solution that makes sense in general is to add you companies Maven repository manager’s public Maven Central mirror to the image. That way at least you will only download the intranet, not the internet.

For a Maven build add a settings.xml file to your image folder with at least these settings if using Nexus:

<mirrors>
<mirror>
  <id>Nexus</id>
  <name>Nexus Public Mirror</name>
  <url>http://nexusmachine/nexus/content/groups/public</url>
  <mirrorOf>central</mirrorOf>
</mirror>
</mirrors>

Then add this to your Dockerfile:

ADD settings.xml /root/.m2/settings.xml

Note this will be overwritten if you later during the run stage mount .m2 folder on top of it, but for most that is fine as the .m2 folder usually contain a relevant settings.xml file anyway.

For a SBT build add this repositories file to your image folder with this content:

[repositories]

local

maven-local

company-repo: http://nexusmachine/nexus/content/groups/public

scala-tools-releases

maven-central

Then add this to your Dockerfile:

ADD repositories /root/.sbt/repositories

Repository manager container

A further solution is to run a repository manager as another container and have all Maven/SBT builds refer to it instead. This avoids the involvement of the host computer yet saves any network traffic as everything is localhost.

Pull down something like https://registry.hub.docker.com/u/mattgruter/artifactory/ .

docker pull mattgruter/artifactory

Configure then run and name it:

docker run -d --name artifactory \
mattgruter/artifactory

Modify the above settings.xml and repositories to refer to you locally linked artifactory name as url instead, e.g this one liner:

company-ivy-repo: http://artifactory/artifactory/repo/,
[organization]/[module]/[revision]/[type]s/[artifact](-[classifier]).[ext]
company-repo: http://artifactory/artifactory/repo/

(The example above added it as an Ivy repository which Artifactory supports).

Hopefully these tips will avoid downloading the internet too often and save a few grey hairs.

Dockerise it all - containerised addiction

noreply@blogger.com (flurdy) — Thu, 13 Nov 2014 10:59:00 +0000

TL;DR: CDD - Container Driven Development

(CDL - Container Driven Life)

Occasionally you come across new interesting technology that is easy to use, and suddenly a veil is unveiled and you become aware of so many opportunities and potential. Maybe it was the same the first time you discovered programming, or OO or FP programming, message queues or NoSQL databases, or distributed source control, or moving from IDE only to proper reproducible build tools (Ant, Maven, Gradle, SBT etc). With certain tools it becomes your favourite hammer that you carry everywhere searching for nails to hit it with.

Docker is such a revelation, at least for me. And I am hammering a wide variety of nails with it. And with the hammer-nail pattern/anti-pattern some of those nails really didn’t need hammering but it was fun.

Docker history

Docker was only revealed in March 2013 by Solomon Hykes and other people at dotCloud . It got a lot of publicity in the hacker news section of the world as people could see the possibly potential in this new tangent of virtualisation. It baked and matured through 2013 as people occasionally showed examples of applying it to their workflows, but still it was too rough for most to use.

But during 2014 especially since the summer the publicity and widespread use of Docker has exploded. Downloads from Docker’s registry hub of ready made images has exponentially rocketed sine the summer of 2014 by 1387%! Now the majority of people I talk to or follow on twitter mention Docker and show examples of how they use it. Conferences and tech news are flooded with new ways to use Docker. And projects that extend or are based on Docker are multiplying all the time.

As someone who has always been interested in virtualisation, automation, reproducibility, build tools and provisioning, introducing containers like Docker has been a welcomed evolution and a revelation. Being already heavily invested in the use of Vagrant for development and systems the migration to/ combination with Docker has been smooth.

Docker addiction step 1. Exploration

First steps are usually to try the easy hello-world-ish examples. Downloading an Ubuntu image and launching bash inside a Docker container. Basically exploring Docker and its commands, which is what I cover in my how to install, basic use patterns and many handy tips with Docker howto.

Docker addiction step 2. Imagination

My howto document also takes you into step 2 which is more about creating your own basic images. Creating a simple Apache or Nginx container with a simple volumised website. Creating images ready with a Java SDK or node.js etc. I have made some simple Dockerfiles of these type of images available at github.com/flurdy/Dockerfiles.

Then you usually also wrap a simple application of your own in an image and running it on other computers. You start to understand how and where Docker can be used.

Docker addiction step 3. Diversity

Once you got one application inside a container you start to experiment with other applications. Might take a look at other types of applications and more likely support applications such as databases inside a container.

You might start to link containers to each other such as an application to a database container and some of the other handy Docker tips in my howto.

Docker addiction step 4. Proliferation

Now you have tried a mixture of applications with Docker you might spread the use to even more core applications and more. You start to link multiple containerised applications to each other.

Replacing third party integration and old legacy applications with a container image, wiring up and switching between different versions of an application via container linking, etc.

As you start to basically recreate environments using Docker containers you might start to facilitate promotion between staging environments only using container images. And eventually production is containerised.

You are probably already sharing and discovering images at work and on the internet. Using the public Docker registry hub and/or internally with tools such as Quay.io

Docker addiction step 5. Eccentricity

When comfortable with writing Dockerfiles and fully aware of the Docker way of layers and process then you start to experiment with more unconventional Docker images.

You might create Android SDK in an image, ready made SSH tunnels to apps or databases in another. You add tools such as a local maven repository manager, local DNS servers, mail servers, etc. into containers. You realise desktop applications can actually be containerised.

Eventually tools to facilitate easier and more manageable Docker life such as Fig, Panamax, Shipyard, Dokku, etc. are maybe among your toolset.

Docker addiction step 6. Ubiquitous

By now all your core applications are run/runnable inside a container.

They link to databases, message queues, 3rd party adapters, and other smaller applications that are all also inside containers.

Your testing environments, staging environments, production environment, development environment and REPL are all running inside containers.

When you create a new application or touch an old legacy application your first step is to write a Dockerfile and containerise the application. You design applications and processes from the ground up to be containerised.

This is what I call CDD - Container Driven Development.

Possibly by now you are convinced everything belongs inside a container, so a better name is perhaps CDL- Container Driven Life.

Pair with people you like and code reviews with people you don't like

noreply@blogger.com (flurdy) — Fri, 08 Nov 2013 16:46:00 +0000

Pair with people you like and code reviews with people you don't like (misquoted from someone way smarter than me).

That sums it up very well for how I feel about both practices.

With large teams, especially if distributed or partially outsourced, code reviews can ensure code quality and are quite essential. It will allow you to share some knowledge and instill best practices. It will be a reminder for people that their code will be viewed by others so don’t take shortcuts.

However code review can also be a total bottleneck if over-bureaucratic. It will add an overhead for all work. If there are some high and mighty gate keepers that will stop you from pushing your code frequently then you have a complete velocity block. If your code is shit then fair enough, but if it is merely nit picking or just disagreements between styles then it is very costly. If however if any compulsory reviewer is not in your office, country, time zone or just very busy then that adds a large delay in the feedback loop.

Another issue with code reviews is that quite a few reviews are of low quality due to lack of context. They do not necessarily know all the discussions, history of why a piece of code works this way, project code style, etc. Especially if they are of the tainted ivory tower architect affliction.

In smaller, agile and especially collocated teams code reviews will flag issue unnecessarily late in the process. Just pair from the start instead to ensure no short cuts or dodgy code slips through, and automatically spread the knowledge. If you do not trust two of your developers combined then you do have a serious problem.

You can though in addition have small and short swarming/tripling/quadrupling sessions in front of 1 computer to look at especially important issues when they are worked on, not afterwards.

If you do neither code reviews nor pairing then you are in trouble.

Expanded from my own Hacker News comment https://news.ycombinator.com/item?id=6693064

One button - The aim - Nearly there

noreply@blogger.com (flurdy) — Sat, 31 Aug 2013 12:00:00 +0000

An aim for a development team (and company) is to achieve one button deploys of your applications to all environment.

By achieving that it involves several solutions which in total results several direct and indirect benefits for the company. Benefits such as:

Quicker releases of new features
Frequent releases of new features
Quick and painless rollback of unwanted releases
Reliable release process
Confidence in testing in different environments
Reduced risk in deployments
Predictable roadmaps with more stable velocities
Less context switching for ops and developers
Developer time focused on delivering features instead of on processes

I joined as a Tech Lead at my current company last year. Since then my team, or more appropriately our fantastic Ops/Devops, Tools as well as other teams, have in a year made great strides towards One button deploys. We are more or less there and the company is rewarded more and more with the benefits mentioned above. We have achieved this by doing the following since last year:

Move to Git and internally hosted repository server

Subversion was once great but no longer. Whilst we used git-svn for a while, we eventually fully moved all applications’ source code to Git.

With the addition of first Stash then GitLab we removed a lot of bottlenecks, restrictions and frustrations. Code could be easily referenced without having to check out. Collaboration via Pull requests for code bases you may not be comfortable with. Code review when needed. Forking and creating new projects became instant. Project discovery and general code share became very very easy.

External configuration and one binary

The main applications in my team used to use Maven profiles to build separate binaries for each environment. This was because the configuration was baked in via Maven filters.

This is not great for QA sign off as you basically have different binaries. It also slows down deployment as you have to rebuild the binary all the time. It definitely makes it difficult to rollback releases or investigate older releases as you have to rebuild an older version from scratch. And it makes it very fragile as each machine might build it differently (different JDK, broken local Maven repositories, etc)

Moving to external configuration meant we had one binary that gets promoted through all environment. The one binary gets upload to the repository manager (initially Nexus then later Artifactory). This binary is then downloaded as part of every deploy job.

Changing configuration is a configuration change and no change to the binary.

Configuration in source control and rolled out with Puppet

With external configuration we enhanced reliability by adding these configurations into source control. This also meant configuration change became very easy. Push changes for the environment to Git and it was then automatically sync and rollout via Puppet. This removed a very annoying and typo prone bottleneck.

Production configuration changes are not activated automatically but are rolled out via a single command.

Process automation, Teamcityfy everything...

We aimed to automate as many processes as possible. Our continuous integration server, TeamCity, have a number of automated builds and a whole range of manual jobs.

Testing and building a binary automatically on every check in. Automatic deployment to development servers on successful builds.

one button jobs to deploy that binary to other environments
one button to tag a release
one button to push release binaries to environments
one button to create bugfix branch
one button to rebuild databases
one button to migrate database schemas
one button to restart servers
one button to test 3rd party APIs and environments
one button to run acceptance tests
one button to smoke test environments
one button to trigger load tests (Gatling)

We did use Fabric directly for deploy tasks but Fabric is now triggered from within TeamCity jobs. Deployinator was nice but we phased it out in preference for all deploys in TeamCity for every environment.

Test separation

There was already an extensive JUnit suite of tests in the applications my team was responsible for. However they were a mix of integration tests masquerading as unit tests which slowed down development and feedback loops.

We relabeled unit test that required frameworks (Spring, Hibernate, etc) or databases as integration tests instead. That reduced our feedback loop time.

We added separate verification builds for just integration test. We separated and migrated 3rd party test to 3rd party libraries. We added stand alone acceptance and smoke tests using Cucumber, Specs2 and Selenium.

Test data

We wrote command line and web applications with Node.js and Spray that quickly create test data and tools that review test data. This sped up development testing and proper QA testing.

3rd party mock applications

Mocking out 3rd party systems or even some internal systems entirely by creating test harness applications in integration environments that pretend to be those systems have speeded up our QA process a lot.

Removed restrictions

Restricting who can commit, run certain jobs or see certain data might sound like a sensible option, but in reality it slows down collaboration. We decided we just trust the majority more than we distrust a minority.

A great benefit was that we opened up all Git repositories to everyone. Anyone can commit, although people less involved in a project prefer to use Pull requests.

Team City jobs can be run by anyone. There is an audit trail, but there has never been an issue of anyone ran a job they should not have. It is a great help if people are in meetings or similar that anyone can push code out to the next environment for example.

Production data for some applications due to PCI and CRB restrictions, production credentials and some production jobs for our critical applications are still restricted but we try to minimise this as well.

Feature toggles

Whilst it is a good practice to keep these toggles to a minimum, adding feature toggles that can be overridden via external configuration has been a good change. We can now quickly disable broken new features. We can dark release features or we can canary release features, then via Git & Puppet enable the feature and it becomes available for all.

Content migration tools

Instead of applying content directly and manually as SQL scripts, or scp/ftp/rsync we started writing tools in Play! to help create data sets of new content, then promote those through environment, cross checking which environment the data is in.

Adding scripts that interact with 3rd party portals was also helpful.

This avoid typos, avoids forgetting to run a script in an environment. And greatly speeds up data migration.

Database migration

Many of our applications use NoSQL solutions such as Cassandra and Redis as some of our flows have to handle millions of interactions, but the core data are still mostly in PostgreSQL.

Whilst we use Flyway and DBDeploy to migrate those database schemas and stub data to some environments, we do not use it all the way to production. This is one area we need to improve.

Environment creation

One element that is important and which our Ops team is just starting to roll out is an internal PAAS/IAAS solution. One button to create a new VM environment or one button to create a database, AWS SQS queue etc. Further enhancements, such as one button to clone an existing VM or database will be nice.

No button

Obviously there were other enhancements that is not really related to "One button", such as phasing out old legacy Java applications with newer Scala applications by applying Strangler Application pattern, replacing Quartz based batch jobs with Akka and Camel, measuring new features effect with AB testing, monitor applications metrics with Graphite, log analysis via Logstash, etc.

Company profitability

All these combined has made certain part of our development and release process so easy and quick. I am sure we have covertly increased the profitability of our company as we now can release quicker, more frequently, with less broken releases or bugs in general.

Future

We still have a lot of work to do. As mentioned we are not quite there with DB migration and Environment creation. We are experimenting with using Vagrant locally and promoting to environments, ZooKeeper for better configuration change without downtime, etc. Eventually we may achieve most of the ideas behind Continuous Delivery.

Heroku commands & tips

noreply@blogger.com (flurdy) — Wed, 30 Jan 2013 18:08:00 +0000

I wrote a quick howto of Heroku commands and tips I use all the time.

It covers creating instances, configurations, common add-ons such as databases & email, deployment strategies and more.

As mentioned in the document I tend to have multiple applications per project, so I always append --remote staging for example, resulting in this long deploy command:

git push staging master && \
heroku logs -t --remote staging; \
heroku open --remote staging && \
heroku logs -t --remote staging

For the eagle eyed you will have noticed the ";" which means as the log tailing never really finishes, when you notice the app is up and running you have to manually end it with control+c to proceed with opening up a browser.

Obviously these commands ties nicely into my use of Play on Heroku and the howtos I wrote on integrating both.

They are perhaps obvious commands and basic but I hope they are of use to some people.

Send email via SendGrid on Heroku using Play! 2.0 with Scala

noreply@blogger.com (flurdy) — Wed, 23 May 2012 01:09:00 +0000

If you have a Play! framework 2.0 application that you want to send email from, here are a few tips.

These tips assumes you deploy to Heroku, but other platforms should work similarly. The examples here are using Scala, but Java should work along similar lines. Finally the specifics are for the SendGrid add-on for Heroku, but other mail server providers should be fine.

First add the free option of the SendGrid add-on to your Heroku app by typing in:

heroku addons:add sendgrid:starter

Then configure your Play! app to use the mail plugin provided by Typesafe:

Add to your dependencies in the project/Build.scala file: (all on one line)

"com.typesafe" %% "play-plugins-mailer" % "2.0.2"

Then create and add these to a conf/play.plugins file: (all on one line)

1500:com.typesafe.plugin.CommonsMailerPlugin

Next configure the mail server settings. You can either add these directly to your conf/application.conf file, but I prefer to share my projects' source code, so my production settings are set via environment variables so that my username/password are not publicly available.

However for the plugin to run smpt.host must be present. Open conf/application.conf and add:

smtp.host=mock

On Heroku I append the settings to the Heroku's propriatory Procfile file. I append these settings to the Procfile to use SendGrid's servers: (all on one line)

-Dsmtp.host=smtp.sendgrid.net -Dsmtp.port=587 -Dsmtp.ssl=yes -Dsmtp.user=$SENDGRID_USERNAME -Dsmtp.password=$SENDGRID_PASSWORD

You may already have other settings in the Procfile, e.g. database URL, so be aware of the 255 char limit, and use a custom properties file instead.

web: target/start -Dhttp.port=${PORT}
-Dconfig.resource=heroku-prod.conf

The SendGrid add-on should create the environment SENDGRID_USERNAME and SENDGRID_PASSWORD variables for you.
You can verify this with:

heroku config

Finally we then create our actual application code to send email:

package notifiers
import com.typesafe.plugin._
import play.api.Play.current
import play.api.Play
import play.Logger
object EmailNotifier {
def sendMail {
val mail = use[MailerPlugin].email
mail.setSubject("Mail test")
mail.addRecipient("Joe Smith <joe@example.com>","sue@example.com")
mail.addFrom("Jon Doe <joe@example.com>")
mail.send( "Test email" )
}
def sendTestMail {
if(Play.current.mode == Mode.Prod){
Play.current.configuration.getString("smtp.host") match {
case None => Logger.debug("Email mock")
case Some("mock") => Logger.debug("Email mock")
case _ => sendMail(participant)
}
} else {
Logger.debug("Email mock")}
}
}

(In case of this posts cant parse tags replace < and > with less than and greater than tags;)

This should be all that is needed.

Play! 1.x did have a handy mock email interface for development and testing. I will try and find a suitable replacement for 2.0 and update this post when I do.

For more information

Play! 2.0: http://www.playframework.org/
Play! 2.0 docs: https://github.com/playframework/Play20/wiki
Heroku and SendGrid: https://devcenter.heroku.com/articles/sendgrid
Mail plugin for Play! 2.0: https://github.com/typesafehub/play-plugins/tree/master/mailer
Heroku environment variables: https://devcenter.heroku.com/articles/config-vars

Play! 1 & 2 command tip

noreply@blogger.com (flurdy) — Sun, 22 Apr 2012 15:18:00 +0000

If you love the Play! Framework, you might be like me and have both version 1.x and 2.x installed.

Version 1.x, in my case 1.2.4, is a well established feature rich stable version. Version 2.x, in my case 2.0, is a new radically different version, that is still in its infancy but released.

You might have both installed as you have older projects using 1.x and new in development projects using 2.x, or similar.

With both version the norm is to install them and expose the main executable as "play". So how do you differentiate between which version to use for which project?

OK, it is no rocket science but here is a quick tip on how I do it:

I have Play! 1.x installed in /usr/local/lib/play-1.2.4

I have Play! 2.x installed in /usr/local/lib/play-2.0

You can add either play folders to the your PATH, e.g. in /etc/environment:

PATH="blahblabh:/usr/local/lib/play-2.0"

But I simple add them to my .bash_aliases file:

alias play1="/usr/local/lib/play1/play"
alias play2="/usr/local/lib/play2/play"

On top of that I symlink this:

$: cd /usr/local/lib;
$: sudo ln -s play-1.2.4 play1;
$: sudo ln -s play-2.0 play2

With this setup I have to make a conscious decision whether to run Play! 1 or 2, and can switch between the two very easily.

$: play1 help;
$: play2 help

Clone local git repository to remote server?

noreply@blogger.com (flurdy) — Wed, 22 Feb 2012 12:34:00 +0000

If you have a local repository that you want to clone/copy to a remote server here is how I do it.

Perhaps you have been scaffolding, testing and initial silver bullet development and have realised the project is mature enough to share with others. Or you just want it backed up remotely.

Simple git clone local remote do not work. Destination path for git clone is always local.

What you need to do is on the remote server create a bare git repository:

remoteserver$ mkdir -p /var/local/repos/myproject;
remoteserver$ cd /var/local/repos/myproject;
remoteserver$ git init --bare;

Then add this remote to your local git repository:

localmachine$ git remote add origin \
ssh://remoteserver.example.com/var/local/repos/myproject;

Now every time you want to push to your remote server, you do a normal push:

localmachine$ git push origin master;

Although I would seriously consider using something github or gitorious instead of your own server.

Continuous Deployment via Stack Overflow

noreply@blogger.com (flurdy) — Mon, 13 Feb 2012 00:55:00 +0000

I do have an interest in Continuous Deployment/Continuous Delivery. Continuous Deployment goes beyond Continuous Integration and automatically deploys builds to production all the time, not once in awhile. It goes well with agile thinking.

With frequent deployments the delta that can go wrong is very small, the feedback is very quick and the knowledge is fresh and active with the developers so fix turnarounds are immediate. By relying heavily on automated integration testing and DevOps that provide automated deployment the result is quick and painless.

This week I answered a difficult question on Stack Overflow. The question by Emwee was for advice on how to use Continuous Deployment with multitude of inter-dependant systems.

It is a tricky question and I did not have an exact answer. My answer was more along the line of how to facilitate an easier deployment by making the dependency and coupling looser and roll-outs smoother.

I referred to Duck Typing, Feature toggles, evolution scripts, version tables, symlink stage & deploy, and referenced a Hacker News discussion on how Amazon deploys their systems, a video on how Netflix builds releases in the cloud and how at IMVU they deploy 50 times a day.

I also referred to Humble and Farley's book on Continuous Delivery.

There was another reply as well by EricMinick referring to his previous answer to a similar question. Eric does in detail describe scenarios of using promotion builds into different isolated test environments and suggest solutions that his company UrbanCode provides in relation with Jenkins.

In the end Continuous Deployment is a great evolution. But with large enterprises you need to keep your tongue straight. But it is worth the investment.

SOPA protest page and 503 redirect

noreply@blogger.com (flurdy) — Tue, 17 Jan 2012 16:14:00 +0000

Tomorrow, 18t January 2012, many websites will protest against the US SOPA and PIPA acts. For example Wikipedia will go blank in protest for 24 hours.

For those intending to blackout their own websites (or in the future intend to do something similar, e.g. a more practical "web site temporarily down message" while doing an upgrade etc, here are a few tips:

First of all Google recommend you do not simple just change your website front page to a blank page or similar. As this can have repercussions on your SEO, ie your search ranking. Read more about it in this post on Google+ by a Googler. They recommend a 503 error response instead. Which indicates the site is temporarily down.

So a simple change of index.html is not recommended. Nor is a simple redirect in the html's meta header, nor a plain http 302 redirect. All this can affect your ranking.

I recommend (if using Apache 2) to use mod_rewrite in this manner:


RewriteEngine on
RewriteCond %{ENV:REDIRECT_STATUS} !=503
Alias /stop-sopa /var/www/stop-sopa
ErrorDocument 503 /stop-sopa/index.html
RewriteRule !^s/stop-sopa$ /stop-sopa [L,R=503]

This uses Alias to another folder so that the same message can be used for several virtual hosts. It uses a custom ErrorDocument to display a human readable blackout page. And it uses RewriteRule to redirect all requests to the stop-sopa page (except for request for /stop-sopa so that you don't get an infinite loop).

If you are looking for a page to use as the blackout page, there is a nice github project page for just that. An example can be viewed here.

Play! 2.0 in IntelliJ IDEA

noreply@blogger.com (flurdy) — Wed, 11 Jan 2012 17:35:00 +0000

Play! Framework 1.x supported creating an IntelliJ IDEA project by the command: play idealize.

However while Play! Framework 2.0 is in beta that command does not work**.

So how do you get your Play! 2.0 project to open in IntelliJ IDEA? There are few different work arounds. Especially regarding integrating sbt.

However I have a quick way. For this to work you need both Play! 2.0 and Play! 1.2.x installed.

Create Play! 2.0 project:
/usr/local/lib/play-2.0-beta/play new helloworld
(I am assuming it was a Java project that you chose)

Rename project folder:
mv helloworld helloworld2

Create Play! 1.x project:
/usr/local/lib/play-1.2.4/play new helloworld

Create IntelliJ project:

cd helloworld; 
/usr/local/lib/play-1.2.4/play idealize

Move IntelliJ files to Play! 2.0 project:

cd ..; 
mv helloworld/helloworld.i* helloworld2/

Remove 1.x project and rename 2.0 folder:

rm -rf helloworld; 
mv helloworld2 helloworld

Now you can open IntelliJ and then go to file/open project, then find and open helloworld/helloworld.ipr.

There will be some issues such as libraries etc but this a good start. For further tips try these suggestions.

** As of 11th of January 2012 it is not present in Play! 2.0. I fully expect Play! to create an idealize, eclipsify, netbeansify etc as soon as 2.0 is stable.

Do not rewrite

noreply@blogger.com (flurdy) — Wed, 16 Nov 2011 12:16:00 +0000

Just don’t do it

Instead evolve

Many (99.99%) of developers continually insist that whichever application they work on needs a rewrite. As in scratch everything, redesign the architecture and technology choices and rewrite every component because the current choice and state is not ideal. I used to feel and say this as well.

Any code I have written (and even worse if written by others) which I need to update/maintain will look bad after only 6 months. After 2 years it smells of bad architecture design and logic. After 5 years the code base feels like a mess, totally unmaintainable.

This is slightly because technology has moved on, and my skills or rather preferences have evolved. But it is mostly due to that over time any application has been updated due to bug fixes and new features, and the maintainance might have been handed over to other people several times, with different skill level or understanding of the architecture. This bloats the code structure and exceeds what the original and probably quite tidy structure was intended for.

Any further maintenance is more and more costly as it takes longer and/or more people to make any changes to the application. And any innovation is muffled as inspiration is dampened.

Many companies makes a good living when other companies outsources the costly maintenance of these systems.

I used to feel the need to rewrite in every new assignment but not anymore. My advice now is: Please do not rewrite

Why are rewrites a bad idea?

No value

If you undertake a rewrite of your core application it will be very costly for your business, with no initial return on investment. Take 6 -12 months of pure losses and in the end the customer (internal or external) has not received any extra value. Most likely they will have less features as scope was reduced, but rarely any new features. Future performance gains and maintainability is a plus but not of value there and then. So it is a huge hit on fragile budgets or even liquidity of smaller companies, with little to no benefit.

This is the main reason not to rewrite, the business side gains nothing from it, it is a pure expense. Any reasonable business person would not approve it. And any future relationship will quite likely be very damaged.

Never finishing

Another risk is that the rewrite takes too long and either never really finishes or is cancelled. Making it an expensive task with no value now nor in the future.

Also rewrites that span a long time usually suffers from that the decisions made at the beginning start to look outdated and wrong before it is even in production and before any future value is gained.

Evolve

However I do not mean you should not evolve your application. Do not keep status quo either. If it is an important core application, you should evolve it, but not perform a total rewrite.

Instead do it in smaller steps while adding value and features for the customer. This way your product is cleaner and up to date, but the business side does not take a huge hit. This is also much more predictable and less volatile. Which make is less likely the project will be cancelled or worse for smaller companies which could go bankrupt.

How do you go about evolving and not rewriting? This very much depends on your type of application.

(If it is a tiny application, go ahead rewrite it. Just make sure the dependant systems are aware of this).

Refactor

On a minor level, continual process of refactoring, with larger refactoring a frequent acceptable task. This should slow down code rot, postponing the need for larger changes a while.

Modularisation

Eventually your application will need a drastic change. If by chance your original design was quite modular or less coupled with other systems you made evolving smoother. If not, try to modularise your current design.

By being modular you can still perform rewrites, but on much smaller scale, taking one module at the time. So that it is not a 12 months loss, but perhaps 1 month loss. While normal work on other modules can still continue. Much less likely to kill your project/company.

Parallel systems

Instead of a hard one day switch over from the old system to the new system, instead run two systems in parallel for awhile, the old legacy system and the new clean system.

Skinny interface adapter layer

If modularised or not, a cleaner way to evolve and replace sections of your application is to introduce an adapter layer for your interface, and keeping it skinny.

This interface adapter only relays between your system/application and other systems. When other systems only talk via the adapter you are free to change your system without affecting external systems.

More importantly you can now split your system into two systems and relay certain calls to the new system instead of the old, without external systems are aware nor affected by this.

The adapter layer is not always possible, or people are resistant to introduce yet another layer/system, but it really makes your much more adaptable in changing your own architecture, you can move elements to a new system and roll back without external costs.

No logic in adapter layer

Do not ever put any logic in interface adapter. Otherwise you end up with yet another system to maintain. Unfortunately this happens quite often when bad planning and management leads to shortcuts which adds logic in the adapter layer. And then you have just multiplied your maintenance costs. And believe me that logic is then rarely temporary.

ESB

Do not interpret this adapter layer requirement as a reason for introducing an ESB. ESB tend to just camouflage the spaghetti integration code, and often introduce above mention unwanted logic. However if you already have an ESB in place it can perform the adapter layer requirement.

Legacy system becomes adapter layer

Another method implementing the adapter layer is to use the existing system to relay calls to the new system instead of handling them themselves. Internally in the existing legacy system you have to do this anyway on each element migrated to the new system, so you can also allow external system to do this.

It is however cleaner to have a separate adapter layer. This will be easier to maintain as probably designed on newer technology and platforms. And allows you switch off the legacy system when you can. But also makes so less likely for people to just decide to use the legacy system instead.

Topic / feature

An even better rewrite strategy/evolution than per module is per topic / feature. This is an even smaller grained change and less risky. As part of more agile strategies you can move one feature at the time to the new clean system.

With an adapter layer this switching is smooth, but not restricted to an adapter layer. Without the adapter you just have more administration of changing every dependant system for each feature moved.

Feature toggles might be part of this strategy.

New feature -> new system
Update feature -> new system

Every new feature request is naturally going to be implemented on the new system. But a good method of choosing which existing features to move is choosing a feature which there is a requirement to modify. Then not touch it on the old system but instead rewrite and update it on the new system. This is then a good carrot/stick way to ensure the migration is performed. And ensures that the company receive some value for each rewrite.

Do not update old(legacy) system/application

Another very important rule is not to update the old legacy system at all when each feature/module is migrated off it. It is so tempting to make a shortcut and update both system as perhaps you have not introduced an adapter layer or instructed enough external systems to use the new system. This will kill your migration/rewrite.

For this step the leadership of your team needs to be firm about and insist on. Do not end up maintaining two systems (or 3 with the adapter layer).

Ensure legacy system delegates

The legacy system will undoubtedly internally refer to the rewritten and migrate module or feature. You need to ensure the legacy system now delegates internally to the new system. This is the only update you should do on the old system. Otherwise again you will need to maintain two systems, and run the risk of the legacy system and the new system executes slightly differently for the same task, leading to people insisting on using the legacy system.

Kill the legacy system

You need to plan and ensure the old system is eventually switched off. Otherwise you will still need to keep people and skills on how to maintain that system. It may be tempting and erroneously to leave some elements behind that should be on the new architecture.

There may be some elements not needed on new system or unrelated that is left on the old system. But probably better to even move these to another new system than keep the old system lying around and increasing the cost of maintaining that small element as well.

Kill the adapter layer?

Once you killed the legacy system, do you want to keep the adapter layer? Keep in mind you might want to move on from the new system when it starts to rot as well. However it may be tidier and less complicated to kill the adapter layer as well. I would kill it and if needed in the future reintroduce it instead.

Do not stop half way

If for some reason the migration to a new system is stopped, either due to reprioritisation, lack of progress etc, then you are stuck with maintaining not 1 but 3 systems. Many companies end up burned by this and is mostly down to not strong enough leadership/management.

So my point is to rewrite small topics/features then modules, but never the whole application. This way value is introduced along the way without a budgetary black hole.

References

Many, but some from former colleague Anders Sveen

Null is okay

noreply@blogger.com (flurdy) — Tue, 30 Aug 2011 09:59:00 +0000

Most of the bugs that you find or are reported to you in java applications are NullPointerExceptions ^[1]. NPE are rife in the beginning of a product’s life cycle, and they never go away.

The consequences are bug fixing which checks if:

input is null

if interface calls return null

if method calls return null

if properties and collections are null

So this original pizza ordering method:

public void orderPizza(PizzaOrder pizzaOrder){

  for( Pizza pizza : pizzaOrder.getPizzas()){

    kitchen.makePizza(pizza);

  }

}

Morphs into this:

public void orderPizza(PizzaOrder pizzaOrder){

  if( pizzaOrder != null) {

    if( pizzaOrder.getPizzas() != null ||

        Collection.isEmpty(pizzaOrder.getPizzas())) {

      if( kitchen != null) {

        for( Pizza pizza : pizzaOrder.getPizzas()){

          if( pizza != null) {

            kitchen.makePizza(pizza);

          } else {

            throw PizzeriaTechnicalException(“No pizza!!!”);

          }

        }

      } else {

        throw PizzeriaTechnicalException(“No kitchen!!!”);

      }

    } else {

      throw PizzeriaTechnicalException(“No pizzas in order!!!”);

    }

  } else {

    throw PizzeriaTechnicalException(“No pizza order!!!”);

  }

}

(Or alternativly many “assert pizzaOrder != null” that are useless in production)

Madness, and very messy.

Blinkered coding standard:
This happens when code ownership is not clear, when desire for clean code is not existant, when fixing bugs with blinkers on.
When developers are time constrained or scared of changing code.
When development is outsourced but authority and knowledge is not, so refactoring is avoided or not part of SLA.

Why are we so afraid of NullPointerExceptions?
What is wrong with null?

Null is okay. Null is exactly that. Don't avoid it like the plague, but try to not pass the problem to others. Tidy up your gateway interfaces, trust internally null does not happen. If it does then don't hide the problem.

Do not check for null inside you application. Check for null on your edges:

Validate user interface inputs. “Users” are always causing trouble, but we can not live without customers. Validate the essentials, but remember duck walking, don’t scare your customers with perfection validation.

Validate external 3rd party api interfaces. Probably validate internal api interfaces. Remember duck walking. If an internal system quite close to your application erroneously passes null values, fix that system instead. It is a gray zone however so some null pointer validation is fine. Just don’t complicate your system because of the laziness of another within your team/company.

Do not validate calls between your applications layers. Trust the edges has validated the data. Use unit tests to ensure integrity.

Dealing with NPE/Null?

If you encounter a Null or anything resembling empty when not possible etc: throw a NullPointerException. Don’t sugar coat and masquerade it as something else. An NPE is an NPE! It says what it is on the tin! It is easily understood by all developers, and easily traced. Obviously don’t show that to the end user, but don’t catch it too far down your stack.

If you get bug report about an NPE. Fix the edge validation, factory generation or inside the api call. Do not add a “== null“ condition inside your application.

It is easier to avoid internal NPE by not passing external DTO and domain objects through your own application. If you act on it, it should be your applications objects, not an external one. This avoids complicated null pointer validation.

“Clean Code” book suggest “specific case object” for method calls that may previously return null. I am not a big fan, but it may solve internal api NPEs. I definitely support the pattern of returning empty collections in find* calls instead of null when nothing was found.

How would I deal with Null in the Pizza Order scenario?

PizzaOrder object should be validated beforehand in the interface layer. It is not the responsibility of this method.

Same layer before this method should call a factory method that sets pizzas collection to an empty collection if no pizzas.

Kitchen is always injected or constructed externally to this method and not its responsibility.

Factories in mapping/creating the PizzaOrder before this method call should also ensure no Pizza objects are Null.

The one possible valid check is if there are any pizzas in the order. The Collection.isEmpty(pizzaOrder.getPizzas()) But it is not a null pointer check, as previous interfaces/factories will ensure a collection object exists. However the interface or application before this method call should probably have validated the actual order, to ensure an order contains pizzas!

So Null is okay and NullPointerException can be your friend.

Create, populate and reset a dynamic database (HSQLDB,Hibernate,SQLMaven,DbUnit)

noreply@blogger.com (flurdy) — Wed, 26 Jan 2011 09:48:00 +0000

Thought I'd jot down how I create, populate and reset databases for development and testing. I use this method in my Java based pet projects such as Snaps ^[app][code] and Wishlist ^[app][code]. I also include this setup as default in my project template.

My projects are Maven based, using Jetty as java container through its maven plugin. In addition I also use JRebel, IntelliJ IDEA and Ubuntu, a setup I described in this howto: Ubuntu + IntelliJ + Maven + Jetty + JRebel, but none of these are required.

This setup is enables a fluid, very dynamic and quick development process. In addition I then use the in memory database HSQLDB for quick and standalone database interaction.

Using JPA/Hibernate with hsqldb, my tables can be created automatically when I start Jetty. However in developement and testing I prefer to have some default data pre populated.

This is where DbUnit comes in to play. With DBunit's maven plugin I can export my current data and populate future databases with the same data. As it is all easy to read XML I can edit manually the basic stub data as well.

However DbUnit can not create the database as it is run before jetty:run action, so I use the SQL-Maven plugin for this purpose. Again SQL Maven needs to know what tables to create, so I use the Maven Hibernate3 Plugin to export a schema from the JPA annotations.

This combination allows me to:

Check out my project anywhere

Create and populate the database with stub data with one command.

Run the application via jetty

Test the application straight away

or develop dynamically with instant feedback

How to set up Hibernate, SQLMaven and DbUnit plugins

Prerequisites/assumptions

Java

Maven

Jetty (can be tomcat maven plugin)

JPA annotations

HSQLDB (using file based hsqldb persistance to survive restarts)

JPA annotations -> Schema: Hibernate

To export the JPA annotations into a database schema we include the Maven Hibernate3 Plugin in your pom.xml. This is quite a long plugin section as it defines a few dependencies which otherwise may be overriden by the plugin and lead to problems like this mapping exception.


<profile>
  <id>hbm-export</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
   <artifactId>hibernate3-maven-plugin</artifactId>
        <version>2.2</version>
        <executions>
          <execution>
            <phase>process-classes</phase>
            <goals>
              <goal>hbm2ddl</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <components>
            <component>
              <name>hbm2ddl</name>
      <implementation>jpaconfiguration</implementation>
            </component>
          </components>
          <componentProperties>
    <persistenceunit>${project.artifactId}</persistenceunit>
            <outputfilename>schema.ddl</outputfilename>
            <drop>false</drop>
            <create>true</create>
            <export>false</export>
            <format>true</format>
          </componentProperties>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>hsqldb</groupId>
       <artifactId>hsqldb</artifactId>
            <version>${hsqldb.version}</version>
          </dependency>
          <dependency>
            <groupId>org.hibernate</groupId>
       <artifactId>hibernate-entitymanager</artifactId>
            <version>${hibernate.version}</version>
            <exclusions>
              <exclusion>
                <groupId>cglib</groupId>
         <artifactId>cglib</artifactId>
              </exclusion>
              <exclusion>
                <groupId>commons-logging</groupId>
    <artifactId>commons-logging</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
          <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>${hibernate.version}</version>
            <exclusions>
              <exclusion>
                <groupId>cglib</groupId>
          <artifactId>cglib</artifactId>
              </exclusion>
              <exclusion>
                <groupId>commons-logging</groupId>
                <artifactId>commons-logging</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
          <dependency>
            <groupId>org.hibernate</groupId>
      <artifactId>hibernate-annotations</artifactId>
            <version>${hibernate.version}</version>
            <exclusions>
              <exclusion>
                <groupId>cglib</groupId>
                <artifactId>cglib</artifactId>
              </exclusion>
              <exclusion>
                <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
              </exclusion>
            </exclusions>
          </dependency>
        </dependencies>                        
      </plugin>
    </plugins>
  </build>
</profile>

To run the schema creation action on its own:

mvn -DskipTests \

 -Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \

    -P hbm-export compile hibernate3:hbm2ddl;

Note: We need to compile the JPA classes so that the plugin can extract annotation information from them.

Note2: that Hibernate/Red Hat have a bad track record of including their plugins and releases properly in maven central repository, so you might either want to adjust your dependeinces and repositories accordingly. Or you can use my proxy repository:


<repositories>
  <repository>
    <id>code-flurdy-repo</id>
    <name>code@flurdy repository</name>
    <url>http://code.flurdy.com/nexus/content/groups/noncentral</url>
    <releases><enabled>true</enabled></releases>
    <snapshots><enabled>true</enabled></snapshots>
  </repository>
</repositories>
<pluginRepositories>
  <pluginRepository>
    <id>code-flurdy-repo</id>
    <name>code@flurdy repository</name>
    <url>http://code.flurdy.com/nexus/content/groups/noncentral</url>
    <releases><enabled>true</enabled></releases>
    <snapshots><enabled>false</enabled></snapshots>
  </pluginRepository>
</pluginRepositories>

Please if your project is popular do not use my repo as it will explode my ec2/S3 data bandwidth usage!

Schema -> Database: SQLMaven

To create your database from this schema with SQL-Maven, add this to you pom.xml


<profile>
  <id>sqlmaven</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>sql-maven-plugin</artifactId>
        <version>1.3</version>
        <dependencies>
          <dependency>
            <groupId>hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>${hsqldb.version}</version>
          </dependency>
        </dependencies>
        <configuration>
          <driver>${db.driverClassName}</driver>
          <url>${db.url}</url>
          <username>${db.username}</username>
          <password>${db.password}</password>
          <autocommit>true</autocommit>
     <srcFiles>
     <srcFile>target/hibernate3/sql/schema.ddl</srcFile>
          </srcFiles>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>

To run the database creation action on its own:

mvn -DskipTests -P sqlmaven sql:execute;

Database population: DbUnit

To populate your database via DbUnit, add this to you pom.xml


<profile>
  <id>dbunit</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>dbunit-maven-plugin</artifactId>
        <version>1.0-beta-3</version>
        <dependencies>
          <dependency>
            <groupId>hsqldb</groupId>
            <artifactId>hsqldb</artifactId>
            <version>${hsqldb.version}</version>
          </dependency>
        </dependencies>
        <configuration>
          <dataTypeFactoryName>org.dbunit.ext.hsqldb.HsqldbDataTypeFactory</dataTypeFactoryName>
          <url>jdbc:hsqldb:file:${project.basedir}/target/db/build;shutdown=true</url>
          <driver>org.hsqldb.jdbcDriver</driver>
          <username>sa</username>
          <password></password>
          <type>CLEAN_INSERT</type>
          <src>src/test/data/stub.xml</src>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>

First you need data to export. I suggest you run jetty:run action and add/use the application to create data in the database. Eg.: Register a user, create default domain objects, or whatever your application uses.

You then stop the application and export the data created:

mvn -DskipTests -P dbunit dbunit:export;

These are by default exported to target/dbunit/export.xml.

These are a good base data. You may try and adjust and extend this file if you like, but be aware that you might brake it, so prehaps try unadjusted initially.

Copy the export file to test/data so that it can be part of your source control etc. You may have noticed I already included the test/data as a src element in the plugin.

cp target/dbunit/export.xml test/data/stub.xml;

Run the database population action (remember to have a new clean database):

mvn -DskipTests -P dbunit dbunit:operation;

Example configuration of all these plugins can be found in this project's pom.xml.

Combined

Above are the basic steps. These I then aggregate into a couple of common one liners:

Create database:

mvn -o -DskipTests \

 -Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \

    -P hbm-export,sqlmaven \

    compile hibernate3:hbm2ddl sql:execute;

Create and populate database:

mvn -o -DskipTests \

 -Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \

    -P hbm-export,sqlmaven,dbunit \

    compile hibernate3:hbm2ddl \

    sql:execute dbunit:operation;

Reset database:

rm -rf target/db;

mvn -o -DskipTests \

 -Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \

    -P hbm-export,sqlmaven,dbunit \

    compile hibernate3:hbm2ddl \

    sql:execute dbunit:operation;

Clean, rebuild database and run jetty:

mvn -o -DskipTests \

 -Dhibernate.dialect=org.hibernate.dialect.HSQLDialect \

    -P hbm-export,sqlmaven,dbunit \

    clean compile hibernate3:hbm2ddl \

 sql:execute dbunit:operation jetty:run;

As you can see these one liners can be quite long so I again wrap these into bash scripts that I put in a bin folder.

You can extend this further to include these plugins as enabled by default and the maven goals as part of your other goals: E.g. include hbm:export and sqlmaven as part of install or test command or dbunit:operation with the jetty:run command, so that they are totally automatic. However I prefer a bit more control of when my data is reset etc.

Summary

With these plugins I can in one command create and populate my database, so that my app is up and running with data very quickly. Hope this is of use to others.

InfoQ Presentation Video Enlarger

noreply@blogger.com (flurdy) — Thu, 06 Jan 2011 19:01:00 +0000

I like Infoq, and especially their presentations.

These presentation page has 3 elements:

Top left: One small video showing the person presenting

Top right: One small window describing the presentation and presenters

Bottom: One large window showing the actual presentation slide

This layout works very well, and basically shows you what you need.

But...

The window is a bit small (320x240), especially on larger resolutions screens. Especially when the presenter also shows some code, which is usually shown in the video window.

So I wrote a little user script which enlarges this video window (480x320).
This script is only for the Google Chrome browser. A similar Firefox based GreaseMonkey script should be easy to convert.

To install click on my
InfoQ Presentation Video Enlarger user script available at
www.ivar.co.uk/cargo/infoq-enlarge-presentation.chrome.user.js.

You can download and look at the js in a text editor first.It really is a very small and easy to understand script.

Anyone can contribute changes if desired.

Accessing Picasa's atom feed API from Spring's REST and XPath templates

noreply@blogger.com (flurdy) — Tue, 19 Oct 2010 17:23:00 +0000

Been swearing at my screen and Google's choice of atom based API for their Picasa web albums. Flickr's REST API took 10 seconds to work out. Picasa: days...

My problem:

Trying to parse album ids from Picasa's API: code.google.com/apis/picasaweb/docs/2.0/developers_guide_protocol.html#ListAlbum.


Source picasaResponse = restTemplate.getForObject( albumsURL, Source.class, parameters );

Jaxp13XPathTemplate xpathTemplate = new Jaxp13XPathTemplate();

Collection albumIds = xpathTemplate.evaluate("//entry",

      picasaResponse, new NodeMapper(){

   public Object mapNode(Node node, int i) throws DOMException {

      Element album = (Element) node;

      log.debug("IN NODE ALBUM");

      return null;

   }

}

This never worked. It should have I thought. Trying different combinations of "/entry", "/feed/entry", "//atom:entry", "/atom:feed/atom:entry", swapping with Jaxen, etc made no impact.

Then after some googlebashing and soulsearching I solved the problem.

The Xpath template needs a namespace to resolve the atom xml. So adding this solved the problem:


Source picasaResponse = restTemplate.getForObject( albumsURL, Source.class, parameters );

Jaxp13XPathTemplate xpathTemplate = new Jaxp13XPathTemplate();

Properties namespaces = new Properties();

namespaces.setProperty("atom",

   "http://www.w3.org/2005/Atom");

xpathTemplate.setNamespaces(namespaces);

Collection albumIds = xpathTemplate.evaluate("//atom:entry",

      picasaResponse, new NodeMapper(){
 
   public Object mapNode(Node node, int i) throws DOMException {

      Element album = (Element) node;

      log.debug("IN NODE ALBUM");

      return null;

   }

}

Ps. code simplified and not using bean injection etc for clarity.

Why IRC is essential for distributed teams

noreply@blogger.com (flurdy) — Fri, 15 Oct 2010 08:25:00 +0000

Realising more every day that IRC is essential for distributed teams.

IMs (MSN, Google Talk, Communicator, etc), Skype & Live Meeting works well but is too sheltered, it is invitational only and time constrained.

Email is recorded, too formal, noisy and too asynchronous/slow.

Yammer or Twitter is nice for distributing comments and discussions, but is too slow for actual problem solving.

To keep teams up to date or solve problems together; a group chat is essential.

For culture build and low barrier of interaction of an informal and not archived discussions; a group chat is essential.

For rubber duck statements or help in minor but blocking issues that would be less likely to ask people directly; a group chat is helpful.

For less direct interruptions than IMs or phone; a group chat message which people can respond to when suitable will improve productivity.

Even for non distributed teams, IRC is an important productivity and culture tool. Continuous inconsiderate interruptions will kill productivity, while quick but asynchronous discussions smooth over the time requirements of the person asking and the person responding.

Unfortunately IRC is just too ugly, too much unix/hacker impression, too old; for an enterprise / self important company to consider.

Alternatives like the now discontinued Google Wave, 37signals's CampFire and others exist, but an internal IRC server does solve all these issues.

Review of Groovy for Domain-Specific Languages

noreply@blogger.com (flurdy) — Tue, 07 Sep 2010 12:48:00 +0000

I recently read the Groovy for Domain-Specific-Languages book by Fergal Dearle. The book essentially covers what Groovy is, what DSL is and how to apply Groovy to incorporate a DSL for your applications.

After reading the book I have to agree that if you intend to create a DSL, which we basically do unintentionally all the time, then Groovy can fulfil these requirements with ease and simplicity.

The book is a easy to read book with clear text, syntax and examples without being too thick and not seem a like a javadoc of information overload. It is rather quick to read through due to the nice flow. The layout do make it a nice reference for myself in the future as it is laid out clearly and contains a large variety of working examples.

The book starts of with first explaining the concepts of what DSL is, what it means in terms of computer science history and how we already use them.

It then goes on to explain in clear but complete detail what Groovy is and does. Actually these chapters are a good introduction to Groovy book in themselves.

Dearle then procedes to explain the main technologies built on top or as part of Groovy such as the web framework: Grails, the Groovy Ant builder scripting tool: Gant, Groovy & Grails ORM database mapper: Gorm, BDD frameworks and similar technologies. This it does in enough detail so that you have a good idea what they do and can experiment with yourself afterwards.

The last half of the book Dearle then brings Groovy and the related technologies back into the DSL fold and explains why and how Groovy is excellent for DSL writing. E.g: Groovy’s dynamic nature, closures, or the multitude of “Builders” which basically are DSLs outright.

These chapters of the book has a large set of basically Cook Book examples of Groovy DSL writing, builders and finishes with how to integrate Groovy with Spring, XML, Ant and existing Java applications.

I was already aware of Groovy’s suitability as a DSL language, but I am now even more convinced of its power, ease and fit as a or rather the DSL. If you would like to have a peak at the book there is a free chapter, or why not buy the book? ( amazon.co.uk, amazon.com, Packt)

(Note: this book was also reviewed on slashdot this summer)

Spring MVC ExceptionHandler needs to be in same class

noreply@blogger.com (flurdy) — Fri, 23 Jul 2010 13:06:00 +0000

Had a frustrating evening (pet project) trying to get Spring 3.0 MVC's annotation based exception handler to work.

All the examples seemed straight forwards:

Just annotate a method in a controller:

import org.springframe...ExceptionHandler;
import org.springframe...ModelAndView;
import org.springframe...Controller;
....
@Controller
public class MyController {
....
@ExceptionHandler(SomeException.class)
public ModelAndView myExceptionHandler(
    SomeException exception){
  blah
}
...
}

But it just would never catch my exceptions. Very frustrating.

But in the end a google search to the end of the world where someone mentioned in the comments that the exceptionhandler method have to be be in the same class as the method throwing the exception.

Very odd restriction compared to Spring's usual very agile and generic annotation.

So I added my handlers to my abstract controller class that all my controllers extend and problem solved, with some hair still intact.

org.hibernate.MappingException: Could not determine type for Set / List

noreply@blogger.com (flurdy) — Tue, 20 Jul 2010 22:11:00 +0000

I was simplifying the domain model of a pet project by removing an unneccesarry entity by using JPA 2.0's ElementCollection annotation:

public class SecurityDetail {

public enum AuthorityRole{
  ROLE_ADMIN,
  ROLE_SUPER,
  ROLE_USER
}
....
@ElementCollection
@CollectionTable(
  name="Authority",
  joinColumns=@JoinColumn(name="username")
)
private Set authorities;
....

However I was getting this error message:

org.hibernate.MappingException: Could not determine type for: java.util.Set, at table: SecurityDetail, for columns: [org.hibernate.mapping.Column(authorities)]

Digging around (googling) I ended up on this blog entry: blog.m1key.me/2010/06/orghibernatemappingexception-could-not.html, which main solution is to make sure you have a newer hibernate, version 3.5.3 or later.

I use maven, with the hibernate3-maven-plugin, sql-maven-plugin and dbunit-maven-plugin to create the schema, database and to populate data. So I bumped the org.hibernate:hibernate-entitymanager dependency from 3.5.0 to 3.5.3-Final. (and fixed my own nexus repository to pull JBoss's latest jars.)

However it did not fix the problem. But by reading the comments of the blog entry above I also realised that my hibernate3-maven-plugin may be using older dependencies. So I explicitly added the recent verions of org.hibernate:hibernate-core dependency to both the plugin and the app, as well as org.hibernate:hibernate-entitymanager to the plugin as well:

<properties>
  <hibernate.version>3.5.3-Final</hibernate.version>
</properties>
....
<dependencies>
  <dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-entitymanager</artifactId>
    <version>${hibernate.version}</version>
    <exclusions>
      <exclusion>
        <groupId>cglib</groupId>
        <artifactId>cglib</artifactId>
      </exclusion>
      <exclusion>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
      </exclusion>
    </exclusions>
  </dependency>
  <dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-core</artifactId>
    <version>${hibernate.version}</version>
    <exclusions>
      <exclusion>
        <groupId>cglib</groupId>
        <artifactId>cglib</artifactId>
      </exclusion>
      <exclusion>
        <groupId>commons-logging</groupId>
        <artifactId>commons-logging</artifactId>
      </exclusion>
    </exclusions>
  </dependency>
</dependencies>
....
<plugins>
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>hibernate3-maven-plugin</artifactId>
    <version>2.2</version>
    <executions>
      <execution>
        <phase>process-classes</phase>
        <goals>
          <goal>hbm2ddl</goal>
        </goals>
      </execution>
    </executions>
    <configuration>
      <components>
        <component>
          <name>hbm2ddl</name>
          <implementation>jpaconfiguration</implementation>
        </component>
      </components>
      <componentProperties>
        <persistenceunit>${project.artifactId}</persistenceunit>
        <outputfilename>schema.ddl</outputfilename>
        <drop>false</drop>
        <create>true</create>
        <export>false</export>
        <format>true</format>
      </componentProperties>
    </configuration>
    <dependencies>
      <dependency>
        <groupId>hsqldb</groupId>
        <artifactId>hsqldb</artifactId>
        <version>${hsqldb.version}</version>
      </dependency>
      <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-entitymanager</artifactId>
        <version>${hibernate.version}</version>
        <exclusions>
          <exclusion>
            <groupId>cglib</groupId>
            <artifactId>cglib</artifactId>
          </exclusion>
          <exclusion>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-core</artifactId>
        <version>${hibernate.version}</version>
        <exclusions>
          <exclusion>
            <groupId>cglib</groupId>
            <artifactId>cglib</artifactId>
          </exclusion>
          <exclusion>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
    </dependencies>                        
  </plugin>
</plugins>

Groovy for DSL

noreply@blogger.com (flurdy) — Fri, 16 Jul 2010 10:58:00 +0000

Started reading through Groovy for Domain Specific Languages, a new book by Fergal Dearle. As I interested in how to apply Groovy in more enterprise applications I would have normally liked to read this book, however I was also asked to write a review of the book, so Id better read it! How to keep DSLs easy to read while easy to create and maintain with Groovy will be something I hope this book covers.

The book was also reviewed on slashdot recently.

You can read the free chapter or buy the book yourself at e.g. amazon.co.uk, amazon.com or Packt.

Any comments of others interested in Groovy, especially in the DSL area are very welcome?

Skinny WAR Assembly in maven

noreply@blogger.com (flurdy) — Fri, 02 Jul 2010 08:53:00 +0000

If you want to create a "Skinny WAR" so that you can package it as part of an EAR or as in my case as part of an assembly later, then you can do this:


  <plugin>
   <artifactId>maven-war-plugin</artifactId>
   <version>2.1-beta-1</version>
   <configuration>
       <packagingExcludes>WEB-INF/lib/*.jar</packagingExcludes>      
       <archive>
           <manifest>
               <addClasspath>true</addClasspath>
               <classpathPrefix>lib/</classpathPrefix>
           </manifest>
       </archive>      
   </configuration>
  </plugin>

If you then later in another pom.xml want to include this war and the excluded libraries:


<dependency>
    <groupId>${project.groupId}</groupId>
   <artifactId>you war module name</artifactId>
   <version>${project.version}</version>
    <type>war</type>
  </dependency> 
  <dependency>
    <groupId>${project.groupId}</groupId>
   <artifactId>you war module name</artifactId>
   <version>${project.version}</version>
    <type>pom</type>
  </dependency>

Ivar's rambling

Rusting, a quick dabble with the Rust language

Rustic Pizza

Experience so far

Install Rust

Learning Rust

Web frameworks

Keep rusting?

ROC theorem: Readable, Optimised and Correct code. Pick two.

ROC theorem

Correct code

Optimised code

Readable code

Not mutually exclusive

All the ROCs

Readable, Optimised and Correct code. ROC. Pick two. Or just rank them.

Evolve or wither slowly

Recap

Big rewrite disadvantages

Continual rewrite benefits

No rewrite disadvantages

Less obvious benefits

Business domain knowledge

Technical knowledge

Quick turnaround

Technical migration

Staff retention

Recruitment

Lean technology - lean organisation

Rewrite when not needed

Rewrite encapsulation

Experiment and rewrite experiments

Rewrite exploration and euthanasia

Summary

Don't download the internet. Share Maven and Ivy repositories with Docker containers

Work around, not solution

Vagrant

Boot2docker

Data volume container

Launch container

Build problem

Maven repository manager

Repository manager container

Dockerise it all - containerised addiction

Pair with people you like and code reviews with people you don't like

One button - The aim - Nearly there

Company profitability

Heroku commands & tips

Send email via SendGrid on Heroku using Play! 2.0 with Scala

Play! 1 & 2 command tip

Clone local git repository to remote server?

Continuous Deployment via Stack Overflow

SOPA protest page and 503 redirect

Play! 2.0 in IntelliJ IDEA

Do not rewrite

Evolve

Null is okay

Why are we so afraid of NullPointerExceptions? What is wrong with null?

Dealing with NPE/Null?

Create, populate and reset a dynamic database (HSQLDB,Hibernate,SQLMaven,DbUnit)

How to set up Hibernate, SQLMaven and DbUnit plugins

Prerequisites/assumptions

JPA annotations -> Schema: Hibernate

Schema -> Database: SQLMaven

Database population: DbUnit

Combined

Summary

InfoQ Presentation Video Enlarger

Accessing Picasa's atom feed API from Spring's REST and XPath templates

Why IRC is essential for distributed teams

Review of Groovy for Domain-Specific Languages

Spring MVC ExceptionHandler needs to be in same class

org.hibernate.MappingException: Could not determine type for Set / List

Groovy for DSL

Skinny WAR Assembly in maven

Why are we so afraid of NullPointerExceptions?
What is wrong with null?