Luca Grulla

Evolutionary Architecture and Microservices: a real world example

2015-07-01T00:00:00+00:00

Microservices have received a lot of attention over the last few years. As an architectural style it delivers a lof of benefits compared to a Monolith application, but there’s no doubt that it comes with the cost of a distributed system (such as discoverability, fault tolerance, data consistency just to name a few).

I believe a Microservice architecture shouldn’t be your starting point. Instead, it should be the result of applying Evolutionary Architecture and Domain Driven Design principles to your system, balanced with a analysis of the maturity stage of your product (both from a business and technology perspectives).

While building uSwitch car insurance we followed exactly these principles; since the beginning of the project until today we refined our architecture based on our increased understanding of the business needs, moving from a monolith by choice system to a well defined microservices architecture that keeps evolving and improving.

If I look back at these 3 years I can identify five major stages:

Fast development cycle
Support business workflows
Separation of concerns
Decouple and scalability
Leveraging uSwitch technology ecosystem

Stage 1: Fast development cycle

We wanted to build a new product from scratch and we wanted to go live as soon as possible. Our first architecture was optimized to enable us to have a fast development cycle and the shortest path to production.

Even if at the beginning of our journey we identified two separate concerns:

serving the actual website
integration with multiple insurance APIs for real-time brokerage

Although our experience was already suggesting that Microservices could be an excellent solution for the problems we were going to solve, we wanted a fast development cycle minimum friction between changes and access to production; we also didn’t want to get distracted in building solutions that won’t add much value for our Release 1. We explicitly chose to build a monolith Ruby on Rails application, with a well defined internal boundary between the web code and the integration layer.

Stage 2: Support business workflows

Once the new product went live sales reports from our partners started to flow in (mostly as CSV files over FTP). With sales data we started to identify KPIs and draw correlation between the web traffic, partner offers and the different conversion rates. We knew that the data aggregation and analysis could be done manually for a short time of period, but we had to automate the workflow before the volume was growing. We therefore introduced a new system responsible for sales data ingestion and for producing analytical dashboard.

In order to give reliable visits to sales conversion rates the newly built dashboard had to have access to some web-centric data(such as number of unique visitors, number of completed journeys etc etc). We introduced a simple REST APIs on the web layer and use cron jobs to do scheduled data transfer from one system to another. This solution allowed us to move forward, and because it was flawed by choice (data duplication, poor resiliency) it forced us to think over time about what we really need in that space.

Stage 3: Separation of concerns

It was time to move our architecture towards SOA to benefit from much clearly defined bounded contexts.

We knew from the very beginning that the integration layer and the websites were two different contexts; while one is responsible to collect and present information the other is responsible to communicate with the external partners and transform data between multiple representations. The integration layer had been encapsulated at the boundary of the web system so far, but it was now time to reify it in an independent service and let it evolve independently from the web. The new service was responsible for pure data transformation: receiving the internal representation of a user profile, transforming it to the different XML flavours expected by the different integrations and transforming the responses back in a normalized representation ready to be used by our website. We felt that the problem space was a good fit for a functional language and we embraced Clojure. The integration between the website and the new aggregation service was HTTP POST with JSON as data type. We also started understanding more of our analytics need: a Postgres instance logically close the Integration layer became our analytics data source.

Stage 4: Decouple and scalability

At this stage we had consistent traffic and sales to really benefit from a more mature data pipeline and analytics platform. REST style integrations brilliantly supported us until this point but we were at a stage where we wanted to increase system resiliency. uSwitch teams have been long time users of Kafka. Kafka is a distributed message system with an eye on message durability. We decoupled the integration layer from the analytics data store(the data transformation was a stateless service by definition anyway) with a set of Kafka topics and few workers. Now the integration layer was just publishing useful information to the topics, enabling multiple consumers to fetch data to support different needs.

Stage 5: Leveraging uSwitch technology ecosystem

We are now 30 months from the go live and this is our current architecture. The amount of data we produce and collect grew together with our business, and Postgres is not a viable solution anymore. We moved all our long-term analytical data to Redshift, getting us aligned to other uSwitch product teams that were already using it. Redshift now works as cross-channel data warehouse, enabling more in depth analysis and reports than before. We also replaced some of the REST based integrations with more Kafka topics: this gives us better resiliency and the benefit of a set of internal tools build for replayability.

Javascript: how to create partial functions

2014-12-30T00:00:00+00:00

Currying and partial application functions are very powerful techniques widely adopted by many functional langagues.

If you work with Javascript you can use the core function bind to easily create partial applications.

Let’s look at an example.

We have an array of objects:

var cities = [{name: "Paris", country: "France"}, {name: "London", country: "UK"}, {name: "Rome", country: "Italy"},{name: "Manchester", country: "UK"}];

and a function that given a city object and a specific country returns true if the city belongs to the given country:

var isIn = function(country, city) {
  return city.country === country;
};

isIn("UK", {name: "London", country: "UK"}) //true

What if we’d like to filter the array looking for cities that are in a specific country?

A partial function is a good approach to solve the problem, and bind makes the creation of it very simple.

The syntax of bind is the following:

fun.bind(thisArg[, arg1[, arg2[, ...]]])

Where the first parameter is the value we want to bound the this keyword to when the created function is invoked while the others are the actual parameters of the bound function.

var UKCity = isIn.bind(null, "UK");//don't need to set the this keyword, just passing null

UKCity({name: "London", country: "UK"}) //true

We can now use the partial function UKCity as argument of the filter function:

var UKCities = cities.filter(UKCity);
//UKCities === [{name: "London", country: "UK"}, {name: "Manchester", country: "UK"}]

//get the list of the Italian cities is as simple as:
cities.filter(isIn.bind(null, "Italy"));
//[{name: "Rome", country: "Italy"}]

In the example the creation of a partial application function allows us to seamlessly compose our initial isIn function with a higher order function like filter to achieve more sophisticated outcomes.

A gist with the whole example code is available here.

Clojure: graceful shutdown of an embedded Jetty instance

2013-07-15T00:00:00+00:00

At uSwitch, several of our services are actually Clojure applications deployed with an embedded Jetty server(via the Ring adapter) which are managed via the standard Unix service interface.

For the business critical services, zero downtime is a fundamental requirement: here comes Jetty graceful shutdown.

Once the ready to be released uberjar is deployed, we signal the Unix service that wraps our Clojure application with a restart. As part of the shutdown process Jetty will immediately close the HTTP listener(freeing the used HTTP port and making it available to the newly deployed embedded instance) while it will still take care of completing the ongoing HTTP connections.

Ring 1.3.x and Jetty 7

If you are using Ring 1.3.x or older you are embedding Jetty 7. In that scenario this is the necessary configuration:

Ring 1.4.x and Jetty 9

with Jetty 9 the web container is now capable of keeping track of the open threads and to completely shut down itself once the threads are completed(rather than waiting for a fixed amount of time). With Ring 1.4.x and higher the code will then look like:

Web development in the large

2013-03-10T00:00:00+00:00

Web development has drammaticaly changed over the past 5 years. Frameworks like Rails imposed themselves as the go-to choice for an increased productivity; new stacks like NodeJS emerged and offered alternatives ways of tackling web products development.

Despite all these goodies offered by these tools, web development often ends up being way harder than what it should be. Teams sink in a never ending bug list, often adding unneeded techical complexity in the attempt to offer a better user experience and hence very quickly transforming the application in an uncontrolled ball of mud.

From my direct experience in building public facing web products, there are few core ideas that have to be the pillars of any web products; these are language/framework agnostic – even if certain tools support these principles better than others.

These reccomendations are mostly related to a scenario where a pure single page web app is not a viable solution(check the this section for a more in depth analysis). Some of these principles are appliable even if you are targeting that style but might be less effective.

Understand your target audience

You want to have a very clear idea of the browsers you have/want to support. Building a product that works only on HTML5 friendly browsers while the 50% of your audience is on IE8 is not the best way to give your product the right boost. If you are replacing an exisiting product use Google Analytics historical data to understand the current browsers segmentation, while if you are building something totally new invest some time in understanding who your customers will be. Knowing your audience can influence heavily the architecture of your application: a single page app can be a perfect fit for a mobile only app( most of the mobile browsers are HTML5 friendly) but it might be a bad choice for a product that has to be consumed by less modern browsers.

Use aggressively progressive enhancement techniques

Build every feature ground up. The more your website works without javascript, the more extensible it wil end up being.

What this really means?

Your product has to work without javascript. Of course this is heavily influenced by the previous guideline, but unless you are building a 100% single page app you should always ask yourself how do build a feature starting from the HTML over HTTP flow.
Once the HTML over HTTP feature is there, enhance components via javascript in order to improve the experience.
Go for multi layered enhancement approach.
1. Look at what HTML5 features are available on the browsers you want to support and if possible rely on them(datepicker and validation are two good examples of HTML5 features that are getting more and more support).
2. If there’s no native support for certain browsers then smoothly degrade to a javascript version
3. Finally if the javascript version results in an heavy experience on specific devices just leave the basic functionality without any bells and whistles.

The advantage in this case is a consistent reduction of the code you have to take care of over time; because HTML5 features are natively built into the browsers you will have a performace boost for free. Also they are often already supported on mobile browsers, giving for free a native experience in the mobile version of the product.

Prefer html snippet over JSON

JSON is extremely powerful and easy to use, but it’s not the solution for everything. Often we need to pass data from the server side to the browser as a consequence of an AJAX request to build new parts of the interface.

Unless you are thinking of consuming the exposed JSON data from an heteregenous set of clients you should just expose HTML snippet that you can then embed directly into your page.

This approach gives you:

Progressive enanchement out of the box.
A substantial reduction in client side computation, something that is critical for less modern browsers.
A massive reduction of of your front-end code spread If you let the server-side to build the HTML snippet you you can use your favourite markup/markdown without the need of hand rolling your version client side or importing yet another dependency into the browser.

Use HTTP Status codes

The HTTP protocol contains everything you need to control and define the interaction between different parties in a scalable and fault tolerant way. When you need to return a specific status to your ajax call there’s no need of reinventing the wheel sending any custom JSON object to say that an error happened or that the specific operation is not acceptable, just use the right HTTP status code. Embracing a HATEOAS architectural style can help you in shaping your systems and APIs, going beyond the basic CRUD with a more holistic view of your system as a good citizen in the HTTP stack.

Leverage HTTP caching

Thr HTTP protocol gives you a wide range of caching options through header directives. Before introducing a cache at the domain model layer ask yourself if the same couldn’t be achieved at HTTP level.

Think about resource shareability (urls with resource id)

The web is really about sharing and URLs are the way we share resources.

Use URLs containing the resource id to epxose the different states of your resources to the world and use authentication and authorization layers to control resource access.

Don’t be tempted to to trade off unicity of the resource URL with the pretty link bells; links that are easy to read and rember for humans are a nice to have, but not all the time this style supports the needs of your application. Even if your web application doesn’t demand the share sharing this is a valid test to verify the solidity of your system: would it be easier to expose a resource to a third party? Is a specific state of a resource uniquely identified by a URI? If the answer to these questions is no there’s a clear smell that the application is not using the right semantic to move from an internal state to another.

Flow control in Javascript

2011-09-26T00:00:00+00:00

Mark Needham recently wrote a blog post on how his team worked around a Javascript asynchronous unwanted behaviour.

They want to iterate over a collection, executes some code for each element of the collection and only when all the collection has been traversed execute a final step.

This didn’t work as expected. Due to the asynchronous nature of Javascript the do something with grid block is invoked before the grid itself is filled and nothing interesting happens.

Their solution to this was removing any possible event driven behaviours and handling control in a fullly imperative way, explicitly calling functions in the expected sequence.

This works but it’s not leveraging Javascript nature; also the code itself results harder to understand then what it should be.

How the same problem can be solved embracing asynchronous thinking ?

The way to achieve control flow in Javascript is with events.

When a block is completed it emits an event. All the interested parties will be listening to the specific event and execute their specific task, contributing to our overarching business flow. This alternative solution is probably also more idiomatic and as a consequence more coincise and easier to read.

Introducing node-tail: a NodeJS tail library

2011-09-15T00:00:00+00:00

In the previous blog post I described the architecture of the firehose we built at Forward with NodeJS.

At the lowest level each node has to tail a log file. Tom Hall and I couldn’t find any useful cross-platform (i.e. not relying on the unix tail command) node module for that task so I ended up writing node-tail.

Using node-tail is very simple:

node-tail is also available via npm, just install it with: npm install tail

Building a firehose with NodeJS

2011-09-12T00:00:00+00:00

In Forward we handle a huge stream of real time data and we are always looking for interesting ways to use that.

We already have a Hadoop cluster for high latency analysis (mostly reporting), but recently we started building a set of tools that can give us a near real-time view of what’s going on. With this goal in mind I have been recently involved in building a data firehose with NodeJS.

The result is the following:

The lowest layer of the firehose is a thin component installed on each server that tails the log file we care about and publishes each log entry to a collector (called firehose-master) via ZeroMQ. The master collects the log entries from all the nodes and republishes everything to the rest of our software ecosystem as a single stream via a single ZeroMQ end point.

With this architecture we easily preserve the horizontal scalability of our main service, in fact adding a new node to the firehose is as simple as installing the tail component on the new server and adding its IP address to the master configuration file.

This stream can now be the core foundation of clients that consume the firehose for different purposes, from real-time trends visualisation to HDFS data bulk load.

Javascript testing

2010-06-15T00:00:00+00:00

Javascript has become in the last year the language for rich web applications, but still it rarely receives the level of attention it deserves.

Libraries such jQuery boost our productivity but the code we often end up writing tend to be a big ball of mud, with an entangled mix of presentation logic, busines logic and server side interaction, all incredibly hard to test and to maintain.

It’s time to move away from this approach and start writing better quality Javascript.

The very first step required to avoid Javascript spaghetti code is start thinking to Javascript as a first class language, and start dealing with it with the same mindset and approach we would use for any server side language.

With this new approach in the same way we identify roles and integration points in server side code we want to start building abstractions in our Javascript codebase.

With the right abstractions in place we are defining clear boundaries between different parts of the system, and as a consequence our code is simpler, we promote reuse and the DRY principle, and we are finally enabling better testing.

Let’s look at any standard Web 2.0 Javascript code from this new perspective: now the DOM and HTTP are two clear integration points.

Our javascript code manipulates the DOM adding &nodes or changing existing nodes content in the same way any other language would interact with a database.Each call to a server over HTTP via Ajax is exactly the same of calling a web server from our server side code.

With these very two first abstractions in mind, we can start rewriting our code, isolating these interactions behind clearly defined objects.

Now the javascript code is not a ball of mud anymore, but a network of objects that collaborate. With these smaller objects in place we can now favour interaction based tests, moving away from any dependency on the DOM and on the HTTP protocol.

The identified abstractions let me actually mock and stub things out and test if the different tiers in my javascript code are exchanging the right messages. This separation of concerns keep the business logic nicely isolated from the user interface transformations, enabling also a cleaner state based testing for that specific part of the code (there are no more dependencies on the DOM).

Italian Agile Day 2009: date announced

2009-08-02T00:00:00+00:00

Friday, 20th of November 2009, the 6th Italian Agile Day will be held in Bologna. It's a great opportunity to share experiences and practices, learn, discuss and meet other praticioners part of the Agile community...and it's free :-) I'll see you there !

What has to be ready for the beginning of a project?

2009-07-05T00:00:00+00:00

The beginning of a project is always a hectic period where several things have to be put in place in order to be able to start the actual development from a solid position.

Interestingly enough, I see that what is important to have ready for ThoughtWorkers most of the time is not what has to be ready for other people.

Given that obviously every project is different and deserves specific attention, here is my list of things that has to be ready before the kick off of iteration 1.

Infrastructure

A repository, a Continuos Integration Environment with Cruise (or suitable alternatives) installed and working, a QA environment where we can deploy every successful build whenever we want, a basic build script (i.e. build/run tests/package). Pairing boxes, each one with exactly the same configuration.

Architecture

Just identify the core services/components, there’s no need to go into detail for each one at this time of the project. If we identify what is the main responsibility of a service is good enough for now, the details will be discovered later in the project. If we can easily identify the way services will communicate (web service? message broker?) good, if not through good OO principles we can abstract the low level mechanism and reduce the cost of change later, deferring the final decision to the point in the project where we have more understanding of the technological constraints.

Patterns

Pairing and frequent pair rotation will help in spreading the knowledge of the approach to solve specific problems and in maintaining consistency throughout the code base, so most of the time there is no need of defining a sort of “project dictionary” of valid patterns at day 1. If we’re introducing new approaches to solve a specific problem, it’s important to highlight pros and cons of the approach so that people know what they are doing once pairing.

Most of the time these are probably enough to start Iteration 1.

All the other decisions can (and sometimes should…) be deferred to a later stage.

QA productivity metrics

2009-03-31T00:00:00+00:00

In a lot of companies the QAs productivityi s measured through the number of defects she raised: the more defects she finds, the harder she works, therefore the better she is.

This approach has an interesting corollary: if you have really good QAs you don’t have good developers. If your QAs are finding a lot of defects this means that your developers are quite poor and are regularly introducing defects in the code base.

This approach isn’t obviously team oriented: QA and Development are seen as two separate and independent entities that interact through a specific contract.

If we truly believe in collaboration the most important productivity metrics are not related to a specific role/skills set but to the team capability of deliver quality software in time and in budget.

We need to ask ourselves why the defects are finding their way into the application deployed in the live environment (incomplete acceptance criteria? not enough analysis or understanding of the domain? gap in the test suite?), and fix the issue as a team, not just using that number as the boundary between two work streams.

Local optimization doesn't necessarily mean improvement

2009-02-14T00:00:00+00:00

Delivering software is a pretty complex activity that requires interaction between people with different skill sets. One of the cornerstone of Agile Development is continuous improvement, and one of the tool often used to learn and improve is the retrospective.

In a context where the collaboration is not effective, people tend to look for local optimization instead of seeing the big picture, and you have things such the “QA retrospective” or the “Developer retrospective”.

In this scenario a “QA retrospective” (as the Dev’s one, or a BA’s one) is probably more harmful than anything else; the specific issues that will be identified won’t address the whole activity of delivering software but will only be focused on that specific step (“we need x to do y”).

But what is the impact of that step in the overall process ?

How that step fits into the chain of event that will take a business idea to be delivered as a software artifact (hopefully in a timely and quality fashion ) ?

Don’t get me wrong: it’s definitively through specific improvements that you improve your overall process but if don’t frame your changes in the big picture, it’s more likely that your changes will impact your process in the wrong direction and actually cause an additional waste.

Each attempt of optimization should therefore start from the clear analysis of what is wrong from a high level point of view and only then it’s time to shift our attention to the specific details of each step.

Acceptance Testing of Flex applications

2009-01-26T00:00:00+00:00

Acceptance Testing is a fundamental practice: it gives you confidence that your application behaves as expected from the end customer point of view.

In the Flex world there are some projects that are currently emerging in the Acceptance Testing space, each one with specific advantages and weak points.

Let’s have a look at some of these.

FlexMonkey

FlexMonkey Open source but based on closed source API (the Automation API are released only with FlexBuilder and not with the open source Flex SDK), it’s based on the record-playback approach and as far I have seen is not easily integrable with a Continuous Integration server.

Flash-Selenium

Flash-Selenium is open source, and it works as an extension of SeleniumRC; the tests can be written in Java, .Net, Ruby or Phyton and the integration with a Continuous Integration server is therefore quite out of the box. SeleniumFLex API SeleniumFlex is another open source project. it’s an extension of SeleniumIDE, test should be written in Selenese.

FunFX

FunFX is based on the automation API (therefore you can use it only if you’re owner of FlexBuilder), the fixtures are written in Ruby.

At the moment there isn’t a de-facto solution: the community is quite dynamic and all these different tools are trying to find their space.

Which one is my favourite ?

It’s really hard to say, I think that the context matters a lot. I don’t like solutions that require the Automation API simply because this hooks you to a vendor just for testing; it’s also true that the pure open source solutions require some extra-hack (like exposing methods through ExternalInterface) that is less than ideal.

Looking at the two pure open source tools I like Flash-Selenium for its out of the box integration with Continuous Integration server, while I prefer Selenium-Flex approach for handling the necessary ExternalInterface configuration.

Signs of poor communication

2008-09-28T00:00:00+00:00

Walking in an office and just looking around for 10 minutes is enough to have a feeling of the level of communication in that environment.

When:

co-located people communicate mainly via IMs, mails, or comments on online collaborative tools instead of face-to-face conversation
a heavyweight tool is the preferred way for driving the work flow instead of using it only for backup and tracking
on the whiteboard you can read the outcomes of the retrospective of 7 months before

There’s definitively something wrong.

What about your team? How many of the above bullet points are you ticking off?

Flex: How to achieve proper separation of responsibilities

2008-06-24T00:00:00+00:00

In a previous post I was discussing on my experience with Flex and one of the highlighted pain points is the extremely poor quality of the available tutorials.

Let’s have a look at the source code of the Flick tutorial:

In a single page you have:

The declaration of visual components (TileList)
the declaration of non visual component (HTTPService)
event handling (photoHandler)
data binding (photoFeed)
pure logic (requestPhotos)

If you decide to follow this development model you will obtain something that is extremely fragile to evolve or simply just to maintain.

Let’s start assigning the right responsibilities to the right objects.

The first step is creating our own Application object and use it as root in the mxml file:

Now the HttpService component is not exposed anymore (and ideally can also be injected) and the events handling in the right place.

The second step is to extend the HttpService object to place the requestPhotos behaviour in the service. The result is below:

At the end of this refactoring our mxml file looks like this:

Here now we are declaring only visual components, and its only role is defining what we want to see and how.

Working with Adobe Flex

2008-06-21T00:00:00+00:00

RIA is a pretty hot topic in these days; even in ThoughtWorks a lot of discussions are going on (if you have time check fellow ThoughtWorkers posts here, here and here).

I’m quite involved in the conversation, mostly due to the use of Adobe Flex in my current project.

What is Flex ?

Flex is the Adobe solution for RIA; based on ActionScript, it promises quick development of rich UI, the ability of testing and TDD-ing, good tools for development and out of the box integration with Web Services and REST services.

Everything sounds cool, but my experience with Flex highlighted few elements that worth a bit of attention…let’s analyze those one by one.

Testability

ActionScript is actually really hard to test.

A Unit testing framework (FlexUnit) is available but it cannot be used by command line because the test runner works only in your browser; this slow down your development cycle (red-green-refactor) quite a lot, reducing your effectiveness during the day.

ActionScript is a half way between a static and a dynamic language (with a clear direction towards being a pure static language, my guess is that the language designers are trying to increase the language appeal to the Java and the .Net community).The main concert here is you miss the testing support of static languages(mock libraries mostly) and of a dynamic one you’re missing the complete dynamic approach(all the classes are sealed by default, and you can’t do methods override at runtime even for objects that are explicitly declared dynamic).

In this scenario interaction testing is nearly impossible to do; sure, you can write your own mock library or stub everything out but this is a pretty strategic decision and a lot depends on the scope of the project.

Flex Security Model

Another aspect that gave us more than one trouble is Flex security model.

A Flex application can connect to a WebService or a REST service only if:

the target service is on the same domain where the Flex application is deployed
the target service is on an external domain but in the target domain there is a crossdomain.xml file that declares your domain as secure

What does it mean ? That if I want to access a public feed (let’s say the BBC weather feed) BBC has to have my domain specified on its crossdomain.xml file that lives on its server.

This is a strong limitation: you cannot simply call BBC and ask to add your domain to their list of trusted site, and this take you down the path of over design your application (like adding a proxy web service on your domain just to be able to get the real data you need from BBC).

Tutorials and suggested best practices

A final comment is for the Adobe tutorials: it’s hard to see in 2008 something so poor. The tutorials (have a look at the Flickr one to have an idea) are a collection of bad practices: the suggested development style is spaghetti-code, where you write all the code in the Script tag within your main Flex application file (mxml).

And what about separation of responsibilities ? Do we really think that this is the code you obtain doing TDD ?

The heavily usage of the Script tag will take you down maintenance nightmares, transforming your application in something that is quickly out of your control and you cannot evolve.

Final thoughts

Looking at these pain points I can’t sat that Flex is a mature technology. Adobe promises a lot but a number of the promises are not there; TDD is simply too hard to do due to the lack of mock support(even if looks promising), the set of possible permissions you can give to the Flash player clashes with the new services oriented features (introducing cumbersome security model that take you to overdesign as a possible workaround), and tooling is quite poor (FlexBuilder, the IDE, crashes regularly and the Ant task is pretty basic).

Article: Seniority, Respect, Authority and an Agile Team

2008-05-25T00:00:00+00:00

Here on InfoQ some interesting thoughts around Seniority and Authority in an Agile team.

Article: Patrick Kua on the Agile Coach role

2008-05-25T00:00:00+00:00

Here a really good article from my friend Patrick Kua on the Agile Coach role.

Ivy: how to download dependencies to different folders

2008-04-17T00:00:00+00:00

Ivy is a dependency manager that lets you define in a declarative way directly in the build file the dependency of your project.

In order to download dependencies in different folders (i.e. separate the runtime dependencies from the testing ones) you have to define configurations.

Let’s say we have this project structure:

lib
- prod
- test

We need three different things to achieve our goal:

The first step is defining in your ivy.xml a configuration for each folder:

Then we need to define the actual dependencies assigning to each one the right configuration(in this scenario both are extending the default one):

Finally we need to define the pattern in the ivy:retrieve tag

Integrate Ivy in a dotNet build

2008-04-17T00:00:00+00:00

I’m currently involved in a .Net shop and I’m working on redesigning the build pipeline (good old CI stuff). Ivy appeared in our brainstorming sessions as a solution for having a more defined control on the projects dependencies requirement.

The problem is that currently (but you never know about the future ;-) ) there isn’t a .Net native port of Ivy.

This can be solved using the exec tag from your NAnt script and using java as an external program. Here is the bit to add to your NAnt build script:

Essap 2008

2008-04-09T00:00:00+00:00

For the third year the European Summer School of Agile Programming(Essap) is a great opportunity to learn more about Agile Methodologies. Here the website of the school: check it out.

Just a bit of confusion...

2008-03-29T00:00:00+00:00

From a job offer recently received: [...]Please note it is essential that applicants have worked in a structured development environment ideally **utilising the Agile (SCRUM, XP) methodology or any other similar methods such as Waterfall**, RAD or RUP[...] Just awesome.

Express your business requirements through your code

2008-03-02T00:00:00+00:00

Last week I was working with a colleague on the implementation of a fairly simple validation of a character: the criteria for validation was that the character had to be lower case or a digit.

Looking at the Javadoc we spotted a couple of useful methods in the Character class: isLetterOrDigit -this method is pretty straightforward- and is LowerCase that returns false for upper case letter and everything is not a letter.

The code we wrote looked like:

if (!isLowerCase(aCharacter) &&!isDigit(aCharacter)) {
    //...
    //error !!
}

All the tests passed. Fairly simple and coincise solution.

But..are we actually expressing the business requirements ?

No. Not al all.

Our code is not communicating clearly what we’re trying to achieve: the API is not clearly expressing that punctuations are not allowed.

The expected behaviour is there but the code is still not in an ideal shape: we need to be able to clearly express what we really need to validate.

The refactored code looked like:

if (!isALowerCaseCharacter(aCharacter) &amp;&amp; !isDigit(aCharacter)) {

where isALowerCaseCharacter was:

private boolean isALowerCaseCharacter(char aCharacter) {
    return isLetter(aCharacter) && isLowerCase(aCharacter);
}

This is better. We are might be increasing the actual complexity of the condition but our code is now expressive: if in 3 months time we pick up this code again we clearly know what business rule we’re applying without looking at <any< additional resource.

The code now is communicating our intention.

EasyMock experience

2008-02-23T00:00:00+00:00

For the last two weeks I’ve worked with EasyMock and coming from a JMock background it’s easy to make a comparison between the two libraries.

I have to say that I’m less than impressed by EasyMock: the whole concept of the two different states (recording and active) for the mock library looks unreasonable.

Let’s look on how we can create a mock with EasyMock:

MyInterface mock =EasyMock.mock(MyInterface.class); //expectation mock.myMethod(); //activation step mock.replay(); //actual call to the mock mock.myMethod(); //verification that the method has been called mock.verify();

There’s also a more DSLish style of defining expectations, that I personally prefer (it differentiates clearly the definition of the expectation from the actual method call).

EasyMock.expects(mock.myMethod());

This style is the only one available when the expectation is more complex:

EasyMock.expects(mock.myMethod()).andReturn(true);

But what if I want to define that a certain method will be called on the stub, no matter how many times ?

I can either use a different way of creating the mock: EasyMock.createNiceMock(MyInterface.class);

or using the DSL way:

EasyMock.expects(mock.myMethod()).anyTimes()

I think that having two ways of defining the same expectation is pretty confusing, specially when you’re using the API for the first time. What I’m looking for in a mock library is the capability of defining the expectations in a coincise but expressive way and then inject the dependency in the object I want to test.

JMock2 is pretty close, but I have to admit the the inner class notation with all those curly brackets around is not helping.

A fellow ThoughtWorker have just released Mockito: it looks like it’s taking the best from EasyMock and JMock and put in a single library. Might be worth a try.

The last "D" in TDD means more than just "Development"

2007-12-05T00:00:00+00:00

Here some interesting thoughts on TDD.

Java enum as a strategy pattern

2007-10-27T00:00:00+00:00

I’ve always thought that Java enums can represent a simple but powerful strategy pattern baked into Java.

This idea has been reinforced by this article by Ralph Johnson of GoF fame, where while talking about design patterns and language design he says:

Design patterns are not made of stone. […] Over time, we should change our programming languages so that they build in things that used to be patterns.

This evolution of modern languages will (hopefully) increase the number of people that are really able to write Object Oriented code, modelling behaviours and not only using objects as a dull namespace construct.

Useless comment

2007-04-25T00:00:00+00:00

Can a name attribute in an Entity class represent something different than the name of the entity?