The 90th percentile

Sublime Text Editor tricks for web developers

2012-09-24T15:28:00.001+02:00

Last weekend we were at Apache Barcamp Spain with a talk about cool tools for web developers.

Aside: if you are coming to a single event in Spain on a given year, make it this one. Barcamp Spain has developed its own soul, where the beer+networking experience is so much fun, and the food by itself is totally worth the trip.

Some people were asking for a transcript of the third part, our own set of tips and tricks to boost Sublime Text Editor for web development. So here it comes:

Feel comfortable

Chances are Sublime Text Editor is not your first editor. You probably had a previous life, in which you were already proficient with some editor (Eclipse, Bean, TextMate, whatever). This is about getting things done, so I would recommend to configure Sublime with the typical shortcuts that are hard-coded into your brain. Me, I am used to Eclipse shortcuts.

The idea is to feel comfortable. Get whatever you need to make Sublime as good as your previous editor of choice. Now, awesomeness can start.

Maximize space

I am not quite a fan of the distraction-free mode in Sublime, but I do want to maximize the used space on screen (the data-ink ratio, for Tufte fans). So here it goes:

Toggle sidebar on/off: Ctrl + K, Ctrl + B. This is useful if you rarely use it. In my case, it's Shift+Ctrl+R to open any file, which makes the sidebar mostly ignored.
See several files at once: Shift + Alt + [1234589] (you have to choose one). You can compare them side by side, top vs bottom, etc. Once you are done, you can switch back with Shift+Alt+1.

Surround with tag: Shift+Alt+W

This is a useful shortcut for inline tags (bold, italics, links), but not so much for block tags (div, p, etc). Try it:

Write any text.
Select a word, press Shift+Alt+W, then type b.

Block select: Shift + Right-click drag

This is great when you need to insert tags at the start of each line (p, li).

Prepare your list of data
Select the code and hit Tab to indent it a bit.
While holding the Shift key, right click and drag to select a rectangular selection of code.
Now type the start tag to get it inserted at the beginning of each line.

Multiple cursors

The previous are just concrete examples of using multiple cursors. For the purpose of understanding, let's say you can have multiple instances of a SublimeCursor class (I do not have any insight of the source code, but I found this approach easier to understand). This is a compiled list of things you can do, extracted by trial and error:

Ctrl + click: create new cursor.
Ctrl + double click: create a new cursor selecting the word that was double-clicked.
Shift + click: select using the latest cursor (so: ctrl + click, then shift + click to add a selection to the set of cursors)
Any keyboard action: Operate on all cursors at once. Keyboard operations (like Ctrl+V) will work, and so will normal typing (like hitting <end> to write the ending tag at the end of each line).

This is best understood with a set of examples:

Write any text.
Ctrl+click at the beginning of each line.
Write <p> or <li>

Forgot to enclose tag attributes with quotes?

Ctrl + double click on every attribute
Type "

Ctrl + double click on words that you want to make bold.
If you want to select more than one, Ctrl+Click at the beginning, then Shift+Click at the end.
Hit Shift + Alt + W, then type b

There are tons of possible applications, and we found ourselves doing some really funky stuff. What is your personal trick? Use the comments section or tweet me @nachocoloma.

Using LiveReload on Linux

2012-06-11T10:00:00.000+02:00

This is a guest entry from José Roberto Vidal, our newest acquisition at Extrema Sistemas. It's a perfect combination between me, not having enough time to write something coherent, and Roberto having something interesting to say and nowhere to stick it on. You can also find a Spanish version here.

LiveReload is an application that allows you to refresh automatically your browser whenever any file gets modified, even compiling any Sass / Less / CoffeeScript files. In a nutshell, it avoids you the tedious-to-death Alt+Tab F5.

Its installation in Mac OS or Windows is quite straightforward, but things get a bit trickier with Linux (surprise, surprise). This is what this post is about.

Installation and configuration

Where Ruby gems get involved:

sudo apt-get install ruby1.9.1 ruby1.9.1-dev

Now you can use RubyGems to get the application. Inside your project folder, do:

sudo gem install bundle
sudo gem install guard
sudo gem install guard-livereload
bundle init

Edit the recently created Gemfile and add these two lines:

gem 'guard'
gem 'guard-livereload'

And execute

bundle exec guard init livereload

Monitored files and browser extension

Your generated Guardfile includes some predefined monitoring folders that you may have to modify if yours is not a Rails project. After that, just start the Guard server:

$ bundle exec guard
Guard uses NotifySend to send notifications.
Guard is now watching at '/home/me/workspace/project'
LiveReload 1.6 is waiting for a browser to connect.
>

At this point you probably can't wait to crack your knuckles and give it a try... Download the LiveReload extension for your browser (Chrome, Firefox) and configure it (right click -> options) to specify the hostname and port used by the Guard server (by default localhost:35729).

Once enabled, the guard-livereload server will notice it:

> Browser connected.
Browser URL: http://mymachine:myport/

Disco!

If you get here, any modification to monitored files (CSS stylesheets, images, even JavaScript files) is noticed and applied immediately at the browser without refreshing.

Guard comes whith a lot of different flavors to automatize tasks in different environments. Get your own, or follow their nice screencast.

This is one of the labs that you can find in our HTML5+CSS3 course, together with Bootstrap, Sass and another 25 labs. It's just three days that you may choose to come to one of our public scheduled dates or make one specific for your own company.

Eclipse shortcuts for Sublime Text 2

2011-10-28T23:10:00.000+02:00

I am a Linux guy. I used to pimp gedit with all plugins under the sun in an effort to transform it into TextMate, but no matter what I try it keeps being an awkward coding experience. Still on the search for alternatives, last month I discovered Sublime Text 2 and its entire new way of understanding text editing.

Sublime Text is just perfect for lightweight coding. I am not going to get into the details, since there are better places documenting this (see for example this list of tips and tricks). It is the perfect tool for times where a full-blown IDE is overkill: after all, it makes little sense to open Eclipse if you are going to spend the next eight hours debugging JavaScript or HTML (Agile and all that).
Now, this is the thing: I work with multiple major projects in the same week, performing different tasks and roles. I switch environments A LOT. Having to think "where am I editing?" before duplicating lines or saving files was messing into my flow.
So, without further ado, behold the

Ultimate eclipse shortcuts for Sublime Text 2

Not exactly rocket science, but still: go to Preferences -> Key bindings - User and paste this:

[
 { "keys": ["shift+enter"], "command": "run_macro_file", "args": {"file": "Packages/Default/Add Line.sublime-macro"} },
 { "keys": ["alt+up"], "command": "swap_line_up" },
 { "keys": ["alt+down"], "command": "swap_line_down" },
 { "keys": ["ctrl+alt+j"], "command": "join_lines" },
 { "keys": ["ctrl+alt+down"], "command": "duplicate_line" },
 { "keys": ["shift+ctrl+r"], "command": "show_overlay", "args": {"overlay": "goto", "show_files": true} },
 { "keys": ["ctrl+shift+s"], "command": "save_all" },
 { "keys": ["ctrl+l"], "command": "show_overlay", "args": {"overlay": "goto", "text": ":"} },
 { "keys": ["shift+ctrl+f4"], "command": "close_all" },
 { "keys": ["shift+ctrl+y"], "command": "lower_case" },
 { "keys": ["shift+ctrl+x"], "command": "upper_case" },
 { "keys": ["ctrl+d"], "command": "run_macro_file", "args": {"file": "Packages/Default/Delete Line.sublime-macro"} }
]

Save, and you will have a selection of finest eclipse shortcuts configured right into your editor. It is as far as I could get without getting into macros. You can help yourself and find the complete list of commands under Preferences -> Key bindings - Global, and complement this with our own discoveries.
If you like this, do not miss our new post about Sublime Text Editor tips for Web Developers.

Apache Barcamp Spain: a wrap-up

2011-10-13T10:47:00.000+02:00

After ten years of IT events, one would think that I have already seen everything under the sun: Spain, Germany, Switzerland, Flanders, Java-related or not, we all end up doing similar things. Come, learn something, attend networking, maybe get some book signed by some mega-crack author with a funny name. Exchange business cards or LinkedIn ids or Twitter names or QR codes. That's it.

Then something like Apache Barcamp Spain gets together and it's like - WOW.

If I had to coin some term, it would be "beer-oriented event". This being my first barcamp, I must say that the format is a fresh and new perspective much more interesting that the typical "get together and talk" mumble-jumble.

The talks

First off, you don't know what you are getting into. Literally. You make the trip to Seville, which in our case means getting three people into the A-Team van to co-star a 500+ km of roadtrip movie. You do all that, yet you don't know the talks that will happen the day after. Everyone proposes a talk and a voting process happens, and for all you know you could end-up learning new, surprising flavors of freaky.

Fortunately this was not the case: from Play! framework to GIS, Agile, Maven, Groovy, Web or Mobile, the landscape was full of well-seasoned experts. I loved the talks, but also the pace: instead of the typical 45-minute sessions, these were shortened up to 30 which leaves just the time to get to the point. No fluff, no fillers, just beans.

With three parallel tracks, you know you will miss 66% of the talks. I know I did, and some great talks are not included here. Sorry for that.

The following is my contribution about CSS. Bear in mind that these are introductory talks, but I tried to make them fun and throw something in for the most experienced.

Css for nondesigners

View more presentations from Ignacio Coloma

Css3 101

View more presentations from Ignacio Coloma

The people

CELEBRITIES. All of them. Hey, we got a T-shirt to prove it.

Seriously, the organization made a big effort to make it clear that this show was about the people. I mean, of course all events do, but here you could really feel the love. Beer-time was the required great excuse to meet great people, and I found this to be one of the best formulas to get in touch. Relaxed and natural, like friends at a party.

Pity that we were just 100 people. I know quite a crowd that was left out because of the limited capacity.

The party

This event was co-sponsored by several companies, Extrema between them, but one company really stood out: Atlassian.

Atlassian sponsored the closing party, and a huge one at that. Flamenco, beers, jamón serrano, mojitos when the time was right. It was exactly the kind of party I would ask of Seville.

Funny fact of the night: After one hour of animated chat, I ended up recognizing Carlos Sanchez because of the avatar in his business card. Avatares reales ya :)

Style your forms using JavaScript or CSS

2011-09-22T20:40:00.000+02:00

These days we are hacking like crazy in CSS land, trying to introduce in our project some nice memories from our Java background like Not Reinventing The Wheel or avoiding the NIH syndrome. Which brings us to this little nice thought:

I don't want to style buttons EVER AGAIN.

I simply have some better things to do. Call it "button", "a.button", "input[type=submit]", I don't care. I just want them to look gorgeous, be as lightweight as possible, and look the same in all browsers. Since it's 2011, I will get all cocky and also want them to be free.

Since we are at it, let's take a look at how to style form controls, period.

Style your widgets using JavaScript

Aristo is a great theme for jQuery UI that includes some great styles for everything under the jQuery sun, which includes buttons of several kinds. It's really great if your needs are not that simple (complex layouts with something clever like sliders or datepickers).

The only problem is that it relies on jQuery to initialize the correct Look & Feel artifacts, which means that during the first access to your web page there are a couple of seconds of my patented YourButtonsLookLikeCrap (tm) effect. There are plenty of workarounds for this, like creating the whole button markup using javascript.

If you need something less heavy there is a project to get Aristo buttons using just CSS, but it's still too green. Better go with skeleton.

Style your controls using just CSS

Skeleton is an extension over the 960 grid system. They extend the grid concept by adding extra typography and media queries oriented to mobile, and it absolutely excels at providing lightweight form controls. It's likely that it will not fit perfectly into your own design, but the stylesheet is really easy to understand and you can tune it to your needs. It's also extremely lightweight, in case that's a concern.

Style your select controls

You know those huge combo boxes that happen every now and then, when reality hits your application? With Chosen you can replace them transparently:

They have plenty of examples of controls, including ways to provide for your Ajax needs.

I know there must be other exciting tools out there that I am missing. How do you make your forms rock? Please share!

Aside: These be Crazy Months

I just came back from spending my whole summer in Beijing. It was an incredible experience that I am absolutely planning to repeat in February, this time with Shanghai. So far I now just the survival basics like "hello", "thanks", "chicken" and "tea". With that (and smiling a lot) you can survive a whole month and get a grip of the culture, which by the way is quite impressive. I also managed to save some time to learn chinese chess, and I can proudly say that I'm absolutely lame at it.

Now I just got back to Spain! Next month I'll be at Apache BarCamp Spain with a CSS for non-designers and CSS3 101 talks, hope to see you there to share some beers and plenty of chinese stories.

Firefox 4 and 3.6 at the same time

2011-05-24T09:00:00.005+02:00

With the arrival of HTML5 web development has turned into the sunny side of the hill again. These days the world would be a perfect place if I could launch two browser instances at the same time, one with support for the latest standards (almost) and another without.

This is something that has been documented online, but from fragmented sources: it is possible to have Firefox 3.X and 4 installed and running at the same time. Actually, it's quite simple as the whole installation process takes less than ten minutes.

Step one: Installing the thing

This one is a no-brainer: you should install both Firefox 3.6 and 4 in separate folders. For Windows, this means using different folders and creating separate shortcut icons.

For Linux, it means installing Firefox 4 from your favorite package manager (.rpm or .deb) and downloading Firefox 3.6 as .tar.gz and uncompressing it somewhere like /usr/local/firefox3.6 (depending on your Linux version, you may prefer to do it the other way around).

In both cases, you need the link to download an older version of Firefox, which is somehow hidden in their website (search for "All systems and languages" and "Other Firefox Downloads", or just click here).

Step two: add -no-remote

Firefox will check at launch time if there is a running instance and attach to it instead of starting a new one, which is definitely not what we want since it will not check for versions in the process. Launching 3.6 will result in your existing Firefox 4.0 window getting the focus, but nothing else. No new window. No "ta-daaaa!"

This behavior can be overriden by adding -no-remote to your program shortcuts (Windows or Linux). That will skip the check, and your new window will open.

Step three: use different profiles

You may be able at this point to launch separate windows for Firefox 3.6 and 4 at the same time, but this is no good if they start stepping on the plugins of each other. The plugin release that may be valid for 3.6 is not for 4, and viceversa (fact is, anything other than Firebug has big chances of not having been ported to Firefox 4 yet). You need separate plugin folders.

The good news is that your plugins are stored in a folder relative to your Firefox profile. Most users (99.9% or so) have a single "default" profile that is selected automatically and are not aware about profiles (now you are; lucky you!). Just launch:

firefox -ProfileManager

Create two separate profiles, which will allow for two separate sets of plugins. I like to name them "Firefox 3.6" and "Firefox 4", in an attempt to be original and unexpected. They will be stored inside your user home folder. Accept and close the window.

To avoid the profile selection window over and over again, the profile can be set at your Firefox launcher (or shortcut icon, if you are using Windows). I will assume some locations and profile names here:

Linux

/usr/bin/firefox -no-remote -P "Firefox 4"
/usr/local/firefox3.6/firefox -no-remote -P "Firefox 3.6"

Windows:

"C:\Program Files\Firefox 4.0\firefox" -no-remote -P "Firefox 4"
"C:\Program Files\Firefox 3.6\firefox" -no-remote -P "Firefox 3.6"

That's it. Now you can develop using both versions with separate Firebug installations.

More. More. More.

This is an excerpt of the first lab in our HTML5 and CSS3 course that is starting tomorrow. This 4-day course includes 20+ labs filled with polyfills, mobile and print CSS, Internet Explorer workarounds, transitions, form controls and mostly everything under the CSS3 / HTML5 sun.

Applications dealing with multiple timezones

2011-02-10T10:00:00.006+01:00

This is a requirement that can hardly be considered common. Most applications are not forced to deal with more than one timezone: intranet applications, nationwide deployments (most European Countries enjoy this great invention that is Central European Time), even air navigation systems place everything in UTC.

We recently deployed Koliseo on AppEngine. This is still a work in progress, where some features have raised interesting questions: what happens when a friend tells you to meet on Friday at 17:15? How should you store this in the database?

I love the way computers measure time: milliseconds since 1970. Holy cow. No leap years, no arbitrary seconds-in-a-minute conversion, no Daylight Saving Time and it doesn't matter where you are. Give me a number that I can sort and compare. Computers do not have problems with dates. Humans do.

Then, the only problem is parsing and formatting these dates.

Parse date

Ye olde way of providing time information says that you should put a test field, label it "when" and let the user do his thing. Suppose that he introduces "25 Oct 2010 at 00:00", sitting comfortably in his nice office in Madrid (GMT + 1, where DST applies).

What the user means: "25 Oct 2010 at 00:00" in NiceAndWarmOffice@Madrid
What the server understood: "25 Oct 2010 at 00:00" in local server time. For AppEngine this is UTC, which means one or two hours less, depending on the season.

The user wanted to express Oct 25 00:00 (Madrid time), but the server understood Oct 24 23:00 (Madrid time). This is the value that will get stored in the database.

You need to ask for the timezone of the user. It can be automatically deduced using javascript (it comes in small and jumbo sizes), but it is nice practice to let the user modify the timezone later, just in case.

This timezone must be stored associated to something, which in our case is a venue. From there on, any date introduced in Koliseo must be associated to this entity. Following this example, any Performance must specify where before telling when if we want to be able to parse the introduced timestamp.

The code to parse is quite straightforward thanks to Saint JodaTime:

DateTimeZone dtz = DateTimeZone.forID(timezoneID);
DateTimeFormatter formatter = DateTimeFormat
 .forPattern("yyyy-MM-dd HH:mm") // replace with your favorite format string
 .withZone(dtz)
 ;
Date date = formatter.parseDateTime(dateAsString).toDate();

Format date

How do you want to display dates to the user?

Relative units: This is the best way to display unambiguous dates ("ten minutes ago", "one week ago", etc). This is done in Loom by using l:formatDate, but the first case I can remember of this practice was found in Ruby
User time: This is usually not interesting, since a date is associated to something happening (in Koliseo, a performance) and is usually meaningful inside their own context and timezone.
Original timezone: This is the most common case. Example: tell me that the show is scheduled at 17:30 (local time of the show, in this case Madrid), not browser time (Helsinki or whatever).

To format dates in the original timezone you can use the same snippet of code just replacing formatter.parse() with format().

Javascript code

Whenever you send or receive a Date from the server, the typical JSON serialization uses "milliseconds from 1970", which can be a problem. This value cannot be used as is because the timezone information is missing. 17:35 in Madrid will be deserialized as 16:35 if I am browsing from London, which is wrong.

Twitter solves this by serializing dates as text, then you can parse and separate the parts to display (day, month, year, weekday). To test your javascript code, just add "-Duser.timezone=UTC" to your server launch script and play a little with your application. If you do not see any date mismatch, everything is good.

Loom 2.2 has been released

2010-11-01T18:14:00.001+01:00

We have just released Loom 2.2 with a lot of stuff to make web development easier:

Better javascript/CSS resource bundles

Loom has been supporting javascript and CSS dependencies for some time, and now they just got easier (think sprockets):

JSP file:

<l:css resource="style.css"/>
<l:script resource="main.js"/>

style.css:

/* 
// @include "myfile.css" 
// @include 'classpath:/mypackage/file2.css' 
*/

main.js:

// @include "myfile.js" 
// @include 'classpath:/mypackage/file2.js'

These dependencies can now be redefined without restarting the server. In production environments they will be concatenated, minified and gzipped - which brings us to the next point:

Added support for Google Closure Compiler

According to multiple sources Google Closure Compiler produces a better compression rate than YUI Compressor. This is nice, but what really got us is the awesome error reporting which even warns about IE gotchas (trailing commas, anyone?).

Google Closure compiler is now the preferred javascript compressor and will be used if it is present in the classpath. For CSS, YUI compressor is still the (only?) solution.

New Ajax Paged Table

Loom includes some great server-side paged components. With 2.1 we introduced a new Ajax paged list where the whole pagination process was performed by javascript code, and with 2.2 we are extending this to paged tables:

var paged = new loom.ui.PagedTable($('container2'), { 
        url: '<l:url mapping="Action.event" decorator="link-only"/>', 
        columnNames: [ 'timestamp', 'contents', 'user.name', 'user.avatar' ], 
        // optional fields: specify CSS class names and cell renderer
        columnClassNames: ['date'], 
        columnRenderers: [ function(o) { return o.timestamp == null? '' : new Date(o.timestamp); } ] 
  });

This is an alternative to server-side paging. Just use the solution that suits you.

Better fluent interfaces

Some interfaces got better at handling HTTP headers and cookies:

// the bad: 
Resolution res = forward() 
   .set("foo", "bar"); 
res.addCookie("baz", "bar"); 
return res; 

// the ugly: 
getResponse().addHeader("baz", "bar"); 
return forward(); 

// the new API:
return forward() 
   .set("foo", bar") 
   .addCookie("baz", "bar") 
   .addHeader("Content-Type", "application/javascript") 
;

CacheControl has also been improved:

// Before: 
CacheControl c = new CacheControl(); 
c.setCacheYears(10); 
c.setLastModified(new Date()); 
return forward().withCacheControl(c); 

// After: 
CacheControl c = new CacheControl()
   .withYears(10)
   .withLastModified(new Date());
return forward().withCacheControl(c);

Others

There is a lot of smaller improvements such as:

Customizable reCaptcha L&F
loom-messages.properties has been translated to italian (thanks to Marco Bazzani)
We have switched from commons-logging to slf4j because performance is just better

Work on the next release (2.3) has already started, including the redesign of some internals to address Google AppEngine development with Guice. It's exciting!

Loom 2.2 is almost out: things we learned so far

2010-10-04T11:00:00.005+02:00

We just got out the new e-banking portal for BBVA Suiza. BBVA Suiza is the BBVA Group's International Private Banking unit, one of the most visible projects we had originally developed with Loom 0.8 - now with 2.2 - and a lot has happened since then.

This post is about things we learned during these last years, and how they were combined to make our experience extremely enjoyable.

It's all about the UI

The most interesting discussions during this project were not about technology or banking products, but about the interface. We spent literally hours discussing one single page, trying to fit things in the most natural way, getting all the non-essential out.

Anyone can design an interface that takes two weeks to learn, but it gets much harder to design something intuitive. It takes a lot of work to arrive to your hamburger interface. On the way we have challenged the breadcrumbs location, the tabs metaphor, the nesting of contents, everything.

It's not about stateless or stateful

It's not 1999 anymore. The caveats of stateless and stateful interfaces are well known, and unless you are a zealot of some concrete Church of Thought, your sweet spot will most probably be somewhere in the middle. Free your server resources by moving more to the browser. Remember that sometimes light pages will do better than a super XXL javascript golem. Get faster responses by avoiding a full page refresh. Do not put too much stress on javascript unless you want to spend your life debugging. Hate IE over everything else. Remember your SEO.

The list is long, and you should make your own choices. Just remember not to waste too much time arguing about which way is better.

Speed!

Any idiot can carve lots of features into your application and make it clunky, take years to download, and lose the train of thought of your users in the eternity that happens between one page and the next. If you aim to make your users rock (as you should) your application should feel natural, which means working towards the 1 second goal.

There are lots of tricks to get there. We have posted some here and there, but nothing beats simplicity: there is nothing faster than not including a feature.

Of course, your customer may not always agree. This is were the next point gets relevant.

Get your customer on your same train

You can align your customer with your own goals just by delivering Software That Works, so don't try to agree to all crazy features or deliver the supah-dupah-canvas-based forms with embedded music. It's important to make a distinction between what the user asks and what he needs, and it helps if the rationale behind a refusal is clear and shared by everybody.

So?

We are not there yet. There are things that we would do different and legacy features we would love to drop, but all in all, everyone is happy. Their customers are calling just to send congratulations (how often do you see that), and suddenly we found ourselves talking to another three banks that want to build something similar.

The technology helped a lot. But what really allowed us to deliver was all the weight that was ditched by the way.

SimpleDS 1.0 is released

2010-09-13T15:35:00.001+02:00

This week we are releasing version 1.0 of SimpleDS. SimpleDS is a simple persistence framework for Google AppEngine that provides an alternative to JDO or JPA.

This release brings a lot of new features:

Cached queries

This is the star feature of this release. Starting with 1.0, all query results can be cached. To maximize cache performance only the returned primary keys will be cached, so you still have to use @Cacheable to cache the entity itself.

There are cases were this feature is killer, e.g. to retrieve a User given the email:

User user = entityManager.createQuery(User.class)
  .equal("email", googleUser.getEmail())
  .withCacheSeconds(3600)
  .asSingleResult()
  ;

DEBUG CacheManagerImpl - Level 2 cache hit: qdata{kind=User,pred=[email = test@example.com]}
DEBUG CacheManagerImpl - Level 2 cache hit: User(5)
DEBUG Level1Cache - Level 1 cache hit: User(5)

Cached queries work with FetchOptions (cursors, limit and offset) and support only count(), asSingleEntity(), asList() and asPagedList(). Any invocation to asIterable() / asIterator() will ignore the cache. To clear cached data, just prepare the same query and invoke clearCache().

More about cache here.

Better fluent interfaces

The syntax has been simplified from this (older version):

SimpleQuery query = entityManager.createQuery(User.class).equal("name", "foo");
return entityManager.find(query);

To this:

return entityManager.createQuery(User.class)
  .equal("email", email)
  .equal("enabled", true)
  .sortAsc("email")
  .asList()
  ;

Now we rarely use more than one line for most queries. In this example, if email is null it will just be ignored.

Several methods have been added: asList(), asSingleResult(), asIterator(), asIterable() and PagedQuery.asPagedList(). The old EntityManager methods have been deprecated.

More about queries here.

New Functions

We are also adding some new Functions for transforming persistent entities. Some examples of use:

// save space for your relationships by storing Set instead of Set
Set userIds = user.getFriends();
Collection userKeys = Collections2.transform(userIds, new IdToKeyFunction(User.class));
Collection users = entityManager.get(userKeys);

// transform back
CompositeFunction func = new CompositeFunction(
 new EntityToKeyFunction(User.class), 
 new KeyToIdFunction()
);
user.setFriends(Collections2.transform(users, func));

// just return a collection of email addresses
return Collections2.transform(users, new EntityToPropertyFunction(User.class, "email"));

More about functions and transformations here.

Added JRebel support

I personally use JRebel for development, which means that I rarely restart my development server. This was a problem with SimpleDS, which was unable to detect changes such as new persistent attributes, etc. SimpleDS can now be used with JRebel just by adding this to your startup code:

if (SystemProperty.environment.get().equals("Development"))
   ClassMetadataReloader.register();

Then in eclipse ("Go to your launcher config -> Arguments -> VM Arguments")

-javaagent:/usr/local/java/appengine-java-sdk/lib/agent/appengine-agent.jar -noverify
"-javaagent:${env_var:REBEL_HOME}/jrebel.jar"

Minor changes

Added @Property.converter to override the default converter for one persistent property. You can, for example, store a String attribute as Text.
Added new methods SimpleQuery.withReadPolicy(ReadPolicy) and SimpleQuery.withDeadline(double)
Added new methods SimpleQuery.withStartCursor(Cursor) and SimpleQuery.withEndCursor(Cursor)
Cache settings will be ignored when invoked within a transaction.
We have moved from commons-logging to slf4j. This may break binary compatibility, but is a huge boost in performance.

Let me get off-topic for a while

Last week this very same blog reached the 400-subscribers mark! It's awesome to find so many people interested, and I find extremely rewarding the contacts I receive every now and then, sometimes just to say hi. Last week it came from Australia! (hey Ángel).

So this is me, taking a small detour to say thanks. And yes, I will try to post more often :))

The current state of logging frameworks

2010-09-06T17:40:00.007+02:00

Logging is a commodity, which is argot to mean that I simply don't care as long as it gets out of the way. Recently I had to revisit my own logging decisions based on a single tip: Google Closure Compiler (the library we are using to minify our javascript files) was ignoring my logging setup.

It's like opening the closet you buried ten years ago when log4j was the one and only available choice.

First, the basics

One thing is the logging framework (the thingie that writes stuff out):

log4j: Started by Ceki Gülcü back in the day, it was the framework used by the Java community until Sun decided to include a different logging implementation in the JRE. After Ceki split from the project, it has been mostly dead in the water.
Java Util Logging (JUL): First included with JRE 1.4.2, it is a simple (some would say naïve) implementation that includes just a handful of possible logging setups. You can extend it, but you should code it yourself.
Logback: It seems that Ceki Gülcü didn't stop after leaving the Apache Software Foundation. Logback is an implementation of the SLF4J API that can replace log4j or JUL.

A separate concept is the logging isolation library, because when you are implementing a library you cannot require (or expect) a specific logging framework:

Commons Logging: Supported by the Apache Software Foundation, this framework has been sadly well-known because of their memory leaks. At the time of this writing their last release dates back to 2007.
SLF4J: A logging API layer by Ceki Gülcü that includes adapters for Log4J, JUL, and almost anything you can think of. The adapter plugin mechanism is quite well-thought, just drop your jar in the classpath and you are ready to go.

The Google case

Google makes a point of not including external dependencies with their libraries, and when they do the package name is jarjar'ed to avoid version conflicts. This would not be needed in an ideal world, but as long as framework designers keep making backwards-incompatible changes I think it's a sane decision. Anyone that has used ASM or even Google Collections / Guava knows what I'm talking about (fun trivia fact: the earliest occurrence of this practice I can recall was inside Weblogic 8.1 in 2003, but my memory is not what I would call a trustable source :)

With this strategy logging becomes a problem because at some point you cannot keep renaming classes. This makes the JDK logging the only remaining alternative - for Google, that is.

The choice

A bit of history here: when log4j was out, it could write anywhere: JMS, Database, Email, the OS event system, Grandma's Cheesecacke Recipes Book... But that was soooo 1999. Today we have aspects, servlet filters and all kind of interceptor mechanisms that make these options obsolete. Of the handful times I have configured a log that goes anywhere other than the standard output, I have always ended up replacing it with a more elaborate solution.

"Zillions of appenders" is thus not exactly an important feature in my book. Performance, stability and support are. A quick search using Google pretty much confirms that the competition is over. SLF4J is being widely used by other frameworks, includes an awesome support (with releases every couple of months, the latest at the time of this writing was July 2010), no major known glitches, and fits perfectly with all the logging frameworks mentioned before.

About the logging framework... well, I will keep playing with JUL and Logback, but log4j is left behind as far as I am concerned.

UPDATE: You can also see a longer explanation by the SpringSource team here.

URL-based locale

2010-08-05T18:35:00.000+02:00

We should assume that users will not always be browsing from a comfortable location (pronounce: home / work). Even if they do, every now and then I have guests at home that do not speak Spanish at all. Try opening the Google home page in a cybercafé in Germany or Finland to see what I mean. It may be convenient, but the browser language is often not good enough for the real world.

If your application supports multiple languages you should give the user an option to change the locale without messing with the browser settings. Specifically, don't force the user to find the settings in a foreign language and foreign browser, and don't assume that he/she is allowed to change them.

There are a couple of possible implementations for this:

Store the locale in the user session: this solution does not persist the locale during browser restarts and implies an existing session for each anonymous user, which is not always an option.
Store the locale in the user settings (database): this is possible only if you do not have anonymous users.
Store the locale in a browser cookie.
Store the locale in the URL

Of these options only the last three have reasonable quality, only the last two work for public websites, and only the last one works for web crawlers. Google recommends to use locale-aware DNS names, which is lingo speak to put your locale somewhere in your DNS name. If you are reading this sitting comfortably on your big heap of money, you may start registering "myhost.es" and "myhost.de", but the rest of us will go with "es.myhost.com" and "de.myhost.com".

Implementing this in your own application is easy: Loom has PrefixLocaleResolver, but with other frameworks you may just use your own web Filter to resolve the user locale as well. Naïve version follows:

public class LocaleResolverFilter implements Filter {

  @Override
  public void doFilter(ServletRequest req, ServletResponse response, FilterChain chain)
      throws IOException, ServletException {

     HttpServletRequest request = (HttpServletRequest) req;
     String serverName = request.getServerName();
     String lang = StringUtils.substringBefore(serverName, ".");
     if (lang.length() != 2) {
         lang = "en";
     }
     
     final Locale locale = new Locale(lang);
     chain.doFilter(new HttpServletRequestWrapper(request) {
      
          @Override
          public Locale getLocale() {
              return locale;
          }
      
     }, response);
  }
  
  @Override
  public void destroy() {
      // empty
  }

  @Override
  public void init(FilterConfig filterConfig) throws ServletException {
      // empty
  }
}

In order to test this, you should add this entry to your hosts file:

127.0.0.1 localhost es.localhost en.localhost www.localhost

Me, I find it funny to think about my "English localhost" :) If you have any interesting ways of detecting your user language, I would love to hear it!

Markdown and your website

2010-06-15T13:00:00.002+02:00

Some ideas take a while to be shared between communities. Markdown is a concept originally developed in Perl that all GitHub developers are familiar with since it is the format behind GitHub pages, and basically it shares the fresh, lightweight smell of wiki pages:


This is a title
===============

You can use *italics* or **bold** or ***both***.

  - Lists
  - [Links][1] 
  - etc

  [1]: http://foo.net/

You can edit markdown with a real-time preview using WMD, which has been open sourced by stackoverflow.com. For the server-side, the best java implementation I could get my hands on is pegdown.

There are some reasons that make this solution appealing:

Simple: Thousands of people are contributing to Wikipedia just because this format is much easier than HTML.
WYSIWYG editor, which is a strong selling point in our case.
Secure: As long as you are storing the markup instead of the HTML output, you have removed most of the security concerns associated with HTML contents introduced by the user.
Solid, coherent UI design: You cannot specify your own CSS classes and styles (nobody wants another myspace :)

In our case we replaced a bunch of fields in a domain object (three url attributes, two phone numbers, description, etc) with a single Markdown description. That's it.

Markdown is well-known in the Rails community (mostly located on GitHub), but Java developers are disseminated between java.net, google code, sourceforge and whatnot. I suppose this is the reason why these technologies are taking some time to root, which is a pity because there are lots of things in the Rails world that could make a difference also for Java.

Loom 2.0 is out

2010-05-10T18:00:00.008+02:00

A major upgrade of the Loom web framework has been released. We made it a major version change since it includes some breaking changes in places where backwards-compatible was just not worth it.

Tag documentation has been improved a lot, but there are other features also worth seeing:

Works with AppEngine

We have been developing our own share of projects with Loom, and some are running in AppEngine. To make this possible we have forked a subproject with AppEngine image validation, datastore type converters (Key, GeoPt, etc), and transparent support for binding SimpleDS paged queries into paged tags.

Cache page fragments

We have added session and application-level caching of page fragments, such as:


<l:cache id="application-menu" scope="application">  
   <!-- application menu goes here -->
</cache>

Fragments can be cached separated by locale or not. This implementation is really simple, and things like cache timeout are not included yet.

Event parameters

Events now accept parameters:


public class BlogEntriesAction extends AbstractAction {  

  // parameter name is required
  @Path("baz")
  public Resolution baz(@QueryParam("id") int id) { ... }  

  // parameter name is not required when embedded in the URL
  @Path("bar/{foo}")
  public Resolution bar(String foo) { ... }

  // You can mix embedded and non-embedded parameters, but
  // embedded are expected to be resolved left-to-right
  @Path("bazbar/{foo}")
  public Resolution bazbar(String foo, @QueryParam("bar") int bar) { ... }

}

New JSON support

We finally deprecated our quick-and-dirty JSON implementation to adopt jackson. This makes some nice features possible:


// return a JSON response
public Resolution doFoo() {
   return json(myObject);
}

// receive JSON objects as parameters
public Resolution bar(@JSON @QueryParam("baz") baz) {   /* ... */ }

// render as JSON inside a page
public Resolution doFoo() {
   return forward("mypage.jsp").setAttribute("bar", bar);
}

<script>
   alert(${l:json(bar)});
</script>

Uploaded files

A breaking change introduced in 2.0 is that file uploads are no longer being automatically bound to java attributes. From now on you must handle these parameters yourself, and this affects the way file validations are being performed:


@FileValidation(
  parameterName="uploadedDocument", 
  formats={ "txt", "rtf" },
  maxFileSize=1024000)
public Resolution saveDocument() { 
  fileManager.merge(getRequest().getFileParameter("uploadedDocument"));
}

@ImageValidation(
  parameterName="uploadedImage", 
  formats={ "jpg", "gif" },
  maxWidth=100, 
  maxHeight=100, 
  maxFileSize=1024000)
public Resolution saveDocument() {    
  fileManager.merge(getRequest().getFileParameter("uploadedDocument"));
}

New paged features

Now you can use more than one paged container (table or list) in the same HTML page, or inject a plain Collection to be conveniently displayed as a single page.

Also: JSON paged lists! PagedListData instances can now be serialized JSON and handled by javascript to render the same HTML contents generated by the server-side tag.

More details about the current paged tags status can be found here.

New information methods

A new set of static methods MessageUtils.info(), warn() and error() have been added, which can be invoked anywhere in your code.


public class MyServiceImpl implements MyService {  

   public void myService() {    
      entityManager.put();
      MessageUtils.info("save.success");  
   }

}

Messages are serialized in case of a redirect, and will be displayed on the next web request.

SSLPolicy

A new annotation has been added to indicate the level of encryption on the annotated events:

@SSLPolicy(REQUIRES_INSECURE)
public class FooAction extends AbstractAction {

        public Resolution event1 {
                // this event requires http:
        }

        @SSLPolicy(REQUIRES_SECURE)
        public Resolution event2 {
                // this event requires https:
        }

}

Recaptcha support

In this release we are migrating from JCaptcha into ReCaptcha, since it seems to be the only implementation designed to face Human Computation attacks. It is also much easier than JCaptcha to configure and use, which is a plus.

@Recaptcha
public Resolution save() {  ...}

<l:form action="Foo" event="save">
 <a:recaptcha publicKey="[publicKey]"/>
</l:form>

The bad news is that the scaffolding tool is not yet up-to-date so we had to drop it from this release. It will be added again when time permits :)

The full changelog is here, the list of breaking changes is here, and comments, as usual, are welcome :)

SimpleDS 1.0_RC1 for AppEngine has been released

2010-05-04T12:32:00.001+02:00

For those that know me from somewhere else, SimpleDS is our open source framework for persistence in AppEngine. It has been compared with Objectify and Twig, and has been mentioned in the Google App Engine Blog a couple of months ago.

This is our first feature-complete release of SimpleDS. Lots of things have been included in so little time.

Getting up-to-date with AppEngine

It's hard to keep up the pace with these guys. This release includes:

Unindexed attributes.
Cursors support.
IN and != clauses.

Cache

We have included a great Level 1 and Level 2 cache. If you come from JDO/JPA, you may already know what this means:

Level 1 cache: This is basically a Map bound to the current thread. Until the end of the current request, any get() invocation will check this cache first. If a match is found, no invocation will be propagated to GAE.
Level 2 cache: Datastore entities are also stored in memcache, which is a second chance to get a positive match.

Cacheable entities must be marked with @Cacheable, with an optional expiration time. This feature will work with single and batch get(), and the cache entries are updated with put() and delete() invocations.

// Invoke memcache or the datastore
List data = entityManager.get(key1, key2, key3);

// this does not invoke anything (resolved by the Level 1 cache)
MyData d2 = entityManager.get(key1);

Functions

We are going extremely functional these days. This release includes a package with functions to transform collections and PagedList instances, which can be combined with batch get() to get even better performance results. This is a simple example, equivalent to a situation quite common when using relations:

// n + 1 requests to the datastore
List data = entityManager.find(query);
Collection parents = Lists.newArrayListwithCapacity(data.size());
for (MyData d : data) {
  parents.add(entityManager.get(d.getKey().getParent());
}

// Transformations: 2 requests 
List data = entityManager.find(query);
Collection parentKeys = Collections2.transform(data, new EntityToParentKeyFunction(MyData.class));
Collection parents = entityManager.get(parentKeys);

That's all it takes to get all entities and their parents, and store them in the Level 1 cache so any request by PK will not hit memcache or the datastore. This release includes functions to retrieve parent and foreign keys, and also works with PagedList. We have taken two real-world snapshots with just this optimization (transformation + cacheable), applied to a single loop:

Before: http://www.flickr.com/photos/koliseocom/4575062969/in/set-72157623904289518/
After: http://www.flickr.com/photos/koliseocom/4575696456/in/set-72157623904289518/

Notice that "before" are also requests to the datastore, while most of the "after" are requests to memcache.

Background tasks

We are aware that background tasks are in the roadmap for AppEngine, but we needed these today. This started as an exercise to upgrade the datastore schema (add properties, delete entities etc) and ended up as a full reusable implementation of tasks that I expect to deprecate once that AppEngine includes its own, probably better, implementation.

Background tasks require adding a servlet to web.xml (and optionally appengine-web.xml as an admin-console entry) and configuring the tasks on application start using plain Java. Tasks can be invoked directly by cron triggers, queues or by POST requests.

public class WriteBehindCacheTask extends IterableTask {

 protected WriteBehindCacheTask() {
  super("my-task-id");
 }

 @Override
 protected SimpleQuery createQuery(TaskRequest request) {
  return entityManager.createQuery(MyClass.class);
 }

 @Override
 protected void process(MyClass entity, TaskRequest request) {
  // ...modify entity...
  entityManager.put(entity);
 }

}

Query cursors and execution deferral will be done transparently (no need to limit or handle cursors). There are some implementation superclasses depending on what you need to do, and some of them just use the raw AppEngine datastore and don't even require SimpleDS to work.

Other features can be checked out at the SimpleDS home page and the changelog. Any feedback is welcome :)

Our "Web Development in 2010" slides are out

2010-04-22T16:44:00.000+02:00

We just got back from Universidad Europea de Madrid after delivering a conference about current trends in web development.

This is an introductory talk of one hour, which implies that we had to leave a lot unsaid. Even so, the number of links to promote further reading this time is much higher than what we are used to.

Enjoy!

Developing web applications in 2010

View more presentations from Ignacio Coloma.

Create your own bit.ly using Base58

2010-03-17T18:00:00.007+01:00

Following the example of bit.ly, tinyurl, owl.ly, even coca-cola has joined the short URL bandwagon. Maybe it's because of Twitter, maybe the world is focusing on mobile devices or just saving bandwidth. In any case, it's not that hard to develop your own.

The implementation is quite straightforward: store the underlying URL in a database and return your primary key. This primary key when displayed as a number (Base10) is longer than strictly necessary; since each character can hold more than numbers, you are wasting space by constraining yourself to a decimal [0-9] range.

Enter Base58

Base58 is what you get after taking Base62 [a-zA-Z0-9] and removing any character that may induce to error when introduced by hand: 0 (zero), O (uppercase 'o'), I (uppercase 'i'), and l (lowercase 'L'). This concept was introduced to the general public by Flickr, which uses the following String:

123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ

To me, it makes more sense to use the natural order in the ASCII chart:

123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz

I will post here an example of the latest. For a Flickr implementation you can go here.


public class StringUtils {

 private static final char[] BASE58_CHARS = 
     "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz".toCharArray();

 public static String numberToAlpha(long number) {
  char[] buffer = new char[20];
  int index = 0;
  do {
   buffer[index++] = BASE58_CHARS[(int) (number % BASE58_CHARS.length)];
   number = number / BASE58_CHARS.length;
  } while (number > 0);
  return new String(buffer, 0, index);
 }
 
 public static long alphaToNumber(String text) {
  char[] chars = text.toCharArray();
  long result = 0;
  long multiplier = 1;
  for (int index = 0; index < chars.length; index++) {
   char c = chars[index];
   int digit;
   if (c >= '1' && c <= '9') {
    digit = c - '1'; 
   } else if (c >= 'A' && c < 'I') {
    digit = (c - 'A') + 9;
   } else if (c > 'I' && c < 'O') {
    digit = (c - 'J') + 17;
   } else if (c > 'O' && c <= 'Z') {
    digit = (c - 'P') + 22;
   } else if (c >= 'a' && c < 'l') {
    digit = (c - 'a') + 33;
   } else if (c > 'l' && c <= 'z') {
    digit = (c - 'l') + 43;
   } else {
    throw new IllegalArgumentException("Illegal character found: '" + c + "'");
   }
    
   result += digit * multiplier;
   multiplier = multiplier * BASE58_CHARS.length;
  }
  return result;
 }
}

An example of the expected output size:

44 = m
1431117682956369 = abc123ABC
// Long.MAX_VALUE
9223372036854775807 = CFq8pKn6mQN

This is hardly rocket science. You can find libraries to do this for ruby, javascript, PHP or even perl.

It's Base58 everywhere

Once you start doing this, it's nothing short of addictive. You can configure your web framework to use a Base58 converter for the key attributes in your URLs and start using '/shows/cr5W' instead of '/shows/234324323423'.

Everybody is doing this right now: Flickr, Facebook or YouTube (which seems to be using [0-9a-zA-Z_-]).

News - News - News

These days life is getting interesting by the minute. Last week our SimpleDS pet project got referenced by Google AppEngine Blog, and next April we are bringing the Javaspecialist Master Course to Madrid!

It's going to be awesome. This is the most advanced Java course I can refer to, and after some conversations with Dr. Kabutz he agreed to bring it to Madrid. It consists of four days of intense performance tuning, concurrency debugging, introspection, memory profiling and some neat, challenging Java puzzles. If you feel interested, you should consider to join us in April!

Generate your TLD using java annotations

2010-02-23T09:00:00.005+01:00

If you are stuck with JSP and keeping your TLD files by hand (like me), you are probably ignoring the elephant in the room:

Tag classes must be kept up-to-date with the TLD by hand. This means duplicate effort, a.k.a. "XML sucks".
For any non-trivial tag library, inheritance is a fact. In order to avoid replication in the TLD, your libraries are bound to use some dark XML magic.
IDEs are extracting JSP coding hints from the TLD file, so you should be introducing your documentation in the TLD by hand.
You are probably not sacrificing Javadoc in the process, which implies maintaining duplicate documentation: java(doc) and TLD.
Make it triple, if you want online HTML documentation.
There is no way to document deprecated behavior.

This March I will be 35, which sucks. Last year my brother Carlos Coloma celebrated his 25-year-old anniversary reading his MSc project with his colleague Javier Reyes Alonso: Generating TLD and HTML documentation from java annotations. Today we are releasing it as an open source project, TLDGen.

Instant gratification: click here to check out the generated HTML documentation.

TLDGen is a standalone library that does not depend on Loom. This is an important requirement to be able to generate our own TLD files.

Download tldgen.jar and add it to your classpath

First you should extract tldgen.jar from the tldgen zip file or from maven (group org.extrema-sistemas, artifact maven). This jar file contains just the annotations to get you running:


package com.acme.tags;

import org.loom.tldgen.annotations.Attribute;
import org.loom.tldgen.annotations.Tag;
import org.loom.tldgen.annotations.BodyContent;

/** 
 * This comment will be included in the generated TLD and HTML 
 */
@Tag(  
  dynamicAttributes=true,
  bodyContent=BodyContent.SCRIPTLESS,
  example="<l:sample name=\"foo\"/>"
)
public class SampleTag extends SimpleTagSupport {

  @Attribute(required=true)
  private String name;

  /** @deprecated use name instead */
  @Attribute  public void setFoo(String value) {
     name = value;  
  } 
}


package com.acme.tags;
import org.loom.tldgen.annotations.Function;
public class Functions { 

  /**
   * bar javadoc <p>this is a paragraph</p>
   */
  @Function(
     displayName="bar display name",
     example="Function example"
  )
  public static void bar() {    }

  @Function
  public static void baz() {    }
}

Execute tldgen

You should download tldgen.zip from google-code, uncompress it somewhere and execute all in the same line (we are assuming a standard gradle project structure):


tldgen
   -sourcepath src/main/java
   -subpackages com.acme.tags
   -tldFile src/main/resources/META-INF/acme-tags.tld
   -displayName "Acme Tag Library"
   -name acme
   -uri "http://acme.com/acme-tags.tld"
   -htmlFolder build/docs/tlddoc

This should generate the TLD docs in build/docs/tlddoc and one TLD file in src/main/resources/META-INF/acme-tags.tld. As an example, you can check out the Loom taglib documentation.

Caching user-supplied images in the browser

2009-09-16T11:55:00.002+02:00

We have been engaged in a big project for the last months. Despite of the nice performance marks, there was a warning in Google PageSpeed that was itching us badly: "The following resources are missing a cache expiration".

PageSpeed was complaining about images uploaded by the user. Resources referenced from CSS are conveniently handled by Loom, but user-provided files were not setting their cache headers appropriately.

The following is, to the best of my knowledge, the same algorithm implemented by blogger and wordpress to avoid delivering the same image file over and over again. The theory has been already covered here.

I will be using Loom in this example, but you can reproduce this using any web framework.

Choosing your Strong Cache Validator

Cache Validators are introduced in the http 1.1 spec. Here we will use a Strong Cache Validator to indicate that a resource should never expire. You should include both validator and resource ID in the URL, like "/resources/{id}/{cacheValue}".

The most well-known candidates are:

MD5: The file checksum can be calculated when the file is saved or when the application starts up. This is how Loom (and others) set the cache headers for CSS and javascript files.
Version numbers: any @Version field can be used to identify persistent JPA entities.

In this case we will be using PersistentFile, which includes both fields. After trying both, we found that using @Version yields much shorter URLs ("/resources/151/2" instead of "/resources/151/3b4efe6405d4fb1ada4a081f4dbef6a9").


@Entity
public class PersistentFile { 

 /** the primary key value */ 
 @Id @GeneratedValue private Integer id; 

 /** the MD5 hash of this file */ 
 @Column(length=32) private String MD5;  

 /** the revision number */ 
 @Version private Integer version;  

 /** the last update timestamp */ 
 private Date lastModified; 
 
 /** ... etc ... */

}

This field will be used to set "Expires" and "Cache-Control" headers to expire ten years in the future. As long as the user is clicking links to browse normally these resources should never be fetched again, but it will be ignored if the user clicks the "Refresh" button, which is where Weak Cache Validation comes into place.

Weak Cache Validator

With Weak Cache Validation the browser must ask for the resource, giving the server a chance to answer with a 304 (not modified) or 200 (OK) response.

For this purpose we will use the "Last-Modified" header. The first request by the browser (the empty cache experience or any "Shift + Refresh" click) will be answered by:

On a browser refresh the server will return a 304 response based on the value of the If-Modified-Since header:

The big picture

During the sample navigation to the homepage, images have run down from this:

To this:

Saving bandwidth is great, but keep in mind that we are doing it to reduce response time. In this case, 2-3 seconds have been reduced to 0.3 just by introducing this change.

Code


public class FileAction extends AbstractAction {  

 @Autowired private FileManager fileManager;  

 /** the id of the file */ 
 private Integer id; 

 /** the version number of the resource */ 
 private Integer version;  

 /** the retrieved file */ 
 private PersistentFile pfile;  

 /**  
  * This method will be invoked before getFile(). If the current request includes a If-Modified-Since  
  * header that matches the lastModified attribute of the file, a 304 (NOT MODIFIED) response will be   
  * automatically sent, and getFile() will not be invoked.  
  */ 
 @Cache(on="getFile") 
 public CacheControl getCacheControl() {  
  pfile = fileManager.find(id);  
  CacheControl c = new CacheControl();  
  if (pfile.getVersion().equals(version)) {   
   c.setLastModified(pfile.getLastModified());   
   c.setCacheForever();  
  }  
  return c; 
 } 

 @GET @Path("/{id}/{version}") 
 public Resolution getFile() {  
  if (pfile.getVersion().equals(version)) {   
   return send(pfile);  
  } else {   
   // if the version number differs, send a 301 redirect   
   return redirect(FileAction.class, "getFile")    
     .addParameter("id", pfile.getId())    
     .addParameter("version", pfile.getVersion())    
     .permanent();  
  } 
 }
}

CacheControl is the class that does all the thinking. It sets the http headers and decides whether to return a 304 or 200 response.

We will be in London on Sept 23rd

We are quite excited that Loom has been selected finalist of the ACCESS-IT awards in the web 2.0 category. We will be in London on September 22 and 23, which is my first trip there. Should be fun!

30 slides about testing ajax components

2009-06-25T18:02:00.002+02:00

Last Sun Open Community Forum is over, and like past events (the name changes, but people remains the same) it was the perfect opportunity to get the spanish java community together to enjoy some beer and - incidentally - to talk about software too :)

This year has been great. Due to my nemesis-agenda I could not be there for the second day, which means that I missed the groovy lab by Nacho Brito (if you are talking about groovy in Spain, that's Nacho) and Daniel "greeneyed" Lopez - seriously, I wanted to see him running five languages in the JVM during a 2-hour lab.

The ones that I managed to attend were great: Jorge Sanchez introduced some alternatives for large-scale integration and presented the new spanish MySQL community, and Alfredo Casado fired some great slides about content negotiation and all things REST.

We were also there: Sergi Almar had a really succesful lab about what's coming in Spring 3.0 which enjoyed some massive assistance. Due to lack of space, some people had to spend the entire lab with their laptops over their kneels but they still were not leaving. I delivered a brief (30-min) talk about how to mix progressive enhancement, java, and javascript tests.

Developing and testing ajax components

View more Keynote presentations from Ignacio Coloma.

Lots of other things to share, but that will be in a couple of weeks. Cheers!

Loom 1.5 is out; are you doing something interesting today?

2009-05-21T13:36:00.003+02:00

Today we are releasing Loom 1.5. This is our first release intended for public consumption, so if you are thinking about giving it a twirl we will be more than glad to help.

I am not going to talk about things included in this release, such as a YSlow! 'A' rate for the demo application (94 out of 100, yay!), the ImageValidation annotation or the simplified spring config. Instead, I wanted to comment something present since our 1.0 release.

Things that your web framework should be doing for you

Last entry was about how your javascript and server-side code should talk HTML at both sides, because generating javascript from the server is a bad practice. This one is about how you (and by "you" I mean your web framework) should be generating your HTML markup.

What follows is what we like to call a Contributed View, meaning that each piece of the web framework can contribute aspects to the generated HTML.

Form Fields

An integer field validated between 5 and 100 can contribute this to the generated HTML:

Converter: inject a 'number' css class to be used by javascript and CSS style sheets.
Validator: inject attributes (scale="0" min="5" max="100") to be used by javascript validations. Better yet, inject maxlength="3" to make the browser enforce the maximum length of this field. It can get tricky since you must take into consideration negative values and decimal point.

Notice what the server should not be generating: javascript code. You can still use prototype, jquery or dojo to validate user input, as the server side is just providing the required metadata.

Links

The easiest example of URL decoration affects links pointing to forbidden resources. If the application developer is using annotations to specify permissions (with JSR-250 @RolesAllowed, for example) the security interceptor can contribute an "unauthorized" css class to links and forms. Again, it is your responsibility as the application developer to use CSS or javascript to hide or disable them.

About our next few weeks

I have been seriously lectured by some friends about how we are not dedicating enough time to make visible the awesome things we are into these days. I hate sounding like a salesman, but people got so serious that I thought I should make an exception just for today:

We are an heterogeneous software company, which means that the PHP guy sits next to the java freak most of the time and we still somehow manage to keep the peace - most of the time. When we can choose, we have been using the Loom-Spring-Hibernate combo for almost two years and are quite happy with it. There are lots of cool details like the browser cache annotation that do not have an equivalent in other web frameworks.

If you test and like it, I want to know. If you test and find things that you don't like, hey, I still want to know!

About our last few weeks

Our training courses have been a great success! We are really proud of the feedback received, 80% of the attendees to both courses have given us the highest qualification possible. In our first Core Spring Course in Madrid, people with previous spring experience started learning something new from day 2, which is a remarkable achievement for an introductory course.

On a related note, a friend from Tenerife was quite pissed at me because he had to investigate a big deal to find our Agile Project Management Course Contents. It's more or less the same thing you can see in this blog, but four hours a day. Last week lots of people confessed that they were expecting to fall asleep in the middle of the course, but instead had lots of fun - and with 'we', I mean 'me included'.

On the "things to improve" comments, we have been asked to provide more practical examples for both courses. We are certainly looking forward to improve that.

Guess who is the first SpringSource partner in Spain

2009-04-16T10:15:00.001+02:00

Yep. We are.

These last weeks have been like playing the violin with one hand while running across a Super Bowl match - with the ball -, but results are finally here: we are quite excited to announce the partnership between Extrema Sistemas and SpringSource.

This agreement is the conclusion of lots of work on both sides, notably Sergi Almar who started the whole thing in the first place. Extrema Sistemas will be delivering the SpringSource official courses in Spain, starting in Madrid on Apr 27-30 and later in Barcelona on May 25-28.

On a related note, I will be in Tenerife on May 4-8 to deliver our Agile Project Management Course on-site for a coalition of software development companies. It will be a full house, and as an interesting extra I will give a quick talk (well, "quick" be pronounced as "like two hours") at my former University.

I am really excited with the perspective of looking at my old college rooms from the other side :)

Start testing your Javascript server-less

2009-03-23T16:00:00.004+01:00

Are you still launching tomcat to test your javascript code? You shouldn't.

When applying the MVC pattern, HTML as always been considered to be part of the View, which is only correct from a server point of view. In browser-land HTML is the Model, the Controller is our javascript code, and the View is - well, it can be your chrome, your CSS, whatever. The important thing is, there are two MVCs involved:

Two years ago we started applying some simple rules for javascript development:

No javascript code generated by the server. Nil. Zero. Nada. That would only make things harder to port and reuse. Instead, the server generates the HTML attributes (some standard and some not) to be used by javascript code. Combine this with progressive enhancement and you will have an almost functional interface to add your javascript icing.

Javascript is always tested using static HTML files, which represent the approved API between server and client software. You can test easily your javascript code, and the server only has to stick to this API.

Javascript unit testing may be added later. We use UnitTest, but there are others: JQuery, Firebug, TestMonkey... the list goes on. Notice that I am not considering Selenium here, as we are not testing interfaces but javascript code.

No matter the programming language, life is easier if you do not try to generate your Controller. Now your test environment can be 100% static.

These simple rules have made our lives so much easier. Instead of the classic mixup of technologies to test Ajax interfaces, there is just a javascript test engine and some html files to test things up (including some static JSON response examples). Before this, something like testing our validation code was a killing effort. This can still be integrated with ant or maven without requiring a java server in the middle.

Going static include some additional advantages:

You can interact with the interface by hand. When the tests are done, you still have a working HTML page to pick at.

You can trace and debug any failing assertion using Firebug or the IE debugger.

There is no roundtrip to the server. Things can only be fixed up to the HTML API, so there is no point in thinking in java at this point.

Think Divide-And-Conquer, but separating client from server using HTML. Once all javascript code is tested, you can start integrating things with the java server just by generating the expected HTML.

My Devoxx slides: 15 minutes about improving your users cache experience

2009-02-09T12:04:00.003+01:00

I finally managed to save some time to upload my slides about the browser cache at Devoxx 2008, in what seems like a life ago (long story short, converting to Powerpoint XP back and forth is not a nice idea if you plan to upload it sometime later).

Most people missed it since it was scheduled as a quickie, which is fine since I'm a lame speaker (specially when throwing foreign languages into the mix). The slides were OK though, so I wanted to share. And here they are.

Caching web contents in the browser

All in all, devoxx was an awesome experience. I'm looking forward to repeat this year!

Loom 1.0 final is out. Let there be AWESOME.

2008-11-26T09:00:00.000+01:00

After two years of work, 700+ java classes, 400+ tests, 35 javascript test pages and a hell of a week polishing rough edges, we finally got the release out the door.

(I enjoy an entire night of pure insomnia once a month, which looking behind is a great boost for productivity :)

With lots of bottled awesomeness^tm inside, the following is a selection of the new features that come included in Loom 1.0:

This release includes a scaffolding tool

We finally got tired of starting projects from scratch, so Rafa got a week dedicated to it and prepared a tool that can kick start a project in five minutes, assuming a decent internet connection. Just download scaffolding.zip from the download page, unzip and run:


# Create a project named myapp 
scaffolding create myapp

# Create the eclipse project 
# (requires GRADLE_HOME defined)
cd myapp
gradle eclipse

# Deploy on your favorite web server 
# (requires CATALINA_HOME or JETTY_HOME defined)
gradle deploy-tomcat
gradle deploy-jetty

Sometimes gradle timeouts while waiting for maven repositories; in that case, just wait a little and repeat the command.

The created template will be based on gradle, which is (some artistic license, here) a combination of ivy + groovy + steroids. Notice that Loom does not use groovy but as a substitute for ant (and a great one at it!), the rest is 100% java.

Open the project using eclipse, where compiled java files go directly into WEB-INF/classes. Open "Run as" and select "myapp - run tomcat" or "myapp - run jetty" (again, CATALINA_HOME or JETTY_HOME are required). No plugins are needed, no build process, nothing. Just compile and open a browser at http://localhost:8080/myapp

Now, create a new JPA entity with some attributes using eclipse (Loom does not require JPA, but this is easier this way) and add your own CRUD interface:


# execute this in your project home dir after MyPersistentClass has been compiled
scaffolding crud com.acme.model.MyPersistentClass

That's it. Restart the server and test your interface. The form should validate your JPA constraints, which can be modified at any time without re-creating the interface or anything.

This release includes Javascript and CSS repositories

Your javascript and CSS files can be concatenated, minified and gzipped in production without involving any ant or maven script.

This is similar to what JAWR does, but easier to configure and use. Its main features are:

Configured using spring: it does not require external files.

All files (javascript, CSS, referenced images) can be loaded from the classpath, which means that they can be provided by other jars. You can still use images inside CSS files, and they will be served - even from the classpath.

All files (again - css, javascript AND image files!) are aggressively cached based on their MD5 checksums. If you have ever worked with GWT, you may be familiar with the concept - if not, drop by our Devoxx talk where we will be talking about this as long as time allows.

You can choose between YUICompressor (javascript and CSS) and JSMin (javascript) just by dropping them on the lib folder. They are included in the /lib/compressors folder of the Loom distribution.

This should be easier to digest using an example. The spring file:


<bean class="org.loom.resources.WebResourceBundleRepository">
<property name="bundles">
<bean class="org.loom.config.SpringResourceMapFactory">
<property name="resources">
<value>
    css=/css/base.css /css/basemod.css 
    ie=/css/yaml-3.0.5/core/iehacks.css /css/my_iehacks.css
    demo=classpath:/js/prototype/prototype-1.6.0.3.js classpath:/js/loom/core.js \
 classpath:/js/loom/format.js 
</value>
</property>
</bean>
</property>

The JSP file:


<l:css resource="css"/>
<l:css resource="ie" ie="any"/>
<l:script resource="demo"/>

The generated html with config.development=true:


<link href="/myapp/resources/css/0/2008a6c2ba1c6009f5f84db99e6a76e0/base.css" rel="stylesheet" type="text/css"></link>
<link href="/myapp/resources/css/1/f6c4b21a2f6c9d265d14081243c2da40/basemod.css" rel="stylesheet" type="text/css"></link>

<!--[if IE]>
<link href="/myapp/resources/ie/0/658801c193faa8e7173e2cd220093e67/iehacks.css" rel="stylesheet" type="text/css"></link>
<link href="/myapp/resources/ie/1/658801c193faa8e7173e2cd220093e67/my_iehacks.css" rel="stylesheet" type="text/css"></link>
<![endif]--> 

<script src="/myapp/resources/demo/0/b5684120e496c310977713be34be4868/prototype-1.6.0.3.js" type="text/javascript"></script>
<script src="/myapp/resources/demo/1/07139c2fd5d77ee9fa619ff335996068/core.js" type="text/javascript"></script>
<script src="/myapp/resources/demo/2/bd7026fb11a52e8fe8c671e894885e78/format.js" type="text/javascript"></script>

The generated html with config.development=false:


<link href="/myapp/resources/css/all/7c66dcfb53bb87201e74aef4f1211a63/" rel="stylesheet" type="text/css"></link>

<!--[if IE]>
<link href="/myapp/resources/ie/all/da188973e7712595d03da0bbbfa059f4/" rel="stylesheet" type="text/css"></link>
<![endif]--> 

<script src="/myapp/resources/demo/all/1c004c851402d83a84e3d7a2fb73a375/" type="text/javascript"></script>

These resources have been automatically concatenated, gzipped, MD5-hashed (just once!) and set to expire ten years from now. The browser will never ask for them again, as long as they do not get modified or the user hits the "refresh" button. During normal navigation only the html file will be retrieved.

But there's more: during development, your changes will be detected on real time, but the browser will only ask for the resources that have changed. The typical 304 traffic between server and client will just not happen.

Go test the feature live on the new demo, just turn on / off the debug switch (at the top right corner) and open firebug to see the number of files involved for each request. Now, navigate between pages and keep an eye on your firebug net tab. No files are being requested.

Wait, there's more.

This release includes partial JAX-RS support

We have added support for the mapping annotations of the JAX-RS spec. In fact, the CRUD interface created by scaffolding does just that:


public class MortgagesAction extends AbstractAction {

 @RetrieveEntity(on={"save", "edit"})
 private Mortgage mortgage;

 @GET @Path("/")
 public Resolution list() { ... }

 @GET @Path("create")
 public Resolution create() { ... }
 
 @GET @Path("/{mortgage.id}")
 public Resolution edit() { ... }
 
 @POST @Path("/{mortgage.id?}")
 public Resolution save() { ... }
 
 @DELETE @Path("/{mortgage.id?}")
 public Resolution delete() { ... }

}

Combine with JavaRebel to enjoy The Extra Cool Development Experience. Open your Action, modify your mappings, refresh the browser - and only your Action class gets reloaded. New Actions cannot be added on-the-fly and there are other limitations, but this first release is really promising.

This release includes some serious pagination components.

I have tested almost every pagination solution in the wild since 1999, checking them off with my old list of (almost) reasonable requirements:

Pagination should be done by the database if possible. Having said that, sometimes I don't have a database, and the web framework should do its work.

There is almost no difference between a paged list (<ul>) and a paged table (<table>).

Pagination and sorting should be transparent, or almost. That is tricky when it involves the database.

It should include a professional look and feel by default. This is not my merit, kudos go to Antonio Lupetti.


<l:pagedTable data="${action.mortgages}" id="mortgages">
  <l:column sortable="false">
     <l:inputCheckbox name="selectedRows" class="selectRow checkbox" value="${row.id}" renderLabel="false"/>
  </l:column>
  <l:column property="id" />
  <l:column property="name" action="Mortgages" event="edit">
     <l:param name="mortgage.id" value="${row.id}"/>
  </l:column>
  <l:column property="address" />
  <l:column property="principalLoanBalance" title="Loan" />
  <l:column property="creationDate" class="date" />
</l:pagedTable>

Again, you can check examples with tables and lists in the demo application.

This release includes LocaleAwareException and HttpException

This is just a pet feature, but I love it. You can throw exceptions that will get to the user as any normal input validation errors, or throw HTTP errors like 404 at any point of the code. It's handy.

This release includes extensibility

Two years are a lot of time, in framework years. We kept the original goal of a lightweight and focused web framework, and moved all the server and javascript components that started cropping everywhere to the addons project (take it easy - it's not that mature yet).

The point is not about the addons project but about this whole new approach to extensibility: just drop a META-INF/loom-descriptor.properties in your jar file like this:


# The list of annotation processors provided by this jar
annotation-processors=org.loom.addons.multiupload.MultiUploadAnnotationProcessor, \
 org.loom.addons.confirmation.RequiresConfirmationAnnotationProcessor, \

# The list of Converter factories provided by this jar 
converter-factories=org.loom.addons.autocompleter.AutocompletedConverterFactory

# The list of messages locations provided by this jar
messages=classpath:resources/addons-messages

Loom will load it from the classpath and the application developer may use its contents:

Java annotations

Validators and Converters

Javascript components, including CSS and image files

Internationalized messages

And, of course, its own JSP tags as usual.

And that's it?

Damn, no. This is the framework we are using to build applications, every day. We expect it to keep growing with a steady pace with all the know-how in our way for years to come.

I only wish "steady" could mean "slower" in this context. I can live without the stress of the last few weeks :)