The 90th percentile

Caching user-supplied images in the browser

2009-09-16T11:55:00.002+02:00

We have been engaged in a big project for the last months. Despite of the nice performance marks, there was a warning in Google PageSpeed that was itching us badly: "The following resources are missing a cache expiration".

PageSpeed was complaining about images uploaded by the user. Resources referenced from CSS are conveniently handled by Loom, but user-provided files were not setting their cache headers appropriately.

The following is, to the best of my knowledge, the same algorithm implemented by blogger and wordpress to avoid delivering the same image file over and over again. The theory has been already covered here.

I will be using Loom in this example, but you can reproduce this using any web framework.

Choosing your Strong Cache Validator

Cache Validators are introduced in the http 1.1 spec. Here we will use a Strong Cache Validator to indicate that a resource should never expire. You should include both validator and resource ID in the URL, like "/resources/{id}/{cacheValue}".

The most well-known candidates are:

MD5: The file checksum can be calculated when the file is saved or when the application starts up. This is how Loom (and others) set the cache headers for CSS and javascript files.
Version numbers: any @Version field can be used to identify persistent JPA entities.

In this case we will be using PersistentFile, which includes both fields. After trying both, we found that using @Version yields much shorter URLs ("/resources/151/2" instead of "/resources/151/3b4efe6405d4fb1ada4a081f4dbef6a9").


@Entity
public class PersistentFile { 

 /** the primary key value */ 
 @Id @GeneratedValue private Integer id; 

 /** the MD5 hash of this file */ 
 @Column(length=32) private String MD5;  

 /** the revision number */ 
 @Version private Integer version;  

 /** the last update timestamp */ 
 private Date lastModified; 
 
 /** ... etc ... */

}

This field will be used to set "Expires" and "Cache-Control" headers to expire ten years in the future. As long as the user is clicking links to browse normally these resources should never be fetched again, but it will be ignored if the user clicks the "Refresh" button, which is where Weak Cache Validation comes into place.

Weak Cache Validator

With Weak Cache Validation the browser must ask for the resource, giving the server a chance to answer with a 304 (not modified) or 200 (OK) response.

For this purpose we will use the "Last-Modified" header. The first request by the browser (the empty cache experience or any "Shift + Refresh" click) will be answered by:

On a browser refresh the server will return a 304 response based on the value of the If-Modified-Since header:

The big picture

During the sample navigation to the homepage, images have run down from this:

To this:

Saving bandwidth is great, but keep in mind that we are doing it to reduce response time. In this case, 2-3 seconds have been reduced to 0.3 just by introducing this change.

Code


public class FileAction extends AbstractAction {  

 @Autowired private FileManager fileManager;  

 /** the id of the file */ 
 private Integer id; 

 /** the version number of the resource */ 
 private Integer version;  

 /** the retrieved file */ 
 private PersistentFile pfile;  

 /**  
  * This method will be invoked before getFile(). If the current request includes a If-Modified-Since  
  * header that matches the lastModified attribute of the file, a 304 (NOT MODIFIED) response will be   
  * automatically sent, and getFile() will not be invoked.  
  */ 
 @Cache(on="getFile") 
 public CacheControl getCacheControl() {  
  pfile = fileManager.find(id);  
  CacheControl c = new CacheControl();  
  if (pfile.getVersion().equals(version)) {   
   c.setLastModified(pfile.getLastModified());   
   c.setCacheForever();  
  }  
  return c; 
 } 

 @GET @Path("/{id}/{version}") 
 public Resolution getFile() {  
  if (pfile.getVersion().equals(version)) {   
   return send(pfile);  
  } else {   
   // if the version number differs, send a 301 redirect   
   return redirect(FileAction.class, "getFile")    
     .addParameter("id", pfile.getId())    
     .addParameter("version", pfile.getVersion())    
     .permanent();  
  } 
 }
}

CacheControl is the class that does all the thinking. It sets the http headers and decides whether to return a 304 or 200 response.

We will be in London on Sept 23rd

We are quite excited that Loom has been selected finalist of the ACCESS-IT awards in the web 2.0 category. We will be in London on September 22 and 23, which is my first trip there. Should be fun!

30 slides about testing ajax components

2009-06-25T18:02:00.002+02:00

Last Sun Open Community Forum is over, and like past events (the name changes, but people remains the same) it was the perfect opportunity to get the spanish java community together to enjoy some beer and - incidentally - to talk about software too :)

This year has been great. Due to my nemesis-agenda I could not be there for the second day, which means that I missed the groovy lab by Nacho Brito (if you are talking about groovy in Spain, that's Nacho) and Daniel "greeneyed" Lopez - seriously, I wanted to see him running five languages in the JVM during a 2-hour lab.

The ones that I managed to attend were great: Jorge Sanchez introduced some alternatives for large-scale integration and presented the new spanish MySQL community, and Alfredo Casado fired some great slides about content negotiation and all things REST.

We were also there: Sergi Almar had a really succesful lab about what's coming in Spring 3.0 which enjoyed some massive assistance. Due to lack of space, some people had to spend the entire lab with their laptops over their kneels but they still were not leaving. I delivered a brief (30-min) talk about how to mix progressive enhancement, java, and javascript tests.

Developing and testing ajax components

View more Keynote presentations from Ignacio Coloma.

Lots of other things to share, but that will be in a couple of weeks. Cheers!

Loom 1.5 is out; are you doing something interesting today?

2009-05-21T13:36:00.003+02:00

Today we are releasing Loom 1.5. This is our first release intended for public consumption, so if you are thinking about giving it a twirl we will be more than glad to help.

I am not going to talk about things included in this release, such as a YSlow! 'A' rate for the demo application (94 out of 100, yay!), the ImageValidation annotation or the simplified spring config. Instead, I wanted to comment something present since our 1.0 release.

Things that your web framework should be doing for you

Last entry was about how your javascript and server-side code should talk HTML at both sides, because generating javascript from the server is a bad practice. This one is about how you (and by "you" I mean your web framework) should be generating your HTML markup.

What follows is what we like to call a Contributed View, meaning that each piece of the web framework can contribute aspects to the generated HTML.

Form Fields

An integer field validated between 5 and 100 can contribute this to the generated HTML:

Converter: inject a 'number' css class to be used by javascript and CSS style sheets.
Validator: inject attributes (scale="0" min="5" max="100") to be used by javascript validations. Better yet, inject maxlength="3" to make the browser enforce the maximum length of this field. It can get tricky since you must take into consideration negative values and decimal point.

Notice what the server should not be generating: javascript code. You can still use prototype, jquery or dojo to validate user input, as the server side is just providing the required metadata.

Links

The easiest example of URL decoration affects links pointing to forbidden resources. If the application developer is using annotations to specify permissions (with JSR-250 @RolesAllowed, for example) the security interceptor can contribute an "unauthorized" css class to links and forms. Again, it is your responsibility as the application developer to use CSS or javascript to hide or disable them.

About our next few weeks

I have been seriously lectured by some friends about how we are not dedicating enough time to make visible the awesome things we are into these days. I hate sounding like a salesman, but people got so serious that I thought I should make an exception just for today:

We are an heterogeneous software company, which means that the PHP guy sits next to the java freak most of the time and we still somehow manage to keep the peace - most of the time. When we can choose, we have been using the Loom-Spring-Hibernate combo for almost two years and are quite happy with it. There are lots of cool details like the browser cache annotation that do not have an equivalent in other web frameworks.

If you test and like it, I want to know. If you test and find things that you don't like, hey, I still want to know!

About our last few weeks

Our training courses have been a great success! We are really proud of the feedback received, 80% of the attendees to both courses have given us the highest qualification possible. In our first Core Spring Course in Madrid, people with previous spring experience started learning something new from day 2, which is a remarkable achievement for an introductory course.

On a related note, a friend from Tenerife was quite pissed at me because he had to investigate a big deal to find our Agile Project Management Course Contents. It's more or less the same thing you can see in this blog, but four hours a day. Last week lots of people confessed that they were expecting to fall asleep in the middle of the course, but instead had lots of fun - and with 'we', I mean 'me included'.

On the "things to improve" comments, we have been asked to provide more practical examples for both courses. We are certainly looking forward to improve that.

Guess who is the first SpringSource partner in Spain

2009-04-16T10:15:00.001+02:00

Yep. We are.

These last weeks have been like playing the violin with one hand while running across a Super Bowl match - with the ball -, but results are finally here: we are quite excited to announce the partnership between Extrema Sistemas and SpringSource.

This agreement is the conclusion of lots of work on both sides, notably Sergi Almar who started the whole thing in the first place. Extrema Sistemas will be delivering the SpringSource official courses in Spain, starting in Madrid on Apr 27-30 and later in Barcelona on May 25-28.

On a related note, I will be in Tenerife on May 4-8 to deliver our Agile Project Management Course on-site for a coalition of software development companies. It will be a full house, and as an interesting extra I will give a quick talk (well, "quick" be pronounced as "like two hours") at my former University.

I am really excited with the perspective of looking at my old college rooms from the other side :)

Start testing your Javascript server-less

2009-03-23T16:00:00.004+01:00

Are you still launching tomcat to test your javascript code? You shouldn't.

When applying the MVC pattern, HTML as always been considered to be part of the View, which is only correct from a server point of view. In browser-land HTML is the Model, the Controller is our javascript code, and the View is - well, it can be your chrome, your CSS, whatever. The important thing is, there are two MVCs involved:

Two years ago we started applying some simple rules for javascript development:

No javascript code generated by the server. Nil. Zero. Nada. That would only make things harder to port and reuse. Instead, the server generates the HTML attributes (some standard and some not) to be used by javascript code. Combine this with progressive enhancement and you will have an almost functional interface to add your javascript icing.

Javascript is always tested using static HTML files, which represent the approved API between server and client software. You can test easily your javascript code, and the server only has to stick to this API.

Javascript unit testing may be added later. We use UnitTest, but there are others: JQuery, Firebug, TestMonkey... the list goes on. Notice that I am not considering Selenium here, as we are not testing interfaces but javascript code.

No matter the programming language, life is easier if you do not try to generate your Controller. Now your test environment can be 100% static.

These simple rules have made our lives so much easier. Instead of the classic mixup of technologies to test Ajax interfaces, there is just a javascript test engine and some html files to test things up (including some static JSON response examples). Before this, something like testing our validation code was a killing effort. This can still be integrated with ant or maven without requiring a java server in the middle.

Going static include some additional advantages:

You can interact with the interface by hand. When the tests are done, you still have a working HTML page to pick at.

You can trace and debug any failing assertion using Firebug or the IE debugger.

There is no roundtrip to the server. Things can only be fixed up to the HTML API, so there is no point in thinking in java at this point.

Think Divide-And-Conquer, but separating client from server using HTML. Once all javascript code is tested, you can start integrating things with the java server just by generating the expected HTML.

My Devoxx slides: 15 minutes about improving your users cache experience

2009-02-09T12:04:00.003+01:00

I finally managed to save some time to upload my slides about the browser cache at Devoxx 2008, in what seems like a life ago (long story short, converting to Powerpoint XP back and forth is not a nice idea if you plan to upload it sometime later).

Most people missed it since it was scheduled as a quickie, which is fine since I'm a lame speaker (specially when throwing foreign languages into the mix). The slides were OK though, so I wanted to share. And here they are.

Caching web contents in the browser

All in all, devoxx was an awesome experience. I'm looking forward to repeat this year!

Loom 1.0 final is out. Let there be AWESOME.

2008-11-26T09:00:00.000+01:00

After two years of work, 700+ java classes, 400+ tests, 35 javascript test pages and a hell of a week polishing rough edges, we finally got the release out the door.

(I enjoy an entire night of pure insomnia once a month, which looking behind is a great boost for productivity :)

With lots of bottled awesomeness^tm inside, the following is a selection of the new features that come included in Loom 1.0:

This release includes a scaffolding tool

We finally got tired of starting projects from scratch, so Rafa got a week dedicated to it and prepared a tool that can kick start a project in five minutes, assuming a decent internet connection. Just download scaffolding.zip from the download page, unzip and run:


# Create a project named myapp 
scaffolding create myapp

# Create the eclipse project 
# (requires GRADLE_HOME defined)
cd myapp
gradle eclipse

# Deploy on your favorite web server 
# (requires CATALINA_HOME or JETTY_HOME defined)
gradle deploy-tomcat
gradle deploy-jetty

Sometimes gradle timeouts while waiting for maven repositories; in that case, just wait a little and repeat the command.

The created template will be based on gradle, which is (some artistic license, here) a combination of ivy + groovy + steroids. Notice that Loom does not use groovy but as a substitute for ant (and a great one at it!), the rest is 100% java.

Open the project using eclipse, where compiled java files go directly into WEB-INF/classes. Open "Run as" and select "myapp - run tomcat" or "myapp - run jetty" (again, CATALINA_HOME or JETTY_HOME are required). No plugins are needed, no build process, nothing. Just compile and open a browser at http://localhost:8080/myapp

Now, create a new JPA entity with some attributes using eclipse (Loom does not require JPA, but this is easier this way) and add your own CRUD interface:


# execute this in your project home dir after MyPersistentClass has been compiled
scaffolding crud com.acme.model.MyPersistentClass

That's it. Restart the server and test your interface. The form should validate your JPA constraints, which can be modified at any time without re-creating the interface or anything.

This release includes Javascript and CSS repositories

Your javascript and CSS files can be concatenated, minified and gzipped in production without involving any ant or maven script.

This is similar to what JAWR does, but easier to configure and use. Its main features are:

Configured using spring: it does not require external files.

All files (javascript, CSS, referenced images) can be loaded from the classpath, which means that they can be provided by other jars. You can still use images inside CSS files, and they will be served - even from the classpath.

All files (again - css, javascript AND image files!) are aggressively cached based on their MD5 checksums. If you have ever worked with GWT, you may be familiar with the concept - if not, drop by our Devoxx talk where we will be talking about this as long as time allows.

You can choose between YUICompressor (javascript and CSS) and JSMin (javascript) just by dropping them on the lib folder. They are included in the /lib/compressors folder of the Loom distribution.

This should be easier to digest using an example. The spring file:


<bean class="org.loom.resources.WebResourceBundleRepository">
<property name="bundles">
<bean class="org.loom.config.SpringResourceMapFactory">
<property name="resources">
<value>
    css=/css/base.css /css/basemod.css 
    ie=/css/yaml-3.0.5/core/iehacks.css /css/my_iehacks.css
    demo=classpath:/js/prototype/prototype-1.6.0.3.js classpath:/js/loom/core.js \
 classpath:/js/loom/format.js 
</value>
</property>
</bean>
</property>

The JSP file:


<l:css resource="css"/>
<l:css resource="ie" ie="any"/>
<l:script resource="demo"/>

The generated html with config.development=true:


<link href="/myapp/resources/css/0/2008a6c2ba1c6009f5f84db99e6a76e0/base.css" rel="stylesheet" type="text/css"></link>
<link href="/myapp/resources/css/1/f6c4b21a2f6c9d265d14081243c2da40/basemod.css" rel="stylesheet" type="text/css"></link>

<!--[if IE]>
<link href="/myapp/resources/ie/0/658801c193faa8e7173e2cd220093e67/iehacks.css" rel="stylesheet" type="text/css"></link>
<link href="/myapp/resources/ie/1/658801c193faa8e7173e2cd220093e67/my_iehacks.css" rel="stylesheet" type="text/css"></link>
<![endif]--> 

<script src="/myapp/resources/demo/0/b5684120e496c310977713be34be4868/prototype-1.6.0.3.js" type="text/javascript"></script>
<script src="/myapp/resources/demo/1/07139c2fd5d77ee9fa619ff335996068/core.js" type="text/javascript"></script>
<script src="/myapp/resources/demo/2/bd7026fb11a52e8fe8c671e894885e78/format.js" type="text/javascript"></script>

The generated html with config.development=false:


<link href="/myapp/resources/css/all/7c66dcfb53bb87201e74aef4f1211a63/" rel="stylesheet" type="text/css"></link>

<!--[if IE]>
<link href="/myapp/resources/ie/all/da188973e7712595d03da0bbbfa059f4/" rel="stylesheet" type="text/css"></link>
<![endif]--> 

<script src="/myapp/resources/demo/all/1c004c851402d83a84e3d7a2fb73a375/" type="text/javascript"></script>

These resources have been automatically concatenated, gzipped, MD5-hashed (just once!) and set to expire ten years from now. The browser will never ask for them again, as long as they do not get modified or the user hits the "refresh" button. During normal navigation only the html file will be retrieved.

But there's more: during development, your changes will be detected on real time, but the browser will only ask for the resources that have changed. The typical 304 traffic between server and client will just not happen.

Go test the feature live on the new demo, just turn on / off the debug switch (at the top right corner) and open firebug to see the number of files involved for each request. Now, navigate between pages and keep an eye on your firebug net tab. No files are being requested.

Wait, there's more.

This release includes partial JAX-RS support

We have added support for the mapping annotations of the JAX-RS spec. In fact, the CRUD interface created by scaffolding does just that:


public class MortgagesAction extends AbstractAction {

 @RetrieveEntity(on={"save", "edit"})
 private Mortgage mortgage;

 @GET @Path("/")
 public Resolution list() { ... }

 @GET @Path("create")
 public Resolution create() { ... }
 
 @GET @Path("/{mortgage.id}")
 public Resolution edit() { ... }
 
 @POST @Path("/{mortgage.id?}")
 public Resolution save() { ... }
 
 @DELETE @Path("/{mortgage.id?}")
 public Resolution delete() { ... }

}

Combine with JavaRebel to enjoy The Extra Cool Development Experience. Open your Action, modify your mappings, refresh the browser - and only your Action class gets reloaded. New Actions cannot be added on-the-fly and there are other limitations, but this first release is really promising.

This release includes some serious pagination components.

I have tested almost every pagination solution in the wild since 1999, checking them off with my old list of (almost) reasonable requirements:

Pagination should be done by the database if possible. Having said that, sometimes I don't have a database, and the web framework should do its work.

There is almost no difference between a paged list (<ul>) and a paged table (<table>).

Pagination and sorting should be transparent, or almost. That is tricky when it involves the database.

It should include a professional look and feel by default. This is not my merit, kudos go to Antonio Lupetti.


<l:pagedTable data="${action.mortgages}" id="mortgages">
  <l:column sortable="false">
     <l:inputCheckbox name="selectedRows" class="selectRow checkbox" value="${row.id}" renderLabel="false"/>
  </l:column>
  <l:column property="id" />
  <l:column property="name" action="Mortgages" event="edit">
     <l:param name="mortgage.id" value="${row.id}"/>
  </l:column>
  <l:column property="address" />
  <l:column property="principalLoanBalance" title="Loan" />
  <l:column property="creationDate" class="date" />
</l:pagedTable>

Again, you can check examples with tables and lists in the demo application.

This release includes LocaleAwareException and HttpException

This is just a pet feature, but I love it. You can throw exceptions that will get to the user as any normal input validation errors, or throw HTTP errors like 404 at any point of the code. It's handy.

This release includes extensibility

Two years are a lot of time, in framework years. We kept the original goal of a lightweight and focused web framework, and moved all the server and javascript components that started cropping everywhere to the addons project (take it easy - it's not that mature yet).

The point is not about the addons project but about this whole new approach to extensibility: just drop a META-INF/loom-descriptor.properties in your jar file like this:


# The list of annotation processors provided by this jar
annotation-processors=org.loom.addons.multiupload.MultiUploadAnnotationProcessor, \
 org.loom.addons.confirmation.RequiresConfirmationAnnotationProcessor, \

# The list of Converter factories provided by this jar 
converter-factories=org.loom.addons.autocompleter.AutocompletedConverterFactory

# The list of messages locations provided by this jar
messages=classpath:resources/addons-messages

Loom will load it from the classpath and the application developer may use its contents:

Java annotations

Validators and Converters

Javascript components, including CSS and image files

Internationalized messages

And, of course, its own JSP tags as usual.

And that's it?

Damn, no. This is the framework we are using to build applications, every day. We expect it to keep growing with a steady pace with all the know-how in our way for years to come.

I only wish "steady" could mean "slower" in this context. I can live without the stress of the last few weeks :)

My top list of java tools

2008-10-30T13:00:00.003+01:00

Lack of imagination is one of our worst sins as software developers. We do the same things over and over again, but we rarely modify our ways - me at least. After some years, these are the tools that made it into my tricks box for everyday tasks. Tiresome operations are not my thing.

Chances are you are already using at least some of these, but here we go anyways:

StringUtils

The bread and butter of the commons-lang library, this utility class includes some methods that should seriously have been included in String long time ago.


StringUtils.isEmpty(null) && StringUtils.isEmpty(""); // true
StringUtils.isBlank("   \n\t");                       // true
StringUtils.substringAfterLast("foo.bar.baz", ".");   // "baz"
StringUtils.substringBeforeLast("foo.bar.baz", ".");  // "foo.bar"
StringUtils.split("foo.bar.baz", '.');                // { "foo", "bar", "baz" }
StringUtils.split("foo,  bar,baz", ", ");             // { "foo", "bar", "baz" }
StringUtils.leftPad("1", 3, '0');                     // "001"

IOUtils and FileUtils

A must-have for the rare occasions where you need to manipulate files by hand. Both are pretty much alike (FileUtils for File, IOUtils for InputStream and Reader classes) and come bundled in commons-io.


File file1;
File file2;
InputStream inputStream;
OutputStream outputStream;

// copy one file into another
FileUtils.copyFile(file1, file2);
IOUtils.copy(inputStream, outputStream);

// read a file into a String
String s1 = FileUtils.readFileToString(file1);
String s2 = IOUtils.toString(inputStream);

// read a file into a list of Strings, one item per line
List<String> l1 = FileUtils.readLines(file1);
List<String> l2 = IOUtils.readLines(inputStream);

// put this in your finally() clause after manipulating streams
IOUtils.closeQuietly(inputStream);

// return the list of xml and text files in the specified folder and any subfolders
Collection<File> c1 = FileUtils.listFiles(file1, { "xml", "txt" }, true);

// copy one folder and its contents into another
FileUtils.copyDirectoryToDirectory(file1, file2);

// delete one folder and its contents
FileUtils.deleteDirectory(file1);

Google collections

This is the best implementation of a collections extension that I know of. Some of these are shouting to be included in the JDK:


// create an ArrayList with three arguments
List<String> list = Lists.newArrayList("foo", "bar", "baz");

// notice that there is no generics or class cast,
// and still this line does not generate a warning.
Set<String> s = Sets.newConcurrentHashSet();

// intersect and union are basic features of a Set, if you ask me
Set<String> s = Sets.intersect(s1, s2);

// Example  of multiple values in a Map
ListMultimap<String, Validator> validators = new ArrayListMultimap<String, Validator>();
validators.put("save", new RequiredValidator());
validators.put("save", new StringValidator());
validators.put("delete", new NumberValidator());

validators.get("save"); // { RequiredValidator, StringValidator }
validators.get("foo");  // empty List (not null)
validators.values();    // { RequiredValidator, StringValidator, NumberValidator }

java.util.concurrent

Not everybody needs the heavy lifting of java.util.concurrent, but the concurrent collections are handy:


// a map that may be modified (by the same or different thread) while being iterated
Map<String, Something> repository = new ConcurrentHashMap<String, Something>();

// same with lists. This one is only available with Java 6
List<Something> list = new CopyOnWriteArrayList<Something>();

Hardly a large toolbox, is it? If your favourite library is missing, feel free to add it :)

I WILL BE AT DEVOXX!! I have a 15 minute talk about the browser cache on December 10th. I am really excited about this, this will be my first JavaPolis (now Devoxx) experience.

It will be fun, I plan to talk about 304, Expires / Cache-control, and maybe have a small demo of MD5-based cache if time permits. If you happen to be there, please drop by and say hi!

"Practical API Design", the book

2008-10-14T10:00:00.001+02:00

It got mentioned on the last OpenJavaDay (thanks Ibon!) with such a good review that I got my own copy of Practical API Design by Jaroslav Tulach. The book describes the set of standards developed by the Netbeans team during the last ten years, from framework novices to their current state of maturity.

Even if you don't agree with some of the book recommendations (which is ok, since they aim at being a bit controversial) it is still a complete source of good practices when designing - not an application, but - a platform. If your case is not as much about "design" but "design while ignoring how many people are using your API and which parts are found useful", this book is a must-read.

If I had to complain about something, it spends too much time in the introduction chapters. It sets the foundation stone of good API design and I really needed to grab some perspective about API stability over time, but a bit shorter would have made it perfect.

I love the freshness in the style, the way it seems to be designed with my case in mind. A quote after several pages describing its intended audience:

If you are still asking yourself wether you should read the book or not, there is a much shorter answer: "Yes, you should read this book!"

Indeed :)

The coolest prototype calendar ever

2008-10-08T08:00:00.003+02:00

Roberto has been doing amazing things with calendars these days. With current loom trunk, this java attribute:


@Temporal(TemporalType.TIMESTAMP)
@DateValidation(minValue="today + 1d", maxValue="today + 1y")
private Date start;

With this javascript code:


// create calendar
$$('input.date, input.dateTime').each( loom.ui.createCalendar, {});

Renders this absolute piece of awesomeness:

Dates before today and after one year (from today) are disabled, as specified by @DateValidation. As always, the server side will use the same constraints.

The use of @Temporal (the standard JPA annotation) triggers the interface to ask for date and time. The default would only ask for a date.

It uses the same attributes (min, max) and format (ISO-8601) specified by the HTML 5 draft, which should help replacing it when browser support arrives by 2025 :)

Full implementation of the PHP strftime format.

Other use cases may be tested here, as the javascript library can be used standalone without loom.

MY PRECIOUSSSSSSSSS...

UPDATE: The calendar has been included in the 1.0 release. Just download the jar file, uncompress, and look inside the js folder.
UPDATE 2: As of loom 1.1 we have added a loom-javascript.zip file which includes just the javascript and css files. To use it, just take a look at the included html test files. They are not pretty, but should indicate a clear way to use the components.

My take on the SpringSource license change

2008-09-25T19:13:00.005+02:00

I have real problems getting what all this fuss is about:

There is a global economic crisis, in case you didn't notice. It affects SpringSource as much as anyone else.

They do numbers. They notice that a big share of their costs are related to maintaining legacy versions for non-paying customers.

They react to minimize impact, while still being open.

I DON'T HAVE THEIR NUMBERS. I cannot have an opinion about if this is a good move or not. Neither do any of you. Please stop saying how you would react if you were the boss of a multimillion dollar company and had to face an unknown problem: "Oh, I would stay on top of the Titanic seeing everything sinking in all its glory, but would not move an inch away from openness", or "I would change the business model and put more stress on training and consulting - no, I don't have a clue about how much they make from consulting, is that important?"

It's funny how most people agree that they wouldn't do a better job developing a java integration framework, but they do know better when talking about management. I frankly prefer this move than some kind of Lehman Brothers thing happening to a framework that is at the core of all the systems I have designed in the last few years.

I am only posting this because dissatisfied customers are 4x more visible, and I don't think it's fair to the good work these guys are doing.

Recycle your HTML-fu

2008-09-24T13:00:00.000+02:00

Recycling is good.

If you look around right now, you have high chances of being the oldest piece of furniture in the room. Maybe it's time to consider some recycling, and this post is trying to help by telling things you may already know - or maybe not.

Little did you know, <META name="keywords"> is being ignored by Google and others. Google has confirmed that they do not trust this information.

On your way out of ie6, you may consider using Dean Edwards' ie7.js. It's an excellent library that provides transparent PNGs, advanced CSS selectors like first-child and last-child, multiple CSS classes... It also includes some missing features for ie7.

I already mentioned this a couple of times: you are not using XHTML. It's nice to use well-formed HTML pages, but don't take it too seriously.

When using script tags, you may omit the language and type attributes.

All HTML validators will complain if you try to follow any of these practices, but it is time they get properly updated. For example, according to the standard a label with a nested input field does not need to provide a "for" attribute, which is something that validators tend to forget.

Closely related to the previous point, the only up-to-date (beta) HTML validators I know of are validator.nu and the W3C HTML 5 validator.

When developing your next REST web portal, keep in mind that GET links must be idempotent, which is a cool way to say that they should not modify anything in the database. For anything else, consider using POST links or forms.

Speaking of which, POST is the only safe way to prevent web crawlers from visiting links. Do not use rel="nofollow" for this purpose.

These are random thoughts that were raised to the table in our last web training week in Zurich. As the funny note of the week, I took a break to buy a t-shirt for my sister, who is obtaining her Ph.D. with the very same CERN guys that are working on the Large Hadron Collider.

Talk about twisted sense of humor :)

XSS patch for tomcat, jetty and glassfish

2008-08-25T16:15:00.003+02:00

In case you didn't know, any application with a JSP page that uses ${foobar} with user-introduced data is inherently insecure. The 10-seconds Intro To The Joy Of XSS goes a little like this:

Go to some data input form.

Enter "<script>window.location.href='http://badguys.com'</script>" in any field

Wait until my entry shows up in somebody else's authenticated browser, typically as part of some listing.

This technique can be used for practically anything, from deleting your data to inserting a hidden iframe that points to the Black Hat version of the Yellow Brick Road. This will happen in all java-based app servers, since the standard says not to escape XML contents inside EL expressions (that ${...} thingy).

After the loom 1.0-rc1 release there were lots of people testing the demo application - well, on July 1st we had the same amount of traffic of the entire month of June. The server stats showed many smart attempts at XSS security: special characters, scripts, images and iframes were the general note.

Loom itself escape XSS attacks using an optimized version of the Google doctype recommendation. But two of our favorite servers (tomcat and jetty) were not doing it: these tags got printed wherever a "${customer.name}" was used, so we decided to do something about it. And here is were Rafa Serrano (werkins) took the lead.

Rafa is the smart, not-yet-thirty-year-old guy that works at our Madrid office (three meters to my right, to be exact), that for some reason I have yet to understand has got two university degrees and is working on the third - I can only suppose the TV has been broken at his house for, like, years. Since I had a great summer totally stuck with my tasks, he decided to take this one out of my stack and implement it himself.

The results are two patches filed for both Tomcat and Glassfish, the last one valid for jetty too. Please, if you have a voice in the Tomcat or Glassfish communities cast a vote for these, as they are a step towards a more secure web.

In case you need security NOW and cannot wait for the new release of your server (yep - that sounds definitely as me), we have included a patched jar for our rc2 release, which can be grabbed from subversion right now. The folder includes a jar that should be dropped into your server lib folder, a test JSP page, and instructions to use. Note that if you still want unescaped HTML content you can use <c:out escapeXml="false"> but that should be the exception, not the norm.

Well, have a secure summer - or something. Cheers!

A way to get rid of IE 6

2008-07-01T07:01:00.008+02:00

This got posted at webappers yesterday, a script to hint your IE 6 users to upgrade:

http://www.savethedevelopers.org/

It's still buggy, but I love the idea. It's also good timing, as our team was considering dropping IE 6 support for some of our widgets and render javascript-less HTML instead.

UPDATE: It's a fact. 37signals will also drop support of IE6 starting August 15.

Lessons learned on the web layer

2008-06-30T09:15:00.002+02:00

I am leaving an intense week behind: first, I just posted my Lessons learned on the web layer slides for Open Java Day.

| View | Upload your own

This has been my first, and it has been an amazing experience. Great conversation during lunch, and some Ribeiro and Albariño drinks on Friday that left some photos I surely will regret when they get online... At the end, I got out with a few contacts and a lot of new friends. Totally worth it.

On related news:

LOOM 1.0 IS OUT! (well, it's the first release candidate, but bear with me anyway). I have invested a couple of full-time months getting everything in place before the talk, and though there is still room for improvement you have to cut the release somewhere. So this is it.

I am not suggesting that anybody should _use_ it, not until it grows some community, but I would be grateful for any kind of feedback: what are you missing in it, what may be cut out, and why. If you feel like giving it a try, go to the new demo - which, I must say, does not look like java at all.

P.S.: You cannot possibly imagine how we are living the Euro Cup results here in Spain. It's crazy ^_^

Extend HTML with your own metadata

2008-06-03T10:00:00.008+02:00

You know that thing about space shuttle rockets being two-horse-asses-wide. If you want to save some time, the motto of the story is Specifications and bureaucracies live forever.

I do love HTML despite the fact that we are still tied to the HTML 4 spec (1999). I would find use for a couple more attributes, call it semantic sugar or whatever.

Let's talk about an example use case to validate some number between 0 (inclusive) and 100 (exclusive).

Option 1: inject JSON attributes

<input type="text" name="foo" id="foo"/>

<script>
// one of these must be added per input field
Object.extend($('foo'), {
   min: 0,
   max: 100,
   excludeMin: false,
   excludeMax: true   
});
</script>

pros

Even my grandma can get this code.

cons

With any mid-size form there will be like a ton of script tags opening and closing all the time. Even if you manage to put them all together, you will not feel very proud of it.

Every input field must have a unique ID attribute to locate it for extension.

You should NOT generate javascript code from your server side. Ever.

The most important: the DOM node must exist prior to extending it. This is an uncomfortable restriction about where to put this javascript snippet, most probably after a </form> closing tag which is a serious problem if you plan to develop a tag library where you do not know the surrounding HTML code.

Inject attributes using CSS

Lots of people do this (jQuery plugin authors, for example):

<input type="text" name="field" class="extended __min_0 __max_100 __excludemin_false __excludemax_true"/>

<script>
// this must be called just once for the entire page
document.observe('dom:loaded', function() {
 $$('.extended').each(function(element) {
   element.className.scan(/__(\w+)_(\w+)/, function(p) { element[p[1]] = p[2] });
 });
});
</script>

pros

It does not have any of the JSON problems.

It is a valid use of the CSS class. From the HTML spec: "The class attribute has several roles in HTML: [...] For general purpose processing by user agents."

cons

UGLY. Again. Sure, for simple cases it just seems right, but try to explain that to your web designer next time he asks about a "required date __min_2005-10-20T10:00:00Z __something_ortheother". When the real world challenges your application, your HTML gets a legacy reek.

You are mixing up style and behavior metadata in the same attribute.

You will have to UrlEncode values that contain special characters (double quotes, spaces, etc).

Namespaced attributes

I like this extensibility thing about XHTML:

<html
   xmlns:v="http://myhost/schemas/validation"
   xmlns="http://www.w3.org/1999/xhtml">
<input type="text" name="field" v:min="0" v:max="100" v:excludemin="false" v:excludemax="true"/>

This is used by W3C ARIA and wicket.

pros

At last, something I can look at.

cons

You must develop and maintain your own XML schema for everything you extend.

You think you are going XML? you are not.

Long story short, your browser is parsing your page as HTML (not XML) unless you include an "application/xhtml+xml" http header. Once you do, everything is fine: any subtle mistake in your XML document will render a blank page in the browser, javascript snippets get picky about their contents, and oh, it will not work at all in any IE browser. That is what "fine" means in my book.

The word out is that browser vendors are focusing in supporting HTML 5 instead of XHTML 1 and 2, but as long as you know what you are doing, schemas work just fine.

HTML attributes

The alternative to XHTML is HTML tag soup, which is more or less similar to XHTML. Anything that is not recognized by the browser is valid, and as long as they belong to valid tags they can be reached from your javascript code. This is exploited by the dojo toolkit and HTML 5:

<input type="text" name="field" min="0" max="100" data-exclude-min="false" data-exclude-max="true"/>

In this example I have included some HTML 5 attributes and preceded the rest with "data-". It would be nice if the HTML group takes some advice from John Resig and listen to the community about what is needed in the spec. Out of the ten validation parameters I use frequently, HTML 5 only supports three - min, max, and pattern.

pros

Here goes my grandma again, lecturing me about how much she loves the simplicity in this code.

cons

You may not put cool buzzwords in your resume.

And Loom?

Which one have we tried? All of them. Well, mostly.

It's been a long year, and after having our own cup of pain we ended up settling for CSS classes to indicate the field type (number, string, date, required) since they can be associated to style, and HTML 5 for everything else. Other alternatives are fine, just make sure you get the whole picture before trying.

Drop by Madrid to enjoy some deeper insight

Do not miss the Open Java Day event organized by JavaHispano and Sun on June 26-27. I'll be talking about practical lessons learned from the web tier: CSS frameworks, charset handling, javascript tests, accessibility, etc. I don't know how much I can fit into one hour, but it will be fun.

UPDATE: John Resig has added further insight on this subject here.

Mock Your Spring Config for Fun and Profit

2008-04-24T13:00:00.004+02:00

"Nemesis: An opponent that cannot be beaten or overcome." (Answers.com)

I used to be quite proud of our spring config structure, which is the result of some trial-and-error game while trying to achieve the perfect pluggable system. I would much prefer avoiding the typical LDAP / Oracle / BankingHost-with-potatoes combo just to debug the web interface while my laptop screams wildly. Besides, I'm green: I love saving power by keeping the system startup under 15 seconds.

Note that I said "green", not "impatient". Green. Ecological, say.

But last month I met The Adversary: twelve systems to integrate, a different environment for test, production and disaster recovery, four different web applications that communicate using web services, up to four different databases... Our spring mojo was not cutting it, so we had to take it to the next level.

What follows are all the alternatives that we tried, what worked and what didn't:

PropertyPlaceHolderConfigurer

The bread and butter of application configuration, this class gets into your spring file and you start injecting values anywhere using the classic ant-like syntax we all know and love:

config.properties:


mail.user=foo
mail.password=bar

spring-config.xml:


<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="classpath:config.properties"/>
</bean>

<bean class="com.acme.SomeClass" p:username="${mail.user}" p:password="${mail.password}"/>

PropertyPlaceHolderConfigurer includes a systemPropertiesMode attribute to override configuration using the command line. For example, this example could be overriden adding -Dmail.user=baz to the server startup script.

Spring imports

This is an old habit of mine, a "main index file" that imports everything else. It is a bit cleaner than specifying every file in the web.xml descriptor:


<?xml version="1.0" encoding="UTF-8"?>
<beans>

 <import resource="spring-properties.xml"/>
 <import resource="spring-mail.xml"/>
 <import resource="spring-web.xml"/>
 <import resource="spring-persistence.xml"/>
 <import resource="spring-security.xml"/>
 <import resource="spring-scheduler.xml"/>
 
</beans>

Say I want to work at home, and for the time being I would prefer to replace "spring-mail.xml" with "mock-mail.xml". Well, PropertyPlaceHolderConfigurer cannot be used for this because substitution is applied after the imports are executed, instead of before.

Our first idea was to process this file using ant to replace placeholders like ${spring.mail.file} with the actual file location. It works, but launching ant after every change in the environment is time-consuming, anti-agile, and plain ugly. You also end up manually inspecting files to see which mocks are being applied, which sucks.

Spring JavaConfig

Spring JavaConfig is a promising little project still in its alpha stage. It aims at providing a more flexible way to configure spring using your favorite JVM language (java, groovy, etc). Definitely a good idea.

At first sight it may cut it:


@Configuration
public abstract class MockConfiguration {

  /** set to true to activate mock */
  private boolean mockMail;

  @Bean
  public MailInterface mail() {
    return mockMail? mockMail() : realMail();
  }

  // The main xml file should define both real and mock beans,
  // we just proxy them
  @ExternalBean
  public abstract MailInterface realMail();

  @ExternalBean
  public abstract MailInterface mockMail();

}

This class is like advanced maths courses at my University: it is correct, works, is kinda cute, and is totally useless.

This solution involves too much code per mocked system. With about a dozen external systems, MockConfiguration can get quite huge, and your (integration) tests can get quite complicated since they will include parts of the global system, where no "mail" bean will be defined.

In my experience, Spring JavaConfig is a great solution looking for an appropriate problem, which mine is not. We moved on.

Placeholders in ApplicationContext

I found this while scanning some spring sources. Suppose your web.xml file includes several spring files:


<context-param>
 <param-name>contextConfigLocation</param-name>
 <param-value>classpath:spring/myconfig.xml,classpath:spring/otherconfig.xml</param-value>
</context-param>

Well, according to AbstractRefreshableConfigApplicationContext.resolvePath, you could also have used placeholders:


<context-param>
 <param-name>contextConfigLocation</param-name>
 <param-value>classpath:spring/${SPRING_MAIL},classpath:spring/${SPRING_PERSISTENCE}</param-value>
</context-param>

Then define them as system properties (using -D) or environment variables:


export SPRING_MAIL=mock-mail.xml
export SPRING_PERSISTENCE=mock-mail.xml

- or -


java -DSPRING_PERSISTENCE=mock-persistence.xml -DSPRING_MAIL=mock-mail.xml ...

The only problem with this is that I don't like solutions that involve the system administrators too much: the war file is not fully configured by itself, which could be a concern, and involves an amount of work proportional to the number of servers to deploy. It is also a bit hard to automatize.

Damn. This one has been long, and I hate long posts. Well, here it is in the hope that someone else finds it useful, or decides to contribute with his/her own experience. Back to work!

UPDATE: You will see a comment by Mark Fisher that explains how to use PropertyPlaceHolderConfigurer with import statements, which renders the last part of this post useless so I decided to remove it (again: I hate long posts).
UPDATE(2): For Yet Another Approach at spring config, check out this post by Robert Mark Bram.

Design your products with real-life experience in mind

2008-02-15T01:00:00.001+01:00

Real-life requirements are strokes of a bigger landscape. Designing for a partial set of requirements is bad practice because you do not have the photo where your application is expected to fit. In GoF patterns jargon we are talking about the forces thing that prevents any pattern from being applicable to Each and Every Case.

Things are much easier when you are developing customized software, but with products there are many more decisions to make. The old habit of designing standards from scratch is stepping aside for experience-driven architecture, where existing use cases are the basement of further design. Even doing so, you will not get the whole picture until users start flooding your system, giving you feedback about how well your system fits The Real Thing and how many cases you left unresolved.

My way (well, sorta)

When talking about products, I follow a very simple design process. Hey, I am not saying this is top-notch approach to product design or anything, just "there are many like it, but this one is mine".

First of all, I always start refusing to code. If you don't yet own a copy of Getting Real, I cannot recommend it enough (even more if you live in Europe, since they only charge you 2 € for shipping). From the book,


"Each time you say yes to a feature, you're adopting a child. You have to take your baby through a whole chain of events (e.g. design, implementation, testing, etc.) [...] Don't be a yes-man." - Getting Real, 37signals

I like to call that suicide requirements: every time a customer calls with a feature request, first try to apply standard YAGNI judo techniques, or give a workaround with existing features. When the same request appears frequently, accept the defeat and add it to the schedule. You lost.


“…the cost of adding a feature isn't just the time it takes to code it. The cost also includes the addition of an obstacle to future expansion [...] The trick is to pick the features that don't fight each other.” - John Carmack.

Do NOT develop anything without a real use case first (more than one would be better, but don't be picky). By doing this you are buying some certainty of avoiding features never needed by real life, and as a side effect helps prioritizing features and bugfixes.

Delay the implementation as much as a reasonable schedule permits. Start talking to your Java hens, giving them time to lay the idea that will save the day, improve sexual life and alleviate global warming (or not). I cannot stress this enough: I have lost entire months for not giving it enough pencil time.

When implementation time comes, apply the best brewed design and cross fingers; if you are lucky, you won't have to change it THAT much.

Integrate your framework with JavaRebel - NOW

2008-02-05T01:00:00.000+01:00

In case you missed it, on-the-fly reloading is the hot thing right now.

A great argument in the everything-vs-Java flame war says that you can miss your children growing while waiting for the change-redeploy-test cycle to complete. Matt Raible and Tim Fennell expressed it as a reasonable idea: web frameworks should redeploy actions on the run.

Changing JVM bytecode has been possible since JVM 1.4: your IDE will send updated bytecode to the debugged VM, given that no changes affect the class structure (attributes or methods), and JavaRebel was born to fill that niche. Starting with JavaRebel 1.1M1, there is a SDK to be notified about class reloading events. After seeing how Jevgeni successfully connected JavaRebel with Google Guice, I reserved a weekend to peek at the promising new release (isn't that what weekends are for? no? really?).

Three hours. That's all it took.

Within three hours, Loom was detecting and redeploying modified actions. Configure your eclipse project to use uncompressed folders (which you should have done already), add the javarebel agent ("-noverify -javaagent:${REBEL_HOME}/javarebel.jar"), and that's all. Changes to action classes will be applied on the next browser refresh.

The point you should remember is the following: it took me a couple of hours to get it working. Start making noise in your favorite forums: hibernate, struts, wicket, whatever - there is no such concept as "loud enough". Reclaim your hot-redeploy features.

I would love to see this feature included in the JDK some day :)

P.S.: To be fair, getting the first action reloaded took was really fast, but a real polished behavior in jetty and tomcat, reloading as the affected JPA classes were modified took a bit more.
UPDATE: ZeroTurnaround has issued an open letter to the Java Community expressing their intention of supporting Java frameworks that plan to integrate on-the-fly class reloading.

A brief note about accessibility

2007-12-13T11:00:00.000+01:00

A note to all those who think: "everyone has flash installed and javascript enabled. We are not supporting Amish browser 1.0".

I happened to install ubuntu 64 bits at work some months ago. Installing flash in a 64 bits linux distro is not straightforward and I haven't dedicated the time yet - after all, I seldom need to browse a flash site. But every now and then I happen to browse a website with a big white hole in the middle of the screen and a big NOTHING filling it, and have to guess what I am missing. It's okay (and expected) if I cannot see the last YouTube thing, but requiring flash to see a city map seems like a misfortunate decision. I ended up annotating the address and going to Google Maps instead.

Think about accessibility like this: a little amount of possible customers that are going to move to other sites because your site is thought, designed and oriented for WASPs. Cell phone browsers do not have a mouse, the elderly may prefer bigger fonts, some users cannot have flash installed because his sysadmin is a security freak concerned about demons and aliens invading this dimension (not my case, again; I'm lazy).

Your environment is not THE environment. There is a consistent percentage of your market that is going elsewhere - not because they don't like your website, just because you don't care.

Validate web forms using JPA annotations

2007-12-11T10:30:00.003+01:00

How many times I have repeated my database constraints to validate user input, I don't know. I don't like that. The RoR crowd nailed this with the ActiveRecord implementation and for the last two years the Java community has tried to imitate them.


"The workman of today works every day in his life at the same tasks, and this fate is no less absurd. But it is tragic only at the rare moments when it becomes conscious." - Albert Camus, The Myth of Sisyphus

What about instead of trying to imitate the ActiveRecord strategy (which is counterintuitive since java classes are not intended to be modified on-the-fly) focus on our strengths: since java 5 two different programming languages can be used in the same source file. Use one for behavior (java) and another to define data structure (annotations) that can be used to generate the database tables and validate user input at the same time:

We first tried extending the existing libraries (stripes, struts, jsf) and ended up developing a new web framework entirely from scratch. I am not going to replicate the whole getting started chapter here, but give a 10-sec introduction. With a source file like this:


Mortgage.java:

@Entity
public class Mortgage extends AbstractProduct {

@Basic(optional=false)
@Column(length=80)
private String address;

@Basic(optional=false)
@NumberValidation(maxValue="100", excludeMax=true)
private BigDecimal interestRate;

}

editMortgage.jsp:

<l:form action="Mortgages" event="save">
 <l:inputText name="mortgage.address"/>
 <l:inputText name="mortgage.interestRate"/>
</l:form>

The framework will generate the following HTML:


<label for="mortgage.address"><span>Dirección:</span>
<input type="text" size="40" maxlength="80" name="mortgage.address" id="mortgage.address"
  class="required string validate"/></label>

<label for="mortgage.interestRate"><span>Interés:</span>
<input type="text" size="3" maxlength="3" name="mortgage.interestRate" id="mortgage.interestRate"
  class="_options-excludeMax-true _options-maxValue-100 _options-minValue-0 _options-scale-0 number required validate"/></label>

For the curious, you can see this form in action or view the source files. There are some things you should notice:

Javascript validations are generated from JPA annotations: try to introduce invalid input and see what happens.

The maxlength and size attributes have been generated from the same annotations used to generate the database.

Javascript messages are internationalized. Change from english to spanish (beware: decimal point changes from '.' to ','). Note that the property name is also translated.

The validation CSS classes can define a different CSS style for required, disabled, numeric fields and so on.

The browser and the server use the same validation messages. Javascript and Java share the same message templates.

Notice the size of the JSP file, because real-life applications do not get much bigger than that

And the entire demo site will still work with javascript turned off. Not that it matters that much, but ain't it cool :)

The demo application also includes examples of paginated queries and an entire JPA + Spring configuration.

I am writing this after returning from a vacation week in Zurich. I have the incredible luck of combining one of our best clients and one of my best friends in the same city. Periodically I get the chance to rediscover the city again: five-stories music shops, the Rhine Falls, charming people and oh-my-gawd-the-food with lots, lots of cheese... I definitely should lose some weight before the biggest and greatest of Christmas arrive.

Wish you all the same. Merry Christmas everyone!

Avoid Flash Of Unstyled Content using javascript

2007-11-19T11:30:00.000+01:00

Don't hate it when the browser displays ugly HTML for a second, then thinks it twice and reformats it as expected?

A Flash Of Unstyled Content (from now on FOUC) happens when the browser shows unstyled content before applying the CSS style sheet (wikipedia). It used to be associated to old versions of IE but also with accessible interfaces that offer richer javascript alternatives. Consider the following:

List menu: the bare HTML should show all options expanded (in case javascript is off, the visitor is a search engine... pick your choice) and use javascript to collapse unselected options.

Tabs: same thing, show all tab contents and hide the unselected ones.

Any kind of collapsible interface, Blogger for instance: if the connection is slow you can glimpse the collapsed components while the browser downloads the entire document.

Solutions are not straightforward: components that do not yet exist in the DOM tree cannot be hidden using javascript so they must be created (and maybe displayed) before they can be manipulated.

Register a "DOM document loaded" event handler

An example using prototype:


document.observe("dom:loaded", new function() {
   $('myDiv').hide();
});

This waits until the document is fully downloaded and will flash if the page is big enough or the connection crappy enough.

Place a <>script> tag after the element


 <div id="myDiv">
 ...
 </div>
 <script>
   $('myDiv').hide();
 </script>

Yikes. You are desperate, right? This script requires that the DOM node already exists, and that sometimes means that the script tag must be placed somewhere else (next to the nearest ending DIV tag, for example). This would not work as a generic approach for a JSP tag (TabTag, MenuTag) that does not know how the surrounding HTML looks like.

CSS sleight-of-hand

Create a CSS style on the fly before the affected HTML content:


<script>
 document.write('<style type="text/css">.hide-fouc { display:none; }</style>');
 document.observe("dom:loaded", function() {
   $$('.hide-fouc').invoke('removeClassName', 'hide-fouc');
 });

<script>
<div id="myDiv" class="hide-fouc">
  ...
<div>

A browser with javascript enabled will not see the component, but javascript-less browsers will. The CSS class is removed after the entire document has been loaded to allow the element to reappear later.

Note that this snippet should be included inside the HTML (not as a external javascript file) and that you should achieve better performance by removing the display:none attribute from the CSS class instead of using a selector, but as said before I am a lazy bastard.

HOWTO: Uncompressed web projects in eclipse without plugins

2007-10-29T17:03:00.000+01:00

In case you are still redeploying war files on a daily basis, it is surprisingly easy to configure your eclipse project for deployment with tomcat/jetty without any need for plugins. I'm writing this down because every now and then I have to explain it from scratch, and in case you are of the impatient kind who prefers code to instructions a complete web project download has been included at the end of this post.

First, create a vanilla project structure:

In this example all java classes will be stored at src/java and the web root folder will be src/web. Now, configure the compiled classes folder at WEB-INF/classes:

There is now a complete uncompressed web application in src/web: just add any third party jars to WEB-INF/lib, your java classes will be compiled to WEB-INF/classes and any properties/xml file that should be in the classpath will be copied there.

Now to the deployment part:

Jetty

Copy the following descriptor to your $JETTY_HOME/contexts folder (better using an ant task):


<?xml version="1.0"  encoding="ISO-8859-1"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure class="org.mortbay.jetty.webapp.WebAppContext">

<Set name="contextPath">/started</Set>
<Set name="resourceBase">${basedir}/src/web</Set>

</Configure>

Where "basedir" is the root folder of your eclipse project.

Jetty will recognize your webapp at startup, but we want to launch it inside eclipse so let's give it another turn. Create a normal java application launcher in your project:

Check that the working directory is the jetty home folder. You will need a JETTY_HOME environment variable for this:

It may seem counterintuitive but the only jar that must be included is the jetty startup jar, the rest will be discovered by jetty. It is important that you remove the project from the classpath before launching:

If you intend to debug your java classes (and you do), you have to add your source files to the launcher. Add the project as well as any sources zip file that you may need (here I have included a couple of my own)

That's all. Launch the jetty instance with the Run/Debug toolbar buttons just as you would with any normal java application. Any changes done while debugging will be detected on the fly by the server, given that they do not affect the class structure (method signatures or number of attributes/methods).

Tomcat

Deploying to tomcat is almost the same process as jetty. The application descriptor should be copied to $CATALINA_HOME/conf/Catalina/localhost (you may have to create some folders). Be aware that the file name is the web prefix that the deployed application will have:


<?xml version="1.0"  encoding="ISO-8859-1"?>
<Context
docBase="${basedir}/src/web"
reloadable="false"
 antiResourceLocking="false"
 antiJARLocking="false">

</Context>

(You may have noticed that this application is not reloadable in order to make changes to classes detected by the runtime while debugging).

The rest is the same as the jetty example, using org.apache.catalina.startup.Bootstrap as Main Class and ${env_var:CATALINA_HOME} as Working Directory.

The classpath is different, though. You probably won't need all the java security files unless you are using SSL but hey, having them surely won't hurt:

The following environment variables are also needed:

That's all. I recommend preparing several installations of jetty and tomcat with different ports if you switch projects very often to avoid having like a ton of applications in the same server and wasting hours of startup time.

To download an example web project file with all these changes included, click here. It includes the main project structure, ant targets to deploy the web application, and eclipse launchers included by default in the run/debug drop-down lists. It's Christmas before Christmas. Enjoy!

About Bug Ranking and Why You Should Care

2007-10-03T23:08:00.000+02:00

There is a whole ranking of bug types that drive a big share of my decisions on framework selection and code design given that I spend much more time working on bugs than developing new code.

By ascending difficulty or chronological solving, the order is the same:

Compile time: the easiest, because compilers shout them out loud: IDEs highlight offending code, ant/maven will refuse to build your artifacts, etc. Eclipse will even replace the buggy code with a runtime error ("A typo in line 239, you been drinkin' again?") producing a working class with a runtime exception that can be used by a testcase. This allows you to execute things that don't even compile.

Deployment time: configuration mistakes noticed when the application gets deployed or the testsuite launched (spring xml errors, web.xml, ejbs, persistence...). These are quite easy to fix.

Reproducible execution time (emphasis on 'reproducible'): errors that happen each and every time. Once you get the right scenario you can reproduce it in a testcase and leave it as a regression test for generations to come. Memory leaks are the hardest bugs in this category.

Not reproducible execution time: happens only in production/preproduction environments with no recognizable scenario (jvm cores, concurrency, problems without a stack trace such as blank pages, etc). Your only tools are your wits, your favorite caffeinated beverage and truckloads of patience.

It makes sense that applications should be designed to displace bugs to the easiest (think generics, for example) but too often I see designs that move features from java to XML or optimization tips that rely heavily on singletons, introducing concurrency issues.

Duck typing languages (ruby and javascript) have by design little support included in the interpreter/IDE. The language itself may help the first writer but hurts the rest that have to use that code - again, developing code from scratch is not the primary concern. I prefer strongly-typed languages that compile and review my code as I type, helping me to introduce fewer bugs.

Switching your spring XML file to Ruby/java/whatever does not achieve real improvements as you are only changing one syntactic sugar with another, which has null influence over bug locations or productivity.

Delegating features from JSP scriptlets to Java classes move deployment bugs to compile time, with extra testcases as a bonus. JSP compilation can also be executed at deployment time instead of the first call by tuning your server (like the precompile parameter of weblogic), but I would only recommend this with testing and production environments to avoid penalizing the whole development team (with real applications, the startup time can become one of those "come back later" things).

The scope of your web application objects is also related with bug types: request, flash, conversation, session and application scopes all have concurrency issues bigger from left to right. Classes stored in anything but the request scope should be aware of possible multiple tabs on the same browser, multiple browsers on the same station, and multiple users on the same aplication. Keep your objects with a scope as narrow as possible and you will be fine.

There is a Bug Policy Nazi Role in my team, assumed by the person responsible of reviewing each framework looking for things that make life easier. To me, this is one of the biggest concerns. You should care, too.

Stupid conclusions using google trends

2007-07-12T08:27:00.000+02:00

This whole thing started because of a project I had to integrate with Oracle BPM. Well, If you search for Oracle BPM, there is only the standard marketing stuff and no technical article, which is weird.

There are three kind of lies: Lies, Damned Lies, and Statistics - Benjamin Disraeli

That (and an idle Friday afternoon) was what got this started. With so much hype about SOA (coining such awesome terms like "an SOA" by people without anything better to do) I decided to separate Signal from Noise launching some BPM-related queries to Google Trends.

Any product adoption curve has an S-shaped form, which means that by now (the market starting and the products anything but mature yet) the BPM graph should be rising. Instead, you get a flat line:

Ouch. The tools are buggy, there is close to none documented success stories or best practices, and there seems to be the same interest today than two years ago - in fact, a little LESS (considering actual people searching instead of news coverage).

The next natural question is how well the major application servers are doing these days. Even with all the buzz about spring and lightweight and the no-more-EJBs-thank-you trend, I wasn't expecting this:

Re-ouch. It seems that glassfish is a good product with bad timing, since any other provider is ramming downhill. As an interesting side note, glassfish may be the only server that is not used by India, India, India, Japan, and India (to see the detailed localized stats, use the combo box at the bottom of the page, named "rank by")

Being such a JSF lover, I really believe that no stupid conclusions list would be complete without a web framework survey:

Somewhere between 2005 and 2006, We The Crowd must have noticed that life with JSF sucked big time. The JSF top is still monopolized by India, which may imply that it's the outsourcing technology of choice or not mean anything at all.

At this point I started searching for Really Funny Stuff, since that is what Fridays are for. If you feel curious about the relationship between Windows Vista and a regular souffle:

Which has a curious resemblance with sex, something that sounds logical if you care about how much Microsoft is charging for Vista.

That was all about my Friday. Do you have any other query to launch? Alcoholic beverages? God vs Sex? Go ahead and share!