Mutant World

Jetty Support for Cross-Domain XMLHttpRequests

2009-08-04T18:24:00.004+02:00

Remember the same origin policy that restricts most of our JavaScript applications ?

Seems that we can finally get rid of it, thanks to a new W3C specification already implemented by Firefox 3.5 and, on server side, by the Jetty Servlet Container.

I blogged about all the details here.

Oh, if you're using Ubuntu Jaunty, you can install Firefox 3.5 via apt://firefox-3.5. It is named "Shiretoko" (its codename), but it's exactly Firefox 3.5.

Time to move on !

JavaScript Cometd

2009-07-29T17:01:00.003+02:00

Lately I have been working with JavaScript, in particular I have written a JavaScript library that implements the Bayeux specification for the Cometd project.

I have to say that I really like JavaScript, the language.

Don't be fooled by "Oh, it's only for modifying CSS in web pages" because it's really a nice language, although it's misunderstood.

You can write nice applications in JavaScript and today JavaScript toolkits (such as Dojo or jQuery) helps you to figure out browser differences and make a wonderful job at reducing the code that you have to write, allowing you to concentrate on the business and not on the technical details.

I have blogged about the Cometd library I have written here.

Enjoy !

Creative Webcam Live! on Ubuntu Intrepid Ibex

2009-01-13T00:28:00.002+01:00

I received the Creative Webcam Live! as a gift quite some time ago, but never had the time to use it or install it properly.
My good ol' Thinkpad T43p was maybe bleeding edge in 2005, but not as far as having already an incorporated webcam

Under Ubuntu Intrepid, this webcam does not work out of the box, at least for me, and I have a fresh (re)install of Intrepid.
This camera appears using lsusb as:


Bus 003 Device 008: ID 041e:4036 Creative Technology, Ltd Webcam Live!/Live! Pro

The solution is quite easy though: you need to set this variable before launching the program that uses the webcam:


export LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so

I tested this with camorama and skype, and in both cases it works just fine.
For skype, I created a wrapper script that first set the variable as above and then invokes /usr/bin/skype and modified the menu entry to point to my wrapper script.

Visibility in JavaScript

2008-12-29T14:39:00.008+01:00

The topic of visibility of members and functions in JavaScript has been figured out already, and there are good resources, among which one of the most important is Douglas Crockford's Private Members in JavaScript.

Here I just recap my experience for my own (and for anyone interested) future reference.

I am pretty paranoid in making all members and methods with the least possible visibility in my Java code. Before opening up a method as protected or public, one should always remind that most of the times, once it's opened up, you cannot close it because other code uses it.

JavaScript can have private members and functions, and I mostly use two idioms for this: the constructor function and the the object initializer.

The constructor function

I use the constructor function technique when I want to create few objects that behave the same (in Java you would say "few objects that belong to the same class", but JavaScript does not have classes).


var MyNS = {}; // An optional namespace
MyNS.Service = function(offset)
{
    var _privateMember = offset;

    function _privateFunction(value)
    {
        return _privateMember + value;
    }

    this.myFunction = function(value)
    {
        _privateMember = _privateFunction(value);
        return _privateMember;
    }
};

The common convention is that function names beginning with an upper case letter are constructor functions, in this case Service.

From the example above, private members are defined within the constructor function using the keyword var, private functions are just nested functions within the constructor function, and publicly accessible functions are defined with the this.<functionName> idiom.
The function myFunction in the example above is a privileged function, but for the sake of visibility it can be classified as public because anyone can invoke it.
From the example above, you see that myFunction can refer to private members and functions.

However, referring to publicly accessible functions from private functions is not possible:


// A constructor function without namespace; does not work
function Service2()
{
    var _open;

    function _clean()
    {
        if (_open) close(); // Does not work: ReferenceError
    }
    
    this.close = function()
    {
        ...
    }
}

It is not possible to call privileged functions from private functions. In my experience this is not a big problem, as it is possible to refactor function close() into a private function _close() and call the private version from both _clean() and close().

With a constructor function it is possible to create several objects that behave the same. In the example above you can create several service objects using the following syntax:


var s1 = new MyNS.Service(1);
var result1 = s1.myFunction(5); // returns 6

var s2 = new MyNS.Service(2);
var result2 = s2.myFunction(0); // returns 2

As you would expect, object s1 and s2 have different internal state and can be used independently.

The object initializer

I use the object initializer technique when I want to create a singleton object.
My preferred way of using the object initializer technique is via a function with immediate invocation, which allows to have private members and functions:


MyNS.EventHandler = function()
{
    var _queue = [];

    function _privateFunction(event)
    {
         ...
    }
    
    return {
        handle: function(event)
        {
            _privateFunction(event);
        },
        get size()
        {
            return _queue.length;
        }
    };
}();

Note the parenthesis at the end of the function definition, which perform the immediate invocation of the function.

Similarly to the constructor function technique, it is possible to define private members and functions, but the object initializer technique also allows (for non-crappy JavaScript interpreters) to use the getter and setter notation.
In the example above I used the getter notation to define a read-only property called size, which can be accessed like this:


var s = MyNS.EventHandler.size;

Using the getter notation is much nicer than having a getter function, and much better than having a public member (which would be not only readable but also writable).

Caveats

Both solutions outlined above have the problem that for each object created in those ways, the function definitions are also duplicated: each object will have its own copy of the functions, and this will result in more memory occupation. That's why I suggest the constructor function technique when you create few objects and the object initializer technique for singletons.

At first, I thought that there must have been a way to have private visibility for member and functions, but only one copy of the function definitions, very much like Java does: each Java object has its own copy of the members, but they all share the methods definitions.

Unfortunately I could not figure out how to do it, not even reading JavaScript library code such as Dojo or jQuery.

However, there is a different technique that it is possible to use to share the functions definitions, which I will call here "public members and prototype".

Public members and prototype

When you have to create lots of objects with members and functions, the best technique is to define the functions in the prototype, so that they will be shared by all objects, and have public members holding the state.
You have to give up on restricted visibility (members must be public) to save memory (only one copy of the functions is shared by all objects), because functions defined in the prototype can only (to my knowledge) refer to public members.


MyNS.Event = function(source) 
{
    this._source = source;
};
MyNS.Event.prototype = function()
{
    var _privateStatic;

    function _privateFunction()
    {
    }

    return {
        consume: function() 
        {
            this._consumed = true;
        },
        get source()
        {
            return this._source;
        }
    };
}();

In this example, MyNS.Event is a constructor function that allows to create events via:


var source = window;
var event1 = new MyNS.Event(source);

The constructor function defines a public member called _source, which can be later referred from functions.
Also functions in the prototype can define public members (like _consumed in function consume()).

All function definitions are defined in the prototype. Note how the prototype object is returned via immediate function invocation, which again allows to have private functions and members as in previous techniques.

The interesting part comes when you define private members in the prototype, like _privateStatic.
These members can only be modified by functions defined in the prototype, which are shared by all objects. The result is that modifications made by one object to _privateStatic (via shared prototype functions) are reflected by all other objects created with the same constructor function. This behavior is the same behavior of static members in Java.

Instead, public members modified by functions defined in the prototype are attached to the object (not to the prototype), so that each different object has its own copy of the members, and therefore they can be modified independently:

 
function A(value)
{
    this._field = value;
}
A.prototype = function()
{
    var _staticField;
    return {
        staticGetter: function() 
        {
            return _staticField;
        },
        staticSetter: function(value) 
        {
            _staticField = value;
        },
        instanceGetter: function()
        {
            return this._field;
        },
    };
}();

var a1 = new A(1);
var a2 = new A(2);
a1.instanceGetter(); // returns 1
a1._field; // public field, returns 1
a1._field = 3;
a1.instanceGetter(); // returns 3
a2.instanceGetter(); // returns 2
a2.staticSetter('foo');
a1.staticGetter(); // returns 'foo'

I'd be interested in any solution that will allow private members with prototype-shared functions, if this is possible in JavaScript.

Java's uncertain future

2008-08-14T23:39:00.002+02:00

Java's future looks to me very uncertain in this period.

I started studying Java during 1999 summer, and working with it since then.

Many things happened since the release of JDK 5.

Many key people left Sun for other companies (and wow, the list of those names is truly impressive; among them Josh Bloch, Neal Gafter, Gilad Bracha, most of the Swing core team - Hans Muller, Scott Violet, Chet Haase, etc.)

Sun itself is not performing very well these days (profit plunged 73% in the last quarter).

Sun has open sourced Java.

Yet, there is no JDK 7 plan.
You heard that right, no official JSR has been opened yet, when we were used to have release of version N of the JDK and official JSR for version N+1 almost immediately opened.
But not this time, and noone seems able to predict neither when JDK 7 will be released, nor what it will contain.

There is OpenJDK.
Can anyone tell me the difference between JDK 7 and OpenJDK ?
And I am not meaning the obvious ones, but why two efforts, and how are they synchronizing ?
I never thought that open sourcing the JDK was such a great idea, from the point of view of developers, though it was probably such a good move for other fields that developers could be forgotten.

There is JDK 6 and JDK 6 update 10. Mmm. More confusion.

The closures will (most likely) not be part of JDK 7, but everyone talks about the "upcoming language changes in JDK 7" yet there is no official JSR, nor for JDK 7 nor for closures. Oops.

Java blogs are beginning to label Java as the next legacy language, but it surely will still live for many many years. I hear the same of COBOL.

It appears as if Sun released the command of the Java ship and let its best commanders flew away.
It would not be bad to see a real move from Sun about Java (because strong statements are just that, words), because hey, the ship seems DIW, dead in water.

It's 2008 summer, and I am studying Ruby.

But the worst indicator is that Hani and its Bileblog are quiet: he cannot desecrate the last moments of a dying era, can he ?

Static Fields in Enums

2008-07-27T01:17:00.003+02:00

Enums in Java have a numeric, read-only, property called ordinal that is a strictly consecutive integer (starting from 0).

Sometimes, however, it is necessary to associate to an Enum another numeric property, let's call it code, that it may not be strictly consecutive and normally specifies something different than a sequence number. Furthermore, it may be necessary to be able to lookup the correct Enum from its code.

It seems totally trivial to write an Enum that has a static map from codes to Enum instances:


public enum Port
{
  SSH(22), TELNET(23), HTTP(80), POP3(110), HTTPS(443);

  // The static map from codes to Enum instances
  private static final 
              Map<Integer, Port> ports = new HashMap<Integer, Port>();
  private final int code;

  private Port(int code)
  {
    this.code = code;
    ports.put(code, this);
  }

  public int getCode()
  {
    return this.code;
  }

  // Lookup method that returns the Enum instance from the given code
  public static Port from(int code)
  {
    return ports.get(code);
  }
}

Note the method static from(int code) that returns an Enum instance from the given code.

Unfortunately, this will not compile. The compiler reports that it is illegal to access the static member ports from the constructor.

This makes sense when one imagines how an Enum is first translated into a class by the compiler. Roughly, there is a static initializer that creates the Ports instances, in this case SSH, TELNET, etc.; this static initializer is the first initializer run when the Ports class is referenced, and it is easy to see that when the static initializer runs, no other initializers have been run yet, and in particular the static Map ports has not been created yet.
See http://java.sun.com/docs/books/jls/third_edition/html/classes.html#301020 for further details.

Fortunately, there is an easy workaround: it's enough to use a different class to store the map from codes to Enum instances, and a private static inner class does the job:


public enum Port
{
  SSH(22), TELNET(23), HTTP(80), POP3(110), HTTPS(443);

  private final int code;

  private Port(int code)
  {
    this.code = code;
    Ports.ports.put(code, this);
  }

  public int getCode()
  {
    return this.code;
  }

  // Lookup method that returns the Enum instance from the given code
  public static Port from(int code)
  {
    return Ports.ports.get(code);
  }

  private static class Ports
  {
    // The static map from codes to Enum instances
    private static final 
              Map<Integer, Port> ports = new HashMap<Integer, Port>();
  }
}

Randy Pausch Last Lecture

2008-07-27T00:57:00.002+02:00

I just watched Randy Pausch last lecture, and found it absolutely incredible, one of those gems that cannot be missed.

If you want to spend 1hr 16 mins in something really worth, then go watch it.

My favorite quote:
"The brickwalls are there for a reason: to give us a chance to show how badly we want something. Brickwalls are there to stop the people who don't want it badly enough."

Thanks Randy: for the head fake that I've got from your last lecture.

JSP page encoding

2007-09-20T22:40:00.000+02:00

I recently had a problem with a web application using JSP pages that was not handling non-ASCII characters correctly: submitted text containing characters such as ù or è resulted in garbage characters stored in the database and consequently garbage displayed by the browser.

The application was developed in Linux, using an IDE configured to save files using UTF-8 encoding, and deployed on a Linux box.
The database was configured to use UTF-8 encoding.
Every page contained the directive <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> in the <head> section.

I thought that given these premises, there was no chance of misbehaviors with respect to character encoding, but I was wrong.
The browser kept telling me that the page it was displaying had ISO-8859-1 encoding.

It turned out that the JSP specification says that if the page encoding of the JSP pages is not explicitely declared, then ISO-8859-1 should be used (!).
The Jetty servlet container was correctly setting the HTTP header as: Content-Type: text/html; charset=ISO-8859-1, following the specification.

The fix is simple, just add this to web.xml:


<jsp-config>
    <jsp-property-group>
        <url-pattern>*.jsp</url-pattern>
        <page-encoding>UTF-8</page-encoding>
    </jsp-property-group>
</jsp-config>

It will be interesting to know why the JSP expert group did not pick up UTF-8 as the default character encoding.

Upgrading to SSL an existing socket connection

2007-07-26T21:23:00.000+02:00

SSL support is in the JDK since 1.4 (and even before as a separate jar), so it is fairly common knowledge how to create and use SSLSocket and SSLServerSocket.

What is less known, or it was to me at least, is that it is possible to create a plain socket, connect to a server, exchange some message, then upgrade the existing socket connection to SSL, exchange some private message with confidentiality (for example a password that would have traveled in clear text), then downgrade back to the plain the socket connection; all this without closing the original socket nor creating a new one on a different port.

Let's see how.

The server

The server class creates a plain ServerSocket, and calls accept() on it; when a client connects, accept() returns a socket that is normally handed to a different thread (details omitted):


Socket socket = serverSocket.accept();
threadPool.execute(new Handler(socket));

class Handler implements Runnable
{
 private final Socket socket;
 Handler(Socket socket) { this.socket = socket; }
 public void run()
 {
   Socket currentSocket = this.socket;
   while (isConnectionOpen(currentSocket))
   {
     String message = readMessage(currentSocket);
     if (upgradeToSSL(message))
     {
         SSLSocketFactory sslSocketFactory =
             (SSLSocketFactory)SSLSocketFactory.getDefault();
         SSLSocket sslSocket =
             (SSLSocket)sslSocketFactory.
                 createSocket(currentSocket,
                              currentSocket.getInetAddress().getHostAddress(),
                              currentSocket.getPort(),
                              false);
         sslSocket.setUseClientMode(false);
         sslSocket.startHandshake();
         currentSocket = sslSocket;
       }
       else if (downgradeFromSSL(message))
       {
         currentSocket.close();
         currentSocket = this.socket;
       }
       else
       {
         handleNormalMessage(currenSocket, message);
       }
     }
  }
}

When upgrading, the server wraps the current (plain) socket using SSLSocketFactory.createSocket(socket, address, port, autoClose), then configures the SSLSocket to be a non-client one (for the purposes of the SSL handshake), then starts the SSL handshake to secure the connection.

When downgrading, the current (ssl) socket is just closed and, thanks to the autoClose argument set to false (which will not close the underlying plain socket), the original plain socket is still operative.

In order for this mechanism to work, you have to configure the SSL details as usual for a server. This means having created a private/public key pair in a keystore (and optionally having signed the public key with a CA root) and having set the relevant SSL system properties (or having done the equivalent using the SSL APIs).

The client

The client connects to the server using a plain Socket, then upgrades the socket connection (details omitted):


// Connects to server
this.socket = new Socket(serverAddress, serverPort);
...
Socket currentSocket = this.socket;
while (moreMessagesToSend)
{
  if (upgradeToSSL)
  {
    SSLSocketFactory sslSocketFactory =
      (SSLSocketFactory)SSLSocketFactory.getDefault();
    SSLSocket sslSocket =
      (SSLSocket)sslSocketFactory.
        createSocket(currentSocket,
                     currentSocket.getInetAddress().getHostAddress(),
                     currentSocket.getPort(),
                     false);
    sslSocket.setUseClientMode(true);
    sslSocket.startHandshake();
    currentSocket = sslSocket;
  }
  else if (downgradeFromSSL)
  {
    currentSocket.close();
    currentSocket = this.socket;
  }

  sendMessage(currentSocket);
  readReply(currentSocket);
}

Upgrading and downgrading the socket connection in the client works like in the server code, the only difference being that the SSLSocket is configured to be a client one (for the purposes of the SSL handshake). The autoClose argument prevents the original plain socket to be closed when the SSL socket is closed.

In order for this mechanism to work, you have once again to configure the SSL details as usual for a client. This means having setup properly a trust store (and optionally having imported into it the server certificate if it's not signed by a CA root) and having set the relevant SSL system properties (or having done the equivalent using the SSL APIs).

If you're writing custom network protocols that require confidentiality, the above may come handy. Enjoy !

Jonas Bonér @ JUG Torino - January 19th

2007-01-17T00:46:00.000+01:00

Jonas Bonér of Terracotta Tech will be our guest speaker at the monthly meeting of the JUG Torino on January 19th, 18:30.

The title of his talk is: How to write scalable, highly available web applications.

If you are in the Torino area, this meeting is one which will be really worth attending, both for the speaker importance and for the arguments.

Here you can find more information on this meeting.

UTF-8 handling for ResourceBundle and Properties

2007-01-01T23:49:00.000+01:00

Every new version of the JDK comes with small improvements, often unnoticed, that however correct long-standing bugs or request for enhancements.

One such improvements is - finally! - the ability for java.util.Properties to handle non ISO-8859-1 properties files.
At first seems like a very small improvement, but anyone that worked on localizing an application (for example translating messages into a country specific language) knows that encoding problems are just painful, and that is far too easy to waste days trying to get them right.

JDK 5 added a new API, namely Properties.loadFromXML(InputStream), that allowed to specify properties using an XML syntax, and hence to specify the encoding of the XML file in the XML declaration (the first line of an XML file):


<?xml version="1.0" encoding="UTF-8" ?>
<properties>
    <entry key="hello.world">Καλημέρα κόσμε</entry>
</properties>

JDK 6 went further, adding Properties.load(Reader): in case the encoding is known a priori by the program, it is possible to create a Reader that uses the specific encoding to read the properties file.

However, in JDK 5, java.util.ResourceBundle was only able to read properties files that were encoded in ISO-8859-1, and this forced the translators to put horrible unicode escapes instead of real text. Following the above example, in JDK 5 you have to write the properties file containing translations like this:


hello.world=\u039A\u03B1\u03BB\u03B7\u03BC\u03AD\u03C1\u03B1 \u03BA\u03CC\u03C3\u03BC\u03B5

Pretty ugly, especially when you think that the API to read properties file in XML format is already present in JDK 5, only that ResourceBundle has not been updated to use it.

Fortunately, with JDK 6 it is possible to solve this problem, although requires a bit of coding (see below).

In JDK 6 ResourceBundle has been extended to give the user more control on how to load resources, by subclassing ResourceBundle.Control, and passing an instance of the ResourceBundle.Control subclass like this:


ResourceBundle bundle = ResourceBundle.getBundle("messages", new MyControl());

where MyControl is the ResourceBundle.Control subclass.
The class ResourceBundle.Control allows to tune:

the kind of resource to load (a Java class, a properties file, a properties file in XML format, or any other format you decide)

the policy of expiration of the resource, thus allowing reload the resource upon some condition

the ResourceBundle subclass to instantiate, normally depending on the kind of resource

Here's the JDK 6 class that allows to load XML properties files as resource bundles.


/**
 * JDK 6's {@link ResourceBundle.Control} subclass that allows
 * loading of bundles in XML format.
 * The bundles are searched first as Java classes, then as
 * properties files (these two methods are the standard
 * search mechanism of ResourceBundle), then as XML properties
 * files.
 * The filename extension of the XML properties files is assumed
 * to be *.properties.xml
 */
public class ExtendedControl extends ResourceBundle.Control
{
    private static final String FORMAT_XML_SUFFIX = "properties.xml";
    private static final String FORMAT_XML = "java." + FORMAT_XML_SUFFIX;
    private static final List<String> FORMATS;
    static
    {
        List<String> formats = new ArrayList<String>(FORMAT_DEFAULT);
        formats.add(FORMAT_XML);
        FORMATS = Collections.unmodifiableList(formats);
    }

    @Override
    public List<String> getFormats(String baseName)
    {
        return FORMATS;
    }

    @Override
    public ResourceBundle newBundle(String baseName, Locale locale,
                                    String format, ClassLoader loader,
                                    boolean reload)
            throws IllegalAccessException, InstantiationException, IOException
    {
        if (!FORMAT_XML.equals(format))
            return super.newBundle(baseName, locale, format, loader, reload);

        String bundleName = toBundleName(baseName, locale);
        String resourceName = toResourceName(bundleName, FORMAT_XML_SUFFIX);
        final URL resourceURL = loader.getResource(resourceName);
        if (resourceURL == null) return null;

        InputStream stream = getResourceInputStream(resourceURL, reload);

        try
        {
            PropertyXMLResourceBundle result = new PropertyXMLResourceBundle();
            result.load(stream);
            return result;
        }
        finally
        {
            stream.close();
        }
    }

    private InputStream getResourceInputStream(final URL resourceURL,
                                               boolean reload)
            throws IOException
    {
        if (!reload) return resourceURL.openStream();

        try
        {
            // This permission has already been checked by
            // ClassLoader.getResource(String), which will return null
            // in case the code has not enough privileges.
            return AccessController.doPrivileged(
                    new PrivilegedExceptionAction<InputStream>()
            {
                public InputStream run() throws IOException
                {
                    URLConnection connection = resourceURL.openConnection();
                    connection.setUseCaches(false);
                    return connection.getInputStream();
                }
            });
        }
        catch (PrivilegedActionException x)
        {
            throw (IOException)x.getCause();
        }
    }

    /**
     * ResourceBundle that loads definitions from an XML properties file.
     */
    public static class PropertyXMLResourceBundle extends ResourceBundle
    {
        private final Properties properties = new Properties();

        public void load(InputStream stream) throws IOException
        {
            properties.loadFromXML(stream);
        }

        protected Object handleGetObject(String key)
        {
            return properties.getProperty(key);
        }

        public Enumeration<String> getKeys()
        {
            final Enumeration<Object> keys = properties.keys();
            return new Enumeration<String>()
            {
                public boolean hasMoreElements()
                {
                    return keys.hasMoreElements();
                }

                public String nextElement()
                {
                    return (String)keys.nextElement();
                }
            };
        }
    }
}

Java Memory Model: down to the metal

2006-12-21T23:53:00.000+01:00

Javapolis is over, and among the sessions that can be seen and heard online, there's Brian Goetz's on the Java Memory Model.

Although I knew (since a while) about the new memory model, I wanted to check out if there was something new. Brian's presentation is quite entertaining (considering the thickness of the argument), but no news with respect to what I already knew.

One thing that always intrigued me, however, is: How is the new memory model actually implemented ?

Before we go into the detail, I will make a small preamble about what the problem is in JDK 1.4, and how it has been fixed in JDK 5.

In JDK 1.4, if two threads are executing the now deprecated double checked locking code (see below), there is no guarantee that the singleton is created only once (it may be created twice) or - hard to believe but true - that a fully constructed singleton is returned (a partially constructed singleton may be returned):


public class Singleton
{
    private static Object singleton;
    public static Object getInstance()
    {
        if (singleton == null)
        {
            synchronized (Singleton.class)
            {
                if (singleton == null)
                    singleton = new Object();
            }
        }
        return singleton;
    }
}

How is it possible that the double checked locking does not work ?

It is possible because in JDK 1.4 the memory model, i.e. the interaction between threads and memory, was not well defined.
In a multi-processor machine it was perfectly legal for the first processor, running the first thread, to cache the value of the data member singleton (for example in a registry) so that the second processor, running the second thread, would have read the stale value of null, even if the first thread already updated it.

Fortunately, in JDK 5 the memory model has been fixed, so that now the effects of synchronized, volatile and final with respect to the memory model have been precisely defined.

These effects are outlined in Brian's presentation, and defined here so you may want to refer to these resources for further details.
The short story is that those keywords now impose memory barriers, i.e. they tell the processors to flush the caches to main memory, or to invalidate the caches and therefore read fresh data from main memory.

In JDK 1.4, the synchronized keyword does not guarantee a memory barrier when the lock is released. In JDK 5 the memory barrier is guaranteed. (Now, you may think that in JDK 5 the double checked locking code above works fine, but it does not - not yet).

Now that we are done with the preamble, let's go back to my original question: how are these memory barriers implemented in the JVM ?

It turns out operative systems do not have primitives (system calls) that handle memory barriers, but processors do. So the JVM goes down to the metal, from C++ to assembler, and squeezes in few assembler instructions to tell the processor to perform a memory barrier.
For example to implement a particular memory barrier called fence, in a x86 the assembler instruction is lock addl 0,(sp) where sp is the stack pointer, while in a ia64 there is a dedicated assembler instruction called mf (memory fence).

Conclusion: Once again I am thankful that the JVM takes care of these details for me, although it's quite funny to figure them out :D

Java Closures

2006-12-19T23:17:00.000+01:00

Java Closures seem to be the buzzword of the moment.
Unfortunately I missed the (I'm told great) JavaPolis conference in Antwerpen, Belgium, but fortunately the JavaPolis guys have put videos of the sessions online !

I recommend all people interested to watch or listen to Neal Gafter's session on closures.

The very interesting idea behind closures in Java (how they are shaped right now, at least) is the ability to write code that somehow "extends" the Java language syntax itself, adding what look like new keywords to the Java language itself.
Neal Gafter himself said that if closures where in the language before JDK 5, the new for loop syntax would probably not have been introduced.

How do closures look like ?

The proposed syntax for the closures in Java allows to write code such as:


Collection<T> elements = ...;
forEach (T element : elements) { doSomething(element); }

The forEach statement here looks like a new keyword of the Java language, but in reality is a method call (assume there is a static import such as import static java.util.Collections.forEach).

Another example is iterating over entries of a java.util.Map:


Map<K, V> props = ...
eachEntry(K key, V value : props) { doSomething(key, value); }

where again the eachEntry statement is a static method call to (for example) java.util.Collections.

One can even think of replacing the synchronized keyword with the classes from java.util.concurrent.locks:


final Lock lock = ...;
sync(lock) { oneThreadAtATime(); }

where sync is a method call, and not a keyword.
It is impressive how closures make sync look like a keyword, when compared to the real synchronized keyword.

Possibilities are endless: from automatically closing streams, to measuring elapsed time, to adding functional programming to collections, to simplify java.util.concurrent.Executor usage, etc.

Closures in Java are targeted for JDK 7. Seems strange to say, with JDK 6 released few days ago, but I cannot wait to see closures in Java.

Hibernate blog interview

2006-12-16T22:00:00.000+01:00

Michele Sciabarrà interviewed me on Hibernate in a short, blog-like style interview available here (in Italian).

JUG Torino Meeting

2006-11-16T15:09:00.000+01:00

Today, November 16th, the Java User Group Torino organizes its nth (we lost the exact count :-) meeting.

The meeting agenda is composed of a quickie on Subversion, hosted by me, and a session on EJB 3 and J2EE 5 hosted by Filippo.

Check out the meeting details and directions here: http://www.jugtorino.it/vqwiki/jsp/Wiki?MeetingNovembre2006

How fast are you ?

2006-11-16T01:03:00.000+01:00

A friend sent me this link today: http://www.speedtest.net/
How fast are you ?

ICMP and InetAddress.isReachable()

2006-07-28T18:33:00.000+02:00

In Java it is only possible to work with two types of sockets: stream based ones (or TCP ones - java.net.Socket and java.net.ServerSocket) and datagram based ones (or UDP ones - java.net.DatagramSocket and java.net.MulticastSocket).
The open bug 4727550 asks to support other socket types, that will allow, for example, to perform an ICMP ping and, in general, to access the sockets in raw mode.

To implement an ICMP ping, a partial solution has been introduced in J2SE 5: a way to check if some address is reachable, via java.net.InetAddress.isReachable() methods.
The implementation of these methods goes native and tries to do its best to "ping" the address represented by the InetAddress.

Surprisingly, there are many differences between the Windows and the Linux/Unix implementation of java.net.InetAddress.isReachable().

Windows, as strange as it seems, does not officially support an ICMP "ping" system call. The J2SE 5 implementation hence tries to open a TCP socket on port 7 (the echo service) and hopes to get some sort of reply.

Linux/Unix, instead, supports an ICMP "ping" system call. So the implementation of java.net.InetAddress.isReachable() first tries to perform the "ping" system call; if this fails, it falls back trying to open a TCP socket on port 7, as in Windows.

It turns out that in Linux/Unix the ping system call requires root privileges, so most of the times java.net.InetAddress.isReachable() will fail, because many Java programs are not run as root, and because the target address unlikely has the echo service up and running.
Too bad.

But why on Linux, for example, I can use the ping program without being root ?
It's because ping has the setuid bit set:


$> ls -l /bin/ping
-rwsr-xr-x 1 root root 30724 2005-11-11 01:15 /bin/ping

When a user invokes ping, it will run as root (see the "rws" permission for the owner ?).

But why on Windows there is a ping executable that does the job ?
It seems that this executable is relying on undocumented features, so everyone tries to avoid binding to those features.

So, for now, writing portable Java code that wants to send ICMP ping is quite an adventure.

Why Thread.currentThread().getContextClassLoader() ?

2006-07-13T01:08:00.000+02:00

I recently been involved in a discussion where I questioned that Thread.currentThread().getContextClassLoader() was the right way to implement the accessibility of a ClassLoader from any point in the code from a thread.
I very much preferred that ClassLoader.getContextClassLoader() was available, thus freeing class Thread of any reference to class loading.

Other examples of accessibility of thread local information are present in the Java libraries, most notably TransactionManager.getTransaction(), that returns the Transaction currently associated with the thread (though the call is not static).
There is also a RFE for having the Locale available, see RFE 6197800.

Both follow the model of not being implemented in class Thread, so I wonder why for ClassLoader it was implemented in this way. Some historical reason maybe, and probably right now is not practical to deprecate that call in favor of ClassLoader.getContextClassLoader().

JavaDay event in Torino, July 7th

2006-07-03T17:48:00.000+02:00

JUG Torino is proud to organize the first JavaDay conference in Torino, on July 7th.
JavaDay travelling conferences will be held (in different dates) in many major cities of Italy, and Torino is the first installment.

The conference schedule is pretty good, and features also two international speakers: Jason van Zyl of Maven fame, and Greg Wilkins of Jetty and WebTide fame.

Sessions Highlights: Object-Oriented development, Ajax, Maven, Jetty, Struts and more.

The JavaDay conference is free, just register at the JavaDay site.

See you there !

TestNG and Maven

2006-05-31T22:23:00.000+02:00

I've been working with JDK 5, and some time ago I decided to try TestNG as test framework, leaving back JUnit.

Since then, I got addicted to TestNG, and if possible, will never go back to JUnit, as really TestNG is leaps beyond JUnit.

Now I am working also on a JDK 1.4 application that I build using Maven.
I've been really impressed by the fact that Maven supports TestNG out of the box, even in JDK 1.4 (using Javadoc annotations). Well, time to convert all tests from JUnit to TestNG, and it's very easy:

Before:


public class MyTest extends junit.framework.TestCase
{
    public void testSomething() throws Exception
    {
        assertTrue(true);
    }
}

After:


public class MyTest
{
    /**
     * @testng.test
     */
    public void testSomething() throws Exception
    {
        assert true;
    }
}

Plus, you get all cool TestNG features:

configurable advice methods (that run before/after the suite, the groups, the class, the test)

configurable dependencies among tests

configurable number of runs

configurable parameters to pass to the test

...and much more

Big Changes and 2 Linux hacks

2006-05-26T14:57:00.001+02:00

Long time, no blog entries.
I have to say that my life changed quite a bit: a newborn baby to take care of (Riccardo), a new job and soon a new house.

Another thing that I changed (less important than the above, but daily used) is the operative system: I've switched from Microsoft Windows to Kubuntu Linux.
The move was painful, especially at the beginning, and IMHO still not possible for an average user. For my mother it would have been impossible to figure out how to make wifi and video card working, let alone the modem, on my Lenovo T43p.

However, I recently found 2 useful Linux hacks.

First Hack: How to avoid duplicate commands in the bash history (when you hit arrow up in a shell).
Open ~/.bashrc; at the beginning of the file there is an entry such as:


export HISTCONTROL=erasedups

It's probably remarked, or has another value. Setting it to 'erasedups' made the hack.

Second Hack: How to correctly configure /etc/hosts when using DHCP.
It turns out that the dhcp client may run scripts when it binds (and when it releases).
The scripts for binding are /etc/dhcp3/dhclient-enter-hooks, and all the scripts inside directory /etc/dhcp3/dhclient-enter-hooks.d/.
Correspondent scripts for releasing are /etc/dhcp3/dhclient-exit-hooks and those inside directory /etc/dhcp3/dhclient-exit-hooks.d/.

The script dhclient-enter-hooks was missing in my configuration. I created it and put this inside:


#!/bin/bash
#
# This script updates /etc/hosts with the address received by the DHCP server
#
HOSTS=/etc/hosts
BACKUP=${HOSTS}.original
LOCAL=${HOSTS}.local

make_etc_hosts ()
{
   cp -a $HOSTS $BACKUP
   cat <<EOF > $HOSTS
# This file is generated by a script: do no edit; edit instead $LOCAL
# Created by $0 on `date`
127.0.0.1 localhost localhost.localdomain
$new_ip_address `hostname`

# The following lines are desirable for IPv6 capable hosts
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

# Entries from file $LOCAL, if any
EOF

   test -f $LOCAL && cat $LOCAL >> $HOSTS
}

if [ "$new_ip_address" ]; then
   make_etc_hosts
fi

With this script, /etc/hosts gets regenerated every time the dhcp client runs, and assigns the IP address received via DHCP to the hostname of the computer.
Still not perfect (if you have multiple network interfaces), but it's possible to improve it.
For me, for now, does the job :)

Job Change

2006-01-29T18:06:00.000+01:00

After more than 6 years in Hewlett-Packard, I decided to resign and take new challenges as a consultant.

In these 6 years I've professionally learnt a lot and met great colleagues and friends, and I will certainly miss them, though many live in Torino.

Motion Illusions

2005-11-10T23:13:00.000+01:00

This picture is truly incredible.

Attaching to a Mustang, explained

2005-11-04T00:16:00.000+01:00

I was intrigued by this blog, referenced by Eamonn, so I decided to take a look at the "attach on demand" feature.

Since J2SE 5, it is possible to instrument a JVM using an "agent" with the following:

java -javaagent:<jarpath>[=<options>] <mainclass>

where <jarpath> is the path to the jar file containing the agent implementation (absolute, or relative to the directory from where the JVM has been started).
The option can be present multiple times, thus instantiating multiple agents.

The jar's manifest must contain an entry called Premain-Class that specifies the full qualified name of the agent implementation class.
A side effect of invoking the JVM with the -javaagent option is that the jar specified by <jarpath> is added to the system classpath.

Agent implementations must have a method called premain that is called by the JVM before the method main in <mainclass> is called.
In J2SE 5, the signature of premain() must be:

public static void premain(String options, Instrumentation inst)

In J2SE 6 the signature can only be:

public static void premain(String options)

That is, if the instrumentation implementation is not needed, you can drop it from the signature.

In J2SE 6, the agent mechanism has been extended so it is possible to invoke agents after the JVM has been started, and not only at JVM startup using the -javaagent option.

The mechanism is slightly different, and requires use of com.sun.* classes, so it's specific to Sun's (and derived) JVMs.
Let's take a look at how it works.

First, we need a reference to the JVMs we want to attach to. This is accomplished with the following code (classes are from com.sun.tools.attach.* package, shipped in $J2SE6/lib/tools.jar):

List<? extends VirtualMachineDescriptor> jvms = VirtualMachine.list();

Then it's possible to attach to a JVM using:

VirtualMachineDescriptor vmd = ...;
VirtualMachine jvm = VirtualMachine.attach(vmd);

Once we're attached, we can invoke the agent mechanism using:

String jarPath = ...;
String options = ...;
jvm.loadAgent(jarPath, options);

Once the agent is loaded in the target JVM, we can detach from it:

jvm.detach();

Loading the agent with this mechanism has the same side effect of adding the jar specified by jarPath to the system classpath.
The jar's manifest file must contain, this time, and entry called Agent-Class specifying the full qualified name of the agent implementation class.

However, the agent implementation must be slightly different.
Instead of calling the premain method, the JVM calls this method:

public static void agentmain(String options, Instrumentation inst)

Again, if you don't need it, you can drop the Instrumentation argument from the signature.

Inside agentmain() it's possible to write any code (with the same restrictions of premain()), so it's basically possible to do whatever one wants.

For example, using this mechanism in Mustang allows to do, in any moment, the following:

add a jar to the system classpath

be able to invoke a method of an arbitrary class (the agent)

It is possible to write a completely empty implementation of agentmain() just to be able to load new jars on demand.

Another use is to be able to "hot fix" a running JVM, using the java.lang.instrument.Instrumentation implementation to replace a malfunctioning class (with the restrictions imposed by the instrumentation mechanism).

Or can be used to tell to the target JVM to collect information about what is doing and dump it somewhere for statistical/monitoring purposes.

In Mustang there is an example of this mechanism to start a JMXConnectorServer, so that the JVM becomes remotely manageable. The jar is shipped in $J2SE6/jre/lib/management-agent.jar and it's completely empty (apart the manifest file) since the agent implementation is shipped in rt.jar (it's the sun.management.Agent class).

Feel free to drop a comment if you have other cool ideas of how to use this mechanism !

A weird particular in java.util.concurrent

2005-07-15T19:10:00.000+02:00

I am using J2SE 5, and so far I am quite happy. I got used to the generics syntax, and in general I can write less code to do the same things.

While browsing java.util.concurrent source code, I found this weird pattern:


public class LinkedBlockingQueue extends ...
{
  ...
  private final ReentrantLock takeLock = new ReentrantLock();
  ...

  private void signalNotEmpty() 
  {
    final ReentrantLock takeLock = this.takeLock;
    takeLock.lock();
    try 
    {
      notEmpty.signal();
    } 
    finally 
    {
      takeLock.unlock();
    }
  }
  ...
}

The more I looked at the code, the more it seemed to me unnecessary to declare that final local variable in the signalNotEmpty() method.
The data member is already final, so it's not a question of visibility in multithread environments.

I decided to investigate more, and went back to Bill Pugh's site, which contains a lot of information about the java memory model and its revision through JSR 133, but I had no luck.

Finally I discovered that JSR 166 implementation is also under public domain, here.

The CVS log that added the weird pattern above is marked in this cryptic comment:

"cache finals across volatiles"

Now, if you have some information about, please drop a comment.

This reminds me what JSR 133 experts were saying about the memory model: that not even the JLS writers understood it.
Now I am certain that few more people understand it (e.g. Doug Lea, Bill Pugh), but - alas - I guess they can be counted with just one hand.

I'll let you informed if I found a solution to this arcane.

UPDATE: Doug Lea had the courtesy to answer to my request for clarification. His answer:

"This was a hopefully temporary performance hack due to a limitation in JVMs that don't realize that final fields are an exception to the rule that you must re-read fields after reading a volatile.
These JVMs, including 1.5.0 hotspot are correct wrt the memory model but inefficient unless helped out in this ugly/weird way. Someday these can disappear. It's
only worth bothering (and even then only sometimes, and even then only marginally so)
for performance-critical code like that inside j.u.c.
It's not a recommended practice."

It happens that notEmpty.signal() reads volatile fields and the JVM implementation inefficiently re-reads takeLock to execute takeLock.unlock() in the finally block.