If you are around Milan join us it will be a lot of fun. For your convenience I have pasted here the full agenda, but don’t forget to visit the official group and sign in for partecipation.

9.30: Welcome coffee
9.45: Welcome and first infos about JBUG
10.15: JBoss AS7
11.00: Coffee Break
11.15: JBoss AS7 and webservices
12.00: JBoss in cloud with OpenShift
12.45: Closing
13.00: luch
14:00: Hacking AS7

And don’t miss the hacking party we will have in the afternoon: Alessio and me will be in the public area of Red Hat office working on JBoss AS7, and would be cool to have you hacking with us. Don’t leave your laptop at home, join the community and send your first patch in that afternoon.

And now, for our italian reader, for this italian event…the post in italian language:

Alessio ed io faremo due presentazioni su AS7 durante il primo meeting del JBUG di Milano il prossimo 20 Settembre.

Se passte da Milano, non perdetevi l’evento, ci divertiremo. Per vostra comodità riporto qui l’agenda definitiva dell’incontro, ma non scordatevi di fare un giro sul gruppo ufficiale del JBUG ed iscrivetevi all’evento mandando la mail come indicato in uno dei messaggi.

9.30: Welcome coffee
9.45: Benvenuto e prime informazioni sul JBUG
10.15: JBoss AS7
11.00: Coffee Break
11.15: JBoss AS7 e i webservices
12.00: JBoss in cloud con OpenShift
12.45: Closing
13.00: Pranzo a buffet
14:00: Sviluppo di AS7

E mi raccomando, non perdetevi l’hacking su AS7 nel pomeriggio. Io ed Alessio ci fermeremo in area pubblica dell’ufficio Red Hat di Milano a lavorare su JBoss AS7. E sarebbe bello sviluppare insieme a voi. Non dimenticatevi il computer , partecipate alla community mandando il pomeriggio stesso la vostra prima patch

See you there!

Ci vediamo li!

As you probably already know AS7 has been announced by Red Hat officialy here. I’ll not go in deep details of what is new, cool and fast in AS7. Just have a look to the announce page to get a bird eye view of all the cool stuffs you can find there.
Before start with main contets about data source I just want to recall you 2 main concept in AS7 we will have to deal with in this article:

1. Modular core: JBoss Modules offers true application isolation, hiding server implementation classes and only loads the classes your application needs. Class loading is concurrent for extreme performance. OSGi support is available the moment you install the application server. We will learn in this article to create a new module for new jdbc drivers, and we compare this approach to hot deployed driver analyzing plus and minus of each.
2. Elegant administration Consistent and powerful management is available, ranging from a polished, user-friendly web console to Java and HTTP APIs to a command line tool to direct XML edits. Configuration data is centralized and user-focused. We will learn in this artcile how to add and manage datasources with all three approaches.
3. Domain management JBoss AS 7 can be launched in two different modes. Domain mode allows you to run and manage a multi-server topology. Standalone mode runs a single server instance, a convenience choice for development. Many settings, such as port offsets and network interfaces, can be controlled with a single attribute. Rolling deployments are available. In this article we will discuss how to add and manage datasources in both mode, providing sample for both too. Domain mode is one of the main new concept and coolest feature to learn about AS7. If you want more info read documentation about this concepts here.
   

The easiest way to hot deploy a driver in standalone mode is to copy the jar into the $JBOSS_HOME/standalone/deployments (don’t forget to read the README.located there!). If you are copying a jdbc 4 compliant driver you will get a message like this into console where you have started the standalone server:

12:59:17,663 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) Starting deployment of "mysql-connector-java-5.1.15.jar"
12:59:18,191 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-3) Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
12:59:18,291 INFO  [org.jboss.as.server.controller] (DeploymentScanner-threads - 2) Deployed "mysql-connector-java-5.1.15.jar"

Standalone: Start the server in standalone mode, open another console, launch the jboss-admin.sh tool. Run these commands:

[standalone@localhost:9999 /] connect
Closed connection to localhost:9999
Connected to standalone controller at localhost:9999
[standalone@localhost:9999 /] deploy /dati/drivers/mysql-connector-java-5.1.15.jar
'mysql-connector-java-5.1.15.jar' deployed successfully.

You will get an output to server running console identical to one pasted in section 1.1.1.

Domain: Start the server in domain mode, pen another console, launch the jboss-admin.sh tool. Run these commands:

[standalone@localhost:9999 /] connect
Closed connection to localhost:9999
Connected to domain controller at localhost:9999
[domain@localhost:9999 /] deploy --all-server-groups /dati/drivers/mysql-connector-java-5.1.15.jar
'mysql-connector-java-5.1.15.jar' deployed successfully.

You will get this output to server running console:

[Server:server-one] 13:07:51,933 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-5) Starting deployment of "mysql-connector-java-5.1.15.jar"
[Server:server-two] 13:07:51,934 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-6) Starting deployment of "mysql-connector-java-5.1.15.jar"
[Server:server-two] 13:07:52,344 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
[Server:server-one] 13:07:52,355 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
[Server:server-two] 13:07:52,441 INFO  [org.jboss.as.server.controller] (pool-1-thread-1) Deployed "mysql-connector-java-5.1.15.jar"
[Server:server-one] 13:07:52,441 INFO  [org.jboss.as.server.controller] (pool-1-thread-1) Deployed "mysql-connector-java-5.1.15.jar"

Standalone: Start the server in standalone mode. Open your browser at address http://localhost:9990/console/ and follow this steps.

Domain: Start the server in domain mode. Open your browser at address http://localhost:9990/console/

Using the web console, don’t forget to enable the driver, because as you can see from the screen cast you will not get driver deployed and available until it is excplicit enabled.

There is another way to set up a driver in AS7: create a module providing classes needed and define the driver at domain level. Doing that you have several advantage in a domain based production environment perspective. You are going to define a driver in a unique place, without limits mentioned in 1.1. It is not so practical only if you are developing and you need to deploy/undeploy drivers for you tests and/or if you are building yourself AS7 from source and you are used to rebuild it quite frequently.

But it’s very good for a production environment, having drivers present in domain controller runtime state, and also in domain.xml. It is in my humble opinion more clear and easy to maintain for an administrator who will probably try to focus on the domain and server groups and not on single servers.

But how to create a ne module and use it as driver? With 4 simple steps

1. Create the directory structure for the module. For example to create a module for mysql driver:

   cd $JBOSS_HOME
   mkdir modules/com/mysql
   mkdir modules/com/mysql/jdbc
   mkdir modules/com/mysql/jdbc/main

2. Copy the driver(s) jar into the created directory
3. Create a module.xml file to define dependencies and resources. For our mysql example it looks like that:

   <?xml version="1.0" encoding="UTF-8"?>
   <module xmlns="urn:jboss:module:1.0" name="com.mysql.jdbc">
     <resources>
       <resource-root path="mysql-connector-java-5.1.15.jar"/>
           <!-- Insert resources here -->
     </resources>
     <dependencies>
       <module name="javax.api"/>
       <module name="javax.transaction.api"/>
     </dependencies>
   </module>

4.  Add driver to the domain/standalone. You have 2 option here:
1. Edit standalone.xml/domain.xml adding a driver element into datasources subsystem

   [...]
   <subsystem xmlns="urn:jboss:domain:datasources:1.0">

               <datasources>
                   <datasource jndi-name="java:jboss/datasources/ExampleDS" enabled="true" use-java-context="true" pool-name="H2DS">
                       <connection-url>jdbc:h2:mem:test;DB_CLOSE_DELAY=-1</connection-url>
                       <driver>h2</driver>
                       <pool></pool>
                       <security>
                           <user-name>sa</user-name>
                           <password>sa</password>
                       </security>
                       <validation></validation>
                       <timeout></timeout>
                       <statement></statement>
                   </datasource>
                   <drivers>
                       <driver name="h2" module="com.h2database.h2">
                           <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
                       </driver>
                       <driver name="mysql" module="com.mysql.jdbc"> <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class> </driver>
                   </drivers>
               </datasources>
           </subsystem>

   [...]

2.  Add it using jboss-admin.sh command line interface

   /subsystem=datasources/jdbc-driver=mysql:add(driver-name="mysql",driver-module-name="com.mysql.jdbc",driver-xa-datasource-class-name="com.mysql.jdbc.jdbc2.optional.MysqlXADataSource")

Then you can use the driver to configure a new datasource. Note that you can specify also xa-datasource-class, and it will be used in every xa-datasource you will configure to use this driver. As said in case you are using deployed driver it’s not possible at the moment and you have to configure this class name for each xa-datasource.

Note that if you are creating a module using a non jdbc4 compliant driver you HAVE TO SPECIFY also driver class using <driver-class> tag into <driver> of standalone/domain xml. As said in 1.1 this is the only way to use a non jdbc4 compliant driver.

1.3 installed-driver-list operation. Use and caveats

As said you the driver-name property is important to refer a driver during a datasource configuration. It’s so important to know which drivers are configured and available in a server/domain. To achieve this we are providina specific operation callable from jboss-admin.sh command line interface and used under the wood by web console during datasource configuration. Here you have an example of this call for a server with just h2 driver configured:

[standalone@localhost:9999 /] /subsystem=datasources:installed-drivers-list
{
    "outcome" => "success",
    "result" => [{
        "driver-name" => "h2",
        "deployment-name" => undefined,
        "driver-module-name" => "com.h2database.h2",
        "module-slot" => "main",
        "driver-xa-datasource-class-name" => "org.h2.jdbcx.JdbcDataSource",
        "driver-class-name" => "org.h2.Driver",
        "driver-major-version" => 1,
        "driver-minor-version" => 2,
        "jdbc-compliant" => true
    }]
}

There is a caveat in domain mode, since installed drivers  are read from runtime state in a server; state which does not exist on the domain controller. It depends on runtime state in a server because if a driver is in a deployment we have no idea until it is deployed; i.e. until it goes through the deployers which only exist on the server. IOW you will find into the profile only drivers defined in modules, while deployed drivers are present only at local server level. An example should be more clear than any description:

[domain@localhost:9999 /] /profile=default/subsystem=datasources:installed-drivers-list
{
    "outcome" => "success",
    "result" => "no metrics available"
}
[domain@localhost:9999 /] /host=local/server=server-one/subsystem=datasources:installed-drivers-list
{
    "outcome" => "success",
    "result" => [{
        "driver-name" => "h2",
        "deployment-name" => undefined,
        "driver-module-name" => "com.h2database.h2",
        "module-slot" => "main",
        "driver-xa-datasource-class-name" => "org.h2.jdbcx.JdbcDataSource",
        "driver-class-name" => "org.h2.Driver",
        "driver-major-version" => 1,
        "driver-minor-version" => 2,
        "jdbc-compliant" => true
    }]
}

2. Add a local datasource

Now that we have learned how to add a new driver we can add a new datasource. Also in this case we have 3 different way to add it: editing xml, using command line interface, or using web console. These examples will be just for standalone mode, to don’t be too much verbose, leaving domain mode as exercise for readers and get feedbacks from them.

In all cases we are going to create a Mysql datasource using a driver created as module.

2.1. Adding a datasource in configuartion xml (standalone.xml)

               <datasource jndi-name="java:jboss/datasources/MySqlDS" enabled="true" use-java-context="true" pool-name="MySqlDS">
                    <connection-url>jdbc:mysql://localhost:3306/test</connection-url>
                    <driver>mysql</driver>
                    <pool></pool>
                    <security>
                        <user-name>root</user-name>
                        <password></password>
                    </security>
                    <validation></validation>
                    <timeout></timeout>
                    <statement></statement>
                </datasource>

/subsystem=datasources/data-source="java:jboss/datasources/MySqlDS2":add(jndi-name="java:jboss/datasources/MySqlDS2",pool-name="MySqlPool",driver-name="mysql",connection-url="jdbc:mysql://localhost:3306/test",user-name="root")

Here you have example of the 3 way to create an xa-datasource in AS7 in domain mode. Don’t’ forget that we are using a driver created as module and so the xa-datasource-class is defined at driver level. As said in 1.1 if you are using an hot deployed driver don’t forget to set this class name for each xa-datasource you are going to create.

               <xa-datasource jndi-name="java:jboss/datasources/MySqlXADS" enabled="true" use-java-context="true" pool-name="MySqlXADS">
			    <xa-datasource-property name="ServerName">localhost</xa-datasource-property>
			    <xa-datasource-property name="DatabaseName">test</xa-datasource-property>
			    <xa-datasource-property name="User">root</xa-datasource-property>
			    <xa-datasource-property name="Password"></xa-datasource-property>
			    <driver>
				mysql
			    </driver>
                </xa-datasource>

/subsystem=datasources/xa-data-source="java:jboss/datasources/MySqlXADS2":add(jndi-name="java:jboss/datasources/MySqlXADS2",pool-name="MySQLXA_Pool2",enabled=true,xa-data-source-properties = {"ServerName"=>"localhost","DatabaseName"=>"test","User"=>"root"},driver-name="mysql")

4. How to test a datasource is correctly working without writing code

We provide an operation to test a datasource is able to open a connection to the declared database. You can use it from command line console jboss-admin.sh. It’s very easy to call:

/subsystem=datasources/xa-data-source=java\:jboss\/datasources\/MySqlXADS2:test-connection-in-pool

Positive answer:

{
    "outcome" => "success",
    "result" => [true]
}

Negative answer:

{
    "outcome" => "failed",
    "failure-description" => "failed to invoke operation: Connection is not valid",
    "rolled-back" => true
}

5. Jndi Name

Please note that in current implementation the only valid jndi name for datasources have to start with java:/ or java:jboss/

6. Conclusion

Play with it, try something different, ask question and report bugs on AS7 user forum or (for more internal things) on IronJacamr User forum.

I’ll try to answer questions and collect the frequently asked ones in another post here.

Have fun and contribute to AS7 and IronJacamar

This kind of issues usually pops up through a test / section of the software that simply hangs without no evident reason. You don’t get continuous integration results for a day or such, then wonder what’s happening, go and check Hudson and start thinking and asking yourself what might have gone bad. In my case it was really “asking yourself” as of course this happened exactly when I was back from a vacation week and my colleagues that were working on the project were not online   (to be honest at the end it turned out not to be their fault at all)

OK, so you start by getting a thread dump to confirm you’re seeing a deadlock; that’s a simple “kill -3 pid” when running on Linux. Here is mine:

[junit] Found one Java-level deadlock:
[junit] =============================
[junit] "pool-1-thread-1":
[junit]   waiting to lock monitor 0x000000005d2c60c0 (object 0x00002aaadfb10fd8,
                                          a sun.net.www.protocol.file.Handler),
[junit]   which is held by "main"
[junit] "main":
[junit]   waiting to lock monitor 0x000000005cb02a58 (object 0x00002aaadfb10e38,
                                          a sun.misc.Launcher$AppClassLoader),
[junit]   which is held by "pool-1-thread-1"
[junit]
[junit] Java stack information for the threads listed above:
[junit] ===================================================
[junit] "pool-1-thread-1":
[junit] 	at java.net.URLStreamHandler.getHostAddress(URLStreamHandler.java:412)
[junit] 	- waiting to lock <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
[junit] 	at java.net.URLStreamHandler.hostsEqual(URLStreamHandler.java:439)
[junit] 	at sun.net.www.protocol.file.Handler.hostsEqual(Handler.java:117)
[junit] 	at java.net.URLStreamHandler.sameFile(URLStreamHandler.java:396)
[junit] 	at java.net.URLStreamHandler.equals(URLStreamHandler.java:316)
[junit] 	at java.net.URL.equals(URL.java:842)
[junit] 	at java.security.CodeSource.equals(CodeSource.java:135)
[junit] 	at java.util.HashMap.get(HashMap.java:305)
[junit] 	at java.security.SecureClassLoader.getProtectionDomain(SecureClassLoader.java:233)
[junit] 	- locked <0x00002aaadfb11120> (a java.util.HashMap)
[junit] 	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
[junit] 	at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
[junit] 	at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
[junit] 	at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
[junit] 	at java.security.AccessController.doPrivileged(Native Method)
[junit] 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
[junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
[junit] 	- locked <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
[junit] 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
[junit] 	- locked <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
[junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
[junit] 	at org.xnio.nio.AbstractNioChannelThread.cancelKey(AbstractNioChannelThread.java:298)
[junit] 	at org.xnio.nio.NioHandle.cancelKey(NioHandle.java:56)
[junit] 	at org.xnio.nio.AbstractNioStreamChannel.cancelWriteKey(AbstractNioStreamChannel.java:241)
[junit] 	at org.xnio.nio.NioTcpChannel.shutdownWrites(NioTcpChannel.java:164)
[junit] 	at org.xnio.channels.TranslatingSuspendableChannel.shutdownWrites(TranslatingSuspendableChannel.java:358)
[junit] 	at org.xnio.channels.FramedMessageChannel.shutdownWrites(FramedMessageChannel.java:255)
[junit] 	- locked <0x00002aaadfb11978> (a org.xnio.ByteBufferSlicePool$PooledByteBuffer)
[junit] 	at org.jboss.remoting3.remote.RemoteConnection$RemoteWriteListener.send(RemoteConnection.java:267)
[junit] 	- locked <0x00002aaadfb119a0> (a org.jboss.remoting3.remote.RemoteConnection)
[junit] 	at org.jboss.remoting3.remote.RemoteConnection.sendCloseRequest(RemoteConnection.java:164)
[junit] 	at org.jboss.remoting3.remote.RemoteConnection.handleOutboundCloseRequest(RemoteConnection.java:149)
[junit] 	at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAction(RemoteConnectionHandler.java:173)
[junit] 	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:354)
[junit] 	at org.jboss.remoting3.ConnectionImpl.closeAction(ConnectionImpl.java:48)
[junit] 	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:149)
[junit] 	at org.xnio.IoUtils.safeClose(IoUtils.java:134)
[junit] 	at org.jboss.as.protocol.ProtocolChannelClient.close(ProtocolChannelClient.java:176)
[junit] 	at org.xnio.IoUtils.safeClose(IoUtils.java:134)
[junit] 	at org.jboss.as.protocol.mgmt.ManagementClientChannelStrategy$Establishing.requestDone(ManagementClientChannelStrategy.java:138)
[junit] 	at org.jboss.as.protocol.mgmt.ManagementRequest$DelegatingResponseHandler.readResponse(ManagementRequest.java:178)
[junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel$ResponseReceiver.handleResponse(ManagementChannel.java:375)
[junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel$ResponseReceiver.access$400(ManagementChannel.java:357)
[junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel.doHandle(ManagementChannel.java:120)
[junit] 	at org.jboss.as.protocol.ProtocolChannel.handleMessage(ProtocolChannel.java:158)
[junit] 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:224)
[junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] 	at java.lang.Thread.run(Thread.java:619)
[junit] "main":
[junit] 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:293)
[junit] 	- waiting to lock <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
[junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
[junit] 	at java.net.URL.getURLStreamHandler(URL.java:1144)
[junit] 	at java.net.URL.<init>(URL.java:393)
[junit] 	at java.net.URL.<init>(URL.java:283)
[junit] 	at java.net.URL.<init>(URL.java:306)
[junit] 	at sun.net.www.protocol.file.Handler.openConnection(Handler.java:74)
[junit] 	- locked <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
[junit] 	at sun.net.www.protocol.file.Handler.openConnection(Handler.java:55)
[junit] 	- locked <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
[junit] 	at java.net.URL.openConnection(URL.java:945)
[junit] 	at sun.net.www.protocol.jar.JarURLConnection.<init>(JarURLConnection.java:66)
[junit] 	at sun.net.www.protocol.jar.Handler.openConnection(Handler.java:24)
[junit] 	at java.net.URL.openConnection(URL.java:945)
[junit] 	at org.apache.cxf.common.logging.JDKBugHacks.doHacks(JDKBugHacks.java:67)
[junit] 	at org.apache.cxf.common.logging.LogUtils.<clinit>(LogUtils.java:66)
[junit] 	at org.apache.cxf.jaxws.spi.ProviderImpl.<clinit>(ProviderImpl.java:61)
[junit] 	at java.lang.Class.forName0(Native Method)
[junit] 	at java.lang.Class.forName(Class.java:247)
[junit] 	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:345)
[junit] 	at java.util.ServiceLoader$1.next(ServiceLoader.java:421)
[junit] 	at javax.xml.ws.spi.Provider.getProviderUsingServiceLoader(Provider.java:146)
[junit] 	at javax.xml.ws.spi.Provider.provider(Provider.java:106)
[junit] 	at javax.xml.ws.Service.<init>(Service.java:57)
[junit] 	at javax.xml.ws.Service.create(Service.java:687)
[junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionEJB3Helper.getProxy(ExceptionEJB3Helper.java:48)
[junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionHelper.testRuntimeException(ExceptionHelper.java:81)
[junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionTestCase.testRuntimeException(ExceptionTestCase.java:45)
[junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
[junit] 	at junit.framework.TestCase.runTest(TestCase.java:154)
[junit] 	at junit.framework.TestCase.runBare(TestCase.java:127)
[junit] 	at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] 	at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] 	at junit.framework.TestResult.run(TestResult.java:109)
[junit] 	at junit.framework.TestCase.run(TestCase.java:118)
[junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] 	at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
[junit] 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
[junit] 	at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] 	at junit.extensions.TestSetup.run(TestSetup.java:23)
[junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit]
[junit] Found 1 deadlock.

Confirmed, it was really a deadlock. Not a straightforward one to solve, at least to me, given the blocking part were a bit deep into the JDK (1.6) sources.

A bit of background on what’s been developed here and which are the game “players”: JBossWS provides and integration layer for running Apache CXF on top of JBoss Application Server. The testcase being executed above (in thread “main”) is a simple standard JAX-WS client application that creates a java.xml.ws.Service instance for consuming an endpoint that’s deployed on JBoss AS. The “pool-1-thread-1″ thread should instead be spawned by the remote deployment request that’s issued by the testsuite to deploy a WS endpoint jar archive on JBoss AS before actually running the client test.

The problem was not easily and consistently reproducible locally, so I used a debugger to force the “evil” timing that causes the deadlock: you set a breakpoint on the code executed by both threads above, before each of them goes through one of the “- locked <…>” points. Then have each of them acquire the first lock and you’re done, sure deadlock when each thread later wants the other lock.

So, carefully analysing the problem I understood what was really happening. The deadlock above is between the system classloader and sun.net.www.protocol.file.Handler, due to their synchronized Classloader::loadClass() and Handler::openConnection()/Handler::getHostAddress() methods. During initialization, Apache CXF performs some hacks, including disabling url caching; to achieve that, it opens a new connection as JDK does not have a static method for setting that, despite the flag being actually static. The process of opening a connection goes through the Handler’s synchronized openConnection() method and in there ends up needing the system classloader for loading a class. Unfortunately, the lock on the system classloader might be taken by another thread invoking the synchronized loadClass() on it and also needing to use the CodeSource for locating the class to be loaded. That eventually ends up in needing the URLStreamHandler / Handler for comparing URL instances.

URLStreamHandler? For those not having experience here, Java comes with a configurable architecture for resolving URLs. Simplifying things a lot, the URLStreamHandler is created using the specified URLStreamHandlerFactory, which can be configured setting the java.protocol.handler.pkgs system property. Why is this relevant here? Basically because JBoss needs and uses this mechanism for installing his own handler to deal with vfs URLs. Looking at the code for the configured factory, the returned handler is basically a singleton. Hence the deadlock, regardless of the vfs-aware handler not being needed here.

How to solve the issue? While it might be arguable whether having singleton protocol handler is a good choice or not, I couldn’t change that quickly and easily, as it was “out of my area of action”. I ended up changing the Apache CXF code after having had a chat with Dan Kulp (CXF project lead): instead of actually opening a connection to get an handle to a URLConnection instance for the sake of disabling URL caching, we now simply create a concrete extension of the abstract URLConnection that is only used for setting the static member through its setDefaultUseCaches() non-static method.

The fix is effective for the specific issue I had and in general for reducing the chances of running into deadlock because of Apache CXF. However, few considerations / caveats are still valid:

• be careful when designing and providing your own protocol handlers, as that can have really unexpected side effects
• opening a connection can be time consuming even before URLConnection::connect() due to all the locking that’s required in there; obviously another reason for checking twice if you really need opening a connection before actually doing that
• is JDK really well designed here? (non-static URLConnection.setDefaultUseCaches(..))

It’s been a nice debugging in any case

[junit] Found one Java-level deadlock:
    [junit] =============================
    [junit] "pool-1-thread-1":
    [junit]   waiting to lock monitor 0x000000005d2c60c0 (object 0x00002aaadfb10fd8, a sun.net.www.protocol.file.Handler),
    [junit]   which is held by "main"
    [junit] "main":
    [junit]   waiting to lock monitor 0x000000005cb02a58 (object 0x00002aaadfb10e38, a sun.misc.Launcher$AppClassLoader),
    [junit]   which is held by "pool-1-thread-1"
    [junit]
    [junit] Java stack information for the threads listed above:
    [junit] ===================================================
    [junit] "pool-1-thread-1":
    [junit] 	at java.net.URLStreamHandler.getHostAddress(URLStreamHandler.java:412)
    [junit] 	- waiting to lock <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
    [junit] 	at java.net.URLStreamHandler.hostsEqual(URLStreamHandler.java:439)
    [junit] 	at sun.net.www.protocol.file.Handler.hostsEqual(Handler.java:117)
    [junit] 	at java.net.URLStreamHandler.sameFile(URLStreamHandler.java:396)
    [junit] 	at java.net.URLStreamHandler.equals(URLStreamHandler.java:316)
    [junit] 	at java.net.URL.equals(URL.java:842)
    [junit] 	at java.security.CodeSource.equals(CodeSource.java:135)
    [junit] 	at java.util.HashMap.get(HashMap.java:305)
    [junit] 	at java.security.SecureClassLoader.getProtectionDomain(SecureClassLoader.java:233)
    [junit] 	- locked <0x00002aaadfb11120> (a java.util.HashMap)
    [junit] 	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
    [junit] 	at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
    [junit] 	at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    [junit] 	at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    [junit] 	at java.security.AccessController.doPrivileged(Native Method)
    [junit] 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    [junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
    [junit] 	- locked <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
    [junit] 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    [junit] 	- locked <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
    [junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    [junit] 	at org.xnio.nio.AbstractNioChannelThread.cancelKey(AbstractNioChannelThread.java:298)
    [junit] 	at org.xnio.nio.NioHandle.cancelKey(NioHandle.java:56)
    [junit] 	at org.xnio.nio.AbstractNioStreamChannel.cancelWriteKey(AbstractNioStreamChannel.java:241)
    [junit] 	at org.xnio.nio.NioTcpChannel.shutdownWrites(NioTcpChannel.java:164)
    [junit] 	at org.xnio.channels.TranslatingSuspendableChannel.shutdownWrites(TranslatingSuspendableChannel.java:358)
    [junit] 	at org.xnio.channels.FramedMessageChannel.shutdownWrites(FramedMessageChannel.java:255)
    [junit] 	- locked <0x00002aaadfb11978> (a org.xnio.ByteBufferSlicePool$PooledByteBuffer)
    [junit] 	at org.jboss.remoting3.remote.RemoteConnection$RemoteWriteListener.send(RemoteConnection.java:267)
    [junit] 	- locked <0x00002aaadfb119a0> (a org.jboss.remoting3.remote.RemoteConnection)
    [junit] 	at org.jboss.remoting3.remote.RemoteConnection.sendCloseRequest(RemoteConnection.java:164)
    [junit] 	at org.jboss.remoting3.remote.RemoteConnection.handleOutboundCloseRequest(RemoteConnection.java:149)
    [junit] 	at org.jboss.remoting3.remote.RemoteConnectionHandler.closeAction(RemoteConnectionHandler.java:173)
    [junit] 	at org.jboss.remoting3.spi.AbstractHandleableCloseable.closeAsync(AbstractHandleableCloseable.java:354)
    [junit] 	at org.jboss.remoting3.ConnectionImpl.closeAction(ConnectionImpl.java:48)
    [junit] 	at org.jboss.remoting3.spi.AbstractHandleableCloseable.close(AbstractHandleableCloseable.java:149)
    [junit] 	at org.xnio.IoUtils.safeClose(IoUtils.java:134)
    [junit] 	at org.jboss.as.protocol.ProtocolChannelClient.close(ProtocolChannelClient.java:176)
    [junit] 	at org.xnio.IoUtils.safeClose(IoUtils.java:134)
    [junit] 	at org.jboss.as.protocol.mgmt.ManagementClientChannelStrategy$Establishing.requestDone(ManagementClientChannelStrategy.java:138)
    [junit] 	at org.jboss.as.protocol.mgmt.ManagementRequest$DelegatingResponseHandler.readResponse(ManagementRequest.java:178)
    [junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel$ResponseReceiver.handleResponse(ManagementChannel.java:375)
    [junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel$ResponseReceiver.access$400(ManagementChannel.java:357)
    [junit] 	at org.jboss.as.protocol.mgmt.ManagementChannel.doHandle(ManagementChannel.java:120)
    [junit] 	at org.jboss.as.protocol.ProtocolChannel.handleMessage(ProtocolChannel.java:158)
    [junit] 	at org.jboss.remoting3.remote.RemoteConnectionChannel$4.run(RemoteConnectionChannel.java:224)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    [junit] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    [junit] 	at java.lang.Thread.run(Thread.java:619)
    [junit] "main":
    [junit] 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:293)
    [junit] 	- waiting to lock <0x00002aaadfb10e38> (a sun.misc.Launcher$AppClassLoader)
    [junit] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
    [junit] 	at java.net.URL.getURLStreamHandler(URL.java:1144)
    [junit] 	at java.net.URL.<init>(URL.java:393)
    [junit] 	at java.net.URL.<init>(URL.java:283)
    [junit] 	at java.net.URL.<init>(URL.java:306)
    [junit] 	at sun.net.www.protocol.file.Handler.openConnection(Handler.java:74)
    [junit] 	- locked <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
    [junit] 	at sun.net.www.protocol.file.Handler.openConnection(Handler.java:55)
    [junit] 	- locked <0x00002aaadfb10fd8> (a sun.net.www.protocol.file.Handler)
    [junit] 	at java.net.URL.openConnection(URL.java:945)
    [junit] 	at sun.net.www.protocol.jar.JarURLConnection.<init>(JarURLConnection.java:66)
    [junit] 	at sun.net.www.protocol.jar.Handler.openConnection(Handler.java:24)
    [junit] 	at java.net.URL.openConnection(URL.java:945)
    [junit] 	at org.apache.cxf.common.logging.JDKBugHacks.doHacks(JDKBugHacks.java:67)
    [junit] 	at org.apache.cxf.common.logging.LogUtils.<clinit>(LogUtils.java:66)
    [junit] 	at org.apache.cxf.jaxws.spi.ProviderImpl.<clinit>(ProviderImpl.java:61)
    [junit] 	at java.lang.Class.forName0(Native Method)
    [junit] 	at java.lang.Class.forName(Class.java:247)
    [junit] 	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:345)
    [junit] 	at java.util.ServiceLoader$1.next(ServiceLoader.java:421)
    [junit] 	at javax.xml.ws.spi.Provider.getProviderUsingServiceLoader(Provider.java:146)
    [junit] 	at javax.xml.ws.spi.Provider.provider(Provider.java:106)
    [junit] 	at javax.xml.ws.Service.<init>(Service.java:57)
    [junit] 	at javax.xml.ws.Service.create(Service.java:687)
    [junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionEJB3Helper.getProxy(ExceptionEJB3Helper.java:48)
    [junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionHelper.testRuntimeException(ExceptionHelper.java:81)
    [junit] 	at org.jboss.test.ws.jaxws.samples.exception.ExceptionTestCase.testRuntimeException(ExceptionTestCase.java:45)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
    [junit] 	at junit.framework.TestCase.runTest(TestCase.java:154)
    [junit] 	at junit.framework.TestCase.runBare(TestCase.java:127)
    [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:106)
    [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:124)
    [junit] 	at junit.framework.TestResult.run(TestResult.java:109)
    [junit] 	at junit.framework.TestCase.run(TestCase.java:118)
    [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [junit] 	at junit.framework.TestSuite.run(TestSuite.java:203)
    [junit] 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
    [junit] 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
    [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:124)
    [junit] 	at junit.extensions.TestSetup.run(TestSetup.java:23)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
    [junit]
    [junit] Found 1 deadlock.

More details, as well as the presentation abstract, available on the JBUG Munich website. If you’re interested and happen to be around…

So, what’s up with JBossWS in 2010 ? The project has gone through two major sets of releases. The 3.3.x series kind of finalized the JBossWS move to having the Apache CXF based stack as its preferred one, installed by default on JBoss Application server. Integrating a third-party piece of software is always something non-trivial. And even if the overall quality of what we consume (and contribute to, of course) from Apache is definitely very high, that’s simply not enough when it comes to integrating: many issues were discovered, dealt with and solved… any many are probably still to come.

The type of development you end up doing when providing a solution like JBossWS-CXF is pretty much different from what you are on when developing stuff from scratch. I kind of expected that to be honest. It’s not that unusual to spend days on looking for the best way of re-using / integrating already existing (or partially available) functionalities, trying to figure out an elegant solution for achieving the goal on JBoss side, while enriching the third-party side of the software in a way that makes sense regardless of JBoss needs. Sometime I think it’s like “plumbing”. For sure you need to go on constantly swapping your hats, the JBoss employee one and the Apache contributor one, in my case. And given we operate in an open source world here … you might even find yourself producing fully vendor agnostic solutions just for the sake of solving your own (JBoss here) problem.

Some might dislike this kind of work, thinking satisfaction can come only from creating your own solutions from scratch, feeling they’re your own babies. Others might appreciate the integration work, enjoying the  new challenges in this. To be honest, I think this is a really subjective feeling and you can find yourself excited by the achievements you reach in both cases. You should probably try both in any case.

Anyway, back to bringing Apache CXF in JBoss through JBossWS… implementing the JSR 109 requirements is a good example of integration. Apache CXF provides WS functionalities and successfully verifies those are compliant with the core JCP WS specifications (JSR 224 – JAXWS, for instance). However CXF of course does not care about all the details on how that’s supposed to work with a given application server according to the rest of the JavaEE specification, which JSR 109 is a good example of. In an ideal world the missing bits need to live in the integration project (JBossWS here)… In reality you end up coordinating different needs, reviewing and rationalizing stuff on both sides, to basically make the integration happen and be a success.

The second set of JBossWS 2010 releases has been the 3.4.x series. While the announcement is recent story, that was a multiple months effort from me and the rest of the team. We went through active collaboration on CXF (Apache contributor hat on ) to have it implement JAXWS 2.2 and make it pass the TCK certifaction testsuite for that. While on that, we implemented the proper integration for passing the corresponding ws modules of JavaEE 6 TCK on JBoss side (with *the* red hat on ). This all was done while directly targeting development snapshots of Apache CXF, hence with multiple moving targets (JBoss AS, Apache CXF, our own JBossWS integration layer and even the TCK which was not final yet) to track for potential regressions.

At the end of the year I’m quite satisfied by what we got. From a job point of view, I’m just waiting for Santa to come with a nice present… a final release of JBoss AS 6 (I sent him the jbossws maven artifacts to include in the box , he should have got them in time… despite the snow over north Italy these days).

OK, now it’s time to start relaxing a bit, to enjoy the Christmas spirit…  then I’ll come back with new intentions for the next year, both directly related to JBossWS and not…

Merry X-mas and happy new year!

http://in.relation.to/Bloggers/AS700Alpha1Released

As Title of the post already said I’ve been lucky working with great people and developer contributing on this release (IronJacamar integration mainly).

As you can see from the Jason’s post and a lot of tweet about that there is a lot of fun in this release, performance, new kernel based on service and domain administration and deployment. Check it out and give us your feed back through public forum.

Have Fun!

http://in.relation.to/17567.lace

As said in the original post we have worked mainly on AS7 integration in this release, ad it was a lot of fun

abstract:

[...] collecting metadata set that represent links between classes under test and test classes (and eventually more fine grained links between methods) and using them for a lot of purposes. Metadata can be collected from different sources: annotations on classes under test, instrumentation of test classes during tests’ execution (in future perhaps even from a dedicated user interface). These metadata can be used for various goals: first of all it becomes possible to run only the tests that stress just a particular set of classes (for instance the classes changed since last compilation) [...] We have a Maven plugin, while Eclipse and Hudson ones are in our roadmap.

Full story:

A while ago I’ve written a post about a new approach to unit tests; it has been a quite read and discussed post and it gives the momentum for starting a new community discussing about the idea and working on a software project implementing this idea. The project was named TestedBy and hosted in google code.

That idea has evolved and changed and we have decided to change also the name of the project as a remark of these changes, calling it SibillaTest. Why Sibilla? Because, as you can read in this post and you can get trying our samples, Sibilla (italian name for Cumae Sybil) is always able to prophesy and predict which tests you need to run.

This post has been written to announce the release of a first beta version of SibillaTest and describe how our thoughts about the original idea have changed in these months and how they have been  implemented in SibillaTest.
I’m going to recall the main concepts of the original idea, If you haven’t read the original blog would be nice to take a look to it, to its numerous comments and also to the update I’ve posted few days after. If you are interested in all the genesis of the project you could also have a look to the discussions continued in our forum

Generally speaking what I’ve depicted in my old blog post was a system to keep annotations in classes under test pointing test classes (or in some case to specific test methods). Advantage of this approach were mainly 2:

• Design By Contract (viewing test as contract definition). The sugar here was the opportunity to put TestedBy annotation also on super classes (even if they are abstract or even interfaces) and inherit test annotations.
• Run only tests stressing a specific class. IOW run test stressing a class of your interest (i.e just compiled) giving you the confidence you aren’t breaking test suite without running the entire suite itself.

The main concerns about this approach was about code cluttering putting a lot of annotations in production code (of course you could have a lot of test stressing a class or even a method).

So we have changed a bit the focus from an annotation centric approach to a more general metadata
approach. The idea is to collect a set of metadata that represent link between classes under test and test classes (and eventually more fine grained links between methods) and use them for a lot of purposes.
The pluses of this approach are mainly two:

• Metadata can be collected from different sources: annotations , instrumentation of test during test execution, maybe in the future from a dedicated  user interface.
• Metadata can be serialized and/or used in a second step for various goals: run “right” tests of course will remain a central feature, but also
“graphical” representation, code navigation in IDE, more dynamic use of it

I’m trying to keep this post not too long trying to keep your attention focused on the usefulness of SibillaTest,  I’m going to describe in this the feature already implemented in SibillaTest. But, of course, we have a lot of ideas that will be developed in next future that we would like to discuss with the community and maybe have some contribution from the community. Here you have a mind map depicting a lot of these working areas to get you an idea. If you are interested in a more detailed description of them you  can find a page in our wiki describing it.

So, what is SibillaTest today?

aims at changing the point of view regarding test classes and classes under test. What we get is the focus being moved to the classes under test; from those classes, which of course are the most important ones of your projects, we obtain links to your test classes and test methods. In other words we define the classes and methods’ contract using tests, while also keeping track of tests that need to be run when a class/method under test has been modified. This is possible by collecting metadata sets that represent links between classes under test and test classes (and eventually more fine grained links between methods) and using them for a lot of purposes. Metadata can be collected from different sources: annotations on classes under test, instrumentation of test classes during tests’ execution (in future perhaps even from a dedicated user interface). These metadata can be used for various goals: first of all it becomes possible to run only the tests that stress just a particular set of classes (for instance the classes changed since last compilation), moreover a graphical representation of classes’ links can be derived. We have a Maven plugin, while Eclipse and Hudson ones are in our roadmap. Further cool idea around this is supporting generic tests to inject on existing production code (to verify commonly situation like don’t accept null parameters) and mock verification (are your mocks respecting the contract you have defined on mocked classes with your tests?)

Practically speaking in this beta release we have:

• a core supporting both annotation based metadata definition and instrumentation of classes during test execution to collect metadata. With annotation we support metadata on test stressing a whole hierarchy of classes/interfaces: you can define tests to be run against all implementation of your interface and SibillaTest take care of it Instrumentation do the magic of knowing which test is stressing particular classes and run only tests needed to validate last changed classes (i.e just compiled)
• a maven plugin (docs here) you can use to run test stressing only last changed classes.
• support of Junit tests (not yet testng or other frameworks)

In our wiki you can find some documentation of how to include Sibilla into your maven project, how to run it and documentation about our annotations if you decide to give TestedBy design by contract approach a try. Here you can also find a small project with trivial classes under test and test classes using SibillaTest to run them. Check it out and give it a try…it’s the key to understand the idea and to say “wow it’s cool!”

Have a look and send us comment through this blog or, much better, through our issue tracker and/or our forum. We have also an IRC channel #sibilla active on FreeNode for discussions and suggestions.

Artifact are available on maven central repository, see the docs too.

Some other post with detailed descriptions of instrumentation and annotation based metadata and their use and plans for next versions will follow during next days. Moreover we will discuss on forum and post here also plans of integration with the very cool Arquillian (I had a very interesting discussion about that with Aslak at last JUDCon)….

….stay tuned.

Of course if anyone is interested to join us and help (especially for eclipse plugin) please write us.

Stay tuned, join us and show some love around on the net!

http://sibilla.javalinux.it

(Forum – IRC – GitHub – Sample(github) – Maven Plugin(github) – @SibillaTest on twitter – issues)

See you in Berlin

I’ve written here very rarely of other passions I have apart technology, java and distributed system. I’ve written just one time I think about my passion for Wine and Food (well I’m italian men!).

I’ve never written here about my passion for sports and the recent maniacal passion for running (just something on twitter). But this morning I’ve reached an important goal I can’t resist to share with a rapid blog post: 1000, one thousand! kilometers. When just 9 months ago I’ve started my runner’s career 1000 km seemed to me a goal too far and to hard to get. Today I’m running something like 60/65 km per week and 1000kms isn’t a goal, but a starting point.

I know you can’t understand me and I seem completely mad to you. It was exactly what I thought about runners 10 months ago , but I can assure you I’m not mad, and running I’m feeling much much better that months ago both with my body and with my mind. Running could be considered meditation on movement. Trust me it is!

I run everywhere I am, so I’ll run in Berlin at early morning before next JUDCon…would you join me? We could have a nice geek talk during our run!

I look forward for this geek run (maybe I’ll make a dedicated post about this geek run…but feel free to contact me immediately if you are interested)

Have fun running and hacking!