(content goes here) blog

Reviewing code quality of Apache Sling using Sonar

dev@day.com (Freddy Mallet) — 01 Jul 2009 00:00:01 GMT

This is a cross-post of Freddy's analysis at the Sonar site. We use Sonar internally at Day to track and improve the quality of all our software. Also check out Nemo which is Sonar's platform for analysing various other FOSS projects.

A few weeks ago Michael Marth, who runs dev.day.com (Day’s developer portal), asked us if we could put together our impressions on the code quality of Apache Sling using Sonar. We thought it would be valuable to share the result of this exercise with the community.

Apache Sling in a few words

“Apache Sling is an innovative web framework that is intended to bring back the fun to web development. It uses all those nice cool and new technologies that make up a state-of-the-art framework. This is Apache Sling in five bullets:

REST based web framework
Content-driven, using a JCR content repository
Powered by OSGi
Scripting inside, multiple languages
Apache Open Source project

Some size indications of the project

40 Maven modules
70,707 lines of code
731 Java classes
and 23,043 lines of Javadoc

The strengths in terms of quality

A project that you get and compile with no difficulty by running two commands:
1. svn checkout https://svn.apache.org/repos/asf/sling/trunk/
2. mvn clean install
This sounds like an evidence but is not always the case :-)
Amongst 130,172 physical lines, only 0.9% are involved in a duplication
46.4% of public API are commented with a Javadoc block

The weaknesses

Only 9% of the source code is covered by 338 unit tests
Average cyclomatic complexity by method (excluding getters and setters) is greater than 3 (3.2).
That is kind of a warning saying “your methods are taking too much responsibilities and should be re-factored”. This warning is confirmed by others metrics : 394 methods have a complexity greater than 7 and 86 methods have more than 50 statements. What is true at method level gets also partially confirmed at class level as 60 classes have a Fan Out Complexity greater than 20 (The number of other classes referenced by a class)

Bad programming practices that should be improved

198 times, method parameters are reassigned in the core of the method
68 times, local variables are defined and hide class fields
28 times, NullPointerException are thrown when an IllegalParameterException would be more suitable

Potential bugs that should be quickly analyzed

Correctness - An apparent infinite recursive loop : there is an apparent infinite recursive loop in org.apache.sling.scripting.jsp.jasper.runtime. JspContextWrapper.include(String, boolean)
Multithreaded correctness - Unsynchronized get method, synchronized set method : org.apache.sling.scripting.jsp.jasper.compiler. JspRuntimeContext.getJspReloadCount() is unsynchronized, org.apache.sling.scripting.jsp.jasper.compiler. JspRuntimeContext.setJspReloadCount(int) is synchronized
Multithreaded correctness - Method calls Thread.sleep() with a lock held : org.apache.sling.event.impl. JobEventHandler.runJobQueue(String, JobBlockingQueue) calls Thread.sleep() with a lock held
Malicious code vulnerability - Field is a mutable array : org.apache.sling.jcr.webdav.impl.servlets. SlingWebDavServlet.COLLECTION_TYPES_DEFAULT is a mutable array

This analysis was done with the intention of giving a synthetic overview of the current state of the project. Where should you start from if tomorrow you wake up with a single idea in mind : “Improving quality of the Apache Sling project !” ?

With respectively a cyclomatic complexity of 428, 385 and 343, classes Generator, Parser and XMLEndoginDetector should be first refactored. With no surprise, the Generator.java file has the greatest number of duplicated lines (154) and rules violations (109)
With its 43 cyclomatic complexity and no unit tests, the method ModifyAceServlet.handleOperation(..) is what we call “a crappy method” :-)

More information on the code quality of the project is available on Nemo.

[LOTD] Content Structure in a CMS

dev@day.com (Michael Marth) — 30 Jun 2009 00:00:01 GMT

Via Seth Gottlieb I have found this really good presentation on content modelling for content management systems by Deane Barker:

"Just Put That In The Zip Code Field..."

View more documents from gadgetopia.

Cleve Gibbon's recent well-written series on content modelling is closely related to Deane's presentation. Make sure to have a look if you are interested in that area.

I really like Deane's presentation, at least when I look at it from the paradigm that proper content management needs a-priori content modelling. Lately, I have come to question this idea, but that shall be the topic of a different post.

[ANN] Upcoming Cloud-Computing Events (July 7 and 9)

dev@day.com (Cedric Huesler) — 29 Jun 2009 00:00:01 GMT

The introduction of the simplified clustering in CRX 1.4.1 (back in January 2009) kick-started the efforts to make CRX and CQ5 easily deployable into the Amazon cloud computing infrastructure.

A lot happend since then - most importantly - we got in touch with customers that have exactly the challenges we were looking to solve with cloud-based deployments. One of them being the ability to scale infrastructure for peak usage, without the cost of running all servers all the time. Of course, to set a good example, we run our own sites - such as www.day.com - on EC2.

Next week, we would like to give you an update of what we did so far and share the plans for the future.

On July 7th (Tuesday) we have a half day seminar in London. Sarah Burnett from Butler Group and myself will discuss the advantages and use cases to make best use of cloud computing infrastructure. This is a great event to get yourself familiar with the cloud computing topic and learn how you can apply that to your content management initiatives. Join the free seminar in London. Sign-up here.

On July 9th (Thursday) I'm going to broadcast my speech and discussions from London via a Webinar using WebEx. Feel free to sign-up to get the details to join the Webinar.

Personally, I believe the most exciting part of the cloud computing era are the new ways to solve challenges accepting the fact that almost unlimited computing resources are at your disposal (at a fairly decent price).

Jazoon 09 Slides

dev@day.com (Michael Marth) — 29 Jun 2009 00:00:01 GMT

In case you missed the Day talks at Jazoon conference please find the slides below:

Thomas Mueller: Testing Zen

View more documents from day.

Thomas Mueller: Java Persistence Frameworks

View more documents from day.

Michael Marth and Michael Dürig: Scalable Agile Web Development: REST meets JCR meets OSGI

View more presentations from mmarth.

Michael Dürig and Michael Marth: Building RESTful Web Applications with Scala for Sling

View more presentations from day.

[ANN] Day engineers at the OSGi DevCon (updated)

dev@day.com (Michael Marth) — 22 Jun 2009 00:00:01 GMT

Starting today the conference OSGi DevCon Europe 2009 takes place in Zurich (in association with the Jazoon conference). Two talks will be given by Day's OSGi experts:

Felix Meschberger: Declarative Services: Dependency Injection OSGi style

2009-06-22, 11:20

Applications in general and OSGi applications in particular consist of a host of different modules and services which need to be bound together to form the actual application. In a traditional application services are generally bound by calling factories or instantiating the service classes or accessing a registry of services. In recent years a new buzz-word entered the arena: Dependency Injection. With dependency injection services are provided to the service clients as they become available. Likewise configuration is injected into the services, that is services do manage their configuration themselves. The OSGi specification for dependency injection is the Declarative Services specification: The components are declared and indicate what services they use and require and may in addition be provided with configuration. This talk shows the benefit of using Declarative Services and how the Apache Felix Maven SCR Plugin simplifies the service declaration even more.

Update: find Felix' slides below

Betrand Delacretaz: Tales from the OSGi trenches

2009-06-22, 14:60

In this talk we share our experience of using OSGi for a major rewrite of Day's family of content management products. After more than two years working with OSGi, the impact on our products, developers, customers and service people is very high, in a positive way. OSGi is no silver bullet either. The extreme modularization and dynamic service deployment features of OSGi make our products much more robust and maintainable, but the costs associated with changing people's way of thinking about code and modules, and with testing and debugging highly dynamic systems, must not be underestimated. Based on real-life code samples, we will show how OSGi is used at several levels in our products, from low-level interactions with the framework to very simple creation of (compiled or scripted) services. We will also present some of the automated testing techniques used in our project. Sharing our experience will help you decide if OSGi is for you, and more importantly at which level you should use it.

[ANN] Meet us at Jazoon 09

dev@day.com (Michael Marth) — 18 Jun 2009 00:00:01 GMT

This year's Jazoon conference is to take place next week in Zurich, Switzerland. Some of the presentations will given by speakers from Day. Hope to see you there:

Thomas Mueller: Testing Zen

Technical short talk, Tuesday, 2009-06-23, 16:30

Test driven development not only improves code quality, it also let's you refactor or replace legacy source code with little risk. Testing also saves time because the earlier a problem is found, the faster and easier it can be corrected.

Unfortunately writing tests is boring. There are many possible use cases that should be tested. Writing tests for every feature is a lot of work, and manually writing tests for all possible combinations is almost impossible.

However with fuzz testing (randomized testing), you don't need to write much, and still get very good results. This talk explains how a typical randomized test looks like, and how to combine it with other techniques such as unit tests, integration tests, and measuring code coverage.

Automated unit tests should run quickly. Many applications use a database, and in many cases this database is the bottleneck when running tests. This talk shows how an embedded in-memory Java database speeds up your tests, and can speed up development as well.

Thomas Mueller: Java Persistence Frameworks

Technical short talk, Wednesday, 2009-06-24, 16:00

In Java, there are many ways to access relational databases. Today, there are almost as many persistence frameworks as there are web frameworks.

Some frameworks are based on a standardized interface like JDO and JPA, some have their own API. Some just simplify using JDBC and SQL. Most frameworks need XML configuration, but not all of them. Some provide a string based query language similar to SQL, while some newer ones don't.

Todays main technologies are:

- SQL using and the JDBC API
- JPA (specially Hibernate, JPOX, OpenJPA)
- Apache iBATIS
- JDO (JPOX)
- Apache Cayenne
- Apache Commons DBUtils

We list the market share of each technology, and discuss the key differences.

Last year a whole new breed of persistence frameworks appeared: Frameworks with a fluent API and an integrated DSL (domain specific language); frameworks that don't need strings for dynamic queries. This new technology was strongly influenced by Microsoft's LINQ. Those frameworks support compile-time type-checking, auto-complete in the IDE, and protect against code injection. They better bridge the "object-relational impedance mismatch" than older frameworks, but they are not ready for prime time yet.

SQL injection is the biggest server-side security vulnerability today. SQL injection is a subset of code injection. Unfortunately, most persistence frameworks don't protect against code injection. The talk gives examples how to inject code and how to protect against it.

Michael Marth: Scalable Agile Web Development: REST meets JCR meets OSGI

Technical long talk, Thursday, 2009-06-25, 13:30

This session is a very hands-on lab that shows how a real web application is developed from scratch in a very agile fashion leveraging a heavy-weight enterprise ready back-end yet allowing for unprecedented agility in development in building rest-style web applications. Thinking of a classic j2ee stack this may sound like a contradiction.

Agility of development begins with the amount of tooling and setup we need to get started, so expect to see the entire walk-through from installation of the server software to the development of a complete application within the time constraints of the session.

Agenda:
(1) Web architecture, think outside the box.
(2) Meet: apache sling.
(3) Building a real-life webapp from scratch.

Michael Dürig and Michael Marth: Building RESTful Web Applications with Scala for Sling

Jazoon Cutting-Edge, Thursday, 2009-06-25, 16:00

In this session we demonstrate how to build RESTful web applications for the Sling framework using the Scala programming language.

Apache Sling is a web application framework which eases development of content centric applications. Sling is based on REST principles and uses a JCR content repository (JSR-170/JSR-283) for storage. Based on the JSR 223 specification (Scripting for the Java Platform) it integrates various scripting languages as OSGi bundles.

Scala is a scalable programming language for the JVM which is fully interoperable with Java. It is designed to express common programming patterns in a concise, elegant, and type safe way. Scala smoothly bridges the gap between object oriented and functional paradigms. Despite being strongly typed, Scala has the touch and feel of a genuine scripting language. It has the ability to infer types of expressions rather than relying on the programmer to explicitly declare them. Scala thus combines the best of the two worlds: flexible scripting and strong tool support e.g. documentation, safe refactoring and fail fast compilation. Its flexible syntax lets programmers easily define their own internal DSLs, effectively extending the language without leaving it.

In our session, we show how to take advantage of Scala to create RESTful web applications with Sling. We use its DSL capability and support for XML literals to create type safe web site templates. In contrast to conventional web site template mechanisms (e.g. JSP), we do not rely on a preprocessor but rather use pure Scala code.

Session outline:
- Short introduction to Apache Sling. Just enough to get everyone started.
- Short overview of Scala and its relevant features.
- Demonstration of how Scala can be used with Sling to create RESTful web applications.
- Q&A

Sling graduates from Incubator

dev@day.com (Michael Marth) — 18 Jun 2009 00:00:01 GMT

Excellent news: Apache Sling has just graduated from the Apache incubator and is now a top-level project. ASF Board member and Sling committer Bertrand Delacretaz announced the news on the Sling mailing list:

I'm pleased to announce that our Board of Directors, at yesterday's meeting, approved the graduation of Sling as a top-level project. I abstained from that vote (as working on Sling is part of my job), so it's not my fault ;-)

Felix Meschberger is the chair of the new PMC, composed of

* Alexandru Popescu <apopescu>
* Bertrand Delacretaz <bdelacretaz>
* Christophe Lombart <clombart>
* Carsten Ziegeler <cziegeler>
* Felix Meschberger <fmeschbe>
* Gianugo Rabellino <gianugo>
* Padraic Hannon <hannonpi>
* Juan José Vázquez Delgado <juanjo>
* Karl Pauls <pauls>
* Vidar Ramdal <vramdal>

Congratulations! The graduation is well deserved.

[LOTD] Spring JCR Extension 0.9

dev@day.com (Michael Marth) — 17 Jun 2009 00:00:01 GMT

Salvatore Incandela, committer of the Spring extension for JCR, left a comment on a previous post on dev.day.com where he announces the release of version 0.9. Since many of Day's customers and partners use Spring (and JCR), I am happy to know that the extension is actively maintained.

(thanks for the pointer, Salvatore)

Note on nodes and files

dev@day.com (Michael Marth) — 08 Jun 2009 00:00:01 GMT

Reading "Content Integration Standards -- CMIS, JSR-170, JSR-283" I stumbled over the quote

The conclusion was that JSR-170 is not the best fit for WCM with its nodes and properties equaling to folders and files. In Devnani’s opinion, it fits document management better.

Apparently, there is some confusion around what nodes and properties are in a Java Content Repository. Let's shed some light on this:

First of all, there are two specific node types defined in JSR-170: files and folders. These are node types that behave as you would expect it from a file system. For example, the node type "file" has specific properties for the creation date and the binary stream.

If your content is nothing but files and folders (for example because your application is document management) you would use these node types. As such, in Java Content Repositories files, folders and document management-type applications are special cases of a more generic case.

This more generic case are node types that have different properties than the ones described above. In the most flexible case "unstructured node type" the node can accomodate for any string, date, binary, boolean, etc property. One node could for example have two binary stream properties (thumbnail and full-length). This covers exactly the use case web content management where content is stored on a fine-granular model. The fine-granular model enables the presentation layer to do things like displaying an article in teaser format.

In summary the situation is like this:

Files and folders are special node types in JCR.
On a modelling level DM is a special case of WCM (I recognize that on a application requirements level DM and WCM diverge).

As a consequence I would rewrite the quote from above as:

JSR-170 is a very good fit for WCM because the nodes and properties store content in a fine-granular way. Since folders and files are just special node types it can be equally well used for document management.

The Jackrabbit Wiki has a list of JCR applications that are either in the content management or the document management domain. As a side-note: JCR is flexible enough to provide infrastructure for completely different application domains as well, here is a list of some of them.

Slick JCR Explorer based on JQuery and Sling

dev@day.com (Michael Marth) — 05 Jun 2009 00:00:01 GMT

Renaud Richardet (who won a price in last year's JCR cup) has posted a very slick repository explorer to the Sling Jira. It is based on JQuery and has the usual explorer features like CRUD for nodes and properties. However, what is really neat: the actual code is about 100 lines of Javascript (including blank lines).

In case you want to check it: copy the files into /apps/sling and make sure that the mime types of the html files are correct (e.g. by fixing the property using CRX Explorer :). Then point your browser to http://localhost:7402/apps/sling/servlet/default/explorer.esp (the port depends on your installation, of course).