Let’s start with a short version of this retrospective. Last year was made of:

• 8 releases of Sonar
• 110+ releases in the ecosystem
• 55,000 downloads of Sonar
• 10,000+ messages on mailing lists

So I suppose we can call this a pretty active year for the community. Now, the longer version:

The Plan

One year ago, we had the following ambitions :

Track changes : The next step is to provide the ability to report on code coverage of new source code. This is to ensure that whatever legacy code is there, teams have the ability to monitor the coverage by unit tests on added code if they wish.

Since Sonar 2.7 and with help of the SCM Activity plugin, this magic feature is available.

Code Review : This is really the next strategic move for the Sonar platform : add a manual dimension to the automated one to provide a complete code review tool.

This was a major change to accomodate into the platform and we therefore decided to adopt a baby step approach. The full functionality was delivered over 5 releases, from 2.8 to 2.12. Here is what the platform now covers:

• Review a violation
• Comment, assign, plan a review
• Flag false-positive violations through the UI
• Create manual violations through the UI
• Change the severity of a violation through the UI

Next step will be to provide the ability to customize the review workflow and its related permissions.

Language consolidation : Provide a Squid-like engine to the C# plugin to make it more robust

It took us 6 month with the great help of Alexandre Victoor to rewrite the C# plugin that embeds its own C# parser and natively supports visual studio projects.

Language consolidation : Improve the C parser to gain robustness, support non ANSI-85 extensions and increase significantly the number of rules available

We haven’t worked as much as we wanted on this C plugin and so the number of rules remains limited. That’s why we’ve already started working to implement the MISRA-C standard.

Language consolidation : Improve the PL/SQL plugin to provide currently missing metrics

A PL/SQL parser has been written to provide all those metrics and to start implementing some new rules outside the Toad CodeXpert tool.

Sonar Eclipse : Now that we have a stable version 1.0 of the plugin, we can start building on it. The objective for this year is to provide capability for running local analysis inside Eclipse

This local mode is now available but in fact the greatest new feature of Sonar Eclipse is certainly the integration of Mylyn to manage reviews directly from the IDE.

Support New Bootstrappers : We have started last year some background work to decouple Sonar from Maven. This work will enable us to support 2 new mechanisms for bootstrapping analysis in Sonar 2.6 : an ANT task and a Java runner. Next step is to also provide Gradle bootstrapper.

DONE, DONE, DONE, Sonar is now fully decoupled from Maven but if you want to use the power of Maven along with Sonar, it is of course still possible.

JaCoCo Integration : We intend to make 2 major integrations of JaCoCo into Sonar this year. The first one is to integrate it into Sonar core. The second one is to provide ANT integration of JaCoCo and therefore provide a simple way of gathering code coverage when you execute unit tests from ANT.

DONE & DONE.

Beyond the Plan

Obviously, we did not plan for all upcoming innovations for the year. Here are three major features of Sonar that weren’t planned and that have been implemented:

• Sonar CPD : this new technology introduced in Sonar 2.10 will fully replace PMD CPD in Sonar 2.14 and allows to track cross-projects duplications
• i18n : Since Sonar 2.10 the Sonar UI can be in spanish, french, greek…
• Email notifications : Since Sonar 2.10 a user can subscribe to some events to be notified by email. For instance when a review is assigned to him.

And this is also true in the ecosystem, here are two examples:

• a great effort made on the PHP plugin to resurrect it
• development of an extension for SAP ABAP

So after all this, what could be an exciting challenge for 2012 ? This is going to be the subject of my next post !

Sonar and Gradle Multi-Module Projects
By Gunnar Hillert, 13 January 2012
I love Sonar. It is a wonderful way to collect some metrics for your Java projects – hassle-free and wrapped in a sweet-looking UI. For Maven-based projects Sonar literally works out of the box. Just start up your Sonar instance (assuming you are using the default settings running on localhost) and then you simply fire it off using:

Measuring Code
By David Green, 2 January 2012
How good is your code? If you’re like the other 80% of above average developers, then I bet your code is pretty awesome. But are you sure? How can you tell? Or perhaps you’re working on a legacy code base – just how bad is the code? And is it getting better? Code metrics provide a way of measuring your code – some people love ‘em, but some hate ‘em.

Making a Business Case For Using Codehaus Sonar
By Chris Khoo, 6 January 2012
Augmenting the development process is often a hard sell – processes don’t contribute directly to company profits and there are financial and time costs (for integration and maintenance) to consider. Personally, I’m a firm believer in adopting processes on an as-needed basis rather than trying to build the ultimate workflow/process all at once.

Couverture des tests d’intégration avec JaCoCo, Maven et Sonar
By Jean-Christophe, 23 January 2012
Dans nos projets on rencontre souvent ce cas de figure : certaines portions de code se prêtent mal au test unitaire. Ce sont les interfaces graphiques, le code qui manipule des fichiers, les connexions réseaux… Cela peut poser problème lorsqu’on …

Continuous Integration and Testing
By John Dobie, 16 January 2012
How do you know if using code metrics really does help to produce code with fewer bugs.
I am convinced they do, but how can I possibly prove it? All projects have historic data. This is usually stored in your bug tracking and source code control tools. We can use the data stored in these systems to perform ‘code forensics.’ We use the historic data from real issues to see if they could have been avoided.

Extended search engine

The search engine will now return not only projects but also modules, package and files. A picture is worth a thousand words :

Add a review on any piece of code

Whenever a quality defect is detected “manually”, the person who detected it has the ability to inject a violation directly into Sonar:

The related violation is then displayed within the source code and will be accounted for in metrics after the next analysis of the project:

Action plans

Action plans can be created to group reviews together.

An action plan can be associated to each violation

And then it is possible to review progress in a widget of a dashboard

Hotpots 2.0

The previous release allowed to use hotspot widgets in its own dashboards (see Sonar 2.12 in screenshots). It’s now possible to customize, rename or even delete the default dashboard named “Hotspots”.

Time now to give a try to the new version and to read the installation or upgrade guides.

Sonar and JaCoCo
By Raghu, 1 December 2011
I am a fan (and regular user) of Sonar, a platform to manage code quality. The 2.12 release of Sonar, which happened yesterday, introduced a bunch of new features including Java 7 support and the availability of Jacoco into Sonar core.

Industrialiser vos projets Flex avec Hudson, Maven, Sonar, FlexPMD et FlexCPD
By Fabien Nicollet, 5 December 2011
Dans le monde Java, il existe de nombreux outils permettant de vérifier l’intégrité de votre code (tests unitaires) mais aussi pour vous assurer que la qualité de votre code est respectée. Ces outils vous permettent d’assurer la robustesse des applications que vous créez.

My Testing and Code Analysis Toolbox
By Jens Schauder, 11 December 2011
Last week we kicked of a “Testing Skill Group” at LINEAS, a group for exchanging knowledge about testing. One question that came up over and over again in various flavors was: What tools are there for testing and analyzing your code? So here is my personal answer for this, in the approximately order I tend to introduce them into projects…

Open Source & code Legacy
By Jean-Pierre FAYOLLE, 18 December 2011
There are more and more solutions of analysis of code which allow to measure the quality of your applications. Most are sold by software vendors, and we had the opportunity to verify that these solutions are expensive to buy, to implement and to use (Disposable software). In response, the last decade has seen the rise of the Open Source alternative to proprietary software.

Flex + ( Ant | Maven ) + Sonar
By Jozef Chutka, 5 December 2011
The title may sound like there are two possible ways how you can have your source code analyzed and published to sonar, but you better do not rejoice prematurely. After spending couple of hours trying to figure out how to make it work using ant I may have hit some nice articles, however sonar-ant-task seems to have major issues with sonar version 2.8. The solution is maven!

According to their authors :

The ThoughtWorks Technology Advisory Board is a group of senior technology leaders within ThoughtWorks. They produce the ThoughtWorks Technology Radar to help decision makers understand emerging technologies and trends that affect the market today. This group meets regularly to discuss the global technology strategy for ThoughtWorks and the technology trends that significantly impact our industry.

In its last publication (July 2011), Sonar platform made its first appearance in the “Assess” circle : “Worth exploring with the goal of understanding how it will affect your enterprise”

Of course, SonarSource team was very proud of this but this is not the point here. Indeed, Sonar is a platform to manage quality among other platforms that have been around for a while : CAST Software, McCabe, Klocwork… So why adding now a QA tool to the radar and why choosing Sonar ? Is this because of its growing open source community : 5,000 downloads and 1,000 emails by month ? For its multi-technology capability in Java, C#, COBOL, PHP, PL/SQL, ABAP… ? Is it for is governance extensions: SQALE or Portfolio Management? Maybe, but I am pretty sure that an additional reason has led Thoughtworks experts to make this choice.

From inception, Sonar was not developed as “just yet an other quality reporting tool” but as a mean to continuously manage and fight back technical debt. This might sound like a subtle semantic difference but this actually makes a big difference. SonarSource team has grown with Agile methodologies and with those methodologies, source code is always very much in the center of focus: it should be able to mutate constantly over time to embrace changes. This key capability to do refactoring at any point in time is so important that the Technical Debt metaphor was early introduced by Agile practitioners.

After 4 years of development and 38 releases of Sonar we, at SonarSource, deeply know what “Change” means! In such evolving context, processes, planing, documentation, specifications, … they all matter but what’s about source code ? Some would like to consider source code as a black box without too much value whose development can be easily outsourced. With such an approach, the goal of a quality platform is just to report from time to time how well a blak box comply to some pre-defined quality standards. We do think this is a mistake !

Source code should be considered as a white box that each stakeholder (developer, project manager, IT director, quality consultant, customer, …) can look at any point of time to understand what happens, to initiate discussions and to make decisions. What ever is the quality of your processes, documentation or specifications, bad code will always lead to failure. Therefore when bad code is injected, it should be immediately detected, fought back and analysed to understand why this crappy code has been injected. Waiting several months to detect technical debt is a huge waste as per Lean principles.

Adopting Sonar means much more than simply installing a tool to comply to some QA or security standards … it means that quality of source code really matters and that the ability to daily manage your Technical Debt is key to sustain a continuous delivery approach and to embrace business changes: Continuous Inspection has entered the game !

Hotspots 2.0

3 new widgets have been created that enable to embed hot spots into a project dashboard: most violated rules, most violated resources and hot spots on a specific metric. We continue the “widgetisation” effort of of all Sonar UI components in order to sooner or later provide full customization capability of the Sonar UI.

Support of Java 7 Syntax

This was a great opportunity to collaborate with both Checkstyle and PMD teams in order to help them support all java 7 language changes : Strings in switch statements, try-with-resources statements, improved type inference for generic instance creation (“diamond”), simplified varargs method invocation, better integral literals, and improved exception handling (multi-catch).

Integration of JaCoCo in the Sonar core

JaCoCo is now part of Sonar core, along with Cobertura. This means that not only can the coverage by unit test be calculated by JaCoCo natively, but also that coverage by integration tests is now much better integrated. Results are displayed in the same page than coverage by unit tests :

Note that Cobertura is still the default engine … but that’s just a matter of time to make Jacoco the default java code coverage engine. Just need more feedback from the community to make this switch.

Improved Display of Duplication Blocks

The ‘Duplications’ tab in the code viewer has been fully refactored to make it far more usable :

• Duplication blocks are visually grouped in case of “truplications” or more
• A code snippet is displayed by default on each group of duplicated blocks
• Duplicated blocks on external files can be seen without leaving the page

Don’t forget that since Sonar 2.11, the default PMD CPD engine has been replaced by the Sonar CPD engine which brings a unique feature : ability to track duplications across projects.

Project Events Management

The events management has been greatly improved to make it easier to use from the History page introduced in Sonar 2.11.

Time now to let you give a try to the version 2.12. Release notes are available in the download page. Reading the installation or upgrade guides is much recommended.

Code Quality Analysis in Deployment Pipeline with Gradle, Jenkins and Sonar
By edvorkin, 31 October 2011
Sonar is a tool that integrate a range of quality analysis tools into a single website. It provide one page visibility into quality of project source code. Developers and managers are interested in test coverage, code duplication, adherence to coding standard, cyclomatic complexity of the code and several other parameters…

Intégrer l’outil de supervision Sonar à Team Foundation Server 2010
By Maxime ARNSTAMM & Simon Lehericey, 13 November 2011
Comme on l’a montré dans l’article précédent, Sonar est l’outil indispensable pour évaluer la qualité des projets d’une DSI au fil de l’eau.
Dans cet article nous aimerions présenter l’installation de différents composants de l’analyse Sonar et comment l’intégrer à votre usine de build TFS…

Lunch-N-Learn Ideas? Use Sonar Hotspots
By Clint Shank, 23 November 2011
Are you eager to get your team together for a lunch-n-learn with free pizza (and beer), but can’t come up with any ideas on which to present?

Or maybe you just want to get heads down and make some high impact, quality improvements to your code base, but don’t know where to start.

Setting up Sonar analysis for C# projects
By John M. Wright, 25 November 2011
In an effort to better understand some of the problematic areas of the C# codebase I work on, I recently setup an instance of the Sonar code analysis platform. Sonar is originally written for Java analysis and later added C# support. This posting walks you through my experience attempting to setup, configure and run the analysis.

Measure Code Coverage of HtmlUnit Based Tests with Sonar and JaCoCo
By deors, 20 October 2011
This blog post is the third one in a series about Integration Tests with HtmlUnit. Finally, in this post we are going to show how to measure code coverage of HtmlUnit tests using Sonar, the popular Continuous Quality Assurance tool, and JaCoCo, a very interesting code coverage tool based on JVM agents instead of instrumenting bytecodes.

Separating Code Coverage With Maven, Sonar and Jacoco
By John Dobie, 23 October 2011
In this example I will expand upon my previous example of keeping your unit and integration tests in separate packages, and explain how to also produce code coverage statistics. For this I continue to use Maven, whilst adding Sonar and Jacoco to provide the code coverage.You can run the example below, but you will need to read the first article here to understand it.

Testing the new Sonar plugin for Gradle
By Luciano, 28 October 2011
If you were looking to convince your boss that Gradle is worth a try for your next project, look no further. Gradle 1 release candidate 5, released on October 25, brings the long awaited Sonar integration, and it works ridiculously well. How well? How about a one-liner.

A Free EC2 Cloud Server With Ubuntu, Jenkins And Sonar
By John Dobie, 29 October 2011
This example shows you how to create a free Amazon EC2 cloud based continuous integration and testing environment on Ubuntu. This is a low power server but it is useful for infrequent use. I personally tend to recommend Cloudbees, but this is handy when you need a free Sonar instance.

Code Quality Analysis in Deployment Pipeline with Gradle, Jenkins and Sonar
By Eugene Dvorkin, 31 October 2011
Sonar is a tool that integrate a range of quality analysis tools into a single website. It provide one page visibility into quality of project source code. Developers and managers are interested in test coverage, code duplication, adherence to coding standard, cyclomatic complexity of the code and several other parameters. Sonar is an open source product and can keep all your code metrics in database, as a matter of fact, in any relational database.

This quality gate is good to have but not efficient enough because defects introduced during a sprint have to be fixed all at the end. Instead, they should be fixed as they appear for better efficiency, similarly to code fix when a unit test breaks in continuous integration: this is what we call continuous inspection. We have done a lot of work this year to be able to provide better support for Continuous Inspection in Sonar and have added several services : differential views, SCM information and manual reviews integrated with email notification and with Sonar Eclipse. Manual reviews is really the new hot feature to complements existing services and making code reviews more effective.

How does this all fit together ? Well, this is the subject of this post… Get your Sonar 2.11 started, open Eclipse along with Sonar Eclipse 2.1, and follow the guide!

Develop, test, commit… and sleep well!

Managing code quality is like handling non-regression: while developing, one should not worry about this – a process should do it and notify you in case of an issue. You know already that you can refactor your code serenely because a continuous integration server will check that you did not introduce a regression, don’t you? Same applies when you improve a feature, the integration tests will make sure that you did not break anything, right? Similarly, you can feel comfortable when you think about quality of your code, Sonar will take care of it for you.

If you wish, you can also use Sonar Eclipse during your development to run local analyses and get realtime feedback. This is not yet optimum since you can only run full analysis and we are working hard on supporting incremental analysis.

Morning: code review time

After a good sleep and a cup of coffee, the first thing you want to know is how well you coded the previous day : log into Sonar and activate the “since previous analysis” differential views on your project: in a second, you see if new defects have been introduced. Those may identify – for instance – potential bugs, too complex classes or insufficiently tested methods. But whatever those violations are, you know that they increase the technical debt of your application. Fixing a violation is like fixing a bug: the sooner, the cheaper – as the context of the violation is fresh in your mind.

To track the newly introduced violations, use the differential violations drilldown. For every newly introduced violation – there shouldn’t be too many as you become more and more familiar with quality rules, create a review and assign it (or – when appropriate, flag it as false positive). If your source configuration management tool is supported by Sonar, finding the developer who introduced the violation is even simpler as his identifier appears next to the violation (as long as you installed Sonar SCM Activity plugin).

Though this process should only take a couple of minutes and will maximize the efficiency for reducing the technical debt, the ultimate objective is to provide a mechanism to notify the person who introduces a new violation.

Before developing again, clean your code

Once you’ve created all the reviews for the newly introduced violations, you can get back to your favorite IDE. But before starting coding, maybe you’d like to fix defects that you introduced the day before, wouldn’t you?

If you’re using Eclipse, you are lucky: Sonar Eclipse provides a very efficient way to work with reviews. Thanks to its Mylyn connector, Sonar Eclipse brings all the reviews assigned to you right inside your task view in Eclipse. There too, in a second, you see all the reviews that you have to work on. Open a review, click on a link to open the corresponding file, fix the defect and resolve the review to “fixed” so that it doesn’t show up in your task list any longer: this is that simple to fix a violation! And if it turns out that the fix is not obvious, you can start a thread of discussion on that review by adding a comment.

If you are not using Eclipse, you can still get notified when reviews are assigned to you. Just log into Sonar web application with your account and go to “My Profile” page to activate the email notification for reviews. This way, you won’t miss a single review assigned to you! Actually, you should probably activate email notification in both cases: indeed, if you created a review and assigned it to someone else, you may want to know if the review has been solved, or if the developer added comments on it.

And what about reviews that have been fixed?

Sonar handles code quality for you, but it also makes sure that fixed reviews have correctly been handled. During the next analysis, for each fixed review, if its corresponding violation has actually disappeared, Sonar will set the review to “closed”. If not, Sonar will reopen the review: in the morning, you will then see it again in your task list (or receive a mail) with the “reopened” status.

If you want to monitor more reviews – not only yours, you can use the Sonar review service that allows you to query reviews against their author, assignee, status, resolution, corresponding project or id.

That is it! This is how we are using differential views and manual reviews to run an effective continuous improvement process. Of course, you can adapt it – or even have a different one, to meet your needs. But keep in mind that the most important is to be sure that technical debt is under control!

More features are coming tu support Continuous Inspection further: create reviews on any code, filtering newly created violations by developer… Stay tuned!

Cross Project Duplications

So far the detection of copy paste code was only done inside a project and for Maven projects inside a Maven module. The main reason for this was the difficulty for PMD-CPD to scale as it requires a lot of memory. Thanks to GSoC, we now have a Sonar-CPD engine that is much more scalable and can detect cross-projects duplications. Do not be surprised to see duplications found increased after the next analysis !

New History Widgets

The TimeMachine was not changed since Sonar 2.5 and it was probably the service that has not been improved for the longest time… This is now fixed as we provide a version 2.0 that is widget based to be fully integrated into the dashboards. Note that you can mouse over the timeline to display the different values.

Deletion of Past Snapshots

Whatever the reason (wrong quality profile, issue with analysis…), there was a recurring demand from the community to be able to take a snapshot of the history. This is now possible and you will be able to polish history charts :

Improve General Settings

General settings have been very much improved to be easier to use. For example there is a save button in each tab now :

Highlight the Link to Delete Projects

This minor improvement has been frequently asked on mailing-lists. At last, it is done :

Time now to let you give a try to the version 2.11. Release notes are available in the download page. Reading the installation or upgrade guides is much recommended.