Predicting Defects in SAP Java Code: An Experience Report (with Tilman Holschuh, Markus Päuser, Kim Herzig, Rahul Premraj, Andreas Zeller).
In a study on a large SAP Java system, we evaluated and compared a number of defect predictors, based on code features such as complexity metrics, static error detectors, change frequency, or component imports.
Software Engineering in Practice Track.

Tilman’s presentation

Improving Bug Tracking Systems (with Rahul Premraj, Jonathan Sillito, and Silvia Breu). We present a prototype of an interactive bug tracking system that gathers relevant information from users and automatically identifies files that need to be fixed to resolve a bug.
New Ideas and Emerging Results Track.

Silvia’s poster
Rahul’s presentation

Also visit the official ICSE 2009 Post-Conference webpage, which has many other exciting talks, posters, and videos.

This week I will attend the USENIX 2009 Annual Technical Conference, where Stephan Neuhaus will present one of our papers.

The Beauty and the Beast: Vulnerabilities in Red Hat’s Packages.
In an empirical study of 3241 Red Hat packages, we show that software vulnerabilities correlate with dependencies between packages. With formal concept analysis and statistical hypothesis testing, we identify dependencies that decrease the risk of vulnerabilities (beauties) or increase the risk (beasts).
Session “Bugs and Software Updates”. Friday 2-3:30pm.

DEFECTS 2009 – Second International Workshop on Defects in Large Software Systems, Chicago, July 19, 2009.

Bugs are everywhere in today’s software and because of the huge economic damage they are actively studied by research. The goal of this one-day workshop is to connect the different research communities with each other and with industry. The workshop will provide a forum for researchers as well as practitioners to discuss issues related to all aspects of bugs. Researchers can present their tools and techniques and make them accessible to industry. Participants from industry can share their experiences and help identifying new and promising research directions.

Register now on the ISSTA website.
Early registration ends soon (June 19).

Prediction of software defects works well within projects as long as there is a sufficient amount of data available to train any models. However, this is rarely the case for new software projects and for many companies. So far, only a few have studies focused on transferring prediction models from one project to another. In this paper, we study cross-project defect prediction models on a large scale. For 12 real-world applications, we ran 622 cross-project predictions. Our results indicate that cross-project prediction is a serious challenge, i.e., simply using models from projects in the same domain or with the same process does not lead to accurate predictions. To help software engineers choose models wisely, we identified factors that do influence the success of cross-project predictions. We also derived decision trees that can provide early estimates for precision, recall, and accuracy before a prediction is attempted.

[click for more details...]

A bug report is typically assigned to a single developer who is then responsible for fixing the bug. In Mozilla and Eclipse, between 37%-44% of bug reports are “tossed” (reassigned) to other developers, for example because the bug has been assigned by accident or another developer with additional expertise is needed. In any case, tossing increases the time-to-correction for a bug.

In this paper, we introduce a graph model based on Markov chains, which captures bug tossing history. This model has several desirable qualities. First, it reveals developer networks which can be used to discover team structures and to find suitable experts for a new task. Second, it helps to better assign developers to bug reports. In our experiments with 445,000 bug reports, our model reduced tossing events, by up to 72%. In addition, the model increased the prediction accuracy by up to 23 percentage points compared to traditional bug triaging approaches.

[click for more details...]

Global and distributed software development increases the need to find and connect developers with relevant expertise. Existing recommendation systems typically model expertise based on file changes (implementation expertise). While these approaches have shown success, they require a substantial recorded history of development for a project. Previously, we have proposed the concept of usage expertise, i.e., expertise manifested through the act of calling (using) a method. In this paper, we assess the viability of this concept by evaluating expert recommendations for the ASPECTJ and ECLIPSE projects. We find that both usage and implementation expertise have comparable levels of accuracy, which suggests that usage expertise may be used as a substitute measure. We also find a notable overlap of method calls across both projects, which suggests that usage expertise can be leveraged to recommend experts from different projects and thus for projects with little or no history.

[click for more details...]

Here’s a list of papers which I have co-authored that will be presented at ICSE 2009 in Vancouver.

Do Crosscutting Concerns Cause Defects? (with Marc Eaddy, Kaitlin Sherwood, Vibhav Garg, Gail Murphy, Nachiappan Nagappan, Alfred Aho). We asked the question, “How much does the amount that a concern is crosscutting affect the number of defects in a program?” and conducted three extensive case studies to help answer this question.
TSE/TOSEM Session. Thursday 2-3:30.

Predicting Defects in SAP Java Code: An Experience Report (with Tilman Holschuh, Markus Päuser, Kim Herzig, Rahul Premraj, Andreas Zeller). In a study on a large SAP Java system, we evaluated and compared a number of defect predictors, based on code features such as complexity metrics, static error detectors, change frequency, or component imports.
Software Engineering in Practice Track. Thursday 2-3:30.

Improving Bug Tracking Systems (with Rahul Premraj, Jonathan Sillito, Silvia Breu). We present a prototype of an interactive bug tracking system that gathers relevant information from users and automatically identifies files that need to be fixed to resolve a bug.
New Ideas and Emerging Results Track. Presentation Wednesday 11-12:30. Poster during Wednesday Reception.

I will also be at MSR 2009 and PROMISE 2009.

Here is a preview of recently submitted papers (created with Wordle.net).

Can’t wait? Go read some technical reports like The Beauty and the Beast or Frequently Asked Questions in Bug Reports.

We show in an empirical study of 3241 Red Hat packages that software vulnerabilities correlate with dependencies between packages. With formal concept analysis and statistical hypothesis testing, we identify dependencies that decrease the risk of vulnerabilities (beauties) or increase the risk (beasts). Using support vector machines on dependency data, our prediction models successfully and consistently catch about two thirds of vulnerable packages (median recall of 0.65). When our models predict a package as vulnerable, it is correct more than eight out of ten times (median precision of 0.83). Our findings help developers to choose new dependencies wisely and make them aware of risky dependencies.

[click for more details...]

It is important that information provided in bug reports is relevant and complete in order to help resolve bugs quickly. However, often such information trickles to developers after several iterations of communication between developers and reporters. Poorly designed bug tracking systems are partly to blame for this exchange of information being stretched over time. Our paper addresses the concerns of bug tracking systems by proposing four broad directions for enhancements. As a proof-of-concept, we also demonstrate a prototype interactive bug tracking system that gathers relevant information from the user and identifies files that need to be fixed to resolve the bug.

[click for more details...]