Code Central has been a huge help in distributing coverage numbers throughout the enterprise; a welcome change to shipping around xml data and reports for most customers.  The one issue that has hindered Code Central out of the box is the complication of requiring an installation of Collector on any Code Central server which is also a coverage box.  The common scenario here is when Code Central is installed on a CI server and build and testing is all in one place.  Most customers in this scenario will need to both collect coverage and serve data analysis from the same machine.

Version 4.1 is targeted at simplifying this issue.  The change is welcome, but not entirely without its potential gotchas.  Previously Code Central by default installed an executable named “ncover.codecentral.exe” and its  service name was “ncovercodecentral”.  Additionally the default port for http browsing to the server was 11233.  By enhancing Code Central to include a profiler we have unified the exe’s for all products in the NCover suite to a single executable “ncover.exe”.  The service name now also being “ncover” for all 3 platforms (Desktop, Collector, and Code Central) as well, and the default port for all 3 platforms is 11235.

With all of this symmetry something had to change.  This change requires that each machine host only one variety of the NCover service host.  The primary impact of this change is that servers currently having Code Central and Collector installed together will need to first uninstall Collector before Code Central can be upgraded to version 4.1.

What are the benefits of such a change?

1) It’s now possible to easily set up a Code Central installation on your CI server without the complication of creating user accounts and setting up a connection for a separate Collector agent to locally collect coverage.

2) Sharing the ncover.exe command line allows all three products to have interchangeable command lines.  Upgrading a server from Collector to Desktop to Code Central requires no change in build scripting.  The commands are now identical.

3) Identical service names also make it possible to script server behavior and reuse that scripting regardless of the installed variant of NCover.    The following commands now work identically on Collector, Desktop, and Code Central.

• net stop ncover
• net start ncover

While this type of change can be disruptive because of the manual intervention for the changeover, we know that ultimately this simplifies implementation for all customers.  Reducing the steps for startup and increasing the compatibility of commands across multiple platforms makes all of our lives just a little simpler.

If you have any questions about the latest release of NCover or need further assistance in upgrading your servers, just let us know.

Contact NCover support:   http://ncover.com/support/home

Or email us:  support@ncover.com

Increasingly, customers are using NCover to gather code coverage in production scenarios (or mock production scenarios). In fact, many of the improvements we’ve made in NCover 4 have been focused on gathering coverage with as little performance impact as possible and requiring little to no configuration settings when gathering coverage.

Consider this test scenario… A development team wants to collect coverage data on their code while it’s being run through its paces by an automated test suite. They have a continuous integration process which builds an installation package on their build server. The build server contains all of the code, debugging symbols and other artifacts related to each version of the installer package which is generated.

On the other side is the staging environment, which is built to reflect what their customer’s environment will look like when the application is deployed. When a build completes, they run the newly generated installer on the staging server. This server is configured just like a customer’s installation would be configured. There is no source code, symbol files or other build artifacts on this server other than the installed application.

In this staging environment the team uses manual testing along with automated scripts to test the application. That’s where code coverage comes in. How can we collect coverage data on the application running in the staging environment without access to the debug symbols, or source code? This is where NCover 4 introduced a new innovation that makes it possible…easy even…to collect coverage in this scenario.

NCover 4 splits the coverage process into two separate phases…

• The static analysis phase runs on the build server and requires source and symbols (to be able to report detailed results. It can run without source and symbols to produce more limited metrics).
• The runtime phase runs on the machine under test and does not require any symbols or source code. The two sets of data are later combined together producing the same detailed coverage analysis that would have been otherwise produced.

This has the following benefits:

• The runtime collection is faster and more lightweight. In fact, due to this change, NCover uses several orders of magnitude less memory in many cases than NCover 3.
• The static analysis can be done as a one-off process…even as part of your build script. And it does not need to be done on the same computer as the one on which the tests are running.
• The static analysis only needs to be run once per unique build. Many runtime collections can be run after that using the same static data.

How do we accomplish this? NCover has a new command called “Analyze” that runs the static analysis process on your assemblies and pushes the data to NCover (and up to Code Central, if you are using it). Then later, when coverage is collected at program runtime, the data is combined on the server side to produce detailed coverage results. Let’s walk through an example to see how this works in practice. I’m going to use our sample CodeBlaster game as an example.

On your build server – The Static Phase

I’ve just finished running the build and it has completed successfully. The output files we are interested in are CodeBlaster.exe, Algorithms.dll, and Box2DX.dll.

From the build server, I’m going to run analyze on each file from the command line to register these assemblies with NCover.

C:\BuildOutput\Debug>NCover analyze64 --force --sync CodeBlaster.exe
NCover.Analyze v4.1.2042.572
Copyright (c) 2003 - 2012 NCover LLC

Analyzing assembly [CodeBlaster.exe]...
Original Assembly: CodeBlaster.exe
Temporary Assembly: ...\Temp\nc4F3B6.tmp\CodeBlaster.exe
Hash: C48801AD2396D7921AD720EBA1F587BF7D976A8E

Completed in 5.50874 seconds.

C:\BuildOutput\Debug>NCover analyze64 --force --sync Algorithms.dll
NCover.Analyze v4.1.2042.572
Copyright (c) 2003 - 2012 NCover LLC

Analyzing assembly [Algorithms.dll]...
Original Assembly: Algorithms.dll
Temporary Assembly: ...\Temp\nc457C0.tmp\Algorithms.dll
Hash: 9A7E7EB58D647237C95AA787E751503DE06F00F0

Completed in 5.6304 seconds.

C:\BuildOutput\Debug>NCover analyze64 --force --sync Box2DX.dll
NCover.Analyze v4.1.2042.572
Copyright (c) 2003 - 2012 NCover LLC

Analyzing assembly [Box2DX.dll]...
Original Assembly: Box2DX.dll
Temporary Assembly: ...\Temp\nc4969E.tmp\Box2DX.dll
Hash: EAAE1D2179BC4090E8F95D9C61B2279D6359C6D4

Completed in 14.8811 seconds.

The --sync option tells the local copy of NCover to push the static analysis data to the Code Central server. If I hadn’t specified that option, then the data would remain only on the local system.

The --force option tells the system to go ahead and run the analysis and sync the data even if it already has data for this assembly in the system.

On the Code Central server – Setup the project

Now, we’ll switch over to the server that’s running Code Central. Let’s go ahead and create a project for the CodeBlaster program on the Code Central server. We’ll use a simple RegEx match rule to specify that we want to collect coverage on the CodeBlaster program.

As soon as I save it, the newly created project will sync to the copy of NCover running on the staging environment. So let’s switch to the staging environment and run the program.

On the staging environment – Collecting coverage at Runtime

On the staging environment, normally a test script would install the program that is to be tested. For this simple example, I simply copied the .dlls over to the computer. Note that there are no pdbs or source code being installed, only the program itself. Let’s go ahead and run CodeBlaster. If you check the local NCover window, you should see the local copy of NCover capture coverage and then sync it up to the Code Central server when the program shuts down.

Back to the Code Central server – Viewing the results

Now we go back to the Code Central server to see the results. Here you can see that we get sequence point coverage (which requires debug symbols) for Code Blaster even though it was run on a server without access to debug symbols or source code.

Similarly, we are able to see the source code with coverage highlighting as well even though coverage was collected without access to the source code.

You can see that we are able to drill down to sequence point coverage details and highlighted source code on the Code Central server even though coverage was collected on a computer where none of the build artifacts existed.

Conclusion – So what does this mean?

Normally NCover combines the static and runtime analysis in the background in order to make collecting coverage data as quick and efficient as it can be. However, as we’ve seen in this article, these two processes can be split across machines in order to get better data without having to spread your source code and debug symbols to every testing machine in your development shop.

This capability should make it far easier for you to collect coverage in production or simulated production scenarios where you only have access to the shipping code. Please let us know what you think of this new feature and how you end up using it.

What’s that?  You hadn’t heard?  NCover is on a rapid release cycle, and v4.1 is hot out of the oven.  The team has been hard at work refining the product that has garnered so much praise this past summer.  NCover 4 was a revolutionary update to the code coverage arena, and 4.1 brings refinements that will make it even easier to roll out code coverage to your enterprise.

Floating on air…
NCover now supports floating licenses.  One of the most requested features in the history of the product has been to allow floating licenses for use on multiple machines.  With the advent of Code Central, NCover now has the perfect license manager located right within your environment.  Code Central already had built-in license management to manage Collectors and Desktops, but now floating licenses means that you can check out a license from the Code Central server and then check it back in to the server for use on another machine.  The benefits are flexibility, scalability, and a removal of the barrier to making NCover available across all development and testing machines in your enterprise. Please contact our sales team for more information about purchasing floating licenses.

NCover reporting for duty…
For your email and artifact enjoyment NCover 4.1 can produce a self-contained HTML report for any completed execution.  This report details the trend of coverage for the prior 25 executions and shows an assembly level overview of the coverage breakdown.  When generated from Code Central this report has live hyperlinks directed back to the details of the report within the Code Central GUI.  The coverage summary report is generated from the NCover command line and is ideal for embedding in your build process.

Snapshots from edge…
With the rise of QA coverage zones in the release cycle, many coverage runs for software can last an extended number of hours or days.  NCover 4.1 has a snapshot tool that captures an incremental picture of the coverage accomplished so far within an execution.  This often requested feature gives incremental insight into those long coverage runs most common to QA environments.  The snapshot functionality is available via the GUI and from the NCover command line.  Use stand alone within the GUI or use in conjunction with the coverage summary report for incremental updates on testing progress.

Windows 8 my homework….
Rumor has it that Microsoft has hit the street with some brand new toys.  Windows 8 and Visual Studio 2012 are now supported by NCover 4.1.  Dive in and enjoy the new Metro GUI and while you’re at it get some VS2012 on the side.  The Visual Studio plugin for NCover now works in Visual Studio versions 2008, 2010, and 2012.

Code Central forever alone…
In response to requests to simplify the implementation of Code Central on build servers, we have updated Code Central to be a full-fledged profiling environment.  Code Central no longer requires a local copy of Collector to perform the profiling tasks.  By making this change we have simplified both the licensing for Code Central and the setup.  All 3 components of NCover now share default service names, ports, and command line syntax.  One of our most requested updates to Code Central, it can now stand alone.  Code Central still retains all the project sharing features it was born to provide.  In addition the Auto-configure capability for detecting locally running processes to cover is now available in Code Central.

Filters with a little flash…
It turns out that with all the great compliments about the new NCover UI, we actually had one very consistent request.  In this case, it was a clear consensus that the application of post-coverage filters was not obvious enough to the user.  We’ve remedied that situation by add a highly visible badge showing the active filter in the dataview header.  Because the filter settings can be preserved with bookmarks and shared links, this badge will reappear whenever a link is active which applies a filter.  The application and removal of a filter remains unchanged, but this mechanism will definitely improve visibility.

More 2 tell….
There are literally dozens of additional features and fixes in the 4.1 release of NCover.  Stay tuned for more detail on these features and many more:

• Re-licensing
• Manual static analysis command
• Re-base paths for source
• Resizable grid columns
• Assembly version detail
• Faster profiling
• Data synchronization
• Pre-coverage filter override
• NCover Explorer Enhancements

See the release notes….

Since we launched NCover 4, we have been agonizing over how to provide you with the best user experience possible. Every time you interact with our website, product, or support team, we want you to be successful and have a satisfying experience. You have already seen how we redesigned NCover’s user interface, but we are now applying that attention-to-detail to our support home page.

The redesign of our support home page revolves around answering your most common support questions. We have tried our best to answer ninety percent of those questions up front. Instead of making you dig through layers of documentation that you may or may not need, you can now easily scan questions that link to the answers you need. Our new documentation categories include, Build Server Integration, NCover 4 Reporting, Coverage Scenarios, Project Settings, Connectivity, and Visual Studio. In short, we want to respect your time, and reward your diligence. While you’ll save the most time by looking through our documentation to find the answer you need, we always welcome you to submit a support ticket. No matter what, we are here to help you.

We have some of the smartest and hard working customers in the world, and we want to make sure that you love using NCover, as much as we love serving you.

Covering IIS in NCover 4 is fundamentally the same as profiling any other .NET process, but because of the way IIS runs, profiling this service presents some special challenges.

1. Project Settings

Finding the right process to profile is the key to successful coverage. Profiling is accomplished with match rules, so let’s talk about how they work for IIS and the options for creating them.

From the Home view of the NCover 4 Explorer, click Add New to open the “Create a New Project” window.

From the “General Settings” tab, name your project.

In the Processes to monitor section, specify a Type and a Value to create a Match Rule for the IIS process.

Users sometimes make the mistake of specifying a test DLL as a match rule. Remember that NCover needs to be able to profile the process that is loading your tested assemblies, in this case, the w3wp worker process.

With this in mind, a regex like “w3wp” is a good way to catch the worker process whenever it runs locally.

2. Auto-Configure

Another option is to click on the Auto-Configure button and let Desktop create match rules for you.

Auto-configure

The General Settings tab will switch to the auto-detect view and start monitoring for .NET processes as soon as you click on Auto-Configure.

Note: A common stumbling-block for users trying to cover IIS is the fact that a w3wp process already running in the background is easy to miss.

The NCover profiler needs to be able to attach to a process when it starts (whether this is IIS or a test runner). So, before testing web apps, you should reset IIS so that the NCover profiler can attach to the w3wp.exe as it initializes.

After you reset IIS, start your web application and exercise your code (manually or with unit tests).

The modules that are being loaded by your testing will display in the detection window.

Desktop IIS auto-detect

Review the list of modules detected and uncheck any modules that shouldn’t be covered.

Click Stop Detection when your selections are complete.

Click Show Match Rules to view the settings that NCover created based on the .NET code that was loaded during your testing.

Desktop IIS auto-generated Match Rule

This Exe match rule will cover IIS whenever it runs from the specified path.

You can change this path match rule to a regex instead, which will monitor the w3wp process regardless of where it runs.

Desktop IIS regex

Click Save to accept these selections.

Note: “Include” filters will be automatically created based on the modules that you selected. You can review the filters on the “Pre-Coverage Filter” tab.

Desktop IIS Pre-Coverage filter

3. Collect Coverage

When your auto-configure and filter selections are complete, save your project settings.

Note: Before running your web app, remember to reset IIS.

Start your web application. Go to the Home page and you’ll see that the light beside your project is pulsing green to indicate coverage is being collected.

Immediately to the right of your project name, the Modules Collecting counter displays the number of modules being profiled.

When your tests are complete, close your web application.

Note: Don’t forget to stop or reset IIS to end the w3wp process that NCover is profiling.

This will stop profiling and NCover 4 Desktop will collect and aggregate the coverage data.

Profiling IIS in Desktop uses the same principles as any other kind of coverage, but the IIS environment requires some additional input from the tester to make covering web applications successful.

The universal acclaim that greeted the release of “Code Blaster” last week presented the NCover team with another great opportunity — a Desktop demo using everyone’s favorite code coverage game!

The new “NCover 4 Desktop and Code Blaster” video is a more extensive look at the Desktop functionality that we highlighted briefly in the original “NCover 4 Desktop in 3 Minutes” video.

This screencast has three primary parts, demonstrating how to:

• Install and register NCover 4 Desktop
• Set up a project using Auto-configure and collect coverage data on Code Blaster
• Browse the resulting coverage data in Desktop

This guided tour of Desktop’s most important features has helpful on-screen captions and narration — it might be the next best thing to actually playing Code Blaster!

Any testing worth its asserts is going to do just that, blow up code when there’s a problem. Let’s face it testers are destructive by profession.  Embrace it!  It’s either that or pass broken software along to customers.  So in honor of all our friends who slave away all day and dream about testing all night, the NCover team has written a little ditty of a demo program in the form of a game we’ve dubbed “Code Blaster”.

Introducing:  Code Blaster

The overarching purpose of Code Blaster is to provide an entertaining example of how an application can be covered by NCover 4 during live testing.  Each time CodeBlaster.exe starts it recompiles local source for the Algorithms.dll provided in the zip file.  The Algorithms.dll will have a different binary hash for each run of the game.  This simulates the changing of code between build runs.  This too allows you to drill all the way into the source code for the Algoritms.dll just like you would for your own code.

Get started:
1. Download CodeBlaster from here.
2. Extract the contents of the zip file to a folder.
3. Run CodeBlaster.exe and start blowing up some code.

We are going to pull together an instructional video for collecting coverage on Code Blaster as a demo.  Look for that next week or give it a try on your own….or not, just download the game and start enjoying your weekend.

BTW, there are some hidden game features… kill all and machine gun mode are my personal favs.

DISCLAIMER:  While we’re not under any delusions that you may want to give up your Cheerful Chickens games for Code Blaster, you may still find it oddly addicting. Enjoy….

http://downloads.ncover.com/CodeBlaster/CodeBlaster.zip

There are several scenarios that may require you to reactivate your NCover license:

• You have a trial version of NCover 4 Desktop, you purchased a new license and need to activate it in place of the trial key code.
• You have upgraded from NCover 3 to NCover 4 and are using the same license key code.
• You purchased a subscription for your existing NCover license and need to re-register your key code.
• You have added Code Central and want to convert a standalone Desktop license to one managed by Code Central.

Access the main menu by clicking on the NCover icon and selecting “License Details.”

NCover menu License Details

The License Registration window will display the current licensing information and the option to use the new Reactivate button.

Desktop Reactivate

After clicking on Reactivate, you have the usual options: Connect to Code Central, or enter your key code and start using standalone Desktop.

There’s also a new registration option — Activate Offline, which we’ll cover in the next post.

Desktop registration options

A few weeks ago, I posted about build server integration with NCover 4, and said that the dev team was working to improve TeamCity coverage, so here’s the good news about NUnit integration!

You can still run nunit-console from your build script if you prefer, but making some simple changes to the the TeamCity TestRunner config will allow NCover 4 to profile the integrated NUnit runner without changing your test process.

In the TeamCity project where you want NC4 to profile NUnit, go to Configuration Step 6: Build Parameters:

Adding Build Parameters.

Set the System Property as appropriate for your build. If you’re using the TeamCity test runner, you have probably already configured that setting to look something like the following:

NUnit System Property

For NCover to detect the NUnit tests when they are executed by the build task, the build agent environment needs to be configured to enable NCover profiling.

This is easily done by adding two profiling environment variables:

NCover Environment Variables

These changes will enable NCover 4 to detect the TeamCity process that runs your unit tests. Depending on your test configuration, that exe will be some variation of  “JetBrains.TeamCity.PlatformProcessRunner.v4.0.msil.exe”. If your tests are designedfor the v2 CLR, you’ll need to watch for the v2 exe.

Once you’ve modified the environment variables for the build as described above, you can create an NCover 4 project to collect your unit test coverage.

Simply add a new project and press the Auto-configure button. NCover is now waiting for you to start the application you want to profile. Leave NCover in Auto-config mode, go back to TeamCity and start the NUnit task.

While that task runs, go back to NCover and you will see the processes that are started during the NUnit testing displayed in the detection window. Once the modules you want to cover are listed, select the ones you want to keep and then save the NCover project.

You don’t have to wait for all of your NUnit tests to complete, unless it’s necessary to make sure that all the modules you want to profile are loaded.

Once you save the NCover project settings, re-start the TeamCity NUnit task to collect your first coverage run.

After you set the TeamCity build parameters and configure your NCover project, covering TeamCity’s NUnit tests should be seamless and automatic.

As a follow-up to my post on understanding coverage metric, I want to explain how you can better visualize branch coverage. One of the reasons we feature both sequence point coverage and branch coverage numbers so prominently is that they are both essential to understanding the coverage picture for your project. Sequence point coverage relates directly to your code, so by looking at the code with coverage data overlaid, it’s easy to see which specific lines of code have not been tested.

As I explained in the last article, branch coverage is a better overall metric because it scales directly with the complexity of a method and can’t be distorted by the verbosity of your code the way that sequence point coverage can. So, if branch coverage is a better metric, then why do we even look at sequence point coverage? The main reason is that branch coverage is calculated directly from the compiler generated code, and the connection between the code and what is generated by the compiler is not always clear-cut or easy to visualize.

The primary value of branch coverage is to measure how much of your program’s structure is being exercised by your tests.

The primary value of sequence point coverage is to find the lines of code which are not being executed and are contributing to a lack of coverage.

Over the past few years, we’ve had many customers ask us to explain how the branch coverage numbers were being calculated for a given method. So, we created a technique to visualize branch coverage in the left hand margin of a method in the coverage display window. Let’s dive into an example here and describe how to interpret the diagram.

First, notice at the top of the window that our branch coverage is at 62.5% with 5 of 8 segments covered. The sequence point coverage is at 47.1% with 8 of 17 sequence points covered. The balls and line segments on the left side of the screen demonstrate those metrics visually. Each green ball represents a covered sequence point, and a red diamond represents an uncovered sequence point. If you count them up, you’ll see 8 green balls and 9 red diamonds for a total of 17 sequence points. The branch segments are shown by connecting the balls and diamonds with arrows. You’ll see 8 groups on the left with 3 of them red and 5 green for a total of 8 branch segments.

The sequence point markers are usually setup on the same line where the sequence points occur to make it obvious where they are. In addition, if you hover the mouse over a sequence point marker, or the code corresponding to it, we add additional information. The branch segment will have grey arrows showing which other segments flow into it and which other segments flow out.

In the screenshot above, the mouse is hovering over the Console.WriteLine("Segment 2"); line. You will notice that on the left, that the segment containing that code is highlighted in a slightly darker shade of green. The sequence point corresponding to that point is even darker green. There are only three sequence points in that branch segment. At the top you see a single grey arrow showing that the only way to run that segment of code is to branch there from the if (second) segment. At the bottom of the segment, there are two grey arrows showing that there are two ways to exit this segment. Depending on the value of the if (first) test, it can either execute the Console.WriteLine("Segment 3") segment, or it can skip past that to the end of the enclosing if block.

Exploring the diagram with the mouse makes it easier to understand what the branching structure of a method and what the coverage data really means. Hopefully this makes it easier for you to figure out how coverage is being calculated and why a particular section of code may not be covered by your tests.

Feel free to ask questions in the comments below.