the urban canuk, eh

Execute Batch from Visual Studio

2012-01-23T13:48:00.001-05:00

Since as long as I can remember, I’ve kept a command-line window open while I worked. It’s a warm fuzzy feeling of how computers used to work. I tend to structure commonly used tasks as msbuild or nant scripts, and then add handy batch files that pass the appropriate parameters to the script.

Unfortunately, most of my team-mates don’t live in the command-line, so running a batch file breaks their traditional flow.

Here’s a short tip on how to execute batch files without having to leave the comfort of the IDE. You’ve probably seen this tip before, but as always, I often use my blog as a digital memory. If it helps you, great.

Visual Studio supports the ability to associate tools and alternate editors for different files. Adding support for batch files is simply a matter of opening the context-menu for a file and choosing “Open With..”. Unfortunately, there’s no mechanism to supply parameters to your program, so adding support for a Command Prompt requires a small subtle hack that passes our parameters to the program we want.

To Associate Batch files to a Custom Command

First, we need to create a simple batch file that passes the arguments that Visual Studio provides onto our batch file.

If you haven't already, add your batch file to your solution. These are best treated as Solution items that aren’t part of your compilation process.
Open notepad and save the following script as C:\ExecuteBatch.cmd

@cmd /c %1

Once this is in place,

Associate the batch file in Visual Studio to the command line by right-clicking on the batch file and choose "Open With...".
In the dialog that appears, provide a name and associate it to the ExecuteBatch.cmd.

Optional: You can associate the Command as the default program for this extension, by selecting your custom command and clicking on the “Set as Default” button. Note that if you edit the file frequently you might want to skip this step, but you’ll have to right-click the file and choose “Open With…” anytime you want to run your custom command.

Gotchas & Caveats

Just a few closing points:

If you set your custom command as the default, note that there is no confirmation if you accidentally double-click the batch file. If your script is potentially destructive or long-running, you might want to add a prompt at the beginning of the batch file before running.
The ExecuteBatch.cmd provided above will close the window immediately after the batch terminates. If you want to review the output of the script before the window closes, you might want to add a pause to the end of the script.
Lastly, when adding new scripts to Visual Studio it will perform the default action when the file is added to the solution. If you don’t want to run the script when the file is added, you might want to temporarily assign a different editor (Source Code Editor) before you precede.

Happy Coding.

Fixing Parallel Test Execution in Visual Studio 2010

2011-11-28T08:55:00.000-05:00

As the number of tests in my project grow, so does the length of my continuous integration build. Fortunately, the new parallel test execution of Visual Studio 2010 allow us to trim down the amount of time consumed by our unit tests. If your unit tests meet the criteria for thread-safety you can configure your unit tests to run in parallel simply by adding the following to your test run configuration:

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local" id="5082845d-c149-4ade-a9f5-5ff568d7ae62" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
  <Deployment enabled="false" />
  <Execution parallelTestCount="0">
    <TestTypeSpecific />
    <AgentRule name="Execution Agents">
    </AgentRule>
  </Execution>
</TestSettings>

The ideal setting of “0” implies that the test runner will automatically figure out the number of concurrent tests to execute based on the number of processors on the local machine. Based on this, a single-core CPU will run 1 test simultaneously, a dual-core CPU can run 2 and a quad-core CPU can 4. Technically, a quad-core hyper-threaded machine has 8 processors but when parallelTestCount is set to zero the test run on that machine fails instantly:

Test run is aborting on '<machine-name>', number of hung tests exceeds maximum allowable '5'.

So what gives?

Well, routing through the disassembled source code for the test runner we learn that the number of tests that can be executed simultaneously interferes with the maximum number of tests that can hang before the entire test run is considered to be in a failed state. Unfortunately the maximum number of tests that can hang has been hardcoded to 5. Effectively, when the 6th test begins to execute the test runner believes that the other 5 executing tests are in a failed state so it aborts everything. Maybe the team writing this feature picked “5” as an arbitrary number, or legitimately believed there wouldn’t be more than 4 CPUs before the product shipped, or simply didn’t make the connection between the setting and the possible hardware. I do sympathize for the mistake: the developers wanted the number to be low because a higher number could add several minutes to a build if the tests were actually in an non-responsive state.

The Connect issue lists this feature as being fixed, although their are no posted workarounds and a there’s a lot of feedback that feature doesn’t work on high-end machines even with the latest service pack. But it is fixed, no-one knows about it.

Simply add the following to your registry (you will likely have to create the key) and configure the maximum amount based on your CPU. I’m showing the default value of 5, but I figure number of CPUs + 1 is probably right.

Windows 32 bit:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\Agent]
"MaximumAllowedHangs"="5"

Windows 64 bit:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\Agent]
"MaximumAllowedHangs"="5"

Note: although you should be able to set the parallelTestCount setting to anything you want, overall performance is constrained by the raw computing power of the CPU, so anything more than 1 test per CPU creates contention between threads which degrades performance. Sometimes I set the parallelTestCount to 4 on my dual-core CPU to check for possible concurrency issues with the code or tests.

Epilogue

So what’s with the Connect issue? Having worked on enterprise software my guess is this: the defect was logged and subsequently fixed, the instructions were given to the tester and verified, but these instructions never tracked forward with the release notes or correlated back to the Connect issue. Ultimately there’s probably a small handful of people at Microsoft that actually know this registry setting exists, fewer that understand why, and those that do either work on a different team or no longer work for the company. Software is hard: one small fissure and the whole thing seems to fall apart.

Something within the process is clearly missing. However, as a software craftsman and TDD advocate I’m less concerned that the process didn’t capture the workaround as I am that the code randomly pulls settings from the registry – this is a magic string hack that’s destined to get lost in the weeds. Why isn’t this number calculated based on the number of processors? Or better, why not make MaximumAllowedHangs configurable from the test settings file so that it can be unit tested without tampering with the environment? How much more effort would it really take, assuming both solutions would need proper documentation and tests?

Hope this helps.

iPhone to PC Adapter

2011-11-24T16:07:00.001-05:00

~~Merry~~ Happy Thanks Giving! I had some time on my hands so I decided to try something new.

Here’s a quick review of my iPhone headset to PC adapter that I bought a few weeks ago. Hopefully this video comes just in time for Christmas ideas and Black Friday shopping.

By the way, Thanks Giving was 5 weeks ago.

Static is Dead to Me

2011-11-22T12:07:00.002-05:00

The more software I write with a test-first methodology, the more I struggle with the use of singletons and static classes. They’ve become a design smell, and I’ve grown to realize that if given enough care and thought towards a design most static dependencies aren’t needed. My current position is that most singletons are misplaced artefacts without a proper home, and static methods seem like an amateurish gateway to procedural programming.

Despite my obvious knee-jerk loathing for static, in truth there’s nothing really wrong with it -- it’s hard to build any real-world application without the use of some static methods. I continue to use static in my applications but its use is reserved for fundamental top-level application services. All told, there should only be a handful of classes that are accessible as static members.

From a test-driven development perspective, there are several strong arguments against the use of static:

Lack of Isolation. When a class uses a static method in another type, it becomes directly coupled to that implementation. From a testing perspective it becomes impossible to test the consuming class without satisfying the requirements of the static dependency. This increases the complexity and fragility of the tests as the implementation details of the static dependency leak into many tests.
Side Effects. Static methods allow us to define objects that maintain state that is global in nature. This global state represents a problem from a testing perspective because any state that must be set up for a test fixture must be reset after the test completes. Failure to clean-up the global state can corrupt the environment and lead to side-effects in other tests that depend on this shared state.
Inability to run tests in parallel. A fundamental requirement for reliable tests is a predictable, well-known state before and after each test. If tests depend on global state that can be mutated by multiple tests simultaneously then it is impossible to run more than one test at a time. Given the raw computing power of a hyper-threaded, multi-core machine, it seems a crime to design our code that limits testing to only one core.
Hidden dependencies. Classes that pull dependencies into them from a static singleton or service-locator creates an API that lies to you. Tests for these classes become unnecessarily complex and increasingly brittle.

An Alternative to Static

Rather than designing objects to be global services that are accessed through static methods, I design them to be regular objects first. There are no static methods or fields. Classes are designed to be thread-safe, but make no assumptions about their lifetime.

This small change means that I expect all interaction to be with an instance of my object rather through a member that is Type specific. This suggests two problems: how will consumers of my class obtain a reference to my object, and how do I ensure that all consumers use the same object?

Obtaining a Reference

Not surprisingly, the problem related to obtaining a reference to my object is easily solved using my favourite Inversion of Control technique, Constructor Injection. While there are many IoC patterns to choose from, I prefer constructor injection for the following reasons:

Consuming classes do not have to depend on a specific framework to obtain a reference.
Consuming classes are explicit about their required dependencies. This fosters a consistent and meaningful API where objects are assembled in predictable and organized manner rather than randomly consumed.
Consuming classes don’t have to worry about which instance or the lifetime of the object they have received. This solves many testing problems as concurrent tests can work with different objects.

The difference between accessing my object through a static member versus an object instance is very subtle, but the main distinction is that using the object reference requires some forethought as the dependency must be explicitly defined as a member of the consuming class.

Obtaining the Same Reference

By forgoing the use of static, we’ve removed the language feature that would simplify the ability to ensure only a single instance of our object is consumed. Without static we need to solve this problem through the structure of our code or through features of our application architecture. Without a doubt it’s harder, but I consider the well structured and tested code worth it (so don’t give up).

Eliminating Singletons through Code Structure:

My original position is that most singletons are simply misplaced artefacts. By this I mean static is used for its convenience rather than to expose the class as a global application service. In these situations it’s far more likely that the static class provides a service that is used in one area of the application graph. I’d argue that with some analysis the abstraction or related classes could be rearranged to create and host the service as an instance thereby negating the need for a singleton.

This approach typically isn’t easy because the analysis requires you to understand the lifetime and relationship of all objects in the graph. For small applications with emergent design, the effort is obtainable and extremely rewarding when all the pieces fit together nicely. The effort for larger applications may require a few attempts to get it right. Regardless of application size, sticking with an inverted dependencies approach will make the problem obvious when it occurs.

An inversion of control container can lessen the pain.

Eliminating Singletons through Architecture:

Perhaps my favourite mechanism for eliminating singletons is to use an Inversion of Control container and configure it with the responsibility of maintaining the object lifetime.

This example shows programmatic registration of a concrete type as a singleton.

private void InitializeGlobalServices(IUnityContainer container)
{
   // configure the container to cache a single instance of the service
   // the first time it is used.
   container.RegisterType<MyServiceProvider>(new ContainerControlledLifetimeManager());
}

The real advantage here is that any object can be made static without rewriting code. As a further optimization, we can also introduce an interface:

private void InitializeGlobalService(IUnityContainer container)
{
   container.RegisterType<IServiceProvider,MyServiceProvider(
       new ContainerControlledLifetimeManager());
}

Conclusion

Somewhere in my career I picked up the design philosophy that “objects should not have a top”, meaning that they should be open-ended in order to remix them into different applications. Eventually the "top" is the main entry-point into the application which is responsible for assembling the objects to create the application.

Dependency Injection fits nicely into this philosophy and in my view is the principle delivery mechanism for loosely coupled and testable implementations. Static however works against this in every regard: it couples us to implementations and limits our capability to test.

The clear winner is dependency injection backed by the power of an inversion of control container that can do the heavy lifting for you. As per my previous post, if you limit usage of the container to the top-level components, life is simple.

Happy coding.

Guided by Tests–Wrap Up

2011-10-24T10:14:00.001-04:00

This post is ninth and last in a series about a group TDD experiment to build an application in 5 7 days using only tests. Read the beginning here.

This last post is aimed at wrapping up the series by looking back at some of the metrics we can collect from our code and tests. There’s some interesting data about the experiment as well as feedback for the design available.

We’ll use three different data sources for our application metrics:

Visual Studio Code Analysis
MSTest Code Coverage
NDepend Code Analysis

Visual Studio Code Analysis

The code analysis features of Visual Studio 2010 provide a convenient view of some common static analysis metrics. Note that this feature only ships with the Premium and Ultimate versions. Here’s a quick screen capture that shows a high level view of the metrics for our project.

Tip: When reading the above graph, keep an eye on the the individual class values not the roll-up namespace values.

Here’s a breakdown of what some of these metrics mean and what we can learn from our experiment.

Maintainability Index

I like to think of the Maintainability Index as a high level metric that summarizes how much trouble a class is going to give you over time. Higher numbers are better, and problems usually start below the 20-30 range. The formula for the maintainability index is actually quite complex, but looking at the above data you can see how the other metrics drive the index down.

Our GraphBuilder is the lowest in our project, coming in at an index of 69. This is largely influenced by the density of operations and complexity to lines of code – our GraphBuilder is responsible for constructing the graph from conditional logic of the model. The maintainability index is interesting, but I don’t hold much stock in it alone.

Lines of Code

Lines of Code is the logical lines of code which means code lines without whitespace and stylistic formatting. Some tools, like NDepend, record other metrics for lines of code, such as number of IL instructions per line. Visual Studio’s Lines of Code metric is simple and straight forward.

There are a few interesting observations for our code base.

First, the number of lines of code per class is quite low. Even the NDependStreamParser which tops the chart at 22 lines is extremely low considering that it reads data from several different Xml Elements. The presence of many small classes suggests that classes are designed to do one thing well.

Secondly, there are more lines of code in our test project than production code. Some may point to this as evidence that unit testing introduces more effort than writing the actual code – I see the additional code as the following:

We created hand-rolled mocks and utility classes to generate input. These are not present in the production code.
Testing code is more complicated than writing it as there are many different paths the code might take. There should be more code here.
We didn’t write the code and then the tests, we did them at the same time
Our tests ensured that we only wrote code needed to make the tests pass. This allowed us to aggressively remove all duplication in the production code. Did I mention the largest and most complicated class is 22 lines long?

Cyclomatic Complexity

Cyclomatic Complexity, also known as Conditional Complexity, represents the number of paths of execution a method can have – so classes that have a switch or multiple if statements will have higher complexity than those that do not. The metric is normally applied at the method level, not the class, but if you look at the graph above, more than half of the classes average below 5 and the other half are less than 12. Best practices suggest that Cyclomatic Complexity should be less than 15-20 per method. So we’re good.

Although the graph above doesn’t display our 45 methods, the cyclomatic complexity for most methods is 1-2. The only exception to this is our NDependStreamParser and GraphBuilder, which have methods with a complexity value of 6 and 5 respectively.

In my view, I see cyclomatic complexity as a metric for how many tests are needed for a class.

Depth of Inheritance

The “depth of inheritance” metric refers to the number of base classes involved in the inheritance of a class. Best practices aim to keep this number as low as possible since each level in the inheritance hierarchy represents a dependency that can influence or break implementers.

Our graph shows very low inheritance depth which supports our design philosophy of using composition and dependency inversion instead of inheritance. There are a few red flags in the graph though: our AssemblyGraphLayout has an inheritance depth of 9, a consequence of extending a class from the GraphSharp library and it highlights possible brittleness surrounding that library.

Class Coupling

The class coupling metric is a very interesting metric because it shows us how many classes our object consumes or creates. Although we don’t get much visibility into the coupling (NDepend can help us here) it suggests that classes with a higher coupling are much more sensitive to changes. Our GraphBuilder has a Class Coupling of 11, including several types from the System namespace (IEnumerable<T>, Dictionary, KeyValuePair) but also has knowledge of our Graph and model data.

Class coupling combined with many lines of code and high cyclomatic complexity are highly sensitive to change, which explains why the GraphBuilder has the lowest Maintenance Index of the bunch.

Code Coverage

Code coverage is a metric that shows which execution paths within our code base are covered by unit tests. While code coverage can’t vouch for the quality of the tests or production code, it can indicate the strength of the testing strategy.

Under the rules of our experiment, there should be 100% code coverage because we’ve mandated that no code is written without a test. We have 93.75% coverage, which has the following breakdown:

Interestingly enough, the three areas of with no code coverage are the key obstacles we identified early in the experiment. Here are the snippets of code that have no coverage:

Application Start-up Routine

protected override void OnStartup(StartupEventArgs e)
{
    var shell = new Shell();
    shell.DataContext = new MainViewModelFactory().Create();

    shell.Show();
}

Launching the File Dialog

internal virtual void ShowDialog(OpenFileDialog dialog)
{
    dialog.ShowDialog();
}

Code behind for Shell

public Shell()
{
    InitializeComponent();
}

We’ve designed our solution to limit the amount of “untestable” code, so these lines of code are expected. From this we can establish that our testing strategy has three weaknesses, two of which are covered by launching the application. If we wanted to write automation for testing the user-interface, these would be the areas we’d want early feedback from.

NDepend Analysis

NDepend is a static code analysis tool that provides more detailed information than the standard Visual Studio tools. The product has several different pricing levels including open-source and evaluation licenses and has many great visualizations and features that can help you learn more about your code. While there are many visuals that I could present here, I’m listing two interesting diagrams.

DependencyGraph:

This graph shows dependencies between namespaces. I’ve configured this graph to show two things:

Box Size represents Afferent Coupling where larger boxes are used by many classes
Line Size represents the number of methods between the dependent components.

Dependency Matrix:

A slightly different view of the same information is the Dependency Matrix. This is the graph represented in a cross-tabular format.

Both of these views help us better visualize the Class Coupling metric that Visual Studio pointed out earlier, but the information it provides is quite revealing. Both diagrams show that the Model namespace is used by the ViewModels and Controls namespaces. This represents a refactoring or restructuring problem as these layers really should not be aware of the model: Views shouldn’t have details about the ViewModel; Service layer should only know about the Model and ViewModels; Controls should know about ViewModel data if needed; ViewModels should represent view-abstractions and thus should not have Model knowledge.

The violations have occurred because we established our AssemblyGraph and related items as part of the model. This was a concept we wrestled with at the beginning of the exercise, and now the NDepend graph helps visualize the problem more concretely. As a result of this choice, we’re left with the following violations:

The control required to layout our graph is a GraphLayout that uses our Model objects.
The MainViewModel references the Graph as a property which is bound to the View.

The diagram shows a very thin line, but this model to view model state problem has become a common theme in recent projects. Maybe we need a state container to hold onto the Model objects and maybe the Graph should be composed of ViewModel objects instead of Model data. It’s worth consideration, and maybe I’ll revisit this and blog about this problem some more in an upcoming post.

Conclusion

To wrap up the series, let’s look back on how we got here:

We started things off by breaking the use case down into small logical chunks. This separation of concerns approach allowed us to create small focused classes and reduced the amount of complexity overall.
When we started coding, we started with the area we knew most about and we leveraged our knowledge of the NDepend file format to create a reader that would build up a model object for us.
We worked backward, and we isolated the problematic areas and stubbed them out with an interface until we could revisit their implementation. We created hand-rolled mocks and later used a framework to dynamically generate mock implementations for us.
We iteratively realized parts of our use case, which meant we actively thought about how these components would be used by callers. Under the rules of our experiment, we wrote tests to satisfy those scenarios.
After linking all the parts together and repurposing some XAML, we had our first run of the application. We came back and flushed out the implementation of our FileDialog using tests for what would normally be considered “untestable”.

I hope you have enjoyed the series and found something to take away. I challenge you to find a similar problem and champion its development within your development group.

Happy Coding.

Guided by Tests–Day Seven

2011-10-11T09:00:00.000-04:00

This post is eighth in a series about a group TDD experiment to build an application in 5 7 days using only tests. Read the beginning here.

Today is the day we test the untestable. Early in the experiment we hit a small road block when our code needed to interact with the user-interface. Since unit-testing the user-interface wasn’t something we wanted to pursue, we wrapped the OpenFileDialog behind a wrapper with an expectation that we would return to build out that component with tests later. The session for this day would prove to be an interesting challenge.

The Challenges of Physical Dependencies

Although using a wrapper to shield our code from difficult to test dependencies is a common and well accepted technique, it would be irresponsible not to test the internal implementation details of that wrapper. Testing against physical dependencies is hard because they introduce a massive amount of overhead, but if we can isolate the logic from the physical dependency we can use unit tests to get 80-90% of the way there. To get the remaining part, you either test manually or write a set of integration or functional tests.

The technique outlined below can be used for testing user-interface components like this one, email components and in a pinch it can work for network related services.

Testing our Wrapper

Time to write some tests for our IFileDialog. I have some good news and bad news.

The good news is Microsoft provides a common OpenFileDialog as part of WPF, meaning that I don’t have to roll my own and I can achieve a common look and feel with other applications with little effort. This also means we can assume that the FileOpenDialog is defect free, so we don’t have to write unit tests for it.

The bad news is, I use this common so infrequently that I forget how to use it.

So instead of writing a small utility application to play with the component, I write a test that shows me exactly how the component works:

[TestMethod]
public void WhenSelectAFileFromTheDialog_AndUserSelectsAFile_ShouldReturnFileName()
{
    var dialog = new OpenFileDialog();
    dialog.Show(); // this will show the dialog

    Assert.IsNotNull(dialog.FileName); 
}

When I run this test, the file dialog is displayed. If I don’t select a file, the test fails. Now that we know how it works, we can rewrite our test and move this code into a concrete implementation.

Unit Test:

[TestMethod]
public void WhenSelectingAFile_AndUserMakesAValidSelection_ShouldReturnFileName()
{
    var subject = new FileDialog();
    string fileName = subject.SelectFile();
    Assert.IsNotNull(fileName);
}

Production Code:

public class FileDialog : IFileDialog
{
    public string SelectFile()
    {
        var dialog = new OpenFileDialog();
        dialog.Show();

        return dialog.FileName;
    }
}

The implementation is functionally correct, but when I run the test I have to select a file in order to have the test pass. This is not ideal. We need a means to intercept the dialog and simulate the user selecting a file. Otherwise, someone will have to babysit the build and manually click file dialog prompts until the early morning hours.

Partial Mocks To the Rescue

Instead of isolating the instance of our OpenFileDialog with a mock implementation, we intercept the activity and allow ourselves to supply a different implementation for our test. The following shows a simple change to the code to make this possible.

public class FileDialog : IFileDialog
{
    public string SelectFile()
    {
        var dialog = new OpenFileDialog();

        Show(dialog);

        return dialog.FileName;
    }

    internal virtual void Show(OpenFileDialog dialog)
    {
        dialog.Show();
    }
}

This next part is a bit weird. In the last several posts, we’ve used Moq to replace our dependencies with fake stand-in implementations. For this post, we’re going to mock the subject of the test, and fake out specific methods on the subject. Go back and re-read that. You can stop re-reading that now.

As an aside: I often don’t like showing this technique because I’ve seen it get abused. I’ve seen abuses where developers use this technique to avoid breaking classes down into smaller responsibilities; they fill their classes with virtual methods and then stub out huge chunks of the subject. This feels like shoddy craftsmanship and doesn’t sit well with me – granted, it works, but it leads to problems. First, the areas that they’re subverting never get tested. Secondly, it’s too easy for developers to forget what they’re doing and they start to write tests for the mocking framework instead of the subject’s functionality. So use with care. In this example, I’m subverting one line of a well tested third-party component in order to avoid human-involvement in the test.

In order to intercept the Show method and replace it with our own implementation we can use Moq’s Callback feature. I’ve written about this Moq’s support for Callbacks before, but in a nutshell Moq can intercept the original method and inbound arguments for use within your test.

Our test now looks like this:

[TestMethod]
public void WhenSelectingAFile_AndUserMakesAValidSelection_ShouldReturnFileName()
{
    // setup a partial mock for our subject
    var mock = new Mock<FileDialog>();
    FileDialog subject = mock.Object;

    // The Show method in our FileDialog is virtual, so we can setup
    //    an alternate behavior when it's called.
    // We configure the Show method to call the SelectAFile method
    //    with the original arguments
    mock.Setup( partialMock => partialMock.Show(It.IsAny<OpenFileDialog>())
        .Callback( SelectAFile );

    string fileName = subject.SelectFile();
    Assert.IsNotNull(fileName);
}

// we alter the original inbound argument to simulate
//    the user selecting a file
private void SelectAFile(OpenFileDialog dialog)
{
    dialog.FileName = "Foo";
}

Now when our test runs the FileDialog returns “Foo” without launching a popup. Now we can write tests for a few extra scenarios:

[TestMethod]
public void WhenSelectingAFile_AndTheUserCancelsTheFileDialog_NoFileNameShouldBeReturned()
{
    // we're mocking out the call to show the dialog,
    // so without any setup on the mock, the dialog will not return a value.

    string fileName = Subject.SelectFile();

    Assert.IsNull(fileName);
}

[TestMethod]
public void WhenSelectingAFile_BeforeUserSelectsAFile_EnsureDefaultDirectoryIsApplicationRootFolder()
{
    // ARRANGE:
    string expectedDirectory = Environment.CurrentDirectory;

    Mock.Get(Subject)
        .Setup(d => d.ShowDialog(It.IsAny<OpenFileDialog>()))
        .Callback<OpenFileDialog>(
            win32Dialog =>
            {
                // ASSERT: validate that the directory is set when the dialog is shown
                Assert.AreEqual(expectedDirectory, win32Dialog.InitialDirectory);
            });

    // ACT: invoke showing the dialog
    Subject.SelectFile();
}

Next: Review some of the application metrics for our experiment in the Guided by Tests – Wrap Up.

Guided by Tests–Day Six

2011-10-04T08:31:00.000-04:00

This post is seventh in a series about a group TDD experiment to build an application in 5 7 days using only tests. Read the beginning here.

Today is the day where we put it all together, wire up a user-interface and cross our fingers when we run the application. I joked with the team that “today is the day we find out what tests we missed.” Bugs are simply tests we didn’t think about.

Outstanding Design Concerns

At this point we have all the parts for our primary use case. And although we’ve designed the parts to be small with simple responsibilities, we’ve been vague about how the structure of the overall application. Before we can show a user-interface, we’ve got a few remaining design choices.

Associating the Command to the View

In the last post we looked at the NewProjectCommand. After reviewing our choices about the relationship between the Command and the ViewModel we decided that the most pragmatic choice was to have the Command hold a reference to the ViewModel, though we didn’t define where either object fit into the overall application. The next question presented to the team was whether we make the command independent of the ViewModel and reside in a menu-control or other top-level ViewModel, or should we add a Command property to the ViewModel?

The team decided to take the convenience route and put our NewProjectCommand as a property on our MainViewModel. Ultimately, this decision means that our MainViewModel becomes the top-level ViewModel that the View will bind to. This decision is largely view-centric and should be easy to change in the future if we need to reorganize the structure of the application.

With this decision resolved, we’ve got a better picture of how the user-interface and logical model will work. Before we start working on the xaml, we need to figure out how all these pieces come together.

Assembling the MainViewModel

The parts of our application have been designed with the Dependency Inversion Principle in mind – our objects don’t have a “top” and expect that the caller will pass in the proper configuration. Eventually, something must take the responsibility to configure the object graph. The next question is: where should this happen?

Someone suggested stitching the components together in the application start-up routine. While this is logically where this activity should occur, it doesn’t make sense to burden the start-up routine with these implementation details. If we did, we’d likely have a monster start-up routine that would change constantly. Plus, we’d have little means to test it without invoking the user-interface.

Achieving 100% code coverage, though a noble effort, is not always realistic. In most applications it’s likely that 10% of the codebase won’t have tests because those areas border on the extreme edges of the application. Writing unit tests for extreme edges is hard (“unit” tests for user-interface components is especially difficult) and it helps to have a balanced testing strategy that includes unit, system integration and functional UI automation to cover these areas.

Rather than including these details in the application start-up routine, we’ll move as much code as possible into testable components. The goal is to make the start-up routine so simple that it won’t require changes. This will minimize the untestable areas of the application.

To accomplish this, we decided to create a Factory for a MainViewModel.

Realizations for the MainViewModel Factory

The primary responsibility of our factory is to assemble our ViewModel with a fully configured NewProjectCommand. The test looks something like this:

[TestMethod]
public void CreatingAViewModel_WithDefaults_ShouldSetupViewModel()
{
    var factory = new MainViewModelFactory();
    var viewModel = factory.Create();
    
    Assert.IsNotNull(viewModel, "ViewModel was not created.");
    Assert.IsNotNull(viewModel.NewProject, "Command was not wired up.");
}

In order to make the test pass, we’ll need to wire-up the all the components of our solution within the factory. The argument checking we’ve introduced in the constructors of our command and loader guarantees that. Ultimately, the factory looks like this:

public class MainViewModelFactory
{
    public MainViewModel Create()
    {
        var vm = new MainViewModel();

        IFileDialog dialog = new MockFileDialog();
        IGraphDataParser graphParser = new NDependStreamParser();
        var graphBuilder = new GraphBuilder();

        var loader = new ProjectLoader(dialog, graphParser);

        vm.NewProject = new NewProjectCommand(vm, loader, graphBuilder);

        return vm;
    }
}

With that all in place, the application start-up routine is greatly simplified. We could probably optimize it further, but we’re anxious to see our application running today, so this will do:

public partial class App : Application
{
    protected override void OnStartup(StartupEventArgs e)
    {
        var shell = new Shell();
        shell.DataContext = new MainViewModelFactory().Create();
        shell.Show();
    }
}

Running the App for the first time

Ready for our first bug? We borrowed some XAML from Sacha’s post, and after we discovered that we needed a one-line user control, we wired up a crude Button to our NewProject command. Then we launched the app, crossed our fingers, and upon clicking the “New Project” button….. nothing happened!

We set a breakpoint and then walked through the command’s execution. Everything was perfect, except we had forgotten one small detail: we forgot to notify the user-interface that we had produced a graph.

As a teaching opportunity, we wrote a unit test for our first bug:

[TestClass]
public class MainViewModelTests
{
    [TestMethod]
    public void WhenGraphPropertyChanges_ShouldNotifyTheUI()
    {
        string propertyName = null;

        var viewModel = new MainViewModel();
        viewModel.PropertyChanged += (sender,e) => 
            propertyName = e.PropertyName;

        viewModel.Graph = new AssemblyGraph();

        Assert.AreEqual("Graph", propertyName,
            "The user interface wasn't notified.");
    }
}

And on second attempt (read: messing with xaml styles for a while) the app produces the following graph:

Next: Read about polishing off the File Dialog in Day Seven

Guided by Tests–Day Five

2011-09-20T09:38:00.001-04:00

This post is sixth in a series about a group TDD experiment to build an application in 5 days using only tests. Read the beginning here.

Today is the fifth day of our five day experiment to write an application over lunch hours using TDD. Five days was a bit ambitious, as a lot of time was spent teaching concepts, so the team agreed to tack on a few extra days just to see it all come together. So, my bad.

By the fifth day, we had all the necessary pieces to construct a graph from our NDepend xml file. Today we focused our attention on how these pieces would interact.

Next Steps

If we look back to our original flow diagram (shown below), upon receiving input from the user we need to orchestrate between loading model data from a file and converting into a Graph object so that it can be put into the UI. As we have the loading and conversion parts in place, our focus is how to receive input, perform the work and update the UI.

Within WPF and the MVVM architecture, the primary mechanism to handle input from the user is a concept known as Commanding, and commands are implemented by the ICommand interface:

namespace System.Windows.Input
{
    public interface ICommand
    {
        bool CanExecute( object parameter );
        void Execute( object parameter );

        event EventHandler CanExecuteChanged;
    }
}

While it’s clear that we’ll use a custom command to perform our orchestration logic, the question that remains is how we should implement it. There are several options available, and two popular MVVM choices are:

use the DelegateCommand that ships with Microsoft’s Prism framework, or;
an old school approach where we encapsulate the logic for the command as its own class.

The elegance of the DelegateCommand allows the user to supply delegates for the Execute and CanExecute methods, and typically these delegates live within the body of the ViewModel class. When given the choice I prefer command classes for application-level operations as it aligns well to the Single Responsibility Principle and our separation of concerns approach. (I tend to use the DelegateCommand option to perform view-centric operations such as toggling visibility of view-elements, etc.)

Writing a Test for an ICommand

As a team, we dug into writing the test for our NewProjectCommand. Assuming we’d use the ICommand interface, we stubbed in the parts we knew before we hit our first roadblock:

[TestClass]
public class NewProjectCommandTests
{
    [TestMethod]
    public void WhenExecutingTheCommand_DefaultScenario_ShouldShowGraphInUI()
    {
        var command = new NewProjectCommand();
        
        command.Execute( null );

        Assert.Fail(); // what should we assert?
    }
}

Two immediate concerns arise:

First, we have a testing concern. How can we assert that the command accomplished what it needed to do? The argument supplied to the Execute command typically originates from xaml binding syntax, which we won’t need, so it's not likely that the command will take any parameters. Moreover, the Execute method doesn't have a return value, so we won't have any insight into the outcome of the method.

Our second concern is a design problem. It's clear that our command will need to associate graph-data to a ViewModel representing our user-interface, but how much information should the Command have about the ViewModel and how will the two communicate?

This is one of the parts I love about Test Driven Development. There is no separation between testing concerns and design problems because every test you write is a design choice.

Reviewing our Coupling Options

We have several options to establish the relationship between our Command and our ViewModel:

Accessing the ViewModel through the WPF’s Application.Current;
Making our ViewModel a Singleton;
Create a globally available ServiceLocator that can locate the ViewModel for us;
Pass the ViewModel to the command through Constructor Injection
Have the Command and ViewModel communicate through an independent publisher/subscriber model

Here’s a further breakdown of those options…

Application.Current.MainWindow

The most readily available solution within WPF is to leverage the framework’s application model. You can access the top-level ViewModel with some awkward casting:

var viewModel = Application.Current.MainWindow.DataContext as MainViewModel;

I’m not a huge fan of this for many reasons:

Overhead: the test suddenly requires additional plumbing code to initialize the WPF Application with a MainWindow bound to the proper ViewModel. This isn’t difficult to do, but it adds unwarranted complexity.
Coupling: Any time we bind to a top-level property or object, we’re effectively limiting our ability to change that object. In this case, we’re assuming that the MainWindow will always be bound to this ViewModel; if this were to change, all downstream consumers would also need to change.
Shared State: By consuming a static property we are effectively making the state of ViewModel shared by all tests. This adds some additional complexity to the tests to ensure that the shared-state is properly reset to a neutral form. As a consequence, it’s impossible to run the tests in parallel.

ViewModel as Singleton / ServiceLocator

This approach is a slight improvement over accessing the DataContext of the current application’s MainWindow. It eliminates the concerns surrounding configuring the WPF Application, and we gain some type-safety as we shouldn’t have to do any casting to get our ViewModel.

Despite this, Singletons like the Application.Current variety are hidden dependencies that make it difficult to understand the scope and responsibilities of an object. I tend to avoid this approach for the same reasons listed above.

Constructor Injection

Rather than having the Command reach out to static resource to obtain a reference to the ViewModel, we can use Constructor Injection to pass a reference into the Command so that it has all the resources it needs. (This is the approach we’ve been using thus far for our application, too) This approach makes sense from an API perspective as consumers of your code will be able to understand the Command’s dependencies when they try to instantiate it. The downside to this approach is additional complexity is needed to construct the command. (Hint: We’ll see this in the next post.)

This approach also eliminates the shared-state and parallelism problem but it still couples us the Command to the ViewModel. This might not be a problem if the relationship between the two objects remains fixed – for example, if the application were to adopt a multi-tabbed interface the relationship between the command and ViewModel would need to change.

Message-based Communication

The best form of loose-coupling comes from message-based communication, where the ViewModel and the Command know absolutely nothing about each other and only communicate indirectly through an intermediary broker. In this implementation, the Command would orchestrate the construction of the Graph and then publish a message through the broker. The broker, in turn, would deliver the message to a receiver that would associate the graph to the ViewModel.

Such an implementation would allow both the ViewModel and the Command implementations to be change independently as long as they both adhere to the message publish/subscription contracts. I prefer this approach for large scale applications, though it introduces a level of indirection that can be frustrating at times.

This approach would likely work well if we needed to support a multi-tabbed user interface.

Back to the Tests

Given our time constraints and our immediate needs, the team decided constructor injection was the way to go for our test.

[TestClass]
public class NewProjectCommandTests
{
    [TestMethod]
    public void WhenExecutingTheCommand_DefaultScenario_ShouldShowGraphInUI()
    {
        var viewModel = new MainViewModel();
        var command = new NewProjectCommand(viewModel);
        
        command.Execute( null );

        Assert.IsNotNull( viewModel.Graph,
            "Graph was not displayed to the user.");
    }
}

To complete the test the team made the following realizations:

Realization	Code Written
Tests are failing because Execute throws a NotImplementedException. Let’s commit a sin and get the test to pass.	Implementation: public void Execute(object unused) { // hack to get the test to pass _viewModel.Graph = new AssemblyGraph(); }
Our command will need a ProjectLoader. Following guidance from the previous day, we extracted an interface and quickly added it to the constructor.	Test Code: var viewModel = new MainViewModel(); var projectLoader = new Mock<IProjectLoader>().Object; var command = new NewProjectCommand( viewModel, projectLoader); Implementation: ///<summary>Default Constructor</summary> public NewProjectCommand( MainViewModel viewModel, IProjectLoader projectLoader) { _viewModel = viewModel; _projectLoader = projectLoader; }
We should only construct the graph if we get data from the loader. (Fortunately, moq will automatically return non-null for IEnumerable types, so our tests pass accidentally)	Implementation: public void Execute(object unused) { IEnumerable<ProjectAssembly> model = _projectLoader.Load(); if (model != null) { _viewModel.Graph = new AssemblyGraph(); } }
We’ll need a GraphBuilder to convert our model data into a graph for our viewmodel. Following some obvious implementation, we’ll add the GraphBuilder to the constructor. A few minor changes and our test passes.	Test Code: var viewModel = new MainViewModel(); var projectLoader = new Mock<IProjectLoader>(); var graphBuilder = new GraphBuilder(); var command = new NewProjectCommand( viewModel, projectLoader, graphBuilder); Implementation: public void Execute(object unused) { IEnumerable<ProjectAssemby> model = _projectLoader.Load(); if (model != null) { _viewModel.Graph = _graphBuilder.BuildGraph( model ); } }
Building upon our findings from the previous days, we recognize our coupling to the GraphBuilder and we recognize that we should probably write some tests that demonstrate that the NewProjectCommand can handle failures from both the IProjectLoader and GraphBuilder dependencies. But rather than extract an interface for the GraphBuilder, we decide that we’d simply make the BuildGraph method virtual instead – just to show we can. Only now, when we run the test the test fails. It seems our graph was not created?	Test Code: var viewModel = new MainViewModel(); var projectLoader = new Mock<IProjectLoader>().Object; var graphBuilder = new Mock<GraphBuilder>().Object; var command = new NewProjectCommand( viewModel, projectLoader, graphBuilder); // ...

Finally, in order to get our test to pass, we need to configure the mock for our GraphBuilder to construct a Graph. The final test (shown below) looks like this. Note the IsAnyModel is a handy shortcut for simplifiying Moq’s matcher syntax.

[TestClass]
public class NewProjectCommandTests
{
    [TestMethod]
    public void WhenExecutingTheCommand_DefaultScenario_ShouldShowGraphInUI()
    {
        // ARRANGE: setup dependencies
        var viewModel = new MainViewModel();
        var projectLoader = new Mock<IProjectLoader>().Object;
        var graphBuilder = new Mock<GraphBuilder>().Object;

        // Initialize subject under test
        var command = new NewProjectCommand(
                            viewModel,
                            projectLoader,
                            graphBuilder);
        
        Mock.Get(graphBuilder)
            .Setup( g => g.BuildGraph( IsAnyModel() ))
            .Returns( new AssemblyGraph() );

        // ACT: Execute our Command
        command.Execute( null );

        // ARRANGE: Verify that the command executed correctly
        Assert.IsNotNull( viewModel.Graph,
            "Graph was not displayed to the user.");
    }

    private IEnumerable<ProjectAssembly> IsAnyModel()
    {
        return It.IsAny<IEnumerable<ProjectAssembly>>();
    }
}

Of course, we’d need a few additional tests:

When the project loader or graph builder throw an exception.
When the project loader doesn’t load a project, the graph should not be changed
When the Command is created incorrectly, such as null arguments

Next: Day Six

Guided by Tests–Day Four

2011-09-13T08:41:00.000-04:00

This post is fifth in a series about a group TDD experiment to build an application in 5 days using only tests. Read the beginning here.

As previously mentioned during Day Three, we split the group into two teams so that one focused on the process to load a new project while the other focused on constructing a graph. This post will focus on the efforts of the team working through the tests and implementation of the GraphBuilder.

This team had a unique advantage over the other team in that they had a blog post that outlined how to use the Graph# framework. I advised the team that they could refer to the post, even download the example if needed, but all code introduced into the project had to follow the rules of the experiment: all code must be written to satisfy the requirements of a test.

A Change in Approach

The goals for this team we’re different too. We already had our data well defined and had a reasonable expectation of what the results should be. As such, we took a different approach to writing the tests. Up to this point, our process has involved writing one test at a time and only the code needed to satisfy that test. We wouldn’t identify other tests that we’d need to write until we felt the current test was complete.

For this team, we had a small brain-storming session and defined all the possible scenarios we would need to test upfront.

I love this approach and tend to use it when working with my teams. I usually sit down with the developer and envision how the code would be used. From this discussion we stub out a series of failing tests (Assert.Fail) and after some high level guidance about what we need to build I leave them to implement the tests and code. The clear advantage to this approach is that I can step in for an over-the-shoulder code-review and can quickly get feedback on how things are going. When the developer says things are moving along I can simply challenge them to “prove it”. The developer is more than happy to show their progress with working tests, and the failing tests represent a great opportunity to determine if the developer has thought about how to finish them. Win/Win.

The test cases we identified for our graph builder:

When building a graph from an empty list, it should produce an empty graph
When building a graph from a single assembly, the graph should contain one vertex.
When building a graph with two independent assemblies, the graph should contain two vertices and there shouldn’t be any edges between them.
When building a graph with one assembly referencing another, the graph should contain two vertices and one edge
When building a graph where two assemblies have forward and backward relationships (the first item lists the second vertex as a dependency, the second item lists the first as a “referenced by”), the graph should contain unique edges between items.

By the time the team had begun to develop the third test, most of the dependent object model had been defined. The remaining tests represented implementation details. For example, to establish a relationship between assemblies we would need to store them into a lookup table. Whether this lookup table should reside within the GraphBuilder or pushed lower into the Graph itself is an optimization that can be determined later if needed. The tests would not need to change to support this refactoring effort.

Interesting Finds

The session on the fourth day involved a review of the implementation and an opportunity to refactor both the tests and the code. One of the great realizations was the ability to reduce the verbosity of initializing the test data.

We started with a lot of duplication and overhead in the tests:

[TestMethod]
public void AnExample_ItDoesntMatter_JustKeepReading()
{
    var assembly1 = new ProjectAssembly
                        {
                            FullName = "Assembly1"
                        };
    var assembly2 = new ProjectAssembly
                        {
                            FullName = "Assembly2"
                        };

    _projectList.Add(assembly1);
    _projectList.Add(assembly2);

    Graph graph = Subject.BuildGraph(_projectList);

    // Assertions...
}

We moved some of the initialization logic into a helper method, which improved readability:

[TestMethod]
public void AnExample_ItDoesntMatter_JustKeepReading()
{
    var assembly1 = CreateProjectAssembly("Assembly1");
    var assembly2 = CreateProjectAssembly("Assembly2");

    _projectList.Add(assembly1);
    _projectList.Add(assembly2);

    Graph graph = Subject.BuildGraph(_projectList);

    // Assertions...
}

private ProjectAssembly CreateProjectAssembly(string name)
{
    return new ProjectAssembly()
            {
                FullName = name
            };
}

However, once we discovered the name of the assembly wasn’t important and that they just had to be unique, we optimized this further:

[TestMethod]
public void AnExample_ItDoesntMatter_JustKeepReading()
{
    var assembly1 = CreateProjectAssembly();
    var assembly2 = CreateProjectAssembly();

    _projectList.Add(assembly1);
    _projectList.Add(assembly2);

    Graph graph = Subject.BuildGraph(_projectList);

    // Assertions...
}

private ProjectAssembly CreateProjectAssembly(string name = null)
{
    if (name == null)
        name = Guid.NewGuid().ToString();

    return new ProjectAssembly()
            {
                FullName = name
            };
}

If we really wanted to, we could optimize this further by pushing this initialization logic into the production code directly.

[TestMethod]
public void WhenConstructingAProjectAssembly_WithNoArguments_ShouldAutogenerateAFullName()
{
    var assembly = new ProjectAssembly();

    bool nameIsPresent = !String.IsNullOrEmpty(assembly.FullName);

    Assert.IsTrue( nameIsPresent,
        "Name was not automatically generated.");
}

Continue Reading: Day Five

Guided by Tests–Day Three

2011-09-09T15:03:00.001-04:00

This post is fourth in a series about a group TDD experiment to build an application in 5 days using only tests. Read the beginning here.

By day three, our collective knowledge about what we were building was beginning to shape. After reviewing the tests from the previous day, it was time for an interesting discussion: what’s next?

Determining Next Steps

In order to determine our next steps, the team turned the logical flow diagram of our use-case. We decomposed the logical flow into the following steps:

Receive input from the UI, maybe a menu control or some other UI action.
Prompt the user for a file, likely using a standard open file dialog.
Take the response from the dialog and feed it into our stream parser.
Take the output of the stream parser and build a graph
Take the graph and update the UI, likely a ViewModel.

Following a Separation of Concerns approach, we want to design our solution so that each part has very little knowledge about the surrounding parts. It was decided that we can clearly separate the building of the graph from the prompting the user part. In my view, we know very little about the UI at this point so we shouldn’t concern ourselves with how the UI initiates this activity. Instead, we can treat the prompting the user for a file and orchestrating interaction with our parser as a single concern.

It was time to split the group in two and start on different parts. Team one would focus on the code that would call the NDependStreamParser; Team two would focus on the code that consumed the list of ProjectAssembly items to produce a graph.

Note: Day Four was spent reviewing and finishing the code for team two. For the purposes of this post, I’m going to focus on the efforts of Team one.

The Next Test

The team decided that we should name this concern, “NewProjectLoader” as it would orchestrate the loading of our model. We knew that we’d be prompting for a file, so we named the test accordingly:

[TestClass]
public class NewProjectLoaderTests
{
    [TestMethod]
    public void WhenLoadingANewProject_WithAValidFile_ShouldLoadModel()
    {
        Assert.Fail();
    }
}

Within a few minutes the team quickly filled in the immediately visible details of the test.

Realization

Code Written

Following the first example from the day before, the team filled in their assertions and auto-generated the parts they needed.

To make the tests pass, they hard-coded a response.

Test Code:

var loader = new NewProjectLoader();

IEnumerable<ProjectAssembly> model 
	= loader.Load();

Assert.IsNotNull(model,
    "Model was not loaded.");

Implementation:

public IEnumerable<ProjectAssembly> Load()
{
    return new List<ProjectAssembly>();
}

How should we prompt the user for a file?

Hmmm.

Our Next Constraint

The team now needed to prompt the user to select a file. Fortunately, WPF provides the OpenFileDialog so we won’t have to roll our own dialog. Unfortunately, if we introduce it into our code we’ll be tightly coupled to the user-interface.

To isolate ourselves from this dependency, we need to introduce a small interface:

namespace DependencyViewer
{
    public interface IFileDialog
    {
        string SelectFile();
    }
}

Through tests, we introduced these changes:

Realization	Code Written
We need to introduce our File Dialog to our Loader. We decide our best option is to introduce the dependency through the constructor of the loader. This creates a small compilation error that is quickly resolved. Our tests still passes.	Test Code: IFileDialog dialog = null; var loader = new NewProjectLoader(dialog); IEnumerable<ProjectAssembly> model = loader.Load(); Assert.IsNotNull(model, "Model was not loaded."); Implementation: public class NewProjectLoader { private IFileDialog _dialog; public NewProjectLoader( IFileDialog dialog) { _dialog = dialog; } // ...
Now, how should we prompt for a file? The code should delegate to our IFileDialog and we can assume that if they select a file, the return value will not be null. The test compiles, but the test fails because the dialog is null.	Implementation: public IEnumerable<ProjectAssembly> Load() { string fileName = _dialog.SelectFile(); if (!String.IsNullOrEmpty(fileName)) { return new List<ProjectAssembly>(); } throw new NotImplementedException(); }
We don’t have an implementation for IFileDialog. So we’ll define a dummy implementation and use Visual Studio to auto-generate the defaults. Our test fails because the auto-generated code throws an error (NotImplementedException).	Test Code: IFileDialog dialog = new MockFileDialog(); var loader = new NewProjectLoader(dialog); IEnumerable<ProjectAssembly> model = loader.Load(); Assert.IsNotNull(model, "Model was not loaded.");
We can easily fix the test replacing the exception with a non-null file name.	Implementation: public class MockFileDialog : IFileDialog { public string SelectFile() { return "Foo"; } }
The test passes, but we’re not done. We need to construct a valid model. We use a technique known as “Obvious Implementation” and we introduce our NDependStreamParser directly into our Loader. The test breaks again, this time because “Foo” is not a valid filename.	Implementation: string fileName = _dialog.SelectFile(); if (!String.IsNullOrEmpty(fileName)) { using(var stream = XmlReader.Create(fileName)) { var parser = new NDependStreamParser(); return parser.Parse(stream); } } //...
Because our solution is tied to a FileStream, we need to specify a proper file name. To do this we need to modify our MockFileDialog so that we we can assign a FileName from within the test. In order to get a valid file, we need to include a file as part of the project and then enable Deployment as part of the mstest test settings. (Note: We could have changed the signature of the loader to take a filename, but we chose to keep the dependency to the file here mainly for time concerns.)	Implementation: public class MockFileDialog : IFileDialog { public string FileName; public string SelectFile() { return FileName; } } Test Code: [DeploymentItem("AssembliesDependencies.xml")] [TestMethod] public void WhenLoadingANewProject...() { var dialog = new MockFileDialog(); dialog.FileName = "AssembliesDependencies.xml"; var loader = new NewProjectLoader(dialog); //...

Isolating Further

While our test passes and it represents the functionality we want, we’ve introduced a design problem such that we’re coupled to the implementation details of the NDependStreamParser. Some may make the case that this is the nature of our application, we only need this class and if the parser’s broken so is our loader. I don’t necessarily agree.

The problem with this type of coupling is that when the parser breaks, the unit tests for the loader will also break. If we allow this type of coupling you can draw a logical conclusion that other classes will have tight coupling and thus when the parser breaks it will have a cascade effect on the majority of our tests. This defeats the purpose of our early feedback mechanism. Besides, why design our classes to be black-boxes that will have to change if we introduce different types of parsers?

The solution is to introduce an interface for our parser. Resharper makes this really easy, simply click our class and choose “Extract Interface”.

public interface IGraphDataParser
{
    IEnumerable<ProjectAssembly> Parse(XmlReader reader);
}

Adding a Mocking Framework

Whereas we created a hand-rolled mock (aka Test Double) for our IFileDialog, it’s time to introduce a mocking framework that can create mock objects in memory. Using NuGet to simplify our assembly management, we add a reference to Moq to our test project.

Refactoring Steps

We made the following small refactoring changes to decouple ourselves from the NDependStreamParser.

Realization	Code Written
Stream Parser should be a field.	Implementation: // NewProjectLoader.cs IFileDialog _dialog; NDependStreamParser _parser; public IEnumerable<ProjectAssembly> Load() { string fileName = _dialog.SelectFile(); if (!String.IsNullOrEmpty(fileName)) { using (var stream = XmlReader.Create(fileName)) { _parser = new NDependStreamParser(); return _parser.Parse(stream); } } throw new NotImplementedException(); }
We need to use the interface rather than the concrete type.	Implementation: public class NewProjectLoader { IFileDialog _dialog; IGraphDataParser _parser; // ...
We should initialize the parser in the constructor instead of the Load method.	Implementation: public class NewProjectLoader { IFileDialog _dialog; IGraphDataParser _parser; public NewProjectLoader(IFileDialog dialog) { _dialog = dialog; _parser = new NDependStreamParser(); } // ...
We should initialize the parser from outside the constructor. This introduces a minor compilation problem that requires us to change the test slightly.	Test Code: var dialog = new MockFileDialog(); var parser = new NDependStreamParser(); var loader = new NewProjectLoader(dialog, parser); Implementation: public class NewProjectLoader { IFileDialog _dialog; IGraphDataParser _parser; public NewProjectLoader( IFileDialog dialog, IGraphDataParser parser) { _dialog = dialog; _parser = parser; } // ...
We need to replace our NDependStreamParser with a mock implementation.	Test Code: var dialog = new MockFileDialog(); var paser = new Mock<IGraphDataParser>().Object; var loader = new NewProjectLoader(dialog, parser);

Strangely enough, there’s a little known feature of Moq that will ensure mocks that return IEnumerable collections will never be null, so our test passes!

Additional Tests

We wrote the following additional tests:

WhenLoadingANewProject_WithNoFileSpecfied_ShouldNotReturnAModel

Next: Day Four

Guided by Tests–Day Two

2011-09-08T08:30:00.000-04:00

This post is third in a series about a group TDD experiment to build an application in 5 days using only tests. Read the beginning here.

Today we break new ground on our application, starting with writing our first test. Today is still a teaching session, where I’ll write the first set of tests to demonstrate naming conventions and how to demonstrate how to use TDD with the rules we defined the day before. But first, we need to figure out where we should start.

Logical Flow

In order to determine where we should start it helps to draw out the logical flow of our primary use case: create a new dependency viewer from an NDepend AssembliesDependencies.xml file. The logical flow looks something like this:

User clicks “New”
The user is prompted to select a file
Some logical processing occurs where the file is read,
…a graph is produced,
…and the UI is updated.

The question on where to start is an interesting one. Given limited knowledge of what we need to build or how these components will interact, what area of the logical flow do we know the most about? What part can we reliably predict the outcome?

Starting from scratch, it seemed the most reasonable choice was to start with the part that reads our NDepend file. We know the structure of the file and we know that the contents of the file will represent our model.

Testing Constraints

When developing with a focus on testability, there are certain common problems that arise when trying to get a class under the test microscope. You learn to recognize them instantly, and I’ve jokingly referred to this as spidey-sense – you just know these are going to be problematic before you start.

While this is not a definitive list, the obvious ones are:

User Interface: Areas that involve the user-interface can be problematic for several reasons:
- Some test-runners have a technical limitation and cannot launch a user-interface based on the threading model.
- The UI may require complex configuration or additional prerequisites (style libraries, etc) and is subject to change frequently
- The UI may unintentionally require human interaction during the tests, thereby limiting our ability to reliably automate.
File System: Any time we need files or folder structure, we are dependent on the environment to be setup a certain way with dummy data.
Database / Network: Being dependent on external services is additional overhead that we want to avoid. Not only will tests run considerably slower, but the outcome of the test is dependent on many factors that may not be under our control (service availability, database schema, user permissions, existing data).

Some of the less obvious ones are design considerations which may make it difficult to test, such as tight coupling to implementation details of other classes (static methods, use of “new”, etc).

In our case, our first test would be dependent on the file system. We will likely need to test several different scenarios, which will require many different files. While we could go this route, working with the file system directly would only slow us down. We needed to find a way to isolate ourselves.

The team tossed around several different suggestions, including passing just xml as string. Ultimately, as this class must read the contents of the file we decided that the best way to work with Xml was an XmlReader. We could simulate many different scenarios by setting up a stream containing our test data.

Our First Test

So after deciding that the name of our class would be named NDependStreamParser, our first test looked something like this:

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace DependencyViewer
{
    [TestClass]
    public class NDependStreamParserTests
    {
        [TestMethod]
        public void TestMethod1()
        {
            Assert.Fail();
        }
    }
}

We know very little about what we need. But at the very least, the golden rule is to ensure that all tests must fail from the very beginning. Writing “Assert.Fail();” is a good habit to establish.

In order to help identify what we need, it helps to work backward. So we start by writing our assertions first and then, working from the bottom up, fill in the missing pieces. Our discovery followed this progression:

Realization	Code Written
At the end of the tests, we’ll have some sort of results. The results should not be null. At this point, test compiles, but it’s red.	Test Code: object results = null; Assert.IsNotNull( results, “Results were not produced.”);
Where will the results come from? We’ll need a parser. The results will come after we call Parse. The code won’t compile because this doesn’t exist. If we use the auto-generate features of Visual Studio / Resharper the test compiles, but because of the default NotImplementedException, the test fails.	Test Code: var parser = new NDependStreamParser(); object results = parser.Parse(); Assert.IsNotNull( results, “Results were not produced.”);
We need to make the test pass. Do whatever we need to make it green.	Implementation: public object Parse() { // yes, it's a dirty hack. // but now the test passes. return new object(); }
Our tests passes, but we’re clearly not done. How will we parse? The data needs to come from somewhere. We need to read from a stream. Introducing the stream argument into the Parse method won’t compile (so the tests are red), but this is a quick fix in the implementation.	Test Code: var parser = new NDependStreamParser(); var sr = new StringReader(""); var reader = XmlReader.Create(sr); object results = parser.Parse(reader); // ... Implementation: public object Parse(XmlReader reader) { //...
Our return type shouldn’t be “Object”. What should it be? After a short review of the NDepend AssembliesDependencies.xml file, we decide that we should read the list of assemblies from the file into a model object which we arbitrarily decide should be called ProjectAssembly. At a minimum, Parse should return an IEnumerable<ProjectAssembly>. There are a few minor compilation problems to address here, including the auto-generation of the ProjectAssembly class. These are all simple changes that can be made in under 60 seconds.	Test Code: var parser = new NDependStreamParser(); var sr = new StringReader(""); var reader = XmlReader.Create(sr); IEnumerable<ProjectAssembly> results = parser.Parse(reader); // ... Implementation: public IEnumerable<ProjectAssembly> Parse( XmlReader reader) { return new List<ProjectAssembly>(); }

At this point, we’re much more informed about how we’re going to read the contents from the file. We’re also ready to make some design decisions and rename our test accordingly to reflect what we’ve learned. We decide that (for simplicity sake) the parser should always return a list of items even if the file is empty. While the implementation may be crude, the test is complete for this scenario, so we rename our test to match this decision and add an additional assertion to improve the intent of the test.

Sidenote: The naming convention for these tests is based on Roy Osherove’s naming convention, which has three parts:

Feature being tested

Scenario

Expected Behaviour

[TestMethod]
public void WhenParsingAStream_WithNoData_ShouldProduceEmptyContent()
{
    var parser = new NDependStreamParser();
    var sr = new StringReader("");
    var reader = XmlReader.Create(sr);

    IEnumerable<ProjectAssembly> results =
        parser.Parse(reader);

    Assert.IsNotNull( results,
        "The results were not produced." );

    Assert.AreEqual( 0, results.Count(),
        "The results should be empty." );
}

Adding Tests

We’re now ready to start adding additional tests. Based on what we know now, we can start each test with a proper name and then fill in the details.

With each test, we learn a little bit more about our model and the expected behaviour of the parser. The NDepend file contains a list of assemblies, where each assembly contains a list of assemblies that it references and a list of assemblies that it depends on. The subsequent tests we wrote:

WhenParsingAStream_ThatContainsAssemblies_ShouldProduceContent
WhenParsingAStream_ThatContainsAssembliesWithReferences_EnsureReferenceInformationIsAvailable
WhenParsingAStream_ThatContainsAssembliesWithDependencies_EnsureDependencyInformationIsAvailable

It’s important to note that these tests aren’t just building the implementation details of the parser, we’re building our model object as well. Properties are added to the model as needed.

Refactoring

Under the TDD mantra “Red, Green, Refactor”, “Refactor” implies that you should refactor the implementation after you’ve written the tests. However, the scope of the refactor should apply to both tests and implementation.

Within the implementation, you should be able to optimize the code freely assuming that you aren’t adding additional functionality. (My original implementation details of using the XmlParser was embarrassing, and I ended up experimenting with the reader syntax later that night until I found a clean elegant solution. The tests were invaluable for discovering what was possible.)

Within the tests, refactoring means removing as much duplication as possible without obscuring the intent of the test. By the time we started the third test, the string-concatenation to assemble our xml and plumbing code to create our XmlReader was copied and pasted several times. This plumbing logic slowly evolved into a utility class that used an XmlWriter to construct our test data.

Next: Day Three

Guided by Tests–Day One

2011-09-07T09:14:00.001-04:00

This post is second in a series about a group TDD experiment to build an application in 5 days using only tests. Read the beginning here.

Today is the pitch. We'll talk about what we're going to build and how we're going to do it, but first we need to understand the motivation behind our methodology. We need to understand why we test.

Methodology

Kent Beck's TDD by Example was one of the first books I read about TDD and to this day it's still one of my favourites. I know many prefer Roy Osherove's Art of Unit Testing (which I also highly recommend) because it's newer and has .net examples, but in comparison they are very different books. Roy's book represents the evolved practice of Unit Testing and it provides a solid path to understand testing and the modern testing frameworks that have taken shape since Kent's book was written in 2002. Kent's book is raw, the primordial ooze that walked upon land and declared itself sentient, it defines the methodology as an offshoot of extreme programming and is the basis for the frameworks Roy explains how to use. If you're looking for a book on how to start and the mechanics of TDD, this is it.

As an offshoot of extreme programming, the core philosophy of TDD is:

Demonstrate working software at all times
Improve confidence while making changes
Build only the software you need

My experiment is based on the strategies defined in Kent's book – the premise is to demonstrate working software and get early feedback – but in a nutshell, you make incredibly small incremental changes and compile and run tests a lot (30-50 times per hour). I sometimes refer to this rapid code/test cycle as the Kent Beck Method or Proper TDD. I don't necessarily follow this technique when I code, but it's a great way to start until you learn how to find your balance. Eventually, running tests becomes instinct or muscle memory – you just know when you should.

The Rules

For our experiment, we followed these rules:

1. Tests must be written first.

This means that before we do anything, we start by writing a test. If we need to create classes in order to get the test to compile, that is a secondary concern: create the test first, even if it doesn’t compile, then introduce what you need.

2. Code only gets written to satisfy a test.

This means that we write only the code that is needed to make a test pass. If while writing the code you're tempted to add additional functionality that you might need, don't succumb to writing that code now. Instead, focus on the current test and make a note for additional tests. This ensures that we only write the code that is needed.

3. Code must always compile (at least within 30 seconds of making a change)

This may seem like a difficult practice to adopt, but it is extremely valuable. To follow this rule, make small changes that can be corrected easily and compile frequently to provide early feedback. If you find yourself breaking a contract that takes twenty minutes to fix, you're doing it wrong.

4. Time between red/green must be very short (<1 minute, meaning rollback if you're not sure.)

This is also a very difficult rule to keep, but it forces you to recognize the consequences of your changes. You should know before you run the test whether it's going to pass or fail. Your brain will explode when you're wrong (and that’s a good thing).

The Problem

As mentioned in the first post, my current project uses NDepend to perform static analysis as part of our build process. The build generates a static image. As our solution analyzes over 80 assemblies, the text becomes illegible, and its somewhat difficult to read (image text has been blurred to hide assembly names).

The two primary use cases I wanted the application to have:

Allow the user to select an NDepend AssembliesDependencies.xml file and display it as a graph
Once the graph has been rendered, provide an option to choose which assemblies should be displayed

The graph would be implemented using Graph#, and we could borrow much of the presentation details from Sacha Barber’s post. Using Graph# would provide a rich user-experience, all we’d need to do is implement the surrounding project framework. Piece of cake!

I asked the team if there were any additional considerations that they would like to see. We would take some of these considerations into account when designing the solution. Some suggestions were made:

Multi-tab support so that more than one graph can be open at a time
Ability to color code the nodes
Ability to save a project (and load it presumably)

Getting Started

With the few remaining minutes in the lunch hour, we set up the project:

Create a new WPF Project. Solution Name: DependencyViewer, Project Name: DependencyViewer.
Create a new Test Project, DependencyViewer.Tests
Added a reference from the Test Project to the WPF Project.
Deleted all automatically generated files:
1. App.xaml
2. MainWindow.xaml
3. TestClass1.
Renamed the default namespace of the Test Project to match the namespace of the WPF Application. For more info, read this detailed post which explains why you should.

While I have some preferences on third-party tools, etc – we’ll get to those as they’re needed.

Next: Day Two

Guided By Tests

2011-09-06T08:30:00.000-04:00

Last week I started a TDD experiment at work. I gathered some interest from management and then sought out participants with a very simple concept: build an application in 5 days, over lunch hour, using nothing but unit tests to drive development.

My next few posts will be part of a series that shares the details of that experiment.

Motivation

Over the years, I've had many conversations with colleagues about TDD. One of the most common comments made is that they don't know what to test or where to start. Another common theme is the desire to get into a project at the very beginning where all the project members have bought into the concept. I've always found this comment to be strange. It's a great comment and it makes perfect sense but -- why the beginning? Why not the middle or end of the project after the tests are put into place? I sense they're expressing two things. First they're expressing the obvious, that life for the development team would be much different if they had tests from the beginning. And secondly, they're interested in understanding how the team managed to establish and sustain the process.

The goal of my experiment would show how to start a project with TDD and to walk through the process of deconstructing requirements into testable components. The process would use tests as the delivery mechanism but would also showcase how to design software for testability using separation of concerns and SOLID OO design principles. Requirements and documentation would be deliberately vague, and the application would need to showcase common real world problems. The tricky part would be finding an application to build.

Fortunately, I was recently inspired by a post by Sacha Barber that provides a great example of how to use the open source graphing framework Graph# to build a graph application in WPF. I immediately saw how I could use this. My current project uses NDepend for static analysis and as part of the build process it produces an image of our dependency graph. With over 80 nodes and illegible text, the static image is mostly useless. However, the build process spits out the details for the graph as an XML file. With some simple xml parsing, I could use Sacha's example to build my own graph.

So rather than spending my weekend working on this application, why not turn this into a teaching exercise? (Ironically, I'm spending my weekends blogging about it)

Follow up posts:

On Dependency Injection and Violating Encapsulation Concerns

2011-08-12T08:58:00.000-04:00

For me, life is greatly simplified when dependencies are inverted and constructor injection is used to provide a clean mechanism to introduce dependencies to a class. It's explicit and easy to test.

Others however argue that some dependencies are private implementation details to the class and external callers shouldn't know about them. The argument is that exposing these dependencies through the constructor violates encapsulation and introduces coupling. This argument is usually coupled with resistance to using interfaces or classes with virtual methods.

From my experience, there are times when composition in the constructor makes sense but there's a fine line when constructor injection should be used. I want to use this post to elaborate on these arguments and provide my perspective on this debate.

Does Constructor Injection violate Encapsulation?

Does exposing the internal dependencies of a class in the constructor expose the implementation details of a class? If you are allowing callers to construct the class directly, then you are most certainly breaking encapsulation as the callers must posses the knowledge of how to construct your class. However, regardless of the constructor arguments if callers know how to construct your class you are also coupling to a direct implementation and will undoubtedly create a test impediment elsewhere. There are some simple fixes for this (including more constructor injection or another inversion of control technique) which I'll outline later.

So if constructor injection violates encapsulation, when is it safe to use composition in the constructor?

It really depends on the complexity and relationship of the subject to its dependencies. If the internal dependencies are simple and contain no external dependencies there is likely no harm in using composition to instantiate them in the constructor (the same argument can be made for small static utility methods). For instance, a utility class that performs some translation or other processing activity with limited outcomes may work as a private implementation detail. Problems arise as soon as these dependencies take on further dependencies or produce output that influences the conditional logic of the subject under test. When this happens the argument to keep these as internally controlled dependencies becomes flawed and it may be necessary to upgrade the "private implementation detail" to an inverted dependency.

Before:

This example shows a class that has a private implementation detail that influences logic of the subject under test. The consequence of this design is that the internal details of the validator leak into the test specifications. This leads to very brittle tests as tests must concern themselves with object construction and validation rules -- any change to the object or validation logic will break the tests.

public class MyModelTranslator
{
    public MyModelTranslator()
    {
       _validator = new MyModelValidator();
    }

    public MyViewModel CreateResult(MyModelObject model)
    {
        var result = new MyViewModel();
        
        if (_validator.Validate( model ));
        {
             result.Id = model.Id;
        }
        else {
            result.State = MyResultState.Invalid;
        }
        return result;
    }
}

[Test]
public void WhenCreatingAResult_FromAVaidModelObejct_ShouldHaveSameId()
{
    var subject = new MyModelTranslator();

    // create model object with deep understanding how Id's
    // and expiry dates are related 
    var model = new MyModelObject()
    {
        Id = "S1402011"
        Expiry = Datetime.Parse("12/31/2011");
    };
    var result = subject.CreateResult( model );
    Assert.AreEqual("S1402011", result.Id);
}

After:

This example shows the above example corrected to use a constructor injected dependency that can be mocked. In doing so, the test is not encumbered with object creation or validation details and can easily simulate validate states without introducing brittleness.

public class MyModelTranslator
{
    public MyModelTranslator(MyModelValidator validator);
    {
       _validator = validator;
    }

    // ...
}

[Test]
public void WhenCreatingAResult_FromAVaidModelObejct_ShouldHaveSameId()
{
    var validatorMock = new Mock<MyModelValidator>();

    var subject = new MyModelTranslator(validatorMock.Object);
    var model = new MyModelObject()
    {
      Id = "Dummy"
    };

    // we no longer care how validation is done, 
    // but we expect validation to occur and can easily control
    // and test outcomes
    ValidatorMock.Setup( x => x.Validate( model )).Returns( true );

    var result = subject.CreateResult( model );
    Assert.AreEqual("Dummy", result.Id);
}

Combatting Coupling with Inversion of Control

As the above points out, if we expose the dependencies of a class in the constructor we move the coupling problems out of the class and into the callers. This is not good but it can easily resolved.

An example that shows that our controller class now knows about the MyModelValidator:

public class MyController
{
    public MyController()
    {
        _translator = new MyModelTranslator( new MyModelValidator() );
    }
}

More Constructor Injection

Ironically, the solution for solving coupling and construction problems is more constructor injection. This creates a Russian Doll where construction logic is deferred and pushed higher and higher up the stack resulting in a top level component which is responsible for constructing the entire object graph. While this provides an intuitive API that clearly outlines dependencies, it's tedious to instantiate by hand. This is where dependency injection frameworks like Unity, StructureMap and others come in: they can do the heavy lifting for you. If done right, your application should only have well defined points where the dependency container is used.

public class MyController : IController
{
    public MyController(MyModelTranslator translator)
    {
        _translator = translator;
    }
}

Note that this assumes that the constructor arguments are abstractions -- either interfaces, abstract classes or classes with virtual methods. In effect, by exposing the constructor arguments we trade off the highly sealed encapsulated black box for an open for extensibility model.

Factories / Builders

In some cases, using Constructor Injection everywhere might not be a good fit. For example, if you had a very large and complex object graph, creating everything upfront using constructor injection might represent a performance problem. Likewise, not all parts of the object graph will be used immediately and you may need to defer construction until needed. In these cases, the good ol' Gang of Four Factory pattern is a handy mechanism to lazy load components.

So rather that construct the entire object graph and pass resolved dependencies as constructor arguments, pass the factory instead and create the lower-level sub-dependencies when needed.

public class MyControllerFactory : IControllerFactory
{
    private IUnityContainer _container;
    
    public MyControllerFactory(IUnityContainer container)
    {
        _container = container;
    }

    public IController Create()
    {
        _container.Resolve<MyController>();
    }
}

Service Location

While Service Location is an effective mechanism for introducing inversion of control, I'm listing it last for a reason. Unlike Constructor Injection, Service Location couples all your code to an intermediate provider. Depending on your application and relative complexity this might be acceptable, but from personal experience it's a very slippery slope -- actually, it would be more appropriate to call it a cliff because removing or replacing the container from an existing codebase can be a massive undertaking.

I see service location as a useful tool when refactoring legacy code towards Constructor Injection. The lower "leaf-nodes" of the object graph can take advantage of constructor injection and the higher nodes can temporarily use service location to create the "leaves". As you refactor, the service locator is removed and replaced with constructor injection, which results in an easier to test and discoverable API. This also provides a pragmatic bottom up refactor approach.

Summary

Use "new" when dependencies have no external dependencies or don't influence conditional flow of the subject. Conversely, use constructor injection when dependencies have additional dependencies.
Use an IoC container to construct objects that use constructor injection, but only use the IoC container in high level components that are responsible for construction of the object graph. If constructing the entire object graph is a performance concern, consider encapsulating the container in a Factory that can be used to resolve remaining parts of the object graph when needed.
To prevent violating encapsulation concerns, use inversion of control to promote using abstractions instead of concrete dependencies.

So You've Decided To Go "Full Retard"!

2011-07-14T10:01:00.000-04:00

Congratulations! You've done something very clever! Your out of the box thinking has now retarded the development process!

Oh? You're not sure what I'm talking about? Well remember that thing you did? I'm pretty sure you knew that it felt like a hack when you wrote it but then you quickly convinced yourself that you'd invented something insanely brilliant. Well, it wasn't. It was a hack. And now, somewhere, kittens are dying.

Yes, I know the compiler didn't complain but that doesn't technically make it valid code. Just because you can use operator overloading to concatenate files doesn't mean you should. Oh yeah and remember that non-standard event signature you implemented to save time? Well it turns out that we really did need event arguments and proper error handling, so now Jimmy, who is replacing you btw, is going to have to rewrite it. kthxbai.

It's not clever, or agile or lean or whatever it is you think it is. It's sloppy. I can't think of a profession where cutting corners is okay.

So before you start bringing the protestors to your aid that "retarded" is an offensive word, I mean it for the true sense, for slowing down the development or progress of action, process, etc. Anytime a developer squints and wonders what was going through your mind -- you've failed to help others understand your code. And know what the funny thing is? It doesn't take much more to do it right. It might take an few extra seconds to put xml comments at the top of the method, or a few extra minutes to write a unit test. If you consider that we spend more time trying to understand code than writing it, it really makes sense to ensure that code remains consistent.

For me, clever is often synonymous with stupid. Everyone's entitled to a few embarrassing "clever" moments a year as long as the own up to it. Going full in and swearing by it, well, …sometimes you come back empty handed.

Visual Studio Regular Expressions for Find & Replace

2011-07-12T08:46:00.000-04:00

Visual Studio has had support for regular expressions for Find & Replace for several versions, but I've only really used it for simple searches. I recently had a problem where I needed to introduce a set of changes to a very large object model. It occurred to me that this could be greatly simplified with some pattern matching, but I was genuinely surprised to learn that Visual Studio had their own brand of Regular Expressions.

After spending some time learning the new syntax I had a really simple expression to modify all of my property setters:

Original:

public string PropertyName
{
    get { return _propertyName; }
    set
    {
        _propertyName = value;
        RaisePropertyChanged("PropertyName");
    }
}

Goal:

public string PropertyName
{
    get { return _propertyName; }
    set
    {
        if ( value == _propertyName )
             return;            
        _propertyName = value;
        RaisePropertyChanged("PropertyName");
    }
}

Here’s a quick capture and breakdown of the pattern I used.

Find:

^{:Wh*}<{_:a+} = value;

^ = beginning of line
{ = start of capture group #1
:Wh = Any whitespace character
* = zero or more occurrences
} = end of capture group #1
< = beginning of word
{ = start of capture group #2
_ = I want to the text to start with an underscore
:a = any alpha numerical character
+ = 1 or more alpha numerical characters
} end of capture group #2
“ = value;” = exact text match

Replace:

\1(if (\2 == value)\n\1\t\return;\n\1\2 = value;

The Replace algorithm is fairly straight forward, where “\1” and “\2” represent capture groups 1 and 2. Since capture group #1 represents the leading whitespace, I’m using it in the replace pattern to keep the original padding and to base new lines from that point. For example, “\n\1\t” introduces a newline, the original whitespace and then a new tab.

It’s seems insane that Microsoft implemented their own regular expression engine, but there’s some interesting things in there, such as being able to match on quoted text, etc.

I know this ain’t much, but hopefully it will inspire you to write some nifty expressions. Cheers.

Build Server Code Analysis Settings

2011-06-29T09:06:00.000-04:00

I've been looking at ways to improve the reporting of Code Analysis as part of our Team Build. During my research I found that the RunCodeAnalysis setting as defined in the TFSBuild.proj differs significantly from the local MSBuild project schema options (see this post for a detailed break down of project level analysis settings).

Specifically, TFSBuild defines RunCodeAnalysis as "Always", "Default" and "Never" while MSBuild defines this as a simple true/false Boolean. To make sense of this I hunted this down to this section of the Microsoft.TeamFoundation.Build.targets file:

<Target Name="CoreCompileSolution">

  <PropertyGroup>
    <CodeAnalysisOption Condition=" '$(RunCodeAnalysis)'=='Always'">RunCodeAnalysis=true</CodeAnalysisOption>
    <CodeAnalysisOption Condition=" '$(RunCodeAnalysis)'=='Never'">RunCodeAnalysis=false</CodeAnalysisOption>
    <!-- ... -->
  </PropertyGroup>
  <!-- ... -->
</Target>

From this we can infer that "Default" setting does not provide a value to the runtime, while "Always" and "Never" map to True/False respectively. But this raises the question, what's the difference between Default and Always, and what should I specify to effectively run Code Analysis?

To answer this question, let's take an example of a small project with two projects. Both projects are configured in the csproj with default settings. For the purposes of the demo, I've altered the output folder to be at a folder called Output at the root of the solution:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  <DebugSymbols>true</DebugSymbols>
  <DebugType>full</DebugType>
  <Optimize>false</Optimize>
  <OutputPath>..\Output</OutputPath>
  <DefineConstants>DEBUG;TRACE</DefineConstants>
  <ErrorReport>prompt</ErrorReport>
  <WarningLevel>4</WarningLevel>
</PropertyGroup>

When we specify code analysis to run with the "Always" setting:

Msbuild.exe Example.sln /p:RunCodeAnalysis=True

The Output folder contains the following files:

CodeAnalysisExperiment1.dll 
CodeAnalysisExperiment1.dll.CodeAnalysisLog.xml 
CodeAnalysisExperiment1.dll.lastcodeanalysissucceeded 
CodeAnalysisExperiment1.pdb 
CodeAnalysisExperiment2.dll 
CodeAnalysisExperiment2.dll.CodeAnalysisLog.xml 
CodeAnalysisExperiment2.dll.lastcodeanalysissucceeded 
CodeAnalysisExperiment2.pdb

Close inspection of the console output shows that the Minimal ruleset was applied to my empty projects, despite having not configured either project for code analysis.

If I tweak the project settings slightly such that one project explicitly declares RunCodeAnalysis as True and the other as False

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' "> 
  <DebugSymbols>true</DebugSymbols> 
  <DebugType>full</DebugType> 
  <Optimize>false</Optimize> 
  <OutputPath>..\Output</OutputPath> 
  <DefineConstants>DEBUG;TRACE</DefineConstants> 
  <ErrorReport>prompt</ErrorReport> 
  <WarningLevel>4</WarningLevel> 
  <RunCodeAnalysis>false</RunCodeAnalysis> 
</PropertyGroup>

and…

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' "> 
  <DebugSymbols>true</DebugSymbols> 
  <DebugType>full</DebugType> 
  <Optimize>false</Optimize> 
  <OutputPath>..\Output</OutputPath> 
  <DefineConstants>DEBUG;TRACE</DefineConstants> 
  <ErrorReport>prompt</ErrorReport> 
  <WarningLevel>4</WarningLevel> 
  <RunCodeAnalysis>true</RunCodeAnalysis> 
</PropertyGroup>

...and then specify the "Default" setting:

msbuild Example.sln

The Output folder now contains the following files:

CodeAnalysisExperiment1.dll 
CodeAnalysisExperiment1.dll.CodeAnalysisLog.xml 
CodeAnalysisExperiment1.dll.lastcodeanalysissucceeded 
CodeAnalysisExperiment1.pdb 
CodeAnalysisExperiment2.dll 
CodeAnalysisExperiment2.pdb

From this we can infer that the "Default" setting looks to the project settings to determine if analysis should be done, whereas the "Always" setting will override the analysis and "Never" will disable analysis altogether.

For my purposes, I have some projects (unit tests, etc) that I don't necessarily want to run code analysis on -- for me, "Always" is a bit of a deal breaker. However, there's an interesting lesson to take away here: command line arguments passed to msbuild are carried to all targets. Which means we can override most project settings this way:

// use 'all rules' for projects that have analysis turned on 
msbuild.exe Example.sln /p:CodeAnalysisRuleSet="Allrules.set"

// use a custom dictionary during code analysis 
msbulid.exe Example.sln /p:CodeAnalysisDictionary="..\AnalysisDictionary.xml"

// redirect output to a different folder 
msbulid.exe Example.sln /p:OutputPath="..\Output2"

Cheers!

Happy Coding.

The Mystery of Cascading FxCop Errors

2011-06-28T08:59:00.000-04:00

As I mentioned in my last post, I've been working through FxCop concerns on my current project. To provide some context, this is a large project with more than 500K lines of code, and FxCop was added late to the development process. Naturally, out of the gate there were thousands of errors, so we had to ease it into the development process by slowly turning on rules until we had the warnings under control.

To facilitate this, we had a dedicated team go through the entire code base and address as many of the errors as they could: implementing IDispose correctly, validating arguments, etc. In the end, we had a hand picked set of rules that applied to our scenarios and a code base that adhered to them. Once the existing violations were under control, we flipped the switch to treat new analysis warnings as errors to ensure that the violations were not introduced into source control.

There were some bumps along the way, but in the end it was manageable (royal pain in the butt, but manageable).

Recently however, my team started to see serious problems with code analysis throughout parts of the application. They found that if they made a minor change, a series of FxCop violations would appear -- even in areas that they didn't touch or had previously been reviewed and considered "warning free". The team believed that FxCop was somehow incrementally analyzing files, thereby finding new problems which had previously been skipped. While this theory fit into the symptom of new violations for existing code, it just didn't seem right: FxCop analyzes compiled assemblies, not individual files.

The problem with code analysis errors is that they're not always obvious what is causing the problem. It's easy to fall into a trap to believe that Code Analysis is only looking at the internal structure or logic of a class for correctness. There are many FxCop rules that look for correctness, such as Methods that can be private or static, arguments that can be null, etc. The really obscure errors are the ones that look at how a class is used by other classes. What may seem like a trivial change can introduce a different execution path that suddenly exposes the class and its dependencies for consideration of new FxCop violations.

Let's take a look at a contrived example. Let's say I'm using a Service Locator to wire up a ViewModel and like any good Inversion of Control junkie, I'm using an interface for that ViewModel. According to FxCop, the following example is good and life is fine.

public class CustomApp : Application 
{ 
    public override void OnStartup() 
    { 
        var view = new MainView(); 
        view.DataContext = new MainViewModel(); 
        view.Show(); 
    } 
}

internal class MainViewModel 
{ 
    public ISearchViewModel SearchViewModel { get; set;}

    public MainViewModel() 
    { 
        SearchViewModel = ServiceLocator.Get<ISearchViewModel>(); 
    } 
}

As smart developers, we decide that we don't really need the ServiceLocator to get our SearchViewModel. The concrete SearchViewModel doesn't have any complex construction logic and we probably will never swap it out with a different instance at runtime -- this is a little hit against dependency inversion but from a testing perspective, I can always introduce a Mock through property injection. Upon introducing this change we discover several new FxCop violations that weren't present before -- nearly a dozen warnings in unrelated classes about objects not being properly disposed. Out of all the warnings, only one warning is specific to the change we've made in this class but it doesn't seem obvious: CA2000: Dispose objects before losing scope.

public class MainViewModel 
{ 
    public ISearchViewModel SearchViewModel { get; set;}

    public MainViewModel() 
    { 
        // FxCop: CA2000 - Dispose object before losing scope
        SearchViewModel = new PrimarySearchViewModel();
    } 
}

This violation takes a few minutes to sort out because we're assigning the object to a property, and it isn't going out of scope. Upon further inpsection, it seems that the concrete class implements IDisposable, but the interface does not. FxCop is warning us that once the constructor goes out of scope, our concrete class will become polymorphically assigned to an interface that doesn't support IDispose and as such, we won't have an opportunity to call the Dispose method on the concrete class. Clever girl.

The easy fix is to add IDispose onto the interface and then implement the dispose pattern for the MainViewModel. All of this makes sense, and we've just fixed a memory leak that was there all along. But why did we have violations in all these other classes?

It turns out that PrimarySearchViewModel implemented IDispose improperly and it also constructed several other IDisposable types with the same problem in its constructor. These violations, like the PrimarySearchViewModel one, are legitimate. Now, if you're following along, you'll realize I haven't answered the question. If all these classes implemented IDispose improperly, why weren't these found earlier?

The answer is because we changed the usage. No one was calling the constructor previously -- the ServiceLocator used reflection to hunt down and construct our object for us, but by switching to use the constructor directly the analysis engine did it's part to highlight problems. It's hard to say whether the analysis engine should have caught this potential problem or if it's a feature that only inspects currently active code. At any rate, I hope this help you or your team -- before dismissing FxCop violations as fluke, look at how the usage of your objects has changed.

Visual Studio Code Analysis Settings

2011-06-27T08:55:00.000-04:00

I've just spent the last week or so working with Code Analysis for my Visual Studio 2010 projects. I still haven't figured out if I'm using Visual Studio's built in Code Analysis tool or FxCop 10.0, because the two tools share a lot of the same engine and rulesets. At any rate, one of the pain points I had to work through was ensuring that the configuration settings between my Visual Studio configurations were the same. Strangely enough, I had more confidence in my changes when manually editing the csproj files in Notepad++ than from Visual Studio -- it seemed like each configuration had slightly different settings, and being able to see the msbuild settings seemed to make more sense.

During this process, I found it quite frustrating that none of the CodeAnalysis settings that are configurable from the csproj schema were documented online. However, the intellisense for the schema is quite good. The schema is located at: <Program Files>\Microsoft Visual Studio 10.0\Xml\Schemas\1033\MSBuild\Microsoft.Build.CommonTypes.xsd

Most of the settings generated by Visual Studio are often the default values and can be discarded.

For my reference, here's the documentation, lifted from the schema:

Project Element	Description
CodeAnalysisAdditionalOptions	Additional options to pass to the Code Analysis command line tool.
CodeAnalysisApplyLogFileXsl	Indicates whether to apply the XSL style sheet specified in $(CodeAnalysisLogFileXsl) to the Code Analysis report. This report is specified in $(CodeAnalysisLogFile). The default is false.
CodeAnalysisConsoleXsl	Path to the XSL style sheet that will be applied to the Code Analysis console output. The default is an empty string (''), which causes Code Analysis to use its default console output.
CodeAnalysisCulture	Culture to use for Code Analysis spelling rules, for example, 'en-US' or 'en-AU'. The default is the current user interface language for Windows.
CodeAnalysisDependentAssemblyPaths	Additional reference assembly paths to pass to the Code Analysis command line tool. A fully qualified path to a directory containing reference assemblies to be used by Code Analysis.
CodeAnalysisDictionary	Code Analysis custom dictionaries. Semicolon-separated list of Code Analysis custom dictionaries. Wildcards are allowed.
CodeAnalysisFailOnMissingRules	Indicates whether Code Analysis should fail if a rule or rule set is missing. The default is false.
CodeAnalysisForceOutput	Indicates whether Code Analysis generates a report file, even when there are no active warnings or errors. The default is true.
CodeAnalysisGenerateSuccessFile	Indicates whether Code Analysis generates a '$(CodeAnalysisInputAssembly).lastcodeanalysissucceeded' file in the output folder when no build-breaking errors occur. The default is true.
CodeAnalysisIgnoreBuiltInRules	Indicates whether Code Analysis will ignore the default rule directories when searching for rules. The default is false.
CodeAnalysisIgnoreBuiltInRuleSets	Indicates whether Code Analysis will ignore the default rule set directories when searching for rule sets. The default is false.
CodeAnalysisIgnoreInvalidTargets	Indicates whether Code Analysis should silently fail when analyzing invalid assemblies, such as those without managed code. The default is true.
CodeAnalysisIgnoreGeneratedCode	Indicates whether Code Analysis should fail silently when it analyzes invalid assemblies, such as those without managed code. The default is true.
CodeAnalysisImport	Code Analysis projects (.fxcop) or reports to import. Semicolon-separated list of Code Analysis projects (.fxcop) or reports to import. Wildcards are allowed.
CodeAnalysisInputAssembly	Path to the assembly to be analyzed by Code Analysis. The default is '$(OutDir)$(TargetName)$(TargetExt)'.
CodeAnalysisLogFile	Path to the output file for the Code Analysis report. The default is '$(CodeAnalysisInputAssembly).CodeAnalysisLog.xml'.
CodeAnalysisLogFileXsl	Path to the XSL style sheet to reference in the Code Analysis output report. This report is specified in $(CodeAnalysisLogFile). The default is an empty string ('').
CodeAnalysisModuleSuppressionsFile	Name of the file, without the path, where Code Analysis project-level suppressions are stored. The default is 'GlobalSuppressions$(DefaultLanguageSourceExtension)'.
CodeAnalysisOverrideRuleVisibilities	Indicates whether to run all overridable Code Analysis rules against all targets. This will cause specific rules, such as those within the Design and Naming categories, to run against both public and internal APIs, instead of only public APIs. The default is false.
CodeAnalysisOutputToConsole	Indicates whether to output Code Analysis warnings and errors to the console. The default is false.
CodeAnalysisVerbose	Indicates whether to output verbose Code Analysis diagnostic info to the console. The default is false.
CodeAnalysisPath	Path to the Code Analysis installation folder. The default is '$(VSINSTALLDIR)\Team Tools\Static Analysis Tools\FxCop'.
CodeAnalysisPlatformPath	Path to the .NET Framework folder that contains platform assemblies, such as mscorlib.dll and System.dll. The default is an empty string ('').
CodeAnalysisProject	Path to the Code Analysis project (*.fxcop) to load. The default is an empty string ('').
CodeAnalysisQuiet	Indicates whether to suppress all Code Analysis console output other than errors and warnings. This applies when $(CodeAnalysisOutputToConsole) is true. The default is false.
CodeAnalysisRuleAssemblies	Semicolon-separated list of paths either to Code Analysis rule assemblies or to folders that contain Code Analysis rule assemblies. The paths are in the form '[+\|-][!][file\|folder]', where '+' enables all rules in rule assembly, '-' disables all rules in rule assembly, and '!' causes all rules in rule assembly to be treated as errors. For example '+D:\Projects\Rules\NamingRules.dll;+!D:\Projects\Rules\SecurityRules.dll'. The default is '$(CodeAnalysisPath)\Rules'.
CodeAnalysisRuleDirectories	Semicolon-separated list of directories in which to search for rules when resolving a rule set. The default is '$(CodeAnalysisPath)\Rules' unless the CodeAnalysisIgnoreBuiltInRules property is set to true.
CodeAnalysisRules	Semicolon-separated list of Code Analysis rules. The rules are in the form '[+\|-][!]Category#CheckId', where '+' enables the rule, '-' disables the rule, and '!' causes the rule to be treated as an error. For example, '-Microsoft.Naming#CA1700;+!Microsoft.Naming#CA1701'. The default is an empty string ('') which enables all rules.
CodeAnalysisRuleSet	A .ruleset file which contains a list of rules to run during analysis. The string can be a full path, a path relative to the project file, or a file name. If a file name is specified, the CodeAnalysisRuleSetDirectories property will be searched to find the file. The default is an empty string ('').
CodeAnalysisRuleSetDirectories	Semicolon-separated list of directories in which to search for rule sets. The default is '$(VSINSTALLDIR)\Team Tools\Static Analysis Tools\Rule Sets' unless the CodeAnalysisIgnoreBuiltInRuleSets property is set to true.
CodeAnalysisSaveMessagesToReport	Comma-separated list of the type ('Active', 'Excluded', or 'Absent') of warnings and errors to save to the output report file. The default is 'Active'.
CodeAnalysisSearchGlobalAssemblyCache	Indicates whether Code Analysis should search the Global Assembly Cache (GAC) for missing references that are encountered during analysis. The default is true.
CodeAnalysisSummary	Indicates whether to output a Code Analysis summary to the console after analysis. The default is false.
CodeAnalysisTimeout	The time, in seconds, that Code Analysis should wait for analysis of a single item to complete before it aborts analysis. Specify 0 to cause Code Analysis to wait indefinitely. The default is 120.
CodeAnalysisTreatWarningsAsErrors	Indicates whether to treat all Code Analysis warnings as errors. The default is false.
CodeAnalysisUpdateProject	Indicates whether to update the Code Analysis project (*.fxcop) specified in $(CodeAnalysisProject). This applies when there are changes during analysis. The default is false.
CodeAnalysisUseTypeNameInSuppression	Indicates whether to include the name of the rule when Code Analysis emits a suppression. The default is true.
RunCodeAnalysis	Indicates whether to run Code Analysis during the build.

As this has been a fun week dealing with FxCop, expect to see a few more posts related to the above settings. Stay right here and click refresh endlessly, or subscribe to the rss feed.

Happy coding.

The Tests are Broken, Now What?

2011-05-31T23:04:00.001-04:00

Despite our best intentions to write durable tests, it seems inevitable that tests will break at some point. In some cases, breaking a test can be seen as a very positive thing: we've introduced a side-effect that has broken existing functionality, and our tests have helped prevent a bug. However, in some cases, tests break because of external factors -- maybe the tests weren't isolated enough, maybe a developer went rogue and forgot to run the tests, or some other poorly justified or lousy excuse. While identifying what went wrong in your development process is important, the immediate concern is that these broken tests represent a failed build and developer productivity is at risk -- what should you do with the broken tests?

If the developer looking at the tests knows a bit about the code, they should have enough information in the tests to help sort things out. The developer should drop what they're doing, slow down and research the code and the tests and fix the broken tests. If caught early, it's very manageable. Unfortunately, this isn't the common case -- the developer introducing the breaking changes doesn't understand the tests and is ultimately left with one of two approaches:

Comment out or remove the test
Mark the test as Ignored

Neither of these approaches "fix" anything. The first approach, to comment the test out, is equivalent to pissing on the requirements. The test supposedly represents a piece of functional behaviour of the application, the fact that it's broken suggests a feature is also broken. The only reason to comment out a test is if that functionality has ceased to exist or the test was invalid in the first place. (Take a look at my checklist for unit test code reviews to help qualify invalid tests)

The second alternative to mark the tests as Ignored can be dangerous, depending on whether you're able to track why the test has been disabled and able to monitor when it enters this state. This is one part developer process, one part testing framework, one part reporting.

Within NUnit, the Ignore attribute fits the bill nicely. The attribute accepts a string message which is used to report why the test has been disabled. It should be standard developer process to only allow use of Ignore if a message is provided. (Come to think of it, I should put in a feature request so that it is required.) When NUnit runs, the Ignored tests are not executed but the number of ignored tests are included in the xml output, meaning that your build server can track ignored tests over time. You can also bypass the Ignored attribute within the NUnit user-interface by simply selecting the Ignored test and running it manually.

[Ignore("The xml for this test isn't serializing correctly. Please fix!")]
[Test]
public void SuperImportantTest()
{
    // snip
}

Within MSTest however, Ignore is essentially the same as commenting out the tests. There's no ability to record why the test has been excluded (developers must leave comments), and when MSTest runs the test suite the Ignored tests are simply excluded from the test run. Looking at the MSTest output, the TRX file does not define a status for ignored, meaning that it can't be tracked in your build reports.

<ResultSummary outcome="Completed">
  <Counters 
    total="1" 
    executed="1" 
    passed="1" 
    error="0" 
    failed="0" 
    timeout="0" 
    aborted="0" 
    inconclusive="0" 
    passedButRunAborted="0" 
    notRunnable="0" 
    notExecuted="0" 
    disconnected="0" 
    warning="0" 
    completed="0" 
    inProgress="0" 
    pending="0" />
</ResultSummary>

Though I'm not a big fan of this, I'll spare you the complaining and move on to working solutions.

The best approach within MSTest isn't to disable the test but to indicate that it isn't reliable and needs attention. In addition, we need these tests to stand out without breaking the build. This is exactly what Inconclusive is for.

[TestMethod] 
public void SuperImportantTest() 
{ 
    Assert.Inconclusive("The xml for this test isn't serializing correctly. Please fix!"); 
    
    // snip 
}

The key to using Assert.Inconclusive is to put the statement as the first code-expression in your test. When MSTest encounters the Assert.Inconclusive statement, execution of the test is immediately halted, and the test is recorded as "Inconclusive". The TRX reports this status separately, and the message appears in the xml output.

Next Steps

With the tests marked as problematic, it’s possible to check-in the tests and unblock developers from a failed build while you and the original test author figure out how to fix the problem. It’s a temporary patch and is not intended for all breaking tests, just small fires that happen during critical developer crunches. Once the fire has been put out and the tests corrected, you really should go back and investigate why the test broke in the first place:

Should the developer’s changes have broken this test? If not, could the code/test have been written differently to be more durable?
What could the developer have done differently? Is this a one time occurrence or a disconnect in process?

That’s all for now.

Testing Asynchronous Logic

2011-04-18T08:58:00.000-04:00

I've been doing a fair bit of asynchronous logic recently where I'm designing objects to raise events after long-running operations have completed, and naturally this creates a few challenges during testing. When a method uses the asynchronous pattern, the work is executed on a separate thread and execution returns immediately to the caller. This is a problem for the a unit test as they are by design a series of standalone sequential steps -- the test may finish or execute assertions before the event is called.

The immediate workaround is to add delays into the tests using a Thread.Sleep statement.

[Test]
public void AsynchronousTestsDontAlwaysWork()
{
    bool eventCalled = false;
    
    var subject = new MyClass();
    subject.OnComplete += (o,e) => eventCalled = true;
    
    subject.DoWorkAsync();
    
    Thread.Sleep(100);
    
    Assert.AreTrue( eventCalled ); // can fail because the event might raise after this assertion
}

This strategy will work, but it's dangerous in the long-term because the duration of the asynchronous operation is dependent on the complexity of the implementation and the environment the test is executing in. In other words, you may need to increase sleep interval if the implementation needs to do more work or if the tests fail on slower machines. In the end, the sleep interval reflects the lowest common denominator in your environment (that slow laptop that intern uses) and the entire test run becomes painfully (and unnecessarily) slow.

The solution is to add a little extra plumbing to block execution until the event is fired.

[Test]
public void AsynchronousTestWithBlockingAlwaysWorks()
{
    var reset = new System.Threading.ManualResetEvent(false);
    
    var subject = new MyClass();
    subject.OnComplete += (o,e) => reset.Set();
    
    subject.DoWorkAsync();
    
    if (!reset.WaitOne(1000)) // block for at most 1 second
    {
        Assert.Fail("The test timed out waiting for the OnComplete event.");
    }
}

The above example uses the ManualResetEvent, which in short is a signalling mechanism. The WaitOne method blocks execution until another thread calls the Set method or the timeout value is exceeded. In the above example, I've set up an anonymous delegate for my event so that the test finishes as soon as the asynchronous logic completes.

Cleaning it up...

As I mentioned earlier, I've been doing a lot of this type of testing so the above pattern has appeared in more than one test. It would be nice to encapsulate this a bit and streamline it into a reusable utility. This post on StackOverflow had an interesting concept to wrap the blocking logic into an object that supports IDisposable. The original poster was attempting to use Expressions to access events, which sadly isn't possible from outside the class, but here's an updated example that uses reflection and a named parameter for the event:

public sealed class EventWatcher : IDisposable
{
    private readonly object _target;
    private readonly EventInfo _eventInfo;
    private readonly Delegate _listener;
    private readonly ManualResetEvent _reset;
    private readonly int _timeout;

    public static EventWatcher Create<T>(T target, string eventName, int timeout = 1000)
    {
        EventInfo eventInfo = typeof (T).GetEvent(eventName);
        if (eventInfo == null)
            throw new ArgumentException("Event not found.", "eventName");

        return new EventWatcher(target, eventInfo, timeout);
    }

    private EventWatcher(object target, EventInfo eventInfo, int timeout)
    {
        _target = target;
        _eventInfo = eventInfo;
        _timeout = timeout;

        _reset = new ManualResetEvent(false);

        // Create our event listener
        _listener = CreateEventListenerDelegate(eventInfo.EventHandlerType);

        _eventInfo.AddEventHandler(target, _listener);
    }

    public void SetEventWasRaised()
    {
        _reset.Set();
    }

    private Delegate CreateEventListenerDelegate(Type eventType)
    {
        // Create the event listener's body, setting the 'eventWasRaised_' field.
        var setMethod = typeof(EventWatcher).GetMethod("SetEventWasRaised");
        var body = Expression.Call(Expression.Constant(this), setMethod);

        // Get the event delegate's parameters from its 'Invoke' method.
        MethodInfo invokeMethod = eventType.GetMethod("Invoke");
        var parameters = invokeMethod.GetParameters()
            .Select((p) => Expression.Parameter(p.ParameterType, p.Name));

        // Create the listener.
        var listener = Expression.Lambda(eventType, body, parameters);
        return listener.Compile();
    }

    public void Dispose()
    {
        bool eventWasFired = _reset.WaitOne(_timeout);
        // Remove the event listener.
        _eventInfo.RemoveEventHandler(_target,_listener);

        if (!eventWasFired)
        {
            string message = String.Format("Test timed-out waiting for event \"{0}\" to be raised.", _eventInfo.Name);
            Assert.Fail(message);
        }
    }
}

[Test]
public void DemonstrateEventWatcherWithUsingBlockAndDisposePattern()
{
    var subject = new MyClass();
    
    using(EventWatcher.Create(subject, "OnComplete"))
    {
        subject.DoWorkAsync();
        
    } // throws Assert.Fail() if event wasn't called.
}

Some Refinements...

This solution works, but there are a few drawbacks:

The using block is a very succinct mechanism for controlling scope, but unfortunately if an exception occurs in the body of the using statement it will get swallowed by the exception in our EventWatcher's Dispose method.
This only validates that the event was called, and it doesn't give us access to the event argument, which might be of interest to the test.
The event name is a literal string value which isn't very refactor friendly.

Preventing Hidden Exceptions from the Finally block

To address the first problem where exceptions aren't properly handled inside the using statement, we change the signature slightly to perform the test logic in an Action.

// snip...
public static void WaitForEvent<T>(T target, string eventName, Action action) 
{ 
    var watcher = Create(target, eventName);

    try 
    { 
        action(); 
    } 
    catch (Exception) 
    { 
        watcher.Clear(); 
        throw; 
    } 
    finally 
    { 
        watcher.Dispose(); 
    } 
}
// ...end snip

[Test] 
public void DemonstrateEventWatcherWithAnActionInsteadOfUsingBlock() 
{ 
    EventWatcher.WaitForEvent(subject, "OnComplete", () => subject.DoWorkAsync()); 
}

Gaining Access to the EventArgs

Addressing the second problem requires some changes to the CreateEventListenerDelegate so that the arguments of the event are recorded.

//snip
public void SetEventWasRaised(EventArgs e) 
{ 
    EventArg = e; 
    _reset.Set(); 
}

public EventArgs EventArg { get; private set; }

private Delegate CreateEventListenerDelegate(Type eventType) 
{ 
    // Get the event delegate's parameters from its 'Invoke' method. 
    MethodInfo invokeMethod = eventType.GetMethod("Invoke"); 
    var parameters = invokeMethod.GetParameters() 
        .Select((p) => Expression.Parameter(p.ParameterType, p.Name)).ToList();

    // Setup the callback to pass in the EventArgs 
    var setMethod = typeof(EventWatcher).GetMethod("SetEventWasRaised"); 
    var body = Expression.Call(Expression.Constant(this), setMethod, parameters[1]);

    // Create the listener. 
    var listener = Expression.Lambda(eventType, body, parameters); 
    return listener.Compile(); 
} // end snip

[Test] 
public void DemonstrateEventWatcherCanCaptureEventArgs() 
{ 
    EventArgs e; 
    EventWatcher.WaitForEvent(subject, "OnComplete", out e, t => t.DoWorkAsync()); 
    
    Assert.IsNotNull(e); 
}

Making it Refactor-Friendly...

Tackling the last problem where we don't have type-safety for our event, we need a special workaround because we're unable to use an Expression to capture our event. From a language perspective, C# will only allow external callers to add or remove event handler assignments, so we can't write a lamda expression that points to an event reference. Hopefully this is something that will be added to the language in a future version, but until then accessing the event requires some special hackery.

Taking a play from the Moq source code, we can infer the event name by using a dynamic proxy and intercepting the method invocation when we perform an event assignment. It's a long and awkward way around, but it works, albeit it comes with all the caveats of using a dynamic proxy: non-sealed classes with virtual events, MarshalByRefObject, or interfaces.

A note about virtual events: I’m personally not a big fan of virtual events (Resharper warnings, etc), but if you’re comfortable putting the event declarations on an interface, you can explicitly specify the interface in the generic type argument. EventWatcher.Create<IMyInterface>(…)

public class EventAssignmentInterceptor : Castle.DynamicProxy.IInterceptor
{
    private Castle.DynamicProxy.IInvocation _lastInvocation;

    #region IInterceptor implementation

    public void Intercept(IInvocation invocation)
    {
        _lastInvocation = invocation;
    }

    #endregion

    public string GetEventName()
    {
        string methodName = _lastInvocation.Method.Name;
        return methodName.Replace("add_", "").Replace("remove_", "");
    }
}

public class EventHelper
{
    public static string GetEventName<T>(T target, Action<T> action) where T : class
    {
        var generator = new Castle.DynamicProxy.ProxyGenerator();
        var interceptor = new EventAssignmentInterceptor();

        T proxy;

        if (typeof(T).IsInterface)
        {
            proxy = generator.CreateInterfaceProxyWithoutTarget<T>(interceptor);
        }
        else
        {
            proxy = generator.CreateClassProxy<T>(interceptor);
        }

        action(proxy);

        return interceptor.GetEventName();
    }
}

Examples

Here’s a few examples that illustrate the various syntax. Note, for sake a brevity (this post is crazy large) the overloads that define events as literals are not shown.

Basic usage in a using block. Note that exceptions inside the using may be swallowed by the dispose, and there's no access to the event args.

using(EventWatcher.Create(subject, t => t.OnComplete += null)) 
{ 
    subject.PerformWork(); 
} 

using(EventWatcher.Create(subject, t => t.OnComplete += null, timeout:1000) 
{ 
    subject.PerformWork(); 
}

Condensed usage. Provides access to the EventArgs and avoids the dispose problem.

EventArgs e = EventWatcher.Create(subject, t=> t.OnComplete +=null, () => subject.PerformWork() );

EventArgs e; 
EventWatcher.Create(subject, t=> t.OnComplete += null, out e, ()=> subject.PerformWork() );

Fluent usage. A cleaner syntax that declares the event and the action separately.

EventArgs e = EventWatcher 
            .For(subject, t => t.OnComplete += null) 
            .WaitOne(() => subject.PerformWork()) 

EventArgs e; 
EventWatcher.For(subject, t=> t.OnComplete += null) 
            .WaitOne( () => subject.PerformWork(), out e);

Full Source

The full source is here.

public sealed class EventWatcher : IDisposable
{
    private readonly object _target;
    private readonly EventInfo _eventInfo;
    private readonly Delegate _listener;
    private readonly ManualResetEvent _reset;
    private readonly int _timeout;

    #region Create
    public static EventWatcher Create<T>(T target, string eventName, int timeout = 1000)
    {
        EventInfo eventInfo = typeof(T).GetEvent(eventName);
        if (eventInfo == null)
            throw new ArgumentException("Event not found.", "eventName");

        return new EventWatcher(target, eventInfo, timeout);
    }

    public static EventWatcher Create<T>(T subject, Action<T> eventAssignment, int timeout = 1000) where T : class
    {
        string eventName = EventHelper.GetEventName(subject, eventAssignment);

        return Create(subject, eventName, timeout);
    } 
    #endregion

    #region For
    public static EventWatcherSetup<T> For<T>(T subject, Action<T> eventAssignment) where T : class
    {
        string eventName = EventHelper.GetEventName(subject, eventAssignment);
        return For(subject, eventName);
    }

    public static EventWatcherSetup<T> For<T>(T subject, string eventName)
    {
        return new EventWatcherSetup<T>(subject, eventName);
    } 
    #endregion

    #region WaitForEvent
    public static EventArgs WaitForEvent<T>(T target, Action<T> eventAssignment, Action action) where T : class
    {
        EventArgs e;
        WaitForEvent(target, eventAssignment, out e, action);
        return e;
    }

    public static void WaitForEvent<T>(T target, Action<T> eventAssignment, out EventArgs e, Action action) where T : class
    {
        string eventName = EventHelper.GetEventName(target, eventAssignment);
        WaitForEvent(target, eventName, out e, action);
    }

    public static EventArgs WaitForEvent<T>(T target, string eventName, Action action)
    {
        EventArgs e;
        WaitForEvent(target, eventName, out e, action);
        return e;
    }

    public static void WaitForEvent<T>(T target, string eventName, out EventArgs e, Action action)
    {
        var watcher = Create(target, eventName);

        try
        {
            action();
        }
        catch (Exception)
        {
            watcher.Clear();
            throw;
        }
        finally
        {
            e = watcher.EventArg;
            watcher.Dispose();
        }
    } 
    #endregion

    private EventWatcher(object target, EventInfo eventInfo, int timeout)
    {
        _target = target;
        _eventInfo = eventInfo;
        _timeout = timeout;

        _reset = new ManualResetEvent(false);

        // Create our event listener
        _listener = CreateEventListenerDelegate(eventInfo.EventHandlerType);

        _eventInfo.AddEventHandler(target, _listener);
    }

    public void SetEventWasRaised(EventArgs e)
    {
        EventArg = e;
        _reset.Set();
    }

    public void Clear()
    {
        _reset.Set();
    }

    public EventArgs EventArg { get; private set; }

    private Delegate CreateEventListenerDelegate(Type eventType)
    {
        // Get the event delegate's parameters from its 'Invoke' method.
        MethodInfo invokeMethod = eventType.GetMethod("Invoke");
        var parameters = invokeMethod.GetParameters()
            .Select((p) => Expression.Parameter(p.ParameterType, p.Name)).ToList();

        var setMethod = typeof(EventWatcher).GetMethod("SetEventWasRaised");
        var body = Expression.Call(Expression.Constant(this), setMethod, parameters[1]);

        // Create the listener.
        var listener = Expression.Lambda(eventType, body, parameters);
        return listener.Compile();
    }

    public void Dispose()
    {
        bool eventWasFired = _reset.WaitOne(_timeout);
        // Remove the event listener.
        _eventInfo.RemoveEventHandler(_target,_listener);

        if (!eventWasFired)
        {
            string message = String.Format("Test timed-out waiting for event \"{0}\" to be raised.", _eventInfo.Name);
            Assert.Fail(message);
        }
    }
}

public class EventWatcherSetup<T>
{
    private readonly T _target;
    private readonly string _eventName;

    public EventWatcherSetup(T target, string eventName)
    {
        _target = target;
        _eventName = eventName;
    }

    public EventArgs WaitOne(Action action)
    {
        EventArgs e;

        WaitOne(action, out e);
        
        return e;
    }
    public void WaitOne(Action action, out EventArgs e)
    {
        EventWatcher.WaitForEvent(_target,_eventName, out e, action);
    }

}

public class EventAssignmentInterceptor : Castle.DynamicProxy.IInterceptor
{
    public Castle.DynamicProxy.IInvocation LastInvocation;

    #region Implementation of IInterceptor

    public void Intercept(IInvocation invocation)
    {
        LastInvocation = invocation;
    }

    #endregion

    public string GetEventName()
    {
        string methodName = LastInvocation.Method.Name;
        return methodName.Replace("add_", "").Replace("remove_", "");
    }
}

public class EventHelper
{
    public static string GetEventName<T>(T target, Action<T> action) where T : class
    {
        var generator = new Castle.DynamicProxy.ProxyGenerator();
        var interceptor = new EventAssignmentInterceptor();

        T proxy;

        if (typeof(T).IsInterface)
        {
            proxy = generator.CreateInterfaceProxyWithoutTarget<T>(interceptor);
        }
        else
        {
            proxy = generator.CreateClassProxy<T>(interceptor);
        }

        action(proxy);

        return interceptor.GetEventName();
    }
}

Parsing Nullable Enumerations

2011-03-18T08:58:00.000-04:00

I had an interesting challenge yesterday that involved converting a string to an enumerated value. This seems like it would be really trivial, but I had a few extra considerations that made it harder to deal with.

We're all familiar with the Enum.Parse method that has existed in the .NET Framework since version 1.1. It's a somewhat clunky and verbose method:

MyEnum result = (MyEnum)Enum.Parse(typeof(MyEnum), "First");

Recently, the .NET Framework 4.0 introduced a new static method on the Enum type which uses generics. It's much cleaner, if you don't mind the out parameters.

MyEnum result;

if (Enum.TryParse("First", out result))
{
    // do something with 'result'
}

Unfortunately, the above methods are constrained to work with struct value-types only and won’t work with Nullable types. After some experimenting and fussing with casting between generic types I managed to get a solution that works with both standard and nullable enumerations.

I’m sure there’s a bit extra boxing in here, so as always your feedback is welcome. Otherwise, this free-as-in-beer extension method is going on my tool belt.

public static TResult ConvertToEnum<TResult>(this string source)
{
    // we can't get our values out of a Nullable so we need
    // to get the underlying type
    Type enumType = GetUnderlyingTypeIfNullable(typeof(TResult));
        
    // unfortunatetly, .net 4.0 doesn't have constraints for Enum
    if (!enumType.IsEnum)
        throw new NotSupportedException("Only enums can be converted here, chum.");

    if (!String.IsNullOrEmpty(source))
    {        
        object rawValue = GetRawEnumValueFromString(enumType, source);
        
        // if there was a match
        if (rawValue != null)
        {
            // having the value isn't enough, we need to
            // convert this back to an enum so that we can 
            // perform an implicit cast back to the generic type
            object enumValue = Enum.Parse(enumType, rawValue.ToString());
        
            // implicit cast back to generic type
            // if the generic type was nullable, the cast
            // from non-nullable to nullable is also implicit
            return (TResult)enumValue;
        }
    }
    
    // if no original value, or no match use the default value.
    // returns 0 for enum, null for nullable<enum>
    return default(TResult);
}

private static Type GetUnderlyingTypeIfNullable(Type type)
{
    if (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(Nullable<>))
    {
        // Nullable<T> only has one type argument
        return type.GetGenericArguments().First();
    }
    
    return targetType;
}

private static object GetRawEnumValueFromString(Type enumType, string source)
{
    FieldInfo[] fields = enumType.GetFields(BindingFlags.Public | BindingFlags.Static);

    // attempt to find our string value
    foreach (FieldInfo field in fields)
    {
        object value =  field.GetRawConstantValue();

        // exact match
        if (field.Name == source)
        {
            return value;
        }

        // attempt to locate in attributes
        var attribs = field.GetCustomAttributes(typeof (XmlEnumAttribute), false);
        {
            if (attribs.Cast<XmlEnumAttribute>().Any(attrib => source == attrib.Name))
            {
                return value;
            }
        }
    }

    return null;
}

Heh: “Free as in Beer”. Happy St. Patty’s, everybody.

Add a Custom Toolbar for Source Control

2011-03-04T12:07:00.001-05:00

My current project uses TFS and I spent a lot of time in and out of source control, switching between workspaces to manage defects. I found myself needing access to my workspaces and getting really frustrated with how clunky this operation is within Visual Studio. There are two ways you can open source control.

The Source Control item in the Team System tool window. I don’t always have the Team Explorer tool window open and when I open it, it takes a few seconds to get details from the server.
View –> Other Windows –> Source Control Explorer. Useful, but there’s too much mouse movement and clicking to be accessible.

So, rather than creating a custom keyboard shortcut that I would forget I added a toolbar that is always in plain-sight. It’s so convenient that I take it for granted, and when I pair with others they comment on it. So for their convenience (and yours), here’s how it’s done.

Add a new Toolbar

From the Menubar, select “Tools –> Customize”. It’ll pop up this dialog.

Click on “New” and give your toolbar a name.

Add the Commands

Switch to the Commands tab and select the name of your toolbar in the Toolbar dropdown.

Click on the “Add Command” button and select the following commands:

View : TfsSourceControlExplorer
View : TfsPendingChanges

Now style the button’s accordingly using the “Modify Selection” button. I’ve set mine to use the Default styling, which is just the button icon.

Enjoy

Building Custom Test Frameworks

2011-03-03T09:04:00.000-05:00

Author’s note: this post focuses on writing test tools to simplify repetitive tasks such as complex assertions and is inspired by colleagues that wrote a tool that nearly evolved beyond their control. I am not encouraging developers to write frameworks that mask symptoms of poorly isolated components or other design problems.

For some, evolving code through writing tests is seen as a labour of love. For others, it's just labour. The latter is especially true when you realize that you have to write a lot of repetitious tests that will likely introduce test friction later on. When faced with this dilemma, developers rise to the challenge and write code.

Fortunately, there's a lot of great test framework tools popping up in the .net community that are designed to plug in to your test code. (I’m running out of battery power on my laptop as I write this, so my list of tools is lacking. Send me a note and I’ll list your tool here) These tools can certainly make things easier by removing some obstacles or laborious activities, but if you're planning on writing your own tool or framework there a few pitfalls.

Common Pitfalls

Obscured Intent

One of the goals of well written unit tests is to provide live examples of proper usage so that developers can learn more about the code's intended behaviour. This benefit can be significantly hindered when the usage and clear intent of your code has been abstracted into your framework.

Eventual Complexity

Applying automation to a problem follows the 80/20 rule where the majority of problems fit nicely into your abstraction. The edge cases however have a tendency to add bloat and it doesn't take much to quickly trash the idea of a simple tool. This is rarely a consequence of poor planning or bad design; additional complexity tends to creep in over time as your code evolves or as more consumers of the tool come on board.

There's a consequence to this type of complexity: if few developers understand the tool's implementation, you risk limiting these developers to be tool maintainers. Even worse, if these developers leave your team there's a risk that the entire suite of tests will be abandoned if they start failing.

Dependence on Tool Quality / False Positives

In TDD, application defects hide in the tests you haven't written, so quality is a reflection of the accuracy and completeness of the tests. Likewise, tests that leverage a custom test framework are only as reliable as the completeness and accuracy of the tool. If the framework takes on the lions share of the work, then the outcome of the test is abdicated to the tool. This is dangerous because any change to the tool's logic could unintentionally allow subtle defects in the tested code to go unnoticed. False positives = tests that lie!

Recommendations

Tests for Tools

Oddly enough, if your goal is to write a tool so that you don't need to write tests, you are going to need to write tests for the tool. Having tests for your custom tool ensures that false positives aren’t introduced as the tool is enhanced over time. From a knowledge transfer perspective, the tests serve as a baseline to describe the tool’s responsibilities and enable others to add new functionality (and new tests) with minimal oversight from the original tool author.

Design for Readability

Great frameworks fit in and enable you to express intent; so be careful of over-reaching and trying to do too much. Make sure your tool doesn't abstract away clues as to what the test is intended for, and if possible use descriptive method names to improve readability. Documenting your tool with Xml documentation syntax and comments is also very helpful for intent.

Evolve Naturally

Unless you spend weeks planning out your unit tests, custom test tools are the product of necessity that are realized when you start to write the second or third duplicated test. I tend to realize my test tools as “found treasures” of my normal TDD development cycle.

Rather than diving in and writing a framework or tool first, focus on satisfying the requirement of working software by writing the test using normal TDD best practices (red, green, refactor). During the clean up and refactor step, you will find duplication between tests or even different fixtures. When that happens, promote the duplicated code to a helper method, then a helper class. I like to think that great TDD is about refactoring mercilessly while keeping the original intent of the code clear – it’s about balance, know when you’ve gone too far.

If you a reach a point where you can extract commonalities between a few helper classes into a generic solution, you’ll find yourself standing in the doorway between where your fixtures were self-contained and where they’ve become dependent on shared test logic. Learn to recognize this moment because this is when you should stop writing tests for production code for a moment and write some tests for the tool. It’s also a good idea to keep a few of the manual tests around so that you can go back into the production code and deliberately break it to prove that both sets of tests are equivalent.

…Until next time. Happy coding. (Now where’s my dang power supply?)

Plaintext + Dropbox = Ubiquitous Text

2011-02-08T09:00:00.000-05:00

I am loving this. If you blog or like to jot down notes while on the go then I highly recommend the following setup.

Dropbox

Dropbox is a small cloud-based utility that synchronizes files between all your devices. It works on PC, Mac, iOS, Android and Blackberry. It even has a web interface that you can browse from any computer, and even track previous versions of files. Best of all, it's free (for 2Gb of storage).

There are a number of applications that use Dropbox as their storage medium. My favorite so far is PlainText.

PlainText

PlainText is a bare bones text editor with a minimal UI that uses Dropbox as it's file system. Simply download to your iPhone, iPod Touch or iPad, login with your Dropbox account and your data magically syncs each time you touch the contents. Only limitation is that it doesn't have the ability to move files to different folders, but I can do that on my PC.

Now it's possible for me to sketch down an idea on my iPhone while riding the subway or streetcar, then pickup where I left off and flesh out the details on my iPad while watching TV. I can even switch between devices with almost no wait, so I can proof read or work through an idea whenever I have a free moment.

Getting started

Setup is easy. On your PC or Mac, go to Dropbox and setup an account, the software will download on the next page. Install the application and associate it to your user account. The free version gives you 2 GB of storage (more if you enlist your friends) and they offer more storage through paid upgrades.

On your iOS device, download PlainText. In the settings, link it to your Dropbox account.

PlainText by default will create a folder in your Dropbox called “PlainText”. This is helpful so that you’re only syncing small files. Here’s a quick screen-capture of the available settings.

You can create files and folders on your iOS device. Any changes will automatically sync to your PC/Mac.

Disclosure: This is not a paid advertisement, I just really like this configuration. I should point out that the links to sign-up are to my referral page. If you are vehemently opposed to this you can visit the site referral free here: https://www.dropbox.com/