Jimmy Bogard

A better Model Binder addendum

bogardj — Thu, 19 Nov 2009 18:31:19 PST

A while back, I wrote about a ModelBinder enhancement we use to do arbitrary filtering on types. The underlying matching algorithm only matches on one type, but we like to use layer supertypes for a lot of our domain objects, so we want to use a single model binder for every Entity or Enumeration type in our system.

However, I missed an important piece in our model binding story: ModelState. In the original implementation:

public override object BindModel(
    ControllerContext controllerContext, 
    ModelBindingContext bindingContext)
{
    foreach (var filteredModelBinder in _filteredModelBinders)
    {
        if (filteredModelBinder.IsMatch(bindingContext.ModelType))
        {
            return filteredModelBinder.BindModel(controllerContext, bindingContext);
        }
    }

    return base.BindModel(controllerContext, bindingContext);
}

I didn’t address ModelState. It turns out that this may lead to problems later on when I use the built-in HtmlHelper methods. These helper methods use ModelState to do things like set error message indicators, fill in values and so on. But if one of the IFilteredModelBinder implementations binds a value but doesn’t do the ModelState piece, I start to get some weird behavior in certain scenarios.

If you start to dig pretty deeply in the DefaultModelBinder code, you’ll find that the base implementation of BindModel will set the ModelState values for certain scenarios. In our case, we need to allow the IFilteredModelBinders to do something similar.

Our slight fix

First, we’ll need to fix the IFilteredModelBinder interface to return not just an object, but something that includes the ValueProviderResult object (used by ModelState). Why not just do object? Well, we can’t make an assumption on what the underlying model binder implementations need to do with a BindResult, it’s rather custom for each implementation. So, our IFilteredModelBinder interface becomes:

public interface IFilteredModelBinder
{
    bool IsMatch(ModelBindingContext bindingContext);
    BindResult BindModel(
        ControllerContext controllerContext, 
        ModelBindingContext bindingContext);
}

That BindResult object is not in MVC, it’s something we have to define:

public class BindResult
{
    public object Value { get; private set; }
    public ValueProviderResult ValueProviderResult { get; private set; }

    public BindResult(object value, ValueProviderResult valueProviderResult)
    {
        Value = value;
        ValueProviderResult = valueProviderResult ?? 
            new ValueProviderResult(null, string.Empty, CultureInfo.CurrentCulture);
    }
}

All object adds is the ValueProviderResult, providing a default null-object implementation if the implementing filtered model binder doesn’t support ValueProviderResults. With this in place, we need to fix our SmartBinder:

public class SmartBinder : DefaultModelBinder
{
    private readonly IFilteredModelBinder[] _filteredModelBinders;

    public SmartBinder(IFilteredModelBinder[] filteredModelBinders)
    {
        _filteredModelBinders = filteredModelBinders;
    }

    public override object BindModel(
        ControllerContext controllerContext, 
        ModelBindingContext bindingContext)
    {
        foreach (var filteredModelBinder in _filteredModelBinders)
        {
            if (filteredModelBinder.IsMatch(bindingContext))
            {
                var result = filteredModelBinder.BindModel(controllerContext, bindingContext);

                bindingContext.ModelState.SetModelValue(
                    bindingContext.ModelName, 
                    result.ValueProviderResult);

                return result.Value;
            }
        }

        return base.BindModel(controllerContext, bindingContext);
    }
}

It’s pretty much like the original, except that piece that does the SetModelValue part. That fills in the necessary pieces for the input HtmlHelper methods to do their thing. Interestingly, this made another problem go away, around ModelState.AddModelError. It turns out I just missed a piece in the custom model binding code.

So just a note to custom model binder implementors – don’t forget to populate the appropriate ModelState values, especially if the results of the binding operation will be used in conjunction with the HtmlHelper extensions.

How to annoy your teammates

bogardj — Fri, 13 Nov 2009 06:22:55 PST

In 3 easy steps:

Step 1: Perform a change that affects many, many files

My favorite is a namespace rename. Other choices include deleting a core marker interface, renaming a layer supertype class, or changing the folder structure in a project.

Typically, VisualStudio will crash when I do this, though I’m not sure if it’s VS choking or ReSharper. Either way, a large automatic refactoring will cause a crash.

Step 2: After the VS crash, re-open the solution and choose to recover all unsaved files

ReSharper likely made it quite a ways through your refactoring before VS crashed. All those unsaved files were likely cached for recovery by VisualStudio, so you can probably get a lot of those changes back. Instead of choosing to revert your local changes, go ahead and recover all those files. It’s key for maximum annoyance.

Step 3: Merge these recovered files back to trunk

Since this was an enormous refactoring, maybe affecting hundreds of files, you’ll want to merge your changes back to trunk as soon as possible. After all, you don’t want to be the sucker merging after everyone else has finished their work.

When you’re done, your co-workers will be greeted with one annoying merge, and another fun dialog that never seems to quite go away:

Just when you think you caught the last of the recovered files that had its line endings screwed up, another one pops up. They’ll hate you once because of the core change that affected so many files, making it quite annoying to merge back to trunk. But they’ll form that long-lasting, persistent hate because of this dialog box that keeps coming up. Yes, there is that little checkbox at the bottom, but who reads these things anyway?

Willed and forced design

bogardj — Thu, 12 Nov 2009 18:32:19 PST

Roy Osherove, as a TypeMock employee, presents quite a dilemma from opinionated TDD blog posts simply because whether he has one or not, there’s always the question of agenda. Which is quite unfortunate, everyone has some sort of selfish agenda at some level. One of those posts came up in a commentary on willed and forced design, with respect to the usage of mocking frameworks:

#1 Willed Design

By writing tests, you can observe the usability of your design from a consumer perspective, and can decide whether or not you like it, and change it accordingly

#2 Forced Design

By using a subset of the available isolation frameworks(rhino, moq, nmock) or specific techniques *manual mocks and stubs) you discover cases that are not technically “mockable” or “fakeable” and use that as a sign for design change.

After about 2 seconds of playing around with dynamic languages such as Ruby and JavaScript, the idea that TypeMock is some how “impure” seemed rather silly. Yes, it bends the CLR in crazy ways, but in and of itself, I believe it has its place.

However, I highly disagree with Roy that #2 is bad, that a tool shouldn’t force my hand in a design. But there’s a bit of a straw man here – it’s not the tool forcing my design, it’s the test telling me where I need to change my design. Quite simply, if I run up against something hard to test, my first choice is to isolate that piece. Here’s an example:

public ActionResult Index()
{
    var user = Session["CurrentUser"];

    return View(user);
}

This piece is awkward to test, even though HttpSessionBase or whatever is a class with virtual methods, and it makes the test ugly. Yes, I could mock the crap out of this heavyweight object, but that’s not really helping me out, is it? Instead, I’ll isolate the ugly:

private readonly IUserSession _userSession;

public HomeController(IUserSession userSession)
{
    _userSession = userSession;
}

public ActionResult Index()
{
    var user = _userSession.GetUser();

    return View(user);
}

Instead of using a dictionary directly, I’ll hide that business behind a facade service, keeping the dictionary ugliness and duplication in one highly cohesive class. But it was the _test_ that led me to do this, because I’m tuned in to pieces that are hard to mock. File I/O? Facade. Web service proxies? Facade. Registry, HttpContext, etc. etc? All facade. I want to isolate my core application from the untestable pieces, not because they’re necessary untestable, but because I don’t want to couple my application directly to these external services. Putting these behind targeted facades eliminates all that duplication that directly using these APIs tends to encourage.

There are plenty of APIs that slapping a simple facade over tends to make things worse, not better, such as:

Workflow Foundation
SharePoint

And any other framework that requires you to tiiiiightly integrate with in order to be successful. So where does that leave me and TypeMock? Well, I don’t use any of those frameworks that require tight coupling, and I don’t have any pain with my current tool of choice in .NET (Rhino Mocks), so why switch exactly? For those rare occasions I might need TypeMock (DateTime.Now), I already have a solution I quite like. Yes, TypeMock could be free, but the only compelling reason I’d switch would be to another free tool with a better, clearer to write, read and scan API.

Fields and virtual members

bogardj — Mon, 09 Nov 2009 06:40:25 PST

It seems like the sillier the bug, the more time you’ll spend debugging it, simply because it’s in functionality you just knew that worked correctly (and had the tests to back it up). One problem we hit recently was code that used fields to back virtual properties:

public abstract class Enumeration : IComparable
{
    private readonly int _value;

    protected Enumeration() { }

    protected Enumeration(int value)
    {
        _value = value;
    }

    public virtual int Value
    {
        get { return _value; }
    }

    public virtual int CompareTo(object other)
    {
        return _value.CompareTo(((Enumeration)other).Value);
    }
}

This isn’t a problem in and of itself, except in the behavior of that last method. Notice its logic uses the field and not the property. I can then define a derived type:

public class PercentDiscount : Enumeration
{
    public override int Value
    {
        get { return 5; }
    }
}

But now my CompareTo method breaks because it isn’t using the correct value. It uses the uninitialized field, causing all sorts of bizarre behavior. When you have logic errors in places like GetHashCode, Equals or CompareTo, I can guarantee your application will break in the most hair-pulling ways imaginable.

Let’s fix it

So what’s a better option? We could just “make sure” that we don’t use the private field, and only use the virtual property. But I hate conventions that rely on my memory to not make a mistake, I want to fall into the pit of success. One way to do so is to use automatic properties:

public abstract class Enumeration : IComparable
{
    protected Enumeration() { }

    protected Enumeration(int value)
    {
        Value = value;
    }

    public virtual int Value
    { 
        get;
        private set;
    }

    public virtual int CompareTo(object other)
    {
        return Value.CompareTo(((Enumeration)other).Value);
    }
}

Instead of a backing field, we’ll use an automatic property, making it impossible to use a non-virtual backing field. We still get the same access modifiers (a virtual, overridable getter, a private setter), plus the benefits of automatic properties. So now I’m curious: are there any valid reasons to keep using fields for holding state?

MVC Best Practices

bogardj — Wed, 28 Oct 2009 05:45:26 PDT

Simone has a great post (as usual) on 12 ASP.NET MVC Best Practices:

Controller:

1 - Delete the AccountController

2 - Isolate controllers from the outside world

3 - Use an IoC Container

4 - So NO to “magic strings”

5 - Build your own personal conventions

6 - Pay attention to the verbs

Model:

7 – Domain Model != ViewModel

8 – Use ActionFilters for “shared” data

View:

9 – NEVER use code-behind

10 – Write HTML each time you can

11 – If there is an if, write an HtmlHelper extension

12 – Choose your ViewEngine carefully

This is a fantastic list, and of course I have my own additions/suggestions :) First, I’m not a fan of action filters for shared data. If we’re doing strongly-typed views, using action filters for shared data puts us back into the magic string land, where we have to either use inheritance in our ViewModels, or the dictionary part of ViewData. I don’t like either approach, I’d rather go with something like RenderAction to truly enforce SRP in my ViewModels. Not everyone appreciates the elegance of this approach, but I’m sure they’ll agree eventually.

I also don’t like writing an HtmlHelper extension just because I have logic in the view. For a half-decent view engine, it shouldn’t be that problematic to have view logic in the view. HtmlHelper extensions enforce the Helper object anti-pattern – a bunch of procedural logic hanging off one static class. Instead, we went the route of building intelligent input builders for input elements, and only really using HtmlHelper when we want to eliminate duplication between multiple views. I see these constructs:

HtmlHelper
Partials
Master Pages

All as means of eliminating duplication in our views, but not much more. If there’s logic in a view, that’s fine by me as long as it’s not duplicated. It’s not my fault that C# looks downright bizarre mixed in with HTML markup, but that’s what other view engines are for.

Finally, if there was an award for How Not to Design a Controller, the AccountController would be the runaway champ. I still can’t imagine why it’s necessary, and the idea of portable areas would be a better fit for its functionality than the Thing We Always Delete with Extreme Prejudice.

MVC Web Testing Strategies – verifying content

bogardj — Tue, 27 Oct 2009 18:51:46 PDT

Some of the questions during the C4MVC presentation concerned how I liked to locate data displayed in the rendered HTML for validation purposes. You have quite a few options for doing so, such as:

Traverse HTML directly through the DOM
Using existing semantic HTML information
Adding semantic HTML information
Wrapping data with specific HTML

I’ll dismiss the first option immediately, it’s quite brittle to traverse only the DOM to locate data on a screen. If I have to have intimate knowledge of the entire HTML document rendered, the test will become far too dependent on the layout on the screen. If I decide to move information out of a table and into a definition list, my test will likely break.

From there, I’m left with basically two options – semantic HTML, whether it’s existing or added, or wrapping each data piece with custom HTML. In the presentation, two questions came up:

When locating a piece of information in a row in a table, why not use something more meaningful, such as the underlying entity’s ID to tag the data?
Again in the table, why not use semantic information on the row, instead of adding meaningless HTML?

Before we look at the different ways to verify content, let’s first look at what we want to verify.

The original content

In the scenario I highlighted in the screencast, I walk though editing some information, then verifying that data was changed correctly. I still have tests for controllers, repositories and so on, but I want a high level scenario-based test to make sure the entire pipeline is working as expected. We try and pull the information out of the HTML content rendered, using some sort of UI testing tool. Without modifying the content, here’s what we have to work with:

<div id="main">
    <h2>Products</h2>
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td><a href="/Product/Edit/1">Edit</a></td>
                <td>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</td>
                <td>Insignia</td>
                <td>359.99</td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/2">Edit</a></td>
                <td>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</td>
                <td>Insignia</td>
                <td>189.99</td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/3">Edit</a></td>
                <td>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</td>
                <td>Dynex</td>
                <td>159.99</td>
            </tr>
        </tbody>
    </table>

There’s more rendered than just this HTML, but this is the piece that really matters. We have some semantic CSS information, the DIV with an ID of “main”, but that’s about it. In our CSS, we style our tables based on an ID-Element selector, but we could have just have easily used a class name on the table. I’d like to verify that the price on the first product was changed correctly, but how do I verify this? How do I make sure that the value in that one cell is the new price?

Choosing a strategy

I don’t want false positives or false negatives. I don’t want to look for just any table or any table cell in the markup, I want really the piece just around “359.99”. A few options to do so include:

Locating the outer DIV, finding the table, the first row, last cell, etc.
Apply a special class to the TD just where that price is shown
Add markup only around data-bound elements

When I first started doing UI testing, I assumed that I wanted to be as semantic and pure as possible, and not try to change my markup just for tests. That idea lasted about an hour, and reality set in that in general, things not designed with testing in mind will be hard to test. Hard to write tests, hard to maintain tests.

My next idea was to add information as needed, perhaps a class name to an existing surrounding element that might double for styling. But I’ve moved completely away from that strategy as well, as styling and UI testing are two different concerns that should not mix.

Why keep these concerns separate? Frankly, because both concerns have two completely different reasons to change. Styling and behavior (such as IDs added for jQuery) are about design and interaction. UI testing is about verifying site behavior. Mixing the two concerns means my tests are more likely to break because I’ve changed the design. But design can change without behavior changing, so I don’t want to couple my tests to the visual design. Coupling UI tests to the site design is like unit testing using only the reflection API.

Instead, I’d rather couple my tests to something that better represents the view – and that would be the ViewModel.

Strongly-typed views, strongly-typed tests

We use strongly-typed views because using a dictionary just leads to brittleness and fear of change. Strongly-typed views, along with the concept of the Thunderdome Principle, let us move past the issues of broken views and broken model binding. Instead of writing black-box tests against our UI, we can take advantage of the strongly-typed views, and modify our HTML rendered to get access to the rendered ViewModel. If you look at views from the point of view of “how to render a ViewModel”, then your UI tests can now talk in the language of the ViewModel.

But to help our UI tests out, we need to be able to locate information rendered from a ViewModel. Looking at our original HTML, the most fool-proof, straightforward way to do this would be to add HTML as close to the rendered ViewModel as possible. Let’s look at our view to see where this might be:

    <h2>Products</h2>
    <% var products = Model; %>
    
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>
            </tr>
        </thead>
        <tbody>
        <% foreach (var product in products) { %>
            <tr>
                <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td>
                <td><%= product.Name %></td>
                <td><%= product.ManufacturerName %></td>
                <td><%= product.Price %></td>
            </tr>
        <% } %>
        </tbody>
    </table>

So what’s the ideal spot here? On the table? The table body? Row? Cell? These are all possible…and all coupled to the design of the UI, rather than the shape of the ViewModel. Instead of coupling to the design of the HTML, we can instead change every spot where we render the ViewModel directly to have extra HTML used only for UI tests. In the presentation, I move towards this model:

<tbody>
<% var i = 0; %>
<% foreach (var product in products) { %>
    <tr>
        <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td>
        <td><%= Html.Span(m => m.Name) %></td>
        <td><%= Html.Span(m => m.ManufacturerName) %></td>
        <td><%= Html.Span(m => m.Price) %></td>
    </tr>
<%
    i++;
} %>
</tbody>

In practice, I’ll use a similar concept to the Opinionated Input Builders Eric highlighted a while back. Instead of input elements, I’ll solely render at the core, span tags. Here is the rendered HTML:

<div id="main">
    <h2>Products</h2>
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>

            </tr>
        </thead>
        <tbody>
            <tr>
                <td><a href="/Product/Edit/1">Edit</a></td>
                <td><span id='_0__Name'>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</span></td>
                <td><span id='_0__ManufacturerName'>Insignia</span></td>
                <td><span id='_0__Price'>359.99</span></td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/2">Edit</a></td>
                <td><span id='_1__Name'>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</span></td>
                <td><span id='_1__ManufacturerName'>Insignia</span></td>
                <td><span id='_1__Price'>189.99</span></td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/3">Edit</a></td>
                <td><span id='_2__Name'>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</span></td>
                <td><span id='_2__ManufacturerName'>Dynex</span></td>
                <td><span id='_2__Price'>159.99</span></td>
            </tr>
        </tbody>
    </table>

Notice that each data-bound output is wrapped with a span tag – and a special span tag at that. The span tag has a unique ID that represents the expression used to render that piece of data. The first product name shown comes from the expression “m => m[0].Name”. Because I know the ViewModel used to render this view, I can use an expression combined with a UI testing tool to locate the rendering of any piece of information on the screen.

The downside of this strategy is the amount of HTML added to the final document render. But since we’ve separated HTML used for styling and behavior, and HTML rendered strictly for testing, we can put a switch in our application so that for environments not doing UI testing, the span tag isn’t ever rendered! We can then get the best of both worlds – non-obtrusive JavaScript on top of semantic HTML, and flip a switch to be able to get to any piece of information on the rendered HTML.

Because our UI tests also use the ViewModel types and expressions to locate the same rendered ViewModels that the views used to render, we no longer run into the issue where our UI tests become disconnected from the views rendered. If we delete a property on our ViewModel, guess what, the UI test no longer compiles!

Wrapping it up

Developing a UI testing strategy isn’t the easiest thing in the world. In our current project, I think we had three or four evolutions before we finally settled on a UI testing strategy we like. And it’s still not perfect – we’re still looking at ways to make UI testing easy, expressive, and more valuable. But gone are the days where a styling change broke our tests. When our UI tests fail now, it’s more along the lines of a new required field, changing business rules and so on.

I don’t like mixing styling and behavior with locating data-bound UI elements, simply because both have very different reasons for change. But by taking advantage of our strongly-typed views in our UI tests, we can render the ViewModel in such a way that makes it dirt-simple to locate individual pieces of our ViewModel on the final rendered page.

C4MVC UI Testing screencast posted

bogardj — Thu, 22 Oct 2009 18:37:41 PDT

Yesterday the C4MVC guys hosted a screencast with me on UI testing ASP.NET MVC. It was a lot of fun, but I actually forgot I was being videotaped (webcam’d?), so all those times I turned off my mic to catch a drink of water or cough uncontrollably are caught in glorious 370i. Anyway, check out the video here:

Unfortunately, I didn’t leave enough time for questions at the end, and I had some good ones from the audience. You can also find the complete code and slides here.

Thanks again to the C4MVC guys for hosting me! Since I didn’t have time to answer too many questions, feel free to ask them in the comments, or you can always use the Contact Me link.

C4MVC meeting on UI testing

bogardj — Wed, 21 Oct 2009 06:38:41 PDT

Today I’m showing giving a talk on UI testing MVC applications for the Community for MVC.Net folks. Those that went to my presentations in Arkansas and Houston have seen this talk already, so they’re off the hook on this one. I’ll be covering ways to create UI tests that don’t break when a butterfly flaps its wings in Japan, and how we can design our MVC applications for UI testability.

The presentation is done via LiveMeeting, (http://Meeting.c4mvc.net), so make sure you’ve hit that link before the start time, which is 12:00 PM CST today. For more details, check out the C4MVC site:

http://www.c4mvc.net/

In the WebForms world, it was quite difficult to create maintainable UI tests. But in ASP.NET MVC, we have far more control over the HTML generated, allowing us to design our UI for testability. It should be a good discussion, I’m looking forward to seeing everyone there!

How not to implement a failing test

bogardj — Mon, 19 Oct 2009 06:45:30 PDT

One of the first things I change in ReSharper, along with one of my biggest pet peeves is a failing test that fails because of something like this:

public class CombinedStreetAddressResolver 
    : NullSafeValueResolver<Address, string>
{
    protected override string ResolveCore(Address model)
    {
        throw new NotImplementedException();
    }
}

In the Red-Green-Refactor progression, the Red of a failing test should come from an assertion failure not a “my code is stupid” failure. The Red part is intended to triangulate and calibrate your test, to make sure that your test can fail correctly. A NotImplementedException won’t cause a meaningful failure, and only serves the purpose of getting your code to compile.

Throwing exceptions means your assertions never get executed in RGR until you attempt to make a passing test, in which case you still haven’t proven your test to be correct. If you don’t know that your test is correct, you have two points of failure: your test, and code under test.

That’s why I’ve set ReSharper to return default values instead of throw exceptions. I want meaningful failures from a valid test, otherwise I’m better off skipping the Red step altogether.

More missing LINQ operators

bogardj — Thu, 15 Oct 2009 20:19:55 PDT

Continuing an old post on missing LINQ operators, the wonders of extension methods allow us as developers to fill potential holes in LINQ operators. Whether it’s a Zip method (now included in .NET 4.0), or better methods for IComparer-based operators, I find myself adding more and more helpful LINQ operators, I wish were already in the framework.

Alternate

Do you ever want to weave two collections together, like shuffling a deck of cards? Well I know I do! Suppose we have this collection:

[1, 3, 5]

And this collection:

[2, 4, 6]

I’d like to create new collection that is the alternating items from the first and second list:

[1, 2, 3, 4, 5, 6]

Here’s the code to do it:

public static IEnumerable<TSource> Alternate<TSource>(this IEnumerable<TSource> first, IEnumerable<TSource> second)
{
    using (IEnumerator<TSource> e1 = first.GetEnumerator())
    using (IEnumerator<TSource> e2 = second.GetEnumerator())
        while (e1.MoveNext() && e2.MoveNext())
        {
            yield return e1.Current;
            yield return e2.Current;
        }
}

Very simple, I iterate both enumerables at the same time, yielding the first, then second collection’s current item. So how is this useful? How about this action:

[Test]
public void Word_play()
{
    var source = new[] {"The", "quick", "brown", "fox"};

    var result = source.Alternate(Spaces()).Aggregate(string.Empty, (a, b) => a + b);

    result.ShouldEqual("The quick brown fox ");
}

private IEnumerable<string> Spaces()
{
    while (true)
        yield return " ";
}

I cheated a little bit with an infinite sequence (the Spaces() method), but I found this method useful when I had to split, then reconstruct new sequences of strings.

Append

I really hate this syntax:

[Test]
public void Bad_concat_method()
{
    var ints = new[] {1, 2, 3};

    var oneToFour = ints.Concat(Enumerable.Repeat(4, 1));

    CollectionAssert.AreEqual(new[] { 1, 2, 3, 4 }, oneToFour.ToArray());
}

I want to just stick an item on the end of an existing collection, but I have to use this arcane Enumerable.Repeat method to do so. Instead, let’s create an operator that lets us tack an item on to the end of a collection:

public static IEnumerable<TSource> Append<TSource>(this IEnumerable<TSource> source, TSource element)
{
    using (IEnumerator<TSource> e1 = source.GetEnumerator())
        while (e1.MoveNext())
            yield return e1.Current;

    yield return element;
}

Now our code becomes much easier to understand:

[Test]
public void Easier_concat_with_append()
{
    var ints = new[] {1, 2, 3};

    var oneToFour = ints.Append(4);

    CollectionAssert.AreEqual(new[] { 1, 2, 3, 4 }, oneToFour.ToArray());
}

Prepend

Append wouldn’t be complete without the converse, Prepend, now would it?

public static IEnumerable<TSource> Prepend<TSource>(this IEnumerable<TSource> source, TSource element)
{
    yield return element;

    using (IEnumerator<TSource> e1 = source.GetEnumerator())
        while (e1.MoveNext())
            yield return e1.Current;
}

Now putting something on the beginning of a list is easier as well:

[Test]
public void Easier_concat_with_prepend()
{
    var ints = new[] {1, 2, 3};

    var zeroToThree = ints.Prepend(0);

    CollectionAssert.AreEqual(new[] { 0, 1, 2, 3 }, zeroToThree.ToArray());
}

Much more readable. I have a few more around replacing the IEqualityComparer<T> overloads, but those are a little bit more esoteric in their examples of crazy set-based logic. What’s really cool about all these methods is they still allow all sorts of fun chaining, allowing me to create very terse chains of operations on lists, with what would have taken a bazillion cryptic for..each loops. Cool stuff!

Thanks Arkansas!

bogardj — Thu, 15 Oct 2009 19:01:17 PDT

This week I made the drive from Austin to Fort Smith and Springdale, Arkansas to speak at the Fort Smith DNUG and the Northwest Arkansas DNUG on UI testing ASP.NET MVC. Although I showed tools such as WatiN and Gallio, I focused more on design for UI testability, and covered items such as:

Site navigation
Filling out forms
Verifying results

We stepped through a working, but horribly fragile UI test, and walked through how we can put in a few design techniques to share knowledge between view design and the UI test. Hopefully, my message of “ban test recorders” gets through to someone, as the UI test recorder is something that threw me off for years on creating maintainable UI tests.

You can find the slides and code here:

Testing the last mile in ASP.NET MVC

Thanks to all the guys in Arkansas for the great conversations, and I hope to see everyone soon! One parting shot from a trip to Devil’s Den:

My jeans were soaked up to my knees from that hike…

An AutoMapper success story

bogardj — Thu, 08 Oct 2009 09:38:32 PDT

I got a cool message on the AutoMapper mailing list from Howard Van Rooijen on how they used AutoMapper in a site they recently launched to production:

Hello AutoMapper Community,

I just wanted to let you know that an e-commerce site which uses AutoMapper as part of it's core architecture has just been released into the wild:

http://www.fancydressout fitters.co.uk

And I wanted to say a HUGE thank-you to Jimmy & the Community for this wonderful tool - that helps remove so much commodity plumbing code from the solution.

We were a little sceptical at the start of the project that AutoMapper would "cut the mustard" when it came to the performance requirements of a public facing, high load, e-commerce site because of the amount of reflection AutoMapper uses at its core, but we have been incredibly impressed with the performance of the solution under load.

The site is based on the S#arp Architecture Framework and its become very apparent how well AutoMapper fits into MVC style architecture as it enables easy separation of concerns with regards to object conversion (entities & ViewModels). Once we moved from hand-cranked converters to AutoMapper it was amazing how much cleaner our code became - so much so that we modified the overall solution architecture to incorporate explicit mapping layers (see attached image). Our general pattern of usage within MVC is as follows:

1. Map input into Domain Entities in the Controller

2. Pass Domain Entities into Task Layer to "do stuff"

3. Map output of Task Layer (Domain Entities) into ViewModel

4. Pass ViewModel to ViewEngine

Simple, slick and clean.

To formalise the Mapping Layer and make it testable we implemented a simple interface:

public interface IMapper<TInput, TOutput>

{

TOutput MapFrom(TInput input);

}

Next we'd implement a custom marker interface so that we could resolve the mapper from the DI container and we adopted the naming convention <Input Type><Output Type>Mapper:

public interface IEditModelEntityMapper : IMapper<EditModel, Entity>

{

}

Then finally implement the interface:

public class EditModelEntityMapper : IEditModelEntityMapper

{

public EditModelEntityMapper()

{

Mapper.CreateMap<EditModel, Entity>()

.ForMember(x => x.Property, y => y.MapFrom(z => z.Property));

}

public Entity MapFrom(EditModel input)

{

return Mapper.Map<EditModel, Entity>(input);

}

}

Then to actually use it in the MVC app:

public class CustomController : Controller

{

private readonly ITasks tasks;

private readonly IEditModelEntityMapper editModelEntityMapper;

private readonly IOutputViewModelMapper outputViewModelMapper;

public CustomController(ITasks tasks,

                                           IEditModelEntityMapper editModelEntityMapper,

                                           IOutputViewModelMapper outputViewModelMapper)

{

this.tasks = tasks;

this.editModelEntityMapper = editModelEntityMapper;

this.outputViewModelMapper = outputViewModelMapper;

}

public ActionResult Index(EditEntity input)

{

var entity = this.editModelEntityMapper.MapFrom(input);

                var output = this.tasks.DoSomething(entity)

                var viewModel = this.outputViewModelMapper.MapFrom(output);

return View(viewModel);

}

}

For those who are interested - here's a little more info:

- We ran the project using Scrum and delivered in 20 weeks: 10 x 2 week iterations

- It's based on the S#arp Architecture framework, which we extended to support, AutoMapper, Spark and ViewModels
- Solution performs very well: 1000 concurrent users per web server, generating around 180 pages per second across 2x single quad core 64bit servers.

Again, many thanks,

Howard

Howard also shared a neat little diagram of his architecture:

This is one of the greatest feelings from doing OSS – that something you created basically just for yourself can also help out other folks out there trying to deliver value for their customers. Thanks for all the feedback everyone, as well as kudos to the S#arp Architecture team for building such a great framework for MVC.

Bugs, defects and feedback

bogardj — Wed, 30 Sep 2009 10:58:35 PDT

In my last post, I talked about how we like to track bugs, with just pieces of paper. At lot of the responses were interesting, but I think some came back to the issue of “what is a bug?” Some felt that bugs needed to live in a more durable system (as if paper isn’t durable, and previous to around 1985 no one knew how to manage information). But for us, bugs don’t live more than 1 business day, so there’s no need to keep them around, unless we want to do trend analysis.

But one important issue is how we deal with feedback. A user can come up to us and say, “the site is broken”. That could mean a lot of things. It could mean the site is blowing up, or it could mean that they couldn’t find the “Submit Order” button. To the user, the site is broken.

Instead of referring to negative user feedback as “bugs”, we treat them as “issues”. So what is an “issue”? An issue is negative user feedback with potentially actionable fix. Note “potentially”.

The feedback process

When we get an issue from a user, the issue goes into triage with the Product Owner. The Product Owner is the person/team deciding what gets developed, so it’s logical that they decide what action to take with an issue. An issue falls into two broad categories:

Bug
Defect

A bug is an issue where the site is unusable or acceptance criteria has not been met. This includes things from “the site is down” to “you misspelled a word”. All of our stories have tangible and intangible acceptance criteria, functional and non-functional requirements. The expectations of these have been laid forth a long time ago, so we don’t really negotiate too much on what is or is not a bug. The user may have a different idea of the classification of bug, but they don’t get to decide.

A defect is an issue where the site is usable and acceptance criteria have been met, but the application does not work as expected. Things in this category are “data field should be over here”, “add this label”, “this business rule isn’t quite right” and so on. We don’t expect the users to know all of the stories we’ve developed, but there are still situations where the site isn’t working like it should, but works as agreed. We work hard to do good JIT analysis, but we still miss things.

Actions on bugs

Bugs are fixed immediately, with an SLA of one business day. We do things like branch and release-per-feature, automated builds and deployments, so we’re never more than about 15 minutes away from a commit to production fix. We refuse to keep an inventory of open bugs, so bugs are fixed as they arrive, and the team will stop their work to fix a bug. Because we stop-the-line, we strive to eliminate as many bugs as we can. This includes good design, a strong testing culture, and post-mortems on bugs to see how we can improve the next time around and eliminate the possibility of a bug like that to arise again.

Actions on defects

Defects need to be analyzed just as much as regular stories need to, so defects enter a second queue where they either turn into stories, or something we call “tweak stories”. Stories take 1-5 business days to deliver, while a “tweak story” gets turned around as quickly as a bug. It all comes down to the complexity of the defect. In any case, the Product Owner prioritizes these defects along with stories, and ultimately it’s the PO that decides when (or if) the team takes action on a defect.

The feedback cycle

For critical bugs, like “the site blows up”, there is no need to triage. It’s unacceptable that the site blows up, so we’ll fix those immediately. But for more subtle feedback, the developer team is ill-equipped to make a determination on the nature of an issue. Often, the development team attempts to fix every issue that comes their way, whether it’s a bug or not, in an effort to make the user happy. But the guy that says “the customer is always right”? He goes broke. The customer isn’t always right, so it’s the PO’s responsibility for determining the classification of the issue.

Ultimately, it’s about roles and responsibilities. Developer teams are usually quite poor at determining the priority of work, as they don’t always have the deep business/project understanding of scope, deadlines, priorities and so on. Instead, we place the responsibility of triage and prioritization where it belongs, with the Product Owner.

With this system in place, we have no need to keep what we call “bugs” around in a durable format. Once artifacts become durable, people hesitate to delete them, and now they need to become managed. At one point, I kept bugs. But I found I never looked at them, as our practice of post-mortems on bugs as they are fixed was perfectly adequate. Instead, I (or nowadays, the originator) just toss them when they’re done, and it feels quite liberating.

My favorite bug-tracking system

bogardj — Tue, 29 Sep 2009 10:57:03 PDT

I’m of the opinion that a process must demonstrate the need for software, before software is put in place to manage that process. Bug tracking is a process, but often we jump straight to a software solution for managing/tracking bugs before considering lower-tech, lower-risk and lower-overhead approaches. In our bug-tracking process, we employ a few main principles:

Bugs are a normal part of the development process
Most bugs are rework, therefore waste, and we should strive to eliminate this waste
Bugs are fixed as they are found (stop-the-line), and do not live for more than 1 business day

Given that bugs do not live for more than one business day, it is not necessary for us to maintain an inventory of bugs. Instead, we want a system than can accurately report a bug, and make it easy for it to be verified.

With these constraints in mind, my favorite bug tracking system is a 5x8” orange sticky note:

On this giant sticky note, the person that finds the bug writes:

Their initials
The name of the story
A short description

If there is a screenshot, this will be stapled to the back. When the developer is done with the bug, they write:

Their initials
The revision # of the commit that fixed the bug

The revision number is used to make sure that the currently deployed version is up-to-date enough to check the fix. So how does this bug tracking system work? Here is our process:

Person finds bug. Person could be anyone, developer, BA, product owner.
Finder creates orange card for bug, following the specified template (every piece of information goes in a certain spot on the giant post-it)
If the bug is a blow-up, or blocks the delivery of a story, it is placed on the desk of the tech lead (me)
If the bug is not specifically called out in acceptance criteria, it is placed on the desk of the BA lead.

These bugs go through triage, because sometimes things “work as designed” but not “work as expected”. Those in the latter category need to go through the normal story process.

The tech lead assigns the bug to a developer (or themselves), by placing the bug on the assigned developer’s desk
When the fix is done, the developer writes their initials and the revision number, and gives the orange card back to the originator, with a smile
The originator of the bug verifies the fix, and places the orange card in the trash with a “thumbs up” to let the developer know the bug is verified fixed.

This is about as lightweight as we can get it, without chaos. This process has evolved over time, but we’ve always fit our tools (post-its) to our process, instead of fitting our process to a tool. We’re all in the same room, so there is absolutely zero need to track bugs electronically. Tracking them electronically would create waste, as we would now have inventory to manage in some other system. It really can’t get much easier than a bright piece of paper on my desk.

Fit the tool to the process

This process evolved given the attributes of the constraints of our system. We all sit in the same room. We strive for face-to-face, verbal communication over all other forms. We don’t want to add overhead by tracking information that no one cares about. When someone suggests to use a software bug tracker, it needs to come from a need to have that information in that format, not because geeks like software.

But what if we had different constraints? What if we weren’t all in the same room? Fine, software it is! Until then, I’ll choose the absolute lowest overhead solution.

But how does it scale? Well, if you keep an inventory of bugs for more than a day, not too well. If bugs are fixed when they happen, it’s just not possible to have more than a dozen bugs out at a time. Scaling needs a vector, you have to understand in which direction it can scale. It scales with the size of the team, but not with geography or volume. That’s fine, we keep those two small and constant. Those choices were intentional, as separated teams and higher volume through poor quality or turnaround decreases delivery throughput.

Thanks Houston TechFest!

bogardj — Mon, 28 Sep 2009 05:41:01 PDT

This past weekend I gave a talk on one of my favorite topics – UI testing. In it, I focused almost exclusively on techniques for authoring maintainable UI tests, and how design for testability extends to views, models and controllers in MVC land. It’s trivial to write UI tests that break easily, but not much more difficult to write maintainable tests, given that you’ve intentionally designed your UI layer for UI testability.

Big thanks to the Houston TechFest organizers, who put on a fantastic conference, and to all the attendees, who had to put up with me for an hour. For those that are interested, you can find the code and slides here:

Testing the Last Mile in ASP.NET MVC code and slides

I had a lot of people asking about the wrappers on top of our browser automation tool, WatiN, and if they were open source. The answer is no, not yet, but we’re working on it. A quick road to fragile UI tests is browser automation code directly in your UI tests, so the sample code provided should at least provide some ideas and guidance to craft your own wrappers. And as always, my door is always open for questions. My email door, at least.