Joshua Flanagan

Error detected while processing vimrc error when using UTF-8 characters in listchars variable

Joshua Flanagan — Fri, 16 Jul 2010 16:23:57 GMT

This error stumped me for a while, and I didn’t find any good answers on Google, so hopefully this can help someone else. I’m running gVim 7.2 on Windows – this information may not apply to other versions/platforms.

Problem

I’ve been slowly adding more functionality to my vimrc file. Today I learned about the listchars setting, which allows you to choose what placeholders to use when displaying whitespace (taking the tip from Configuring Vim right).

For my “trail” setting (spaces at the end of a line), I picked the Middle Dot Unicode character · (U+00B7 or RightAlt+0183). When I saved (:w) and reloaded the new settings (:so ~\_vimrc), the setting took effect with no problem. However, when I closed gVim and re-opened it, I was greeted with the error message “Invalid argument: listchars…”

Solution

After much trial an error, I determined that my _vimrc file must be encoded as UTF-8 with Signature. It was previously encoded as UTF-8 (w/o Signature) – which is how gVim encodes the default file. To fix it, I opened _vimrc in my old favorite Notepad2, clicked on the UTF8 in the status bar, selected UTF-8 Signature, and saved the file. The next time I started gVim, there was no error, and my trailing whitespace displayed with the Middle Dot.

Update: A comment from John Weldon suggested a much simpler solution that you can do with vim (no Notepad2 required).

(Optional) If it isn’t already UTF-8 (mine was), set the encoding:

:set fileencoding=utf-8

Tell vim to store the encoding signature:

:set bomb

And of course :w to save the file.

Linux Re-revisited

Joshua Flanagan — Mon, 05 Jul 2010 01:38:55 GMT

I have long been a Windows user, but I have a habit of installing Linux on a home machine every couple years to give it a chance to grow on me. It had never won me over to become my primary machine for various reasons: installation issues, driver issues, unable to run my games, unable to run the software I depended on, and no apparent advantage over what I was comfortable using. Until now.

Motivation

A few months ago I had come across a good Ruby on Rails tutorial, and had started dipping back into Rails on the side. Rails works on Windows, but you always feel like a second class citizen, as there always seem to be a number of tiny workarounds you have to deal with.

Around the same time, Tim Tyrrell had told me how he installed Linux via Wubi. It allows you to dual-boot into linux, without repartitioning your hard drive (very much like the Windows 7 boot to VHD feature), and gives you the ability to uninstall via Windows’ Add/Remove Programs. No commitment required.

A month ago this weekend, I attended the TXJS conference, and reminiscent of NFJS ‘06, it got me really fired up about the software world outside of my day job. Back in ‘06, it was Ruby on Rails. This year it was node.js. At the time, node did not run on Windows (the homepage now reports that it runs on Cygwin), and I felt like I was missing out.

The night I returned from TXJS, I installed Ubuntu 10.04 (via Wubi) on my primary machine. I downloaded the source for node.js, compiled and installed it, and ran my first”hello world” sample. It was all so much easier than I had anticipated. I suddenly felt there was hope for me, that I would be able to participate in this other side of computing that I had avoided for so long.

Impressions

The only installation/hardware issues I had this time around was trying to get my 2 external monitors working properly when my laptop was docked (and closed). It was incredibly frustrating and led to having to manually edit the xorg.conf – something I had hoped was in linux’s past. The good news is I did get it all working. The other usual suspects, wireless networking and printing, just worked, without any fuss.

On the plus side, it’s hard to overstate how nice it is to work on an operating system where 99% of the software is free and available via a package manager. To give you a sense, compare the experience of attempting to use the Subversion client on a machine where it has not been installed. From a Windows 7 Command Prompt, and an Ubuntu Terminal window, I typed: svn [enter]

Windows 7 response: 'svn' is not recognized as an internal or external command, operable program or batch file.

Ubuntu response: The program 'svn' is currently not installed. You can install it by typing: sudo apt-get install subversion

This is not some special case for Subversion – I was given a similar response any time I tried to use a program that was not installed. Do I have Mono installed? Type msc [enter] and Ubuntu tells me which package I need to install to get the Mono C# compiler. It is so much easier to stay focused on the task at hand when you don’t have to search for a website, download a file, and click through an installation wizard.

I’ve been able to do most of my day-to-day tasks without missing a beat. Gmail, Google Reader, and Google Docs had long ago replaced their installed software equivalents in my life. My personal finance software, Moneydance, can run on Windows and Linux, with the same license. I haven’t used my PC for gaming in years. I currently only boot into Windows for three reasons: upload running data from my Nike+ Sportband (no Linux support in sight), working from home in Visual Studio, and using Windows Live Writer for writing blog posts (although I’m sure I can find an alternative).

On a somewhat relevant note, the one suggestion from The Pragmatic Programmer where I’ve always felt the weakest was “Use a Single Editor Well”. I can zip around C# code with VS/ReSharper, but I would never consider VS for general purpose text editing. So I usually end up in Notepad2 – a nice tool, but not the type of editor a programmer should rely on. Vim has been getting a lot of attention lately, and it seemed a reasonable choice to try. I was well aware of Vim’s steep learning curve, and knew I did not want to try and adopt it for C# editing (I already had a good solution for C#, and I don’t think its an area where Vim’s capabilities would shine), but I knew I wouldn’t “get it” unless I used it for real work. I decided I would complete the entire Ruby on Rails tutorial entirely with vim. With a lot of help from a graphical cheat sheet tutorial and A Byte of Vim, it has been going surprisingly well. Becoming a competent vim user has made the Linux experience even more rewarding.

Using rake under IronRuby

Joshua Flanagan — Mon, 14 Jun 2010 22:22:19 GMT

Download and install IronRuby

Go to http://ironruby.codeplex.com/ and click on the Downloads tab. Grab the .zip file that is appropriate for your platform. There is also a Windows Installer, but I don’t know what it does, so I stick with the zip file. Extract it to a folder on your computer – I recommend c:\ironruby. Add c:\ironruby\bin to your path (if you already have a version of ruby installed, I recommend putting IronRuby in the path AFTER your existing ruby).

Download and install Rake

This is easy, as rubygems comes along with IronRuby. Open a command prompt and type:

igem install rake

In your c:\ironruby\bin folder, there are two new files: rake.bat and rake. I made copies of those files and named them irake.bat and irake, respectively. This is completely unnecessary, but helpful if you already have an existing ruby installed. irake will run a script with IronRuby, and rake will run a script with your existing ruby.

To test it, create a new file named rakefile.rb in a temporary folder. Add the following contents:

task :default do
  puts 'hello world'
end

Open a command prompt to the folder containing that file, and type irake. After an unfortunate pause, you should see the "”hello world” message. You now have a working IronRuby installation that can run rake scripts. If you have another version of ruby installed (and in your path), you can execute the same script by typing rake. You should see that it also (though much more quickly) puts the greeting message on screen.

Now for something interesting

If you ran the last demo under both versions of ruby, you witnessed the downside of using rake under IronRuby– the slow startup time. However, IronRuby does have one nice big advantage for .NET developers: the ability to easily call your .NET code from the script.

For example, I recently built a simple website using a Web Application Project, which compiles to a DLL. I wanted to be able to build out the database schema using my Fluent NHibernate configuration. I created a static BuildSchema method that does just that:

       1: public class Database 

       2: {

       3:     public static void BuildSchema()

       4:     {

       5:         new SchemaExport(GetConfiguration()).Create(false, true);

       6:     }

       7:  

       8:     public static Configuration GetConfiguration()

       9:     {

      10:         return Fluently.Configure()

      11:             .Database(MsSqlConfiguration.MsSql2008.ConnectionString(x =>

      12:             {

      13:                 x.Database("mydatabse");

      14:                 x.Server(".");

      15:                 x.TrustedConnection();

      16:             }))

      17:             .Mappings(map => map.AutoMappings.Add(

      18:             AutoMap

      19:             .AssemblyOf<Database>(t => typeof(Entity).IsAssignableFrom(t))

      20:             ))

      21:  

      22:         .BuildConfiguration();

      23:     }

      24: }

I cannot easily execute the DLL from the command-line (to rebuild the database as part of a build script). Normally I would have to create a separate Console application project, which calls BuildSchema from its Main method. However, with IronRuby, I can invoke this method with the following code:

       1: desc "Generate the database (requires IronRuby)"

       2: task :db do

       3:   $:.unshift('.\src\MyApp\bin')

       4:   require 'MyApp'

       5:   MyApp::Database.BuildSchema

       6:   puts 'Database rebuilt'

       7: end

Most of this is standard rake code. The interesting bits are in:

Line 3 - Extend $: (the array of paths that ruby searches for files to load) to include the location of my binary. Otherwise, IronRuby will not be able to find any of the assemblies my assembly depends on.
Line 4 - Load the libary built from the web application project (MyApp.dll)
Line 5 - Invoke the static Database.BuildSchema method

With IronRuby and Rake, it is trivial to create build script tasks that make use of your existing .NET code. No need to create special task libraries, as you would with NAnt or MSBuild.

Austin Code Camp 2010

Joshua Flanagan — Tue, 11 May 2010 18:09:36 GMT

A reminder that Austin .NET User Group’s Code Camp is in a couple days – Saturday, May 15 2010. Code Camp is always free, and registration is still open.

At 9am, I’ll be talking about “Leveraging conventions in .NET”. I’ll be walking through a few examples of how my team establishes and uses conventions in our day-to-day work to help us move fast.

Some technologies I’ll touch on:

jQuery – Should give you a good taste of its power, if you aren’t using it yet. There are a few more jQuery sessions at Code Camp so you can dive in deeper if you like what you see.
Fluent NHibernate – even if you haven’t used FNH or NHibernate, you’ll still be able to appreciate the examples
FubuMVC – should also be interesting to ASP.NET MVC developers

Adding git commit information to your assemblies

Joshua Flanagan — Thu, 08 Apr 2010 04:07:51 GMT

The problem

It is a fairly common practice of .NET projects hosted in Subversion to include the repository Revision number as part of the AssemblyVersion. For example, MyWidgets.dll might have a version of 1.2.0.5309, which means it was built from the snapshot of source code at revision 5309. This can be very valuable, as it makes it very easy to trace a binary in the wild back to its original source.

Many projects are now moving towards distributed version control systems (DVCS), like git, which do not use a simple counter to describe a commit. In git, each commit is represented by a SHA1 hash, which looks something like e5f08d4a9e15943a64014329bd6fd2f348a89b15. Assembly versions in .NET are made up of four 16-bit integers, so it is obvious we cannot include the SHA1 of a commit. We need another way to get a unique number.

A solution

The solution I’ve been using is to take advantage of the “number of commits since the most recent tag”, as made available through the git describe command. In order for this to work, you must have at least one tag in your repository. Every time you change the major or minor components of your version, you should create a new tag for that commit. I’ll start by marking commit e5f08d4a9e15943a64014329bd6fd2f348a89b15 as version 1.2.0 (notice I leave off the last version component, as that is the number we will generate):

$ git tag v1.2.0 e5f08d

(Note: tags are not pushed to remotes, by default. Whenever you create a tag that you wish to make public, you must use the --tags flag when you push. See the Sharing Tags section in Pro Git)

I can now use the describe command like this:

$ git describe --tags --long

which will output:

v1.2.0-115-g4642f26

The --tags flag allows me to use lightweight tags, and the --long flag makes sure the output is always in the same format. In this example, the number 115 is what we are looking for. That means there have been 115 commits since the most recent tag, which is v1.2.0. The characters after the “g” make up the short version of the current commit’s SHA1. Now its just a simple matter of parsing the describe output so that we can use the number in our assembly version.

An example

You can see a real world example in FubuMVC build script. It is a rake (ruby) script that uses the assemblyinfo task from Albacore to generate a CommonAssemblyInfo.cs file at build time, which is referenced by all of the projects in the solution. The relevant section is:

desc "Update the version information for the build"
assemblyinfo :version do |asm|
  asm_version = BUILD_NUMBER_BASE + ".0"
  
  begin
	gittag = `git describe --long`.chomp 	# looks something like v0.1.0-63-g92228f4
    gitnumberpart = /-(\d+)-/.match(gittag)
    gitnumber = gitnumberpart.nil? ? '0' : gitnumberpart[1]
    commit = (ENV["BUILD_VCS_NUMBER"].nil? ? `git log -1 --pretty=format:%H` : ENV["BUILD_VCS_NUMBER"])
  rescue
    commit = "git unavailable"
    gitnumber = "0"
  end
  build_number = "#{BUILD_NUMBER_BASE}.#{gitnumber}"
  tc_build_number = ENV["BUILD_NUMBER"]
  puts "##teamcity[buildNumber '#{build_number}-#{tc_build_number}']" unless tc_build_number.nil?
  asm.trademark = commit
  asm.product_name = "#{PRODUCT} #{gittag}"
  asm.description = build_number
  asm.version = asm_version
  asm.file_version = build_number
  asm.custom_attributes :AssemblyInformationalVersion => asm_version
  asm.copyright = COPYRIGHT
  asm.output_file = COMMON_ASSEMBLY_INFO
end

Lines 26-27: run the git describe the command, parse the output with a regular expression, and store it in the gitnumberpart variable
Line 29: store the full git commit SHA1 number in the commit variable (either using an environment variable available on the build server, or from the git log command)
Lines 31-32: just in case the machine running the build does not have the git client installed
Line 34: create the full version number by concatenating the BUILD_NUMBER_BASE variable with the gitnumber from describe. BUILD_NUMBER_BASE can either be hardcoded in the rake script, or read from a VERSION file. It contains the first 3 components of the version (ex: 1.2.0)
Lines 35-36: We rewrite our TeamCity build number to be the assembly version number followed by the sequence number provided by TeamCity (ex: #1.2.0.115-#330, where 330 is the sequence number from TC)
Line 37: We store the full commit SHA1 in the AssemblyTrademark attribute. We use this within the FubuMVC diagnostics pages to provide a link to the github page for the currently executing version of the framework.
Line 38: We append the full output of the describe command to the product name and store in AssemblyProduct (ex: FubuMVC v1.2.0-115g4642f26)
Lines 40-42: We write the various version attributes. Notice the AssemblyVersion attribute always uses zero for the last component of the version, instead of the ever increasing “commits since tag” number. This is on purpose, since AssemblyVersion is used as part of the assembly name, and should not change unless there are breaking changes. We only adjust the version on every build for AssemblyFileVersion, which is not used by .NET at all, but is visible within the File Properties dialog in Windows Explorer.

Teaching AutoMapper about our conventions

Joshua Flanagan — Fri, 19 Mar 2010 19:32:04 GMT

I often need to send data from my entities to the client in JSON format to enable rich AJAX functionality. It isn’t practical to serialize the entities directly, so they are first flattened to a data transfer object (DTO) before serialization. For example, the Case entity has a reference to a Contact entity, which has a FirstName property. On the Case DTO, I might have a ContactFirstName property to hold that value. We use AutoMapper to handle the chore of populating the properties of the DTO using values from the original entity.

Conventions for mapping our DTOs

AutoMapper requires that we declare up front which which entities will be mapped to which DTOs. It has a lot of intelligent defaults, so once we tell it that a Case can be mapped to a CaseDTO, it can figure out how to populate the ContactFirstName property on the DTO from Case.Contact.FirstName. However, there are some properties on the DTO which aren’t so straightforward, so we have to explicitly configure how they are mapped.

Our DTOs generally follow these rules:

For every entity reference property on the source entity, the DTO will have a property of the same name, but of type GUID, which holds the entity identifier. Ex: Case.Contact of type Contact maps to CaseDTO.Contact of type GUID.
For every entity reference property on the source entity, the DTO will have a property with the name plus the suffix “ViewURL”, which holds the URL for viewing the referenced entity. Ex: CaseDTO.ContactViewUrl holds the URL for the Case.Contact.
For every “list value” (think values that show in a drop-down) property on the source entity, the DTO will have a property with the name plus the suffix “Display”, which holds the localized display value. Ex: Case.Contact.Country will map the value “USA” to CaseDTO.ContactCountry, and the value “United States” to CaseDTO.ContactCountryDisplay.

Implementing our mapping rules with AutoMapper

It was very easy to teach AutoMapper about the first rule. It took just a single line of code in our configuration:

cfg.CreateMap<Entity, Guid>().ConvertUsing(entity => entity.Id);

This tells Automapper that any time it tries to map a property of a type that derives from Entity to a property of type GUID, execute the lambda to get the Id.

The other two rules were not so easy to apply. AutoMapper currently has no built-in support for defining conventions to build mappings based on aspects of the types or their properties (although I hear this is something being considered for the future). Instead, you have to explicitly declare the custom relationship between each type, and each custom property resolution. You end up with code that looks like this:

Mapper.Initialize(cfg =>
{
    cfg.CreateMap<Entity, Guid>().ConvertUsing(entity => entity.Id);
    cfg.CreateMap<Case, CaseDTO>()
        .ForMember(d => d.ContactViewUrl, map => map.ResolveUsing<UrlValueResolver>().FromMember(s => s.Contact))
        .ForMember(d => d.SiteViewUrl, map => map.ResolveUsing<UrlValueResolver>().FromMember(s => s.Site))
        .ForMember(d => d.InstalledPartViewUrl, map => map.ResolveUsing<UrlValueResolver>().FromMember(s => s.InstalledPart))
        // ... and the rest of the ViewUrl properties
        .ForMember(d => d.CaseTypeDisplay, map => map.ResolveUsing<ListValueResolver>().FromMember(s => s.CaseType))
        .ForMember(d => d.StatusDisplay, map => map.ResolveUsing<ListValueResolver>().FromMember(s => s.Status))
        // ... and the rest of the Display properties
        ;
    cfg.CreateMap<Site, SiteDTO>()
        .ForMember(d => d.SupportSiteViewUrl, map => map.ResolveUsing<UrlValueResolver>().FromMember(s => s.SupportSite))
        // ... and the rest of the ViewUrl properties
        .ForMember(d => d.PrimaryAddressCountryDisplay, map => map.ResolveUsing<ListValueResolver>().FromMember(s => s.PrimaryAddress.Country))
        // ... and the rest of the Display properties
        ;
    cfg.CreateMap<Part, PartDTO>()
        // ... all of the ViewUrl properties
        // ... all of the Display properties
        ;
    // ... and the rest of the Entities
});

The sample is abbreviated because showing all of the entities, and all of the ViewUrl and Display property mappings for each of those entities is just too repetitive to be interesting. Aside from the tedium of having to type it up the first time, it makes maintenance a lot more painful. Any time we add an entity or list value property to an entity, we would have to remember to go add the corresponding mapping to the AutoMapper configuration, or risk introducing bugs.

Declaring mapping rule conventions

The benefit of establishing conventions in your code is that different parts of the system can rely on those conventions, so you get a lot of functionality that just works the way you would expect. Since we consistently name our DTOs the same way, and we consistently name properties on the DTO the same way, we should be able to rely on the fact that these will always be populated the same way.

With a couple patches to AutoMapper (now available in the latest source), I was able to build some extension methods that allowed me to express our conventions. The previous (which had to be edited for length) code can now be rewritten as:

Mapper.Initialize(cfg =>
{
    cfg.CreateMap<Entity, Guid>().ConvertUsing(entity => entity.Id);
    cfg.CreateMapsForSourceTypes(x => x.CanBeCastTo<Entity>(),
        type => Type.GetType(typeof (ListItemMethodDTO).Namespace + "." + type.Name + "DTO"),
        (map, source, destination) =>
        {
            map.MapViewUrls(source, destination);
            map.MapDisplayValues(source, destination);
        });
});

Now the code expresses our convention: for every type that derives from Entity, map it to a type (in a given namespace) with the same name with a DTO suffix, and map all the ViewUrl and Display members appropriately.

I’ve posted the full source for our extension methods, but the CreateMapsForSourceTypes method is probably the only one that is generally applicable:

public static void CreateMapsForSourceTypes(this IConfiguration configuration, Func<Type, bool> filter, Func<Type, Type> destinationType, Action<IMappingExpression, Type, Type> mappingConfiguration)
{
    var typesInThisAssembly = typeof (AutoMapperExtensions).Assembly.GetExportedTypes();
    CreateMapsForSourceTypes(configuration, typesInThisAssembly.Where(filter), destinationType, mappingConfiguration);
}

public static void CreateMapsForSourceTypes(this IConfiguration configuration, IEnumerable<Type> typeSource, Func<Type, Type> destinationType, Action<IMappingExpression, Type, Type> mappingConfiguration)
{
    foreach (var type in typeSource)
    {
        var destType = destinationType(type);
        if (destType == null) continue;
        var mappingExpression = configuration.CreateMap(type, destType);
        mappingConfiguration(mappingExpression, type, destType);
    }
}

It simply takes a criteria for identifying your source types, a function for determining the destination type based on the source type, and a set of configuration steps that should be applied to each mapping. Goodbye repetitive code!

Adding variable output behavior to your FubuMVC actions

Joshua Flanagan — Mon, 08 Feb 2010 03:34:34 GMT

Introduction

A question I’ve received multiple times now when discussing FubuMVC’s flexibility: can I make a single controller action render a view, or json, or rss/xml, etc., based on a request header, or some portion of the URL? The short answer is yes, of course, as I imagine it is for just about any web framework. The more interesting questions are around how difficult it will be, how much code do you have to write (and therefore, maintain), and how invasive it will be to the rest of your design. In this post, I’ll demonstrate one potential solution to the problem, and hopefully answer those questions in a way that reflects positively on FubuMVC. All of the code in this post is available in the Ouptuts/VaryByAcceptHeader solution in the fubumvc-examples repository.

I’m going to return to my simple sample application which I discussed in my last FubuMVC post. It allows a user to make a list of movies they want to see. Originally, the movies/list action simply rendered a view to display the current list of movies, as well as a form to add more. I’m going to modify this action so that it can also return the list of movies as json or xml.

Behaviors

FubuMVC has a great mechanism for adding behavior to actions. We creatively call them Behaviors. A Behavior is just a class that contributes to fulfilling a request for a given route. For example, the movies/list route is currently fulfilled by executing two behaviors: OneInOutOutActionInvoker and RenderFubuWebFormView. To solve the given problem, I’m going to introduce a new Behavior: RenderVariableOutput.

RenderVariableOutput is fairly straightforward. It has two dependencies that are injected by the container: the IFubuRequest, and an instance of a ConditionOutput. The IFubuRequest contains all of the information about the request, giving me access to the HTTP headers and query string values. ConditionalOutput is a simple helper class that makes it easy for me to associate a predicate and a Behavior, and store them together in the container. It also holds a reference to another ConditionalOutput which it will pass control to if its own predicate fails. The RenderVariableOutput Behavior effectively gets a chain of these ConditionalOutputs, so that when it calls GetOutputBehavior, it will receive the Behavior for the first predicate that passed. If a matching Behavior is found, it is executed.

protected override DoNext performInvoke()
{
    var detector = _fubuRequest.Get<OutputFormatDetector>();

    var behavior = _outputs.GetOutputBehavior(detector);
    if (behavior != null)
    {
        behavior.Invoke();
    }
    return DoNext.Continue;
}

The other class you see in play is OutputFormatDetector. It is passed as to the predicates to determine which Behavior to use. This is a simple class that has two properties: Accept, and RenderFormat. The IFubuRequest has access to the same mechanism that is used for model binding. This means that when I ask it for an instance of an OutputFormatDetector (or any class), it will attempt to create a new instance of the class, and then populate the public properties with any values that it knows about. The Accept property will be populated with the value in the Accept HTTP request header. The RenderFormat property will be populated if it finds a value with that key – possibly from the query string, form post data, route parameters, etc.

Conventions

I now have a Behavior that can execute one of many potential output Behaviors, based on the Accept header or a RenderFormat parameter. There are two remaining tasks: declare the conditions for each potential output, and declare which routes should be able to render different formats.

One way to associate a Behavior with a route is by creating a Convention. A Convention in FubuMVC is a class that implements IConfigureAction. It has a single method, Configure(), which gives it access to the BehaviorGraph. The BehaviorGraph is the semantic model that represents everything needed to satisfy all of the routes in the application. By manipulating the BehaviorGraph, you can modify how each route is fulfilled.

For my sample application, I’m going to declare the following convention:

Apply the RenderVariableOutput behavior to every action (route) that returns an IEnumerable model, and is currently configured to render a view.

Note that this is just the convention I picked for this sample. The Behavior itself does not require an IEnumerable output model, I could have just as easily stated “every action that starts with the letter ‘G’. IEnumerable just seemed like good indicator of the kind of data someone might want to see in JSON or XML format.

I implemented this convention in a class named VariableOutputConvention. The convention also declares the conditions that would trigger the different outputs:

variableOut.AddOutput(a => a.RenderFormat == "json", json);
variableOut.AddOutput(a => a.RenderFormat == "xml", xml);
variableOut.AddOutput(a => a.AcceptsFormat("text/html"), view);
variableOut.AddOutput(a => a.AcceptsFormat("application/xml"), xml);
variableOut.AddOutput(a => a.AcceptsFormat("application/json"), json);

It uses the RenderFormat parameter as a manual override mechanism, before falling back to the Accept header. This allows me to test the behavior by appending a “?RenderFormat=json” query string, since it is more difficult to manipulate the Accept header sent by a browser.

As a final step, I apply the convention to my application by adding the following line to my FubuRegistry:

ApplyConvention<VariableOutputConvention>();

See it in action

We’ve created the Behavior and tied it to some routes. To see it in action:

Run the website from the sample code and go to http://localhost:59187/movies/list. Note that it renders a view, as it used to.
Add a few movies to the list.
Go to http://localhost:59187/movies/list?renderformat=json. You should see the list in JSON format.
Go to http://localhost:59187/movies/list?renderformat=xml. You should see the list, serialized as XML.

Conclusion

As you can see, adding multiple output capability to a web application built on FubuMVC is not very hard. The hardest part was in trying to figure out how to represent the predicates and their output behaviors in the semantic model so that they would flow out of the container properly. Beyond that, the functionality is implemented entirely in a few small classes.

The best part is how I was able to implement this new functionality without impacting the rest of my application design. I added a new capability to existing actions (well, one action in this example, but it could have applied to multiple actions, if more existed that matched the criteria) without changing the actions at all. I just applied a new behavior to the action. With a little work (for example, make the list of conditions/outputs configurable via a DSL), the code could be packaged up and re-used across multiple projects. You could just add an assembly to your application project, wire up the convention in your FubuRegistry, and enjoy the new functionality. Hopefully this helped demonstrate the power of FubuMVC’s conventionally applied behaviors.

How to resolve a binary file conflict with Git

Joshua Flanagan — Fri, 29 Jan 2010 03:47:35 GMT

When performing a merge in git, you might see the message:

warning: Cannot merge binary files: HEAD:somefile.dll vs. otherbranch:somefile.dll

Auto-merging somefile.dll
CONFLICT (content): Merge conflict in somefile.dll
Automatic merge failed; fix conflicts and then commit the result.

In this scenario, somefile.dll is a binary file that has been modified in both the current branch, and the branch you are attempting to merge in to the current branch. Since the file cannot be textually merged, you need to make a decision: do you keep the version of the file in your current branch, or the version in the other branch.

In TortoiseSVN, I was used to being able to right-click on the file in question and choose “Resolve using mine”, or “Resolve using theirs”. So what is the git equivalent?

Resolve using mine

The file in your working copy is still the copy from your current branch – in other words, it was not modified by the merge attempt. To resolve the conflict and keep this file:

git add somefile.dll 
git commit –m “My commit message for the merge”

Resolve using theirs

If you prefer to resolve the conflict using their copy, you need to get the version of the file from the branch you were trying to merge in:

git checkout otherbranch somefile.dll

Now that you have the correct version of the file in your working copy, you can mark it as resolved (by adding it), and commit:

git add somefile.dll
git commit –m “My commit message for the merge”

Note that in place of otherbranch, you can use any name (treeish) that refers to a branch: a local branch name (otherbranch), a remote branch name (origin/master), a specific commit SHA (980e3cc), etc. For example, if you were merging in from your remote when you received the conflict, and you wanted to resolve using the remote version, you would retrieve that copy of the file using:

git checkout origin/master somefile.dll

You then add the file and commit as described above.

UPDATE: There is a shortcut for getting the copy from the other branch (and it even uses the terminology I was expecting):

git checkout --theirs -- somefile.dll

FubuMVC – Define your actions your way

Joshua Flanagan — Mon, 18 Jan 2010 19:41:14 GMT

In this post I’m going to try and demonstrate the flexibility that FubuMVC allows in how you build your web applications. While FubuMVC is opinionated about some things (lean on your container as much as possible), it mostly gets out of your way to let you work the way you want. We know how valuable it can be to work with an opinionated framework that allows you to move quickly as long as you follow its conventions, but we also know how painful it can be when you find yourself fighting established conventions at every step. With FubuMVC, you define the conventions and build your own opinionated framework, on top of which you build your applications. But don’t let the sound of building your own framework scare you off, as I’m going to demonstrate how little effort that requires.

The sample application

I’ve built a very simple application that will help me illustrate the process. The application allows the user to make a list of movies they want to see.

There are three actions that the application handles:

Url (route)	Description
/movies/list	show the list of movies
/movies/add	add a movie to the list
/movies/remove	remove a movie from the list

I built three different implementations of the same application. All of the code is available in the src/Actions folder of the fubumvc-examples repository.

Controller/Action

The first version of the application (ControllerActionStyle) uses the familiar “controller/action” approach, where multiple actions are defined on a single controller. The first part of the route identifies the controller class, and the second part is the name of a method to call on that class. This convention is fairly easy to declare, since FubuMVC already does a lot of it by default.

Sidebar: Your conventions are declared by defining a class that derives from FubuRegistry and then passing it to a FubuBootstrapper (like FubuStructureMapBootstrapper) at application startup. FubuRegistry exposes a DSL for you to use to describe things like: how to identify methods that should be treated as actions, what the routes for those actions should be, how to decide the output for those actions, and if the output is rendered from a view, which view should it use.

The relevant parts in our registry are:

Actions.IncludeTypesNamed(x => x.EndsWith("Controller"));

Routes.IgnoreControllerNamespaceEntirely();

This tells FubuMVC to look at all available types whose name ends with the word “Controller”. By default, all public methods (with, at most, 1 input parameter) are treated as actions, so nothing further needs to be specified. It also tells FubuMVC to ignore the namespace on controller classes when building routes. By default, routes are defined as parts/of/namespace/classname/methodname (and the “controller” suffix of the classname is removed by default). By stating we want to ignore the namespace, our routes will look like classname/methodname.

Following this convention, I built a MoviesController class which contains methods for List(), Add(), and Remove().

public class MoviesController
{
    public ListMoviesViewModel List()
    {
		//
    }

    public AjaxResponse Add(AddMovieInput model)
    {
		//
    }

    public AjaxResponse Remove(RemoveMovieInput model)
    {
		//
    }
}

To verify that everything is configured correctly, I open my browser to http://localhost/myapp/_fubu/ to view the list of routes and what actions they will call:

Route	Action(s)	Output(s)
movies/add	MoviesController.Add()	Json
movies/list	MoviesController.List()	WebForm View '~/Controllers/List.aspx'
movies/remove	MoviesController.Remove()	Json

Controller-less Actions

The second version of the application (HandlerStyle) defines each action within its own handler class. This is sometimes referred to as the “controller-less action” approach. I started with the code from the ControllerActionStyle and pulled all of the actions out of MoviesController into their own classes ListHandler, AddHandler, and RemoveHandler in a Movies subfolder. They all have a single public method named “Execute” which does all of the work for that action. You might think that with such a drastic change in how actions are organized that I would need to write a lot of code to wire everything up again. But since I’m still following a convention, I just need to describe it to FubuMVC in my registry:

Actions
    .IncludeTypes(t => t.Namespace.StartsWith(typeof(HandlerUrlPolicy).Namespace) && t.Name.EndsWith("Handler"))
    .IncludeMethods(action => action.Method.Name == "Execute");

Routes.UrlPolicy<HandlerUrlPolicy>();

This states that actions are located by looking for public methods named “Execute” on types in the Handlers namespace whose name ends with “Handler”. Since my routes no longer follow the usual classname/methodname pattern, I defined a custom URL policy in HandlerUrlPolicy:

public class HandlerUrlPolicy : IUrlPolicy {
    public bool Matches(ActionCall call)
    {
        return call.HandlerType.Name.EndsWith("Handler");
    }

    public IRouteDefinition Build(ActionCall call)
    {
        var routeDefinition = call.ToRouteDefinition();
        routeDefinition.Append(call.HandlerType.Namespace.Replace(GetType().Namespace + ".", string.Empty).ToLower());
        routeDefinition.Append(call.HandlerType.Name.Replace("Handler", string.Empty).ToLower());
        return routeDefinition;
    }
}

The Matches method states that this policy only applies to actions whose type name ends with “Handler”. This allows us to have different URL conventions for different parts of an application. The Build method is where we state what the route should look like for each action.

Side note: If you are coming from ASP.NET MVC, you may have picked up on a difference in how routes are defined. With ASP.NET MVC, you define route patterns at application startup (ex: {controller}/{action}), and then as a request comes in, a route pattern is matched, and the MvcRouteHandler object uses the parameter values to try and locate an action (which may not exist). In FubuMVC, each route has its own IRouteHandler in the container which is tied directly its action up front. Every addressable action has an entry in the route table, so if a route is matched, we know exactly which action to execute. FubuMVC routes can still have parameters, but they are only used to provides inputs to the action – not to determine the action.

We build the first part of the route by taking the namespace of our action types (SimpleWebsite.Handlers.Movies) and chopping off the beginning part (SimpleWebsite.Handlers), which is the namespace of the HandlerUrlPolicy. The second part of the route is determined by taking the name of the type where the action is defined and chopping off the “Handler” suffix. Now when we browse to _fubu we see that our routes are still the same as they were in the ControllerActionStyle, but they map to our new actions:

Route	Action(s)	Output(s)
movies/add	AddHandler.Execute()	Json
movies/list	ListHandler.Execute()	WebForm View '~/Handlers/Movies/List.aspx'
movies/remove	RemoveHandler.Execute()	Json

REST-like

The final version of the application (EndPointStyle) defines each action in a RESTish manner (I’m not brave enough to claim RESTful). It is similar to the HandlerStyle approach in that each route is handled by a single class, but instead of a single “Execute” method, there is a method for each HTTP method (GET, POST, etc) that is valid for the endpoint. Once again, the convention is defined in our registry:

var httpVerbs = new HashSet<string>(StringComparer.InvariantCultureIgnoreCase)
    {"GET", "POST", "PUT", "HEAD"};

Actions
    .IncludeTypes(t => t.Namespace.StartsWith(typeof(EndPointUrlPolicy).Namespace) && t.Name.EndsWith("Endpoint"))
    .IncludeMethods(action => httpVerbs.Contains(action.Method.Name));

httpVerbs.Each(verb => Routes.ConstrainToHttpMethod(action => action.Method.Name.Equals(verb, StringComparison.InvariantCultureIgnoreCase), verb));

Routes.UrlPolicy<EndPointUrlPolicy>();

Actions are located by searching all types with the “Endpoint” suffix in the Endpoint namespace for methods named after an HTTP method. Routes are constructed in a custom URL policy (EndPointUrlPolicy) that has the same logic we used in the HandlerStyle example (namespace/classnamewithoutsuffix). In line 24, I add a policy that constrains the acceptable HTTP method for a route based on the name of the class method that handles it (ex: the Get() method should only respond to HTTP GETs). This enables us to tie the same route pattern to multiple actions, based on how the request is made.

To demonstrate this functionality, I added a new feature to the sample application: the ability to sort your movies in your order of preference, and have the system remember that order. When the movies are sorted (using the very cool jQuery sortable), a message is posted to movies/list endpoint with the new order. The Post() method on the ListEndpoint is invoked to save the order in the repository, as opposed to the Get() method on ListEndpoint which displays the current list of movies.

public class ListEndpoint
{
    public ListMoviesViewModel Get()
    {
		// show the current list of movies
    }

    public AjaxResponse Post(UpdateMovieListOrder input)
    {
		// save the order in which the movies should be displayed
    }

Another visit to the _fubu diagnostics page confirms the routes are wired up as we intended (note that movies/list shows up twice, with 2 different actions):

Route	Action(s)	Output(s)
[POST] movies/add	AddEndpoint.Post()	Json
[GET] movies/list	ListEndpoint.Get()	WebForm View '~/EndPoints/Movies/List.aspx'
[POST] movies/list	ListEndpoint.Post()	Json
[POST] movies/remove	RemoveEndpoint.Post()	Json

Wrap Up

If you compare the three implementations of the movie application, you will notice that very little had to change as far as FubuMVC is concerned. By changing a couple lines in my FubuRegistry and creating a custom IUrlPolicy with 5 lines of logic, I was able to establish conventions to let me build my application in my preferred style (controllers, handlers, or endpoints). Other .NET web frameworks may have the extension points to allow each of these styles, but I’d be willing to bet you would have to write a lot more custom code and possibly have to give up some of the out of the box functionality the framework provides (if you prove me wrong, leave a comment and I’ll update the post to link to your example).

Readable Regular Expressions Revisited

Joshua Flanagan — Sat, 19 Dec 2009 16:24:47 GMT

Many, many years ago (internet time), I proposed a fluent interface for composing regular expressions. People either loved the idea or hated it (or thought it was just ok). The intention was to try and tackle the opaqueness of regular expressions that might be embedded in your otherwise familiar C# source code.

I’ll confess I never used that approach in production code. It started as a thought experiment, and that’s as far as it went (for me, at least). However, I did pick up a great technique from the comments. William “OmegaMan” Wegerson suggested using RegexOptions.IgnorePatternWhitespace along with liberal usage of in-line comments. Here is a recent example from the fubumvc source:

const string propertyFindingPattern = @"
\{              # start variable
(?<varname>\w+) # capture 1 or more word characters as the variable name
(:              # optional section beginning with a colon
(?<default>\w+) # capture 1 or more word characters as the default value
)?              # end optional section
\}              # end variable";

Notice that the comments violate one of the main rules of good commenting: do not restate what the code says. Usually, someone reading your code is literate enough in the programming language that they can figure out “what” the code does, it just isn’t always clear “why”. But when it comes to regular expressions, I would guess that a majority of C# programmers need to look at a regex reference every time they try and decipher a pattern. Do them a favor and document what each part of the pattern does while you are writing it, since you’re probably looking at the reference already anyway. This should make it much easier for someone to follow (and modify) the code going forward. No fancy fluent interface required.

Big Visible TeamCity

Joshua Flanagan — Fri, 28 Aug 2009 20:00:34 GMT

Big Visible Cruise is a cool utility for adding a build information radiator to your team room. It makes your Cruise Control build status immediately clear with a large green (good) or red (bad) screen. My (admittedly limited) search for an equivalent solution for JetBrains’ TeamCity came up empty. TeamCity has a few built-in notification options, but none of them seemed to be the right fit. I toyed with many different possibilities, all of which would require more development or infrastructure than we wanted to allocate to the task. Luckily, we found a solution that cost us little time and requires very little deployment footprint.

On individual build configuration pages, there is an option to “enable status widget”. This makes the build status publicly available on your TeamCity server at http://<buildserver>/externalStatus.html. This is what it looks like when we expose the status of our two builds:

Now, we can put that up on a big monitor for everyone to see, but I don’t think it “radiates” any information – you still need to purposefully read it. However, through the power of GreaseMonkey and jQuery, we were able to modify that page to look like this:

Now, it is immediately clear to anyone in the room when one of our builds has a problem. The script also refreshes the page automatically every 15 seconds, which the default externalStatus.html page does not do.

You can easily recreate this using Firefox, the GreaseMonkey Add-on, and my bigvisibleteamcity.user.js GreaseMonkey extension. If you aren’t familiar with GreaseMonkey, just install the add-on, then drag the *.user.js file onto your browser window, and you will be prompted to install it.

You will need to edit the bigvisibleteamcity.user.js file to change the URL in the @include line to refer to the externalStatus.html page on your server.

I didn’t spend a lot of time on the styling, but the hooks are there if you want to. If you are familiar with javascript, it should be clear in the source how to change the refresh time, or to add support for multiple rows of boxes, etc. Let me know if you make any cool changes.

Credit due to Dovetail cohorts Chad Myers for the idea and Sam Tyson for bootstrapping me on GreaseMonkey scripting.

How we handle application configuration

Joshua Flanagan — Mon, 13 Jul 2009 00:38:15 GMT

We recently overhauled the way we handle configurable settings within our application (server names, email addresses, polling frequencies, etc) . I’m going to present our solution below, but its new enough that I’d like to hear feedback on how others approach the problem.

The goal

We want a configuration solution with the following traits:

No magic strings. The configurable values will ultimately be stored with an identifying string key, but we didn’t want client code to have to know and use that string. The potential for typos is high, and maintaining a parallel list of constants is tedious.
Strongly-typed values. The configured values will likely be stored as strings in a configuration file, but we never want client code to know that. If the configurable value is numeric, the consumer should get a number. Forcing the consumer to deal with type conversion (and related error handling) would create a lot of duplicate, error prone, “ceremony” code that would obscure the essence of what it is trying to accomplish.
Test friendly. Client code that depends on a configurable value should not be forced to have any “out of process” dependencies (database, file system, etc.). Ideally, we would like to avoid having to stub expectations or use a test fake for configuration. We would also like to avoid adding any “simple type” (int, bool, string) constructor arguments to our classes, since the automocker we use cannot handle them.
Tooling friendly. We will have a large number of configurable values in many different places around the codebase. Our solution should make it easy to build tooling to generate a sample configuration to document all of the possible settings. We also want to be able to validate an existing configuration to make sure all necessary values have been provided.

Though not necessary, it would also be nice to have:

Default values. Users should only have to configure the values that have no sensible default, or values they want to override. It should not be left up to the client code to handle checking for a configured value, or falling back to a hardcoded default. That’s more “ceremony” code, not to mention the fact that multiple clients that use the same configurable settings would all have to know the default.
App.config/web.config Storing application configuration settings in an app.config or web.config is a well-established convention in the .NET world. It already has support for encryption, multiple file overrides, and tooling. We should use it if it doesn’t cause too much pain.

Our solution in action

To explain our solution, I’ll use the example of a service that associates an uploaded image with a user profile. Users might upload images of all sizes, and in many different formats, but we want to make sure they are all stored in the same size and format. We want to give owner’s of our application the ability to set the standard size and format, so they must be configurable. We also want to make sure the path where images are stored is configurable. Consider the following implementation:

public class AvatarService : IAvatarService
{
    private readonly AvatarSettings _settings;
    private readonly IFileSystem _fileSystem;
    private readonly IImageService _imageService;

    public AvatarService(AvatarSettings settings, IFileSystem fileSystem, IImageService imageService)
    {
        _settings = settings;
        _fileSystem = fileSystem;
        _imageService = imageService;
    }

    public void SaveAvatar(User user, Stream originalImage)
    {
        var extension = _imageService.GetImageExtensionFromMimeType(_settings.PreferredMimeType);
        var pathToSave = Path.Combine(_settings.AvatarStoragePath, user.Id + extension);
        user.AvatarFilePath = pathToSave;
        var scaledImage = _imageService.ScaleImageWithCrop(originalImage,
            _settings.PreferredMimeType,
            _settings.DefaultWidth,
            _settings.DefaultHeight);
        _fileSystem.Write(scaledImage, pathToSave);
    }
}

As you can see, in order for the class to make use of these externally configured values, it just needs to take an instance of AvatarSettings in its constructor. At runtime, the settings will be injected by our composition tool (StructureMap), just like the file system and image services. The AvatarSettings itself is a simple POCO class:

public class AvatarSettings : DictionaryConvertible
{
    public AvatarSettings()
    {
        PreferredMimeType = System.Net.Mime.MediaTypeNames.Image.Jpeg;
        DefaultWidth = DefaultHeight = 64;
    }

    [ExpandEnvironmentVariables]
    public string AvatarStoragePath { get; set; }
    public int DefaultWidth { get; set; }
    public int DefaultHeight { get; set; }
    public string PreferredMimeType { get; set; }
}

public abstract class DictionaryConvertible
{
    private readonly List<ConvertProblem> _problems = new List<ConvertProblem>();
    public IEnumerable<ConvertProblem> Problems { get { return _problems; } }

    public void AddProblem(ConvertProblem problem)
    {
        _problems.Add(problem);
    }
}

I’ll discuss the DictionaryConvertible base class later, but I included it here to show that it does not bring along any extra baggage that would hinder the testability of derived classes. When testing client code like the AvatarService, we can prepare a specific test context by setting configuration values directly on an AvatarSettings instance. This is more natural than stubbing values on a fake. If we are testing a scenario that is not impacted by the configuration values, we can just let the automocker pass in a default instance.

To set the configurable values for the application, we just need to add corresponding keys to the appSettings section of the app.config/web.config file:

<appSettings>
  <add key="AvatarSettings.AvatarStoragePath" value="%ALLUSERSPROFILE%\DovetailCRM\Avatars"/>
  <add key="AvatarSettings.DefaultWidth" value="96"/>
  <add key="AvatarSettings.DefaultHeight" value="96"/>
</appSettings>

There are a few things worth noting here. First, I have not included a value for AvatarSettings.PreferredMimeType in the config file. If you look at the source for AvatarSettings, you’ll notice it is set in the constructor. That is how we declare default values. It allows us to avoid cluttering up the config file with a bunch of settings that have reasonable defaults, while still allowing the flexibility to override them if desired. This is illustrated by the DefaultWidth and DefaultHeight settings which have configured values that override the defaults from the constructor. Also notice that the DefaultWidth and DefaultHeight properties are declared as integers (even though the values are strings in the config file) so that client code does not have to do any type conversion.

The final thing to notice is the [ExpandEnvironmentVariables] attribute that decorates the AvatarStoragePath property. In addition to type conversion, the code that populates a settings object can use additional metadata to do further manipulation of the configured values. In this case, the value “%ALLUSERSPROFILE%\DovetailCRM\Avatars” will be converted to “c:\ProgramData\DovetailCRM\Avatars” before any client code ever sees it. This not only simplifies the client code, but eliminates duplication of logic when a settings object has more than one client.

How it works

So how does this all work? The Settings object needs to be populated from the configuration file, so we define an interface for this responsibility:

public interface ISettingsProvider
{
    DictionaryConvertible PopulateSettings(DictionaryConvertible instance);
}

Implementations take in an instance of a DictionaryConvertible derived type, and should return an instance of the same type with all of the settings populated. We have a single implementation, AppSettingsProvider, which gets values from the <appSettings/> section of the app.config/web.config. Under the hood it makes use of a class that knows how to cycle over the properties of an object and set them using values from a dictionary (very similar to the DictionaryConverter in fubumvc). Any problems encountered during the conversion are recorded using the AddProblem method of the DictionaryConvertible, hence the original reason for the base class. We were already using this code to populate the input models for our MVC controller actions (similar to the ModelBinders that were introduced in ASP.NET MVC), so it made sense to re-use it.

The final step is to tell StructureMap how to create instances of the settings classes so that they are properly injected into consumers. In our Registry, we add:

For<AvatarSettings>()
  .Use<AvatarSettings>()
  .EnrichWith((session, original) => 
      session.GetInstance<ISettingsProvider>().PopulateSettings(original));

This is a good example of making use of the advanced power of your composition tool (and a good argument against trying to make a Common Service Locator equivalent for service registration). We use the EnrichWith statement to tell StructureMap “when I ask you for an AvatarSettings instance, before returning it to me, you should first pass it through the PopulateSettings method of my configured ISettingsProvider.”

Of course, it wasn’t long before we started adding a number of these Settings classes to our codebase, and it became a tedious extra step to add this same registration code for each class. The next logical step was to create a custom type scanner:

public class SettingsScanner : ITypeScanner
{
    public void Process(Type type, PluginGraph graph)
    {
        if (!type.Name.EndsWith("Settings") || !typeof (DictionaryConvertible).IsAssignableFrom(type)) return;
        graph.Configure(r => r.For(type).EnrichWith((session, original) =>
        {
            return session.GetInstance<ISettingsProvider>().PopulateSettings((DictionaryConvertible)original);
        }).TheDefaultIsConcreteType(type));
    }
}

We could now remove the previous code from our Registry that registered a single class (AvatarSettings), and replace it with this code which has the same effect for ALL Settings classes in our codebase:

Scan(x =>
{
    x.TheCallingAssembly();
    x.With<SettingsScanner>();
});

Tooling support

One of our goals was that the configurable settings should be easy to document. With a growing number of Settings classes all over the codebase, it could be very easy to lose track of which values are needed in the configuration file. However, since all of the Settings class follow a convention (derive from DictionaryConvertible, and named with the “Settings” suffix), it is trivial to write a little utility to reflect over the code and write out a sample <appSettings /> section.

public void Execute(string[] args)
{
    var settingTypes = _container.Model.PluginTypes
        .Where(t => t.PluginType.Name.EndsWith("Settings") && t.PluginType.BaseType == typeof(DictionaryConvertible))
        .Select(t => t.PluginType);

    dumpSettings(settingTypes, Console.Out);
    Console.WriteLine();
}

private void dumpSettings(IEnumerable<Type> settingTypes, TextWriter output)
{
    var xml = new XmlTextWriter(output) {Formatting = Formatting.Indented};
    xml.WriteStartElement("appSettings");
    settingTypes.Each(t =>
    {
        var settings = Activator.CreateInstance(t);
        var properties = t.GetProperties(BindingFlags.Instance | BindingFlags.DeclaredOnly | BindingFlags.Public);

        properties.Each(p =>
        {
            var key = AppSettingsProvider.DefaultNamingStrategy(p);
            var value = p.GetValue(settings, null);

            xml.WriteStartElement("add");
            xml.WriteAttributeString("key", key);
            xml.WriteAttributeString("value", value == null ? "" : value.ToString());
            xml.WriteEndElement();
        });
    });
    xml.WriteEndElement();
    xml.Close();
}

Summary

All of this comes together to allow a very natural workflow when we decide we need to make a value in our code configurable:

Create a new Settings class (or append to an existing class) that derives from DictionaryConvertible and add the properties for each of the configurable values we need. Optionally initialize the properties with a default value in the constructor.
Add the Settings class as a constructor parameter in the client class that needs the configurable value.
After the client class is done and tested, add an <appSettings /> entry to the application configuration file for each setting without a default value.

Real World Refactoring

Joshua Flanagan — Mon, 29 Jun 2009 00:35:29 GMT

If you’ve ever asked, or been asked, for an example to illustrate a software design principle, you know how frustrating it can be to work with a contrived example. There is rarely any depth to the example, and it lines up so neatly with the concept being explained that the student may have trouble recognizing similar situations in the wild. I’m going to try and relate a real world example of a recent refactoring effort, with the hope that the extra context and narrated walkthrough will help someone make a connection that has failed before.

A little background

My discussion revolves around Docu, which is an open source project for converting .NET XML comments into HTML documentation. (Note: I want to make clear is that this is not a criticism of any of the Docu code. All code is written with specific goals and constraints in mind. As the goals evolve, some designs that worked perfectly in early iterations may start to create friction, and deserve to be reconsidered.) .NET XML comments are constructed by applying a set of top-level tags to code elements such as classes, methods, or properties. Top-level tags are things like <summary/>, <remarks/>, or <param/>. Within the contents of the top-level tags, you can use embedded tags to provide additional contextual information. Embedded tags are things like <see/>, <para/>, or <paramref/>. The following example shows a top level <remarks/> tag, with embedded <paramref/> and <para/> tags:

///<remarks>Make sure the <paramref name=”maxValue” /> is a positive number.
///<para>Do not call more than once.</para></remarks>

In Docu, the job of translating the XML contents of a single top-level tag into a semantic model is handled by the CommentParser. Let’s take a look at the original implementation:

public class CommentParser : ICommentParser
{
    private readonly IDictionary<Func<XmlNode, bool>, Func<XmlNode, bool, bool, IComment>> parsers =
        new Dictionary<Func<XmlNode, bool>, Func<XmlNode, bool, bool, IComment>>();

    private readonly InlineTextCommentParser InlineText = new InlineTextCommentParser();
    private readonly InlineCodeCommentParser InlineCode = new InlineCodeCommentParser();
    private readonly MultilineCodeCommentParser MultilineCode = new MultilineCodeCommentParser();
    private readonly SeeCodeCommentParser See = new SeeCodeCommentParser();
    private readonly ParagraphCommentParser Paragraph;
    private readonly ParameterReferenceParser ParameterReference = new ParameterReferenceParser();
    private readonly InlineListCommentParser InlineList;

    public CommentParser()
    {
        Paragraph = new ParagraphCommentParser(this);
        InlineList = new InlineListCommentParser(this);
        parsers.Add(node => node is XmlText, InlineText.Parse);
        parsers.Add(node => node.Name == "c", InlineCode.Parse);
        parsers.Add(node => node.Name == "code", MultilineCode.Parse);
        parsers.Add(node => node.Name == "see", See.Parse);
        parsers.Add(node => node.Name == "para", Paragraph.Parse);
        parsers.Add(node => node.Name == "paramref", ParameterReference.Parse);
        parsers.Add(node => node.Name == "list", InlineList.Parse);
    }

    public IList<IComment> Parse(XmlNodeList nodes)
    {
        var blocks = new List<IComment>();

        int count = nodes.Count;
        for(int i = 0; i < count; i++)
        {
            XmlNode node = nodes;
            bool first = (i == 0);
            bool last = (i == (count - 1));

            foreach(var pair in parsers)
            {
                var isValid = pair.Key;
                var parser = pair.Value;

                if(!isValid(node))
                    continue;

                var block = parser(node, first, last);

                if(block != null)
                {
                    blocks.Add(block);
                    continue;
                }
            }
        }

        return blocks;
    }

You can see (lines 18-24 above) that in its constructor it builds up a collection of function (Func<>) pairs: one function that is used to identify a specific embedded tag, paired with another function that knows how to parse that tag into its model representation (an instance of IComment). The collection is used by the primary method in CommentParser, which iterates over all of the child nodes of a given chunk of XML, finds a parsing function that can handle that node, invokes the function, and collects the results.

The functions that perform the parsing of individual embedded tags are implemented in separate classes (I’ll call them “node parsers”). The CommentParser creates an instance of each node parser and stores it in a field where it can be referenced by the collection of function pairs. The benefit of moving the embedded tag parsing into separate classes is that they can be developed and tested independent of CommentParser. Unfortunately, you need to make a number of changes to CommentParser every time you add a new node parser. Since there are still quite a few embedded tags that are not yet recognized by Docu, and each embedded tag (generally) requires a new node parser, the CommentParser class will be continuously modified and unstable. The primary goal of my refactoring effort is to create a more stable CommentParser that is open for extension (support for new embedded tags can be added) but closed for modification.

High Cohesion

As stated above, the logic for determining when a specific parsing function should be applied was inside of CommentParser, while the logic to implement that function was in the individual parsing classes. These two pieces of logic are tightly related to each other. You cannot safely apply an arbitrary node parser to any node – it only makes sense to apply a parser to the type of node it was designed for. We can make the node parsers and the CommentParser more cohesive by moving both pieces of logic into the node parsing classes. I’ll add a CanParse(XmlNode) method to each node parser. The method returns true if the parser knows how to parse a given comment node. I implement them by copying the logic for identifying specific tags from the Func<> pairs in CommentParser.

Low Coupling

We still have the problem that the CommentParser creates and stores an instance of each specific node parser class (lines 6-11 above). This high coupling between the CommentParser and the node parsing classes makes it impossible to execute in isolation. You cannot use a CommentParser without bringing all of the functionality of all node parsers along. You also have to modify the CommentParser every time a new type of node parser is added (as when adding support for a new type of comment tag).

I’ll reduce coupling between CommentParser and individual node parsers by introducing an interface (ICommentNodeParser) to describe the functionality exposed by the node parsers. The CommentParser will only interact with this interface, which exposes the CanParse and Parse methods. I then change the node parsers so that they implement the new interface. Since they already had all of the needed functionality, it was just a matter of making sure the method signatures matched correctly.

Slight detour

When I started to add the interface to all of the node parsing classes, I noticed they all derived from CommentParserBase. However, none of the code in the application referred to these classes through this base class. That’s a pretty good indicator that inheritance is being used to share common functionality rather than for polymorphism. Using inheritance just to share some common functionality can lead to more inflexible designs and less cohesive classes. You are usually better off using composition instead of inheritance in these scenarios. In this case, the only shared functionality was a single helper method that had some special logic for string trimming. Since the method didn’t make use of any instance data, it was easy to move it to a separate class as an extension method on string. Now that CommentParserBase was empty, there was no reason to keep it around, so it was eliminated. The fact that no code broke when the base class was deleted (without using a refactoring tool) is a good indicator that it was the right decision.

Composition through dependency injection

At this point we have a bunch of node parsers that all implement a common interface. But the CommentParser is still tightly coupled to the various implementations because it has to create the instances. This is the perfect opportunity to use dependency injection to pass the node parser instances into the CommentParser. I just change the constructor of CommentParser to require an array of ICommentNodeParsers, and delete all the code that was creating the node parser instances. Whoever creates an instance of CommentParser will need to pass in the collection of node parser instances. CommentParser is no longer coupled to the specific node parsers and can be more easily used in isolation. You can see that the newer version is greatly simplified:

public class CommentParser : ICommentParser
   {
       private readonly ICommentNodeParser[] _parsers;

       public CommentParser(ICommentNodeParser[] parsers)
       {
           _parsers = parsers;
       }

       public IList<IComment> Parse(XmlNodeList nodes)
       {
           var blocks = new List<IComment>();

           var count = nodes.Count;
           for(var i = 0; i < count; i++)
           {
               var node = nodes;
               var first = (i == 0);
               var last = (i == (count - 1));

               var parser = _parsers.FirstOrDefault(p => p.CanParse(node));
               if (parser == null) continue;

               var block = parser.Parse(this, node, first, last);
               if (block != null)
               {
                   blocks.Add(block);
               }
           }

           return blocks;
       }

At this point you may be thinking I just “passed the buck” for creating the individual node parser instances to another class up the stack. Someone still needs to create them, and will therefore be tightly coupled to the implementations. That is true. However, if we can pass that responsibility up the stack far enough, it can be handled by code that has no other responsibility than to configure and bootstrap our application. That type of code doesn’t typically have any logic that would be re-used and doesn’t have the same concerns about designing for maintainability. Luckily, in the case of Docu, we already use StructureMap to compose our object instances so we get this functionality for free. I simply had to add a single line to the StructureMap configuration to tell it to make use of any implementation of ICommentNodeParser it finds in the Docu assembly.

public DefaultRegistry() {
  Scan(x =>  {
    x.AssemblyContainingType<DocumentationGenerator>();
    x.WithDefaultConventions();
    x.AddAllTypesOf<ICommentNodeParser>();

Now when an instance of CommentParser is requested from StructureMap, it will automatically have the array of all node parsers injected to its constructor.

Wrap up

So what did we gain? Let’s compare the stories for adding the ability to parse a new XML documentation tag. In the original implementation, we had to:

Create a class with code that parses the new tag
Create a new field in CommentParser to hold an instance of the new parsing class
Modify CommentParser’s constructor to register the new class, along with a predicate that determines when it should be used

And now we simply:

Create a class that implements ICommentNodeParser with code that recognizes and parses the new tag

Notice that you didn’t have to touch CommentParser? We just gave it new functionality without having to change (and potentially destabilize) the code. That’s the open-closed principle in action. The increased cohesion of the node parsers makes them easier to understand and helps localize any future changes to their implementation. We also saw that the use of dependency injection and a composition tool (StructureMap in this case) made it painless for us to pull code apart into separate classes. The code was made easier to maintain by applying a few established object oriented design principles. Hopefully this example helped clarify the application of these principles. Your feedback is appreciated.

For additional context, you can view the commit that contained the changes discussed above.

Integrating a custom test runner with TeamCity

Joshua Flanagan — Thu, 11 Jun 2009 00:45:15 GMT

JetBrains’ TeamCity build server provides a wealth of information about the tests that are run as part of your build. If you use one of the test runners that they support out of the box, you automatically get real-time individual test results, detailed timing reports, and historical performance metrics.

Up until recently, all of the tests for my team were run through NUnit – C# unit tests, QUnit JavaScript tests, and integration tests. Since NUnit is natively supported by TeamCity, we got used to the rich test reporting. However, we recently started to integrate our automated acceptance tests into the build. Our acceptance tests are written using StoryTeller, which are executed using its own test runner. The StoryTeller test runner has slightly different behavior than the NUnit runner. For one, a failed assertion does NOT stop execution of a test – it continues executing so that a single test can have multiple failed assertions. We also have the ability to mark individual tests with their state in the development lifecycle: they start as "acceptance” tests when they are first defined, and then are marked as “regression” once all supporting code has been correctly implemented. The idea is that you can drive your development with failing acceptance tests until they pass, at which point they become regression tests. This is relevant because a failed acceptance test does NOT fail the build – only a failed regression test will fail the build.

The StoryTeller test runner is a console application that will return an error code if any regression tests fail. It also generates an HTML report which contains the details for each test. Our initial attempt at integrating it into our build allowed us to fail the build if any test failed, but we didn’t get much feedback about why it failed. We would have to go searching through the build log messages or HTML report files for details.

I created a new StoryTeller ITestListener (TeamCityTestListener) which is used by the StoryTeller runner to log test results. It writes the test details to the console using TeamCity’s Service Message API (which I mentioned previously). TeamCity picks up these messages, and creates the nice individual test reports, complete with timing and history:

Now we know exactly which tests failed, but we don’t know which assertions in the tests failed. As I mentioned, the complete details of the test execution are recorded in an HTML report. It’d be nice if we could view individual test reports right within TeamCity. Turns out, that is pretty easy to do as well. I followed the documentation for Including Third-Party Reports in the Build Results, and added the following line to our <TEAMCITY>\.ServerConfig\.BuildServer\config\main-config.xml file:

<report-tab title="StoryTeller" basePath="st-results" startPage="st-results.htm" />

We now have a nice new StoryTeller tab next to the other build result tabs:

We configured the build to store all of the HTML reports as artifacts, and point our custom tab to the test results index HTML file generated by StoryTeller. Clicking the StoryTeller tab displays that index file with a color coded overview of all of the tests. Clicking a test brings up the report for that test so that I can see exactly which steps failed.

Let your Inversion of Control tool work for you

Joshua Flanagan — Wed, 10 Jun 2009 03:58:51 GMT

If you are just starting out exploring use of an Inversion of Control tool (IoC), its very easy to go down the wrong path, and make things harder for yourself. Today I had a conversation with a bright developer who was feeling the pain because doing it all wrong. And just a month or so ago, I had the same conversation with another bright developer who was also doing it the same, wrong, way.

The problem may be that people misinterpret the intent of an IoC tool. It is seen simply as a configuration tool – a way to swap out classes using some magical XML or a DSL. The thought is that as long as your code gets all of its dependencies out of the container, you can change the behavior just by configuring the container.

And then you end up with code that looks like this:

public class OrderService : IOrderService
{
    private IRepository _repository;
    private ISystemClock _systemClock;

    public OrderService()
    {
        _repository = Container.Resolve<IRepository>();
        _systemClock = Container.Resolve<ISystemClock>();
    }
}

public class OrderController
{
    private IOrderService _orderService;

    public OrderController()
    {
        _orderService = Container.Resolve<IOrderService>();
    }
}

The OrderService needs an IRepository and an ISystemClock, so you pull them out of the container in the constructor. The OrderController needs an IOrderService, so you pull it out of the container in the constructor. You can now swap out actual implementations using some external configuration.

Except now you’ve got this annoying dependency on the magical Container class. And maybe the Container has a dependency on an external XML file.

And you discover that when you try to unit test your OrderService, you need to make sure you have a Container available, and that it has been configured to serve up whichever instances you want to use during testing.

Your simple, plain-old CLR object (POCO) now has this deadweight Container dependency dragging it down. The Container requires extra care and feeding, in the form of configuration, and now it has infected your entire codebase.

Hopefully, at this point you are feeling enough pain that you go seeking some advice, and hopefully you find out about auto-wiring.

Any IoC tool worth using will support auto-wiring dependencies for objects retrieved from the container. This means that the tool will build up all of the necessary objects needed to satisfy a request. This allows you to rewrite the above code as:

public class OrderService : IOrderService
{
    private readonly IRepository _repository;
    private readonly ISystemClock _systemClock;

    public OrderService(IRepository repository, ISystemClock systemClock)
    {
        _repository = repository;
        _systemClock = systemClock;
    }
}

public class OrderController
{
    private readonly IOrderService _orderService;

    public OrderController(IOrderService orderService)
    {
        _orderService = orderService;
    }
}

But where did the Container go? Exactly. The majority of your code should never have to deal with the container directly. Ideally, you make a single call to the container which builds up an object graph that implements your program. For a console application, you might make a call to the container once in your static Main method. For an MVC web application, you might make a call to the container at the beginning of each request to retrieve the appropriate Controller. The rest of your code is blissfully ignorant of the container. When something asks the container for an OrderController, the container will detect the dependency on the IOrderService. So it goes to build an OrderService, and detects the dependencies on the IRepository and ISystemClock. So it builds up those instances, and any dependencies they may have, and so on.

By the way, my colleague Jeremy Miller wrote this same post just a few months ago. But he assured me it was worth re-writing, because there are new people jumping on all the time, and they’re not likely to go back reading old blog posts. If you find yourself with the problem I describe above, go get the full details by reading his post about auto-wiring in StructureMap.

One thing that I’ll add is that I’ve heard a lot of talk lately that “container” is really a misnomer for what an IoC tool does, and that it should be more appropriately referred to as a “composer”. I think this argument has merit, and wonder if the name may be contributing to people going down the wrong path described above. A “container” sounds like something that stores a bunch of stuff for you, and it is your job to get stuff out of it, hence the explicit calls to the Container littered throughout your code. A “composer” sounds like something that takes little pieces of functionality in your application and puts them together into a usable whole. The bad code example above reflects “container” thinking. The better code example reflects “composer” thinking.