Murray on .NET

FubuMVC Webinar - Compositional and Convention-based Web Development

2011-07-21T12:44:00.001-06:00

I just gave a webinar yesterday for JP Boodhoo’s Develop With Passion® webinar series on Compositional and Convention-based Web Development with FubuMVC. The recorded session is now posted online with a slew of referenced and related links. I apologize for the sound quality and will make corrections if I get another chance to do another webinar/screencast. Here is the link to the recording:

http://webinars.developwithpassion.com/webinars/3

I didn’t mention this in the screencast but before we began switching our company’s production website to FubuMVC, we first devised a way to get the benefit of FubuMVC Behavior Chains while still on the ASP.NET MVC framework. Please feel free to read more about that successful venture at Bob Pace’s blog post on the MvcToFubu project.

FubuMVC Behavior Chains - The BMVC Pattern

2011-06-13T16:01:00.001-06:00

For the last several months at work, we have been gradually migrating our production website to use the .NET open source FubuMVC web framework. Before committing to a whole new infrastructure for our site, some brilliant talent on our team decided to improve our experience on the ASP.NET MVC framework by integrating in one of FubuMVC’s most compelling features, behavior chains. Bob Pace originally blogged about the process in a two part blog post series (part 1 and part 2); since then, he has created a packaged solution called MvcToFubu which he just recently started another blog post series about.

Behavior chains are by far my favorite feature of FubuMVC. I believe they single-handedly revolutionized our use of the MVC pattern in a web environment. In fact, I like to think of it as a whole new pattern, the BMVC Pattern (I was told by a coworker that the ‘B’ should stand for “Better”, and while that’s true, I’ll stick with “Behaviors”). No, I’m not sincerely trying to coin yet another design or architectural pattern buzzword. However, I am most sincerely not joking when I say behavior chains add a whole new dimension to how I design web architectures. I find myself creating composable pipelines of cohesive, testable components in lieu of monolithic controllers and actions.

What are Behavior Chains?

Have you ever looked at your controller actions and thought “Man, there’s a lot going on here!” (validation, authorization, transactions, ORM setup, logging, caching, etc.)? If you haven’t, I urge you to take a conscious look at your controller actions and try to identify code that doesn’t directly relate to the purpose of the page (or sub-view of the page). Shouldn’t there be a way to move some of this secondary infrastructure code out of our controller actions and have it be applied conventionally to the correct actions/routes? Yes, and this is exactly what Behavior Chains are meant to accomplish, the ability to create composable pipelines of reusable behavior nodes for each route in your site.

Behavior chains can be built up in any order you want for each route via Conventions and Policies, which are extension points that allow you to inspect and modify the entire Behavior Graph that was built by FubuMVC during startup bootstrapping. The behavior graph is the modifiable collection of all behavior chains in the site, initially built with just a few default behavior nodes for each route. A typical behavior chain by default usually has an ActionCall node (representing the call to a controller action) and an Output node (often a tie-in to a view rendering engine, but can also be JSON or anything else). Other behavior nodes can be sprinkled into the pipeline in any order in front of or in between the ActionCall and the Output nodes. As mentioned before via the use of Conventions and Policies, it is common practice to filter your list of behavior chains using LINQ extension methods and then add a specific behavior node into the matching chains at a specific location in the chain. This truly elevates coding to conventions to a new level; some of the filtering criteria regarding controller actions that you may come up with could be as follows:

Action method names that contain a certain word.
Actions methods that are located in a particular controller or namespace.
Action method signatures that contain an input/output model that implements a specific interface.
Action method signatures that contain an input/output model that contains a property of a certain type or have a particular C# attribute applied to it.
So on, so forth…whatever you can dream up!

One more aspect of behavior chains that is important to point out is the logistics in how behavior nodes are chained together. The chains are not merely a linked-list of behavior nodes that are traversed linearly. Instead the makeup of the chains is better conceptualized to be an outer behavior node wrapping around an inner behavior node, which is then itself an outer node wrapped around another inner node, and so on. In fact, the FubuMVC developers like to refer to this chaining architecture as the Onion Layer Model or the Russian Doll Model (conjuring up a mental image of the nesting Russian doll toy called a matryoshka doll or a babushka doll). When an outer behavior node has finished executing it’s logic, it can then optionally execute it’s inner behavior node logic (or stop the behavior chain entirely in order to return early with a response, an error, or even transfer to a different route and behavior chain). One of the key advantages that this wrapping architecture can afford is the ability for an outer behavior to run additional logic or clean up code after all inner behavior nodes have completed (i.e., transactions can be committed, connections can be closed, sessions can be disposed, etc.).

How does this change the way I code my controllers?

I find that my controller actions are much thinner and more focused on the base case and the successful code path. All the exceptional code paths and edge cases have been identified and deal with by the other behavior nodes in the chain, prior to my action call being invoked. My action call can now focus on the essential logic that differentiate this route from others. That essential logic doesn’t get lost anymore in the necessary infrastructure code that should be a secondary concern. And all of this logic can be broken up into cohesive, reusable components within a flexible request/response pipeline that can be built up automatically by coding to whatever conventions you deem beneficial. Behavior chains really do add a whole new first-class citizen to the MVC design pattern. Do yourself a favor and check out the MvcToFubu solution mentioned at the top of this post, and bring some FubuMVC behavior chain goodness to your ASP.NET MVC website. I think you might be as impressed as we were and yearn for the other killer features of FubuMVC as well.

.NET Needs More Apprenticeships

2011-06-03T15:50:00.001-06:00

It came to my attention recently that there are a few companies in other dev circles with all-star developers who give back to their communities by doing apprenticeship programs or open door programs at their workplaces. Basically the idea is to let people come into your workplace and pair program with them, sharing knowledge and best practices. I believe we need more of this sense of community going on in the .NET space. I’m sure I’ve probably overlooked something (or perhaps I’m about to spout off craziness and heresies), but it just seems like we should be able to work out the logistics to make this a reality.

The interesting thing is that this really isn’t a new concept in other professions. For example, the culinary world has had the idea of staging for quite a while. Sometimes it’s a final part of an interview process and other times it’s just to swap techniques and recipes. As mentioned above, I was recently informed of a few dev companies that have taken this idea to our profession. In the Rails community, Hashrocket has an apprenticeship program where you can apply to be a part of their team for a week and soak in some of the awesome skills and techniques they possess. It appears they vet their candidates a bit to ensure they get the right kind of people coming in who are motivated and ready to learn, and also because the code being worked on is for real Hashrocket projects for their clients. In the Node.js space, Joyent has a program called Office Hours where devs can register for free in advance to bring in something to work on with a Joyent employee at end of work day and get their feedback & expert advice. They provide snacks, drinks, and a comfortable work environment.

I think these ideas are brilliant and foster real community among developers. Although this idea alone can improve the interview and hiring process, I think it can go further than that to help our profession mature in the sharing of best practices and proven techniques. For most .NET companies out there, I’m willing to bet there are few legitimate roadblocks that would prevent instituting programs like these. Being completely honest with yourself, how much of what you work on each day is top secret intellectual property that is core to the success of your company and absolutely cannot be seen by outsiders? Even if there are parts of your code that should not be seen by strangers, I’m sure there are projects and tasks to be done on the other parts of code that will still prove instructive for your invited guests. I encourage our dev community to do whatever it takes to make this happen: have the visitors sign non-disclosure agreements, interview and re-interview potential candidates to meet satisfactory requirements, don’t allow them to hook up thumb drives to your machine and steal your secrets, whatever! It doesn’t matter if it’s a week-long thing or a few hours; let’s realize we’re all in this together and build this community up to have a higher standard of software craftsmanship.

Develop with Passion bootcamp course - Denver 2011

2011-05-31T09:27:00.000-06:00

A very talented coworker and I recently went to JP Boodhoo’s Develop with Passion bootcamp training course (previously known as Nothin but .NET). I had originally heard of the course from Dave Tchepak who attended the course nearly two years ago. This blog post is a wrap-up of my thoughts and feelings about the course itself, JP Boodhoo as the instructor & experienced craftsman, and the .NET development community in general.

My prior impressions of the course from what I had read was that I would be in awe of JP's coding and design skills and very overwhelmed with the feeling that I didn't really know how to code; also that I would get no sleep at all that week, going from early morning to past midnight every day. I had also heard from a previous course member (off the record) that JP’s organization and direction with course material was a bit disorganized back then (luckily this has changed significantly since he had taken the course). I was really excited to see some good BDD in action, get my TDD passion ignited again (I had actually mentioned on twitter recently that my motivation for test was waning as of late), and see JP's mad Vim skills (since I have gotten into that the last few months).

Overall, I was pleasantly surprised with the fulfillment of most of my positive expectations and initial impressions going into the course. JP's personality and passion are infectious, and his teaching skills are pretty top-notch. The course material was well-structured and there was a clear direction for what we hoped to cover throughout the week (and yet, he was still flexible enough to be able to briefly entertain an off-topic direction the students wanted to take the course). He stayed true to the advertised 9am-9pm every day no matter what. He said that when he did the late nighters with previous courses, people came back the next day with little sleep and trashed as far as coding & learning capabilities go.

Watching him code so efficiently is quite a treat. He automates anything he does more than once so that a particular task doesn’t ever get in his way and hold him back the next time. His AutoHotkey scripts and automation tools that he has put together are absolutely incredible. He briefly touched the mouse two times the entire week, and it only for when one of his programs or Windows VM was wigging out. Screens and dialogs flew by so fast, you could tell he wasn't going to let any UI (even Visual Studio) get in the way of his productivity to code. He mentioned he will be open sourcing some of his scripts that he uses to keep his development environment consistent across OS platforms. Definitely cool stuff.

As far as JP's actual coding/design skills, I am absolutely floored. He certainly understands C# and .NET well. But it's his Object-Oriented design skills that blew us away. I've realized I don't really know OO design. I think too mechanically about the implementation of the solution instead of focusing on breaking down the problem into smaller manageable chunks. His TDD skills (both Top Down Design and Test Driven Development) are amazing. I've never been a fan of the TDD samples and intros I've seen (where the instructors do the absolute dumbest thing to make the test pass; JP admitted that style didn’t jive with him either). I've always thought that Behavior Driven Development and Top-Down Design were the way to go if I were to have tests drive my designs (or have tests at all for that matter; I'm still having trouble finding enthusiasm for tests written after development for modules and units). So it was great to see a master at that particular craft. It made me excited to use tests as a tool to guide my OO design, behavior discovery, and problem solving.

Some of you may recall that I had previously investigated BDD, Machine.Specifications, and Top-Down Design a year and a half ago doing the Anagram Code Kata. I think I can recognize better design and responsibility break-up and assignment nowadays. I likely will blog on these subjects again this year with new & improved passion and knowledge.

There was one sad takeaway for me personally from the course, which was a sense of disappointment in the .NET development community in general. Giving myself some time since the course to really think it through, I’m realizing I’m probably being too tough on our community, as I know each Dev community out there suffers from it’s own set of problems. But I don’t want to excuse our shortcomings either; rather, I would like to point them out so that we can focus on them and root those weaknesses out. I think our community in specific is unique in that there is a big shot company that dictates the direction our community should take, sometimes taking credit for all the innovation. Things are improving lately no doubt; it appears Microsoft is listening more and more and trying to let outsiders get involved and contribute.

But we as .NET devs need to step up and take matters into our own hands, we need to take better control of the code we write and encourage best practices within the realms of our responsibilities and influence. I believe many .NET shops rely on the tools sent down from above so heavily that we don’t really solve problems anymore. JP’s focus on problem solving and code katas from the course attack this deficiency at its core. I do somewhat blame our university Computer Science programs for not teaching real OO (a great discussion on formalism versus hermeneutics in a book I highly recommend, Object Thinking), but at some point we have to take what we were given and no matter our current standing, focus on sharpening our skills through practice and continuous learning. It boils down to individual responsibility for our current skill set and progression as a software craftsman. JP’s course does a terrific job of waking you up to that realization and inspiring you to be a better professional at your craft.

I have a few takeaways from the course that I’ll publicly spell out now. First, I (and hopefully our whole team) need to better understand design patterns and have them become integral to our code discussions. JP was all about efficiency in keystrokes and related dev tasks, and it seems natural that our language and code discussions could benefit from more efficient communication using design pattern terminology to convey a lot about a specific design concept in only a few words. Second, I hope that I (and our team again) can really put our tests to work for us, helping us recognize the problem to be solved or the behavior that needs to be captured, and then guiding us toward a simple solution. The third takeaway is related to that last part, and that is to have simple solutions that are distributed across simple object implementations (focusing on behaviors) that adhere to SOLID principles. It appears the key is to not focus on implementation specifics until the very last responsible moment; if done right, the implementations to create should be so clear cut and trivial, almost spelled right out in the tests themselves. It’s hard to put into words, but it seems crucial to focus first on the collaborators needed while designing code at the higher levels, finding ways to offload as much responsibility as possible (at least that’s what it felt like) onto dependencies that have yet to be designed and implemented. Designing Top-Down in this way gave such simple and elegant solutions that were totally extensible and adhered to the Open/Closed Principle better than any other code I have ever seen before. Perhaps as I blog more about this, I’ll be able to better put this style of coding into words (especially as I try to better understand it myself; it all seems so magical still!).

Overall, a great course that I highly recommend to all. JP has mentioned doing a similar course in Ruby, which I personally find especially interesting, given the same design/coding techniques would seem to run against a lot less friction compared to the heavy reliance on interfaces and mocking that C# require.

Stay tuned as I try to put some of this stuff into practice!

LINQ Query Syntax and Type Argument Inference

2010-12-03T15:29:00.004-07:00

In LINQ, you can query in-memory data collections using Method Syntax (the IEnumerable extension methods) or Query Syntax (the SQL-like, declarative syntactic sugar on top of the underlying method calls). I was using the query syntax to join a few in-memory collections and then project (like a SQL SELECT) the denormalized data into a new object that only contained a few pieces of information from each of the separate data collections. The code could look similar to this (we'll go with a shopping example in honor of the upcoming holiday season):

var shoppingLists =
    from person in people
    join wishlist in wishlists on person.Id equals wishlist.personId
    join store in stores on store.Id equals wishlist.storeId
    select new
    {
        PersonName = person.Name,
        StoreName = store.Name,
        NumItems = wishlist.Items.Count
    };

However, I ran into the following intellisense red squiggly error in the Visual Studio C# code editor on the second join statement:

The type arguments for method 'IEnumerable<TResult> System.Linq.Enumerable.Join<TOuter,TInner,TKey,TResult>(this IEnumerable<TOuter>, IEnumerable<TInner>, Func<TOuter,TKey>, Func<TInner,TKey>, Func<TOuter,TInner,TResult>)' cannot be inferred from the query.

The error might also end with a statement that says "Try specifiying the type arguments explicitly."

Let’s Take a Closer Look…

The underlying problem is best understood by examining the method signature of the LINQ Join extension method and how the LINQ query syntax for the keyword join maps to that method call in the compiler. The type signature for the extension method is as follows:

public static IEnumerable<TResult> Join<TOuter,TInner,TKey,TResult>(
    this IEnumerable<TOuter> outer,
    IEnumerable<TInner> inner,
    Func<TOuter,TKey> outerKeySelector,
    Func<TInner,TKey> innerKeySelector,
    Func<TOuter,TInner,TResult> resultSelector)

Consistent with all extension methods, it is public static and has a this keyword in front of the first method parameter, allowing you to call the Join method as if it had been declared in the original definition of the IEnumerable<> class or one of its inheriting children classes.

The first two parameters are the two data collections being joined, and in the case of my LINQ query above, they would be the joined collection of people & wishlists and then the stores collection, respectively.

The next two method parameters take delegates or lambda expressions that select the key value from each collection participating in the join, outer collection key selector first and then the inner key selector second. The key values of each item in each collection will compared to each other for equality. It will use the normal definition of the Equals method for your object type (or if you specified a custom one in your class definition), or there is another Join extension method signature that takes in another method parameter that an object that conforms to the IEqualityComparer<TKey> interface. The key selectors from our LINQ query sample would be the two member access expressions in the "on store.Id equals wishlist.storeId" statement.

The last parameter is a delegate or lambda expression that takes in a item from each participating data collection and projects the items into a new form (like the Select LINQ query method and its SQL equivalent). The two items being passed into the Func have passed the equality comparison of each item’s key value, exactly the same as how rows from two tables match up in an inner join statement in SQL.

As for the generic type parameters in this extension method definition, TOuter and TInner are the types from the two collections participating in the join. TKey is the type returned by the key selectors; notice that in order to compare the equality of each key selected, they need to be of the same type. In our example, TKey is likely int or long (or maybe Guid) since we’re dealing with Id properties of all of our objects. TResult is the type returned by the projection Func<TOuter,TInner,TResult> resultSelector; in our case, we returned an anonymous type consisting of three properties (PersonName, StoreName, and NumItems).

Ok, So Where Did I Screw Up Then?

To answer this, let’s take a look one more time at my LINQ query syntax, how it would map to LINQ method syntax, and the Join method signature again all side-by-side:

var shoppingLists =
    from person in people
    join wishlist in wishlists on person.Id equals wishlist.personId
    join store in stores on store.Id equals wishlist.storeId
    select new
    {
        PersonName = person.Name,
        StoreName = store.Name,
        NumItems = wishlist.Items.Count
    };

// Equivalent LINQ method syntax
var shoppingLists2 = people
    .Join(wishlists,
        person => person.Id,
        wishlist => wishlist.personId,
        (person, wishlist) => new { person, wishlist })
    .Join(stores,
        store => store.Id,
        something => something.wishlist.storeId,
        (obj, store) => new
        {
            PersonName = obj.person.Name,
            StoreName = store.Name,
            NumItems = obj.wishlist.Items.Count
        });

public static IEnumerable<TResult> Join<TOuter,TInner,TKey,TResult>(
    this IEnumerable<TOuter> outer,
    IEnumerable<TInner> inner,
    Func<TOuter,TKey> outerKeySelector,
    Func<TInner,TKey> innerKeySelector,
    Func<TOuter,TInner,TResult> resultSelector)

The issue is lines 19 and 20 above; they need to be switched. The compiler can’t properly infer the generic type parameters because we have our key selectors out of order in the argument list when invoking the Join method. If you wanted to be explicit about your generic type parameters assignments, then Join<TOuter, TInner, TKey, TResult> would be Join<Temp1, Store, int, Temp2>, where Temp1 and Temp2 represent the magical type definitions that the compiler creates underneath the covers when you use anonymous types in your C# code. Notice the outerKeySelector needs to come before the innerKeySelector. It appears the "on store.Id equals wishlist.storeId" syntax is not a commutative operation.

One last thing, I just realized that the error(s) given in Visual Studio when you actually compile the code are different than the intellisense red squiggly error message. It appears an actual build of the code is able to be a little more omniscient than the quick compile that intellisense does as you type. The error (and intelligent suggestion) the compiler offers says something like this:

The name 'store' is not in scope on the left side of 'equals'. Consider swapping the expressions on either side of 'equals'.

Additional Info

While reading the MSDN article on the LINQ Join method, I thought the "Remarks" section has some really great stuff about deferred execution, default equality comparison, differences from SelectMany, sort order perseverance, and so on:

This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach in Visual C# or For Each in Visual Basic.

The default equality comparer, Default, is used to hash and compare keys.

A join refers to the operation of correlating the elements of two sources of information based on a common key. Join brings the two information sources and the keys by which they are matched together in one method call. This differs from the use of SelectMany, which requires more than one method call to perform the same operation.

Join preserves the order of the elements of outer, and for each of these elements, the order of the matching elements of inner.

In query expression syntax, a join (Visual C#) or Join (Visual Basic) clause translates to an invocation of Join.

In relational database terms, the Join method implements an inner equijoin. 'Inner' means that only elements that have a match in the other sequence are included in the results. An 'equijoin' is a join in which the keys are compared for equality. A left outer join operation has no dedicated standard query operator, but can be performed by using the GroupJoin method. See Join Operations.

Thanks, I hope this proves helpful to someone, as it was certainly eye-opening to me.

Maybe Monad Extensions

2010-09-30T17:35:00.001-06:00

I learned a lot from my last foray into implementing my own Maybe Monad. Apparently, it got noticed by and was actually useful to Derick Bailey, as it was one of the resources he used in his own personal exploration into Monads. Of course, Derick did the topic a bit more justice, as he has a propensity to do.

Nonetheless, I have done as I said I would and reworked my Murray Monads into a cleaner, more concise implementation using extension methods, much like the implementation I introduced from Dmitri Nesteruk in the last post. So without further ado, I’d like to introduce you to the MaybeMonadExtensions.

Monads 2.0 - “These Go To Eleven”

If you want to get an introduction to what Monads are and how I got to where I am today with this little library, then you really should check out the previous post on this topic. I will however give a few brief code samples to show you what it can do to tidy up all those null checks using the Maybe Monad.

Here is code we tend to write all the time:

string postCode;
if (person != null)
{
  if (HasMedicalRecord(person) && person.Address != null)
  {
    CheckAddress(person.Address);
    if (person.Address.PostCode != null)
      postCode = person.Address.PostCode.ToString();
    else
      postCode = "UNKNOWN";
  }
}

It can look like this with some Maybe Monad love:

string postalCode = person.If(HasMedicalRecord)
  .Access(p => p.Address)
  .Apply(CheckAddress)
  .Access(a => a.PostCode)
  .Convert(pc => pc.ToString(), "UNKNOWN");

I know! Groovy, huh?!

The source can be found here: http://bitbucket.org/murrayondotnet/maybemonadextensions/

Let me know if you put this to good use!

Maybe from Murray Monads

2010-09-21T18:33:00.000-06:00

Off and on for the last six months, I have probably seen and read at least a half dozen introductions and tutorials on monads, a functional programming construct based on the mathematical monads from category theory. Monads are known for their headache-inducing explanations, and I’ve had my fair share. Every tutorial I’ve committed time to mentions the endless possibilities of using such constructs to improve your code, but I’ve had a hard time finding real-world examples of using this stuff out in the wild (much like this Stack Overflow poster, whose link I only found just now). It has recently become a quest of mine to find some C# code we all write that could be improved by a monad.

I am now going to attempt to introduce a Maybe monad that I cooked up, but I warn you that I have no idea what I’m doing or if I’m staying within the rules for what makes a monad by definition a monad. Don’t worry though, you have nothing to lose; I’m the only one taking a risk at looking foolish, and it’s all at the expense of my reputation and not yours. Should be fun!

“It is not the spoon that bends, it is only yourself”

From the Wikipedia article linked to above:

“In functional programming, a monad is a kind of abstract data type constructor used to represent computations (instead of data in the domain model). Monads allow the programmer to chain actions together to build a pipeline, in which each action is decorated with additional processing rules provided by the monad. Programs written in functional style can make use of monads to structure procedures that include sequenced operations, or to define arbitrary control flows (like handling concurrency, continuations, or exceptions).

…

“A programmer will compose monadic functions to define a data-processing pipeline. The monad acts as a framework, as it's a reusable behavior that decides the order in which the specific monadic functions in the pipeline are called, and manages all the undercover work required by the computation.”

The promised tenets of monads are increased code readability and algorithm composability, an example of preferring declarative code over imperative, procedural code. In the case of the Maybe monad, it is a wrapper around types that might have an invalid state like null or nothing. The lure of the Maybe monad is the ability to deal with values that might be in this invalid state without having to explicitly check for null before continuing or exceptions being thrown in your algorithms.

I first found a post by Rinat Abdullin where he introduced his version of a C# Maybe monad. I downloaded the library and attempted to figure out how to use it in code I could possibly write any given day. I tried mimicking the HttpContext.Request.Params collection commonly used in ASP.NET pages and HTTP handlers to retrieve the passed in query string or post values on the Request object. If you ask that collection for a parameter that was not passed in on the request, it returns a null instead of the string value. Common practice is to then use a bunch of if statements and key off the fact that certain query string or post parameters may or may not exist.

The problem I ran into with Rinat’s implemenation is that I wrote the code using the monad how I thought it should help me, and then ran into compiler errors and runtime exceptions because of the expectation that Maybe objects couldn’t be constructed around null values. Here is the twitter conversation I had with Rinat in regard to this misconception that I had:

murrayondotnet says:

@abdullin Can you make your MaybeMonad in Lokad lib accept null at construct or explicit/implicit cast? Maybe<SomeType> monad = varThatsNull

abdullin says:

@murrayondotnet No, this would violate the idea of MaybeMonad (as in Lokad libs). But you can copy code and tweak it to your needs locally.

murrayondotnet says:

@abdullin Figured this was case; what does it violate? I'm still new to monads. Thought one would be able construct monad around maybe null

abdullin says:

@murrayondotnet check out the article on Maybe monads in Lokad libs: http://bit.ly/1GJlNN

murrayondotnet says:

@abdullin Thanks for the link, thats actually how I found your lib. I didnt see where you say why cant construct maybe monad on null. Help?

abdullin says:

@murrayondotnet in essence monad in Lokad guarantees: there will be no nulls around (hence no need to check for them), it replaces null.

murrayondotnet says:

@abdullin I may try to create my own implementation that hides whether null or not. If so probably base mine off of the nice things in yours

And so I did.

Introducing MurrayMonads

I don’t even like the name. I’m also not sure this code will ever be used again…by anyone, let alone me (especially since I like some other guy’s implementation better; read on to watch me embarrass myself). But whatever, Visual Studio asks for a project/solution name and so there it is. I need some cool theme like Rhino, as used by Ayende.

Speaking of Oren (I will use his proper name instead so that he doesn’t get stuck at the airport unable to board his flight), he posted a link on Twitter to an article where Dmitri Nesteruk created his own version of the Maybe monad, just as I was starting my experiment. He used extension methods instead of an explicit monad object wrapper in order to enable the pipelining composability that you can also do with Rinat’s implementation. Really clever, but there was one small thing that bothered me a bit, and that is how the monad chaining begins with Dmitri’s API. (Actually, I’ll prove I’m wrong by the end of the post; this is the cause of upcoming embarrassment mentioned above, hooray!) Because of this, I decided to use his example problem as the basis for my sample that I will show, in order to increase comparability.

So I set off to try and merge the good points of both Rinat’s and Dmitri’s implementations. I have put my source code up for viewing at Bitbucket.org: http://bitbucket.org/murrayondotnet/murraymonads

Are We There Yet?!

Yes, we are. Let’s see some code finally.

Here is the code we don’t like writing all the time:

string postCode;
if (person != null)
{
  if (HasMedicalRecord(person) && person.Address != null)
  {
    CheckAddress(person.Address);
    if (person.Address.PostCode != null)
      postCode = person.Address.PostCode.ToString();
    else
      postCode = "UNKNOWN";
  }
}

This is actually code that you wrote; I took it just yesterday from your current project at work. I know you’re feeling a bit defensive because of this, but I’m here to help so put down the attitude please. We’re gonna make your life easier…I hope…

All in all, it’s code you see every day and there’s not much really wrong with it per se. I just think it could be cleaner I suppose. So using Dmitri’s extension methods, you can get that logic down to this:

string postCode = this.With(x => person)
  .If(x => HasMedicalRecord(x))
  .With(x => x.Address)
  .Do(x => CheckAddress(x))
  .With(x => x.PostCode)
  .Return(x => x.ToString(), "UNKNOWN");

As long as that last ToString method call doesn’t return a null (which one of my use cases does in the TestConsoleApp included in my source code), you will either get the value or “UNKNOWN”. Looking good!

So my version of the Maybe monad comes out pretty close to this last one:

string postCode = person.If(p => HasMedicalRecord(p))
  .Access(p => p.Address)
  .Apply(a => CheckAddress(a))
  .Access(a => a.PostCode)
  .Convert(pc => pc.ToString(), "UNKNOWN");

// or even
string postalCode = person.If(HasMedicalRecord)
  .Access(p => p.Address)
  .Apply(CheckAddress)
  .Access(a => a.PostCode)
  .Convert(pc => pc.ToString(), "UNKNOWN");

The difference between Dmitri’s and mine being how the chain is first started, on the Maybe person object instead of Dmitri’s use of the this keyword.

Here’s what you’ve been really waiting for where I eat my own words. Dmitri’s code can also be written like this:

string postCode = person.If(HasMedicalRecord)
  .With(x => x.Address)
  .Do(CheckAddress)
  .With(x => x.PostCode)
  .Return(x => x.ToString(), "UNKNOWN");

…which appears to take away any advantage of my library. I officially like his solution better, I just don’t like some of the method names. So perhaps I will create another blog post and Bitbucket repo with a new library of extension methods instead. I will leave up this mess-of-a-post public (as well as the source code) so that perhaps others can see the process I went through and learn from my mistakes.

Your Turn to Criticize Now

Let me have it. Let me know how crazy I am or if this is even useful. Let me know if I’m completely misusing the monad principles. Or, by some weird miracle, let me know if I’m on to something. Actually, if Dmitri is on to something. Either way, I want to hear from you. Thanks.

Additional Monad Learning Resources

http://blogs.msdn.com/b/wesdyer/archive/2008/01/11/the-marvels-of-monads.aspx
http://www.infoq.com/presentations/Demystifying-Monads
http://importantshock.wordpress.com/2009/01/18/jquery-is-a-monad/
Also, LINQ is based on monadic principles.

F# Pipeline Operator in C#?

2010-09-06T20:22:00.001-06:00

I am currently reading the Real-World Functional Programming book from Manning Publications. It’s mostly an exercise in learning F#, but also to improve my C# code as well. I finally learned what the F# pipelining operator (|>) is all about, and I was so impressed I thought I would try to figure out how to bring it into C# if possible.

Introducing the F# Pipeline Operator

Here is some F# code that defines a list of integers and a pair of functions. They are then used in a few “higher-order” functions (functions that utilize other functions as parameters or return values, according to my book). List.filter is equivalent to the LINQ IEnumerable.Where() extension method, and List.map is equivalent to IEnumerable.Select().

let numbers = [ 1 .. 10 ]
let isOdd(n) = n % 2 = 1
let square(n) = n * n

List.filter isOdd numbers
List.map square (List.filter isOdd numbers)

The interesting part is the last line of code, where it is argued that you lose a little bit of code readability because the function that is actually executed first (List.filter) is found second in the expression statement. Enter the F# pipelining operator:

let squared =
  numbers
  |> List.filter isOdd
  |> List.map square

Even though we’re no longer a compact, one-liner expression, I think we’ve gained some code readability. It says to take the list numbers and perform the following manipulations in the exact order listed. Cool stuff, if I say so myself.

“There is No Charge for Awesomeness…or Attractiveness”

(Sorry, I’ve been watching the movie Kung Fu Panda a lot with my kids lately. Funny stuff.) So can this awesomeness be brought over into C#? I opened up Visual Studio and tried to see if I could figure out how to define my own operator or reuse one of the existing ones.

I realize C# is a pretty tough language to flip keywords and operators around and invent new syntax with (ok, it’s impossible), but I tried anyway. The type of code readability issue I wanted to remedy is the same as with the F# code snippet above. I thought it would be nice to make C# code like the following statement more readable and digestible (especially the execution order):

Convert.ToInt32(Context.Request[“requestParam”].Trim());

As I started to (unsuccessfully) beat out some pretty rough code, I ran into problems with trying to make extension methods that are operator definition overloads and errors saying “One of the parameters of a binary operator must be the containing type.” It just wasn’t working and then I began to realize that C# already has a pipelining syntax, even if only in one small subset of the .NET API. You can find a decent pipelining syntax with the Fluent Interface used for the LINQ extension methods. Its power is in the fact that the functions return the object type that was passed in and acted upon, so that more functions can be chained on afterward in a composable manner. The ability to do things like collection.Where().Select().OrderBy() is a much more declarative and functional solution to imperative for loops (and it makes it more readable and intention revealing in my opinion). It’s a difference in describing more what you want to be accomplished rather than how it is to be accomplished (especially when the how in programming languages is more readable to a computer than it is a human).

So the good news is that you indeed can have a nice pipelining syntax in C#, especially for APIs that are designed with Fluent Interfaces like LINQ. However, it doesn’t solve code readability issues with code snippets like the C# fragment I showed above. For those regular, non-fluent method calls in the rest of the .NET API, you must use old-fashioned Object-Oriented Programming practices of breaking code up into meaningful, atomic units of statements and naming and encapsulating functions cohesively in order to get more readable code.

Reactive Framework Finally Explained

2010-07-14T17:59:00.000-06:00

I have been intrigued by the Reactive Extensions for .NET (Rx Framework) for quite a while now, especially the fundamental shift from a “push” data model as opposed to the “pull” model most familiar to us. I have thought to myself that this must be a game changer in how data and events are consumed in every-day software. I didn’t quite understand how it all works or how to use it from the few samples I had seen, but I knew if I gave myself a little time to really sit down and look at it, my life would be changed forever.

However, after reading some articles, watching some recorded presentations, and even attending a MIX ‘10 session in-person on the Reactive Extensions, I felt increasingly lost and confused by what this framework could offer me. I wasn’t understanding some fundamental ideology of the framework and it was jading my learning experience when watching/reading sample code. Well, I think I have finally figured out what the mental roadblock was and will attempt to explain the framework here as I best as I understand it, in hopes that it can be of help to others not getting what the fuss is all about. I can sense that I’m not the only one who has been turned off by the unsuccessful attempts by Microsoft to explain this revolutionary framework to the average developer. Apparently those who had a hand in designing the framework are just so incredibly intelligent that they are having a hard time discerning that the common man is not immediately getting it.

IEnumerable<T>, Meet IObservable<T>

Essentially what Microsoft did is create a mathematical dual of the IEnumerable and IEnumerator interfaces using Category Theory, and named them IObservable and IObserver respectively. In its bare simplicity, this is done by merely reversing the direction of your arrows in your mathematical functions and compositions. The following images of a few slides from Erik Meijer’s MIX ‘10 Session that were the best visual explanation I have seen:

Duality Example with De Morgan’s Laws

Creating a Dual of IEnumerable

Creating a Dual of IEnumerator

The reason the mathematical proof part is significant is because we can guarantee that LINQ syntax and operators will work on these new collection and data subscriber interfaces, just like they did with IEnumerable collections in LINQ-to-Objects. This new LINQ provider is commonly called “LINQ over Events.”

Reacting to Data is Asynchronous

This is actually where I think some of the confusion came in for me. Mostly what I didn’t understand is how you can write LINQ queries into pushed data (already sent and also yet to be sent), when essentially that task is indeterminate by nature. What I have figured out is that the LINQ syntax is not entirely for the data being pushed, but also used for molding the subscriptions themselves. What the LINQ syntax provides you is a way to compose, filter, project, aggregate, and throttle data and event subscriptions in a powerful, declarative way.

A common workflow seems to be to create one or more observable collections (and there are many ways to do so) and then customize and compose a subscription based on the subscription(s) using LINQ syntax. All that’s left then is to set the subscription in motion, giving it some sort of callback or lambda expression to be called as data is asynchronously pushed in and processed.

Here is a very simple code example that illustrates this process quite plainly:

using System;
using System.Linq;

namespace TestRX
{
    public class Program
    {
        public static void Main(string[] args)
        {
            var oneNumberPerSecond = Observable.Interval(TimeSpan.FromSeconds(1));

            var numbersTimesTwo = from n in oneNumberPerSecond
                                  select n * 2;

            Console.WriteLine("Numbers * 2:");

            numbersTimesTwo.Subscribe(num =>
            {
                Console.WriteLine(num);
            });

            Console.ReadKey();
        }
    }
}

It is interesting to watch the code run and think about what it’s actually doing. It sits on the Console.ReadKey() line while the observable collection is asynchronously pushing data to the lambda expression containing the Console.WriteLine() statement.

It looks like the Reactive Framework is not just a better way to handle events and pushed data, but it also appears from what I’ve read that it could be used as a nicer API for sending off asynchronous process threads.

Additional Reading

Microsoft DevLabs has created the Reactive Extensions for .NET (and it’s associated languages), but it has also came up with the Reactive Extensions for Silverlight and JavaScript (can be found on same webpage). This proves very interesting as JavaScript (and Silverlight) tends to be quite event driven because of its close interactions to UI elements in HTML (or XAML). In fact, one of the most encouraging example use cases that I’ve heard was that of having some HTML text field that you wanted to auto-search as the user types, but not until the user pauses for a second; they can also hit Enter. What the Reactive Framework could provide you is a way to aggregate more than one event source (alphabetic and Enter key strokes and the timer) into a single event handler using clear, declarative framework statements.

Here are other links I would recommend for additional examples and explanation:

Enjoy! Let me know via comments if you start using it for something real cool and useful.

Parallel.For Loops in .NET 4

2010-06-12T12:19:00.004-06:00

I was having fun working on the Toughest Developer Puzzle Ever 2 when I came across a problem that asked for the coordinates into a grid of numbers of the upper left corner for a 4x4 sub-grid with a magic sum of 34, meaning all rows, columns, and the 2 diagonals all sum up to 34. Instead of trying to figure it out manually, I probably did what many programmers would do and wrote a quick and dirty program to accomplish the task for me.

I didn’t try to use any optimized search algorithm or anything, as I knew that the problem space was very small and specific and that computers are really fast. So I figured I would just write a brute-force iterate through every row and column type of an algorithm (in Big O notation being O(mn) or just O(n2) if the original grid is square), knowing it would return in milliseconds.

I quickly realized that each iteration was not dependent any previous iteration for its calculations, so I decided it might be a good time to try out the new Parallel.For loops in the .NET 4 Framework that would multi-thread the iterations of the loops. Here’s the implementation I threw together:

using System;
using System.Linq;
using System.Threading.Tasks;

namespace Find34Grid
{
    public class Program
    {
        private static readonly int[][] Grid = new []
        {
            new [] {16,3,2,13,15,10,3,6,41,15,14,4,12,8,7,1,12},
            new [] {5,10,22,8,4,5,16,7,9,7,6,12,5,11,10,8,5},
            new [] {9,6,7,12,14,11,2,13,16,3,2,13,15,10,3,6,5},
            new [] {41,15,14,4,12,8,7,2,5,10,11,8,4,5,16,9,15},
            new [] {16,3,2,13,15,10,3,6,15,10,16,2,3,13,16,2,3},
            new [] {5,10,11,8,4,5,16,9,4,5,5,11,10,8,5,11,10},
            new [] {9,6,7,12,14,11,2,13,14,11,9,7,6,12,9,7,6},
            new [] {41,15,14,4,12,8,7,1,12,8,4,14,15,1,14,15,1},
            new [] {9,7,6,12,5,11,10,8,5,11,10,3,6,41,15,14,4},
            new [] {4,14,15,1,9,7,6,12,9,7,5,16,9,9,7,6,12},
            new [] {12,8,13,13,4,14,15,1,4,14,11,2,13,16,3,2,13}
        };

        private const int MagicSum = 34;

        public static void Main(string[] args)
        {
            Parallel.For(0, Grid.Length - 3, i =>
            {
                Parallel.For(0, Grid[i].Length - 3, j =>
                {
                    int[] sums = new []
                    {
                        Grid[i].Skip(j).Take(4).Sum(), // Row 1
                        Grid[i+1].Skip(j).Take(4).Sum(), // Row 2
                        Grid[i+2].Skip(j).Take(4).Sum(), // Row 3
                        Grid[i+3].Skip(j).Take(4).Sum(), // Row 4
                        new [] { Grid[i][j], Grid[i+1][j], Grid[i+2][j], Grid[i+3][j]}.Sum(), // Column 1
                        new [] { Grid[i][j+1], Grid[i+1][j+1], Grid[i+2][j+1], Grid[i+3][j+1]}.Sum(), // Column 2
                        new [] { Grid[i][j+2], Grid[i+1][j+2], Grid[i+2][j+2], Grid[i+3][j+2]}.Sum(), // Column 3
                        new [] { Grid[i][j+3], Grid[i+1][j+3], Grid[i+2][j+3], Grid[i+3][j+3]}.Sum(), // Column 4
                        new [] { Grid[i][j], Grid[i+1][j+1], Grid[i+2][j+2], Grid[i+3][j+3]}.Sum(), // Diagonal 1
                        new [] { Grid[i][j+3], Grid[i+1][j+2], Grid[i+2][j+1], Grid[i+3][j]}.Sum() // Diagonal 2
                    };

                    if (sums.All(x => x == MagicSum))
                    {
                        Console.WriteLine("y (row): {0}\nx (column): {1}", i, j);
                    }
                });
            });

            Console.ReadLine();
        }
    }
}

Nothing special and certainly did the job. I didn’t really notice a performance improvement in anyway, but that was expected for such a small problem. I don’t really expect a whole lot of feedback on such straightforward code, but if there is a cleaner or more efficient way to utilize Parallel.For or Parallel.ForEach, please don’t hesitate to let me know via commenting to this post. Thanks!

Anagram Code Kata Part 5 – Domain Objects Over Language Primitives

2010-02-20T17:55:00.001-07:00

This post is part of a series on coding Kata, BDD, MSpec, and SOLID principles. Feel free to visit the above link which points to the introductory post, also containing an index of all posts in the series.

In this post, we will discuss some reasons why you might want to avoid using language primitives directly in place of domain objects. Specifically, I have been using String variables and objects to represent words up to this point. Esteban suggested I create a Word class for a few good reasons which I’ll layout for you. Out of sheer luck and good timing, Dru Sellers of CodeBetter.com also wrote on this subject shortly afterward and confirmed Esteban’s reasoning.

Use String Class or Create Word Domain Object?

I mentioned in the previous post that Esteban had suggested creating a Word class instead of just passing around strings everywhere through my application. One good reason is that we don’t own the String class and so further modification to our string-based implementation is difficult because the logic is scattered throughout the application, instead of centralized under the responsibility and definition of one domain class. It is much more difficult to refactor the word-specific logic with its behaviors and attributes spread throughout the code. Even if your Word class doesn’t grow any more beyond a seemingly unnecessary wrapper of the String class, the code is more cohesive and ready for change should the need arise.

But more importantly, you designed the code thinking in a true object-oriented mindset. You have to keep in mind that the String class is someone else’s implementation, and is hardly ever sufficient in and of itself as a domain object within your solution. Think about it, the behaviors that a string object performs are so generic and multipurpose that you likely don’t need two-thirds of the class as defined (and there’s likely a few behaviors you really need that aren’t there). Of course nearly every application on earth makes use of the String class; but because of this fact, it has no meaning in and of itself within any given application. You would have to look at how the stings are actually used within the code in order to understand its unique application within the app’s context. Of course that task of research is much easier for everyone (including the original author of the code) if it’s all encapsulated within a dedicated domain object class. True object-orientation means describing in code form the properties, behaviors, and interactions/relationships of real world objects within your problem domain.

Esteban gave me a great example to illustrate these points. He said that you can always represent money as a decimal (and even when you use a domain object, it’s got to have a decimal language primitive underneath the covers). However, what happens when you need to attach metadata to the amount (like currency denomination), or if you need to change decimal precision? You would have to go through all of the code and make sure your use of decimal language primitives is modified uniformly in order to retain consistency. Also, mathematic operations involving money are hardly ever the same as their counterparts involving standard decimals, because currency deals with discrete values to a certain decimal precision. Typically when the behaviors and properties within our system begin to get complex, we are cognizant enough to create domain objects in order to bring it all under one class. We definitely don’t want to over-architect features and interactions before we need them, but I think there is power in this principle of abstracting away language primitives and instead encapsulating their use within domain objects located in just one place in your codebase. I believe it is one thing we can keep in mind to help guide us to better object-oriented thinking, and avoid language-oriented coding.

As mentioned above, Esteban’s thoughts were confirmed nearly verbatim by Dru Sellers of CodeBetter.com in his blog post that he wrote just a few days after I had the conversation with Esteban. A great coincidence no doubt, and worth a read; here’s the link:

Business Primitives (1/2)

Word Class Implementation

So basically I created the following class implementation and then replaced string with a reference to this new class, Word:

public class Word
{
    private string wordStr = string.Empty;

    public Word(string wordStr)
    {
        this.wordStr = wordStr;
    }

    public override string ToString()
    {
        return wordStr;
    }

    public override bool Equals(object obj)
    {
        return wordStr == ((Word)obj).ToString();
    }

    public override int GetHashCode()
    {
        return base.GetHashCode();
    }

    public int GetCanonicalHashCode()
    {
        char[] letters = wordStr.ToCharArray();
        Array.Sort<char>(letters);
        return new string(letters).GetHashCode();
    }
}

I have defined an overridden implementation for Equals(object) (so that the test assertions and other IEnumerable.Contains() queries work) and GetHashCode() (solely to satisfy a compiler warning). I also moved the GetCanoncialHashCode() method from AnagramGrouper, in order to better encapsulate it as a behavior a Word knows how to do innately.

One other change I had to make was to convert strings into Word objects in our NewlineFileParser, which I accomplished by using an IEnumberable.Select() call as shown below:

return File.ReadAllLines(filePath).Select<string, Word>(x => new Word(x));

Let’s Revisit AssertWasCalled One More Time

Trust me, I am groaning with you, even as I wrote that heading text. The last thing we need is for me to rehash the topic again and flip flop my stance yet another time. Yes, that’s right I’ve changed my mind again. First I couldn’t understand what utility asserting methods were called would have under normal test scenarios. Then I changed my mind that perhaps using it would help my test specifications have clearer intent of what I am asserting. After an email conversation with David Tchepak, I think I’m now back to my original stance. Here is what Dave said that had me reconsidering, and I think it’s pretty sound reasoning (emphasis added by me):

…

“But seeing as you asked for it, here goes. :)

“I try and avoid AssertWasCalled like the plague. Generally I don't care that some method was called, I care that my class does the work it needs to. If that involves calling a dependency then great, but that is not my class' reason for existence.

“I prefer your original approach of stubbing out everything in the setup and having that tested indirectly. One reason I prefer this is I find it makes it easier to refactor: the assertions in my test don't change, the class still does the same thing. However I can add or change dependencies by changing some wiring in the setup, and then make sure I haven't stuffed anything up as my assertions still pass. I prefer that my tests specify what I want, not how it does it. To me, AssertWasCalled reeks of over-specification. The one exception is where I hit the external boundaries of my code, so where I want to send an email or something without side effects that I can test. Then the core of the behaviour is the call itself, so I'm happy to assert on that then.”

And with that I’ll promise to never bring this up again…unless of course I get swayed by someone else. :) In all seriousness, I think this is an interesting discussion and so if you have insight, please share via comment below.

Summary

Please leave your thoughts in the comments below in regard to creating domain objects over using language primitives (or heaven forbid, the AssertWasCalled debate). As far as this coding kata exercise, I need to take a high-level look at where this should go next. Perhaps the next post will tie up loose ends and see how our code performs on large text files as input. It may be that we will need to refactor our architecture to achieve better runtimes. If not, maybe we can still discuss where would could have headed if it had been necessary.

Anagram Code Kata Part 4 – Will it Write My Code for Me?

2010-01-26T22:27:00.001-07:00

In this post, we work on implementing the other main dependency, IAnagramsFinder. In doing so, you will discover that I have been a little naïve with this BDD methodology. Apparently I expected a different experience than what I ran into doing the exercise. In gathering feedback, I had a very interesting discussion about Object-Oriented Programming with Esteban that I will summarize and ask for further discussion from all of you. I’ll also show you one way to debug your code when running the MSpec tests via the console test runner.

New Thoughts on Mocks vs. Stubs

During Part 2 of this series, Tobias left a comment about using mocks and AssertWasCalled, rather than stubs and defining fake responses. I didn’t understand how it would help, as it appeared they would accomplish roughly the same thing. With stubs and fake responses, if you don’t get back the return you are looking for, then you can infer that the dependencies weren’t called since you had previously hard-coded their return values when called. With mocks and asserting methods were called, you directly assert that the manager class did in fact call it’s dependencies.

When I looked more closely into using the AssertWasCalled method in Rhino.Mocks, I noticed that my strategy of using stubs and faked return values wasn’t very explicit in signifying that the ultimate responsibility of the manager class was to delegate to its dependencies. I mean the test context does rigorously stub out the dependencies, but the actual assertions weren’t clear what we were testing. Therefore, I decided to pop in a pair of AssertWasCalled statements (as found below), leaving the rest of the test the same as it was back in Part 2.

It should_result_in_list_of_anagram_sets_and_both_dependencies_should_have_been_called = () =>
{
    fileParser.AssertWasCalled(x => x.ExtractWordListFromFile(filePath));
    anagramGrouper.AssertWasCalled(x => x.FindAnagramSets(wordListFromFile));

    result.ShouldEqual(expected);
};

However, when I thought to simplify the test code by removing most of stubbing setup, it appeared I had to leave most of the stub logic in tact in order to test if anagramGrouper.FindAnagramSets() was called with parameter wordListFromFile, which was one of the stubbed fake responses. The one dependency (the file parser) needed to pass it’s return value to the next dependency (the anagrams finder). I could not think of a way to accomplish testing that the manager class facilitated that without the stubbing logic (fake responses) that I created.

It would be great if someone could enlighten me since I’m such a newbie to mocking and stubbing. Otherwise, I feel the intent of the test is now more clear with the AssertWasCalled checks that I added, even if that I means I’m needlessly mixing stubbing and mocking. But one last question: should I try to stick more closely to the rule of thumb guiding me to stick to one all-encompassing assertion per test if possible? Let me know your thoughts.

Implementing the AnagramGrouper Dependency

So let’s continue onward with some more actual design and code. We are tackling the AnagramGrouper dependency, whose sole responsibility is to take in the parsed collection of words and find anagram sets. Here is the test spec (one thing to note is that in some of my earlier tests for the AnagramsFinder manager class and IFileParser, I had used string arrays liberally; I decided to go with IList<string> collections instead):

public class AnagramGrouperSpecs
{
    [Subject(typeof(AnagramGrouperSpecs))]
    public class when_given_word_list_collection
    {
        static AnagramGrouper sut;
        static IEnumerable<IList<string>> result;
        static IEnumerable<string> wordList = new[] { "wordA", "wordB", "Aword", "wordC" };

        Establish context = () =>
        {
            sut = new AnagramGrouper();
        };

        Because of = () =>
        {
            result = sut.FindAnagramSets(wordList);
        };

        It should_result_in_anagram_sets_collection_of_length_1 = () =>
        {
            result.Count().ShouldEqual(1);
        };

        It should_contain_2_specific_words_in_the_anagram_set = () =>
        {
            result.First().ShouldContain("wordA", "Aword");
        };
    }
}

Here too I have basically two assertions (but split it up differently than last time); should I also try harder to avoid this as well?

When contemplating how to further breakdown this responsibility of finding anagram sets into subtasks, I figure we would need one piece of code that would compare two words and answer whether they are anagrams of each other, and another piece of code that would keep track of the various anagram sets (perhaps a collection or dictionary of string arrays). With what BDD has revealed to me thus far, I figure this means we are going to see two more dependencies (and interfaces, and spec tests) added to the codebase.

So I start with the word comparison task, thinking I could perhaps leverage the String.GetHashCode() method for assigning meaningful value to the list of characters in the string. However, string hash codes give importance to letter casing and order of characters, so a modified strategy would have to be utilized, even though GetHashCode() seems quite close to filling our need.

Now I probably should have come up with this ingenious algorithm all by myself, but…it is what it is and this is the best, most concise solution that I can put together to solve the problem. With that said, I ended up doing some internet searching in order to proof-of-concept my GetHashCode() idea. I came across pure genius in the form of Brian Mullen’s blog post entitled LINQ group by and GroupBy, specifically the second-to-last code snippet. Just String.ToCharArray() the word and then Array.Sort() the resultant array; now turn it back into a string and do GetHashCode() now. (As a side note, we can worry about case-insensitivity later if desired by simply doing a String.ToLower() on the word before getting the hash code.) Now anagrams will have the same hash code (Brian uses the term “canonical”, which I think is very fitting); it’s literally perfect in every way!

So now for the other dependency, I figure we could use a Dictionary<int, IList<string>>, where the key is the canonical hash code and the value is a string list collection representing an anagram set of words. These two dependency implementations are so simple and straight forward that I have a hard time seeing a reason to continue breaking out full-fledged object dependencies with interfaces and Dependency Injection. However, I will make a case for moving the canonical hash code logic outside this AnagramsGrouper class, but that will come later.

So let’s write some code to pass our test/spec:

public class AnagramGrouper : IAnagramGrouper
{
    public IEnumerable<IList<string>> FindAnagramSets(IEnumerable<string> wordList)
    {
        Dictionary<int, IList<string>> results = new Dictionary<int, IList<string>>();

        foreach (string word in wordList)
        {
            int canonicalHashCode = GetCanonicalHashCode(word);

            if (results.ContainsKey(canonicalHashCode))
            {
                results[canonicalHashCode].Add(word);
            }
            else
            {
                results.Add(canonicalHashCode, new List<string>(new[] { word }));
            }
        }

        return results.Values;
    }

    public static int GetCanonicalHashCode(string word)
    {
        char[] letters = word.ToCharArray();
        Array.Sort<char>(letters);

        return new string(letters).GetHashCode();
    }
}

Maybe you noticed already, but there is a bug in my logic. At that time, I didn’t see it and needed to debug the code while running. Setting a debug break point in your code doesn’t work like you’re probably used to, and this is because we’re running the MSpec tests via its console test runner. I had to do some searching to figure this one out, and that’s what we’ll cover next.

How to Debug MSpec Specifications and the Code They Test

After a short ride on the Google super highway, I found a blog post containing a November 2009 comment from Aaron Jensen, author of MSpec, that gave a few possible solutions. I opted to go with the Debugger.Break() solution, which I put at the very top of my FindAnagramSets method. It is a tad awkward because it will show a Windows error dialog box that says the program encountered a problem and Windows is searching the web for a solution. If you wait through that, it will give you an option to debug the program via a Visual Studio dialog where you select the currently running instance of Visual Studio as your debugger instance. You just remove the statement when you’re done (and you’ll really want to remember to remove it from your production code). Does the trick, though it is a bit obtrusive to your code base.

If you haven’t already figured it out, the bug is that my FindAnagramSets method is returning all anagram sets found in the dictionary. An anagram set of just one word is not really an anagram set, which is why our test specification which is based on that assumption failed. The fix is to filter out anagram word sets with less than two words before returning, like this:

// Return anagram sets with more than one word in it
return results.Values.Where(x => x.Count > 1);

That will make the code pass the tests now. Remember to remove the debugger statement we added.

BDD, OOP…WTH?!

Just kidding. But I do want to discuss a reaction I had to BDD after implementing this anagram finding solution. After I completed the code and finished the tests, I was feeling pretty great. But then I started thinking about how I didn’t really go down the path that I felt BDD was originally trying to lead me down with AnagramGrouper, which was to create two more dependencies, just as I had done for the first managing class AnagramsFinder. I mean I felt pretty frustrated because I was digging BDD to this point. I’m not quite sure what my original expectations were, but I might have thought BDD was magically going to right my algorithms for me as well.

I decided to discuss the whole thing with Esteban to get some perspective on the issue. He told me he didn’t know a thing about BDD and how it’s supposed to work, but he figured BDD would lead me to decoupled, cohesive code design, but he couldn’t see how it would aide with algorithmic solutions to the problems at hand. I’m not sure why I was so naïve, but it makes complete sense. I mean these methodologies are designed to keep you disciplined to some proven, guiding principles, but you still need to use your brain and creativity to actually solve the problem. I guess it would help to get some feedback and suggestions from you all in regard to knowing when it is time to switch from BDD architecture design mode to algorithm design mode within the broken down dependencies that handle the very specific sub-responsibilities. Please let me know if you have found some guiding concept or question you’ve come up with that you put to the test in order to lead you into the right coding mindset for that particular context (design or algorithm; or do I even have the right categories?).

Perhaps it’s just that this design process feels so new to me. Maybe I have solely been in the coding, algorithm mindset this whole time throughout my career. Perhaps it will become more natural with practice and experience. I guess it just feels like I’m learning to drive a manual, stick-shift car or something, and I am a little rough with the clutch when transitioning between gears.

To take this discussion further, Esteban made a comment in our conversation about his personal thoughts on these methodologies (like BDD) and Object-Oriented Programming. He has come to believe that we invent these methodologies like flavors of the week “because as programmers we don’t really get OOP.” Perhaps OOP just comes more naturally to his thought process that most people (or he admits it’s possible he could just be completely ignorant), but he feels that there’s really just a few simple rules and guidelines to keep in mind at all times in order to keep your objects behaving like real life objects and not acting like data constructs. I encouraged him to give his theory some additional thought and make it a little more formal by writing it down. He did this and the following is a link to the resultant blog post:

3 Simple Rules to Good Object Oriented Code

Please read through that post and comment on it. His idea needs to be criticized and/or validated. He admits he may have no clue what he’s talking about, but I said that’s why it needs to be put to the test by posting it in the open. We both think it’s an interesting theory, but we’re unsure if it even holds any water. Perhaps Esteban’s theory is too simplistic to be practical for many programmers. Provide feedback and lots of it.

Summary

So I think I need to wrap this post up about now. I think I have raised enough discussion points to write a book, but please do take the time to comment on what I’m doing wrong, what I’m doing right, or how to make the BDD mentality more flowing and natural. I promised to talk about having a Word class as opposed to using primitive strings everywhere, and that will definitely be in the next post. Also, I think our AnagramGrouper class may need some more testing and flushing out of how it should handle spaces, punctuation, casing, and so on.

Thanks for sticking with me thus far.

Anagram Code Kata Part 3 – Use Common Sense

2009-12-18T18:10:00.001-07:00

In this post, we will discuss how far to take the idea of creating mockable contracts via interfaces for testing purposes and to keep responsibilities modular. As I solicited feedback from commenters, I was getting hung up on applying this concept to every object and dependency. We’ll talk about how I’m now thinking that overly strict adherence to the practice can drive you nuts and decrease productivity. It is meant to help guide your design process along rather than hinder it. We’ll finish this post off with one of our two dependencies implemented in an extremely simple manner.

IFileParser Dependency

When we left off last time, we had defined two separate responsibilities that AnagramsFinder was going to push down into its dependencies. As I started to write a specification test for the newly created NewlineFileParser, I realized it would be difficult to stub based on providing a file path to a text file as the only parameter. I began having thoughts that maybe I could break up another pair of dependencies for file parsing, namely obtaining a file stream (or file contents string) from a path and then parsing the file contents into a list of multiple lines. But then again, my first of the two dependencies is still dependent on the physical file system and I cannot get away from this unless I somehow mock up an in-memory file system. This began to feel silly to me as I was hoping I would just use the built-in .NET APIs for file systems and string parsing.

After emailing Dave and Esteban asking for guidance, the revelation I received via these two fine gentlemen is that file parsing is not an essential purpose of this application nor exercise (at least not yet; we won’t be focusing on performance unless it appears to be grossly deficient as we go along). What we could do is write integration tests instead, where we have a set of simple test files sitting in a folder that we run our NewlineFileParser against to ensure it working as we expect. But I don’t think I’m even going to do this at this point. I really do want to get to the core of the application. Plus, I feel certain that I can trust the implementation of File.ReadAllLines() from the .NET Framework.

Unless it needs to be changed further down the road, the following code implementation will likely be sufficient for the remainder of this coding exercise:

public class NewlineFileParser : IFileParser
{
    public IEnumerable<string> ExtractWordListFromFile(string filePath)
    {
        return File.ReadAllLines(filePath);
    }
}

I feel great about this because I haven’t gone insane trying to mock and test everything in sight, nor have I broken up every single responsibility in such small chunks that it becomes tedious. Instead, I have broken up the two core responsibilities of AnagramsFinder which means I can change and test the two dependencies independent of each other and from their manager class. Even though I haven’t written a ton of code yet, I feel the tests specs have driven my design into a cohesive, decoupled design that should prove flexible if requirements change or if I encounter friction later on and need to rethink my design. A flexible architecture for the application is being flushed out based on a “top-to-bottom” design view of the requirements.

Create Stubs or Assert Methods were Called

Tobias Walter left a comment on my last post with some suggestions that I would like to get clarification on and discuss further with my growing audience of three now. He compared some of my implementation to Dave’s implementation on his Favour test driving logic over data post, but the comparison was lost on me. It seemed Tobias was saying that I had gone too far breaking up the responsibilities into such small dependencies. I am unsure what his argument was, but as far as I could tell both Dave and I broke up a managing class into two separate responsibilities so that the more crucial of the two could be tested independently. Also in doing so, we both created stubs that were given specific dummy responses to method calls using Rhino.Mocks. One more comment I would like to say is that I broke the file parsing responsibility away into its own dependency more because of the need to isolate the anagrams grouping than because file parsing is of itself a difficult task. If Tobias or anyone else could try to clarify his argument, that would really help me understand the merits of his suggestions.

Another suggestion that Tobias brought up that I would like to petition feedback on is to use the mocking feature of testing whether a method was called instead of creating a stub programmed with a dummy response. I have to admit, I’m not sure when to use one method over another. I guess I’ve felt in the past that testing whether a method was called seems silly, since it seems odd to have your test know the implementation and inner method calls of the subject under test. But I am now realizing that my stubs appear to be doing the same thing, but with dummy input and output programmed in. If you have an opinion or experience on this subject, please leave some feedback in the comments of this post.

What’s Next?

In my next post I hope to summarize this mocking debate brought up above regarding two differing methods of verifying my manager class is correctly managing its dependencies. I also hope to begin driving out the design and implementation of the AnagramGrouper dependency via behavior specification tests. Thanks and any feedback is greatly appreciated.

Anagram Code Kata Part 2 – Mocking and SRP

2009-11-17T00:59:00.000-07:00

In this post, we will rewrite our first specification/test from the previous post, where we didn’t feel very confident about the direction we were heading. We got some great feedback and will now better focus on test driving logic and not just data; we will also be more careful in extracting the requirements from problem statement and in giving our classes a Single Responsibility (the first of the SOLID principles of Object-Oriented Design). As we are test driving our design from a “top-down” perspective, we will encounter the need for dependencies not yet implemented while creating the higher level classes. Instead of halting our rhythm and progress, we will utilize a mocking framework to stub out non-existent implementations according to interfaces. This also encourages us to use Dependency Inversion (last of the SOLID principles; same link as above) and we can prove the validity of our modules by testing them in complete isolation from their dependencies.

Don’t Focus on Data (Yet) and SRP (Single Responsibility Principle)

I had a feeling I was focusing too much on testing data, and I also knew it seemed to odd to be writing my very first test to be dependent on solving a specific instance of the problem (especially a test case with thousands of words in it), instead of working on solving the problem generally. After some terrific feedback from David Tchepak in the comments of my last post, I now understand that wasn’t my only mistake. I was also trying to give my AnagramsFinder class too many responsibilities (especially with just one ParseTextFile method), namely extracting the words out of the text file and then grouping the anagrams together. Not only that, I also was returning a count of anagram sets when the problem statement specifically asked for output of the anagram sets themselves. Even though this is a simple problem scenario, it can be foolish to make assumptions or simplifications like I did that are not part of the requirements.

Let us first focus on the responsibilities required of our AnagramsFinder class. It needs to parse a list of words out of a text file given the file path and it also needs to group words together into sets that are anagrams of each other. However, we just named two responsibilities; implementing both in the same class would make it less cohesive and therefore less maintainable. This is because our class would have more than one responsibility that could change in the future, and because these responsibilities are in the same class, they could be considered coupled and one could be affected by modifications to the other. All of this train of thought falls under the Single Responsibility Principle from the SOLID principles (see links at the beginning of the post).

To solve this predicament, we will make AnagramsFinder a managing class of dependencies that individually solve these separate concerns. Each of these dependencies will adhere to the Single Responsibility principle, as does this higher abstraction manager class. Its responsibility can be summarized as managing the inputs, outputs, and execution order of its dependencies. We shall name the dependencies directly after each of their responsibilities, namely FileParser and AnagramGrouper. However, I don’t want to go implement these dependencies and throw off the flow of fleshing out the design and logic of my manager class. We are trying to design from a more “top-down” approach, instead of focusing on the lower level data concerns of the solution too early in the design process. To accomplish completing the design of our manager class without actually implementing the dependencies, we will code to interfaces (namely IFileParser and IAnagramGrouper).

DIP (Dependency Inversion Principle)

The advantages of using interfaces are that we abstract out the actual implementation of the dependencies, allowing us to be more modular and less coupled between dependencies. One class is not strongly tied to a specific implementation of its dependency, but rather the general contracts made available by the interface’s defined method signatures. What this really means is that we can swap out actual implementations of the dependency without the consuming class being modified in the least bit. This makes for truly maintainable code, especially when requirements change down the road after version one of the application is up and running. This is what the Dependency Inversion Principle (again, from SOLID) is all about.

We can take this a step further by passing the necessary dependencies into our consuming class at construction. This allows for us to “inject” into the object any implementation of the interface we see fit; this technique is aptly named Dependency Injection (DI). Our manager class doesn’t have to fret about any concrete details regarding its dependencies’ actual implementations, nor about any of the construction ceremony associated with “newing” up and initializing said dependencies. We won’t be using any DI containers (as our dependency needs are very light in this exercise), but the use of the Dependency Inversion Principle does set us up nicely to use a mocking framework so that we can finish fleshing out our tests without implementing any of the dependencies yet.

Mocking Dependencies

We would like to finish the design of our top level AnagramsFinder manager class, but not fully commit to how its dependencies will be implemented yet. We have a general idea of how we want our consuming class to interact with its dependencies via interfaces. Let’s go ahead and look at our interfaces we named earlier:

public interface IFileParser
{
    IEnumerable<string> ExtractWordListFromFile(string filePath);
}

public interface IAnagramGrouper
{
    IEnumerable<string[]> FindAnagramSets(IEnumerable wordList);
}

Now, we could create concrete implementations of the interfaces to be used in our tests that return simple dummy results. An example could look like this:

public class TestFileParser : IFileParser
{
    public IEnumerable<string> ExtractWordListFromFile(string filePath)
    {
        return new[] { "wordA", "wordB", "Aword", "wordC" };;
    }
}

This is helpful because it keeps the one class we are interested in from being dependent on any real logic in its dependencies. We want to be able to test our modules in isolation from any dependencies, so that no secondary or outside influences skew the test results. We are basically providing dummy dependencies to our tests that have no logic that could cause side effects.

However, the software development community has provided tools called mocking frameworks that can create these dependency stubs for you. This can save you from creating concrete interface implementations that have no use except for in testing. The mocking framework I will try out is Rhino.Mocks from Ayende. To get set up, you merely need to download the latest build zip archive, extract out the Rhino.Mocks.dll library to our Libraries folder created in the last blog post, and add a reference to the DLL in our Visual Studio solution. I will next show you how to use Rhino.Mocks as we rewrite our first specification test.

Rewrite of Our First Specification/Test

Armed with SRP, DIP, mocking, a more strict adherence to the actual problem statement, and a renewed focus on “favoring test driving logic over just testing data,” we now rewrite our first specification as follows:

[Subject(typeof(AnagramsFinder), "Finding Anagrams")]
public class when_given_text_file_with_word_on_each_line
{
    static AnagramsFinder sut;
    static IEnumerable<string[]> result;
    static string filePath = "dummy_file_path.txt";
    static IEnumerable<string> wordListFromFile = new[] { "wordA", "wordB", "Aword", "wordC" };
    static IEnumerable<string[]> expected = new[] { new[] { "anagram", "gramana" } };

    Establish context = () =>
    {
        var fileParser = MockRepository.GenerateStub<IFileParser>();
        fileParser.Stub(x => x.ExtractWordListFromFile(filePath)).Return(wordListFromFile);

        var anagramGrouper = MockRepository.GenerateStub<IAnagramGrouper>();
        anagramGrouper.Stub(x => x.FindAnagramSets(wordListFromFile)).Return(expected);

        sut = new AnagramsFinder(fileParser, anagramGrouper);
    };

    Because of = () =>
    {
        result = sut.ExtractAnagramSets(filePath);
    };

    It should_result_in_list_of_anagram_sets = () =>
    {
        result.ShouldEqual(expected);
    };
}

I should mention a few notes about the code. First, we need to add a using Rhino.Mocks statement at the top of our specification class file. Second, our AnagramsFinder instance variable named sut stands for the “Subject Under Test,” a mannerism I picked up from David Tchepak, whom I’ve mentioned several times throughout this post series.

Our use of Rhino.Mocks is found when we call the MockRepository.GenerateStub method against an interface. We then proceed to tell the stubbed object how to behave by specifying dummy return values when given methods of the object are called with given parameters. The last line of our “context” setup is to then inject these two newly generated, stubbed dependencies into our manager class for testing. It’s also interesting to note that none of my parameters and expected outputs really make much sense. This is done on purpose to show that the data really doesn’t matter to this class, as we are not testing the logic of any data processing by the dependencies (remember, test in isolation). It is true that this test really isn’t testing any real logic at this point, and it may even never evolve into testing any real logic either. However, I think the important point is that it aided us in fleshing out the design, which we previously didn’t know how we were going to implement. Not all tests will created equal in regard to validating our logic, but they all will play a part in driving the design of our code. Hopefully these statements are true, and I would love to hear feedback on this topic in the comments.

I also want to touch on the Specification pattern of Behavior Driven Design and testing. Specification tests are commonly set up into a workflow of context establishing, behavior performing, and results asserting. The MSpec framework is designed to encourage this test organization.

Red, Green, Refactor and MSpec Test Runner Output

To make our first test run, we need to create the basic outline of our AnagramsFinder class as outlined in our test. This includes a constructor that takes our two dependencies and an ExtractAnagramSets method:

public class AnagramsFinder
{
    private IFileParser fileParser;
    private IAnagramGrouper anagramGrouper;

    public AnagramsFinder(IFileParser fileParser, IAnagramGrouper anagramGrouper)
    {
        this.fileParser = fileParser;
        this.anagramGrouper = anagramGrouper;
    }

    public IEnumerable<string[]> ExtractAnagramSets(string filePath)
    {
        throw new NotImplementedException();
    }
}

I am using the ConsoleRunner according to the method described in Rob Conery’s introductory MSpec and BDD post, including output to an HTML report. After running our specifications, we get the following output:

Specs in AnagramCodeKata:

AnagramsFinder Finding Anagrams, when given text file with word on each line
¯ should result in list of anagram sets (FAIL)
System.NotImplementedException: The method or operation is not implemented.
   ... Stack Trace here ...

Contexts: 1, Specifications: 1
  0 passed, 1 failed

…and the HTML report:

A highly encouraged tenet of Test Driven Development is the practice of “Red, Green, Refactor.” It is meant to denote the evolution of the results and state of your tests. You are encouraged to write the test and then do the minimum work necessary to get the code base to compile and run. You are first running your test in a control state where you know it should fail. Most test runners will show the color Red in regard to failed tests, and thus the name of the first step. Your next stage is to implement the code to make the test pass and turn the output color to Green, commonly indicating passing tests. The Refactor stage is a time to pause and see if any code can be reorganized or simplified. Then “rinse and repeat as necessary”, an applicable instruction from shampoo bottles.

To make our test pass, we implement real logic to coordinate the calls into our dependencies, like so:

public IEnumerable<string[]> ExtractAnagramSets(string filePath)
{
    var wordList = fileParser.ExtractWordListFromFile(filePath);
    return anagramGrouper.FindAnagramSets(wordList);
}

…and the successful outputs of our test runner:

Specs in AnagramCodeKata:

AnagramsFinder Finding Anagrams, when given text file with word on each line
¯ should result in list of anagram sets

Contexts: 1, Specifications: 1

Final Thoughts and Questions

So what do you think of the new test? Are we better able to see the Single Responsibility of the AnagramsFinder manager class? Are we more inline with solving the actual problem statement? Your feedback would be much appreciated.

In the next post, we will begin work on test driving the design and implementation of the dependencies…unless of course I get feedback that I should be focusing on a different area first or that this last rewrite still needs more help.

Anagram Code Kata Part 1 – Getting Started

2009-11-12T15:03:00.001-07:00

This post will provide resources that serve as an introduction to the MSpec BDD testing framework. We will also set up our Visual Studio solution, explain the Anagram Code Kata problem statement, and write our first specification and test.

Machine.Specifications (MSpec)

The author of this BDD testing framework is Aaron Jensen of CodeBetter.com. Probably the best introduction to the framework (and lightly touching on general principles of BDD) is Rob Conery’s post entitled Make BDD Your BFF. MSpec is hosted on GitHub using the Git source control software (and here is the link for the repository again). Download the code and build it to get the necessary libraries that you need into a Bin folder. My MSpec libraries are a few months old, as I haven’t upgraded my source since my initial download back then. I have not set up TestDriven.NET and am not sure it will be necessary for this simple coding exercise. I am using NUnit for the actual testing framework underneath MSpec; the following article may be of interest to you if you would like to use a newer version of NUnit than what MSpec targets:

Using Latest NUnit Version with MSpec

MSpec helps you organize your code into Specifications, a core BDD concept. It is inspired by early BDD testing frameworks like Ruby’s RSpec. The syntax and arrangement can be a little odd for a static language like C#, but I think I like the direction it is heading. Again, the best way to get familiar with the syntax, set up the MSpec Test Runner, and configure HTML reporting output is to read Rob’s introductory post on BDD and MSpec.

Setting Up the Visual Studio Solution

Let’s begin by creating a C# console application named AnagramCodeKata targeting .NET Framework 3.5. The solution name (and solution folder) will be the same. For our dependencies on MSpec (and NUnit), I have created a physical file system folder named Libraries and made it a sibling folder to the project’s physical folder (same level as the solution file). That folder contains the following DLL libraries from the Bin folder of the compiled MSpec source code:

CommandLine.dll
Machine.Specifications.ConsoleRunner.exe
Machine.Specifications.dll
Machine.Specifications.NUnit.dll
Machine.Specifications.Reporting.dll
nunit.framework.dll

Then within our Solution Explorer, we only need to make references to 3 of these libraries (namely Machine.Specifications.dll, Machine.Specifications.NUnit.dll, and nunit.framework.dll). I have also created two folders within our project in Solution Explorer, naming them Reports and Specifications. The Reports folder will contain the HTML-generated report of running our tests every time. The Specifications folder will hold our specifications and tests. This is what our current Solution Explorer looks like now:

Also notice the MSpecRunner button at the top, which I’ve created to execute a custom External Tool that points to Machine.Specifications.ConsoleRunner.exe and passes in the appropriate parameters. This helpful setup was put forth by Rob Conery in the link to his introductory blog post that I provided above.

Anagram Problem Statement

So let’s go ahead and look at the Anagram problem we are trying to solve. Here’s a quick summary of what we are trying to accomplish:

“The challenge is fairly simple: given a file containing one word per line, print out all the combinations of words that are anagrams; each line in the output contains all the words from the input that are anagrams of each other.”

Unfortunately, Dave Thomas’ wordlist.txt link is broken on his blog post, and when responding to commenters asking for a fixed link to the text file, he pointed us all to some website that aggregates links to several word lists from around the Web. Many of us readers wanted to know the exact word list file he used so that we could compare our results to his. In his post, he states that there should be 2,530 sets of anagrams and 5,680 total words participating in those anagram sets. The tough part about this is that we’ve got to have the same word list text file. I searched the web and found a few, but I only found one that might be the same list. It is from some guy’s personal GitHub repository:

http://github.com/krist0ff/code_kata/tree/ba975c3a64f11a81db2f3716b40de046f1ca7ef4/kata6

If you’re visiting this blog post, months or years after it was originally written, I obviously can’t guarantee this file will still be out there. You can try emailing me or commenting here and I can hopefully send it to you via email if I still have it. But more importantly, as we progress with this series, we’ll find out if it’s even the right word list or not.

Lastly, I’d like to look at the objectives for the coding kata that Dave Thomas outlines:

“Apart from having some fun with words, this kata should make you think somewhat about algorithms. The simplest algorithms to find all the anagram combinations may take inordinate amounts of time to do the job. Working though alternatives should help bring the time down by orders of magnitude. To give you a possible point of comparison, I hacked a solution together in 25 lines of Ruby. It runs on the word list from my web site in 1.5s on a 1GHz PPC. It’s also an interesting exercise in testing: can you write unit tests to verify that your code is working correctly before setting it to work on the full dictionary.”

I don’t think we’ll focus on runtime as much, but we’ll certainly revisit our algorithm as necessary if we have unacceptable wait times for the code to finish. To complete this section, let’s go ahead and add the wordlist.txt to our solution at the root of of our Project node.

How About Some Actual Code?!

So let’s now start off by creating our first specification. I have created a new C# file named AnagramsFinderSpecs.cs under our Specifications folder in our solution. I attempted to write our first test, but already feel like I’m not inline with “favoring test driving logic over just testing data.”

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Machine.Specifications;

namespace AnagramCodeKata.Specifications
{
    [Subject(typeof(AnagramsFinder), "Finding Anagrams")]
    public class when_given_text_file_with_word_on_each_line
    {
        static int result;
        static AnagramsFinder sut;

        Establish context = () =>
        {
            sut = new AnagramsFinder();
        };

        Because of = () =>
        {
            result = sut.ParseTextFile("wordslist.txt");
        };

        It should_result_in_count_of_2530_anagram_sets = () =>
        {
            result.ShouldEqual(2530);
        };
    }
}

If you can’t make out the funky structure and syntax, again refer back to Rob Conery’s introductory post to all of this stuff that I link to at the top. He does a great job of explaining what’s going on here.

Also, please remember that I have no clue what I’m doing and I’m just trying to copy people who know what they are doing. I am hoping that I’ll be corrected by those who have a better idea of how to approach all of this. This is a good stopping point because of my desire for feedback and the feeling of uncertainty I’m already getting.

Seriously, please guide me in the right direction or validate where I’m headed, because even I’m not sure where that is yet. I will say this, I’m pretty sure I shouldn’t be reading in wordslist.txt yet, and that this might be where I need to start using a mocking framework.

Stay tuned, there’s more to come. I just can’t tell what it will be about yet. It’ll still be fun though, I promise.

Custom Setup Bootstrappers

2009-11-11T15:03:00.001-07:00

At work, we have a line of desktop products for one of clients installed on the user’s Windows machine via MSI installers. We use Visual Studio Setup Projects to create the MSI installers. Visual Studio also provides the ability to create setup bootstrappers to make sure the user has all the prerequisites needed for the MSI install technology and any dependencies required by your desktop product. These bootstrappers are unmanaged code, because you can’t always depend on the .NET framework already being there (at least not until everyone is off of XP and lower). I have developed a solution that has worked for the last year or more, but has felt clumsy and slapped together. Recently I found the open source project called dotNetInstaller which has better provided me with the control and flexibility I need for custom logic during bootstrap and install time. I’ll first introduce you to my “hack” solution.

It has been difficult for me to find ways to tie into the bootstrapper logic workflow to add in custom logic during installs, and Visual Studio is very rigid in its flexibility and basic in its configuration options in this regard. Until recently, I had settled on writing managed, C# code for this custom logic meant to run before or after the MSI installer, and then creating another unmanaged setup.exe bootstrapper to ensure the .NET framework was installed just so that executable could run. So the user would run this new unmanaged setup.exe bootstrapper (making sure .NET framework was installed), which would then call my managed custom logic executable, which would then call the MSI installer’s unmanaged, Visual Studio-generated setup.exe bootstrapper (ensuring more prerequisites were installed), which would then finally install the MSI, and which would then lastly return to my managed custom logic executable for any post-install, clean up logic. I describe this solution in full (and how to generate the same bootstrappers that Visual Studio creates using MSBuild outside the IDE) in a StackOverflow.com answer that I submitted:

http://stackoverflow.com/questions/1106768/how-can-i-reverse-engineer-an-installer-that-was-written-with-ghost-installer/1121846#1121846

Well, recently I found the dotNetInstaller project on CodePlex.com and have found it to be the control and flexibility I need for custom logic when bootstrapping installers. Not only does it give you great control over prerequisite install order, it also gives you the ability to run code after MSI install and to also filter on CPU architecture types, OS versions, and other registry or file system conditions. I have currently setup branching scenarios for the prerequisites based on if the user has a 32-bit (x86) or 64-bit (x64) OS. This utility will even let you brand the bootstrapper with a banner and icon, which is a change of pace from the plain and unlabeled bootstrapper that Visual Studio and MSBuild produce.

If you have custom bootstrapping needs, I have to highly recommend you give dotNetInstaller a look. Hopefully it will prove as flexible and powerful for your needs as it has for mine.

Anagram Code Kata – BDD & MSpec

2009-11-10T21:32:00.004-07:00

Many forms of martial arts utilize the kata as a means for detailed, focused practice and training. The power of this practice comes from repetition and focus on simple fundamentals, “while attempting to maintain perfect form.” The principle goal is to “try out different combinations of techniques in a safe, practice environment to ultimately find out how to defeat your opponent”. In the last few years, this concept has taken hold within the software development world in the form of Code Kata, a phrase coined by Pragmatic Programmer Dave Thomas (and here is his blog dedicated to the Code Kata).

My inspiration for all of this is David Tchepak’s Calculator Kata series of posts. Hopefully he will stop by and provide some helpful guidance and advice. I have selected the Anagram Kata (code kata number 6) for me to practice. My goals for this series of posts are:

To practice coding and share my learning experience with you, while also soliciting feedback and suggestions.
To more fully understand the concepts driving Behavior Driven Development (BDD) via a testing framework designed for such a practice, namely MSpec.
To keep to the SOLID principles of Object-Oriented Design, especially in favor of test driving logic over just testing data.

Below I will keep an updated list of blog posts in this series. Hope you enjoy it!

UPDATE (07/18/10): I was asked where the source code to this post series could be found. I decided to host it at BitBucket.org using Mercurial for the source control: http://bitbucket.org/murrayondotnet/anagramkata/

Trying Out SyntaxHighlighter

2009-10-23T19:54:00.006-06:00

This is just a test post to see if I'm smart enough to add the javascript SyntaxHighlighter to my blog. ;)

public int ScoreDice(int[] selectedDice)
{
    int score = 0;

    foreach (IScoringCombo scoreCombo in ScoringCombos)
    {
        IEnumerable<int> satisfyingIndices = scoreCombo.SatisfiedBy(selectedDice);

        if (satisfyingIndices.Count() > 0)
        {
            score += scoreCombo.ScoreDice(selectedDice);

            // Remove the dice at satisfyingIndices from selectedDice collection so not double-scored
            selectedDice = selectedDice
                .Select((value, index) => index)
                .Except(satisfyingIndices)
                .ToArray();
        }
    }

    return score;
}

Web Deployment Tool

2009-09-29T17:45:00.001-06:00

It appears that the IIS team at Microsoft has officially completed a 1.0 version of the deployment utility I mentioned at the end of my previous post about Visual Studio Web Deployment Projects almost two years ago. It includes a command-line tool named MSDeploy that helps with deploying and syncing web projects across all of the web servers in your server farm. The installation also includes all kinds of packaging and deployment features that seem really handy. You can even do migrations from IIS 6 to IIS 7 or package your apps to be distributed via the Microsoft Web Platform Installer.

I thought that maybe the tool creates MSI installers, but it appears the package may actually just be a ZIP file that you can import and export through the IIS Manager or various other command-line tools or PowerShell Cmdlets.

Here is the page where you can read more about the toolset and download the installer: http://www.iis.net/extensions/WebDeploymentTool

Variable Declaration Restrictions

2009-09-23T16:09:00.000-06:00

I came across a surprising restriction on variable declarations that I thought I would share.

My own investigation into C# was prompted by an answer given to a Stack Overflow question regarding an odd Objective-C compiler error. The answer seemed to allude to a language specification somewhere that doesn’t allow variable declarations in one-liner embedded statements of control structures (loops, switch statements, if-else statements, etc.). Not only that, Objective-C wasn’t unique in this restriction; it appears Java and even C itself have had this restriction for quite a while as well.

Naturally this got my curiosity going and I decided to see if C# had also carried over such a restriction. The answer is yes; it appears that Java, C#, and Objective-C have all kept this language restriction from their common ancestor C. Let’s get right to what it looks like:

The red-underlined errors read “embedded statement cannot be a declaration or labeled statement” and the green-underlined warnings read “the variable ‘i’ is assigned but its value is never used.” You’ve probably seen the warning quite a few times before, but the compiler error really caught me by surprise. I mean it makes sense that declaring a variable in the one-line body of a control structure is pointless, as it would immediately go out of scope (and usefulness) after declared. But I thought it odd for the compiler to actually deny such a program statement, as if it put the compiler in an invalid state or maybe it was somehow dangerous.

One thing I do want to point out that appears to be different from the other languages (or at the very least Objective-C) is that the variable declaration is allowed in the switch statement, as illustrated in the image above.

To figure out why this isn’t allowed, I decided to actually look into the C# 3.0 Language Specification to see if there were any comments or explanations regarding this issue. I ended up finding the C# language grammar in Appendix B. I was able to parse through the grammar and find this restriction in the grammar itself. Take a look:

B.2.5 Statements

statement:

labeled-statement
declaration-statement
embedded-statement

embedded-statement:

block
empty-statement
expression-statement
selection-statement
iteration-statement
jump-statement
try-statement
checked-statement
unchecked-statement
lock-statement
using-statement
yield-statement

block:

{ statement-list_optional }

statement-list:

statement
statement-list statement

...

selection-statement:

if-statement
switch-statement

if-statement:

if ( boolean-expression ) embedded-statement
if ( boolean-expression ) embedded-statement else embedded-statement

switch-statement:

switch ( expression ) switch-block

switch-block:

{ switch-sections_optional }

switch-sections:

switch-section
switch-sections switch-section

switch-section:

switch-labels statement-list

switch-labels:

switch-label
switch-labels switch-label

switch-label:

case constant-expression :
default :

...

You’ll notice that there is the concept of a statement and an embedded-statement; also note that statement is a superset of embedded-statement, adding two other types of statements: labeled-statement and declaration-statement. If a particular grammar rule requires an embedded-statement, then those two latter types of statements are not allowed in that context. If we wanted to allow those two statement types, our grammar rule should specify the generic statement category instead.

Also noteworthy is that an embedded-statement can resolve to a block, which includes braces around a statement-list. A statement-list is one or more statement entities, which would now allow us to use the two statement types excluded from embedded-statement.

Something interesting that is not included in the grammar snippet above, the following statement constructs (and language keywords) require the block statement construct, bypassing the option of a grammar resolution to embedded-statement directly and allowing you to leave off surrounding braces: try, catch, finally, checked, and unchecked.

Almost all of the other statement constructs allow you the option of a one-liner embedded-statement (or which can be resolved into a block), including if, if-else, while, do-while, for, and foreach. The one surprising exception is the switch construct, which allows for a statement-list after each switch-label. As a side note, this allows you to put useless braces (via block) around your statement-list (or anywhere inside it) after any of your switch-label statements if you wanted. But back to the point, these facts explain the existence of (and in one case, lack of) compiler errors regarding embedded statements in the code screenshot near the top.

Now you will know exactly what the compiler means when it gives you the error that an “embedded statement cannot be a declaration or labeled statement.”

BeginInvoke Methods and OneWay Attribute

2009-05-21T16:29:00.001-06:00

In a previous post I talked about Windows Forms UI Threading using delegates. Recently I needed to replicate what I had learned, and I happened to do so without reviewing my notes in the former blog post. Apparently I didn’t remember everything I had learned and became confused over thread invocation techniques, introducing a bug into my thread messaging. The UI thread was blocking while the background worker thread was processing.

The cause was related to figuring out which BeginInvoke method to call when kicking off the worker thread, the BeginInvoke of the Form object or the BeginInvoke of the delegate. The answer is the BeginInvoke method of the delegate. All of the examples linked to in the previous post had it right, but I got it mixed up with making Invoke or BeginInvoke calls from the background thread using the Form object’s version of these methods. If you don’t care whether your messaging from the background thread is synchronous or asynchronous, then you can use either methods and you can use the ones from the Form object.

The problem is using these Form object methods in order to kick off your background worker thread. The description of the BeginInvoke method on the Form object states that it “executes the specified delegate asynchronously on the thread that the control’s underlying handle was created on.” I translate this to mean that calling your Form object’s BeginInvoke will process on your UI thread. Apparently the delegate BeginInvoke method has no such wording in its description.

Rereading this MSDN article from the former post and reading the Form object’s BeginInvoke method description which I quoted above set me straight and got rid of my blocking UI problem (again).

During this research session, I came across another informative resource when trying to figure out what was meant by potential resource leaks when using BeginInvoke with out later calling EndInvoke. From reading the article, I gather you can substitute the EndInvoke call with the OneWay attribute on the method being invoked, as long as it has a void return signature and no ref nor out parameters. Please let me know if my interpretation is wrong. I added the attribute to my code in the applicable spots and have seen no side effects of doing so. Here is the article on read mentioning this attribute:

Defensive Event Publishing in .NET – Part I

PowerShell 1.0 for Windows XP

2009-03-24T14:08:00.001-06:00

I just noticed in today's Tuesday round of Microsoft Updates that there is an optional software download for PowerShell 1.0 on Windows XP. PowerShell has been one of those things that I really want to learn, but just haven't devoted the time. I desperately would like to replace my use of the Command Prompt with Windows PowerShell.

PowerShell really is more than just your regular Command Line Interpreter that you have been used to, especially compared to the Windows Command Prompt. You get the power of the command line shells that you may have tried out in Linux environments, but it really does go beyond those as well. PowerShell is (from Wikipedia) "Microsoft's new extensible command line shell and task-based scripting technology." Here is a quote from an MSDN document that better explains the advantages PowerShell brings:

"Unlike most shells, which accept and return text, Windows PowerShell is built on top of the .NET common language runtime (CLR) and the .NET Framework, and accepts and returns .NET objects. This fundamental change in the environment brings entirely new tools and methods to the management and configuration of Windows.

"Windows PowerShell introduces the concept of a cmdlet (pronounced "command-let"), a simple, single-function command-line tool built into the shell. You can use each cmdlet separately, but their power is realized when you use these simple tools in combination to perform complex tasks. Windows PowerShell includes more than one hundred basic core cmdlets, and you can write your own cmdlets and share them with other users.

"Like many shells, Windows PowerShell gives you access to the file system on the computer. In addition, Windows PowerShell providers enable you to access other data stores, such as the registry and the digital signature certificate stores, as easily as you access the file system."

If you want to know more about PowerShell, I would suggest the following readings:

Also, James Kovacs from CodeBetter.com has been doing a series of posts on the basics of PowerShell that is a great light introduction to how a developer would find the tool most useful. Here are two of the starting posts from that series:

The "Yield" Keyword in C# (AKA Execution Control Hot Potato)

2009-03-02T17:51:00.000-07:00

I actually read about this topic a while back when I saw a C# code sample containing an unfamiliar keyword -- yield. Well just today I saw another code sample using this keyword again, and had since forgotten everything I had learned. As I Googled the topic again, I happened across the same article I read back then, a terrific blog post by Joshua Flanagan. Here is a link to Joshua's simple introduction to the yield C# keyword:

The power of yield (return)

I would also suggest the following MSDN article as well:

yield (C# Reference)

I'll give you the gist of this .NET 2.0 feature. If you have a method that returns an iterative collection and the consumers of your function will only need foreach, read-only access to that collection, then it is good practice to have your method return the universal IEnumerable collection (as opposed to you arbitrarily selecting an implementation-specific collection to be forced upon the consumers of your code). In C#, when you are returning an IEnumerable collection from a method, you are allowed to use the yield keyword in order to transform your method into what Microsoft calls an "iterator block". Here is a quote from the MSDN article I mentioned above to explain this phrase and the yield keyword:

"The yield keyword signals to the compiler that the method in which it appears is an iterator block. The compiler generates a class to implement the behavior that is expressed in the iterator block. In the iterator block, the yield keyword is used together with the return keyword to provide a value to the enumerator object. This is the value that is returned, for example, in each loop of a foreach statement. The yield keyword is also used with break to signal the end of iteration."

You are essentially building an enumerator (or iterator), and with each iteration of your foreach loop in the calling, consuming code, you are executing your method containing the iterator block until it reaches a yield return statement. At this point, only one item at a time in the collection is returned back to the consuming foreach loop for immediate logic execution by the body of the loop. On the next iteration of the consuming foreach loop, the method containing the iterator block will be called again and another single item in the collection will be returned for similar logic execution.

This game of execution control Hot Potato has its benefits, in that you can stop the flip-flop of execution control when the music has stopped. In this analogy, I am comparing the "music stopping" to your code satisfying some logic that makes the rest of your foreach loop iterations unnecessary (perhaps you have found the one item in the collection you were looking for by meeting the sufficient requirements of an if statement). The benefit is that when you break out of the consuming foreach loop before completing the iterations, you prevent wasteful code execution and memory use. This feature prevents one piece of code from fully populating a potentially large collection of items, just to have another consuming piece of code iterate partially through the list and then prematurely exit. Consider the savings especially when adding each item to the collection may be computationally intensive or if each item in the collection is itself a very large data structure. In the Hot Potato analogy, this would be as if the kids in the circle who didn't touch the "hot potato" never existed in the game, saving floor space and unnecessary tosses to people you don't really want to target for humiliating removal from the game.

For good sample code snippets of using the yield keyword, see the links provided above.

I hope to identify situations in the near future where I could benefit from using this C# keyword. Hopefully then I won't forget a year down the road from now and then have to re-search the internet for Joshua's blog post and the MSDN article to remind myself what I already learned.

FOLLOWUP (03/26/09): So I'm not sure how I missed it, but it took reading a very recent article by Scott Mitchell of 4GuysFromRolla.com to understand the real reason the yield keyword was introduced. The introduction of this keyword in .NET 2.0 was most helpful in the case of fulfilling the requirements of implementing the IEnumerable interface, which typically required you to create a sub-class that implemented IEnumerator. In many cases, it allowed you to take about 50 lines of code and turn it into 2 lines of code using the yield keyword. The introduction of LINQ brought heavy reliance on enumerating using the yield keyword in order to pipeline query operators together. Scott Mitchell takes you through this whole journey, tying all of the loose ends together to create a complete history of this keyword and how it's most commonly utilized. Please visit the following article to get the whole picture:

An Extensive Examination of LINQ: The Ins and Outs of Query Operators

SQL Server Express 2005 Remote Connections

2009-02-19T10:36:00.000-07:00

I developed an ASP.NET site connecting to a local SQL Server Express 2005 instance on my development box. To further my informal testing, I wanted to host the site under IIS 7 on a Windows Server 2008 box, but I wanted to just use the same database remotely. I got the following error message when the site tried to hit the database:

"An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open connection to SQL Server)"

Apparently SQL Server Express 2005 is not automatically configured for remote access during installation (which really isn't surprising as it is the Express version). Remote connections can be enabled though with a few relatively painless steps. See the following link for the procedure:

Configuring SQL Server Express 2005 for Remote Access when SQL Server does not allow remote connections

Hope this is helpful.

IIS 7 and ASP.NET Providers (Membership, Roles, and Profile)

2009-02-17T17:38:00.000-07:00

I talked at length about the ASP.NET Provider Model pattern and the ease of implementing several built-in services you can have for free with ASP.NET 2.0 in my previous series of posts. In this post, I will be discussing the advantages of using the ASP.NET Provider system under IIS 7 found in Windows Vista and Server 2008. I will also talk about an issue with using this Provider system to try and add authentication to an ASP.NET website that contains non-ASP.NET content, like static HTML pages. I think I'll start with the discussion about non-ASP.NET content first.

IIS 7.0 Integrated Pipeline

Since I don't really understand much about ISAPI filters and such, I'm just going to quote stuff from random pages on the official Microsoft IIS site:

"IIS 6.0 and previous versions allowed the development of .NET application components via the ASP.NET platform. ASP.NET integrated with IIS via an ISAPI extension, and exposed its own application and request processing model. This effectively exposed two separate server pipelines, one for native ISAPI filters and extension components, and another for managed application components. ASP.NET components would execute entirely inside the ASP.NET ISAPI extension bubble and only for requests mapped to ASP.NET in the IIS script map configuration.

"IIS 7.0 integrates the ASP.NET runtime with the core web server, providing a unified request processing pipeline that is exposed to both native and managed components known as modules."

and...

"This allows developers to fully extend the IIS 7.0 server with the richness of ASP.NET 2.0 and the .NET Framework, instead of using the lower level IIS C++ APIs. Existing ASP.NET applications also immediately benefit from tighter integration using existing ASP.NET features like Forms Authentication, Roles, and Output Caching for all content."

Note the last three words there: "for all content." When I started this endeavor to implement the ASP.NET Provider system, I didn't realize that it would really only apply to ASP.NET content and pages. I didn't realize on IIS 6 I would have to fiddle with ISAPI filters and content filters to make my implementation work. And when I tried to doing a mapping (like *.html to the ASP.NET ISAPI DLL), I either didn't do it right or it just isn't very reliable and stable; here is a link to the thread I followed in trying to make the appropriate changes in IIS 6. I will note however that UltiDev's Cassini Web Server can host such a site with absolutely no extra modification, where even IIS 7 required a little configuration.

By default in IIS 7, an application pool runs using the Integrated mode for the Managed Pipeline configuration. But there is also the option of Classic mode, which is helpful if you are needing strict compatibility with how you ran a website under IIS 6. In many cases, however, an existing ASP.NET website runs under Integrated mode without any problems.

Configuring an ASP.NET Website to Run on IIS 7

First, I think I will point you to a great video and document on Microsoft's official IIS site that explains everything you really need to know. Then I will emphasize a few of the important points and also a gotcha or two. Here are the links you should reference:

The video above is also a terrific introduction to IIS 7. Below is a screenshot of IIS's UI for configuring a website:

Notice the ASP.NET section and how it's able to use your Providers to bring back Users, Roles, and Profile data back from the database. This UI reads from and writes to your Web.config and is really slick in my opinion.

The only real "breaking change" for running your website under IIS 7 is the fact that your Web.config might need a little migration to be compatible with Integrated mode. Apparently IIS gives you helpful error messages if it detects you haven't run through the migration yet. The two most common things that ASP.NET websites have that require this migration are if you have declared httpModules or httpHandlers in your Web.config. The fix is to either copy or move these sections under system.web to their corresponding system.webServer sections. If you duplicate the sections, then you should be able run under both Integrated and Classic modes without any problems. You can even still run on IIS 6 I believe, as IIS 6 will ignore this new section called system.webServer which is IIS 7 specific. IIS 7 even provides a utility that can perform this migration for you; here is the what you would type in a Command Prompt:

%windir%\system32\inetsrv\APPCMD.EXE migrate config <Application Path>

...where <Application Path> would be something like "Default Web Site/" or "Default Web Site/someApp". If you manually migrate your Web.config and are still getting migration error messages, here is a snippet you can add right under the system.webServer node to disable the error message:

Also, I want to make note of something I couldn't get to work that they showed in the tutorial video. At about the 13:50 mark in the video, they go into the Modules section of the UI. These are the modules of the IIS pipeline for both managed .NET code and unmanaged code. In order to make their video file hidden behind ASP.NET Forms Authentication, they change a property on two authentication modules to allow these modules to execute on non-managed content as well. However, this did not work for me for some reason when I was trying to secure HTML pages behind Forms Authentication. What did work for me (and may have been a drastic solution) was to put the following attribute on the modules node under system.webServer:

I believe the issue in my case was their was another few managed modules that I needed to clear that checkbox for in the UI. But I didn't take the time to find out which ones I was missing and opted for the drastic solution. I will also say there is a way to configure these managed modules through your Web.config so you don't have to have your IT Admin configure properties manually in the UI. In fact, it seems that much of IIS 7 configuration for your website can be done this way.

A Few Error Messages (and Their Solutions)

If you get error messages while using the IIS Manager user interface like this...

"This feature cannot be used because the default provider type could not be determined to check whether it is a trusted provider.

"You can use this feature only when the default provider is a trusted provider. If you are a server administrator, you can make a provider a trusted provider by adding the provider type to the trusted providers list in the Administration.config file. The provider has to be strongly typed and added to the GAC (Global Assembly Cache)."

...then you should visit the following article for detailed information about resolving the error message:

Adding Management Trusted Providers <add>

Getting rid of this error message is really only for convenience in letting the IIS Manager UI be able to query your database using your providers and give you a basic view of your Membership, Roles, and Profile fields. Also, if you are trying to find your Administration.config file, it is likely located under the C:\Windows\System32\inetsrv\config\ folder.

The other error message I received was not in the IIS Manager UI, but rather when you attempt to view your website in a browser (after you think you got all the configuration bugs out!). A BadImageFormatException is thrown and the error details say something like this:

Could not load file or assembly 'AssemlyNameHere' or one of its dependencies. An attempt was made to load a program with an incorrect format.

The likely cause of this error is that the assembly referenced in the error message was compiled only for 32-bit machines and you are running a 64-bit operating system. With Windows Server 2008 x64, the application pools in IIS have an Advanced Setting called Enable 32-Bit Applications, that is disabled by default. Enable this setting and you're probably good to go. See the following link for more detailed information and some nice screenshots:

Could not load file or assembly 'name' or one of its dependencies

Additional Reading

I just barely noticed that 4GuysFromRolla.com recently posted an article about the exact same stuff I was attempting to explain. To read additional details about IIS 7's Integrated Pipeline and ASP.NET Forms Authentication with static content, check out the following link:

Apply ASP.NET Authentication and Authorization Rules to Static Content with IIS 7.0's Integrated Pipeline Feature

Conclusion

I hope you've enjoyed this series on the ASP.NET Provider System. My desire is that you've found these posts helpful in getting you started with implementing such a versatile and customizable model for adding common membership and authorization functionality to your new and existing websites. There are quite a few advantages in hosting your site on IIS 7 in Windows Server 2008, as I've outlined in this specific post. Hopefully these discussions about errors and gotchas will also help you get going in no time!