CodeBetter.Com - Stuff you need to Code Better!

MSpec latest build download now available

aaronjensen — Thu, 19 Nov 2009 18:30:00 GMT

Many thanks again to Alexander Groß for adding packaging to MSpec. You can now download builds from herewithout having to build it yourself! Just grab the Artifact from the latest build.

Call for help with MSpec

aaronjensen — Thu, 19 Nov 2009 06:51:25 GMT

Thankfully, the community has created a number of very helpful blog posts about MSpec. Unfortunately, I’ve been quite bad at aggregating them into a single authoritative source and documenting the few but somewhat obscure APIs and tools that make up MSpec. It is on my list of things to do but time has been rather… tight. I’d love to get it up sooner than later, as MSpec is getting more and more eyes on it and I want to make things as smooth as possible for new users.

That’s where this plea comes in, I’m looking for someone to help put together some documentation in one place, probably the github README.markdown. I obviously don’t have anything to offer other than fame and fortune (minus the fortune and probably the fame). Rest assured, though, that myself and the rest of the community (like kreiggers) would be forever grateful.

I’d be happy to edit and keep up to date the documentation once created so this wouldn’t be an ongoing effort, but I could use some help getting started. Ping me on twitter if you’re interested and we can brainstorm a bit. Thanks in advance and sorry for the spam.

Introduction to the Reactive Framework Part V

Matthew.Podwysocki — Wed, 18 Nov 2009 22:48:20 GMT

In the previous post of the Introduction to the Reactive Framework series, we covered how to create new IObservable<T> instances, either from scratch or from existing sequences. What this allowed us to do was turn an operation which was previously interactive, such as iterating over a collection, to a reactive, where I could have multiple listeners reacting to the same collection concurrently. This has nice implications for allowing us to scale our application in interesting ways. This time, let’s take another angle on making asynchronous programming easier.

Let’s get caught up to where we are today:

From Asynchronous To Observables

In the first post in the series, we talked about some of the problems we face with asynchronous programming today. For example, for any given operation, how do we handle exceptions, or handle cancellation, or even manage the lifetime of resources? Let’s take a look at the story of asynchronous programming today. For example, let’s take the fairly straight forward approach of sending a tweet in Twitter, but with the added pressure of non-blocking calls for all IO operations. Now we took an example which should have been straight forward, and now it looks something like this anti-pattern:

var request = WebRequest.Create("http://twitter.com/statuses/update.xml");
request.Credentials = new NetworkCredential("foo", "bar");
request.Method = "POST";

request.BeginGetRequestStream(rsAr =>
{
    var requestStream = request.EndGetRequestStream(rsAr);
    var tweet = HttpUtility.HtmlEncode("status=Hello from an async method #bacon");
    var bytes = Encoding.UTF8.GetBytes(tweet);
    requestStream.BeginWrite(bytes, 0, bytes.Length, beginWriteAr =>
    {
        requestStream.EndWrite(beginWriteAr);
        request.BeginGetResponse(responseAr =>
        {
            var response = request.EndGetResponse(responseAr);
            var responseStream = response.GetResponseStream();

            var readBuffer = new byte[response.ContentLength];
            responseStream.BeginRead(readBuffer, 0, readBuffer.Length, readAr =>
            {
                var read = responseStream.EndRead(readAr);
                var readText = Encoding.UTF8.GetString(readBuffer);
                ResultsText.Text = readText;
            }, null);
        }, null);
    }, null);
}, null);

Not only is this code an absolute mess, but it has a number of bad practices, including not handling cleanup of our resources, as well as not handling exceptions, cancellation checking, or even checking if our buffer was completely read. Adding those constraints to the above code would make it a bit more unreadable. The situation we run into is that programming in a non-blocking fashion has caused all sorts of heartache where we become so wrapped up in the non-blocking code that we lose sight easily of the domain logic we should have been interested in, in the first place.

I’ve covered in the past that with F# async workflows, solving this is a rather easy task and that kind of rich functionality has been unavailable to C# developers directly. But, with the Reactive Extensions for .NET, we have the ability to also take advantage of composable asynchronous actions as well. So, where do we get started?

Let’s take a simple example of creating a function and then invoking it asynchronously.

Func<int, int, int> add = (x, y) => x + y;

add.BeginInvoke(3, 4, ar =>
{
    var result = add.EndInvoke(ar);
    Console.WriteLine(result);
}, null);

Many asynchronous methods in .NET are written in this way that their signature has a BeginXXX and EndXXX where the XXX is the method name that is being executed asynchronously. The Begin takes the arguments to execute the method, an AsyncCallback which is an action that takes an IAsyncResult and returns nothing, and finally object state. The End method takes the IAsyncResult which is passed in from the AsyncCallback in order to retrieve the value of the asynchronous call.

void BeginXXX(args, AsyncCallback, state);

Value EndXXX(IAsyncResult);

Luckily, the Reactive Extensions give us a couple of ways to make this happen. First, let’s take an existing method such as the System.IO.Stream.WriteAsync. In this example, we’ll create an extension method which allows us to turn this asynchronous method into a IObservable<T>.

public static IObservable<Unit> WriteAsync(
    this Stream stream, 
    byte[] buffer, 
    int offset, 
    int count)
{
    return Observable.FromAsyncPattern<byte[], int, int>(
        stream.BeginWrite,
        stream.EndWrite)(buffer, offset, count);
}

Because the Write method returns void, we have issues in C# as we cannot have a generic type as being void. To get around this, much like F#, the Reactive Extensions introduced the Unit type to express that we have no value. Quite frankly this is the way I think C# should have been to allow for this and not need these unnecessary hacks. Anyways, the Observable class has a method called FromAsyncPattern which takes in both the BeginXXX and EndXXX and returns a Func delegate that takes the arguments required to execute it. In this case, the Observable.FromAsyncPattern returns a Func which takes the buffer, offset and count and returns an IObservable<Unit>.

There is also another way in that we can take our existing add function from above and using an extension method called ToAsync, can also work the IObservable magic.

Func<int, int, int> add = (x, y) => x + y;
Func<int, int, IObservable<int>> obvervableAdd = add.ToAsync();

IObservable<int> result = from added in obvervableAdd(3, 4)
                          select added;
IDisposable sub = result
    .SubscribeOnDispatcher()
    .Subscribe(added => Results.Text = added.ToString());

In this instance, we took our existing add method and then turned it into an asynchronous method by using the ToAsync extension method. That returns a Func delegate which takes our two arguments and returns an IObservable<int> which contains our asynchronous result. We can then take our new observableAdd and execute it inside of a LINQ statement, then subscribe to it on the Dispatcher thread in order to rid ourselves of synchronization context issues in WPF.

Going back to our original example here, let’s see how we might follow some of the ideas presented here into creating a way to send a tweet completely in a LINQ statement.

var tweetObservable =
    from request in 
        CreateWebRequest(
            "http://twitter.com/statuses/update.xml", 
            userName, 
            password, 
            "POST")
    let tweet = HttpUtility.HtmlEncode("status= Hi! #bacon")
    let buffer = Encoding.UTF8.GetBytes(tweet)
    from _ in request.SetContentLength(buffer.Length)
    from requestStream in request.GetRequestStreamAsync()
    from __ in requestStream.WriteAsync(buffer, 0, buffer.Length)
    from response in request.GetResponseAsync()
    let responseStream = response.GetResponseStream()
    let reader = new StreamReader(responseStream)
    select reader.ReadToEnd();
    
var tweetsObservable = tweetObservable
    .Throttle(TimeSpan.FromMinutes(3))
    .Repeat(25);

We can create extension methods to the WebRequest for GetRequestStream and GetResponse to enable observable behavior via the Begin/End methods that the WebRequest exposes. As well, we can extend the System.IO.Stream as well to both write and read streams asynchronously. We have other more trivial methods such as setting the content length and creating a web request as an observable as well, and in order to do that was covered in the previous post using the Observable.Create method. After we create this IObservable value, we can then throttle it to every three minutes and to repeat 25 times. One concern we might have is around the resource usage and ensuring the cleanup, and we’ll cover that next time.

Conclusion

As I’ve stated before, the LiveLabs Reactive Framework gives us the ability to harness reactive programming and treat events as the first class citizens they should have been using LINQ expressions and other standard LINQ combinators. Still, there is much to cover in this series with the standard LINQ combinators, the monadic heritage of this solution as well as managing resources, and turning Tasks Observables.

Astoria, SSIS Adapters and BDD – Projecting Projections

hdierking — Wed, 18 Nov 2009 22:19:00 GMT

The big idea behind ADO.NET Data Services is that it enables a data source (generally a relational data source) to be exposed via a RESTful service. In the context of fetching data – which is the principle concern of our SSIS data source component, data is returned as either an ATOM feed or as a JSON list. Despite the particular data format, the general principles are the same – you’re taking a relational model and projecting it in a more serial structure. Now, if all you’re querying is a single table, it doesn’t matter all that much – you’re simply taking a tabular data set and serializing each row as an entry in the ATOM feed. However, ADO.NET Data Services provides an expansion feature that traverses a relationship and inserts the child data inline in the resulting feed. Keep in mind that the ADO.NET Data Services API is based on HTTP – and the expansion feature is exposed simply as a querystring parameter.

Looking At the Data

Consider the following query: http://localhost:7558/northwind.svc/Products

This will return the following ATOM feed (fragment):

<feed xml:base="http://localhost:7558/Northwind.svc/" 
   xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
   xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
   xmlns="http://www.w3.org/2005/Atom">
   <title type="text">Products</title> 
   <id>http://localhost:7558/northwind.svc/Products</id> 
   <updated>2009-11-17T22:45:22Z</updated> 
   <link rel="self" title="Products" href="Products" /> 
   <entry>
      <id>http://localhost:7558/Northwind.svc/Products(1)</id> 
      <title type="text" /> 
      <updated>2009-11-17T22:45:22Z</updated> 
      <author>
         <name /> 
      </author>
      <link rel="edit" title="Products" href="Products(1)" /> 
      <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Categories" 
         type="application/atom+xml;type=entry" 
         title="Categories" 
         href="Products(1)/Categories" /> 
      <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Order_Details" 
         type="application/atom+xml;type=feed" 
         title="Order_Details" 
         href="Products(1)/Order_Details" /> 
      <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Suppliers" 
         type="application/atom+xml;type=entry" 
         title="Suppliers" 
         href="Products(1)/Suppliers" /> 
      <category term="NorthwindModel.Products" 
         scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> 
      <content type="application/xml">
         <m:properties>
            <d:Discontinued m:type="Edm.Boolean">false</d:Discontinued> 
            <d:ProductID m:type="Edm.Int32">1</d:ProductID> 
            <d:ProductName>Chai</d:ProductName> 
            <d:QuantityPerUnit>10 boxes x 20 bags</d:QuantityPerUnit> 
            <d:ReorderLevel m:type="Edm.Int16">10</d:ReorderLevel> 
            <d:UnitPrice m:type="Edm.Decimal">18.0000</d:UnitPrice> 
            <d:UnitsInStock m:type="Edm.Int16">39</d:UnitsInStock> 
            <d:UnitsOnOrder m:type="Edm.Int16">0</d:UnitsOnOrder> 
         </m:properties>
      </content>
   </entry>
   ...
</feed>

As you can see, for a single resource query, the resulting feed is pretty simple with a feed consisting of multiple entries. Each entry (in addition to the basic metadata) contains properties for the actual row values and links for relationships. To leverage the expansion functionality – in this case, we’ll expand Suppliers, we would modify the query to look like the following: http://localhost:7558/northwind.svc/Products?$expand=Suppliers

From this query, we would get the following, expanded result:

<entry>
   <id>http://localhost:7558/Northwind.svc/Products(1)</id> 
   <title type="text" /> 
   <updated>2009-11-17T23:04:46Z</updated> 
   <author>
      <name /> 
   </author>
   <link rel="edit" title="Products" href="Products(1)" /> 
   <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Categories" 
      type="application/atom+xml;type=entry" title="Categories" href="Products(1)/Categories" /> 
   <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Order_Details" 
      type="application/atom+xml;type=feed" title="Order_Details" href="Products(1)/Order_Details" /> 
   <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Suppliers" 
      type="application/atom+xml;type=entry" title="Suppliers" href="Products(1)/Suppliers">
      <m:inline>
         <entry m:etag="W/"'49%20Gilbert%20St.','London','EC1%204SD'"">
            <id>http://localhost:7558/Northwind.svc/Suppliers(1)</id> 
            <title type="text" /> 
            <updated>2009-11-17T23:04:46Z</updated> 
            <author>
               <name /> 
            </author>
            <link rel="edit" title="Suppliers" href="Suppliers(1)" /> 
            <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Products" 
               type="application/atom+xml;type=feed" title="Products" href="Suppliers(1)/Products" /> 
            <category term="NorthwindModel.Suppliers" 
               scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> 
            <content type="application/xml">
               <m:properties>
                  <d:Address>49 Gilbert St.</d:Address> 
                  <d:City>London</d:City> 
                  <d:CompanyName>Exotic Liquids</d:CompanyName> 
                  <d:ContactName>Charlotte Cooper</d:ContactName> 
                  <d:ContactTitle>Purchasing Manager</d:ContactTitle> 
                  <d:Country>UK</d:Country> 
                  <d:Fax m:null="true" /> 
                  <d:HomePage m:null="true" /> 
                  <d:Phone>(171) 555-2222</d:Phone> 
                  <d:PostalCode>EC1 4SD</d:PostalCode> 
                  <d:Region m:null="true" /> 
                  <d:SupplierID m:type="Edm.Int32">1</d:SupplierID> 
               </m:properties>
            </content>
         </entry>
      </m:inline>
   </link>
   <category term="NorthwindModel.Products" 
      scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> 
   <content type="application/xml">
      <m:properties>
         <d:Discontinued m:type="Edm.Boolean">false</d:Discontinued> 
         <d:ProductID m:type="Edm.Int32">1</d:ProductID> 
         <d:ProductName>Chai</d:ProductName> 
         <d:QuantityPerUnit>10 boxes x 20 bags</d:QuantityPerUnit> 
         <d:ReorderLevel m:type="Edm.Int16">10</d:ReorderLevel> 
         <d:UnitPrice m:type="Edm.Decimal">18.0000</d:UnitPrice> 
         <d:UnitsInStock m:type="Edm.Int16">39</d:UnitsInStock> 
         <d:UnitsOnOrder m:type="Edm.Int16">0</d:UnitsOnOrder> 
      </m:properties>
   </content>
</entry>

While this example demonstrates expansion of a single link resource, ADO.NET Data Services allows you to expand a single link, multiple links, or multiple nested links. The funny thing about this is that the SSIS data flow engine takes much more of a relational view of the world with its stream architecture. This means that we’re starting with a relational model, projecting into a hierarchical model, and then projecting back to a relational model. The question, then, is how we go about designing (and testing) this projection.

To start, here’s a few high level design principles:

Every resource usage (primary and all expanded) should yield a unique output stream in an SSIS data flow which is visible in the data flow designer
Composition of output streams (e.g. joining) should be done via data flow designer transform components

A Unique Stream Per Resource

Every resource that is returned as part of an ADO.NET Data Services query, whether it is the primary resource (e.g. Products) or an expanded resource, should have a unique output in the data flow designer. Moreover, if a resource is referenced in more than 1 role (e.g. http://localhost:7558/northwind.svc/Products?$expand=Suppliers/Products), an output stream should be created for each role. The goal of having multiple output streams is that each one can then by manipulated independently using the data flow designer.

Enabling Composition

One of the primary reasons behind creating unique streams per resource *role* is to enable granular composition/branching inside the data flow designer (as shown above). The service metadata provided by a data service provides a type definition for each resource managed by the service. This makes it initially straightforward for determining data shape from a data services query. However, in order to effectively project the parent/child relationship specified with the expansion feature, we need to make sure we include the identifiers of the parent record in context of the child record. I started with the following specifications:

public class when_employees_resource_query_expands_territories : with_northwind_CSDL
{
    Establish context = () => {
        queryBuilder.ResourcePath = "Employees";
        queryBuilder.OperationsString = "$expand=Territories";
    };

    Because of = () => query = queryBuilder.GetQuery();

    It should_declare_2_resources = () => query.DataShape.Count().ShouldEqual(2);

    It should_have_1_reference_property_for_territories =
        () => query.DataShape.ElementAt(1).Properties
            .Where(p => p.GetType() == typeof (RefProperty)).Count()
            .ShouldEqual(1);

    It should_have_2_properties_for_territories =
        () => query.DataShape.ElementAt(1).Properties
            .Where(p => p.GetType() == typeof (Property)).Count()
            .ShouldEqual(2);

    It should_have_correct_query_string = 
        () => query.ToString().ShouldEqual(
            string.Format("{0}/Employees?$expand=Territories", Constants.SERVICE_URI));

    It should_have_employees_as_first_resource = 
        () => query.DataShape.ElementAt(0).Name.ShouldEqual("Employees");

    It should_have_employees_as_the_first_resource =
        () => query.DataShape.ElementAt(0).Name.ShouldEqual("Employees");

    It should_have_Territories_as_second_resource =
        () => query.DataShape.ElementAt(1).Name.ShouldEqual("Territories");

    It should_have_territories_as_the_second_resource =
        () => query.DataShape.ElementAt(1).Name.ShouldEqual("Territories");
}

First, note the with_northwind_CSDL base class. Aaron brought up in an earlier post that this “with…” convention was one that he was actually moving further away from, so in this point, I want to call out more specifically what is happening here. The basic point of this shared context is to setup a mock reader to get the XML metadata from a file rather than from the $metadata clause, which is what is used at runtime.

public class with_northwind_CSDL
{
    const string NORTHWIND_METADATA_PATH = "svc_metadata.xml";
    protected static DataServicesQuery query;
    protected static IDataServicesQueryBuilder queryBuilder;

    Establish context = () => {
        var mockMetadataReader = new Mock<IReader>();
        mockMetadataReader.Setup(r => r.GetData(Constants.SERVICE_URI + "/$metadata"))
            .Returns(new StreamReader(NORTHWIND_METADATA_PATH));

        queryBuilder = new DataServicesQueryBuilder(new CSDLResourceBuilder(), 
            mockMetadataReader.Object) {ServiceURI = Constants.SERVICE_URI};
    };
}

Setting this up in a base class rather than in every spec class saved me a fair amount of repetitive typing – especially since my specifications here are pivoting on how the query shapes the data shape rather than on how the metadata is acquired.

So once I had my basic specs setup, all that remained was the implementation. As you can see from the following, the basic task of creating the data shape is effectively divided among 3 types:

the data reader is responsible for getting the metadata from some source (in the running service, this is done using a HTTP GET request – for my specs, this was mocked with an XML file reader)
the resource builder is responsible for producing resources (the main elements of the data shape) from the metadata provided by the data reader
the query builder is responsible for knowing how to interpret the query URI and ensuring that parent/child projections are correctly constructed.

Based on this separation of roles, you can see that the bulk of the projection logic resides in the query builder class – specifically in the GetResources method. This logic looks like the following:

IEnumerable<Resource> GetResources() {
    var metadata = _metadataReader.GetData(string.Format("{0}/$metadata", ServiceURI));
    _resourceBuilder.Initialize(metadata);

    // initialize resourcs collection with primary resource
    var resources = new List<Resource> {
                                           _resourceBuilder.GetResource(ResourcePath.Split(new[] {'/'}).Last())
                                       };

    //add resources specified by the presence of an $expand query operation
    if (!string.IsNullOrEmpty(OperationsString)) {
        var expandOperation = QueryOperation.ParseQueryOperations(OperationsString).Expand();

        foreach (var expandedPath in expandOperation.ResourcePaths)
            if (expandedPath.Contains("/")) {
                var pathAxes = expandedPath.Split(new[] {'/'});
                for (var i = 0; i < pathAxes.Length; i++)
                    resources.Add(_resourceBuilder.GetResource(pathAxes[ i ], resources.Last()));
            }
            else
                resources.Add(_resourceBuilder.GetResource(expandedPath, resources.Last()));
    }

    return resources;
}

As you can see, this method is basically getting resource names from the data services query string and then asking the resource builder to build those resources. For expanded resources, the builder is asked to produce the resource incorporating the keys of the parent resource.

In sum, I think that dealing with $expand is one of the more interesting aspect of this component, so I wanted to spend a bit more time going through it. To see all of the components working together, you can download the entire source code at http://code.msdn.microsoft.com/ssisastoria. But in sum, the problem looks like: DB (relational) –> Data Services (hierarchical) –> SSIS (relational).

Astoria, SSIS Adapters and BDD – Class Design

hdierking — Wed, 18 Nov 2009 20:11:00 GMT

In the process of writing a post about one specific feature of the design, I realized that I hadn’t actually described the moving parts in the solution. There are 3 projects in the VS solution: the SSIS-specific code, the core application logic, and the application logic tests (specifications). I’ll talk more about the SSIS part later and focus here on the core application logic and tests.

At a high level, there are 2 major logical components that interact to support SSIS at design time and runtime – those are DataServicesQuery and DataServicesQueryBuilder.

The DataServicesQueryBuilder Object

The query builder is responsible for getting the metadata from ADO.NET Data Services and constructing the resources that are used by SSIS during the design time experience (those resources are also passed to the query to be used at runtime for making sure that a data element gets put in the right place in the buffer). The basic elements of the query builder are shown below:

As you can see, responsibilities are further decomposed as follows:

The DataServicesQueryBuilder class is responsible for understanding an ADO.NET Data Services query and producing a query that can be consumed by the SSIS runtime
The IReader interface is responsible for getting stream data from some data source – the concrete implementation used by the query builder is the HttpRequestReader. This implementation is used by the query builder to get service metadata as follows:

var metadata = _metadataReader.GetData(
    string.Format("{0}/$metadata", ServiceURI));

The IResourceBuilder interface is responsible for producing Resource instances from whatever stream data is handed to it. Since ADO.NET Data Services currently has a dependency on EDM (more specifically, CSDL), the only relevant concrete class at the moment is CSDLResourceBuilder.

The only other thing I want to point out from the query builder classes is that we have a class for both Property and for RefProperty. Resource properties are the equivalent of columns (if looking at the data from a tabular perspective) and RefProperties are the equivalent of links. This will become more obvious when I discuss use of the $expand keyword in a future post, but the difference between parent/child (and subsequently, the driver behind inheritance) is based on the following runtime behavior (when extracting data from results data):

public class Property
{
    ...

    public virtual object GetValue(XElement element) {
        var propertyElement = element.Element(XName.Get("content", XmlNamespaces.AtomXmlNamespace))
            .Element(XName.Get("properties", XmlNamespaces.DataServicesMetadataXmlNamespace))
            .Element(XName.Get(Name, XmlNamespaces.DataServicesXmlNamespace));

        var propertyVal = propertyElement.Value;

        var val = Convert.ChangeType(propertyVal, PropertyType);

        return val;
    }
}

public class RefProperty : Property
{
    public RefProperty(string name, Type type, int? maxLength, bool? unicode)
        : base(name, type, maxLength, unicode) { }

    public override object GetValue(XElement element) {
        return element.Parent.Name.LocalName=="feed" 
            ? base.GetValue(element.Parent.Parent.Parent.Parent) 
            : base.GetValue(element.Parent.Parent.Parent);
    }
}

The DataServicesQuery object

Ultimately, the SSIS code simply uses the query builder in order to get a DataServicesQuery object. The query builder configures the query with the collection of resources that it created (called the data shape). The query is responsible for executing the actual query and extracting the results from the returned XML (ATOM feed) into a tabular format which can be used to populate SSIS output buffers. As you can see below, the query class layout is much simpler than the builder:

As you can see, the reader is again used to pull down stream data. The query then uses it’s data shape (resources) to take the XML data stream and create a ResultSet for each resource. 1..n ResultSets are then handed back as the result of executing the query – this is what gets handed back to SSIS and allows the SSIS adapter code to populate the SSIS buffers.

You can download and look at the complete source at http://code.msdn.microsoft.com/ssisastoria. As always, please let me know if there’s stuff that doesn’t make sense or that could be refactored to be improved.

MSpec v0.3

aaronjensen — Wed, 18 Nov 2009 18:20:00 GMT

IMPORTANT NOTE: This is beta (still). The assertion library is new and may have bugs. I’d appreciate you testing it on your specs, but don’t fight specs breaking too hard if they do, let me know what happened and revert, I wouldn’t want to cost you your day chasing down a bug in MSpec. Thanks!

It’s been more or less forever since I’ve incremented the version number of MSpec so I went ahead and did it today. This is just a brief post to let you know what’s new and what I broke—maybe I’ll post some more detail later.

MOST IMPORTANT NOTE: v0.3 removes the dependency on NUnit/XUnit. If you’re upgrading, delete your Machine.Specifications.NUnit.dll and Machine.Specifications.XUnit.dll and remove their references from your projects. MSpec has an all new assertion library (with a bit of borrowed code from XUnit, turns out equality is hard).

I’ve attempted to format the new error messages to minimize debugging, so they’re rather verbose. Please let me know if they are too verbose or if there’s anything else you’d like to see.

You may need to tweak a few specs (ex. ShouldEquals is now generic, so you have to pass it the same type as the thing you’re observing) but most things should “just work”. Let me know if any of the new assertions give pain.

I also added a XCopy Deployable Test Runner manifest for TD.NET so no more fiddling with the registry or dealing with version issues. Pretty slick really. Thanks Jamie for pointing this out to me.

Of course there are plenty of other changes from 0.2 to 0.3 since it’s been like, a year. Here’s some others off the top of my head:

R# runners for 4.0, 4.1, 4.5 and 5.0 and they’re better than ever thanks to Alexander Groß.
Xml output from the command line runner with --xml for CC.NET/Hudson thanks to Barry Woods.
Concern is now Subject
There is no longer a difference between “Establish context” and “Establish context_once”, the context is always established only once. The idea is that observations are observations, they should not mutate state so there is no reason to execute the context more than once. Same for cleanup. If you *really* want to do execute the context for each specification, first try to change your mind, then if you still want it use: [SetupForEachSpecification] on your context class.
You can implement an IAssemblyContext in your spec assembly to set things up before or clean things up after all of your specs in that assembly have run.
You can implement a ICleanupAfterEveryContextInAssembly to perform cleanup after every context (think cleaning up static state, resetting your ServerClock/DateTime replacement for example.
Selenium RC support

Many thanks again to Alexander Groß for adding packaging to MSpec. You can now download builds from herewithout having to build it yourself! Just grab the Artifact from the latest build.

And, as always, the best way to get me is on twitter.

Reactive Extensions for .NET (Rx) released on DevLabs

Matthew.Podwysocki — Tue, 17 Nov 2009 22:50:12 GMT

As you may have noticed, I’ve been blogging a bit about the Reactive Framework, now the Reactive Extensions for .NET (Rx) and how it will change the way we approach asynchronous and event-based programming. Just as LINQ to Objects changed the way we deal with collections, The Reactive Extensions will change the way we deal with asynchronous and event-based programming. Fast forward to today, Rx has been released on Microsoft DevLabs for not only .NET 4 Beta 2, but also .NET 3.5 SP1 and Silverlight 3 as well.

If you’re not familiar with Rx, it is a library for composing asynchronous and event based programs together using observable collections (IObservable<T>). This is especially important in the development of Silverlight, which disallows blocking networking and threading calls from happening and instead forces you into an asynchronous model, but just as well, it’s quite useful for other applications such as traditional desktop applications and web applications. With that comes its own set of challenges. How do I compose these asynchronous events together? How do I handle exceptions? How about cancellation?

The Reactive Extensions, the contain a superset of the standard LINQ operators we’ve come to know and use quite frequently in our applications which are based not off the IEnumerable<T>, but instead IObservable<T>. The IObservable<T>/IObserver<T>, being the mathematical dual of IEnumerable<T>/IEnumerator<T> allows us to go from the pull model to the push model with ease. With that, we can create abstractions over asynchronous methods, events, Parallel Extensions for .NET Tasks, StreamInsight, F# First Class Events and more.

Download it today and give the team feedback.

The High Cost of Losing a Developer

Dave Laribee — Tue, 17 Nov 2009 06:18:00 GMT

You've probably heard statistics about the "high cost of losing a customer." The theory is simple and true (fact?): keeping existing customers and repeat business is much more profitable than attracting new customers. Here are a few facts from BusinessCoach.com:

- For every customer who bothers to complain, there are 26 others who remain silent.
- The average “wronged” customer will tell 8 to 16 people.
- 91% of unhappy customers will never purchase services from you again.
- It costs about five times as much to attract a new customer as it costs to keep an old one.
- Each one of your customers has a circle of influence of 250 people or potential customers who hear bad things about you!

The point of this post is simple: RETAIN YOUR GOOD DEVELOPERS. Getting someone up to speed on a legacy codebase takes a long time and is an expensive undertaking. A large part of the software coach's or manager's job is attracting, developing and keeping talent. And that talent becomes more valuable over time.

Yes, you might be able to replace someone at a lower annual salary, but you have to take into account the complexity of your code portfolio in how long it'll take to make that person productive. A $60K/annum employee may very well take $120K before reaching the productivity level and contribution of the developer who left the team.

Pablo's Fiesta: Open Spaces in Austin

Jeremy D. Miller — Mon, 16 Nov 2009 22:25:00 GMT

Looks like all my buddies at Los Techies are putting on their first Open Spaces event in Austin during February. I'll see you there.

More info here.

Building extensible RIAs at PDC

Glenn Block — Mon, 16 Nov 2009 18:23:00 GMT

This week I’ll be attending PDC in Los Angeles speaking about using MEF in Silverlight.

Building Extensible Rich Internet Applications with the Managed Extensibility Framework

In Hall F on Thursday at 11:30 AM

Learn how the Microsoft .NET Framework 4 introduces the Managed Extensibility Framework (MEF) for building customizable applications that can easily be extended by third parties. Whether you are building an extensible data grid, a custom rules engine, a pluggable editor, or a composite application such as a pluggable CRM system, you want to learn about MEF. Hear how to use MEF to decouple your applications into more maintainable and testable pieces, and partition your application into dynamically deployable chunks that download on-demand.

If you haven’t heard, MEF is available in Silverlight similar to it’s availability on the desktop. But what does it mean to build an extensible app in Silverlight? Yes it’s technically possible, but why should you care? In this talk we’ll explore different application scenarios for when MEF is a good fit. We’ll also explore some of the SL specific features we’ve introduced in order to enrich the MEF experience.

Hope to see you there!

Glenn

Back to Basics: Usage of Static

Patrick Smacchia — Mon, 16 Nov 2009 13:06:00 GMT

Normal 0 21 false false false FR X-NONE X-NONE

The keyword static is somewhat awkward in a pure Oriented-Object world. I would like to expose here the usages of static I came up after 15 years of OO Programming.

The 2 rules of thumb are:

A: Static fields that are not pure value constants should be prohibited.
B: Static classes, that contain only pure functions, that compute outputs from inputs parameters, are allowed. Personally, I am used to suffix such class with XXXHelper.

You might not agree with these 2 simple rules so let me justify them.

What’s wrong with static fields that are not pure value constant? The underlying motivation for creating a static field comes from the need for an object that lives during all the program execution lifetime. Generally this is a central object, shared by many methods. So instead of passing the object as a parameter to all these methods, making it global makes it more convenient, easy or readable. I see 3 main problems in this reasoning:

First, everytime I stumbled on a need for an object that lives during all the program execution lifetime I found out sooner or later (generally later) that I was wrong. There is no such thing. For example imagine that your program become hosted as an addin (in VisualStudio for example). Addin can be unload and reloaded. Thus the program execution lifetime become session lifetime. And you certainly don’t want that the global object lives between sessions. The cornerstone here, to get my point, is the concept of session. It is virtually impossible to forecast up-front what will be sessions in which our code will live in the future. But there is a very common example of session: Automatic tests. An automatic test (or a set of automatic tests) is a mini session in itself. Having static mutable fields makes the code much less testable because all these mini-sessions must shared the same mutable state, with the need to initialize/clean up it.

Tests are generally ran sequentially. But what if several sessions are ran in parallel and they all share the same mutable objects. Then you need to synchronize access to the global object. And trust me, synchronizing accesses will be a major pain, not even taken account that this will slow down significantly your program. I wrote an article on Managing states in a multi-threaded environment without the synchronization pain. One of my preferred tip in the article is Thread / Resource affinity: The idea is as simple as it is efficient here also: making sure that an object is always accessed by the same thread. Basically each thread hold its own instance. If N threads holds N states then states must be attached to N objects and states cannot be static. To do so, using the ThreadStaticAttribute on your static field can do the job. But it is much more elegant to create a dedicated object for each thread that holds the complete thread-bounded context.

Finally, the third problem with static fields comes from the fact that this makes the program much less maintainable. Static fields represent global state. A global state can be accessed from anywhere in source code. Thus from anywhere in the source code, in any scope, you can potential create a dependency to the global state. The dependency is generally non-obvious because it is generally indirect (because the static field is generally encapsulated by a static static property). But the result is that accesses to the global state become anarchical. It is virtually impossible to enforce accesses to a global state and you will end up with some sort of side-effect nightmare. Sooner or later, the need to refactor the global state in order to control its usage will urge (this is another empirical fact I learnt the hard way).

A consequence of all this is that singleton class must be prohibited. From what we just saw, the single static field of singleton makes program harder to test, harder to maintain, and singletons foster concurrency problems. In my opinion, the problem with singleton is that it is the easiest OOP design pattern to learn. As a consequence, every OOP beginner knows the singleton and has a tendency to use it. Instead of using singleton, I use what I call Managers. A manager is a purely non-static class especially created in the spirit that only one instance will live at a particular point in time. A manager doesn’t have the dirty static instance field trick. Instead managers are generally referenced by other more high level managers. All this grape of managers represent the context in which the code is running. Manager can also be easily passed as a parameter to methods that needs locally to access the context. Also, manager are generally free to initialize and clean up, this make them convenient for testing.

Concerning the second rule B, yes I love purely static helper classes, that don’t have state. The simple CQL query…

SELECT METHODS WHERE IsStatic

…just showed that 36% of my code resides in static methods. Object-Oriented is a great mean to model a problematic and creates proper abstraction. However, too often classes contain functions that computes well-defined output from well-defined input. These functions have a tendency to be potentially static and to be implemented with a lot of lines of code. Doing so result in fat classes hard to understand and maintain. This is why I prefer to nest this code inside dedicated helper classes. Not only this makes my instance classes smaller and clearer, but also automatic tests for helper classes are easy to write: tests just need to create input, call a static method, and check output done by computation. There is no hidden state to take care of initialize. Ironically, a helper class might encapsulate its own private mini-object model that models its own mini-concern.

I know that some others might see these helper classes like trash code that result from lack of design. I agree, such helper classes could be avoided but for what? For more instance classes whose purpose is not clear and obvious. Indeed, if their underlying purpose was obvious in the first place, then the idea to put the code in a static helper wouldn’t’ have even come to the mind. Keep It Simple Stupid!

Introduction to the Reactive Framework Part IV

Matthew.Podwysocki — Sun, 15 Nov 2009 23:54:11 GMT

In the previous post in this series, we covered how to turn .NET events into first class values through IObservable instances. By doing so, we were able to do much more interesting things than just subscribe and unsubscribe, instead we were able to create a mouse drag event with little effort through composition. In this post, let’s look at going from push to pull by turning collections into IObservable<T> instances.

Let’s get caught up to where we are today:

From IEnumerables To Observables

As you may remember from the first post and second post in this series, we covered going from IEnumerable<T> to IObservable<T> and the duality between the two. But, how can we switch back and forth between them? Remember that we can create an IObservable from a couple of angles. Let’s first create one by using the chaining together Empty and StartsWith combinators on the Observable class to create an IObservable<T>. Note that this is not the publicly available release of the Reactive Framework, so bear with any name changes.

var observable = Observable.Empty<int>()
    .StartWith(1)
    .StartWith(2)
    .StartWith(3);

This allows me to start out with an empty IObservable<int> and then pre-append the value to the front of the instance, so in this case it would render 3, 2, 1. Next, let’s look at how we might generate a sequence IObservable from 1 to 10.

var observable = Observable.Generate(
    1,                        // Initial value
    value => value <= 10,    // Predicate
    value => value,            // Selector
    value => value + 1);    // Iterator
observable.Subscribe(Console.WriteLine);

In this instance, we supply an initial value, then a predicate to determine when to stop, a result selector, and then finally an iterator. There are plenty of overloads as well on the Generate to suit your needs when creating IObservable<T> values. But, what about taking existing IEnumerable<T> values and converting them to IObservable<T> values? One could use the the Create function which allows us to create arbitrary IObservable<T> instances.

var enumerable = Enumerable.Range(1, 10);
var observable = Observable.Create<int>(observer =>
{
    try
    {
        foreach (var item in enumerable)
            observer.OnNext(item);
        observer.OnCompleted();
    }
    catch (Exception exception)
    {
        observer.OnError(exception);
    }

    return () => { };
});
observable.Subscribe(
    value => Console.WriteLine(value),
    exn   => Console.WriteLine(exn.ToString()));

What we’re doing in the above code is using the Create method which gives us an IObserver<T> that we can post values to. In this case, we’re iterating through a collection and for each item, we’re calling OnNext. At the end of the iteration, we call the OnCompleted method which indicates that there are no more values. If there were some sort of exception when iterating the values, it would be caught and the exception posted to the observer. We can then decide how to handle it during our Subscribe method call, if at all. Finally, we return an Action which is to be called once we unsubscribe from the given IObserver<T>, which in this instance, we do nothing. That’s all fine and good, but there is an easier way of course, by using the ToObservable extension method.

var enumerable = Enumerable.Range(1, 10);
var observable = enumerable.ToObservable();
observable.Subscribe(Console.WriteLine);

Going from IEnumerable<T> to IObservable<T> as you an see is quite easy. Just as well, going back is easy as well through either of two methods, one being calling the ToEnumerable extension method, and the other being the GetEnumerator method.

// To Observable
var enumerable = Enumerable.Range(1, 10);
var observable = enumerable.ToObservable();

// And back again
var enumerableAgain = observable.ToEnumerable();
var enumerator = observable.GetEnumerator();

This ultimately leaves us in control of when we want to switch between the pull and push model, depending on our circumstances. It’s sort of like a buy one get one free in that respect. What makes this interesting is to take a collection and spread it out to multiple instances, for example reading in a file and then you could have multiple subscriptions each process the data in turn. Those possibilities are endless if we want to do certain actions concurrently.

var text = File.ReadAllLines("DocumentToProcess.txt");
var textObservable = text.AsObservable();

// Process concurrently
var textLetterSub = text.Subscribe(CountLetters);
var textWordSub = text.Subscribe(CountWords);
var textVowelSub = text.Subscribe(CountVowels);

There are also more things to come when dealing with collections that we’ll get into in later posts in this series, but this is enough for now.

Conclusion

Code Coverage: What Is It Good For?

Dave Laribee — Fri, 13 Nov 2009 20:06:00 GMT

Absolutely nothing. Or so the story goes...

I had a mini-exchange with my pal Glenn Block last night on Twitter. He was recycling the somewhat old meme that code coverage is a cockamamie metric that, as he puts it, "[code coverage is] a false security blanket on quality".

Measurements (other than net profit, of course) are merely indicators. They give us insight into what we're doing, red flags that help us know when to follow up on and drill into with more qualitative and hands on tools.This is, more-or-less, the idea behind Autonomation or "automation with a human touch."

For example: while code coverage isn't proof that our testing efforts will yield higher maintainability, it does tell us a teams commitment to a test practice.

I tend to believe the most interesting indicators are composite metrics. What happens when we start thinking about combining code coverage with cyclomatic complexity? I'd like to read a (hopefully) inverse relationship over time, coverage rising while complexity falls, as a good indicator that we're doing simple design.

The important thing to remember about metrics is they are only single purpose tools which, correctly applied, gift us with little tidbits of insight. It's up to you to find meaningful and creative uses for the various metrics and their various combinations. Once you start doing that, measurements like code coverage can certainly help you achieve your immediate goals.

Introduction to the Reactive Framework Part III

Matthew.Podwysocki — Fri, 13 Nov 2009 04:06:55 GMT

In my previous post in the Introduction to the Reactive Framework series, I discussed the duality between the IEnumerable/IEnumerator and IObservable/IObserver and how that enables us to generalize the ideas from LINQ to Objects to LINQ to Observables/Events. After the last post, I was asked how we go from events to Observables, so let’s tackle that in this post.

Before we get started, let’s get caught up to where we are today:

From Events...

Let’s start out by looking at the way we’d usually do event-based programming in .NET. First, if we have the need to pass data as arguments to our event, we’ll need to create a class which inherits from EventArgs, and in this case we have a class called SomeEventArgs. This class holds nothing more than just a simple object value.

public class SomeEventArgs : EventArgs {
    public object Value { get; private set; }

    public SomeEventArgs(object value) {
        Value = value
    }
}

Then we’d have a class which exposes an event with an EventHandler of our SomeEventArgs. At some point along the way we’d have a method which invokes our event, keeping in mind that we need to check for nulls before invoking.

public class SomeClass
{
    public event EventHandler<SomeEventArgs> SomeEvent;
    public event EventHandler<SomeEventArgs> OtherEvent;
    
    public void InvokeEvent(object value) {
        var handler = SomeEvent;
        if(handler != null)
            handler(this, new SomeEventArgs(value));
    }
    // Other event invoking code here
}

Now to do something useful with it, you can subscribe and unsubscribe by using the += and –= operators. In this case, I actually declared a method called SomeHandler which will react to the event arguments that I give it.

var someClass = new SomeClass();

// Add handler
someClass.SomeEvent += SomeHandler;

// Remove handler
someClass.SomeEvent -= SomeHandler;

Our SomeHandler method must follow the signature with the sender as an object and the EventArgs derived class, whatever it might be. At this point we react to anything that we’re subscribed to in this method.

private void SomeHandler(object sender, SomeEventArgs e) {
    // Some handler code
}

The problem with this solution is that it isn’t quite compositional. What do I mean by that? What if I want to combine this SomeEvent firing with another event? Or if I want to ensure that an event gets posted on a certain synchronization context? Or if I want to throttle the firing of an event to a certain threshold? All of those are a tad more difficult using standard out of the box .NET events, but, we can with Observables. So, how do we get there?

To Observables

Unlike F#, we don’t get Observables for free when we use events, so we have a little work ahead of us. In order for us to go from events to Observables , we simply need to use the Observable.FromEvent method. There are several flavors that we can use. The first option is to use a little “magic string” juice to wrap our event in an IObservable. This method takes the object to bind for the event and the event name as a string.

public static class EventExtensions {
    public static IObservable<Event<SomeEventArgs>> GetSomeEvent(
        this SomeClass someClass) {
    
    return Observable.FromEvent<SomeEventArgs>(
        someClass, "SomeEvent");
}

This method then returns to us an IObservable<Event<SomeEventArgs>> which allows us to get access to the EventArgs and the Sender through their respective properties on the Event class. Personally, I’m not a fan of magic strings at all and if I can get away with not using them, all the better. Luckily, there are other options. The next option is to use Action<EventHandler<EventArgs>> arguments for both the subscribe and unsubscribe. Keep in mind that this only works for those events which are declared to use EventHandler<TEventArgs>.

public static class EventExtensions {
    public static IObservable<Event<SomeEventArgs>> GetSomeEvent(
        this SomeClass someClass) {
    
    return Observable.FromEvent<SomeEventArgs>(
        h => someClass.SomeEvent += h,
        h => someClass.SomeEvent -= h);
}

If your given event doesn’t have the signature of EventHandler<TEventArgs>, you can still take advantage of the Reactive Framework as well. In this instance, let’s extend the WPF IInputElement to wrap the MouseMove event as an IObservable<Event<MouseEventArgs>>.

public static IObservable<Event<MouseEventArgs>> GetMouseMove(
    this IInputElement inputElement)
{
    return Observable.FromEvent(
        (EventHandler<MouseEventArgs> h) => new MouseEventHandler(h),
        h => inputElement.MouseMove += h,
        h => inputElement.MouseMove -= h);
}

What this does for us behind the scenes is create an IObservable/IObserver to handle the events as they happen. Each invocation of the event causes the OnNext to be fired with the Event<TEventArgs> argument. In this case, the OnComplete and OnException are not implemented as part of the IObserver instance.

An Example

Let’s create a little example of drawing on a canvas in WPF. In order to do that, we’ll need not only the MouseMove extension as we have above, but also MouseLeftButtonDown and MouseLeftButtonUp. Each of those extension methods should follow the same pattern as above in terms of needing the extra parameter to convert the EventHandler<MouseButtonEventArgs> to a MouseButtonEventHandler.

First, we need to capture position of the mouse during our mouse moves. After that, we’ll need to zip together the mouse move and another mouse move that is one ahead in order to capture a delta. Finally, we’ll capture the mouse downs and the mouse deltas until the mouse ups happen. Listed below is code to make that happen.

var mouseMoves = from mm in mainCanvas.GetMouseMove()
                 let location = mm.EventArgs.GetPosition(mainCanvas)
                 select new { location.X, location.Y};
                 
var mouseDiffs = mouseMoves
    .Skip(1)
    .Zip(mouseMoves, (l, r) => new {X1 = l.X, Y1 = l.Y, X2 = r.X, Y2 = r.Y});

var mouseDrag = from _  in mainCanvas.GetMouseLeftButtonDown()
                from md in mouseDiffs.Until(
                    mainCanvas.GetMouseLeftButtonUp())
                select md;

The code should read rather straight forward as we attach to the mainCanvas object in order to draw. The interesting functions are the mouseDiffs and the mouseDrag. The mouseDiffs takes the mouseMoves and then skips 1 ahead and zips it with another mouseMoves, and projects to hold both left and right values. The mouseDrag function takes events from left mouse button down and then from mouseDiffs until the left mouse button up Observable happens.

Now that we have our mouseDrag, what can we do with it? Well, let’s go ahead and draw on our canvas:

var mouseSub = mouseDrag.Subscribe(item =>
{
    var line = new Line
                  {
                      Stroke = Brushes.LightSteelBlue,
                      X1 = item.X1,
                      X2 = item.X2,
                      Y1 = item.Y1,
                      Y2 = item.Y2,
                      StrokeThickness = 5
                  };
    mainCanvas.Children.Add(line);
});

Our mouseSub subscription creates a new line with our coordinates from zipped mouseDrag Observable and then adds it to our canvas. Consider how little code was required to make this happen, and try doing that with standard .NET events. Quite frankly, this kind of solution makes event-based programming using .NET events expressive enough to be usable.

Conclusion

Astoria, SSIS Adapters and BDD – On BDD and MSpec

hdierking — Fri, 13 Nov 2009 01:04:00 GMT

Let me preface this post by saying that it is about my (early) experiences practicing BDD in the context of a specific specification framework – MSpec. There are plenty of folks out there who are much more authoritative on this subject than I. And if you happen to be one of those people, I would love to get your feedback on what I’ve done here and how I could do it better.

In developing a SSIS data source to talk to an ADO.NET Data Services endpoint, shifting my approach from a basic TDD style to specifications was not originally part of the plan. However, as I got deeper into development, I felt as though I actually wanted a bit more rigor in terms of semantic restrictions (e.g. arrange/act/assert) than I was getting out of the box from my unit test framework. MSpec had been on my radar for a little while (I was introduced to both this framework and BDD in general at ALT.NET), so I took a quick look - and after staring oddly for a moment at the =()=> syntax decided that it provided the kind of semantic consistency I was really looking for. Additionally, the fact that it could plug into the ReSharper unit test runner (with underscores removed from boxcar casing no less!) made it quick and very easy to get up and going on the journey writing specifications rather than tests. Going through the specification writing exercise on this project has me convinced at this point that as someone else once said, BDD is TDD done right. I certainly plan to keep pursuing specifications over traditional unit tests in future efforts.

So where to start?

I’m not going to spend time in this post talking about how to use MSpec – there’s already lots of good guidance out there. That said, because the syntax initially looks a bit funky, I will say 2 quick things that are helpful to keep in mind when you first look at specifications built on top of MSpec.

Everything in your specification (test) class is a field. And all of the work gets down via delegate fields which are of types with names like “Establish”, “Because”, and “It”
Regarding the =()=> syntax, you are probably already comfortable with lambda expressions, so you can ignore the => on the right - and you can get the assignment operator on the left. Therefore, that just leaves the empty parenthetical – and this just means that your delegate (lambda) is for a function that has no arguments. There, all better?

I already had some unit tests written – though some of them were more of integration tests and revealed some pretty serious problems in my design – but we’ll deal more with those topics later. So my starting point was to try and re-write some of my existing tests as specifications. For example, take the following:

[Test]
public void Given_Null_URI_Throw_Exception() {
    try {
        new DataServicesQuery(_dataServicesQueryBuilder, null, "Suppliers");
    }
    catch(ArgumentNullException ex) {
        Assert.AreEqual("serviceUri", ex.ParamName);
    }
}

Re-writing this as a specification resulted in the following:

public class when_service_uri_is_null : with_dummy_resource_builder_and_reader
{
    static NullReferenceException lastException;

    Because service_uri_is_null = () => {
        try {
            queryBuilder.ServiceURI = null;
            queryBuilder.ResourcePath = "Products";
            queryBuilder.GetQuery();
        }
        catch (NullReferenceException ex) {
            lastException = ex;
        }
    };

    It should_have_specific_error_message = 
        () => lastException.Message.ShouldEqual("Service URI must be provided");

    It should_raise_an_exception = 
        () => lastException.ShouldNotBeNull();
}

A couple things to point out here.

First and foremost, there is a more rigid semantic that is enforced by both convention and even by the type system. The convention is that classes follow a “when X [with Y]” pattern (I found that this class name should map very tightly to the instructions contained in the Establish and/or Because expressions – if it does not, you may have a problem in your design). Also, assertion (“It” expressions) delegates should follow a naming convention whereby “should …” is followed by a description of the assertion. MSpec also uses the type system to enforces this convention in how the delegate types are named (e.g. – “Because”, “It”).
All of the NUnit attributes identifying test fixtures and tests are gone. MSpec is smart enough to figure out what is a test vs. what is a regular old class based on the MSpec delegates that make up the class. When looked at through the NUnit lens, the specification class remains the test fixture, and each “It” delegate instance becomes an NUnit test.
Notice the with_dummy_resource_builder_and_reader class. As with unit tests, inheritance can be a great way to provide shared context to multiple test fixtures. This base class is responsible for 2 things: 1) exposing a few data fields for its child classes, and 2) providing its own Establish delegate to setup those data fields.
For me, this format of specifications seemed to yield another benefit – without explicit effort. That benefit was more tests with each test having a much better definition around its responsibility.

This particular type of test – one where you’re expecting an exception – was one where I wasn’t 100% confident in my approach. However, I didn’t find any baked in support from MSpec – and frankly, for the framework to provide a feature similar to NUnit’s ExpectedExceptionAttribute doesn’t really make a lot of sense to me for writing specifications anyway. You can see above the approach that I took – simply trap the exception and then run a couple tests over the exception to assert various characteristics of it. If there’s a better way to go about this, I would love to hear it.

After I started getting the hang of writing specifications by rewriting my tests, I began venturing into new parts of my application – and this is where I really started realizing just how much ceremony exists in traditional unit testing frameworks compared to something like MSpec. Because mostly everything in MSpec is a field, there’s no need to provide any kind of method body, etc. after declaring the title of the specification. So, for example, when I wanted to write specifications around executing a query, I was able to quickly capture the specs as follows:

public class when_query_is_executed_for_products_with_expanded_suppliers
     : with_query_shape_factory
{
    It should_have_2_result_sets;
    It should_have_a_result_set_named_products;
    It should_have_8_columns_in_products_result_set;
    It should_have_77_rows_in_products_result_set;
    It should_have_a_result_set_named_suppliers;
    It should_have_13_columns_in_suppliers_result_set;
    It should_have_77_rows_in_suppliers_result_set;
}

Remember, “It” is simply a type and “should_have_2_result_sets” is simply a field declaration – no need to initialize it with anything just yet. When the MSpec runner is executing, it checks to see whether the field has been initialized – if it hasn’t, as in this case, it flags it as if you used NUnit’s IgnoreAttribute.

Finally, I mentioned earlier in this post about using ReSharper to run MSpec specifications. While I would probably still be interested in MSpec if this integration didn’t exist, the fact that it was there removed any question or hesitation. We talk a lot about the readability of tests, and of tests being the spec – and this is, based on what I’ve experienced so far, where specification frameworks like MSpec really shine. Bringing that goodness directly into Visual Studio ala ReSharper mades the feedback loop even tighter, and really made for a happy development process.

If you’ve read this far and you have no clue what this project is that I’m talking about, see the overview here. If you want to download the code and look at it yourself, check it out at http://code.msdn.microsoft.com/ssisastoria

Switching virtual networks the easy way

pvanooijen — Thu, 12 Nov 2009 10:40:00 GMT

For my laptop I am using server 2008 as OS. The main reason is Hyper V, for every different project I can fire up one or more appropriate virtual servers. For the details on setting up such an environment see this post. On the advantages of using Server 2008 R2 see this post. Traveling around the laptop meets a lot of different networks, from the Office domain of which it is a member to some kind of wireless in the wild open. Sometimes I want my virtual servers to see and be seen by the outer world, sometimes I don’t.

You can do a lot of network configuration in Hyper-V, in this post I described a rather baroque configuration where all kinds of network connectivity can be switched on and off. Which worked well but is far more complicated than needed. By now I have found out that switching networks in Hyper –V can be as simple as this:

Create a virtual network for every adapter on the laptop.

My laptop has an internal (fixed IP address) adapter and an external USB adapter (depending on DHCP). The Hyper-V virtual network manager does not allow you (yet ?) to pick Wireless or dial-up adapters for a virtual network.

Assign each virtual machine just one network adapter.
Select the active adapter as needed.

The good thing is that this can be done while the virtual machine is running. Changing the network is instantaneous. Just like pulling the cable and inserting another one. So much simpler than all the previous multi network fiddling.

[ANN] DC ALT.NET – 11/24/2009 “Stop Talking, Start Teaching” with Jeff Casimir

Matthew.Podwysocki — Tue, 10 Nov 2009 19:48:14 GMT

As a group, we’ve been focusing a lot on technology on such things as JavaScript, MongoDB and so on. This month, I’d like to take a step back as a group and instead focus on some soft skills. When I started this group back in 2007, I had the idea that this group would help others not only become better technologists in a wide array of technologies, but also better teachers as well. On November 24th, we have Jeff Casimir to help us understand how to become a better teacher with his presentation “Stop Talking, Start Teaching: 5 Rules for Successful Presentations and Demos”. Listed below are the details:

Stop Talking, Start Teaching: 5 Rules for Successful Presentations & Demos

Some of the most important work we do as programmers involves pitching ideas. Whether it's inside the team, searching for funders, or running demos for potential clients, we all need to present ideas and cultivate an audience. The past decade has seen the rise of the presentation as an important communication medium, but few people do it well. There are many resources on how to make good looking slides, but we need to focus on the method, structure, and content.

We don't need to be "speakers" – we need to be teachers. In this session we'll explore the five most essential rules to successful teaching and how they apply to you.

Attendees will, by the end of the session, have a framework for designing and critiquing their work and that of others. This new understanding will make them better teammates and more successful salespeople.

About Jeff Casimir

After majoring in Computer Systems Engineering I joined Teach for America and began a career in education. I taught high school Computer Science for four years before moving into school administration. As a Vice Principal I was responsible for evaluating and hiring new teachers, observing and coaching existing teachers, and defining much of the school's academic process. Most of all, I hate boring presentations. I want these smart, experienced people to Stop Talking, and Start Teaching.

Hope to see you there!

Design Conundrum for StoryTeller -- Opinions Wanted

Jeremy D. Miller — Mon, 09 Nov 2009 21:31:00 GMT

I made the preview release of StoryTeller a couple months ago as soon as I got some sort of "intelligent editor" functionality going in the WPF client so you very rarely ever had to expose your tender eyes to the horrors of the underlying Xml storage. Fine, except for one little problem: the UX and usability suck (and I know because my team uses it and I'm irritated with it when I use it. Hooray for dogfooding). It's also butt ugly.

Flash forward to the past couple months. My team at Dovetail is putting the finishing touches on our online rules engine editor feature in our product (I'm blogging about it soon). We basically use StoryTeller as our "DSL" engine, but we built a completely web based editor for the StoryTeller grammars and integrated that into our web application. I had originally envisioned this module as being written in Silverlight because of the complexity of the dynamic layout and behavior. Instead, we decided to build the UI out with client side JavaScript. We do generate HTML templates on the server representing the DSL grammars that embed behavioral directives with class attributes and copious usage of the jQuery Metadata plugin, but all the behavior of the UI is accomplished through client side scripting. My team might not completely agree, but frankly I'm thrilled with how it's gone. The UI looks pretty good and the screen behavior was relatively easy to create with the HTML/JavaScript/CSS/jQuery combination. We were even able to TDD quite a bit of the JavaScript development using QUnit.

So here's my dilemma, I'm committed to making StoryTeller rock and I absolutely need to replace the existing test editor functionality. At this point I have two options -- either write a new version using WPF that closely mimics the improved usability that we built at Dovetail or stick with JavaScript/jQuery/HTML and embed a WebBrowser control into StoryTeller's WPF client (which I already do). Here are the facts as I see them and I'm obviously biased at the moment toward the HTML/JavaScript model -- but I'm looking for other people's opinions, especially about WPF:

Pro-WPF

It's an excuse/reason to learn a lot more about WPF for me. I think I'd finally have to dig into data templates, styling, and attached behaviors to do this well.
Embedding a WebBrowser control into a WPF app and letting more COM stuff in the door is kloogey and occasionally error prone. The integration is odd
Might run faster
Richer editing controls

Pro-HTML/JavaScript/jQuery

The amazing jQuery ecosystem of plugins and support. At a minimum, I'd be using the jQuery UI sortable/draggable stuff, jQuery Validation, and jQuery Metadata. I went looking today and I just don't see any substantial OSS community around WPF other than more copycat MVVM frameworks than you can shake a stick at.
I'm much more familiar with the jQuery way of doing things and HTML
I've got a successful working prototype to start from
Many of the mechanics are much simpler with jQuery than with WPF. For example, I want users to be able to reorder parts of the StoryTeller specification by dragging elements around. This is just a matter of some declarative class names in HTML with jQuery, but the WPF equivalent looks exceptionally nasty.
I actually like coding with JavaScript and the dynamic features make it relatively easy to extend the model with less ceremony than a static typed language
The feedback cycle is faster with JavaScript than C#
I'd really love to be able to allow users to embed the StoryTeller editor in their own applications, and the HTML/JavaScript model has much more reach than a WPF or even a Silverlight model would.
I think this model is actually much easier to test than the WPF model. It's hard to write automated tests against WPF without something special like TypeMock that can get around how the WPF team sealed so many of the routed event classes or an expensive 3rd party tool that screen scrapes.
Truth be told, I don't really like WPF. It's weird and has more "oddball-ness" to it than any other technology I've ever used. There are parts of it that I love (the layout system comes to mind), and others that drive me batty (XAML's runaway verbosity, binding expressions, routed events). I don't hate it on first sight the way I did WebForms back in the day, but something about it just doesn't agree with me.

Call it a Wash

I've got JetBrains RubyMine for the JavaScript editing, so the tooling support is not really an advantage for the WPF/C# solution anymore
qUnit isn't that hard to use, so I'm not sure that the C# code is any easier to test and I have a recipe for integrating qUnit tests into an automated build anyway.

Anyway, it's been interesting. Now, what do you think?

On being Culture Aware

Patrick Smacchia — Mon, 09 Nov 2009 14:26:00 GMT

Normal 0 21 false false false FR X-NONE X-NONE

A typical development pitfall comes from forgetting about the culture setup of the machine on which the code is executed in production (typical it works on my machine mistake). Imagine that a program running on a French machine stores a float as the string "3,14". The float value cannot be restored from the same program running on a US machine. It would expect the string to be "3.14" with a dot, not a coma.

To prevent such mistake in the NDepend code I created the following CQL rule. Basically, it forces the code base to use only Parse() and ToString() methods that accept a parameter of type IFormatProvider. As far as I know, the problem occurs on DateTime, float (Single), double and decimal.

// <Name>Float and Date Parsing must be culture aware</Name>

WARN IF Count > 0 IN SELECT METHODS

FROM TYPES "System.DateTime",

"System.Single",

"System.Double",

"System.Decimal"

// The NameLike CQL clause operates on the signature

// "methodName(type1,type2...typeN)"

WHERE (NameLike "Parse\(" OR NameLike "ToString\(") AND

!NameLike "IFormatProvider"

With this rule set up, in the future we won’t forget about caring for culture when converting from/to string. Previously, I already wrote a post concerning this Nurture Knowledge Database agile state in mind.

FxCop has a similar rule named Specify IFormatProvider. We are inclined to append this CQL rule to the set of default CQL rules, what is your opinion?

Using Latest NUnit Version With MSpec

hdierking — Sun, 08 Nov 2009 20:11:00 GMT

As I’ve been working another blog post about my adventures in learning BDD ala MSpec, I ran into something that I wanted to share as the problem wasn’t immediately obvious (to me anyway).

The problem is that MSpec has a dependency on version 2.4.6.0 of the nunit.framework assembly – and frankly, it’s not really a very big problem. In fact, I didn’t even notice it until now because I had been running all of my specs exclusively through the Resharper runner (enabled by the MSpec plugin).

** As a side note, using R# with the MSpec plugin absolutely rocks, and while the runner is needed for integrating MSpec with your automated builds, I would totally recommend R# for your regular dev rhythm.

In my project, I originally had some integration tests in the same assembly as my specifications. As a result, I had a direct reference to NUnit (version 2.5.2.9222) in addition to the reference to Machine.Specifications and Machine.Specifications.NUnit (which has a reference to version 2.4.6.0). As a result, the later version of NUnit was being copied to my bin folder and throwing a file load exception for every specification when run using the console runner.

My initial reaction was to simply move all my integration tests to a different assembly and remove the direct NUnit reference in my specifications assembly (remember that referencing Machine.Specifications.NUnit will copy the version of NUnit to the bin folder that MSpec expects. No problem – everything works as you would expect.

However, it seems reasonable to me that, at least in larger projects that are wanting to incorporate specs and not have to re-write all existing tests, that it would be good to simply reconfigure the MSpec console runner to use the latest version of NUnit. This can be done easily using the standard .NET assembly binding redirection feature. Simply create a file named Machine.Specifications.ConsoleRunner.exe.config and drop it into the MSpec folder that contains your console runner (Machine.Specifications.ConsoleRunner.exe). The contents of the file should look like this.

<?xml version="1.0"?>
<configuration>
    <runtime>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
          <assemblyIdentity name="nunit.framework"
                            publicKeyToken="96d09a1eb7f44a77"
                            culture="neutral" />
          <bindingRedirect oldVersion="2.4.6.0"
                           newVersion="2.5.2.9222"/>
        </dependentAssembly>
      </assemblyBinding>
    </runtime>
</configuration>

Run the MSpec console again, and voila! Works like a charm.

Astoria, SSIS Adapters and BDD – Code

hdierking — Fri, 06 Nov 2009 19:57:00 GMT

I’m currently working on the post talking about how, in the course of writing my SSIS data source for ADO.NET Data Services, I stumbled into replacing my unit tests with specifications. However, in advance of that, I wanted to go ahead and provide the link to the source code. I would love to get feedback on what’s working and what’s not so that I can get better – and so that ideally, those the random stranger who downloads and looks at it on some idle Tuesday gets better as well.

The code can be found at http://code.msdn.microsoft.com/ssisastoria

December 2-3 London Workshop Update

Dave Laribee — Thu, 05 Nov 2009 15:29:00 GMT

Forgive the self-promotional interruption, but quick housekeeping note:

I've asked the company hosting my workshop in London to reduce the price of my 2-day workshop on December 2-3 aimed at tools and techniques technical leads, folks interested in lean, standard work, etc. to try to get as many people there (it's a group collaboration focused thing). I also want to get as much feedback as I can. If you book before November 15th, the price is £550.

The code is the truth, but it is not the whole truth

Patrick Smacchia — Wed, 04 Nov 2009 13:55:00 GMT

In a recent interview from Grady Booch, co-creator of UML, Grady said:

When Jim, Ivar, and I began our journey that became manifest in the UML, we never intended it to become a programming language. I think that there's a fairly narrow domain for which model-driven development makes sense but that we should return to the roots of the UML, which was to be a language for visualizing, specifying, constructing, and documenting the artifacts of a software-intensive system—in short, a graphical language to help reason about the design of a system as it unfolds. Most diagrams should be thrown away, but there are a few that should be preserved, and in all, one should only use a graphical notation for those things that cannot easily be reasoned about in code. As I've also often said, the code is the truth, but it is not the whole truth, and there are things such as rationale, cross-cutting concerns, and patterns that cannot easily be recovered or seen from code.... These are the things for which a graphical notation adds value, and any such notation should be used only if it has predictive power or reasoning power (meaning, you can ask questions about it).

I especially cannot agree more with:

As I've also often said, the code is the truth, but it is not the whole truth, and there are things such as rationale, cross-cutting concerns, and patterns that cannot easily be recovered or seen from code....

The code is the truth, the code represents the whole asset of a software shop. But code base is a complex asset. Call them patterns, rationales, intentions or decisions, there are many valuable intellectual artifacts buried in a code base that cannot easily be recovered or seen from code.

This is something I realized in 1997 while trying for the first time to demystify a C++ legacy code base. I was angry! What, in 1997, we, the professional developers, haven’t any tooling to visualize code? I remember also, in the architecture department next door, the architects were spending days doing diagrams that had nothing to do with what actually the code was. Putting hands in the code was something considered with low value. It was let for fresh graduated like me.

Hopefully, the result of this period was the seed from where the idea of a tool such as NDepend came to me. Code Metrics, Treemap, Dependency Graph, Dependency Matrix, Instability vs Abstractness diagrams… are natural artifacts to shed light on all these precious intentions buried in the code. I especially like the word intentions that was often used by Bruno Boucard, a former colleague I have great respect for. The idea of a language such as Code Query Language came to me when the need to make all these intentions concrete and verifiable, was clear.

What we do is the result of what we did and who we met, no matter whether it was good or bad experiences. There is no such thing as: today, I gonna find an idea to make my own business. Good ideas come naturally along the way, through what, Paulo Coehlo, would call signs.

Introduction to the Reactive Framework Part II

Matthew.Podwysocki — Tue, 03 Nov 2009 05:28:37 GMT

In my previous post, I talked a little bit about the parts of the Reactive Framework that are coming to the .NET 4 BCL as well as their implementation in F# as part of F# First Class Events. This time, let’s come back to the Reactive Framework itself from which the IObservable<T> and IObserver<T> originated. As you may remember, you can play with the bits of the Reactive Framework from the Silverlight 3 Toolkit.

In the first post in the series, I gave a basic introduction to the basic problems we have with asynchronous and reactive programming. We covered some of the evolution of the .NET framework with asynchronous programming and where the Reactive Framework fits. This time, I’ll pick up where we left off to talk about one of the important ideas behind the Reactive Framework, the duality of Enumerable and Observable.

The Duality of Enumerable & Observable

During Erik’s appearances on Channel 9, an important, yet recurring theme was the mathematical duality of the Enumerable to the Observable. What does that mean exactly though?

We remember from our first post in the series where we talked about the pull (interactive) versus the push (reactive) model. The pull model, represented by the iterator pattern of IEnumerable<T>/IEnumerator<T> states that we must explicitly call a method in order to get each item from our abstracted collection. On the other hand, our push model, represented by the observable pattern of IObservable<T>/IObserver<T> states that we register an interest through a subscription and then items are subsequently handed to us from some abstracted collection.

First, let’s look at the iterator pattern, which is a common interactive programming pattern that’s used quite frequently inside .NET. This consists of two interfaces, the IEnumerable<T> and the IEnumerator<T>. Listed below are some of the highlights of these two interfaces. I cut down what they are to their essential bits.

public interface IEnumerable<T>
{
    IEnumerator<T> GetEnumerator();
} 

public interface IEnumerator<T> : IDisposable
{
    T Current { get; }
    bool MoveNext();
}

The IEnumerable<T> interface exposes a single method which returns an IEnumerator<T> to iterate some object. The IEnumerator<T> gives us the ability to get the current item and determine whether there are more items to iterate.

To think about basic asynchronous programming, we must consider a basic Gang of Four Pattern, the Observer Pattern. This pattern describes where an object called an Observable, maintains a list of its dependent Observer classes and notifies them automatically of any state changes. In Java, this should be rather familiar territory with the java.util.Observer class and the java.util.Observable interfaces. In the Reactive Framework, this approach is no different, as we have an IObservable<T> interface which maintains a list of dependent IObserver<T> interfaces and notifies them automatically of any state changes. Let’s look at the signatures below:

public interface IObserver<T>
{
    void OnCompleted();
    void OnError(Exception exception);
    void OnNext(T value);
} 

public interface IObservable<T>
{
    IDisposable Subscribe(IObserver<T> observer);
}

What you may notice, and what you may have seen on some of Erik Meijer’s Channel 9 videos is that there is a duality between the Enumerable and Observable, given the signature of these interfaces. To explain what I mean, let’s first look at the flow of the Enumerator:

The GetEnumerator function takes no arguments and returns the IEnumerator<T> which has a no-argument function and returns the next item in the list through MoveNext and Current. Now, let’s look at the inverse of it.

The duality of the Enumerable solution is quite simple. We have our observer on the left hand side which has an action function which takes a T argument and returns void. The right hand side is the notification that is sent when something happens. This is all managed by the Observable as it maintains a list of all observers so that it can notify them.

To make this a little more concrete, let mash these two sets of interfaces together in order to dualize. First, let’s start with the IEnumerator<T> interface.

public interface IEnumerator<T> : IDisposable
{
    T Current { get; } // throws exception
    bool MoveNext();
}

What we’ll notice is that we have two items the Current property with a get accessor and the MoveNext method which takes nothing and returns a boolean that indicates if it can move next. You’ll also notice that our Current property also could throw an exception. If we’re in the Java world, we’d explicitly mark this interface with throws Exception, but as C# doesn’t have that, this comment will do. Now, let’s look to dualize this interface.

public interface IDualEnumerator<T>
{
    void SetCurrent(T value | Exception ex);
    void MoveNext(bool canMove);
}

At first glance, all we have to do is flip our Current property to have a setter. Because the Current yielded a value or threw an exception, that’s the way we’ll model it for now. Secondly, our MoveNext will now take a boolean to indicate whether we’re done. Let’s refactor this a little bit more and break the SetCurrent into two distinct operations, one for yielding the next value, and one for throwing an exception.

public interface IDualEnumerator<T>
{
    void Yield(T value);
    void Throw(Exception ex);
    void MoveNext(bool canMove);
}

Now that we have that under control, let’s then move onto our MoveNext method. As you’ll notice, we’ll continue to send in a true until we’re finished pushing values. Instead of calling MoveNext repeatedly, let’s instead call this only once when we’re finished.

public interface IDualEnumerator<T>
{
    void Yield(T value);
    void Throw(Exception ex);
    void Break();
}

What we then end up with is pretty much the signature of our IObserver<T>. By renaming our Yield to OnNext, our Throw to OnException and Break to OnCompleted, we’re left with our original IObserver<T> interface.

public interface IObserver<T>
{   
    void OnNext(T value);
    void OnError(Exception exception);
    void OnCompleted();
}

What about the dual of IEnumerable<T>? Let’s first look at what that interface is.

public interface IEnumerable<T>
{
    IEnumerator<T> GetEnumerator();
}

This interface has a single method called GetEnumerator which returns an IEnumerator<T>. How might we dualize this interface? Well, we already know that the dual of the IEnumerator<T> is IObservable<T>, just as well, we need a way to attach interest in a given dualized Enumerator, so we’ll create a method called Attach. Just as well, when we attach interest, we also need a way to detach our interest for a given observer. That might look like the following.

public interface IDualEnumerable<T>
{
    void Attach(IObserver<T> observer);
    void Detach(IObserver<T> observer);
}

If we look at the Java interfaces for Observer and Observable, these are pretty close in signature. There ultimately is a problem with this scenario in terms of composition. At each level, it needs to remember which things maps to which so that you could undo any of the observers if need be. Not doing so could lead to space leaks and is generally a bad idea. How could we solve this then? Ultimately, we need a way to track our attachment once we call the Attach through some form of a return value. At this point, the tracking is nothing more than needing the ability to detach our given Observer. If you’ll notice in our IEnumerator<T> interface, it inherits the IDisposable interface which gets one thinking. Could we instead, change our Attach method so that when we call it, we return an IDisposable instead which gives us the ability to track our subscription and be able to clean up after we’re done? That’s exactly the approach that the IObservable<T> takes in the final version of the interface.

public interface IObservable<T>
{
    IDisposable Subscribe(IObserver<T> observer);
}

Now that we’ve proven the duality, why does it matter? Because of this duality, the laws that apply to LINQ to Objects now can apply to LINQ to Observables (or Events). And in fact, we can see that through the Observable class which has many of our standard LINQ combinators such as:

Aggregate
First/FirstOrDefault
GroupBy
Join
Last/LastOrDefault
Select
SelectMany
Single/SingleOrDefault
Skip/SkipWhile
Take/TakeWhile
Where
Zip

Because of this duality, we can do such simple things as fire the ProgressChanged event until the RunWorkerCompleted event happens.

var worker = new BackgroundWorker();

var percentage = from progress in worker.ProgressChangedEvent()
                           .Until(worker.RunWorkerCompletedEvent())
                 select progress.EventArgs.ProgressPercentage;

var subscription = percentage.Subscribe(
    p => Console.WriteLine("Percentage Complete: {0}", p));

That’s only scratching the surface of the power of the Reactive Framework. We’ll go in depth in the next part of the series on the combinators. We could go into the fact that the Enumerable solution is really just the lazy list monad and the Observable solution is really just the continuation monad, but that’s for another time as well.

Conclusion

Should I use MEF with an IoC container? - Part 1

Glenn Block — Sun, 01 Nov 2009 06:36:00 GMT

In my post "Should I use MEF for my IoC needs" we took a look at the question of whether or not it is appropriate to use MEF instead of a traditional IoC container. In this post we'll discuss the question of whether or not you should use both an IoC container and MEF in the same application.

MEF and IoC container concerns

As we've mentioned in the past, MEF's primary design in V1 is around open-systems whose components (parts) are dynamically discovered at runtime. IoC containers on the other hand are primarily designed for decoupling a closed (fixed) set of application components in order to improve the maintainability of the system by the engineering team. In other words each is addressing a specific set of concerns. So if that is the case then is it plausable to use both in an application? The answer is Yes. Not only that, but we specifically designed MEF with the idea of integration with other containers in mind. Below will explore one of the key-scenarios for doing this.

Enabling third-party extensibility in an IoC container-dependent application.

You have an existing customer management system that uses an IoC container such as Castle Windsor which you use in order to promote separation-of-concens and testability. The application leverages advanced Windsor facilities for integrating with NHibernate and for applying cross-cutting-concerns (AOP) such as logging, etc. The loosely coupled design of the system has proven to be greatly beneficial and has minimized the friction for developing and deploying the system through several releases.

At some point requests start pouring in from customers want to create their own extensions in order to customize the application in their specific environments. Additionally several ISVs express interest in providing third-party value-adds such as telephony integration, and scheduling. You determine that adding a third-party extensibility model to the application makes sense in order to address this opportunity. Through your research you come across MEF and realize that it provides the infrastructure you need for extensibility. However it does so in a manner that overlaps with many of the functions that Windsor is providing. What to do? Let's explore various options.

a. Migrate the existing code to use MEF and remove the IoC container completely.

One option is to consider ripping out Castle and just using MEF. Technically this sounds reasonable, however after investigation it is not feasible. There are literally thousands of components that are currently registered in the Windsor container. Moving these to MEF involves modfying all of these existing classes in order to make them discoverable by MEF, not to mention refactoring all the places in the application where the container is accessed. Because the changes are so widespread, the likelihood of introducing breakages / bugs in the system is very high. Additionally there are the various Castle facilities such as Dynamic Proxy, and NHibernate integration which MEF does not yet support, which will have to authored by the team. The cost / risk of such changes is too prohibitive, not to mention that it seems like it should be unnecessary.

b. Implement a custom extensibility mechanism built on top of the IoC container?

Another option is to author a home-grown solution. This can be a viable option depending on the needs, and the willingness to author your own solution. For simple cases you can easily facade an extensibility mechanism on top of an IoC. Once you start getting into more advanced features like lazy instantiation, metadata, recomposition and stable composition, you are in for quite a bit of work.

c. Have both the IoC and MEF both present in the same application?

The final option I want to discuss (and the one I recommend for these situations) is to use both. Use the IoC container for addressing internal application decoupling needs, and use MEF for third-party extensibiltiy needs, that is the IoC container for IoCish things and the MEF container for MEFfish things :-)

This path is appealing for several reasons.

It works with existing codebases whether they be applications or frameworks. it does not require any significant rewrite.
It allows leveraging the strengths of each container for the things it does best. For example leveraging MEF's metadata model for discriminating between different extensions vs the dynamic proxy support in Castle.

However, there are several caveats of this approach.

It raises questions such as:
- Which component (part) should live in which container?
- Who owns the lifetime?
- Can a MEF part access services coming from the IoC? How?
- Can the IoC access exports coming from MEF? How?
It increases complexity in that now there are components living in mulitple places.
It increases the potential points of failuire within the application.

All of these issues are managable. In my next post we'll see how.