Will Beattie's Technical Blog

Windows Azure AppFabric Service Bus Queues API

noreply@blogger.com (Will Beattie) — Tue, 28 Jun 2011 14:00:00 +0000

Recently Microsoft released a new App Fabric SDK 2.0 CTP including some great new features,

You can grab all the bits and pieces from here and/or read the release notes.

Some of the highlights include:

Publish/Subscribe which are called Topics
Message Queues
Visual Studio Tools
AppFabric Application Manager
Support for running WCF & WF

The part that interested me the most is the Queues feature and that is what I’m going to be exploring in this post.

Overview of Queues

Message Queues are not a new concept allow for more reliable and scalable communication between distributed systems than pure request/response. Solutions like MSMQ, NServiceBus already exist to solve this problem for locally connected systems. What the Queues API provides is that it provides similar features but the messages are being transported across the internet and persisted in the cloud.

There is currently a Message Buffer available in Azure but this has serious limitations:

Messages only persisted for a maximum 10 minutes
Maximum of 50 messages
Requires the client to be connected
Charged per connection not per message
Complex API
Max message size of 8 KB

The Queues API addresses a number of these issues by adding.

Long term persistence (No clarification on the SLA yet)
Simpler API
Max message size of 256 KB
Sessions
Larger size of queue (100 MB currently but set to increase)

To start testing out the Queues feature you need to first Login to the AppFabricLabs portal and create a test Service Namespace.

Then you will need to add Microsoft.ServiceBus and Microsoft.ServiceBus.Messaging dlls to your project. They can be found in C:\Program Files\Windows Azure AppFabric SDK\V2.0\Assemblies\NET4.0

Wrapping It Up

Before starting on the implementation I’m going to define a simple interface around the ServiceBus bits and also add an IMessage interface which all Messages will need to implement.

IServiceBus

    public interface IServiceBus
    {
        void Send(string queueName, IMessage message);
        
        T Receive<T>(string queueName) where T : IMessage; 
    }

IMessage

Although not required by the API it’s good practice to have a unique identifier for each message.
When architecting message based systems you need to allow for idempotence meaning that an operation should be able to be executed multiple times without changing the result. Most message queue systems guarantee that the messages will be delivered “at least once” so you need to allow for this in your code and having an unique identifier on the message makes implementing this a trivial exercise.

    public interface IMessage
    {
        Guid Id { get; set; }
    }

Note: All messages must be Serializable

Authentication

In order to authenticate against the service bus you need three values:

Issuer Name
Issuer Key
Service Namespace

The service namespace is just the name which you created in the portal, in this example it’s “appfabricdemo1”.

The issuer name & key can be found under the Default Key section in the Portal.

There is a little bit of ceremony in setting up the correct objects but I’ve put this into a single method called InitClient which is called from the Send and Receive methods.

       private readonly string issuerKey;
       private readonly string issuerName;
       private readonly string serviceNamespace;
       private TransportClientCredentialBase clientCredentials;
       private MessagingFactory messagingFactory;
       private ServiceBusNamespaceClient namespaceClient;
       private IList<Queue> queueList; 

       public AzureServiceBus(string issuerName, string issuerKey, string serviceNamespace)
       {
           this.issuerName = issuerName;
           this.issuerKey = issuerKey;
           this.serviceNamespace = serviceNamespace;
       }

       private void InitClient()
       {
           clientCredentials = TransportClientCredentialBase
               .CreateSharedSecretCredential(issuerName, issuerKey);

           var uri = ServiceBusEnvironment
               .CreateServiceUri("https", serviceNamespace, string.Empty);
           
           var runtimeUri = ServiceBusEnvironment
               .CreateServiceUri("sb", serviceNamespace, string.Empty);

           namespaceClient = new ServiceBusNamespaceClient(uri, clientCredentials);

           messagingFactory = MessagingFactory.Create(runtimeUri, clientCredentials);
       }

The key classes here are:

Creating the Queue

Before we can start sending and receiving messages you first have to create a queue. The API doesn’t currently have a method to check if a queue exists already so it has to be hand rolled as calling CreateQueue throws an exception if it already exists.

This is fairly easily achieved with two simple methods

       private Queue GetOrCreateQueue(string path)
       {
           path = path.ToLower();

           var queues = GetQueues();

           var queueExists = queues.Any(q => q.Path == path);

           if (queueExists)
           {
               return queues.FirstOrDefault(q => q.Path == path);
           }

           var queue = namespaceClient.CreateQueue(path);

           queues.Add(queue);

           return queue;
       }

       private IList<Queue> GetQueues()
       {
           if (queueList != null)
           {
               return queueList;
           }

           queueList = new List<Queue>();
           var queues = namespaceClient.GetQueues();

           foreach (var queue in queues)
           {
               queueList.Add(queue);
           }

           return queueList;
       }

As the GetQueues method needs to make a remote call I keep an in-memory collection of the Queue objects so that it only calls the first time.

Note: Queue names are converted to lowercase when created.

Sending a Message

In order to send a message you need to create a QueueClient, MessageSender and then convert the message to a BrokeredMessage.

        public void Send(string queueName, IMessage message)
        {
            InitClient();

            var queue = GetOrCreateQueue(queueName);

            var queueClient = messagingFactory.CreateQueueClient(queue);

            using (var messageSender = queueClient.CreateSender())
            {
                var brokeredMessage = ConvertToBrokeredMessage(message);

                messageSender.Send(brokeredMessage);
            }

            queueClient.Close();
            messagingFactory.Close();
        }

        private static BrokeredMessage ConvertToBrokeredMessage(IMessage message)
        {
            var brokeredMessage = BrokeredMessage.CreateMessage(message);

            brokeredMessage.MessageId = message.Id.ToString();

            return brokeredMessage;
        }

The BrokeredMessage has a bunch of useful properties, most notable of which are:

ContentType
CorrelationId
DeliveryCount
Properties – which is a Dictionary<string, object>
MessageId

Receiving a Message

Receiving a message is much the same as sending a message in that it requires a QueueClient & MessageReceiver.

There are two different modes when receiving a message.
ReceiveAndDelete which deletes the message immediately after reading or PeekLock which only locks the message and leaves you to manage the delete.

       public T Receive<T>(string queueName) where T : IMessage
       {
           InitClient();

           var queue = GetOrCreateQueue(queueName);

           var queueClient = messagingFactory.CreateQueueClient(queue);

           queueClient.CreateReceiver();

           BrokeredMessage brokeredMessage;

           using (var messageReceiver = queueClient.CreateReceiver(ReceiveMode.ReceiveAndDelete))
           {
               brokeredMessage = messageReceiver.Receive();
           }

           if (brokeredMessage == null)
           {
               return default(T);
           }

           queueClient.Close();
           messagingFactory.Close();

           var message = brokeredMessage.GetBody<T>();

           return message;
       }

The Demo App

I wanted to pull this together and so created an example application which is two console apps one being the client and one being the server. The server sends a message to the client on one queue and when the client receives the message it sends a reply message on another queue.

Server

    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Welcome to the Queue Demo Server");
            Console.WriteLine("Press any key to start:");
            Console.ReadLine();


            var serverQueue = "serverqueue";
            var clientQueue = "clientqueue"; 

            var serviceBus = new AzureServiceBus(ConfigurationManager.AppSettings["IssuerName"], 
                ConfigurationManager.AppSettings["IssuerKey"], 
                ConfigurationManager.AppSettings["ServiceNamespace"]);

            var message = new TestMessage("I've travelled along way just to get here.");

            
            serviceBus.Send(serverQueue, message);

            Console.WriteLine("Sent Message:" + message.Id + " at " + DateTime.Now);


            while (true)
            {
                message = serviceBus.Receive<TestMessage>(clientQueue);

                if (message == null || message.Id == Guid.Empty)
                {
                    Console.WriteLine("No messages found yet I'll keep trying");
                }
                else
                {
                    Console.WriteLine("Read Message: " + message.Id + " at " + DateTime.Now);
                    Console.WriteLine(message.Message);
                    break;
                }
            }


            Console.ReadLine();
        }
    }

Client

   internal class Program
   {
       private static void Main(string[] args)
       {
           Console.WriteLine("Welcome to the Queue Demo Client");
           Console.WriteLine("Press any key to start:");
           Console.ReadLine();
           
           var serverQueue = "serverQueue";
           var clientQueue = "clientQueue"; 

           var serviceBus = new AzureServiceBus(ConfigurationManager.AppSettings["IssuerName"], 
               ConfigurationManager.AppSettings["IssuerKey"], 
               ConfigurationManager.AppSettings["ServiceNamespace"]);


           while (true)
           {
               var message = serviceBus.Receive<TestMessage>(serverQueue);

               if (message == null || message.Id == Guid.Empty)
               {
                   Console.WriteLine("No messages found yet I'll keep trying"); 
               }
               else
               {
                   Console.WriteLine("Read Message: " + message.Id + " at " + DateTime.Now);
                   Console.WriteLine(message.Message);
                   break;
               }
           }


           var responseMessage = new TestMessage("And so have I");

           serviceBus.Send(clientQueue, responseMessage);

           Console.WriteLine("Sent Message:" + responseMessage.Id + " at " + DateTime.Now);



           Console.ReadLine();
       }
   }

Fiddler

If you’re curious you can see what’s going on under the covers using Fiddler.

API Change Requests

Having played around with the API it’s pretty good and much better than the MessageBuffer API which is really awkward.

I’d like to see on the ServiceBusNamespaceClient a method that tells you if a Queue exists or not, something like:

bool QueueExists(string path)

Currently MessagingFactory, ServiceBusNamespaceClient, QueueClient do not implement interfaces which makes it much harder to Unit Test. Creating an abstracting around these classes would be a big win IMO.

Conclusion

All in all the Queue API is a much needed and welcomed addition to the Azure offerings and greatly simplifies messaging communication between not always connected clients and the server. It also appears to be very fast as well.

Grab the code.

In my next post I’m going be looking at the Publish/Subscribe (Topics) features.

Getting started with SpecFlow, WatiN, ATDD and BDD

noreply@blogger.com (Will Beattie) — Tue, 21 Jun 2011 14:00:00 +0000

After completing my Scrum Master course I felt it was high time I review some of our existing engineering practices and consider ways to improve.

Whilst we practice strict test-first TDD and have 100% code coverage as part of our “definition of done” the one area that we’ve been missing is automated UI testing and acceptance testing. After all, high code coverage doesn’t mean a thing if the requirements are not met.

ATDD & BDD

This motivated me to rediscover what is known as Acceptance Test Driven Development or ATDD which is about defining automatable tests from a customer perspective that closely reflect the requirements described in the User Stories. This is also associated to Behaviour-Driven Development (BDD).

The cool kids in the Ruby world caught on to this way back when we were still doing web forms development.

Some of the Tools available in this space are:

RSpec (Ruby)
Cucumber (Ruby)
Robot Framework (Java & Python)
FITNESSE
WatiR Web UI Testing Framework for Ruby

Of all these tools & frameworks Cucumber is the most interesting in that it allows to describe behavioural tests in plain (non-technical) language allowing to the tests to be authored by the customers or customer proxies.

But What About .NET?

So I’ve mentioned a couple of the leading tools and frameworks in this area which are available in the Java and dynamic language world. In the .NET world there are a number of frameworks around which are either inspired by or ports of their counterparts.

NSpec (Inspired by RSpec)
SpecFlow (Inspired by Cucumber and conforms to Gherkin syntax)
WatiN (Inspired by WatiR)

NSpec and RSpec focus on BDD style Unit-Testing where as Cucumber and SpecFlow are more suited to acceptance tests.

For that reason I’m going to be looking at SpecFlow and WatiN for the remained of this post.

SpecFlow

For those of you without any knowledge of Cucumber or SpecFlow essentially how it works is that you specify Features which are files in your project.
Features firstly define the requirement from a customers perspective using non-techie vocabulary and then specify the scenarios or steps using the Given, When, Then words.

Here is a simple example of a how a feature can be written (I’m not an expert on this language yet).

Feature: Authentication
    In order to allow site personalization
    As a user
    I want to be able to login 

Scenario: Navigation to Home 
    When I navigate to /Home/Index 
    Then I should be on the home page

Scenario: Logging in
    Given I am on the home page
    And I have entered will into the username field
    And I have entered test into the password field 
    When I press submit
    Then the result should be a Logged In title on the screen

Relating this back to Agile you can think of a feature as a user story.

It should be noted that developers should not be writing these, they should be written by either the customer or a customer proxy such as a tester or business analyst. If these are written by non-technical people then you can ensure that they are focusing on the requirement and not the technical implementation.

Setting up the Test Project

After downloading and installing SpecFlow (make sure to close Visual Studio when doing so) you’ll need to create a standard Unit Test project. e.g. SpecFlowDemo.AcceptanceTests.

Add a reference to TechTalk.SpecFlow.dll which you can find in C:\Program Files\TechTalk\SpecFlow.

If you’re using MSTest then you’ll need to add the following to your App.Config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <specFlow>
    <unitTestProvider name="MsTest" />
  </specFlow>
</configuration>

I like to create a separate folder for the Features and Steps.

At this point your project should be looking something like.

Defining a Feature

As I mentioned earlier a feature should try and map to a single user story.

For this example I’m going to create a Search feature which searches google for BDD related articles.

Add New Item > SpecFlow Feature File

This will add a Feature file which comprises of the .feature part and the .cs code-behind. Make sure you don’t touch the .cs file as this contains designer generated code.

The Feature

For this fictitious example I’m going to use a Search feature. This is described below using the Gherkin language.

Feature: Search
    In order to find articles on BDD
    As a BDD fanatic 
    I want to enter a keyword into a search engine and be shown a list of related websites


Scenario: Navigate to Search Engine
    When I enter http://www.google.com in the address bar
    Then I should be on the home page


Scenario: Perform search
    Given I am on the home page
    And I have entered BDD into the keyword textbox
    When I press the btnG button
    Then I should see a list of articles related to BDD

Adding the Steps

Now we are going to add a Step definition for each of the scenarios described in the Feature file.

Right Click on Project > Add New Item > SpecFlow Step Defintion

Once you’ve added the Step you have to add the methods for each of the Given, When, Then lines in your scenario and annotate them with the correct attribute.

The key thing here is that the class needs to be annotated with the Binding attribute.

You can use the (.*) expression to represent variables coming from your Feature file.

NavigateToSearchEngine Step

[Binding]
 public class NavigateToSearchEngine
 {
     [When("I enter (.*) in the address bar")]
     public void when_i_enter_the_url(string url)
     {
     }

     [Then("I should be on the home page")]
     public void then_i_should_be_on_the_home_page()
     {
     }
 }

PerformSearch Step

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
           
        }

        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string button)
       {

       }
    }

WatiN

Now that we have our Feature and Steps defined in order to execute the UI tests we’re going to need WatiN. For more info on WatiN check out the documentation.

Download WatiN

Add references to Waitn.Core.dll & Interop.SHDocVw.dll which can be found in C:\Program Files\WatiN Test Recorder

Ensure that Interop.SHDocVw has Embed Interop Types set to False

Static Browser Class

This class is a static helper class for accessing the browser. WatiN has support for Internet Explorer & Firefox but I haven’t been able to get it to work with Firefox 4 I just tend to use the IE implementation.

#region

using TechTalk.SpecFlow;
using WatiN.Core;

#endregion

namespace SpecFlowDemo.AcceptanceTests
{
    [Binding]
    public class WebBrowser
    {
        public static Browser Current
        {
            get
            {
                if (!ScenarioContext.Current.ContainsKey("browser"))
                {
                    ScenarioContext.Current["browser"] = new IE();
                }
                return (Browser) ScenarioContext.Current["browser"];
            }
        }

        [AfterScenario]
        public static void Close()
        {
            if (ScenarioContext.Current.ContainsKey("browser"))
            {
                Current.Close();
            }
        }
    }
}

Implementing the Steps

NavigateToSearchEngine

    [Binding]
    public class NavigateToSearchEngine
    {
        [When("I enter (.*) in the address bar")]
        public void when_i_enter_the_url(string url)
        {
            WebBrowser.Current.GoTo(url);
        }

        [Then("I should be on the home page")]
        public void then_i_should_be_on_the_home_page()
        {
            Assert.AreEqual(WebBrowser.Current.Title, "Google");
            Assert.IsTrue(WebBrowser.Current.TextFields.Exists(tf => tf.Name == "q"));
        }
    }

PerformSearch

    [Binding]
    public class PerformSearch
    {
        [Given("I am on the home page")]
        public void given_i_am_on_the_home_page()
        {
            WebBrowser.Current.GoTo("http://www.google.com");
        }

       
        [Given("I have entered (.*) into the keyword textbox")]
        public void and_i_have_entered(string keyword)
        {
            var field = WebBrowser.Current.TextField(tf => tf.Name == "q");
            
            field.TypeText(keyword);

        }

       [When("I press the (.*) button")]
       public void when_i_press_the_button(string buttonName)
       {
           var button = WebBrowser.Current.Button(b => b.Name == buttonName);

           button.Click();
           
           WebBrowser.Current.WaitForComplete(); 

       }

       [Then("I should see a list of articles related to (.*)")]
       public void then_i_should_see_a_list_articles_related_to(string keyword)
       {
           WebBrowser.Current.WaitUntilContainsText("Advanced search");
           
           var searchDiv = WebBrowser.Current.Div("res");

           Assert.IsTrue(searchDiv.Children().Count == 3); 
           
       }
    }

Running the Tests

Now you’re ready to run the tests and with a bit of luck they should both pass. You will notice during the tests that Internet Explorer will open and you can actually watch the UI test happen.

Console Output

You get a nice descriptive console output as well which is nice.

When I enter http://www.google.com in the address bar
-> done: NavigateToSearchEngine.when_i_enter_the_url("http://www.google...") (15.5s)
Then I should be on the home page
-> done: NavigateToSearchEngine.then_i_should_be_on_the_home_page() (2.9s)

Given I am on the home page
-> done: PerformSearch.given_i_am_on_the_home_page() (6.4s)
And I have entered BDD into the keyword textbox
-> done: PerformSearch.and_i_have_entered("BDD") (3.2s)
When I press the btnG button
-> done: PerformSearch.when_i_press_the_button("btnG") (0.3s)
Then I should see a list of articles related to BDD
-> done: PerformSearch.then_i_should_see_a_list_articles_related_to("BDD") (1.4s)

The Code

You can download the code here.

Conclusion

If you’re not doing ATDD or Automated UI Testing then I strongly urge you to consider SpecFlow and WaitN which can add huge value to your testing process. I’m really looking forward to expanding our definition of done to include these.

There is also a WatiN Test Recorder which can assist in automating some of your tests or at the least show you the capability of WatiN. At last look though it doesn’t seem to be an active project anymore.

IE Security Warning with QuickTime Object tags

noreply@blogger.com (Will Beattie) — Tue, 14 Jun 2011 14:00:00 +0000

I just received a bug report about the infamous Internet Explorer Security Warning for one of the pages in our application that serves video content over HTTPS.

After viewing the source I found the offender which turned out to be the codebase attribute set to http://www.apple.com/qtactivex/qtplugin.cab.

<object id="videoObject" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B" 
            codebase="http://www.apple.com/qtactivex/qtplugin.cab" 
            width="330" height="292"> 
            <param name="src" value="https://securedomain/video.mp4" /> 
            <param name="controller" value="true" /> 
            <param name="autoplay" value="False" /> 
            <param name="scale" value="aspect" /> 
            <param name="cache" value="true"/>
            <param name="saveembedtags" value="true"/>
            <param name="postdomevents" value="true"/> 
               
            <!--[if IE] --> 
            <EMBED name="movie"
                height="292"
                width="330"
                scale="aspect"
                src="https://securedomain/video.mp4"
                type="video/quicktime"
                pluginspage="www.apple.com/quicktime/download"
                controller="true"
                autoplay="False"
            /> 
            <!--[endif]--> 
        </object>

The fix was just to change this to https. https://www.apple.com/qtactivex/qtplugin.cab.

<object id="videoObject" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B" 
            codebase="https://www.apple.com/qtactivex/qtplugin.cab" 
            width="330" height="292"> 
            <param name="src" value="https://securedomain/video.mp4" /> 
            <param name="controller" value="true" /> 
            <param name="autoplay" value="False" /> 
            <param name="scale" value="aspect" /> 
            <param name="cache" value="true"/>
            <param name="saveembedtags" value="true"/>
            <param name="postdomevents" value="true"/> 
               
            <!--[if IE] --> 
            <EMBED name="movie"
                height="292"
                width="330"
                scale="aspect"
                src="https://securedomain/video.mp4"
                type="video/quicktime"
                pluginspage="www.apple.com/quicktime/download"
                controller="true"
                autoplay="False"
            /> 
            <!--[endif]--> 
        </object>

Fortunately our users have the option of choosing the HTML5 Video player meaning they don’t need to install any 3rd party plugins to view videos.

Entity Framework 4 with Amazon RDS

noreply@blogger.com (Will Beattie) — Tue, 17 May 2011 14:00:00 +0000

In my last post I demonstrated how you can use MySQL with Entity Framework 4.

In this post I’m going to show you how to use Amazon RDS. Amazon RDS is a Relational Database Service which is similar to SQL Azure except that it supports MySQL & Oracle is coming soon. This is the actually the first time I’ve attempted to use the service and am going to be writing this as I go.

Amazon RDS takes care of all the critical database management tasks like software updates, backups & replication.

Signing Up

This post assumes you already have an AWS (Amazon Web Services) account, if you don’t go to the Sign In page.

As with all Amazon Web Services you have to explicitly sign up, you can do this by going to http://aws.amazon.com/rds/ and clicking the “Sign Up For Amazon RDS” button.

Note: Signing up is not instant, it took about 12 hours for me to receive the confirmation email.

Launching the DB Instance

Log into the AWS console and go to the Amazon RDS tab.

Step 1 – Launch DB Instance

Click on the big Launch DB Instance button.

Step 2 – DB Instance Details

After clicking the Launch DB button you will get the below screen.

For information on the DB Instance Class go to the AWS RDS page.

Multi-AZ Deployment is the ability to have the DB instance replicated in another availability zone with Amazon managing the failover, we’re not going to worry about that for now. This can be changed at anytime after the instance is created.

Note: See the maximum size of 1024 GB, compared with SQL Azures offering of 50GB.

Step 3 – Additional Configuration

On this screen you can specify the Database name, port & availability zone.

Step 4 – Management Options

On this screen you set your preferences for DB backups and maintenance. You’ll want to tweak this depending on where your customers are geographically located.

Step 5 – Review

Finally you can review all of your settings and when you’re satisfied click the Launch DB Instance and let Amazon work their magic.

Done

After a about 5 minutes or so the instance will be available in the DB Instances page.

Security

Now that we’ve created our instance we need to set some security rules in order to be able to access it from EC2 instances and any other external machines. I’m going to add a rule to allow access from machine so that I can do the data & schema import.

You can either choose from CIDR/IP or EC2 Security Group when setting your rule. When selecting the EC2 Security Group you write the name of the security group used in EC2.

For more information on security groups go to Working with DB Security Groups.

Importing the Data & Schema

The preferred method for importing schema and data into your DB Instance for datasets of 1GB and less is to extract the data with mysqldump and pipe it directly to your instance. If you dataset is larger than 1GB checkout the Amazon RDS Customer Data Import Guide for MySQL.

Connection

First you need to get the Endpoint for the instance from the AWS console. This can be found be going to the DB Instances page and clicking on your DB Instance.

Before starting you’ll want to check that you can connect to the instance properly using telnet:

telnet efdemo.cauwsk2cfjqz.ap-southeast-1.rds.amazonaws.com 3306

If you experience any problems here then check your security group rules and local firewall settings.

Import

Here is the command for the import:

"C:\Program Files\MySQL\MySQL Server 5.1\bin\"\mysqldump efdemo --host=efdemo.cauwsk2cfjqz.ap-southeast-1.rds.amazonaws.com --port 3306 --user=willbt --password

If everything went to plan you should get the “Dump completed” message in the command prompt and we’re ready to test the connectivity from EC2.

If you have issues, make sure the version of your DB Instance and local MySQL Installation are the same.

Of course you could also just connect to your instance using MySQL Administrator & MySQL Query Browser.

Accessing from EC2

Once you’ve got your instance setup in EC2 you can deploy your application to it.

If you’ve added the same security group that your EC2 instance uses to the DB Instance security groups then you should have no problems connecting to your DB Instance. Make sure the port you specified when creating the DB Instance is configured in any firewall rules.

To double check go onto your EC2 instance and run the telnet command again, if you have any issues check the instance firewall rules and the security group settings again.

You will need to also install the MySQL Connector onto your EC2 instance or make sure you set Copy To Local on the assemblies MySql.Data and MySql.Data.Entity.

Finally the connection string by setting the server attribute to the Endpoint of your DB Instance.

<connectionStrings>
    <add name="EFDemoEntities" connectionString="metadata=res://*/EFDemo.csdl|res://*/EFDemo.ssdl|res://*/EFDemo.msl;provider=MySql.Data.MySqlClient;provider connection string=&quot;server=efdemo.cauwsk2cfjqz.ap-southeast-1.rds.amazonaws.com;User Id=willbt;Password=password;database=efdemo;Persist Security Info=True&quot;" providerName="System.Data.EntityClient" />
  </connectionStrings>

Pricing

When you compare the pricing between a MySQL solution and the SQL Server solution it’s very easy to see benefits of going MySQL.

	Annual USD	Monthly USD
My SQL RDS (Large)	$ 3854.40	$ 321.20
SQL Server Standard (Large)	$ 9460.80	$ 788.40

That’s a pretty big difference & also bear in mind that with a SQL Server solution you’re not going to get automated backups, updates & failover.

Conclusion

Writing this blog was my first experience with Amazon RDS and I have to say I am very impressed with just how easy it is to get setup.

Amazon EC2 & RDS with MySQL for .NET applications is a very attractive deployment option and the next greenfield project I work on I’ll be seriously considering Amazon RDS with MySQL.

I’ve got a few pet projects kicking around with would be well suited for this platform so stay tuned.

Using Entity Framework 4 with MySQL

noreply@blogger.com (Will Beattie) — Tue, 10 May 2011 12:00:00 +0000

If you’re on the .NET Platform then MS SQL Server is usually the de-facto choice for the RDBMS. However if you’re at all cost conscience then you will realize that scaling and replication is going to cost you a fair chunk of change in licensing fees.

For that reason open source RDBMSs and in particular MySQL offer a much cheaper alternative.

In this post I’d like to demonstrate how you can use Entity Framework 4 with MySQL.

MySQL Connector Net 6.3.6

The first thing you’ll need to do is download and install the latest version of the MySQL Connector for .NET from http://dev.mysql.com/downloads/connector/net/

Make sure that Visual Studio is closed when you install.

Pascal Case Table Names

Because we are going to generate our Entity Framework Model of an existing database we want to make sure that the entity names use pascal casing. By default MySQL on Windows forces lowercase table names.

You can change this behaviour by adding lower_case_table_names=2 to your my.ini file which will located in
C:\Program Files\MySQL\MySQL Server <YOURVERSION>\

Read more about Identifier case sensitivity on the MySQL site.

NOTE: This will need to be done before you create your Schema and Tables.

Creating the Entity Framework Model

Right click on the solution and go to Add New Item then select the ADO.NET Entity Data Model.

Then choose “Generate from Database”.

You will then want to create a connection that points to your MySQL Database.

Go to New Connection. Note: by default the Data Source will be set to Microsoft SQL Server (SqlClient)

Click “Change” and select the Data source and Data Provider as shown below.

After doing this you enter your connection properties, shown below.

Once you press OK you will be presented with your database objects like below. Select all the Tables, Views & Stored Procedures you want to include in your model and press Finish

Follow the “Next” buttons through the wizard until your Model has been created.

Testing the Object Context

To make sure that we can use the ObjectContext correctly I wrote a quick test in a console app.

internal class Program
    {
        private static void Main(string[] args)
        {
            using (var context = new EFDemoEntities())
            {
                var category = new Category
                                   {
                                       Name = "Developer Tools"
                                   };

                var product = new Product
                                  {
                                      Name = "Visual Studio 2010"
                                  };

                category.Products.Add(product);

                context.Categories.AddObject(category); 

                context.SaveChanges();

                category = context.Categories.FirstOrDefault();

                Console.WriteLine("CategoryId: " + category.Id);

                product = context.Products.FirstOrDefault();

                Console.WriteLine("ProductId: " + product.Id);

            }
        }
    }

And the output.

If you receive “The specified value is not an instance of a valid constant type Parameter name: value” when trying to do an insert of a related entity then make sure that the foreign key column is not UNSIGNED. The MySQL connector does not support UNSIGNED columns as foreign keys,

Conclusion

I’ve only covered the basic steps to using MySQL with Entity Framework 4.

I have not used this yet in production, however it seems that MySQL and Entity Framework 4 is now a viable solution for those not married to SQL Server.

Till next time.

Downloading Amazon Kindle app for iPad2 in Singapore

noreply@blogger.com (Will Beattie) — Fri, 29 Apr 2011 01:01:00 +0000

Last week I got my brand spanking new iPad 2.

One of the major reasons I wanted this piece of technical indulgence is to use as a e-book reader on the daily commute.

So after plugging it in for the first time and going through the arduous ceremony of setting it up and doing the software update you can imagine my dismay when I discovered that the Kindle App was not available in the Singapore App Store.

Now I’m already used to the fact that there is no Music or Video content in the Singapore version of iTunes and get by quite happily without it. But disallowing the downloading of a free app is taking it a bit too far.

Fortunately for those in Singapore there is a workaround.

VPost

The first step is to create a VPost account.

For those who don’t know VPost is a service run by Singapore Post that effectively gives you a shipping address in either Japan, Europe or the United States. You can then get online shopping orders delivered to this address and they will forward the items to your Singapore based address.

It’s a great service which has many uses and I thoroughly recommended it to anyone.

Another iTunes Account

Once you’ve got your VPost address in the US you’ll need to create a new iTunes account using your US based address and not linked to your credit card as iTunes validates the billing address of the credit card against the address entered in the sign up form and prevents from signing up if the country is not the same.

The full list of steps are listed on the Apple Support site http://support.apple.com/kb/ht2534.

Downloading the Kindle App

Now that you have your US based iTunes account you can back to your iPad go to Settings > Store then sign-out of your existing account and sign-in using your new account.

Next up go to the App Store the search for “kindle” and low and behold it should appear and be able to be downloaded.

Because your US account has no payment information tied to it, you’ll need to remember to sign out and sign in using your Singapore based account whenever you want to buy any apps.

Amazon Account

Now that you have the Kindle app you’ll need to create an Amazon account in order to be able to buy books.

It is possible to use your Singapore address for this account, but bear in mind that there are quite a lot of books which are not available in the Asia-Pacific region. So to get around this you can use your US based VPost address and you will get access to all

To get started go to the sign in page and select “No, I am a new customer”.

Conclusion

Despite the hoops you have to jump through to both download the App and then get the content this workaround does make it all possible.

Hope this helps someone.

Specification Pattern, Entity Framework & LINQ

noreply@blogger.com (Will Beattie) — Tue, 22 Feb 2011 14:00:00 +0000

Firstly just to clarify I am going to be talking about the OOP Specification Pattern not the data pattern commonly found in the SID (Shared Information & Data) model.

Much has been said about the specification pattern so I’m not going to go into that, if you want an overview check out these posts:

http://www.lostechies.com/blogs/chrismissal/archive/2009/09/10/using-the-specification-pattern-for-querying.aspx
http://devlicio.us/blogs/jeff_perrin/archive/2006/12/13/the-specification-pattern.aspx

In this post I’m going to demonstrate how you can make use of the specification pattern to query Entity Framework and create reusable, testable query objects and eliminate inline LINQ queries.

The Smell

When I first got started with Entity Framework way back in 2008 when EF was still in it’s infancy we had lot’s of inline LINQ all over the code base and specific methods on our repositories for querying requirements (which any OOP purist will tell you is bad).

We had a service layer method which more or less looked something like:

     public IEnumerable<Product> FindAllActiveProducts(string keyword)
     {
         return productRepository.FindAllActive(keyword); 
     }

And then on our repository a method which looked like:

        public IEnumerable<Product> FindAllActive(string keyword)
        {
            var query = context.CreateObjectSet<Product>().Include("Category");


            var products = from p in query
                           where p.IsActive
                                 && p.Name.Contains(keyword)
                           select p;

            return products.ToList();

        }

Now whilst this is not all bad it does present a few code smells:

You end up with lot’s of methods on your repositories to handle different query scenarios
There’s no way to test the query in isolation
Magic strings for the Include path

I’m not going to touch on whether or not you should be using repositories for this type of query scenario because that’s a whole other topic and a quick Google search will yield many posts debating this very subject.

However I will say that if you find yourself with lot’s of methods on your repositories that only perform query operations then this is a big code smell.

The Specification Interface

The first step is to define an interface for our Specification.

   public interface ISpecification<T>
   {
       Expression<Func<T, bool>> Predicate { get; }

       IFetchStrategy<T> FetchStrategy { get; set; }

       bool IsSatisifedBy(T entity);
   }

If you’re familiar with this pattern you will notice the addition of two properties Predicate and FetchStrategy.

The Predicate is ultimately what will be using to perform the query. You will notice this is read-only which forces it to be defined within the specification implementation.

The FetchStrategy is an abstraction which defines the child objects that should be retrieved when loading the entity. More on this below.

Fetch Strategy

For those of you who don’t know when you load an entity from EF & other ORMs you can choose to either load just the root properties or load the related entities at the same time. The way to do this is in EF is by using the .Include method on the ObjectQuery.

This works fine, however fetch strategies are likely needed to be reused in different places so having the .Include with magic strings everywhere becomes a real maintenance headache.

In order to alleviate this pain I’ve created an abstraction on the concept.

   public interface IFetchStrategy<T>
   {
       IEnumerable<string> IncludePaths { get; }

       IFetchStrategy<T> Include(Expression<Func<T, object>> path);

       IFetchStrategy<T> Include(string path);
   }

Here is a generic implementation of this IFetchStrategy.

    public class GenericFetchStrategy<T> : IFetchStrategy<T>
    {
        private readonly IList<string> properties;

        public GenericFetchStrategy()
        {
            properties = new List<string>();
        }

        #region IFetchStrategy<T> Members

        public IEnumerable<string> IncludePaths
        {
            get { return properties; }
        }

        public IFetchStrategy<T> Include(Expression<Func<T, object>> path)
        {
            properties.Add(path.ToPropertyName());
            return this;
        }

        public IFetchStrategy<T> Include(string path)
        {
            properties.Add(path);
            return this;
        }

        #endregion
    }

    public static class Extensions
    {
        public static string ToPropertyName<T>(this Expression<Func<T, object>> selector)
        {
            var me = selector.Body as MemberExpression;
            if (me == null)
            {
                throw new ArgumentException("MemberException expected.");
            }

            var propertyName = me.ToString().Remove(0, 2);
            return propertyName;
        }
    }

This is all fairly self explanatory, all it does is maintain a list of the include paths and provides a fluent interface. All the ToPropertyName extension does is take a LINQ expression and returns the name of the property.

Do note however that there is still one Include method that takes a string as the parameter.
This is here to support really deep object hierarchies which can’t be represented as an expression.

You could easily create your own implementations for each scenario e.g. FullProductFetchStrategy and use that, however I tend to define the fetch strategy within the specification itself as you will soon see.

The Specification Implementation

First off we have a base class which contains the basic functionality and implements the ISpecification interface.

    public abstract class SpecificationBase<T> : ISpecification<T>
    {
        protected IFetchStrategy<T> fetchStrategy;
        protected Expression<Func<T, bool>> predicate;

        protected SpecificationBase()
        {
            fetchStrategy = new GenericFetchStrategy<T>();
        }

        public Expression<Func<T, bool>> Predicate
        {
            get { return predicate; }
        }

        public IFetchStrategy<T> FetchStrategy
        {
            get { return fetchStrategy; }
            set { fetchStrategy = value; }
        }

        public bool IsSatisifedBy(T entity)
        {
            return new[] {entity}.AsQueryable().Any(predicate); 
        }
    }

I have given the fetch strategy a getter & setter as it provides a bit of flexibility to the consumers but arguably you could make this read only and force instantiation in the constructor.

Now returning to the original example of finding active products with a similar name to the keyword provided here is the ActiveProductsByNameSpec.

   public class ActiveProductsByNameSpec : SpecificationBase<Product>
   {
       public ActiveProductsByNameSpec(string keyword)
       {
           predicate = p => p.Name.Contains(keyword) && p.IsActive;

           fetchStrategy = new GenericFetchStrategy<Product>().Include(p => p.Category);
       }
   }

As you can see everything is defined in the constructor.

Now you could expose a property for the keyword argument and have a single function which builds the predicate.

My preference however is to do everything in the constructor as it is immediately obvious what the requirements for this class are and by exposing properties you risk having required values not set leading to subtle bugs.

Testing the Specification

Testability for me is one the greatest benefits of using this pattern. With deep object graphs LINQ queries can soon grow in size and complexity. More code == More chance of bugs.

I can count 4 different test cases for this spec and here’s how we can test them.

        [TestMethod]
        public void When_Product_Not_Active_Predicate_Should_Find_No_Match()
        {
            var product = new Product {IsActive = false, Name = "Resharper"};
            
            var spec = new ActiveProductsByNameSpec("Resharper");

            var actual = spec.IsSatisifedBy(product); 

            Assert.IsFalse(actual);

        }

        [TestMethod]
        public void 
            When_Product_IsActive_But_Does_Not_Contain_Keyword_Predicate_Should_Find_No_Match()
        {
            var product = new Product { IsActive = true, Name = "Visual Studio" };

            var spec = new ActiveProductsByNameSpec("Resharper");

            var actual = spec.IsSatisifedBy(product);

            Assert.IsFalse(actual);

        }

        [TestMethod]
        public void 
            When_Product_Does_Not_Contain_Keyword_And_Is_Not_Active_Predicate_Should_Find_No_Match()
        {
            var product = new Product { IsActive = false, Name = "Visual Studio" };

            var spec = new ActiveProductsByNameSpec("Resharper");

            var actual = spec.IsSatisifedBy(product);

            Assert.IsFalse(actual);
        }

        [TestMethod]
        public void 
            When_Product_IsActive_And_Contains_Keyword_Predicate_Should_Find_Match()
        {
            var product = new Product { IsActive = true, Name = "Resharper" };

            var spec = new ActiveProductsByNameSpec("Resharper");

            var actual = spec.IsSatisifedBy(product);

            Assert.IsTrue(actual);
        }

The Generic Repository

Now that we have our spec we need to create a generic repository which takes the ISpecification interface and returns some Entities.

   public interface IGenericQueryRepository
   {
       T Load<T>(ISpecification<T> spec);

       IEnumerable<T> LoadAll<T>(ISpecification<T> spec);

       bool Matches<T>(ISpecification<T> spec);
   }

And this is implemented by:

    public class GenericQueryRepository : IGenericQueryRepository
    {
        private ObjectContext context;

        #region IGenericQueryRepository Members

        public T Load<T>(ISpecification<T> spec)
        {
            var query = GetQuery(spec.FetchStrategy);

            return query.FirstOrDefault(spec.Predicate);
        }

        public IEnumerable<T> LoadAll<T>(ISpecification<T> spec)
        {
            var query = GetQuery(spec.FetchStrategy);

            return query.ToList();
        }

        public bool Matches<T>(ISpecification<T> spec)
        {
            var query = GetQuery(spec.FetchStrategy);

            return query.Any(spec.Predicate);
        }

        #endregion

        private IQueryable<T> GetQuery<T>(IFetchStrategy<T> fetchStrategy)
        {
            ObjectQuery<T> query = context.CreateObjectSet<T>();

            if (fetchStrategy == null)
            {
                return query;
            }

            foreach (var path in fetchStrategy.IncludePaths)
            {
                query = query.Include(path);
            }

            return query;
        }
    }

Pulling it all together

Now that we have a generic repository we can safely get rid of the FindAllActive method on our ProductRepository and instead change our service layer to depend on the IGenericQueryRepository and instantiate the specification like so.

       public IEnumerable<Product> FindAllActiveProducts(string keyword)
       {
           var spec = new ActiveProductsByNameSpec(keyword);

           return queryRepository.LoadAll(spec); 
       }

Conclusion

Well that’s it. I hope I’ve demonstrated how you can reduce the number of inline LINQ queries and ease the testability of such queries. On my team it is now a rule that all LINQ to Entities queries are defined as Specifications.

The only element of this approach that is specific to Entity Framework is the fetch strategy approach I’ve used and I’m sure this could be easily adapted to fit with other ORMs that support LINQ.

As always I’m happy to hear any feedback and feel free to contact me should you need any clarification.

Managing Change in Long Running Workflows Part 2

noreply@blogger.com (Will Beattie) — Tue, 01 Feb 2011 14:00:00 +0000

Recently I wrote about how some of the Serialization problems you can face when dealing with long running workflows. In this post I’m going to cover how I deal with logic changes in persisted workflows and also how you can make logic changes to your workflows without having to deploy code.

Hosting the Workflow

When it comes to hosting your workflows there are really only two options to consider.

Self-Hosting
WCF Workflow Service

To be honest I only looked at the WCF Workflow Service model briefly and decided it wasn’t the best fit for the requirements as it couldn’t provide the same benefits of self-hosting within a Workflow Application.

Workflow Instance Store

In order to persist Workflow instances you need to first create the Workflow Instance Store database.

For this you will need to use the following scripts:

C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreSchema.sql
C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en\SqlWorkflowInstanceStoreLogic.sql

The Workflow

The scenario I’m going to use for this post is that of an order approval workflow. So after an order is created it goes into a “Pending” state, a workflow is then started requiring two users to approve before it can be changed to an “Approved” state.

This is represented in the below diagram.

Bookmarks

If you already know how to implement long running workflows using Bookmarks then you can skip down to the next section.

The way to have points in your workflows that trigger persistence and can wait for external input are called Bookmarks.

In above diagram the “Wait for Approval” activity is a Bookmark and looks like the following:

 [Serializable]
 public sealed class WaitForApproval : NativeActivity<bool>
 {
     public InArgument<int> UserId { get; set; }

     protected override bool CanInduceIdle
     {
         get { return true; }
     }

     protected override void Execute(NativeActivityContext context)
     {
         var bookmarkName = "waitingFor_" + UserId.Get(context);

         context.CreateBookmark(bookmarkName, OnReadComplete);
     }

     private void OnReadComplete(NativeActivityContext context, Bookmark bookmark, object state)
     {
         var input = Convert.ToBoolean(state, new CultureInfo("EN-us"));

         context.SetValue(Result, input);
     }
 }

Take note of the bookmark name as this will be used later when resuming the workflow.

TIP: The bookmark is stored in the BlockingBookmarks field on your InstancesTable in your workflow instance database.

The Problem

The problem you can face after you’ve persisted your workflow instance is when the workflow definition changes and you try to load the instance using WorkflowApplication.Load.

A number of changes detailed below can lead to invalidating the workflow instance and will cause a System.Activities.ValidationException to be thrown.

The somewhat “official" word on this found here is that “WF4 is not able to handle runtime changes to the workflow definition”. This was elaborated a bit more and here are the “official” lists of breaking and non-breaking changes.

Non-Breaking Changes:

Changes to the activities code
Changes which don’t affect the number or types of arguments/variables
Addition or removal of member fields or properties

Breaking Changes:

Renaming or removing methods that had been used as bookmark, fault or completion callbacks
Adding new arguments
Adding new variables
Adding new children

Basically the rules of serialization/deserialization versioning apply also to workflows, which when you think about it makes sense.

Don’t fret though as this can be avoided without too much effort, so keep reading to find out how.

Incorporating Workflow into the Architecture

Very early on in my foray with Workflow Foundation I realised the power and flexibility it could to add to the application, specifically the ability to use different workflows for different users/accounts and changing workflows at run-time.

Because of this it was decided that the workflows should be treated as a first class citizen in our domain model and not just left as an abstract technical implementation of the business process.

Below is a simple class diagram which should give you a good idea of the approach taken.

As you can see in the Order class there is the WorkflowInstanceId and WorkflowXaml.
The WorkflowInstanceId is there because it allows to identify the Workflow by using the OrderId meaning that you don’t have to expose the InstanceId in other parts of your application.

The WorkflowXaml is the raw definition of the workflow that gets copied from the WorkflowDefinition at instantiation time. The purpose of this is to allow the WorkflowDefinition Xaml to be changed without affecting the persisted workflows and solves the problem described above.

Arguably you could add a separate Class/Table which stores the active Xaml and InstanceId for the Order, but for simplicities sake I’ve just added them to the Order.

It should also be said the WorkflowInstanceId and WorkflowXaml can be cleared from the Order table once the workflow has completed.

This will all become clearer as you read on.

Generic Workflow Host

For the hosting I have come up with a GenericWorkflowHost which allows to start instances and resume from bookmarks. There’s not much that is very exciting in here although do take note of the following methods:

CreateActivityFrom
StartPersistableInsance
LoadInstanceWithBookmark

The key part is the XamlServices.Load method which allows to create an Activity using just the Xaml. This is important because this allows you to change the Xaml without re-compiling and deploying code.

using System;
using System.Activities;
using System.Activities.DurableInstancing;
using System.Activities.XamlIntegration;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Reflection;
using System.Runtime.DurableInstancing;
using System.Xaml;

public static class GenericWorkflowHost
{
    private static ConcurrentDictionary<Guid, WorkflowApplication> runningWorkflows;

    #region Private Helper Methods 

    private static Activity CreateActivityFrom(string xaml)
    {
        var sr = new StringReader(xaml);

        //Change LocalAssembly to where the Activities reside
        var xamlSettings = new XamlXmlReaderSettings
                               {LocalAssembly = Assembly.GetExecutingAssembly()};

        var xamlReader = ActivityXamlServices
            .CreateReader(new XamlXmlReader(sr, xamlSettings));

        var result = XamlServices.Load(xamlReader);

        var activity = result as Activity;

        return activity;
    }

    private static InstanceStore CreateInstanceStore()
    {
        var conn = ConfigurationManager.ConnectionStrings["WorkflowDbConn"].ConnectionString;

        var store = new SqlWorkflowInstanceStore(conn)
                        {
                            InstanceLockedExceptionAction = InstanceLockedExceptionAction.AggressiveRetry,
                            InstanceCompletionAction = InstanceCompletionAction.DeleteNothing,
                            HostLockRenewalPeriod = TimeSpan.FromSeconds(20),
                            RunnableInstancesDetectionPeriod = TimeSpan.FromSeconds(3)
                        };

        var handle = store.CreateInstanceHandle();

        var view = store.Execute(handle, new CreateWorkflowOwnerCommand(),
                                 TimeSpan.FromSeconds(60));

        store.DefaultInstanceOwner = view.InstanceOwner;

        handle.Free();

        return store;
    }

   
    #endregion

    public static void InvokeInstance(object input, string xaml, Guid instanceId)
    {
        var inputs = new Dictionary<string, object>();

        if (input != null)
        {
            inputs.Add(input.GetType().Name, input);
        }

        var wf = CreateActivityFrom(xaml);

        var activity = wf;

        WorkflowInvoker.Invoke(activity, inputs);
    }

    public static Guid StartPersistableInstance(IDictionary<string, object> inputs, string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }


        var activity = CreateActivityFrom(xaml);

        var workflowApp = new WorkflowApplication(activity, inputs)
                              {
                                  InstanceStore = CreateInstanceStore(),
                                  PersistableIdle = OnIdleAndPersistable,
                                  Completed = OnWorkflowCompleted,
                                  Aborted = OnWorkflowAborted,
                                  Unloaded = OnWorkflowUnloaded,
                                  OnUnhandledException = OnWorkflowException
                              };

        workflowApp.Persist();

        var instanceId = workflowApp.Id;

        workflowApp.Run();

        runningWorkflows.TryAdd(instanceId, workflowApp);

        return workflowApp.Id;
    }

    public static bool LoadInstanceWithBookmark(string bookmarkName,
                                                Guid instanceId,
                                                object input,
                                                string xaml)
    {
        if (runningWorkflows == null)
        {
            runningWorkflows = new ConcurrentDictionary<Guid, WorkflowApplication>();
        }

        BookmarkResumptionResult result;

        if (runningWorkflows.ContainsKey(instanceId))
        {
            var workflow = runningWorkflows[instanceId];
            workflow.Completed = OnWorkflowCompleted;
            workflow.PersistableIdle = OnIdleAndPersistable;

            result = workflow.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }
        else
        {
            // Setup the persistance
            var store = CreateInstanceStore();

            var activity = CreateActivityFrom(xaml);

            var application = new WorkflowApplication(activity)
                                  {
                                      InstanceStore = store,
                                      Completed = OnWorkflowCompleted,
                                      Unloaded = OnWorkflowUnloaded,
                                      PersistableIdle = OnIdleAndPersistable,
                                  };

            application.Load(instanceId, TimeSpan.FromSeconds(60));

            result = application.ResumeBookmark(bookmarkName, input, TimeSpan.FromSeconds(60));

            runningWorkflows.TryAdd(instanceId, application);

            Console.WriteLine(instanceId + " resumed @ " + bookmarkName);
        }

        return result == BookmarkResumptionResult.Success;
    }

    public static void UnloadInstance(Guid instanceId)
    {
        if (!runningWorkflows.ContainsKey(instanceId))
        {
            return;
        }

        var workflow = runningWorkflows[instanceId];
        workflow.Unload();

        runningWorkflows.TryRemove(instanceId, out workflow);
    }

    #region Events 

    public static void OnWorkflowCompleted(WorkflowApplicationCompletedEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.CompletionState);
    }

    public static PersistableIdleAction OnIdleAndPersistable(WorkflowApplicationIdleEventArgs e)
    {
        return PersistableIdleAction.Unload;
    }

    public static void OnWorkflowAborted(WorkflowApplicationAbortedEventArgs e)
    {
        Console.WriteLine(e.Reason);
    }

    public static void OnWorkflowUnloaded(WorkflowApplicationEventArgs e)
    {
        if (runningWorkflows != null && runningWorkflows.ContainsKey(e.InstanceId))
        {
            WorkflowApplication workflowApp;
            runningWorkflows.TryRemove(e.InstanceId, out workflowApp);
        }

        Console.WriteLine(e.InstanceId + " unloaded");
    }

    public static UnhandledExceptionAction OnWorkflowException(WorkflowApplicationUnhandledExceptionEventArgs e)
    {
        //log the exception here using e.UnhandledException 
        return UnhandledExceptionAction.Terminate;
    }

    #endregion
}

As you may have noticed the GenericWorkflowHost is a static class so I choose to abstract this away behind an interface so that it’s nice and Unit-Test friendly:

   public interface IOrderApprovalWorkflowHost
   {
       void Resume(Guid instanceId, string xaml, int userId, bool isApproved);

       Guid Start(int orderId, string xaml);
   }

Which is implemented like so:

    public class OrderWorkflowApprovalHost : IOrderApprovalWorkflowHost
    {
        public void Resume(Guid instanceId, string xaml, int userId, bool isApproved)
        {
            var bookmark = string.Format("waitingFor_{0}", userId);

            GenericWorkflowHost.LoadInstanceWithBookmark(bookmark, instanceId, isApproved, xaml); 
        }

        public Guid Start(int orderId, string xaml)
        {
            var inputs = new Dictionary<string, object>()
                             {
                                 {"OrderId", orderId}
                             };  
            return GenericWorkflowHost.StartPersistableInstance(inputs, xaml);
        }

    }

Starting the Workflow Instance

Now that we have our workflow host it’s time to start the workflow. I like to abstract any workflow calls behind a service interface that the other layers can consume without the hard-dependency on the workflow libraries.

    public interface IOrderApprovalService
    {
        void SubmitApproval(int orderId, int userId, bool isApproved);

        void StartApprovalWorkflow(int orderId); 
    }

The StartApprovalWorkflow method is implemented like so:

        public void StartApprovalWorkflow(int orderId)
        {
            using (var unitOfWork = unitOfWorkFactory.Create())
            {
                var order = orderRepository.Get(orderId);

                var workflow = workflowRepository.Get(1); // this id should come from configuration

                var instanceId = orderApprovalWorkflowHost.Start(orderId, workflow.Xaml);

                order.StartWorkflow(workflow, instanceId);// this sets the workflow xaml & instanceid


                unitOfWork.Commit(); 

            }
        }

All we do here is load the Order & workflow from their respective repositories and then start the workflow and assign the workflow to the order. All pretty simple stuff.

Resuming the Workflow Instance

Resuming the instance is as easy as loading the Order and then calling the OrderApprovalWorkflowHost passing the persisted Xaml & InstanceId that was saved when starting the workflow.

        public void SubmitApproval(int orderId, int userId, bool isApproved)
        {
            var order = orderRepository.Get(orderId);

            orderApprovalWorkflowHost
                .Resume(order.WorkflowInstanceId, order.WorkflowXaml, userId, isApproved);
        }

Conclusion

The approach I’ve outlined in this post has some major benefits:

Ability to change the workflow definition without code deployment
Ability to change workflow definition without invalidating persisted instances
Provides a simple way to view active instances

When you want to change the logic flow in a persisted instance the simplest approach is either to force the workflow to end and then restart it or to just rollback any state changes the workflow has done and simply restart a new instance then delete the old instance. This needs to be carefully thought out though especially if any of the activities send emails.

Well that’s all I’ve got for this post, I’m really keen to hear any feedback and as always I’d be happy to answer any questions.

Managing Change in Long Running Workflows Part 1

noreply@blogger.com (Will Beattie) — Thu, 25 Nov 2010 13:00:00 +0000

I’ve been using Workflow Foundation now for over a year and it has become an integral part of our architecture and has by and large been very successful. However it is not without it’s issues, the single most being that official documentation is light at best and relevant blog posts are quite rare.

I recently ran into a fairly serious problem after deploying a new release of our Software.

The Problem

I received this lovely email from our exception tracker:

System.Runtime.DurableInstancing.InstancePersistenceCommandException: The execution of the InstancePersistenceCommand named {urn:schemas-microsoft-com:System.Activities.Persistence/command}LoadWorkflow was interrupted by an error. ---> System.Runtime.Serialization.SerializationException: Deserialized object with reference id '73' not found in stream.

The key part is “Deserialized object with reference id '73' not found in stream.” As I’m sure you can tell this does not really provide any helpful information.

The stack trace does give you some more information as to which property failed, but depending on the object graph the stack trace can be fairly massive.

This is the part which gives you the biggest clue:

at ReadArrayOfPersonFromXml(XmlReaderDelegator , XmlObjectSerializerReadContext , XmlDictionaryString[] , XmlDictionaryString[] )

The Cause

Basically what has happened here is that when trying to load an unloaded workflow instance from the SqlWorkflowInstanceStore the deserializer spits the dummy because a property on the object that existed at the time the Workflow was persisted no longer exists on the class.

Once you understand the cause it does seem reasonable, sort of. In my effort to clean up the solution and get rid of unused properties I had effectively broken every workflow instance which was persisted before the date of the latest release.

As it turns out it’s possible to create this issue by either removing properties or renaming existing properties.

I made the fatal mistake of using passing my Domain Entities into the workflow as an argument.

The Patch

In order to fix this issue I ended up scrambling through the check-in history over the last 6 weeks trying to figure which properties had been removed. Once I figured it out, I added them back and deployed a patch. This solved the problem and the crisis was averted for the short-term.

The Solution

When a workflow is unloaded and persisted to the data store it is serialized and all internal variables & arguments are serialized along with it.

Rules for data objects in workflows:

Never ever under any circumstances use Domain Entities
Always use DTO’s for Arguments or Variables in Long Running Workflows
Set IsRequired=false on the DataMember attribute for all Nullable Properties
Set the Name property on the DataMember attribute
Set the Order Property on the DataMember attribute
Set the Name on the DataContract attribute

This is an example of a class that could create problems in the future.

    [DataContract]
    public class Category
    {
        [DataMember]
        public int Id { get; set; }

        [DataMember]
        public List<Product> Products { get; set; }
    }

Based on the above lessons here is how I would suggest coding this class for use in a Workflow.

   [DataContract(Name="Category")]
   public class Category
   {
       [DataMember(IsRequired = false, Name = "Id", Order=0)]
       public int Id { get; set; }

       [DataMember(IsRequired = false, Name = "Products")]
       public List<Product> Products { get; set; }
   }

By setting the Name property on the DataMember you are then free to change the property name without fear of breaking the deserialization.

Setting IsRequired to False allows you to remove the property in the future without breaking deserialization, obviously you should set this for True for any properties that are truly required by the Workflow to function correctly.

Conclusion

This is just an example of how to deal with data that changes in workflows, in my next post I’m going to cover how you deal with workflows whose logic changes after an instance has been created and persisted.

I hope this helps anyone who’s working with Workflow Foundation or about to start using it.

Till next time.

MediaInfo deprecated attributes in Version 0.7.37

noreply@blogger.com (Will Beattie) — Wed, 24 Nov 2010 13:00:00 +0000

I’ve just discovered that the latest version (0.7.37) of MediaInfo has some deprecated attributes and I’ve compiled the full list of deprecated attributes split by the Track type.

General

Menu_Codec_List
Format/String
Codec
Codec/String
Codec/Info
Codec/Url
Codec/Extensions
Codec_Settings
Codec_Settings_Automatic

Video

Codec
Codec/String
Codec/Family
Codec/Info
Codec/Url
Codec/CC
Codec_Profile
Codec_Description
Codec_Settings
Codec_Settings_PacketBitStream
Codec_Settings_BVOP
Codec_Settings_QPel
Codec_Settings_GMC
Codec_Settings_GMC/String
Codec_Settings_Matrix
Codec_Settings_Matrix_Data
Codec_Settings_CABAC
Codec_Settings_RefFrames
Resolution
Resolution/String
Colorimetry
Interlacement
Interlacement/String

Audio

Codec
Codec/String
Codec/Family
Codec/Info
Codec/Url
Codec/CC
Codec_Description
Codec_Profile
Codec_Settings
Codec_Settings_Automatic
Codec_Settings_Floor
Codec_Settings_Firm
Codec_Settings_Endianness
Codec_Settings_Sign
Codec_Settings_Law
Codec_Settings_ITU
Resolution
Resolution/String
Video0_Delay
Video0_Delay/String
Video0_Delay/String1
Video0_Delay/String2
Video0_Delay/String3
Video0_Delay/String4

Image

Codec
Codec/String
Codec/Family
Codec/Info
Codec/Url
Resolution
Resolution/String

Text

Codec
Codec/String
Codec/Info
Codec/Url
Codec/CC
Resolution
Resolution/String
Video0_Delay
Video0_Delay/String
Video0_Delay/String1
Video0_Delay/String2
Video0_Delay/String3
Video0_Delay/String4

Chapters

Codec
Codec/String
Codec/Info
Codec/Url

If you want the full list of all available attributes, then download the DLL version from the MediaInfo download page and go to the Developers/List_Of_Parameters folder.

Consuming the SSRS ReportExecutionService from a .NET Client

noreply@blogger.com (Will Beattie) — Tue, 23 Nov 2010 13:00:00 +0000

I’ve just finished writing a nice wrapper which internally calls the SSRS ReportExecutionService to generate reports.
Whilst it was fairly simple to implement there has been some major changes between 2005 and 2008 and the majority of online and documentation is based on the 2005 implementation.

The most important change is that the Report Server and Report Manager are no longer hosted in IIS which will be a welcomed change to Sys Admins but makes the security model and hosting model vastly different. So far I’ve yet to figure out how to allow Anonymous Access, if anyone knows how to do this leave a comment and it will be most appreciated.

Getting Started

To get started you’ll want to add a service reference to http://localhost/ReportServer_SQL2008/ReportExecution2005.asmx where ReportServer_SQL2008 is the name you configure in the Reporting Services Configuration Manager.

The Web Application files are located in C:\Program Files\Microsoft SQL Server\MSRS10.SQL2008\Reporting Services, you’ll want to to create a dedicated User and grant it permissions to these application folders.

Once you’ve done that you’ll need to browse to the Report Manager logging in as an Administrator account and then add a Role for your new user under the Security page http://localhost/Reports_SQL2008/Pages/Settings.aspx?SelectedSubTabId=SecurityLinkID and also under the Properties tab in Home http://localhost/Reports_SQL2008/Pages/Folder.aspx?SelectedTabId=PropertiesTab.

Calling the API

This post is mainly about solving some of the problems you may face, for detailed information on calling the API check out MSDN which has a wealth of information on the topic, here is the Overview and more specifically how to Render a Report.

However note that you may have to pass a TrustedUserHeader type whenever calling a Web Service method, it’s ok for this value to be Null.

WCF Security

Once you’ve followed the guide and coded your solution appropriately you may come across this exception or one similar when trying to call a Web Service method.

The authentication header received from the server was 'Negotiate,NTLM'.

The key here is to set the Network Credential property before to a Windows user who has access to the Reports and has the correct Role Assignments in the Report Manager.

var rsClient = new ReportExecutionServiceSoapClient(); 

            rsClient.ClientCredentials.Windows
                .ClientCredential = new NetworkCredential("username", "password");
            rsClient.ClientCredentials.Windows
                .AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;

Also make sure that in the Web.Config in C:\Program Files\Microsoft SQL Server\MSRS10.SQL2008\Reporting\Services\ReportServer has the following values set.

These are the bindings I use and work well. Make sure to set all the properties to the Maximum allowed values as Report Server returns some fairly large arrays.

<system.serviceModel>
    <bindings>
      <basicHttpBinding>
        <binding name="ReportExecutionServiceSoap" closeTimeout="00:01:00"
            openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
            allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
            maxBufferSize="65536" maxBufferPoolSize="65536" maxReceivedMessageSize="65536"
            messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
            useDefaultWebProxy="true">
          <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
              maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
          <security mode="TransportCredentialOnly">
            <transport clientCredentialType="Ntlm" proxyCredentialType="None" realm="" />
            <message clientCredentialType="UserName" algorithmSuite="Default" />
          </security>
        </binding>
      </basicHttpBinding>
    </bindings>
    <client>
      <endpoint address="http://localhost/ReportServer_SQL2008/ReportExecution2005.asmx"
          binding="basicHttpBinding" bindingConfiguration="ReportExecutionServiceSoap"
          contract="Services.Impl.ReportExecution2005.ReportExecutionServiceSoap"
          name="ReportExecutionServiceSoap" />
    </client>
  </system.serviceModel>

Conclusion

I’m all about using the right tool for the job so for Reporting when you’re using SQL Server 2008 as your data store you really can’t go past SSRS which even comes free with SQL Server Express Edition.

Also included in SSRS 2008 is the Report Builder which is a Report Designer tool accessible from the Report Manager allowing users to design reports without having to install the Business Intelligence Studio.

Hope this helps someone.

SharpKit C# to Javascript Convertor

noreply@blogger.com (Will Beattie) — Tue, 05 Oct 2010 13:00:00 +0000

Recently I discovered SharpKit whilst looking at an online IDE called CodeRun.

SharpKit is described as “a Web Toolkit that enables you to write C# and convert it to JavaScript during compilation.”

There are a number of other similar tools around that try and achieve the goal of writing in a Static language and then converting it to JavaScript.

Some of those include:

GWT (Google Web Toolkit)
Script #
jsc

I can’t speak for jsc or Script # but I’ve had a good look at GWT and although I can see the benefits it reminded me a bit of the ASP.NET WebForms model in that the high level of abstraction has a big learning curve.

The SharpKit feature list is very impressive.

This is the first time I have tried SharpKit so I’m going to do a very simple walkthrough and how to get started.

Installation

Grab the Download from the SharpKit website and run the Installer.

As SharpKit is a commercial product they require you to Register before Downloading, but don’t be put off by this.

After installing you will get the following folders.

C:\Program Files\SharpKit\Addin
C:\Program Files\SharpKit\Assemblies
C:\Program Files\SharpKit\Samples
C:\Program Files\SharpKit\Templates
C:\Program Files\SharpKit\Utils

The Samples folder contains really good examples of each project type so is the best way to get started.

Project Structure

In my ignorance I assumed that you could just reference the DLLs and away you go, however this was an incorrect assumption. You actually have to use one of the SharpKit Visual Studio project templates.

I really don’t like this and would expect that there should be a Command-Line interface that you can call in the Post-Build event to do the conversion. Correct me if I’m wrong but I couldn’t find any such Utility.

Defining the Class

When you create a class that you want to be exported to JavaScript you have to annotate it with the JsType attribute. This defines how you want the Class to be converted at Build-Time.

There are 4 distinct modes you can use

CLR – Adds .NET Support to the Client including LINQ
Global – Exports all function as global static functions
JSON – Not exported but treats as a JSON object when instantiated
Prototype – Exported as a function with an instance constructor

If you want to use jQuery then you need to inherit from jQueryContext.

The other 3rd party libraries following the same pattern and inheriting a base class.

Below is a simple example:

    [JsType(JsMode.Global, "common.js")]
    public class common : jQueryContext
    {
        public static void btn_click()
        {
            jQuery("#div1").show(1000, onShowEnded); 
        }
        static void onShowEnded()
        {
            jQuery("#div1").css("color", "red"); 
        }
    }

As you can see that you have to specify the physical name of the file in the JSType Attribute and the file will be generated relative to where the CSharp file is located.

The generated JavaScript for the above class looks like this:

//@AutoGenerated
function btn_click()
{
    $("#div1").show(1000, onShowEnded);
}
function onShowEnded()
{
    $("#div1").css("color", "red");
}

The code that’s generated is of a high standard. There’s not much else to show you on this as it’s fairly self-explanatory, check out the How-Tos for more detailed information.

3rd Party Library Support

Whilst you can use SharpKit to write raw JavaScript there’s not a lot of point these days with so many great libraries around.

SharpKit comes with very comprehensive support for Libraries and currently supports the follwing:

jQuery
jQTouch
jQuery UI
YUI
ExtJS
GoogleMaps
ASP.NET AJAX

It’s yet to be seen how they manage to keep up with library changes.

Pricing

At the time of writing they have a Freemium pricing model.

SharpKit Lite – Free version with a limit of 1000 lines of code
SharpKit Pro - $649 / Developer

Having a free version is good but if you’re considering a Tool of this sort at all then it’s likely you’re doing some heavy JavaScript development so will very quickly go over 1000 LOC.

At $649 I think it is pretty good value, however it would likely be too much for lone developers or Open Source projects.
What would be nice is an Open Source Project License which is heavily discounted or better yet free. This could be combined with an affiliate badge linking back to SharpKit that you need to display on your website, similar to Kentico.

Pros

All in all I was pretty impressed with this Tool and it shows great promise. Here’s what I think are good about it:

Comprehensive 3rd Party Library support
.NET CLR Support
Good Documentation & Support
Generates Clean JavaScript Code

Cons

However there were a few things I do not like about it.

Forced to use the SharpKit Visual Studio Project Template
No Visual Studio 2010 Support
Pricing
Heavy use of Attributes
No Command-Line tool to perform the conversion

Conclusion

As a Software Architect I’ve never been able to decide on the best way for JavaScript to be organized in Web Projects and usually leave it up to the Lead Web Developer who’s going to be working with it on a daily basis.

However this inevitably leads to lot’s of scattered JavaScript throughout the project and it quickly becomes a maintenance headache.

So to me the idea of writing C# and have it converted to JavaScript at build-time is very attractive indeed from both a maintenance and testability point of view. However I’m thinking that in practice it’s only really going to help .NET Developers with basic JavaScript knowledge as more advanced JavaScript developers would feel highly constrained by losing the Dynamic goodness of the language.

For those reasons I’m not entirely sold on SharpKit and similar tools just yet, however it’s definitely worth investigating further and I’m probably going to use this on the next piece of non-trivial JavaScript that needs writing.
Stay tuned for more on that.

I’m keen to know your thoughts on this and similar tools, so feel free to comment.

Till next time.

Invoking System.Actions with Retries using ReliableActionInvoker

noreply@blogger.com (Will Beattie) — Tue, 28 Sep 2010 13:00:00 +0000

Although some would argue that Exceptions should be thrown and handled one level up. I tend to think there are plenty of cases where by after an Exception is thrown it makes total sense to Retry the operation N number of times before finally Re-throwing the Exception up the Call stack.

One of these such cases is the simple act of Downloading a file from a Remote Server. Although the code is very simple, WebClient.DownloadFile can and does fail frequently for a host of different reasons.

Consider the following example which simply tries to download a remote Uri to a local file path.

        private int maxRetries = 5;
        private int retriesCounter;

        public void Download(string uri, string targetFilePath)
        {
            try
            {
                var webClient = new WebClient();

                webClient.DownloadFile(uri, targetFilePath);

                retriesCounter = 0; 
            }
            catch (WebException webEx)
            {
                if (retriesCounter >= maxRetries)
                {
                    throw;
                }

                retriesCounter++;

                Download(uri, targetFilePath);
            }

        }

As you can see with a little bit of Recursion we retry the Operation a maximum of 5 times and then finally Throw the exception.

This type of code is messy to read, very repetitive and also common to errors which can lead to the dreaded StackOverflowException.

To get around this I’ve come up with what I call a ReliableActionInvoker which essentially provides a generic way to Invoke System.Actions and retry them if any Exception or an explicit Exception occurs.

Interface

   public interface IReliableActionInvoker
   {
       /// <summary>
       /// Tries to Invoke the supplied action and if any Exception is thrown will
       /// retry the number of times supplied as maxRetries
       /// </summary>
       /// <param name="action"></param>
       /// <param name="maxRetries"></param>
       void Invoke(System.Action action, int maxRetries);

       /// <summary>
       /// Tries to Invoke the supplied Action and if the Exception is of the 
       /// supplied ExceptionType will retry the number of times supplied as 
       /// maxRetries
       /// </summary>
       /// <param name="action"></param>
       /// <param name="maxRetries"></param>
       /// <param name="exceptionType"></param>
       void Invoke(System.Action action, int maxRetries, Type exceptionType);

       /// <summary>
       /// Tries to Invoke the supplied Action and if the Action throws
       /// an Exception that is contained in the supplied exceptionTypes array
       ///  will retry the number of times supplied as maxRetries
       /// </summary>
       /// <param name="action"></param>
       /// <param name="maxRetries"></param>
       /// <param name="exceptionTypes"></param>
       void Invoke(System.Action action, int maxRetries, Type[] exceptionTypes);
   }

If you’re not familiar with System.Action then I suggest you check it out on MSDN.

Implementation

    public class ReliableActionInvoker : IReliableActionInvoker
    {
        private int retriesCounter;
        public void Invoke(Action action, int maxRetries)
        {
            var exceptionTypes = new Type[0];

            Invoke(action, maxRetries, exceptionTypes);
        }

        public void Invoke(Action action, int maxRetries, Type exceptionType)
        {
            Invoke(action, maxRetries, new[] {exceptionType});
        }

        public void Invoke(Action action, int maxRetries, Type[] exceptionTypes)
        {
            try
            {
                action.Invoke();
                retriesCounter = 0; 
            }
            catch (Exception ex)
            {
                if (retriesCounter >= maxRetries)
                {
                    retriesCounter = 0; 
                    throw; 
                }
                if (exceptionTypes == null || exceptionTypes.Length == 0)
                {
                    retriesCounter++;
                    Invoke(action, maxRetries, exceptionTypes);
                    return;
                }

                if (exceptionTypes.Contains(ex.GetType()))
                {
                    retriesCounter++;
                    Invoke(action, maxRetries, exceptionTypes);
                    return;
                }

                throw;
            }
        }

  }

This code is pretty self explanatory. I first call Action.Invoke and catch the Exception. When no Exception Type is supplied then Retry, otherwise check if the caught Exception exists in the supplied Collection and Retry.

When the maxRetries has been reached Throw the Exception.

Usage

    public class WebClientDownloader : IUriDownloader
    {
        private readonly IReliableActionInvoker reliableActionInvoker;

        public WebClientDownloader(IReliableActionInvoker reliableActionInvoker)
        {
            this.reliableActionInvoker = reliableActionInvoker;
        }

        public void Download(string uri, string targetFilePath)
        {
            uri.ValidateArgumentNull();
            targetFilePath.ValidateArgumentNull(); 

            var webClient = new WebClient();

            reliableActionInvoker
                .Invoke(() => webClient.DownloadFile(uri, targetFilePath), 5, typeof(WebException));
        }

    }

Finally here’s how you could use it.

That’s all there is to it, it’s pretty simple really. I’m keen here to what you think or how you’ve approached this problem in a generic way before so feel free to comment.

The nice thing about this is it opens up the possibility to implement this using AOP but I’ll leave that to another post.

Till next time.

OpenRasta vs WCF for RESTful Services

noreply@blogger.com (Will Beattie) — Tue, 21 Sep 2010 13:00:00 +0000

Not long ago I discovered OpenRasta. If you’re not familiar with it it’s described as a “resource-oriented framework for .NET enabling easy ReST-ful development of websites and services”.

After reading Rails 3 in a Nutshell I started looking at .NET a bit differently and have started being envious of the simplicity of Rails. When it comes to REST the ROR guys have definitely approached it the right way and it is baked right into the Rails framework.

When it comes to .NET there are a few different approaches you can take to create RESTful services.

ASP.NET MVC
WCF
OpenRasta
Other MVC Frameworks
Standard HTTP Handlers

In this post I’m going to cover how you can implement such a service using WCF and OpenRasta.

The goal is to expose a Product as JSON and XML using a RESTful uri.

product/1

WCF Approach

First let’s have a look at how to do this in WCF

Project Structure

Just create a standard WCF Service Application Project

DataContract

The first thing we have to do is create a DataContract which is just a Data Transfer Object that you want to be exposed by the web service.

    [DataContract]
    public class Product
    {
        [DataMember]
        public int Id { get; set; }

        [DataMember]
        public string Name { get; set; }
    }

You have to annotate the class with the DataContract attribute and properties with the DataMember attribute.

Service Contract

Now we have to define our Service Contract.

    [ServiceContract]
    public interface ICatalogService
    {
        [OperationContract]
        [WebGet(UriTemplate = "product/{id}", ResponseFormat = WebMessageFormat.Json)]
        Product GetProductById(string id);

    }

The important part here is the WebGet attribute. The UriTemplate specifies the route which you can access the Resource on and the ResponseFormat specifies the format or Content-Type to return the resource as, you can choose from Json or XML.

Having to limit to a single Content-Type is already a serious limitation and the UriTemplate being an attribute property makes it hard to change this route easily.

Also notice that the Id argument is a string, this is deliberate as the UriTemplate only supports having path segments of type string. If you try otherwise you will see an exception similar to below.

Operation 'GetProductById' in contract 'ICatalogService' has a path variable named 'id' which does not have type 'string'. Variables for UriTemplate path segments must have type 'string'.

Service Implementation

Next we go to the Service implementation, this is pretty standard.

    [AspNetCompatibilityRequirements(RequirementsMode= AspNetCompatibilityRequirementsMode.Allowed)]
    public class CatalogService : ICatalogService
    {
       
        public Product GetProductById(int id)
        {
            var product = new Product { Name = "WCF", Id = 1 };

            return product; 
        }

     }

The only difference here from a standard service implementation is the use of the AspNetCompatibilityRequirements attribute on the class.

Web.Config

The only additional config required is the standardEndpoints configuration. The full config is below.

 <system.serviceModel>
    <standardEndpoints>
      <webHttpEndpoint>
        <standardEndpoint defaultOutgoingResponseFormat="Json" name="" helpEnabled="true" automaticFormatSelectionEnabled="true" />
      </webHttpEndpoint>
    </standardEndpoints>
    <behaviors>
      <serviceBehaviors>
        <behavior>
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
  </system.serviceModel>

Consuming the Service

For the service to return JSON you have to specify in the Content-Type as "application/json” in the HTTP Header, for XML use application/xml.

You can consume using URI http://myrestsite.com/CatalogService.svc/product/1

Observations

Ok well that wasn’t so bad, however I’m sure you’ve already identified a few limitations.

Inflexible routing options, only one Uri per method
Only able to return XML or JSON
Heavy use of Attributes

OpenRasta Approach

OpenRasta deal with Resources, Handlers and Views, but we aren’t concerned with Views in this post.

Handlers
”Handlers are the objects responsible to act upon resources through the HTTP uniform interface and will always get called first. They're the equivalent of Controllers in other MVC frameworks.”

Resources
”Your resources are what you expose on various Uris. In OpenRasta, each thing you want to expose on the web needs to have a Resource class associated with it.”

Project Structure

To host OpenRasta you need to create a standard Web Application project but your Resources and Handlers can be contained in a standard Class Library project.

After installing OpenRasta you’ll need to add references to the following DLLs:

OpenRasta.dll
OpenRasta.Codecs.WebForms.dll
OpenRasta.Hosting.AspNet.dll

The project structure is very simple, you can split it into Handlers and Resources as shown below.

Resource

The Resource or DTO is just a standard POCO.

   public class Product
   {
       public int Id { get; set; }

       public string Name { get; set; }
   }

Notice there’s no base class to inherit or any attributes. This is important as it makes it very easy to utilise Classes that you already have defined.

Handler

Handlers are equivalent to MVC Controllers or compared with WCF it’s the Service implementation.

    public class ProductHandler
    {
        public object Get(int id)
        {
            var product = new Product {Name = "OpenRasta", Id = 1};

            return product;
        }
    }

Again no base class or attributes required.

OpenRasta Config

One of the great things about OpenRasta is that the Configuration of how the Resources are accessed is completely decoupled from the implementation. The configuration has a nice fluent interface for configuring Handlers and Resources.

You can create a class with any name and implement the OpenRasta.Configuration.IConfigurationSource interface

   public class OpenRastaConfig : IConfigurationSource
   {
       #region IConfigurationSource Members

       public void Configure()
       {
           using (OpenRastaConfiguration.Manual)
           {
               ResourceSpace.Has.ResourcesOfType<Product>()
                   .AtUri("/product/{id}")
                   .HandledBy<ProductHandler>()
                   .AsJsonDataContract()
                   .And
                   .AsXmlDataContract();
           }
       }

       #endregion
   }

.AtUri specifies the Uri you can use to access the Resource. It’s also possible to have multiple Uris for the same resource.

                    .AtUri("/product/{id}")
                    .And
                    .AtUri("/products/{id}")

.HandledBy<ProductHandler> specifies the Handler to use.

.AsJsonDataContract() tells OpenRasta you want to expose the resource as Json

.AsXmlDataContract() tells OpenRasta you want to expose the resource as Xml

Web.Config

There is a little bit of Web.Config configuration to be done as well, this is all you need:

<configuration>
  <system.web>
    <compilation debug="true" targetFramework="4.0" />
    <httpHandlers>
      <add verb="*" path="*.rastahook" type="OpenRasta.Hosting.AspNet.OpenRastaHandler, OpenRasta.Hosting.AspNet"/>
    </httpHandlers>
    <httpModules>
      <add name="OpenRastaModule" type="OpenRasta.Hosting.AspNet.OpenRastaModule, OpenRasta.Hosting.AspNet"/>
    </httpModules>
  </system.web>
  <system.webServer>
    <validation validateIntegratedModeConfiguration="false"/>
    <modules>
      <add name="OpenRastaModule" type="OpenRasta.Hosting.AspNet.OpenRastaModule, OpenRasta.Hosting.AspNet"/>
    </modules>
    <handlers>
      <add name="OpenRastaHandler" verb="*" path="*.rastahook" type="OpenRasta.Hosting.AspNet.OpenRastaHandler, OpenRasta.Hosting.AspNet" />
    </handlers>
  </system.webServer>
</configuration>

Consuming the Service

As with WCF you need to specify the correct Content-Type to get the format you desire.

You can consume using a URI like http://myrestsite.com/product/1 or http://myrestsite.com/products/1

Observations

All in all OpenRasta is a nicer development experience and is more closely tied to the definition of REST. It is fully extensible and allows to return a Resource in any Format, including an HTML View.

Conclusion

I’ve only touched on a very basic use of OpenRasta, it has tons more functionality which makes developing RESTful services a breeze and also stacks up as a very good MVC framework if you’re having issues with the ASP.NET implementation. It has pretty good documentation and a strong community around it.

Check out the Comparisons table for a round up of features.

IMO doing REST in WCF just doesn’t seem very intuitive to me and is much less flexible especially when you dive deep into using HTTP Status Codes and HTTP Verbs.

If you have a requirement to develop a RESTful service instead of defaulting to WCF I urge you consider OpenRasta instead.

Till next time.

Defensive Programming & Argument Validation

noreply@blogger.com (Will Beattie) — Tue, 14 Sep 2010 13:06:00 +0000

The other day I found myself debugging the dreaded NullReferenceException “Object Reference not set to an instance of an object” on a piece of code written by a fairly Senior Developer.

If you’ve been developing on .NET for more than 5 minutes then I’m sure you have come across this in your daily work and it is incredibly frustrating to try and debug.

With this post I hope to demonstrate how by using Defensive Programming you can make your API’s more robust and as a result reduce the number of bugs in your code.

If you’re a bit hazy on Defensive Programming and it’s cousins then I suggest you read the following:

Defensive Programming
Fail Fast
Design by Contract (DbC)

Validating method arguments for public methods should be a very simple concept to grasp and for most skilled developers is a as natural as writing If statements.

The guiding principle I follow is to trust no single input and ensure that code only executes when the supplied arguments satisfy the requirements (pre-conditions). If it does not it should Fail Fast and throw an exception.

Take One

Here’s an example of what I consider to be a poorly written method.

        public OperationResult ChangeProductName(int productId, string name)
        {
            using (var uow = uowFactory.Create())
            {
                var product = productRepository.Get(productId);

                product.ChangeName(name); 

                uow.Commit(); 
            }

            return new OperationResult {WasSuccessful = true}; 
        }

Now what’s going to happen if the productId supplied is less than zero?

Well either the ProductRepository is going to throw an ArgumentOutOfRangeException or it will return a null Product and the call to Product.ChangeName is going to throw a NullReferenceException.

Either way an exception will be thrown after we have created a new Unit Of Work, which in practice could be an expensive operation.

Take Two

So here is an example of how you could write the same method in a better way by first checking the arguments supplied satisfy the needs of the method.

        public OperationResult ChangeProductName(int productId, string name)
        {
            if (productId < 1)
            {
                throw new ArgumentOutOfRangeException();
            }

            if (string.IsNullOrEmpty(name))
            {
                throw new ArgumentNullException();
            }

            using (var uow = uowFactory.Create())
            {
                var product = productRepository.Get(productId);

                if (product == null)
                {
                    throw new ProductNotFoundException(); 
                }

                product.ChangeName(name);

                uow.Commit();
            }

            return new OperationResult { WasSuccessful = true }; 
        }

Take Three

Now the previous approach is bounds better than the first but we can reduce the lines of code by making use of some Extension Methods like so:

        public OperationResult ChangeProductName(int productId, string name)
        {
            productId.ValidateArgumentRange(1);
            name.ValidateArgumentNull(); 

            using (var uow = uowFactory.Create())
            {
                var product = productRepository.Get(productId);

                if (product == null)
                {
                    throw new ProductNotFoundException(); 
                }

                product.ChangeName(name);

                uow.Commit();
            }

            return new OperationResult { WasSuccessful = true }; 
        }

Extension Methods

Here are the extension methods that I use to take care of this.

   public static class ArgumentExtensions
   {
       public static void ValidateArgumentNull(this object value)
       {
           if (value == null)
           {
               throw new ArgumentNullException();
           }
       }

       public static void ValidateArgumentRange(this int value, int min)
       {
           if (value < min)
           {
               throw new ArgumentOutOfRangeException();
           }
       }

       public static void ValidateArgumentRange(this int value, int min, int max)
       {
           if (value < min || value > max)
           {
               throw new ArgumentOutOfRangeException();
           }
       }
   }

You maybe thinking to yourself that this is really simple stuff and I’d agree, but I felt it needed to be shared for those Devs who are still learning.

There is a new project called Code Contracts which provides a way to explicitly define pre-conditions and post-conditions, this also allows to make use of Automatic testing tools such as Pex.

IMO a few years from now Code Contracts will be as fluent to a developer as Boolean operators and we will end up with better quality API’s because of it, however right now it is only a Research project so I’m hesitant to depend too highly on it until it’s finalised and becomes part of the core .NET Framework.

There are also ways to approach this problem using Aspect-Oriented Programming but I will leave that to another post.

Till next time.

Load Balancing with Amazon EC2 and Elastic Load Balancing

noreply@blogger.com (Will Beattie) — Tue, 07 Sep 2010 13:00:00 +0000

Hosting your applications in the cloud yields many benefits, however it can be at times a very scary place when thing’s go wrong, and they always do.

If you love your customers then you owe it to them to make sure you have in place redundancy and ensure you provide a high level of availability.

The Amazon EC2 Infrastructure provides a really easy way to set this up using the Elastic Load Balancing feature.

Before starting you need the basic concepts of how the EC2 infrastructure works and how it can help you in your quest for high availability.

Regions

Regions in EC2 are geographical locations where the Data Centres reside and at the time of writing there are four Regions:

US East – North Virginia
US West – California
EU – Ireland
APAC – Singapore

It goes without saying that the Region you select should be always be as close to your customers as possible.

Availability Zones

Each Region has 2 or more Availability Zones. These are important to understand as they are independent from each other in that a failure of one zone does not affect the other.

Network Diagram

This is a fairly typical approach to load balancing a web application but it’s important to note that the level of redundancy is achieved by putting different instances in different Availability Zones.

The key thing to note here is that the instances you load balance should be in separate Availability Zones.

This post will be covering the Load Balancing aspect as I will leave the database redundancy to another post.

Creating The Load Balancer

Creating the Load Balancer is easy using the AWS Console

Under Networking & Security > Load Balancers you can “Create Load Balancer”.

This should be fairly self explanatory.

First we are assigning a name to our Load Balancer and then we are setting up the Ports the Load Balancer should listen on and forward to.

If you want to use HTTPS then you will need to do Port Forwarding as show in the highlighted image.

Configure Health Check

Next you need to setup your Health Check, this is how the Load Balancer decides if your Instance can have requests forwarded to it.

You can do this by setting the Ping Path which is an HTTP resource that is polled at an interval you define.

If it returns an HTTP result code other than 200 it will cause your instance to be deemed unhealthy and subsequently removed from the Load Balancer.

The Ping Path can be a static HTML resource, however it’s a good idea to run other any system checks and so I like to use a .ASPX file with Code-Behind instead.

Adding Instances

Here you can select Instances from all Availability Zones in your Region, as I mentioned before you should select Instances in different Availability Zones to provide a higher level of availability. Although you can have a single instance, it goes without saying you should have at least two.

DNS

When a load balancer is created it automatically gets assigned a DNS Name. To point your website address to your load balancer all you have to do is create a CNAME record for it.

Managing The Load Balancer

Now that you have setup your load balancer you can add more instances easily and get visibility of the Health of each instances by selecting the Load Balancer in the AWS Console.

Healthy

Unhealthy

And this is what it looks like if you have an unhealthy instance.

Key Issues

Although EC2 alleviates lot of pain out of setting up a load balanced solution there are a few issues you need to be aware of before diving headfirst into this solution.

Connection Timeout Limit

At present the load balancer does not hold connections open for more than 60 seconds by design. So if you have any requests which take longer than a minute you are going run into problems. If you do find yourself in this position you should really be looking at how to reduce the Response times by using one-way Messaging or reducing the operation.

Static IP Address Support

Unfortunately there is currently no support for a Static IP address for your load balancer. So if you are integrating with third parties who have strict firewall policies then you may have problems. I’m hoping though that Amazon add this feature in the future as it is needed in a lot of scenarios.

SSL Support

To enable HTTPS you have to use port forwarding at the load balancer level, which can be achieved by listening on 443 and then forwarding to another port for instance 8443.

In IIS you then need to change the port in the Site Bindings, and don’t forget to open 8443 on your Firewall.

Sticky Sessions

The Elastic Load Balancer does support Sticky Sessions via Cookies or QueryString however it is only works for Port 80 traffic.
So if you’re using InProcess Session State you will need to move to the SQL based provider.

Cache

If you’re using System.Web.Caching.Cache to cache objects then you’re also going to run into issues because of the Sticky Session problem on HTTPS. If Caching is a critical factor in your applications performance then you will need to consider a Distributed Cache solution like Memcache or Shared Cache.

Well that’s about all there is to it. I hope this helps someone.

Till next time.

Implementing Custom Castle Windsor Facilities

noreply@blogger.com (Will Beattie) — Thu, 02 Sep 2010 13:00:00 +0000

If you’ve been following my posts you would know that I love Castle Windsor.

One of the many useful features I have found is the Facility and I’m going to try and give a good example how you can make use of this.

In a recent post I showed how you can add Cross-Cutting concerns to your application by using Interceptors.

Now when configuring the Container you can explicitly configure each Interceptor per Service but when you have lot’s of components it can get pretty hard to maintain after a while and can also introduce subtle issues if someone forgets to configure it correctly.

Below is how you would configure your Container without using a Facility. On the last line we are specifying the Interceptor explicitly.

        public void Configure()
        {
            container = new WindsorContainer();

            container.Register(
                Component.For<CacheInterceptor>(),
                Component.For<ICacheProvider>()
                    .ImplementedBy<WebCacheProvider>().LifeStyle.Singleton,
                Component.For<ICatalogQueryService>()
                    .ImplementedBy<CatalogQueryService>()
                    .LifeStyle.Transient
                    .Interceptors(new InterceptorReference(typeof (CacheInterceptor))).Anywhere);
        }

Or the XML version

<?xml version="1.0" encoding="utf-8" ?>
<castle>
  <components>
    <component id="CacheInterceptor"
             service="MyApp.Aop.CacheInterceptor, MyApp"
             type="MyApp.Aop.CacheInterceptor, MyApp"
             lifestyle="transient">
    </component>
    <component id="ICatalogQueryService"
             service="MyApp.Services.ICatalogQueryService, MyApp"
             type="MyApp.Services.Impl.CatalogQueryService, MyApp"
             lifestyle="transient">
      <interceptors>
        <interceptor>${CacheInterceptor}</interceptor>
      </interceptors>
    </component>
    <component id="ICacheProvider"
             service="MyApp.Core.Cache.ICacheProvider, MyApp"
             type="MyApp.Core.Cache.Impl.WebCacheProvider, MyApp"
             lifestyle="transient">
    </component>
  </components>
</castle>

What would be nice would be to be able to specify the Interceptor on all Components automagically and Castle allows for this with Facilities. Below is an implementation which achieves the desired result.

   public class CacheInterceptionFacility : AbstractFacility
   {
       protected override void Init()
       {
           Kernel.ComponentRegistered += Kernel_ComponentRegistered;
       }

       private void Kernel_ComponentRegistered(string key, IHandler handler)
       {
           if (typeof(IMustBeCached).IsAssignableFrom(handler.ComponentModel.Implementation))
           {
               handler.ComponentModel.Interceptors
                   .Add(new InterceptorReference(typeof(CacheInterceptor)));
           }
       }
   }

All you have to do is Inherit from the Castle.MicroKernel.Facilities.AbstractFacility and then override the Init.

You’ll see that we use the Interface IMustBeCached to identify the Component to apply the Interceptor too.

Now in your Bootstrapper all you have to do is call AddFacility on the WindsorContainer and you’re good to go.

        public void Configure()
        {
            container = new WindsorContainer();

            container.Register(
                Component.For<CacheInterceptor>(),
                Component.For<ICacheProvider>()
                    .ImplementedBy<WebCacheProvider>().LifeStyle.Singleton,
                Component.For<ICatalogQueryService>()
                    .ImplementedBy<CatalogQueryService>().LifeStyle.Transient);

            container.AddFacility("CacheInterceptionFacility", new CacheInterceptionFacility()); 

        }

Or the XML version

<?xml version="1.0" encoding="utf-8" ?>
<castle>
  <facilities>
    <facility
      id="CacheInterceptionFacility"
      type="MyApp.Aop.CacheInterceptionFacility, MyApp">
    </facility>
  </facilities>
  <components>
    <component id="CacheInterceptor"
             service="MyApp.Aop.CacheInterceptor, MyApp"
             type="MyApp.Aop.CacheInterceptor, MyApp"
             lifestyle="transient">
    </component>
    <component id="ICatalogQueryService"
             service="MyApp.Services.ICatalogQueryService, MyApp"
             type="MyApp.Services.Impl.CatalogQueryService, MyApp"
             lifestyle="transient">
    </component>
    <component id="ICacheProvider"
             service="MyApp.Core.Cache.ICacheProvider, MyApp"
             type="MyApp.Core.Cache.Impl.WebCacheProvider, MyApp"
             lifestyle="transient">
    </component>
  </components>
</castle>

Aside from reducing the configuration overhead another benefit of using this approach is that in your Debug or Test Environments you can easily deploy without any Interceptors and only enable them in your Staging and Production environments.

Caching as Cross-Cutting Concern Part 2 – Invalidating the Cache

noreply@blogger.com (Will Beattie) — Tue, 31 Aug 2010 13:00:00 +0000

In my previous post I showed how you can very easily add Caching to your application by using Castle Windsor.

The example used was a very basic implementation and whilst it can be useful for a large number of cases it didn’t cover the all important cache invalidation.

In this post I hope to explain how with a bit of convention over configuration how you can invalidate your cache when an object changes.

In the last post I was using the Method name combined with the arguments to build the cache key. Now to allow us to build a cache key that can be used when invalidating the cache the key needs to be related to the Result type.

For this I’m going to create a simple interface called IAmCacheable to identify the object we want to cache.

    public interface IAmCacheable
    {
        object Key { get; }
    }

And make sure the object we want to cache implements this interface.

    public class Product : IAmCacheable 
    {
        public object Key { get; }

        public string Name { get; set; }
    }

Now we have to change the CacheInterceptor to build the Cache Key based on the combination of the Type and the Id.

   public class CacheInterceptor : IInterceptor
   {
       private readonly ICacheProvider cacheProvider;

       private const int CacheExpiryMinutes = 1; 

       public CacheInterceptor(ICacheProvider cacheProvider)
       {
           this.cacheProvider = cacheProvider; 
       }

       #region IInterceptor Members

       public void Intercept(IInvocation invocation)
       {
           //check if the method has a return value
           if (invocation.Method.ReturnType == typeof(void))
           {
               invocation.Proceed();
               return; 
           }

           //check to see if the return type implements IAmCacheable
           var returnTypeIsCacheable = typeof (IAmCacheable)
               .IsAssignableFrom(invocation.Method.ReturnType); 

           if (!returnTypeIsCacheable)
           {
               invocation.Proceed();
               return;
           }

           var cacheKey = BuildCacheKeyFrom(invocation); 
          
           //try get the return value from the cache provider
           var item = cacheProvider.Get(cacheKey); 

           if (item != null)
           {
               invocation.ReturnValue = item;
               return;
           }

           //call the interecepted method
           invocation.Proceed();

           if (invocation.ReturnValue != null)
           {
               cacheProvider.Put(cacheKey, CacheExpiryMinutes, invocation.ReturnValue); 
           }

           return;
       }

       #endregion

       private static string BuildCacheKeyFrom(IInvocation invocation)
       {
           var cacheableTypeName = invocation.Method.ReturnType.Name;
           var cacheKey = "{0}-{1}";

           var keyArgName = "id";

           var prms = invocation.Method.GetParameters();

           var keyIndex = prms.TakeWhile(prm => prm.Name != keyArgName).Count();

           var keyValue = invocation.Arguments[keyIndex]; 

           return string.Format(cacheKey, cacheableTypeName, keyValue); 

       }

   }

The main change here is in the BuildCacheKeyFrom method. What I’m doing here is looking for the value of the Argument named “id” and using this combined with the Type name to build the key which would end up like “Product-1”.

Now that we’ve changed the logic to build the Cache Key we need to be remove the object from the Cache when it changes.

Similar to the IMustBeCached interface in the first article we create an interface called IInvalidateCache to identity which call to apply the invalidation interceptor on.

   public interface IInvalidateCache
   {
   }

And we’ll make the IProductRepository implement this like so.

    public interface IProductRepository : IInvalidateCache
    {
        Product Get(int id);

        void Update(Product product);
    }

Now we need a new type of Interceptor called CacheInvalidationInterceptor which removes any IAmCacheable objects from the cache when they are passed as arguments to any method.

    public class CacheInvalidationInterceptor : IInterceptor
    {
        private readonly ICacheProvider cacheProvider;

        public CacheInvalidationInterceptor(ICacheProvider cacheProvider)
        {
            this.cacheProvider = cacheProvider;
        }

        #region IInterceptor Members

        public void Intercept(IInvocation invocation)
        {
            //check to see if any argument implements IAmCacheable
            var hasCacheableArgument = invocation.Arguments.Any(a => a is IAmCacheable);

            if (!hasCacheableArgument)
            {
                invocation.Proceed();
                return;
            }

            //call the intercepted method
            invocation.Proceed();

            var cacheKey = BuildCacheKeyFrom(invocation);

            //remove the item from the cache
            cacheProvider.Remove(cacheKey); 

            return;
        }

        #endregion

        private static string BuildCacheKeyFrom(IInvocation invocation)
        {
            var cacheable = invocation.Arguments.First(a => a is IAmCacheable) as IAmCacheable; 

            var cacheableTypeName = cacheable.GetType().Name;

            var cacheKey = "{0}-{1}";

            return string.Format(cacheKey, cacheableTypeName, cacheable.Key);

        }

    }

All we do here is look for any Arguments which implement IAmCacheable and then use a combination of the Type name and the IAmCacheable.Key to build the cache key.

We also make sure the method is called first by calling Invocation.Proceed before removing the item from the Cache.

Finally we need to wire up this new Interceptor in the Bootstrapper.

        public void Configure()
        {
            container = new WindsorContainer();

            container.Register(
                Component.For<CacheInterceptor>(),
                Component.For<ICacheProvider>()
                    .ImplementedBy<WebCacheProvider>().LifeStyle.Singleton,
                Component.For<ICatalogQueryService>()
                    .ImplementedBy<CatalogQueryService>()
                    .LifeStyle.Transient
                    .Interceptors(new InterceptorReference(typeof(CacheInterceptor))).Anywhere);


            container.Register(
                Component.For<CacheInvalidationInterceptor>(),
                Component.For<IProductRepository>()
                    .ImplementedBy<ProductRepository>()
                    .LifeStyle.Transient
                    .Interceptors(new InterceptorReference(
                        typeof(CacheInvalidationInterceptor)))
                        .Anywhere);


        }

This is my final post on the subject. Hope this gives you some ideas on how to implement Caching or other Cross-Cutting concerns using AOP.

Till next time.

Caching as a Cross-Cutting Concern using Castle Windsor

noreply@blogger.com (Will Beattie) — Tue, 24 Aug 2010 13:00:00 +0000

This post assumes knowledge of Dependency Injection and AOP.

AOP or Aspect-Oriented Programming is a very powerful way to add Cross-Cutting concerns to a system without impacting the core code base. Cross-Cutting concerns can cover non-functional requirements and also functional requirements.

There are generally three approaches when implementing AOP.

1. Generate Dynamic Proxies at runtime to intercept method calls

2. Post-Compilation Assembly Transformation

Post Sharp

3. Attributes

Just using Plain Old Attributes

I really really don’t like annotating classes & members with Attributes so for me Dynamic Proxies is the only option and the performance hit is worth the productivity gains.

The very common example of AOP is Logging. Logging is by all accounts a Cross-Cutting concern and so lends itself very well to the example.

However I tend to think of Caching as a Cross-Cutting concern and really hate to see code like the below.

       public Product GetProductById(int id)
       {
           var cacheKey = string.Format("Product-", id);

           var cache = HttpRuntime.Cache;

           var product = cache.Get(cacheKey) as Product; 

           if (product != null)
           {
               return product; 
           }

           product = productRepository.Get(id);

           cache.Add(cacheKey, product, null, 
               System.Web.Caching.Cache.NoAbsoluteExpiration, 
               System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.High, null);

           return product; 

       }

IMO the above code block only adds noise to your code base and makes it harder to read. It’s also prone to logic errors if it’s being implemented in many places and so is a prime candidate for AOP.

In this example I will be using Castle Windsor just because I am most familiar with it, but any good Dependency Injection framework should support AOP.

In Castle Windsor there is a concept of an Interceptor which as the name implies intercepts any method calls on a class. Behind the scenes this is implemented using DyanmicProxy.

Let’s first have a look at a fairly contrived example of a Service that we want to add Caching too.

    public interface ICatalogQueryService
    {
        Product GetProductById(int id); 
    }

Now in order to configure the Service to use our Interceptor we need a way to identify which class we want to apply the interceptor too. You can use Attributes or explicitly set the interceptor when configuring the Container however I prefer is to use an Interface to explicitly mark which Interface should be intercepted. This allows you to easily configure a Facility but that’s a topic for another post.

    public interface IMustBeCached
    {
    }

So after that I change the ICatalogQueryService to implement the IMustBeCached interface.

    public interface ICatalogQueryService : IMustBeCached 
    {
        Product GetProductById(int id); 
    }

The next and most important part is creating the Interceptor, this has to implement the Castle.Core.Interceptor.IInterceptor interface.

So here it is, in just a few lines of code we have implemented a reusable Cache policy that caches all return values for one minute and our ICatalogQueryService implementation is none the wiser.

   public class CacheInterceptor : IInterceptor
   {
       private readonly ICacheProvider cacheProvider;

       private const int CacheExpiryMinutes = 1; 

       public CacheInterceptor(ICacheProvider cacheProvider)
       {
           this.cacheProvider = cacheProvider; 
       }

       #region IInterceptor Members

       public void Intercept(IInvocation invocation)
       {
           //check if the method has a return value
           if (invocation.Method.ReturnType == typeof(Void))
           {
               invocation.Proceed();
               return; 
           }

           var cacheKey = BuildCacheKeyFrom(invocation); 
          
           //try get the return value from the cache provider
           var item = cacheProvider.Get(cacheKey); 

           if (item != null)
           {
               invocation.ReturnValue = item;
               return;
           }

           //call the intercepted method
           invocation.Proceed();

           if (invocation.ReturnValue != null)
           {
               cacheProvider.Put(cacheKey, CacheExpiryMinutes, invocation.ReturnValue); 
           }

           return;
       }

       #endregion

       private static string BuildCacheKeyFrom(IInvocation invocation)
       {
           var methodName = invocation.Method.Name;

           var arguments = (from a in invocation.Arguments select a.ToString()).ToArray();
           var argsString = string.Join(",", arguments);

           var cacheKey = methodName + "-" + argsString;

           return cacheKey; 

       }
   }

Now all that’s left to do is tie it to configure the Windsor Container to use the Interceptor.

     public class Bootstrapper
     {
        private WindsorContainer container; 
        public WindsorContainer Container
        {
            get { return container;  }
        }

        public void Configure()
        {
            container = new WindsorContainer();

            container.Register(
                Component.For<CacheInterceptor>(),
                Component.For<ICacheProvider>()
                    .ImplementedBy<WebCacheProvider>().LifeStyle.Singleton,
                Component.For<ICatalogQueryService>()
                    .ImplementedBy<CatalogQueryService>()
                    .LifeStyle.Transient
                    .Interceptors(new InterceptorReference(typeof(CacheInterceptor))).Anywhere);

        }
     }

The other great thing about this is that you can turn caching on or off just by configuring the Container.
Usually I will only enable Interceptors when in the Staging & Production environments and so keep any Cache out of the Debug environment as it can be problematic.

That’s all there is to it. Feel free to comment if you have any questions.

Till next time.

Top 10 Job Interview Tips for Software Developers

noreply@blogger.com (Will Beattie) — Sun, 15 Aug 2010 22:00:00 +0000

This post maybe a little off the usual thread of my usual Technical posts but I feel it has to be said.

Since moving to Singapore and trying to build a development team I have found myself inundated with resumes, in the last 6 months alone I have probably read in excess of 500 resumes and interviewed less than 5% of that number and thought I would share some tips for any hopeful developers.

Most of this should be common sense, but the funny thing about common sense is that it’s not so common.

1. Prepare
Research the company before going for an interview. If you know who’s interviewing you then Google them and see what they’re about. This can help give you an idea of the types of questions you will get in the interview.

There’s no excuse for not even having a basic idea of what the companies core business is.

2. Acronyms != Interesting Resume
There are no prizes for the most acronyms you list. As a rule only list those that you are able to discuss in detail for two minutes or more.

3. Relevance
Customize your resume for the specific role you are applying for.

If it’s a C# role, then a full page dedicated to a project you did using COBOL back in the 80’s is not going to win you any points.

4. Keep It Simple Stupid (KISS)
Keep your resume to four pages or less. Write it so that all the important information can be viewed in 10 seconds or less.
Bullet points are as always a great way of conveying concise information.

I usually read resumes on my iPhone whilst on the daily commute so less is best.

5. Be Honest
Don’t exaggerate your years of experience. The fact you did a one month internship in your final year of study does not equate to one years work experience.

6. Read the Ad
Read the job ad carefully and follow any instructions. If the ad says “Please apply via our Job Portal” then actually apply via their job portal and not via the website the ad is on.

Attention to detail and the ability to follow instructions are paramount to the success of a developer.

7. Not too Early, Not too Late
There is a fine line between arriving too early and arriving too late, for me that line is 15 minutes either side of the interview time.

Even though it is annoying I am less concerned with candidates who arrive more than 15 minutes early. But if you’re more than 15 minutes late and don’t call then to let me know you’ll be late then you shouldn’t bother coming in at all.

8. Communication
Communication is key to the success of any development team & it’s important to demonstrate this in the interview.
Listen to the questions and take the time to formulate answers before speaking, avoid mono-syllabic answers but also be careful not to hijack the interview. An interview is not the time for you to rant incessantly about the latest “X” technology.

9. Ask Questions
Asking questions shows that you are inquisitive mind and able to communicate effectively, again both skills that are important for an aspiring Developer.

10. Keep your Web Presence Clean
The first thing I do and many recruiters do is type the name of a candidate into Google and see what results it brings up.

No presence at all can show that you’re not engaged with the community. On the flipside those photos of you wasted at the weekend do nothing to help your cause.

My usual process is to search Google, Facebook, Twitter and LinkedIn.

Well that’s my list.

If you’re a C# Developer or JavaScript whizz and fancy working in Singapore then check out some of our job openings or drop me a line on will@willbeattie.net

Cannot load reference assembly Error 13 .NET 4.0 SvcUtil

noreply@blogger.com (Will Beattie) — Thu, 29 Jul 2010 04:49:00 +0000

Just ran into this nice error while trying to generate a proxy using the SvcUtil and /reference option in a .NET 4.0 Project.

Error 13 Cannot load reference assembly 'C:\MySource\bin\Debug\MyReference.dll'

The solution makes total sense as the current version of SvcUtil is for .NET 3.5 only.

You can get around this by changing your path to:

C:\Program Files\Microsoft SDKs\Windows\v7.0A\bin\NETFX 4.0 Tools\svcutil.exe

Alternatively you can download the latest Microsoft Windows SDK for Windows 7 and .NET Framework 4

And then change the SvcUtil path to:

C:\Program Files\Microsoft SDKs\Windows\v7.1\bin\NETFX 4.0 Tools\svcutil.exe

Hope this helps someone.

Flex Builder 3 changes not reflected in compiled SWF

noreply@blogger.com (Will Beattie) — Tue, 20 Jul 2010 03:26:00 +0000

I have just spent the last 2 hours trying to figure out why my none of my changes were reflected in the compiled SWF.

Thankfully I came across this post where one of the comments had the solution.

If you have a SWF named <PROJECTNAME>.swf in the html-template then deleting it will fix this issue.

Apparently the existence of this file means that when you do a Build or Release Export it just copies this file and does NOT compile the latest source code.

Makes sense to me!

It turns out that it’s a known bug which has never been fixed, but I hope it’s been fixed in Flex Builder 4.

Hope this helps someone having the same problem in the future.

CDN Sync Version 1.0 Released

noreply@blogger.com (Will Beattie) — Sun, 13 Jun 2010 08:54:00 +0000

I have just published a small project on CodePlex called CDN Sync.

The primary goal of this project is to provide a set of tools to synchronize local files and folders with Cloud Storage Providers during the Build and Deployment process.

Using a CDN (Content Delivery Network) for all your static website files should be the default these days if you’re concerned providing the optimal client experience, however deploying these files to your CDN Provider can be a manual process and prone to versioning issues.

Version 1.0 includes an MSBuild task and support for synchronizing with Amazon S3.

Pre-Requisites:
Visual Studio 2010
Amazon Web Services Account

You can download the Release from here and read the documentation

Future features:

Support for Cloud Files
Support for Windows Azure
Command Line Interface

Any questions or issues then please post them to the CodePlex project or comment here.

HTML5 Video Tag Codec Support

noreply@blogger.com (Will Beattie) — Sun, 16 May 2010 03:29:00 +0000

Recently there was a question asked over on Stackoverflow.com regarding best practices around H.264 and the Video Tag.
The answer with the most votes has IMO provided an inadequate solution and I thought I would take the time to clear this up with a new post.

Unfortunately and to the frustration of developers that deal with Video content, HTML 5 has dropped the Codec as part of the specification leaving the Browser vendors to implement whatever they want.
However the consensus is to use H.264 with even Microsoft adopting this for the forthcoming Internet Explorer 9.

Here’s how it stacks in terms of browser support as of writing.

Browser	H.264	Ogg Theora
Google Chrome	Yes	Yes
Firefox	No	Yes
Safari	Yes	No
IE9	Yes	No
Opera	No	Yes

H.264 Only – Chrome, Safari & IE9

If you see this your browser does not support H.264

<video width="480" height="360" preload autobuffer controls>
    <source src="http://blog.willbeattie.net.s3.amazonaws.com/framecount.mp4" type="video/mp4" />
    <p>If you see this your browser does not support H.264</p>
</video>

If you view this on Firefox you will see that their implementation does not ignore the Video tag and display the paragraph tag, instead it shows show the big X.

Ogg Theora Only – Firefox, Opera, Chrome

If you see this your browser does not support Ogg Theora

<video width="480" height="360" preload autobuffer controls>
    <source src="http://blog.willbeattie.net.s3.amazonaws.com/framecount.ogv" type="video/ogg" />
    <p>If you see this your browser does not support Ogg Theora</p>
</video>

H.264 & Ogg Theroa – Firefox, Chrome, Safari, Opera & IE9

If you see this your browser does not support H.264 or Ogg Theora

<video width="480" height="360" preload autobuffer controls>
    <source src="http://blog.willbeattie.net.s3.amazonaws.com/framecount.mp4" type="video/mp4" />
    <source src="http://blog.willbeattie.net.s3.amazonaws.com/framecount.ogv" type="video/ogg" />
    <p>If you see this your browser does not support H.264 or Ogg Theora</p>
</video>

If you view this page on all of the aforementioned browsers you will notice that the current Video tag implementations are not very helpful to developers. The ideal would be that the Browser detects the Codec that they don’t support and then fallback to whatever is in the Video tag.

Now if you are just displaying video content then this doesn’t pose too much of a problem as you can just support a single codec and then fallback to a plug-in based media player of your choice. However if you want to start implementing features using Canvas then this becomes a major roadblock.

The only solution right now is to have an H.264 version and an Ogg Theora version of the same video which is a real PITA especially if you have thousands if not millions of videos to deal with.

If you do go the route of converting all your videos from H.264 to Ogg Theora then I’d recommended using the FFMpeg2Theora convertor as it’s very easy to use.

Hope this clarifies some of the challenges presented when dealing with Video content using HTML5.

Unit Testing Workflow Activities in .NET 4.0

noreply@blogger.com (Will Beattie) — Sat, 01 May 2010 07:43:00 +0000

Recently during a catch up with my buddy Keith Patton I was (as I tend to do) singing the praises of the Workflow in .NET 4.0. The all important question about Unit Testing support was raised, I tried as best as I could to explain the new In and Out Arguments but I didn’t feel I was convincing enough, so I though I would clarify with a blog post.

If you developed Workflows in .NET 3.5 then you will be well aware of the lack of Unit Test support which was due to many reasons but mostly in part to the complex Workflow hosting environment.

When moving to .NET 4.0 Workflow Foundation from .NET 3.5 it pays to be conscious of that fact that there is no longer a distinction between Activities and Workflows. Everything derives from the System.Activities.Activity class. So the definition of a Workflow is just a collection of of 1 or more Activities.

I have designed a very simple Rental Car Activity which takes the an Applicants age as the input and outputs a True or False depending on the age.

One of the key benefits of .NET 4.0 is that Activities can be run by passing in an IDictionary<string, object> of your In arguments and the result is an IDictionary<string, object> of Out Arguments, that’s right no more Dependency Properties.

This is the Workflow Diagram.

As you can see above there is an Age In argument and an IsApproved Boolean is the Out argument.

Here is the definition of the Age Check Condition.

Now when coming to do the Unit Test the key part of the equation is the WorkflowInvoker class which allows to run any Activity synchronously. Specifically the WorkflowInvoker.Invoke method which takes the Activity and IDictionary<string, object> arguments and returns an IDictionary<string, object>.

And here is the code to test our Activity.

        [TestMethod]
        public void Invoke_RentalCarApprovalWorkflow_WithApplicatantOver25_Should_Return_True()
        {
            var age = 26;

            var arguments = new Dictionary<string, object>
                                {
                                    {"Age", age}
                                };


            var activity = new Workflow.RentalCarApprovalWorkflow();

            var actual = WorkflowInvoker.Invoke(activity, arguments);

            Assert.IsNotNull((actual));
            Assert.AreEqual(1, actual.Count);
            Assert.AreEqual(true, actual["IsApproved"]);

        }

        [TestMethod]
        public void Invoke_RentalCarApprovalWorkflow_WithApplicatantUnder25_Should_Return_False()
        {
            var age = 24;

            var arguments = new Dictionary<string, object>
                                {
                                    {"Age", age}
                                };

            var activity = new Workflow.RentalCarApprovalWorkflow();

            var actual = WorkflowInvoker.Invoke(activity, arguments);

            Assert.IsNotNull((actual));
            Assert.AreEqual(1, actual.Count);
            Assert.AreEqual(false, actual["IsApproved"]);

        }

It’s really that simple.

Drop me a line if you have any questions or problems implementing this.

Till next time.