Restless C#ding

Safely Swallowing Exceptions

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Mon, 11 Feb 2013 07:39:00 -0400

While using exceptions for flow logic is never a good idea, sometimes you need to have the ability to swallow Exceptions thrown that can happen during the reasonable course of use of your product.

Take for example an arbitrary design requirement that allows an enduser to upload a binary file of a given type... let's say a word document... to your site, and read it in.

Now there's a few things you can do initially to verify this. You may check a file's extension. If given an address of a file (path or URL), you may verify that it actually exists and you can get a read handle to it. But what about some more exceptional circumstances: an enduser takes Word.exe and renames it to Word.doc. You built a whole framework to analyze their information, but either a component you have used to read Word files or your own code is throwing exceptions because this is not what you think it is.

A handy way to handle this situation cleanly is to use a marker Exception. As deep as you can (in the example of an external component, this would usually be the Open or Load or even a constructor) wrap the call that is 'untrustworthy' in a try/catch.

Create a special exception to mark this behavior:

public class CouldNotReadWordFileException : Exception
{
   public CouldNotReadWordFileException(string message,
   Exception innerException) :
   base(message, innerException)
   {
   //marker only.
   }
}

Now the fun part. Take the exception, and wrap it with your own (in this case, it's in some library class. Let's call it "WordParser"):

try
{
   someWordObject.OpenFile(inputFileStream);
} catch (Exception e)
{
   throw new CouldNotReadWordFileException("Unable to open word file.",e);
}

So what does that actually get us? Well, we can try/catch outside, and only
catch that specific exception type. Other exceptions will still be thrown
and bubbled out to whatever default exception handling logic is in place.

What happens when something is failing with openfile and you want to trace
locally? Well, we can conditionally throw from outside. I find a static settings
class is most useful for this, since it's the type of setting that will be done at the environment level.

public static class ConstantExceptionHandlingSettings
    {
        private static bool _doRethrowWordOpeningExceptions = false;
        public static bool DoRethrowWordOpeningExceptions
        {
            get { return _doRethrowWordOpeningExceptions }
            set { _doRethrowWordOpeningExceptions = value; }
        }
    }

So in the code that calls our WordParser class, you can do the following:

try
{
   var parser = new WordParser(aFile);
   parser.Parse();
} catch (CouldNotReadWordFileException notWord)
{
   if (ConstantExceptionHandlingSettings.DoRethrowWordOpeningExceptions)
   {
      throw notWord; //inner exception will have details.
   }

   //now if we're not just bombing out, handle it cleanly:
   //perhaps this occurs if they rename a word file with an exe extension.

   aFile.IsUnparsable = true;
   aFile.IsValid = false;
   return; //no need to go any further
}

This technique is useful for when you want to turn on/off the way Exceptions are bubbled. In your text fixture's setup method, you can just call:

ConstantExceptionHandlingSettings.DoRethrowWordOpeningExceptions = true;

Global Static Settings

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Tue, 22 Nov 2011 08:36:00 -0400

Sometimes if you have a complex system, you'll want the equivalent of 'permanent' environment settings, where 'permanent' is within the context of your execution. The normal path for these things would be some sort of config setting, and perhaps constructor injection to get those settings to your worker bees.

Here's a technique to make it simpler to get such a setting deep into your code, without some of the clutter of injecting yet another dependency. Is this an anti-pattern? I'd love to hear feedback against such a thing; I'll briefly address some of the potential drawbacks I can think of off the top of my head after exploring the code.

First a business scenario. Lets say your library / application is talking to an unreliable system, however it's a necessary evil due to the fact that your company, Acme Lottery Predictions, is dependent on. Once a day, every day, you go out to every newspaper in America's web sites, and find out what last night's winning lottery numbers are. You then try to predict what the next night's numbers will be in your library through fancy number-crunching.

Now unfortunately some of those sites will be down, and some will decide that they just didn't get around to it that day, and your business needs to go on, with or without that data. So you write this library that can handle the exceptions (truly these are sometimes exceptions -- let's say a server 500 error, or just a domain name that can't resolve, can't ping, etc... any number of 'horrible, really bad, no good' things) gracefully by logging and moving on (and possibly retrying later).

But you want to test your code individually -- sometimes you want to throw exceptions while testing, sometimes you don't. Enter a global settings class.(Note: you’ll see this sometimes in third-party libraries to set things like License Keys)

public static LotteryRemoteConnectivitySettings
{
    public static int NumberOfAttemptsBeforeGivingUp { get { return 5; } }

    private static bool _doRethrowExceptionsAfterLogging = false;
    public static bool DoRethrowExceptionsAfterLogging 
    {
        get { return _doRethrowExceptionsAfterLogging; }
        set { _doRethrowExceptionsAfterLogging = value;}
    }
}

And we have a unit... (well, integration test, really -- since this test goes against remote systems, by definition it is NOT a unit test) test of our system, to represent our client.

[TestFixture]
public class TestsRemoteConnectivityToWisconsinJournal
{
    [SetUp]
    public void Setup()
    {
        LotteryRemoteConnectivitySettings.
         DoRethrowExceptionsAfterLogging=true;
    }

    [Test]
    [Category("Integration Tests")]
    [Category("Remote Site Tests")]
    public void 
     Test_Wisconsin_Journal_Has_Lottery_Numbers_For_Date()
    {
        var dateToRetrieve = DateTime.Now;
        var lotteryNumberGetter = 
               new WisconsinJournalLotteryGetter();
        var expected = true;
        var actual = 
                  lotteryNumberGetter.
                  CanGetLotteryNumbersFor(dateToRetrieve);
        Assert.AreEqual(expected, actual);
    }
}

Simply put, in our test fixture's setup, we just have to call that one property, and the behavior should be that if the setting is true, then wherever we were catching exceptions before, we should throw them now. And how to accomplish this?

public class WisconsinJournalLotteryGetter 
    : IGetLotteryNumbers
{
    public bool CanGetLotteryNumbersFor
        (DateTime dateToRetrieve)
    {
        var canGetNumbers = false;
        
        try 
        {
            canGetNumbers = 
                DoTheVoodooThatYouDoSoWell();
        }
        catch (Exception exc)
        {
            Log(exc);
            if             (LotteryRemoteConnectivitySettings.
                DoRethrowExceptionsAfterLogging)            
            {
                throw;
            }
        }
        return canGetNumbers;
    }
}

Now we have a default behavior of 'false' when an exception is thrown and it's not allowed to rethrow exceptions; if it IS allowed, it will just throw the exceptions as normal. Either way it gets logged properly. This is not a substitute for proper exception handling, however it does allow a mission-critical system to stay alive even when unreliable dependencies disappear.

You will always have to make the business decision of 'What does it mean when the data that I retrieve from multiple sources is incomplete or untrustworthy?' Along that vein, you will have to decide how to crunch that data or retry. The best solution is to learn from your logged data what causes your dependencies to collapse, however that is not always under your control.

Now as far as what is bad about this method of using a global setting, let's take a look.

1. It's a hidden dependency. It's not immediately clear that WisconsinJournalLotteryGetter has a dependency on LotteryRemoteConnectivitySettings.

2. It's coupled code. This one can at least be mitigated through factories. If you had a factory that returned you back an interface matching LotteryRemoteConnectivitySettings, then you could provide the static class sometimes, and perhaps other times read it from the registry, etc. That way WisconsinJournalLotteryGetter could be dependent on an ILotteryRemoteConnectivitySettings.

3. Exception handling not ideal -- well, there are other uses of this global settings code; so for the given topic, I'm going to say that's not really relevant here.

4. If you 'miss' using this setting in 1 of your 500 newspaper concretes, good luck finding that.

Questions / Comments are always welcome.

Exceptions During EventHandler Invocation and Allowing No Subscribers

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Wed, 24 Aug 2011 20:51:00 -0400

Ever create a middle-tier library and have a ton of EventHandlers / EventArgs on your classes because you think it will be convenient in some use cases that just aren't needed up front?

In my experience, this always winds up as an annoyance on invocation as the end user of the library would need to wire up all those events even if they weren't being used, or risk a NullReferenceException saying "Object reference not set to an instance of an object".

This post shows a quick and easy pattern to use across any application or library to make consuming events completely optional. You can tell your classes "if no one is listening, don't invoke".

Lets start out with a simple console application and a few classes. We have a forest with some trees:

public class Forest
{
 public int TreeCount { get; private set; }
 public Forest(int numberOfTrees)
 {
  TreeCount = numberOfTrees;
 }

 public void RemoveATree()
 {
  TreeCount--;
 }

 public bool HasTrees { get { return TreeCount > 0; } }
}

And a Chopper that knows how to cut down trees in that forest.

public class TreeChopper
{
    private readonly Forest _forest;
    public TreeChopper(Forest aForest)
    {
        _forest = aForest;
    }

    public void ChopATreeDown()
    {
        if (!_forest.HasTrees)
        {
            throw new NoTreesLeftException("There are no trees in the forest to chop down.");
        }

        OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));
        OnTreeChopped.Invoke(this, new TreeChoppingEventArgs("Finished chopping."));
        OnTreeFalling.Invoke(this, new TreeChoppingEventArgs("Tree is starting to fall down."));
        _forest.RemoveATree();
        OnTreeFell.Invoke(this, new TreeChoppingEventArgs("Tree has fallen. There are "+_forest.TreeCount+" trees left in the forest."));
    }

    public EventHandler<TreeChoppingEventArgs> OnTreeChopping;
    public EventHandler<TreeChoppingEventArgs> OnTreeChopped;
    public EventHandler<TreeChoppingEventArgs> OnTreeFalling;
    public EventHandler<TreeChoppingEventArgs> OnTreeFell;

}

Note that there's 4 events for stages of chopping down that tree, and a simple message EventArgs:

public class TreeChoppingEventArgs : EventArgs
{
    public string Message { get; set; }
    public TreeChoppingEventArgs(string message)
    {
        Message = message;
    }
}

public class NoTreesLeftException : ArgumentOutOfRangeException
{
    public NoTreesLeftException(string message) : base(message){}
}

This is all well and good, and if we set up a runner, we can chop down all the trees in a little console app. The chopping:

public static void Run()
{
    //1. set up a forest.
    var forest = new Forest(numberOfTrees:5);

    //2. wire up all events.
    var chopper = new TreeChopper(forest);

    chopper.OnTreeChopping += OnTreeChoppingEvent;
    chopper.OnTreeChopped += OnTreeChoppingEvent;
    chopper.OnTreeFalling += OnTreeChoppingEvent;
    chopper.OnTreeFell += OnTreeChoppingEvent;


    //3. cut down all trees in the forest
    while (forest.HasTrees)
    {
        chopper.ChopATreeDown();
    }
}

private static void OnTreeChoppingEvent(object sender, TreeChoppingEventArgs e)
{
    Console.WriteLine(e.Message);
}

So this displays the following output:

Starting to chop a tree.
Finished chopping.
Tree is starting to fall down.
Tree has fallen. There are 4 trees left in the forest.
Starting to chop a tree.
Finished chopping.
Tree is starting to fall down.
Tree has fallen. There are 3 trees left in the forest.
Starting to chop a tree.
Finished chopping.
Tree is starting to fall down.
Tree has fallen. There are 2 trees left in the forest.
Starting to chop a tree.
Finished chopping.
Tree is starting to fall down.
Tree has fallen. There are 1 trees left in the forest.
Starting to chop a tree.
Finished chopping.
Tree is starting to fall down.
Tree has fallen. There are 0 trees left in the forest.

But that's pretty verbose. Lets say from the outside perspective, we don't care when the tree chopping started... or even finished. The only thing we care about is when that tree fell, so we can drag it away and use the wood. How do we invoke all of the methods, in case they are used, but get protected in case they are not listened to?

The answer lies in how events work. An event handler is simply a way of keeping track of how to call the delegates -- the places that the event truly gets 'handled'.

Let's take a look at the event handler's definition (.NET 4.0 EventHandler<TEventArgs> where TEventArgs:EventArgs )

EventHandler
Of course the real meat in Delegate
And the internals in GetInvocationList : Delegate[]

This is essentially a subscription list. If you've wired up a handler for the event, it gets added to the list (really an Array). If not, the list is empty.

So to make an event that ensures it's only fired when there's someone listening the assumption is that we simply have to do the following:

Before:

OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));

After:

if (OnTreeChopping.GetInvocationList() != null)
{
OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));
}

Yet in practice, you'll find that this still throws the NullReferenceException. Why? Well, the OnTreeChopping object (the EventHandler itself) turns out to be null. So the first reaction is just to do the following. Check to see if the EventHandler is null, then invoke it if it's not.

Before:

OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));

After:

if (OnTreeChopping != null)
{
OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));
}

So now, if we wrap all of our events in this little block, they'll be 'safe', right? Well, lets assume the answer to that is true for the moment. Writing the same thing over and over again would be smelly; we've just made it 4 lines of code (well, 2 + braces) every time we want to invoke an event. Enter a handy extension method:

public static class EventHandlerExtensions
{
    public static void TryInvoke<T>
        (this EventHandler<T> anEvent, object sender, T eventArgs)
        where T:EventArgs
    {
        if (anEvent != null)
        {
            anEvent.Invoke(sender, eventArgs);
        }
    }    
}

So in our calling code:

Before:

OnTreeChopping.Invoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));

After:

OnTreeChopping.TryInvoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));

And if we only care about the tree being done falling, we can comment out the wiring for the other events as so:

//chopper.OnTreeChopping += OnTreeChoppingEvent;
//chopper.OnTreeChopped += OnTreeChoppingEvent;
//chopper.OnTreeFalling += OnTreeChoppingEvent;
chopper.OnTreeFell += OnTreeChoppingEvent;

The invocation as so:

OnTreeChopping.TryInvoke(this, new TreeChoppingEventArgs("Starting to chop a tree."));
OnTreeChopped.TryInvoke(this, new TreeChoppingEventArgs("Finished chopping."));
OnTreeFalling.TryInvoke(this, new TreeChoppingEventArgs("Tree is starting to fall down."));
_forest.RemoveATree();
OnTreeFell.TryInvoke(this, new TreeChoppingEventArgs("Tree has fallen. There are " + _forest.TreeCount + " trees left in the forest."));

And our output looks like so:
Tree has fallen. There are 4 trees left in the forest.
Tree has fallen. There are 3 trees left in the forest.
Tree has fallen. There are 2 trees left in the forest.
Tree has fallen. There are 1 trees left in the forest.
Tree has fallen. There are 0 trees left in the forest.

It's all nice and easy; no messages about chopping, which I don't care about. If the event never gets wired, it doesn't get called. Great. But that begs the question of why to even provide GetInvocationList anyway. Is it useful to us at all? Well, it turns out that there are cases where a misbehaving (read: exception throwing) event handler can spoil invocation for the rest of the handlers.

Chris Tacke had a great writeup on this here (http://blog.opennetcf.com/ctacke/2010/05/27/WhyYouShouldUseEventHandlerGetInvocationList.aspx), and I'm going to apply this scenario to our example. Lets modify our code somewhat to create something bad happening. Lets say we have two event handlers. One transmits the message about the tree having fallen to an RSS feed, and another logs it to some sql database. The sql database server just went down, but we still want to publish the live events to external consumers depending on the data for estimating how much wood they could have available for their production line. First of all, our extension method is now going to swallow the Exceptions, and write them nicely to our console output:

public static void TryInvoke<T>(this EventHandler<T> anEvent, object sender, T eventArgs)
    where T:EventArgs
{
    if (anEvent != null)
    {
        try
        {
            anEvent.Invoke(sender, eventArgs);    
        }
        catch (Exception e)
        {
            Console.WriteLine(e.Message);
        }
    
        
    }
}

Second, we're going to create a 'good' event handler, and a 'bad' one:

private static void OnTreeChoppingEventLogToSql(object sender, TreeChoppingEventArgs e)
{
    Console.WriteLine(e.Message);
    throw new InvalidDataException("sql database does not exist");
    Console.WriteLine("Logged to sql.");
}

private static void OnTreeChoppingEventLogToRssFeed(object sender, TreeChoppingEventArgs e)
{
    Console.WriteLine(e.Message);
    Console.WriteLine("Logged to RSS feed.");
}

And when we wire them up:

chopper.OnTreeFell += OnTreeChoppingEventLogToSql;
chopper.OnTreeFell += OnTreeChoppingEventLogToRssFeed;

So what does the output look like?
Tree has fallen. There are 4 trees left in the forest.
sql database does not exist
Tree has fallen. There are 3 trees left in the forest.
sql database does not exist
Tree has fallen. There are 2 trees left in the forest.
sql database does not exist
Tree has fallen. There are 1 trees left in the forest.
sql database does not exist
Tree has fallen. There are 0 trees left in the forest.
sql database does not exist

Wait... what happened there? What about that critical RSS feed? Well, it turns out that the exception thrown blocked further invocation of other event handlers. Keep in mind that event handlers are invoked in the order that they are added to the invocation list. Also, an exception by default prevents further invocation. So if we want to change that behavior, we'll need to manually call each EventHandler<T>, rather than just let the MulticastDelegate handle it.

So our extension method changes to:

public static void TryInvoke<T>(this EventHandler<T> anEvent, object sender, T eventArgs)
    where T:EventArgs
{
    if (anEvent != null)
    {
        foreach (EventHandler<T> wiredHandler in anEvent.GetInvocationList())
        {
            try
            {
                wiredHandler(sender, eventArgs);
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }
        }
    }
}

Which makes our output work:
Tree has fallen. There are 4 trees left in the forest.
sql database does not exist
Tree has fallen. There are 4 trees left in the forest.
Logged to RSS feed.
Tree has fallen. There are 3 trees left in the forest.
sql database does not exist
Tree has fallen. There are 3 trees left in the forest.
Logged to RSS feed.
Tree has fallen. There are 2 trees left in the forest.
sql database does not exist
Tree has fallen. There are 2 trees left in the forest.
Logged to RSS feed.
Tree has fallen. There are 1 trees left in the forest.
sql database does not exist
Tree has fallen. There are 1 trees left in the forest.
Logged to RSS feed.
Tree has fallen. There are 0 trees left in the forest.
sql database does not exist
Tree has fallen. There are 0 trees left in the forest.
Logged to RSS feed.

Note that this won't really work as we want it to if we need to get our exceptions properly bubbled out after the TryInvoke call. To this end, we can introduce some Exception collecting,

including a wrapper for event errors:

public class EventInvocationException : Exception
{
    public List<Exception> Exceptions { get; set; }
    public string Message { get; set; }
    public EventInvocationException(string message, List<Exception> exceptions)
    {
        Message = message;
        Exceptions = exceptions;
    }
}

And then collect them as they are thrown:

public static void TryInvoke<T>(this EventHandler<T> anEvent, object sender, T eventArgs)
    where T:EventArgs
{
    var bubbledExceptions = new List<Exception>();
    if (anEvent != null)
    {
        foreach (EventHandler<T> wiredHandler in anEvent.GetInvocationList())
        {
            try
            {
                wiredHandler(sender, eventArgs);
            }
            catch (Exception e)
            {
                bubbledExceptions.Add(e);
            }
        }
    }

    if (bubbledExceptions.Count>0)
    {
        throw new EventInvocationException("At least one event handler threw an exception", bubbledExceptions);
    }
}

Now since this will be used in a high-level library, allow the invoker to specify allowed behavior, with a default of allowing all of the exceptions to bubble out:

public static class EventArgsExtensions
{
    public static void TryInvoke<T>(this EventHandler<T> anEvent, object sender, T eventArgs, bool allowMultipleExceptions=true)
        where T:EventArgs
    {
        if (allowMultipleExceptions)
        {
            anEvent.TryInvokeWithMultipleExceptions(sender, eventArgs);
        }
        else
        {
            anEvent.TryInvokeWithOnlyOneExceptionAllowed(sender, eventArgs);
        }
    }

    private static void TryInvokeWithOnlyOneExceptionAllowed<T>(this EventHandler<T> anEvent, object sender, T eventArgs)
        where T:EventArgs
    {
        if (anEvent!=null)
        {
            anEvent.Invoke(sender, eventArgs);
        }
    }

    private static void TryInvokeWithMultipleExceptions<T>(this EventHandler<T> anEvent, object sender, T eventArgs)
        where T:EventArgs
    {
        var bubbledExceptions = new List<Exception>();
        if (anEvent != null)
        {
            foreach (EventHandler<T> wiredHandler in anEvent.GetInvocationList())
            {
                try
                {
                    wiredHandler(sender, eventArgs);
                }
                catch (Exception e)
                {
                    bubbledExceptions.Add(e);
                }
            }
        }

        if (bubbledExceptions.Count>0)
        {
            throw new EventInvocationException("At least one event handler threw an exception", bubbledExceptions);
        }
    }
}

Now we can still call the TryInvoke on our event like before:

OnTreeFell.TryInvoke(this, new TreeChoppingEventArgs("Tree has fallen. There are " + _forest.TreeCount + " trees left in the forest."));

However if we want the first exception to really blow up, we can use the old-style plain invoke:

var allowMultipleExceptions=false;
OnTreeFell.TryInvoke(
this, 
new TreeChoppingEventArgs("Tree has fallen. There are " +         _forest.TreeCount + 
    " trees left in the forest."), 
allowMultipleExceptions);

And the best part is that all you have to do from this point on is to call TryInvoke instead of Invoke whenever you're publishing an event, and import the namespace of your extension method.

Unit Testing Highly-Dependent Classes (Stubbing Classes such as System.Web.UI.Page)

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Fri, 19 Aug 2011 08:00:00 -0400

After Jetbrains’ recent releases of DotCover and Resharper 6, along with a mostly isolated set of libraries, I forced myself to completely cover the code. I started out with a fairly TDD set of libraries, which allow me to have some classes prepopulated based on contextual information from the System.Web.UI.Page object that will serve as the destination for communication between two pages.

To start out, I had several classes with their own unique responsibilities, and then one base page (which had a generic type constraint) that allowed for the automatic population of the object in the page.

public ReceivingPageBase<T> : System.Web.UI.Page
where T : class, IPagePopulatableObject, new()
{
public T RecievedObject {get;set;}
protected Override void OnPreLoad(EventArgs e)
{
//attempt to load the object, persist in viewstate;
//if postback, load it from viewstate instead.
}
}
public ReportingInformation : IPagePopulatableObject
{ //some properties, implementation of IPagePopulatableObject }

Now the tricky part. As I went to write tests, and kept analyzing code coverage, I was also reading through The Art of Unit Testing by Roy Osherove. The book’s very well paced, and a fairly light read. It’s pretty easily explained stubbing and mocking, which have eluded me for a bit of time. I had used constructor injection for my repositories and stubbed them out so that my tests were non-db-dependant, and got my overall coverage up to about 85%.

As an aside, I had run into a closing-brace bug with DotCover itself that was tricky to workaround (see my next post for some details on that particular gem).

As I went to write tests against the remaining bit of code (the base page shown in the code sample above), I had tons of trouble writing a stub that would essentially be a fake page. Oh how easy it would be if only I could tag System.Web.UI with an interface… but alas, it’s in a framework library and out of my control.

So I slept for a couple days on that particular issue, doing some research, when I came across one of Martin Fowler’s solutions to this particular issue… using a design pattern known as Passive View in order to wrap the page. The real system can use a concrete class that directly maps and wraps System.Web.UI.Page, while the tests can use a stub that simulates page context information. Let’s jump to code to see how it works (note the last image in the Passive View link was the clearest application to this problem). First, we need to isolate the ReceivingPageBase<T> itself from the loading of the object. The ReceivingPageBase<T> class will be moved out of the library and close to the applications which consume it. In it’s place will be:

//In library:
public PagePopulatableObjectReciever<T>
where T:class, IPagePopulatableObject, new()
{
private readonly Page _page;
public T RecievedObject {get;set;}
public PagePopulatableObjectReciever(Page page)
{
_page = page;
Received = DoLoadObject();
}
private T DoLoadObject()
{
//attempt to load the object from _page, 
//persist in _page.viewstate;
//if postback, load it from _page.viewstate instead.
}
}
//In application:
public AppSpecificRecievingPageBase<T> : System.Web.UI.Page
where T : class, IPagePopulatableObject, new()
{
public T RecievedObject {get;set;}
protected Override void OnPreLoad(EventArgs e)
{
//attempt to load the object, persist in viewstate;
//if postback, load it from viewstate instead.
}
}

Note that this only gets us halfway there. We’ve split apart the page from the receiving, so now the page is relatively ‘dumb’ (read: decoupled; SRP is good!). But this still couples the ObjectReciever to the System.Web.UI.Page, where we want to use some stub instead. Well, it’s time to apply the PassiveView pattern. Create the Passive View of System.Web.UI.Page, with an interface and a concrete.

//1. Create the passive view interface
public interface IStatefulPage
{
StateBag ViewState {get;}
bool IsPostBack {get;}
NameValueCollection QueryString {get;}
}
//2. Make the application base page conform
public AppSpecificRecievingPageBase<T> : 
System.Web.UI.Page, IStatefulPage
where T : class, IPagePopulatableObject, new()
{
public T RecievedObject {get;set;}
protected Override void OnPreLoad(EventArgs e)
{
//attempt to load the object, persist in viewstate;
//if postback, load it from viewstate instead.
}
//explicit interface declaration
//needed to avoid conflict with 
//existing ViewState variable
StateBag IStatefulPage.ViewState
{
get {return ViewState;}
}
public NameValueCollection QueryString
{
get { return Request.QueryString;}
}
//IsPostback is 'free'; 
//signature already conforms.
}
//3. Replace the ObjectReciever's page with IStatefulPage.
public PagePopulatableObjectReciever<T>
where T:class, IPagePopulatableObject, new()
{
private readonly IStatefulPage _page;
public T RecievedObject {get;set;}
public PagePopulatableObjectReciever(IStatefulPage page)
{
_page = page;
Recieved DoLoadObject();
}
private T DoLoadObject()
{
//attempt to load the object from _page, 
//persist in _page.viewstate;
//if postback, load it from _page.viewstate instead.
}
}

Now at this point, we’ve broken the coupling of object receiving from the Page object. This is fantastic… the last prep step is to create our stub to use for tests. Obviously depending on your tests, your behavior may vary, but I wanted to add in a sample set of data to receive, and have a working state to store it in for postback, giving me the following stub:

public class StubObjectReceivingPageWithValidObject :
IStatefulPage
{
public StubObjectReceivingPageWithValidObject
(string objectKey)
{
_queryString.Add("MASTERKEY",objectKey);
IsPostBack = false;
}
private readonly StateBag
_viewState = new StateBag();
public StateBag ViewState
{
get { return _viewState;}
}
public bool IsPostBack {get;set;}
private readonly NameValueCollection
_queryString = new 
NameValueCollection();
public NameValueCollection QueryString
{
get { return _queryString;}
}
}

Now I can use that stub for testing, and it will simulate page and context without the need for any sort of test web server, or ‘real’ building of a web page and context information.

What have we learned? Well, wrapping any framework class in an interface can be done through the use of wrapper objects, essentially making a Passive View. Our stub page winds up being a Test Double (aka Imposter) which allows us to test without using any true Page instances. That’s all good.

What’s the downside to doing this? Shallow interfaces for one. I made one that had ViewState, IsPostBack, and QueryString, because I knew those were what I needed to know. If some other class has other parts of the page, I could have either expanded IStatefulPage, or created another Passive View interface. Perhaps these can get out of control if you have too many. Also another downside is that I’m using trainwreck notation (see number 5 here from “Uncle” Bob Martin’s seminal Clean Code) for things like mapping IStatefulPage’s QueryString to Request.QueryString. I can live with that, at least for the present.

All of this leads to increasing testability, and speed of testing. It’s important to keep test speeds high in order to keep your team running them often.

Jetbrains Dottrace, Linq 2 Sql, and the case of the static DataContext

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Thu, 21 Jul 2011 20:00:00 -0400

I've been in head-down, job-focused, non-blogging mode for a while now, but this issue has been enough to wake me out of hibernation. I've always been best at learning things on the job, since I tend to run into real-world issues that the examples never hit (happy path, anyone?), and I've had an intermittent memory exception in an app that I've been trying to track down on nights & weekends for a while. It's not severe enough to linger, but it's frequent enough (once a month or two) to nag at me for not fixing it.

Enter Jetbrains Dottrace Memory. I've used it for performance analysis a few times in the past, and discovered it's invaluable at tracking down where your problems are. Seriously, I can't recommend the Jetbrains suite of tools enough. Sometime in the past year or so, they split apart DotTrace performance from DotTrace memory & I'm at about a half-upgrade (DotTrace memory 3.5), with a currently bad integration story with Studio & Resharper 6, but that's a side story.

I figured I would need to run a memory snapshot on a machine that runs my application (x64 machine, .NET 4, 32-bit console application in case your curious), since the network connectivity the application relies on doesn't exist on good-old localhost. (In the back of my mind I'm thinking I should take some of the remote server's connection logs, and mock up the remote side using an interface... I still may go down that path, but for now I need to rule out the TcpStream as a factor in the leak).

So I install DotTrace, and didn't bother putting in my personal product key, since I was hoping to grab the dump and analyze it on my development workstation anyway. I fire it up, start my console app, run the dump, and boom. I get stuck mid-dump, my app crashes, and no snapshot. I'll save hours and days of summary here, but how I figured this out was to put in "Console.ReadLine()"s in my code, doing deeper and deeper dumps until I found the line of code causing the problem.

It turns out that my Linq2Sql code is at fault (or DotTrace is... an argument can be made for either side). I have a repository that on construction stores a reference to the DataContext needed for queries. DotTrace failed to allow a memory dump after the data context was used. Wrapping it in dispose did not resolve the issue. A deeper change resolves the single code issue I had, however I would need to analyze the rest of the code I have that still uses Linq2Sql (I'm in favor of the NHibernate / Fluent NHibernate / Lambda Extensions kit for data access these days, but I still have some retro code lying around). The greater concern here is code changes just to allow tooling to work. Part of me is curious if the memory leaks are in fact due to stale DataContext objects that do not get disposed.

I'd be curious to see disposal patterns for Linq2Sql DataContext objects, however part of me thinks disposal of Linq2Sql altogether is the more prudent option. I had found this article on options for context instantiation (http://blog.stevensanderson.com/2007/11/29/linq-to-sql-the-multi-tier-story/), and my app used one of the options that relied on being run in single-thread mode (names changed to protect the guilty):

public static class DataContextHelper
{
   public static MyRequestModelDataContext RequestContext = 
      new MyRequestModelDataContext(ConnectionStrings.MyConnectionString);
   public static MyResultModelDataContext ResultContext = 
      new MyResultModelDataContext (ConnectionStrings.MyConnectionString);
}

Unwittingly, by putting a constructor on my repo that went around the static property's context construction, and instead putting the context instantiation inside the repository itself, even having the context being used doesn't prevent the memory dump. I know that having a static class means one instance of that class, and in this case its properties as well, around in memory. I'm not sure what that open data context does to DotTrace, but clearly the result is a crash while attempting to dump. So at this point I start trying to eliminate DataContextHelper in favor of some more atomic operations, and explicit lifecycle of the DataContext.

I started with the above, and a call to create a context in a private member in the repository, and moved to:

public class MyRepository
{
   private MyRequestModelDataContext _context = 
      new MyRequestModelDataContext(ConnectionStrings.MyConnectionString);
   public MyResponse GetResponse()
   {
      var query = from x in context.MyResponses
                  where x.Property.Equals(false)
                  orderby x.AnotherProperty
                  select x;
      var result =  query.FirstOrDefault();
      return MyEntityConverter.ConvertFrom(result);
   }
}

Now the interesting thing is that this alone seems to solve the issue... when MyRepository goes out of scope, the garbage collector would need to handle both the repository instance and its own instance of the DataContext.

The only downside to this would be that the instantiation of a context is by nature expensive, however the typical use case would be to create a single repository instance and use that for all of the data calls within a specific operation set or transaction anyway. So I make the change, and see the behavior of crashing memory dumps; clearly this is not the answer. So lets make things a little more interesting. One more change to the repository to make sure the context is cleaned up:

public class MyRepository : IDisposable
{
   ...
   public void Dispose()
   {
      if (_context != null)
      {
         _context.Dispose();
      }
   }
}

And then back a layer. The original code was:

var repo = new MyRepository();
return repo.GetResponse();

And the changed code is:

MyResponse result;
using (var repo = new MyRepository();
{
   //dump 1
   result = repo.GetResponse();
   //dump 2
}
//dump 3
return result;

The dump comments are Console.ReadLine's injected in to track when the dumps fail. While the console application is set up to wait, I can run over to DotTrace's "Control Profiling" window and dump the memory into another snapshot (or try to, anyway). The results of the above change are that the three dump points work fine. The next step of course would be to extend out these changes to any other repositories / calls. The repositories are easy... the calls are not quite as easy. I find usages of the repository class, and wrap everything in using statements; the next step is verifying at least that the functioning of the application is not impaired, using a test run. So far so good. I then go and run memory dumps at various points throughout the process -- mission succeeded. I'm still unclear as to the nature of the behavior, however the code change allows me to proceed with my diagnostic regiment.

What is a bit unsettling still is that there are points in the application during execution where I am unable to dump out memory without crashing. I need to keep an eye on where there are objects around that depend on open data contexts, and those seem to be directly related to when I am able to successfully generate snapshots.

Quick and Dirty: TSQL to generate random Social Security Numbers (SSNs)

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Wed, 06 Apr 2011 10:12:00 -0400

Here's a quick and easy way to generate fake social security numbers, which is very useful for sample data.

create table #ssns
(
   ssn varchar(11)
)
go
declare @cnt int
DECLARE @maxRandomValueA int = 999, @minRandomValue int = 0;
DECLARE @maxRandomValueB int = 99
DECLARE @maxRandomValueC int = 9999

Set @cnt = 1
While @cnt <= 587 Begin
Insert Into #ssns(ssn)
select 
   replicate('0', 
      3-len(gen.a))+gen.a+'-'+
   replicate('0', 2-len(gen.b))+gen.b+'-'+
   replicate('0', 4-len(gen.c))+gen.c
from
   (
      SELECT 
         convert(varchar(3),CAST(((@maxRandomValueA + 1) - @minRandomValue) * RAND() + @minRandomValue AS int)) as a,
         convert(varchar(2),CAST(((@maxRandomValueB + 1) - @minRandomValue) * RAND() + @minRandomValue AS int)) as b,
         convert(varchar(4),CAST(((@maxRandomValueC + 1) - @minRandomValue) * RAND() + @minRandomValue AS int)) as c
   ) gen
Set @cnt = @cnt + 1
End
select * from #ssns

Search For Text in Files using Powershell

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Sat, 11 Sep 2010 09:50:00 -0400

Using Microsoft Powershell, you can quickly search for text in files:

Get-ChildItem –recurse | where {$_.extension –eq “.cs”} | foreach ($_) {Select-String StringToSearchFor $_}

Theexample above recurses through all folders below the current one, checking to see if the extension is *.cs (C# Code Files), and looking for the text "StringToSearchFor".

Tutorial: Installing SSL in IIS on Windows Server 2008

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Mon, 16 Aug 2010 20:05:00 -0400

I'm surprised at the changes I'm still discovering in IIS 7.5 in Windows Server 2008. I've managed quite a few Windows Server 2003 boxes over the years, and now I'm exclusively on 2008 boxes. One of the recent surprises I ran into was just how different it is to set up SSL on a hosted site in IIS 7.5 . This was not exactly a simple task on previous versions of IIS, but it had become comfortable if only by rote.

Getting close is intuitive enough -- open up Information Services Manager, and click on the site you want to work on. You should see an SSL settings icon, but if you click on it, you'll get a surprising message:

"The site does not have a secure binding (HTTPS) and cannot accept SSL connections."

While this seems like an annoyance, it's actually a very powerful addition to the functionality that can be hosted from IIS. You want to handle FTP? Add an FTP binding and a managed assembly. Your own proprietary protocol? Go for it!

Back to the issue at hand, how to enable secure binding for a website. All you need to do is right click the site to edit, and choose "Edit Bindings". From here, choose "Add", and select "HTTPS" from the list, assign to the IPs du jour, and pick your certificate. I won't get into certificate generation / purchasing in this article, but there's a self-signed one installed by default that can be used provided a browser warning is not too intrusive for your purposes.

Keep in mind that for a client-facing site, you should always have a legitimate signed certificate to maintain a professional atmosphere.

Once this is done, you can optionally go back to the SSL settings, and force a site to only function over SSL if that's your ideal scenario.

Your site should now be functioning over SSL.

Predictive Analytics -- Data Mining

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Wed, 11 Aug 2010 19:55:00 -0400

After some frenetic development at my new position, I've had the opportunity to try out some new technology (read that as 'new to me', rather than new in general): Sql Server 2008 Data Mining. I'll be sharing quite a lot of perspective on this in upcoming posts, with some practical examples where possible.

One thing that's surprised me so far is how small the community seems to be around this. It doesn't seem to be very well publicized, even withing Sql circles. Most of the documentation that's out there are from members of the Sql Server development team at Microsoft that have come up with the server implementation.

After having some hands on, there's plenty to love and quite a bit of frustration, however I believe ultimately that most developers and information workers will find a great deal of value in this little-hyped tool.

Here's some trivia that gives you a picture about what it takes to learn how to effectively use this:

1) Data Mining is tightly integrated as a part of SSAS -- Sql Server Analysis Server (with all that goes with that -- Windows Auth only included)

2. Data Mining queries (predictions) are done using DMX queries (not to be confused with MDX queries for cubes, although you can blend a statement with a DMX query and an MDX subselect)

3) Data Mining started out as an ODBC set of extensions -- literally Data Mining eXtensions for ODBC (aka DMX)

4) Mining Models and Structures are stored under the covers as cube-ish metadata. Don't believe me? Check out the tooling, and note how attributes are made... there's a lot of similarities there.

5) The Data Mining add-on for office is really something that should be exploited -- essentially making an on-the-fly cube based on a table in excel, and then having the ability to create live trending forecasts / clustering / market basket associations? Sign me up!

6) The tooling is inconsistent, depending on the type of data you're working with. If you predict nested details, you lose some tooling functionality; if you only predict master-level fields, you lose other functionality.

7) Although the technology has been out since at least Sql Server 2000, the adoption rate looks low, or people aren't sharing their learned lessons with the community at large. I've gathered the blog feeds I've found so far here: http://www.google.com/reader/bundle/user%2F09775912853343203303%2Fbundle%2FSqlServerDataMining

8) This book is invaluable -- there's learned lessons and documentation here that can't even be found in the Books Online documentation in Sql Server. Data Mining with Microsoft Sql Server 2008 by Jamie McLennan, ZhaoHui Tang, and Bogdan Crivat.

Fixing Home Server VM Restore for Win7

jnadal.nospam@nospam.jnadal.com (Jason Nadal) — Thu, 10 Jun 2010 21:52:00 -0400

I've been trying to restore my development virtual machine in order to test out some apps, and found out the vmware image must have gotten corrupted. So Windows Home Server backs up all my machines, but in this case wouldn't restore all that well -- yielding a no-boot scenario. So after replacing the Home Server Restore disk with the Windows 7 install disk and booting, I expected to just adjust the boot partition location in BOOT.INI, but was surprised to see that had been done away with.

In it's place is a new command line tool "BOOTREC", but there's requirements on it actually being able to poll the active drive paritions ("active" being the key here).

This link shows how to fix the boot partition for Windows 7. Notice the difference between the legacy use of the hidden, system, readonly BOOT.INI with a command line app.

http://social.technet.microsoft.com/Forums/en/w7itproinstall/thread/7791044e-db7f-4144-a96c-945299811f58

Now for me, the initial bootrec command wouldn't let me rebuild the boot configuration, instead giving an "element not found" error. Insead, the DISKPART instructions in the post about setting the active disk partition allowed me to get to an BOOTMGR not found error.

Now Kevon Walcott's solution didn't work on it's own for me, but was the first step:

Diskpart

LIST DISK

SELECT DISK (followed by the number of the disk . most likely 0)

LIST PARTITION

SELECT PARTITION (followed by your partition number. most likely 0)

ACTIVE

EXIT

Second step was from Vivian Xing [MSFT], and rebuilt the boot config for the drive itself and the boot list:

Bcdboot C:\windows

NOTE: Replace C with the drive letter where the system is installed.

BOOTREC /FIXMBR

BOOTREC /FIXBOOT

BOOTREC /REBUILDBCD

BOOTREC /SCANOS

Rebooting gave me a "BOOTMGR is missing" error. Bouncing the machine a 3rd time, and using the windows system recovery option to repair the installation at least let me try. However so far, the error "BOOTMGR is missing" is still present. So far, Win7: 1, Me:0.

Restarting the VM and again rebooting into windows system recovery gave me the message that the disk I was trying to repair with was not the right media for the installation (note: took a bit to realize the installation was Win7 Pro x64, and I had Win7 Ultimate media in the drive).

Putting the right media in this time, and now it's just saying Windows. Use the command prompt, bcdboot c:\windows AGAIN, and choose startup repair again. More repeated failures, and another restore from home server -- this time adjusting the partitions in Disk Manager in the restore window.