Mob's Dev Blog

FubuMVC Automatic Binding of Request Properties

2012-02-08T15:03:00.001-08:00

One very nifty feature of FubuMVC is the automatic binding of request properties to your input model. From what I can tell, any item in the AspNetAgregateDictionary will be bound if the name/type of your input model property match. So for example, the following input model would have the user agent bound to the UserAgent property, talk about loose coupling:

public class InputModel
{
    public string UserAgent { get; set; }
}

The following items can be bound to your model:

Name	Type
AcceptTypes	string[]
ApplicationPath	string
AppRelativeCurrentExecutionFilePath	string
Browser	System.Web.HttpBrowserCompatibilityBase
ClientCertificate	System.Web.HttpClientCertificate
ContentEncoding	System.Text.Encoding
ContentLength	int
ContentType	string
Cookies	System.Web.HttpCookieCollection
CurrentExecutionFilePath	string
FilePath	string
Files	System.Web.HttpFileCollectionBase
Filter	System.IO.Stream
Form	System.Collections.Specialized.NameValueCollection
Headers	System.Collections.Specialized.NameValueCollection
HttpMethod	string
IsAuthenticated	bool
IsLocal	bool
IsSecureConnection	bool
LogonUserIdentity	system.Security.Principal.WindowsIdentity
Params	System.Collections.Specialized.NameValueCollection
Path	string
PathInfo	string
PhysicalApplicationPath	string
PhysicalPath	string
QueryString	System.Collections.Specialized.NameValueCollection
RawUrl	string
RequestType	string
ServerVariables	System.Collections.Specialized.NameValueCollection
TotalBytes	int
Url	Uri
UrlReferrer	Uri
UserAgent	string
UserHostAddress	string
UserHostName	string
UserLanguages	string[]

Adding Build Time and Error Details to TeamCity Email Notifications

2012-02-06T10:20:00.001-08:00

After setting up TeamCity I wanted to add some more information to the notification email. TeamCity allows you to customize all notifications (Not just email) through templates. The following resources were helpful in figuring out how to do this:

TeamCity: Customizing Notifications Templates

TeamCity: Server-side Object Model

FreeMarker Manual (Templating engine)

First I wanted to add build time to the subject of successful build notifications so that we could easily monitor it:

build_successful.ftl:

<#global subject>${project.name} ${buildType.name} v${build.buildNumber} Succeeded

(${((build.duration / 60) % 60)}:${((build.duration) % 60)?string?left_pad(2, "0")})</#global>

This will give you a subject along the lines of:

Frogger Stable CI v1.0.0.331 Succeeded (11:58)

When a build fails, TeamCity will include failed test results in the email but not other failure information. I wanted to add the failure summary (Like the one seen on the build summary page) in every failed build notification:

build_failed.ftl

<#global bodyHtml>
    ...
    <div style="color:red">
        <#list build.buildLog.errorMessages as message>
            ${message.text?replace("\n", "\lbr/\g")}<br/>
        </#list>
    </div>
    ...
</#global>

This will give you a notification along the lines of:

Build Frogger::Stable CI #1.0.0.336 failed
Agent: ENDOR1

bt3
Execute build_projects_ui
RuntimeError: MSBuild Failed. See Build Log For Detail

Stacktrace:
C:/.../gems/albacore-0.2.7/lib/albacore/support/failure.rb:12:in `fail_with_message'
... stack trace ...
C:/Program Files/TeamCity/BuildAgent1/plugins/rake-runner/lib/rb/runner/rakerunner.rb:40:in `'
Rake aborted!

Changes included (2 changes).
Change f735f1f299dc10005d8cc33d80003ea6915cbfa4 by mobrien (2 files): Added frog.
Change 4b023210511568f5159a28e493ec01134285c363 by mobrien (1 file): Added cars.

MVC vs MVP

2012-01-31T09:30:00.001-08:00

After working on the backend for so long I’m having a bit of a frontend renaissance. One area of confusion is the difference between MVC and MVP. There is a lot of conflicting/confusing information out there. I noticed that many of the posts and comments get it partially right which is more confusing than getting it completely wrong. So after trudging through many posts, the ones linked below seem to explain the difference accurately and clearly.

TL;DR, here was my take, for better or worse:

…had a view/controller pair for each element on the screen (textbox, button, etc).
…the controller handled raw user input and decided if it was relevant to its view (i.e there was a mouse click or keystroke, was it within the coordinates of my view? Yes, I’ll handle it; no, I’ll ignore it).
…the controller did not manipulate the view. It just handled user input and updated the model…period.
…the view observes the model and updates itself.

…has one view/presenter for multiple widgets, perhaps an entire screen.
…the presenter handles high level input that is passed along by widgets on the screen.
…the presenter manipulates the view…

supervising controllers manipulate the view only in complex scenarios but let view update itself as much as possible, either by data binding to the model or observing the model.
passive views are manipulated only by the presenter and they have no knowledge of the model. Changes in the model are strictly observed by the presenter which in turn manipulates the model.

Comparison

Controllers in MVC were focused on processing low level, raw input (Which was a lot of work), updating the model was a by product. Handling of low level, raw input is no longer necessary these days as it is handled by widgets; widgets then fire higher level events (Like button click instead of mouse click). So since the job of handling of low level input is now on the widgets, the MVP presenters are focused on updating the model in response to higher level events.
Controller/view pairs in classic MVC were responsible for one “dumb” UI element. The view and presenter in MVP are responsible for an ensemble of “smart” widgets.
MVC controllers did not manipulate the view whereas MVP presenters do (By varying degrees with supervising controller and passive view).

To me the biggest difference between the two is the relationship between the view and controller/presenter. The rest seems less relevant, especially now-a-days; who cares if the controller is handling low level or high level events or if it is handling one UI element or many widgets. These seem to simply be a function of the available technology. In MVC the controller does not manipulate the view, the view only updates itself as a result of changes in the model. In MVP the presenter manipulates the view to varying degrees. That to me is the real conceptual difference that has relevance today. I think it also explains why MVC is not used today in the UI; it’s just not practical to restrict the controller from manipulating the view (Although the presentation model & mvvm attempt to successfully do this). That restriction would make it difficult to handle and test non-trivial UI logic. It also explains why MVC (flavored a bit differently) is used in many server side web frameworks as this separation between view and controller is practical and valuable in a stateless environment.

History/differences:

http://aviadezra.blogspot.com/2008/06/mvc-model-view-controller-design.html

http://aviadezra.blogspot.com/2007/07/twisting-mvp-triad-say-hello-to-mvpc.html

http://martinfowler.com/eaaDev/uiArchs.html

http://lostechies.com/derekgreer/2007/08/25/interactive-application-architecture/

Definitions:

http://martinfowler.com/eaaCatalog/modelViewController.html (Just a stub with the structure and a pointer to POEAA where you can read more)

http://martinfowler.com/eaaDev/ModelViewPresenter.html

http://martinfowler.com/eaaDev/SupervisingPresenter.html

http://martinfowler.com/eaaDev/PassiveScreen.html

http://martinfowler.com/eaaDev/PresentationModel.html (Good to understand in comparison to the other patterns)

JavaScript Prototyping Reference

2011-12-20T09:30:00.001-08:00

I’ve had problems remembering what is actually happening when working with constructor functions and prototyping. I created the following table to demonstrate what's going on. If I’m missing something please let me know as I’m not a JavaScript ninja by any means.

A couple of things to note:

The __proto__ property is deprecated and non standard. I include it only as a way to indicate an object’s prototype.
I included the underscore.js extend function only to show how it compares to protypal inheritance in JS or CS. I think the name of this function is not clear.
The backbone.js extend method is functionally equivalent to the CoffeeScript extends operator.

JavaScript
var x = function() {}	x = { // Only meaningful for constructors not functions. prototype: { constructor: x, __proto__: Object.prototype }, __proto__: Function.prototype }
var Y = function(name) { // In a constructor 'this' refers to the // new object. this.name = name; } Y.prototype.printName = function() { // In a method, 'this' refers to the // declaring object (Unless overridden). console.log(this.name); } var y = new Y('Fritz London');	Y = { // This will be set as the new objects prototype. prototype: { constructor: Y, __proto__: Object.prototype, printName: function() { console.log(this.name); } }, __proto__: Function.prototype } y = { __proto__: Y.prototype, name: 'Fritz London' }
var z = { name: 'Richard Feynman' }	z = { __proto__: Object.prototype, name: 'Richard Feynman' }
var a = { name: 'Werner Heisenberg' } var createObject = function(prototype) { var F = function() {}; F.prototype = prototype; return new F(); } var b = createObject(a); // Or the built in var b = Object.create(a);	a = { __proto__: Object.prototype, name: 'Werner Heisenberg' } F = { prototype: a, __proto__: Function.prototype } b = { __proto__: a }
CoffeeScript
class Parent constructor: -> @id = ++Parent.identity name: 'Niels Bohr' @identity: 0	function Parent() { this.id = ++Parent.identity; } Parent.prototype.name = 'Niels Bohr'; Parent.identity = 0; --- Parent = { prototype: { constructor: Parent, __proto__: Object.prototype, name: 'Niels Bohr' }, __proto__: Function.prototype, identity: 0 }
parent = new Parent()	parent = { __proto__: Parent.prototype, id: 1 }
class Child extends Parent url: 'http://www.google.com'	__extends(Child, Parent); function Child() { Child.__super__.constructor.apply(this, arguments); } Child.prototype.url = 'http://www.google.com'; --- Child = { prototype: { constructor: Child, __proto__: Parent, url: 'http://www.google.com' }, __proto__: Function.prototype, // CoffeeScript generated property __super__: Parent.prototype }
child = new Child()	child = { __proto__: Child.prototype, id: 1 }
backbone.js
// [backbone.js].extend(protoProps, classProps) var Person = Backbone.Model.extend({ constructor: function() { this.id = ++Person.identity; }, name: 'Niels Bohr' }, { identity: 0 });	Person = { prototype: { // If a constructor property is not // specified, a new function is used // that calls the parent constructor // in the context of the new object. constructor: protoProps.constructor, __proto__: Backbone.Model.prototype, name: 'Niels Bohr' }, __proto__: Function.prototype, // Backbone added property __super__: Backbone.Model.prototype // Backbone attaches this to new objects extend: Backbone.Model.extend, identity: 0 }
underscore.js
var person = { id: 5 } var a = { name: 'Werner Heisenberg' } var b = { occupation: 'Physicist' } _.extend(person, a, b);	person = { id: 5, name: 'Werner Heisenberg', occupation: 'Physicist' }

Dissecting the output of the CoffeeScript class and extends keyword

2011-12-19T19:33:00.001-08:00

CoffeeScript has a convenient way of emulating class and inheritance semantics via the class and extends keyword. I wanted to better understand what was happening under the covers so the following is an analysis of the transpiler output. We’ll look at a simple relationship between two CS “classes”:

class Parent
    constructor: ->
        @id = ++Parent.identity
    @identity: 0

class Child extends Parent
    url: 'http://www.google.com'

CS outputs the following (As of 1.2):

var Child, Parent;
var __hasProp = Object.prototype.hasOwnProperty, 
    __extends = function(child, parent) {/* discussed later */};

Parent = (function() {
    function Parent() {
        this.id = ++Parent.identity;
    }
    Parent.identity = 0;
    return Parent;
})();

Child = (function() {
    __extends(Child, Parent);
    function Child() {
        Child.__super__.constructor.apply(this, arguments);
    }
    Child.prototype.url = 'http://www.google.com';
    return Child;
)();

Some things I noted:

The CS transpiler builds the constructor functions in IIFE’s so we don’t clutter up the global object.
The CS “constructor” property is the body of the constructor function.
The CS @ prefix…
- … refers to the new object only when used in the constructor.
- … refers to the function object outside of the constructor (to declare “static” properties).
Properties not prefixed with the CS @ are prototype properties.
The “parent” constructor function can be executed in the context of the new child object. This happens automatically (As shown above) if the child does not explicitly declare a constructor. If the child object does supply a constructor (And you want to call the parent constructor) you will need to manually call super(…) in the child constructor with zero or more arguments.

Now lets take a look at the __extends method that is generated by CS. This method sets up the prototype chain and a property that links to the “parent” prototype.

var __hasProp = Object.prototype.hasOwnProperty;
var __extends = function(child, parent) { 
    for (var key in parent) {
         if (__hasProp.call(parent, key)) 
            child[key] = parent[key]; 
    } 
    function ctor() { 
        this.constructor = child; 
    } 
    ctor.prototype = parent.prototype; 
    child.prototype = new ctor; 
    child.__super__ = parent.prototype; 
    return child; 
};

Some things I noted:

All constructor function properties (Or “static” properties) are copied from the “parent” to the “child” constructor function.
An intermediate prototype object is created so that you can augment the “child’s” prototype without changing the “parent’s” prototype.

It’s prototype is the “parent” prototype.
It serves as the prototype of “child” objects.
It’s constructor is set to the “child” constructor function.

A special property called __super__ is created on the “child” constructor function. Evidentially CS likes to be backwards compatible as it doesn't make use of the relatively new Object.getPrototypeOf method. Calling super() in the CS constructor, as noted above, will make use of this property to call the “parent” constructor function.

From this we can see that CS sets up the prototype chain like so:

One more thing I was interested in was how the CS extends keyword compares to the underscore.js method of a similar name:

_.extend = function(obj) {
    each(slice.call(arguments, 1), function(source) {
        for (var prop in source) {
            if (source[prop] !== void 0) obj[prop] = source[prop];
        }
    });
    return obj;
};

Clearly they are very different. The underscore extend method basically does a mixin as it simply copies properties from the source objects to the target object.

Prototyping in Lua

2011-12-19T12:16:00.001-08:00

One of the hardest things to get my head around in JavaScript (Besides “this”) was prototyping. I wasn't sure if it was prototyping itself that was hard to grok (It seemed amazingly simple to me in principle) or if it was how JavaScript implemented it. I thought it would be an interesting exercise to explore how another prototypal language worked to see where the difficulty lay. So let’s take a look at how Lua implements prototyping (Disclaimer: I only have a passing knowledge of Lua so please excuse my naïveté).

Lua “tables” form the basis of objects. They are essentially hash tables, à la JavaScript objects. Prototyping in Lua is enabled by two actions; first setting the objects’ metatable (Which can be any table) and second, setting the prototype field in that metatable (Which is the __index field). Here is how it works (NB: see here about the use of colon vs. dot. Compare it to call/apply in JavaScript):

-- Create the prototype
Greeting = { hello = 'hello' }

function Greeting:say_hello()
    print(self.hello)
end

-- Create an object
lolcat = { hello = 'hai' }

-- Set the meta table (A table where metadata is held), in this case we’ll just set it to be the object itself
setmetatable(lolcat, lolcat)

-- Set the prototype
lolcat.__index = Greeting

Other than the setting of the metatable I’d have to say this is incredibly straight forward. You can encapsulate the creation logic into a constructor which simplifies the process:

-- Constructor
function Greeting:new(o)
    o = o or {}
    o.__index = self
    return setmetatable(o, o)
end

lolcat = Greeting:new{ hello = 'hai' }

-- https://github.com/silentbicycle/lunatest
assert_equal('hai', lolcat.hello)

Although prototyping still presents a mental shift for me the Lua approach feels much cleaner and simpler. I find the dot/colon approach (mentioned above) much cleaner as well.

Welcome to the FubuMVC Cafeteria

2011-06-25T17:37:00.001-07:00

I was wondering what FubuMVC had wired up in the IoC container out-of-the-box and it happens to be quite a bit. Here is a dump of the container configuration (Using the spark view engine):

PluginType	Description
Bottles.IActivator	FubuMVC.Spark.SparkActivator, FubuMVC.Spark FubuMVC.Core.Packaging.PackageFileActivator, FubuMVC.Core FubuMVC.Core.Content.ScriptGraphConfigurationActivator, FubuMVC.Core
FubuCore.Binding.AggregateDictionary	FubuCore.Binding.AggregateDictionary
FubuCore.Binding.IBindingContext	FubuCore.Binding.BindingContext, FubuCore
FubuCore.Binding.ICollectionTypeProvider	FubuCore.Binding.DefaultCollectionTypeProvider, FubuCore
FubuCore.Binding.IConverterFamily
FubuCore.Binding.IModelBinder
FubuCore.Binding.IModelBinderCache	FubuCore.Binding.ModelBinderCache, FubuCore
FubuCore.Binding.IObjectResolver	FubuMVC.Core.Diagnostics.Tracing.RecordingObjectResolver, FubuMVC.Core
FubuCore.Binding.IPropertyBinder
FubuCore.Binding.IPropertyBinderCache	FubuCore.Binding.PropertyBinderCache, FubuCore
FubuCore.Binding.IRequestData	FubuMVC.Core.Diagnostics.Tracing.RecordingRequestData, FubuMVC.Core
FubuCore.Binding.ISmartRequest	FubuMVC.Core.Runtime.FubuSmartRequest, FubuMVC.Core
FubuCore.Binding.IValueConverterRegistry	FubuCore.Binding.ValueConverterRegistry, FubuCore
FubuCore.Binding.ObjectResolver	FubuCore.Binding.ObjectResolver, FubuCore
FubuCore.IDisplayFormatter	FubuCore.DisplayFormatter, FubuCore
FubuCore.IFileSystem	FubuCore.FileSystem, FubuCore
FubuCore.IObjectConverter	FubuCore.ObjectConverter, FubuCore
FubuCore.ITypeResolver	FubuCore.TypeResolver, FubuCore
FubuCore.Reflection.ITypeDescriptorCache	FubuCore.Reflection.TypeDescriptorCache, FubuCore
FubuCore.Reflection.TypeDescriptorCache	FubuCore.Reflection.TypeDescriptorCache
FubuMVC.Core.Behaviors.IJsonReader	FubuMVC.Core.Behaviors.JavaScriptJsonReader, FubuMVC.Core
FubuMVC.Core.Conneg.IFormatter	FubuMVC.Core.Conneg.JsonFormatter, FubuMVC.Core FubuMVC.Core.Conneg.XmlFormatter, FubuMVC.Core
FubuMVC.Core.Conneg.IMediaProcessor`1<T>	FubuMVC.Core.Conneg.MediaProcessor`1, FubuMVC.Core
FubuMVC.Core.Content.IContentFolderService	FubuMVC.Core.Content.ContentFolderService
FubuMVC.Core.Content.IContentRegistry	FubuMVC.Core.Content.ContentRegistryCache, FubuMVC.Core
FubuMVC.Core.Content.IMimeTypeProvider	FubuMVC.Core.Content.DefaultMimeTypeProvider
FubuMVC.Core.CurrentRequest	Instance is created by Func<object> function:System.Func`2[StructureMap.IContext,FubuMVC.Core.CurrentRequest]
FubuMVC.Core.Diagnostics.DiagnosticsIndicator	FubuMVC.Core.Diagnostics.DiagnosticsIndicator
FubuMVC.Core.Diagnostics.IDebugDetector	FubuMVC.Core.Diagnostics.DebugDetector, FubuMVC.Core
FubuMVC.Core.Diagnostics.IDebugReport	FubuMVC.Core.Diagnostics.DebugReport, FubuMVC.Core
FubuMVC.Core.Diagnostics.IRequestHistoryCache	FubuMVC.Core.Diagnostics.RequestHistoryCache, FubuMVC.Core
FubuMVC.Core.Diagnostics.Tracing.BehaviorTracer	FubuMVC.Core.Diagnostics.Tracing.BehaviorTracer, FubuMVC.Core
FubuMVC.Core.Diagnostics.Tracing.DiagnosticBehavior	FubuMVC.Core.Diagnostics.Tracing.DiagnosticBehavior, FubuMVC.Core
FubuMVC.Core.IEndpointService	FubuMVC.Core.EndpointService, FubuMVC.Core
FubuMVC.Core.Packaging.IPackageFiles	FubuMVC.Core.Packaging.PackageFilesCache, FubuMVC.Core
FubuMVC.Core.Registration.BehaviorGraph	FubuMVC.Core.Registration.BehaviorGraph
FubuMVC.Core.Registration.Querying.IChainResolver	FubuMVC.Core.Registration.Querying.ChainResolver, FubuMVC.Core
FubuMVC.Core.Routing.IRoutePolicy	FubuMVC.Core.Routing.StandardRoutePolicy, FubuMVC.Core
FubuMVC.Core.Runtime.IBehaviorFactory	FubuMVC.StructureMap.PartialBehaviorFactory, FubuMVC.StructureMap
FubuMVC.Core.Runtime.IFubuRequest	FubuMVC.Core.Diagnostics.Tracing.RecordingFubuRequest, FubuMVC.Core
FubuMVC.Core.Runtime.IJsonWriter	FubuMVC.Core.Runtime.JsonWriter, FubuMVC.Core
FubuMVC.Core.Runtime.IOutputWriter	FubuMVC.Core.Diagnostics.RecordingOutputWriter, FubuMVC.Core
FubuMVC.Core.Runtime.IPartialFactory	FubuMVC.Core.Runtime.PartialFactory, FubuMVC.Core
FubuMVC.Core.Runtime.ISessionState	FubuMVC.Core.Runtime.SimpleSessionState, FubuMVC.Core
FubuMVC.Core.Runtime.IStreamingData	FubuMVC.Core.Runtime.StreamingData, FubuMVC.Core
FubuMVC.Core.Security.IAuthenticationContext	FubuMVC.Core.Web.Security.WebAuthenticationContext, FubuMVC.Core
FubuMVC.Core.Security.IAuthorizationFailureHandler	FubuMVC.Core.Security.DefaultAuthorizationFailureHandler, FubuMVC.Core
FubuMVC.Core.Security.IAuthorizationPolicyExecutor	FubuMVC.Core.Diagnostics.Tracing.RecordingAuthorizationPolicyExecutor, FubuMVC.Core
FubuMVC.Core.Security.IAuthorizationPreviewService	FubuMVC.Core.Security.AuthorizationPreviewService, FubuMVC.Core
FubuMVC.Core.Security.IChainAuthorizor	FubuMVC.Core.Security.ChainAuthorizor, FubuMVC.Core
FubuMVC.Core.Security.IEndPointAuthorizor	FubuMVC.Core.Security.NulloEndPointAuthorizor
FubuMVC.Core.Security.IEndPointAuthorizorFactory	FubuMVC.Core.Security.EndPointAuthorizorFactory, FubuMVC.Core
FubuMVC.Core.Security.ISecurityContext	FubuMVC.Core.Web.Security.WebSecurityContext, FubuMVC.Core
FubuMVC.Core.SessionState.IFlash	FubuMVC.Core.SessionState.FlashProvider, FubuMVC.Core
FubuMVC.Core.SessionState.IRequestDataProvider	FubuMVC.Core.SessionState.RequestDataProvider, FubuMVC.Core
FubuMVC.Core.UI.Configuration.IElementNamingConvention	FubuMVC.Core.UI.Configuration.DefaultElementNamingConvention, FubuMVC.Core
FubuMVC.Core.UI.ICssLinkTagWriter	FubuMVC.Core.UI.CssLinkTagWriter, FubuMVC.Core
FubuMVC.Core.UI.IPartialInvoker	FubuMVC.Core.UI.PartialInvoker, FubuMVC.Core
FubuMVC.Core.UI.Scripts.IScriptTagWriter	FubuMVC.Core.UI.Scripts.BasicScriptTagWriter, FubuMVC.Core
FubuMVC.Core.UI.Scripts.ScriptGraph	FubuMVC.Core.UI.Scripts.ScriptGraph
FubuMVC.Core.UI.Security.IFieldAccessRightsExecutor	FubuMVC.Core.UI.Diagnostics.RecordingFieldAccessRightsExecutor, FubuMVC.Core
FubuMVC.Core.UI.Security.IFieldAccessService	FubuMVC.Core.UI.Security.FieldAccessService, FubuMVC.Core
FubuMVC.Core.UI.Tags.ITagGenerator`1<T>	FubuMVC.Core.UI.Tags.TagGenerator`1, FubuMVC.Core
FubuMVC.Core.UI.Tags.TagProfileLibrary	FubuMVC.Core.UI.Tags.TagProfileLibrary
FubuMVC.Core.Urls.IUrlRegistry	FubuMVC.Core.Urls.UrlRegistry, FubuMVC.Core
FubuMVC.Core.Urls.IUrlTemplatePattern	FubuMVC.Core.Urls.NulloUrlTemplate, FubuMVC.Core
FubuMVC.Core.View.Activation.IPageActivationRules	FubuMVC.Core.View.Activation.PageActivationRuleCache, FubuMVC.Core
FubuMVC.Core.View.Activation.IPageActivationSource
FubuMVC.Core.View.Activation.IPageActivator	FubuMVC.Core.View.Activation.PageActivator, FubuMVC.Core
FubuMVC.Core.View.Activation.PageActivationRuleCache	FubuMVC.Core.View.Activation.PageActivationRuleCache, FubuMVC.Core
FubuMVC.Spark.Rendering.IRenderStrategy	FubuMVC.Spark.Rendering.NestedRenderStrategy, FubuMVC.Spark FubuMVC.Spark.Rendering.AjaxRenderStrategy, FubuMVC.Spark FubuMVC.Spark.Rendering.DefaultRenderStrategy, FubuMVC.Spark
FubuMVC.Spark.Rendering.IViewEntryProvider	FubuMVC.Spark.Rendering.ViewEntryProviderCache, FubuMVC.Spark
FubuMVC.Spark.Rendering.IViewModifier	FubuMVC.Spark.Rendering.PageActivation, FubuMVC.Spark FubuMVC.Spark.Rendering.SiteResourceAttacher, FubuMVC.Spark FubuMVC.Spark.Rendering.ContentActivation, FubuMVC.Spark FubuMVC.Spark.Rendering.OnceTableActivation, FubuMVC.Spark FubuMVC.Spark.Rendering.OuterViewOutputActivator, FubuMVC.Spark FubuMVC.Spark.Rendering.NestedViewOutputActivator, FubuMVC.Spark FubuMVC.Spark.Rendering.ViewContentDisposer, FubuMVC.Spark FubuMVC.Spark.Rendering.NestedOutputActivation, FubuMVC.Spark
FubuMVC.Spark.Rendering.IViewModifierService	FubuMVC.Spark.Rendering.ViewModifierService, FubuMVC.Spark
FubuMVC.Spark.Rendering.NestedOutput	FubuMVC.Spark.Rendering.NestedOutput, FubuMVC.Spark
FubuMVC.Spark.Rendering.ViewOutput	FubuMVC.Spark.Rendering.ViewOutput, FubuMVC.Spark
FubuMVC.Spark.SparkModel.ITemplateRegistry	FubuMVC.Spark.SparkModel.TemplateRegistry
Microsoft.Practices.ServiceLocation.IServiceLocator	FubuMVC.StructureMap.StructureMapServiceLocator, FubuMVC.StructureMap
Spark.ISparkViewEngine	Spark.SparkViewEngine
StructureMap.IContainer	StructureMap.Container
System.Web.HttpContextBase	System.Web.HttpContextWrapper, System.Web
System.Web.HttpRequestWrapper	Instance is created by Func<object> function:System.Func`2[StructureMap.IContext,System.Web.HttpRequestWrapper]

Building a Better MVC Filter Attribute Provider

2011-04-20T18:39:00.001-07:00

As I mentioned here the filter attribute design in MVC is terrible. That other post explains why so I’m not going to rehash. This post is about how we can do it better. I don’t really have anything against using the attributes to annotate an action or controller it’s mainly that MVC makes the attribute itself the filter. What if, instead, the attribute was simply a marker that said that a particular filter should be applied and allowed you to customize it:

public interface IFilterAttribute
{
    Type FilterType { get; }
    void InitializeFilter(object filter);
}

This interface specifies the filter type then a method to initialize the filter. We could then implement this interface on an attribute like so:

public class SecureAttribute : Attribute, IFilterAttribute
{
    private readonly bool _enabled;

    public SecureAttribute(bool enabled)
    {
        _enabled = enabled;
    }

    public Type FilterType { get { return typeof(SecureFilter); } }

    public void InitializeFilter(object filter)
    {
        ((SecureFilter) filter).Enabled = _enabled;
    }
}

This attribute allows us to turn security on and off on an action or controller. You can see that it specifies the type and initialization of the filter. The filter is nothing special; in this case it implements IMvcFilter so it can set the multiplicity (In this case we only want one filter to be applied) and IAuthorizationFilter for authentication. It is also setup to be constructor injected:

public class SecureFilter : IAuthorizationFilter, IMvcFilter
{
    private readonly IAuthenticationService _authenticationService;

    public bool Enabled { get; set; }

    public SecureFilter(IAuthenticationService authenticationService)
    {
        _authenticationService = authenticationService;
        Enabled = true;
    }

    public bool AllowMultiple { get { return false; } }
    public int Order { get; set; }

    public void OnAuthorization(AuthorizationContext filterContext)
    {
        if (Enabled && !_authenticationService.Authenticate())
        {
            // Access denied
        }
    }
}

Now our filter provider can look for this new interface, IFilterAttribute and create the filters from the information provided from the attribute:

public class FilterAttributeProvider : IFilterProvider
{
    private static readonly Dictionary<string, IEnumerable<FilterAttributeMetadata>> AttributeCache =
        new Dictionary<string, IEnumerable<FilterAttributeMetadata>>();

    private readonly Func<Type, object> _filterFactory;

    public FilterAttributeProvider(Func<Type, object> filterFactory)
    {
        _filterFactory = filterFactory;
    }

    public IEnumerable<Filter> GetFilters(ControllerContext controllerContext, ActionDescriptor actionDescriptor)
    {
        return GetFilterAttributes(actionDescriptor).Select(x => CreateFilter(x, _filterFactory));
    }

    private static IEnumerable<FilterAttributeMetadata> GetFilterAttributes(ActionDescriptor actionDescriptor)
    {
        IEnumerable<FilterAttributeMetadata> attributes;
        if (!AttributeCache.TryGetValue(actionDescriptor.UniqueId, out attributes))
        {
            attributes = LoadFilterAttributes(actionDescriptor);
            lock (AttributeCache) AttributeCache.Add(actionDescriptor.UniqueId, attributes);
        }
        return attributes;
    }

    private static IEnumerable<FilterAttributeMetadata> LoadFilterAttributes(ActionDescriptor actionDescriptor)
    {
        return actionDescriptor.GetCustomAttributes(typeof(IFilterAttribute), true).Cast<IFilterAttribute>().
                                Select(x => new FilterAttributeMetadata(x, FilterScope.Action)).
                   Union(actionDescriptor.ControllerDescriptor.GetCustomAttributes(typeof(IFilterAttribute), true).
                                Cast<IFilterAttribute>().
                                Select(x => new FilterAttributeMetadata(x, FilterScope.Controller)));
    }

    private static Filter CreateFilter(FilterAttributeMetadata filterMetadata, 
                                       Func<Type, object> filterFactory)
    {
        var filter = filterFactory(filterMetadata.Attribute.FilterType);
        filterMetadata.Attribute.InitializeFilter(filter);
        return new Filter(filter, filterMetadata.Scope, GetFilterOrder(filter));
    }

    private static int? GetFilterOrder(object filter)
    {
        var mvcFilter = filter as IMvcFilter;
        return mvcFilter != null ? mvcFilter.Order : (int?)null;
    }

    private class FilterAttributeMetadata
    {
        public FilterAttributeMetadata(IFilterAttribute attribute, FilterScope scope)
        { Attribute = attribute; Scope = scope; }

        public IFilterAttribute Attribute { get; private set; }
        public FilterScope Scope { get; private set; }
    }

    public interface IFilterAttribute
    {
        Type FilterType { get; }
        void InitializeFilter(object filter);
    }
}

Using the type on the attribute, the provider can create the filter with a factory passed into the constructor (Which would be our IoC container). Once the filter is created it is initialized by the InitializeFilter() method on the attribute. This completely decouples attributes from filters and allows us to constructor inject filters even when they are applied via attributes. So as an example lets say that we want to globally apply security but in some actions or controllers we want to turn it off so they are public. We can register the secure filter we defined above and our authentication service in our IoC container (In this example StructureMap):

public class MvcApplication : HttpApplication
{
    protected void Application_Start()
    {
        ObjectFactory.Configure(x =>
                                    {
                                        x.For<IAuthenticationService>().Use<AuthenticationService>();
                                        x.For<IAuthorizationFilter>().Use<SecureFilter>();
                                    });

        DependencyResolver.SetResolver(x => (x.IsAbstract || x.IsInterface) ?
                                                ObjectFactory.TryGetInstance(x) :
                                                ObjectFactory.GetInstance(x), 
                                       x => ObjectFactory.GetAllInstances(x).Cast<object>());

        FilterProviders.Providers.Add(new FilterAttributeProvider(ObjectFactory.GetInstance));
    }
}

Now our entire site requires authentication. For an action that we want public we can turn authentication off:

public class HomeController : Controller
{
    [Secure(false)]
    public ActionResult Index()
    {
        return View();
    }
}

We only allow one SecureFilter to be applied, so since the filter associated with the attribute has an “Action” scope it will take priority over the global filter which has a “Global” scope.

Creating an Overridable MVC3 Global Filter

2011-04-18T16:37:00.001-07:00

The new global filters in MVC3 are a nice way to apply behavior globally. Sometimes though you’ll want to opt out an action or controller from that behavior. Lets take authentication as an example; we want everything to be secure by default but we may have individual controllers or actions that will be public. The following authorize attribute enables us to create a global filter and then override the global behavior where we need to:

// Make sure we specify that this attribute should only be specified once.
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class SecureAttribute : AuthorizeAttribute
{
    private readonly bool _enabled;

    // Default constructor with security enabled, used as a global filter
    public SecureAttribute()
    {
        _enabled = true;
    }

    // Attribute constructor
    public SecureAttribute(bool enabled)
    {
        _enabled = enabled;
    }

    protected override bool AuthorizeCore(System.Web.HttpContextBase httpContext)
    {
        return !_enabled || AuthenticationService.Authenticate();
    }
}

A few things to note. First we need to specify that this attribute should only be applied once with the AttributeUsageAttribute. Under the covers the FilterAttribute class looks for this attribute to determine if multiple instances of the filter can exist. In the default constructor we specify the default behavior; in this case security being enabled. In our authorization method we can check to see if authentication is enabled and proceed accordingly.

Here we can register the global default. This filter will have a scope of “Global” which is a lower priority scope:

public class MvcApplication : HttpApplication
{
    protected void Application_Start()
    {
        GlobalFilters.Filters.Add(new SecureAttribute());
        // ...
    }
}

We can then override the default behavior where appropriate. The action attribute has a scope of “Action” which is a higher priority scope. So this filter will take precedence over the globally added one:

public class SomeController : Controller
{
    [Secure(false)]
    public ActionResult SomePublicAction()
    {
        return View();
    }
}

On Filters in MVC3

2011-04-15T16:41:00.001-07:00

I’ve been digging into the new filter functionality in MVC3 and this is a brain dump of what I’ve learned and some thoughts.

Filters currently must implement one or more of these interfaces: IActionFilter, IResultFilter, IExceptionFilter, and IAuthorizationFilter. The IMvcFilter interface allows you to tack on some additional metadata like the sort order and multiplicity. There are a couple of convenience classes that make creating filter attributes easier: FilterAttribute, ActionFilterAttribute, AuthorizeAttribute, HandleErrorAttribute and a few more. FilterAttribute implements IMvcFilter and so provides that metadata out of the box. When deriving from any of these attribute classes its important to understand that disallowing multiple instances of a filter is accomplished by applying the AttributeUsageAttribute to your derived class and setting AllowMultiple to false. The FilterAttribute class looks for that attribute to determine multiplicity and exposes it through the IMvcFilter.AllowMultiple property.

Originally filters (Which is a terrible name BTW. What do they filter??) were applied as .NET attributes to actions or controllers. At the beginning of the request, the controller and action filter attributes were pulled from the action method and controller class and run. One thing to note is that .NET attributes are only created by the CLR once (they are essentially singletons) so every time you reflect them in a particular app domain you will get the same object instances. I think the whole attribute approach for filters is badly implemented. One of the problems is that if you want to inject dependencies into an filter you have to rig up property injection. Additionally since they are attributes, and essentially singletons, they cannot hold any request specific state. This may be fine for something like authentication but difficult for transaction management as you will not be able to use IoC to inject request specific dependencies. I think a better design would have been to make the attributes more like WCF behaviors where the attribute itself is not the filter but simply acts as a way to create the filter. But that’s neither here nor there.

Now moving on to MVC3; things have changed a bit. We now have this concept of the IFilterProvider interface. This interface has one method, GetFilters() which as you can probably guess returns filters. Filter providers are now the source of all filters. There are three filter providers registered out of the box:

GlobalFilters.Filters: This is simply a place where you can ad-hoc register individual filters that will be applied globally. These should be stateless as they will be reused for each request.
FilterAttributeFilterProvider: This provider uses reflection to pull filters applied as attributes to controller classes and action methods; it basically does the job that the pre MVC3 code did. So the filters that this provider returns are not global, they are scoped to the controller and action. Since they are attributes they are essentially singletons and will be reused for each request so they should be stateless. Also keep in mind that the FilterAttributeFilterProvider will only discover attributes that derive from FilterAttribute. So deriving from Attribute and then implementing a filter interface will not get your attribute picked up by this provider.
ControllerInstanceFilterProvider: This provider allows the controller servicing the request to act as a filter (ee gads!).

MVC calls the FilterProviders.Providers.GetFilters() method, to aggregate all the filters from all the filter providers, for each individual request. You can also register your own filter providers by adding an instance of a type that implements IFilterProvider to the FilterProviders.Providers collection. FilterProviders wrap filters in the Filter class which adds some additional information such as the scope and order. You can override the order in the Filter class constructor if you are writing your own filter provider or extending an existing one. If you don’t pass in an order it will check and see if the filter implements IMvcFilter and get the order from there otherwise it defaults to -1. One interesting thing about the code that aggregates all these is that it removes duplicate filters for those filters that do not allow duplicates (Exposed by the IMvcFilter.AllowMultiple property). The way it determines which ones to remove is by sorting the filters by the order and then by the scope and preserving the first instance it encounters and rejecting the subsequent instances. This means that higher orders win and then within an order, narrower scopes will win (Which is actually a higher scope number).

This post demonstrates how to pull global filters from your favorite IoC container.

Building a Better MVC Dependency Resolver

2011-04-08T13:24:00.001-07:00

Updated to include global filters.

Unfortunately the latest effort to add DI to MVC falls short in a number of ways. There are 3 areas in particular that are causing me pain:

No distinction between infrastructure and a request.
No container per request.
No request container create and release mechanism.

In order to accommodate functionality specific to a request there would need to be a distinction between building up infrastructure related objects (created once at startup, e.g. IControllerFactory, IControllerActivator, etc.) and request related objects (created for each request, e.g. controllers, views, etc.) Unfortunately, all objects are built up from the same two methods regardless of if they’re infrastructure or request related. I use a StructureMap nested container per request. I want all dependencies for an individual request to be managed by that nested container. Also once the request completes I want to “clean up” the nested container (Like commit a transaction or whatever). Currently the dependency resolver offers no way to do this. Here is a better interface IMO:

public interface IDependencyResolver
{
    // Infrastructure
    object GetInfrastructureService(Type serviceType);
    IEnumerable<object> GetInfrastructureervices(Type serviceType);

    // Per-request
    object CreateActionContainer();
    object GetActionService(object container, Type serviceType);
    IEnumerable<object> GetActionServices(object container, Type serviceType);;
    IEnumerable<object> GetActionFilters(IContainer container, IEnumerable<Type> filterTypes);
    void ReleaseActionContainer(object container);
}

I’m sure its unlikely that we’ll ever see anything like this from the MVC team but we can put together something on top of the current interface that gets us there:

public abstract class DependencyResolverBase<T> : IDependencyResolver, IFilterProvider where T : class
{
    private const string MetadataKey = "___DependencyResolverContainer___";
    private static readonly Type[] ActionFilterTypes = new[] { typeof(IAuthorizationFilter), typeof(IActionFilter), 
                                                               typeof(IResultFilter), typeof(IExceptionFilter) };
    private readonly Func<IDictionary> _getRequestMetadata ;

    protected DependencyResolverBase()
    {
        _getRequestMetadata = () => HttpContext.Current.Items;
    }

    protected DependencyResolverBase(Func<IDictionary> getRequestMetadata)
    {
        _getRequestMetadata = getRequestMetadata;
    }

    // Infrastructure Services
    public abstract object GetInfrastructureService(Type serviceType);
    public abstract IEnumerable<object> GetInfrastructureServices(Type serviceType); 

    // Action Services
    public abstract T CreateActionContainer();
    public abstract object GetActionService(T container, Type serviceType);
    public abstract IEnumerable<object> GetActionServices(T container, Type serviceType);
    public abstract IEnumerable<object> GetActionFilters(T container, IEnumerable<Type> filterTypes);
    public abstract void ReleaseActionContainer(T container);
 
    public void ReleaseActionContainer()
    {
        var container = CurrentContainer;
        if (container != null) ReleaseActionContainer(container);            
    }

    public void RegisterHttpApplication(HttpApplication application)
    {
        application.EndRequest += (s, e) => ReleaseActionContainer();
    }

    object IDependencyResolver.GetService(Type serviceType)
    {
        return IsInfrastructureService(serviceType) ?
            GetInfrastructureService(serviceType) : 
            GetActionService(GetActionContainer(), serviceType);
    }

    IEnumerable<object> IDependencyResolver.GetServices(Type serviceType)
    {
        return IsInfrastructureService(serviceType) ?
            GetInfrastructureServices(serviceType).
                Union(GetBuiltInInfrastructureServices(serviceType)) :
            GetActionServices(GetActionContainer(), serviceType);
    }

    IEnumerable<Filter> IFilterProvider.GetFilters(ControllerContext controllerContext, 
                                                   ActionDescriptor actionDescriptor)
    {
        return GetActionFilters(GetActionContainer(), ActionFilterTypes, 
                                controllerContext, actionDescriptor).
                    Select(x => new Filter(x, FilterScope.Global, null));
    }

    private IEnumerable<object> GetBuiltInInfrastructureServices(Type serviceType)
    {
        return serviceType == typeof(IFilterProvider) ? 
                        new List<object> {this}:
                        Enumerable.Empty<object>();
    }

    private T GetActionContainer()
    {
        var container = CurrentContainer;
        if (container == null) CurrentContainer = container = CreateActionContainer();
        return container;
    }

    private T CurrentContainer
    {
        get { return (T)_getRequestMetadata()[MetadataKey]; }
        set { _getRequestMetadata()[MetadataKey] = value; }
    }       

    private static bool IsInfrastructureService(Type type)
    {
        return type.Namespace != null && type.Namespace.StartsWith("System.Web.Mvc");
    }
}

This class uses the template method so you can fill in the details in a derived class. Here is an example with StructureMap:

public class StructureMapDependencyResolver : DependencyResolverBase<IContainer>
{
    public override object GetInfrastructureService(Type serviceType)
    {
        return Resolve(ObjectFactory.Container, serviceType);
    }

    public override IEnumerable<object> GetInfrastructureServices(Type serviceType)
    {
        return ObjectFactory.Container.GetAllInstances(serviceType).Cast<object>();
    }

    public override IContainer CreateActionContainer()
    {
        return ObjectFactory.Container.GetNestedContainer();
    }

    public override object GetActionService(IContainer container, Type serviceType)
    {
        return Resolve(container, serviceType);
    }

    public override IEnumerable<object> GetActionServices(IContainer container, Type serviceType)
    {
        return container.GetAllInstances(serviceType).Cast<object>();
    }

    public override IEnumerable<object> GetActionFilters(IContainer container, IEnumerable<Type> filterTypes)
    {
        return container.Model.AllInstances.
            Join(filterTypes, x => x.PluginType, y => y, (x, y) => x).
            Select(x => x.ConcreteType).
            Distinct().
            Select(container.GetInstance);
    }

    public override void ReleaseActionContainer(IContainer container)
    {
        container.Dispose();
    }

    private static object Resolve(IContainer container, Type serviceType)
    {
        return (serviceType.IsAbstract || serviceType.IsInterface) ?
            container.TryGetInstance(serviceType) :
            container.GetInstance(serviceType);
    }
}

Registration is pretty straight forward. The DependencyResolverBase class needs to subscribe to the request end event in order to release the container. This event needs to be subscribed to for each http application (And there can be many) thus the override of Init(). The dependency resolver automatically registers itself as a filter provider so no need to manually register it.

public class MvcApplication : HttpApplication
{
    /...

    protected void Application_Start()
    {
        ObjectFactory.Configure(x =>
                                    {
                                        x.For<IDatabase>().Use<MongoDB>();
                                        x.For<IActionFilter>().Use<SomeFilter>();
                                        x.For<IResultFilter>().Use<SomeFilter>();
                                    });

        var dependencyProvider = new StructureMapDependencyResolver();
        DependencyResolver.SetResolver(dependencyProvider);
        // ...
    }

    public override void Init()
    {
        ((StructureMapDependencyResolver)DependencyResolver.Current).RegisterHttpApplication(this);
    }
}

Adding the Current Time to the Git Bash Prompt on Windows

2011-04-01T10:37:00.001-07:00

Many times I’ve wanted to look back a my git bash shell to see *when* I did something. Turns out its pretty trivial to modify the prompt to your liking. This page covers all sorts of things you can do. In order to make the prompt format permanent you’ll have to modify it in the profile here: [Program Files]\Git\etc\profile. Scroll down to about line 160 and you’ll see the prompt format. I wanted to just replace the current user with the current time, \@ (See the highlighted portion):

Save these changes and open a new git bash shell:

Setting up a Secure WCF Service that uses a UserNamePasswordValidator and runs on Cassini

2011-03-09T16:18:00.001-08:00

I spent the better part of today trying to figure this out so here is the brain dump. Basically I needed to do some prototyping that required a web service that used a custom UserNamePasswordValidator. Normally I have SSL setup in IIS and use credentials over a secured transport layer but I just wanted to whip up something simple for my prototype that would run in Cassini (Yes I know about IIS Express, haven't had time to try it out so it may have been a better route). Although WCF supposedly supports a new option to enable credentials over any transport, unsecured or not, it’s the usual half baked Microsoft “feature” that doesn't really get you anywhere and wont generate the WSDL (I’m lazy and I didn't want to wire up the client config by hand). In any event you can just use message security and forgo the transport level security with this configuration:

  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="UsernameAuthentication">
          <security mode="Message">
            <message clientCredentialType="UserName"/>
          </security>
        </binding>
      </wsHttpBinding> 
    </bindings>
    <services>
      <service name="WebServices.ContactService" behaviorConfiguration="UsernameAuthentication">
        <endpoint address="" binding="wsHttpBinding" bindingConfiguration="UsernameAuthentication" 
                  name="WebServices.ContactService" bindingNamespace="urn:Contacts" 
                  contract="WebServices.IContactService"/>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="UsernameAuthentication">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceCredentials>
            <userNameAuthentication userNamePasswordValidationMode="Custom" 
                customUserNamePasswordValidatorType="WebServices.UserNamePasswordValidator, WebServices"/>
            <serviceCertificate storeName="My" storeLocation="LocalMachine" 
                x509FindType="FindBySubjectName" findValue="MyPrototype"/>
            <clientCertificate>
              <authentication certificateValidationMode="None" revocationMode="NoCheck" />
            </clientCertificate>
          </serviceCredentials>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

Now the only other step is that you have to generate an X509 cert. I created a batch file to do this automatically (Requires the Windows/.NET SDK):

certutil.exe -delstore My MyPrototype

"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\makecert.exe" -sr LocalMachine -ss My -a sha1 -n CN=MyPrototype -sky exchange –pe

The last thing I needed was for the client not to try and validate the cert (As the CA cannot be validated):

[Test]
public void Contacts_Test()
{
    var contacts = new ContactServiceClient();
    contacts.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = X509CertificateValidationMode.None;
    contacts.ClientCredentials.UserName.UserName = "admin";
    contacts.ClientCredentials.UserName.Password = "admin";
    // ...
}

At this point everything works like a champeen!

Using Ruby/Rake/Albacore to Create and Deploy Nuget Packages

2011-03-04T18:33:00.001-08:00

Ruby/Rake/Albacore is a top notch build and deployment stack. If you aren't familiar with it take a look at this quick start guide. As of version 0.2.3 Albacore offers a couple of tasks to aid in the creation of Nuget packages. We’ll cover how to create and deploy Nuget packages using these tasks. The following assumes that the build script will be run on a build server but you can easily adapt it to run locally. It also assumes that the build server establishes a working folder for the source code and scripts when the build runs.

Installing Ruby/Rake/Albacore

If you haven't already, you will need to install either Ruby or IronRuby. After running the installers you should be good to go.

http://www.ironruby.net

http://rubyforge.org

Installing Rake is simply a matter of running (i)gem install rake:

Installing Albacore is simply a matter of running (i)gem install albacore:

Nuget Gallery

If you want to publish your package to the Nuget gallery you will need to follow the steps below:

If you haven't already, setup a gallery account here. Once you verify your email you’ll be able to login and continue.
Next you will want to reserve your package name. To do this click on the “Contribute” tab. Next click on the “Register Package Id” link. Here you can enter the id of the package you want to publish and it will be reserved for you. The documentation states the following about package ID’s: “Package IDs may not contain any spaces or characters that are invalid in an URL. In general, they follow the same rules as .NET namespaces do. So “Foo.Bar” is a valid ID, “Foo!” and “Foo Bar” are not.”
Now you will need to note your access key which you will use later on. This can be viewed by clicking the “MY ACCOUNT” link in the upper right corner. There you will see your access key which is just a guid. If it is compromised you can generate a new one here.

Package Prep

The next thing we’ll need to do is setup a folder that will contain the contents of the package. Under the build server working folder we’ll have our script dynamically create the following structure:

/deploy
/package
/lib

Everything under the “package” folder will be the contents of the Nuget package. Nuget has a number of conventions for the folder structure that you can read about here. But to keep it simple we’ll only cover the convention for deploying an assembly that does not target a specific version of the .NET framework. In that simple case we can just place the binaries from our project under a folder called “lib” (the following code makes use of this module, filesystem.rb, for file system operations):

require "albacore"
require "release/filesystem"

# ... Other tasks ...

desc "Prep the package folder"
task :prepPackage => :unitTests do
    FileSystem.DeleteDirectory("deploy")
    FileSystem.EnsurePath("deploy/package/lib")
    FileSystem.CopyFiles("src/Gribble/bin/Release/Gribble.dll", "deploy/package/lib")
    FileSystem.CopyFiles("src/Gribble/bin/Release/Gribble.pdb", "deploy/package/lib")
end

Here we make sure the “deploy” folder is cleared out and a fresh folder structure is recreated each build. Then we copy the assembly and pdb from our project “release” folder to the “lib” folder.

Creating the Nuspec File

Now we need to create the Nuspec file. You can find more information on that here. This file contains a description of the Nuget package and is bundled with it. For that we will use the Albacore nuspec task:

desc "Create the nuspec"
nuspec :createSpec => :prepPackage do |nuspec|
   nuspec.id = "gribble"
   nuspec.version = ENV["GO_PIPELINE_LABEL"]
   nuspec.authors = "Mike O'Brien"
   nuspec.owners = "Mike O'Brien"
   nuspec.description = "Gribble is a simple, Linq enabled ORM designed to work with dynamically created tables."
   nuspec.summary = "Gribble is a simple, Linq enabled ORM designed to work with dynamically created tables."
   nuspec.language = "en-US"
   nuspec.licenseUrl = "https://github.com/mikeobrien/Gribble/blob/master/LICENSE"
   nuspec.projectUrl = "https://github.com/mikeobrien/Gribble"
   nuspec.working_directory = "deploy/package"
   nuspec.output_file = "gribble.nuspec"
   nuspec.tags = "orm sql"
end

You’ll notice that this task depends on the prepPackage task to get our package folder initialized. You can look at the link noted earlier to get more information about these properties so I’ll just cover a couple of important ones. First the id property is going to be set to the package id we reserved earlier in the gallery. The version must be set so that this package can be differentiated from others that you push. In this case I’m using the version number created by the build server. The working directory and output file determine where the spec will be created. We want it to be created in the root of the package folder. This produces the following nuspec file:

<?xml version='1.0'?>
<package xmlns='http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd'>
  <metadata>
    <id>gribble</id>
    <version>1.0.28</version>
    <authors>Mike O&apos;Brien</authors>
    <description>Gribble is a simple, Linq enabled ORM designed to work with dynamically created tables.</description>
    <language>en-US</language>
    <licenseUrl>https://github.com/mikeobrien/Gribble/blob/master/LICENSE</licenseUrl>
    <projectUrl>https://github.com/mikeobrien/Gribble</projectUrl>
    <owners>Mike O&apos;Brien</owners>
    <summary>Gribble is a simple, Linq enabled ORM designed to work with dynamically created tables.</summary>
    <tags>orm sql</tags>
  </metadata>
</package>

Creating the Nuget Package

Now that we have a package folder and nuspec file we can package all this up. To do this we can use the nugetpack task (No documentation on that yet):

desc "Create the nuget package"
nugetpack :createPackage => :createSpec do |nugetpack|
   nugetpack.nuspec = "deploy/package/gribble.nuspec"
   nugetpack.base_folder = "deploy/package"
   nugetpack.output = "deploy"
end

This task is pretty simple; we need to provide the path to the nuspec file (nuspec), the path to the package folder (base_folder) and the folder to put the generated package into (output). The filename for the package is [Id].[Version].nupkg (i.e. gribble.1.0.28.nupkg). The actual package is just a zip file. You can open it with your favorite zip utility and view the contents. It’s basically the same as your package folder, save a few additions:

Push the Nuget Package to the Gallery

Now that we have the package we can push it to the gallery. To do this you will need to download the Nuget command line utility onto your build server. You can download it here (“NuGet.exe Command Line”). You can place it wherever you want but you will need to include the parent folder in your system path so that your build server can find it (This may require you to restart your build server service after you modify the system path). If you don't know how to modify the system path just follow the steps here. Now Albacore doesn't have a task to handle the pushing of the package so we’ll create our own (nuget.rb):

require "yaml"

class NugetPush
    attr_accessor :apiKey, :package

    def run()
        @apiKey = YAML::load(File.open(ENV["USERPROFILE"] + "/.nuget/credentials"))["api_key"] unless !@apiKey.nil?
        system("nuget", "push", @package, @apiKey)
    end
end

def nugetpush(*args, &block)
    body = lambda { |*args|
        rc = NugetPush.new
        block.call(rc)
        rc.run
    }
    Rake::Task.define_task(*args, &body)
end

And here is how we would use it:

require "albacore"
require "release/filesystem"
require "release/nuget"

# ... Other tasks ...

desc "Push the nuget package"
nugetpush :pushPackage => :createPackage do |nugetpush|
   nugetpush.package = "deploy/gribble.#{ENV['GO_PIPELINE_LABEL']}.nupkg"
end

Not too much to it. All you have to do is provide the path to the package we created in the last step. But there is the matter of the api key which is needed to authenticate you. The task we created can take this directly in the build script but that is generally not a good idea as it will be visible to everyone who has access to the script (If your project is open source then that means everybody). So we take a cue from ruby gems and allow you to cache the key in the user profile (Which is evidently on its way in the next version of Nuget). You’ll notice that it looks for a yaml file called “[UserProfile]\.nuget\credentials” with an item called “api_key”. If your build server service is running under a user account this path will probably be something like “C:\Users\ServiceAccount\.nuget\credentials”. If its running as LocalSystem it gets a bit more tricky. If your build server is running on an x64 operating system the path will be “C:\Windows\SysWOW64\config\systemprofile\.nuget\credentials” and if it is an x86 operating system it will be “C:\Windows\System32\config\systemprofile\.nuget\credentials”. To create the “.nuget” folder you will have to do it from the command line as Windows Explorer doesn't like the period prefix. After you create the folder, create a file called “credentials” and enter the following:

api_key: [Your API Key]

Once that's set the script will pull the key from there.

At this point you should be ready to run your build script and have it create your nuget package and push it to the gallery. If you’re interested in doing this with ThoughtWorks Go check out this post and substitute the ruby gems related steps with the steps above. The code samples above were taken from the Gribble project. You can see full build script here.

Creating Custom Linq Query Operators that Support both IQuerable and IEnumerable

2011-02-25T15:21:00.001-08:00

If your creating a custom query provider you may end up creating some custom query operators. I’ve found this to be a very powerful way to add additional functionality. If your doing this I would suggest also adding versions of your operators that work with IEnumerable<T>. The benefit of this is you can swap out your query provider for a List<T> (Or some other in memory collection that implements IEnumerable<T>) for unit testing.

Lets start with an sample query operator that represents a bubble sort:

public static class Queryable
{
    public static IQueryable<TSource> BubbleSort<TSource>(this IQueryable<TSource> source)
    {
        if (source == null) throw new ArgumentNullException("source");
        return source.Provider.CreateQuery<TSource>(
            Expression.Call(null, ((MethodInfo)MethodBase.GetCurrentMethod()).MakeGenericMethod(new[] { typeof(TSource) }), 
                            new[] { source.Expression }));
    }
}

Here we have some code that calls this custom query operator:

public static IQueryable<string> Sort(IQueryable<string> items)
{
    return items.BubbleSort();
}

This query operator likely maps to an out of process call somewhere (Like on a database server) but in our unit tests we want to keep it in memory. Instead of passing in our query provider we can pass in an in memory collection like List<T>:

[Test]
public void Test_Sort()
{
    var items = new List<string> { "Pauli", "Bohr", "Dirac", "Heisenberg" };
    var sorted = Sort(items.AsQueryable());
    // ... Asserts
}

Now if you were to do this it would fail because List<T> has no idea what BubbleSort() is. Although List<T> implements IQuerable<T> everything it does happens in memory as expected. When you treat an IEnumerable<T> as an IQueryable<T> it actually gets wrapped by a class called EnumerableQuery<T> which, when it executes, rewrites and compiles the expression tree. The interesting thing about this class (Technically delegated to the internal EnumerableRewriter class) is that it rewrites the expression tree so that arguments and return types of type IQueryable<T> get rewritten to be IEnumerable<T>. Same is true of Expression<Func<>> and Expression<Action<>>; they get rewritten to Func<> and Action<>. So if you want to be able to use an IQueryable<T> operator in memory, simply create a corresponding operator in the same type (This is required as far as I can tell) that operates on IEnumerable<T>:

public static class Queryable
{
    public static IQueryable<TSource> BubbleSort<TSource>(this IQueryable<TSource> source)
    {
        if (source == null) throw new ArgumentNullException("source");
        return source.Provider.CreateQuery<TSource>(
            Expression.Call(null, ((MethodInfo)MethodBase.GetCurrentMethod()).MakeGenericMethod(new[] { typeof(TSource) }), 
                            new[] { source.Expression }));
    }

    public static IEnumerable<TSource> BubbleSort<TSource>(this IEnumerable<TSource> source)
    {
        if (source == null) throw new ArgumentNullException("source");
        return SortLibrary.BubbleSort(source);
    }
}

Now your custom query operator will work against your custom query provider or an in memory collection. Obviously, the code that works against your in memory collection should behave the same as your out of process call or this is all for naught. There is one caveat however for query operators that execute the expression but still return an IQueryable<T>, for example:

public static IQueryable<TSource> Clone<TSource>(this IQueryable<TSource> source)
{
    if (source == null) throw new ArgumentNullException("source");
    return source.Provider.Execute<IQueryable<TSource>>(
        Expression.Call(null, ((MethodInfo)MethodBase.GetCurrentMethod()).MakeGenericMethod(new[] { typeof(TSource) }),
                        new[] { source.Expression }));
}

In this case you will need to have the corresponding IEnumerable<T> operator convert to and return an IQueryable<T>:

public static IQueryable<TSource> Clone<TSource>(this IEnumerable<TSource> source)
{
    if (source == null) throw new ArgumentNullException("source");
    return Enumerable.Clone(source).AsQueryable();
}

If you don’t do this the query provider will throw a type mismatch exception on the IEnumerable<T> your returning.

Getting Distinct Results on a Finite Set of Columns or Projections

2011-01-24T14:02:00.001-08:00

I have a situation where I need to return a result set that is distinct on only a few columns or projections. The DISTINCT keyword does not fit the bill as it applies to the entire projection. Before we explore this lets start with some sample data in a table we'll call Contacts.

id email                name
-- ------------------- -------
1   foxydog@yahoo.com    Bob
2   foxydog@yahoo.com    Robert
3   foxydog@yahoo.com    Dick
4   william@hotmail.com Bill

In this example we want to get distinct records by email address, discarding duplicate records (As in this context they are not important). There are a couple of ways to approach this. You could do a GROUP BY on email and an aggregate the other fields (Like MAX) but this could mess things up if the other columns are related somehow (Like an address for example, you could get mismatched street, city, state and zip). You could also do a WHERE id IN on a sub query that does group by on email and a max on id which would solve the problem of mismatched columns:

SELECT * FROM Contacts
WHERE id IN (SELECT MAX(id) FROM Contacts GROUP BY email)

I don't like this however because it requires two table scans (BTW I have indexes on id and email):

Another option is to use the ROW_NUMBER aggregate and PARTITION BY. You can partition by the columns or projections you choose (And I say "projections" because it doesn't strictly have to be a column name, it could be LEN(email) for example). So we could say the following:

SELECT *
FROM (SELECT *,
ROW_NUMBER() OVER(PARTITION BY email ORDER BY email) PartitionId
FROM Contacts) T

This returns the following:

id email                name     PartitionId
-- ------------------- ------- -----------
1   foxydog@yahoo.com    Bob      1
2   foxydog@yahoo.com    Robert   2
3   foxydog@yahoo.com    Dick     3
4   william@hotmail.com Bill     1

Notice how the row numbers restart (PartitionId) when the email address changes? We can then take this one step further and only return records with a partition id of 1 giving us a distinct result set:

SELECT *
FROM (SELECT *,
ROW_NUMBER() OVER(PARTITION BY email ORDER BY email) PartitionId
FROM Contacts) T
WHERE PartitionId = 1

This returns the following:

id email                name     PartitionId
-- ------------------- ------- -----------
1   foxydog@yahoo.com    Bob      1
3   william@hotmail.com Bill     1

And voila! Distinct on only one column. Even though this solution requires a bit more TSQL, the execution plan looks much nicer as we are only doing one table scan:

Three Layer Tea (AKA Sarawak Tea)

2011-01-16T14:19:00.001-08:00

Aside from roti boom (roti canai with butter and sugar) my favorite treat from Malaysia was the three layer tea (Or Sarawak tea). Its made up of three layers, in the following order from top to bottom: black tea, evaporated milk and gula melaka (Brown coconut sugar in syrup form). Its is served chilled (I mix it up) and not only does it taste awesome its unbelievably refreshing in the stifling Malaysian heat. Most America grocers wont carry brown coconut sugar syrup (Sometimes called palm sugar) so you'll have to shop around at your local Asian grocers. I'd definitely give it a try!

Building a Better Expression Visitor

2011-01-14T11:25:00.001-08:00

After trying to hack the new expression visitor class in .NET 4.0 to fit my needs I finally just gave up and wrote a new one from scratch. I did not need the rewriting capabilities of the FCL implementation as I am going directly from an expression tree to an object model. Here are the key features I needed to make this happen:

Ability to control which nodes are visited next. I 'm using the approach of having different visitors for different parts of the expression. For example I have a select visitor that just handles the IQueryable methods, a where clause visitor that handles the where expression (Unary and binary expressions only) and a visitor to handle expressions that stem from constants or from properties on an entity. This composition greatly simplifies things and makes the visitors reusable. When I hit a point where I want to pass the torch to a different visitor I want to be able to specify that I don't want the current visitor to continue visiting a particular child node. You can do this manually with the current implementation but why not just tell the visitor which nodes you want it to skip and let it handle that?
Need to know what the parent was. Sometimes the parent node can effect how the child is processed. For example if the parent of an expression is a binary expression I need to handle it a little differently. Currently you have no information about the parent expression.
Ability to pass along state to visitors of children. I needed to be able to pass along state or a lambda from parent to child in order to build up my object model. I tried a dictionary approach with the current implementation but this was hacky. Not to mention that I didn't want to be doing a ton of dictionary lookups.

So I put together an expression visitor that enables all these features and its made things much easier:

https://gist.github.com/780050

Here are a couple of examples:

Each virtual method has a counterpart that allows you to control what child expressions are visited. For example the VisitMethodCall method has the following overload (In addition to the default):

protected void VisitMethodCall(MethodCallExpression node, TState objectState, TState argumentsState, 
                               bool visitObject, bool visitArguments, params Expression[] argumentsNotToVisit)

Here you can control what is visited as demonstrated by the following implementation:

protected override void VisitMethodCall(Context context, MethodCallExpression node)
{
    // ...
    VisitMethodCall(node, context.State, context.State, true, true, node.Arguments.Skip(1).ToArray());
}

Here we're visiting all the children but skipping every parameter except the first one as we are handing those (Not shown).

We may also need to know what the parent was, as in this example:

protected override void VisitMember(Context context, MemberExpression node)
{
    if (!context.HasParent || context.Parent.IsBinaryLogicalOperator()) //...;
    else //...;
}

Here we need to know if the member expression itself is a predicate (i.e. x => x.Active) or part of a binary logical operation (i.e. x => x.Active == true). In my implementation these two scenarios are handled differently so I needed a way to check this.

The last thing is passing of state. Here I have a visitor for the where clause as follows:

public class WhereVisitor<TItem> : ExpressionVisitorBase<Action<Operand>> {...}

The generic type argument is the visitor state. The state in this instance is an action that has one parameter, an Operand (The Operand type is irrelevant, just an example). This allows us to pass a lambda to a child expression visit method which in turn can pass in some state. This doesn't have to be a lambda, it can be anything that you want passed around from visit to visit.

protected override void VisitBinary(Context context, BinaryExpression node)
{
    // ...
    var operand = // ...;
    context.State(operand);
    
    VisitBinary(node, 
        x => operand.Operator.LeftOperand = x,
        null,
        x => operand.Operator.RightOperand = x,
        true,
        false,
        true);
}

In this example we are creating an operand and passing it into the parent lambda. We are also specifying lambdas that will be passed to the visitor methods that evaluate the children. As you can see they are building up an object model.

In conclusion there are two major things I've learned so far:

If there is a lot of friction using the built in expression visitor, write a custom one from scratch that fits your needs. You can look at the one in the FCL with Reflector or the one I have linked above as an example. It's a bit of work to do but I've found it's a much simpler and cleaner approach and will save you headaches in the long run.
Go from an expression tree to an object model then from the object model to the domain language (i.e. TSQL). I've found that going directly from an expression tree to a domain language gets complicated very fast, especially when you want to add optimizations. Splitting it up into two steps greatly simplifies things. NHibernate does this as well.

Linq Providers Made Easy (Well, sorta)

2010-12-22T19:42:00.001-08:00

Currently I'm implementing a simple Linq enabled ORM to handle an edge case in our system (For everything else we use NHibernate). I've found that the hard part about writing a Linq provider is parsing expression trees; the provider is actually the easy part. In the course of writing it I've created a simple base class that makes it even easier to create the provider:

public abstract class QueryableBase<T> : IOrderedQueryable<T>, IQueryProvider
{
    private readonly Func<QueryableBase<T>> _createQuery;

    protected QueryableBase(Func<QueryableBase<T>> createQuery)
    {
        _createQuery = createQuery;
        Expression = Expression.Constant(this);
    }

    public abstract object Query(Expression expression);

    // ---------------------- IOrderedQueryable Implementation -----------------

    public Expression Expression { get; set; }
    public Type ElementType { get { return typeof(T); } }
    public IQueryProvider Provider { get { return this; } }

    public IEnumerator<T> GetEnumerator()
    {
        return (IEnumerator<T>)((IEnumerable)this).GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        var result = (IEnumerable)Provider.Execute(Expression);
        return result == null ? Enumerable.Empty<T>().GetEnumerator() : result.GetEnumerator();
    }

    // ---------------------- IQueryProvider Implementation -----------------

    public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
    {
        return (IQueryable<TElement>)CreateQuery(expression);
    }

    public IQueryable CreateQuery(Expression expression)
    {
        var query = _createQuery();
        query.Expression = expression;
        return query;
    }

    public TResult Execute<TResult>(Expression expression)
    {
        var value = Execute(expression);
        return value == null ? default(TResult) : (TResult)value;
    }

    public object Execute(Expression expression)
    {
        return Query(expression);
    }
}

This base class provides everything you need, save the expression tree parsing, for a Linq provider. All you have to do is inherit from this class and implement the Query method as follows:

public class SqlTable : QueryableBase<DataRow>
{
    private SqlConnection _connection;
    private string _tableName;

    public SqlTable(SqlConnection connection, string tableName) : 
        base(() => new SqlTable(connection, tableName))
    { 
        _tableName = tableName;
        _connection = connection;
    }

    public override object Query(Expression expression)
    {
        // TODO: Parse the expression tree, generate TSQL, execute it and return the results.
    }
}

You'll notice that the QueryableBase constructor requires a lambda that creates a new instance of your provider (In this case SqlTable). This is used by the base class in the implementation of IQueryProvider to construct new queries based on this one but with a different expression (So that's why we are creating the new provider with the exact same parameters that were passed into the constructor of this one). Now all the magic happens in the Query method. Here is where you would parse the expression tree and translate it into something useful; in this case a TSQL statement that will be executed. Parsing expression trees is a subject all its own (And covered pretty well on the interwebs) so we'll gloss over that. Here is how we could use our new table query provider:

using (var connection = new SqlConnection("..."))
{
    connection.Open();

    var table = new SqlTable(connection, "Products");

    var cheapProducts = table.Where(x => (double)x["Price"] < 5.0).
                              OrderBy(x => x["Name"]).
                              Take(50).
                              Select(x => new { Name = (string)x["Name"], 
                                                Price = (double)x["Price"]}).
                              ToList();

    foreach (var product in cheapProducts) 
        Console.WriteLine("{0} = ${1:0.00}", product.Name, product.Price);
}

Setting up MonoDevelop and F# on Windows

2010-12-20T16:04:00.001-08:00

Based in part on Tomas Petricek's instructions here and here.

Either
1. Install Visual Studio 2008+ (Which you may already have installed).
2. Or download and install the .NET framework 3.5+ here: http://msdn.microsoft.com/en-us/netframework/aa569263
Download and install Gtk# for .NET here: http://monodevelop.com/Download
Download and install MonoDevelop for Windows here: http://monodevelop.com/download
Either
1. Install Visual Studio 2010+ (Which ships with F#) and select the F# option during setup or modify your existing installation to include F# (via Control Panel | Programs and Features | Microsoft Visual Studio...)
2. Or install the F# stand alone package here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=f8c623ae-aef6-4a06-a185-05f59be47d67
Install the F# MonoDevelop language bindings
1. In MonoDevelop, click Tools | Add-in Manager and click the "Install Add-ins..." button.
2. Next click the "Repositories..." button, then the "Add" button.
3. Make sure the "Register and on-line repository" radio button is selected and enter http://functional-variations.net/addin/ and click "OK".
4. Close out of the "Add-in Repository Management" dialog, expand the "Language bindings" node, check the "F# Language Binding..." and click "Forward".
5. Click through default options in the add-in installation and restart MonoDevelop after it completes.
6. Now you should see the F# project and file templates when creating new file or project.

Setting up MonoDevelop and F# on Mac OS

2010-12-20T08:26:00.001-08:00

Based on Tomas Petricek's instructions here and here.

Download and install Mono for Mac here (Open the .dmg and run the .pkg): http://www.go-mono.com/mono-downloads/download.html
Download and install MonoDevelop for Mac here: http://monodevelop.com/download
Download, unzip and install F# for Mac here: http://fsxplat.codeplex.com/releases
Install the F# MonoDevelop language bindings
1. In MonoDevelop, click Tools | Add-in Manager and click the "Install Add-ins..." button.
2. Next click the "Repositories..." button, then the "Add" button.
3. Make sure the "Register and on-line repository" radio button is selected and enter http://functional-variations.net/addin/ and click "OK".
4. Close out of the "Add-in Repository Management" dialog, expand the "Language bindings" node, check the "F# Language Binding..." and click "Forward".
5. Click through default options in the add-in installation and restart MonoDevelop after it completes.
6. Now you should see the F# project and file templates when creating new file or project.

Exploring The F# do Binding

2010-12-05T14:13:00.001-08:00

I've often wondered how the F# compiler compiles the do binding. Here is the skinny on that (Using v4.0.30319.1 of the compiler).

Definition

The docs give the following definition of the do binding:

"A do binding is used to execute code without defining a function or value. Also, do bindings can be used in classes...

Use a do binding when you want to execute code independently of a function or value definition. The expression in a do binding must return unit. Code in a top-level do binding is executed when the module is initialized. The keyword do is optional.

Attributes can be applied to a top-level do binding. For example, if your program uses COM interop, you might want to apply the STAThread attribute to your program. You can do this by using an attribute on a do binding, as shown in the following code."

In a Module

First off lets talk about using the do binding directly in a module. To set the stage, lets say we have 2 modules with exactly the same code except one says hello and one says goodbye. Notice that the Hello module is the last module in the project (Remember, in F# file order matters).

Goodbye.fs:

module Goodbye

    open System

    let getTime () = DateTime.Now.ToString()
    let message = "goodbye there: "

    Console.WriteLine(message + getTime ())

    do
        Console.WriteLine("hasta luego")

    do
        Console.WriteLine("oh bai")
        Console.ReadKey() |> ignore

Hello.fs:

module Hello

    open System

    let getTime () = DateTime.Now.ToString()
    let message = "hello there: "

    Console.WriteLine(message + getTime ())

    do
        Console.WriteLine("hola")

    do
        Console.WriteLine("oh hai")
        Console.ReadKey() |> ignore

The previous two files are compiled as follows (Represented in C#, with some of the compiler noise removed).

public static class Goodbye
{
    public static string getTime() { return DateTime.Now.ToString(); }
    public static string message { get { return "goodbye there: "; } }
}

public static class Hello
{
    public static string getTime() { return DateTime.Now.ToString(); }
    public static string message { get { return "hello there: "; } }
}

internal static class $Goodbye
{
    static $Goodbye()
    {
        string message = Goodbye.message;
        Console.WriteLine(Goodbye.message + Goodbye.getTime());
        Console.WriteLine("hasta luego");
        Console.WriteLine("oh bai");
        ConsoleKeyInfo info2 = Console.ReadKey();
    }
}

internal static class $Hello
{
    public static void main@()
    {
        string message = Hello.message;
        Console.WriteLine(Hello.message + Hello.getTime());
        Console.WriteLine("hola");
        Console.WriteLine("oh hai");
        ConsoleKeyInfo info2 = Console.ReadKey();
    }
}

A couple of things to note here. First of all, all the code (Whether defined in a do binding or directly on the module) gets rolled up into either a static constructor (If not the entry point) or the entry point method. The module's let bindings are placed in another class where value bindings are defined as properties and function bindings are defined as, well, methods. So you can see, as mentioned in the docs, that a do binding is not required when writing top level code in a module (The exception to this is if your writing a GUI app. You will need to use a do binding so that you can apply the STAThread attribute to it). The second thing to note is that it's the code in the last module that is compiled as the entry point for the application (If its not a library). And the entry point code does not need to be in a do binding (Unless it's a GUI app as mentioned above).

If it is a GUI app you can use the do binding to define the STAThread attribute as follows:

open System
open System.Windows.Forms

[<STAThread>]
do
    let form = new Form()
    Application.Run(form)

Which gets compiled to:

internal static class Startup
{
    [STAThread]
    public static void $main@()
    {
        Form form = new Form();
        Application.Run(form);
    }
}

All in all I don't see any point in using a do binding directly in a module unless you need to apply an attribute. If you, dear reader, have any insights into where a do binding in a module could be useful (Aside from applying attributes) I'd love to hear about it.

In a Type

As the docs mention, you can use the do binding in a type. Using a do binding in a type is more interesting as it forms the type constructor (Either static or instance) and, unlike a within module, is required if you want to define any initialization logic (Other than let bindings which form the private fields). Here is an example (And yes, I know this is a contrived example as we could have just done the assignment in the let binding):

namespace LawnCare
    open System

    type WeedWacker (manufacturer : string) =

        static let mutable random = null;
        let mutable maker = manufacturer
        let mutable sn = null

        static do
            random <- new Random()

        do
            sn <- maker.ToUpper() + "-" + random.Next().ToString()

        do
            maker <- maker.ToLower()

        member x.SerialNumber = sn
        member x.Maker = maker

This code gets compiled to the following:

internal static class $LawnCare
{
    static $LawnCare()
    {
        WeedWacker.random = null;
        WeedWacker.random = new Random();
    }
}
 
public class WeedWacker
{
    internal string sn;
    internal string maker;
    internal static Random random;

    public WeedWacker(string manufacturer)
    {
        maker = manufacturer;
        sn = null;
        sn = maker.ToUpper() + " " + random.Next();;
        maker = maker.ToLower();
    }

    public string SerialNumber { get { return sn; } }
    public string Maker { get { return maker; } }
}

Interesting to note that the type and its static constructor are separated into two different types. Also, just as with a module, you can define multiple do bindings and they will get rolled up into one instance and/or static constructor.

Not the most exciting topic I must admit, but good to know none the less.

Why Updating a Clean Property with the NHibernate Pre Update Event Listener, with Dynamic Update Set on the Entity, Doesn't Work

2010-11-13T16:15:00.001-08:00

Wow, that's a mouthful. Anyways, if your trying to update a property in an NHibernate (2.1.2.4000) pre update event listener that is "clean" (In other words wasn't updated in the calling code) and your entity is set to dynamic update the update will not make it to the database. Here is why that is the case. The pre update event gets called in the NHibernate.Action.EntityUpdateAction class. Problem is that by the time the event is called the dirty field list has already been set in this class. So no matter what you do the dirty list is already set in stone as an invariant:

public sealed class EntityUpdateAction : EntityAction
{
    private readonly int[] dirtyFields;

    // ...

    public EntityUpdateAction(object id, object[] state, int[] dirtyProperties, bool hasDirtyCollection, 
                              object[] previousState, object previousVersion, object nextVersion, object instance, 
                              IEntityPersister persister, ISessionImplementor session)
        : base(session, id, instance, persister)
    {
        this.dirtyFields = dirtyProperties;
        // ...
    }

    public override void Execute()
    {
        // ...
        bool flag2 = this.PreUpdate();
        // ...
        if (!flag2)
        {
            persister.Update(id, this.state, this.dirtyFields, this.hasDirtyCollection, this.previousState,
                             this.previousVersion, instance, null, session);
        }
        // ...
    }
}

So as you can see, there is no way the pre update event can effect the dirty fields value that is passed into the update call. What's the solution? Create an interceptor and implement the OnFlushDirty method. The easiest way to do this is to inherit from EmptyInterceptor and override the method as follows:

public class TimestampInterceptor : EmptyInterceptor
{
    public override bool OnFlushDirty(object entity, object id, object[] currentState, object[] previousState, 
                                      string[] propertyNames, global::NHibernate.Type.IType[] types)
    {
        return SetTimestamp(entity, propertyNames, currentState, x => x.Modified);
    }
}

Then register your interceptor and your good to go.

The Great Guid vs Int PK Debate

2010-10-01T16:52:00.001-07:00

Updated 10/11/2010

This has come up so often that I figured its time to do a blog post and just point people to it.

First, like Voltaire said, lets define our terms. We're talking about a primary key which according to Wikipedia is a value that "can uniquely identify each row in a table". When I'm talking about ints I'm implying that it is an integer key that is incremented by the database (IE "identity" in SQL Server). When I'm talking about guids I'm talking about sequential guids.

Reasons for using guids:

They are managed by the application not the database. I think this is my biggest reason for preferring them. I can have the application (domain or ORM) manage them which allows the application to be in control. Having the database manage key creation (Whether int or guid) buys me absolutely nothing and constrains the application for no good reason. I really think the application is where key management belongs, not the backing store. In relation to NHibernate, Tuna has an interesting post here and Fabio here on this subject. I summarize:

Unit of work. NHibernate has the concept of unit of work. Objects are tracked and when you are ready to commit the changes it will figure out what needs to be done at that time and then do it. Using an int identity hijacks this ability when doing inserts because NH must immediately perform an insert (On Save()) in order to get the PK of the entity. If your doing a transaction and you roll it back then prior inserts have to be rolled back in the database, instead of simply having been in memory.
Batching on insert. For example creating the trite order and line items, committing and then having the line items be inserted in one shot instead of individually. If you're using an int identity you will not be able to take advantage of this type of batching because NHibernate needs to return the PK of each line item, which is generated by the database. Note that this only applies to inserts; as other operations do not generate a PK that needs to be returned (Like when doing an update).

Easy to merge data. This may or may not be a good reason depending on your application. Recently this has not been an issue for me but earlier in my career I worked for a payroll company that moved data around the system like crazy (It was a SAAS app that that had one DB per client). Some of the tables had int PKs and others had guid PK's. For obvious reasons it was much easier to move data from one db to another when you had guid PK's. Int PK's required regeneration so were a nightmare to deal with. And again, int PK's didn't buy us anything at all.
They are not as guessable as integers. Notice I didn't say that weren't guessable period. Some things to think about:
1. Guids are not necessarily guaranteed to be truly random, as the UUID spec notes, they are guaranteed to be unique not random (Some interesting discussion here on this subject). So the randomness, if any, depends on the algorithm used to generate them. If your developing on .NET 4.0 (Haven't checked older versions) your using Guid.NewGuid() which uses CoCreateGuid (You can see this in Reflector). This in turn calls the function UuidCreate (So its creating a UUID). BTW, this means that the algorithm is OS dependent if your using .NET 4.0 (and probably older versions I'd imagine) as the guid generation does not happen in managed code. CoCreateGuid on both Windows Server 2003 SP2 x86 and Windows Server 2008 R2 yields a v4 UUID (See here for the layout). The spec states that this version "is meant for generating UUIDs from truly-random or pseudo-random number". What does this mean exactly? All depends on the algorithm Microsoft is using and AFIAK there's no official information on it. I've seen a lot of talk about it on the web but it seems like its all anecdotal, nothing official. In any event it's probably safe to say that its pseudo random. So the bottom line is its not as guessable as say, incrementing an int, but if someone really wanted to guess prior and subsequent values they could do it. Maybe its like having a 10 foot high wooden fence around your yard as apposed to a 10 foot high, razor wire fence and guys with M4's? The wooden fence will stop the casual intruder but someone could get past it if they really really wanted to. Now what about the guid comb (Because this is what your going to want to use as its partially sequential and wont cause index fragmentation)? The NHibernate implementation (NHibernate.Id.GuidCombGenerator) actually uses Guid.NewId() to get the base guid but then modifies part of it so that its sequential. According to the UUID spec the first 10 bytes are the temporally unique portion, and the last 6 bytes contain the spatially unique portion aka "node". For version 4 of the UUID, the random one, everything is random, including the temporally unique portion except for 6 bits (Which specify the version and a reserved portion). The NHibernate guid comb implementation replaces the last 6 bytes with a temporally unique value (Making it sequential), while leaving the rest (74 bits) alone. So you go from having 122 bits of psuedo random data to having 74 bits of pseudo random data. Now your 10 foot tall wooden fence gets cut down to 6 feet. So that's why I say its not as guessable as an int identity.
2. If your app is secure this really shouldn't matter. If being able to predict PK's allows an attacker to compromise your app then you have a problem and relying on security by obscurity is bad. And as we just discussed, guids are not truly random, or at least you shouldn't count on them being so. On the other hand you could say that its not possible to make sure there are absolutely no security holes in your application so using guid PK's raises the bar by making less information available to an attacker or at least making it harder to obtain. So I guess it all comes down to personal philosophy; I can see compelling arguments for both sides. I've sided on trying to make the app as secure as possible while giving attackers as little information as possible and making what information is there harder to obtain.

Bunk reasons for using guids:

Ints are limited and we want a PK that will not run out of id's. Yeah, ints are limited.....to 2.1 billion!!! I don't buy this argument in the least. How many people doing line of business apps are running into that limit? Not saying it cant happen but we have a 400+ GB database and were not even scratching the surface of the the int upper limit. And you can always go to a big int if you want...
Required for replication. Ok this may not be a bunk reason, it may actually be a good reason for your application. But I see this reason thrown around a lot and I get the feeling that many people will not do replication down the road and just use this as yet another reason (Totally an anecdotal statement, no hard data on that one).
We want to be able to change our DBMS if we need to so we want to be completely decoupled from it. I like the decoupling part, but the "change our DBMS part"? Does anyone actually swap out their DBMS on a regular basis, if ever? This is a bunk argument IMO because this does not happen in the real world. Plus I believe that most modern DBMS's support some sort of int identity so in the off chance this happens this will be the least of your worries.

Reasons for not using guids:

They are slower and 4 times bigger: There are a number of posts out there demonstrating that sequential guids are marginally slower than ints. Also they are 128 bits as opposed to 32 bits. In particular it will increase the size of your indexes (I don't include the size of the tables because 16 bytes for a column is relatively small which probably pales in comparison to the size of other columns in the table). Does that marginal difference in performance and increased index size mean you should definitely use ints? That really depends on your application and your development philosophy. Obviously if there is some concrete reason to go either way then the decision is already made (For example if you will do replication or if you've tested guid PK's and the performance hit is really that critical). If that isn't the case, which is probably where most LOB app developers are, then it may be less clear. Since the performance hit is marginal it may not be critical or even noticeable for your application. Be careful not to "strain the gnat" (Perceived guid vs int PK performance penalty) while "gulping down the camel" (All those real performance issues in your application). If the tables are big then the index size pales in comparison, so is that really an issue? I think it really comes down to philosophy. If your using a data centric approach then guids will seem pointless. So you might as well go the int route as guids aren't going to buy you anything and you will have the slight performance boost and smaller indexes. If your using an ORM and/or a DDD approach then the guid will be an attractive option as it puts the control where you want it; the application. In that case its definitely worth the slight performance hit or increased index size because there is a payoff.
Guid's are not human readable. This is a personal preference thing and again depends on development philosophy as mentioned above. As someone who prefers a DDD approach I could care less if its human readable because I'd rather be spending my time in the domain, not mucking around in the db where I would be oft encountering guids. If your taking a data centric approach then you will be looking at them all the time and will probably want the simplicity and elegance of an integer.

Bunk reason for not using guids:

They cause page splits in a clustered index and cause index fragmentation in non clustered indexes. Guids yes, sequential guids no. Guid's are all psuedo random whereas guid combs have a sequential portion which avoids this. Sequential guids are on par with int identities performance-wise, check out Jimmy Nillson's guid comb article for some stats and further explanation. NHibernate also ships with a guid comb identity strategy so you can take advantage of it there.

Conclusion

As mentioned earlier, I see the lines being drawn according to development philosophy. If you're using an ORM (And possibly doing some flavour of DDD) then there will be advantages to using a sequential guid, both concrete and philosophical, and no advantages to using an int. If your approach is data centric then guids will seem distasteful and int will seem like a trusted old friend.

Novacut - Distributed Video Editing

2010-09-29T11:08:00.001-07:00

A buddy of mine is developing an OSS collaborative video editor inspired by the distributed nature of existing OSS tools like git. This seems like a very innovative approach as a team scattered around the world could shoot and post process video in a collaborative way. I just recently watched La gran final (Great movie BTW) and I bet Novacut would have been perfect for a project like that. The project primarily aims to facilitate indy film and TV production which lately, IMO, outshines the big production houses. Anyways, check it out here and back it financially here.