Theory to Practice to Theory

Behind the scenes of events

noreply@blogger.com (Elisha) — Wed, 22 Feb 2012 19:23:00 +0000

Events are a classic implementation of the observer pattern. Support for events syntax exists in many languages, such as C#. In this post we’ll try to understand the internals of events.

Delegates background

The most abstract way to describe a delegate is a “pointer to a method”. A very relevant feature of delegates is that they can “point” at multiple methods. In order to do so we use the =+ operator and combine to other delegates, for example:

[Test]
public void Invoke_TwoDelegatesCombined_BothCalled()
{
    bool wasACalled = false;
    bool wasBCalled = false;

    Action delA = () => wasACalled = true;
    Action delB = () => wasBCalled = true;

    Action combine = null;
    combine += delA;
    combine += delB;

    combine.Invoke();

    Assert.That(wasACalled);
    Assert.That(wasBCalled);
}

But, what actually happens here? Let’s take a look at this code:

combine += delA;

This code compiles to the following IL:

IL_0031: ldloc.2
IL_0032: ldloc.0
IL_0033: call class [mscorlib]System.Delegate [mscorlib]System.Delegate::Combine(class [mscorlib]System.Delegate, class [mscorlib]System.Delegate)
IL_0038: castclass [mscorlib]System.Action
IL_003d: stloc.2

Which is equivalent to:

combine = Delegate.Combine(combine, delA);

The result of the compiled code is direct call to Delegate.Combine, which makes any future call to the combined result forwarded to both delegates.

Default event

If we use default event implementation, the compiler generates two methods and a backing field. The backing field is a delegate storing the subscribers, the methods are add and remove the subscribers from the delegate. This implementations allows us to add and remove subscribers for whom the event is visible and raise the event from the type itself. For example:

public class Publisher
{
    public event EventHandler MyEvent;

    public void Publish()
    {
        MyEvent(this, EventArgs.Empty);
    }
}

The event compiles into a field, which is a delegate of the event type:

.field private class [mscorlib]System.EventHandler MyEvent

And into two methods for adding and removing subscribers:

.event [mscorlib]System.EventHandler MyEvent
{
    .addon instance void Events.Publisher::add_MyEvent(class [mscorlib]System.EventHandler)
    .removeon instance void Events.Publisher::remove_MyEvent(class [mscorlib]System.EventHandler)
}

With the signatures:

.method public hidebysig specialname 
 instance void add_MyEvent (
  class [mscorlib]System.EventHandler 'value'
 ) cil managed

.method public hidebysig specialname 
 instance void remove_MyEvent (
  class [mscorlib]System.EventHandler 'value'
 ) cil managed

The bodies of the events, not surprisingly, manipulates the backing field, we can ignore the bodies for now.

So up to here we see what the declaration of event compiles into – an event declaration, a backing field which is a delegate of the event type and two methods for adding and removing subscribers. All this magic for single C# line of code.

The other side of the event is what happens as we raise it. The event can be raised only from within the type that declares it. For example:

MyEvent(this, EventArgs.Empty);

Compiles into:

IL_0000: nop
IL_0001: ldarg.0
IL_0002: ldfld class [mscorlib]System.EventHandler Events.Publisher::MyEvent
IL_0007: ldarg.0
IL_0008: ldsfld class [mscorlib]System.EventArgs [mscorlib]System.EventArgs::Empty
IL_000d: callvirt instance void [mscorlib]System.EventHandler::Invoke(object, class [mscorlib]System.EventArgs)

All this code does is accessing the delegate backing field and invoking it.

Custom event

In fact, the event is not custom but the add/remove methods are. Custom add/remove for events is a feature in C# which I think is not very commonly used (in contrast to properties). It allows the developer provide an alternative implementation to the event subscription.

public event EventHandler MyCustomEvent
{
    add {}
    remove { }
}

The compiled class in this case does not contain a backing field. It contains the declaration of the event and the two methods with the custom provided body.

A difference which derived from the compiled code difference is that there’s no way to raise the event directly. This makes sense since the custom code can do many things (or nothing) with the subscribers and not store them in a common place for later invocation. If we’d try to raise MyCystomEvent in the same way we tried to raise MyEvent we’ll get a compilation error.

Retrieving property value by name using dynamic method

noreply@blogger.com (Elisha) — Sun, 19 Feb 2012 21:08:00 +0000

In the previous post we compared some alternatives of the dynamic keyword. One important alternative which is very interesting is based on reflection emit. Reflection emit enables us to generate code using IL at runtime, compile it and execute it right away.
In this post we’ll see how to extract from an unknown type a string property named ‘Name’ using a dynamic method.

The code

public static string GetNameByDynamicMethod(object arg)
{
    Type type = arg.GetType();

    Func<object, string> getterDelegate;
    if (!typeToEmitDelegateMap.TryGetValue(type, out getterDelegate))
    {
        string typeName = type.Name;

        PropertyInfo nameProperty = type.GetProperty("Name");
        Type returnType = typeof (string);
                
        // Define a new dynamic method
        // The method returns a string type
        // The method expects single parameter
        var method = new DynamicMethod("GetNameFrom" + typeName, 
                                       returnType, new[] {typeof(object)});
                
        ILGenerator ilGenerator = method.GetILGenerator();

        // Load to the stack the first method argument.
        //In our case, this is an object whose type we already know
        ilGenerator.Emit(OpCodes.Ldarg_0);
        
        // Cast the object to the type we already know
        ilGenerator.Emit(OpCodes.Castclass, type);

        // Call the getter method on the casted instance
        ilGenerator.EmitCall(OpCodes.Call, nameProperty.GetGetMethod(), null);

        // Return the value from Name property
        ilGenerator.Emit(OpCodes.Ret);

        // Compile the method and create a delegate to the new method
        getterDelegate = (Func<object, string>)method.CreateDelegate(typeof(Func<object, string>));

        typeToEmitDelegateMap.Add(type, getterDelegate);
    }

    return getterDelegate(arg);
}

What we did here was to define a new method, generate its code with IL, compile it and execute it. This new method is equivalent in many ways to a method we’d generate in the original program. This new method will be hosted in a dynamic module in the memory.

The advantage of this kind of method over reflection is that compiles the code once and doesn’t need to explore the type again any time we need to get the property value.

Performance

A quick comparison for calling these alternatives 10,000,000 times each:

	Seconds	Ratio to directly
Directly	0.0131	1
Dynamic	0.4609	35
Expression	0.9154	70
Reflection emit	0.9832	75

As can be seen, using the dynamic keyword works much faster than compiling an expression or a dynamic method at runtime.

Another interesting data is the time that each alternative takes to set up (The time to perform the first call):

	Seconds
Directly	0.00003
Dynamic	0.08047
Expression	0.00114
Reflection emit	0.02169

Performance of the dynamic keyword

noreply@blogger.com (Elisha) — Fri, 17 Feb 2012 11:33:00 +0000

In .NET 4.0 a new keyword was introduced, the dynamic keyword. One of the things it enables is calling methods on an instance and bypassing the compile time type checks. It can be useful in many scenarios such as duck typing.
In this post, we’ll see that in some cases the keyword might have an unnecessary performance hit. Another thing we’ll see is how to save some of that time.

Simple performance measure

Let’s compare the performance of 3 versions for getting a property value - directly, using dynamic and using reflection:

public static string GetName(Student arg)
{
    return arg.Name;
}
public static string GetNameByDynamic(dynamic arg)
{
    return arg.Name;
}
public static string GetNameByReflection(object arg)
{
    Type type = arg.GetType();

    MethodInfo getter;
    if (!typeToMethodMap.TryGetValue(type, out getter))
    {
        PropertyInfo property = type.GetProperty("Name");
        getter = property.GetGetMethod();
        typeToMethodMap.Add(type, getter);
    }

    return (string) getter.Invoke(arg, null);
}

Calling each method 10,000,000 times sums to: GetName=0.02 seconds, GetNameByDynamic=0.47 seconds, GetNameByReflection=15.41. Meaning, dynamic compared to strong type call is ~20 times slower.

Improving performance using interface

One way to deal with this performance hit is to extract an interface from all possible objects, using it, we can get back working with strong type:

public interface INameProvider{
    string Name { get; set; }
}

And change our method to:

public static string GetNameByInterface(INameProvider arg)
{
    return arg.Name;
}

Luckily this code run at 0.07 seconds, which is ~7 times faster than the dynamic version. The conclusion from this is that if our code is in critical performance area, we better extract an interface (as long as it makes sense – don’t abuse the interface if the types has no logical commonality!).

Improving reflection version using expressions

What should we do if our code is written in pre-.NET 4.0 version and our solution based on reflection? In this case, our code runs ~750 times slower than the strong type version. Since we can’t use dynamic, which was introduced first at .NET 4.0, we should find some other solution. A simple one is generating a method using expressions. The main advantage of expressions here is that they can be compiled into a new method which we can reuse.

public static string GetNameByExpression(object arg)
{
    Type type = arg.GetType();

    Func<object, string> getterDelegate;
    if (!typeToDelegateMap.TryGetValue(type, out getterDelegate))
    {
        var parameterExpression = Expression.Parameter(typeof (object), "arg");
        var castExpression = Expression.TypeAs(parameterExpression, type);
        MemberExpression memberExpression = Expression.Property(castExpression, "Name");
        var lambda = Expression.Lambda<Func<object, string>>(memberExpression, parameterExpression);

        getterDelegate = lambda.Compile();
        typeToDelegateMap.Add(type, getterDelegate);
    }

    return getterDelegate(arg);
}

This code here is basically equivalent to generating a lambda which looks like:

(object arg) => ((Student)arg).Name;

Since we compile the code once we can skip the reflection invocation each time and end with much faster code. Running this method times 10,000,000 takes 0.86 seconds, which is ~18 times faster than the reflection solution.

Conclusion

If you writing code which must run as fast as possible, this is the performance summary:

	Seconds	Ratio to directly
Directly	0.02	1
Through interface	0.07	3.5
Using dynamic	0.47	23.5
Using expression	0.86	43
Reflection	15.41	770

Monitoring execution using Mono Cecil

noreply@blogger.com (Elisha) — Sat, 04 Feb 2012 20:58:00 +0000

This post will demonstrate how to monitor the execution of .Net code using Mono Cecil. This can be useful for logging, for performance analysis and mainly for fun. The concept is obviously IL weaving. We’ll look for entry points and existing IL instructions to weave around the new IL. In this post we’ll show only four types of monitoring, in reality we have some more. The four types are: Enter method, Exit method, Jump from method and Jump back to method. Jump in this context means call other method and return from other method.
In our example we’ll assume we have some simple ‘notifier’ which the weaved code will call:

public class Notifier
{
    public static Action<string> Enter;
    public static Action<string> Exit;
    public static Action<string> JumpOut;
    public static Action<string> JumpBack;

    public static void NotifyEnter(string methodName)
    {
        if (Enter != null)
        {
            Enter(methodName);
        }
    }

    public static void NotifyExit(string methodName)
    {
        if (Exit != null)
        {
            Exit(methodName);
        }
    }

    public static void NotifyJumpOut(string methodName)
    {
        if (JumpOut != null)
        {
            JumpOut(methodName);
        }
    }

    public static void NotifyJumpBack(string methodName)
    {
        if (JumpBack != null)
        {
            JumpBack(methodName);
        }
    }
}

Monitoring enter

This is the most trivial weave, this one inserts before the first instruction in the method body a call to Enter callback. In order to do so, we need first to load the assembly and find all the methods into which we can weave:

public void Weave()
{
    AssemblyDefinition assembly = AssemblyDefinition.ReadAssembly(assemblyPath);

    IEnumerable<MethodDefinition> methodDefinitions = assembly.MainModule.GetTypes()
        .SelectMany(type => type.Methods).Where(method => method.HasBody);
    foreach (var method in methodDefinitions)
    {
        WeaveMethod(assembly, method);
    }

    assembly.Write(assemblyPath);
}

Now we add reference into the weaved assembly to the callbacks. This is not yet the weaving, this is required definition for the assembly to use in the weaved assembly. We’ll get the called methods using reflection:

Type notifierType = typeof (Notifier);
enterMethod = notifierType.GetMethod("NotifyEnter", BindingFlags.Public | BindingFlags.Static, null, new[] {typeof (string)}, null);
exitMethod = notifierType.GetMethod("NotifyExit", BindingFlags.Public | BindingFlags.Static, null, new[] {typeof (string)}, null);
jumpFromMethod = notifierType.GetMethod("NotifyJumpOut", BindingFlags.Public | BindingFlags.Static, null, new[] {typeof (string)}, null);
jumpBackMethod = notifierType.GetMethod("NotifyJumpBack", BindingFlags.Public | BindingFlags.Static, null, new[] {typeof (string)}, null);

Afterwards, we’ll add the references to the weaved assembly:

MethodReference enterReference = assembly.MainModule.Import(enterMethod);
MethodReference exitReference = assembly.MainModule.Import(exitMethod);
MethodReference jumpFromReference = assembly.MainModule.Import(jumpFromMethod);
MethodReference jumpBackReference = assembly.MainModule.Import(jumpBackMethod);

So our weave method looks like:

private static void WeaveMethod(AssemblyDefinition assembly, MethodDefinition method)
{
    MethodReference enterReference = assembly.MainModule.Import(enterMethod);
    MethodReference exitReference = assembly.MainModule.Import(exitMethod);
    MethodReference jumpFromReference = assembly.MainModule.Import(jumpFromMethod);
    MethodReference jumpBackReference = assembly.MainModule.Import(jumpBackMethod);

    string name = method.DeclaringType.FullName + "." + method.Name;
            
    WeaveEnter(method, enterReference, name);
    WeaveJump(method, jumpFromReference, jumpBackReference, name);
    WeaveExit(method, exitReference, name);
}

Now, we have everything ready to weave the enter monitoring code:

private static void WeaveEnter(MethodDefinition method, MethodReference methodReference, string name)
{
    var ilProcessor = method.Body.GetILProcessor();

    Instruction loadNameInstruction = ilProcessor.Create(OpCodes.Ldstr, name);
    Instruction callEnterInstruction = ilProcessor.Create(OpCodes.Call, methodReference);

    ilProcessor.InsertBefore(method.Body.Instructions.First(), loadNameInstruction);
    ilProcessor.InsertAfter(loadNameInstruction, callEnterInstruction);
}

The ILProcessor is a helper utility which Cecil provides to make the weaving simpler. The first instruction we weave is loading of a string which is the name of the method being entered. The second instruction we weave is a call instruction which uses as argument the loaded string. We insert the instructions in the beginning of the method and from now on every time the method is entered the callback will be invoked.

Monitoring exit

Monitoring exit is a little more interesting. In contrast to enter where we have single weaving point, exit may have multiple exit points – multiple return statements, thrown exceptions, etc…
Here we’ll monitor for simplicity return statements only:

private static void WeaveExit(MethodDefinition method, MethodReference exitReference, string name)
{
    ILProcessor ilProcessor = method.Body.GetILProcessor();

    List<Instruction> returnInstructions = method.Body.Instructions.Where(instruction => instruction.OpCode == OpCodes.Ret).ToList();
    foreach (var returnInstruction in returnInstructions)
    {
        Instruction loadNameInstruction = ilProcessor.Create(OpCodes.Ldstr, name);
        Instruction callExitReference = ilProcessor.Create(OpCodes.Call, exitReference);

        ilProcessor.InsertBefore(returnInstruction, loadNameInstruction);
        ilProcessor.InsertAfter(loadNameInstruction, callExitReference);
    }
}

As can be seen, we first find all the return instructions. Afterwards, we insert before them call to our callback in a similar way to the enter callback.

Monitoring method jumps

This monitoring type will enable us knowing when we jump to another method. If we are doing performance measuring, in “ideal” world (where we have single thread and no context switches) this would be the place where we stop and resume measuring the time for the executed method. Here for simplicity we’ll weave around simple call instructions, ignoring other types of call (like callvirt).

private static void WeaveJump(MethodDefinition method, MethodReference jumpFromReference, MethodReference jumpBackReference, string name)
{
    ILProcessor ilProcessor = method.Body.GetILProcessor();

    List<Instruction> callInstructions = method.Body.Instructions.Where(instruction => instruction.OpCode == OpCodes.Call).ToList();
    foreach (var callInstruction in callInstructions)
    {
        Instruction loadNameForFromInstruction = ilProcessor.Create(OpCodes.Ldstr, name);
        Instruction callJumpFromInstruction = ilProcessor.Create(OpCodes.Call, jumpFromReference);

        ilProcessor.InsertBefore(callInstruction, loadNameForFromInstruction);
        ilProcessor.InsertAfter(loadNameForFromInstruction, callJumpFromInstruction);

        Instruction loadNameForBackInstruction = ilProcessor.Create(OpCodes.Ldstr, name);
        Instruction callJumpBackInstruction = ilProcessor.Create(OpCodes.Call, jumpBackReference);

        ilProcessor.InsertAfter(callInstruction, loadNameForBackInstruction);
        ilProcessor.InsertAfter(loadNameForBackInstruction, callJumpBackInstruction);
    }
}

Here, we find all the call instructions and insert before them a call to JumpFrom and after them call to JumpBack. This way we get a call before leaving and returning to the method.

Example

public void MethodA()
{
    MethodB();
}


private void MethodB()
{
}

If we execute MethodA we’re about the receive these calls:

Enter MethodA
JumpFrom MethodA
Enter MethodB
Exit MethodB
JumpBack MethodA
ExitMethod A

Summary

Mono Cecil can be used for low level AOP where the aspects targets are IL instructions. There are already some great tools out there for AOP like PostSharp, but it is cool to know how simply a solution can be implemented using Cecil.

The synchronized keyword

noreply@blogger.com (Elisha) — Fri, 18 Nov 2011 16:05:00 +0000

What is does

A not very known feature of .NET is the synchronized keyword. The keyword can be used on methods and it ensures:

Instance method – can be executed in single thread on the instance (different instances are not synchronized). Equivalent to lock(this).
Static method – can be executed in single thread. Equivalent to lock(typeof(TypeName)).

Usage in C#

If you’ll look at the C# specification you’ll see that there’s no mention to this keyword. The reason is that the keyword is an IL keyword and not a C# one. In order to instruct the compiler to mark the method as synchronized we can use the MethodImplAttibute with Synchronized MethodImplOptions. For example:

[MethodImpl(MethodImplOptions.Synchronized)]
public void MethodWithSyncAttribute()
{
}

The IL result

Using synchronized keyword

The MethodWithSyncAttribute() looks in IL:

.method public hidebysig instance void MethodWithSyncAttribute() cil managed synchronized
{
// Code size 2 (0x2)
.maxstack 8
IL_0000: nop
IL_0001: ret
}

It is very clear that this method has no explicit lock instructions like Monitor.Enter for example. Yet, it’ll still behave the same as if we used a lock block around the method body.

Using lock block

The previous method is equivalent to the next:

public void MethodWithExplicitLock()
{
    lock(this)
    {
    }
}

This method translates into:

.method public hidebysig instance void MethodWithExplicitLock() cil managed
{
// Code size       36 (0x24)
.maxstack 2
.locals init ([0] bool '<>s__LockTaken0',
           [1] class Sync.Logger CS$2$0000,
           [2] bool CS$4$0001)
IL_0000: nop
IL_0001: ldc.i4.0
IL_0002: stloc.0
.try
{
    IL_0003: ldarg.0
    IL_0004: dup
    IL_0005: stloc.1
    IL_0006: ldloca.s   '<>s__LockTaken0'
    IL_0008: call       void [mscorlib]System.Threading.Monitor::Enter(object,
                                                                        bool&)
    IL_000d: nop
    IL_000e: nop
    IL_000f: nop
    IL_0010: leave.s    IL_0022
} // end .try
finally
{
    IL_0012: ldloc.0
    IL_0013: ldc.i4.0
    IL_0014: ceq
    IL_0016: stloc.2
    IL_0017: ldloc.2
    IL_0018: brtrue.s   IL_0021
    IL_001a: ldloc.1
    IL_001b: call       void [mscorlib]System.Threading.Monitor::Exit(object)
    IL_0020: nop
    IL_0021: endfinally
} // end handler
IL_0022: nop
IL_0023: ret
}

As can be seen, the lock block translates naturally into a try/finally block with calls to Montior.Enter and Monitor.Leave.

Summary

The synchronized keyword is an IL keyword that synchronizes the marked method calls. It causes the method to behave in equivalent way to the one where the whole body is surrounded with lock block. It is interesting to note that locking instructions are generated only during JIT when using the keyword.
The bottom line is that for C# developers it mostly provides another syntactic sugar for defining trivial lock.

Visual Studio 2010 code complexity extension

noreply@blogger.com (Elisha) — Sat, 02 Jul 2011 17:16:00 +0000

An Alpha version of code complexity addin for Visual Studio 2010 is available. The extension can be found at project page at CodePlex.
The extension shows the complexity of the methods in the IDE near the method and measures the method “health”.
For example, view of healthy methods (low complexity):

And example of too complex method:

Currently, the complexity metric shown is simple complexity (defined by Code Complete). This metric counts the number of possible paths in the method. A health method is one with low complexity, and method with 10 paths is not so good and worth simplifying.

Keep it green!

Intercepting unmanaged call in managed code

noreply@blogger.com (Elisha) — Sat, 02 Jul 2011 17:01:00 +0000

This post will demonstrate how to intercept unmanaged calls in the executing process. There are many reasons for intercepting unmanaged calls, among them monitoring, debugging and some other hacks.
In this post it’ll be demonstrated how to intercept calls to CreateFile from Kernel32 library. The CreateFile method is called for opening an existing file and creating new one. For example, calls to File.OpenText will initiate a call to CreateFile.

Hooking CreateFile in unmanaged code

We’ll define a function pointer type for CreateFile:

typedef HANDLE (WINAPI *FileCreateFunction)(
    LPCWSTR lpFileName,
    DWORD dwDesiredAccess,
    DWORD dwShareMode,
    LPSECURITY_ATTRIBUTES lpSecurityAttributes,
    DWORD dwCreationDisposition,
    DWORD dwFlagsAndAttributes,
    HANDLE hTemplateFile);

Afterwards, we’ll store the original CreateFile method and create a hook to which will redirect calls:

FileCreateFunction OriginalCreateFile = (FileCreateFunction)GetProcAddress(GetModuleHandle(L"kernel32"), "CreateFileW");

HANDLE WINAPI CreateFileHook(
    LPCWSTR lpFileName,
    DWORD dwDesiredAccess,
    DWORD dwShareMode,
    LPSECURITY_ATTRIBUTES lpSecurityAttributes,
    DWORD dwCreationDisposition,
    DWORD dwFlagsAndAttributes,
    HANDLE hTemplateFile)
{
    bool hasListener = createFileCallback != NULL;
    if(hasListener)
    {
        createFileCallback(lpFileName);
    }

    return OriginalCreateFile(
        lpFileName,
        dwDesiredAccess,
        dwShareMode,
        lpSecurityAttributes,
        dwCreationDisposition,
        dwFlagsAndAttributes,
        hTemplateFile);
}

For now, ignore the code about the callback, this code we’ll be used later for notifying the managed when a file is created.

As we can see, the hook function has the same signature as the original one. This is obvious since the function caller will not be changed, just the target method. The hook observes the function arguments and forwards the call to the original call.

Now, we’ll forward the calls from CreateFile to the new hook using mhook library:

BOOL APIENTRY DllMain( HMODULE hModule,
    DWORD  ul_reason_for_call,
    LPVOID lpReserved)
{
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        Mhook_SetHook((PVOID*)&OriginalCreateFile, CreateFileHook);
        break;
    case DLL_PROCESS_DETACH:
        createFileCallback = NULL;
        Mhook_Unhook((PVOID*)&OriginalCreateFile);
        break;
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
        break;    
    }

    return TRUE;
}

This code will hook the CreateFile method (OriginalCreateFile) to our new defined hook when the library is loaded into the process.

Using the library in managed code

In order to load the unmanaged library we’ll use P/invoke calls:

[DllImport("kernel32", SetLastError = true)]
static extern IntPtr LoadLibrary(string lpFileName);

[DllImport("kernel32.dll", SetLastError = true)]
static extern bool FreeLibrary(IntPtr hModule);

Loading the library:

static void Main(string[] args)
{
    IntPtr library = LoadLibrary(@"..\..\..\Debug\InterceptionLibrary.dll");
    FreeLibrary(library);
}

Right now, all the calls will behave the same, but all will be routed through our new hook (a user of the software will experience no difference).

Preparing a callback in unmanaged code

We would like to know what file is being created, so we’ll define a callback function pointer that matches it:

typedef void (*NotifyCallbackFunction)(const TCHAR* fileName);

The managed code will register a callback using the method:

extern "C" __declspec(dllexport) void RegisterFileCreateListener(
    NotifyCallbackFunction callback )
{
    createFileCallback = callback;
}

This method is exposed so it can be used by the managed code using:

extern "C" __declspec(dllexport)

Registering the callback through managed code

First, in order to call the register method, we’ll need to define a delegate into the register method will be loaded (ignore the callback delegate for now):

[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
private delegate void RegisterFileListenerDel(OnFileCreatedDel callback);

In order to load the register method into the managed assembly, we’ll need to use:

[DllImport("kernel32", SetLastError = true)]
static extern IntPtr GetProcAddress(IntPtr hModule, string procName);

Loading the method is done by:

IntPtr pAddressOfFunctionToCall = GetProcAddress(library, "RegisterFileCreateListener");
var registerListener = (RegisterFileListenerDel)
                       Marshal.GetDelegateForFunctionPointer(
                           pAddressOfFunctionToCall,
                           typeof (RegisterFileListenerDel));

Now, we’ll have to define a callback delegate and method:

[UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Unicode)]
private delegate void OnFileCreatedDel(string fileName);

private static void OnFileCreated(string fileName)
{
    Console.WriteLine("File opened: {0}", fileName);
}

Now, all needed is to register the callback using the method loaded in the previous step:

registerListener(OnFileCreated);

That’s all, as long as the library is loaded our callback we’ll be notified on every CreateFile call.

Example

This is our final version of the main method in the managed code:

static void Main(string[] args)
{
    IntPtr library = LoadLibrary(@"..\..\..\Debug\InterceptionLibrary.dll");
    
    IntPtr pAddressOfFunctionToCall = GetProcAddress(library, "RegisterFileCreateListener");
    var registerListener = (RegisterFileListenerDel)
                           Marshal.GetDelegateForFunctionPointer(
                               pAddressOfFunctionToCall,
                               typeof (RegisterFileListenerDel));
    registerListener(OnFileCreated);

    File.OpenText(@"C:\Development\wow.txt").Dispose();
    File.CreateText(@"C:\Development\wow2.txt").Dispose();

    FreeLibrary(library);
}

Running this code will notify these files created:

Summary

In order to be notified about a native call in managed code:

Create an unmanaged library
Hook the requested method using mhook
Create and expose a callback registration method
In managed code load the library
Register a callback

You can download the source code example here.

Alternatives

There several other ways, another possible way to intercept calls in unmanaged calls is Detours, a library developed by Microsoft. Another possible solution is EasyHook, a library that allows intercepting unmanaged calls directly from managed code.

Agile means a release is ready today

noreply@blogger.com (Elisha) — Sat, 18 Dec 2010 21:22:00 +0000

The trigger for this post is an interesting experience I had this week our weekly demo. I lead the demo and made the client worried. This demo had a positive side – only a few crashes, our system had stability problems during the previous phases. Being able to work with almost no exception is great progress. So far, so good. The downside was that we had to set the environment in some points during the demo, like:

Run a script to fix the registry
Watch the task manager to ensure it’s done processing before sending new request
Delete local DB between sessions
…

It repeated itself during the last few demos, it annoyed us – the developers and it definitely annoyed the client. The result was a request from the client to change the architecture. This is rarely a legitimate request, the client asks for features, for a “demoable” value. Architecture and design should not be the client concerns. This helped learning important basics about the demo: how did a demo with such progress made a client so worried?

The purpose of demo

The demo has two parallel purposes, they can live side by side, but it’s crucial to know which one each dome serves:

Feedback for new concepts

This is clear characteristic of demos of a new products. The client has hard time telling what are the exact features that solves the problem the product is aimed for. In this phase, the developers are in charge of demonstrating how the initial ideas “feels” in real life – a POC the client can interact with. This phase is important in the product lifetime. In this phase the features must be perceptible, but not perfect. The ideal result of this demo is a client decision whether the feature is good or not, should it be thrown or should it go into the product.

Show the value the product provides

Hopefully, this is the characteristic of most demos. These demos give the client a close view of the development progress – the client knows what value the product provides. In this point decisions can be made – use the new value (like selling the new version) and choose what is the next important value the product should provide.
Choose wisely – which purpose this demo serves?Defining before the demo what it’s supposed to achieve is crucial.

When should we choose?

The answer is easy – before the iteration plan. The reasons are clear: For a POC the least possible should be done, the feature may not be part of the product, no need to put effort there at this moment. For a progress demo a feature will be planned differently – coding will take longer (this is not a throw away code – refactoring, unit-testing…), satellite tasks will be required (like installer, licensing, etc.). In this iteration the feature will require more tickets, so it must be planned accordingly.

How to choose?

If there’s confidence the feature is what the client wants in the product – plan an iteration in such way the feature is in the product and it’s “productized”. If there’s a doubt, plan in such a way the feature will be demoable but with the least effort as possible.

What is done

“What is done” is important in a demo that show the current value of the product. Then what is done? The answer is very simple: The product can be released right after the demo. In these demos there’s a simple but important guideline – perform the demo on a neutral machine from an installed version. Meaning – running the product from the development IDE on a development machine is not good enough.
If the demo purpose is to show what the client can use, show the client what can be used. The client can use only what is already “productized”, so if the feature is not part of the installer, it doesn’t help. If the the feature is not linked to the license, it doesn’t help. If the feature crashes and constantly requires workarounds, it doesn’t help.

Bottom line

All developers know how to achieve a successful demo. The problem is identifying what is the next demo purpose. Before planning the next iteration be clear about the iteration “What is done”. One of the things which could cause a failure is acknowledging the product passed the POC phase and it’s a real product (This is where me and my team failed this week). A good parameter for moving between the phases concluded from the question “when users will use it?”.
After identifying the demo purpose, plan the iteration accordingly and make sure the demo fits it’s purpose.

Automatic generation of View-Model - test drive

noreply@blogger.com (Elisha) — Thu, 26 Aug 2010 19:06:00 +0000

In the previous post I tried to present an approach for automatic View-Model generation. When trying to use it in real life project, as simple as registration form, many missing features were revealed.

Here’s a list of issues which were noticed on simple implementation:

There’s no way to access the model from the abstract View-Model
There’s no way no pass arguments to the constructor
No built in way to define the order of validations
No way to raise event of PropertyChanged other than the property being set
No way to declare additional errors on properties which are not mapped to the model

Actual attempt to use the framework in registration form

This is the simple registration – all fields are mandatory, email must match some regular expression and password verification must be same as password. The model has 3 properties – Email, Name and Password. These properties are mapped to the View-Model. Let’s see how this code looks like in the View-Model using the new framework:

public abstract class UserRegistrationViewModel : 
                                        INotifyPropertyChanged, IDataErrorInfo, IUserViewModel
{
    private static readonly ViewModelGenerator viewModelsGenerator = new ViewModelGenerator();

    public static UserRegistrationViewModel CreateUserRegistrationViewModel(User user)
    {
        return viewModelsGenerator.Generate<UserRegistrationViewModel>(user);
    }

    protected UserRegistrationViewModel() { }

    private const string MANDATORY_FIELD_ERROR_MESSAGE = "This field is mandatory";
    private const string PASSWORD_VERIFICATION_PROPERTY_NAME = "PasswordVerification";

    [Model]
    private readonly User user;
    private ICommand save;

    public event PropertyChangedEventHandler PropertyChanged = (sender, args) => { };

    [Validation(typeof (MandatoryValidator))]
    public abstract string Name { get; set; }

    [Validation(typeof(EmailValidator))]
    [Validation(typeof (MandatoryValidator))]
    public abstract string Email { get; set; }

    [Validation(typeof(MandatoryValidator))]
    [RelatedProperty(PASSWORD_VERIFICATION_PROPERTY_NAME)]
    public abstract string Password { get; set; }

    private string passwordVerification;
    public string PasswordVerification
    {
        get { return passwordVerification; }
        set
        {
            passwordVerification = value;
        }
    }

    private string GetPasswordVerificationValidation()
    {
        if(string.IsNullOrEmpty(passwordVerification))
        {
            return MANDATORY_FIELD_ERROR_MESSAGE;
        }
        
        if (user.Password != passwordVerification)
        {
            return "Password and Verification must be same";
        }

        return null;
    }

    public virtual string this[string columnName]
    {
        get
        {
            if (columnName == PASSWORD_VERIFICATION_PROPERTY_NAME)
            {
                return GetPasswordVerificationValidation();
            }

            return null;
        }
    }

    public string Error { get { return null; } }

    public ICommand Save
    {
        get
        {
            if (save == null)
            {
                save = new SaveCommand(user);
            }

            return save;
        }
    }
}

Let’s see how this code demonstrates the solutions to some of the missing features.

Accessing the model – in order to to get an instance of the model, a new attribute is presented [Model]. Decorating a field with it will make sure that field is initialized with the model instance.

Raising PropertyChaned event on other property than the one being set – in order to deal with it a new attribute is presented [RelatedProperty]. When the property is being set, the PropertyChanged event will be raised for all properties – the one being set and the related properties.

Defining additional errors for non-mapped properties – the generated View-Model tries to resolve errors for mapped properties, if it find none it forwards the call the base View-Model (the abstract class) and checks for additional errors.

Download

The framework source can be found in CodePlex. It is still very partial and contains many bugs but it already works on the basic scenarios :)

Automatic generation of View-Model - first attempt

noreply@blogger.com (Elisha) — Thu, 19 Aug 2010 21:23:00 +0000

I recently found myself writing the same code again and again. As can be guessed, I’m writing a WPF application based on MVVM architecture. In order to avoid writing the same code again I am trying to generate the View-Model automatically using Castle Dynamic Proxy.

Simplest scenario – forwarding calls to model

This is probably the simplest case we encounter. This is so simple we are tempted to bind the view directly to the model.

Naive implementation

public class Model
{
    public object Prop { get; set; }
}

public class ViewModel
{
    private readonly Model model;

    public ViewModel(Model model)
    {
        this.model = model;
    }

    public object Prop
    {
        get { return model.Prop; }
        set { model.Prop = value; }
    }
}

The generated alternative

So, what we’d like to achieve is skipping the forwarding implementation. The View-Model can look like:

public abstract class ViewModel
{
    public abstract object Prop { get; set; }
}

So far, it’s very simple :) Let’s see how it’s being used with the Model:

[Test]
public void Generate_SetPropertyValue_ModelPropertyUpdated()
{
    var viewModelGenerator = new ViewModelGenerator();
    var model = new Model();
    
    ViewModel generatedViewModel = viewModelGenerator.Generate<ViewModel>(model);

    object valueToAssign = new object();
    generatedViewModel.Prop = valueToAssign;

    Assert.That(model.Prop, Is.SameAs(valueToAssign));
}

This example shows that we’ve created a View-Model based on a Model instance, simulated a call to a property setter and the new value automatically reflected the Model. That was simple, wasn’t it?

Next scenario – Implementing INotifyPropertyChanged

This is a very common scenario, which has very common implementation. It’s so common I’ll skip the naive implementation and jump to the generated version directly:

The generated alternative

public abstract class ViewModel : INotifyPropertyChanged
{
    public abstract object Prop { get; set; }
    public event PropertyChangedEventHandler PropertyChanged;
}

[Test]
public void Generate_SetPropertyValue_PropertyChangedRaised()
{
    var viewModelGenerator = new ViewModelGenerator();
    var model = new Model();
    ViewModel generatedViewModel = viewModelGenerator.Generate<ViewModel>(model);

    bool wasRaised = false;
    generatedViewModel.PropertyChanged += (sender, args) => wasRaised = true;

    generatedViewModel.Prop = new object();

    Assert.That(wasRaised, Is.True);
}

In this case we’ve generated a View-Model that implements INotifyPropertyChanged. We had to do nothing but declaring the View-Model implements INotifyPropertyChanged. Whenever a property value is changed the event will be raised with the property name.

Last scenario – Implementing IDataErrorInfo

This case is trivial too so I’ll skip again the naive solution.

The generated alternative

First we’ll take a look at the format of the abstract View-Model and the validation declaration:

public abstract class ViewModel : IDataErrorInfo
{
    public abstract string this[string columnName] { get; }
    public abstract string Error { get; }

    [Validation(typeof(DummyValidator))]
    public abstract object Prop { get; set; }
}

public class DummyValidator : IValidator
{
    public string Validate(object value)
    {
        return "error";
    }
}

The view model is implementing now the IDataErrorInfo interface. The interfaces require implementation of indexer that maps from property to error message. The automatic View-Model generation should take care of it. In order to declare the required validations we’ll use the Validation attribute. The attribute defines which validator to use on the property. The validation result will be mapped to the property name.

For example:

[Test]
public void Generate_SetInvalidPropertyValue_PropertyErrorIsCorrect()
{
    var viewModelGenerator = new ViewModelGenerator();
    var model = new Model();

    ViewModel generatedViewModel = viewModelGenerator.Generate<ViewModel>(model);

    generatedViewModel.Prop = new object();

    Assert.That(generatedViewModel["Prop"], Is.EqualTo("error"));
}

Conclusion

MVVM is used many times in common and similar scenarios. The code is usually a template which we can generate dynamically. This way the code duplication will be drastically reduced and allow uniform implementation.

The examples here were simplified but the concept should be clear. There’s much more work on the framework in order to cover more real life scenarios, soon to come :) I’ll upload framework the source code to CodePlex before the next post.

Balancing working hours toward better productivity

noreply@blogger.com (Elisha) — Sat, 24 Jul 2010 14:53:00 +0000

I'll start by saying - I don't know what's the best balance. Then, what can I tell?

The obvious

The ratio between our working hours and productivity is not linear. We all know that, nothing new here. If we work 12 hours a day instead of 10 we won’t produce 20% more features.
When we need to get more work done, we work more hours.

The almost obvious

How’s our productivity looks like during the day?

Everybody agrees about this graph until the productivity gets closer to zero. But, is possible to have negative productivity ?

Negative productivity

This is the unintuitive part, we encountered it so many times but it’s still hard to accept it. From some part of the day, we do more harm than good – we cause more bugs, produce fragile design and less readable code. Even though our feature might work just fine, we did more harm than good – next week, when we have to extended the feature, we’ll have to understand code in inferior standard. We’ll fight the design and most likely, we’ll fight the bugs we missed before.
From my personal experience, during the last year I found myself fighting with features for very long hours, in most of the times where I gave up and left office the next morning was extremely effective. I could throw away all the mess I did during the previous evening and write nice code within an hour.

Conclusion

The conclusion is very straight forward – when you’re getting to the zone of overwork, Go Home! You’re wasting your time, you’re wasting your boss’s money and you’re planting seeds in code that’ll annoy your teammates in the near future. We all know this is counter intuitive, but working less, closer to balanced hours, makes us more productive.

More thoughts

We have concentrated on the productivity of a single day. A bit more interesting can be to analyze a whole week, is it possible that working one day less a week will improve our productivity? If we’d find it to be true, will we act and shorten our work week? If we’d reduce shorten our work week, should we get paid less or more? On one hand we’re more productive, so we should be paid more, but on the other hand, we work less, so we should be paid less. This lead to an interesting question - are we paid for our time or for our results?
There’s a lot to think about here, but we must find first the optimal balance of working hours and days.

Worker thread using parallel tasks

noreply@blogger.com (Elisha) — Thu, 22 Jul 2010 05:04:00 +0000

Worker thread is a known pattern – there’s work to do, it needs to be done asynchronously and we want to get all the work results when it’s ready. What we’re going to see is an implementation of it as an alternative to the common implemenatations. This implemenation will take advantage of the new parallel tasks library.
To formalize the requirements:

The worker queues items to process
The items are processed asynchronously
Only one item can be proceesed at a time
The items are proceesed in the order they were queued
The worker will store the processed results in the order they were processed

The worker class

public class Worker<TItem, TResult>
{
    private readonly IItemsProcessor<TItem, TResult> itemsProcessor;
    private Task lastTask;

    public IList<TResult> ProcessedItems { get; private set; }

    public Worker(IItemsProcessor<TItem,TResult> itemsProcessor)
    {
        this.itemsProcessor = itemsProcessor;
        ProcessedItems = new List<TResult>();
        InitializeNullTask();
    }

    private void InitializeNullTask()
    {
        lastTask = new Task<TResult>(() => default(TResult));
        lastTask.Start();
    }

    public void ProcessItem(TItem item)
    {
        var nextTask = lastTask
            .ContinueWith(task =>
                              {
                                  var processItem = itemsProcessor.ProcessItem(item);
                                  ProcessedItems.Add(processItem);
                              });
        lastTask = nextTask;
    }

    public void WaitForPendingItems()
    {
        using (var sync = new ManualResetEvent(false))
        {
            lastTask.ContinueWith(task => sync.Set());
            sync.WaitOne();
        }
    }
}

The worker creates a a task for each item needs to be processed. Each task is executed in the thread pool, the point where we ensure that the tasks are ran in the correct order is the ContinueWith call. ContinueWith takes care for the order of the tasks execution.

The InitializeNullTask creates a task that, suprisingly, does nothing but to set a head to the tasks queue. This task helps us avoid in ProcessItem to check if this is the first item to process. The first task starts with call to Start while all the other starts with ContinueWith.

WaitForPendingItems also enques a task. This time, the task is only waiting to be executed, which means all other items were already processed. When the task starts it releases the enquing thread.

Usage example

In this example we’ll download a list of web pages and check print their sizes. The downloader impelments the IItemsProcessor we’ve seen the worker expects.

public class WebUrlsDownloader : IItemsProcessor<string, byte[]>
{
    public byte[] ProcessItem(string url)
    {
        using (var webClient = new WebClient())
        {
            return webClient.DownloadData(url);
        }
    }
}

And the actual usage:

public void DownloadFiles()
{
    var worker = new Worker<string, byte[]>(new WebUrlsDownloader());
    worker.ProcessItem(@"http://msdn.microsoft.com/en-us/library/dd537608.aspx");
    worker.ProcessItem(@"http://msdn.microsoft.com/en-us/library/dd537609.aspx");
    worker.ProcessItem(@"http://msdn.microsoft.com/en-us/library/dd997405.aspx");
    worker.WaitForPendingItems();
    Console.WriteLine("Finished downloading files:");
    foreach (var processedItem in worker.ProcessedItems)
    {
        Console.WriteLine("Downloaded file with size: {0}", processedItem.Length);
    }
}

Unit testing F#

noreply@blogger.com (Elisha) — Mon, 12 Jul 2010 21:19:00 +0000

F# is a cool and exiting language. It’s much more than an academic language, it can solve many real life (coding) problems in its functional manner. As to production code, it must be covered with unit tests. I’ll show a simple example of unit test written in C# with moq against F# code.
The code under test is a simple lottery calculator, it takes a list of participants and tells how much every participant won - 5 times the number of hits. Simple, isn’t it?
Let’s take a look at the code under test, it could probably be written better, but ignore it for now :)

type LotteryCalculator(winningNumbers:System.Collections.Generic.IList<System.Int32>) = 
    let calculatePrize(participatingNumbers) =
        let numOfHits = participatingNumbers |> 
                        Seq.filter (fun participatingNumber -> winningNumbers.Contains participatingNumber) |>
                        Set.ofSeq |> 
                        Set.count
        numOfHits * 5
    
    member this.CalculatePrizes(participants:System.Collections.Generic.IEnumerable<IParticipant>) =
        participants
        |> Seq.map (fun participant -> (participant, calculatePrize(participant.GetTicket())))

So, we have an API method, CalculatePrizes(), which take a collection of participants, look at their tickets and returns a tuples list of participant and prize. So we have something like this: CalculatePrizes: System.Collections.Generic.IEnumerable<IParticipant> –> seq<IParticipant * int>

After dealing with some issues, on which I’ll tell in a future post, I wrote this test:

[Test]
public void CalculatePrizes_RecievesTwoParticipant_MatchCorrectPrizes()
{
    var firstParticipantMock = new Moq.Mock<IParticipant>();
    IParticipant firstParticipant = firstParticipantMock.Object;
    firstParticipantMock.Setup(fake => fake.GetTicket()).Returns(new List<int> {1, 4, 5});
    
    var secondParticipantMock = new Moq.Mock<IParticipant>();
    IParticipant secondParticipant = secondParticipantMock.Object;
    secondParticipantMock.Setup(fake => fake.GetTicket()).Returns(new List<int> { 1, 2, 4 });

    var lotteryCalculator = new LotteryCalculator(new List<int> {1, 2, 3});
    var participantPrizes = lotteryCalculator.CalculatePrizes(new[] { firstParticipant, secondParticipant });

    var prizes = participantPrizes.ToDictionary(tuple => tuple.Item1, tuple => tuple.Item2);

    Assert.That(prizes[firstParticipant], Is.EqualTo(5));
    Assert.That(prizes[secondParticipant], Is.EqualTo(10));
}

It’s nice that we can keep our unit tests written in C#, it’s also important in cases where the users of the code (most likely us in other module) will use it in C# as well.

Code contracts – Where invariant method called?

noreply@blogger.com (Elisha) — Sat, 09 Jan 2010 14:23:00 +0000

I assume you’re already know code contracts and heard about invariant methods. This post will demonstrate how simple code compiles with invariant method.

This simple class declares a method as invariant method:

public class InvariantExample
{
    public void Method()
    {
        Console.WriteLine("In Method");
    }

    [ContractInvariantMethod]
    protected void InvariantMethod()
    {
        Contract.Invariant(2 > 1);
    }
}

Using reflector we can see that Method compiles to:

public void Method()
{
    Console.WriteLine("In Method");
    this.$InvariantMethod$();
}

As we can see the compiled method ends with a call to our invariant method. Assuming we had more methods in our class we would have seen similar call in each one of them. The invariants is less explicit than most contracts features but it's a powerful part of it, use wisely!

Easy Wins Optimizations – HashSet

noreply@blogger.com (Elisha) — Mon, 07 Dec 2009 07:16:00 +0000

HashSet is a data structure introduced in .Net 3.5 framework. The main advantage of HashSet is that it adds and fetches items in very low complexity (constant amortized time). We’ll try to see how faster can it be over List. In the example we’ll go over code that stores all historical lottery results and checks if some results appear among the historical ones.

Registering results

public abstract class ResultsRegistrarBase
{
    protected ICollection<long> codes;

    public void AddResult(int num1, int num2, int num3, int num4, int num5, int num6)
    {
        long code = ConvertToCode(num1, num2, num3, num4, num5, num6);

        codes.Add(code);
    }

    public bool ContainsResult(int num1, int num2, int num3, int num4, int num5, int num6)
    {
        long code = ConvertToCode(num1, num2, num3, num4, num5, num6);

        return codes.Contains(code);
    }

    private static long ConvertToCode(int num1, int num2, int num3, int num4, int num5, int num6)
    {
        return num1 + (long) Math.Pow(10, 0) + num2 * (long) Math.Pow(10, 2) +
               num3 * (long) Math.Pow(10, 4) + num4 * (long) Math.Pow(10, 6) +
               num5 * (long) Math.Pow(10, 8) + num6 * (long) Math.Pow(10, 10);
    }
}

This class is in charge of storing and finding the results. It assumes it gets the numbers sorted and the ConvertToCode method maps all the numbers to one “big" number”. For example: 1,5,20,30,45,49 will be mapped to 494530200501. The codes collection will be initialized in a base class:

The naive code:

public class NaiveResultsRegistrar : ResultsRegistrarBase
{
    public NaiveResultsRegistrar()
    {
        codes = new List<long>();
    }
}

The code based on HashSet:

public class HashSetResultsRegistrar : ResultsRegistrarBase
{
    public HashSetResultsRegistrar()
    {
        codes = new HashSet<long>();
    }
}

How faster is it over the the naive code?

The difference is huge, on a simple case where we have 250,000 historical results and we’re querying the collection 50,000 times the average running time of the naive code is 1:28 minutes and the code based on HashSet takes only 0.2 seconds. The HashSet code run ~360 times faster than the naive code. The reason for this major change is that List has to iterate over all the items in order to determine if it contains the requested item. HashSet uses a hash method, without getting to implementation details, it lets enables the contains look in very few entries in order to determine if an item is contained or not in the collection.

This is a major improvement was received using minimal change in the code. So in cases where we query a large collection whether some items are in it, usage of HashSet can give us major running time improvement with minimal change.

How to pass LINQ-To-SQL data objects with associations through WCF service?

noreply@blogger.com (Elisha) — Thu, 03 Dec 2009 09:24:00 +0000

Sometimes we need to pass a simple data object through WCF service. It has few simple steps for performing it.

Make data objects declare columns as Data Members

This step is very simple, all needed is to define in the DBML editor that the serialization mode is Unidirectional.

By defining this serialization mode the generated code will be decorated with DataContract and DataMember attributes on classes and columns. By doing so the objects are compatible with WCF service and declared as part of the contract.

Visit the objects associations properties before returning them from the service

For example, if we’re dealing with this simple schema and our service returns a customer with requested ID.

Assuming we’re having this naive code:

public class StoreService : IStoreService
{
    public Customer GetCustomer(int id)
    {
        using (var store = new StoreDataContext())
        {
            return store.Customers.First(dbCustomer => dbCustomer.ID == id);
        }
    }
}

The client will get the correct customer, but it won’t have the Orders property filled, even if the customer has orders the property value will be null. The reason for this behavior is that the generated code doesn’t execute the query against the database when it’s serializing. The solution is to access all data members before it serializes.

public class StoreService : IStoreService
{
    public Customer GetCustomerWithRelations(int id)
    {
        using (var store = new StoreDataContext())
        {
            Customer customer = store.Customers.First(dbCustomer => dbCustomer.ID == id);
            VisitDataMembers(customer);            
            return customer;
        }
    }

    public static void VisitDataMembers(object dataMember)
    {
        if (dataMember == null)
        {
            return;
        }

        bool isEnumerable = dataMember is IEnumerable;
        if (isEnumerable)
        {
            foreach (var item in (IEnumerable)dataMember)
            {
                VisitDataMembers(item);
            }
        }

        var properties = dataMember.GetType().GetProperties();
        IEnumerable<PropertyInfo> dataMemberProperties = 
            properties.Where(property => property.IsDefined(typeof(DataMemberAttribute), true));
        foreach (PropertyInfo dataMemberProperty in dataMemberProperties)
        {
            var dataMemberValue = dataMemberProperty.GetValue(dataMember, null);
            VisitDataMembers(dataMemberValue);            
        }
    }
}

This VisitDataMembers method ensures all data members properties are accessed at least once before serialization and the client will be able to use all the data which is accessible for the data object tree. For example in the sample schema, the client will be able to get from customer to orders to products to materials.

Easy Wins Optimizations – Sorted List

noreply@blogger.com (Elisha) — Wed, 25 Nov 2009 20:05:00 +0000

This is the first post of easy wins optimizations posts. The need for these posts comes from the fact the most of us hardly ever need it :) Sounds wired? It should. What it means is that we don’t need in most of the time, the most naive code does the best work for us. When we encounter the case that requires optimization we hardly remember the simplest ways of doing it.
In this post on sorted list we’ll go over an example of optimizing simple case.

How to find a person in a collection whose age is closest mine?

Assume we have a person class with age defined on it:

public class Person
{
    public double Age { get; private set; }

    public Person(double age)
    {
        Age = age;
    }
}

What we want is to collect many people into a list, and any time we asked about an age, we find the closest person to this age. We can think of it as an application for dating service that match people by age.

The naive solution

So what can be simpler than collecting all persons to a list? Probably nothing, this is how we do it:

public class NaivePersonFinder : IPersonFinder
{
    private readonly List<Person> persons = new List<Person>();

    public void Add(Person person)
    {
        persons.Add(person);
    }

    public Person FindPeopleInRange(double requiredAge)
    {
        Person closestPerson = null;
        double smallestDiffrence = double.MaxValue;
        foreach (var person in persons)
        {
            double difference = Math.Abs(person.Age - requiredAge);
            if (difference < smallestDiffrence)
            {
                smallestDiffrence = difference;
                closestPerson = person;
            }
        }

        return closestPerson;
    }
}

The code iterates over the collection in the most straight forward way there is and updates the closest person whenever it encounter closer person. Simple! Isn’t it?

But I need faster results!

The sorted list solution
How does sorted list helps us here? Sorted list allow us to search items using Binary Search. Binary search saves a lot of time since it has to query very few items in the collection in order to find the closest one. For example, to find the closest item in a collection of 1,000 items it’ll have to query ~10 and saves iteration over the hole 1,000.

Some code

public class SortedPersonFinder : IPersonFinder
{
    private readonly SortedList<double, Person> persons = new SortedList<double, Person>();

    public void Add(Person person)
    {
        persons.Add(person.Age, person);
    }

    public Person FindPeopleInRange(double requiredAge)
    {
        if (persons.ContainsKey(requiredAge))
        {
            return persons[requiredAge];
        }

        IList<double> sortedAges = persons.Keys;

        int index = BinarySearchIndex(requiredAge);

        var closestPersons = new List<Person>(3);

        if (index > 0)
        {
            closestPersons.Add(persons[sortedAges[index - 1]]);
        }

        if (index < sortedAges.Count - 1)
        {
            closestPersons.Add(persons[sortedAges[index + 1]]);
        }

        closestPersons.Add(persons[sortedAges[index]]);

        return closestPersons.OrderBy(person => Math.Abs(person.Age - requiredAge)).First();
    }

    private int BinarySearchIndex(double requiredAge)
    {
        var sortedAges = persons.Keys;

        var scopeStart = 0;
        var scopeEnd = sortedAges.Count - 1;

        int index = 0;
        while (scopeEnd > scopeStart )
        {
            index = (scopeStart + scopeEnd) / 2;
            double ageAtIndex = sortedAges[index];
            if (requiredAge > ageAtIndex)
            {
                scopeStart = index + 1;
            }
            else
            {
                scopeEnd = index;
            }
        }
        return index;
    }
}

The code is more complex than the naive version, but it finds the results much faster. It uses .Net Sorted List collection. This is a collection that updates a collection of keys whenever the collection changes. The keys kept ordered and we can easily access them and binary search them.

How much faster is it?

On 100,000 persons collection and 50,000 queries the running time of the sorted list is much more faster than the naive:
The naive solution did it in average time of ~100 seconds
The sorted list solution had average time of ~6 seconds. This is a major improvement, it means that it ran more the 15 times faster than the naive solution.

Conclusion

If we have a collection on which order can be applied AND we also need quick results for queries based on the order sorted list can easily improve running time. We must always remember that the running time is usually negligible even in the most naive code. If no optimization required – don’t optimize, more gain we’ll be earned from simple and maintainable code.