Tobi + C# = T#

Part 3: Adding a dynamic decorator to an object – the “Why not”

Tobias Hertkorn — Mon, 21 Dec 2009 16:44:50 +0000

Krzysztof pointed me to his post Castle Dynamic Proxy FAQ: why there’s no “class proxy with target” why the dynamic decorator is not part of Castles’s DynamicProxy implementation yet.

Looking at the code one realizes that the real problem is, that IsAdult() is not virtual. That’s why the decorator is not able to intercept and redirect that call to the target. There is no way around it – that is a real bug and it does show why a class proxy with a target is indeed dangerous. Thanks for pointing it out to me, Krzysztof!

The question we have to ask now is: If we refuse to generate the proxy, if any member more visible than protected is non-virtual, would that get rid of the bug? I would argue that it will, since the proxy will now be able to redirect the call to the target and everything works out fine. Or am I again missing something?

Tweet this

Part 2: Adding a dynamic decorator to an object – the “Why”

Tobias Hertkorn — Sun, 20 Dec 2009 16:03:31 +0000

I got the question "Why would we do that, we can't guarantee that nobody would use the original reference to manipulate stuff."

Well, this question is answered best by an example.

Imagine an object with a complicated creation and/hydration path. We are in a multithreaded environment and the object is passed between theads often, etc. There are to approaches to ensure that on has no data corruption because of multi threading. Either locking or immutability. Immutability is a strong feature but usually it is quite clumsy to use. Especially since with purely immutable objects there can be a lot of memory pressure with all the copying and creating of objects. Think "creating a complex immutable tree".

A solution to circumvent the memory pressure is to have a mutable object during creation and marking it as immutable after it is fully hydrated. This usually leads in C# to code similar to this:

PLAIN TEXT

C#:

public class MyImmutable
{
private string m_property;
public string Property
{
get { return m_property; }
set
{
if (m_immutable) { throw new MemberAccessException("Type is marked as immutable"); }
m_property = value;
}
}
private bool m_immutable = false;
public void MarkAsImmutable()
{
m_immutable = true;
}
}
public class MyImmutableFactory
{
public MyImmutable CreateMyImmutable()
{
MyImmutable myImmutable = new MyImmutable();
myImmutable.Property = "Test";
myImmutable.MarkAsImmutable();
return myImmutable;
}
}
public static class ImmutableOldStyleTest
{
public static void Test()
{
MyImmutableFactory myImmutableFactory = new MyImmutableFactory();
MyImmutable myImmutable = myImmutableFactory.CreateMyImmutable();
bool caughtException = false;
try
{
myImmutable.Property = "Test2";
}
catch (MemberAccessException)
{
caughtException = true;
}
Debug.Assert(caughtException);
Debug.Assert(myImmutable.Property == "Test");
}
}

For each set it is the programmer's resposibility to check within the setter, if it may or may not proceed with the setting of the value. Which can be quite tiring for multiple properties (DRY for the check!) and multiple immutable types (DRY for the MarkAsImmutable() method).

An alterative approach would be to use a decorator:

PLAIN TEXT

C#:

public class ImmutableInterceptor : IInterceptor
{
public void InterceptGetter(IProxy proxy, MyType @object, string propertyName, out object value)
{
proxy.ProceedGetter(propertyName, out value);
}
public void InterceptSetter(IProxy proxy, MyType @object, string propertyName, object value)
{
throw new MemberAccessException("Type is marked as immutable");
}
}
public class MyTypeFactory
{
public MyType CreateMyType()
{
MyType myType = new MyType();
myType.Property = "Test";
return MarkAsImmutable(myType);
}
private MyType MarkAsImmutable(MyType myType)
{
ImmutableInterceptor immutableInterceptor = new ImmutableInterceptor();
return GenerateProxy(myType, immutableInterceptor);
}
// This method belongs in a proxy generator
private static MyType GenerateProxy(MyType myType, IInterceptor interceptor)
{
return new MyTypeProxy(myType, interceptor);
}
}
public static class ImmutableTest
{
public static void Test()
{
MyTypeFactory myTypeFactory = new MyTypeFactory();
MyType myType = myTypeFactory.CreateMyType();
bool caughtException = false;
try
{
myType.Property = "Test2";
}
catch (MemberAccessException)
{
caughtException = true;
}
Debug.Assert(caughtException);
Debug.Assert(myType.Property == "Test");
}
}

Way more straight forward, isn't it?

Another application could be the old INotifyPropertyChanged problem, read-only collections, ...

Tweet this

Adding a dynamic decorator to an object

Tobias Hertkorn — Sat, 19 Dec 2009 16:01:29 +0000

I am really confused that non of the dynamic proxy frameworks out there allow me to add dynamic behavior after the fact - I have an instantiated object and need to add behavior when a property is accessed. Like this:

PLAIN TEXT

C#:

var myType = new MyType();
myType.Property = "Test";
var interceptor = new MyInterceptor();
MyType wrappedMyType = proxyGenerator.GenerateProxy(myType, interceptor);
wrappedMyType.Property = "Test2";
Debug.Assert(myType.Property == "Test2");
Debug.Assert(interceptor.SetterWasCalled);

All current frameworks need me to instantiate the object using the proxy generator. Which is unfortunate, because, unlike in the example, I plan to introduce the decorator in a part of my application that is conceptionally in a completely different region of the application's source code. It would just not make sense to introduce the knowledge about the additional behavior at the point of object creation.

It should be fairly easy to pull off, since I am already able to do it by hand:

PLAIN TEXT

C#:

public class MyInterceptor : IInterceptor
{
public void InterceptGetter(IProxy proxy, MyType @object, string propertyName, out object value)
{
// do something before get
proxy.ProceedGetter(propertyName, out value);
// do something after get
}
public void InterceptSetter(IProxy proxy, MyType @object, string propertyName, object value)
{
// do something before set
proxy.ProceedSetter(propertyName, value);
// do something after set
m_setterCalled = true;
}
private bool m_setterCalled = false;
public bool SetterWasCalled
{
get { return m_setterCalled; }
}
}
public class MyType
{
public virtual string Property { get; set; }
}
/////// AUTOGENERATED PROXY ///////
public class MyTypeProxy : MyType, IProxy
{
private MyType m_object;
private IInterceptor m_interceptor;
public MyTypeProxy(MyType @object, IInterceptor interceptor)
{
m_object = @object;
m_interceptor = interceptor;
}
public override string Property
{
get
{
object value;
m_interceptor.InterceptGetter(this, m_object, "Property", out value);
return (string)value;
}
set { m_interceptor.InterceptSetter(this, m_object, "Property", value); }
}
public void ProceedGetter(string propertyName, out object value)
{
if (propertyName == "Property")
{
value = m_object.Property;
}
else
{
throw new MissingMemberException();
}
}
public void ProceedSetter(string propertyName, object value)
{
if (propertyName == "Property")
{
m_object.Property = (string)value;
}
else
{
throw new MissingMemberException();
}
}
}
/////// SUPPORTING CODE ///////
public interface IProxy
{
void ProceedGetter(string propertyName, out object value);
void ProceedSetter(string propertyName, object value);
}
public interface IInterceptor
{
void InterceptGetter(IProxy proxy, T @object, string propertyName, out object value);
void InterceptSetter(IProxy proxy, T @object, string propertyName, object value);
}
/////// TEST ///////
public static class Test
{
public static void ProxyTest()
{
var myType = new MyType();
myType.Property = "Test";
var interceptor = new MyInterceptor();
MyType wrappedMyType = GenerateProxy(myType, interceptor);
wrappedMyType.Property = "Test2";
Debug.Assert(myType.Property == "Test2");
Debug.Assert(interceptor.SetterWasCalled);
}
// This method belongs in a proxy generator
private static MyType GenerateProxy(MyType myType, IInterceptor interceptor)
{
return new MyTypeProxy(myType, interceptor);
}
}

Who will help me to adapt the API to the Castle DynamicProxy API and to write the actual proxy/decorator generation?

Tweet this

Silverlight 4 Beta released

Tobias Hertkorn — Wed, 18 Nov 2009 19:35:06 +0000

AWESOME. This just made my day:

Silverlight 4 beta

This sounds like a great release that smoothes the edges around ADO, out of browser and code reuse between WPF and Silverlight. That's HUGE!

Plus PRINTING! Finally. And the mic and webcam features are nothing to sneeze at either. The list goes on and on. Seriously, they really packed a lot of stuff in. And it seems to be all community feedback driven. They really listened to us over at uservoice. 8 out of the top 10 features actually made it into that release. Great work!

Enjoy!

Tweet this

Bootstrapping easy_install for python 2.6 on Windows

Tobias Hertkorn — Fri, 04 Sep 2009 12:49:06 +0000

I had to setup Python 2.6 on Windows. And of course I wanted to use the magnificent Setuptools, specifically easy_install. Unfortunatelly there is only an msi for 2.5 on the website. But actually bootstrapping the 2.6 egg is not that hard:

Download the 2.6 egg from the setuptools download site. Put it in a temporary directory like c:\downloads\setuptools. Then unzip the egg to the same folder.

Open a command line, go to c:\downloads\setuptools and use the following command:

python -c "import sys; sys.path.insert(0, 'C:/Downloads/setuptools/setuptools-0.6c9-py2.6.egg'); from setuptools.command.easy_install import bootstrap; sys.exit(bootstrap())"

That should be it.

Tweet this

C# is not (yet) a high perf language

Tobias Hertkorn — Sun, 02 Aug 2009 16:57:12 +0000

Disclamer: The lcg used in this article has drawbacks but it was sufficiently random for my purposes. Please be sure to check if that assumption holds for you as well before using!

Well, this won't come to you as a shocker - but C# is not a high performance language... No, wrong, the C# compiler is not primarily optimized for performance. I noticed it while playing around with a linear congruential generator, a fast and lightweight pseudo random generator.

The lcg is in essence this code fragment:

PLAIN TEXT

C#:

PRN = (A * PRN + C) % M; // [0;M[

which generates the next pseudo random number (PRN) which is a number between 0 and M (0 inclusive, M exclusive), using carefully chosen constants A, C and M.

In order to generate a double between 0 and max the naive way to do so is like this:

PLAIN TEXT

C#:

private static double RandomLcg(double max)
{
PRN = (A * PRN + C) % M; // [0;M[
return max * PRN / M;
}

Now what would you say, if I told you that this optimization, no not optimization - reordering,

PLAIN TEXT

C#:

private static double RandomLcg(double max)
{
PRN = (PRN * A + C) % M; // [0;M[
return max * (PRN * (1.0 / M));
}

is almost TWICE at fast as the original? Curious, isn't it?!

The fun part is - manually inlining the PRN write and access gives another 10% improvement. So the final version

PLAIN TEXT

C#:

private static double RandomLcg(double max)
{
return max * ((PRN = (PRN * A + C) % M) * (1.0 / M));
}

generates 1,000,000,000 pseudo numbers in 11,4 seconds on my machine - the original version needs 20,7 seconds.

This kind of performance improvement is not uncommon in any language, but especially the manual inlining surprised me. I guess it is a sign that the compiler team is - understandibly - focusing on other things than extracting the last bit of performance from C#. After all, who needs to regularly generate a billion PRNs using C#?

Download source for crude test program.

Tweet this

Optimizing heavy arithmetics in Mono

Tobias Hertkorn — Mon, 29 Jun 2009 18:38:33 +0000

Sowhat as a tribute to this rather foolish letter.

---

This will be an article about how I optimized the md5 algorithm in Mono and the Microsoft .NET Framework. I haven't completed it yet, but there is quite a bit about optimization in my blog. Check out all the mentions of md5 in my blog for some pointers.

If you need my work on this right now, raw and unpolished - please feel free to email me at tobias at hertkorn dot com. Or to get some hand-on example, go to my post about the optimized Md5CryptoServiceProvider in mono.

Optimizing the md5 algo on little endian machines

This decoding step is a major part within the md5 algo. It translates a 64 byte array into a 16 uint array. The general endian-safe step looks like this:

PLAIN TEXT

C#:

for (i = 0; i <16; i++)
{
buff[i] = (uint)(inputBuffer[inputOffset + 4 * i])
| (((uint)(inputBuffer[inputOffset + 4 * i + 1])) <<8 )
| (((uint)(inputBuffer[inputOffset + 4 * i + 2])) <<16)
| (((uint)(inputBuffer[inputOffset + 4 * i + 3])) <<24);
}

If this step is performed on a little endian machine, the bytes are already safed in memory in the right order and there is no need for all the bitlogic. Therefore optimizing it using unsafe code and direct copying 4 consecutive bytes as one uint gives a huge speed increase:

PLAIN TEXT

C#:

unsafe
{
fixed (byte* bFixed = inputBuffer)
{
byte* inputPointer = bFixed + inputOffset;
for (i = 0; i <16; i++)
{
buff[i] = *(uint*)(inputPointer);
inputPointer += 4;
}
}
}

As we are already in unsafe context using an additional pointer for buff gives an additional speed increase:

PLAIN TEXT

C#:

unsafe
{
fixed (byte* bFixed = inputBuffer)
{
fixed (uint* aFixed = buff)
{
byte* inputPointer = bFixed + inputOffset;
uint* bufferPointer = aFixed;
for (i = 0; i <16; i++)
{
*bufferPointer = *(uint*)(inputPointer);
inputPointer += 4;
bufferPointer++;
}
}
}
}

Be sure to still make this Md5 implementation endian safe by introducing a if:

PLAIN TEXT

C#:

if (BitConverter.IsLittleEndian)
{
// unsafe decode
}
else
{
// bitshifting decode
}

Avoiding unnecessary stloc and ldloc ops

If possible do as much arithmetic as possible in one go. For example:

PLAIN TEXT

C#:

b += (((d ^ a) & c) ^ a) + (uint)K[7] + buff[7];
b = (b <<22) | (b>> 10);
b += c;

This b += c is not smart to use in this situation and actually produces a performance hit. Better would be:

PLAIN TEXT

C#:

b += (((d ^ a) & c) ^ a) + (uint)K[7] + buff[7];
b = ((b <<22) | (b>> 10)) + c;

Altering the whole segment like this in the md5 algorithm adds up to a 10-15% performance increase!

Let's look at the IL:

PLAIN TEXT

IL:

ldloc.1
ldloc.3
ldloc.0
xor
ldloc.2
and
ldloc.0
xor
ldsfld unsigned int32[] MD5CryptoServiceProviderMonoOrig::K
ldc.i4.7
ldelem.u4
add
ldarg.0
ldfld unsigned int32[] MD5CryptoServiceProviderMonoOrig::buff
ldc.i4.7
ldelem.u4
add
add
stloc.1
ldloc.1
ldc.i4.s 22
shl
ldloc.1
ldc.i4.s 10
shr.un
or
stloc.1 // stores b
ldloc.1 // loads b
ldloc.2
add
stloc.1

the second version's:

PLAIN TEXT

IL:

ldloc.1
ldloc.3
ldloc.0
xor
ldloc.2
and
ldloc.0
xor
ldsfld unsigned int32[] MD5AddOptimization3::K
ldc.i4.7
ldelem.u4
add
ldarg.0
ldfld unsigned int32[] MD5AddOptimization3::buff
ldc.i4.7
ldelem.u4
add
add
stloc.1
ldloc.1
ldc.i4.s 22
shl
ldloc.1
ldc.i4.s 10
shr.un
or
// store, load eliminated
ldloc.2
add
stloc.1

Tweet this

Version 1.2.1 of Hg Commit Monitor released

Tobias Hertkorn — Sun, 28 Jun 2009 23:32:32 +0000

Just finished a new version of Hg Commit Monitor. This version now includes ssh support, additional error handling and prevents a bug in the atom exporter of hgweb from influencing the updates. Enjoy.

Please leave a comment if I should follow Richard Stallman's plea to drop C# for Hg Commit Monitor - and rewrite the whole thing in say ... vb?

Tweet this

Stylishly improving Silverlight Toolkit Chart’s Performance

Tobias Hertkorn — Fri, 26 Jun 2009 22:13:16 +0000

TL;TR: Skip all the way down and download the complete style without "Show" storyboard for the Silverlight Toolkit March 2009 release.

I am currently looking at a couple of Silverlight features and extensions. And what's about the first thing one wants to see - GRAPHS. Well, that's me at least - maybe my background in Physics does push me in weird ways ~~most of the times~~ sometimes...

What I found was that the performance of the charts is not that great - when going to more than a couple datapoints ... to say like 1000?! Oh, and of course those datapoints should change all the time, right? Turns out that does kinda have some impact on performance - since every single datapoint has an animation attached to it when it gets shown or hidden.

I am a total noob when it comes to XAML in general and Silverlight in particular - so it was a huge break-through for me to find out that one can change the style of existing controls. Yeah I know. Knocked me out though. So I got the trial of Expressoin Blend 2, did all the steps that were necessary to get to the templates. And found out that now all my datapoints where bright orange. Umm, that's not fun.

So after a bit more of fumbling around I found out that it's all very easy - when one knows how to do it... I know this is one of the "aha, actually he is not that bright" kinda revelation blog posts. Aaaanyway.

The new template for a datapoint (in this particular case a LineDataPoint for LineSeries) that has no fade in translation looks like this:

PLAIN TEXT

XML:

TargetType='charting:LineDataPoint' x:Key='LineNoTransition'>
Opacity='1' x:Name='Root'>
:VisualStateManager.VisualStateGroups>
:VisualStateGroup x:Name='CommonStates'>
:VisualStateGroup.Transitions>
:VisualTransition GeneratedDuration='0:0:0.1' />
:VisualStateGroup.Transitions>
:VisualState x:Name='Normal' />
:VisualState x:Name='MouseOver'>
>
BeginTime='00:00:00' Duration='00:00:00.0010000' Storyboard.TargetName='MouseOverHighlight' Storyboard.TargetProperty='(Shape.Fill).(SolidColorBrush.Color)'>
KeyTime='00:00:00' Value='#FFFFDF00' />
>
BeginTime='00:00:00' Duration='00:00:00.0010000' Storyboard.TargetName='MouseOverHighlight' Storyboard.TargetProperty='(UIElement.Opacity)'>
KeyTime='00:00:00' Value='0.24' />
>
>
:VisualState>
:VisualStateGroup>
:VisualStateGroup x:Name='SelectionStates'>
:VisualStateGroup.Transitions>
:VisualTransition GeneratedDuration='0:0:0.1' />
:VisualStateGroup.Transitions>
:VisualState x:Name='Unselected' />
:VisualState x:Name='Selected'>
>
BeginTime='00:00:00' Duration='00:00:00.0010000' Storyboard.TargetName='SelectionHighlight' Storyboard.TargetProperty='(UIElement.Opacity)'>
KeyTime='00:00:00' Value='0.18' />
>
>
:VisualState>
:VisualStateGroup>
:VisualStateGroup x:Name='RevealStates'>
:VisualState x:Name='Shown' />
:VisualState x:Name='Hidden' />
:VisualStateGroup>
:VisualStateManager.VisualStateGroups>
.ToolTip>
Content='{TemplateBinding FormattedDependentValue}' />
.ToolTip>
Fill='{TemplateBinding Background}' Stroke='{TemplateBinding BorderBrush}' />
RenderTransformOrigin='0.661,0.321'>
.Fill>
GradientOrigin='0.681,0.308'>
Color='#00FFFFFF' />
Color='#FF3D3A3A' Offset='1' />
>
.Fill>
>
Fill='Red' Opacity='0' x:Name='SelectionHighlight' />
Fill='White' Opacity='0' x:Name='MouseOverHighlight' />
>
>

Notice that I set the opacity of the "Root" grid to 1 (it is 0 in the original style) and removed all storyboards from the "Shown" and "Hidden" VisualStates. So now we have a ControlTemplate that shows the datapoint right away (opaycity to 1) and does not need any additional CPU cycles for animations (no storyboards).

In order to be able to attach these animation-less templates to the respective data series we need an additional template:

PLAIN TEXT

XML:

TargetType='charting:LineDataPoint' x:Key='LineDataPointStyleBlue'>
Property='Background'>
.Value>
>
.RelativeTransform>
>
CenterX='0.5' CenterY='0.5' ScaleX='2.09' ScaleY='1.819' />
X='-0.425' Y='-0.486' />
>
.RelativeTransform>
Color='#FFB9D6F7' />
Color='#FF284B70' Offset='1' />
>
.Value>
>
Property='BorderBrush' Value='Gray' />
Property='BorderThickness' Value='1' />
Property='IsTabStop' Value='False' />
Property='Width' Value='8' />
Property='Height' Value='8' />
Property='Template' Value='{StaticResource LineNoTransition}' />
>

Notice that I went to the source of the chart library and copied the original style for "Background" getting rid of the hideous "orange". Plus notice the Value for the Template that points to our new animation-less template.

Last but not least attach the new dataline template to the chart:

PLAIN TEXT

XML:

:Chart x:Name="Performance">
:LineSeries
x:Name="SleepTimez"
ItemsSource="{Binding Path=Performance.SleepTimez}"
Title="T"
DependentValuePath="Count"
IndependentValuePath="Time"
DataPointStyle="{StaticResource LineDataPointStyleBlue}" />
:LineSeries
x:Name="ListSleepTimez"
ItemsSource="{Binding Path=Performance.ListSleepTimez}"
Title="LT"
DependentValuePath="Count"
IndependentValuePath="Time"
DataPointStyle="{StaticResource LineDataPointStyleRed}" />
:LineSeries
x:Name="ChartSleepTimez"
ItemsSource="{Binding Path=Performance.ChartSleepTimez}"
Title="CT"
DependentValuePath="Count"
IndependentValuePath="Time"
DataPointStyle="{StaticResource LineDataPointStyleLightGreen}" />
:Chart>

And that's pretty much it. Unfortunatelly I was not able to make the style palette work again - so right now I have to manually choose and attach colors to the various data series. But that's good enough for me right now.

I have not figured out yet how to remove the "hide" transition. It seems as if the removal of datapoints from a graph still trigger an animation, but I don't know why. Any hints?

Download: Complete LineDataPoint style

Tweet this

Falling into the deep hole of Constructor Autodiscovery

Tobias Hertkorn — Wed, 24 Jun 2009 16:14:46 +0000

I am using Unity right now do some IoC. All I wanted to do is get an instance of the WCF Client that was auto-generated for me. So naively I thought this would be enough:

PLAIN TEXT

C#:

m_container.RegisterType();

But that greeted me with a kinda weird error message:

Microsoft.Practices.ObjectBuilder2.BuildFailedException: The current build operation (build key Build Key[Test.ServiceReference.ServiceClient, null]) failed: The current build operation (build key Build Key[Test.ServiceReference.ServiceClient, null]) failed: The type ServiceClient has multiple constructors of length 2. Unable to disambiguate. (Strategy type DynamicMethodConstructorStrategy, index 0) (Strategy type BuildPlanStrategy, index 3) ---> Microsoft.Practices.ObjectBuilder2.BuildFailedException: The current build operation (build key Build Key[Test.ServiceReference.ServiceClient, null]) failed: The type ServiceClient has multiple constructors of length 2. Unable to disambiguate. (Strategy type DynamicMethodConstructorStrategy, index 0) ---> System.InvalidOperationException: The type ServiceClient has multiple constructors of length 2. Unable to disambiguate. bei Microsoft.Practices.ObjectBuilder2.ConstructorSelectorPolicyBase`1.FindLongestConstructor(Type typeToConstruct) bei Microsoft.Practices.ObjectBuilder2.ConstructorSelectorPolicyBase`1.SelectConstructor(IBuilderContext context) bei Microsoft.Practices.ObjectBuilder2.DynamicMethodConstructorStrategy.PreBuildUp(IBuilderContext context) bei Microsoft.Practices.ObjectBuilder2.StrategyChain.ExecuteBuildUp(IBuilderContext context) --- Ende der internen Ausnahmestapelüberwachung --- bei Microsoft.Practices.ObjectBuilder2.StrategyChain.ExecuteBuildUp(IBuilderContext context) bei Microsoft.Practices.ObjectBuilder2.DynamicMethodBuildPlanCreatorPolicy.CreatePlan(IBuilderContext context, Object buildKey) bei Microsoft.Practices.ObjectBuilder2.BuildPlanStrategy.PreBuildUp(IBuilderContext context) bei Microsoft.Practices.ObjectBuilder2.StrategyChain.ExecuteBuildUp(IBuilderContext context) --- Ende der internen Ausnahmestapelüberwachung --- bei Microsoft.Practices.ObjectBuilder2.StrategyChain.ExecuteBuildUp(IBuilderContext context) bei Microsoft.Practices.Unity.ObjectBuilder.NamedTypeDependencyResolverPolicy.Resolve(IBuilderContext context) bei BuildUp_Test.ServiceReference.ServiceClient(IBuilderContext )

Turns out: I am used to Windsor Castle... Apparently Unity thinks of itself high enough to decide what it can and can't construct. Meaning - instead of using the constructor it could satisfy with all the types I registered (in other words the default constructor since I did not register anything else) it went straight for the longest constructor it could find - and got rightfully confused since there are multiple in any service client of length two.

Well, after a bit of soul searching I came up with this solution: "Specify the type of constructor I want Unity to use"

PLAIN TEXT

C#:

m_container.RegisterType()
.Configure()
.ConfigureInjectionFor(new InjectionConstructor());

Now the default constructor is used when resolving IService to ServiceClient. (Note that the constructor of InjectionConstructor has no parameters...)

Still think that the Constructor discovery process of Unity is straight brain-dead. But now it works...

Tweet this

Silverlight, TextBox, TwoWay Binding and the TextChanged event

Tobias Hertkorn — Mon, 22 Jun 2009 16:37:47 +0000

Unfortunatelly Silverlight does not allow to set a UpdateSourceTrigger on the Binding. Therefore when using a TextBox the value of a twoway bound property only changes after the textbox looses the focus.

PLAIN TEXT

XML:

x:Name="Description"
Text="{Binding Path=Description, Mode=TwoWay}" />

This is not the kind of behavior I was looking for when I designed a small tool that should filter a list on the fly. Of course it should update the list on every key-stroke I make to get an immediate GUI response for our filtering.

On the web one only finds the "workaround" to manually make the control loose and gain focus on ever textchanged event. I think that is rather silly. Instead of using that workaround I used the following method:

Switch to OneWay Binding
Manually set the property of my object in the TextChanged event

The resulting XAML looks like this:

PLAIN TEXT

XML:

x:Name="Description"
Text="{Binding Path=Description, Mode=OneWay}"
TextChanged="Description_TextChanged" />

And the code-behind like this:

PLAIN TEXT

C#:

private void Description_TextChanged(object sender, TextChangedEventArgs e)
{
Filter.Description = ((TextBox)sender).Text;
}

It's that simple.

Tweet this

The one and only reason Wolfram Alpha will fail: …

Tobias Hertkorn — Tue, 16 Jun 2009 18:49:00 +0000

... Ego Surfing does not work!

That's all. No need to add anything further - I predict it to fail, because it does not return any meaningful results when I enter my name.

What about your name?

Tweet this

Hg Commit Monitor – Get notified of new pushs to Mercurial repositories

Tobias Hertkorn — Wed, 03 Jun 2009 20:05:45 +0000

I didn't find any commit monitor for Mercurial repositories. I am currently working on a project with multiple people pushing to one repository - and of course I want to know as soon as possible when one of the others did commit new source code. That way we can resolve merge conflicts soon and get a nice code review loop going.

Hg Commit Monitor uses the atom feeds of repositories to detect new commits. It was quite important for me that it does get out of my way and sight - unless I need to be notified. So it will just sit in the tray and only communicate via balloons.

If you are interested in such a tool, you can download it from here. I did publish both a compiled version and a snapshot of the mercurial repository.

Currently Hg Commit Monitor needs .NET 3.5. But if you want to run it on mono, please contact me and I will replace the reference to System.ServiceModel.Syndication with a third party Atom parser.

Screenshot Main Window V. 1.0.1

Tweet this

Ignoring https related AuthenticationException when using a WebClient

Tobias Hertkorn — Sun, 31 May 2009 14:43:43 +0000

I am currently working on a little pet project that scrapes and parses html pages for information. Nothing fancy, just a little tool that sits in my taskbar and informs me on changes on that URL.

In order to do that I issue an http request using the System.Net.WebClient class.

PLAIN TEXT

C#:

public string GetPage(Uri uri, string username, string password) {
WebClient wc = new WebClient();
wc.Credentials = new NetworkCredential(username, password);
return wc.DownloadString(uri);
}

The page I want to crawl is secured by basic authentication. That's why I have to set the appropriate Credentials.

Recently I have switched from hosting the page under http to hosting it under https. As this is my private server I did not bother to request a signed ssl certificate. I just used a self signed one. Unfortunatelly this rendered my little tool unusable, since the WebClient started to throw:
failed: System.Net.WebException : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ----> System.Security.Authentication.AuthenticationException : The remote certificate is invalid according to the validation procedure.

That's of course due to my self signed certificate not being trusted. And it's a good thing that the .NET Framework does not ignore these kinds of errors. After all there could be a man in the middle attack going on. Fortunatelly the framework also allows us to specify that this check should be overridden. All we need to do is specify a custom certificate verification method by attaching a custom ServerCertificateValidationCallbackHandler to the ServicePointManager.

PLAIN TEXT

C#:

public string GetPage(Uri uri, string username, string password) {
WebClient wc = new WebClient();
wc.Credentials = new NetworkCredential(username, password);
return wc.DownloadString(uri);
}
public static void SetBypassSslCertificateValidation() {
ServicePointManager.ServerCertificateValidationCallback
+= new RemoteCertificateValidationCallback(BypassSslCertificateValidation);
}
private static bool BypassSslCertificateValidation(object sender, X509Certificate cert, X509Chain chain, SslPolicyErrors error) {
return true;
}

Well, just don't do this stunt, when there actually is a security risk.

Tweet this

x86-only software is … unfortunate

Tobias Hertkorn — Sun, 17 May 2009 09:55:23 +0000

I guess one can get away with regular software only being x86 compatible. It is not too big a problem that e.g. firefox runs as 32 bit. Seriously, if any given website needs more than 2GB to render its content - there's something wrong with the website anyway. Unfortunately it's a totally different story for software that touches the hardware or explorer shell extensions.

One of the examples for a deeply embedded software would be the Cisco VPN Client. Right now I have to use a virtual winxp 32bit in order to connect to my office's VPN. WTF?

Unfortunately I ran across another program: The shell extension tortoiseHg only comes in 32bit. Hmm, what? It's a software that specifically targets programmers and development started in 2007. How could that possibly come in 32bit only?

It's 2009 for crying out loud. Not only is 4GB common amongst developer, 4GB is fast becoming "annoyingly little RAM".

Tweet this

Dabbling in Mercurial

Tobias Hertkorn — Sun, 17 May 2009 01:58:48 +0000

What can I tell you - I have been using git for a bit a while ago, and I really liked it ... in a pure linux environment. It was a bit awkward when using windows. So I went back to subversion when doing my .NET development.

But I really did miss the offline functionality that is one of the biggest advantages (in my eyes) of DSCMs. Listening to a stackoverflow podcast lately I heard Joel rave on about Mercurial. So I thought I have to check it out myself - and I have to say, I am more than a bit impressed. I was able to set up a windows machine and a linux machine with hg within minutes. Creating a hg push repository using apache was a breeze as well. All with authentication.

When it comes to using hg: Even using the command line interface is a not too annoying. I went through update, pull, push, merge, ... I even simulated merge conflicts. Easy! And using the graphlog extension it is quite simple to view the logs and all the paths that lead to the current version. All in all, it seriously was not a bit of trouble. And I haven't even installed tortoiseHg yet... But that's for tomorrow.

Conclusion: I guess I do have to switch from svn to hg. There simply is no need to put it off anymore. Should I dare to ask my boss, if we could switch at work as well?!

Tweet this

Does Microsoft claim exclusiveness – what would YOU need TrueCrypt for anyways?

Tobias Hertkorn — Wed, 06 May 2009 07:55:21 +0000

I heard only the best things about the Windows 7 beta - so, with the RC being released I did set aside a bit of time to do my backups (which is breeze when using TrueImage), get the necessary drivers for my system, ...

Now a little more infos about my system: I have one main drive for the operating system and two additional drives that are 100% TrueCrypt volumes, one for my personal data and one for my backups.

Well, alright, we are already kinda used to Windows overwriting the MBR on a disk that was booting to linux before, we all know that - but did you know that the installer now initializes ALL drives in the system? Imagine my surprise when both my TrueCrypt partitions were SHOT! Apparently the Windows Installer found it appropriate to write a boot sector and a boot manager on all disks in my system. And a encrypted disk without its headers is ... just a blob of random data.

~~To be fair, of course I used an unsupported software on a beta operating system. Who's to blame but me?~~

BUT ANYWAY: W.T.F.!

Conclusion: USB Disk FTW. Remove ALL TrueCrypt Disks. Not Developers, Developers, Developers - but Backups, Backups, Backups.

Tweet this

Mapping the uncharted waters of LINQ

Tobias Hertkorn — Wed, 11 Mar 2009 01:48:28 +0000

Preamble

Today I had a really interesting discussion with Jon Skeet on stackoverflow.

It got started by a question by me: When does a library deserve the be called “Linq-something” or “something-Linq”?

In it I kinda rant about libraries that have "Linq" in their name even though they do not provide extension methods for IQueryable and IEnumerable but instead just for IEnumberable. And I asked the community if it is OK for libraries that just extend IEnumberable to have Linq in their name.

Jon Skeet's (who is kind of a stackoverflow celebrity) answer made me wonder though. What are the classifications for Linq related libraries? Especially since he seemed to mix and match Linq-to-... libraries with his MoreLinq library. In the comments (Click "Comments" to see them) I tried to clarify why I thought there was a difference but unfortunately I was not clear enough to get my point across in 300 characters. That's why I chose to write this blog post. Hope it will clarify my view on what parts LINQ is made of.

Classifying the moving parts of LINQ

First I want to clarify what I think that comparing a Linq-to-... library with MoreLinq is kinda flawed. Linq-to-... are what I (and others) call a Linq Provider library and something like MoreLinq is a Query Operators library.

Linq Provider libraries

To quote from the .NET Framework Glossary:

LINQ provider
A library that implements the functionality of the standard query operators for a specific kind of data source. A LINQ provider translates queries expressed in Visual Basic or C# into the native language of the data source. For example, LINQ to SQL is a LINQ provider for SQL Server databases that translates queries into dynamic SQL and marshals the results into objects defined on the client computer.

LINQ Provider are the base of LINQ. They connect the logical description of a data transformation to the actual data storage (which does not have to be a relational database but may be). Loosely speaking they can transform a "persisted" form of data or an in-memory representation of data into an alternative in-memory representation of data. We can give those providers abstract transformation rules and the provider will convert them into concrete transformation rules specific for the kind of data store it "owns". Abstract transformation they individually do not understand will get deferred until a rudimentary memory representation is formed and will be applied in memory to form the final in-memory representation of data.

A typical Linq Provider library will contain:

An implementation of IQueryProvider
At least one implementation of IQueryable
Usually one or multiple ExpressionVisitor
Usually one or multiple ExpressionConverter

Examples of Linq Provider libraries and their concrete IQueryProvider implementation:

Linq to Sql (System.Data.Linq.Table)
Linq to Nhibernate (NHibernate.Linq.NHibernateQueryProvider)
Linq to Objects (System.Linq.EnumerableQuery)

Linq Operator libraries

Linq Operator libraries provide a rich set of abstract operations that can be used to construct the abstract transformation rules we can pass to any LINQ Provider. Most notably are the Standard Linq Operations actually baked into the C# 3.5 compiler as syntactical sugar.

Some examples for these operations:

Where
GroupBy
Sum
Max
...

To be able to be used across multiple Linq Provider and let the provider benefit from deep inspection of the abstract transformations the Linq Operators must extend IQueryable. Unfortunately the parsing and compiling of these abstract transformation expressions is CPU intensive and can severely impact the performance of in-memory transformations.
That's where LINQ to Objects comes (unfortunately) into play a second time. In this context Linq to Objects means using abstract Linq queries directly on IEnumerable without the intermediate use of a Linq Provider.

Because of these two forms of abstract transformations, one with Linq Provider, one without, good Linq Operation libraries will provide both an implementation of the operation for IQueryable and for IEnumerable. Remember though - IEnumerable could still be queried using the Linq to Objects Provider EnumerableQuery. Fortunately writing Linq Operations for IEnumerable is by far easier than implementing the same operation for IQueryable.

The Linq Operator library FilterByExample

To clarify what the difference is we will implement the Linq Operation "FilterByExample" both for IEnumerable and IQueryable. FilterByExample will enable us to filter a given data set via a provided instance of the class within the data set. FilterByExample will allow us to specify certain properties of the given class. This is useful for quickly creating advanced filtering forms in GUI development.

Download source for Com.Hertkorn.Framework.FilterByExample

Most of the magic happens in the method FilterByExample in Queryable (and Enumerable almost in the same way). Please notice that in this private version of FilterByExample relevant properties are listed. The public version gets properties that should be excluded (look at the unit tests for examples).

PLAIN TEXT

C#:

private static IQueryable FilterByExample(this IQueryable source, T example, params PropertyInfo[] relevantPropertyz)
{
// since this method is private no additional precondition check.
if (relevantPropertyz.Length == 0)
{
// no filtering
return source;
}
var exampleExpression = Expression.Constant(example, typeof(T));
var otherParameterExpression = Expression.Parameter(typeof(T), "other");
var filterExpression = Expression.Equal(
Expression.MakeMemberAccess(otherParameterExpression, relevantPropertyz[0]),
Expression.MakeMemberAccess(exampleExpression, relevantPropertyz[0])
);
for (int i = 1; i Length; i++)
{
var next = Expression.Equal(
Expression.MakeMemberAccess(otherParameterExpression, relevantPropertyz[i]),
Expression.MakeMemberAccess(exampleExpression, relevantPropertyz[i])
);
filterExpression = Expression.AndAlso(filterExpression, next);
}
Expression> filter = Expression.Lambda>(filterExpression, otherParameterExpression);
return source.Where(filter);
}

The summary explanation of this code snippet is: In line 15 through 28 all relevant properties are tested to be of equal value between two instances of an object of type T. These tests are linked via a logical AND in line 27 so that the resulting expression returns true if the value all relevant properties match in the two instances. This expression is passed to a where clause in line 32. So the returned IQueryable is a representation of the data where only object will be contained in the result where the relevant properties have matching values.

Comparing FilterByExample's extension for IQueryable and IEnumerable

The question is now. Why go through the stress of implementing FilterByExample twice. We can't really see any advantages in the library and its tests. On the contrary. The performance tests show that the IQueryable implementation is about three times slower than the IEnumerable implementation. Part of the reason why we don't see any advantage of the IQueryable implementation is that EnumerableQuery does not use an expression optimizer. So for any effect to appear we must switch to a different Linq Provider.

Download LinqToSql Example - If you want to run it make sure to adjust the connection string in the app.config

Using FilterByExample operator with Linq-to-Sql

When using Linq to Sql we can actually see the SQL Query that gets sent to the DB Server based on the abstract transformation we specify using Linq. That way we can verify that the abstract transformation the operator creates is in fact what we were looking for.

A simple query using standard linq operators would look like this:

PLAIN TEXT

C#:

IQueryable filter = from t in c.TestClasses
where t.TestString == "Test"
select t;
PerformQuery("standard version", c, filter);

and would result in this output:
Performing standard version: '1' 'Test' 'Test' '4' 'Test' 'Me' '5' 'Test' 'TestMe' SELECT [t0].[TestInt], [t0].[TestString], [t0].[TestString2] FROM [TestClass] AS [t0] WHERE [t0].[TestString] = @p0 -- @p0: Input NVarChar (Size = 4; Prec = 0; Scale = 0) [Test] -- Context: SqlProvider(Sql2005) Model: AttributedMetaModel Build: 3.5.30729.1

As expected we see a where clause in the select method which shows that the Linq Provider understood our abstract transformation and was able to translate it into SQL. That way not the whole data table "TestClass" gets pulled across the wire. Just the part we need.

Now we use the FilterByExample operator:

PLAIN TEXT

C#:

TestClass tc = new TestClass();
tc.TestString = "Test";
IQueryable filter2 = c.TestClasses.FilterByExample(tc, x => x.TestInt, x => x.TestString2);
PerformQuery("one relevant prop", c, filter2);

This means "filter the TestClass table by tc and ignore the values in TestInt and TestString2.
And it results in:

Performing one relevant prop: '1' 'Test' 'Test' '4' 'Test' 'Me' '5' 'Test' 'TestMe' SELECT [t0].[TestInt], [t0].[TestString], [t0].[TestString2] FROM [TestClass] AS [t0] WHERE [t0].[TestString] = @p0 -- @p0: Input NVarChar (Size = 4; Prec = 0; Scale = 0) [Test] -- Context: SqlProvider(Sql2005) Model: AttributedMetaModel Build: 3.5.30729.1

Again lo and behold the actual SQL that gets produced looks exactly as if we would have used the standard where query.

Finally a more complex query - filter the list by two relevant properties:

PLAIN TEXT

C#:

tc.TestString = "Test";
tc.TestInt = 1;
IQueryable filter3 = c.TestClasses.FilterByExample(tc, x => x.TestString2);
PerformQuery("two relevant props", c, filter3);

which results in:

Performing two relevant props: '1' 'Test' 'Test' SELECT [t0].[TestInt], [t0].[TestString], [t0].[TestString2] FROM [TestClass] AS [t0] WHERE ([t0].[TestInt] = @p0) AND ([t0].[TestString] = @p1) -- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [1] -- @p1: Input NVarChar (Size = 4; Prec = 0; Scale = 0) [Test] -- Context: SqlProvider(Sql2005) Model: AttributedMetaModel Build: 3.5.30729.1

As expected a second parameter further specifies the where clause plus both are linked by an AND. Now just one object got sent across the wire.

Seems like our custom Linq Operator gets interpreted correctly by the SQL Linq Provider and the filter and transformation happens right where it belongs. Within the SQL Server itself. And even though I did not test it, I strongly believe that it would be the same with e.g. the Linq to NHibernate provider.

In contrast the IEnumerable version

Now to show the difference between the IQueryable and the IEnumerable operator we have to force the Linq to Sql to use the IEnumerable operator:

PLAIN TEXT

C#:

tc.TestString = "Test";
tc.TestInt = 1;
IEnumerable filter4 = FBE.Enumerable.FilterByExample(c.TestClasses, tc, x => x.TestString2);
PerformQuery("IEnumerable", c, filter4);

Notice how we don't get an IQueryable but an IEnumerable as result of the operation. Now the SQL reads:

Performing IEnumerable: '1' 'Test' 'Test' SELECT [t0].[TestInt], [t0].[TestString], [t0].[TestString2] FROM [TestClass] AS [t0] -- Context: SqlProvider(Sql2005) Model: AttributedMetaModel Build: 3.5.30729.1

It is not surprising that the Linq Provider was not able to analyse the abstract transformation we specified. There simply is no expression that could be analysed. So there is no where clause specified in the SQL query. Which results in the Linq Provider sending the whole table across the wire and the filtering being done in-memory. This is of course no biggie in our test scenario. But imagine the table having millions of rows. That certainly would be a big performance problem. Especially if we throw in some joins, etc.

Conclusion

There is a difference between MoreLinq and Linq-to-...

I hopefully did clarify that part.

Real Linq Operators libraries extend IQueryable and IEnumerable

If there is a Linq Operator that is worth putting into a library there have to be two versions of the operator. One for IQueryable to enable smart LINQ Provider to do the data transformation where it fits best. And one for IEnumerable to have the performance benefits when dealing with data that is already in-memory.

No library is going to be able to extend what every provider can do...

... because they would have to know all about all providers.

This is simply wrong. Providers are separate parts in Linq. And Operators certainly do not have to know anything about the provider that will use them. On the contrary, the abstract representation using Expressions is universal. There are just providers that can interpret and optimize those expressions better than other providers. By using Operations that manipulate IQueryable to impose the desired transformation the Linq Provider is given the chance to incorporate the abstract transformation when compiling the concrete transformation. When just using IEnumerable this chance is denied from the very beginning.

Downloads:
Source for Com.Hertkorn.Framework.FilterByExample - the Operator example
LinqToSql Example - using the Operator on Linq to SQL - If you want to run it make sure to adjust the connection string in the app.config

Tweet this

When a great experience suddenly turns into … horror

Tobias Hertkorn — Tue, 24 Feb 2009 06:38:37 +0000

I was upgrading my server which is hosted by large provider from Debian etch to the current stable version lenny. And I was again amazed how simple upgrading a whole box can be with Debian. It seriously was a blast. A couple of questions about touched configurations, a bit of waiting and all was happily done.

At least I thought it was.

As usual I started doing the upgrade - even though I should have known better - at about 11 p.m. which I though would give ample time to finish up and go to bed. Well, it just was a good time to upgrade, traffic on the site was low, all services where not in use, so I was happy to not disturb too many people. And the best part was: I seriously was done at about half past twelve, spending most of the time doing some readjusting of the exim configuration after the upgrade. I seriously doubt that any other non-debian based system would have given me such a fast a smooth upgrade path. After verifying everything worked, I was eager to test - as a last thing before hopping into bed - if the server was still coming up without a glitch after a reboot/power-loss.

After giving ample time for the server to come up again, I tried to remote ssh to it. And got nothing but timeouts. Which was the point where I slightly started to panic. I immediately started thinking about kernel failures, data loss, mbr corruption, ... Oh the horror.

Lucky me, my hoster does provide terminal access even from remote, so I could check if it was disaster or if it was "just" connectivity - missing module in the new kernel or something similar. To make a long story short - somehow the dhcp client got deinstalled during the upgrade! D'oh! And of course I did not have the installation package locally. Chicken and egg problem, right? How should I fix the network connection without getting the dhcpclient package via the network connection. But trying the usual gateways like .1 or .254 via static routes did not seem to work out so well...

Fortunately I did find the correct settings for 1&1 (which are a bit exotic) eventually - so for anybody having the same problem here they are:

iface eth0 inet static address 212.227.xxx.xxx netmask 255.255.255.255 network 212.227.xxx.xxx broadcast 212.227.xxx.xxx up /sbin/route add -net 10.255.255.1 netmask 255.255.255.255 dev eth0 up /sbin/route add default gw 10.255.255.1

Substitute all the 212.227.xxx.xxx with your server address.

After setting the static interface information, reinstalling the dhcp client was not a bit of a problem. But how in hell could I have missed that the upgrade uninstalled such a vital package?
In the end I thought "All's well that ends well" and went to bed around 3 in the morning. Though one question remains: Will I be wiser in choosing the time to start an upgrade next time? Why do I seriously doubt that?

Tweet this

Tweeting

Tobias Hertkorn — Sat, 14 Feb 2009 19:24:54 +0000

Boring update: I finally got to twitter. Almost a week ago. I did not post this earlier here on the blog, because I wanted to see if it sticks. And I have to say it is a nice way of passing the time while code is compiling. But I hope to keep it purely "professional" in the sense that I don't want to start talking about anything else than programming problems or at least programming related topics.

So, if you want to join me - and maybe yell at me for not posting anything of value to my blog in way too long - go to http://twitter.com/dun3 and look me up. Looking forward to your tweets.

Tweet this