Jb in a nutshell

IL fun fact: not is not !

Jb Evain — Mon, 30 Jan 2012 16:00:00 +0100

Picture yourself working on crafting a specific piece of CIL.

You need to write the compiled equivalent of:

bool b = ...;
bool n = !b;

It would be tempting to write:

ldloc b
not
stloc n

Except that it would not always work. not doesn’t negate booleans, it computes the bitwise complement of the value on the stack. It’s also not to be confused with the neg opcode, which negates a value, as in a multiplication by -1.

The usual pattern to negate a boolean is:

ldloc b
ldc.i4.0
ceq
stloc n

But that’s not really fun, is it?

Actual fun fact: there’s a misused “not” in the ECMA 335 in the example of the section 14.5 of the partition II.

Fun fact: C# methods whose bodies span over multiple source files

Jb Evain — Thu, 19 Jan 2012 22:00:00 +0100

While working on Mono.Cecil (your lovely library to analyze and manipulate .net binaries that is used by legions), one thing that struck me as odd for a while, was the fact that a method could have debug symbols for instructions that are defined in multiple files.

Cecil has this type, Instruction. When you’re analyzing a module with debug information (think .pdb or .mdb files), an Instruction may have its SequencePoint property set. A sequence point is nothing but the location of the code in a file that relates to the instruction.

What got me wondering, is that the APIs to retrieve those sequence points make it so that a method can have sequence points in different documents. So be it, this is how I ended up representing it in Cecil, but it’s only recently that I stumbled upon a case where indeed, a C# method had instructions defined in multiple files:

Foo.Bar.cs:

public partial class Foo {
    private List<Bar> _bars = new List<Bar>();
}

Foo.Baz.cs:

public partial class Foo {
    private Baz _baz;

    public Foo (Baz baz)
    {
        _baz = baz;
    }
}

Do you see what’s going on here?

The constructor of Foo is defined in Foo.Baz.cs, but there’s a field initializer in Foo.Bar.cs that is going to be compiled inside Foo’s constructor. When you debug the constructor, you’ll effectively end up jumping between the two files. Crazy right?

Can you think of another case where a C# (or VB.NET for that matters) method would have instructions defined in different files?

Constraining generic constraints

Jb Evain — Fri, 13 Jan 2012 14:30:00 +0100

This is the last part of an update about Mono.Linq.Expressions, a tiny helper library to complement the System.Linq.Expressions namespace for .net 4 and Mono.

And it's not really about Mono.Linq.Expressions itself. If you're interested about it though, you can read the part about fluently creating expression trees, and the other one about combining lambda expressions together.

Proper generic constraints

Have a look at the signature of the factory method Expression.Lambda:

public static Expression<TDelegate> Lambda<TDelegate> (
    Expression body,
    params ParameterExpression [] parameters) {}

And at the signature of Expression<T>:

public sealed class Expression<T> : LambdaExpression {}

Given the post title, and the <h3> a few lines up there, it's obvious where I'm going: those generic parameters have no constraint. A C# compiler will happily compile:

class Potato {}

Expression<Potato> e = Expression.Lambda<Potato> (
    Expression.Constant (new Potato ()));

Of course this won't get you far, but it sure is a heart breaker that because of a little language oddity, you delay to until runtime the problem resolution: this code will throw, Potato sure isn't a Delegate type, and Expression.Lambda, the only way to create an Expression<T> is making sure of that.

So much for type safety, generics!

Another famous issue is the following. Again, for the compiler, it's perfectly legit to write:

int i;
if (!Enum.TryParse ("42", out i))
    Console.WriteLine ("This is so unfair!");

And why is that? Because of the signature of Enum.TryParse:

public static bool TryParse<TEnum> (string value, out TEnum result) where TEnum : struct {}

To sum the issue up, C# doesn't allow generic constraints on delegates and enums. And why am I rumbling about this? Because Mono.Linq.Expressions has this method:

public static Expression<T> Combine<T> (
    this Expression<T> self,
    Func<Expression, Expression> combinator)

And if Expression<T> is not constrained on Delegate, then I sure as hell won't let this stand in my own public API! Which is why the actual signature of the Combine method is:

public static Expression<T> Combine<[DelegateConstraint] T> (
    this Expression<T> self,
    Func<Expression, Expression> combinator) where T : class

And all of this was just a perfect excuse to write a little Mono.Cecil based utility tool, appropriately named patch-constraints, that is used as a post-compile step to fixup the Delegate and Enum constraints of any generic parameter decorated respectively with a DelegateConstraintAttribute or EnumConstraintAttribute, turning them into actual constraints to delegates and enums, which, funnily enough, the C# compiler is more than happy to honor.

Obviously I could have used the most famous Jon Skeet's unconstrained-melody tool, but 2002 called and it wants its ildasm based, IL text parser back.

Mono.Linq.Expressions update 2

Jb Evain — Thu, 12 Jan 2012 12:25:00 +0100

This is the second part of an update about Mono.Linq.Expressions, a tiny helper library to complement the System.Linq.Expressions namespace for .net 4 and Mono.

The first part is about fluent creation of expression trees.

Combining expression trees together

I keep reading questions on StackOverflow about this. How to combine two lambda expression together? If we have:

Expression<Func<User, bool>> isUserOver18 = u => u.Age >= 18;
Expression<Func<User, bool>> isFemaleUser = u => u.Gender == Gender.Female;

If we want to combine this lambda expression with a “and” logical expression, the natural way would be to write:

Expression<Func<User, bool>> isFemaleUserOver18 = u =>
    isUserOver18(u) && isFemaleUser(u);

This works just fine if you compile the expression into a delegate to actually execute this code. But most of the time questions of StackOverflow are about using the resulting lambda expression to create a query for LINQ to a database provider, which will analyze the expression tree and create an according SQL request.

By combining expression trees this way, the LINQ provider may or may not be unable to turn the two invocations into actual SQL.

That's one of the reason I wrote about an updated PredicateBuilder. The obvious solution is to inline the two combined representation of lambda expressions into a new lambda expression tree.

The update of Mono.Linq.Expressions comes with a new type, CombineExtensions, which exposes extension methods that you can use to combine fully created (into lambda expressions) expression trees.

Using those, combining the two expression trees is as simple as:

Expression<Func<User, bool>> isFemaleUserOver18 = isUserOver18.Combine(
    isFemaleUser,
    (left, right) => left.AndAlso(right));

And indeed, if you print the code representation of this expression tree, you'll have both lambda bodies inlined into another one:

user => user.Age >= 18 && user.Gender == Gender.Female

Or if you want to negate the boolean expression:

Expression<Func<User, bool>> isNotFemaleUserOver18 = isFemaleUserOver18.Combine(
    e => e.Not());

The cool thing about those Combine extension methods is that they're completely generic, they don't work only on simple predicates. For instance, you can use those to chain constructions of mathematical expressions.

Mono.Linq.Expressions update

Jb Evain — Wed, 11 Jan 2012 17:00:00 +0100

I just tagged the 1.2 version of Mono.Linq.Expressions, and pushed an updated nuget package.

Mono.Linq.Expressions is a utility library to complement the System.Linq.Expressions namespace, and works with .net 4.0 just as fine as it does with Mono. With a bit over 220 downloads of the nuget package, it's short of roughly 160,400 downloads to be the most downloaded nuget package : a stunning success to put it simply.

This post is the first of a short series to detail what's awesome and new in this version.

Extension methods for a fluent construction of expression trees.

Have you been using the expression tree API to build a representation of code at runtime ? If so you're familiar with the Expression class, and it's load of factory methods.

You're also familiar with this kind of code:

var user = Expression.Parameter(typeof (User), "user");

var isFemaleUserOver18 = Expression.Lambda<Func<User, bool>>(
    Expression.AndAlso(
        Expression.GreaterThanOrEqual(
            Expression.Property(user, "Age"),
            Expression.Constant(18)),
        Expression.Equal(
            Expression.Property(user, "Gender"),
            Expression.Constant(Gender.Female))), user);

If you take some time to parse this code, the intent is to create an expression tree similar to the one the compiler would emit if you were to write:

Expression<Func<User, bool>> isFemaleUserOver18 =
    user => user.Age >= 18 && user.Gender == Gender.Female;

Mono.Linq.Expressions 1.2 contains a code generated series of extension methods to simplify the manual construction of expression trees by fluently chaining the invocations. This allows you to write instead:

var user = typeof (User).Parameter("user");

var isFemaleUserOver18 = Expression.Lambda<Func<User, bool>>(
    user.Property("Age").GreaterThanOrEqual(18.Constant())
    .AndAlso(
        user.Property("Gender").Equal(Gender.Female.Constant())), user);

Not only is the code shorter, but it's also easier on the eyes, and easier to comprehend. Using this, almost the factory methods calls to the Expression class can be written fluently.

TechDays 2011

Jb Evain — Mon, 07 Feb 2011 23:01:00 +0100

Mercredi, j’aurai l’opportunité de présenter Mono et son écosystème pendant les TechDays Microsoft.

Ce sera l’occasion de faire le point sur les dernières nouveautés de Mono, et de montrer comment réutiliser ses compétences .net et partager son code C# pour cibler des plateformes en vogue, comme l’iPhone et l’iPad avec MonoTouch, Android avec MonoDroid et Mac OS X avec MonoMac.

Si vous ne pouvez pas assister à la session, je serai aussi disponible sur le stand alt.net pour discuter de tous ces sujets. Venez nombreux !

Custom Linq Expressions

Jb Evain — Mon, 17 Jan 2011 18:45:00 +0100

Between .net 3.5 and .net 4.0, the Linq Expression Tree API changed quite a bit, going from a simple compiler supporting expressions only, to a full fledged compiler supporting expressions as well as statements, on top of which are implemented all DLR based language. For anyone working closely with this API, the Expression Tree spec is a great source of information, the MSDN isn’t very helpful there.

When the spec comes to describing the object model for nodes, it differentiates several kind of nodes:

The core nodes and the common nodes are those which are implemented inside System.Linq.Expressions, and common nodes are implemented on top of the core nodes, using a process that is called reducing nodes.

A reducible node is simply a node that can be compiled as a compound of core nodes. We’re not going to talk about library specific extensions, which are nodes that you would write if you were to implement, say, a LINQ provider. We’ll focus on reducible nodes.

I took some time last year to implement a few helper nodes that are not available in .net 4.0, but, according to the spec, may be included in a future release. They’re mapped to similar C# constructs, and are pretty useful it you generate complex code using expression trees. Here’s the list of custom expressions I added in Mono.Linq.Expressions, my project to complement the System.Linq.Expression namespace:

Even if those map to a C# statement, they’re suffixed with `Expression` for consistency with the whole System.Linq.Expressions namespace. Using them is as simple as, heh, adding a using directive:

using Mono.Linq.Expressions;

And using the factory methods on the CustomExpression class to create those new nodes:

var d = Expression.Parameter (typeof (DisposableType), "d");

var printAndDispose = Expression.Lambda<Action<DisposableType>> (
    CustomExpression.Using (
        d,
        Expression.Call (typeof (Console).GetMethod (
            "WriteLine", new [] { typeof (object) }), d)),
    d);

The Expression Tree API makes it easy to write custom reducible expressions, all you have to do, is to inherit from Expression, override CanReduce and NodeType, and implement the Reduce method:

public abstract partial class CustomExpression : Expression {

    public override ExpressionType NodeType {
        get { return ExpressionType.Extension; }
    }

    public override bool CanReduce {
        get { return true; }
    }

    /* ... */
}

If we take, for instance, our UsingExpression, mapping to the C# using statement, Reduce is implemented as follows:

public override Expression Reduce ()
{
    var end_finally = Expression.Label ("end_finally");

    return Expression.Block (
        new [] { variable },
        Expression.Assign (variable, disposable),
        Expression.TryFinally (
            body,
            Expression.Block (
                Expression.Condition (
                    Expression.NotEqual (variable, Expression.Constant (null)),
                    Expression.Block (
                        Expression.Call (
                            Expression.Convert (variable, typeof (IDisposable)),
                            typeof (IDisposable).GetMethod ("Dispose")),
                        Expression.Goto (end_finally)),
                    Expression.Goto (end_finally)),
                Expression.Label (end_finally))));
}

Just like a C# using, it will try to execute a block, and whether the block triggered an exception or not, it will call IDisposable.Dispose on the subject of the using statement.

Wasn’t that easy? Even if those nodes are pretty general, writing custom and specific nodes is a powerful and simple way to factorize code when generating code using the Expression Tree API.

In the end, I added in Mono.Linq.Expressions a CustomExpression type, our base class for well, custom expressions. We provide a CustomExpressionVisitor which extends the standard ExpressionVisitor to support our custom expressions, and the CSharpWriter has been updated to support them as well:

[Test]
public void Using ()
{
    var arg = Expression.Parameter (typeof (IDisposable), "arg");

    var lambda = Expression.Lambda<Action<IDisposable>> (
        CustomExpression.Using (
            arg,
            Expression.Call (typeof (Console).GetMethod (
                "WriteLine", new [] { typeof (object) }), arg)),
        arg);

    AssertExpression (@" 
void (IDisposable arg)
{
    using (arg)
    {
        Console.WriteLine(arg);
    }
}
", lambda);
}

Next feature for Mono.Linq.Expressions: using Mono.Reflection to turn a delegate into an expression of this delegate.

IQueryable support for WP7

Jb Evain — Wed, 13 Oct 2010 17:35:00 +0200

Or how to use APIs missing from the WP7 SDK.

Monday saw the official launch of Windows Phone 7, at this occasion, I downloaded the SDK to see how it compares to MonoTouch and MonoDroid. Basically, it’s based on the Compact Framework Base Class Library, on top of which is exposed the Silverlight UI API.

The Compact Framework has historically targeted devices with limited capabilities. The Compact Framework BCL is a trimmed down version of the vanilla .net BCL. One notable missing feature is runtime code generation through System.Reflection.Emit. As such, you can’t use DynamicMethods, nor can you compile assemblies on the device, and nor can you compile Expression Trees. You can expect to find missing tidbits sprinkled throughout the whole BCL. But luckily for us, there’s Mono.

The Mono BCL being licensed under the MIT/X11, you can easily take C# code from our libraries and use it inside your own applications. And that whenever our code is not tied to a runtime feature. This means that you couldn’t bring System.Reflection.Emit to WP7, but that you could probably bring XmlDocument with a bit of work.

Basically, this reminded me of what db4o did to bring support for Expression Trees to the Compact Framework using code from Mono. I figured I’d try to do the same for WP7. If you played with the WP7 SDK, you probably know that you have access to Expression Trees, and the compiler can emit them without any issue. But LambdaExpression and Expression of T don’t have their Compile method you’re used to. And WP7 is missing IQueryable and folks.

So as an example, I extracted our Expression Tree interpreter, which is used, guess what, when you can’t compile Expression Trees at runtime, and our IQueryable support. Here we are, by just referencing this System.Linq.dll, you can write code such as:

using System.Linq;

var data = new [] { 1, 2, 3, 4, 5, 6, 7, 8 };

var query = data.AsQueryable ();
query.Where (i => i % 2 == 0);

int count = query.Count ();

Or:

using System.Linq.Expressions;

var p1 = Expression.Parameter (typeof (string), "x");
var p2 = Expression.Parameter (typeof (string), "y");

Expression<Func<string, Func<string, Func<string>>>> e =
    Expression.Lambda<Func<string, Func<string, Func<string>>>> (
        Expression.Lambda<Func<string, Func<string>>> (
            Expression.Lambda<Func<string>> (
                Expression.Call (
                    typeof (string).GetMethod ("Concat", new [] { typeof (string), typeof (string) }),
                    new [] { p1, p2 }),
                new ParameterExpression [0]),
            new [] { p2 }),
        new [] { p1 });

var f = e.Compile ();
var f2 = f ("Hello ");
var f3 = f2 ("World !");

var result = f3 ();

(Crazy, I know)

You can simply take the assembly and reference it from your projects, or compile yourself the code. Let me know if this has been useful to you.

To conclude, I’d say don’t hesitate to take code from Mono to fill the voids in the WP7 SDK. Who knows if you won’t end up contributing missing parts or filing bugs afterwards.

IsAssignable what?

Jb Evain — Fri, 01 Oct 2010 10:30:00 +0200

I’ve tweeted it before, Jackson tumblered it in return, but after commenting on a StackOverflow answer where the poster got it wrong, commenting:

I always seem to turn that call around.

I felt that I needed to broaden this grand hack. If you’re like me, and apparently like a lot of coders out there, you tend to always get Type.IsAssignableFrom wrong on the first try, then you need this extension method:

public static bool IsAssignableTo (this Type self, Type type)
{
    if (self == null)
        throw new ArgumentNullException ("self");
    if (type == null)
        throw new ArgumentNullException ("type");

    return type.IsAssignableFrom (self);
}

Somehow, even if IsAssignableFrom is more «left to rightish», preserving the side of the arguments a real assignment would have, I understand IsAssignableTo faster. For what it’s worth, I’ve used it extensively while writing our .net 3.5 implementation of System.Linq.Expressions, and it served me well.

Have a great friday.

A river of T

Jb Evain — Mon, 09 Aug 2010 01:20:00 +0200

Generic collections, iterators and Linq all have in common a single interface: IEnumerable<T>. If you’ve been writing C# code lately, I bet you wrote your fair share of methods taking or returning IEnumerables.

Boo (among other marvels) has a cute syntactic shorthand for it. You can write T* instead of IEnumerable[of T]. Of course people would really like to have that in C# as well. Sure T* would be nice, but it’s already used for pointers. Some other suggestions in the stackoverflow question involve using the # suffix, or {}. On my side, I quite like the idea of re-using ~. It’s only used by the bitwise complement operator in C# (and by finalizers, thanks Scott for the heads up), and if you look at it with a bit of poetry, it kind of look like a river, a stream. And what’s an IEnumerable<T> but a stream of T?

Now discussing wishes for the next version of C# is nice and all, but I felt like hacking a bit on mcs, Mono’s C# compiler tonight, and in less than half an hour of grepping through the code, I had a working version of a patch that would enable this syntactic sugar.

Let’s take a very poorly implemented set of Linq operators:

using System;
using System.Collections.Generic;

static class Enumerable {

	public static IEnumerable<T> Concat<T> (this IEnumerable<T> source, IEnumerable<T> other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static IEnumerable<TResult> Select<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static IEnumerable<TResult> SelectMany<TSource, TResult> (this IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static IEnumerable<T> Where<T> (this IEnumerable<T> source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

With my patch, you could write instead:

using System;

static class Enumerable {

	public static T~ Concat<T> (this T~ source, T~ other)
	{
		foreach (var item in source)
			yield return item;

		foreach (var oitem in other)
			yield return oitem;
	}

	public static TResult~ Select<TSource, TResult> (this TSource~ source, Func<TSource, TResult> selector)
	{
		foreach (var item in source)
			yield return selector (item);
	}

	public static TResult~ SelectMany<TSource, TResult> (this TSource~ source, Func<TSource, TResult~> selector)
	{
		foreach (var item in source)
			foreach (var sub_item in selector (item))
				yield return sub_item;
	}

	public static T~ Where<T> (this T~ source, Func<T, bool> predicate)
	{
		foreach (var item in source)
			if (predicate (item))
				yield return item;
	}
}

I let the reader decides for himself which version is the most readable. It's pretty much the same debate as the one for nullable types. Also note that this is just a toy patch, mainly for the sake of digging a bit in mcs, but it’s an interesting experiment nevertheless.