i have a framework...

Introducing Kiddo: A Ruby DSL for building simple WPF and Silverlight applications

Thu, 27 May 2010 20:08:00 GMT

Read the original article here...

As a long time Ruby lover and deep rooted .NET supporter, I was probably more psyched than anyone I knew when IronRuby 1.0 was finally released. I immediately grabbed and started building some apps with it to see where the boundaries were going to lie between IronRuby and ruby.exe, and so far I've been pleasantly surprised by how many things just work as I'd expect. I then started to try out some of my favorite libs that I was sure would not work with IronRuby, and I wasn't surprised at all when _why's amazing Shoes library didn't work. Being somewhat familiar with Shoes (it's a great DSL for building simple UIs in Ruby) I felt it wouldn't be that difficult to port it over and as it turned out, someone else had already started the work. As cool as this was, I was never quite satisfied with good 'ol shoes. While it was quite complete, it lacked simple extensibility points, and although easy, it wasn't quite "kid friendly". At the same time on the .NET side of the fence, IronRuby could easily compile XAML to create WPF and Silverlight UIs, but trying to do it declaratively in plain Ruby was no fun at all. And so, the Shoes-inspired, WPF/Silverlight GUI DSL was born. (and it lives here: http://bitbucket.org/fdumlao/kiddo/src)

Introducing Kiddo

Tell you what. Let's start with a quick code example first. We'll build a useful app that we can use to quickly reverse strings whenever we need it.

Read the complete article here...

Building Functions vs. Building Expressions: Performance Comparison

Tue, 24 Feb 2009 04:47:00 GMT

Yesterday I posted an entry describing a method for building functions as opposed to building expressions to get a more succinct syntax, and in general a more functional approach. There was a touch of reflection in there though, so I wasn’t too sure what the performance would be like.

Even though the post was specifically about an alternative to building expressions, the commenter suggested that it was more efficient, based on his own benchmarks. I was shocked.

Not being the type to believe something without seeing it, I found myself compelled to write my own simple benchmark and the results were shocking.

Building functions is FASTER than building expressions. WAY faster. Let’s take a look:

The Code:

The function building method I created was this (again sorry for the formatting – I’m trying to make it fit):

   1:  public static Func<T, bool> BuildEqFuncFor<T>

   2:      (string prop, object val)

   3:  {

   4:      return t => t.GetType()

   5:          .InvokeMember(prop,

   6:              BindingFlags.GetProperty,

   7:              null,

   8:              t,

   9:              null) == val;

  10:  }

And the commenter suggested I use the approach where you build up an expression, and then compile it:

   1:  public static Func<T, bool> ExBuildEqFuncFor<T>

   2:      (string prop, object val)

   3:  {

   4:      var o = Expression.Parameter(typeof(T), "t");

5:

   6:      Expression<Func<T, bool>> expression =

   7:          Expression.Lambda<Func<T, bool>>(

   8:              Expression.Equal(

   9:                  Expression.PropertyOrField(o, prop),

  10:                  Expression.Constant(val)), o);

11:

  12:      return expression.Compile();

  13:  }

The Results:

I ran each method 1, 100, and 10,000 times and here were the results (in seconds):

It’s unbelievable how much faster the function building method ran, compared to creating an expression tree and compiling it. In fact, within 100 runs there was almost no change at all!

I looked into what was making the expression building function so long to create, and as I suspected it was the “Compile” method call that is used to turn the whole tree into a function. I changed the method to just return the expression and not compile it and re-ran the tests. It still took twice as long just to build the expression as it did to build an actual function.

If you think about it though, it makes sense. Compiling on the fly was never supposed to be considered a performant (yes I said “performant”) method of metaprogramming in C# and if you are going to use it, it would be wise to implement some caching to prevent the need to constantly build the same functions again and again.

Or, you can just build the functions directly.

C# Dynamic Function Factory

Mon, 23 Feb 2009 06:36:00 GMT

Building functions instead of expression trees.

The functional programming features of the latest versions of C# allow developers to harness a very expressive programming paradigm to solve challenges in new, elegant ways. Two of the very interesting new features are the “Func<>” delegate and the Lamda expression.

A Lamda expression is a simple expression that is interpreted by the compiler as either a delegate or an expression tree, depending on the Type called for in the given context. You see them used often in LINQ. For example:

   1:  var bostonPeople = people.Where( p => p.City == "Boston" );

Some time has been devoted in explaining how a LINQ query like the above can be created dynamically by building expression trees (see the excellent article: Creating Dynamic Queries in LINQ by Bill Wagner). While this approach to creating a Lambda expression dynamically is very powerful, some may find that it is far more than they need and instead could get by with a simpler implementation. Others may find it difficult to read or understand and therefore provides a challenge in maintenance.

One possible solution is to use the powerful new Func<> and a lambda expression to easily create a function on the fly. This solution is not quite as powerful as building an expression tree but in situations where it’s possible it will save quite a bit of code.

To demonstrate, how about a completely contrived example. Let’s say you have this object in your domain:

   1:  class Person

   2:  {

   3:      public string Name { get; set; }

   4:      public string City { get; set; }

   5:      public string State { get; set; }

   6:  }

and then let’s assume you need to have a way to return back a subset of a collection of “people” based on some criteria that won’t be known until runtime. We can create a factory that builds a function on the fly that we can pass into a LINQ method. Essentially what the “BuildEqFuncFor<T>” method does is build a Func<T, bool> that compares a given value against a named property of T. The signature of the built function is compatible with most LINQ query parameter expectations. (Please excuse the code formatting and naming conventions – I’m trying to fit it into this post.)

   1:  class FunctionFactory

   2:  {

   3:      public static Func<T, bool> BuildEqFuncFor<T>

   4:          (string prop, object val)

   5:      {

   6:          return t => t.GetType()

   7:              .InvokeMember(prop,

   8:                  BindingFlags.GetProperty,

   9:                  null,

  10:                  t,

  11:                  null) == val;

  12:      }

  13:  }

Now this factory only supports building functions that test for equivalency between some property of some object and a value, however more advanced functions could be constructed using the same concept.

We can now use this FunctionFactory to create a function on the fly and use it in a LINQ method to query our people collection. Let’s see what that looks like:

   1:  var personIsFromBoston =

   2:      FunctionFactory.BuildEqFuncFor<Person>("City", "Boston");

3:

   4:  var bostonPeople = people.Where( personIsFromBoston );

5:

   6:  if (bostonPeople.Any())

   7:      bostonPeople.ToList().ForEach(p => Console.WriteLine(p.Name));

Obviously this is a simple example but it could easily be expanded upon (and optimized) to create a more advanced function factory that could generate functions to solve problems beyond just creating dynamic LINQ queries. An issue you should take into account, is there is no compile time type checking so you should take care to handle cases where the type of the property doesn’t match the type of the value, or that the property even exists on the type provided at all.

As an interesting aside… this could be made even a bit more flexible by adding another function layer:

   1:  Func<string, Func<Person, bool>> personIsFrom =

   2:      s => FunctionFactory.BuildEqFuncFor<Person>("City", s);

3:

   4:  var bostonPeople = people.Where( personIsFrom("Boston") );

I hope you found this interesting, I look forward to your comments.

Silverlight Adaptive Streaming: How it works

Mon, 03 Nov 2008 16:11:12 GMT

Recently an announcement was made by Akamai that it was partnering with Microsoft to provide an Adaptive Streaming solution for Silverlight and IIS 7.0. Since I work in the online video industry I found the announcement very interesting, especially considering Move Network's previous announcement that it had formed a "Strategic Releationship" with Microsoft to provide this exact functionality.

I'm not sure if this represents the fruits of this relationship or not, but I can say that the two technologies are extremely similar in how they provide an almost "instant-on", high-definition stream of video to the client. Which brings me to the "meat" of this article - how the heck to they do that?

Specifically, you can see an example of what I'm talking about at www.smoothHD.com. You'll need Silverlight of course, but once you do you'll notice that the time it takes the video to begin playing is almost negligible and the quality is outstanding... unless you have a crappy connection... in which case you'll notice that it starts up quickly and has so-so to poor quality; which is by design. This is because Adaptive Streaming "adapts" to your bandwidth so that you get the best experience available to you. The thought is that it would be better for a client with slower access to get "something" rather than "buffering".

Step By Step - What makes this thing tick?

Let me start with the caveat that this is a high level overview of the technology - it's not perfect, but should provide you the jist of it. Now that I've said that, let's get started.

It all starts with encoding. In order to provide users with a stream appropriate for their bandwidth, the video needs to be encoded at various bitrates from low to very high quality. The more versions created the better the stream will be able to adapt.

Besides just creating several versions, the video needs to be "cut up" into many pieces. Most likely, this is done by cutting at particular times or frames in the video instead of cutting at particular sizes. So for example, if my video was encoded at 3 different bitrates (by the way 3 is far to few but makes my example easier) I might slice each of them at 5 second intervals ( I would probably make the interval shorter than that but again, this is an example). This way, the first slice of the video encoded using bitrate A would be 2kb, the first slice of the one using bitrate B would be 5kb, and the first slice of bitrate C would be 8kb. They are all different sizes but contain the exact same portion of video as their peers.

This is where Akamai comes in (although you could likely use any modern CDN). You need to ensure that access to these video "pieces" is fast, reliable, etc... So using a CDN like Akamai puts the files closer to the video consumer making delivery problems less likely. Once the video is on CDN, it's all up to the player.

The player bears the brunt of the work of making sure that the video stays smooth regardless of network conditions. The first thing that happens when the player starts up, is it grabs a manifest file containing information about all of those video pieces. It then makes a quick determination of the user's network bandwidth and grabs the first piece of video that is appropriate for the client's connection and begins playing. As that first piece begins playing, it looks closer at the bandwidth of the client and then adjusts the video quality when it grabs the next piece of video. It will continue to monitor the client's performance as it plays, and adjusting which piece of video to grab based on the information it has most recently gathered.

If you look at what it's doing here, you'll see that we aren't really talking about a "stream" but instead the player is playing small video files in a seamless sequence so that there is no interruption to the client. Because the video files are very small they load quickly and don't require waiting for "buffering". The video can begin playing when the first "piece" is downloaded (even before if the codec is one that allows for partial file playback). This also allows a client to "jump" to another spot in the video without having to wait for the player to re-buffer, because it just has to grab the small video file at the point you want to jump to.

One cool thing to note here; if you think about how the Internet works, you'll realize that there are many many more significant benefits to this than just smooth video for clients. As more and more clients watch a particular video, proxy caches all across the Internet will cache these small "pieces" of video files (as they tend to do with smaller files that are requested frequently). In essence these proxies become an extension of your CDN, except that it's free; because the proxies will serve up these small cached files without ever talking to Akamai or whatever CDN you are working with to deliver your video.

Hopefully this has provided some insight for those of you who wondered about this technology - I am excited to see more of this coming soon.

LINQ to Concurrency Problems

Tue, 28 Oct 2008 17:19:00 GMT

LINQ to SQL is fascinating - the more I work with it, the more I hate it, and then love it again all at the same time. Recently we had an issue while using it in a slightly older version of our N-Tier WCF application which uses LINQ to SQL as it's primary ORM solution (yes, I know).

I'll try to briefly familiarize you with the part of the system that became interesting here. In our system we have a domain object called "Asset". Asset has a property called "AssetType" that is an "AssetType" type. Here's a super simplified version:

The interesting thing about our system is that it is strongly tied to a legacy system which keeps all of it's data in XML files organized using a folder structure. The legacy system can't go away and we can't yet migrate the data to our new system's database so we actually pull our asset objects from either the database or the XML files. Once we have the asset built based on the XML data, we need to assign the AssetType, which is only stored in the database. The first solution was to use LINQ to SQL to select an appropriate AssetType and attach it to the asset created from XML. The problem here, is that getting an asset from XML (which happens more often than from the db) took about twice as long as we wanted it to.

Solution ? Cache Baby, Cache!

We started by creating a very basic cache. We decided to just create a static class called "DataCacheHelper" with a single static property that was a static collection of AssetType. By selecting all of the AssetTypes from the database the first time we needed them, and storing them in the static collection, we could quickly grab the one we wanted from any OperationContext when we needed it, without having to go to the database. It was fast and it worked great in when we unit tested it. We deployed it to our test environment, went through the usual workflows and test cases, and everything seemed to check out, so we were cleared to go to production.

That's when it all got... interesting.

Everything was going fantastic. More than fantastic. The system was 3 times faster for some calls, and just more performant all around. We were all sitting here, hitting the system from various vectors, gleefully proving to ourselves how great we all are, never imagining that we had overlooked something critically important in our design. Then:

Person 1: "Uh... I have a failure."

Me: "Which method?"

Person 2: "Testcase 3."

Person 3: "It passes for me."

Me: "Yeah me too... wait... it just failed for me now."

And then they all failed. The whole thing kept on failing and we hurriedly rolled back the production deployment. But what happened?

System.ArgumentException: Could not modify EntitySet.
   at System.Data.Linq.EntitySet`1.CheckModify()
   at System.Data.Linq.EntitySet`1.Add(TEntity entity)
   at Domain.Entities.Asset.set_AssetType(AssetType value) in DomainDataEntities.designer.cs:line 2933

So this was unexpected, but at least I had a place to start. The generated code for setting the AssetType property for Asset was throwing an exception for some reason. I looked at the code and that's when the first problem dawned on me. LINQ to SQL will automatically create the reverse reference for you by default. So my AssetType had a property called "Assets" that was a collection of Asset objects. Specifically, it was an EntitySet<Asset>. I realized instantly that EntitySet is probably not thread safe, and even if it were, it certainly wasn't OperationContext safe (as you can have more than one operation per thread). So the design had failed because LINQ to SQL was doing it's job and making assumptions about how we wanted to work with our data, and we weren't doing our job and customizing what LINQ to SQL had created to fit the specific way we wanted to work with our domain objects. Fortunately this was REALLY easy to fix.

If you look at the properties for the relationship you'll se a property named "Child Property". All you have to do is set that to false and voila! The AssetType object no longer has an "Assets" property. We rebuilt, retested, and are now happy with a system that is both more performant AND it has the added benefit of actually working.

As a side note, while researching this issue, I found absolutely 0 results for a Google search on "EntitySet.CheckModify()". Unfortunately you can't get source code for System.Linq.Data yet to see what that method is checking before it throws, but I used Reflector (search for it) and was able to see exactly how it worked and why it was throwing. I'd love to post more about that but I think MS frowns on posting decompiled source code, so I'll have to come up with a better way.

Troubleshooting ASP.NET MVC Routing

Wed, 10 Sep 2008 02:33:00 GMT

One thing I absolutely love about the ASP.NET MVC framework is that there isn't too much "magic". Magic can be nice if you are able to adhere to a framework's "Golden Path", but as soon as some customization becomes necessary you find that the magic can get in the way (DataContractSerializer anyone?).

One of the places in the ASP.NET MVC framework (and now part of .NET SP1) where magic is happening is routing. Routing takes a url, looks for the best fit from the pre-defined list of routes, breaks it up into parameters and feeds the whole thing to a handler. Pretty sweet, but if you have done something wrong it'll be tough to tell since all that is happening behind the scenes.

I've created a personal checklist to follow when I start to have routing troubles, if you have any additional steps you've found useful please post them in the comments and I will try to keep this checklist up to date. This article covers troubleshooting routes when using the ASP.NET MVC framework, however some of the steps mentioned will be applicable no matter how you are using routing.

1. Verify your controllers and actions.

I'm a contract first developer. I'll typically define my routes and views before I develop the controllers to use them. It's happened more than once that I started testing before I had written the method in my controller that I'm trying to access.

2. Look for duplicate routes.

This may not be as obvious as it seems. It's easy to create duplicate routes that read different but are hard for the routing system to distinguish. Remember that every parameter in your route is basically an empty slot where some data can be, so if you have 2 routes with the same number of slots in the same locations, they will be indistinguishable by the routing system.

3. Make sure your most specific routes are first.

Routes are tested against the URL in the order that they were added. If there is a less specific route that your URL can "fit" into before the one you actually wanted to execute, the routing system will choose that one and never get to your more specific route.

These 3 steps have helped me to successfully solve every routing issue I've come across so far, but if you have any other tips, I'd love to hear them.

Thinking In Rest - Part 2

Thu, 14 Aug 2008 17:43:14 GMT

So last time I covered the concept of resources (if you haven't read it yet, you should check that out here first) and the idea that when browsing the web you aren't looking at "pages" but instead you are looking at resources that happen to be formatted as pages. As it turns out, an HTML page is a pretty convenient format for a resource when a human is accessing it. Unfortunately, HTML is pretty poor for machine consumption - so how do we provide resources to client programs?

Representations

As is talked about in the very excellent book RESTful Web Services (OReilly) your resources can have more than one representation. A representation is basically a different way of delivering the same information - kind of like Tylenol. If you think about Tylenol you can get it in many forms: Gelcaps, Pills, Liquid, and Chewables. No matter what you are still getting Tylenol, it's just up to you to select the representation that is easiest for you to consume.

Obviously the most typical representation on the web is HTML. RESTful services however are most commonly found using XML, JSON, or both. The thing to understand is that it really doesn't matter. You could use any representation you like and it would still be REST. Just be sure to choose representations that your clients are likely going to be able to consume (imagine how well a new Tylenol delivered in convenient 30 Ton block form would go over).

Give Them What They Want

Many of us web surfers take certain parts of the web experience for granted. Such as getting back some HTML when we ask for a web page. It just seems natural that this would happen automatically; but it doesn't. Behind the scenes your web browser is telling the server what it wants and the server is paying attention. Here's an example of what I'm talking about. When I visit my weblog, I instruct my browser to go to: http://weblogs.asp.net/FreedomDumlao. My browser responds by doing the following:

GET /freedomdumlao/ HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml
Accept-Language: en-us

As you can see - the browser actually tells the server that it will accept HTML, XHTML, or XML. It even tells the server what language it will accept content in! It is then up to you, the service provider to attempt to comply with the request. Likewise, if you can't do what the client wants, you should tell them so.

Using the Accept header is a perfectly reasonable way of having your clients tell you what they want - but not always practical. Another common method is for the client to specify what they want by appending an extension onto the end of the URI. I first saw this practice used by the folks over in the Ruby on Rails camp, and it is also mentioned as a best practice in RESTful Web Services (OReilly). Using this method, the client would request one of the following URLs:

http://fakehost.com/posts.html
http://fakehost.com/posts.xml
http://fakehost.com/posts.json

No matter which one they call, they'll get the same data, with the representation that is implied by the extension. if the extension is not practical, you might consider adding the desired representation to the url path instead:

http://fakehost.com/html/posts
http://fakehost.com/xml/posts
http://fakehost.com/json/posts

You might think that this is inconsequential for the service you plan on building - because you your service will only have 1 representation. This is fine - however your client should still indicate in some way what it expects.

Making Things Happen

So we understand now that REST is about resources and resources can have any representation that we want them to. In the next article I'll discuss how we can create, update, and delete our RESTful resources.

Thinking in REST

Wed, 06 Aug 2008 20:31:00 GMT

I've been reading blogs around the .NET community lately and there seems to be some serious confusion about what exactly REST is. Is it a SOAP replacement? Is it a definition of how to prepare data for consumption by AJAX clients? And what does it have to do with all this new MVC stuff we've been hearing about? What is it exactly?

In the .NET and Microsoft world - we have been conditioned to believe in SOAP as the way to do machine conversations over the web (although SOAP of course is capable of much more than just web based services). In fact if you ask any .NET engineer what a web service is I'd bet that you'd here him/her talk about SOAP and probably ASMX or WCF as well. It's just as well too because honestly SOAP is a very well defined and robust way to do that job, and because it is so well defined we as .NET engineers often don't even have to think about the details, instead having them be handled automatically for us by the framework and tools we all love so much.

So then why are we even talking about this? Well, it just so happens that the entire world of technology doesn't actually speak SOAP. If you are a Java or .NET engineer you'll have no problem finding tools to parse out the WSDL and spin up clients for you automatically, but if you are using PHP or ActionScript you are likely going to have problems (especially if the service is using SOAP 1.2). But almost all technologies today have do have one thing in common - they can make web requests. Almost every technology has the ability to make POST and GET request, and most of those support the extended set of HTTP verbs as well (HEAD, PUT, DELETE, etc...). The nature of REST is to use those verbs, those verbs that are as old as the HTTP spec itself, to represent actions and to use that information to allow a service to make decisions about how it processes requests.

Now then - let's start thinking in REST.

So what is REST? Perhaps if we expand the acronym that will help us understand it better: REST = Representational State Transfer. Hmmmmm. I don't think that adds any clarity to the situation. So instead let's talk about what really sets it apart from the web services technologies/concepts you are already familiar with. The first think you need to be able to understand that REST is all about resources. "That's nice." you say, "But what exactly do you mean by 'resources'?" Well, resources are the things that make your application what it is. In a blog you have resources; posts, comments, pictures, etc. In an ecommerce site you have resources; products, user reviews, users, orders, and so on. REST is all about thinking of your application in terms of those resources instead of t

Technorati Tags: ASP.NET,WCF,Web Services

hinking about how you will provide interaction to those resources to your clients. Let's look closer at the blog resources I mentioned.

If I wanted to provide a SOAP API to my weblog I'd probably start by writing 2 methods: GetPost and GetRecentPosts right? But wait - I already HAVE a way to tell the server that I want to get a post...

http://mycoolbutfictionalblog.com/posts/my_cool_post

See? If that were a real weblog, I bet I could follow that to a post called "My Cool Post" or something along those lines right? So let's take a look at what the browser is doing when we follow that link.

GET /posts/my_cool_post HTTP/1.1 Host: mycoolbutfictionalblog.com (more stuff we'll talk about later...)

Look at that! The browser is doing EXACTLY what I want to do with my web service! It's GET-ing the post. But what happens if I want to get a list of all the posts? I guess I would go to:

http://mycoolbutfictionalblog.com/posts

And the browser would do:

GET /posts HTTP/1.1   
Host: mycoolbutfictionalblog.com    
(more stuff we'll talk about later...)

So all by itself our blog has managed to support 2 methods that I wanted to create in my web service. I already have "GetPost" because of the GET /posts/my_cool_post and I have "GetPosts" by means of GET /posts. The only problem of course is that these posts are returning in HTML format right now. Not ideal for using in a web service.

The truth is, at this point, we can truly claim that we have a RESTful web service to retrieve a Post or a collection of Posts. REST doesn't define what format the resource is in - and that is where a lot of the confusion comes in. Until learning about REST, you probably thought about that blog post as a web page that had a blog post on it. Now that you are beginning to understand REST you should see that it is a blog post formatted as a web page. The post is what we are asking for, it just happens that the current delivery of the post is using an HTML container.

Once that logic is in place then, we can see how we could do the exact same thing but instead of formatting the post as an HTML web page, we could choose to format it as XML. We could choose to format it as JSON, or AMF, or any other way we might want to return that resource to the client.

There is quite a bit more to be written on this subject, and I'll pick it up again in a later post. For now though, remember, if you aren't talking about resources - you aren't talking about REST.