the Joy of Code

Project Liike

Josh Twist — Mon, 31 Oct 2011 18:17:44 GMT

Pronounced LEEE-keh, project Liike is the latest set of guidance from Microsoft’s Patterns & Practices team. Here’s an outline in their own words:

“Project Liike is an effort to produce guidance that helps organizations build mobile web applications for modern browsers. There are countless devices available today and all indications suggest this number is only going to grow, without convergence. The factors to help choose between native apps and web apps are fairly well-known. Those who choose the web face numerous decisions and challenges based on their requirements. This project aims to help.”

The project is just starting up and they’ve identified a number of scenarios they _could_ target. They need your help to work out which is the most important. So, your action here is to read through these:

And then hit Like (get it?) on the one you think is most important.

RIA/JS: Deleting Entities

Josh Twist — Mon, 17 Oct 2011 21:47:12 GMT

I got an e-mail from somebody who had watched the first few screencasts and wanted to know about deletes. Seemed like a good topic for our next screencast so here it is: Deleting with client-side change tracking:

Here’s the source as it’s entered into the console (plus the part that creates the array and datasource that I embed in the page).

var contacts = [];
 
var ds = $([contacts]).dataSource({
  serviceUrl: '/RiaJs-Demo1-ContactService.svc',   
  queryName: 'GetContacts',   
  bufferChanges: true
});
 
ds.refresh();

contacts.deleteEntity(contacts[1]);

// should output "ClientDeleted"
ds.getEntityState(contacts[1]); 
// deletes from the DB 
    
// and removes the item from the array   
ds.commitChanges();

Next time I hope to start working with some UI!

Building SPA Applications - the webcasts

Josh Twist — Sat, 08 Oct 2011 00:40:19 GMT

I wanted to create a 'header' page where I can link to each webcast about building rich, single-page applications with RIA/JA. This is that page and here are the links.

RIA/JS The barebones demo
In this demo we take a look at how easy it is to stand up a Domain Service and read/write data using the ria.js libraries and implicit-commit.
RIA/JS: Buffer, Commit or Revert – the choice is yours
This time we forego implicit-commit in favor of buffered changes, change tracking and more control.
RIA/JS: Deleting entities
Here we take a look at how to 'non-destructively' delete an entity with client-side change tracking.

I'll keep adding the links as more arrive

RIA/JS: Buffer, Commit or Revert – the choice is yours

Josh Twist — Fri, 07 Oct 2011 17:20:43 GMT

Here is the second in a series of webcasts about the RIA/JS project. This extends from the last webcast (RIA/JS- The barebones demo) and switches from implicit-commit to look at the bufferChanges: true mode.

It’s only 8 minutes, so check it out – you can download the original to your ‘device’ of choice from vimeo here: http://www.vimeo.com/30170543 (the link to download is on the right halfway down the page).

Here is the code that I entered into the ‘console’ during the demo for reference (but watch the video, ok?).

var contacts = [];

var ds = $([contacts]).dataSource({
serviceUrl: '/RiaJs-Demo1-ContactService.svc',
queryName: 'GetContacts',
bufferChanges: true });

ds.refresh();

$.observable(contacts[0]).setProperty('FirstName' : 'Barry');

ds.revertChanges();

$.observable(contacts).insert(0, {
'FirstName' : 'Erik',
'LastName' : 'Porter'
});

$.observable(contacts[0]).setProperty('FirstName' : 'Jonjo');

ds.commitChanges();

Back soon with more…

RIA/JS: The barebones demo

Josh Twist — Wed, 05 Oct 2011 17:43:27 GMT

Earlier this week I promised some blog posts about my new project: RIA/JS and I'm pleased to bring you the first, of hopefully many. I wanted to start with a series of short webcasts that show the technology and the pieces that are currently available. Note that these pieces are just an early preview, but you should get some ideas of where we're headed and this is a great time to provide us feedback on what you like/don't like and what you need.

I wanted the videos to be very short, around 5 minutes. This first one is a little longer as it has to set the scene - but it's still only 12 mins and I promise it will be 12 minutes well spent! Off you go:

Things you’ll need to get started:

Visual Studio 2010
RIA Services V1.0 SP2

Also, the NuGet package you want is: RiaServices.jQuery

If you don’t NuGet yet, you should, but you can still get the scripts you need over here on codeplex: http://wcf.codeplex.com/releases/view/69862

And about the demo:

I'd really recommend watching the web cast, but in case you want the code I entered into the console:

// create an array to hold my data
var contracts = [];

// create the data source and
// refresh it (populate the array)
$([contacts]).dataSource(
{
    serviceUrl: "/RiaJs-Demo1-ContactService.svc",
    queryName: "GetContacts"
}).refresh();

// add a new contact, write back
// to the DB using implicit-commit
$.observable(contacts).insert(0,
{
    FirstName: "Howard",
    LastName: "Dierking"
});

Stay tuned as next time I plan to show more features of data source and this time, I promise we’ll be closer to 5 mins.

Note: There is a little ‘bugette’ in the RiaServices.jQuery NuGet package at the moment that adds two JSON endpoints:
<system.serviceModel>
  <domainServices>
    <endpoints>
      <add name="soap" type="Microsoft …
      <add name="JSON" type="Microsoft …
      <add name="json" transmitMetadat …
    </endpoints>
  </domainServices>
You can delete either of the highlighted lines and you’ll be good to go!

A new challenge

Josh Twist — Tue, 04 Oct 2011 01:49:41 GMT

Most people interested in software development using Microsoft tools will be aware of a recent re-org at Microsoft – no surprise, churn at a large organization is to be expected and welcomed. As a result some people were moved around the business and I was given the opportunity to try something new. I’d had a lot of fun working with my friends on Workflow but I felt ripe for a new challenge and was particularly excited by the new project – which I’ll talk about in significantly more detail over the next few days.

Best of all, I got to leave Workflow with a little bit of a bang – presenting Building Applications with Workflow and Azure at BUILD 2011. Best of luck to my friends in the WF team!

Now, looking forward I’m jolly (British for super) excited to be working with the WCF RIA Services team – in particular joining their RIA/JS jQuery client effort and looking at how we can build the ultimate end-to-end experience for HTML and JavaScript developers. In particular I’ll be looking at rich HTML applications (think JavaScript heavy and fewer pages) and how they can rock with our end-to-end story including for mobile devices; both mobile web applications and even cross-platform native applications such as those built with frameworks like PhoneGap.

My good buddy Jeff Handley introduced RIA/JS with this blog post a few months back and it’s a great overview: RIA/JS - jQuery client for WCF RIA Services

To add to this I plan to generate a stack of step-by-step screencasts and blog posts very shortly. Stay tuned and come back soon!

Building Applications with Workflow and Azure – BUILD 2011

Josh Twist — Wed, 21 Sep 2011 19:20:22 GMT

Thanks to everybody who attended my session at BUILD – I was blown away by how many of you made it to the last slot of the conference. For those who couldn’t make it, and everybody else who didn’t make it to BUILD – the session recording is available on channel 9:

http://channel9.msdn.com/events/BUILD/BUILD2011/SAC-867T

Be sure to check it out. I discuss how you can leverage Windows Workflow in Azure applications TODAY. Also, at the end of the session I give a preview of some super-early bits of the ‘Azure Workflow Service’ which we’re hoping to preview in 2012.

As promised in the session, I have the code from the first two demos for you.

You can get my slides from the channel 9 site (link above) and I’d love to hear your feedback, either here in the comments or on the channel 9 website.

Enjoy!

Speaking at BUILD

Josh Twist — Fri, 19 Aug 2011 17:54:03 GMT

Flights: booked. Accommodation: mm, hopefully booked soon.

I guess that means I should be pretty sure I'm going.

See you there hopefully?

Radio buttons are great, but…

Josh Twist — Thu, 30 Dec 2010 19:45:56 GMT

Are two clicks better than one? There’s a few places I see radio buttons used where I really wish the UI designer had used separate, ‘action’ buttons. For example, in Sharepoint 2010 when opening an Office Document:

Which I would prefer as:

Or this one from the software I’m using right now (Windows Live Writer)

where I would prefer

(Apologies for the crappy hacking together of my sample images, but you get the idea)

Not only would these different approaches reduce the number of clicks necessary to achieve your goal – they’d better satisfy Fitts’ law too – with the buttons representing a bigger target than a little radio button (and label, if the developer remembered to wire it up).

Cloud artwork - now as Visio Stencils

Josh Twist — Tue, 30 Nov 2010 18:33:58 GMT

A few weeks ago I shared some artwork I created for a presentation: Cloud Artwork.

A chap called John Goldsmith recently got in touch and asked (very politely) if I minded him converting them into Visio Stencils. Of course not, and they're available for everybody to share on his blog at: http://visualsignals.typepad.co.uk/vislog/2010/11/cloud-shapes.html.

Thanks John!

Synchronizing multiple nodes in Windows Azure

Josh Twist — Mon, 29 Nov 2010 19:56:41 GMT

My latest MSDN Magazine article is now available online:

Cloud Computing: Synchronizing Multiple Nodes in Windows Azure

It's based on some real-world experiences I had working on an Azure solution with a customer in the UK, with a focus on building systems that deal gracefully with elasticity (didn't you know, scalability is boring - it's elasticity that makes the cloud *really* interesting :).

My last blog post working for Microsoft UK

Josh Twist — Fri, 19 Nov 2010 23:12:33 GMT

Weirdly, it's exactly four years since I joined Microsoft in the UK as an ADC (Application Development Consultant). Spooky then, that this should be my last day as an ADC and working for Microsoft Services. Without a doubt the last four years has been the best and some of the most enjoyable of my career, with my proudest moment being in November 2009 when I was lucky enough to win the title of Systems Developer of the year at the UK IT Industry Awards organised by the BCS and Computing Magazine.

I was even interviewed by Computing Magazine as part of a follow up story which, embarrassingly for me, you can read here.

It was a very short interview and therefore I probably shouldn't have been surprised to feel somewhat misquoted. But I did (especially the bit about my work with the open source community) - I remember how this particular sentence stood out to me: '“I don't intend to leave Microsoft, but I do plan to work with them in the US in the future and this accolade makes all the difference,” he says.'

At the time it was just an answer to a direct question about whether I planned to leave Microsoft. A passing comment that I didn't take all that seriously. Weird then that this is exactly why today is my last day working for Microsoft UK, as I start work for Microsoft US as a Program Manager on Monday working with the Workflow team.

As you can imagine, I'm pretty excited about getting my teeth into actually shaping the pieces that developers use. I'm hoping to take my experience from the field and use it to help improve the experience and productivity of all the people I've been working with for the last four years, and hopefully many more (I'm very keen to continue with customer interaction so if you work with Workflow - I want to hear from you, if you don't - I want to know why).

However, on the flip side I'm sad to be leaving a team packed with the brightest minds I've ever had the pleasure of working with, many of whom I still regard amongst my closest friends. It was a blast - thanks guys.

Josh

PS - The ADM team in the UK are hiring. If you want to work with that fabulous team drop me a line.

Getting good at Parallel: 1 - Cancellation

Josh Twist — Wed, 03 Nov 2010 11:46:16 GMT

This is part of a short series of posts following my presentation ‘Getting Good at Parallel’ for the NxtGenUG Southampton user group. This post is all about the subtleties of Task cancallation in the TPL. The expectation is that you’re already at least vaguely familiar with Tasks (maybe because you attended my session) but to recap quickly.

The TPL provides a CancellationTokenSource that can be used to cancel Tasks. The code might look something like this:

CancellationTokenSource cts = new CancellationTokenSource();

Task taskA = Task.Factory.StartNew(() =>
    {
        // do some work
        Thread.Sleep(1000);
    }, cts.Token);



Task taskB = Task.Factory.StartNew(() =>
    {
        // this task decides to cancel
        cts.Cancel();
    }, cts.Token);

In this very basic sample we have two tasks and both share the Token provided by our CancellationTokenSource. Notice that taskB decides (for whatever reason you like) to cancel the Token and any tasks that are sharing that token. But what actually happens when Tasks are cancelled? running Tasks plucked from the processor and brutally stopped? Do I get an exception? This post hopefully answers a few of these questions with the aid of some simple diagrams that represent the flow of tasks, like this one:

Here we have two tasks, A and B. The arrow represents the time the Task is Running (that is, actively undergoing execution). As you can see, both tasks begin executing at approximately the same time. Task A will run for significantly longer than Task B, which calls Cancel() on the CancellationTokenSource. Both tasks share the same token from this CancellationTokenSource.

So what happens when Task B’s body calls cts.Cancel(). With the code above, pretty much nothing.

Note that Task A ISN’T going to be plucked from existence and aborted. Task A will happily continue to its natural conclusion unless the developer has coded for co-operative cancellation inside the body of Task A, i.e. the code checks to see if cancellation has been requested and exits the method (via return or an exception).

The behaviour of cancellation gets more interesting when we try to do some co-ordination by waiting on Task A or B.

Task.WaitAll(taskA, taskB);

If, for example, we call Task.WaitAll and specify both Task A and Task B, the WaitAll invocation would return at time t3 shown on the diagram; that is, when all tasks have completed.

Task.WaitAny(taskA, taskB);

If we call Task.WaitAny and specify both Task A and Task B, the WaitAny invocation would return at time t2 shown on the diagram. This is when the first task completes. Note that it isn’t when the cancellation is invoked. Task 1 would continue to process in the background until time t3.

OperationCanceledException

Both WaitAll and WaitAny (static methods of the Task type used above) support an overload that accepts a CancellationTokenSource parameter. If this token source is cancelled (as above) then WaitAll and WaitAny will throw an OperationCanceledException the moment the Cancel() is called. One exception (ha!) to this rule of course is if a task is completes before the CancellationTokenSource is cancelled. In that case, WaitAny will return the index of that task.

What happens if we add a third task, C, that is scheduled (taskC.Status == TaskStatus.WaitingToRun) but doesn’t get executed until around the time Task B completes, as shown below:

How does this change the behaviour of WaitAll and WaitAny?

Task.WaitAll(taskA, taskB, taskC);

If we call Task.WaitAll and specify all three tasks, the WaitAll invocation would throw an invocation exception at time t3. The AggregateException would contain a TaskCancelledException created for Task C – because its execution never took place.

Task.WaitAny(taskA, taskB, taskC);

If we call Task.WaitAny and specify all three tasks the WaitAny invocation would not throw any exception – it would return 1 to indicate that taskB completed and released the wait at time t2 on the diagram. If you’d need to know about cancellation at this point – you would simply use the overload that allows you to specify a CancellationTokenSource as discussed in the OperationCancelledException callout above, e.g.

try
{
    Task.WaitAny(taskA, taskB, taskC, cts);
}
catch (OperationCanceledException)
{
    // operation was cancelled
}

Exceptions, exceptions everywhere

During my session one of the attendees raised the point that if feels, to some degree, that the TPL tends toward driving control flow with exceptions. I can see why one would think this but I've still not had a chance to decide whether I really agree. After all, lots of the framework requires the handling of exceptions that would, to some degree, drive your business flow. The important thing is that your business logic sufficiently abstracts that I guess. Anyway, the good news is there are ways to avoid using exceptions as shown above if you prefer, e.g.

public static void WaitUntilTasksCompleteOrCancellation1
    (Task[] tasks, CancellationToken cancellationToken)
{
    var mres = new ManualResetEventSlim();
    using (cancellationToken.Register(() => mres.Set()))
    {
        Task.Factory.ContinueWhenAll(tasks, _ => mres.Set());
        mres.Wait();
    }
}

public static void WaitUntilTasksCompleteOrCancellation2
    (Task[] tasks, CancellationToken cancellationToken)
{
    var allCompleted = Task.Factory.ContinueWhenAll(tasks, delegate { });
    var allCompletedHandle = ((IAsyncResult)allCompleted).AsyncWaitHandle;
    WaitHandle.WaitAny(
        new WaitHandle[] 
        {allCompletedHandle, cancellationToken.WaitHandle} );
}

Not pretty, but if you have to avoid exceptions – there are options.

Next up in the series – cancelling a long running ‘async’ task. Huge thanks to Stephen Toub for all his help and support in dealing with my questions. Thanks!

Getting Good at Parallel (with NxtGenUG)

Josh Twist — Wed, 03 Nov 2010 11:45:25 GMT

On Thursday last week I presented a session on .NET 4.0’s Task Parallel Library (TPL) for the NxtGenUG Southampton user group. It’s a long way for me (I live in the North West of England) but it’s always worth the trip. I had a lot of fun, enjoyed the (challenging!) questions the audience threw at me and enjoyed the post session beers and discussion. Here’s the slides and demos from the presentation:

GettingGoodAtParallel.pptx (1.2MB)
Demo Source Code (zip 0.2MB)

During the session there was a veritable stack of ‘what ifs’ we pondered. Some I couldn’t answer because we were running out of time but most, I simply didn’t know. And so I promised to do some further investigation and experimentation and follow up with a blog post or two. This post will serve as the index for a few posts to hopefully close those questions.

The posts:

Getting Good at Parallel: 1 - Cancellation
Coming soon…

Generating images using WPF on the Server

Josh Twist — Mon, 11 Oct 2010 16:58:12 GMT

… and ‘plugging any nasty leaks you might see’.

Recently I worked on a proof of concept where we wanted to leverage the power of WPF and DataTemplates on a server to generate images. This is actually surprisingly easy and, unsurprisingly, very powerful.

The project planned to use WPF to generate images for a PivotViewer collection. If you haven’t checked out PivotViewer yet you really should – it even comes in Silverlight form these days and it’s all kinds of great. A phenomenal way to add simple BI like slicing and dicing of data with mind-blowing visuals and transitions.

To enhance the raw visuals in the source data (just portraits of people in this case) we wanted to generate new source images with a data overlay. Something like this:

With a mugshot at the back and some overlaid visuals to create a composite image. In this example we simply have a simple KPI indicator and the name of the person. A bit of colour adds an extra visual dimension to the PivotViewer and can be used to great effect.

Of course, the idea of overlaying the PivotViewer source images is such a good idea that the Pivot team already thought of it. If you use their PAuthor.exe tool then you can specify a HTML template to be used for an overlay. However, in this exercise we had a dynamic collection with a dynamic collection of images and wouldn’t be using the command line tool. Also, with the developers familiar with Silverlight it made sense to opt for the infinitely more powerful WPF/DataTemplate approach.

The DataTemplate would perhaps look something like this (btw, this is a different template to that shown above):

<DataTemplate 
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <Grid Background="White" Height="500" Width="350">
        <Viewbox>
            <Image Source="{Binding ImageSource}"/>
        </Viewbox>
        <StackPanel Margin="35">
            <TextBlock 
       Text="{Binding Attributes[Name], StringFormat='Name: \{0\}'}" 
       FontSize="20" FontWeight="Bold"/>
            <TextBlock 
       Text="{Binding Attributes[HireDate], StringFormat='Hired: \{0:d\}'}" 
       FontSize="20" />
            <TextBlock 
       Text="{Binding Attributes[TargetPercentage], StringFormat='\{0\}'}" 
       FontSize="20" />
        </StackPanel>
        <Border Name="TargetBorder" BorderBrush="Red" 
              BorderThickness="30" Opacity=".5"/>
    </Grid>

    <DataTemplate.Triggers>
        <DataTrigger Binding="{Binding Attributes[OnTarget]}" 
             Value="True">
            <Setter TargetName="TargetBorder" 
                   Property="BorderBrush" Value="Green" />
        </DataTrigger>
    </DataTemplate.Triggers>

</DataTemplate>

Notice how it makes heavy use of data binding and even a DataTrigger that would switch the border color from Red to Green if the OnTarget value is true. This DataTemplate would need some DataContext object it would be bound to, which might look like this (notice, this is a one-time only binding so no need for INotifyPropertyChanged here).

public class TemplateBindingSource
{
    public ImageSource ImageSource { get; set; }
    public Dictionary<string, object> Attributes { get; set; }
}

You’ve probably already guessed that this little chap will carry the ‘mugshot’ image (in-memory) and the Attributes dictionary is a simple name-value pairing of keys and data. You could easily imagine inflating this dictionary from almost any data source (an object via reflection, or a row of data from a database with each column representing a key).

The meat of the processing takes place in this method:

public byte[] ComposeImage(
       Dictionary<string, object> attributes, 
       byte[] imageData)
{
    byte[] result = null;

    _dispatcher.Invoke((Action)delegate
        {
            using (MemoryStream stream = new MemoryStream(imageData))
            {
                //// Load the bitmap image from image bytes
                BitmapImage image = new BitmapImage();

                image.CacheOption = BitmapCacheOption.None;
                image.BeginInit();
                image.StreamSource = stream;
                image.EndInit();
                image.Freeze();

                // Create the data context that the template will use
                TemplateBindingSource dc = new TemplateBindingSource
                {
                    ImageSource = image,
                    Attributes = attributes
                };

                DataTemplate template = TemplateService.GetTemplate();

                ContentControl element = new ContentControl
                {
                    ContentTemplate = template,
                    Content = dc
                };

                // Measure and arrange the tile
                element.Measure(new Size { 
                   Height = double.PositiveInfinity, 
                   Width = double.PositiveInfinity });
                element.Arrange(new Rect(0, 0, 
                   element.DesiredSize.Width, 
                   element.DesiredSize.Height));

                element.UpdateLayout();

                result = WriteVisualAsJpeg(element);
            }
        });

    return result;
}

The TemplateService.GetTemplate() call simply loads the DataTemplate into memory from disk, something like this:

using (Stream steam = File.OpenRead(filename))
{
    return (DataTemplate) XamlReader.Load(stream);
}

Of course, if you’re processing a lot of images then you wouldn’t want to read and parse the DataTemplate xaml file repeatedly. You’d want to cache this somewhere (be careful here, only the thread that created the DataTemplate can play with it!). The other interesting methods involved in this process are CopyStream (taken from this post) and the genuinely interesting WriteVisualAsJpeg:

private byte[] WriteVisualAsJpeg(FrameworkElement element)
{
    double factor = 1d;

    int w = Convert.ToInt32(element.ActualWidth);
    int h = Convert.ToInt32(element.ActualHeight);

    RenderTargetBitmap render = new RenderTargetBitmap(
        w, h, 96 * factor, 96 * factor, PixelFormats.Default);
    render.Render(element);

    BitmapEncoder encoder = new JpegBitmapEncoder();
    encoder.Frames.Add(BitmapFrame.Create(render));

    using (MemoryStream stream = new MemoryStream())
    {
        encoder.Save(stream);

        return stream.GetBuffer();
    }
}

To write the result to disk you can just File.WriteAllBytes() with the result of ComposeImage() above. Lemon Squeezy.

When I’m leaking images…

However, we did see one really notable problem with our first shot at this technique…

This was the memory footprint of our application over time:

Which resulted in this:

Ouch! Yup, an ever increasing amount of memory followed by an OutOfMemoryException. A sure sign that we’re haemorrhaging memory somewhere.

With some help of the smart folk on the WPF team (thanks guys!) we determined that the problem lay with the fact that the WPF code was running on a thread without an active Dispatcher to process any queued frames and we had been missing a key call to update the layout of the element. This was fixed by, amusingly enough, introducing a running Dispatcher the the equation and calling element.UpdateLayout() as highlighted above.

The bigger problem of course is running a Dispatcher on the server – not to mention that we need an STA thread to do any WPF work at all. Enter the BackgroundStaDispatcher class:

public class BackgroundStaDispatcher
{
    private Dispatcher _dispatcher;
 

    public BackgroundStaDispatcher(string name, TraceSource logger)
    {
        AutoResetEvent are = new AutoResetEvent(false);

        Thread thread = new Thread((ThreadStart)delegate
        {
            _dispatcher = Dispatcher.CurrentDispatcher;
            _dispatcher.UnhandledException += 
            delegate(
                 object sender, 
                 DispatcherUnhandledExceptionEventArgs e)
            {
                logger.TraceData(TraceEventType.Error, 0, e.Exception);
                if (!Debugger.IsAttached)
                {
                    e.Handled = true;
                }
            };
            are.Set();
            Dispatcher.Run();
        });

        thread.Name = string.Format("BackgroundStaDispatcher({0})", name);
        thread.SetApartmentState(ApartmentState.STA);
        thread.IsBackground = true;
        thread.Start();

        are.WaitOne();
    }
 

    public void Invoke(Action action)
    {
        _dispatcher.Invoke(action);
    }
 

    public void BeginInvoke(Action action)
    {
        _dispatcher.BeginInvoke(action);
    }
}

This class just creates a new STA Thread and gets a Dispatcher running on it. As you can see in the highlighted section in ComposeImage above (_dispatcher.Invoke) we’re using an instance of this guy to do all of the WPF grunt work. With this, and the call to element.UpdateLayout our memory profile looks like this:

Much better!