Some background

In v4 Macros were a bit of a pain to ship, in case you didn't know they were stored in the database and their data model was... less than ideal mainly as they evolved from being XSLT components to also supporting .NET, Iron* and eventually Razor.

Well here's a fun fact about v5 Macros they aren't in the database, Macros in v5 are actually stored in Hive. Out of the box they will be run of the file system Hive provider but since they are in Hive you could (in theory) stick them into the Database, on a FTP or anywhere crazy that you want. But really, them being on the file system is pretty fantastic as it means that it's really easy to include them in Source Control, something that was a huge problem with the Umbraco projects I'd worked with in the past.

You will find your macros (by default) at the location ~/App_Data/Umbraco/Macros and this contains a serialized XML version of your macro, which looks a lot like the exported Macro definitions in v4, just not quite as confusing. Also this is a configuration value that comes from the /configuration/umbraco/macros[@rootPath] section of the web.config. Again this is something that you can change but you probably shouldn't :P.

Installing Macros from your package

In my last post I introduced tasks and again this is what you'll want to use to install your macro with your package.

The first thing you want to do is copy your macro file from the Macros folder into somewhere that'll include it in your package. I use a folder called Macros that sites at the root of my package, but it's ultimately a personal preference thing so you can put that folder anywhere inside your package.

Side note Matt Brailsford has a great post on how to create v5 packages, a must-read until there's a UI to do it.

Part of the v5 source includes a task that can help you with this and it's called CopyFileTask and it allows you to copy a single file from one location to another. You'll see this task used as part of the DevDataset that's shipping with the RC builds but what you want to do is something like this:

  <add type="Umbraco.Cms.Web.Tasks.CopyFileTask, Umbraco.Cms.Web.Tasks" trigger="post-package-install">
    <parameter name="source" value="Macros/MyMacro.macro" />
    <parameter name="destination" value="~/App_Data/Umbraco/Macros/MyMacro.macro" />
  </add>

Adding that to your web.config in your packages (as described in my last post) will copy the file from the Macros folder in your package (or what ever folder you've put them into) to the Umbraco Macros folder.

Conclusion

Yes, that's it.

Seriously, it's so much easier to ship Macros in v5 that v4 and the fact that they are running off disk (don't be a dick and use some crazy Hive provider for them like Examine!) makes installing them as simple as copying a file.

But I think there's some room for improvements around this still and as I work on my v5 tasks set I'm going to be doing a simpler task for installing Macros, but in the meantime the above will work nicely today.

For this I have had to do a bit of digging into the Task system which is coming in v5, which is acting as a replacement for the traditional .NET event system, and in this article I'm going to share some tips when building installer tasks.

Right task for the job

In v5 there are two kinds of tasks available, Standard Tasks (my name) and Configuration Tasks and depending on what you're wanting to do you'll need to choose the right kind of task. Here's a quick overview of the two task types:

Standard Task

This is the most common type of task that you'll be creating; a task inherits from Umbraco.Cms.Web.Tasks.AbstractWebTask and requires a Umbraco.Framework.Tasks.TaskAttribute to be added so that the Umbraco framework layer will be able to find it (and you need to provide the attribute with a Guid for identification). This task type is very basic and can be used for any task that is raised in the system and then execute a piece of code, because of this you can think of it as being very similar to the event handlers that were in the Umbraco 4 system (or that you'll find in any .NET application).

Configuration Tasks

This task is primarily used in the install/ uninstall process of Umbraco 5 and inherits from Umbraco.Cms.Web.Tasks.ConfigurationTask. Where the previous task type you require an attribute the Configuration Tasks don't and you'll get some very undesired results if you do include the attribute. The power of this task type though is it allows you to specify values in the configuration file for the task, providing static values into the task as it is executed.

Side note - there is another task type Umbraco.Framework.Tasks.AbstractTask which is the base class for the AbstractWebTask but instead of relying on the web-side of Umbraco 5 it can be run without any web references. This would what you want if you are using the Umbraco framework outside of a web context, which it can do in-theory, but it's well beyond the scope of this post :P.

Task configuration

In addition to creating a class you'll also need to add a section in your configuration file that your task definition will reside within. There are two ways to do this:

1. Add to the master web.config file (not recommended as it can have upgrade issues)
2. Add your own package

I'm going to make the assumption that you're creating your own package here and you'll have your own web.config that you want to work against. First off you need to ensure you have the right web.config section:

<configuration>
  <configSections>
    <sectionGroup name="umbraco.cms">
      <section name="tasks" type="Umbraco.Cms.Web.Configuration.Tasks.TasksConfiguration, Umbraco.Cms.Web" requirePermission="false" />
    </sectionGroup>
  </configSections>
 </configuration>

This is the basis of your web.config file (and assuming there's nothing else in it yet) and what we've done is created a new web.config section called umbraco.cms and in that included the tasks section which uses a type provided by Umbraco.

Next we need to register our tasks:

  <umbraco.cms>
    <tasks>
      <add type="MyPackages.Tasks.MyAwesomeTask, MyPackage" trigger="post-package-install" />
    </tasks>
  </umbraco.cms>

This section would appear after the </configSections> node and adds the section which we defined and then within that we add our tasks. There are two pieces of information we have to provide it:

1. The fully qualified type of our task (namespace + classname + assembly)
2. A trigger for the task, for install tasks there is one called post-package-install

So that's the setup, now to make a task.

Creating your first task

So you're working on the next awesome package for Umbraco 5 and you need some stuff to happen when you install your package, well let's get cracking and make your first task. We'll do a basic task which will email you on package install, kind of a basic pingback to tell you when someone has installed the package. First up we'll make a class:

using System;
using Umbraco.Cms.Web.Context;
using Umbraco.Cms.Web.Tasks;
using Umbraco.Framework;
using Umbraco.Framework.Tasks;

namespace TaskDemo
{
    [Task("{C1C251E1-CACF-447A-9516-694251C16B08}", TaskTriggers.PostPackageInstall)]
    public class EmailOnInstall : AbstractWebTask
    {
        public EmailOnInstall(IUmbracoApplicationContext applicationContext) : base(applicationContext)
        {
        }

        public override void Execute(TaskExecutionContext context)
        {
            throw new NotImplementedException();
        }
    }
}

This is as empty a file as you can possibly have for an Umbraco 5 task, currently this will just error on install, pretty useful!

The important stuff will be happening within the Execute method, this is the method that is invoked when task is run and obviously where you want to put your logic, so let's build it out:

public override void Execute(TaskExecutionContext context)
{
    var email = new MailMessage
                    {
                        From = new MailAddress("phone-home@demo.com")
                    };
    email.To.Add(new MailAddress("new-install@demo.com"));
    email.Subject = "A new install has happened!";
    email.Body = "Hey dude,\r\nSomeone has installed your awesome package!\r\nH5YR!";

    var smtpClient = new SmtpClient();
    //server config skipped
    smtpClient.Send(email);
}

There we go, a very basic implementation of a task has been done! Here's the config for this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <sectionGroup name="umbraco.cms">
      <section name="tasks" type="Umbraco.Cms.Web.Configuration.Tasks.TasksConfiguration, Umbraco.Cms.Web" requirePermission="false" />
    </sectionGroup>
  </configSections>

  <umbraco.cms>
    <tasks>
      <add type="TaskDemo.EmailOnInstall, TaskDemo" trigger="post-package-install" />
    </tasks>
  </umbraco.cms>
</configuration>

Something you may notice is that in my config and in my class I've had to specify the trigger, this could be a mistake that I've made in my understanding thus-far but it seems to me that that is needed, someone feel free to correct me ;).

Creating a configuration task

As stipulated the above task is very basic but it does show you how you can work with the basics of a task. Well let's say that you want to create something a bit more advanced, say you want to have a task that will grant permissions to your custom application when the package is installed (this is a common problem solved in the PackageActionContrib in v4). For this we'll leverage the Configuration Task type so that we can make it a reusable task.

using System;
using Umbraco.Cms.Web.Context;
using Umbraco.Cms.Web.Tasks;
using Umbraco.Framework.Tasks;

namespace TaskDemo
{
    public class GrantPermissions : ConfigurationTask
    {
        public GrantPermissions(ConfigurationTaskContext configurationTaskContext) : base(configurationTaskContext)
        {
        }

        public override void Execute(TaskExecutionContext context)
        {
            throw new NotImplementedException();
        }
    }
}

So again we've got our skeleton class but this time we inherit from ConfigurationTask so that we can provide it with configuration values.

Inside the Execute method we can access the ConfigurationTaskContext.Parameters property which will contain the parameters that are passed in from our configuration file, like so:

public override void Execute(TaskExecutionContext context)
{
    if (!ConfigurationTaskContext.Parameters.ContainsKey("application"))
        throw new ArgumentException("No application supplied");
}

A simple check to make sure that we did get an application supplied, I want that as a pre-condition so that people don't break things on me!

But let's do something with the application provided:

Note: We're diving into the Hive here, I'm going to glance over how Hive works here, that's beyond the scope of this article, just believe me when I say that the code does work :P

public override void Execute(TaskExecutionContext context)
{
    if (!ConfigurationTaskContext.Parameters.ContainsKey("application"))
        throw new ArgumentException("No application supplied");

    var controller = (Controller)context.EventSource;
    //Get the ID of the current user
    var id = ((UmbracoBackOfficeIdentity)controller.User.Identity).Id;
    //Access the Hive user store
    using (var uow = ApplicationContext.Hive.OpenWriter<ISecurityStore>())
    {
        //find the current user in Hive
        var entity = uow.Repositories.Get<User>(id);
        //Add the specified app to their permissions
        var apps = new List<string>(entity.Applications)
                        {
                            ConfigurationTaskContext.Parameters["application"]
                        };
        //Update their permissions
        entity.Applications = apps;
        //Tell Hive to update the object -- possibly not needed
        uow.Repositories.AddOrUpdate(entity);
        //tell Hive that we want to save the changes to its store
        uow.Complete();
        controller.HttpContext.CreateUmbracoAuthTicket(entity);
    }
}

I've put some comments inline to explain the code as it goes but the important part is that we are reading the task parameters out and adding it to the users permissions.

Once this is all updated it amazingly will just give a new icon in the applications tray on the install of the package!

Now let's have a look at the config:

    <tasks>
      <add type="TaskDemo.GrantPermissions, TaskDemo" trigger="post-package-install">
          <parameter name="application" value="my-awesome-app" />
      </add>
    </tasks>

Conclusion

While the code above may look a little bit scare to begin with it's actually not that bad when it comes to creating tasks. There's a few simple rules which you need to remember:

1. Pick the right type of task for your work, do you want to pass in config values or can you compile everything together?
2. Do you need to work with anything web specific or is just the base FrameworkContext going to be enough?
3. Make sure you subscribe to the right event!

There's a few tasks built into the core of Umbraco 5 for copying files so that can also provide a good reference source.

For hosting I wanted to go with Heroku as I just love how simply I can get a site from my local machine to deployed with the platform and I also love how many add-ons there are available.

To send emails there's a couple of choices, I decided to go with SendGrid for no reason other than they were the first that I saw :P.

So install SendGrid into your heroku app (I'm using the free version):

heroku addons:add sendgrid:starter

And now you need something to send emails from NodeJS, for this I've gone with node_mailer as it was the first in my search results and it's got a dead simple API. What's really cool about Heroku is that when you have add-ons such as SendGrid installed you get the config options injected, meaning sending an email is as simple as this:

var email = require('mailer');

email.send({
	host: 'smtp.sendgrid.net',
	port: '587',
	authentication: 'plain',
	username: process.env.SENDGRID_USERNAME,
	password: process.env.SENDGRID_PASSWORD,
	domain: 'heroku.com',
	to: 'someone@somewhere.com',
	from: 'someone@somewhere-else.com',
	subject: 'You sent an email',
	body: 'Hey look at that!'
}, function (err, result) {
	//Do your error handling
});

You have to hard-code these settings:

• host: 'smtp.sendgrid.net'
• port: '587'
• authentication: 'plain'

But Heroku will inject the username & password for you, both of which will be on the process.env object, like so:

• process.env.SENDGRID_USERNAME
• process.env.SENDGRID_PASSWORD

And there you have it, you're now ready to send emails from NodeJS on Heroku.

But there's the obvious problem, I want to push data to the UI but I don't have any way to get the data.

Luckily I wrote tbd recently which can solve one of the problems, it can generate data to pump into my new UI and this is where AmplifyJS really comes to shine.

Introduction to faking data with amplify.request

To simplify my front-end routing I'm going to be using the Request API from AmplifyJS and if you're not familiar with it check out the docs before going further as I only plan to cover the testing side of it.

Let's say I have a route defined like so:

amplify.request.define('get-data', 'ajax', {
	url: '/data-service',
	dataType: 'json',
	type: 'GET'
});

And later in my app I'm accessing it:

amplify.request('get-data', function (data) {
	//using templating pump out the UI from the data
});

So where does the faking data come in? Well the cool thing about how Amplify is designed means that you can replace a defined request!

Say what?!

First thing we need to understand is the request types. When you define your request in AmplifyJS the 2nd argument you pass in is the request type, generally speaking this will be ajax as that is the provided request type in the API. You can define your own types so if you were wanting to pull in from an OData service you can setup that, add a new key to the request types and then it's all sweet (sorry how to do that is beyond this articles scope).

Where it gets really interesting is that if your provide only two arguments to the define method, a key and a function this works as well. In this case your function is executed when you invoke the request. Now let's add this code:

amplify.request.define('my-data', function (settings) {
	settings.success({
		status: 'success',
		items: { }
	});
});

This will make it so that whenever I call my request I will get a successful response with no data. There are properties which you need to set, first is the status to success so that AmplifyJS knows the response was successful, second is the items property which will contain any data you want returned to the method.

Setting up your project

Now you've got the basics down I thought I'd just give a bit of an insight into how I go about including this into a project. As mentioned you can override a defined request as many times as you want:

amplify.request.define('my-data', 'ajax', { ... });
amplify.request.define('my-data', 'odata', { ... });
amplify.request.define('my-data', function (settings) { ... });

Here I've setup the request three times but the last one to be executed is the one included.

The way I setup my project is that I have a file which I define my requests in, all of them together (or at least logically broken down into groups of common requests). For when I'm wanting to stub out my requests I create a secondary file and include the stubbed out requests in there and then include it directly after the main file. This means that once the real request is created it's immediately replaced with fake out.

With the fake requests in a separate file I can include or exclude them as I please, as my services come online or even use them in unit tests.

Building your data

The idea of doing this all with AmplifyJS was shown to me by Elijah Manor. He sent me this jsfiddle which shows it all setup.

The problem with examples like this is that they are using fixed data, every reload of that page will show you exactly the same thing and clicking the refresh button on the UI will reload the data with exactly the same data. Now in this demo it's not really that bit a deal, the data doesn't really need to look different each time it's not going to make much difference. But what if you are doing something that will look different based on the data, say you're doing some charting?

I've created a jsfiddle to demonstrate this, when you click the button the chart will be rebuilt with different data.

Now here's my mock request:

amplify.request.define('get-data', function (settings) {
    var data = tbd.from({})
                .prop('category').use(tbd.utils.random('a', 'b', 'c', 'd', 'e')).done()
                .prop('value').use(tbd.utils.range(10, 100)).done()
                .make(tbd.utils.range(3, 8)());

    settings.success({
        status: 'success',
        items: data
    });
});

With tbd I'm scaffolding out a data series for my charting API, using the alphabet as the label and then a randomly chosen number between 10 and 100 for the value. This means that as I generate new data my UI will change (I'm leveraging one of tbd's util methods to generate a random number of results as well (I might clean up the API to make that simpler in the future).

Conclusion

And there we have it an example of how we can combine a couple of helpful JavaScript libraries to make it easier to:

• Simplify out UI request layer
• Make sure our development isn't halted while data services are under development
• Have less hard coded data responses

Some of the tests I've had to go beyond what matchers are available out of the box so I thought I'd share them here (mostly so I've got an easy point for myself to find them again :P).

A quick into to adding your own matcher

I've you're new to Jasmine and haven't added your own matchers here's a quick tutorial.

You need to use the beforeEach and call addMatchers, like so:

beforeEach(function() {
   this.addMatchers({
       alwaysTrue: function(){
           //put your logic in here to determine truthy results
           return true;
       }
   });
});

Now you can call it like so:

expect('something').alwaysTrue();

This test extension that I've created isn't really useful as it will always return true but you can put in any logic you want.

The this object has an actual property, which is the value from your expect method and the arguments passed to the call are passed in.

My matchers

beforeEach(function() {
    this.addMatchers({
        toBeInArray: function() {
            return ~[].slice.call(arguments).indexOf(this.actual);
        },
        toBeInDateRange: function(min, max) {
            var actual = this.actual.getTime();
            return actual <= max.getTime() && actual >= min.getTime();
        },
        toBeInNumericalRange: function (min, max) {
            var actual = this.actual;
            return actual <= max && actual >= min;
        }
    });
});

Here's how you use them:

expect('a').toByInArray('a', 'b', 'c');
expect(3).toBeInNumericalRange(0, 10);
expect(new Date(2011, 11, 01)).toBeInDateRange(new Date(2011, 10, 01), new Date(2011, 11, 30));

Hopefully this is helpful to someone else :)

While last year I declared to be the year of the conferences but of course this year was just as crazy with conferences.

I...

• Got a Microsoft MVP award!
• Went to the USA for my first time to go to MIX11 and talked FunnelWeb at the Open Source Fest
  • Vegas is insane and I don't think I've ever been as hung over in my life as I'd been
  • Hanging out with Glenn Block was heaps of fun, he's such a top guy
  • I got to have lunch with Scott Hanselman, Phil Haack and Rob Conery which was pretty awesome from a nerd point of view
  • The pre-party also allowed me to meet some of my idols in Douglas Crockford, Dave Ward and Elijah Manor
• Next stop was Melbourne and DDD Melbourne where I got to present on JavaScript craziness
  • And watching Steve Godbold rap was just hilarious
• Next up was REMIX where I spoke on being a web developer and had to do an impromptu session where I talked about JavaScript again (although admittedly I lost the audience on the second talk :P)
• It wasn't long until I was back on a plane to get to Denmark for CodeGarden 11 where I talked about everything from ASP.NET MVC to what is interesting in Umbraco 5
  • I have since been removed from the Umbraco project but I stand by what got me removed, instead I am trying to get people involved from the outside
• Back in Australia and it was time for DDD Sydney and a revisit my DDD Melbourne talk about getting freaky with JavaScript and talk about Open Source in the panel session run by Nick Hodge
• But no rest for wicked it was time for Teched NZ (slides) and then Teched AU (slides & video)
  • I got voted one of the top sessions in the web track which was pretty exciting!

Phew!

For the first time in 2 years though I didn't resign from my job, I know, shockingly I'm still working at the same company I was working for this time last year.

Generally speaking I've scaled back my Open Source work, I've still tried to be active in the various communities but instead of actively developing I've tried to be more of a voice of reason. I've release a few small things such as:

• Another pub/sub library in JavaScript
• A JavaScript quiz website
  • Which I then rewrite in NodeJS
  • But admittedly it's not been getting a lot of love these days from myself or the community
• A library to building test data in JavaScript

So here's to a more JavaScript filled 2012!

Hi, my name's Aaron and I'm a former member of the Umbraco core team. Before I departed the core team something I was pushing for was greater involvement between the core (and HQ) and the Umbraco developer community.

Let me make sure I clear one thing up first, Umbraco has a great user community, our.umbraco thrives with huge number of contributors helping everyone out from the beginner to the advanced. The extensions community is also a hive (sic) of activity.

The community that I'm talking about is the one around developing Umbraco as a product.

If you're reading this you most likely have some vested interest in Umbraco, you're doing freelancing and implementing it, your company sells solutions based around it or you are like me and just find CMSs sexy (well let's hope that's not the case :P). Whatever the case may be the direction of the product does have an impact on you so you should make sure your voice is heard. Even if you're not a developer your voice is important, feedback about things you/ your clients find challenging, features you'd like to see, testing alpha/ beta/ RC released or even just questioning why something was done a particular way.

How do I get involved?

This all sounds well and good but what should you do?

A few months ago the Umbraco 5 contrib was opened up and it's already starting to get some stuff around this up there.

There's a post on contributing and then there's the discussion around automated UI testing which Matt Braildsford took into a larger discussion last week.

Lastly there's a room on JabbR but it's a little quiet still these days.

Keep an eye on the Umbraco twitter stream and grab nightly Umbraco builds.

Raise a discussion if you don't understand why something was implemented some way (but keep implementation discussions to our.umbraco).

Monitor change set commits and make sure they are still conforming to what standards there are.

So c'mon, get involved!

This is a situation that I find myself in quite often and it always left me thinking about how I would throw together some data to do the UI. Generally speaking it'd involve a bunch of copy and pasted lines of JavaScript which builds up an object graph. This does work but it's not a great way to simulate data, especially if you want to change the data volumes and see how the UI will react.

Coming from a .NET background I've used libraries like NBuilder and Fabricator in the past. These libraries take an input object and will generate a series of fake data from it.

So I thought "hey, why not create that in JavaScript" and from there tbd was born!

tbd - Test Data Builder

tbd, or Test Data Builder is a project I started to create (fake) data using JavaScript. There's a bit of a joke in the name, when I was trying to pick a name I was thinking "what'd be quirky, it's for building test data, oh sweet, tbd since it can be Test Data Builder or To Be Defined, which makes for a good play on words". Now the astute reader will notice the mistake immediately but for those with reading problems like me you'll need a hint, Test Data Builder is actually tdb.

The idea behind tbd was to be able to take a JavaScript object and create a bunch more of them, as many as you want! I also wanted it to be disconnected to the browser so you could run it in both Node.js and the browser. For running it in the browser you need to add a reference to the file and with Node.js it's up on npm.

Here's a basic example of how to use it:

var data = tbd.from({ hello: 'world' }).make(10);

This will create an array of 10 objects all which are identical. Since JavaScript doesn't have reflection like .NET there isn't a way to get an inferred type of a property, instead you have to assign it a 'default' value. This can have an advantage though as it means you can also only have tbd generate values for properties you actually want random data for. So how do you do values?

var data = tbd.from({ foo: 1 })
            .prop('foo').use(function() { return Math.random(); }).done()
            .make(10);

This will create an array of 10 items which have a unique random number (well as unique as a random number can be :P) for the property foo. To break down the way it works you need to understand the fluent API for properties.

1. Pass the property name to the prop method as a string
   1. Pro tip - if the property name doesn't exist on the source object it'll still be added!
2. Pass a value or function into the use method
   • If you pass a value that will be used for each object
   • If you pass a function it'll be invoked for each object
   • There's some helper methods we'll look at later
3. Call done to signify you've finished with that property so you're back to the root API and allow you to modify more properties

The last thing you always call is make and specify the number of objects you want.

And that's it, you can go off and create all the data you could ever want.

Making better fake data

Since tbd is a really dumb API if you don't tell it what to do with a property it wont do anything. So how do you produce better fake data? That might sound like a silly question but say you're trying to build some graphs, you don't want all the data to be the same do you?

Well to simplify this tbd ships with a number of useful utilities for generating better fake data. The full list you can get off the readme but are some of the most useful IMO. All of these reside in the tbd.utils namespace.

Pick a random value

Say you want to randomly choose a value from a set of values, say a bunch of different word, there's a handy method that'll do that called random and you use it like so:

var data = tbd.from({ hello: 'world' })
            .prop('hello').use(tbd.utils.random('me', 'no me', 'why not me!')).done()
            .make(10);

This method takes n number of arguments and it will randomly choose one of them for each object. You can pass in any data type that you want to this and it'll take a random from the list so you're not just restricted to strings.

Better random numbers and dates

While the random method is great if you have a small set of data to go through but what if you don't? What if you want a random number between 1 and 1000? Typing that out would suck. Well luckily there is the range method:

var data = tbd.from({ hello: 'world' })
            .prop('hello').use(tbd.utils.range(1, 1000)).done()
            .make(10);

For range you pass in a min and max value and something from in there will be used. This method also supports dates so you can randomly choose a date from within a range.

Sequences

Sometimes you just want an ordered list of values and that's where sequential comes in. Sequential you provide a start point and you'll get a value incremented by one each time from there:

var data = tbd.from({ hello: 'world' })
            .prop('hello').use(tbd.utils.sequential(1)).done()
            .make(10);

Note: I had a mistake in the initial post, the method is sequential not sequence.

Date sequences

One cool thing about the sequential api is that you can provide it a date and it will increment that. By default the dates will shift one day at a time to get you to your new date:

tbd.from({ date: new Date })
    .prop('foo').use(tbd.utils.sequential(new Date() /* optional parameter for date property the increment, default is 'd' */)
    .make(10);
//the 'day' property will be incremented by 1 from the starting value

But you can overload this to increment by other date parts:

• y -> Year
• M -> Month
• d -> Day (default)
• h -> Hour
• m -> Minutes
• s -> Seconds

Conclusion

So this wraps up our look at tbd, a useful little tool I wrote to make it easier to build out some fake data for when you're mocking a UI or to pump into a test.

Grab it today!

For anyone who's not familiar with the proxy method is allows you to take a function and specify a context (the this value) so when you pass it around for execution you always know what you're going to have as the context.

The method has been around for a while and it did serve a good purpose but these days its usefulness is becoming limited and I'm going to look at a few reasons as to why you shouldn't be using it.

Understand this

The most common reason I see that people will use proxy is because they don't understand how this works in JavaScript. While there are dozens of articles through your favourite search engine explaining this (here's a good start) I'll do my best to give a quick overview.

When people come from C# they already have a notion of this and what it represents, unfortunately this is a broken assumption when moving to JavaScript. In C# this represents the current class which you're working within but since JavaScript isn't a classical language there isn't the concept of a class so you couldn't really associate it to one could you?

Instead this represents the context for which a function is executed for and different kinds of execution result in different contexts. Here's a few examples:

• Invoking a function from an object will set the context to the owner object
• Invoking a function literal will set the context to the global object (window in a browser)
• Invoking a function with apply/ call allows it to be controlled

And this is why a lot of people get utterly confused with this in JavaScript.

But what does this have to do with the $.proxy method? As I said one of the most common reasons I see people using it is because they want to be able to access members of a particular object within a callback (such as a function called from an AJAX success). Because they are aware that in their outer function they can go this.foo they expect it to be available within the callback (after all isn't the callback in the same class?) but it will fail so they use $.proxy to ensure that they can access the member(s) they require.

So why is it a poor choice in this scenario? Well the reason is that you can solve the problem in a much simpler fashion, by understanding JavaScript closures. You can draw a lot of similarities between JavaScript closures and C# closures, but the simplest explanation is that a variable will be within scope until all functions that require it have been descoped. But with the case of this, since it changes between function scopes the var that = this pattern emerged in JavaScript.

Let's have a look at some code:

var foo = {
    makeRequest: function () {
        $.get('/foo', $.proxy(function (result) {
                this.update(result);
            }, this)
        );
    },

    update: function (data) { /* ... */ }
};

//somewhere later in the code
foo.makeRequest();

Here we're using the proxy method to make it possible to access the update method, since when we called the makeRequest method its this is a reference to foo (we're assuming that the assignment of foo is out of scope for the makeRequest method). So let's update it to use variable closures:

var foo = {
    makeRequest: function () {
        var that = this;
        $.get('/foo',function (result) {
            that.update(result);
        });
    },

    update: function (data) { /* ... */ }
};

foo.makeRequest();

The difference here is that I'm assigning the value of this to a variable before the callback is created and inside the callback I refer back to the variable.

So why is this better than using proxy? The primary reason is readability, if you look at the first snippet you're intention is obscured by the use of proxy to change the scope. When someone who understands JavaScript comes to that snippet they have to know what the use of proxy is and why the this context needs to be controlled. With the second snippet it's clearer to see that you want to call the update method on the same object which makeRequest was invoked from.

Use built-in methods

An often missed note of JavaScript (well ECMAScript really) is that the functionality that is provided by proxy (and the similar methods in the other libraries) is built into the language, through the bind method. The bind method was added as part of ECMAScript 5 and

Creates a new function that, when called, itself calls this function in the context of the provided this value, with a given sequence of arguments preceding any provided when the new function was called.

Hmm that sounds pretty much like what you get from the proxy method yet it's built into the language.

This means that you can (potentially) get a performance boost (check it out on JSPerf) by using a native browser API rather than the wrapper. Also at the time of writing jQuery (1.7.1) doesn't use the native browser method it does the code itself.

Performance aside the main difference between the jQuery implementation and the ECMASCript 5 specification is the jQuery proxy method does not throw a type error if the first argument is not a function which the spec states bind will do. So although they are named differently they are providing the same functionality except for a critical check.

The take away from this point is that it's built into the language so using it makes a lot of sense and when you're in browsers that don't support it it's easy to polyfil the missing API (the mdn docs include the polyfil).

Conclusion

That wraps up my "rant" against using the proxy method in jQuery. The goal of this article was to teach you a bit more about the JavaScript language and that things people commonly try and work around have simpler solutions.

By understanding language concepts such as this and closures you can avoid manipulating scope.

By knowing what's new in the language you can use built-in APIs and make faster and more portable code.

The one thing I do remember from my brief foray into that scary other world is the reliance on INotifyPropertyChanged and INotifyPropertyChanging interfaces. I've always thought that the idea behind these two interfaces was a good one, the primary problem though is how you actually have to implement them. Seriously, there's a lot of shit code you have to implement.

So I decided to do something a bit silly, I decided to implement the two interfaces in JavaScript.

A dumb idea with a point

Now I see no real reason to use the code that I'm going to look at in any current development (I'll explain why later) but more importantly I want to look at something that is part of ECMAScript 5 that doesn't get the attention it deserves, Object.defineProperty.

JavaScript properties throughout history

JavaScript, unlike languages such as C#, doesn't really have this concept of a property like you get there, the idea of a get and set operating being something that you can control. Really this is how a class in JavaScript with some public properties looks:

var person = {
    firstName: 'Aaron',
    lastName: 'Powell'
};

And when we want to update a property we'd do something like this:

person.firstName = 'John';

Now there's nothing wrong with this, it does what you'll want to do in a lot of scenarios, the problem is when you're wanting a slightly more complex scenario, say you want to react to a change to the firstName property, maybe perform some validation.

Let's assume we want to have an age property on our person. Obviously we want to make sure that age is at least 0 and probably less than 110 (sounds reasonable :P), well how do you do this?

• Validation before assigning the property?
  • That'll work, but what if we're exposing it to external API's? How can we enforce the validation to them?

Functions as properties

The general way which this problem is solved is to rather than use assignable properties you use functions as properties, making your code look like this:

var person = (function() {
	var _age;

	return {
		//firstName, lastName, etc
		age: function(val) {
	      	if(val !== undefined && (val >= 0 && val <= 100) {
	        	_age = val;
	      	} else {
	        	//Raise an error
	      	}
	      	return _age;
    	}
	}
})();

Now we use the age property like so:

person.age(27);
console.log(person.age()); //outputs 27

Now this isn't really that bad, the main pain point to it is that we now have a different way to assign the value, we do it through a function invocation rather than through an assignment statement. This can come to light if you're writing a JavaScript templating engine, you need to check if the property is actually a property or a function property. But we do get some nice stuff like the fact that in JavaScript you don't need to do overloads so we can have the one function perform both the get and set operation for our property.

Libraries such as KnockoutJS use this pattern for properties to do their UI binding but it can cause confusion, like in KnockoutJS if you want to bind to a property you'd do something like this: data-bind="css: { someClass: someBoolean }" which Knockout will understand it's an observable property and bind to the result of the function, but if you want to use the false value you need to do data-bind="css: { someClass: !someBoolean() }". Note that this time it's invoked the property as a function rather than just using the property.

The can be a bit confusing and I've seen more than one developer (including myself) getting stumped as to why their bindings weren't working only to realise that it'd because they are binding to !someBoolean which equates to !function() { } rather than the result of the function. It's a very face-palm moment.

Introducing ES5 properties

As part of the ECMAScript 5 spec the concept of properties was addresses and has resulted in the Object.defineProperty API (and an API to define multiple at once, being Object.defineProperties) and this allows us to (among other things) define get and set method bodies for our properties.

Let's revisit our person.age property example from above, but do it using an ECMAScript 5 property:

var person = (function() {
	var _age;

	var newPerson = { 
		firstName: 'Aaron',
		lastName: 'Powell'
	};

	Object.defineProperty(newPerson, 'age', {
		get: function() { return _age; },
		set: function(value) {
			if(value && value >= 0 && value <= 110) {
				_age = value
			} else {
				//raise error
			}
		}
	});

	return newPerson;
})();
person.age = 27;
console.log(person.age);

Hopefully you can see here the difference between the function property and the ES5 property.

With ES5 the property with a function body looks just like a public field. Now there's a few other things you can do here, such as make properties read-only and exclude them from for...in loops, and for that check out the MDN (you can also do body-less properties), but it makes it very easy to build smarts into your objects. Also like .NET if you want to provide a get/set you need to have a backing store, but that's easy to get around with closure scope.

So that covers a basic look ES5 properties. Now back to our bad idea...

Implementing INotifyPropertyChange in JavaScript

As I demonstrated above it is possible to add a body to your properties let's do something with that idea.

If you're not familiar with INotifyPropertyChang* then you should read the MSDN docs. The TL;DR is that you use trigger the Changing event before you assign the property and then the Changed event after it's assigned and the UI can react.

As I said I think there's a lot of value in this pattern, it's just that as most Xaml devs will tell you implementing it is a real pain in the ass.

So say you wanted to implement it in JavaScript, it's not overly hard, ultimately we need to do something like this:

Object.defineProperty(foo, 'prop', {
	get: function() { return _prop; },
	set: function(val) {
		propertyChanging(this, 'prop');
		_prop = val;
		propertyChanged(this, 'prop');	
	}
});

I've ignored the guff code like what the propertyChang* methods are doing as well as subscribing handlers to the events but you get the idea. This really don't look any different to the C# version though does it? So what's the point?

Making it better through the magic of JavaScript

As you can see there's a lot of boilerplate code that you need to get this working. In .NET there's no real way to avoid this (unless you do some magic under the covers). But one of the cool things about JavaScript being a dynamic language is that we can modify an object pretty damn easily. Let's go back to our person object:

var person = {
	firstName: 'Aaron',
	lastName: 'Powell',
	age: 27
};

Now imagine that I want to implement my JavaScript version of INotifyPropertyChang* on it so that my UI can react whenever I update the values, but I don't want to be going through and writing this all out myself, I've got a bunch of objects that I want to promote to implementing the interfaces.

Well since JavaScript doesn't actually have interfaces in the language it's a bit tricky, and this is where my funky little script comes in.

Hello Xamlizer!

So I've created a little JavaScript code snippet which I've called Xamlizer that'll take an object and implement INotifyPropertyChang* on it. Now the script isn't really that smart, all it does is goes through all the properties of the object and then converts them into properties that implement our pattern.

You can then use it like this:

var person = {
	firstName: 'Aaron',
	lastName: 'Powell',
	age: 27
};

xamlizer(person);
person.addPropertyChanging(function(object, property) {
	console.log('Property ' + property + 'changing');
});

person.age = 28;

And there we go, we've got a script that'll turn our normal JavaScript objects into something that can notify subscribers when the property changes.

If you dig into the code for Xamlizer you'll see that it doesn't do anything really complex, it just modifies some properties. Note: As I said it's not really that smart, it actually modifies anything public on the object, so if you have a function that is public it might get crazy :P. But hey, it's just demo code!

And if you want to see it in action check out the jsfiddle.

Conclusion

Well this wraps up our look at the limitations in how you have to do properties in ES3, the changes which ES5 provides you with (although their usefulness at the moment is debatable since we have to support ES3 browsers for a while still) and finished off with looking at how to implement a generic library to change fields to properties with debatable usefulness.

Since then I've come to realise that it isn't the platform I want wanting to go with. One of my main problems with it is its comment management system. Anyone who has submitted an answer to me will know what I'm talking about, the excessive spam which you end up with when I do publish all the answers.

Well because of this I decided to move away from Posterous and go with a new platform. As my new platform I decided that I wanted to use Node.js because well this is a JavaScript quiz so why not use JavaScript!

The software

When looking at what I wanted to do with the new site I decided I wanted something that was easy to create a site in and also easy to update content it. A lot of people are raving about Jekyll of recent, which is a Ruby CMS which runs a flat file system website and Markdown as an editing language.

This seemed ideal, JavaScript Quiz isn't a big site nor is it a dynamic site so something that runs off flat files is very ideal. I'm also quite a fan of Markdown (which we use in FunnelWeb) so being able to write my posts in that is very nice an idea.

So I started looking for a Node.js alternative as I'd prefer to use something than write it myself (I'm a bit over developing a CMS at the moment) and I came across a project called Docpad.

Intro to Docpad

Docpad is a Node.js CMS in a similar style to Jekyll written by a guy from Sydney named Benjamin Lupton (and I like supporting home-grown software so that was a big plus). It's got a good set of templating engines to pick from so you don't have to use raw HTML if you want something a bit more cool for your templates (more shortly) and best of all it's shit simple to use.

You need to install the following npm packages and you're off and running:

• coffee-script
• express
• docpad

You're better off installing both coffee-script and docpad globally since they both have executables but you don't have to.

Note: I had problems using Node.js with cygwin on Windows, I couldn't get docpad to install but that seemed to be a cygwin issue as it worked fine on both my Linux and OSX machines, just something to watch out for :).

This isn't a Docpad tutorial, go check out the docs if you want to learn more.

Templates

As I mentioned Docpad has a number of different HTML templating engines available, you can use Eco, Jade, Haml or the one I chose, CoffeeKup!

CoffeeKup is a way of using CoffeeScript as a HTML template engine. It's pretty cool and it means that you're able to do some really powerful things with the templates and interacting with the document you're rendering. Plus it means that we're using JavaScript/ CofeeScript for most of our site (one language to rule them all!).

CSS

I'm not using any of the CSS templating engines (despite submitting a request for CCSS to be included :P) mainly because I'm using HTML5 Boiler Plate's css and I don't want to have to convert it every time I upgrade.

The rest of the CSS is really basic and I've just cobbled together so I can get the site live, expect it to be improved as I get more time.

Fixing commenting

As I mentioned commenting is something that was really a pain to anyone who was entering the quiz each week as you'd get spammed up with emails (don't worry, I got them all as well so it was very annoying). Good news is that the new site wont have this problem, I've gone with Disqus for comments (still moderated) which means that it should be much nicer an experience.

From an admin point of view it's much nicer as well :).

Hosting

One advantage of Posterous was that it was a hosted solution so it wasn't costing me any more and this is something that I wanted to ensure didn't change. I decided that I'd go with Heroku for my hosting since they have offered Node.js hosting for a while now.

This means that I am also using Git to store the site and I have it hosted on GitHub at the moment (sorry it's not a public repo :P).

Because of this I have a nice workflow of being able to edit my content, run it through the Docpad 'compiler' and commit in the generated HTML. This then goes up to Heroku and just runs off the flat files.

Ideally I'd not be committing the generated files and have part of the app startup code generate the files but so far I've had nothing but trouble getting it working that way. Heroku's cedar stack (which is where node.js runs) is a writable file system but something still seems to be going amiss (and it's not exactly easy to dig into...).

Wrap up

So this is how I've gone about the relaunch of JavaScript Quiz. The new site should be active soon (awaiting the DNS to change over :P). I wont be porting the old comments so the old site will stay active. Hopefully I've got the redirects all sorted out (yes the 404 page is pretty shit so far :P). Hopefully this provides a nice new home for the site.

I had a bit of tech with me, an iPad, iPhone, laptop and 2 kindles so I thought I'd share some of my experiences and tips for travelling as a geek.

Stay connected

Something that I find is very useful when travelling is having access to the internet. It allows me to do those useful things such as email/ skype my parents, use google maps and check in on foursquare.

So my first pointer when going overseas is work out how you plan to stay connected. Most hotels I've stayed in recently have had free wifi so if that's good enough for you then check out the places you plan to stay. Also a lot of cafe's and bars (particularly ones targeting travellers) offer free wifi so that can allow you your mid-day twitter fix.

The other option is picking up a local sim. I've been to the USA, Denmark and Vietnam this year and all those countries have prepaid sims which you can pick up and drop into your device. For the USA check out things like GoPhone from AT&T. In Denmark I picked up a sim card from the post office (don't remember who the provider was though) for 99 Danish Krone that lasted for 1 week and in Vietnam I got a sim for a month for the whole of $1USD!

So before you go check out the country and you'll probably find an easy way to pick up a sim card. We found this really useful in Vietnam as it meant we could look up an address rather than relying on shitty maps in guide books, saving an argument or two with the girlfriend :P.

Before you go for a local sim card make sure your phone is network unlocked. I ended up in Denmark with a network locked phone and my sim wasn't usable in it :(

Internet beats books

Like a studious traveller we picked up our copy of the Lonely Planet but by the end of the trip we were only use it for one purpose, the find out where not to go.

Now I don't want to rag on Lonely Planet too much but it's really hard for a print book to keep pace with the internet. Instead we turned to good ol' technology (since I had a local sim) to find out stuff to do. Now I want to talk about two sites that are invaluable if you're travelling.

Wikitravel

Url: http://wikitravel.org

Wikitravel is the wikipedia of travel websites. It's got lots of great tips on history of a place, what to see while you're there how to get in, out and around. The kind of stuff you can get out of a Lonely Planet guide book but it is able to be kept up to date (say around pricing of cabs). It can even give you those handy tips that you wont find out until you've hit them (such as Melbourne trams having coin-only ticket machines).

That said be careful of vandalism/ shameless self promotion on the site, you'll occasionally find companies promoting themselves on there. It's generally pretty easy to pick them though.

Trip Advisor

Url: http://www.tripadvisor.com/

Trip Advisor is a must when you're planning your trip and when you're away. The site is full of user generated content and allows for people to enter information about places they've visited, stayed, eaten at, etc and then vote against them.

Again this is something that kills Lonely Planet. Where Lonely Planet can only have a finite amount of places listed and gets out of date, a site based around generate content can reflect the actual mood of travellers to an area.

We used this to find recommendations for hotels, places to eat or just check out others opinions for places we got recommended by friends.

Have adapters

It goes without saying that when you're travelling having local power adapters is a valuable thing but what I found more valuable was carrying a multi-port adapter. Don't go crazy and take like a 10 port power board if you only have 2 devices but they can be handy (particularly if you're travelling for work as well as play).

Travel insurance

It goes without saying that you should have travel insurance with you but make sure that your policy will cover you for the devices you are carrying with you. Last thing you want is to lose your laptop and find out that you only had $500 of coverage!

I went with Travel Insurance Direct who have a reasonably well priced set of plans including yearly world-wide plans.

TL;DR

• Get a local sim
• Use WikiTravel and Trip Advisor
• Make sure your travel insurance will cover your gear

var todoViewModel = function() {
    this.items = new ko.observableArray(['Item 1', 'Item 2', 'Item 3']);
    this.selectedItem = new ko.observable('Item 1');
};

What I'm trying to point out here is that the viewModel is being defined in JavaScript and that the items within it are coded into your JavaScript.

While you can argue that this is demo code and it should only be treated as such something I've noticed is there isn't any other examples. I haven't seen any example where they are talking about getting the data initially from the server for their viewModel.

So how do you approach this? In this article I'm going to look at how to create a viewModel from the server using ASP.Net MVC.

Note: I'm talking about doing a viewModel as part of the initial page load since generally speaking you'll have been doing data layer interaction as part of the request. Building a viewModel using an AJAX request is a different story and I wont be covering.

From the server to the client

Let's get started with an example of our controller:

public class TaskController : Controller
{
    public ActionResult Index()
    {
        var vm = new TaskViewModel
                     {
                         Tasks = new[] { new Task("Write Blog Post"), new Task("Publish Blog Post") }
                     };

        return View(vm);
    }
}

I'm just going to have a reasonably simple ViewModel that just has a collection of tasks that I want to display as part of my KnockoutJS-built UI but the tasks are to be pulled in from my data layer (obviously this is demo code and it's hard coded so you'll have to use your imagination for that part :P).

For the view I'm just creating something that is very simple for the task list:

<form data-bind="submit:addTask">
    Add task: <input type="text" data-bind='value:taskToAdd, valueUpdate: "afterkeydown"' />
    <button type="submit" data-bind="enable: taskToAdd().length > 0">Add</button>
</form>

<p>Your values:</p>
<select multiple="multiple" height="5" data-bind="options:tasks"> </select>

<div>
    <button data-bind="click: removeSelected, enable: hasTasks">Remove</button>
    <button data-bind="click: sortTasks, enable: hasTasks">Sort</button>
</div>

Now we have a conundrum, how do I as part of my response create a KnockoutJS ViewModel that I can then use in my UI?

It's all about the serialization

When I was prototyping this for my current project I remembers that Shannon has mentioned that he'd done something similar himself and I've shamelessly taken his approach and am using it :P.

His approach was to use a serializer to create a JSON object from the model (there was some other stuff in the skype message he sent me but I'll confess to having not read that :P). For the serialization you can use the JavaScriptSerializer, the DataContractJsonSerializer or Json.NET. Personally I prefer Json.NET and it's what I'll be using in this demo.

So let's make a little HTML helper to do this for us:

public static class HtmlHelperExtensions
{
    public static IHtmlString KnockoutFrom<T>(this HtmlHelper<T> html, T obj)
    {
        var serializer = new JsonSerializer
                             {
                                 ContractResolver = new CamelCasePropertyNamesContractResolver()
                             };

        return new HtmlString(JObject.FromObject(obj, serializer).ToString());
    }
}

All we're doing here is creating an instance of the JsonSerializer from Json.NET and telling it to use the CamelCasePropertyNamesContractResolver. This is why I like Json.NET, it allows me to convert my .NET naming conventions into JavaScript conventions without a lot of effort. Lastly we just return the serialized object. Not really anything special happening in here.

Now in my View I can do this:

@Html.KnockoutFrom(Model)

Hmm but this isn't really helpful, we're just getting out JSON blob in our view, I still would have to do a bunch of work to actually make it usable and especially if I am doing this on a lot of pages it's a lot of code that I'd prefer not to do every time. So let's see if we can improve our extension method.

Setting up the viewModel

So what do we want from our improved version? Well I'd like the observables to be set up for me and I'd like it to avoid global variables.

To do this what I'm going to do is update my extension method to use the Knockout Mapping plugin. This plugin is really sweet as it allows me to map a JSON object into a KnockoutJS object and is great when you're working with AJAX data, you can easily pull down some data from the server and then use the plugin to extend it into your ViewModel.

In this case though I'm going to use it to map the JSON version of our server ViewModel into our KnockoutJS one:

    public static IHtmlString KnockoutFrom<T>(this HtmlHelper<T> html, T obj)
    {
        var serializer = new JsonSerializer
                             {
                                 ContractResolver = new CamelCasePropertyNamesContractResolver()
                             };

        var sb = new StringBuilder();
        sb.Append("(function() {");

        var json = JObject.FromObject(obj, serializer);

        sb.Append("var vm = ko.mapping.fromJS(" + json + ");");

        sb.Append("ko.applyBindings(vm);");

        sb.Append("})();");

        return new HtmlString(sb.ToString());
    }

The main updates here are:

• I'm using a StringBuilder to build up some JavaScript (normally I hate server-generated JavaScript but here it serves a good purpose)
• I'm creating an immediately-invoked function expression to prevent leakage
• I'm doing my binding straight away, hiding the need for that too

Excellent, this works, at least it works to an extent as we still have a few problems:

• What if I want to restrict where the binding happens?
• What about adding methods to my KnockoutJS viewModel?

Improving interactivity

While the above will work fine for simple scenarios it's not great if you have a complex UI that you want to work with, and realistically it's not likely you'll have a viewModel you don't want to extend with dependantObservables or anything, so let's do some refactoring.

I'm going to change the end of my extension method to look like this:

        sb.Append("var vm = ko.mapping.fromJS(" + json + ");");

        var type = obj.GetType();

        var ns = JavaScriptify(type.Namespace);
        sb.Append("namespace('" + ns + "');");
        sb.Append(ns + "." + JavaScriptify(type.Name) + " = vm;");

        sb.Append("})();");

        return new HtmlString(sb.ToString());

What I've done here is instead of doing the bindings I'm just going to create a global object which the viewModel will be assigned to (but I am namespacing it so it's a bit better). This object I can then interact with in my JavaScript and add methods/ properties/ etc to myself.

I'm also using a helper method to make the .NET namespace & type names friendlier for JavaScript:

    private static string JavaScriptify(string s)
    {
        return string.Join(".", s.Split('.').Select(x => x[0].ToString().ToLower() + x.Substring(1, x.Length - 1)));
    }

With this new extension method I can update my View to play around with the viewModel before binding:

<script>
    @Html.KnockoutFrom(Model)

    $(function() {
        var model = knockout.serverViewModels.models.taskViewModel;

        model.addTask = function() {};
        model.taskToAdd = new ko.observable('');
        model.removeSelected = function() {};
        model.hasTasks = function() {};
        model.sortTasks = function() {};

        ko.applyBindings(model);
    });
</script>

Conclusion

This wraps up my post on how to convert your server ViewModel into something that can be used in your KnockoutJS, allowing you to push all data down in the initial request rather than subsequent ones.

Thanks to Shannon for the initial idea, hopefully this little extension will make it even easier.

If you want to grab the code it is available here.

One final note, the Json.NET serializer does support the DataMember attributes, so you can also selectively include properties from your server ViewModel by attributing them too.

I wish Shannon, Alex, Matt, Niels and the rest of the team the best for the Umbraco 5 release.

In this part we're going to look at how you can use DOM interactions in your QUnit tests and still run them under Node.js.

Working in a DOM-less JavaScript environment

One thing that can trip people up when they first come to Node.js is they don't realise that JavaScript isn't tied to the browser. In reality JavaScript is just a language that happens to be used predominately in the browser, meaning that the window object isn't part of the JavaScript specification, it's just something that's part of the runtime.

So when you're running your code under Node.js you can't just document.getElementById or $('#foo'). Uh-oh, how are we going to run tests against the DOM, after all if you want to test a library like Knockout.Unobtrusive you kind of need to be able to do that!

Well don't fear, the smart people in the Node.js community have solved this problem with a nice little package, jsdom!

jsdom is essentially an implementation of the DOM in Node.js allowing you do create a document object, a window object and interact with it as though it was in a browser.

Note: There are limitations to jsdom, it doesn't do everything you'd want but it's a great way to do basic interactions such as we want to do in our tests.

Updating out test runner

In my previous post I showed you how to set up a basic test runner in your Cakefile. For our tests we're going to need to add some new stuff into our runtime (Node.js) and to do that we can use the dep option when we set up the runner:

test = 
  deps: ["./tests/test-env.js"]
  code: "./#{output}/#{file}.js",
  tests: "./tests/#{file}.tests.js"

What I'm going to do is create a test-env.js file which will be executed before the tests, allowing me to set up our pesudo-DOM.

Creating our test environment

I'm going to set up a few new variables:

var jsdom = require('jsdom'),
    fs = require('fs'),
    dom = fs.readFileSync("./tests/knockout.unobtrusive.tests.html").toString(),
    document = jsdom.jsdom(dom, null, { features: { QuerySelector: true } }),
    window = document.createWindow(),
    navigator = {
      userAgent: 'node-js'
    };

First thing I'm doing here is importing jsdom and fs. This will mean I can work with jsdom and the file system.

The next step is to pull in our test HTML page as a string. We've got some base HTML which we'll be interacting with during our tests. This is also the HTML we'd be running in our browser for our browser-based tests because keep in mind we want to share our tests between Node.js and the browser.

Now that we have our DOM as a string we'll create our document object from it. One other thing we are doing is specifying that we do want querySelector and querySelectorAll available, and this is done with the { features: { QuerySelector: true } } argument to jsdom.

Lastly we need to create a window object and a navigator object.

So when we have all these local variables we need to make sure that they'll be available everywhere:

global.window = window;
global.navigator = navigator;
global.document = window.document;

Unlike the browser Node.js's global object is called global, once we add our variables to that they'll be available in any of the other files we use in the runner.

Augmenting our tests

In the original set of tests that were in the Knockout.Unobtrusive project there was a heavy reliance on jQuery. Now admittedly there is a jQuery npm package but I couldn't get it running on Node.js 0.4.10 but it's not really important (I'd prefer not to rely on jQuery in my tests anyway).

So I'm going to do a check for jQuery (our Node.js tests wont have it):

var get, getAll, camalizer, data,
  dataMatcher = /data-(.+)/,
  dashAlphaMatcher = /-([a-z])/ig;

if(typeof $ !== 'undefined') {
  getAll = get = $;
} else {
  camalizer = function(x, letter) {
    return letter.toUpperCase();
  };

  data = function(el) {
    var attribute,
        attributes = el.attributes,
        data = {};

    for(var i = 0, il = attributes.length; i < il; i++) {
      attribute = attributes[i];
      if(dataMatcher.test(attribute.name)) {
        data[attribute.name.match(dataMatcher)[1].replace(dashAlphaMatcher, camalizer)] = attribute.value;
      }
    }

    return function(attr) {
      return data[attr];
    };
  };

  get = function(id) {
    if(id.indexOf('#') === 0) {
      id = id.substring(1, id.length);
    }

    var el = document.getElementById(id);

    if(!el) {
      el = document.querySelectorAll(id)[0];
    }

    el.data = data(el);
    return el;
  };

  getAll = function(selector) {
    var el,
        elements = document.querySelectorAll(selector);

    if(elements[0] && !elements[0].dataset) {
      for(var i=0, il=elements.length; i < il; i++) {
        el = elements[i];
        el.data = data(el);
      }
    }

    elements.each = function(fn) {
      var that = this;
      that.forEach(function(value, index) {
        fn.apply(that, [index, value]);
      });
    };

    return elements;
  };

Ok so the obvious first step is to check for jQuery and then we're deferring everything to that, but instead of just exposing $ I'm going to expose two methods, get and getAll. The former will be useful for getting a single element, the latter for multiple elements.

Next I'm creating two helper methods, the first being a camel case method (handy for working with data-* and the second simulating the .data API which you get from jQuery itself.

Note: The data method isn't exactly the same as using the $.data API, but I'm only replicating what I need at the current time.

In our simulated data API it will iterate through all the attributes and find any data-* ones (using a regex to look for them) then turning them into camel cased strings (like the spec, so data-foo-bar becomes fooBar). It's probably a bit more complicated than it needs to be but it works nicely as I want.

The only other interesting point of note is that the getAll method will also simulate the .each API from jQuery so that we can use those loops in our tests.

And there you go you're essentially done. Once you replace all your usages of jQuery in your tests for the get and getAll API then you'll be ready to roll!

Be careful with DOM manipulations

Something that I got tripped up with when porting the Knockout.Unobtrusive tests was that you don't want your DOM manipulations to persist across the tests.

If you've done work with QUnit you'll know that if you a DOM element with an id of qunit-fixture then it'll get rebuilt after every single test.

Well there's a problem, the Node.js implementation of QUnit isn't designed to work with the DOM so naturally this doesn't work. But it's an easy one to get around, QUnit exposes a method called reset that you can use to force a reset of the qunit-fixture element. Since the Node.js one doesn't worry about the DOM it doesn't have this method.

To implement this ourselves we'll create a module for our test that we can have a teardown method on the end of it:

QUnit.module('createBindings', {
  teardown: function() {
    if(!QUnit.reset) {
      //do the reset
    }
  }
});

Note: you need to either access module via QUnit.module or assign that to module yourself because module is a reserved word in Node.js.

For the reset method I've created a helper function back in the test-env.js file:

global.rebuildDom = function() {
  global.document = jsdom.jsdom(dom, null, { features: { QuerySelector: true } });
  global.window = global.document.createWindow();
};

This will rebuild both the document and window objects from the original DOM string. So we can update our module like so:

QUnit.module('createBindings', {
  teardown: function() {
    if(!QUnit.reset) {
      rebuildDom();
    }
  }
});

Conclusion

I'm aware that this has been a bit of a long and complicated post but hopefully it gives you some starting points for how you could approach doing online & offline JavaScript tests.

Here's the assets from the talk:

• Slides
• Recording

And here's a list of the tools which I looked at in my presentation:

• HTML5 Boilerplate
• Modernizr
• YepNope.js
• RaphaelJS
• AmplifyJS
• Knockout

Recently I wrote an article on a preparser I've written for Knockout. Interestingly enough at the same time Brendan Satrom had the same idea. I quite like the approach that Brendan has taken so I decided to have a poke around in the code and see if we could even merge the two projects.

The first thing I noticed when looking into the code was that it was written using CoffeeScript. The second thing I noticed was that the tests were all written using QUnit and were to be run in the browser. But there was a bit of a nuisance, the tests were against the compiles JavaScript, not the raw CoffeeScript (it was running against the generated file), to do any modifications, and test them, you have to copy the CoffeeScript to the online compiler, the back to the compiled file and then run the tests.

I'm sure you can see where the problems can come into this solution.

Well I've done a few small projects using CoffeeScript in the past and I've also included some tests into it so I decided at having a crack at getting this to work.

Getting your tools together

So to get started I'm using Windows and I'm going to be using Node.js to do the browser-less coding. Although I'm aware there is a version of Node.js for Windows I'm still using a self-compiled version with cygwin because npm works fine under cygwin but not with the Windows compiled version.

Additionally I'm going to be using a few npm packages:

• CoffeeScript
• QUnit
• Colors

Getting started by watching cake

If you've done much work with CoffeeScript you'll probably have come across the concept of a Cakefile, if you haven't, a Cakefile is a CoffeeScript version of a Rakefile (or MSBuild is a similar concept if you're coming from .NET just a lot more horrible), so I'm going to start off by using Cake to create a file system watcher.

The basic idea if I want to have a Cake task which will monitor for changes on the file system (specifically our CoffeeScript file) and when a change happens we'll compile it to JavaScript and run our tests.

First off I'll define some constants in our Cakefile:

fs = require 'fs'
path = require 'path'
CoffeeScript = require 'coffee-script'
file = 'knockout.unobtrusive'
source = 'coffee'
output = 'js'

Next it's time to setup our watch task:

task 'watch', 'Watch prod source files and build changes', ->
    msg = "Watching for changes in #{source}"
    console.log msg

    fs.watchFile "#{source}/#{file}.coffee", (curr, prev) ->
        if +curr.mtime isnt +prev.mtime
            console.log "Saw change in #{source}/#{file}.coffee"
            try
              invoke 'build'
              console.log 'build complete'
              invoke 'tests'
            catch e
              msg = 'Error with CoffeeScript build'
              console.log msg
              console.log e

We're using the standard watchFile method in Node and in the callback we'll ensure that the change times aren't equal (double-checking for false positives) and if there's a valid change we want to execute the following two tasks:

• build
• tests

Additionally we're wrapping this in a try/ catch so that if it fails we can provide a useful message but have the watcher keep running (say if you save while you're half-way through a change you wont get a major failure or anything).

Now let's have a look at the build task. This task will allow us to compile our coffee file into JavaScript. This is just going to be a standard Cake task as well so you can use it elsewhere if you want:

task 'build', "builds #{file}", ->
    console.log "building #{file} from coffeescript"
    code = fs.readFileSync "#{source}/#{file}.coffee", 'utf8'
    fs.writeFile "#{output}/#{file}.js", CoffeeScript.compile code

Since Node (well JavaScript) is a callback-based programming model generally speaking you'll be doing asynchronous operations, even with the file system. Node has provided some changes though to allow for synchronous programming. In this case I'm going to be using the synchronous read operation. The main reason for this is so that I don't have to pass around a callback which will then do the tests (this could get messy in the Cakefile).

Once the read operation is completed we pipe the output into the CofeeScript compiler API (which we can call from CoffeeScript/ JavaScript) and write the output of that into a JavaScript file.

When the build task is done the next step in our watcher is to call out to our test runner. As mentioned above I wanted to reuse the QUnit tests that already shipped in source of Knockout.Unobtrusive I plan to use the Node.js implementation of QUnit. It's rather simple and again we'll create a Cake task:

task 'tests', "run tests for #{file}", ->
    console.log 'Time for some tests! '
    runner = require 'qunit'
    sys = require 'sys'
    colors = require 'colors'
    test = 
      code: "./#{output}/#{file}.js",
      tests: "./tests/#{file}.tests.js"

    runner.options.summary = false

    report = (r) ->
      if r.errors
        msg = "Uh oh, there were errors"
        sys.puts msg.bold.red
      else
        msg = 'All test pass'
        sys.puts msg.green

    runner.run test, report

So a few things of note:

• We're using the runner which comes with QUnit for Node.js
• We're creating an object with our tests info which includes:
  • The file under test
  • The tests to execute
• I'm suppressing the summary (we execute the tests a lot so there's no need to see it)
• Lastly there's a callback for when the runner finishes which will either dump a message on success or failure (with a pretty colour!)

From the point of view of the Cakefile that's really all we have a need for, we've got our watcher up and running and we can just kick it off:

cake watch

Now whenever we edit our CoffeeScript file it'll go nicely.

Wrapping up

So this wraps up the first part of migrating our tests out of the browser to make a more automated series of JavaScript tests.

Next time we'll look at how to deal with some of the limitations of working in a DOM-less environment.

Here's the slides from the talk:

• Slides

And here's a list of the tools which I looked at in my presentation:

• HTML5 Boilerplate
• Modernizr
• YepNope.js
• RaphaelJS
• EaselJS
• AmplifyJS
• Knockout
• Backbone

Now I'm a pretty firm believe that if you are going to criticise something then you better make it constructive. Just saying "I don't like blah" isn't helpful to a) the author of blah or b) people wanting to learn more about blah, so I decided that I would follow up my criticism of KnockoutJS with a way I would go about fixing it.

Introducing the KnockoutJS preparser

To address my issue with KnockoutJS use of JavaScript in HTML I started looking at how I would go about it and the solution I kept coming back to was using the data-* attributes to describe in my HTML an intention. I decided that if I could do this in a good, convention approach then I should be able to translate it back into a KnockoutJS binding with minimal impact.

Out of this idea (and a challenge from my colleague Ducas) I set about taking my idea from my brain and putting in code.

Essentially what I came up with was taking this:

<span data-ko-text="firstName"></span>

And turning it into this:

<span data-bind="text: firstName"></span>

The idea is that you take data-ko-* as a prefix and use that to describe what could (well will...) become your binding syntax. Once getting this done I threw the code up on github and you can grab it from the KnockoutJS Pre-Parser project page.

What does it address

The ultimate goal is to be able to use the data-ko-* to describe out what you want in your HTML and the pre-parser will pick that up and Knockout-ify it for you. Ideally I'd like to support all common scenarios and so far I support:

• Basic bindings with text, css, value, etc
• Template name binding
• Pre-parser syntax within a template
• Template options (but not completely, you still have to embed JSON in your template options attribute. I added it about 3 hours ago so it's still being worked on :P)
• Event handler wire ups (if you want a complex handler like defining scope, etc you'll still have to embed JavaScript for now, I'm trying to work out how to get around that)

How to use it

If you really want to know how to use it I suggest you read the Readme or the tests but the quick and dirty is:

• Include it after you include KnockoutJS
• ???
• Profit

The pre-parser will actually hijack the ko.applyBinding method and perform the pre-parsing at that point, no custom stuff needs to be done to make it work :).

Where to get it

At the moment the only way to get it is via Github, depending on interest/ motivation I'll put it up on Nuget for people to get it via as well.

Conclusion

So this wraps up my introduction to the KnockoutJS Pre-parser and an approach I am taking to address one of the issues which I have with KnockoutJS. Feel free to give me any feedback you have on the library and the idea in general.

As the years went by we’d still see him sitting on his corner, but gradually he was looking less dishevelled. We still threw him money and chuckled to ourselves when he’d say “You’re not the only one who does this, everyone listens to me, I’m the most popular out there”. I mean who could really take the guy seriously? We'd put up with him for years now, we knew some of his quirks but we never really paid him the time of day unless we wanted to get lost in pain and suffering.

Then one day he was gone; there was no dishevelled old man sitting on our corner in tatty military gear. We felt sad, had our old friend passed away at the end of the first decade in the 21st century? Should we have paid more attention to him; maybe actually loved him? In reflection maybe he wasn't so crazy; maybe he did have a point; maybe there really was a time when his uniform sparkled and we'd have looked at him with awe.

But then we noticed someone looking at us. We turned to see a proud looking man in a shiny new military uniform and as we looked longer we noticed that it wasn’t some stranger looking at us but it was our old friend JavaScript. He was now sober, he’d been given a shave, a shiny new uniform and an array of new armaments. But what was most important was that he was looking for action.

He saw us looking back at him, he smiled and shouted “I’m ready for round two, let's get this browser war started”.

And with a quiet dignity he wandered off to help us fight the next generation of wars.