Coding for Fun and Profit

Introducing Stack Mechanics

andrewh@uglybugger.org — Thu, 05 Oct 2017 12:00:00 +1000

After living under a rock for far too long, I'm excited to introduce Stack Mechanics.

I've teamed up with two great friends of mine, Damian Maclennan and Nick Blumhardt to launch a series of deep-dive courses in distributed systems, microservices architectures and continuous delivery.

Don't panic - I'm not leaving ThoughtWorks :)

The longer version: Damian, Nick and I, alongside numerous other great engineers, have spent a lot of time solving the hard problems that organisations encounter when trying to move away from monolithic, legacy systems. We've seen lots of places like the potential that microservices offer with respect to organisational agility and independence, but generally they've been completely unprepared for the inherent complexities in such systems and how to manage the trade-offs. Usually we weren't lucky enough for it to be a green-fields project, and as a result we've all inherited our share of legacy, tighly-coupled systems with ugly, scary integrations, unknown black boxes, business logic hidden in integration layers (or ESBs, or stored procedures, or BPM layers), and with the many and various failure modes inherent in such ecosystems.

We arrived at a set of practices, patterns, techniques and tools that helped us solve these kinds of problems and we've had a lot of success at it. Unfortunately, we still see so many organisations making the same kinds of mistakes, through the best of intentions, that we first encountered many years ago. Many teams start off well but are unaware of the pitfalls; many have no idea how to even get started.

We've decided to get together and offer deep-dive training based on our real-world experiences. We're kicking off in November 2017 with a three-day, hands-on workshop on .NET architecture, microservices, distributed systems and devops. There will be both theory and practical sessions, covering topics like:

Design Patterns for maintainable software
Test Driven Development
DevOps practices such as monitoring and instrumentation
Continuous delivery
Configuration patterns
REST API implementation
Microservices
Asynchronous messaging
Scaling and caching
Legacy system integration and migration

Attendees will work with us and together to build an actual ecosystem to solve a small but real-world problem.

Book your ticket for the November 2017 (Brisbane) workshop now to lock in early-bird pricing.

We'll be scheduling other workshops in other cities (and probably some more in Brisbane), so register your interest in other dates and cities via stackmechanics.com, follow us on Twitter via @stack_mechanics and on Facebook as Stack Mechanics.

Command-line Add-BindingRedirect

andrewh@uglybugger.org — Tue, 04 Oct 2016 11:00:00 +1000

One of the things I try to do as part of a build pipeline is to have automatic package updates. My usual pattern is something along the lines of a CI build that runs on every commit and every night, plus a Canary build that updates all the packages to their latest respective versions.

The sequence looks something like this:

The Canary build:
1. pulls the lastest of the project from the master branch;
2. runs nuget.exe update or equivalent;
3. then compiles the code and runs the unit tests.

If everything passes, it does (roughly) this:

git checkout -b update-packages
git add -A .
git commit -m "Automatic package update"
git push -f origin update-packages


# Note: There's a bit more error-checking around non-merged branches and so on,
# but that's fundamentally it.

The CI build then:
1. picks up the changes in the update-packages branch;
2. compiles the code (yes, again), to make sure that we didn't miss anything in the previous commit);
3. runs the unit tests;
4. deploys the package to a CI environment;
5. runs the integration tests; and
6. if all is well, merges the update-packages branch back down to master.

For what it's worth, if a master build is green (and they pretty much all should go green if you're building your pull requests) then out the door it goes. You do trust your test suite, don't you? ;)

All of this can be done with stock TeamCity build steps with the exception of one thing: the call to nuget.exe update doesn't add binding redirects and there's no way to do that from the console. The Add-BindingRedirect PowerShell command is built into the NuGet extension to Visual Studio and there's no way to run it from the command line.

That's always been a bit of a nuisance and I've hand-rolled hacky solutions to this several times in the past so I've re-written a slightly nicer solution and open-sourced it. You can find the Add-BindingRedirect project on GitHub. Releases are downloadable from the Add-BindingRedirect releases page.

Pull requests are welcome :)

ConfigInjector 2.2 is out

andrewh@uglybugger.org — Tue, 06 Sep 2016 11:00:00 +1000

ConfigInjector 2.2 is out and available via the nuget.org feed.

This release is a small tweak to allow exclusion of settings keys via expressions as well as via simple strings. Thanks to Damian Maclennan for this one :).

To exclude settings keys via exact string matches, as per before:

ConfigurationConfigurator.RegisterConfigurationSettings()
                         .FromAssemblies(ThisAssembly)
                         .RegisterWithContainer(configSetting => builder.RegisterInstance(configSetting)
                                                                        .AsSelf()
                                                                        .SingleInstance())
                         .ExcludeSettingKeys("DontCareAboutThis", "DontCareAboutThat"))
                         .DoYourThing();

To exclude settings keys via expression matches:

ConfigurationConfigurator.RegisterConfigurationSettings()
                         .FromAssemblies(ThisAssembly)
                         .RegisterWithContainer(configSetting => builder.RegisterInstance(configSetting)
                                                                        .AsSelf()
                                                                        .SingleInstance())
                         .ExcludeSettingKeys(k => k.StartsWith("DontCare"))
                         .DoYourThing();

Introducing NotDeadYet

andrewh@uglybugger.org — Sat, 02 Apr 2016 10:00:00 +1000

NotDeadYet is a simple, lightweight library to allow you to quickly add a health-checking endpoint to your .NET application.

It has integrations for ASP.NET MVC, WebApi and Nancy.

Why do I want this?

To easily generate something like this: http://www.uglybugger.org/healthcheck.

When scaling out a web applications, one of the first pieces of kit encountered is a load balancer. When deploying a new version of application we generally pull one machine out of the load-balanced pool, upgrade it and then put it back into the pool before deploying to the next one.

NotDeadYet makes it easy to give load balancers a custom endpoint to do health checks. If we monitor just the index page of our application, it's quite likely that we'll put the instance back into the pool before it's properly warmed up. It would be a whole lot nicer if we had an easy way to get the load balancer to wait until, for instance:

We can connect to any databases we need.
Redis is available.
We've precompiled any Razor views we care about.
The CPU on the instance has stopped spiking.

NotDeadYet makes it easy to add a /healthcheck endpoint that will return a 503 until the instance is ready to go, and a 200 once all is well. This plays nicely with New Relic, Amazon's ELB, Pingdom and most other monitoring and load balancing tools.

Awesome! How do I get it?

Getting the package:

Install-Package NotDeadYet

In your code:

var healthChecker = new HealthCheckerBuilder()
    .WithHealthChecksFromAssemblies(ThisAssembly)
    .Build();

Doing a health check

var results = healthChecker.Check();
if (results.Status == HealthCheckStatus.Okay)
{
    // Hooray!
} else {
    // Boo!
}

Adding your own, custom health checks:

By default, NotDeadYet comes with a single ApplicationIsOnline health check which just confirms that the application pool is online. Adding your own (which is the point, after all) is trivial. Just add a class that implements the IHealthCheck interface and off you go.

public class NeverCouldGetTheHangOfThursdays : IHealthCheck
{
    public string Description
    {
        get { return "This app doesn't work on Thursdays."; }
    }

    public void Check()
    {
        // Example: just throw if it's a Thursday
        if (DateTimeOffset.Now.DayOfWeek == DayOfWeek.Thursday)
        {
            throw new HealthCheckFailedException("I never could get the hang of Thursdays.");
        }

        // ... otherwise we're fine.
    }

    public void Dispose()
    {
    }
}

Or a slightly more realistic example:

 public class CanConnectToSqlDatabase : IHealthCheck
{
    public string Description
    {
        get { return "Our SQL Server database is available and we can run a simple query on it."; }
    }

    public void Check()
    {
        // We really should be using ConfigInjector here ;)
        var connectionString = ConfigurationManager.ConnectionStrings["MyDatabaseConnectionString"].ConnectionString;

        // Do a really simple query to confirm that the server is up and we can hit our database            
        using (var connection = new SqlConnection(connectionString))
        {
            var command = new SqlCommand("SELECT 1", connection);
            command.ExecuteScalar();
        }
    }

    public void Dispose()
    {
    }
}

There's no need to add exception handling in your health check - if it throws, NotDeadYet will catch the exception, wrap it up nicely and report that the health check has failed.

Framework integration

Integrating with MVC

In your Package Manager Console:

Install-Package NotDeadYet.MVC4

Then, in your RouteConfig.cs:

var thisAssembly = typeof (MvcApplication).Assembly;
var notDeadYetAssembly = typeof (IHealthChecker).Assembly;

var healthChecker = new HealthCheckerBuilder()
    .WithHealthChecksFromAssemblies(thisAssembly, notDeadYetAssembly)
    .Build();

routes.RegisterHealthCheck(healthChecker);

Integrating with Nancy

In your Package Manager Console:

Install-Package NotDeadYet.Nancy

Then, in your bootstrapper:

var thisAssembly = typeof (Bootstrapper).Assembly;
var notDeadYetAssembly = typeof (IHealthChecker).Assembly;

var healthChecker = new HealthCheckerBuilder()
    .WithHealthChecksFromAssemblies(thisAssembly, notDeadYetAssembly)
    .Build();

container.Register(healthChecker);

FAQ

How do I query it?

Once you've hooked up your integration of choice (currently MVC or Nancy), just point your monitoring tool at /healthcheck.

That's it.

If you point a browser at it you'll observe a 200 response if all's well and a 503 if not. This plays nicely with load balancers (yes, including Amazon's Elastic Load Balancer) which, by default, expect a 200 response code from a monitoring endpoint before they'll add an instance to the pool.

Does this work with X load balancer?

If your load balancer can be configured to expect a 200 response from a monitoring endpoint, then yes :)

Can I change the monitoring endpoint?

Of course. In MVC land, it looks like this:

var healthChecker = new HealthCheckerBuilder()
    .WithHealthChecksFromAssemblies(typeof(MvcApplication).Assembly)
    .Build();

routes.RegisterHealthCheck(healthChecker, "/someCustomEndpoint");

and in Nancy land it looks like this:

HealthCheckNancyModule.EndpointName = "/someCustomEndpoint";

Does this with with my IoC container of choice?

NotDeadYet is designed to work both with and without an IoC container. There's a different configuration method on the HealthCheckerBuilder class called WithHealthChecks which takes a Func<IHealthCheck[]> parameter. This is designed so that you can wire it in to your container like so:

public class HealthCheckModule : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        base.Load(builder);

        builder.RegisterAssemblyTypes(ThisAssembly, typeof (IHealthCheck).Assembly)
            .Where(t => t.IsAssignableTo<IHealthCheck>())
            .As<IHealthCheck>()
            .InstancePerDependency();

        builder.Register(CreateHealthChecker)
            .As<IHealthChecker>()
            .SingleInstance();
    }

    private static IHealthChecker CreateHealthChecker(IComponentContext c)
    {
        var componentContext = c.Resolve<IComponentContext>();

        return new HealthCheckerBuilder()
            .WithHealthChecks(componentContext.Resolve<IHealthCheck[]>)
            .WithLogger((ex, message) => componentContext.Resolve<ILogger>().Error(ex, message))
            .Build();
    }
}

This example is for Autofac but you can easily see how to hook it up to your container of choice.

Why don't the health checks show stack traces when they fail?

For the same reason that we usually try to avoid showing a stack trace on an error page.

Can I log the stack traces to somewhere else, then?

You can wire in any logger you like. In this example below, we're using Serilog:

var serilogLogger = new LoggerConfiguration()
    .WriteTo.ColoredConsole()
    .WriteTo.Seq("http://localhost:5341")
    .CreateLogger();

return new HealthCheckerBuilder()
    .WithLogger((ex, message) => serilogLogger.Error(ex, message))
    .Build();

Do the health checks have a timeout?

They do. All the health checks are run in parallel and there is a five-second timeout on all of them.

You can configure the timeout like this:

var healthChecker = new HealthCheckerBuilder()
    .WithHealthChecksFromAssemblies(typeof(MvcApplication).Assembly)
    .WithTimeout(TimeSpan.FromSeconds(10))
    .Build();

What does the output from the endpoint look like?

It's JSON and looks something like this:

{
  "Status": "Okay",
  "Results": [
    {
      "Status": "Okay",
      "Name": "ApplicationIsRunning",
      "Description": "Checks whether the application is running. If this check can run then it should pass.",
      "ElapsedTime": "00:00:00.0000006"
    },
    {
      "Status": "Okay",
      "Name": "RssFeedsHealthCheck",
      "Description": "RSS feeds are available and have non-zero items.",
      "ElapsedTime": "00:00:00.0005336"
    }
  ],
  "Message": "All okay",
  "Timestamp": "2015-11-14T11:42:35.3040908+00:00",
  "NotDeadYet": "0.0.10.0"
}

Sensitive setting support in ConfigInjector

andrewh@uglybugger.org — Wed, 30 Mar 2016 10:00:00 +1000

ConfigInjector 2.1 has been released with support for sensitive settings.

This is a pretty simple feature: if you have a sensitive setting and want to be cautious about logging it or otherwise writing it to an insecure location, you can now flag it as IsSensitive and optionally override the SanitizedValue property.

If you just want to mark a setting as sensitive, just override the IsSensitive property to return true. This will allow you to make your own judgements in your own code as to how you should deal with that setting. You can, of course, still choose to log it - it's just an advisory property.

If you want to be a bit more serious, you can also override the SanitizedValue property to return a sanitized version of the value. By default, if you're logging settings to anywhere you should log the SanitizedValue property rather than just the Value one.

public class FooApiKey: ConfigurationSetting<string>
{
    public override bool IsSensitive => true;

    public override string SanitizedValue => "********";
}

It's worth noting that these properties do not change the behaviour of ConfigInjector; they simply allow us to be a bit more judicious when we're dealing with these settings.

Talk video: Back to basics: simple, elegant, beautiful code

andrewh@uglybugger.org — Fri, 25 Mar 2016 10:00:00 +1000

This is the video of the talk I gave at DDD Brisbane 2015.

As a consultant I see so many companies using the latest and greatest buzzwords, forking out staggering amounts of cash for hardware and tooling and generally throwing anything they can at the wall to see what sticks. The problem? Their teams still struggle to produce high-quality output and are often incurring unsustainable technical debt. Codebases are still impossible to navigate and there's always that underlying dread that one day soon someone is going to discover what a mess everything is.

How can this happen? It wasn't supposed to be this hard! Don't we all know all this stuff by now?

Let's take a look at some patterns and practices to reduce the cognitive load of navigating a codebase, maintaining existing features and adding new ones, and all while shipping high-quality products. Fast.

What's new in ConfigInjector 2.0

andrewh@uglybugger.org — Mon, 14 Dec 2015 10:00:00 +1000

ConfigInjector 2.0 has hit the NuGet feed.

What's new? A handful of things:

Support for overriding settings via environment variables (useful for regression suites on build servers).
Support for loading settings from existing objects.
Logging hooks to allow callers to record where settings were loaded from and what their values were.

There are some breaking changes with respect to namespaces so it's a major version bump but unless you're doing anything really custom it should still be a straight-forward upgrade.

Building Tweets from the Vault: Twitter OAuth

andrewh@uglybugger.org — Sun, 01 Feb 2015 21:00:00 +1000

In Building Tweets from the Vault: NancyFX tips and tricks we took a look at some of the refactoring-friendly approaches we can take when building a NancyFX application.

In this post we'll see a very simple example of how Tweets from the Vault uses Twitter and Tweetinvi, a nice-to-call .NET library for Twitter. Tweetinvi has a whole lot more features than authentication but authentication in general is the main focus of this post, so here goes.

I'll state in advance that this advice only briefly touches upon some really elementary application security. I'll write a bit more about that in subsequent posts but please don't treat this advice as anything other than a few brief pointers on the most basic of things. Do your homework. This stuff is important.

Requiring authentication in NancyFX

To begin with, our Nancy application has a base module class from which almost all others derive:

public abstract class AuthenticatedModule : RoutedModule
{
    protected AuthenticatedModule()
    {
        this.RequiresAuthentication();
    }
}

Our AuthenticatedModule just demands authentication and leaves everything else to its derived classes. It's worth noting that there's also a convention test (which we'll discuss in another post) that asserts that every single module in the app must explicitly derive from either AuthenticatedModule or UnauthenticatedModule so as to leave no room for "Oh, I forgot to set the security on that one."

In Tweets from the Vault, we're using NancyFX's StatelessAuthentication hook. We actually add an item to the request pipeline to check for 401 responses and send a redirect. In this way, our individual modules can just demand an authenticated user and return a 401 if not. It's up to the rest of our pipeline to figure out that we should probably present a kinder response.

In our bootstrapper:

protected override void ApplicationStartup(ILifetimeScope container, IPipelines pipelines)
{
    ConfigureLogging(container);
    using (Log.Logger.BeginTimedOperation("Application starting"))
    {
        // A bunch of irrelevant stuff elided here
        ConfigureAuth(pipelines);
    }
}

private static void ConfigureAuth(IPipelines pipelines)
{
    // Yes, we're using the container as a service locator here. We're resolving
    // a .SingleInstance component once in a bootstrapper so I'm okay with that.
    var authenticator = IoC.Container.Resolve<Authenticator>();

    StatelessAuthentication.Enable(pipelines,
                                   new StatelessAuthenticationConfiguration(authenticator.Authenticate));

    pipelines.AfterRequest.AddItemToEndOfPipeline(ctx =>
    {
        if (ctx.Response.StatusCode == HttpStatusCode.Unauthorized)
        {
            var response = new RedirectResponse(Route.For<SignIn>());
            ctx.Response = response;
        }
    });
}

Let's have a look at what this is doing. We can see that the item is being added to the end of the pipeline. That means that it will be executed after our module has done its thing and returned. If the module does an early exit and returns a 401, that will be observable in ctx.Response.StatusCode and we'll mess with it; otherwise we'll just pass the response straight through.

If we've observed a 401, we clobber the 401 response with a 302 and bounce the user back to the SignIn page using the Route.For expression that we looked at in Building Tweets from the Vault: NancyFX tips and tricks. It's noteworthy that the browser will never see a 401; just a 302.

What about Twitter and OAuth?

The assumption I'm making here is that you'll actually want to do something on behalf of a user using the Twitter API. That's pretty obvious as it's what Tweets from the Vault does, but I'm going to state up-front: if all you want is an identity via OAuth, this is a harder way to do it than you need. If you want API access, however, then read on.

The first thing you'll need is an application on Twitter. Go to apps.twitter.com to create one.

The next thing you'll need is an x.509 certificate. You'll be telling Twitter to pass keys to access other people's accounts via GET parameters, so don't be sending those around the place in plaintext. Incidentally, Twitter does support localhost as a valid redirect URL target, so you'll be fine for your own testing. Just make sure that you never present a sign-in/sign-up page other than via HTTPS, and likewise make sure your callback URL is HTTPS as well.

You'll also want the Tweetinvi package:

Install-Package Tweetinvi

Once we've hit our SignIn page, it creates a Twitter sign-in credentials bundle using Tweetinvi. This isn't exactly the code in Tweets from the Vault as there are a few abstractions here and there - I've inlined a few things - but it's pretty close. In our SignIn module:

// You'll want to stash these somehow as there's a single-use token in there
// that you'll need to decode the response.
var temporaryCredentials = TwitterCredentials.CreateCredentials(userAccessToken,
                                                                userAccessSecret,
                                                                _twitterConsumerKey,
                                                                _twitterConsumerSecret);
var authenticationUrl = CredentialsCreator.GetAuthorizationURLForCallback(temporaryCredentials, redirectUrl);

// Hack because the Tweetinvi library doesn't seem to support just authentication - it wants to make an
// authorize call all the time. This will happen anyway on the first time someone uses your app but
// forever after an authenticate call will just bounce straight back whereas an authorize call will
// continue to prompt.
authenticationUrl = authenticationUrl.Replace("oauth/authorize", "oauth/authenticate");

We redirect the user to that authenticationUrl, which will be somewhere on twitter.com, and Twitter will present them with an "Authorize this App" page.

Then, in our SignInCallback module:

var temporaryCredentials = /* fetch these from wherever you stashed them */
var userCredentials = CredentialsCreator.GetCredentialsFromCallbackURL(callbackUrl, temporaryCredentials);
var twitterUser = User.GetLoggedUser(userCredentials);

At this point, we have a valid Twitter user who's been verified for us by Twitter (thank you :) ) We'll also have a set of keys to allow us to make API calls as that user to the extent permitted by the privileges that your app was granted by the user.

Of course, if the user declines to authorise the Twitter app to use their account then you'll get back a different response. Be sure to handle that.

Now what?

Now we have a user who's just presented us with a valid set of callback tokens from Twitter via a redirect URL. That's nice, but we shouldn't be leaving those lying around. What we should be doing is generating our own authentication token of some sort and sending that back as a cookie. (Remember to give people some way to destroy that cookie once they leave a machine, too - you need a "Sign out" button[1]!)

A good way to do this is using a JSON Web Token or similar. There are a bunch of libraries (and opinions) out there on The One True Way™ to do it but the general principle is roughly the same as HTTP cookies: you shove a bunch of claims into a JSON object, sign it and give it to the browser. When it makes a request it can supply that via a cookie.

The JWT standard doesn't specify encryption - it's about sending information in plaintext but making it verifiable. That said, if you don't have to inter-operate with anyone else (i.e. you're just doing your own sign-on, not implementing SSO across a group of sites) then go ahead and encrypt it. It will help prevent other people stickybeaking into what you've bundled into there but still let you use someone else's library code rather than hand-rolling your own. It should go without saying[2] that if you're going to put any sensitive information into it then 1) have a careful think about whether you actually need to do that, and 2) make sure you're using a reputable encryption algorithm with a decent-length key.

Using this approach you can put pretty much anything into your token. As a general rule, I'd like to be able to load a page and only hit persistent storage for data specific to that page. Loading a user's name, profile picture URL or anything else that is part of the ambient experience goes into the encrypted token. This means that I can render most pages without hitting a dbo.Users or similar. The token doesn't need to be readable by anyone else but it does need to be relatively small as it's going to be transmitted by the browser on every request. Also think about what you'll do in the case of wanting to disable a user account - if you're not checking dbo.Users every request then how will you know to return a 403?

Be sensible. Don't create another ViewState. Don't treat it like session state[3].

So we're done?

Not quite. You'll probably want to create your own representation of a user once you have a confirmed Twitter identifier. I'd also use the Twitter 64-bit int as your foreign key, not the username, as that may well change.

It's worth bearing in mind that Twitter's OAuth solution does not provide users' email addresses so that's something you'll have to either request for yourself or live without. That's up to you :) Likewise, we're relying on Twitter's anti-spam measures to prevent malicious sign-ups. That's not unreasonable in the first instance but don't expect it to be perfect.

In the next post in this series, we'll take a look at some interesting domain event modelling as part of implementing payments using Stripe.

[1] And it should generate a POST request, not a GET one, but that's a story for another day.

[2] Hence, of course, needing to say it...

[3] An evil to be discussed some other time...

Building Tweets from the Vault: NancyFX tips and tricks

andrewh@uglybugger.org — Mon, 19 Jan 2015 21:00:00 +1000

In Building Tweets from the Vault: Azure, TeamCity and Octopus, I wrote about the hosting and infrastructure choices I made for Tweets from the Vault. This article will cover a bit more about the framework choices, notably NancyFX.

NancyFX

NancyFX may sound like a bit more of an esoteric choice, especially to the Microsoft-or-die crowd. I've been having a pretty fair amount of success with Nancy, however. I love the way that it just gets out of the road and provides a close-to-the-metal experience for the common case but makes it simple to extend behaviour.

By all means, it's not perfect - I'm not a huge fan of "dynamic, dynamic everywhere" - but it's way better than MVC for my needs. The upgrade path is a whole lot less troublesome, too - the best advice I've found for upgrading between major versions of MVC is to create a new project and copy the old content across.

Application structure

The equivalent of an MVC controller in NancyFX is the module. In a typical MVC controller, there are lots (usually far too many) methods (controller actions) that do different things. While this isn't strictly a feature of the framework, all the sample code tends to guide people down the path of having lots of methods on an average controller, with a correspondingly large number of dependencies.

In MVC, routing to controller actions is taken care of my convention, defaulting to the controller's type name and method name. For instance, the /Home/About path would (by default) map to the About() method on the HomeController class.

Nancy routes are wired up a little bit differently. Each module gets to register the routes that it can handle in its constructors, so if I were to want to emulate the above behaviour I'd do something like this:

public class HomeModule : NancyModule
{
    public HomeModule()
    {
        Get["/Home/Index"] = args => /* some implementation here */;
        Get["/Home/About"] = args => /* some implementation here */;
        Get["/Home/Contact"] = args => /* some implementation here */;
    }
}

Obviously, if we want the same Nancy module to handle more than one route then we just wire up additional routes in the module's constructor and we're good.

This is nice in a way but it's also a very easy way to cut yourself and I tend to not be a fan. Not only that, but it still leads us down the path of violating the Single Responsibility Principle in our module.

My preference is to have one action per module and to name and namespace each module according to its route. Thus my application's filesystem structure would look something like this:

app
    Home
        Index.cs
        About.cs
        Contact.cs

This makes it incredibly easy to navigate around the application and I never have to wonder about which controller/module/HTTP handler is serving a request for a particular path.

My About.cs file would therefore look something like this (for now):

public class About : NancyModule
{
    public About()
    {
        Get["/Home/About"] = args => /* some implementation here */;
    }
}

RoutedModule

One problem with the above approach is that it's not refactoring-friendly. If I were to change the name of the About class then I'd also need to edit the route registration's magic string. Magic strings are bad, mmmkay?

A simple approach for the common case (remembering that it's still easy to manually register additional routes) is to just derive the name of the route from the name and namespace of the module. (Hey, I didn't say that all of MVC was bad.)

public abstract class RoutedModule : NancyModule
{
    protected RoutedModule()
    {
        var route = Route.For(GetType());
        Get[route, true] = (args, ct) => HandleGet(args, ct);
        Post[route, true] = (args, ct) => HandlePost(args, ct);
    }

    protected virtual async Task<dynamic> HandleGet(dynamic args, CancellationToken ct)
    {
        return (dynamic) View[ViewName];
    }

    protected virtual Task<dynamic> HandlePost(dynamic args, CancellationToken ct)
    {
        throw new NotSupportedException();
    }

    protected virtual string ViewName
    {
        get { return this.ViewName(); }
    }
}

This now allows for our About.cs file to look like this:

public class About : RoutedModule
{
}

Routes

We're not quite there yet. I'm not a fan of magic strings and in the above example you can see a call to a static Route.For method. That method is where the useful behaviour is, and it looks like this:

public static class Route
{
    private static readonly string _baseNamespace = typeof (Index).Namespace;

    public static string For<TModule>() where TModule : RoutedModule
    {
        return For(typeof (TModule));
    }

    public static string For(Type moduleType)
    {
        var route = moduleType.FullName
                              .Replace(_baseNamespace, string.Empty)
                              .Replace(".", "/")
                              .Replace("//", "/")
                              .ToLowerInvariant();
        return route;
    }

    public static string ViewName(this RoutedModule module)
    {
        // Left as an exercise for the reader :)
    }
}

This allows us to have a completely refactor-friendly route to an individual action. There are a couple of similar routing efforts for MVC, notably in MVC.Contrib and MvcNavigationHelpers, but this lightweight approach doesn't require building and parsing of expression trees. (It's worth noting that it doesn't account for a full route value dictionary, either, but you can add that if you like.)

In our views, our URLs can now be generated like this:

<a class="navbar-brand" href="@(Route.For<Index>())">
    Tweets from the Vault
</a>

and in our modules, like this:

return new RedirectResponse(Route.For<Index>());

A quick ^R^R (Refactor, Rename, for all you ReSharper Luddites) of any of our modules and you can see that we haven't broken any of our links or redirects.

In the next post in this series, we'll take a quick look at authenticating with Twitter using OAuth.

Building Tweets from the Vault: yet another Bootstrap site?

andrewh@uglybugger.org — Sat, 17 Jan 2015 21:00:00 +1000

There's even a Tumblr for this.

The reality, however, is that Bootstrap is incredibly popular for good reason. It's responsive out-of-the-box, is delivered via other people's CDNs (for which I thank you :) ) and provides a relatively familiar UI paradigm.

In keeping with the "minimum viable product" theme, Bootstrap allows for very quick... err... bootstrapping of a pleasant, clean, simple web application with the minimum of fuss.

This is just a quick post to get the "Yet another Bootstrap site?" question out of the way. In the next post in this series I'll look in a bit more detail at NancyFX and some sneaky tricks to make it refactoring-friendly.

Building Tweets from the Vault: Azure, TeamCity and Octopus

andrewh@uglybugger.org — Thu, 15 Jan 2015 21:00:00 +1000

In the previous post in this series, Building Tweets from the Vault: Minimum Viable Product, I wrote about the absolute minimum feature set to get Tweets from the Vault off the ground.

In this post I'm going to write a bit more about some of the technology choices. So... what were they, and why?

Azure
TeamCity
Octopus Deploy
NancyFX
Bootstrap
Tweetinvi
Stripe + Stripe.NET
Serilog + Seq

Azure + TeamCity + Octopus Deploy

Obviously, I was going to need a hosting platform that was easy to get started with but that could scale if (when? ha!) my app hits the big-time. The app (and the article you're currently reading) is running on Azure VM-hosted IIS deployed to via Octopus Deploy.

To ship a feature

git add -A .
git commit -m "Added some feature"
git push

TeamCity will pick up that change, build it, run my tests, push a package to Octopus Deploy and then deploy that package to my test environment. A quick sanity-check there[1] and then it gets promoted to the live Azure environment. Using this tool suite a change can go from my MacBook to production via a sensible build + test + deploy pipeline in under two minutes.

For anything more complicated, I'll use a feature branch. TeamCity will automatically pick up and build refs/heads/* so all of my branches get the same treatment, all the way through to packaging in Octopus and deploying to a test site.

Hotfixes are treated in the same way as feature branches. If I have to revert to any particular revision, it's simple:

git checkout some-hash
git checkout -b hotfix-some-fix
git add -A .
git commit -m "Fixed some bug"
git push

That build will go straight to my test environment through the normal build + test + deploy pipeline and I can then tell Octopus to promote that hotfix package to production. No mess; no fuss.

In the next posts in this series, I'll write a bit about Bootstrap and NancyFX.

[1] I trust my test suite. You should trust yours, too - or write better ones ;)

Building Tweets from the Vault: Minimum Viable Product

andrewh@uglybugger.org — Tue, 13 Jan 2015 21:00:00 +1000

Tweets from the Vault is a service that will take a random[1] historical item from your RSS feeds and tweet a link to it.

When I started building the service, my goals were simple:

Solve my own problem
Get to a minimum viable product as quickly as possible

In this series of posts I'm going to look at each of those points in a little more detail.

Solve my own problem

There's a bunch of content in my blog that is still very relevant today. Lots of stuff on agile; lots on software principles and some valuable odds and ends that were starting to be lost to the archives.

I have IFTTT tweeting content as soon as it hits my RSS feeds, which is great, and obviously that leads to people's visiting those posts while that tweet appears in their timeline. Once that tweet falls off their timeline, though, all bets are off - nobody's likely to see that tweet ever again.

I wanted a solution that would periodically fish out a historical but relevant article and tweet a link to it and there wasn't a service that did that for me in a way that I liked. The closest I could find was an outdated Wordpress plugin (and I don't use Wordpress). Well... I blog mostly about software so why wouldn't I write one for myself? And, if I were to write one for myself, perhaps I could tweak it a bit and make it useful to other people. And thus, Tweets from the Vault was born.

Mimimum viable product

The minimum viable product for me was pretty simple:

Sign in using a Twitter account
Set up a small, recurring payment
Add and remove RSS feeds
On a schedule:
- Pick a random article from that set of RSS feeds
- Tweet it

And that's pretty much it.

In the next post in this series, I'll start looking at some of the technology choices for the app.

[1] For a given definition of "random".

Tweets from the Vault

andrewh@uglybugger.org — Wed, 07 Jan 2015 13:00:00 +1000

I built a thing!

I keep linking people to old blog posts of mine. Sometimes it's to solve a problem that was solved a long time ago; other times it's to make a point that the argument they're having isn't new. Either way, there's a whole bunch of valuable content locked up with nothing but an "Archives" link on a web site to show that it ever existed.

I went looking for a way to re-publish some of this content and couldn't find anything that did what I wanted. Thus, Tweets from the Vault was born.

Tweets from the Vault is a paid service that will pick a random item out of any set of RSS feeds you give it and tweet it from your account.

I've priced the lowest plan of one tweet per day at $1/month. That's less than the average person would spend on electricity to power their laptop for the month, and way less than you'd spend on even a half-decent coffee.

You'll be seeing it in my Twitter feed - and I hope I'll see it in yours :)

Am I interviewing you? Here's what I'm going to ask.

andrewh@uglybugger.org — Thu, 20 Mar 2014 10:45:00 +1000

If you have an interview scheduled with me, here's what I'm going to ask you.

Brisbane Azure User Group talk on Azure Service Bus Made Easy

andrewh@uglybugger.org — Tue, 18 Mar 2014 13:00:00 +1000

Damian Maclennan and I did a talk at the Brisbane Azure User Group on Azure Service Bus Made Easy. Here's the video :)

And here are Damian's slides from the night:

Support for long-running handlers in Nimbus

andrewh@uglybugger.org — Thu, 13 Mar 2014 11:30:00 +1000

Shiny, new feature: Nimbus now allows command/event/request handlers the option to run for extended time periods.

The Readify Firehose: An aggregated feed of a bunch of random Readifarians

andrewh@uglybugger.org — Wed, 12 Mar 2014 12:45:00 +1000

In true Readify style, we had the idea for a thing the other day and launched it the next morning. It's a small thing but still a thing.

Your domain model is too big for RAM

andrewh@uglybugger.org — Tue, 11 Mar 2014 13:00:00 +1000

Here's the video from my DDD Brisbane 2013 talk.

Stopping a Visual Studio build on first error

andrewh@uglybugger.org — Sat, 08 Mar 2014 20:00:00 +1000

I can't believe I've lived without this for so long.

ConfigInjector now supports static loading of settings

andrewh@uglybugger.org — Thu, 06 Mar 2014 19:45:00 +1000

ConfigInjector 1.1 has just been released.

Request and response with Nimbus

andrewh@uglybugger.org — Wed, 05 Mar 2014 19:45:00 +1000

In this article we're going to have a look at the request/response patterns available in Nimbus.

Eventing with Nimbus

andrewh@uglybugger.org — Thu, 27 Feb 2014 19:45:00 +1000

In this article we're going to have a look at some of the eventing patterns we have in Nimbus

Command handling with Nimbus

andrewh@uglybugger.org — Wed, 26 Feb 2014 20:00:00 +1000

We've had a quick introduction to Nimbus, so let's look at some messaging patterns in a bit more detail.

Handler interface changes in Nimbus 1.1

andrewh@uglybugger.org — Tue, 25 Feb 2014 13:45:00 +1000

We've tweaked the handler interfaces slightly for the 1.1 release of Nimbus.

Nimbus: What is it and why should I care?

andrewh@uglybugger.org — Sun, 23 Feb 2014 15:55:00 +1000

So Damian Maclennan and I built a thing. We're quite proud of it.

RSS as a primary source of truth

andrewh@uglybugger.org — Tue, 18 Feb 2014 18:02:00 +1000

I'm experimenting with making RSS my authoritative source for blog posts.

Introducing ConfigInjector

andrewh@uglybugger.org — Fri, 04 Oct 2013 13:35:00 +1000

So I've been using this pattern for a while and promising to blog it for almost as long.

Code is on GitHub; package is on NuGet. Here you go :)

Quick demonstration of continuous delivery

andrewh@uglybugger.org — Wed, 24 Jul 2013 08:51:00 +1000

I'm running Readify's Making Legacy Apps Awesome workshop right now.

The story so far: a fairy tale.

andrewh@uglybugger.org — Thu, 11 Apr 2013 21:23:00 +1000

I've just pushed the latest to the Making Legacy Apps Awesome workshop's Git repo.

Almost sold out: only 5 tickets left to Making Legacy Apps Awesome.

andrewh@uglybugger.org — Thu, 11 Apr 2013 14:23:00 +1000

We're almost sold out for the Making Legacy Apps Awesome workshop - there are only five tickets left.

Two-day workshop: Making Legacy Applications Awesome

andrewh@uglybugger.org — Tue, 12 Mar 2013 20:53:00 +1000

I'm running a two-day Readify workshop with Krzysztof Kozmic in Brisbane in April.

"We tried agile and it didn't work."

andrewh@uglybugger.org — Sun, 10 Mar 2013 16:43:00 +1000

So, you tried agile and it didn't work?

Video from my #dddbrisbane talk yesterday is now online

andrewh@uglybugger.org — Sun, 02 Dec 2012 16:41:00 +1000

DDD Brisbane 2012 yesterday was great fun. If you weren't there, you really missed out.

Vote for my @dddbrisbane talk: Inversion of Control from First Principles: Top Gear Style

andrewh@uglybugger.org — Sat, 03 Nov 2012 09:24:00 +1000

So I'm throwing my hat into the ring again to present at DDD Brisbane.

DDD Brisbane 2012 is on the 1st of December (a Saturday) and sessions are peer-voted so you get to see what you want to see.

Inversion of Control from First Principles: Top Gear Style

Tonight: James May writes "Hello, World!", Richard Hammond cleans up the mess and Clarkson does some shouting.

When most people first try to apply good OO design the wheels fall off as soon as their app starts to get complex. TDD, Mock<T>, IoC, WTF? What are these TLAs, why should you care and where's that owner's manual when you need it, anyway?

Most people are afraid of trying TDD and IoC because they don't really know what they're doing. In true Top Gear spirit we're not going to let ignorance prevent us from having a go, so sit back and watch us point a compiler in the general direction of France and open the throttle.

In this talk we're going to introduce inversion of control from first principles so that it's not just an abstract concept but a real, "I finally get it" tool in your toolbox. We'll start with "Hello, world!" and finish by writing a functioning IoC container - live, in real-time and without a seat-belt - and you can take the code home afterwards and test-drive it yourself.

In the right hands, IoC is a very sharp tool. Just don't let Clarkson drop it on his foot...

*Actual Top Gear presenters may not be present. But it will be awesome anyway.

You should submit something, too.

Don't forget to vote for me :)

In software, the iron triangle is a lie

andrewh@uglybugger.org — Fri, 31 Aug 2012 11:58:00 +1000

Everyone's heard the old adage, "Fast, good, cheap: pick two." It's called the Iron Triangle or Project Triangle.

I'm not going to make this argument about the world in general but in software this just doesn't work.

Why? Because software quality is paramount and poor-quality software is a complete showstopper as far as "fast" is concerned. You can't build any decent-sized piece of software on a poor foundation. If the code is good it will be easy and quick to change. If the code is poor it will be slow and painful to change.

Cheap will start out cheap and nasty by design but will morph into "expensive and nasty" very, very quickly, and then you'll be stuck with your expensive-yet-cheap-and-nasty legacy application and a team of developers quickly heading for the door before the midden hits the windmill.

In software your best options are "fast and good" (if you can find a crack team) or "slow and good" but neither of those is cheap.

What risks are you taking with your business?

andrewh@uglybugger.org — Tue, 21 Aug 2012 15:43:00 +1000

I had a potential client contact us a while ago. We hadn't dealt with them before and they didn't end up retaining us - largely, I think, because the message about how much trouble they were in might have been a bit too unpalatable to heed.

They're in a world of pain through a combination of bad luck and poor planning although, to be fair, it's more of the latter.

I can't help you with bad luck but I can prompt you to plan for it.

If you ship software, please ask yourself these questions:

If you had to ship a build tomorrow, could you?
How long would it take? Be honest - a day? A week? A month?
What dependencies do you have that could cause you to need to ship one?
- Third-party web services?
- iOS provisioning profiles?
- Expired x.509 certificates?
- Changes to certificate revocation lists?
- A critical security flaw?
- A leap-year bug?
- A leap-second bug?
- An operating system patch?
What monitoring do you have in place so that you're the first to know about any of these problems?
How much will it hurt if any of these fails?
How quickly do you need to be back up and running?
How many people are going to sue you if your software/platform/application falls down? And for how much?
How much do you stand to lose?

Back to that potential client: I honestly don't think their business is going to survive this particular flavour of disaster. In other words, I think the entire company is going to fold - and all because someone else moved their cheese and they didn't have a contingency plan. I wish them the best but I can't help them now - not at this late stage :(

I can't help them but I can remind you that the unexpected does happen, and will to you at some point. If your answers to any of the questions above frighten you... better me than fate :)

UPDATE: It brings me no happiness to report that they indeed did go bankrupt. Please don't let that happen to you for such a preventable reason.

Introducing YACLP: Yet Another Command-Line Parser

andrewh@uglybugger.org — Thu, 28 Jun 2012 17:47:00 +1000

It's on NuGet:

Install-Package YACLP

Why another one?

Because there were a bunch out there but all of them focused more on the parsing than on being quick and easy to call.

I want my command-line parser to not only parse arguments (which it does, but isn't very flexible about) but to automatically generate a usage message so that I don't have to.

Simple Usage

var options = DefaultParser.ParseOrExitWithUsageMessage<MyCommandLineParameters>(args);

Recommendations

I'd recommend using an IConfiguration or similar interface so that anything that depends on it doesn't need to know about command-line parameters.

Our main program would look like this:

public class Program
{
    private static void Main(string[] args)
    {
        var configuration = DefaultParser.ParseOrExitWithUsageMessage<CommandLineParameters>(args);

        new Greeter(configuration).SayHello();
    }
}

... and our Greeter like this:

public class Greeter
{
    private readonly IConfiguration _configuration;

    public Greeter(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    public void SayHello()
    {
        var message = string.IsNullOrEmpty(_configuration.Surname)
                          ? string.Format("Hi, {0}!", _configuration.FirstName)
                          : string.Format("Hello, Mr/Ms {0}", _configuration.Surname);

        Console.WriteLine(message);
    }
}

Note that our Greeter takes a dependency on an IConfiguration, which looks like this:

public interface IConfiguration
{
    string FirstName { get; set; }
    string Surname { get; set; }
}

... and that IConfiguration interface is implemented by our CommandLineParameters class:

public class CommandLineParameters : IConfiguration
{
    [ParameterDescription("The first name of the person using the program.")]
    public string FirstName { get; set; }

    [ParameterIsOptional]
    [ParameterDefault("Smith")]
    [ParameterDescription("The last name of the person using the program.")]
    public string Surname { get; set; }
}

The key point here is that our Greeter knows absolutely nothing about command-line parameters as everything is segregated using the IConfiguration interface.

The Principle of Least Privilege and other fallacies

andrewh@uglybugger.org — Thu, 07 Jun 2012 08:32:00 +1000

The Principle of Least Privilege states that a user (or service) should be given the absolute bare minimum privileges required in order to fulfil its function.

On the surface, how could this possibly be bad? If I have everything I need in order to do my job then by definition I have everything I need. Likewise, if my app has all the privileges it needs in order to function correctly then, again, by definition it can function correctly. Right?

For the purpose of this post I'm going to focus on application security. The parallels between that and user-level permissions are obvious, so I'll leave you to draw your own conclusions.

Where this all falls down is in defining "least privilege" in a sensible manner. How do we normally decide what privileges an application will require? When we decide on what the application will do, of course. And how do we decide what an application will do? We gather our requirements, of course. And when do we do this? We (of course, of course) gather all our requirements up-front, because that's how we roll.

To rephrase that:

We gather our requirements up-front.
We know these requirements to be inaccurate, incomplete or just plain wrong.
We set our security policies according to these requirements.
We have our policies "signed off" by some governance group or other.
We send our security requirements off to our sysadmins to implement in the form of AD security groups etc.

In other words:

We send our known-broken security requirements, based on our known-broken application requirements, off to be set in stone before we ever even ship our application. Now try telling me that it makes sense. Of course, we can change security policies after they've been written - and constitutional reform is theoretically possible, too, but how long did it take for women to get the vote?

If you're going to set strict security policies for your app then your development team should be responsible - and held accountable - for setting sensible policies and updating them quickly according to changing requirements. If you're going to wrap security policies in endless red tape then don't be surprised when 1) people ask for more privileges than they need just to avoid administrative pain; and 2) your project ends up with a sub-optimal result because of a bunch of stupid security work-arounds that decrease your overall security anyway.

TL;DR: Hire smart people. Trust them. Get out of their road. Hold them accountable.

If your DBA makes your schema changes, you're doing it wrong

andrewh@uglybugger.org — Wed, 06 Jun 2012 17:11:00 +1000

Does your DBA make schema changes for you? Here's a simple question: why?

One of the fundamental principles of an agile team is that of cross-functionality. Everyone should have at least a passing familiarity with everyone else's role, and there should be no one bottleneck within the team. There should also be minimal external dependencies that could prevent the team from delivering its stated sprint goal. If you have an external dependency then you're betting your own team's success on something that you don't control. Tell me how that makes sense, someone?

If you have a crack DBA within your team then that's one thing. I still don't think it's wise, but at least they're operating within your team. Even so, they're a bottleneck: if more than one person needs to make a schema change then all but the first person can hurry up and wait for the DBA to be available.

Is your DBA a developer? Does s/he have commit rights and responsibilities just like any other member of your team? Will s/he fix the build if it breaks? Does s/he decide on your class names? Or on your project/solution structure? Then why have them act as a gatekeeper for same? Your database is just a persistent record of your domain model, and should change accordingly. The schema should be updated by your own scripts, kept in your own source repository, and applied automatically by your application. It is part of your application.

Have infrastructure people do infrastructure and software developers write software. Database servers are infrastructure. Databases themselves are software components, not infrastructure.

This might sound like I'm against DBAs in principle. Not entirely, but I am against the kind who feel the need to insert themselves into application design decisions after the fact. To be fair, I'm also against developers who treat databases as an abstraction that they don't have to understand. My position is that both attitudes are irresponsible.

As a developer using a database you're responsible for knowing your tools and using them well, and that includes SQL. Likewise, as a DBA responsible for any component of a software development project you're responsible for knowing your tools, and that includes being able to code to the extent that you can write migrations if that's what your team needs.

Dear DBAs

andrewh@uglybugger.org — Sat, 02 Jun 2012 21:43:00 +1000

Applications need to own their own data.

The job of a DBA is a relatively thankless one. To make things easier for all parties, there needs to be a better understanding of where the responsibilities lie between DBAs and applications developers.

Applications should be perfectly capable of maintaining their own schemas and data. A database is just a big variable store, in the same way as are the stack and the heap. It's a clever store, yes, but still a variable store. The structure of that variable store and the access to it should be governed by the application itself. The application should be able to migrate its own schema up and down in the case of a rollout or rollback, and should need no human intervention for any part of its release.

Making changes to an app's database outside of the build pipeline (for instance, adding uniqueness constraints that may end up crashing the app), is putting the application into an inconsistent state with what's in development and what will be deployed when it next goes to production. This isn't going to help anybody.

A good software engineer will keep mechanical sympathy in mind when doing database work, and will ask for help when out of his/her depth. A good software engineer will know about nth normal forms, indices, sharding and will be as responsible with the database(s) owned by his/her app to the same degree that he/she would be responsible with the stack and heap.

A good DBA will ensure that each app sharing a database server behaves as a good citizen and doesn't unnecessarily or unfairly utilise resources. A good DBA will be able to help identify and debug poorly-performing queries, and contribute to changing them via the normal build/deployment pipeline.

We can play nicely in the sandpit together. Let's do that :)

This week's version control workflow

andrewh@uglybugger.org — Mon, 06 Feb 2012 16:36:00 +1000

So this is my current workflow in order to commit a single change from my development VM to the client's environment:

Push to github from My_VM
Pull from github to My_Laptop.
Push from My_Laptop to USB stick.
Transfer USB stick to Client_Workstation.
Pull from USB stick to Client_Workstation.
Push from Client_Workstation to NTFS share.
Pull from NTFS share to Client_VM

Software Project Rescue: A Fairy Tale (@QALMUG on Friday the 3rd)

andrewh@uglybugger.org — Mon, 30 Jan 2012 08:12:00 +1000

I'm presenting this on Friday morning at the QLD ALM User Group:

This is a tale of a naïve protagonist, misguided advisors, princesses[1], dragons[2] and knights[3] in shining armour[4].

Like most fairy tales, this story has an idyllic beginning, a middle, and a happy ending. Also like most fairy tales, the middle of this tale is a grim, dark, scary journey through the Woods of Requirements, blithely past the Ivory Tower of Architecture, into the Depths of Design Despair, under the Mountain of Technical Debt and finishing with an agile leap of faith over the Waterfall of Doom to reach the rainbow on the other side.[5]

This talk starts with a post-mortem of a 3.5-year, $2m project that went horribly wrong. We’ll look at where the project failed: the architectural choices; the management strategies; the personalities involved and some sample code. We’ll also look at the changes that were made to bring the project back on track, get its wildly spiralling technical debt under control, re-release a functioning version and refactor it to something testable – and all in 3.5 weeks.

Finally, we’ll discuss ways to identify the issues encountered in this project so that you can spot them before they bite, strategies for regaining control over a project that’s already in trouble, and effective methods for managing troublesome stakeholders.

[1] There may or may not be actual princesses.
[2] Or dragons.
[3] Or knights.
[4] Motorcycle helmets.
[5] Bingo!

I hope people think it's worth getting up early for :)

Wow. DISQUS rocks.

andrewh@uglybugger.org — Mon, 09 Jan 2012 21:27:00 +1000

Wow. I was introduced today by Andrew Tobin (@tobin) to DISQUS.

I'd tweeted about my replacement blog engine, and mentioned in my previous post that I hadn't yet implemented commenting. He suggested DISQUS, which is an online commenting service that I'd somehow never heard of. How had I not heard of this?!

From sign-up to comments working and tested, it's taken about half an hour of effort. If you need a public commenting solution, I genuinely can't think why you'd write your own any more. Nice work :)

New Blog Engine

andrewh@uglybugger.org — Sun, 08 Jan 2012 23:42:00 +1000

As per my New Beginnings post, I've tried a couple of times recently to move to FunnelWeb. I've failed. The reasons for my failure are simple:

I wanted to host everything on AppHarbor.
I wanted everything (including posts and comments) under source control.
I wanted a custom look and feel.

Everything that I wanted could be done using FunnelWeb, but when I started doing it I realised that I was re-working lots of code that I didn't actually expect to use in production and that, in a nutshell, a blog is just a web site and making a database-driven web site only really makes sense when there are frequently-changing data.

In addition, I kept finding myself firing up Visual Studio in order to write code snippets, then doing horrible things to them (I'm looking at you, Windows Live Writer) in order to make them appear. If I'd used FunnelWeb then I could use markdown (good) but then I'd have had to use Visual Studio to write any code snippets anyway.

I thought about it some more, and decided that I like writing code in Visual Studio, and can live with using it as my main blog text editor. So... blog posts are all now written using VS.

The other advantage of having a blog that's entirely under source control is that it's trivial to back up, restore and redeploy - and I can deploy it anywhere.

There's a bunch of stuff that doesn't work yet, but I can live with that. RSS isn't hooked up properly; hence feeding to Twitter (TwitterFeed via RSS) doesn't work. Comments aren't ready for prime-time either, but I expect they'll be done shortly. There's also no mobile browser detection, but that's in the pipeline.

Why did I want to change in the first place? Because there have been lots of blog posts that I've found myself wanting to write, but needing to include too many code snippets. Gists are fine but hurt to include via script tags; pre-formatted code blocks are ick and screenshots are kind of pointless when my goal is to allow people to easily copy/paste the code I'm posting. So... now that my roadblocks are mostly out of the way, expect to see more useful stuff here. You can hold me to that :)

The Book of Process

andrewh@uglybugger.org — Fri, 14 Oct 2011 04:50:00 +1000

Once upon a time, a company's youthful founder lucked upon a successful method of performing a task.
The task was profitable, and therefore it was good.
The founder wrote down that method and bestowed it unto his/her minions.
S/he said unto them, "This is The Process, and it is good."
The minions performed The Process until the end of days.
And they all lived happily ever after.

Not quite.

The adage, "If it ain't broke, don't fix it" has a corollary best expressed by Tess Ferrandez: If broken it is, fix it you should. Or at least, "If broken it is, don't inflict it on everyone just because it's all too hard to bother changing it."

I'm referring to internal corporate processes that serve no purpose other than demonstrating to some ISO 9000 certification minion that a documented process exists.

It seems as if every organisation, once it reaches a certain size, goes into the "create process and perish" stage. If it's a private enterprise it'll die a long, slow, horrible death of three-thousand triplicate signatures, but if it's a government enterprise then it's never going to die and we're all going to hate it.

Is hate too strong a word? I don't think so. Show me a single person who's dealt with a government department and left happy, and I'll show you someone who's on far too many psychedelics to be on the same planet as the rest of us. We hate government agencies because they're slow, bloated and inefficient. (We hate individual governments, too, but for different reasons. That's just not the point of this post.)

So, given the choice, why do organisations choose to have processes that make them slow, bloated and despised? Your guess is as good as mine, but I think it's to do with some misguided idea that they should be able to have any human follow the process and get the same result. Guess what, ladies and gentlemen: if you have a muppet-followable process then you'll end up hiring muppets to implement it - which is at best embarrassing while the process makes sense, but an utter disaster once the process becomes obsolete and everyone refuses to recognise it.

The Book of Process above should read something like this:

Once upon a time, a company's youthful founder lucked upon a successful method of performing a task.
The task was profitable, and therefore it was good.
The founder wrote down that method and bestowed it unto his/her minions.
S/he said unto them, "This is The Process, and it is good."
The minions performed The Process until the end of days.
The end of days arrived with a pitchfork-waving, torch-brandishing mob of angry citizens who burned the company's offices down around the minions.
The minions could not find ~~their backsides with both hands~~ the "In Case of Fire" process document quickly enough to escape.
And the citizens all rejoiced, and lived happily ever after.

Is process hurting your company? If so, it might be worth considering whether you actually need all your documented processes, or whether you can just set desired outcomes and performance metrics, and leave your smart people to figure things out for themselv...

... oh. I get it. Smart people. They've already left. Never mind.

Farewell, Steve

andrewh@uglybugger.org — Thu, 06 Oct 2011 11:46:00 +1000

There's nothing I can say that hasn't been said before by someone else, about someone else, for similar reasons. Nonetheless: today the world has lost a giant and we are all the poorer for it.

Steve Jobs changed the game so many times that people lost count. His visionary genius, his personal drive and his dedication to making absolute perfection commonplace have left an indelible legacy in which we all share.

His arrogance, his of-course-my-way-is-better approach and his unwillingness to compromise cultivated dislike amongst many, but then, nobody else gave the world the iPhone or the MacBook. Steve's arrogance was justified and, well, his way generally was better.

Steve's example should prompt all of us - in all industries - to treat elegant, beautiful design as a first-class consideration when creating something. If you build something people love, they will love you for it.

Farewell, Steve.

The Forgotten Convention-Based Test Harness

andrewh@uglybugger.org — Thu, 29 Sep 2011 09:31:00 +1000

I'm writing another MVC3 app. I'm in the same world of pain with respect to magic strings and anonymous classes. I don't like it here.

I'm sorry, but who on earth thought that this was a good idea for a method signature?

I mean, seriously? Six strings and a Dictionary<string, object>? And that's a sensible collection of arguments? Why not add in a RouteValueDictionary (another string/object dictionary) for good measure? Oh, never mind.

But surely there are smarter overloads than that, right? Well, yes - and honestly, you just couldn't make this stuff up:

Oh My Friendly Geranium, but who on earth thought of this - and why wasn't it knocked on the head by someone sensible? MVC team: I'm really sorry, but IMO you've completely missed the point of a strongly-typed language. I know you've taken a lot of flak for this over the past few years but, to be honest, it's kind-of deserved :(

So, what on earth do we do about it?

When we're in a world of magic-string pain, the first thing to do is generally start creating some conventions. The second thing, of course, is that we test those conventions using unit tests. Hold on a second, though: we're using a strongly-typed language and yet we're *writing unit tests *to test our conventions because we're using strings and anonymous classes? Why don't we have a tool that does this for us?

We want a convention-based test harness that:

Runs on every build.
Has a set of conventions that are clear and unambiguous.
Doesn't make us manually write tests.
Will auto-generate test cases for every new piece of code we write.
Is refactoring-friendly.
Is fast.
Will fail the build if something's wrong.

I think we've forgotten something important. Can anyone point to a tool that's all of the above, comes pre-installed with every version of Visual Studio and requires zero integration work, no NuGet packages and just works?

Anyone? Anyone? No? Here's one: csc.exe. Yep, that's right: use the compiler.

Call me old-school, but a compiler is all of the above. Consider this method:

Think about it: why don't I have to test that a and b are integers? Sure, I should be testing for edge cases here, but I don't have to type-check or null-check my inputs. Why not? Because the compiler is enforcing the convention that when I say "int", I mean a 32-bit integer and it simply won't let anyone pass in anything else.

I don't have to write a single unit test to enforce these conventions. The compiler will provide me with fast and reliable feedback - at compile time - if I've broken anything, which is far better than getting the feedback at run-time using unit tests (or worse, at run-time when a user hits an issue).

I think we as developers can afford to take a bit more time to write strongly-typed code, e.g. HtmlHelpers for controller actions. Try this one:

You can make your code infer the controller name, the action name, the area and all sorts of things without ever having a magic string. You could even add strongly-typed parameters to it (built using a fluent interface) so that it's effectively impossible to get it wrong without the compiler complaining.

So why don't more people use such a great convention-based test tool? I have no idea.

iPhone/MonoTouch Unit Testing with Team Foundation Server

andrewh@uglybugger.org — Mon, 26 Sep 2011 09:19:00 +1000

I know, I know: apples, oranges etc. It's not really, though - this is actually quite straight-forward. But first, some background.

I was recently involved in building another iPhone application for an enterprise customer. We had previously dealt with that customer and had a good working relationship and level of trust with them, but a huge part of that trust was the visibility that we provided as to what was going on. It's mostly off-topic for this post, but the way we did it involved using Team Foundation Server, giving the client's people accounts and making sure that the client's product owner could log in at any time and see how the project was going.

With the iPhone application, we wanted to do exactly the same. The problem was that we hadn't had that much experience using TFS to build iPhone apps. Most of our collective efforts had either been in Objective C using git (the stock-standard approach that Xcode pushes) or C#/MonoTouch using Mercurial (my experience). While both of those approaches are fine for personal and small projects, we really wanted all the other bonus points that TFS provides (continuous integration, work item tracking, reporting, web-based project portal etc.)

So how'd we do it? Well, the first thing to note is that we're not actually building the application bundle using TFS - yet. That still requires a MacBook, MonoDevelop and a bunch of other stuff. We'll probably get there soon using custom build tasks, rsync, ssh and a few other things, but we're not quite there yet.

What we do have is a working continuous integration and nightly build, plus running tests using MSTest. The nice thing is that it actually wasn't that hard.

Open the project in Visual Studio (not MonoDevelop). You'll probably need something like Chris Small's MonoTouch Project Converter to make this work happily.
Include monotouch.dll in your /lib directory and reference it from there rather than from the GAC. (It won't be in the GAC on your build server, and nor should it be.)
If you have other dependencies (e.g. System.Data), copy those from your MacBook into /lib as well and reference those from there.
Done :)

The key point to note when you're building your app is that you're not going to be able to easily test your ViewController classes using MSTest, so make them dumb. If there's business logic in there, extract it out into your domain model. If there's data access logic in there... well... you're doing it wrong anyway and you should definitely extract that out :)

You'll end up with an app that has a dumb(ish) UI shell wrapped around a bunch of well-tested business logic classes. The added bonus of doing it this way is that you can then re-use a lot of that code when you write your WP7 or Android version.

The outcome? The visibility we wanted from the reporting and work item tracking side of things, plus a CI build that didn't require witchcraft to configure, plus automated unit tests.

The only real down-side of this approach is that the build we're unit-testing isn't the build we're shipping - we still have to build that manually on a MacBook somewhere. It does, however, give us a good indication of our overall code quality and a reliable safety net for refactoring.

An iPhone Eye for the C# Guy at @dddbrisbane

andrewh@uglybugger.org — Sat, 27 Aug 2011 10:28:00 +1000

I just submitted this abstract for DDD Brisbane 2011. Don't forget to vote for me!

An iPhone Eye for the C# Guy

iPhone Development using MonoTouch

This session will cover the basics of developing an iPhone application using C#/MonoTouch, from how to create a "Hello, world!" app through to a look at a real-world, production codebase.

We'll cover the use of web services, threads, databases, generics (yes, you can use generics), reflection, inversion of control (yes, you can use IoC, too!) and general application architecture, and finish with a look at some tools, tips and tricks to make life as an iPhone developer much less painful.

This session will assume prior knowledge of threading, reflection, generics, inversion of control and why you'd want to use all of these, but don't let that scare you :)

Crash Logging in a MonoTouch App

andrewh@uglybugger.org — Fri, 26 Aug 2011 10:58:00 +1000

Customer: Your app crashed again.
Developer: How? What were you doing when it crashed? What happened?
Customer: I don't know. I was playing with it and it crashed.
Developer: Do you remember which page you were on?
Customer: ?
Developer: bangs head against wall
Sound familiar?
One of the first things I do when I start a project (or when I inherit one) is set up logging. You'd be amazed and depressed at how many projects just don't have any, or bolt it on as an after thought. Here's a hint: it's much easier to debug your app while you're developing it if you know where it's breaking. Ground-breaking, I know
There are a couple of things we want to do:
1. Log when the app crashes. Do it quickly and reliably, and don't rely on any app infrastructure (e.g. injected loggers) as it's already been torn down at this point. 2. Send the log message the next time the app starts. This will allow us to use all our nice web services etc. and means we can use just the one logging mechanism rather than having several different ones. MonoDevelop creates a fairly standard-looking Main.cs for us:

Let's change that to add a simple try/catch block:

The key here is line 11 - it makes it simple and obvious as to what it's doing.
So... the CrashLog class itself is a static class and doesn't do very much at all. The idea is that it's simple to call and doesn't rely on having any of the app's components still available.

We're using the My Documents special folder as that's where we're reliably allowed to write things to on the filesystem.
So now that we have our crash logger, let's hook things up so that we can log when the app starts again. In our AppDelegate class:

Yes, we're using our IoC container as a service locator. This isn't good, but we can't use constructor injection in our AppDelegate as it's the class responsible for creating our IoC container.
So how does the logger work? Well, that's up to you. You can choose to make a web-service call; you could spit it to another text file and periodically upload it; you could even send an email if you really wanted to.
My preference is using a web service as I tend to just hook it straight to the server-side logger (which usually uses log4net under the covers), but your mileage may vary.
The next time a customer tells you that your app crashed, though, you'll be able to respond with, "I know - and I've already fixed that bug."