Dylan Beattie's Blog

GiveCamp UK 2011 – A Retrospective

2011-10-26T00:52:00.001+01:00

I spent this last weekend at UCL’s Bloomsbury campus in London, with a hundred or so charitable geeks, at the first UK GiveCamp. I came away from it amazed – at the generosity of the volunteers and sponsors alike; at the sophistication of some of the solutions that were delivered within a single weekend, and at the reactions from the charities involved when we presented our projects on Sunday afternoon. It was a wonderful experience, and one I’d happily do again, but in the spirit of agile and continuous improvement, here’s my own personal retrospective on GiveCamp UK 2011.

What went well?

The project. I was lucky enough to be working with a team who were helping Scene and Heard, a mentoring project based in north London that arranges for children to work with volunteer theatre professionals to write and perform plays. Simma , their Head of Development, showed up on Friday with a great pitch, obvious enthusiasm, a wealth of knowledge about the domain, and a “wish list” of projects and ideas for us to investigate. Top of the list was a ticket booking system – and I was immensely heartened when our entire team of coders – you know, people who like to build stuff – unanimously agreed that the pragmatic solution would be to hook them up with EventBrite. By 8pm on Friday, we’d drawn up a backlog of requirements, made sure EventBrite would actually deliver what they needed, and a couple of the team were already working on the integration. This left the rest of us free to start looking at some of the more blue-sky ideas on Simma’s list… and somehow by Saturday afternoon we were building a full relational database, MVC web front-end, and a set of data migration and normalisation routines to import their data from a collection of Access and Excel sheets into SQL Server. It’s not quite live yet – we’re having some problems getting the Entity Framework config code to work with AppHarbor’s SQL Server databases – but by the end of the week we hope to have it online, hosted, and handed over to them, for immediate use, or as a solid platform for future development.
The venue. The power worked. There was ample desk space and seating. The wi-fi was pretty solid – for most of the weekend we were actually using their wi-fi network for all our database access and development, as well as simple web browsing & e-mail, and other than an occasional IP address change, it worked.
The catering. The food was excellent, there were ample supplies of drinks, snacks, awesome tea thanks to Teapigs, fresh fruit and enough Haribo to keep you wired and coding well into the small hours. Result.
The wrap-up. Without Paul & Phil imposing a strict code cut-off, we’d have happily coded until they threw us out – but in retrospect, having a hard deadline with plenty of notice really focused our efforts towards the end of the project.
The platforms. We used Github for hosting, Trello for organising our backlogs and a whole lot more besides; EventBrite solved the ticketing problem, and we’re in the process of deploying to AppHarbor. All fantastic, powerful tools – and all completely free.
The swag. 80Gb SSDs for all the attendees? Best. Swag. EVER. Plus a hugely generous range of licenses, software, books and ebooks. Check out the list of sponsors at http://www.givecamp.org.uk/sponsors – I’m hugely grateful to every single one of them for making this amazing event a reality.
The people. Everyone was helpful, cooperative, collaborative and enthusiastic, and I hope everyone learned as much working with each other as I did working with them.
The demos. I was really, really impressed at what the teams delivered. From discovery to implementation to – in some cases – deployment onto a live website, in under 48 hours… as Ben Hall put it, it “makes you wonder what we normally do all day…” I am also not at all biased by how genuinely delighted Simma and Jasmine looked when we showed them our work. Honest.

What could have gone better?

I thought the Friday start was too early. Lots of people – including several of the charities and one of the organisers – were held up in transit and missed the opening pitches and kick-off. The only downside of a venue as wonderfully accessible as UCL Bloomsbury is that at 5pm on a Friday it’s smack in the middle of the worst rush hour in the country. Just an idea – but I’d suggest next time a preliminary session, maybe earlier in the week, with the team leaders and the charity reps? A solid couple of hours capturing requirements and understanding the domain. We’d have been up to speed much quicker if one or two of us had had the chance to think things over ahead of time.
That’s actually the only criticism I have of the event itself. The rest are memos to myself and the team for next time:
- Bring an ergonomic keyboard! By Sunday afternoon my arms were starting to seize up… I’m not a laptop coder by choice, and having one of my trusty MS Natural 4000s would have made a *huge* difference. Plus a proper mouse. And a second monitor.
- Stick to what you know. This is not an event for trying out new technology – unless someone on your team knows it well, don’t go near it. We spent a good half-day investigating Visual Studio Lightswitch before concluding that we just didn’t have the expertise to tailor it to our requirements… it was gone 3pm on Saturday when we finally settled on ASP.NET MVC 3 with Entity Framework, and an hour later we were absolutely flying.
- Get a Skype chat or IRC channel set up ASAP. You’re going to be sharing lots of addresses, API keys, URLs – stuff that’s time-consuming to read out loud or write on paper.

In conclusion – wonderful event, I’m really pleased to have been part of it, and I hope the rest of our team will be at the next one because I’d love to work with them again some time. And huge thanks to Paul Stack, Rachel Hawley, Phil Winstanley, Kendal Miller and Dave Sussman, who all looked absolutely worn out by Sunday evening – it wouldn’t have happened without you!

Moleskine + Kindle = ... Moleskindle?

2011-09-17T16:10:00.001+01:00

Faced with the harrowing prospect of trying to fit the entire Song of Ice and Fire into hand luggage on my next holiday, I bought a Kindle. It's quite magic. It won't switch off, it doesn't light up, it looks utterly fake - like some sort of plastic prop tablet device where they've used a printed cardboard screen... and it's absolutely lovely. My paperback books tend to end up rather battered from being slung around in bags all the time, and I wanted a case to keep the Kindle safe from knocks and scratches. Rather than spend money on one of the ridiculously overpriced cases you can get for it, I wanted to try something a bit different. You remember reading spy books as a kid where people would hide stuff inside hollowed-out books?

I found an old Moleskine notebook that was just the right size for it, and started hacking away - a couple of happy hours playing with craft-knives and glue, and here it is: the Moleskindle

It's a bit fiddly - and messy - getting the cutouts just the right shape; I found wood glue worked just fine - and once it's dried, the compacted glued paper is quite easy to carve & trim using a sharp craft knife. I cut a notch in the right-hand side so I can reach the page-turn buttons whilst it's in the case, but you need to pop it out to reach the power button or recharge it. Still, I think it looks pretty cool, it'll stop the Kindle getting knocked and scratched, and you can fool people on the Tube into thinking you're reading something incredibly intellectual that's been hand-written in a Moleskine notebook when you're secretly reading rock star autobiographies.

Software Development - HORSE-style

2011-09-11T22:31:00.001+01:00

There's as many ways to lose at poker as there's ways to fail at delivering software, but one variation I have yet to experience is a game called HORSE. In HORSE, each hand follows a different set of rules - you'll play a hand of Hold'Em, a hand of Omaha, a hand of Razz, a hand of Stud, and a hand of Stud Hi-Lo; then you go back to the beginning and do it all over again, until my brother has all my chips.

Inspired by this, I've devised the following brilliant software methodology for all those teams who can't quite settle on a system that works for them. It's called WALKS, and you work in two-week sprints, using a different methodology for each sprint to ensure you get the maximum efficiency from all these wonderful processes and systems:

Weeks 1-2: Waterfall
You spend the first two weeks making bold, ambitious, big-design-up-front plans, and not actually writing any code or shipping any features.

Weeks 3-4: Agile
You spend the next two weeks trying desperately to get *something* built and releasable.

Weeks 5-6: Lean
Realizing that your "big design" is probably killing your attempts to be agile, you start hacking out unnecessary features and trying to pare the design back to something you might actually be able to build.

Weeks 7-8: Kanban
You still don't know what you're doing, so you decide to write everything on Post-It notes and stick them to a board, figuring that if you start pulling jobs off the queue, you might at least get *something* done.

Weeks 9-10: Scrum
You have two weeks of daily stand-up meetings, in a desperate attempt to try and get a handle on things. Finally, you sit down on Friday afternoon, have a two-hour timeboxed retrospective, and decide that what you really need is a full set of requirements and a definitive spec.

Then you take a weekend off, come in on Monday, and start at the top again.

If that sounds familiar, it's OK - you're not hopelessly lost, confused or unproductive; you're just taking a structured approach to being multi-disciplinary...

(This is a joke post. Please don't use WALKS to build software. Ever. The world has enough problems as it is...)

SkillsMatter Progressive.NET Tutorials 2011

2011-08-01T12:59:00.001+01:00

Some of you may have heard about - or attended - SkillsMatter's Progressive.NET Tutorials over the last couple of years. This is a three-day program of in-depth workshops covering the latest languages, frameworks and techniques in .NET development. The great thing about the workshop format is that it provides enough time to actually get some hands-on experience; instead of the rapid-fire 45-minute lectures you'll find at most conferences, you can actually try things out, ask questions, work through examples, and really get to grips with the techniques or frameworks you're exploring.

This year, Ian Cooper - whom you may know from the London .NET User Group - has put together a great programme of speakers and topics, and I'm really excited to say that, for the first time, I'll be there as a speaker instead of an attendee.

My workshop is called "Front-End Tips for Back-End Devs". I'll be looking at how many of the techniques we take for granted in back-end development - including DRY, abstractions, packaging and dependency management - can be applied to your page layouts, stylesheets and scripts. We'll cover semantic markup, we'll take a whirlwind tour of all the wonderful new tags introduced by HTML5, we'll look at CSS sprites and media queries, and we'll see how you can keep your web UI as clean, elegant and maintainable as the rest of your codebase.

There's also Damjan Vujnovic talking about TDD in JavaScript; I went along to one of Damjan's talks earlier this year and was really impressed, so I'm looking forward to the opportunity to go over his ideas in a little more depth. Jon Skeet will be talking about async/await programming on C# 5 (no word yet on whether Tony the Pony will be there). There's workshops on continuous deployment, on web development in F# using the WebSharper framework, on packaging and dependency management, on REST, on Nancy, on SimpleData - in fact, if you've heard the .NET community buzzing about anything in the last year or so, chances are there's a workshop here that will show you what they're all so excited about.

It's at the SkillsMatter eXchange in London, from 5-7th September 2011. The cost is £425 (yes, it's not free, but you do get three days of top-notch content without giving up your evenings or weekends) - and you can use the promo code PROGNET50 to get £50 off.

Sign up online here. To keep up with event news, follow the hashtag #prognet on Twitter, and it'd be great to see you there.

Just Do It: Command-Query Segregation, Nike-Style

2011-06-29T21:10:00.001+01:00

OK, CQRS is a hugely mis-applied and mis-understood architectural style. The insight I'm sharing here is based on my attending Udi Dahan's Advanced Distributed Systems Architecture course, then applying what I'd learned to a project we were building for 3-4 months, then having one of my colleagues go on the same course, and then have him come back and point out everything we'd done wrong. Let's assume you are already using CQRS. Maybe you're using it appropriately; maybe it's over-engineering; maybe it's completely misapplied. Doesn't matter, for what I have to say here; there are smarter folks than I who can tell you whether you should be doing it in the first place. No, I'm here to share a particular insight about CQRS with you.

Your commands should be like a psychotic drill sergeant screaming orders.

When you issue a command, your work is done. End of story. Maybe it happens immediately. Maybe there's a delay, and someone has to wait a few seconds. Maybe it doesn't go according to plan, and somebody else notices afterwards, and they call someone else, and it gets fixed up. Maybe it fails spectacularly. You don't care. (Hell, you're probably dead by now. Nobody would ever have won any wars if everyone threw an exception when they found the sarge face-down in a fox-hole with a bullet-hole in his head.)

You're the sarge. You are IN COMMAND. You order someone to destroy the ammo dump in North Camp, then you get on with your life. Tomorrow morning, you'll get a fresh intel report. Maybe it'll say that ammo dump in North Camp has been destroyed - maybe it won't. You'll review the fresh intelligence, decide what to next, issue a fresh batch of orders - and get on with your life. That's CQRS. Your data is stale, your word is LAW, and you have better things to do than hang around wondering if you maybe did the wrong thing. If you give a command, and it doesn't get obeyed, there's exactly two potential outcomes:

Nobody notices
Somebody notices

Nobody notices? Cool. No problem. Somebody notices? Well - that's where you hope it's one of your guys (i.e. alerts, logging, infrastructure, monitoring) instead of one of THEIR guys (i.e customers/clients) That's how you do CQRS. Get your intelligence - your queries. The freshest data you can get, but don't bust a gut if it's a little out of date. Give your orders. Trust your intelligence. Get on with your life. Rinse. Repeat.

How to install "Active Directory Users and Computers" on your Windows 7 Workstation

2011-06-11T15:12:00.001+01:00

This one stumped me until I hit upon the magical combination that makes it work.

Install Windows 7 Service Pack 1.
Download the Remote Server Administration Tools for Windows® 7 with SP1
Install them.
Reboot
Go into Start -> Control Panel -> Programs and Features, and go to "Turn Windows features on or off" - because the installer will download and install the admin tools, but won't actually switch them on. Helpful.

What makes it fun is that it just fails silently until you get it right. No error messages, no warnings... installer didn't even tell me I was missing SP1 - which I thought I already had.

Why Cloning Classic Games in Javascript Makes for a Great Hack Day

2011-06-09T00:12:00.001+01:00

Hack days should be about code. Anything that stops you writing code (or talking about / editing / refactoring code) is friction.

There's two things in any software project that tend to cause huge amounts of friction - certainly during the early stages. One is tooling. Installing compilers takes time. Installing libraries takes time. Configuration takes time. I remember a day at Snowcode last year when we spent literally five hours installing Ruby, various build tools, make files, modules, browser automation components, plug-ins... I don't think I wrote a single line of code that day. It was interesting, and educational, but a hack day should be about building stuff.

The other huge source of friction in software development? Debates. There's X ways of doing something, and you can't make any progress until you've chosen one. Should we allow HTML in the comments? How big do we make the gallery thumbnails? What colour should we paint the bike-shed? On a "real" project, these decisions are made by the product owner - but for something like a hack-day, if you one person in charge of all the design, decision-making and prioritisation, they'll rapidly become a rather frustrated bottleneck.

So - pick a problem that's clearly-defined and well-understood, and solve it using a language that everyone's already got, that doesn't need a compiler, linker or build environment, and that everyone can run just by opening a web browser.

In other words - clone a classic game in JavaScript.

Choose something everyone's played. Bomberman. Tetris. Lemmings. Asteroids. Pac-man. Have a copy of the game on hand - on a laptop, or an emulator, or bring a console, or whatever - so if anyone asks questions, you can just refer to your definitive reference implementation and get back to work. That'll eliminate debate without handing anyone the poisoned chalice of product ownership on a volunteer-based project.

And embrace the awesome lightweight expressiveness of JavaScript - the only language that you can write and run, out of the box, on every single computer since Windows 98. There's no compiler. There's no IDE, no build chain, no runtime or virtual machine or standard libraries to install. People can use vi, Visual Studio, TextMate, Notepad - whatever they like. (Personally, WebStorm is rocking my world right now - JavaScript intellisense and refactoring with built-in Git support... it's fantastic. Just remember that Ctrl-Y doesn't redo by default and you'll love it.)

OK, this weekend we were using NodeJS, so the build/run/test cycle involved restarting the node server (which is Ctrl-C, up, enter) - but the guys working on the renderer didn't even need a server. They built a client-side test harness (index.html), and their build and deployment cycle was Ctrl-S, Alt-Tab, F5.

That's low friction. That's walking in off the street, opening up your laptop, pulling the code, and starting to build stuff straight away. And I like that.

Notes from the KaboomJS! LonDev Hack-Day

2011-06-05T16:01:00.001+01:00

A couple of weeks back, I had this crazy idea to build Bomberman, in Javascript, in a day. I floated the idea on Twitter and got a pretty enthusiastic response, and so I set up the first LonDev hack day. 12 people, in a room, for one day, working together to build and ship a working game.

Did we do it? You'll have to read all about it on the new LonDev wiki, but personally, I'm really pleased at how it went, and really excited at the idea of organising the next one.

What's really encouraging is the mix of people and expertise who contributed. We had a couple of .NET coders, some Ruby/Rails guys, some JS web hackers who'd not done node/socket stuff before, and @palfrey who, as far as I can tell, spends his days switching between Erlang and PHP to stop himself getting bored.

A fun day. Some really solid code. Some really interesting lessons learned. And there's a couple of us hanging out in #kaboomjs on Freenode over the next few days to get it finished off and up and running.

Want to work for Spotlight?

2011-04-26T13:55:00.001+01:00

Spotlight are hiring! We're looking for someone to join our software team full-time, in a senior development position. An experienced scrum master, who knows how to work with business and technical people to make things happen. Somebody who understands how to create great software. From database optimization to SOLID principles to TDD to user experience and accessibility, you understand what makes software great - great to use, great to maintain, great to extend.

We’re at a turning point. Five years ago, we were an award-winning publishing company who maintained our own website. Five years from now, we’re going to be a software company who publish award-winning directories. It’s a great place to work, and it’s a really exciting time to be here. It’s full-time, permanent job, working at our office just off Leicester Square. We’re upstairs from the Prince Charles Cinema – London home of Sing-along-a-Sound-of-Music and The Room – and surrounded by excellent bars, restaurants and theatres.

If you understand 90% of the postings on my blog, you’re probably in the right ball-park in terms of technology – but if you want buzzwords, it’s C#, .NET, agile, scrum, MVC, Castle Windsor, NHibernate, NServiceBus, jQuery, IIS, SQL Server, NUnit, SOA, TeamCity, FinalBuilder, msdeploy, and various other bits that are occasionally referred to as the “alt net stack”.

Interested? Read the full job spec, and details of how to apply, at www.spotlight.com/jobs/developer.html

NO AGENCIES. Seriously. If we want to deal with agencies, we’ll call you. If you call me, I will put my phone handset in a drawer, close the drawer, and let you talk to my stationery while I wander off and make some coffee. If you’re lucky, it’ll only waste 90 seconds of your time. If you’re unlucky, your phone system still uses analogue-switched PSTN and you’ll find you can’t hang up. It’s hard to earn commission when you can’t use your phone, and you’d be surprised how long it takes to make a really good cup of coffee.

Slides and Notes from “So You Think You Know JavaScript”

2011-04-10T21:00:00.001+01:00

The slides, notes and references from my JavaScript talk are now online, at

http://www.dylanbeattie.net/javascript/

A huge thanks to everyone who came along, to Ian Cooper and the LDNUG User Group for organising, and to SkillsMatter for the venue, the projector, the publicity, the video and the ginger tea. A full video of the talk is also available on the SkillsMatter website – and you’ll be pleased to hear that their awesome new video processing rig means you can now see my grinning face AND read the code samples on the slides.

The NodeJS demo code is open source and is online at https://github.com/dylanbeattie/BomberJS – fork it, pull it, do whatever you like with it. No warranties as to whether it’s any good or not… but it’s there and it works.

A couple of people asked afterwards about running Node on Windows, as I was doing in the demos. I was using a compiled binary from http://node-js.prcn.co.cc/, which worked absolutely fine for little demo apps with 5-6 concurrent client connections. I've no idea how it scales, but the general consensus seems to be that you should stick to Linux / MacOS for hosting any significant Node applications.

Churn-down Charts

2011-04-10T17:05:00.001+01:00

Our team have just finished a sprint on a project that’s using loads of new technology – MSMQ, NServiceBus, WCF – that we’ve not worked with before, and it’s played havoc somewhat with our estimates of how long everything was going to take. We hit our deadline, but only thanks to the product owner shifting a group of features into the next sprint, and at the retrospective everyone agreed that the process worked just fine but we didn’t have any really good way of visualising it. We have a burn-down chart – actually, we have two, ‘cos there’s one drawn on the whiteboard and there’s one in FogBugz as well – and what we’ve been doing is at the end of every day, we’ll just mark the number of hours left. On days when we discover more problems than we ship features, this looks like we’re moving backwards… which is true in terms of monitoring progress and planning, but isn’t great for morale, and it doesn’t really explain what’s going on.

So, I’ve come up with this, as a way of tracking estimation accuracy and churn as well as straightforward progress. I’ve no idea if it’s original or not, I don’t know whether it has a name, but I’ve called mine a churn-down chart. I’ve annotated this example to show you what happens over the course of the project – click for a bigger version.

Basically – when the green line hits the red line, you’re done. The product owner controls the red line, by adding and removing features from the sprint. Yes, I know you’re not allowed to add stories to a sprint that’s in progress - I think the chart actually demonstrates why. The little blue tails were inspired by this fantastic visualisation of budget forecasts compared with reality (which I found via Chart Porn), and they track how many hours of features we actually delivered that day, as opposed to how many hours we have left at the end of the day. This clearly shows the difference between days when we were productive but discovered lots of unplanned work, as opposed to days when we were just stuck.

I like the way it empowers the product owner to actually work with the team to hit the deadline – you’re not working towards a fixed target, you’re both dealing with shifting requirements as you find bugs and have ideas, and you can see at a glance whether you’re on target or not, and if not, why not.

Development Methodologies

2011-04-04T09:48:00.001+01:00

By now, everyone’s seen test-driven development (TDD), behaviour-driven development (BDD) and domain-driven design (DDD) – but there’s some other, so far development paradigms that haven’t got nearly the attention that they deserve.

Attention Deficit Disorder Driven Design (ADDDD)

Most commonly seen in open-source projects. You begin by implementing a core feature. After a couple of days, when either it gets boring or you’ve coded yourself into a corner and can’t work out how to get out, you pick a new feature and start implementing that one instead. Advantages of this approach are that you can tick “in development” on the feature comparison charts when evaluating your solution against the alternatives. Disadvantages are that it leads to crappy software that doesn’t work.

Attention Deficit Hyperactive Disorder Driven Development (ADHDDD)

Just like ADDDD, but features are only ever added in brief caffeine-fuelled bursts of manic coding, usually around 4am, accompanied by dozens of tweets, blog posts and Facebook status updates.

Developer Developer Developer Driven Development (DDDDD)

Projects are started twice a year, normally the week immediately after the popular DDD community event at Microsoft headquarters, and generally involves building something really ground-breaking like a wiki or a blog engine, just to “get your head around” all the amazing new stuff you’ve seen at DDD. You’ll generally lose interest about two days after you put the code up on Github as a “pre-alpha technology demo”, and then six months later you’ll do the whole thing all over again.

Advanced Dungeons & Dragons Driven Development (AD&DDD)

Everyone sits around drinking Red Bull, eating Doritos, boasting about their accomplishments and pretending to be some sort of tenth-level software architect when deep down they’re still not quite sure what a pointer is. A “dungeon master” (also known as a “project manager”) occasionally rolls some dice or reads a Gartner report, and then tells them that their project has died. Then they do it all over again, once every couple of months, sometimes continuing well into middle age.

Acronym Driven Development (ADD)

The HORSE of development methodologies; you consistently blame the failure of your last project on the fact that you picked the wrong methodology, and resolve to try something different on your next project. The conventional approach is to go test-driven, then behaviour-driven, then domain-driven, then extreme, then back to domain-driven. It’s a very educational way of wasting your employer’s time and money, and there’s normally someone in a back room happily coding away who doesn’t have the faintest idea what the rest of you are doing, but is probably shipping enough features to keep your company afloat.

I’ll be talking about JavaScript at Skills Matter on April 5th

2011-03-22T17:41:00.001Z

On April 5th I’ll be giving a talk on JavaScript at SkillsMatter here in London. It’s being organized by the London .NET User Group, but it’s not a .NET talk. Instead, it’ll cover a range of topics related to JavaScript’s history, the current state of the language, and the future of this widely-used and widely-misunderstood language.

JavaScript is fifteen years old, and the principles that influenced JavaScript’s architecture go back to the very dawn of computer science. It’s a powerful, expressive, dynamic language, that’s now being used to deliver some of the biggest and most popular software application in the world – and yet a whole generation of developers still thinks of JavaScript as being a scripting language that’s barely good enough to make pop-up windows appear on a web page.

There’s a lot of very cool stuff going on in the JavaScript world right now. With HTML5’s offline storage, you can use JavaScript to write client applications that you can install to your phone or your laptop and run them even when you’re offline. With CommonJS, there’s finally a unified effort to create a standard runtime library for JavaScript so we can write JS programs that support file systems, networking, loadable modules and unit tests. With NodeJS, there’s a fast, scalable framework for writing HTTP servers as collections of discrete JavaScript components. With frameworks like KnockoutJS, there’s declarative support for building rich web user interfaces in JavaScript - and it’s still pretty good at doing pop-up windows as well.

So you think you know Javascript? Sign up, come along and find out. I’ll bet you a pint there’s something in there you’ve never seen before.

True Names, and Other Dangers: What Dr. Seuss Can Teach Us About SoA

2011-02-26T23:23:00.000Z

Did I ever tell you that Mrs. McCave, Had twenty-three sons, and she named them all Dave?

Well, she did. And that wasn't a smart thing to do; You see, when she wants one, and calls out "Yoo-Hoo!
Come into the house, Dave!" she doesn't get one; All twenty-three Daves of hers come on the run!

- from “Too Many Daves” by Dr. Seuss

They say there’s only two hard problems in software – cache invalidation, naming things, and off-by-one errors. Cache invalidation’s hard because it’s difficult to clarify requirements. Off-by-one errors are hard because the joke wouldn’t work without them. But naming things? How hard can that be?

If you’ve ever worked in IT support, you’ll have had calls saying “the system is down”. Sometimes, a more enlightened caller will helpfully tell you that it’s ‘the network’ or ‘the database’ that’s broken. I once started at a job where everybody referred to everything as “sequel.” There had been a big database migration a few years earlier, resulting in a new website and new desktop software, and the whole process had been referred to as “upgrading to SQL Server”. Everyone kept hearing the techies talk about “upgrading to sequel”, and so when they got something new on their desktops, they conclude “Ah – this must be that sequel thing that everyone’s been talking about!”. Two days later, they call you up and say there’s a problem with ‘sequel’ – and in this context, ‘sequel’ could refer to just about anything. The name was overloaded to the point of uselessness.

“Ah yes - but it’s
services
all the way down!”

What’s scary is that this happens all over the industry. People talk about “Software as a Service”, when what they’re actually dealing with is an XML web service, that’s connecting to a WCF service hosted in a Windows service to provide a business service. Like dear old Mrs. McCave, we’re finding out that names are great if they’re unique, but when different things start laying claim to the same names, you’re going to end up cross-eyed.

So, as my team start teasing apart our proverbial big ball of mud, I’m trying out a new naming policy for the new components and modules we’re building. Pick a word that sounds nice and doesn’t mean anything within our business.

The last three projects I worked on were called Rosemary, Tarragon and Kamogelo. No namespace conflicts, no semantic overloading and no clashing with reserved keywords. Rosemary’s almost like an employee – it has an event log, and a mailbox, and a sufficiently clear sense of identity that people seem to get it. When they say there’s a problem with Rosemary, they’re right; it doesn’t take 15 minutes to work out what they mean, and that’s a good thing. It also encourages clear separation of concerns, and facilitates good discussion thereof – lots of “does this feature belong in Rosemary or Tarragon?” instead of just adding another class to the legacy codebase.

So it’s goodbye, “customer e-mail service” and “accounts system” and “web shop”, and hello to Sundance, Monolith and Aquarius. And before too long, Moonface and PuttPutt and Shadrack, in recognition of dear old Mrs. McCave.

Making HttpContext.Current Available Within a WCF Service

2011-02-24T14:28:00.001Z

I needed to add a quick’n’dirty WCF service to an ASP.NET MVC web application, so I could call a handful of methods from a different application.

The MVC app in question is using Windsor, NHibernate and the repository pattern, so we’ve got a fairly standard pattern where we spin up a ManagedWebSessionContext in the Application_BeginRequest handler (in global.asax.cs) and then flush and close the session in Application_EndRequest(). I used the Windsor WCF facility to inject a bunch of dependencies into a little WCF service, but I was finding that SessionFactory.GetCurrentSession() was always returning null – because when you’re using the ManagedWebSessionContext, your NHibernate session is bound to your HttpContext.Current, and by default you don’t have one of these inside a WCF service.

However - if you can live with tight coupling between your WCF service and IIS hosting, there’s a couple of little config things you’ll need to do to get this working. What doesn’t help is that until you’ve got all this just right, you’ll get a really helpful “Failed to Execute URL” error from IIS that’ll tell you absolutely nothing about what’s wrong.

First, make sure WCF HTTP activation is installed on your server – in Windows 2008, it’s under Server Manager –> Features:

Next, make sure you’ve registered the WCF service model with IIS, by running:

C:\WINDOWS\Microsoft.NET\Framework\v3.0\Windows Communication Foundation\> ServiceModelReg.exe –i

Next, make sure your web service is running in ASP.NET compatibility mode. First, check you’ve got this:

<system.serviceModel>
<serviceHostingEnvironment aspNetCompatibilityEnabled="true" />
</system.serviceModel>

in your web.config file, and then decorate your service implementation with the AspNetCompatibilityRequirements attribute:

[AspNetCompatibilityRequirements(RequirementsMode= AspNetCompatibilityRequirementsMode.Required)]
public class WcfMagicService : IMagicService {
. . .
}

The last thing I had to do was necessitated by WCF not supporting multiple host headers; I had to hard-wire the WCF endpoint to listen on a specific hostname. In this case, this involved tweaking the serviceHostingEnvironment section of web.config, which now looks like this:

<serviceHostingEnvironment aspNetCompatibilityEnabled="true">
    <baseAddressPrefixFilters>
        <add prefix=http://services.mydomain.com” />
    </baseAddressPrefixFilters>
</serviceHostingEnvironment>

And then adding another attribute to the service implementation class:

[ServiceBehavior(AddressFilterMode=AddressFilterMode.Any)]
[AspNetCompatibilityRequirements(RequirementsMode= AspNetCompatibilityRequirementsMode.Required)]
public class WcfMagicService : IMagicService {
}

Once that’s done, you’ll have an instantiated HttpContext.Current inside your service methods, so your code – and useful things like NHibernate’s ManagedWebSessionContext – will behave just as they do in normal MVC controllers or WebForms code-behind.

Check Out smtp4dev if You Build Mail-Enabled Software on Windows

2011-02-21T23:46:00.001Z

One of my cow-orkers pointed me at a great little utility a while back called smtp4dev. It’s an SMTP server that listens on your local machine, and instead of relaying e-mail, it’ll capture them and store them in a queue so you can review and open them.

It’s brilliant – simple and elegant and incredibly easy to use. Just configure your application (website, debugger, logging framework – whatever it is you’re building) to send mail on localhost:25, fire up smtp4dev, and watch the messages pile up. I’ve been building SMTP appenders for log4net this evening, and it’s been really, really useful.

Binaries and source are at smtp4dev.codeplex.com – well worth a look if you ever write software that sends e-mail.

How to host Git in the same Apache server that comes with CollabNet Subversion

2011-02-07T23:28:00.001Z

CollabNet Subversion Edge is a great Subversion distro that includes Apache 2.2 and the ViewVC web-based repo browser, and makes it really, really easy to get up and running with Subversion and WebDAV. I’m setting up a project server to host something we’re working on, and it’s been generally decided that whilst Subversion is all very well for keeping Word documents in, we’d quite like something a touch more… distributed for the actual source code repo. And when James Gregory mentioned on Twitter that git would mean “no more tree conflicts ”, I may have actually started salivating… ahem.

Anyway, yes. Apparently Git is quite good.

Jeremy Skinner has some fantastic notes on how to get git up and running with Apache 2.2. on a Windows server – I followed these pretty much to the letter to get my first incarnation up and running, but had to comment out a bunch of the Collabnet/Subversion settings in the Apache config files to get the Git server running properly. A bit of tinkering, though, and I’m pretty much there. What makes this interesting is that CollabNet includes a web-based admin console, which makes configuring the built-in modules very straightforward, but it does mean several of the config files have this rather ominous warning at the top:

#
# DO NOT EDIT THIS FILE IT WILL BE REGENERATED AUTOMATICALLY BY COLLABNET SUBVERSION
#

so – any changes we make in there will be just peachy until someone touches the web interface, at which point BOOM! they’ll spontaneously stop working. So whatever we’re going to do, we need to do it without touching any of those files. I wasn’t sure at first whether this would be possible, but it seems to be up and running now and hanging together quite nicely.

Fire up the Apache httpd.conf file in your favourite editor – by default it’ll be in C:\Program Files\Subversion\data\conf\ – and add the following lines at the end:

# Configure Apache to listen for named virtual hosts on port 80
NameVirtualHost *:80

# Include the configuration file for our git http hosting
Include "C:\Program Files\Subversion\data\conf\git_httpd.conf"

Now create a new document called – yep - C:\Program Files\Subversion\data\conf\git_httpd.conf – and make it look something like this:

# HTTP settings for using Apache with MSysGit on Windows
# Based on Jeremy Skinner's notes at http://www.jeremyskinner.co.uk/2010/07/31/hosting-a-git-server-under-apache-on-windows/

<VirtualHost *:80>

    # Set this to the root folder containing your Git repositories.
    SetEnv GIT_PROJECT_ROOT D:/Git/

    # Set this to export all projects by default (by default,
    # git will only publish those repositories that contain a
    # file named “git-daemon-export-ok”
    SetEnv GIT_HTTP_EXPORT_ALL

    # Route specific URLS matching this regular expression to the git http server.
    ScriptAliasMatch "(?x)^/git/(.*/(HEAD | info/refs | \
        objects/(info/[^/]+ | [0-9a-f]{2}/[0-9a-f]{38} | pack/pack-[0-9a-f]{40}\.(pack|idx)) | \
        git-(upload|receive)-pack))$" \
        "C:/Program Files (x86)/git/libexec/git-core/git-http-backend.exe/$1"

    # The canonical DNS hostname that you want to use for your git server
    ServerName my_git_server

    # Any other DNS aliases that point to your git server
    ServerAlias my_git_server my_git_server.mydomain.com my_git_server.my_intranet.local

    # The root folder for non-GIT-hosted documents (e.g. phpgit or some other Web front end)
    DocumentRoot "D:\gitserver\htdocs\"
    <Location />
        # This section is duplicated from the Collabnet SVN LDAP authentication
        AuthType Basic
        AuthName "Spotlight GIT Repository"
        AuthBasicProvider csvn-file-users ldap-users
        Require valid-user
    </Location>
</VirtualHost>

Check your configuration by running httpd.exe from the command line, like so:

C:\Program Files\Subversion\bin>httpd.exe -f "c:\program files\Subversion\data\conf\httpd.conf" –t
Syntax OK

and if all looks good, go into services.msc and restart the CollabNetSubversionServer service (which is actually Apache)

Finally, I followed Jeremy’s instructions to get GitPhp running, but then replaced it with a different project – also called GitPhp – from http://www.xiphux.com/programming/gitphp/, which provides a full repository browser, revision history, etc.

All I had to do to get GitPHP running was to copy the gitphp.conf.php.example file to gitphp.conf.php, and then tweak the following settings:

/* The root folder of my Git repositories */
$gitphp_conf['projectroot'] = 'D:\git';

/* On 64-bit Windows, C:\Program Files (x86) ends up as C:\Progra~2\ so these need to be configured manually */
$gitphp_conf['gitbin'] = "C:\Progra~2\Git\bin\git.exe";
$gitphp_conf['diffbin'] = "C:\Progra~2\Git\bin\diff.exe";

Job done. I now have:

http://my_svn_server/ returning regular HTML pages hosted by Apache
http://my_svn_server/svn/ giving me a WebDAV browser view on my raw Subversion repositories
http://my_svn_server/viewvc/ giving me ViewCV web-browsable revision historys and diffs on my Subversion repos
http://my_git_server/ giving me GitPHP browsing of my Git repositories
http://my_git_server/git/ exposing URLs for GIT cloning, fetching and pushing
http://my_iis_server/ is still running IIS 7 with .NET 4 and MVC 3

and that’s all on one box, with two IP addresses, with the svn and git servers sharing an instance of Apache on one address, the IIS server running on the other, and DNS records pointing svn and git at the first address and IIS at the second.

Running IIS and Apache on the same Windows 2008 R2 Server

2011-02-07T19:18:00.001Z

I’m trying to get a Composite C1 site and the Apache WebDAV front-end to Subversion running on the same Windows 2008 R2 server, and doing so requires a bit of trickery with IP address bindings and such, and I thought I’d share it – partly ‘cos it’s useful, and partly because I’m bound to have to do this again in three months time and there’s no way I’ll remember how I did it. First off, make sure your box has (at least) two IP addresses – I’ve bound mine to 192.168.0.13 and 192.168.0.14

To get IIS to listen on ONLY 192.168.0.13, you’ll need to run the netsh.exe utility.

C:\Users\dylan.beattie>netsh

netsh>http add iplisten ipaddress=192.168.0.13

IP address successfully added

netsh>http show iplisten

IP addresses present in the IP listen list:
-------------------------------------------

192.168.0.13

netsh>exit

(note that netsh.exe is a Windows 2008 utility – if you’re running Windows 2003 or earlier, look up the docs on using httpcfg.exe to achieve the same thing)

If you now fire up a web browser and go to http://192.168.0.13/, you should get the default IIS7 “Welcome” screen, and http://192.168.0.14/ shouldn’t return anything at all. Now to get Apache listening on 192.168.0.14. Find your httpd.conf file – if you’ve just installed CollabNet Subversion (like I have) it’ll be in the \data\conf folder of wherever you put your SVN install.

You’ll need to find the Listen directive in httpd.conf, and modify it to say:

Listen 192.168.0.14:80

That’s all. Next time – to get Git running on the same Apache installation… until then, happy hacking.

“Choose Life” For DBAs

2011-01-17T23:11:00.001Z

I’m really really sorry. Someone tagged #sqlmoviequotes on Twitter and I got carried away…

SELECT TOP(1) * FROM LIFE.

SELECT TOP(1) * FROM JOB.

SELECT TOP(1) * FROM CAREER.

SELECT TOP(1) * FROM FAMILY.

SELECT TOP(1) * FROM television ORDER BY SIZE DESC

SELECT * FROM washing_machine CROSS JOIN car CROSS JOIN compact_disc_player CROSS JOIN electrical_tin_opener

SELECT * FROM health, cholestorol, dental_insurance
    WHERE health.status = 'good' and cholestorol.level < 5

SELECT * FROM mortgage WHERE interest_rate = 'fixed'

SELECT TOP(1) * FROM home WHERE TYPE = 'starter'

SELECT * FROM person
    INNER JOIN friendship ON person.id = friendship.person_id and friendship.friend_id = 'ME'

SELECT * FROM leisurewear
    INNER JOIN luggage ON leisurewear.color_scheme = luggage.color_scheme

SELECT TOP(3) * FROM lounge_furniture WHERE payment_plan = 'hire purchase'
    AND range_id IN (SELECT range_id FROM fabric_option GROUP BY range_id HAVING COUNT(*) > @RANGE_SIZE)

SELECT 'diy', CURRENT_USER FROM activity WHERE DATEPART(dw, activity_date) = 1 AND DATEPART(hh, activity_date) < 12

DECLARE @your_mouth INT
DECLARE junk_food CURSOR FOR SELECT * FROM FOOD WHERE TYPE = 'junk'
WHILE @@CURRENT_SHOW IN (SELECT * FROM SHOW WHERE keyword IN ('mind_numbing', 'spirit-crushing')) BEGIN
    FETCH NEXT FROM junk_food INTO @your_mouth
END

BEGIN

    sp_start_job 'brat'
    sp_start_job 'brat'
    sp_start_job 'brat'

    KILL @@SPID
END

SELECT * FROM events WHERE EventDate > GETDATE()

SELECT TOP(1) * FROM LIFE

Mapping a Drive Letter to a Subversion Repository with CollabNet, WebDrive and WebDAV

2011-01-10T19:26:00.001Z

This is quite neat. As part of a business-wide agile initiative, I’m looking into solutions for storing and collaborating on documents – something that gives *me* the history and auditing capabilities of something like Subversion, but gives the rest of the team something clean and easy that fits well with current working practises.

So… mapped drive letters. Everyone knows about drive letters – “just stick it on the R: drive” is nice and easy, and as long as everyone’s R: drive points to the same place and the fileserver behind it’s getting backed up, you’re sorted. Unless you want to revert a document that’s been corrupted, or accidentally deleted, or you just want to get back an earlier revision because you realize you’ve done something dumb. Then you need to mess around with tapes and stuff, and that’s just no fun at all.

Plus, of course, we have a wiki, which isn’t much fun to edit because the constant round-tripping from WYSIWYG->markup->HTML->WYSIWYG tends to clobber newlines and formatting, but it *is* a great place to keep stuff because it doesn’t get lost.

So, requirements for document storage:

As easy to use as a drive letter.
Security. Windows / LDAP authentication to control who can read and who can write.
Revision history – just a record of who made changes, and when.
Ability to revert changes to an earlier revision
HTTP accessible so you can read stuff with just a web browser.

There’s two things you can do with a list of requirements like that… speak to vendors, or hack something together yourself. You wanna speak to vendors, you go ahead; I shan’t stop you. Still here? Good. Let’s hack.

1. Install Subversion on the server.

For this, I’m using the CollabNet Subversion Edge stack – a single installer combining Subversion, Apache and the ViewVC web front-end. It’s very, very neat, and (having done this the hard way) much easier than setting up mod_dav_svn yourself. Once it’s up, use the web interface (linked from the Start menu on your server) to set up a new repo – call it doc_repo – and then verify that if you browse to http://myserver/viewvc/doc_repo/ you get the ViewVC web front-end view of your new, empty repository.

If you’re after Windows/LDAP authentication, that’s also configurable from the Subversion Edge web interface – and CollabNet has detailed notes on how this works.

2. Install WebDrive on your workstation.

This is a commercial package that’ll map a Windows drive letter to a WebDAV share. This is supposedly something that Windows is capable of doing natively, but I have never, ever got this to work, not even once. I would be glad to hear recommendations for free / open-source alternatives for this, since it’s currently the only bit of this set-up that costs money. There’s apparently also a netdrive.exe floating around but licensing for NetDrive seems to be a little confused.

3. Hack the Subversion config file.

On the server, open up C:\Program Files\Subversion\data\conf\svn_viewvc_httpd.conf. We’re not going to edit this file – you’ll need to find the bit that looks like:

<Location /svn/>
   DAV svn
   SVNParentPath "C:\Program Files\Subversion\data\repositories"
   SVNReposName "CollabNet Subversion Repository"
AuthzSVNAccessFile "C:\Program Files\Subversion\data/conf/svn_access_file"
SVNListParentPath On
Allow from all
AuthType Basic
AuthName "CollabNet Subversion Repository"
AuthBasicProvider csvn-file-users ldap-users
Require valid-user
</Location>

and copy it. Then open up C:\Program Files\Subversion\data\conf\httpd.conf – which is the regular Apache configuration file – and paste the copied section right at the end, and make the following changes:

# Change Location to be the URL path of your WebDAV repo – I’ve used webdrive here
<Location /webdrive/>
    DAV svn
    SVNParentPath "C:\Program Files\Subversion\data\repositories"
    SVNReposName "Subversion WebDAV"
    AuthzSVNAccessFile "C:\Program Files\Subversion\data/conf/svn_access_file"
    SVNListParentPath On
    Allow from all
    AuthType Basic
    AuthName "Document Repository"
    AuthBasicProvider csvn-file-users ldap-users
    Require valid-user
# Add the two lines below
    ModMimeUsePathInfo on
    SVNAutoversioning on
</Location>

Now restart the web server (using ApacheMonitor.exe from C:\Program Files\Subversion\bin\ on your server) and check that you can see http://myserver/webdrive/ in a normal Web browser – the screen should say “Collection of Repositories” with your doc_repo repository listed underneath.

4. Connect WebDrive to your new WebDEV-Enabled Repository

Nearly there. Finally, fire up WebDrive on your workstation and create a new connection. Enter the Site Address/URL as http://myserver/webdrive/doc_repo/ – note that you must put the repo name in the URL otherwise WebDrive will complain with an error something like:

Unable to connect to server, error information below

Error: Socket receive failure (4507)
Operation: Connecting to server
Winsock Error: WSAECONNRESET (10054)

Correct settings will look something like this:

Hit Connect, and Windows explorer will fire up a new N: drive window pointing at your repo.

5. Witness the Awesome Power of Autoversioning!

Create an empty folder, create a text file in it, then browse to http://myserver/viewvc/doc_repo/ and you should see your new folder, and file, along with the Subversion history recording who created the file and when:

I like that a lot.

Christmas, Crisis and the Cloud

2010-12-29T00:20:00.001Z

This Christmas, I volunteered at Crisis, the UK charity that provides food, shelter and help for homeless people and rough sleepers over Christmas. It’s the first time I’d done anything like this, and – amongst stints of washing-up, manning doors, looking after showers, mopping and playing Articulate – I spent quite a lot of time helping with the suite of PCs that were provided for our guests to use.

It was a profound and enlightening experience. I use Gmail and Google Docs because they’re convenient – I regularly use 3-4 different PCs and online services are just a convenient way of keeping things accessible and in sync. It never really occurred to me that for someone who doesn’t have a computer of their own, or any regular access to one via a university or workplace, something as simple as a webmail account can be the difference between staying in touch and disappearing completely. People with no regular income, no phone and no fixed address were using Yahoo Mail to keep in touch with family and friends, apply for jobs and look for accommodation. It was quite amazing. I don’t know whether the folks at Yahoo, Gmail, Hotmail et al realize just what a profound difference this makes – but if any of them are reading, well done. You rock, and I never realized quite how much until this week.

Another interesting observation – Microsoft’s market saturation is so complete that even people who store their worldly possessions in a plastic shopping-bag insist on Microsoft Word. Their CVs are in Word format, they know the Word menus and commands – many of them are remarkably accomplished Word users. This is sad, because Word means Windows, and – even with the awesome job the people at the Aimar Foundation had done setting up a locked-down Citrix system - some machines became infected with a malware product called “Security Shield”. Security Shield masquerades as an anti-virus package, makes a lot of bad noise about (non-existent) viruses and then asks the user for a credit card number to “activate the virus protection”. Well, Security Shield, this Christmas you actually asked the homeless for money. Well done. You scared the hell out of frightened, vulnerable people who thought they’d broken the computer. You scared and upset the volunteers who were giving up their Christmas holiday to help those same people. I really hope you take all the money you’ve made from your little scam and spend it on something that’ll bring you joy in this life, because karma’s a bitch and your next life is going to suck.

Anyway. Karmic retribution aside, it really got me thinking. For every iPad-toting hipster raving about how they keep all their stuff “in the cloud” because it’s the Next Big Thing, there’s a rough sleeper out there who genuinely doesn’t have anywhere else to keep it. There’s web mail, web docs, online tools for retouching photos, online games (spidersolitaire.com proved very popular with some of our guests), online video – you have no idea how cool YouTube is to someone who doesn’t own a PC or a television - and I started wondering what else technology could offer.

What about a complete online tool set and sandbox for people looking to learn about software development? Tutorials, exercises, a lab, workshop, CV and portfolio, accessible anytime, anywhere, for the cost of a virtual machine and a few gigs of disk space (i.e. practically nothing). A full development environment where they can learn to code, store projects, compile, test and deploy applications and websites – but accessible entirely within the sort of stripped-down web browser you'd find in most internet cafés. There’s no material reason why someone couldn’t learn to build software, establish an online presence, contribute to open-source projects, develop a reputation on Q&A sites like StackOverflow, and generally become a demonstrably employable developer, entirely without any investment in physical resources. The days of needing expensive computer time or equipment to become a coder are gone. The financial barrier to entry in our industry is effectively zero. We should be shouting this from the rooftops, doing everything possible to put information and resources in the hands of anyone who cares to take advantage of it, and celebrating this amazing consequence of the openness that’s sustained our industry for so long.

Anyway. Crisis was awesome, you should all do it next year, and here’s to a 2011 filled with compassion, kindness, enthusiasm and excitement.

Happy New Year.

Git Logos and Icons

2010-11-28T19:25:00.001Z

I’ve recently started working on a couple of projects that are hosted on Github, most notably the Kurejito payment gateway project for which I’m a member of the core team on an OS project for the first time (yeah, scary…)

Git is pretty cool. A bit of a learning curve but I suspect once I get the hang of frequent commits and local branching and work out how stop shooting myself in the foot it’s going to be quite hard going back to Subversion.

Thing is, I really don’t like the Git icon and logo that are shipping with Mingw32 Git for Windows. I mean, I really don’t like them. I don’t like them so much that every time I fire up git bash, I get completely distracted by how much I don’t like them and end up wanting to redesign them instead of doing whatever it was I was supposed to be doing… so I did.

This started out as me just wanting a decent Windows icon for the link to Git Bash in my Start menu, I got a bit carried away, and ended up with this. And since Git appears to use a collaboratively-edited Wiki page as the closest thing it has to brand and logo guidelines, if you want to use any of these logos or icons for your Git-related shortcuts, projects or pages, go ahead. I’m not going to stop you, and I suspect they won’t, either.

If you’re interested, they’re released under a Creative Commons Sharealike Attribution License – and there’s downloads (including Windows and Mac icon formats) and original artwork files at http://www.dylanbeattie.net/git_logo/

Direct download links:

Git Icon – Windows .ICO Format – git_icon.ico
Git icon in MacOS .icns format – git_icon.icns
Git icon original artwork – git_icon.ai or git_icon.png
Git logo in PNG format – git_logo.png
Git logo in Adobe Illustrator format – git_logo.ai
Git logo in SVG format – git_logo.svg

or grab the whole package as git_icon_and_logo.zip

Sending Templated E-mail using the Spark View Engine

2010-11-26T11:46:00.001Z

We have a couple of systems that send personalized e-mail notifications to our users, and for a while I’ve been looking for a nice way to use a proper MVC-style view engine to populate the templates for personalizing these e-mails. The problem is, most of the ASP.NET view engines are so tightly bound to System.Web and things like HttpContext.Current and the VirtualPathProvider that running them in a standalone console application is really quite unpleasant.

Well, with a bit of hacking around and some help from @RobertTheGrey, I’ve finally got the awesome Spark view engine running within a console app. No fake VirtualPathProviders, no mocking or spoofing HttpContext.Current – it just works.

The secret is this little snippet of code here:

var templateFolder = @"D:\Templates\";
var viewFolderParameters = new Dictionary<string, string> {
{"basePath", templateFolder}
};
var settings = new SparkSettings();
settings.SetPageBaseType(typeof(TemplateBase));
settings.AddViewFolder(ViewFolderType.FileSystem, viewFolderParameters);
engine = new SparkViewEngine(settings);

which spins up a fresh SparkSettings configuration object, tells it to use your own TemplateBase class and the templates folder you’ve specified. The method that actually does the population looks like this:

public string Populate(string templateFileName, object data) {
var writer = new StringWriter();
var descriptor = new SparkViewDescriptor();
descriptor.AddTemplate(templateFileName);
var view = (TemplateBase)engine.CreateInstance(descriptor);
try {
    view.ViewData = new ViewDataDictionary(data);
    view.RenderView(writer);
} finally {
    engine.ReleaseInstance(view);
}
return (writer.ToString());
}

so you end up with little snippets like this:

foreach(var user in userRepository.RetrieveUsersWhoShouldGetWelcomeEmails()) {
    var htmlBody = templater.Populate("welcome_html.spark”, user):
    var textBody = templater.Populate(“welcome_text.spark”, user):
    mailServer.SendMail(“me@mysite.com”, user.Email, “Welcome!”, textBody, htmlBody);
}

There’s a full working example in the Spark repository on GitHub if you’re interested.

Should “Cancel” cancel the cancellation, or just cancel the cancellation cancellation?

2010-11-03T19:02:00.002Z

This is from SagePay’s new MySagePay admin system, which has brought us all so much joy these past weeks and will go down in history as my second favourite piece of web design ever, after Kibo’s Optimised for WebTV! page.

Today’s happy little discovery. You click to view a transaction. You think “Oops, wrong transaction.” Years of Windows experience means you instinctively hit Cancel – the button marked Cancel, that’s at the bottom-right of the window, just like it is on every other dialog box you’ve seen in the last 25 years. But guess what? In this case, the Cancel button actually cancels the transaction, rather than closing the window and backing out of the operation.

Of course, it wouldn’t do anything that drastic without a confirmation dialog… guess what that dialog looks like?

“Cancel” means exactly the same thing in practically every application, window manager and operating system that’s been released for the last 25 years. The idea that anyone might think that using it for a different purpose in a different context is… just unbelievable.

Moral of the story: if your ubiquitous language contains a word or phrase that everyone knows means something else – change the ubiquitous language. Think of a better word. Here’s some to get you started… Unauthenticate. Withdraw. Retract. Deauthenticate. Discard. Abandon. Reject. Decline. Even “Cancel Transaction” would have been better than this.

So – to cancel a transaction, you hit Cancel, then Cancel again. To cancel the cancellation, you hit Cancel, but DON’T press Cancel, press X and then X.

(and yeah, to cancel the dialog as opposed to the transaction, you click the groovy little X in the top corner, because this is Web 2.0, dontcha know?)

Obvious, really.

“If It Ain’t Broke…” – Why Good Code Doesn’t Guarantee Happy Users

2010-10-26T02:23:00.001+01:00

For years, my sites have processed online payments using a firm called Protx. Protx was acquired by Sage a few years back, rebranded as SagePay, and since then they’ve been gradually making changes to their system. Nothing too drastic, and always with plenty of warning that changes were forthcoming. Their payment API is generally rock-solid, powerful, flexible, and has allowed us to cope with all sorts of complicated payment scenarios without too much fuss. So far, so good.

About two months ago, SagePay announced that a preview of their new administration system was running on their test servers, and they invited everyone to go and have a look. Their old system was kinda clunky, but it worked, and our accounts team had been using it quite happily for years. So I went. I looked. The new system had lots of shiny Ajax – OK, fair enough, this is 2010 after all – and horizontal scrolling. No, really. Check this out:

I have a 1600x1200 screen because information is nice and I like being able to see lots of it at once. SagePay have decided, for some reason, that our payment transactions are best presented in a fixed-size 800x300px window. Fine, though – that’s what technical previews are for; lots of people reported this and were assured it would be sorted out.

Well, the new system launched over the weekend. The old one’s gone, they new one’s live, and things like the horizontal scrolling have not been sorted out, and customers’ reactions to the new system is, well, a bit negative. A thread on their support forums is full of gems like:

“A complete crock. Should be rolled back immediately.” (here)

“this is a complete and utter disaster. This system needs rolling back as soon as possible.” (here)

“The system is rejecting the order because of the delivery address line 2 which contained 58-62 xxxxx Road. Your system is not allowing the - in the address (This address is the formatted address that is provided by Royal Mail). This needs to be fixed quickly.” (here)

OK, fine. Big company rolls out shiny upgrade, upsets customers, breaks APIs, whatever. Big companies do this a lot.

But - I was expecting something different here, because last week, Mat – SagePay’s “Chief Nerd” – published this blog post, in which he talks quite rationally and eloquently about the forthcoming improvements. And it sounds like quite good stuff, too. Stuff like:

“To accommodate this we’ve changed the way we develop software, not only at a language level, but also at the process level. We’ve embraced Test Driven Development, Agile methodologies, Continuous Integration and parallel automated testing.”

“By writing tests first and only then writing code that passes those tests, we know that our software does what it is supposed to.”

“This demonstrable code quality gives our developers much more confidence in their code, frees them to refactor software that behaves sub-optimally, and ensures the test team’s time isn’t wasted on trivial bugs.”

(quotes from http://sagepay.wordpress.com/2010/10/18/if-it-aint-broke/)

OK. Mat, I believe you. If your comments about improved capacity and security are correct, this actually represents a huge achievement for the SagePay technical team. But I have to wonder… if the chief nerd is doing everything right, why are the customers so upset?

“…if the CHIEF NERD is doing EVERYTHING RIGHT, why are the CUSTOMERS so UPSET?”

Well, this is what I think probably happened. First, they’ve been working on this since June 2009; the first real customer preview was in September 2010, and it’s now live, just over a month later. That’s not a series of incremental improvements, delivering discrete chunks of business value every few weeks or months. That’s the big rewrite wearing a not-very-convincing Agile disguise.

Second, one of the core tenets of the Agile Manifesto is “customer collaboration over contract negotiation” – and to get that right, you have to know who your customers really are, and collaborate with them. This is tricky, because to the developers, the ‘customer’ is probably the product owner – but in reality, the customer should be the person who’s going to use the product. Now a firm like SagePay probably can’t call their customers in off the street to collaborate on a big project like this – but they can talk to them.

I think it’s really easy – particularly in big companies – to get this wrong. I have made this mistake many times. You launch version 1.0, you spend a couple of years babysitting it and fielding the support calls, and you end up thinking you know exactly what all your customers want… except you don’t, because happy users never call support. They just use the product and get on with their lives. You can find out what your users don’t like from support calls and complaints, but to find out what they do like, you need to get out there and talk to them, and it sounds like many of the features in the new SagePay system were based on feature requests and complaints from a vocal minority who weren’t really representative of the user base as a whole. It feels like you’re doing exactly the right thing, but you’re not actually collaborating with your customer.

“REAL USERS don't care about TEST DATA”

Second – technical previews are all very well, but real users don’t care about test data. When I tried the new MySagePay tech preview last month, it had about a dozen transactions in it, from 2006, when we were testing an upgrade to our own payment system. That’s not a realistic test of the system, because there’s no incentive to actually do anything with it. The only way to get real feedback is to get real people to use the software to do real work – and you can’t do real work with test data.

In a nutshell, I think they missed the distinction between have we built the right thing? and have we built it right?

TDD, Continuous Integration, refactoring – these will all help you build it right, but agile’s also about making sure you’re building the right thing, and I think SagePay dropped the ball on this one.

Mat even says (my emphasis)

“…any change introduces risk. … We might produce new software that performs identically to the old for a given payment protocol only to find that two thousand customers are using a non-documented “feature” of the old system that we’ve now written out."

Well, that’s exactly what they did. On Friday, https://live.sagepay.com/txstatus/txstatus.asp was returning real-time status for previous payment transactions – and now it’s gone. Vanished. What used to take 500 milliseconds in an automated script now takes a real person 90 seconds or so – including all that lovely horizontal scrolling they have to do.

Still, all is not lost. If SagePay really do have a clean, new architecture, and full test coverage, and a decent agile process in place, then it should be straightforward to respond to this customer feedback. Respond to change, instead of following a plan. Some tweaks at the presentation layer, maybe a couple of new properties on various view models and controllers, a handful of new methods on the supporting services, and it shouldn’t take long to deliver a product that combines the scalability and security of the new system with a UI that does exactly what the customers need.

Mat, if you’re reading this, I’d be really interested to hear what you guys did today whilst your support team were running around putting out fires. I’d love to see what your product backlog looks like right now, and how you’re condensing the torrent of feedback into user stories and work items. There’s not enough sharing in this industry, and if you and your team can be as open about things today as you were this time last week, you’ll probably help a whole bunch of people – including me – next time we find ourselves babysitting a tricky launch.