Hugo+

What I want from an ad network

2015-09-10T11:16:00.003+01:00

There’s been a lot of fuss made recently about the inclusion (or possibility of inclusion) of ad-blockers in iOS9, and what that might mean for the web as a whole. On the one hand, no one wants a Flash ad banner that morphs into a dinosaur and then rampages across the screen and up onto the companion skyscraper ad**; on the other hand, ads do pay for most of the content on the web that we, as consumers, get for free. It’s the Faustian pact of the web.

So, thinking about this, and my responsibility, as a nice chap, to accept ads as the price for free content, I came up with this (totally impractical) solution.

I would happily accept adverts on the proviso that I could log in to the ad serving network behind the ads, and:

1. View all of the ads that I have been served
2. View all of the sites I have visited (as recorded by them)
3. Modify what content I want to see (no more car ads)
4. Wipe all of my stored data (even if that was restricted to once per month)

Is that too much to ask?

** Of course you do - agencies, you can have this idea for free, no royalties required

On natural lighting

2014-08-21T08:24:00.000+01:00

Short post this - I just wanted to share something that I've noticed over the past three months that gets very little attention in the office workspace press, which is currently obsessing over whether to sit or stand whilst at work.

Natural light.

I've started getting in to the office earlier, and I have an east-facing window next to my desk. When I get in to the office, the first thing I used to do was turn on the lights. Now I don't and I sit at my desk for at least an hour with just the natural daylight coming in. And I can't tell you how jarring it is when someone from the 'west-side' (i.e. dark in the morning) comes in and turns the lights on.

Daylight is far easier on the eyes, and more relaxing and comfortable than standard office strip lights - and it's surprising how dark it can be outside whilst continuing to work. Give it a try.

Joining the Tour

2014-07-29T17:41:00.003+01:00

YunoJuno is a very bicycle-friendly company. All three co-founders cycle to work, we've done London-Brighton and Oxford-Henley-via-the-Chilterns, and we're avid followers of Le Tour.

So much so, in fact, that this time last year Shib (our CEO) announced that in 2014 he was going to head to Paris for the final stage. Fast forward twelve months, and we've moved the office over to Paris for the week in order to make this happen. Airbnb sorted us out with a great apartment in the 7th, complete with discordant jazz clarinet coming from the neighbouring flat.

On the morning before the final we took to the Vélib' (the original city bike share) and cycled much of the final lap - including the full length of the Champs Elysee (up and down) and a circuit around the Arc de Triomphe.

Since we had to do a fair bit of research to work out where to watch, I thought I'd share our experience...

First thing is when to arrive. The timetable for the stage (available online) is pretty accurate, and the first 'event' is the arrival of the 'caravan' - basically a bunch of honking, hooting, carnival floats that precede the peloton by a couple of hours. We figured we should get to our spot at about the same time they did (so anticipating a two hour wait for the action). This is about right - we found a spot on the river, within the lap (so we got to see the peloton eight times), and the crowds were pretty thin at this time.

Barely-there crowds a hour before the action

Shib wanted to be close to the action, so he took the river side of the road, whilst I stayed on top in the Tuileries gardens, with a view down on to the road.

And then we waited. And I fell asleep. And then it all kicked off.

We had a fantastic view - I've posted some of my pics on Flickr, and Shib has some spectacular roadside action shots - we had about £4k worth of camera equipment between us, and of course the best shots came from an iPhone.

The Yellow Jersey on his way to victory

And then it all ended. Being a little way from the end we weren't entirely sure it had happened - you lose count of the laps, and then suddenly you realise they're not coming round again. The medal ceremony is short and snappy, and an hour later the whole thing's disappeared. And throughout it all the crowds really never became a problem - someone came and stood by me and had a perfect view and he only turned up halfway through the race.

Altogether it's a great experience - definitely to be repeated.

The current state of TV

2014-05-22T17:58:00.004+01:00

A couple of weeks ago I decided I needed to get up to speed with GoT, and set about watching series 1-3, so that I could catch up to the latest series as it played out live.

This was my experience.

Sign up to NowTV, Sky's pay-as-you-go monthly streaming service, in order to access the old series.
Watch half of series 1 using Chromecast from Android app - it's a great experience.
Enjoy it so much I fork out for a NowTV box, so I don't even need to cast. Brilliant. This is future.
Sky's licensing window with HBO expires - all episodes disappear from Sky. I am bereft.
Cannot find old GoT on any other streaming service in the UK.
Purchase US VPN to access HBO-Go.
Discover that in the US, you have to sign up via your "TV Provider" - as I live in the UK, my "TV provider" is a metal stick on my roof. It doesn't support oauth.
Send an SOS out on Twitter for solutions. Nothing comes back.
Give in, and download series 2 via BitTorrent. I've already spent about £40 just to watch 6 episodes of first series, so figure I'm owed something.
Wait about four days for torrent to download.
Watch downloads using Videostream chrome app and Chromecast - it too is brilliant, but feel guilty about my crime.
Guilt takes control, eventually finding series 3 available on Blinkbox, from Tesco. This is ironic, given that I spent two years building Tesco's first streaming service, before it was folded into Blinkbox.
Buy series 3.
Find out that BlinkBox's Android app doesn't support casting, so have to watch from laptop. There's a terrible lag casting from a Silverlight app running in chrome, so I switch to mirroring via AppleTV. The Silverlight app threatens to melt my laptop.

So far, I've watched (most of) three series of GoT, and along the way:

I've subscribed to two new services (NowTV and VPN).
I've bought one new set-top box (NowTV).
I've used a further two set-top boxes (Chromecast and AppleTV).
I've used two separate device (laptop and tablet)
I've downloaded two new apps (NowTV, Blinkbox, Videocast)
I've used two further apps (TunnelBlick, uTorrent)
I've broken various Terms of Service agreements
I may even have broken the law?

This the current state of TV.

It's a clusterf*ck.

**UPDATE**

Despite my best efforts, I wasn't fast enough - so although I have now finished series three, and series four is available via my NowTV subscription - the first three episodes have already expired. And because the series is still in progress, it's not available for streaming anywhere else.

How did we end up with this?

2013-12-17T09:06:00.001+00:00

I, along with several million other people, got caught up in both the LinkedIn and Adobe account hacks, and I've had "stop using 'password' as my password for every online account" in my to-do list for at least a year.

A couple of weeks ago I decided to do something about it. I activated two-factor auth on every account that supported it, clicked the 'generate temporary password for devices' link on Twitter, and then hit the nuclear button by downloading LastPass and beginning to upgrade my passwords (I don't really use 'password' as my password - I also use 'P@ssword', 'Passw0rd!' and 'pa55w0rd' - I'm always thinking, see).

It was a PITA changing old passwords, but I just about got around to changing the important things (accounts attached to credit cards, and core identity providers) when my new phone arrived (Moto-G, which is fantastic, btw, although I am comparing it to a 2.5 yr old HTC).

What a clusterf*k. Activating two-factor auth and various device checks means that I am constantly being asked to re-enter passwords, only this time I have no idea what those passwords are, as I've changed them from the ones I know to the brilliantly uncrackable long, complex, passwords generated by LastPass.

On the plus side I've discovered that Google Keep is a very efficient way of copying complicated passwords in plain text between devices. On the downside, lots of apps don't support 'Paste' on Android, so I have to write them down instead (hence a pocketful of Post-It notes). I think I'm now technically less secure, as I'm far more prone to social engineering hacks that I was before.

I really hope that those who hold our secrets (no, not the NSA) are working on solutions to the secure login conundrum, because no one outside of the tech industries would use a computer if it was as complicated as the situation I find myself in.

Marketing reality distortion - a case study.

2013-12-13T09:30:00.002+00:00

It's a slow day at work (warming up for the Christmas party), so I thought I'd blog about Accenture instead. Actually, this is a rant about marketing departments, not Accenture per se, they just happen to have riled me this morning.

A friend of mine works at Accenture, and this morning retweeted an image from their current recruitment marketing campaign. It's a great pic (see below), and contains a very clear message. They're looking for Big Thinkers, the ambitious, those who aspire to a Better Future. The message itself is literally spelled out, "Future > Present", and visually it taps into something many men (possibly a tad too gender specific, but maybe I'm being over-sensitive) will immediately understand: I want to be an astronaut. That's my dream.

It's a seductive message - and for anyone remotely ambitious just starting out (it's aimed at people just starting their working lives) it makes Accenture look like somewhere you'd like to work. Maybe not quite as cool as Google (who do genuinely have a "Moonshot" division), but a good option nonetheless. So you click through.

And that's when the wheels come off. Apparently the recruitment budget was spent, in its entirety, on the campaign. Having nothing left they got the guy in Accounts' brother to knock up the campaign microsite. He's jazzed it up with a carousel of incredibly happy people in grey suits (and ties - really, who wears a tie these days) - but I'm pretty sure none of them are Felix Baumgartner or Commander Hadfield.

Pretty much everything on this page (font, use of white space, formatting, images) is bad. I imagine it's just the default Sharepoint "Add new microsite" template, but that's not really the point. This page, which is where things get real, also has a very clear message: we're accountants, or possibly management consultants, but either way you're going to spending a lot of your time in front of a spreadsheet, and it's not memorising launch code sequences. In addition, we really don't give a sh*t about design, the process of creation, attention to detail, refinement or finesse. We're robots. With spreadsheets. And sometimes we hand out awards to other spreadsheet-toting robots.

In reality I've known lots of people who've passed through the Accenture machine, and they're not all number-crunching automatons - they're generally intelligent, hard-working, interesting people, with a lot to say, on a lot of subjects.

This is a masterclass of old-school marketing-reality distortion, and rather than making Accenture an attractive place to work, it makes them look desperate ("we couldn't think what to do, so we just made it up"). There are lots of good reasons to work at Accenture - wanting to be an astronaut isn't one of them.

Spies stole my laptop. Possibly.

2013-08-14T09:05:00.004+01:00

I've bought a tin-foil hat.

I was burgled last month. Nothing dramatic, they broke a window, stole a laptop. They didn't make any mess (suspiciously neat as it happens, I'd almost claim they vacuumed the place), and it was more of an inconvenience than anything else.

The police came round and dusted for fingerprints - nothing - no blood where they'd crashed the window, no greasy fingerprints on anything. They may have worn gloves (professional thieves, in Stockwell?)

And then I realised that a couple of other things were missing - principally two other (old, not used) laptops (resale value < £50), and rather bizarrely, and most annoyingly, my v. expensive Logitech wireless keyboard.

At about the same time the Snowden story was unfolding, and I was tweeting and posting occasionally on the subject.

Oh, and I live less than a mile from MI6, the UK's foreign intelligence service (007).

So here's my tin foil hat theory - spies stole my laptops, and more than that, they stole my keyboard as they have backdoor key-logging software installed on popular wireless keyboards. Don't say I didn't warn you.

Over and out - heading off-grid for a while...

Status codes are not for humans.

2013-04-03T13:03:00.002+01:00

Last week I finally managed to provoke our tech lead into using the 'F-word' in conversation. The trigger for this outburst was none other than HTTP status codes, and my desire to invent a new one. This was clearly beyond the pale.

I have since retreated from my original stance (a custom status code), but have committed a change to our dev branch to use an existing, but rarely seen, status code - 422 "Unprocessable Entity" (more on the situation in which I return this later).

This code is an extension to the base set of codes, proposed as part of the WebDAV extensions to HTTP 1.1 in June 2007 (citation):

The 422 (Unprocessable Entity) status code means the server understands the content type of the request entity (hence a 415(Unsupported Media Type) status code is inappropriate), and the syntax of the request entity is correct (thus a 400 (Bad Request) status code is inappropriate) but was unable to process the contained instructions. For example, this error condition may occur if an XML request body contains well-formed (i.e., syntactically correct), but semantically erroneous, XML instructions.

This status code has seen increasing adoption within the HATEOS / REST community, however it's not without its detractors - as the comment thread with this postillustrates.

My particular crime is not just to use this code, but to use it in a classic vanilla HTML scenario - in the event of form POST validation errors.

It has always seemed to me a peculiar anomaly that whilst the PRG pattern and the use of 3xx status codes (albeit typically the 'wrong' code, 302, instead of the code specifically designed for this situation - 303) has become ubiquitous, it is perfectly acceptable to return a 200 OK status when the form POST is rejected (because some validation has failed).

The common argument against this is that form validation is an application specific issue, and therefore not the responsibility of the protocol used to the transmit the data, but I think that's a thin argument at best. Additional, accepted, status codes exist for "Payment Required (402)", and "Forbidden (403)", both of which relate to the underlying application, and not HTTP per se.

The 2xx code block specifically states:

This class of status code indicates that the client's request was successfully received, understood, and accepted.

But what if the request was successfully received, understood, and notaccepted. Surely this is a valid use case - in fact I would suggest that it's one of the canonical HTML/HTTP use cases. And whilst the specific rules around acceptance are application specific, the generic concept of 'Request Declined' is ubiquitous - in 15 years of web development I've never seen an application that does not support this scenario. And the rules around this are no more application-specific than the 'Payment Required' example.

It is (IMO) unfortunate that people who are looking to support this as a valid 4xx exception are having to shoe-horn this into the 422 WebDAV extension, but my suggestion for a new 421 code (421 on the basis that it's between 422, which is the closest formal code, and 420, which Twitter appropriated and therefore also made-up) seems to have really upset people, so I'm happy to stand down on that. (And in fact, re-reading 422 now, it is basically what I'm looking for - just would rather it wasn't within the context of a WebDAV submission. My ideal outcome would be to rewrite the description of 422 in a more generic sense, and have it adopted as the de facto response in the event of a request validation error.)

All of which brings me to the real purpose of this post. In researching 422, I have come across endless discussion around the rights and wrongs of HTTP code assignments, but not one post or comment on what, to me, is the main reason for adopting status codes at all (beyond the functioning of the web, of course) - which is to facilitate testing and analytics, i.e. making it easier for computers to understand what is going on without having to read HTML.

Status codes are transparent and invisible to end users, but stick out like a sore thumb in logs, which makes them invaluable for analysis. A log file that includes nothing but 200s or the occasional 404 provides very little insight in to what is really going on.

Similarly, having to interrogate the HTML body of a response to understand whether the HTTP request was "received, understood and accepted" or not is painful at best, and often misleading.
HTTP status codes aren't there for humans, they are there for non-humans, whether that be the routing infrastructure (understanding what to cache), or HTTP clients (understanding what to do next).

And remember, browsers are not the only clients.

(PS - to everyone who disagrees with me, bear in mind that I do understand your objections, I just don't agree with them. I'm trying to make the web more useful, not studying for my Masters.)

[UPDATE]

I found this in the original HTTP 1.1 specification:

HTTP status codes are extensible. HTTP applications are not required to understand the meaning of all registered status codes, though such understanding is obviously desirable. However, applications MUST understand the class of any status code, as indicated by the first digit, and treat any unrecognized response as being equivalent to the x00 status code of that class, with the exception that an unrecognized response MUST NOT be cached. For example, if an unrecognized status code of 431 is received by the client, it can safely assume that there was something wrong with its request and treat the response as if it had received a 400 status code. In such cases, user agents SHOULD present to the user the entity returned with the response, since that entity is likely to include human- readable information which will explain the unusual status.

You can make up your own mind from this (I have).

Programmers are not interchangeable

2013-02-21T11:40:00.001+00:00

[UPDATE: just found this in my drafts folder from a few months ago - not sure why I never published it at the time, but since YJ is out-and-proud, I thought I should push the button.]

This essay by Paul Graham (YC founder) from back in 2007 has been doing the rounds recently (trending on HN), and it struck a chord with me because of the following extract and the new project with which I am involved - YunoJuno - www.yunojuno.com.

YJ is founded on precisely this belief - that certain 'craft skills' (designers, developers, ux etc.) are not interchangeable, and that the success of a project is very largely dependent on the specific individuals involved.

To that end, we want to create exactly the kind of relationship that Paul outlines below - "Maybe we could define a new kind of organization that combined the efforts of individuals without requiring them to be interchangeable."

It's a vision we share, and one that I hope YunoJuno will be a part of (specifically the part that looks after the individuals). If you have a skill, and you want to treated as an individual, but to feel part of something greater, then head on over and 'Join the Family'.

~~(We're in beta at the moment, launching in earnest in the new year.~~ Not any more - we are open for business.)

One of the defining qualities of organizations since there have been such a thing is to treat individuals as interchangeable parts. This works well for more parallelizable tasks, like fighting wars. For most of history a well-drilled army of professional soldiers could be counted on to beat an army of individual warriors, no matter how valorous. But having ideas is not very parallelizable.

And that's what programs are: ideas.

It's not merely true that organizations dislike the idea of depending on individual genius, it's a tautology. It's part of the definition of an organization not to. Of our current concept of an organization, at least.

Maybe we could define a new kind of organization that combined the efforts of individuals without requiring them to be interchangeable. Arguably a market is such a form of organization, though it may be more accurate to describe a market as a degenerate case—as what you get by default when organization isn't possible.

Probably the best we'll do is some kind of hack, like making the programming parts of an organization work differently from the rest. Perhaps the optimal solution is for big companies not even to try to develop ideas in house, but simply to buy them. But regardless of what the solution turns out to be, the first step is to realize there's a problem. There is a contradiction in the very phrase "software company." The two words are pulling in opposite directions. Any good programmer in a large organization is going to be at odds with it, because organizations are designed to prevent what programmers strive for.

Desktop ambivalence (in a post-OS world)

2013-02-21T11:28:00.000+00:00

There was a time when your choice of desktop OS said something about you - you were a Windows nerd or an Apple hipster - hell, Apple even based an entire advertising campaign on it. That was then. This is now.

A couple of years ago I was exclusively a Windows person. I wrote software for the .NET platform, which made Visual Studio my tool of choice, and the software I wrote ran on Windows servers. Even if I was working with VMs, I was running Windows Server VMs inside my Windows desktop OS.

I was always aware of Apple, but as I wasn't a hipster, and had never opened Photoshop, I left them to one side. When I was forced to use OSX I felt quite strongly that it was inferior to Windows.

Then I went to work at a creative agency, where I was swimming in an ocean of Mac-luvin. I started using one on the odd occasions when I need to borrow an office laptop. I still didn't like OSX, but I could get along with it.

At the same time I started playing around with Python, and I decided to scratch an itch I had around my complete lack of understanding of Linux as an OS. So I installed Ubuntu on a VM, and started playing with that.

Then I was given an iPad by work. And I bought myself a Windows 7 phone. So now I had Windows 7, Windows Phone 7, iOS, OSX, and Ubuntu in my life. I even installed Windows 8 when it came out (and paid for the upgrade!), but uninstalled it a week later (you can read about that here).

I finally realised that I had become ambivalent to the OS - and started concentrating on the tools I was using instead. I dropped Visual Studio in favour of Sublime Text, and dropped Word in favour of anything that worked with Markdown (which is anything).

I still don't like OSX - the fact that maximising a window doesn't, you know, maximise it, and the woeful Finder being my pet hates - but I bought myself a MacBook Air as my primary computer simply because I like the hardware, and frankly, I don't really care about the OS.

A beautifully designed piece of hardware running beautifully designed, and simple, software (stand up iA Writer), running on an OS that just gets out of my way, is all I need these days. Are we in the post-OS-as-brand world?

Postscript: the logical conclusion of this is no (discernible) OS at all. Over Christmas I thought I'd be a good son and back up the laptop I bought for my mother a couple of years ago. She uses it infrequently, but enough, and I figured she probably had something of value on it worth saving. In fact there was nothing - at all - on it. Every single thing she does on her computer is web-based. She is the Chrome OS poster child (grandmother).

Testing HTML emails

2012-12-01T12:01:00.003+00:00

Someone asked me yesterday if I could send the contents of an HTML file as an HTML-formatted email to a client, as they wanted to see what it looked like. Sending HTML emails is usually something that's either done through an online service (MailChimp et al.) or from the internals of a platform build where you (as a dev) have access to templates and SMTP servers and all that that entails.

Just sending a file as an HTML-formatted email is surprisingly hard to do (i.e. you can't type raw HTML into your email client and have it render in the recipient's inbox.)

So I put a script together to do it - https://gist.github.com/4176850

It requires Python (it's a Python script), but runs from the command line, thus:

$ python send_html_email.py --filename='test_file.html' --recipient=hugo@example.com

It runs through Gmail's SMTP service, so you will need a Gmail account - and you'll be prompted for your email and password when you run the script.

Feel free to use, feedback, fork, etc.

[UPDATE]

I have recently come across the service mailtrap (mailtrap.io), which is a brilliant companion to this sort of solution. Mailtrap operates an SMTP service that doesn't go anywhere. Sound crazy, but it's great - you set up an account, get a set of SMTP credentials, and then everything you send to that server gets parked in the Mailtrap inbox - where you can see the emails received as HTML, plain text and even source. Try it out.

Windows 8 - first thoughts

2012-11-03T23:53:00.003+00:00

tl;dr I've been through the pain so you don't have to - please do not install
Windows 8 on a desktop, non-touch, computer - you can thank me later.

Last weekend I installed (and paid for) a copy of Windows 8 on my home desktop.
I complained about it at the time (display driver issues), but persevered, so I
thought I'd share my experience.

Before starting, and to give context to my story, here's my history with MSFT:
for many years I made my living building application with Microsoft technologies.
I've created Excel spreadsheets, Access databases, VB6 applications, eMbedded
Visual Basic mobile applications, .NET applications (Windows and Web, from 1.0
through to the current period), and large enterprise implementations of their
core server products; I worked for a year with Redmond on the 'Jupiter' project,
which became BizTalk 2004, and earned a living as a BizTalk consultant for some
time after that. I've owned mobile Windows products since the early days of
WinCE - I had black and white Cassioepia PDA, and all the terrible phones in
between. I'm not a fanboy, but it's not far off to say that Microsoft and their
platform have paid for my house, my holidays and most of my belongings. I owe
them.

Recently I've branched out (not entirely of my own volition): I was given an
iPad (1) as a present a couple of years ago, which is great; I bought myself a
Nexus 7, which is better (for me, as reading is my main activity on a tablet).
I have an Android phone (which I hate), but only because I lost my Windows Phone
7 (which I loved). At work I switch between a MacBook Pro laptop (pretty good),
and a desktop which runs both Windows 7 and Ubuntu, which I use for development.

So I think it's fair to say that I have a pretty good grasp of the main desktop,
tablet and phone operating systems.

Which makes my experience with the Windows 8 all the more extraordinary. Quite
simply it's the worst operating system I've yet encountered. And one week after
installing it I have returned to Windows 7, which is the best desktop operating
system to date (IMO - I know Mac-lovers think OSX is great, but really, it
isn't. Finder, anyone?)

Windows Phone 7 is fantastic, a real break from the competition, and a bold,
imaginative statement from Redmond. It was clearly created by a team with great
talent, given the freedom to do what they are good at. WP7 is what happens when
you get the right people in the room, and let them do their best work. The problem
is that on the back its critical (if not commercial) acclaim, something incredible
appears to have happened.

It feels (and I have no inside knowledge on this) as if the WP7 team were fired,
and then their work was handed to a group with less talent, no background on the
project, and simply told to "make Windows look like this."

I cannot imagine any circumstances under which the new start screen would seem
appropriate on a desktop, operated by a mouse. Everything about it screams
"touch me", which is quite annoying on a 22" non-touch screen. On which subject,
ENORMOUS touch-friendly buttons looking f*king ridiculous on a large screen.
The installed apps (Photos, People etc.) look absurd.

All of which I could forgive, if they hadn't also messed with the 'old-style'
desktop. There is no start button, for which the person responsible should be
fired, and the Metro-style translated to the rich windowed multi-tasking envir-
onment that is Windows, looks like they just forgot to finish the product.

Windows 8 is a catastrophic downgrade in User Experience, which is a shame as I
believe it covers a bunch of really quite important internal changes. Most
companies use pointless eye-candy to distract from the lack of real features,
but it appears as if MSFT have pulled off a first - drawing attention away from
the good stuff to concentrate on the terrible UI/UX.

Do not upgrade.

Why Google should get behind Markdown

2012-10-31T11:10:00.000+00:00

In the war against complexity, Markdown has emerged as the winner in the
simple-text-formatting battle. There are others - notably Textile (supported
by Basecamp prior to its refresh) and reStructuredText (heavily supported by the
Python community) - but Markdown is the clear leader in its field.

In recent days there's been a bit of a hoo-hah about the lack of a formal spec.
for it (@gruber, the BDFL of Markdown doesn't seem that bothered), and the pro-
liferation of different variants / flavours. For instance, StackOverflow and
GitHub both have subtle extensions that favour their own specific use case.

I don't want to get involved in that debate, but now that it has appeared in
the 'public' domain, I thought I'd add one idea that occurred to me this
morning (whilst writing and sharing some notes).

I use iA's writer for taking notes in meetings, which supports .md, and I use
Sublime as my general purpose writing / coding tool, which also has a great
'preview in browser' function for .md files. What I'm missing however is a good
online tool for a.) making notes online, and b.) sharing those notes as HTML
pages (as per the preview in browser function) with other people.

At the same time, I use Google Docs for sharing docs online; one of the
ongoing complaints about gdocs is that it's such a poor relation to Office. So
here's what I suggest (to Google) - instead of chasing Office and adding
features, how about removing features, and simply becoming the de facto Markdown
editor / browser / publishing platform of choice. Put your weight behind the
specification effort (i.e. write a bunch of formal compliance tests for .md),
then allow people to write in .md in gdocs, and to publish them as HTML for
sharing.

They already have partial support for *bold* and _italic_ markup on Google+
posts, so they clearly have thought about this.

(And yes, I know that people who understand Markdown are a tiny niche audience,
but then I'm not expecting anyone from Google to actually read this, so I don't
need to be practical.)

[draft] Estimation is guesswork.

2012-10-22T09:46:00.002+01:00

Estimation (in software development circles) is neither an art nor a science - it's guesswork. And yet we continue to perpetuate the myth that it's possible to predict how long something will take to build.

I have a post in me about the 'immutablility of developers' and their inability to change the rate at which they work, a fact of life that very few project managers seems to have grasped.

This is just a placeholder (note to self to write it!)

Experiments in rebase.

2012-09-13T09:29:00.004+01:00

Thought I'd post this as a visual for anyone else getting their head around the difference between rebase and merge.

I created an project, and then created a series of commits to that project (by 'touch'ing a text file), across two branches, 'master' and 'branch':

- [master] touch readme1 'first commit to master'
- [master] touch readme2 'second commit to master'
- git checkout -b branch
- [branch] touch readme3 'first commit to rebase'
- [branch] touch readme4 'second commit to rebase'
- git checkout 'master'
- [master] touch readme5 'third commit to master'

At this point I had two diverged branches.

Running $ git log --oneline on [master] i have:

518f971 third commit to master
4a6d74b second commit to master
0f788ce first commit to master

on [branch] I have:

10e8078 second commit to branch
0122b8b first commit to branch
4a6d74b second commit to master
0f788ce first commit to master

I now copied the entire directory so that I have two versions of the project in exactly the same state, one of which I will rebase, the other I will merge.

Rebase

$ git checkout branch
$ git rebase master

7b9e782 second commit to branch
58f65f3 first commit to branch
518f971 third commit to master
4a6d74b second commit to master
0f788ce first commit to master

At this point, master is behind branch, but can be 'fast-forwarded' by merging:

$ git checkout master
$ git merge branch

Merge

$ git checkout master
$ git merge branch

a06b559 Merge branch 'branch'

518f971 third commit to master
10e8078 second commit to branch
0122b8b first commit to branch
4a6d74b second commit to master
0f788ce first commit to master

This time you get an extra commit - which is the merge commit itself. This method preserves the exact history - i.e. the fact that development branched, and was then merged back in successfully. If this is important (and I'm definitely not in the 'always rebase' camp, then go with merge. Otherwise, rebase will give you a cleaner history. (NB like a supermodel, the better looking the history, the more deceptive the reality. You may prefer to preserve the history of branching.)

GitHub goes down, and everyone just carries on...

2012-09-11T18:10:00.000+01:00

GitHub has been on & off all day today, and at first one would imagine that that would cause a gigantic PITA for developers, and that productivity in all places 'silicon' (valley, fen, glen, roundabout, savannah, desert, glacier,...) would plummet.

However, since GitHub uses git, and git is a DVCS, whilst users will have lost all the shiny unicorns that GH provides, they could, if their work really mattered, have simply pushed their repository to another hosted git service - like Codebase, BitBucket, Assembla, ..., and then carried on as normal. A little bit of coordination required to get teammates the new repo URL, and they'd be back online in minutes.

Or simply continued working locally, and committing, until GH gets back online. I've often done the reverse - I have private repos in BitBucket (free!) which I have periodically pushed to GH in order to take advantage of some of the branch visualisations.

Try doing that with TFS or SVN.

Django's extended user profile...

2012-08-09T22:35:00.000+01:00

Django, as Python's (arguably) most popular web framework, follows the 'batteries included' philosophy of the language with a large number of built-in web features. One of these is the 'auth' system, which provides simple user and group authentication and permissions management.

The core User class is 'skeletal' in nature - containing as it does just email, username, first name, last name and password profile fields (as well as some other utility attributes such as 'is_staff' etc). This is unlikely to ever satisfy a real-world scenario, and so Django includes a documented mechanism for extending this base user record with your own application-specific extended profile. It's simple to set up and use - you define a profile model, and then add that model as the AUTH_PROFILE_MODULE setting. Once this is set up, you can simply call the get_profile() method of any user object, and it will return the profile object.

Unfortunately, there is one fundamental limitation to this pattern - it is assumed that every user registering on your site has the same profile. So if, for instance, you wanted teachers and pupils, or perhaps suppliers and customers; or buyers and sellers, each with their own profile type, you are out of luck.

Google this particular issue and you will find a plethora of workarounds and hacks, all of which seem to start with the same underlying assumption - that the core django framework does some kind of magic that cannot be subverted; get_profile() is sacrosanct.

One of the joys of using open source software, of course, is that you can open up frameworks and just take a look at how they work. So that's what I've done, and this is what I've found...

When using extended user profiles within django, there are really only two key requirements - first, that when a new user signs up, the user profile is created; and second, that when you call get_profile() on a user, you get the correct profile object back again. Let's dig into both of those in turn, and consider how we might adapt them to use multiple profile types.

1. Creating the user profile.

When creating the extended user profile, the django framework provides precisely no assistance. It is entirely up to the application developer to manage this process. To quote from the docs:

The method get_profile() does not create a profile if one does not exist. You need to register a handler for the User model'sdjango.db.models.signals.post_save signal and, in the handler, if created is True, create the associated user profile:

So, in this case, it makes no difference whether you have one or more user profile classes to use - it's entirely up to you. You would probably have something similar to this in your views:

  if kwargs['type']=='teacher':
      profile = TeacherProfile(user=request.user, subject=kwargs['subject'])
  elif kwargs['type']=='student':
      profile = StudentProfile(user=request.user, year=kwargs['year'])
  profile.save()

2. Retrieving the user profile.

This is where django, in theory, starts to add value. The framework includes the user.get_profile() method, which will return the relevant profile object for the user. The problem being that it will only support one, fixed, profile class for all users in the system. This is where the source code helps us:

def get_profile(self):
"""
Returns site-specific profile for this user. Raises
SiteProfileNotAvailable if this site does not allow profiles.
"""
if not hasattr(self, '_profile_cache'):
from django.conf import settings
if not getattr(settings, 'AUTH_PROFILE_MODULE', False):
raise SiteProfileNotAvailable(
'You need to set AUTH_PROFILE_MODULE in your project '
'settings')
try:
app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
except ValueError:
raise SiteProfileNotAvailable(
'app_label and model_name should be separated by a dot in '
'the AUTH_PROFILE_MODULE setting')
try:
model = models.get_model(app_label, model_name)
if model is None:
raise SiteProfileNotAvailable(
'Unable to load the profile model, check '
'AUTH_PROFILE_MODULE in your project settings')
self._profile_cache = model._default_manager.using(
self._state.db).get(user__id__exact=self.id)
self._profile_cache.user = self
except (ImportError, ImproperlyConfigured):
raise SiteProfileNotAvailable
return self._profile_cache

Observation #1 is that it's not a very complicated method; we should be able to work out what this does. Let's remove some of the error checking and reduce it to what we really want:

def get_profile(self):
from django.conf import settings
app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')
model = models.get_model(app_label, model_name)
return model._default_manager.using(
self._state.db).get(user__id__exact=self.id)

Now let's reduce that to pseudocode:

  # pseudocode - do not attempt to run!
def get_profile(self):
determine_model_used_as_profile_form_settings_file()
get_model_from_db_using_current_user_id()

Observation #2 is that this is trivial - it does exactly what you would think, and no more. There is nothing to be scared of here. The only problem is that is only expects one value to exist in the settings file, and is fixed on that assumption.

So, how do we take this knowledge and use it to our advantage. My own personal view is that, having read through the code, you should feel perfectly happy and comfortable to ignore it, and roll your own. It really isn't doing anything magic. Nothing will break.

Using our teacher/student app above, you can simply create your own get_profile() method, and hard-code the 'get' code (you can stick the method anywhere you feel comfortable using it - I have a profiles app that defines the models, and have the get_profiles() method in __init__.py file):

# pseudocode - do not attempt to run!
def get_profile2(user):

if user is teacher:
return get_teacher_profile(user_id = user.id)

elif user is student:
return get_student_profile(user_id = user.id)

The obvious problem in the pseudocode above is how to check if the user is a teacher or student, given that there is nothing on the user object that we can use to determine what type the user is. The simplest answer to this, and it is a hack, is to use Groups. By assigning users to relevant groups when they register you can do the following:

def get_profile2(user):
if user.groups.filter(name='teachers').exists():
return get_teacher_profile(user_id = user.id)

elif user.groups.filter(name='students').exists():
return get_student_profile(user_id = user.id)

This is a workable solution, but it does require hard-coding the groups, which makes it more of a copy-and-paste solution than a pluggable django app or extension.

The ideal answer is for the core django project to support multiple profile types within a project, by changing the way it currently works. The fix is trivial:

Change the AUTH_PROFILE_MODULE to use a dictionary instead of a string to store a mapping of user types and their profile models
Add an additional field to the core user model, called something like 'user_type' (or, heaven forbid, 'profile')
Update auth.models.User.get_profile() to look up the model based on the user_type / profile attribute and use that instead

I have hacked my local copy of django to do just this, and this method works as expected. A fully-integrated, multi-profile, django installation. Of course, this isn't supportable by any hosting company, so is not a viable short-term solution. In fact it's only supportable if it gets pulled into the next version of django. I'd love to see this happen, so if you think this is a good idea, please add any feedback below; if enough people want it to happen...

Our economy is based on The Wealth of Nations, and it’s out of date…

2011-08-24T23:15:00.001+01:00

This is a post I’ve been meaning to write for some time, and have never got around to as it’s just too big a thing to get straight in my head. I’m writing something now (and it’s little more than the title) just because I think I should lay down a marker. Apologies in advance to anyone who was looking for something more insightful.

The summary is this: The Wealth of Nations by Adam Smith is the de facto bible of western economics, and its core tenet is the Division of Labour, which he studied through the prism of industrialisation and specifically the manufacture of pins.

Problem is, we’re moving beyond a classic industrial society – possibly to something identified as the Information Age. And in this age, classical economics no longer apply. This is most clearly seen in my own industry – software development, which works in precisely the opposite fashion: productivity increases as work is consolidated into a smaller group of more talented (and expensive) developers. The Division of Labour is not only unproductive, but positively destructive, as anyone who has scrambled through an offshore project can attest.

This paradox undermines many well-meaning corporate initiatives, and until we have a clear, and open, debate on this we are, frankly, a bit f*ked.

More to follow.

^[1] Notwithstanding the “Invisible Hand”, which is the other core message.

Yes, Facebook has lots of users. So what?

2011-07-07T20:43:00.001+01:00

Another day, another article about Facebook's global takeover. Yes, I know that FB now has 750m members, which is clearly a lot of people. However, the obsession with registered users feels very like the rush for pixels in the digital camera business a few back, or the stampede for ever larger app stores amongst the mobile vendors.

I don't care if FB has 1bn members (anecdotal evidence suggests that registrations peaks at apprx. 50% internet users within a country), a camera has 50m pixels or my mobile app store has 1m applications - if they're not my friends, my favourite apps, or the pictures still come out blurry it's a problem.

Google+ is the victim in the latest article - with the Facebook-Skype deal apparently sounding the death-knell for the new Hangouts video-chat service. I don't think it will kill it, and what is more, I think the niche audience that G+ is rapidly acquiring (albeit fanned by the limited access to invites) is more valuable than the mass of humanity represented by FB*.

In a similar vein, if I was a camera vendor I would pay a lot more to advertise on Flickr than Facebook (yes, I know Flickr doesn't have ads, it's a theoretical point), because people on Flickr care about photos, and people on Facebook don't. (I'm also sure there are hundreds of amateur photo groups on Facebook - but if they really cared they'd be on Flickr, as it's a destination defined by the quality of images available.)

I am sure that it makes sense for all commercial entities to have a presence on Facebook, and to invest in a "social media strategy", but they are supporting acts; the web is the superset of all properties, and will outlive Facebook.

* NB This obviously doesn't apply to the drinks brand WKD - whose natural habitat is clearly Facebook on a Friday night.

What agile means to me

2011-06-05T20:33:00.001+01:00

[Update: in response to some of the comments here and on HN, I thought it might be worth updating with a note on my personal experience with Agile, which appears at the bottom of the post.]

Agile (with a big 'A') has become so mainstream now that it has started to become the problem. An alarming number of people who espouse the virtues of Agile, and who quote the Agile Manifesto believe that Agile is a project management methodology, and that Agile really means SCRUM, XP, Kanban, and that it is embodied in the daily stand-up, whiteboards or writing requirements on post-it notes. I was once told by an Agile Trainer (LOL) that the correct way to phrase the requirement "we need advertising placeholders on the site, and some way to manage which ads appear where" was "As a User, I wish to be marketed to." Needless to say his company lost a $m project on the back of such BS.

As a result of all this, Agile (big 'A' again) has become a bit of an albatross - it doesn't really work, it doesn't deliver the benefits it promised, and it inevitably involves a lot of arguing amongst the team about who should be doing what, and when.

All of which distracts from the fact that agility (small 'a' this time) is a wonderful thing, that can be achieved, that does provide enormous benefits, and that should be encouraged in everything we (as an industry) do. Agile (small 'a') refers to team dynamics, delivery processes, the software itself (how easy is it to change), and operations. Everyone should aim to be agile.

So, a few lessons I've learned along the (hard) way:

The best way to achieve your goal is to start with the right people
If you've found the right people, let them get on with the job you're paying them for
The people with the best ability to plan the delivery are the people doing the work
A small team with a manageable backlog needs very little Project Management
Everyone on the team “needs to know”.

A few things that help this happen on a practical level:

Try and avoid building towards a pre-determined marketing / launch date
If you do have a fixed time, then scope is variable
Never try messing around with team sizes to massage delivery
Automate everything in sight - builds, deployments, testing, progress reporting, tea-making
Measure as much as you can - no feedback == no direction

And finally some other things that will prevent you from getting there:

If you have to have everything on your requirements list, you can't be agile
If you need to plan beyond the next sprint with any degree of accuracy, you're not agile
If you think you can "fix" an iteration by adding more people, you're not agile
If you are prepared to change the end date of an iteration to "fit something in", you're not agile
If you have to give a fixed date for delivery - it's very, very difficult to be agile.

Postscript: my personal experience with Agile.

I worked for several years at a company that was at the forefront of the Agile movement in the UK, and I believe that at one point they had more registered SCRUM Master practitioners than anyone else in the country. (I think Sensai Schwaber visited in person at one point, although I’d left by then.) They were generally acknowledged to be the leaders in the field.

The incident in the post occurred when I returned to the company as a client, parachuted into a large project that they were running with the full SCRUM toolkit (“Agile Coach” included), that was running into trouble, principally the fact that it was not agile. The project was rescued by hiring a team of people with the right combination of aptitude & attitude.

As I say in the post – agility is a great goal, and everyone should aim for it – but don’t make the mistake of thinking that Agile is the means to achieve it.

Driving to the airport, agile-style.

2011-06-05T19:59:00.001+01:00

If you were travelling from A-B, and planning the journey yourself, you might look it up on Google Maps. GM would give you a route, and an estimated time for the journey - let's say it's one hour for our mythical trip. Now let's assume that the destination for our journey is the airport, as we have a plane to catch - i.e. we absolutely have to be there on time.

In this scenario, would you leave the house one hour before the flight departs (let's ignore the reality of parking / check-in / security etc. - we're on the company jet, just driving up to the plane)? Of course not - if you had to be there on time, and you were taking someone else's directions, even someone with the 'authority' of Google, you'd probably add at least 50%, and probably 100% to the estimate. Even then, you're still subject to the vagaries of the traffic - something that affects everyone, regardless of whether they've driven the route before.

If you were to approach the journey from an agile point of view, you wouldn't book the airline ticket in advance - you'd catch the plane when you got there. And you wouldn't plan the route either - you'd hire a driver, who knew the route, and simply tell him the destination. He'd make up the route as he went along, avoiding traffic problems if possible by using his previous experience. It’s pretty clear that if you just set out from A, travelling to B as fast as you could, it would take less time than if mentally started at B, then worked backwards to determine when you ought to leave A.

Agile behaviour in a software development team requires a similar faith in the ‘driver’ – in this case the people in the team. The quickest way to reach your destination is to have faith in the team to deliver, and to make sure that they understand the destination. If you then insist on them following a specific route, then you have just removed their capacity to avoid delays and dead-ends – removing the reason you hired them in the first place.

If you want to be agile, you have to trust in your team.

(And yes, this does mean that outsourcing a project cannot be agile. It also means that agile is NOT a project management methodology. It’s a way of working.)

Application Lifecycle Management

2011-05-16T00:49:00.000+01:00

[Update: a couple of people have called me out on this post, and suggested that it would only work from the bottom up if you hire good developers. This is true. This entire blog is based on the philosophy that the cheapest and most efficient way to achieve any given goal in software is to hire a smaller number of better developers. If you have a different philosophy, then you may find this site more helpful http://hrb.fm/keHPzD.]

ALM is described on Wikipedia as:

“…a continuous process of managing the life of an application through governance, development and maintenance. ALM is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management.”

The phrase itself is enough to make the blood run cold, but I’ve been struggling recently to justify why I think it’s a Bad Thing, and I think I’ve worked it out: it’s all about the direction in which processes are created and applied.

ALM comes from the Old World (aka I.T.), where working practices are created by Architects and applied downwards to development teams. It’s a process designed to commoditise development, and to manage human beings out of the equation. If the process is sufficiently detailed and watertight, then developers can become replaceable – which in turn allows you to find cheapers versions.

This is lowest common denominator thinking – by creating a process that allows poor developers to work effectively, that same process will prevent good developers from doing what they’re good at. It’s mediocrity by design. Which in I.T. terms is not necessarily a bad thing – ERP, Accounting, Warehouse Management Systems – these are things that are: a.) operated in a controlled environment, b.) used by specially-trained staff, c.) defined by stability, not innovation.

None of this applies to web application development – which is defined in contrast by: a.) public access, b.) self-service, c.) continuous innovation, and pace of change. In this New World (otherwise known as The Web), companies are defined by the quality of their development staff – they are the asset, and they need be allowed to flourish, and not capped by process.

The kicker to this is that process is still a good thing in this New World – but the new processes are designed not to prevent developers from getting things wrong, but to make it easier for them to get things right. ALM should exist in practice, but it should (in fact must) come from the bottom up – the processes are organic and designed by the developers, for the developers.

There’s no requirement for management to even be aware of these processes – they should manage by outcome, not intervention.

NB If this rings any bells with you I would recommend Orson Scott Card’s “software developers as bees” article - http://hrb.fm/lIOR0w

Web app or native - looks like HTML is winning the war.

2011-05-09T14:42:00.001+01:00

When the iPhone launched, one of the major gripes was the lack of support for native apps. Steve Jobs launched the web-apps-look-like-native-apps program, was roundly criticised for the lack of a native SDK, and sure enough it turned out it was already in the pipeline. The App-Age was born.

(Not long after, Palm released webOS, a web-app platform from inception, and it was roundly praised for its innovative approach.)

Circle back round a few years, and the plethora of native platforms (iOS, Android, RIM, webOS, WP7) and form factors (phone, tablet) combined with the emergence of HTML5 as a "standard" (albeit one implemented (in)differently by each platform developer) means that web-apps are now back in favour.

PhoneGap in particular seems to have captured the mindshare of the mobile developer community; it will be interesting to see how this battle plays out over the coming year. Gaming aside, once local storage becomes a common feature, web apps look unstoppable.

Ode to Dropbox

2011-04-14T14:28:00.001+01:00

I’ve been using Dropbox for about 18 months, and before that I was a Mesh fan, and in the dim and distant past I spent many, many hours fighting with Plaxo and ActiveSync (see here for a post from 2004 on my frustrations at the time) - so I like to think I know a thing or two about file syncing issues.

I have a separate work Dropbox account, which for the past couple of months has been so far over its (free) limit that I've had to stop using it (there's only so much you can legitimately clear out!) In the meantime, two things have been happening - I've been adding to the local directory, in anticipation of a future upgrade, and other people have been adding content to the remote folders. I had enough faith in Dropbox already to be able to cope with this – I’d get another 3GB of new content coming down, and Dropbox would get a few MB of changes from my local machine. Easy.

Unfortunately, having upgraded, I discovered that in some overzealous spring-cleaning I had removed critical user profile files, and so the Windows user account associated with the work Dropbox account was corrupt, and I couldn't login and start Dropbox. This meant that all of my changes were now stuck in a local folder that couldn’t be synced, hidden amongst 2GB of data that had already been synced to the cloud.

My solution was to create a brand new Windows account and to copy over the entire 2GB archive to this new profile before hitting the sync button - but this meant that instead of adding just my recent changes, I would be syncing the entire 2GB local directory to the new user profile, 99% of which was already on Dropbox, uploaded from a different Dropbox sync. I was convinced this would result in conflict armageddon (and about 48 hours of file transfers – 2GB up, 3GB down, merge, sync, index.)

I left it running over lunch, came back and discovered... nothing. It had just worked.

Dropbox is an amazing piece of software, and if they don't sell up to Microsoft (Windows 8 - Live Mesh replacement), Apple (MobileMe replacement) or Google (Chrome OS file system) for several billion dollars something ain't right.

Google lives in Perpetual Beta, whilst Microsoft shoots for RTM.

2011-03-31T23:53:00.001+01:00

Something came up in conversation today that struck a chord with the (my) favourite topic of the moment – why is MSFT not responding to everything going on around it?

I don’t really know anything at all about MSFT’s business – I’m a .net guy, I’ve been to Redmond, I’ve seen the inside of the machine, but I don’t know all the ins-and-outs. But I do know what they do – and more importantly, how they do it, and that gives me enough to form an opinion.

They produce software. And the software they produce, even to this day, is distributed on physical disks. That’s right – they burn their software onto shiny plastic disks, which are then distributed around the world (“filling the channel”). And this is their problem.

Culturally, MSFT is still working in a physical disk world. If you’re burning your software onto 100m disks, you can’t afford to get it wrong, and you can’t easily get it back again once it’s left the building. So they create enormous product development cycles, building towards huge, monolithic, releases. They still call their final releases RTM (Release to Manufacturing). Their development processes are built around this concept, their product development process are built around this concept. Worse still, they have partnerships with disk-pressing plants and disk-selling retail channels. And all of this baggage they haul around with themselves to this day.

Back in the 90s, the talk was of how MSFT missed the boat, misunderstood the internet, and then woke up. They suddenly Got It. The built a better browser, killed Netscape, and now they owned it. Except that they didn’t. They thought that the internet was the browser – and since they owned the desktop, they could fold in a browser for free and their job was done. Wrong.

They still don’t get the internet; only 18 months ago I was being told, in Redmond, that whilst the internet was nice, desktop applications were nicer, so could we please put our internet-only Silverlight idea on hold, and just get with the Windows 7 program.

Google has none of this this heritage – it was born from the internet. As was Facebook. They can innovate and release at internet speed. A Google search results page is a continuously evolving multi-variate test. When they say that they live life in beta – they’re not lying – they really are trying new ideas every single day, live. Google is never “RTM”.

And Microsoft cannot compete – because of its legacy. They have a vault of cash, and some fantastic talent, but until they start to shed some of their physical dead-weight, until they acknowledge what the internet has done to their business (a business they invented), they are heading into retirement. The world they built is innovating away from them, and they’re just too slow to react.