While perusing Stackdriver’s blog I happenstanced across this little nugget about common AWS problems and it immediately solidified the “Chickens vs Cattle” metaphor for me.

Read these, and tell me how your legacy enterprise crapplication would cope if this happened to it!!  I particularly love the ephemeral disk corruption one and the answer?  SHOOT THE CHICKEN, GET ANOTHER!

Git yer cloud game on.

Availability

Ephemeral Disk Corruption

When running an instance with ephemeral disks available, they can become corrupt. The filesystem will become read-only, and you will be unable to write to disk. Messages will also show up in dmesg related to filesystem corruption.

• Solution: Relaunch the Instance

DNS Unavailable

It is possible for the internal DNS resolver on the hypervisor to crash which causes DNS lookups to fail from within the instance.

• Solution: Change your DNS Resolver, Relaunch (Instance Store), Stop/Start (EBS Backed)

Disk Full

Although one of the draws of AWS (and cloud computing in general) is that you can scale up as needed, it is an unfortunate reality that ephemeral disks don’t do so. You can upsize EBS but not automatically. You can check how full your disk is by checking disk usage metrics.

• Solution: Upsize disk or Remove unused files

Performance

Instance at Capacity

One of the best features of EC2 is its range of instances, but this can also make it more difficult to find the right mix between cost and size. Many people misestimate the size that they require, resulting in an instance at capacity. You can can discover that your instance at capacity through checking the CPU, memory, and disk IO.

• Solution: Upsize your EC2 Instance

RDS at Capacity

Like instances, databases can also hit capacity. This can be discovered through checking the CPU, memory, or disk IO on the RDS instance. You may also notice slow application response times, or queries taking longer than expected.

• Solution: Resize RDS or Diagnose Slow Queries

Host Contention

When distributing server space, Amazon will often oversubscribe a server under the assumption that all users won’t operate at full capacity. Using a command such as ‘iostat 1’ you can measure the amount of CPU Steal your instance is experiencing. High CPU Steal is usually an indicator of noisy neighbors.

• Solution: Relaunch (Instance Store), or Stop/Start (EBS Backed) in order to get a new host.

ELB Misconfiguration

Suboptimal configuration of your ELB can cause problems like latency, timeouts and 503 errors. It can be discovered by comparing your configuration with best practices. All of the instances in the load balancer should be healthy, and only AZs with instances in them should be configured for your load balancer.

• Solution: Reconfigure ELB

Cost

Underutilization of EC2 or RDS

Another effect of EC2’s wide variety is that resources can be underutilized also and eat into your bill. This can be discovered through looking at CPU, Memory or Disk metrics. Consider your availability as well before removing an instance.

• Solution: Reduce instance size or the number running

Application

Security Group Misconfiguration

Not configuring your security groups correctly can cause problems like timeouts, 50x errors or app unavailability. Often you remember to open the local firewall and port on an instance, but forget to open it in the EC2 security group.

• Solution: Open port on security group

Following on from my recent quick fire posts on hybrid cloud (see Make your own composite hybrid cloud monster! and Hybrid cloud / Composite cloud detailed definition), it was great to see the venerable Forrester brainboxes James Staten and Stefan Reid co-author a post on a different angle with HYBRID CLOUD FUTURE? TOO LATE!

Their premise is that, at a business process level, you are already experiencing the hybrid cloud “hybrid cloud means a cloud service connected to any other corporate resource”.

If you read my last posts and add the Forrester view to it, you get an irresistible urge to draw a diagram… or is that just me?  But what kind of diagram can it be?

• It can’t be a layer diagram because there are optional layers, and there are no dependencies between them.
• It’s not a maturity diagram because in addition to optional and non-dependant layers there is no logical progression from one aspect of hybrid cloud to another.
• It’s not a decision tree either.
• If hybrid cloud is an abstract term that can be represented by different (and not connected) concrete implementations, then it’s just a class diagram: right?

Anyhow, here’s an in-exhaustive list of what might constitute a hybrid cloud and you can pick none, one, many or all of them!  Hybrid cloud can be implemented:

• At the business process level (think BPaaS, where your business processes use internal services and an external cloud service like http://salesforce.com)
• At the business support system level (think share service catalogue like VMware vCloud Connector, think about billing across clouds to a central point for one invoice)
• At the operational support system level (think monitoring multiple clouds, think provisioning to multiple cloud, think “bursting” depending on capacity conditions)
• At the service definition level (think allowing PaaS to deploy to many clouds, or IaaS to stretch across clouds).
• At the deployment model level (think of a method to deploy a private cloud with an extension into a public cloud resource model, using something like Amazon VPC).
• At the resource level (think of ensuring compatibility between virtual resources and using standards like OVF from DMTF to allow application mobility).

So you see, hybrid cloud is many things and your mileage may vary!

1. more than one cloud
2. cloud compatibility to allow the same workload to run on any node in he composite cloud without modification
3. exposure of node metadata to allow programmatic decision making for auto placement of workloads into the composite
4. workloads encapsulated as blueprints to reduce fragility
5. one composite catalogue that contains items that can be deployed to composite cloud
6. composite-aware network balancing and naming services
7. one composite UI/API
8. control, monitor and report multiple clouds as one hybrid/composite (ie. composite-aware OSS)
9. a metering and billing capability to create a composite bill (ie. composite-aware BSS)
10. an optional network path bridging between tenant resources on multiple clouds to create a composite cloud fabric
11. recommended that migratory workloads fit a nimble profile, especially with regards to data to mitigate data gravity
12. recommended embedding of network and security intelligence into mobile workload blueprint so security and governance travels with the workload
13. reconsider centralised command/control view of hybrid into a distributed cloud view
14. apply DRS-like method across clouds
15. apply FAST-like method across clouds

Legend is full of mythical hybrid monsters, but there’s always room for some more!

Hasn’t it delighted (and terrified) mankind for millenia to conjour up combinations of man and beast and then attribute some kind of terrible reasoning to it?  Mostly this happens out of ignorance of the reality of things, sometimes just for fun stories, other times as an example of why you should eat your greens.

Should we feel sorry for the beast, rather than revulsion?  Or, if it’s mythical, should we even care?  But what if we found out the beast was real?!  What if the Minotaur was living in a villa in Cyprus, and its only crime was bad breath and the occasional G&T before 3pm?

The inimitable Christian Reilly posted a little twitpic the other day showing a snippet from some kind of exec brochure that described hybrid cloud in a way he disagreed with.  The definition was:

Hybrid cloud computing gives companies control over multiple cloud instances with the power to move data and applications back and forth almost instantaneously.

If you don’t know Christian he’s one of those guys that has the magic combo of being both a cloud consumer and a cloud provider, and having those two perspectives gives you a certain clarity when it comes to business/marketing BS.

Anyhow, putting the problems with that statement aside we have to all face up to something:

Cloud consumers, aka as business people who pay the bills and Really Matter, want “hybrid cloud” (whatever it might be!)

Let’s throw some iodine in that murky pool before we drink it though: hybrid cloud is an abstract term with many different design patterns and even more concrete implementations, right?  So you if you just ask for “hybrid cloud” then who knows what you might get.  But if you ask for a specific concrete implementation of specific pattern of hybrid cloud, then that would be better, wouldn’t it?

My appeal to my fellows in the business is not to rush to take the abstract hybrid cloud term into a limited singleton concrete straight-jacket too soon with the resultant tiresome handbags at dawn.  Maybe the time is to have the top-level hybrid cloud abstract term, followed by a mid-level set of patterns that describe abstract implementations and are technology agnostic, followed by low-level concrete implementations that are product/vendor specific?

One Abstract Definition -> Few Design Patterns -> Many Concrete Implementations

That way there’s room for everyone, and like all good neighbours we can either get on or fall out but we all have the right to live on the street.  I just wonder who will be the house with no income and all the santa decorations?

Hybrid Cloud Abstract Definition

From NIST Definition of Cloud Computing:

Hybrid cloud. The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).

Hybrid Cloud Design Pattern for Composite Catalogue

Here’s a sample stake in the ground for one kind of design pattern for hybrid cloud, and some important points are:

1. Apart from the implementation example, this is technology agnostic so could be implemented by anyone with anything.
2. This is just one example, I’m not oppressing anyone, Lorreta.
3. Patterns are wonderful in an evolutionary way in that they open things up (understandable variation) and they close things down (eliminate poor approaches, encourage fewer better methods).
4. You can make composite patterns, such as using the Catalogue and Buffer patterns together.
5. This is just focused on one context (composite catalogue)

With these things in mind, here’s an example of a Hybrid Cloud Design Pattern:

Name	Hybrid Cloud Catalogue
Intent	Provide a way to share one cloud service catalogue across multiple clouds
Motivation	Provide a single consistent catalogue across multiple clouds.
Applicability	A consumer has access to multiple providers and multiple clouds and requires a UI/API to present a single catalogue view across them all.  This is could be manifest as a single "composite cloud catalogue" that allows local (none hybrid) and composite (hybrid) catalogue items.
Structure
Participants	The Consumer access the Composite Cloud Catalogue from a single UI/API.  The Cloud Broker presents the catalogue (might be a function of a single primary Provider).  The Developer submits services and applications into the catalogue with metadata that describes how that service/app behaves in the composite function (is it local only or can it run anyway?  Does it have any characteristics like data sovereignty etc).  The Provider offers the functionality for integration with the composite catalogue.  The Auditor has controls to demonstrate compliance is adhered to so the framework and controls must exist.
Collaboration	Consumer interacts with the Broker/Provider to access the catalogue.  The Developer/Auditor also interact with the Broker/Provider but for different functions (manage and audit catalogue).
Consequences	Simplified interfaces and consumption methods increase the liquidity of the cloud.  To make this happen there needs to be as little friction and stickiness to local cloud instances.  Tradeoff is that "clouds with a purpose" like community clouds have inherent stickiness and there ability to compose with other similar clouds is limited.
Implementation	This is possible today between a VMware vCloud on-premise private cloud and a vCloud Provider using the vCloud Connector:

Initially started by less than ten IT personalities with nothing more than a sense of humour and a twitter hash tag, the joke was directed at Wall Street who continually show breathtaking ignorance of IT and also at the Silicon Valley bubble that lives in a distorted reality field generated by alien technology buried in the Rockies.

One clouderati member who didn’t want to be named said, “This prank has got out of hand just like the Office Space round up collection program did. People out there really believe that computers are ‘elastic’ and there is ‘infinite capacity’: when they learn of the reality they are going to be shocked!”

I asked the aource the obvious question: So what is the reality? “Did you ever wonder where computers went to die? That’s right: all of the old and disused PCs, Commodore 64s and Sinclair Spectrums that have gone to Silicon Heaven are being cabled together to provide a recycled cloud engine in what might just be the recycling story of the century.”

Nobody knows how this news will be taken by the IT masses who might not react well to being awoken from their mass delusion, but the clouderati source finished with a wry smile “It proves the original point, that Wall St doesn’t get IT: they will probably miss this week’s news just like they missed the last seven years of cloud.”

I wanted to share my simple 3-step fubar smoke test for any process. For each point, score 0 if it is absolutely true; score 1 if its sometime true; score 2 if its never true.

1. Watch people do the work (also known as the Reality Test) – does it match the process? (0, 1, 2)

2. Is the process updated easily to reflect reality (is there a Closed Loop system in place? Do the workers have the ability to improve the process?) (0, 1, 2)

3. Using police interrogation techniques, separate two or more people who know the process and ask them explain it as they would to a new employee (known as the FNG Test). Do the two accounts match? (0, 1, 2)

There you have it. Do that for any process. The higher you score, the more fubar the process is. There’s more, but these three are fundamental.

I’ll invite Sir Florian the Acerbic to come and comment… Should be fun (expect swearing!)

The converged infrastructure market is really into its stride now with all the big vendors having offerings, with each presenting different approaches.  The excellent David Vellante of Wikibon wrote a great piece titled Converged Infrastructure Takes The Market By Storm where they are predicting the total addressable market (TAM) to reach $402Bn by 2017.

(Disclosure I work for VCE and my company sells Vblock Systems into this market).

This Infrastructure Continuum has just one piece of data driving it:  How “off-the-shelf” is the offering, meaning how complete and usable is the product when I get it.  There’s no clutter in this diagram about which market the offerings are targeted at, I’m keeping it simple.

Simply, the Infrastructure Continuum visualises a level of effort on behalf of the consumer during design, procurement, deployment and support: at the left side your product is more customized, less off-the-shelf, and to the right hand it is very productized.

There really are differences between the offerings when you dig into them, which I’ve highlighted with the following terms (which I’m sure will make people excited):

• Semi-converged means the product is either not complete when you receive it, or it only has limited capabilities/capacity compared to a fully converged offering: ie. you the consumer still have some work to do.
• Fully-converged means the product has complete network, compute, storage and virtualization capabilities and is ready to use when you get it.
• Super-converged means the product contains additional functionality such as data protection, perhaps installed applications ready to go, and is most likely to represent a complete application appliance, deployed in pairs for DR.
• Hyper-converged is the super small form factor appliances.

The new folks, and the reason this post was triggered, are Nutanix, Simplivity and Scale Computing.  Each of them have reduced typical converged infrastructure into tiny 2U appliances/nodes that have virtualization,compute and storage contained.  They don’t offer networking like a converged system, such as a pair of redundant access switches but they will contain virtual switches.

The target market for these is typically small-medium business but doesn’t have to be: a large, multi-national might use them for remote office/branch office, and the same large company might by lots of these nodes to create farms from them.

Each product is similar in that it has some smarts in software.  Nutanix, for example, uses an NFS-based distributed filesystem across its nodes.  The offerings from Simplivity and Scale have their own operating system to offer features such as clustering, dedupe and replication.

Management wise, all of them plug into VMware’s vSphere vCenter and as each of the offerings is aimed at VMware installations, this means the management is simple.

Questions over the products are those targeted at all new companies and products they are all time-related questions about company longevity, scale, performance, supportability, product development.

It’s not possible to compare features and price amongst them because some (Simplivity) haven’t announced pricing.

All should be at VMworld US – so go and see them!  In the meantime, find out more:

• Scale Computing’s HC3 FAQ
• Simplivity’s OmniCube Overview
• Nutanix Complete Cluster Resources

The criteria he’s using are grouped under a very telling header, Cloud Promises, and he’s been to each provider’s web site to get the data.  Thanks, Thoran!

I hope Thoran doesn’t mind but I’m going to paraphrase his post by arranging his (very good) criteria into three major groupings – Price, Capabilities, Features.  While providers will differentiate themselves on all three, many people just consider the first aspect and this is where Thoran’s work is really useful, to open eyes to the wider understanding of what IaaS providers are giving (or not!) for the price they want.

I had two takeaways from Thoran’s work:

• If you are going to consume an IaaS service, and you are shopping around, then start with the Features, then the Capabilities, and lastly Price.  Not the other way around.  If you pick cheapest and damn the other too, then you might be hitching your wagon on a ride to the bottom, so be sure you know what you’re doing (ie. you can provide the other Capabilities/Features and you know how to exit your provider quickly should it go wrong).
• If you are going to provide an IaaS service, you should really see the Features as the area you can really differentiate.  The first two categories are very competitive today, and I find it hard to believe a new provider is going to beat AWS, Rackspace etc on Price and Capabilities without their service being a loss-leader.

Price

• Variety of pricing plans
• Average monthly price
• Cost of Outbound data transfer (how strong is their data gravity?)
• Cost of Inbound data transfer
• Storage costs (of persistent external storage)

Capabilities

Thoran has graded these as Poor, Average or Extensive.

• Scale Up (bigger VM)
• Scale Out (more VMs)
• APIs (to the VM/resources)
• Monitoring
• (Number of) data center locations
• (Number of) instance types (of VMs)
• (Number of) supported operating systems

Features

Again measured as Poor, Average or Extensive.

• Security Certifications
• Security Protection
• Open Standards – does cloud have any standards yet?
• VM Upload
• Service Age
• Service Level Agreements – this is a topic on its own!
• Support

Brian has a different style to Linus but stick with it – it’s decent content.

Some key points:

• It’s fast, and performance is key in development so you don’t get distracted while waiting for the tool SQUIRREL!
• The distributed architecture helps performance by reducing load on any central servers.
• Python can, if you’re clever, be as fast as C (disclaimer: no immediate proof!) :)
• “Collaborating with yourself as your clue develops over time”