As has been noted before, a blanket ban on legitimate scanning activity by customers of their own infrastructure (whether outsourced or not) undermines security assurance processes and can make regulatory compliance impossible; e.g. PCI DSS mandates network vulnerability scanning as a control.

Vulnerability scanning is a stalwart practice of the Information Security community. Enterprises invest considerable time and money developing vulnerability management programs to help assess IT security risk across applications and infrastructure. Specifically, vulnerability scanners help identify potential security weaknesses at scale; e.g. missing patches, default passwords, coding or configuration weaknesses.

Vulnerability scanning is front of mind for Internet exposed or partner connected infrastructure. However, when said infrastructure is owned and/or operated by a service provider, some of the existing challenges associated with vulnerability scanning are magnified:

• Scans can cause outages.  This can happen if the scanning policy includes Denial of Service checks or the scanning engine is configured with “aggressive” settings; e.g. connection entries in firewall state tables get exhausted.  Its also possible for scans to tickle obscure bugs in the target - or devices enroute to the target.  Even without a full-on outage, poorly configured scans can still negatively impact performance or availability for other customers of shared infrastructure.
• Identifying unauthorised scans. Without a trusted, robust process for “blessing” or approving source IP addresses of customer scan engines, service providers cannot distinguish legitimate scans from scans with the evil bit set.  Sure, they can use whois to determine source network ownership but even if the scan originates from a customer owned network, this does not necessarily mean it is authorised!  Given this, many providers take the stance that all scans are treated as hostile unless pre-agreed.
• Scanning may trigger automated or manual actions by the provider. A common automated response from a provider is to apply traffic shaping to slow down the scan, or simply block the client IP address via an ACL update.  This can lead to false negatives; i.e. vulnerabilities present are not discovered as the scanner IP was automagically identified as a noisy vulnerability scanner and auto-throttled/blocked.  Even half smart attackers can quickly deduces the presence of auto-response mechanisms (”huh, no response now”) so either switches to slow probes from multiple sources or goes for gold with a one-shot exploit.

Enterprise customers on dedicated infrastructure at Tier 1 web hosting providers will either contract the hosting company (or their security partner) to perform vulnerability scans or do it themselves.  Either way, for scanning to happen, agreement will need to be reached on scan scope, types of scans to be run (scanning tools & policies), time windows and source IP addresses used.  Beyond that are the process issues of how results will be communicated, integration with ticketing systems etc.

The provider will limit the scan scope to the dedicated infrastructure allocated to the customer - the scanning of shared infrastructure by the customer is generally a ‘no no’.  This, along with management networks will be scanned by the provider to meet customer compliance mandates or security policies.

With Cloud “Infrastructure as a Service” providers, things get a little more complicated.

• A cloud is multi-tenant; i.e. the cloud platform is shared to multiple customers through software abstraction.  The provider will naturally be concerned with the impact of any scanning activity, particularly if it causes any SLA violations.
• Further, cloud customers can spin up infrastructure on demand.  New virtual servers can be  brought to life automagically to handle increased load.  This increased infrastructure footprint is still subject to the same compliance mandates though; i.e. it must be scanned within some time period of its appearance.  Even if spinning up copies of “known good/secure” virtual machine (VM), you still need to scan them.   New vulnerablities are published all the time, along with corresponding vulnerability checks - hence the need for both regular scans and representative scans.  Further, vulnerbility scanning isn’t just testing the VM, its also helping you verify the security controls outside the VM that are designed to protect it; e.g. a providers’ software firewall.  Picking and choosing which pieces of your hosted infrastructure to scan is a slippery slope to selective exposure if not handled with care.
• Finally, we shouldn’t discount the “Clouding around” factor.  Credit card payments for “instant on” infrastructure changes the dynamic between cloud consumer and cloud provider.  Similar to low end, consumer oriented shared hosting before it, you may never speak with, let alone meet, an employee of your provider before you use their services.  There simply isn’t a conversation about scanning (the “conversation” today is a monologue found in the Terms of Service).  Plus, if the provider fails to meet your needs, you can drop them at a moments notice and switch to another (Cloud baggage permitting…).  In other words, its either not possible, or not convenient to call up your provider to agree the principle and logistics of scanning the services they host on your behalf.  Enterprise customers - or at least their security teams - will be wanting that conversation and can likely strike a deal with a modified ToS to allow scanning of some sort but this seems unncessarily exclusionist to me.

We can address these issues through a mix of provider open-mindedness, policy, process, technology and contract.

For cloud providers to attract certain customers, they may need to soften their policy on vulnerability scanning.  Taking a hardline “no” stance precludes some workloads from ever entering the cloudosphere (with bigger consequences for enterprises seeking a strategic cloud partner).  A preferred scenario has the cloud provider showing some understanding of enterprise prospects assurance needs and defining scanning parameters acceptable to their own operations risk tolerance.

Scanning is not an “unknown” risk, rather its a very well understood activity with quantifiable elements (packet rate, state table usage etc).  Normal rate limiting could be temporarily or permanently loosened for customer approved IP addresses to enable scans against a customers cloud IP addresses (not API endpoints or cloud providers websites!) to complete in a reasonable time window.  Besides, Internet systems are scanned, probed and attacked constantly by script kiddies, Internet surveyors and an assortment of bots and other lifeforms.  So the bad guys get to scan because they don’t care and yet the customer, who wants to do the “right thing”, is not allowed to.  Is that rational?

Assuming a cloud provider with a more measured approach towards vulnerability scanning of customer cloud infrastructure, we now need a simple, mutually trusted mechanism to agree scan sources, rate limits etc.  Something like an “ScanAuth” (Scan Authorize) API call offered by cloud providers that a customer can call with parameters for conveying source IP address(es) that will perform the scanning, and optionally a subset of their Cloud hosted IP addresses, scan start time and/or duration. This request would be signed by the customers API secret/private key as per other privileged API calls. The provider receiving the request can rely on the digital signature as proof that a scan is authorised with the associated parameters. After the provider has processed the scan authorisation request, the provider could return a status code approving or denying the request (with a possible reason code to allow resubmission with more acceptable parameters).  This response can optionally include rate limits which the customer can use to tune the intensity of their scanner.

The provider can now whitelist the customer provided scanner IP(s) for the duration of the requested scanning window such that active countermeasures like anti-DoS controls are not triggered, resulting in a ‘cleaner’ scan (and hence a more accurate report).

Should the scanning activity exceed any specified limits, or communicate with IP addresses not associated with customer virtual machines, the provider could instantly blacklist the scanning IP or apply traffic shaping.

The bottom line: when everyone is clear on the need, approval process, scan parameters and abuse policy, this can be done with very little fuss.

A “ScanAuth” API call empowers the customer (or their nominated 3rd party) to scan their hosted Cloud infrastructure confident in the knowledge they won’t fall foul of the providers Terms of Service. This avoids a situation where either a customers Cloud services are interrupted by an angry provider (availability fail!) or in the worst case, getting kicked off the Cloud entirely.  Clearly, a lose/lose scenario.

What do you think?

The scene goes something like this:

The provider rolls their eyes as yet another customer security team sends in their 500 deeply probing security questions, transmitted in some homegrown template in Word, Excel or $diety forbid, Powerpoint.  The customer security team, naturally suspicious of the provider and irked by managements apparent keenness to outsource the farm, has created the security questionnaire from hell:

• it’s the result of 100 hours of internal team meetings
• it’s gone through 14 drafts, 20 reviewers inboxes, 76 yellow highlighter comment fields and was printed at least 6 times
• it only asks IT security questions (no input from other relevant functions such legal/compliance/audit - HA!)
• it’s laced with a few tricky landmine questions based on potential security issues raised (but not satisfactorily answered) in online forums and provider support forums
• it contains 25 attachments detailing all the company security policies that *must* be followed (huh, Bluetooth policy requirements for a cloud storage provider…interesting)

In the context of cloud providers, they are slammed - a raft of audits in progress right now - with more expected soon.  The provider is experiencing an ADoS (Audit Denial of Service).  Instead of innovating new service offerings (including security!), the talented security professional at the provider is stuck cut and pasting answers from internal cheatsheets to customers questionnaires in the knowledge that the customer likely has no idea how much money it would cost to fulfill some of these security requests.  The sheer number of questions is confusing given that the customer IT team had stated they were only looking to host non-critical, non-sensitive data…

Audits are time consuming, repetitive across customers, costly and generally a motivational drain for everyone involved.  Moreover in the context of Cloud, time consuming audits seriously delays a key benefit of cloud - agility.  Its the “on demand” part of “Infrastructure on demand” that is a primary benefit of cloud.  If the security review process takes 3 months to complete, how much business opportunity has your employer lost?  Don’t like that question?  OK, another one: how much time could you have spent doing something more interesting?

Which leads me to some questions:

• what does the cost/benefit ratio look like of the “questionnaire security review method”? (to be clear, I’m not arguing against the need for security reviews)
• why do we all use different format questionnaires? (note: format)
• why are we asking these questions? (are the bulk of our questions simply an expression of our policy asked in a question format?)
• how many of these questions/policies are predictable and duplicated?  As in, you and I ask some of the same questions…we may differ in the details (e.g. password complexity..eek!) but we both probably ask the same base question even if our thresholds around answers are slightly different.
• what if we were to agree a set of common questions/policy statements?  We don’t all have to subscribe to them, we can pick the ones that reflect our policy…  There could be thousands, you search, pick and mix just like an iTunes playlist (Ed: Genius!)
• for those standard policy questions, could we “digitize” them and express them electronically?  Could the provider host a policy oracle that we could post these questions to?
• for those “uncommon” questions that the providers oracle cannot automagically answer, could we agree a standard way to “ask/transmit” those with some simple agreements about response formats? (um, freetext fields ;-).
• ultimately, could we “digitize” a significant portion of our questions to get near instant answers? (and could we make that multi-lingual…)
• would the provider recognise this as a benefit too?
• would the provider also see the legitimate opportunity this presents to charge for higher assurance services around cloud compute/storage/network based on our policy requirements?  “You want triple cycle, double buffering?  You got it - for an extra 5c per MB”).  Yes, the cost of  your security policies in a pay per drink model are revealed!
• would the provider recognise the opportunity to offer incentives to customers for choosing this low friction path of policy compliance instead of tying up their skilled employees filling out ad-hoc questionnaires?

Is there an existing system/application/protocol whereby I can transmit my policy requirements to a provider, they can respond in real-time with compliance level and any additional costs, with less structured/known requirements responded to by a human (but transmitted the same way)?  In other words, I’m looking for human driven, machine to machine policy exchange/agreement.

I propose that the benefit of quickly ascertaining policy compatibility along with any additional costs involved would reduce the on-ramp to cloud, reduce switching costs, drive a form of policy interoperability and take us closer to where we need to be in the long run: the ability to express security policy for a single unit of compute/storage/network in a cloud.  Ultimately, I want to be able to tie my security policy to the information asset I need to protect and push that to a cloud broker who performs policy reconciliation to determine which of my approved provider(s) can meet my needs without any human intervention (yeah, I can hope ;-).

And before everyone jumps on me and says ‘but the point of an on-site audit/security review is to get assurance that the provider is doing what they claim they are doing” I’d like to point out that policy and assurance are two different things.  Before you and the provider invest time in the optional on-site audit, why not get the bulk of the policy questions out the way in a fast and low cost manner? (i.e. “death to the questoinnaire?”).

If you’re following along thus far, you’ll also see the possibility for trusted 3rd party auditors to digitally ’sign’ individual policy statements made by cloud providers they have audited. That signature could itself reflect the assurance level you need.  This in turn could help drive the nascent cyberinsurance market for cloud…assuming the auditor is open to counterclaims by the insurer ;-).

If you do need to go on-site (and assuming the cloud provider tells you where “on-site” is ;-), you’ll have a list of items the provider categorically stated they do, meaning you can cherry pick the areas where you want to deep dive for assurance.  If upon inspection you find reality does not match stated policy, you can scream bloody murder.  Providers that mislead customers will soon get known.

Thoughts?

On the one hand I totally understand the need for a nation to protect its own interests - particularly where national critical infrastructure is concerned, but on the other, it “feels” a bit strange that an initiative like this is coming from a vendor with a vested interest in Cloud.

Here’s the quote:

Sun’s UK chief technology officer is working with major British public and private organisations to set up a cross-sector forum to resolve cloud-computing security issues.

Cloud-computing systems could become as important as the UK critical national infrastructure, and they need to be secured in an appropriate manner, Wayne Horkan told ZDNet UK on Thursday. The Sun executive said he is working on setting up the forum alongside organisations such as the CBI, Microsoft and Accenture; government departments such as Berr, Dius and the Treasury; and the government’s chief scientific advisor, Professor John Beddington.

“I’m concerned about the security of the supply,” Horkan said at the Cloud Expo Europe conference in London. “If cloud computing becomes a utility, it’s important to me that the UK as a nation state has good security of supply. It’s important that the UK has the appropriate capability in cloud computing.”

He then goes on to cite privacy concerns.

It’s plain to see that the majority of Cloud offerings are from US based companies.  Nearly every briefing I’m invited to is EST or PST.  In fact, I can’t remember even speaking with a UK Cloud provider.   Of the many media requests for comments, all but one were from the US.

I can’t help smelling fear in this effort. As a Brit, I would love to see a UK group coming together to innovate, support and promote the fledgling UK Cloud industry.  Perhaps that will be one of the goals of the group - if so, I don’t think that is ’security’ specific (unless we are talking security innovation).

Development of UK specific Data Privacy guidance in relation to Cloud should be led and enforced by the Information Commissioners Office.

I also feel this will do little to advance security of the Cloud overall. With the positive news yesterday that the UK based Jericho forum and the Cloud Security Alliance (CSA) have formally agreed to “work together”, isn’t this inward looking approach just fragmenting our efforts?  Why not direct the security talent that would comprise this group towards the CSA or ENISA.

Security is a *global* issue.  I’m struggling to see how country specific cloud security interest groups “fit” when we talk about globally distributed systems.  What next - Cloud UN? ;-).

I don’t disagree with the need to protect supply, but I would much prefer to see the UK government driving an initiative like this as part of their critical infrastructure protection strategy.  A strategy around UK Cloud innovation would be nice too ;-).

Perhaps I am being overly pessimistic or missing something. What do you think of a country specific Cloud security group set up by a technology company? A US based technology company no less… ;-).

Launched last month, the founders are security professionals from Cloud customers and Security in the Cloud providers (with sponsorship coming from the latter).  The Technical Adviser is friend and fellow security professional Chris Hoff.

From the Introduction on page 5 of the guidance document:

The Cloud Security Alliance is a grassroots effort to facilitate the mission to create and apply best practices to secure cloud computing. Incorporated as a not-for-profit organization, our efforts will seek to provide a voice for security practitioners. However, recognizing that a secure cloud is a shared responsibility, we will be inclusive of all organizations and points of view to fulfill this mission.
What follows is our initial report, outlining areas of concern and guidance for organizations adopting cloud computing. The intention is to provide security practitioners with a comprehensive roadmap for being proactive in developing positive and secure relationships with cloud providers. Much of this guidance is also quite relevant to the cloud provider to improve the quality and security of their service offerings. As with any initial foray, there will certainly be guidance that we could improve upon. We will quite likely modify the number of domains and change the focus of some areas of concern. We seek your help to improve this guidance to make version 2.0 of this document an even better asset to the security practitioner and cloud provider.

How To Get Involved

This is a real opportunity to shape the future security of Cloud. With sufficient participants, a mature guidance document and strong awareness, I believe a group like this can make a real impact on the future of Cloud Security. Its my view that this advances the Cloud Security conversation which is a major reason why I started this blog and will be contributing as I can.

If you’ve been sitting on the sidelines up to now, I encourage you to get involved and contribute as little or as much as you can.

Getting started is easy:

1. Join the CSA linkedin.com group to become an official member of the group (I’m already a member).

2. Review and give feedback to the CSA guidance document via the CSA Google Group.

Finally, the CSA have a number of  events planned to spread the word, including Gluecon (Denver), ISSA CISO Forum (Chicago) and the Cloud Computing Expo Europe in Prague, Czech Republic.  More info here.

Legal Cloud today announced that several top, international law firms had signed up as early testers of its virtual data center services for the legal market. The Legal Cloud is operating a ‘closed beta’ with select law firms interested in reducing the costs of their existing collocation facilities, finding a way to implement a business continuity program without duplicating private infrastructure or simply planning for their future primary and secondary infrastructure facilities.

What Makes This Different?

From their blurb:

The founders of the Legal Cloud have been working in the legal technology industry for over a decade. We understand that the needs of international law firms are different to other industries. Our data centers are optimized to meet the needs of law firms. Our choice of technologies, performance, data storage, latency, service level agreements, security and features have all been specifically devised to support the needs of the legal industry (source).

Why This Is Important From a Cloud Security Perspective?

• This cloud is designed around the needs of a specific industry:  with a well defined set of clients in mind it can cater to the groups specific operational and security needs
• These are not just “any customers”: international law firms that will have legal, compliance and security requirements over and above your “average” cloud customer today.  This needs to be a cloud with ‘higher assurance’ features to gain the trust and buy-in of legal CIOs
• The security conversation suddenly becomes a lot more focused: we are not talking about a general “one size fits all” cloud anymore and facing the disharmony of varying customers security needs and provider capabilities.  This may sound trivial but security conversations can get painful fast when customer and provider come from different worlds.
• In a view I’ve held for a longtime, its a taste of things to come: banking clouds, healthcare clouds, federal clouds (happening now).  Yes, there are other industry specific clouds (e.g. Salesforce Service Cloud) and they have their own security requirements, but arguably less assurance will be demanded by customers.
• The customers become an important lobby group for future security feature requests: instead of X voices asking for completely different things, the community of Legal Cloud users will state requirements “loud and clear” and if nScaled doesn’t listen, provide an opportunity for “Another Legal Cloud” to steal customers.
• The success of this cloud will be judged by many: if nScaled delivers on their promise, they will benefit from first mover advantage and become the “standard” for legal cloud.  From my UK experience, the legal community is cautious about new technologies and is a pretty tight-nit group, so if sufficient “established” legal firms move its not hard to imagine many more following (well, I’m sure that’s what nScaled hopes ;-).

What Is On Offer?

Legal Cloud is offering the following on a services basis:

• Fully virtualized data centers
• Business Continuity Service
• Active Cloud Servers
• Unlimited Storage
• Snapshot recovery points

And here’s how it looks from a 50,000ft:

What Do They Say About Security?

After a brave headline of “Security Guaranteed” (sure to rile anyone in information security), they go on to state:

The security of your data is of paramount importance. Here is how we guarantee it’s security.

Secure Data Centers

Our data centers are highly secure and redundant precision environments backed by the Fanatical Support of Rackspace. (SAS-70 Compliant)

Secure Virtual Private Networks

We extend your network into the Legal Cloud using VPN (Virtual Private Network) and VLAN (Virtual LAN) technologies. Your data is encrypted during transit with IPsec. Within the Legal Cloud, your data is segregated in logically separate areas from other clients data and attached only to your private networking. This gives each client their own private network and storage in the Cloud.

Data Encryption

Client Data is encrypted from client source servers to target devices using strong encryption protocols.

Not on the public Internet

The legal Cloud is not exposed to the public Internet. It is actually an extention of each clients internal network, each seperated by strong security protocols.

Service Level Agreements

We are working on appropriate SLA’s for our legal customers during the beta period.

Psychologically, I suspect the most significant reassurance for many CISOs will be this one single sentence: “Not on the public Internet”.  Beyond that, use of IPsec will make this feel very much like a standard 3rd party ‘partner’ connection.  I don’t see any mention of storage encryption options as yet, nor any further detail on the logical separation - once I’ve had a briefing and can speak more to the security aspects, I’ll post more.

P.S nScaled have annouced a couple of webinars aimed at their target audience here.

From comments I received afterwards, I got positive feedback despite running out of time (my fault entirely).  I’ve been pleasantly surprised by the number of people asking for copies of the slides, but do bear in mind the slides are somewhat ‘terse’ as they are primarily talking points for me to bounce off of (as it were).

Should anything not be clear, feel free to leave a comment below and I’ll do my best to clarify.

I’d also like to take this chance to thank Jeff Moss, Ping and the rest of the Black Hat crew for doing such a professional job running the conference - it was confidence inspiring to be in such capable hands.

Introducing the slide, I remarked that its important to differentiate the two:

• “Cloud Security”: this refers to the security of “the Cloud”, or more usefully, of a given cloud.  Stepping back, we can use the term to refer to the general security aspects of Cloud Computing.
• “Security in the Cloud”: this is about delivering security services via “the cloud”.

Back in April 2008, when I was naming this blog, I initially planned to call it ‘Security in the Cloud’ but after 30 minutes of Googling and reading, it became evident that I was mistaken as this term had already been adopted to refer to services delivered via the Internet (primarily Security MSSPs).  Hence cloudsecurity.org was born.

Having said all that, I’m now seeing newer “security in the Cloud” providers referring to themselves as ‘the Cloud Security Leader’ which only serves to add to the confusion.

[This post was inspired by "The Real Meaning of Cloud Security Revealed" by Lori MacVittie]

Now’s your chance to express them…

ENISA (the European Network and Information Security Agency) is conducting a security risk assessment of cloud computing.

If ENISA is unfamiliar to you, here’s how they describe themselves:

• Is a Centre of Expertise for the EU Member States and EU Institutions in Network and Information Security, giving expert advice and recommendations
• Is a switchboard of information for best practices
• Facilitates contacts between the EU-institutions, the Members States and the private business & industry actors

For the Cloud Risk Assessment, the group (of which I’m a member) will focus on three scenarios:

1. A user perspective on Cloud Computing (i.e. Small and Medium Enterprises)
2. Cloud Computing in a eGovernment environment (i.e. national health service)
3. Cloud Computing and Resilience

In pursuit of the first scenario, ENISA is seeking feedback:

“…aimed at giving advice to (among others) SME’s on the most important risks in adopting cloud computing technologies, as well as ways to address those risks.

As part of this study, we want to look in detail at the perspective of SME end-users of cloud computing infrastructures and applications (either current users or those considering adoption). As a first step, we have decided to base our study on a survey of the actual needs, requirements and expectations for cloud computing infrastructures.”

Take the 10 minute survey here (results will be shared).

This post is a summary of a very interesting call I had with George Reese, CTO of enStratus and author of the forthcoming “Cloud Application Architectures” book.  Please note: this isn’t a comprehensive review of the full service, rather it reflects the pieces that we delved into based on some of the common concerns we have around Cloud Security (to give you some idea, we spoke for over 90 minutes…).

enStratus offers cloud infrastructure management tools “aimed at the needs of enterprise IT”.  Today, they support Amazon EC2, with support for other clouds to follow.

Their tag line is ‘Confidence in the Cloud’ and their offering focuses on 3 key areas addressing the twin cloud adoption barriers of security and reliability:

• protecting cloud based data through encryption
• offering service levels above that of the underlying cloud provider (99.9999% for EC2)
• achieving Recovery Time and Recovery Point Objectives “in the face of the most extreme disasters”.

George outlined 3 concerns his customers have about cloud providers such as Amazon:

• Amazon controls the physical systems on which the data resides, meaning Amazon malfeasance, Amazon misfeasance, or even 3rd party subpoenas put that data at risk.
• The complexity of resource orchestration in the context of credential management; i.e. when do your credentials need to be in the cloud versus when their presence is just a security risk
• User management, even via traditional identity management systems, can be dysfunctional.

The enStratus Approach to Cloud Key Management

One of my pet peeves with AWS is the “one key to rule them all” security model (the dysfunctional user management George alluded to).  Any disclosure of that key results in an attacker gaining access to all your infrastructure.  But to make privileged API calls, every developer must have a copy of the key…

Its not unknown for AWS users bundling an AMI (creating a virtual machine image) for public consumption to leave their AWS credentials in the AMI itself.  Oops. This is obviously a Bad Thing ™ as a malicious user that opts to use that AMI can steal their access key, gain access to their Amazon hosted infrastructure and run up bills in their name.

One of the things I really like about the enStratus offering, is the relentless focus on controlling the use and hence exposure of a customers’ ‘cloud masterkeys’. Their implementation keeps the keys away from the AMI, and therefore the cloud, PLUS out of the hands of an org’s IT/dev people.

enStratus acts as a trust broker. After signing up for the service, the customer loads their “all powerful” Amazon credentials via a shared enStratus Provisioning Server into a Credentials Server (no direct internet connectivity).

From that point forward, the customers’ IT people access the enStratus service and manage their cloud infrastructure via named user accounts assigned specific privilege levels.

Permissions include;

• administrator
• start/stop servers
• uptime retrieval and
• audit trail review.

Non-administrative users have no direct access to the AWS keys.

Here’s a peek at the architecture of enStratus.

When an authorised enStratus user issues cloud infrastructure management requests via the Web Services and Console server, the Provisioning server issues the cloud API calls on behalf of the users.  This eliminates the need for every user needing a copy of the key to sign requests.  Given they are mediating API requests, adding logging functionality was a no-brainer and means the next time you need to know ‘who spun up that unpatched AMI image with an allow-all security group?’, you can find out.

Its important to note that there is nothing preventing anyone with your AWS key from just making API calls directly to the AWS API endpoint - totally bypassing the enStratus infrastructure.  Therefore, careful key lifecycle management is still necessary; i.e. load fresh AWS credentials straight into enStratus and follow a “no sharing” policy.

I should point out that the EC2 ecosystem players can’t do anything about this as the AWS platform doesn’t offer a mechanism to tie IP level controls to AWS key usage or EC2 (yet).  One way Amazon could implement this (nothing announced) is with their new JSON based Access Policy Language.  Despite this, enStratus can still detect new EC2 instances spun-up by API calls they didn’t mediate, through telemetry used for operational monitoring - they just won’t be able to tell you who started it.

enStratus can help customers build their AMIs, including bundling in HIDS (Host based Intrusion Detection) via ossec, with centralised agent reporting.  Another example of how they protect the AMI key is through error checking in their scripted AMI builds to ensure key material is not left in an AMI accidently.  In addition, users are prevented from accessing partially provisioned AMIs (to eliminate potential key snarfing shenanigans).

Root access to EC2 images is disabled by default (unlike with vanilla EC2).  Privileged access is granted via sudo.

Filesystem Encryption

enStratus offers optional filesystem encryption through a checkbox.  Keys are temporarily passed into the EC2 instance when required; i.e. mounting.

Encrypted filesystem support is implemented via 2 block volumes configured as RAID 0.  2 sets of encryption keys are used.  One for encrypting and remounting the ephemeral drive (this is a “non-persistent store” automagically attached by EC2 to each running AMI).  The second key pair is used to encrypt and mount filesystems attached as Elastic Block Storage (EBS).  EBS is off-instance, persistent storage.  To avoid potential exposure of key material, the 2nd set of keys are stored on the encrypted ephemeral drive during mount.

Worth noting is that in testing, George found that 2 EBS volumes, configured as RAID 0 with an encrypted XFS filesystem  offers similar performance to a single, unencrypted EBS volume with an ext3 filesystem.

George is keen to stress that enStratus is not looking to control both customers data and their keys.  So whilst he recommends and can help customers make use of the EBS snapshot feature to clone/backup storage volumes to Amazon S3 (Simple Storage Service), he isn’t offering a hosted backup service (to avoid a potential conflict).  Of course, an evil and privileged enStratus employee could access your live data as the keys are stored in their Credential server.  Today though, enStratus is a small company so figuring out ‘who dunnit’ would not require the services of Sherlock Holmes.

Futures

Today, the enStratus management infrastructure sits outside of the cloud (at a colo) for cloud monitoring and isolation reasons.  George is exploring the possibility of also offering an on-premise offering for customers wishing for more control.

I offered a few short and medium term suggestions around additional integrity checks, encryption ideas, assurance processes (source code security reviews, penetration testing) and consideration to using a Hardware Security Module (HSM) for key storage to further bolster both security and trust.  George seemed genuinely open and receptive to these ideas and also shared a few interesting customer requests they are actively working on today.  Some of the more expensive line items would become practical if they can attract additional funding.

Overall, I have to say I’m impressed with their approach, technology and attitude.  Definitely worth a hands-on evaluation if current Cloud controls don’t fall within your risk tolerance.

Good luck to George and the rest of the enStratus team as they prepare to present at Under The Radar!

Attending Under The Radar?

As a special offer to cloudsecurity.org readers, the organisors of Under The Radar are offering $100 off the list price for VIP tickets.  To claim yours, click here.

A Quick SQS Reminder

For those unfamiliar with SQS, here’s the elevator pitch from Amazon:

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. Amazon SQS makes it easy to build an automated workflow, working in close conjunction with the Amazon Elastic Compute Cloud EC2 and the other AWS infrastructure web services.

Amazon SQS works by exposing Amazon’s web-scale messaging infrastructure as a web service. Any computer on the Internet can add or read messages without any installed software or special firewall configurations. Components of applications using Amazon SQS can run independently, and do not need to be on the same network, developed with the same technologies, or running at the same time.

So, a very handy data structure that makes perfect sense in distributed programming.  However, access control options were limited…until today.

The New Secret Sauce

AWS is also introducing additional permission features that control access to Amazon SQS and to each of its fundamental actions on a very fine-grained basis. You can exercise this control at two levels:

* At the higher level you can use the new AddPermission and RemovePermission functions to set and remove particular access rights for each queue. Access rights, including the ability to send, receive, or delete messages, change message visibility, or to retrieve queue attributes, can be granted to any AWS user via their AWS account number.
* At the lower level you can use our new Access Policy Language. This expressive language makes its debut as part of this SQS release; over time, we plan to employ this Access Policy Language with our other services. The Access Policy Language enables the creation of complex rules to enable access to queues based on identity (AWS account number), source IP address, date, time, and more.

With this new permission system you can now use Amazon SQS queues to connect non-AWS applications to AWS applications and to connect AWS applications from different organizations. You could use an open, public queue as a drop box, allowing outside applications to submit work items for processing. This could be a fully public drop box, or it could be limited to requests from a single country by using a policy based on an IP address or address range. Communication between organizations can be established based on IP addresses or AWS accounts, as appropriate.

For me, the most significant news is not so much that SQS now has fine grained access control, but that Amazon have introduced a Access Policy Language and they plan to apply it to other AWS services.  This is a very positive development and could be the mechanism they use to overcome some of the longstanding security concerns I blogged about recently.

Architectural Overview

For the visually inclined (source):

Where:

1. You, the resource owner.

2. Your resources (contained within the AWS service; e.g., SQS queues).

3. Your policies.  Typically you have one policy per resource, although you could have multiple. The AWS service itself provides an API you use to upload and manage your policies. For information about the content of the policies, see How to Write a Policy.

4. Requesters and their incoming requests to the AWS service.

5. The access policy language evaluation code.

This is the set of code within the AWS service that evaluates incoming requests against the applicable policies and determines whether the requester is allowed access to the resource. For information about how the service makes the decision, see Evaluation Logic (Ed: note there are soft and hard denials).

An Example

Here’s an example from the developer docs showing a simple IP based control (multiple controls can be applied to a single resource):

The following example policy gives all users permission to use all possible SQS actions that can be shared for the queue named 987654321098/queue1, but only if the request comes from the 192.168.143.0/24 range.

{
  "Version": "2008-10-17",
  "Id": "Queue1_Policy_UUID",
  "Statement":
     {
        "Sid":"Queue1_AnonymousAccess_AllActions_WhitelistIP",
        "Effect": "Allow",
        "Principal": {
           "AWS": "*"
         },
        "Action": "SQS:*",
        "Resource": "/987654321098/queue1",
        "Condition" : {
            "IpAddress" : {
               "SourceIP":"192.168.143.0/24"
            }
     }
}

Conclusion

Notice the values for the ‘Action’ and ‘Resource’ tags.  Now imagine those with different AWS service identifiers and resource types and things start to get really interesting.

Now all we need is an user-friendly, hard-to-shoot-yourself-in-the-foot policy generator/front-end to open this feature up to the masses.

All in all, its great to see the introduction of a consistent policy language from the cloud pioneer.

I’m off to learn more about the language…

Update: in case it isn’t obvious from the example, the policy language is expressed using JSON (thanks @lmacvittie for the prompt)