Versioning For All

To begin with, the new S3 beta versioning feature is now available in all regions. This means that you can retain past versions of all your S3 objects regardless of where your bucket is located.

The latest JetS3t code has full support for versioning that makes it very easy to use. You can enable versioning for a bucket like so:

Then should you ever need to recover some data — such as after accidentally deleting an object or overwriting data with a corrupted file — you can find and retrieve the prior versions:

The Second Factor

As well as rolling out broader availability of the versioning feature Amazon has (somewhat quietly) added another interesting feature: the first API-level support for multi-factor authentication. Multi-factor authentication (MFA) adds an extra level of security to systems by requiring users to prove ownership of a token or device of some kind in addition to their normal login credentials. This means that even if someone steals or guesses your credentials they will be unable to perform actions on your account because they do not possess the device.

In Amazon’s case, like PayPal and some banks before them, the additional factor comprises a small electronic device that generates code numbers. Once you have purchased one of these devices and enabled it in your AWS account you will be required to provide an extra code number when performing certain tasks.

Previously the additional MFA device code was only required when you logged in to the AWS Console but as of today you can turn on MFA for your S3 buckets in tandem with versioning. When versioning with MFA is enabled not only will the bucket’s owner be the only user who can permanently delete object versions, but this user will be required to provide a time-limited MFA code to do so.

Again, this is relatively straight-forward to use in JetS3t:

The addition of MFA support at the API level in S3 is particularly interesting because this is the first time Amazon has done so, and because it raises some interesting challenges for developers who are accustomed to building fully-automated systems. To take advantage of the protection the MFA provides a system will need to prompt the user for her MFA code every 30 seconds or so when she wishes to permanently delete data. I am keen to see how — and if — developers actually build this feature into their applications.

Hello BitBucket

Finally, repeating the news I posted recently on the JetS3t discussion forums, I have decided to move the JetS3t codebase from it’s old home at java.net over to the BitBucket service: http://bitbucket.org/jmurty/jets3t/

BitBucket has the advantage of being a more modern, easy-to-navigate site, and has seamless support for Mercurial which is my favorite source code management tool. So it’s farewell to java.net and CVS, you served us well but it’s time for some new blood.

Try out the latest code and let me know what you think. Head over to the JetS3t BitBucket repository and grab the latest code via a pull (if you’re familiar with Mercurial) or simply download it via the “get source” link.

Once you enable versioning for one of your S3 buckets, any time you change an object in that bucket a version of the prior object will be stored in addition to the latest one. You can then perform operations on prior object versions such as retrieving older data, restoring “deleted” objects, and generally maintaining a fail-safe history of everything that happens in the bucket.

This will be a boon to anyone who is worried about their S3 data being accidentally deleted or corrupted by user/computer error.

The feature is currently in early beta form and is available for testing with buckets located in the “us-west-1″ location. You can read about the current functionality here: Versioning Beta Design.

Better still, you can grab the latest JetS3t code from CVS and try it out for yourself! The code samples file CodeSamples.java now includes a section called “Bucket Versioning (Beta)” to get you started.

Both the versioning feature itself and JetS3t’s support for it are in an early stage so watch out for warts.

This release includes some bug fixes, more sophisticated configuration options for the “filecomparer” component that manages file synchronizations, and supports the two major new CloudFront API features: private distributions and streaming distributions.

Visit the JetS3t web site to download the latest release and view the latest documentation such as code samples and the API Javadoc.

You can read about the complete list of changes in the release notes. And for the Maven-ites among you the official Maven2 repository has also been updated.

This feature allows you to control access to S3 objects you distribute through CloudFront by making them available only through specific distributions, or by requiring the use of signed URLs that you generate and provide to privileged users.

As of this evening the latest JetS3t codebase (available from the CVS repository) has full support for the new features, including the ability to:

• create and update private distributions
• manage Origin Access Identifiers, which are required for private distributions
• generate canned and custom-policy signed URLs for private distributions that require request signing.

These new features are not yet available in a stable packaged release but I plan to provide the next stable version before the end of November.

The Cloud Servers product allows you to rent computing resources and is a competitor to Amazon’s Elastic Compute Cloud (EC2) service. Rackspace has a comparison page that describes, from their perspective, the advantages of their offering over EC2.

Some key differences from EC2 include:

• Cloud Servers has a wider range of server sizes available at the low end, with a minimum size of 256MB RAM that has a price of only $10.95 per month.
• Public IP addresses can be shared among multiple servers.
• The service supports dynamic resizing (vertical scaling) of servers to a degree. Unlike EC2, you can increase or decrease the computing power available to a single server without the need to manually start a new instance and redeploy your application to the new instance. However this scaling, while easy, isn’t instantaneous — behind the scenes Rackspace’s service actually starts a new server and copies everything across for you, so there is likely to be some downtime.
• A simpler RESTful API with support for JSON messages in addition to XML.

I am not yet familiar enough with Cloud Servers to give a detailed comparison with EC2, but it seems to be a full-featured service that is aiming to address some of the difficulties people face when using Amazon’s offering. If Rackspace can learn from Amazon’s missteps they should be able to provide a compelling cloud computing platform.

It has taken some time for a strong, low-level “Infrastructure as a Service” competitor to EC2 to arrive, but we may finally have it in Cloud Servers. I hope so, because the more active competition we have in this space the more quickly the products and technology will improve, and the better off we cloud computing users will be.

I guess that means that not only has the designation “beta” been stripped of any real meaning on the modern web, but even the decision to leave beta status can be taken on a whim. Perhaps there is a tangible reason — aside from marketing towards the big end of town — that prompted Google to remove the beta status from their products today. Perhaps. But if so, the reason isn’t explained in these blog posts.

Don’t get me wrong, I am a long-time user of both of these products and I consider them invaluable tools of very high quality. The tech is great. I just get annoyed when large companies, through carelessness or malice, misuse words and destroy their semantic value.

You can read about this new feature on the AWS Blog and watch an introductory video here.

Unified Identity

A single account (identity) with which you can log in to many sites, removing the need to create and remember a separate username/login for every web site you interact with.

Openness

A decentralized authentication system with multiple providers. This means that you can choose a provider (or even a few) from the many options available to vouch for your identity, and switch providers if you find a better one. Or you can even be your own provider.

Delegation

I think Delegation is the most attractive feature of OpenID because it means your own web site can act as your identity, while delegating the authentication process to one (or more) OpenID providers.

In short, with delegation you can log in to sites using a URL you own like jamesmurty.com, while taking advantage of the strong authentication options offered by providers such as Verisign’s PIP. Although my Verisign PIP identity happens to be jmurty.pip.verisignlabs.com, I can use my own web site as an alias for this provider-specific identity.

By decoupling your identity from your OpenID provider you can take advantage of the fact there are many providers and easily switch providers later on without losing your identity, and without having to update your associated OpenID identity on every web site. After all, if you had to do that you might as well have created your own username/password on every site in the first place.

But…

Unfortunately, the complexity of OpenID and the challenge ordinary people can have getting it to work properly is preventing widespread adoption of the system in general, and of the Openness and Delegation features in particular. Although big players like Google and Yahoo are supporting (parts of) the specification, they are understandably encouraging people to adopt their branded OpenID identities rather than extolling the advantages of controlling your own identity.

After all, every web company would love to take on the “burden” of managing your unified web identity. It’s the ultimate in vendor lock-in.

Setup OpenID delegation for your web site

If you have your own web site or blog and are able to edit the HTML pages directly, you can set up delegation by adding special

tags to the

section of one of your site’s pages. You will most likely want to do this on the site’s home page so you can use a short URL like jamesmurty.com instead of jamesmurty.com/my-openid-page.html.

Below are the

tags I use on my site to delegate to my jmurty Verisign PIP identity. You will need to use your own provider-specific identity URL in your links, and the format could vary quite a lot depending on the OpenID provider you choose so check your provider’s documentation. Also, I’m not sure that all OpenID providers actually support delegation, so you should research this before you sign up with a provider.

It is important that these

tags be included inside a valid HTML

section in your web page, or many web sites will be unable to find your delegate settings.

More Complexity, aka Taming Blogger.com

You may have noticed that the OpenID information is provided twice, once for the original OpenID specification (

tags) and again for version 2 of the spec (

tags).

I don’t know why the second lot of settings is necessary, since presumably the spec is supposed to be backwards-compatible, but I have found that some sites won’t work properly unless the version 2 settings are provided.

One example of version incompatibility quirks is Google’s Blogger.com, which allows you to comment on blog posts after logging in with an OpenID. Prior to adding the

tags I found that although Blogger would allow me to authenticate and post comments, it would replace my delegating identity jamesmurty.com with the delegated version jmurty.pip.verisignlabs.com. This meant that the delegation was essentially useless, since anyone clicking on the nickname for my comment would end up at an empty Verisign PIP landing page instead of my own site.

I’m not sure if this is Google’s fault, or a fault in the OpenID spec. Either way it was annoying having to track down and fixing this issue. It just serves as yet another example where OpenID is not quite living up to the promise of simplifying identity management.

First, here’s a reminder of what this project does:

XML Builder is a utility that creates simple XML documents using relatively sparse Java code.

It is intended to allow for quick and painless creation of XML documents where you might otherwise be tempted to use concatenated strings, and where you would rather not face the tedium and verbosity of coding with JAXP.

The new features include:

• Parse existing documents into an XMLBuilder object, so you can now easily add nodes to pre-existing documents.
• Use XPath queries to locate a specific element in your document. This is especially useful if you have parsed a document and you need to add new nodes at different locations in the DOM. Type in your XPath query and you can now jump directly to the right place.
• The project now has a Maven-friendly structure, complete with a repository from which you can obtain the Jar file — see the Downloads section on the project page for instructions. This great leap forwards is thanks to Dan Brown’s instructions for using Wagon to deploy Maven artifacts to Google’s SVN.
• JUnit tests are now public in the repository, to help keep me honest.

People familiar with the project may notice that I have changed the version numbering scheme. The latest version is 0.3, not 3. I think the “0.” prefix better indicates the maturity of this tool.

There are some real gems here, such as:

• [They all] used Amazon services, and most if not all of them seemed to use RightScale to manage them.
• Cost: cloud is more expensive than real machines. Cloud is good for elastic computing, not for high constant demand.
• You need monitoring services external to your cloud!