• AWS does not force developers to use any particular programming model, language, or operating system.
• AWS does not force developers to use the entire suite of services - they can use any of our infrastructure services individually or in any combination.
• AWS does not limit developers to a pre-set amount of storage, bandwidth, or computing resources they can consume - they can use as much or as little as they wish, and only pay for what they use.

Our customers love this flexibility. Today, a developer can run more experiments and achieve results much faster than before. If something does not work in a particular environment, the developer can drop that idea, click a few buttons, dispose all of his infrastructure and move on to the next experiment; starting with a fresh, new environment. Developers can try out several new ideas simultaneously by running multiple projects concurrently. Once the ideas are implemented, they can be further battle-tested using more resources in the AWS cloud until they become finished products. Developers love this because they are able to convert their concept/idea into a successful finished product quickly. As a result, we are seeing tremendous innovation happening at break-neck speed. The Cloud is becoming a platform for Innovation. 

The inherent flexibility of the AWS cloud enables customers to use it as a Platform in variety of different ways, including: 

• The AWS cloud as a Platform for Collaboration
• The AWS cloud as a Platform for Computation
• The AWS cloud as a Platform for Software and Data Delivery
• The AWS cloud as a Platform for Hot and Cold Storage
• The AWS cloud as a Platform for Research Development and Experimentation

Every day, I find customers in each of categories mentioned above. Some of them share their stories and architectures with us.

It does not stop there!

Its inspiring when I see the AWS cloud being used as a Platform for Platforms.

AWS is not only a rich platform to build solutions but also a platform for building specialized platforms. Customers can choose to either use the AWS cloud directly or take advantage of these value-added platforms. Customers can also mix and match platforms from this rich ecosystem.  

In this post, we look at some of the best examples of specialized platforms built on AWS:

Ruby Platforms
Heroku, as most of you may already know, is one of the early platforms built on top of Amazon Web Services. This "Instant Ruby Platform" enables any Ruby developer to take their existing Ruby code and move it to cloud. Customers of Heroku do not have to worry about scaling or managing their server farm, in case there is a success disaster. Heroku deploys a Ruby app in a single step without changing the app or the process. They recently launched commercially and offer a similar pay-as you-go pricing for enterprises and hobbyists. They also offer a free tier to try and test your prototypes.  By offering Deployment platform-as-a-Service on the top of reliable Amazon's platform, everybody wins.  The end-user gets all the things they need to build a modern web-scale application quickly while Heroku manages the "magic" (via “Slugs” and “Dynos”) of Ruby deployment without worrying about the complexity of maintaining and managing the underlying infrastructure.

Engine Yard offers a rich and open Ruby deployment platform on Amazon Web Services. Developers can take advantage of Engine Yard's pre-configured standardized stacks which makes it easy to deploy a Ruby on Rails application. Using Engine Yard's wizard-style web interface, developers can create the entire environment including different Unix packages and Ruby Gems to install, setting frequencies of database backups. They have a nice video on How to deploy your Ruby app on EC2 in 10 minutes. 

Language-agnostic Development Platform
I stumbled upon CodeRun – Online Development Platform on Amazon EC2 – few months ago and was tracking it closely. The interface looked just like Visual Studio but in-browser. End-user can code in PHP, AJAX, ASP.NET in an in-browser IDE which is fully hosted on multiple Amazon EC2 instances and then deploy the code (by clicking on “Debug” or “Run”) using several backend services that in-turn run on various other Amazon EC2 instances (all managed). Free accounts may share instances while premium accounts (which will be available in August) will run on stand-alone instances. Developers may deploy their code to Amazon EC2 more than once for testing, debugging and production purposes. Code snippets are shared on Amazon S3 and can be shared among developers (Check out AWS Code Samples). Amazon EBS is used to store users files and data while Amazon CloudFront is used distribute static files. Logs are stored on Amazon SimpleDB. They almost have a full house!

The platform includes a custom elasticity mechanism that monitors resource usage and performs automatic scaling based on dynamic set of predefined business rules. Developers can code in existing technologies (.NET/PHP/JS) and “outsource” scalability to CodeRun. With a single-click, you can deploy your app. CodeRun leverages AWS API to completely automate the deployment process. This includes allocating resources (instances, addresses and storage), copying files, synchronizing database structure and configuring the web servers. The entire platform is not fully baked yet but, I think, it has tremendous potential.

Voice Platform
Twilio is a voice platform built on the top of AWS. As Twilio’s website suggests, developer could build innovative voice apps like sales automation systems, order inquiry lines, CRM solutions, call routing apps, appointment reminders, custom voicemail apps. Platform developers at Twilio are focusing on building a powerful telephony platform on the top of Amazon Web Services. Twilio is drop-dead simple and easy to get started (“friction-free development”). With this simplicity, I think, it won’t be too long until a developer will be able to write a phone tree app that calls up all your friends from your social network about an upcoming party and get their RSVP over phone which can then be viewed on a website. Take a look at Twilio’s presentation (from the AWS Start-Up Tour in Seattle), and you will be convinced that they are AWS experts and they know what they are doing.

At VoiceCon, Siemens Enterprise Communications pre-announced that their Voice and Unified Communications product suite will be available as-a-Service on the AWS cloud. It will be interesting to see how this platform evolves.

Bottom line
Heroku, Engine Yard, Twilio, CodeRun are all different in nature and behavior. All of them are built using different technologies and methodologies. All are targeting different market segments. All share one thing in common. They are all built on AWS. All of them are built to scale and take advantage of flexibility. Innovation thrives in an environment that permits flexibility. AWS gives them the flexibility they need along with the scalability and elasticity their customers require.

This, to me, is very inspiring. What do you think ?

- Jinesh Varia (evangelists at amazon dot com)

Here’s another story that I found fascinating: nominally it is about how The Server Labs uses Amazon Web Services as a scale-out solution that also implements Oracle databases; however it’s really about space exploration (or should I say “nebula computing”). It began with an email asking whether there would be a problem running up to 1,000 Amazon EC2 High-CPU Extra-Large instances.

The Server Labs is a software development/consulting group based in Spain and the UK that works closely with the European Space Agency, and they needed to prove the scalability of an application that they helped build for ESA's Gaia project. In addition to the instances, they also requested 2 large and 3 X-Large instances to host Oracle databases that coordinate the work being performed by the high-CPU instances.

Gaia’s goal is to make the largest, most precise three-dimensional map of our Galaxy by surveying an unprecedented number of stars - more than one billion. This, by the way, is less than 1% of all stars! The plan is to launch a mission in 2011, collect data until 2017; and then publish a completed catalog no later than 2019.

I had the opportunity to see a PowerPoint deck created and presented by The Server Lab’s founder, Paul Parsons, and their software architect, Alfonso Olias, who is currently assigned to this project.

The deck explained that the expected number of samples in Gaia is 1 billion stars x 80 observations x 10 readouts, which is approximately equal to 1 x 1012 samples—or as much as 42 GB per day transferred back to Earth. There’s a slide in the deck that says “Put another way, if it took 1 millisecond to process one image, the processing time for just one pass through the data on a single processor) would take 30 years.”

As the spacecraft travels, it will continuously scan the sky in 0.7 degree arcs, sending the data back to Earth. Some involved algorithms will come into play in order to process the data; and the result is a fairly complex computing architecture that is linked to an Oracle database. Scheduling the cluster of computational servers is not quite so complicated, and is based on a scheduler that is focused on keeping each machine as busy as possible.

However the amount of data to process is not steady—it will increase over time. Which means that infrastructure needs will also vary over time. And of course idle computing capacity is deadly to a budget.

The opportunity to solve large computational problems usually turns to grid computing. No difference this time either – except that as mentioned above, the required size of the grid is not constant. Because Amazon Web Services is on-demand, it’s possible to apply just enough computational resources to the problem at any given time.

In their test, The Server Labs set up an Oracle database using an AWS Large Instance running a pre-defined public AMI. Then they mounted 5 EBS volumes of 100 GB each, and mounted them to the instance.

Then they created Amazon Machine Images (AMIs) to run the actual analysis software. These images were based on large instances and included Java, Tomcat, the AGIS software and an rc.local script to self-configure an instance when it’s launched.

The requirements break down as follows:

To process 5 years of data for 2 million stars, they will need to run 24 iterations of 100 minutes each, which works out to 40 hours running a grid of 20 Amazon EC2 instances. A secondary update has to be run once and requires 30 minutes per run, or 5 hours running a grid of 20 EC2 instances.

For the full 1 billion star project numbers extrapolate out more or less as follows: They calculated that they will analyze 100 million primary stars, plus 6 years of data, which will require a total of 16,200 hours of a 20-node EC2 cluster. That’s an estimated total computing cost of 344,000 Euros. By comparison, an in-house solution would cost roughly 720,000 EUR (at today’s prices) – which doesn’t include electricity or storage or sys-admin costs. (Storage alone would be an additional 100,000 EUR.)

It’s really exciting to see the Cloud used in this manner; especially when you realize that an entire set of problem solutions that were beyond economic possibility before the Cloud became a reality.

Mike

Amazon Web Services

Along with Michael Crandell and Edward Goldberg of RightScale, Simone Brunozzi of Amazon Web Services and Patrick Kerpan of CohesiveFT will show you how you can save time and money by running your entire testing application testing infrastructure in the cloud. They will discuss creation of an agile approach to rapid prototyping, creation of a test and development environment which replicates the final deployment environment, and will also show how to build a secure VPN environment.

The webinar is free but registration is required.

-- Jeff;

After selecting the tab you will see a list of all of your CloudFront distributions:

We've also got a brand new getting started video.

This is just one of many exciting new features that we have in store for users of the console. Stay tuned to this blog for further developments.

-- Jeff;

A product of Alamofire, Gowalla runs on EC2 and makes extensive use of S3, SQS, and SimpleDB. To learn more, you can watch Alamofire founder Scott Raymond's presentation at the Scotland on Rails conference. In the video, Scott talks about the pros and cons of using AWS, how they scaled their MySQL database tier, Facebook interaction, and much more.

-- Jeff;

The service is free for the first month and then $5 per month thereafter.

-- Jeff;

The new version is hosted on EC2 and includes advanced search features such as relevance ranking, clustering (by topic, author, publication, or date), and exporting of search results in popular citation formats. The site indexes content in 15 different topic areas including agricultural sciences, astronomy & space, biology & nature, chemistry, computers & technology, defense technologies, earth & environmental sciences, energy, health & medicine, materials science, physics, and mathematics. It also indexes patents and science news.

Abe will be presenting a paper titled Journey to Ten Thousand Sources at the SLA (Special Libraries Association) conference this week. The paper contains a lot of interesting information about the architecture of this highly scalable federated search engine including details of the ways in which queries are partitioned into batches and then spread across multiple crawlers.

-- Jeff;

The HITS come with some fairly specific requirements, so please read the instructions with care. You can get started here or by searching the HITS for the word 'favorites'.

-- Jeff;

The following paragraphs are straight out of Paul’s full blog post, which of course you should read in its entirety if this topic is relevant to your interests.

Oracle recently made available a set of AMI images suitable for use with the Amazon EC2 cloud computing platform. I found the two images (32-bit and 64-bit) that contain Weblogic (along with Oracle Enterprise Linux 5 and JRockit) the most interesting of the lot. This article will explain how to set up a basic two-node Weblogic cluster using the 32-bit Weblogic image provided by Oracle with an Amazon Elastic Load Balancer (ELB). In future articles, I will demonstrate how to set up a more complicated cluster with Apache Web Server instances balancing the load between many weblogic cluster members.

You can set up a Weblogic cluster in EC2 in very little time which makes it great for testing complicated Weblogic setups without having to mess around on your local machine or trying to scrape together the necessary hardware. This type of configuration would also be suitable for deploying a production application, though you’d have to check the licensing implications with Oracle if you wanted to do this.

Note that this article assumes a basic level of familiarity with using Amazon web services.

Mike

The EC2 tab of the console has a new button:

You can see your existing set of Reserved Instances:

And you can purchase additional Reserved Instances:

-- Jeff;