• Important updates to our Security Center including a revised AWS Security White Paper and the results of our SAS 70 type II audit.
• A plan to bring AWS to datacenters in Asia.
• Selection of the finalists in the 2009 Start-Up Challenge. Congratulations to Bizo, FlightCaster, Gazaro, GoodData, Involver, Motally, and ReTel Technologies. The final event will take place on December 9th in Sunnyvale, California. Register if you'd like to attend.
• The AWS SDK for .NET Developers.
• Brand new Developer Centers for .NET, PHP, Ruby, and Java.
• Support for Amazon CloudFront Private Content.
• The worldwide availability of the MFA (Multi-Factor Authentication) devices. Learn more about MFA or just go ahead and purchase a device.

-- Jeff;

PS - As you can see, we've been kind of busy, but we're not done yet. We have plenty of work to do, and a veritable panoply of open positions on the AWS team

The SDK includes the following goodies:

• A new AWS .NET Library - This library provides a set of developer-friendly APIs that hide much of the low-level plumbing associated with programming for the AWS cloud, including authentication, retries, and error handing. The library supports the Amazon Elastic Compute Cloud, Auto Scaling, Amazon CloudWatch, Elastic Load Balancing, the Amazon Virtual Private Cloud, Amazon SimpleDB, the Amazon Simple Storage Service, Amazon CloudFront, the Amazon Simple Queue Service, the Amazon Relational Database Service, and Amazon Elastic MapReduce.
• Code Samples - The SDK includes practical examples in C# to show you how to build applications that make use of AWS. 
• Visual Studio Support - The SDK includes Visual Studio templates to give you a running start on your application.
• Documentation - The SDK has a Getting Started Guide along with reference documentation for the library and for AWS.

We also set up a dedicated .NET Forum and created a Windows & .NET Developer Center. As is always the case, we'd love to hear from you once you've had a chance to dive in to this new SDK, and we'd enjoy learning more about how you have put it to use.

-- Jeff;

• Access only allowed after a specified date/time.
• Access only allowed between a pair of dates/times.
• Access only allowed before a specified date/time.

You can then enable access from a designated IP address or IP address range by using the CIDR notation. You can allow a single IP address (one user or client system) or a range of addresses (perhaps a school or a corporate network) to access the content.

The system is very flexible. Here's what you need to do to start distributing your private content:

1. Store your content in Amazon S3, protected by a restrictive ACL (Access Control List).
2. Create a CloudFront origin access identity. This identifier is effectively a special AWS user identifier.
3. Update the ACL on the private S3 objects to grant read permission to the origin access identity generated in the previous step. This must be done separately for each object; it is not sufficient to make this change at the bucket level.
4. Create a new private content distribution or mark an existing distribution as private.
5. Create a key pair (or locate the one that you already use for Amazon EC2). You can also upload an existing key pair, or you can use a set from another AWS account. If you want to allow other AWS accounts to sign your CloudFront requests, you must also designate these accounts as Trusted Signers in your CloudFront distribution.
6. Design the access policy for the content. You can use a simple canned access policy that specifies nothing more than an expiration date, or you can create a more sophisticated policy which can include a date range, an IP address range, an a resource identifier (a URL to the private content) using optional wildcard characters.
7. Construct a URL using the base-64 encoded form of the policy, the public portion of the key pair, the partial URL of the private content, and the base URL of the CloudFront distribution.
8. Sign the URL using the RSA-SHA1 signature algorithm. Note that this is a different algorithm than the HMAC-SHA1 algorithm used by other parts of AWS.
9. Distribute the signed URL as desired.

You can generate and sign multiple URLs for the same content, each with a unique access policy, if necessary. This could be useful if you are making the private content available to members of a mailing list, or if you want to generate the URLs after one of your user has registered and/or paid for the content.

The newest version of the CloudBerry S3 Explorer includes full support for this exciting new feature, as does the newest Bucket Explorer. Here's a picture of Bucket Explorer in action:

The JetS3T toolkit for Amazon S3 already supports CloudFront Private Content.

We expect other tools and other toolkits to do so in the very near future. Stay tuned to this blog for more information. If you are are a tool or toolkit developer and are in need of any special help in order to implement this feature, please post a note to the CloudFront forum.

A number of blogging tools and content management systems now support CloudFront. Here's a quick roundup:

• Joomla!:
  • Joomla! Discussion Forums: CloudFront Joomla Plugin
  • Rockstar Apps: Amazon WS Tools
• Drupal:
  • CloudFront Installation
• WordPress:
  • Mark's Blog: How to copy your Wordpress files to CloudFront efficiently
  • The Inquisitr: Amazon Cloudfront CDN with a WordPress Blog
  • WordPress.org: Plugin Directory: CloudFront
  • OM4: Using Amazon CloudFront with WordPress and WordPress MU
• BitsyBox:
  • CMSWiRE: BitsyBox Web CMS, Built for Developers Too
• Bits On The Run:
  • Use your own Amazon S3 account

We are interested in hearing how you plan to put this new feature to use. Feel free to post a comment!

-- Jeff;

Here's the scoop:

Developers who want to build scalable, cost-effective cloud-based solutions will be interested in this session. Learn about Melodeo's client/server technical architecture which runs entirely on Amazon's AWS  cloud computing platform. Since Melodeo's service with nuTsie streams media and is a large-scale service, we’ll also cover scaling and redundancy. Technical challenges will be discussed, as well as what was gained and lost during the transition.

Bob's session will take place on Wednesday, November 11th at 9:45 AM.

-- Jeff;

• We started out with the November 2004 introduction of the Amazon Simple Queue Service - When we rolled this out five years ago, I think the general reaction was "Huh? Why would Amazon do that?" I think it is now pretty obvious that a scalable messaging model is an essential component of a scalable system architecture.
• The next step was Amazon S3 in the spring of 2006. That service has grown and grown (and grown) and now holds a remarkable 64 billion objects. Our early adopters quickly realized that Amazon S3 was an efficient and reliable way to store any amount of data.
• In the summer of 2006 I took a break from my summer vacation to announce Amazon EC2 and with it the concept of computing in "the cloud." As we've added features such as the Elastic Block Store, Elastic IP Addresses, Elastic Load Balancing, Amazon CloudWatch, and additional instance types, Amazon EC2 has become more flexible and more powerful, step by step.
• Amazon SimpleDB was announced at the end of 2007, giving developers a fast and highly scalable way to store and retrieve indexed data.
• We also introduced Amazon DevPay at the end of 2007, consistent with out plan to allow developers to build and to monetize their own web services. At the beginning of 2009 we also introduced the S3 Requester Pays model to make it easier to monetize content stored in Amazon S3.
• With increasing adoptions came increasing requests for more support options, so we introduced AWS Premium Support option in the spring of 2008, along with the Service Health Dashboard.
• We rolled out Amazon CloudFront in the fall of 2008, making it easier and more efficient to distribute content stored in Amazon S3.
• 2009 started off right, with the introduction of the AWS Management Console, making it even easier to manage the use of AWS resources.
• In the spring of 2009, Apache Hadoop became an official part of AWS with the introduction of Elastic MapReduce.If you read my 2006 post introducing EC2, you'll see that I thought that EC2 was a perfect host environment for Hadoop way back then!
• Come the summer of 2009, we introduced the Amazon Virtual Private Cloud, giving our customers the ability to connect their existing networks up to an isolated set of AWS resources across a VPN connection.
• This past fall we continued to push forward, giving developers access to the Amazon Relational Database Service, making it easy for them to set up and run a relational database in the cloud.
•  Over the years we've also reduced our prices for bandwidth, Amazon EC2, and SQS requests. We also lowered prices and introduced a free tier of usage for Amazon SimpleDB.

We continue to listen to our customers and to innovate on their behalf. We've got plenty of work ahead of us, so stay tuned to this blog and keeps those cards and letters (and comments) coming!

-- Jeff;

To enable further scaling up for these workloads, we are introducing a new family of memory-heavy EC2 instances with the Double and Quadruple Extra Large High-Memory instance types. Here are the specs (note that an ECU is an EC2 compute unit, equivalent in CPU power to a 1.0-1.2 GHz 2007-era AMD Opteron or Intel Xeon processor):

• Double Extra Large - 34.2 GB of RAM, and 13 ECU (4 virtual cores with 3.25 ECU each), 64-bit platform.
• Quadruple Extra Large - 68.4 GB of RAM, and 26 ECU (8 virtual cores with 3.25 ECU each), 64-bit platform.

These new instance types are available now in multiple Availability Zones of both EC2 regions (US and Europe). Double Extra Large instances cost $1.20 per instance hour and the Quadruple Extra Large instances cost $2.40 per instance hour (these prices are for Linux instances in the US region).

These new instances use the most recent generation of processor and platform architectures. In order to get the best possible performance you should experiment with compiler settings and may also want to check out specialized compilers such as Intel's Professional Edition and AMD's Open64 Compiler Suite. As with all EC2 instances where the processor architecture underlying the virtualized compute resources may vary, you may want to think about ways to detect and adapt to the processor type at launch time if this will make a difference for your particular workload.

You can launch new Double Extra Large and Quadruple Extra Large instances today using the AWS Management Console or ElasticFox.

-- Jeff;

This represents a reduction of up to 15% from the current prices for Linux instances and is a direct result of our policy of working non-stop to drive our operating costs down for the benefit of our customers. This does not affect the price of our two new instance types.

This isn't the first time we've lowered our prices in order to make AWS an even better value. In the past we've done this by adding tiered pricing to Amazon S3, reducing the storage and processing charges for SimpleDB, reducing the per-request pricing for SQS, and reducing bandwidth pricing for all services.

We've also reduced the prices for a number of IBM platform technologies. Take a look at&Amazon Running IBMfor a complete list of what we have to offer, along with the new prices.

Update: The first version of this post had the wrong prices for the SQL Server m1.large instances.

-- Jeff;

Today I'd like to tell you about our newest service, the Amazon Relational Database Service, or Amazon RDS for short. Now in beta, RDS makes it easier for you to set up, operate, and scale a relational database in the cloud. You get direct database access without worrying about infrastructure provisioning, software maintenance, or common database management tasks.

Using the RDS APIs or the command-line tools, you can access the full capabilities of a complete, self-contained MySQL 5.1 database instance in a matter of minutes. You can scale the processing power and storage space as needed with a single API call and you can initiate fully consistent database snapshots at any time.

Much of what you already know about building applications with MySQL will still apply. Your code and your queries will work as expected; you can even import a dump file produced by mysqldump to get started.

Amazon RDS is really easy to use. I'll illustrate the most important steps using the command-line tools, but keep in mind that you can also do everything shown here using the APIs.

The first step is to create a database instance. Here's how to create a database named mydb with room for up to 20 GB of data:

You can specify any one of five server instance classes:

• db.m1.small (1.7 GB of RAM, $0.11 per hour).
• db.m1.large (7.5 GB of RAM, $0.44 per hour)
• db.m1.xlarge (15 GB of RAM, $0.88 per hour).
• db.m2.2xlarge (34 GB of RAM, $1.55 per hour).
• db.m2.4xlarge (68 GB of RAM, $3.10 per hour).

During the beta you can create up to twenty databases per AWS account, and each one can consume up to 1 TB of storage. You can specify an availability zone (which you should do if you plan to access it from an EC2 instance) or you can let RDS choose one for you.

You can check on the status of your new database at any time:

When the status changes from creating to available your database is ready for use and the command output will include the server name and connection string you use to access the database instance from the command line or your application:

The final step is to edit the database's security group so that it allows inbound connections. You can enable connections from any (or all) of your EC2 security groups:

You can also enable connections from a particular IP address or address range using CIDR notation:

And that's all it takes to create a fully functioning DB Instance of MySQL 5.1 in the cloud. There's nothing to install, nothing to configure, and nothing to tune. The DB Instance will use the InnoDB storage engine.

If you need more storage space, you can expand your instance like this:

If you don't use the --apply-immediately parameter, the expansion will be performed during your DB Instance's maintenance window. The maintenance window opens once per week for four hours. During that time we will apply the latest MySQL patches (if necessary) and take care of any pending scaling operations. Your DB Instance will be unavailable only for the minimum amount of time required to make system changes on your behalf. You should set the maintenance window to a time that will result in the least possible inconvenience and down time for your users.

You can set up a two hour backup window and a retention period for automated backups, and you can also create a database snapshot at any time:

You can create new Amazon RDS DB Instances from either type of backup.With the automated backup feature, you can restore to any point in time within your backup retention period, up to the last five minutes.

Each DB Instance exports a number of metrics to CloudWatch including CPU Utilization (percent), Free Storage Space (bytes), and Database Connections (count).

At this point in the post I'd usually talk about the uses for the service, but I think that's pretty obvious! I should point out that RDS enables a lot of really enticing development and test scenarios. You can set up a separate database instance for each developer on a project without making a big investment in hardware.

Once you've deployed RDS for production use, you can easily scale up to larger instance sizes, add additional storage space (up to a total of 1 TB per RDS instance), and make backups with ease. You can easily snapshot a production database and then bring it back to the lab to dig in to a problem.

RDS usage is charged by the DB Instance hour. As noted above, there are five instance sizes and corresponding hourly rates. You'll also pay 10 cents per GB per month for your provisioned storage and 10 cents for every million I/O requests. You get backup space to store 100% of your provisioned storage at no additional charge, with additional space priced at 15 cents per GB per month. The usual AWS charges for data transferred in and out of the cloud also apply.

We have a number of enhancements planned for the future. Here are some of the features planned for the coming months:

• Reserved DB Instances so that you can pay a low one-time fee and then receive a substantial discount on your hourly usage charges.
• A High Availability offering so that you can easily and cost-effectively provision synchronously replicated RDS instances in two different availability zones.

We believe that RDS will make a really nice complement to Amazon SimpleDB and that each of the services has a number of unique features and use cases. A number of our early adopters are planning to use both of the services in their AWS applications.

As always, we have plenty of documentation, libraries, and FAQs. I look forward to seeing how you put RDS to use in your own applications.

-- Jeff;

PS - You may also enjoy reading RightScale's take on today's releases.

This data represents some of the first individual human genomes to be sequenced and peer-reviewed (the full story is here). This article contains full information about this remarkable and ground-breaking effort.

In order to get better assembly and data accuracy you determine the order of bases n times. With older sequencing technologies you collected longer reads and coverage was typically in the n=4-6 range. The sequencing process also took a very long time (several months) to collect sufficient data. Modern, or next generation, sequencing technologies yield shorter reads but you get results much faster (days to weeks) and at much lower cost, so you can repeat the experiment many times to get better coverage. Higher coverage depth gives you the ability to detect low frequency common variations (which is how we are differentiated from one another, and can be characteristic of certain diseases) and improved genome assemblies.

Suggested uses for this data include:

• The development of alignment algorithms.
• The development of de novo assembly algorithms.
• The development of algorithms that define genetic regions of interest, sequence motifs, structural variants, copy number variations, and site-specific polymorphisms.
• To test the viability of annotation engines that start with raw sequence data.

By the way, this data set is big (700 GB), but you can create an EBS volume, attach it to an EC2 instance, and start processing it in minutes!

-- Jeff;

If you are a System Integrator or Independent Software Vendor, Enterprise Architect or a entrepreneur, this will be a great opportunity to meet and learn more about AWS.

Seats are limited and prior registration is required:

AWS Workshop in Beijing
Oct 24th : http://sd2china.csdn.net
(in conjunction with CSDN conference)

Details of the Workshop

AMAZON WEB SERVICES DEEP DIVE - CLOUD WORKSHOP

INTRO TO AWS INFRASTRUCTURE SERVICES – 30 MIN
Learn how to create an AWS account, understand SOAP, REST and Query APIs and learn how to use the tools like AWS management Console

DEEP DIVE INTO AWS INFRASTRUCTURE SERVICES - 60 MIN
Amazon EC2 - Learn how to create, bundle and launch and AMI, Setting up Amazon EBS volumes, Elastic IP
Amazon S3 buckets objects and ACLs and Amazon CloudFront distributions
Amazon SQS queues
Amazon SimpleDB Domains, Items, Attributes and Querying
Amazon Elastic MapReduce JobFlows and Map Reduce

Exercise/Assignments:
Architect a Web application in the Cloud – to be discussed in class

ARCHITECTING FOR THE CLOUD: BEST PRACTICES – 30 MIN
Learn how to build highly scalable applications in the cloud. In this session, you will learn about best practices, tip, tricks and techniques of leveraging the highly scalable infrastructure platform: AWS cloud.
 
MIGRATING APPLICATIONS TO THE CLOUD - 30 MIN
Learn a step by step approach to migrate your existing applications to the Cloud environment. This blueprint will help enterprise architects in performing a cloud assessment, selecting the right candidate for the Cloud for a proof of concept project and leveraging the actual benefits of the Cloud like auto-scaling and low-cost business continuity. Jinesh will discuss migration plans and reference architectures of various examples, scenarios and use cases.

Laptop required
Knowledge of Java language preferred

See you at the workshop!

- Jinesh