Numerous U.S. government agencies, systems integrators and other companies that provide products and services to the U.S. government are using AWS services today. Now all U.S. government agencies can save significant time, costs and resources by leveraging the AWS Department of Health and Human Services (HHS) ATO packages in the FedRAMP repository to evaluate AWS for their applications and workloads, provide their own authorizations to use AWS, and transition workloads into the AWS environment. Agencies and federal contractors can immediately request access to the AWS FedRAMP package by submitting a FedRAMP Package Access Request Form and begin to moving through the authorization process to achieve an ATO with AWS.

What is FedRAMP? Check-out the answer to this and other frequently asked questions on the AWS FedRAMP FAQ site.

-- Jeff;

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

-- Jeff;

Here's what's new:

• Apple HTTP Live Streaming (HLS) Support. Amazon Elastic Transcoder can create HLS-compliant pre-segmented files and playlists for delivery to compatible players on iOS and Android devices, set-top boxes and web browsers. You can use our new system-defined HLS presets to transcode an input file into adaptive-bitrate filesets for targeting multiple devices, resolutions and bitrates.  You can also create your own presets.
  
  
• WebM Output Support. Amazon Elastic Transcoder can now transcode content into VP8 video and Vorbis audio, for playback in browsers, like Firefox, that do not natively support H.264 and AAC.
  
  
• MPEG2-TS Output Container Support. Amazon Elastic Transcoder can now transcode content into transport stream containing H.264 video and AAC audio, which are commonly used in broadcast systems.
• Multiple Outputs Per Job. Amazon Elastic Transcoder can now produce multiple renditions of the same input from a single transcoding job. For example, with a single job you can create H.264, HLS and WebM versions of the same video for delivery to multiple platforms, which is easier than creating multiple jobs and saves you time.
  
  
• Automatic Video Bit rate Optimization. With this feature, Amazon Elastic Transcoder will automatically adjust the bit rate in order to optimize the visual quality of your transcoded output. This takes the guesswork out of choosing the right bit rate for your video content.
  
  
• Enhanced Aspect Ratio and Sizing Policies. You can use these new settings in transcoding presets to precisely control scaling, cropping, matting and stretching options to get the output that you expect regardless of how the input is formatted.
  
  
• Enhanced S3 Options for Output Videos. Amazon Elastic Transcoder now enables you to set S3 Access Control Lists (ACLs) and storage type options without needing to use the Amazon S3 API or console. By using this feature, your files are then created with the right permissions in-place, ready for delivery to end-users.
  

To learn more about these features, visit the Elastic Transcoder Documentation and sign up for our free What's New with Amazon Elastic Transcoder webinar. The webinar will be held on May 29th at 10 AM PT.

As usual, these features are availble now and you can start using them today.

-- Jeff;

Parallel Scans
As you may know, DynamoDB stores your data across multiple physical storage partitions for rapid access. the throughput of a DynamoDB Scan operation is constrained by the maximum throughput of a single partition. In some cases, this means that a Scan cannot take advantage of the table's full provisioned read capacity.

In order to give you the ability to retrieve data from your DynamoDB tables more rapidly, we are introducing a new parallel scan model today.  To make use of this feature, you will need to run multiple worker threads or processes in parallel. Each worker will be able to scan a separate segment of a table concurently with the other workers. DynamoDB's Scan function now accepts two additional parameters:

• TotalSegments denotes the number of workers that will access the table concurrently.
• Segment denotes the segment of table to be accessed by the calling worker.

Let's say you have 4 workers. You would issue the following calls simultaneously to initiate a parallel scan:

• Scan(TotalSegments=4, Segment=0, ...)
• Scan(TotalSegments=4, Segment=1, ...)
• Scan(TotalSegments=4, Segment=2, ...)
• Scan(TotalSegments=4, Segment=3, ...)

The two parameters, when used together, limit the scan to a particular block of items in the table. You can also use the existing Limit parameter to control how much data is returned by an individual Scan request.

The AWS SDK for Java comes with high-level support for parallel scan. DynamoDBMapper implements a new method parallelScan, which handles threading and pagination of individual segments, which makes it even easier to try out this new feature.

To learn more about the parallel scan model, read the conceptual introduction and the best practices guide.

Provisioned Throughput Changes
You can now change the provisioned throughput of a particular DynamoDB table up to four times per day (the previous limit was twice per day). This will allow you to react more quickly to changes in load.

Read Capacity Metering
We are changing the way that we measure read capacity. With this change, a single read capacity unit will allow you to do 1 read per second for an item up to 4 KB (formerly 1 KB). In other words, larger reads cost one-fourth as much as they did before.

This change is being rolled out across all AWS Regions over the next week. Don't be alarmed if you see that your consumed capacity graph shows a lot less capacity than before.

With this change, scanning your DynamoDB table, running queries against the tables, copying data to Redshift using the DynamoDB/Redshift integration, or using Elastic MapReduce to query or export your tables, are all more cost-effective than ever before.

I hope that you can make good use of the new parallel scan model, and that the other two changes are of value to you as well.

-- Jeff;

-- Jeff;

We are pleased to announce three new AWS OpsWorks features that make it even easier to manage your applications: Elastic Load Balancing support, a monitoring view of your stack’s Amazon CloudWatch metrics, and support for additional Amazon EC2 instance types.

Elastic Load Balancing Support
...

You can now use Elastic Load Balancing to automatically distribute traffic across your application’s instances. Some of the advantages of using Elastic Load Balancing with your OpsWorks applications are

• Elastic Load Balancing automatically scales its request handling capacity in response to incoming application traffic.
• Elastic Load Balancing spans multiple AZs for reliability, but provides a single DNS name for simplicity.
• Elastic Load Balancing metrics such as request count and request latency are reported by Amazon CloudWatch.
• SSL certificates are stored using IAM credentials, allowing you to control who can see your private keys.

To get started, once you have created your ELB in the EC2 console, simply add it to the layer you want to load balance, such as your Rails app server. The layer can have a fixed pool of instances or it can use instance-based scaling to grow the capacity based on load or time. OpsWorks automatically takes care of adding and removing the instances in your layer with the load balancer.

Monitoring View
The new monitoring view is a convenient way to see the status of the instances running your application. OpsWorks sends thirteen 1-minute metrics to CloudWatch for each instance, including CPU, memory and load. The metrics are automatically grouped and filtered by each layer in the stack. You can specify a time period, select a particular metric that you want to view, or drill down to specific instances to get a more detailed view.

Additional Instance Type Support
OpsWorks now supports EBS-backed EC2 instances to give you more instance types to choose for your development needs, including the AWS Free Usage Tier-eligible micro instance.

Go For It
You can use all of these new features with a few clicks of the AWS Management Console.

You may also want to sign up for our upcoming AWS OpsWorks Webinar (May 23 at 10:00 AM PST). In the webinar you will learn about key concepts and design patterns for continuous deployment and integration using technologies like AWS OpsWorks and Chef.

-- Chris Barclay, Senior Product Manager

In order to help you to do this for your own applications, I'd like to review some important EC2 concepts and then take a look at each of the instance types that make up the EC2 instance family.

Important Concepts
Let's start with some concepts, just to make sure that we are all on the same page.

An Amazon Machine Image (AMI) is a template that defines your operating environment, including the operating system. A single AMI can be used to launch one or thousands of instances.

Instances provide compute power and are the fundamental building blocks. Instances are created by launching an Amazon Machine Image (AMI) on a particular instance type. You can scale the number of instances you are running up or down on demand, either manually or automatically, using Auto Scaling.

Instance Types comprise various combinations of CPU, memory, storage, and networking capacity and give you the flexibility to choose the appropriate mix of resources for your applications. Each instance type has one or more size options that address different workload sizes. For the best experience, you should launch on instance types that are the best fit for your applications.

Instance Families are collections of instance types designed to meet a common goal. To make it easier for you to select the best option for your applications, Amazon EC2 instance types are grouped together into families based on target application profiles.

A vCPU is a virtual Central Processing Unit (CPU). A multicore processor has two or more vCPUs.

Meet the Family
Today, Amazon EC2 gives you the option of choosing between 10 different instance types, distributed across 6 instance families. You have the flexibility to choose the combination of instance types and sizes most appropriate for your application today, and you can always change the type you use later as your business and application needs change. So what are the available instance families and instance types?

General-Purpose. This family includes the M1 and M3 instance types, both of which provide a balance of CPU, memory, and network resources making them a good choice for many applications. For many of you, this family is often the first choice, with sizes ranging from 1 vCPU with 2 GiB of RAM to 8 vCPUs with 30 GiB of RAM. The balance of resources makes them ideal for running small and mid-size databases, more memory-hungry data processing tasks, caching fleets, and backend servers for SAP, Microsoft SharePoint, and other enterprise applications.

M3 instances are the newest generation of general-purpose instances, and give you the option of a larger number of virtual CPUs (vCPUs) that provide higher performance. M3 instances are recommended if you are seeking general-purpose instances with demanding CPU requirements. M1 instances are the original family of general-purpose instances and provide the lowest cost options for running your applications. M1 instances are a great option if you want smaller instance sizes with moderate CPU performance, and a lower overall price.

Compute-Optimized. This family includes the C1 and CC2 instance types, and is geared towards applications that benefit from high compute power.

Compute-optimized instances have a higher ratio of vCPUs to memory than other families and the lowest cost per vCPU of all the Amazon EC2 instance types. If you are running any CPU-bound scale-out applications, you should look at compute-optimized instances first. Examples of such applications include front end fleets for high-traffic web sites, on-demand batch processing, distributed analytics, web servers, video encoding, and high performance science and engineering applications like genome analysis, high-energy physics, or computational fluid dynamics.

CC2 instances are the latest generation of compute-optimized instances and provide the lowest cost for CPU performance for all Amazon EC2 instance types. In addition, CC2 instances provide a number of advanced capabilities: Intel Xeon E5-2670 processors; high core count (32 vCPUs); and support for cluster networking. These capabilities allowed us to create a cluster of 1064 CC2 instances that achieved a Linpack score of 240.09 Teraflops, good for an entry at number 42 in the November 2011 Top500 supercomputer list.

C1 instances are the first generation of compute-optimized instances. They are available in smaller sizes and are ideal for massively scaled-out applications at massive scale. Most examples of customers launching 1000s of instances to transcode videos or for virtual drug design are likely to take advantage of C1 instances.

Memory-Optimized. This family includes the M2 and CR1 instance types and is designed for memory-intensive applications. Instances in this family have the lowest cost per GiB of RAM of all Amazon EC2 instance types. If your application is memory-bound, you should use these instances. Examples include high performance databases and distributed cache, in-memory analytics, genome assembly, and larger deployments of SAP, Microsoft SharePoint, and other enterprise applications. In general, if you are running a performance-sensitive database you should first look at this family.

CR1 instances are the latest generation of memory-optimized instances and provide more memory (244 GiB), faster CPU (Intel Xeon E5-2670) compared to M2 instances. CR1 instances also support cluster networking for bandwidth intensive applications.

M2 instances are available in smaller sizes, and are an excellent option for many memory-bound applications.

Storage-Optimized. This family includes the HI1 and HS1 instance types, and provides you with direct-attached storage options optimized for applications with specific disk I/O and storage capacity requirements. Currently there are two types of storage-optimized instances.

HI1 instances are optimized for very high random I/O performance and low cost per IOPS. These instances can deliver over 120,000 4k random read IOPS making them ideal for transactional applications. In particular, we designed these instances to be the best platform for large deployments of NoSQL databases like Cassandra and MongoDB.

HS1 instances are optimized for very high storage density, low storage cost, and high sequential I/O performance. HS1 instances give 48 TB of storage capacity across 24 hard disk drives, high network performance, and are capable of supporting throughput performance of as much as 2.6 GBps. These instances are designed for large-scale data warehouses, large always-on Hadoop clusters, and for cluster file systems. Indeed, HS1 instances are the underlying instance type for our petabyte-scale data warehousing service, Amazon Redshift.

Micro Instances. Micro, or T1, instances are a very low-cost instance option providing a small amount of CPU resources. Micro instances may opportunistically increase CPU capacity in short bursts when additional cycles are available. They are well suited for lower throughput applications like bastion hosts or administrative applications, or for low-traffic websites that require additional compute cycles from time to time.

Micro instances are available in the AWS Free Usage Tier to allow you to explore EC2 functionality at no charge. Due to the opportunistic scheduling used by Micro instances, you should not use them for applications that require sustained CPU performance. You can learn more about the characteristics of Micro instances and appropriate workload characteristics in the Amazon EC2 documentation.

GPU Instances. This family includes the CG1 instance type, and allows you to take advantage of the parallel performance of NVidia Tesla GPUs using the CUDA or OpenCL programming models for GPGPU computing. GPU instances also provide high CPU capabilities and support cluster networking. For applications like AMBER, a molecular dynamics application, you can get 4-5x improvement in performance compared to CC2 instances. Many of you are running computational chemistry, rendering, and financial analysis applications on CG1 instances today to take advantage of the speedup you can get from GPGUs.

Your Choice
I hope that this classification will help you to select the instance type that best fits your application. Because you can launch and terminate instances as desired, profiling and load testing across a variety of instance types is simple and cost effective. Unlike a traditional environment where you are locked in to a particular hardware configuration for an extended period of time, you can easily change instance types as your needs change. You can even profile multiple instance types as part of your Continuous Integration process and use a different set of instance types for each minor release.

The availability of multiple instance types, combined with features like EBS-optimization, and cluster networking allow applications to be optimized for increased performance, improved application resilience, and lower costs.

In particular, you should evaluate the most important performance metrics for your application. For applications that benefit from a low cost per CPU, you should try compute-optimized instances (C1 or CC2) first. For applications that require the lowest cost per GiB of memory, we recommend memory-optimized instances (M2 or CR1). If you are running a database, you should also take advantage of EBS-optimization or instances that support cluster networking. For applications with high inter-node network requirements, you should choose instances that support cluster networking. You can get all the detailed specifications for Amazon EC2 instances types on the EC2 Instance Types Table.

Our goal is to continue to provide you with instance types that address the needs of a broad swath of applications and we welcome feedback on how the currently available instance types are addressing those needs. Post a message in the EC2 forum and we'll make sure that the team sees it.

Hopefully the information provided in this post will help you get your applications revved up right away.

 -- Jeff (with help from Deepak Singh and Paul Duffy);

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

-- Jeff;

I have created the AWS Road Trip site to give you the opportunity to follow my journey. I will be posting photos, videos, maps, and more as I make my way from Boston to Seattle, with Austin as the southern extreme and San Francisco to the west. The clever and generous folks at MapBox have created a very nice interactive map of my journey.

Of course, this map (and many other parts of my trip) are powered by AWS. To learn more about how EC2, CloudFront, S3, and the Simple Email Service work together to create beautiful maps, read Will White's post, How We Serve Faster Maps from MapBox.

The AWS Road Trip site is hosted in Amazon S3, with DNS provided by Route 53. Content is generated in Octopress and pushed to S3 using S3cmd (see the first post for more information). I will write more about my setup after I return from my trip.

Some of the user groups still have space available (again, see the first post for more information). If I am speaking in your city,sign up now in order to secure a spot for yourself. I have a fresh new presentation, hundreds of AWS T-shirts, some stickers, and some other goodies to distribute.

As much as I would like to, I won't have time for any other meetings along the way. If you'd like to meet me in person, plan on coming to the user group in your city. I hope to see you there!

-- Jeff;

We would also like to take this opportunity to encourage filmmakers and artists to make use of cloud-based animation and rendering  technology as part of the production process. In order to help make this happen, we are providing aspiring filmmakers with AWS free usage credits and access to mentoring through an online forum.

Entries in the Animated Short Films category can be up to 20 minutes long, contain a significant amount of animation, and must utilize the credits provided by AWS. All entries must be submitted via Withoutabox by June 30, 2013 and completed films must be received by July 31, 2013.

At the recent AWS Summit in San Francisco, Ryan Tudhope of Atomic Fiction played a very impressive demo reel to show off some of the work that they have done on AWS:

AWS customer Zync can help you to make use of AWS if you are interested in creating and rendering an entry in this category.

For more infomation, please consult the NVFF press release and our Napa Valley Film Festival Sponsorship page.

-- Jeff;

We are making two important announcements today. First, we are opening up an AWS Direct Connect location in Seattle. Second, we are introducing AWS Direct Connect support for AWS GovCloud (US).

AWS Direct Connect in Seattle
AWS customers in the Pacific Northwest can now make use of the AWS Direct Connect location at the Equinix SE2 facility in Seattle. If you are running your own equipment in SE2 you can use Direct Connect to optimize the connection to AWS. If your equipment is located elsewhere, you will need to establish a connection from your location to an AWS Direct Connect location in order to connect to AWS.  The best way to do this is to work with an APN Partner that supports AWS Direct Connect. They will be happy to work with you to establish your connection.

AWS Direct Connect Support for AWS GovCloud (US)
Direct Connect now allows you to transfer data from any AWS Direct Connect location in the United States to AWS GovCloud (US). As you may know, this AWS Region was designed for the specific regulatory and compliance requirements of Controlled Unclassified Information (CUI). Depending on your needs, you can also run unclassified workloads in AWS GovCloud (US) to take advantage of the unique capabilities of the Region.

To help you learn more about this exciting new aspect of AWS GovCloud (US), we have set up a pair of special online events:

• AWS GovCloud (US) Office Hours will run from 1:00 PM to 3:00 PM EST on Tuesday, May 14th.
• The Introduction to AWS GovCloud (US) Region webinar will be held on May 15th at 1:30 PM EST.

Both of these events are free but space is limited and preregistration is recommended.

-- Jeff;