AWS Compute Blog

Migrating your Java applications to AWS Graviton using AWS Transform custom

Hahnara Hyun — Wed, 27 May 2026 15:04:10 +0000

For Java applications, modern JVMs like Amazon Corretto and OpenJDK are highly optimized for Arm64 and modern applications that are pure Java often require zero changes to run on Graviton. In many cases, applications aren’t fully modernized or purely Java and have a range of dependencies. When you’re responsible for migrating workloads, it’s helpful to use a systematic approach that surfaces issues, proposes solutions, and does the transformation work for you at scale.

That’s why we built the Java x86 to Graviton Migration transformation for AWS Transform custom (ATX). This is an AI-powered agent that analyzes your Java codebase, creates a migration plan, and executes the transformation—complete with version-controlled commits at every step. With ATX you can efficiently assess hundreds of Java applications simultaneously and quickly learn which applications require no changes and which ones need modifications. This streamlines the process of estimating the scope of effort, while also having suggested code updates before you even start.

ATX is available as a Kiro power, a VS Code extension, and an Agent Skill if you’d like to use it directly within other AI assistants to reduce context switching. While we will be using ATX to highlight how you can rapidly accelerate a Graviton migration, we have also published an open source Graviton universal skill based on the Agent Skills open standard so that you have the flexibility to use the skill natively within Kiro, Claude Code, Codex, or the platform of your choice.

AWS Graviton processors, based on the Arm64 architecture, can provide up to 40% better price performance over comparable x86-based instances for a wide variety of workloads. Now customers can use AI tools to quickly migrate workloads to Graviton.

The Java x86 to Graviton migration transformation

At a high level, we recommend customers finish any major version Java updates prior to migrating to Graviton and there’s a separate Java Version Upgrade transformation available for this use case. The Java x86 to Graviton Migration transformation requires a minimum of Java 8 and won’t incorporate Java version updates into the code changes.

The Java x86 to Graviton Migration completes multiple steps with work divided across multiple AI agents within the AWS Transform service, covering things like:

Native library analysis – Identifies Java Native Interface (JNI) dependencies and finds Arm64-compatible alternatives
Dependency updates – Updates libraries to versions with Arm64 support
Build configuration – Modifies Maven/Gradle configs for multi-architecture builds
Architecture-specific code – Refactors hard-coded x86 assumptions
Unit Test – Verifies compatibility at runtime given unit tests are in the project
Documentation – Creates migration notes and runbooks for your team

The agent automatically detects your Java version, manages runtime switching as needed during analysis, and handles much of the environment complexity for you such as multi-module project detection or Maven or Gradle auto-detection. Transformation completion times vary, but for many applications you can expect it to take roughly an hour (ATX works well with repos under 300K lines of code).

In this post, we:

Walk through the requirements for running the Java x86 to Graviton Migration transformation.
Help you familiarize yourself with ATX using a single Java application with Interactive Mode
Outline how to assess Graviton compatibility across the Java applications that you want to migrate to Graviton in a single batch and summarize the results with Campaign Mode.

By the end, you should have a good idea of how Java x86 to Graviton Migration transformation functions and have a summary of the expected code changes and dependency updates needed for each of your Java applications, along with version-controlled code updates.

Graviton transformation requirement

The Java x86 to Graviton migration transformation should run on an Arm64 machine.

The agent doesn’t just read your code, it builds, loads native libraries, and validates your application’s runtime behavior on Arm64. If you run the transformation on an x86 machine, the agent can identify compatibility issues but can’t execute build validation or run tests.

If you try to run on x86, you will see the following error message:

⚠  This transformation requires Arm64 architecture.    
Detected: x86_64        
Please run ATX on an Arm64 environment. See documentation for options.

To get started you need a Graviton instance or Apple Mac laptop running Arm64 with the ATX CLI, build tools, and Java JDKs that your project requires. The project source code should also be loaded locally onto the machine running the ATX CLI. Because Apple silicon is Arm64-based, it’s possible to build, load, and verify Arm64 based dependencies for a quick proof-of-concept. However, we recommend running the transformation in an environment that reflects what you plan to deploy in production to surface any potential OS level incompatibilities.

Requirements

Requirement	Details
AWS Transform custom permissions	AWS Identity and Access Management (IAM) policies for the Transform service (see Authentication docs)
Arm64 execution environment	Amazon Elastic Compute Cloud (Amazon EC2) Graviton instance or Apple Silicon Mac. Running on x86 limits validation to static analysis only. Phase 3 (build/test) requires Arm64.
Node.js 20+	Required by the AWS Transform CLI. Use the official installer at nodejs.org/en/download. Package managers (dnf, yum) can install an older version.
Git	AWS Transform custom uses local Git for version control during the transformation.
AWS Transform CLI	Installed using the setup script (see Client Setup for the curl command).
Java build tooling	A JDK (Arm64 build, e.g. Amazon Corretto or OpenJDK), Maven and/or Gradle as required by the target project. These are not optional for Java transformations. The agent needs them for dependency analysis, native library scanning, and build validation.

Running the Graviton transformation with Interactive Mode

With your code on an Arm64 environment and all the prerequisites for the transformation, we can begin the transformation.

Step 1: Navigate to Your Project and create or clone a git repo

cd /home/developer/workspace # Docker 
# or 
cd ~/workspace # AMI
git init

We recommend not pointing to the main branch of the repository of your application. You can work in a local git environment or create a separate branch. ATX needs the ability to commit changes as it iteratively transforms your code. The final decision on which commits are pushed is up to the developer.

Step 2: Launch ATX Interactive Mode

Enter the following command to launch ATX interactive mode.

atx

ATX starts in interactive mode:

To view available transformations, in a separate terminal enter:

atx custom def list > custom_list.txt

The AWS Managed transformations will be listed first, followed by User-created transformations that you’ve developed.

Step 3: Select the Graviton transformation

Enter the following into atx cli:

>AWS/early-access-java-x86-to-graviton

ATX will prompt you for next steps and your project details:

> Would you like to:+ c to abort or provide feedback)
1. View the entire transformation definition
2. View specific sections of the transformation definition
3. Apply this transformation to your code
4. Modify this transformation
> 3
> What is the file system path to the code repository where you want to apply this transformation?
> .

Note that because this is an AWS Managed Transformation, you can’t view the complete transformation definition or modify it. However, you can provide additional context customized to your use case. Keep in mind that the Transformation won’t make permanent changes to your code through the transformation process.

Step 4: Provide additional context

ATX might ask clarifying questions to tailor the transformation:

Step 5: Review the analysis

ATX analyzes your codebase:

Analyzing your codebase...[████████████████████████████████████████] 100%
✓ Parsed 847 Java files
✓ Analyzed 234 dependencies  
✓ Identified 12 native library usages
✓ Scanned build configuration
✓ Evaluated test coverage
Analysis complete. Preparing migration plan...

Step 6: Review the migration plan

ATX presents a detailed plan before making changes:

═══════════════════════════════════════════════════════════════
                    MIGRATION PLAN
═══════════════════════════════════════════════════════════════

I've analyzed your project and here's what I'll do:

PHASE 1: Dependency Updates (12 changes)
────────────────────────────────────────
 • Update netty-transport-native-epoll 4.1.85 → 4.1.100
   Reason: Arm64 support added in 4.1.94
   
 • Update snappy-java 1.1.8.4 → 1.1.10.5  
   Reason: Current version lacks Arm64 native binaries
   
 • Update lz4-java 1.7.1 → 1.8.0
   Reason: Arm64 optimization improvements
   
 • ... (9 more dependency updates)

PHASE 2: Build Configuration (3 changes)
────────────────────────────────────────
 • Add Arm64 classifier to native dependency declarations
 • Configure multi-architecture Docker build
 • Add Graviton-optimized JVM flags to startup scripts

PHASE 3: Code Modifications (2 changes)
────────────────────────────────────────
 • Refactor ProcessorArchitecture.java
   Current: Hard-coded x86_64 assumption
   Change: Add Arm64 detection logic
   
 • Update NativeLibraryLoader.java
   Current: Loads x86-specific .so files
   Change: Architecture-aware library loading

PHASE 4: Testing & Validation
────────────────────────────────────────
 • Run build and tests on Arm64
 • Generate migration report and documentation

═══════════════════════════════════════════════════════════════

? How would you like to proceed?
  ❯ Execute the full plan
    Execute phase by phase (review each)
    Modify the plan
    Ask questions about the plan
    Export plan and exit

Step 7: Ask questions, request changes

This is your chance to understand and adjust:

? Ask questions about the plan:

You: Why are you updating netty to 4.1.100? 

ATX: Good concern. Netty 4.1.94 was the first version with Arm64 
     native transport support, which is required for optimal 
     performance on Graviton. 
     
     However, I can adjust this:
     
     Option 1: Use 4.1.94 (minimum Arm64 support)
     Option 2: Use 4.1.97 (includes Arm64 bug fixes)
     Option 3: Keep 4.1.100 (latest stable with Arm64)
     
     Which would you prefer?

You: Let's go with 4.1.100.

Step 8: Execute the Transformation

When you’re ready:

? How would you like to proceed?
  ❯ Execute the full plan

Starting transformation...

Phase 1: Dependency Updates
───────────────────────────
[1/12] Updating netty-transport-native-epoll...
       ✓ Modified pom.xml
       ✓ Committed: "chore(deps): update netty to 4.1.97 for Arm64 support"

[2/12] Updating snappy-java...
       ✓ Modified pom.xml  
       ✓ Committed: "chore(deps): update snappy-java to 1.1.10.5 for Arm64"

... (progress continues)

Phase 2: Build Configuration
───────────────────────────
[1/3] Adding Arm64 classifiers...
      ✓ Modified pom.xml
      ✓ Committed: "build: add Arm64 native classifiers"

... (progress continues)

Phase 3: Code Modifications
───────────────────────────
[1/2] Refactoring ProcessorArchitecture.java...
      ✓ Modified src/main/java/com/example/util/ProcessorArchitecture.java
      ✓ Committed: "feat: add Arm64 architecture detection"

... (progress continues)

Phase 4: Validation
───────────────────
Running build... ✓
Running tests... ✓ (847 passed, 0 failed)

═══════════════════════════════════════════════════════════════
                 TRANSFORMATION COMPLETE
═══════════════════════════════════════════════════════════════

Summary:
 • 17 files modified
 • 3 files created  
 • 14 commits made
 • All tests passing

Generated Artifacts:
 • GRAVITON_MIGRATION_REPORT.html  - Full migration report
 • MIGRATION_RUNBOOK.md            - Deployment guide for your team
 • commit-log.txt                  - All commits with descriptions

Your code is now Graviton-ready!

After the transformation is complete, you can now performance test and load test on Graviton instances to configure your scaling policies or target thresholds to maximize price/performance on Graviton. For more guidance on performance testing, see the AWS Graviton Technical Guide.

What you get after transformation

Version-controlled history

Every logical change is a separate commit:

$ git log --oneline -10

a3f2b1c (HEAD) docs: add Graviton migration runbook
b82d4e5 test: add Arm64 architecture verification tests
c9a1f3d feat: add Arm64 architecture detection
d4e7c2a build: configure multi-arch Docker build
e5f8d1b build: add Arm64 native classifiers
f6a9e2c chore(deps): update lz4-java to 1.8.0
g7b0f3d chore(deps): update snappy-java to 1.1.10.5
h8c1a4e chore(deps): update netty to 4.1.97 for Arm64 support
...

Each commit is atomic and revertible. If something doesn’t work, you can git revert specific changes.

Migration report

A comprehensive markdown report covering:

What was changed and why
Dependencies that were updated
Code modifications with before and after diffs
Performance optimization recommendations

Migration runbook

A deployment guide for your team:

Pre-deployment checklist
JVM flags designed for Graviton
Monitoring and rollback procedures

Additional resources on migrating to Graviton on an infrastructure level can be found in the Transition Guide.

Assessing Graviton compatibility for multiple Java applications with Campaign Mode

When you’re ready to start migrating multiple applications, you might want to opt for an automated process that removes the manual effort of going back and forth with the transformation agent after each transformation step with campaign mode. The following command allows ATX CLI to go through a full transformation that you can check back in with after it’s completed. This limits the additional customization and context that you might want to provide the agent.

As mentioned in the first step of running a Graviton Transformation, the environment that the code is transformed in and decision of which commits are pulled into the main repo is up to the developer. Running in campaign mode across several applications doesn’t require accepting and pushing code changes. Therefore, this automated method is most useful when you want to gauge a high-level overview of effort required to migrate across several or even hundreds of applications.

atx custom def exec \
--code-repository-path /path/to/myapp \
--non-interactive \
--trust-all-tools \
--campaign  \
--repo-name myapp \
--add-repo

This command can be added into scripts, allowing further automations to be built into continuous integration and delivery (CI/CD) pipelines or scaling transformation jobs across several repos without manually entering prompts as previously shown through interactive mode.

The status of transformations running with campaign mode will be displayed in the AWS Transform Web UI. Setting up the Web UI is a prerequisite to running a transformation in campaign mode.

In addition to this view, if you run the transformation across multiple applications, you can generate a consolidated dashboard with an agent of your choice. Gather the transformation results into a centralized directory, then use the following prompt for example:

Analyze all Java application Graviton transformation summaries in <directory>/<path>/ and create a comprehensive dashboard that includes: 
 
1. Executive summary with key metrics (total apps, compatibility rate, code changes required) 
2. Application summary table with columns: Application name, Type, Java version, Dependencies count, Code changes, Compatibility %, Status 
3. Code changes analysis - which apps needed changes and why 
4. Dependency transformation analysis - common dependencies and their ARM64 status, any upgrades required 
5. Native library analysis - which apps use native libs and their compatibility 
6. Performance expectations - JWT/crypto improvements, general performance gains, cost-performance ratios 
7. JVM optimization patterns - common flags used across applications 
8. Build system patterns - Maven/Gradle usage, Docker multi-arch support 
9. Test results summary - pass/fail rates, pre-existing vs ARM64 issues 
10. Common libraries requiring changes (or note if none) 
11. Deployment readiness assessment 
12. Risk assessment with mitigation strategies 
13. Migration recommendations with phased approach 
14. Documentation summary - total docs created and their coverage 
 
Read graviton-validation/00-summary.md from each application subdirectory. Consolidate findings into a single comprehensive markdown dashboard with tables, metrics, and actionable insights. 
 
Focus on: compatibility rates, code change requirements, dependency issues, performance expectations, and migration readiness.

Keep in mind that agents might output outcomes of the migration that aren’t sourced from the transformation summaries. As a result, we recommend that you use the summary as a high-level estimate of the technical effort required for migrating to Graviton.

Conclusion

The AWS Transform custom Java x86 to Graviton Migration transformation alleviates the guesswork in Graviton migrations by using AI for dependency analysis, compatibility assessment, code refactoring, and runtime validation. Development teams can evaluate hundreds of Java applications simultaneously, with each transformation providing atomic version-controlled commits for straightforward rollback and clear change tracking. The tool offers two modes: 1) interactive mode for hands-on, application-by-application migration with developer review at each step, or 2) campaign mode for automated assessment across multiple applications. ATX converts unknown Graviton migration effort into defined requirements through automated compilation and runtime testing. This provides a more efficient way to evaluate workload compatibility and migrate to Graviton.

The Java x86 to Graviton Migration transformation is one of a range of pre-built AWS Managed Transformations but you can also create custom transformations unique to your own use case that can be scaled to drive migrations across your organization. Learn more on the AWS Transform custom website or documentation.

Resources

ATX Documentation: https://docs.aws.amazon.com/transform/latest/userguide/custom.html
AWS-Managed Transformation Definitions: https://github.com/aws-samples/aws-transform-custom-samples/tree/main/aws-managed-definitions
Graviton Getting Started: github.com/aws/aws-graviton-getting-started
Agent Skills for Graviton migration: https://github.com/aws/aws-graviton-getting-started/tree/main/tools/skills

Streamline your infrastructure: Automating AMI creation with Kiro CLI and EC2 Image Builder

Malini Chatterjee — Fri, 22 May 2026 21:01:36 +0000

Managing infrastructure at scale requires robust automation tools that reduce manual effort while maintaining consistency and security. The combination of Kiro CLI and AWS EC2 Image Builder offers a powerful solution for automating the creation, testing, and deployment of Amazon Machine Images (AMIs).

The challenge of manual image management

Traditional approaches of creating and maintaining AMIs often involve manual processes that are time-consuming, error-prone, and difficult to scale. Teams struggle with:

Inconsistent configurations across development, testing, and production environments
Security vulnerabilities from outdated base images and missing patches
Compliance gaps due to manual validation processes
Slow deployment cycles caused by repetitive manual tasks

With EC2 Image Builder and Kiro CLI, teams can replace these manual workflows with automated, and secure AMI pipelines. EC2 Image Builder provides the fully managed automation engine, while Kiro CLI brings AI-powered assistance to help you build, iterate, and troubleshoot those pipelines faster — using natural language.

EC2 Image Builder

EC2 Image Builder is a fully managed AWS service that simplifies the creation, maintenance, and deployment of customized, secure, and up-to-date server images. The service provides the following key capabilities:

Automated build pipelines: Define your image configuration once, automatically build images on a schedule or trigger basis, and manage the lifecycle of the AMI. Image Builder handles the entire lifecycle of custom AMI creation, testing, distributing and managing the lifecycle of the AMIs.
Built-in security: Automatically apply the latest security patches and validate images against AWS security best practices. EC2 Image Builder can enforce security with every created AMI using update-linux/update-windows components patch OS vulnerabilities at build time, IMDSv2 can be enforced at the pipeline level, and Amazon Inspector validates CVE posture before image distribution — all automated, no manual intervention
Testing and validation: Run automated tests to verify your images meet functional and security requirements before deployment. This ensures only validated images reach production environments.
Multi-region distribution: Automatically distribute your AMIs across multiple AWS regions and share them with specific AWS accounts, streamlining deployment across complex organizational structures.

Kiro CLI: AI-powered infrastructure automation

Kiro CLI brings generative AI capabilities directly to your terminal, enabling natural language interactions with AWS services. This AI-powered command-line interface transforms how developers and operators interact with infrastructure automation tools.

What makes Kiro CLI powerful

Natural language commands: Instead of memorizing complex CLI syntax or hand-authoring CloudFormation templates, simply describe what you want to accomplish. Kiro CLI interprets your intent and generates Infrastructure as Code — such as CloudFormation or CDK — that you can review, version-control, and deploy through your existing CI/CD pipelines. For quick, non-destructive exploration (e.g., listing resources or describing configurations), Kiro can also execute AWS API calls directly.
Context-aware assistance: Kiro understands your AWS environment and provides intelligent suggestions based on your current context, resources, and best practices. You can connect Kiro CLI to remote tools and systems via Model Context Protocol (MCP), for example, you can connect to AWS MCP servers for and documentation and troubleshooting assistance.
Workflow automation: Chain multiple operations together using conversational commands, reducing the cognitive load of managing complex infrastructure tasks.
Integration with AWS services: Seamlessly interact with EC2 Image Builder, Systems Manager, and other AWS services without switching between different tools or interfaces.

The synergy: Kiro CLI + EC2 Image Builder, automated pipeline creation

When combined, these tools create a streamlined workflow infrastructure automation:

Faster onboarding: Seamless AMI creation and faster maintenance with Kiro CLI. Rather than switching between the AWS Console and AWS CloudFormation documentation during initial exploration, Kiro CLI lets you describe your requirements conversationally — giving you a fast path to a working pipeline that you can then manage and refine through the Console or CloudFormation as your production needs mature.
Improved security posture: Automated patching and compliance validation built into every image. Describe your patching requirements conversationally, and Kiro CLI includes the appropriate build components that apply OS-level patches, kernel updates, and CVE fixes directly into the AMI at build time.
Consistent deployments: Version-controlled AMI pipelines that produce identical, pre-tested images promoted across dev, staging, and production without manual changes. EC2 Image Builder ensures every build follows the same recipe, components, and validation steps.
Reduced operational overhead: Eliminates manual, repetitive tasks around image creation, distribution, and lifecycle management accelerating iteration cycles for pipeline builds.
Faster troubleshooting: Kiro CLI parses error output and explains root cause in plain language, cutting the time spent deciphering CloudFormation stack traces and Image Builder build logs.

Getting started

Before implementing this solution, ensure you have the pre-requisites:

Kiro CLI installed (installation guide: for Linux, macOS or Windows) and configured.
Configure the AWS Documentation MCP server , refer the detailed steps here.
AWS account with access permissions for the following services:
- EC2 Image Builder
- IAM (for role creation and policy attachment)
- EC2 (for AMI management)
- Systems Manager
- VPC (for network configuration)
An existing VPC with public/private subnets configured

To begin automating your infrastructure using Kiro-CLI, here are some sample prompts that you can use as a baseline:

Example 1: Amazon Linux for EKS nodes

Use case: Teams running Kubernetes on Amazon EKS need custom node AMIs that include the correct container runtime, kubelet version, and security hardening — and that stay current with weekly base image updates. This prompt automates that pipeline and keeps your EKS node groups up to date automatically.

Prompt:

Create a production-ready EC2 Image Builder pipeline using a direct APIs 
for custom EKS-optimized Amazon Linux 2023 AMIs with the following requirements:

- Weekly automated builds triggered by base AMI updates
- AWS managed components for container runtime, kubelet and CloudWatch agent
- Automatic launch template updates for EKS managed node groups

What Kiro CLI generates:

Kiro CLI produces the API calls and supporting configuration to set up:

An EC2 Image Builder pipeline with a weekly schedule and base AMI change detection
Image recipe based on the EKS-optimized Amazon Linux 2023 AMI
Component definitions for container runtime (containerd), kubelet, and CloudWatch Agent
Automation to update EKS managed node group launch templates with the new AMI ID after each build
If we use a short prompt, Kiro will pick the default values, which customer can definitely change/edit accordingly. However, if we want to be more presriptive, then one can follow a detailed prompt like Example 2 below.

Example 2: Windows server golden image

Use case: Enterprise teams running Windows-based workloads often need a standardized, hardened base image that meets compliance requirements (such as CIS benchmarks) and includes approved software. Manually maintaining this image is error-prone and time-consuming. This prompt automates the full pipeline — from build to distribution.

Prompt:

Create a production-ready EC2 Image Builder pipeline for a Windows Server 2025 
golden image as a single CloudFormation template:

- Monthly automated builds via cron schedule
- Using latest public Windows Server 2025 AMI from AWS
- Components: AWS-managed CloudWatch Agent, AWS CLI, Windows Updates
- Apply AWS-managed STIG components (stig-build-windows) for build-time hardening 
  and corresponding stig-validate-windows for validation.
- For the EC2 instance profile role, use only these AWS-managed policies: 
  EC2InstanceProfileForImageBuilder, EC2InstanceProfileForImageBuilderECRContainerBuilds,
  and AmazonSSMManagedInstanceCore. Do NOT use any policy containing "FullAccess".
- Create a KMS multi-region primary key (MRK) in the pipeline region for AMI
  encryption, with a key policy granting cross-account access to
  [ACCOUNT_1, ACCOUNT_2, ACCOUNT_3] for kms:CreateGrant, kms:DescribeKey,
  and kms:Decrypt. Include a KMS alias. Output the key ARN for replica
  creation in target regions.
- Amazon Inspector vulnerability scanning
- Single pipeline deployed in one region. Use EC2 Image Builder
  DistributionConfiguration to share the output AMI to accounts
  [ACCOUNT_1, ACCOUNT_2, ACCOUNT_3] in regions us-east-1 and us-west-2.
  Do NOT create separate pipelines or stacks per region.
- In the DistributionConfiguration, use AmiDistributionConfiguration's
  built-in SsmParameterConfigurations to write the output AMI ID to
  /golden-image/windows-server-2025/latest in each distribution region.
  Do NOT use Lambda functions or custom resources for SSM parameter updates.
- Create an SNS topic for build notifications. Use the
  InfrastructureConfiguration's built-in SnsTopicArn property for pipeline
  status notifications. Do NOT create EventBridge rules for notifications.
- Lifecycle policy: Disable AMIs after 180 days, delete after 360 days
- Least-privilege IAM roles for Image Builder, EC2 instance profile,
  and lifecycle
- All resource names (KMS alias, IAM roles, SNS topics, Image Builder
  components, recipes, pipelines, infrastructure configs, distribution
  configs, lifecycle policies, EventBridge rules, and SSM parameter paths)
  must include !Sub "${AWS::StackName}" or a parameterized prefix to ensure
  uniqueness. This prevents conflicts if the template is deployed multiple
  times in the same account/region.
- Use AWS-managed components where available
- Parameterize account IDs and regions
- Do NOT create multiple stacks or deploy resources in multiple regions

What Kiro CLI generates:

Kiro CLI interprets this prompt and produces a complete CloudFormation template that includes:

An EC2 Image Builder pipeline with a monthly build schedule
Image recipe referencing the latest Windows Server 2025 AMI from AWS Systems Manager public parameter
AWS-managed components for CloudWatch Agent, AWS CLI, and Windows Updates
STIG hardening build component with corresponding validation component
KMS key and encryption settings applied to the output AMI
Amazon Inspector integration for CVE scanning before distribution
Distribution configuration targeting 3 AWS accounts across 2 regions
Built-in SsmParameterConfigurations writing the AMI ID to /golden-image/windows-server-2025/latest in each distribution region
SNS topic and subscriptions for build success/failure notifications
Lifecycle policy: disable AMIs after 180 days, delete after 360 days
Least-privilege IAM roles for Image Builder service, EC2 instance profile, and lifecycle management

Once the execution is complete, you can navigate to the EC2 Image Builder console. Once you are in the AWS Console EC2 Image Builder, you will be on the page for Image Pipelines. You will see in the screenshot below that the new pipeline is now Enabled.

Please note that the name of the pipeline will vary based on your specific inputs. This image is just a sample “enabled” pipeline looks like in EC2 Image Builder console.

Fig 1: Sample EC2 Image Builder console, after the pipeline is “enabled”.

For more examples and scenarios, you can check Infrastructure Automation with Kiro CLI and EC2 Image Builder workshop.

Cleanup

To avoid ongoing charges, remove all resources created during this walkthrough. The cleanup steps depend on which example you followed.

Example 1: Amazon Linux for EKS nodes cleanup

If you created resources via direct API calls, delete them in the following order:

Disable and delete the Image Builder pipeline — this stops the weekly automated builds triggered by base AMI updates.
Delete the image recipe based on the EKS-optimized Amazon Linux 2023 AMI.
Delete the component definitions for container runtime (containerd), kubelet, and CloudWatch Agent.
Delete the infrastructure configuration and distribution configuration.
Revert your EKS managed node group launch templates to their previous AMI ID, or point them to a known-good image, before removing the custom AMIs.
Deregister any AMIs produced by the pipeline and delete their associated EBS snapshots.
Remove IAM roles and instance profiles created for Image Builder and the EC2 instance profile.

Example 2: Windows server golden image cleanup

If you deployed the CloudFormation template, navigate to the AWS CloudFormation console, select your stack, and choose Delete. This removes the pipeline, recipe, components, IAM roles, KMS resources, SNS topic, and lifecycle policy in a single operation.

After the stack is deleted, manually clean up these resources that CloudFormation does not remove:

Deregister distributed AMIs — In each target account (ACCOUNT_1, ACCOUNT_2, ACCOUNT_3) and region (us-east-1, us-west-2), deregister the shared Windows Server 2025 AMIs and delete their associated EBS snapshots.
Delete SSM parameters — Remove /golden-image/windows-server-2025/latest in each distribution region where it was written by the SsmParameterConfigurations.
Schedule KMS key deletion — If the multi-region primary key (MRK) was replicated to other regions, delete the replica keys first, then schedule deletion of the primary key. Revoke any cross-account grants issued to ACCOUNT_1, ACCOUNT_2, and ACCOUNT_3.
Remove Amazon Inspector associations — If Inspector was enabled solely for this pipeline, disable it to avoid ongoing scanning charges.
Verify lifecycle policy cleanup — Confirm that the lifecycle policy (disable after 180 days, delete after 360 days) was removed with the stack. If any AMIs were already marked for lifecycle action, manually deregister and delete them.

Please note that AMI de-registration and snapshot deletion must be performed in every account and region where images were distributed. Ensure receiving accounts also deregister their copies to stop incurring storage costs.

Conclusion

The combination of AI-powered tools like Kiro CLI with robust automation services like EC2 Image Builder represents the future of infrastructure management. Whether you’re managing dozens or thousands of instances, automating your AMI creation pipeline is no longer optional—it’s essential for maintaining security, consistency, and agility in modern cloud environments.

In this post, we highlighted the benefits of AI-assisted infrastructure management using Kiro CLI. You can start using the workshop Infrastructure Automation with Kiro CLI and EC2 Image Builder for detailed prompts for building production-ready golden AMI pipeline with minimal manual coding.

Sharing Capacity Blocks for ML Across Your AWS Organization

Tyler Klimas — Mon, 18 May 2026 15:47:16 +0000

When your data science team reserves GPU instances for a two-week training job but completes it in four days, that capacity has the potential to sit unused while your computer vision team waits another week to start their project. Now you can eliminate this GPU waste and scheduling conflict by sharing Capacity Blocks for ML across your AWS Organization. This scheduling mismatch between teams creates bottlenecks that delay product launches, increase infrastructure costs, and slow your ability to deliver machine learning (ML) powered features to customers. With cross-account sharing for Amazon Elastic Compute Cloud (Amazon EC2) Capacity Blocks for ML, you can now distribute reserved graphics processing unit (GPU) capacity across teams based on actual demand rather than rigid scheduling predictions. This means your computer vision team can use the capacity as soon as the data science team is done.

In this post, we’ll show you how to configure cross-account sharing for Capacity Blocks for ML, set up monitoring for your shared resources, and optimize instance utilization through alerting. By increasing the utilization rates and reducing over-provisioning, you improve your resource efficiency and cost optimization for your organization.

You can reduce idle resources in your ML team’s account by sharing capacity with other teams waiting for GPUs. Additionally, you can maintain Capacity Blocks for ML centrally. This lets you control which teams have access to the capacity and helps you reduce waste and bottlenecks in your organization. Before starting into the tutorial, let’s review how Capacity Blocks for ML and AWS RAM work together.

Overview

Capacity Blocks for ML let you reserve GPU-based accelerated compute instances ahead of time for short duration ML workloads. When you launch instances in Capacity Blocks for ML, Amazon EC2 automatically places the instances in Amazon EC2 UltraClusters, giving you low-latency, petabit scale networking. UltraClusters provide the high performance networking your training workloads require.

You see exactly when GPU capacity is available and schedule your Capacity Blocks for ML to start when it makes sense for your project. You pay upfront for the entire reservation period. This makes Capacity Blocks for ML useful when you need GPUs for days to months. It provides predictable capacity without long-term commitments.

When you purchase Capacity Blocks for ML, you can share it with other accounts in your AWS Organization using AWS Resource Access Manager (AWS RAM). With AWS RAM, you can share AWS resources across accounts within your organization. When you share with other accounts, those accounts become consumer accounts that can launch instances using your capacity. As the owner account, you pay the upfront reservation cost and retain ownership. If you’re launching instances from a consumer account, you are responsible for additional costs such as operating system licensing charges. Capacity Blocks can be shared to multiple accounts simultaneously, with the entire Capacity Block reservation being shared on a first come, first served basis.

Figure 1: Capacity Block sharing using Resource Access Manager.

With the share feature, you benefit from flexible GPU capacity management when your priorities shift, or teams finish work at different times. Now, when your data science team completes experimentation early, your other teams can use that capacity for production training. If priorities shift mid-quarter, you can move capacity where it’s needed most.

In this tutorial, you’ll share a Capacity Block for ML across accounts and then create an alarm to monitor utilization when it drops below a threshold. Before you start, complete the following prerequisites.

Prerequisites

To share Capacity Blocks for ML, you must first find and purchase a Capacity Block. Only standard Capacity Blocks for ML can be shared using AWS RAM. UltraServer Capacity Blocks are not eligible for sharing.

You can share Capacity Blocks only within your AWS Organization. Verify the owner of the Capacity Blocks as well as the consumer(s) are within the same organization. For guidance, see Creating and configuring an organization.

Before sharing Capacity Blocks, you must configure resource sharing with AWS Organizations. Only the management account with the following required AWS Identity and Access Management (IAM) permissions can enable resource sharing within an Organization:

ram:EnableSharingWithAwsOrganization

iam:CreateServiceLinkedRole

organizations:EnableAWSServiceAccess

organizations:DescribeOrganization

Using the AWS Management Console of the management account:

Navigate to the AWS RAM console.
In the left navigation pane, choose Settings.
Select Enable sharing with AWS Organizations.

Figure 2: Enable sharing with AWS Organizations in AWS RAM.

Using the AWS Command Line Interface (CLI):

Run this command to give AWS RAM trusted access to your organization’s account structure:
```
    aws organizations enable-aws-service-access --service-principal ram.amazonaws.com
```
Turn on resource sharing within your organization so accounts and OUs can access shared resources without manual acceptance:
```
    aws ram enable-sharing-with-aws-organization
```

After you turn on sharing in your organization, you need the following IAM permissions to create resource shares:

ram:CreateResourceShare

ram:AssociateResourceShare

ram:GetResourceShares

Now that you’ve completed the prerequisites, you’ll learn how to share the Capacity Blocks for ML to other accounts of your organization.

Tutorial

You’ll complete this sharing process in four steps:

Create a resource share.
Attach Capacity Block to the resource share.
Verify the share in your consumer account.
Monitor the resource share.

Verify Capacity Reservation (console)

In your Capacity Block owner’s account, navigate to the Amazon EC2 console.
In the left navigation pane, choose Capacity Reservations.
Confirm your Capacity Blocks for ML is in Active or Scheduled state.
If you have a Resource share already configured, choose Actions, Share and select your Resource share.

Figure 3: EC2 Capacity Reservation

Share Capacity Blocks for ML (console)

You now will create a Resource Share and associate the following resources.

Navigate to the AWS RAM console in your Capacity Block owner’s account.
In the left navigation pane, choose Resource shares.
Choose Create resource share.

Figure 4: Create Resource share in AWS RAM

Enter a name for your resource share.
Under Select resource type, choose Capacity Reservations.
Select your Capacity Block from the list.
Under Principals, specify the accounts, organizational units, or organization to share with.Figure 5: Select principals to share resources with
Choose Create resource share.

Share Capacity Blocks for ML (AWS CLI)

Replace the placeholder values in the following CLI commands below with your actual values:

arn:aws:ec2:us-east-2:123456789012:capacity-reservation/cr-1234abcd56EXAMPLE → Your Capacity Reservation ARN
111122223333 → The AWS account ID of the principal you’re sharing with
arn:aws:ram:us-east-2:123456789012:resource-share/7ab63972-b505-7e2a-420d-6f5d3EXAMPLE → Your RAM resource share ARN

Create resource share with Capacity Block and principals:

    aws ram create-resource-share \
         --name capacity-block-share \
         --resource-arns arn:aws:ec2:us-east-2:123456789012:capacity-reservation/cr-1234abcd56EXAMPLE \ 
         --principals 111122223333

To add a Capacity Block to existing resource share:

    aws ram associate-resource-share \
         --resource-share-arn arn:aws:ram:us-east-2:123456789012:resource-share/7ab63972-b505-7e2a-420d-6f5d3EXAMPLE \
         --resource-arns arn:aws:ec2:us-east-2:123456789012:capacity-reservation/cr-1234abcd56EXAMPLE

Access and Launch shared Capacity Blocks (console)

After you add the Capacity Block to a resource share, your consumer accounts automatically gain access when you share the Capacity Block within the same AWS Organization.

Navigate to the AWS RAM console in your consumer account.
In the left navigation pane, choose Shared with me, Resource shares. Verify the Resource share is Active.Figure 6: In your consumer account, verify the resource share
Navigate to the Amazon EC2 console. In the left navigation pane, choose Capacity Reservations.
Confirm the shared Capacity Block appears and is in Active or Scheduled state. Because sharing is asynchronous, the Capacity Block may take a few moments to appear even after the resource share shows Active.
Navigate to the Amazon EC2 console and choose Launch instance.
Configure your instance as required (AMI, instance type, key pair, etc.).
Under Advanced details, for Purchasing option, choose Capacity Blocks.
For Capacity reservation, choose Specify Capacity Reservation.
For Capacity reservation targeted ID, select or enter your Capacity Block reservation ID.
Launch the instance.

Access shared Capacity Blocks (AWS CLI)

Replace the placeholder values in the following CLI commands below with your actual values:

ami-0abcdef1234567890 → Your AMI ID
cr-0c54f6734d944345a → Your Capacity Reservation ID

List resource shares in your consumer account:

    aws ram get-resource-shares --resource-owner OTHER-ACCOUNTS

Verify that capacity reservation is available:

    aws ec2 describe-capacity-reservations

Launch EC2 instance from Capacity Block:

    aws ec2 run-instances \
         --image-id ami-0abcdef1234567890 \
         --count 1 \
         --instance-type p5.48xlarge \
         --key-name my-key-pair \
         --subnet-id subnet-0abcdef1234567890 \
         --instance-market-options MarketType='capacity-block' \
         --capacity-reservation-specification CapacityReservationTarget={CapacityReservationId=cr-0c54f6734d944345a}

Monitor usage (console)

You can create Amazon CloudWatch alarms to proactively identify low utilization of your Capacity Block. This helps you to improve the usage of your capacity reservation. This section shows you how to create an Amazon Simple Notification Service (Amazon SNS) email notification when the number of running instances drops below a certain threshold.

In addition to monitoring usage, AWS CloudTrail logs capture API events related to your Capacity Block, including the CapacityReservationId. As the owner, you can see which accounts are consuming instances and when.

Step 1: Create an SNS Topic for Notifications

Open the Amazon SNS console.
In the left navigation pane, choose Topics.
Choose Create topic.
For Type, select Standard.

Figure 7: Create SNS Topic

For Name, enter capacity-block-alerts.
Choose Create topic.

Step 2: Create an SNS Subscription:

In the left navigation pane, choose Create subscription.Figure 8: Create SNS Subscription
For Protocol, choose Email.
For Endpoint, enter your email address.
Choose Create subscription.

Step 3: Create the CloudWatch Alarm

Navigate to the Amazon CloudWatch console.
In the left navigation pane, choose Alarms, All alarms.
Choose Create alarm.
Choose Select metric.
Choose EC2 Capacity Reservations.
Choose By Capacity Reservation.
Find your Capacity Block ID (e.g., cr-12345678abcdef).
Select the checkbox next to InstanceUtilization.
Choose Select metric.

Step 4: Configure the Metric

Under Metric:
For Statistic: Select Average.
For Period: Select 5 minutes.
Under Conditions choose Threshold type: Select Static.
Whenever InstanceUtilization is…: Select Lower than…: Enter 20 (This metric is measured in percentage).
Choose Next.

Step 5: Configure Actions

Under Notifications:
Alarm state trigger: Select In alarm.
Select an SNS topic: Choose Select an existing SNS topic.
Send a notification to…: Select capacity-block-alerts.Figure 9: Configure CloudWatch Alarm
Choose Next.

Step 6: Name and Create Alarm

For Alarm name, enter: CapacityBlock-LowUtilization-cr-123456789abcdef.
For Alarm description, enter: Alert when Capacity Block utilization drops below 20%.
Choose Next.
Review your configuration and choose Create alarm.

Monitor usage (AWS CLI)

Replace the placeholder values in the following CLI commands below with your actual values:

123456789012 → Your 12-digit AWS account number
cr-0c54f6734d944345a → Your Capacity Reservation ID
7ab63972-b505-7e2a-420d-6f5d3EXAMPLE → Your RAM resource share ID
your_email@example.com → Your email address for notifications

Create the SNS topic:

    aws sns create-topic \
        --name capacity-block-alerts

Using the TopicArn from the output, subscribe your email:

    aws sns subscribe \
        --topic-arn arn:aws:sns:us-east-2:123456789012:capacity-block-alerts \
        --protocol email \
        --notification-endpoint your_email@example.com

Create the full CloudWatch alarm:

    aws cloudwatch put-metric-alarm \
        --alarm-name "CapacityBlock-LowUtilization-cr-1234EXAMPLE" \
        --alarm-description "Alert when Capacity Block utilization drops below 20%" \
        --namespace "AWS/EC2CapacityReservations" \
        --metric-name "InstanceUtilization" \
        --dimensions Name=CapacityReservationId,Value=cr-0c54f6734d944345a \
        --statistic Average \
        --period 300 \
        --evaluation-periods 1 \
        --threshold 20 \
        --comparison-operator LessThanThreshold \
        --alarm-actions arn:aws:sns:us-east-2:123456789012:capacity-block-alerts

Clean up (console)

As the owner of the Capacity Block, you retain the ability to modify the resource share. However, owners cannot modify instances that consumers launch into Capacity Blocks they have shared. This section outlines how to clean up your previous work.

Using the AWS Management Console:

Stop sharing the Capacity Block

Navigate to AWS RAM console.
In the left navigation, choose Shared by me, Resource shares.
Select your resource share.
Choose Modify.
Remove the Capacity Block from the resource share or delete the entire resource share.

Delete the CloudWatch Alarm

Navigate to the Amazon CloudWatch console.
In the left navigation, choose Alarms, All alarms.
Select the alarm you created.
Choose Actions, Delete.
Confirm deletion.

Delete the SNS Topic and Subscription

Navigate to the Amazon SNS console.
In the left navigation, choose Subscriptions.
Select the subscription and choose Delete.
In the left navigation, choose Topics.
Select capacity-block-alerts and choose Delete.
Confirm deletion.

Clean up (AWS CLI)

Replace the placeholder values in the following CLI commands below with your actual values:

123456789012 → Your 12-digit AWS account number
7ab63972-b505-7e2a-420d-6f5d3EXAMPLE → Your RAM resource share ID
cr-0c54f6734d944345a → Your Capacity Reservation ID
a1b2c3d4-5678-90ab-cdef-EXAMPLE → Your SNS subscription ID

Remove the Capacity Block from the resource share

    aws ram disassociate-resource-share \
        --resource-share-arn arn:aws:ram:us-east-2:123456789012:resource-share/7ab63972-b505-7e2a-420d-6f5d3EXAMPLE \
        --resource-arns arn:aws:ec2:us-east-2:123456789012:capacity-reservation/cr-0c54f6734d944345a

Delete the resource share

    aws ram delete-resource-share \
        --resource-share-arn arn:aws:ram:us-east-2:123456789012:resource-share/7ab63972-b505-7e2a-420d-6f5d3EXAMPLE

Delete the CloudWatch Alarm

    aws cloudwatch delete-alarms \
         --alarm-names "CapacityBlock-LowUtilization-cr-123456789"

Delete the SNS Topic and Subscription

List subscriptions to get the subscription ARN

    aws sns list-subscriptions-by-topic \
         --topic-arn arn:aws:sns:us-east-2:123456789012:capacity-block-alerts

Delete the subscription

    aws sns unsubscribe \
         --subscription-arn arn:aws:sns:us-east-2:123456789012:capacity-block-alerts:a1b2c3d4-5678-90ab-cdef-EXAMPLE

Delete the Topic

    aws sns delete-topic \
         --topic-arn arn:aws:sns:us-east-2:123456789012:capacity-block-alerts

Conclusion

In this post, we showed you how to share Capacity Blocks for ML across your AWS Organization using AWS RAM. We covered configuring the AWS RAM integration with Organizations, creating resource shares, and accessing shared Capacity Blocks for ML from consumer accounts. Finally, we showed you how to monitor and alert on low instance utilization.

By sharing Capacity Blocks across your organization, you can reduce idle GPU capacity, eliminate scheduling bottlenecks between teams, and maximize the return on your reserved compute investment. To take this further, consider building dashboards in Amazon CloudWatch to track utilization trends across multiple Capacity Blocks.

You can get started by purchasing Capacity Blocks for ML and sharing it across your organization today. For more details on other resources you can share with AWS RAM, visit the Shareable AWS resources in the user guide. If you have questions, contact your AWS account team or leave a comment below.

Enhancing network observability with new AWS Outposts racks LAG metrics

Adam Duffield — Thu, 30 Apr 2026 19:14:52 +0000

When you deploy AWS Outposts racks, you can run AWS infrastructure and services in on-premises locations. Maintaining seamless connectivity, both to the AWS Region and your on-premises network, is fundamental to delivering consistent, uninterrupted service to your applications. Implementing an observability strategy that uses available network metrics is key to understanding the health of this connectivity.

In August 2025, we launched two new Amazon CloudWatch metrics, VifConnectionStatus and VifBgpSessionState, that helped provide greater visibility into these Layer 3 networking constructs. However, insight into Layer 2 networking was still missing. AWS has released a new metric LagStatus, that provides greater visibility into the hybrid infrastructure connectivity for both first-generation and second-generation Outpost racks.

Link aggregation group overview

Link aggregation combines multiple physical Ethernet connections into one logical link, referred to as a link aggregation group (LAG). This consolidation delivers benefits such as increased aggregate bandwidth and built-in redundancy through fault-tolerant connections between network devices. AWS Outposts uses LAG connections between Outpost network devices (ONDs) and customer network devices (CNDs). The links from each Outpost network device are aggregated into an Ethernet LAG to represent a single network connection.

Figure : Second-Generation Outposts Rack network connections

Each LAG between an Outpost network device and a customer local network device is configured as an IEEE 802.1q Ethernet trunk. This enables the use of multiple VLANs for network segmentation between data paths. Each Outpost has the following VLANs to communicate with local network devices:

Service link VLAN – Enables communication between the Outpost and customer network devices to establish a service link path to the AWS Region.
Local gateway VLAN(s) – (If exists, and as single or multiple LGW routing domains), enables communication between Outpost and the customer network devices to establish a local gateway path to connect your Outpost subnets to the local area network.

Figure : Second-Generation Outposts Rack VLAN layout

Using the LagStatus metric

The new LagStatus metric in CloudWatch provides visibility into the operational status of LAG connections between Outposts networking devices and on-premises infrastructure. The metric reports a binary status (1 for the LAG being UP, 0 for the LAG being down) and includes the OutpostId and LagId as dimensions to quickly identify non-operational resources.

You can view this metric on the CloudWatch console. As with all operational telemetry, access to these metrics should be appropriately restricted to authorized principals. The metric data points are published at 5-minute intervals, and like all CloudWatch metrics, there might be a time lag in the metric data being published. In the navigation pane, choose All metrics, followed by Outposts under the AWS namespaces section. The Outposts namespace can only be viewed by the Outposts owner account, unless CloudWatch cross-account observability is configured.

Figure : CloudWatch Metrics view of the LagStatus metric

While the LagStatus metric alone provides insight into the Outposts network connectivity, combining it with VifConnectionStatus and VifBgpSessionState delivers more immediate, actionable insights that expedite troubleshooting. In addition, to improve the clarity of the existing metrics, the related LagId is added as a new Outposts metric dimension. By observing the values of all three metrics, you can narrow down the potential cause of any issues. The following table gives some possible connectivity issue scenarios and how they can be identified using these metrics:

LagStatus	LGW BGP	ServiceLink BGP	Potential issue
UP	UP	UP	Recommended state – all components working
UP	UP	DOWN	ServiceLink BGP issue – configuration issue
UP	DOWN	UP	LGW BGP issue – configuration issue
UP	DOWN	DOWN	Both BGP sessions down – configuration issue
DOWN	DOWN	DOWN	Lag configuration issue or Physical failure

With these metrics, you can use CloudWatch Composite Alarms to alert operational teams when any of the components aren’t running as expected.

To create a composite alarm, alarms must first be defined for all three of the individual metrics. This can be done from the console, CLI, or AWS CloudFormation. Following the principle of least privilege, ensure that IAM permissions are restricted to the minimum actions required for CloudWatch alarm creation. For more information, see the CloudWatch documentation. If you prefer, you can configure these individual alarms without notification actions enabled to reduce potential notification noise. Each virtual interface (VIF) has its own set of metrics, so you would need to configure alarms for all VIFs used with your Outpost. The number of total VIFs will vary depending on the Outpost generation that’s deployed because of the different networking architectures.

First-generation Outposts racks use four VIFs per rack (two for Service Link, two for Local Gateway). Second-generation racks require a minimum of eight VIFs (four for Service Link, four for Local Gateway), because they support multiple local gateway routing domains, each with its own VIFs.

An example alarm configuration as seen in the console for a single VIF is shown in the following figure 4.

Figure : Individual CloudWatch alarms for VIF status

After these individual alarms are created, a composite alarm can be created that monitors for any of the component metrics going into an alarm status. In the following example, the AWS Command Line Interface (AWS CLI) is used to create the composite alarm called composite-alarm-lag1 and send a notification using an Amazon Simple Notification Service (Amazon SNS) topic called outpost-network-alarms. As this topic carries infrastructure health data, it’s recommended to encrypt it using an AWS Key Management Service key and restrict the subscription policy to authorized principals.

aws cloudwatch put-composite-alarm \
  --alarm-name "composite-alarm-lag1" \
  --alarm-rule "ALARM(VifBgpSessionState-lgw-vif-xxxxxxxxxxxx) OR ALARM(VifConnectionStatus-lgw-vif-xxxxxxxxxxxx) OR ALARM(VifBgpSessionState-sl-vif-xxxxxxxxxxxx) OR ALARM(VifConnectionStatus-sl-vif-xxxxxxxxxxxx) OR ALARM(LagStatus-op-lag-xxxxxxxxxxxx)" \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:outpost-network-alarms \
  --region us-east-1

You can use this granular monitoring to quickly identify and troubleshoot connectivity issues, particularly in scenarios where LAG status is up but VIF BGP status is down.

Conclusion

This post provides details about the newly released LagStatus CloudWatch metric, and how this metric can be used with existing metrics such as VifConnectionStatus and VifBgpSessionState to build a comprehensive network connectivity observability solution. The LagStatus metric is now available in all commercial AWS Regions and the AWS GovCloud (US-East) and AWS GovCloud (US-West) Regions where Outposts racks are available, for both first-generation and second-generation racks at no additional cost.

For more information about Outposts rack networking patterns, see the Networking section of the Outposts High Availability Design and Architecture Considerations whitepaper.

Reach out to your AWS account team, or fill out this form to learn more about observability for Outposts.

Serverless ICYMI Q1 2026

Julian Wood — Thu, 30 Apr 2026 15:58:24 +0000

Stay current with the latest serverless innovations that can improve your applications. In this 32nd quarterly recap, discover the most impactful AWS serverless launches, features, and resources from Q1 2026 that you might have missed.

In case you missed our last ICYMI, check out what happened in Q4 2025.

2026 Q1 calendar

Serverless with Mama J

Serverless with Mama J

If you really want to know whether you understand something, try explaining it to your mom!

That’s exactly what Eric Johnson did. His mom, everyone calls her Mama J, wanted to know what serverless actually means and why it matters. So he walked her through it: what servers do, why they’re a headache to manage, and how AWS Lambda lets you skip all that by running code only when it’s needed, scaling automatically, and charging you nothing when nobody’s using it.

Watch the video on the AWS Developers YouTube channel.

Build serverless apps faster with AI

AWS is providing a growing set of AI-powered tools to bring serverless expertise directly into your coding assistants. From Model Context Protocol (MCP) servers and Anthropic Claude plugins to Kiro Powers. These tools provide contextual guidance for architecture decisions, implementation patterns, and deployment automation across the full serverless development lifecycle.

For more information on the tools available, see the resources page.

Serverless Patterns Collection

The open source Serverless Patterns Collection on Serverless Land now provides a direct link to download pattern .zip files. You can also clone the whole repo and explore more patterns.

Serverless Patterns .zip download

AWS Lambda

Build fault-tolerant, long-running applications using familiar programming patterns using AWS Lambda durable functions. You can use Lambda durable functions to write multi-step workflows in your preferred programming language, using built-in methods that automatically handle progress checkpointing and error recovery. This can improve your architecture so that you can focus on your business logic and optimize costs by charging only for active compute time.

You can build durable functions in Python and TypeScript and there is a durable execution SDK for Java in preview with the code available on GitHub.

Eric Johnson has a new video deep dive showing how to upload videos and scan them with AI. Learn how to coordinate multiple AWS services like Amazon Rekognition and Amazon Transcribe, implement human-in-the-loop approval workflows, and crate a live dashboard for real-time updates.

To find out how durable functions work, see the blog post which also provides testing and best practices guidance. You can also watch the re:Invent Breakout Session video: Deep Dive on AWS Lambda durable functions (CNS380)

Lambda now supports the .NET 10 runtime, including support for file-based apps. Developers can take advantage of the latest .NET 10 performance improvements, new language features, and improved startup times for Lambda functions.

You can now see Availability Zone (AZ) metadata in function execution environments. This allows you to determine the AZ ID (e.g., use1-az1) of the AZ your function is running in. This helps build functions that can make AZ-aware routing decisions, such as preferring same-AZ endpoints for downstream services to reduce cross-AZ latency. Operators can also implement AZ-aware resilience patterns like AZ-specific fault injection testing.

Payload size increase

AWS has increased the maximum payload size from 256 KB to 1 MB for a number of services such as asynchronous Lambda invocations, Amazon SQS, and Amazon EventBridge. This gives you more room to build and maintain context-rich event-driven systems and reduce the need for complex workarounds such as data chunking or external large object storage.

This blog post explores a real-world example using rich event context in agentic event-driven architectures

Payload size increase workflow

Amazon Bedrock

Amazon Bedrock expanded its model availability with a new set of fully managed open-weight models spanning frontier reasoning and agentic coding. Other model releases include Anthropic Claude Opus 4.6 and Claude Sonnet 4.6, and NVIDIA Nemotron 3 Super. You can invoke them through the unified Amazon Bedrock API without managing any underlying infrastructure, making it straightforward to experiment and swap models as your workload evolves.

Amazon Bedrock AgentCore is the infrastructure layer for securely deploying and operating AI agents. It works with popular open source frameworks, including Strands Agents, LangGraph and CrewAI, giving you the flexibility to build with your preferred tools without vendor lock-in.

AgentCore Gateway now includes semantic tool search, so you can discover the right tool for a task using natural language queries instead of manually browsing a catalogue. It also adds custom KMS encryption, debugging messages, and resource tagging to give you stronger governance over tool integrations.

Policy in Bedrock AgentCore allows you to define precise boundaries on agent actions and run continuous quality monitoring. This helps you maintain predictable, auditable agent behavior in production without embedding guardrail logic inside each individual agent.

AgentCore Runtime now supports stateful MCP server features, allowing agents to maintain session context across tool calls for richer, more coherent multi-step interactions.

Strands Agents

Strands Agents SDK

Strands Agents is an open source SDK for building and running AI agents in just a few lines of code, working with models available in Amazon Bedrock. Strand s Labs is a new dedicated GitHub organization for experimental agent projects, including robotics and code agents. This gives you early access to cutting-edge agentic techniques before they reach production frameworks. See the introduction blog post for more information.

AWS Step Functions

AWS Step Functions introduces an enhanced TestState API that enables API-based testing for validating workflows before deployment. The new API supports testing individual states in isolation or complete workflows end-to-end, making it easier to verify state machine logic without incurring runtime costs.

By integrating TestState API testing into CI/CD pipelines, you can validate workflow logic before deployment, reducing the risk of production issues. Find complete code examples and testing framework in the GitHub repository.

Amazon EventBridge

Amazon EventBridge Scheduler now provides resource count metrics to help you monitor quota usage. These new metrics make it easier to track the number of schedules and schedule groups in your account and proactively manage service quotas.

Amazon DynamoDB

You can replicate Amazon DynamoDB table data across multiple AWS accounts and Regions. This enhances resiliency through account-level isolation, supports tailored security and data-perimeter controls. You can align workloads by business unit or environment and simplify governance requirements.

Amazon DynamoDB global replication

Amazon ECS

Amazon ECS Managed Instances can now integrate with Amazon EC2 Capacity Reservations. This allows you to make sure there is capacity availability for your container workloads while benefiting from the management automation of ECS Managed Instances.

ECS also now supports Network Load Balancer (NLB) for linear and canary deployment strategies. This helps you perform gradual traffic shifting using NLBs, providing more flexibility in deployment pipelines for latency-sensitive applications.

Serverless blog posts

January

February

March

Serverless Office Hours

Join our livestream every Tuesday at 11 AM PT for live discussions, Q&A sessions, and deep dives into serverless technologies. Watch episodes on-demand at serverlessland.com/office-hours.

Still looking for more?

The Serverless landing page has overall information about building serverless applications. The Lambda resources page contains case studies, webinars, whitepapers, customer stories, reference architectures, and even more Getting Started tutorials.

You can also follow the Developer Advocacy team to see the latest news, follow conversations, and interact with the team.

Julian Wood: @julian_wood, https://www.linkedin.com/in/julianrwood/
Eric Johnson: @edjgeek, https://www.linkedin.com/in/singledigit/
Erik Hanchet: @ErikCH, https://www.linkedin.com/in/erikhanchett/
Salih Gueler: @salihgueler, https://www.linkedin.com/in/salihgueler/
Marcia Villalba: @mavi888uy, https://www.linkedin.com/in/marciavillalba

And finally, visit Serverless Land for your serverless needs.

AWS Outposts monitoring and reporting: A comprehensive Amazon EventBridge solution

Matt Price — Tue, 14 Apr 2026 16:18:12 +0000

Organizations using AWS Outposts racks commonly manage capacity from a single AWS account and share resources through AWS Resource Access Manager (AWS RAM) with other AWS accounts (consumer accounts) within AWS Organizations. In this post, we demonstrate one approach to create a multi-account serverless solution to surface costs in shared AWS Outposts environments using Amazon EventBridge, AWS Lambda, and Amazon DynamoDB. This solution reports on instance runtime and allocated storage for Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Services (Amazon RDS), and Amazon Elastic Block Store (Amazon EBS) services running on Outposts racks. In turn, teams can track the cost of infrastructure associated with their workloads across AWS accounts. This solution is a framework that can be customized to meet your organization’s specific business objectives.

Solution overview

The following is the Terraform-based reference architecture used to represent the solution, including EventBridge, DynamoDB, and Lambda across a multi-account environment. Relevant launch events are tracked in EventBridge that invoke Lambda functions, which are logged in DynamoDB tables (see sample code). This allows reporting on captured event data through the AWS SDK for Python (Boto3).
Figure 1: Reference architecture for reporting solution on AWS Outposts

Prerequisites

The following prerequisites are necessary to implement this solution:

At least two active AWS accounts in the same AWS Organization as the Outposts owner account.
- One AWS account, which is the data collection account to store the event data (this doesn’t have to be the account that owns the Outposts).
- Workload accounts where resources are deployed on Outposts.
AWS Command Line Interface (AWS CLI) installed and configured on an administrative instance. For more information, see Installing, updating, and uninstalling the AWS CLI in the AWS CLI documentation.
Terraform installed on the same administrative instance. For more information, see the Terraform documentation.
Make sure that you have the necessary AWS Identity and Access Management (IAM) permissions necessary to create the AWS resources using Terraform in all accounts.
Prior Experience with Terraform deployments on AWS Cloud. To increase your familiarity, you can explore Get Started – AWS on the HashiCorp website.
Access to clone the AWS Outposts Monitoring and Reporting git repository.
SDK for Python installed and configured on a local machine.

Walkthrough

The following sections walk you through how to deploy this solution.

Deploying in data collection account

Step 1: Create a bucket in-Region to hold the Terraform state file in the data collection account.

aws s3 mb s3://state-bucket-name

Step 2: Clone the repository.On your local machine, clone the repository that contains the sample by running the following command:

git clone https://github.com/aws-samples/sample-outposts-monitoring-and-reports.git

Navigate to the cloned repository by running the following command:cd sample-outposts-monitoring-and-reports/data_collection

Step 3: Edit the providers.tf to configure the AWS provider.



provider "aws" {
  region = ""
}

Step 4: Edit the backend.tf to provide the Terraform state bucket and Outposts anchored AWS Region.

terraform {
  backend "s3" {
    bucket = ""
    key    = "terraform.tfstate"
    region = ""
  }
}

Step 5: Modify the variables.tf.From the root directory of the cloned repository, modify the variables.tf file with the target Region and workload accounts as shown in the following example. The target Region is the collection destination.

variable "aws_region" {
  description = "AWS region for resources"
  type        = string
  default     = ""
}

variable "allowed_account_id" {
  description = "AWS account ID allowed to put events to the event bus"
  

}

Initialize the configuration directory of the data collection account to download and install the providers defined in the configuration by running the following command:

terraform init

All resources are deployed with minimal permissions to serve as an example. We recommend viewing all configurations to make sure that they meet your organizational security policies. Step 6: Deploy infrastructure in the data collection account.Run terraform plan on the configuration to and review which resources are created:

terraform plan

When you have reviewed the plan, run the following command and enter “yes” to accept the changes and deploy:

terraform apply

Deployment should take less than 5 minutes. If you receive any errors, review the previously mentioned steps to ensure that you followed them in their entirety. If the errors persist, reach out to AWS Support for additional guidance.

Deploying in workload account

The data collection account receives events from EventBridge and performs intelligent analysis and storage from the AWS Outposts resource data.Step 1: Navigate to the workload account directory by running the following command:

cd ../workload_account

Step 2: Edit variables.tf to set up the Region and event bus Amazon Resource Name (ARN).

variable "aws_region" {
  description = "AWS region for resources"
  type        = string
  default     = ""
}

variable "event_bus_arn" {
  description = "target event bus arn"
  type        = string
  default     = ""
}

Edit the code to update the event bus name.

Step 3: Run the following command to create the backend.tf and create the Terraform state bucket for each workload account.

./init-backend.sh

This is an idempotent operation that creates a file from the template and a bucket with a fixed name including the account ID if it doesn’t exist.

Step 4: Initialize the configuration directory of the Data Collection Account to download and install the providers defined in the configuration by running the following command:

terraform init

Step 5: Deploy the infrastructure in the Data Collection Account.Run a terraform plan on the configuration and review which resources are created:

terraform plan

After you have reviewed the plan, run the following command and enter “yes” to accept the changes and deploy:

terraform apply

Deployment should take less than 5 minutes. If you receive any errors, follow the troubleshooting steps in the previous section.

At this point, any Amazon EC2 or Amazon RDS instances and Amazon EBS volumes are logged to the DynamoDB tables in the data collection account. Repeat Steps 3–5 for each workload account running resources on AWS Outposts with appropriate account credentials. If you’re deploying at scale and using AWS Control Tower consider using AWS Control Tower Account Factory for Terraform (AFT).

Running monthly reports

With this solution in place, reports can be generated on demand. These reports can be customized by modifying the Python example scripts shown to support your needs. Reports can be created from a local machine with credentials that have access to the DynamoDB tables in the data collection account. The examples were created from the source directory of the data collection account git repository. Run the following command to view the report for Amazon RDS usage in September 2025:

./rds_runtime_calculator.py --year 2025 --month 9 --output rds_report.csv

Figure 2: Example of RDS runtime report

Run the following command to view the report for Amazon EBS usage in September 2025:

./ebs_volume_reporter.py --year 2025 --month 9 --output ebs_report.csv

Figure 3: Example of EBS usage report

Run the following command to view the report for Amazon EC2 usage in September 2025:

./ec2_runtime_calculator.py --month 9 --year 2025 --output ec2_report.csv

Figure 4: Example of EC2 runtime report

Cleaning up

Complete the following steps to clean up the resources that were deployed by this solution. For each workload account, complete the following:

cd sample-outposts-monitoring-and-reports/workload_account

terraform destroy

Enter “yes” to proceed. You can then manually empty and remove the terraform state S3 bucket for that account.

For the data collection, complete the following:

cd ../data_collection

terraform destroy

Enter “yes” to proceed. You can then manually empty and remove the terraform state S3 bucket for that account.

Conclusion

Customers who have shared multi-account Outposts deployments can use this solution to create account level reporting for Outposts resources using real-time event capture and processing, state analysis and categorization, historical usage metrics, and serverless architecture. Teams can use this to visualize and report on the costs of running their workloads on Outposts. The event-driven design supports accurate tracking while maintaining low operational overhead. The solution scales effectively across multiple Outposts and accounts, providing a unified view of hybrid infrastructure. Keep in mind that you can extend the functionality described here to meet your business objectives.

Deploy this solution today using the GitHub repository to gain financial insights to share with the tenants of your Outposts workload accounts. Reach out to your AWS account team, or fill out this form to learn more about Outposts.

Building Memory-Intensive Apps with AWS Lambda Managed Instances

Guy Haddad — Fri, 10 Apr 2026 19:54:44 +0000

Building memory-intensive applications with AWS Lambda just got easier. AWS Lambda Managed Instances gives you up to 32 GB of memory—3x more than standard AWS Lambda—while maintaining the serverless experience you know. Modern applications increasingly require substantial memory resources to process large datasets, perform complex analytics, and deliver real-time insights for use cases such as in-memory analytics, Machine Learning (ML) model inference, and real-time semantic search. AWS Lambda Managed Instances gives you a familiar serverless programming model and experience combined with the flexibility of being able to choose the underlying Amazon EC2 instance types and providing developers with access to large memory configurations.

In this post, you will see how AWS Lambda Managed Instances enables memory-intensive workloads that were previously challenging to run in serverless environments, using an AI-powered customer analytics application as a practical example. You’ll see cost savings of up to 33% compared to standard Lambda for predictable workloads, while eliminating the operational overhead of managing EC2 instances.

Understanding AWS Lambda Managed Instances

AWS Lambda Managed Instances runs your AWS Lambda functions on the Amazon EC2 instance types of your choice in your account, including Graviton4 and memory-optimized instance types. AWS handles underlying infrastructure lifecycle including provisioning, scaling, patching, and routing, while you benefit from Amazon EC2 pricing advantages like Savings Plans and Reserved Instances.

Key benefits include:

Flexible instance selection: Choose from compute-optimized (C), general-purpose (M), and memory-optimized (R) instance families
Configurable memory-CPU ratios: Optimize resource allocation for your workload
Multi-concurrent invocations: One execution environment handles multiple invocations simultaneously, improving utilization for I/O-heavy applications
Dynamic scaling: Instances scale based on CPU utilization without cold starts

AWS Lambda Managed Instances is best suited for high-volume, predictable workloads that benefit from sustained compute capacity and larger memory configurations.

Memory-Intensive Workloads Work Best with AWS Lambda Managed Instances

This blog focuses on one of AWS Lambda Managed Instances’ most powerful capabilities: running memory-intensive workloads that require more than the standard AWS Lambda’s 10 GB memory and 250MB ZIP limits. Here are the use cases where AWS Lambda Managed Instances helps:

In-Memory Analytics — Load gigabytes of structured data into memory at initialization and serve sub-millisecond analytical queries across thousands of invocations
ML Model Inference — Keep large model weights resident in memory across invocations for consistent, low-latency inference without a dedicated endpoint.
Real-Time Semantic Search — Build vector similarity search over large embedding indexes held entirely in memory, enabling natural language queries over millions of records without an external vector database.
Graph Processing — Hold large graph structures in memory for traversal algorithms that require the full graph to be accessible at once.
Scientific & Numerical Computing — Run simulations, Monte Carlo methods, and large matrix operations that require substantial working memory and benefit from memory-optimized Amazon EC2 instance families.
Large-Scale Report Generation — Aggregate and transform multi-gigabyte datasets in memory to generate complex reports or dashboards on demand, without staging data through intermediate storage.

Use Case: AI-Powered Customer Analytics with AWS Lambda Managed Instances

To demonstrate the power of AWS Lambda Managed Instances for memory-intensive applications, we built an AI-Powered Customer Analytics application that combines in-memory data processing with ML-based semantic search. The application loads in memory 1 million customer behavioral records (sessions, purchases, browsing patterns) from a Parquet file in S3 into a Pandas DataFrame and an embeddings cache consuming 200MB, then responds for analytics queries:

Customer Analysis — Deep-dive into individual customer behavior: engagement scores, conversion rates, purchase patterns, and AI-generated customer segments
Semantic Search — Natural language queries powered by FastEmbed (sentence-transformers/all-MiniLM-L6-v2) that find similar customers using vector similarity
Cohort Analysis — Real-time segmentation by device, country, age group with aggregated metrics

Architecture Overview

Our AI-powered customer analytics application demonstrates this in practice: 1 million records in memory (200MB), a compact sentence transformer model for semantic search, sub-second query performance, and zero infrastructure to manage. The solution uses a simple, serverless architecture:

Customer transaction data (Parquet format) is stored in Amazon S3
Amazon Cognito User Pool authenticates users and issues JWT tokens for API access
Amazon API Gateway routes requests with Cognito authorizer validation, rate limiting (5 requests/second, burst 10), X-Ray tracing, and access logging
AWS Lambda function with AWS Lambda Managed Instances loads the entire dataset (200MB) and all-MiniLM-L6-v2 model (900MB) into memory during initialization while also performing a threaded embeddings cache generation. This step can consume about 14GB of the allocated memory, exceeding standard AWS Lambda’s 10 GB limit
Analytics queries execute against the in-memory data using the model
Results are returned in milliseconds for interactive analysis

Deploy the Application

The below steps walk you through deploying the application to AWS using the AWS Serverless Application Model (SAM). The deployment process packages your Lambda function code, uploads artifacts to Amazon S3, and provisions all required AWS resources including Lambda functions, IAM roles, and any configured VPC networking via AWS CloudFormation.

Prerequisites

Make sure you have the following tools installed locally:

AWS CLI configured with credentials
SAM CLI installed
Python 3.13+ installed locally
Docker or Finch (required for container builds)
AWS account with appropriate permissions
A VPC with at least 2 subnets (across different Availability Zones) and a security group — required for the Lambda Managed Instances capacity provider
Supported regions: Check AWS Capabilities by Region for supported regions

Getting Started

The complete source code for this application is available in our GitHub repository. To deploy it yourself follow the below steps and refer to the full deployment instructions hosted on GitHub.

1. Clone the repository

git clone https://github.com/aws-samples/sample-lambda-managed-instances-analytics.git

2. Navigate to the project folder

cd sample-lambda-managed-instances-analytics

chmod +x setup-data.sh deploy-lambda.sh

3. Generate sample data and upload to S3

./setup-data.sh

This script will create an S3 bucket (if needed), generate 1M rows of sample data, and upload the data to S3.

4. Build and deploy the Lambda function

./deploy-lambda.sh

This script will build the container image with FastEmbed, push it to ECR, and deploy the Lambda function along with Capacity Provider, API Gateway, and Cognito User Pool. After deployment, it automatically generates the UI authentication configuration and prompts you to create a test user.

Run the Application

1. Start the UI

The application includes a simple HTML-based UI through which you can test the AWS Lambda function using Amazon API Gateway:

cd ui && python3 -m http.server 8000

2. Open your browser at http://localhost:8000 and click ‘Sign In’ to authenticate via Cognito using the username/password that you created during deployment

3. Enter your API endpoint URL. Test connection and click system Info.

Test the Application

a. Customer Analysis — Enter one or more User IDs to get more information on the customer behavior: engagement scores, conversion rates, purchase patterns, and AI-generated customer segments

b. Semantic Search – Enter natural language queries like “list high value customers from USA” in the Semantic Search and verify the results. Note that the response is very fast as the analytics data and FastEmbed models are loaded into memory during init stage

c. Cohort Analysis — Enter the query data to get Real-time segmentation by device, country, age group with aggregated metrics

Observability

AWS Lambda Managed Instances automatically publishes metrics to Amazon CloudWatch, giving you visibility into function performance and capacity utilization. Monitor InitDuration to track dataset and model load time at startup, MaxMemoryUsed to confirm your data fits within configured memory, and ProvisionedConcurrencySpilloverInvocations to detect when AWS Lambda Managed Instances capacity is exhausted.

Enable AWS Lambda Insights for enhanced per-invocation metrics including CPU time and memory utilization over time. Use Amazon CloudWatch Log Insights to query INIT_START, INIT_END, and REPORT log entries for initialization and memory details per invocation.

What Makes This Better with AWS Lambda Managed Instances

Without AWS Lambda Managed Instances, building this same application would require one of these alternatives:

Option A: EC2 with auto-scaling — Full control, full responsibility: patching, scaling policies, load balancing, and deployment pipelines — all on you.
Option B: Redesign for standard Lambda — Swap in-memory data for an external database and replace the ML model with Amazon SageMaker endpoint. More latency, more cost, more complexity.

With AWS Lambda Managed Instances, you write a single AWS Lambda function, define a Capacity Provider, and deploy with SAM. AWS Lambda handles the Amazon EC2 instances, scaling, and lifecycle, giving you the memory you need with the operational simplicity you want. The in-memory approach eliminates network latency and disk I/O, delivering consistent sub-200ms response times for complex analytics.

Cost Considerations

AWS Lambda Managed Instances uses Amazon EC2-based pricing with a management fee. For predictable workloads, you can leverage Amazon EC2 Savings Plans or Reserved Instances to reduce costs significantly.

Example cost comparison (us-east-1, 32 GB memory, 1M invocations/month):

AWS Lambda (standard): ~$267/month (on-demand pricing)
AWS Lambda Managed Instances: ~$180/month (with 1-year Compute Savings Plan)
Savings: 33% reduction

The cost benefits increase with higher memory configurations and sustained workloads that can take advantage of Amazon EC2 pricing discounts.

Best Practices

Based on experience building this solution, here are key recommendations:

Memory sizing: Start with your dataset size plus 50% overhead for processing. Monitor Amazon CloudWatch metrics to optimize.
Initialization strategy: Load large datasets during the init phase to amortize the cost across multiple invocations.
Concurrency configuration: Set PerExecutionEnvironmentMaxConcurrency based on your workload’s I/O characteristics. Higher values work well for I/O-bound analytics.
Data format: Use columnar formats like Parquet for efficient memory usage and fast loading.
Monitoring: Track initialization duration, memory utilization, and invocation latency in Amazon CloudWatch to identify optimization opportunities.

Cleanup

When you’re done exploring the solution, it’s good practice to remove all provisioned resources to avoid ongoing charges. For the full cleanup commands and exact steps, refer to the project’s README.md in GitHub repository.

Conclusion

AWS Lambda Managed Instances opens up a new class of serverless applications that support larger AWS Lambda layer packages and more memory. Memory-intensive workloads — in-memory analytics, ML inference, graph processing, scientific computing — can now run with the simplicity of AWS Lambda and the resources of Amazon EC2. The customer analytics example demonstrates how in-memory processing with AWS Lambda Managed Instances delivers performance improvements over traditional database queries while maintaining serverless benefits like automatic scaling and pay-per-use pricing.

Ready to get started? Explore the AWS Lambda Managed Instances documentation and try building your own memory-intensive serverless application. You can find the complete code for this example on GitHub.

Accelerate CPU-based AI inference workloads using Intel AMX on Amazon EC2

Santosh Kumar — Mon, 30 Mar 2026 16:43:10 +0000

This post shows you how to accelerate your AI inference workloads by up to 76% using Intel Advanced Matrix Extensions (AMX) – an accelerator that uses specialized hardware and instructions to perform matrix operations directly on processor cores – on Amazon Elastic Compute Cloud (Amazon EC2) 8th generation instances. You’ll learn when CPU-based inference is cost-effective, how to enable AMX with minimal code changes, and which configurations deliver optimal performance for your models.

Many organizations find that CPU-based inference is more suitable for their production Artificial Intelligence/Machine Learning (AI/ML) workloads after evaluating factors like cost, operational complexity, and infrastructure compatibility. As more organizations deploy AI solutions, improving how models run on standard CPUs has become a critical cost control strategy for workloads where CPU inference provides the right balance of performance and economics.

IDC, a global market intelligence and advisory firm, projects that worldwide AI spending will reach $632 billion by 2028, growing at a 29% compound annual growth rate from 2024, with inference costs representing a significant portion of operational expenses. Deloitte, a leading professional services firm specializing in technology consulting and research, forecasts that inference – the running of AI models – will make up two-thirds of all AI compute by 2026, far exceeding initial training costs. This makes optimizing AI/ML inference on CPU crucial for controlling long-term AI/ML operational expenses.

At the core of AI inference workloads are matrix multiplication operations – the mathematical foundation of neural networks that drives computational demand. These matrix-heavy operations create a performance bottleneck for CPU-based inference, resulting in suboptimal performance for AI/ML workloads. This creates three key challenges for organizations: balancing cost optimization with performance requirements, meeting real-time latency demands, and scaling efficiently with variable workload demands. Intel’s Advanced Matrix Extensions (AMX) technology addresses these challenges by accelerating matrix operations directly on CPU cores, making CPU-based inference competitive and cost-effective.

AMX capabilities and architecture

AMX supports multiple data formats including BF16 which preserves the range of 32-bit floating point operations in half the space, INT8 maximizes throughput when accuracy can be slightly compromised, and FP16 offers a balance between the two. This flexibility lets you match precision to your specific needs.

Introduced in 2023 with 4th Generation Intel Xeon Scalable processors, AMX consists of eight 1KB tile registers (specialized on-chip memory for matrix data) and a Tile Matrix Multiply Unit (TMUL – dedicated hardware for matrix calculations) that enables processors to perform 2048 INT8 operations or 1024 BF16 operations per cycle. These tile registers provide efficient matrix storage, reducing memory access overhead and improving computational efficiency for matrix operations central to neural networks. For real-world customer workloads, this translates to significantly faster inference times for transformer models, recommendation systems, and natural language processing tasks, while reducing the total cost of ownership through improved resource utilization and lower infrastructure requirements.

Figure 1: AMX Architecture showing AMX tile registers, processing units, and data flow within CPU core

Note: AMX operations, including tile setup and memory-to-tile data movement (which are handled automatically by the system), introduce small overhead that may outweigh benefits for smaller models or single-batch processing where insufficient matrix operations cannot amortize these costs, making batch size optimization critical for performance gains.

When to choose CPU inference with AMX

CPU inference with AMX acceleration benefits workloads including:

Batch processing and traditional ML: Content summarization, recommendation systems, and analytical workloads benefit from CPU’s cost efficiency and ability to handle sparse data structures and branching logic.

Small to medium-sized models: Models under 7B parameters and batch sizes of 8-16 samples achieve excellent performance through optimized threading, making CPUs ideal for applications like fraud detection and chatbots.

Variable demand workloads: E-commerce systems and applications with unpredictable traffic patterns can quickly scale CPU instances up or down based on demand, avoiding the fixed costs of dedicated accelerator hardware that sits idle during low-traffic periods.

Complex business logic: Applications like financial risk assessment and content moderation that need to combine ML predictions with business rules and conditional logic work well on CPUs, which handle mixed workloads better than specialized accelerators.

Implementation: AMX optimization with PyTorch

PyTorch, a popular open-source machine learning framework, includes built-in Intel optimizations through oneDNN (Intel’s Deep Neural Network library) that automatically use AMX when available. Setup requires installing dependencies and configuring environment variables for optimal performance.

Install dependencies

# Install transformers and torch
pip install torch transformers

Configure environment variables

These environment variables tell oneDNN library how to optimize your inference workload for AMX.

Enable AMX instruction set (tells oneDNN to use AMX tiles for matrix operations):
```
export DNNL_MAX_CPU_ISA=AVX512_CORE_AMX
```
Optimize thread affinity (binds threads to CPU cores for better cache performance):
```
export KMP_AFFINITY=granularity=fine,compact,1,0
```
Use all available CPU cores for parallel processing:
```
export OMP_NUM_THREADS=$(nproc)
```
Cache compiled kernels (avoids recompilation overhead on subsequent runs):
```
export ONEDNN_PRIMITIVE_CACHE_CAPACITY=4096
```
Set default precision to BF16 (enables automatic AMX acceleration):
```
export ONEDNN_DEFAULT_FPMATH_MODE=bf16
```
(Optional) Enable verbose logging to verify AMX activation:
```
export ONEDNN_VERBOSE=1
```

BF16 optimization example

With environment variables configured, implementing BF16 optimization requires minimal to no code changes. The following example demonstrates how PyTorch automatically leverages AMX tile registers for matrix operations when BF16 precision is used.

Note: This is a simplified example for demonstration purposes; adapt the code to your specific use case and requirements.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import time

# Load model and tokenizer from HuggingFace
model_name = "google/gemma-3-1b-it"

model_revision = "dcc83ea841ab6100d6b47a070329e1ba4cf78752"
tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    revision=model_revision
)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    revision=model_revision
)
# Fix tokenizer padding issue for batch processing
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# Enable BF16 precision for automatic AMX acceleration
model = model.to(dtype=torch.bfloat16)
model.eval()  # Set to inference mode

# Inference function with BF16 autocast
def run_optimized_inference(prompts):
    inputs = tokenizer(prompts, padding=True, 
                      return_tensors="pt")  # Tokenize input
    
    with torch.no_grad():  # Disable gradients for inference
        with torch.amp.autocast('cpu',
                               dtype=torch.bfloat16):  # BF16 autocast
            outputs = model.generate(
                **inputs,
                max_length=100,     # Set maximum sequence length 
                do_sample=False     # Use greedy decoding
            )
    return outputs

# Example usage with performance measurement
prompts = ["What are the benefits of cloud computing?"]
start_time = time.time()
results = run_optimized_inference(prompts)  # Run BF16-optimized inference
elapsed_time = time.time() - start_time
tokens_generated = len(results[0]) - len(tokenizer.encode(
    prompts[0]))  # Count new tokens

# Display results and performance metrics
print(tokenizer.decode(results[0], skip_special_tokens=True))
print(f"Latency: {elapsed_time*1000:.1f}ms, "
      f"Throughput: {tokens_generated/elapsed_time:.1f} "
      f"tokens/sec")

Performance benchmarks

To validate AMX performance benefits, we conducted benchmarks across multiple popular language models representing different use cases and model sizes.

Benchmarking methodology and environment

We tested two improvements: hardware generation advances (m8i vs m7i) and AMX optimization impact (FP32 vs BF16). This shows you both upgrade paths for your workloads.

Models tested: BigBird-RoBERTa-large (355M), Microsoft DialoGPT-large (762M), Google Gemma-3-1b-it (1B), DeepSeek-R1-Distill-Qwen-1.5B (1.5B), Llama-3.2-3B-Instruct (3B), YOLOv5 (tested with 30 images at ~1200×800 resolution with 5 iterations for each image)
Amazon EC2 instance types: m8i.4xlarge, m7i.4xlarge (8^th & 7^th gen general-purpose Amazon EC2 instances with 16 vCPUs and 64 GiB memory, both AMX-capable)
Batch sizes: 1, 8, 32 (number of input samples processed simultaneously in a single inference call)
Iterations: 5 runs per configuration
Comparison types:
- Instance generation comparison (m8i vs m7i performance)
- AMX optimization impact (32-bit floating-point (FP32) vs Brain Floating Point 16 (BF16) on same instance)
Optimizations: FP32 baseline vs BF16 AMX
Framework: PyTorch 2.8.0 (which has built-in Intel optimizations)
Region: AWS us-west-2
Measurement methodology: In our benchmarks, ‘inference latency’ represents the complete model inference execution time including input tokenization and full sequence generation (for generative models) or complete forward pass (for non-generative models). Each measurement is the average of 5 iterations after warm-up iterations, excluding model loading time. We use this metric because AMX’s matrix multiplication acceleration improves performance throughout the complete forward pass.

Note: Throughout this blog, FP32 refers to the default 32-bit floating-point precision, while BF16 refers to Brain Floating Point 16-bit precision with AMX acceleration enabled.

Disclaimer: Performance results are based on internal testing and may vary depending on specific workloads, configurations, and environments.

Detailed result: BigBird-RoBERTa-large

This benchmark represents document classification, content summarization, and text analysis workloads typical in batch processing where high throughput is desirable and offline inference scenarios where strict latency requirements are not critical.

Figure 2: m7i.4xlarge vs m8i.4xlarge inference latency comparison for model BigBird-RoBERTa-large (355M parameters)

Figure 3: m7i.4xlarge vs m8i.4xlarge throughput comparison for BigBird-RoBERTa-large model across batch sizes 1, 8, and 32

Figure 4: FP32 vs BF16 inference latency comparison for model BigBird-RoBERTa-large (355M parameters) on m7i.4xlarge and m8i.4xlarge instances across batch sizes

BigBird-RoBERTa-large model benchmarking demonstrates three key performance improvements. Figure 2 shows m8i hardware delivers 4-20% latency reduction across batch sizes compared to m7i for both FP32 and BF16 with AMX, providing immediate benefits without application changes. With AMX and BF16, performance gains decrease at higher batch sizes as AMX overhead exceeds benefits for smaller models like BigBird-RoBERTa-large. Figure 3 validates these improvements with corresponding 4-25% throughput gains, enabling better resource utilization for production applications. Figure 4 demonstrates that enabling AMX with BF16 optimization provides the most significant impact, reducing m8i latency by 55-67% compared to non-AMX FP32 baseline, enabling 2-3x higher processing capacity and reduced compute costs.

The analysis above demonstrates the methodology for interpreting benchmark results using BigBird-RoBERTa-large as a representative example. The remaining models (DialoGPT-large, Gemma-3-1b-it, DeepSeek-R1-Distill-Qwen-1.5B, and Llama-3.2-3B-Instruct) follow identical testing procedures and exhibit similar performance patterns, with variations primarily in the magnitude of improvements based on model size and architecture. The comprehensive analysis of five models and their performance implications are synthesized in the following section.

Benchmarking result for additional models

To validate AMX’s effectiveness across diverse AI workloads, we benchmarked five additional models representing different use cases and model sizes. Each model follows the same testing methodology described above, with performance patterns showing how AMX benefits vary based on model architecture, parameter count, and batch size.

DialoGPT-large (762M) – Conversational AI

This benchmark represents conversational AI, chatbots, and real-time dialogue systems where low latency and consistent response times are critical for user experience.

Figure 5: m7i.4xlarge vs m8i.4xlarge inference latency comparison for model DialoGPT-large (762M parameters)

Figure 6: m7i.4xlarge vs m8i.4xlarge throughput comparison for DialoGPT-large model across batch sizes 1, 8, and 32

Figure 7: FP32 vs BF16 inference latency comparison for model DialoGPT-large (762M parameters) on m7i.4xlarge and m8i.4xlarge instances across batch sizes

Gemma-3-1b-it (1B) – General Purpose

This benchmark represents general-purpose language understanding tasks, content generation, and smaller model deployments suitable for cost-sensitive applications and variable demand workloads.

Figure 8: M7i.4xlarge vs M8i.4xlarge inference latency comparison for model Gemma-3-1b-it (1B parameters)

Figure 9: m7i.4xlarge vs m8i.4xlarge latency and throughput comparison for Gemma-3-1b-it across model batch sizes 1, 8, and 32

Figure 10: FP32 vs BF16 inference latency comparison for model Gemma-3-1b-it (1B parameters) on m7i.4xlarge and m8i.4xlarge instances across batch sizes

DeepSeek-R1-Distill-Qwen-1.5B (1.5B) – Reasoning

This benchmark represents reasoning and analytical workloads, including complex decision-making systems, financial analysis, and applications requiring sophisticated logic processing.

Figure 11: m7i.4xlarge vs m8i.4xlarge inference latency comparison for model DeepSeek-R1-Distill-Qwen-1.5B (1.5B parameters)

Figure 12: m7i.4xlarge vs m8i.4xlarge latency and throughput comparison for DeepSeek-R1-Distill-Qwen-1.5B model across batch sizes 1, 8, and 32

Figure 13: FP32 vs BF16 inference latency comparison for model DeepSeek-R1-Distill-Qwen-1.5B (1.5B parameters) on m7i.4xlarge and m8i.4xlarge instances across batch sizes

Llama-3.2-3B-Instruct (3B) – Large model

This benchmark represents larger model deployments for complex instruction-following tasks, advanced content generation, and applications requiring higher model capacity while maintaining cost efficiency.

Figure 14: m7i.4xlarge vs m8i.4xlarge inference latency comparison for model Llama-3.2-3B-Instruct (3B parameters)

Figure 15: m7i.4xlarge vs m8i.4xlarge latency and throughput comparison for Llama-3.2-3B-Instruct model across batch sizes 1, 8, and 32

Figure 16: FP32 vs BF16 inference latency comparison for model Llama-3.2-3B-Instruct (3B parameters) on m7i.4xlarge and m8i.4xlarge instances across batch sizes

Yolov5 – Computer vision model

This benchmark represents computer vision workloads including object detection, image classification, and real-time video processing applications where consistent throughput is important for production deployments.

Instance type	Inference latency in Sec (Processing time per image)		Throughput (Image processed per sec)
Instance type	FP32	BF16	FP32	BF16
m8i.4xlarge	0.034	0.029	29.23	34.63
m7i.4xlarge	0.038	0.031	26.39	32.28
m8i improvement	10.5%	6.5%	10.8%	7.3%

Key insights: m8i instances deliver 7-11% better performance than m7i across both precision formats. Combining hardware upgrade with AMX optimization, m8i with BF16 delivers up to 24% lower latency and 31% higher throughput compared to m7i with FP32.

Benchmark result summary

The detailed graphs above demonstrate consistent performance patterns across tested models. Key findings:

M8i vs M7i instance performance

m8i instances deliver 9-14% average and up to 20% better performance than m7i across the tested models through hardware advances: up to 4.6x larger L3 cache, higher base frequencies, up to 2.5x higher DDR5 bandwidth, and enhanced AMX execution with FP16 support.

Model	Use Case	m8i average latency improvement*
BigBird-RoBERTa-large (355M)	Document analysis	10%
DialoGPT-large (762M)	Conversational AI	14%
Gemma-3-1b-it (1B)	General purpose	10%
DeepSeek-R1 (1.5B)	Reasoning tasks	11%
Llama-3.2-3B (3B)	Large model deployment	12%
YOLOv5	Computer vision	9%

* Average across all tested configurations (FP32 and BF16 at batch sizes 1, 8, and 32)

AMX acceleration impact (FP32 vs BF16)

BF16 precision with AMX delivers 21-72% performance improvements at batch sizes of 8 and above compared to FP32 baseline on the same instance type. These results compare FP32 vs BF16 performance on m8i.4xlarge, with performance gains varying by model size and batch configuration. Larger batch sizes show greater AMX benefits.

Model	Latency improvement (%)
Model	Batch 1	Batch 8	Batch 32
BigBird-RoBERTa-large	55	67	63
DialoGPT-large	– 44*	21	30
Gemma-3-1b-it	6	42	24
DeepSeek-R1	24	68	59
Llama-3.2-3B	27	72	68

* At batch size 1, DialoGPT-large’s autoregressive decoding generates tokens sequentially, producing many small matrix operations where AMX tile setup overhead exceeds the acceleration benefit. At batch sizes 8 and above, multiple sequences are processed in parallel, creating larger matrix operations that amortize this overhead and deliver 21-30% improvement.

Performance patterns by batch size

Larger models (1B+ parameters) show consistently better AMX performance across the tested batch sizes:

Batch size 1: Mixed results – larger models show 6-27% improvement, smaller models may experience AMX overhead
Batch size 8: Strong performance gains of 21-72% across the tested models, with larger models showing greater benefits
Batch size 32: Significant improvements of 24-68% for most models, demonstrating AMX’s batch processing strength

Batch size optimization guidelines

AMX performance scales with batch size, with optimal range varies by model size. Performance saturates beyond batch 16 due to hardware limits including memory bandwidth and compute bottlenecks.

Model Size	Performance Gain	Recommended Batch Size	Notes
<1B parameters	21-67%	8-32	Batch 1 results vary by architecture*
1-2B parameters	42-68%	4-16	6-24% gains even at batch 1
3B+ parameters	27-72%	1-8	Benefits across batch sizes

* Encoder models (BigBird) show 55% gains at batch 1; autoregressive models (DialoGPT) may experience overhead.

Combined performance benefits

When we combine AMX optimization with 8th generation instances (m8i), the performance improvements compound significantly. For example, Llama-3.2-3B-Instruct running with BF16 AMX on m8i instances can achieve up to 76% better performance compared to FP32 inference on m7i instances at optimal batch sizes (batch 8: m7i FP32 45.51s vs m8i BF16 10.93s = 76% improvement; batch 32: m7i FP32 62.60s vs m8i BF16 17.47s = 72% improvement).

Throughput scaling

Across the tested models, throughput (tokens/sec) increases proportionally with latency reduction. This consistent relationship demonstrates that AMX optimizations translate directly to improved inference efficiency.

Price-Performance Analysis: Gemma-3-1b-it Model

While m8i.4xlarge instances are priced slightly higher than m7i.4xlarge ($0.847 vs $0.806 per hour in us-west-2), they deliver superior price-performance. To illustrate the economic benefits, we analyzed cost per 1 million tokens using Gemma-3-1b-it as a representative example. M8i delivers up to 13% better price-performance over m7i through hardware generation advances, with both instances running BF16 AMX.

Batch Size	Data Type	m7i.4xlarge		m8i.4xlarge		Price-Performance improvement
Batch Size	Data Type	Throughput (tokens/sec)	$ per 1M token	Throughput (tokens/sec)	$ per 1M token	Price-Performance improvement
1	BF16(AMX)	14.3	$15.66	17.2	$13.67	13%
8	BF16(AMX)	71	$3.16	82.3	$2.86	9%
32	BF16(AMX)	119.1	$1.88	127.8	$1.84	2%

Combining the hardware upgrade with BF16 AMX optimization delivers up to 44% better price-performance compared to FP32 on m7i.

Batch Size	m8i.4xlarge			m7i.4xlarge			Price-Performance improvement
Batch Size	Data Type	Throughput (tokens/sec)	$ per 1M token	Data Type	Throughput (tokens/sec)	$ per 1M token	Price-Performance improvement
1	BF16(AMX)	17.2	$13.67	FP32	14.9	$15.03	9%
8	BF16(AMX)	82.3	$2.86	FP32	44.1	$5.08	44%
32	BF16(AMX)	127.8	$1.84	FP32	89.2	$2.51	27%

Key findings from the price-performance analysis:

Combined optimization delivers up to 44% better price-performance: m8i with AMX and BF16 outperforms m7i with FP32 at batch size 8 – consistent with our batch size optimization guidelines where batch sizes of 4-16 deliver optimal results for 1B models like Gemma-3-1b-it, achieving $2.86 per 1M tokens for applications like chatbots and fraud detection.
Larger batches maximize cost efficiency: Batch size 32 reduces costs further to $1.84 per 1M tokens, a 27% improvement over m7i FP32 – ideal for throughput-oriented workloads like content summarization and recommendation systems where latency requirements are flexible.

Production deployment recommendation

BF16 AMX: Delivers 21-72% performance improvements at recommended batch sizes while maintaining model accuracy, making it suitable for production workloads including fraud detection systems, content moderation, and real-time recommendation engines
Batch processing: Target batch sizes of 4-16 based on your use case – smaller batches (1-4) for latency-sensitive applications like chatbots, larger batches (8-16) for throughput-focused scenarios like document analysis and offline processing
Instance selection: m8i instances provide consistent 9-14% performance improvements over m7i, delivering immediate ROI for existing CPU inference workloads without requiring application changes
Model size consideration: Larger models (1B+ parameters) show better AMX utilization across batch sizes, making them ideal candidates for m8i deployment in complex reasoning and content generation applications

Conclusion and next steps

By using Intel AMX on Amazon EC2 8th generation instances, you can achieve substantial performance improvements for AI inference workloads. Our benchmarks demonstrate up to 72% performance improvements across popular language models, making CPU inference more competitive for batch processing, real-time applications, recommender systems, and variable demand workloads while delivering substantial cost savings through improved resource utilization.

Key takeaways:

BF16 AMX optimization delivers up to 72% performance improvements across model sizes, with batch 8 showing 21-72% gains and batch 32 showing 24-68% gains
Batch sizes of 4-8 provide optimal performance for most models—DialoGPT achieves 21% improvement in latency at batch 8, while Llama-3.2-3B achieves 72% improvement
8th generation instances deliver up to 14% performance improvements over m7i across the tested workloads
Combined optimizations (m8i + BF16 AMX) can achieve compound performance improvements up to 76% in optimal configurations (vs m7i FP32), making CPU inference highly competitive for cost-sensitive applications
M8i instances deliver up to 13% better price-performance vs m7i (lower cost per 1M tokens), based on our analysis of the Gemma-3-1b-it model
Proper environment configuration is critical for AMX activation

You can implement these optimizations immediately. AMX hardware acceleration combined with PyTorch’s Intel-specific enhancements requires configuring environment variables while delivering substantial speed gains. Begin with BF16 optimization on your existing models, then explore INT8 quantization for additional gains.

Next steps:

Launch an Intel based Amazon EC2 8th generation instance (m8i.4xlarge)
Install PyTorch (includes built-in Intel optimizations)
Configure AMX environment variables
Measure performance improvements
Scale your optimized inference workloads

Additional resources

Build high-performance apps with AWS Lambda Managed Instances

Debasis Rath — Mon, 30 Mar 2026 14:53:01 +0000

High-performance applications such as CPU-intensive processing, memory-heavy analytics, and steady-state data pipelines often require more predictable compute resources than standard AWS Lambda configurations provide. AWS Lambda Managed Instances (LMI) addresses this by letting you run Lambda functions on selected Amazon EC2 instance types while preserving the Lambda programming model. You can choose over 400 Amazon Elastic Compute Cloud (Amazon EC2) instance types from general purpose, compute optimized, or memory optimized instance families to match workload requirements. AWS Lambda continues to manage infrastructure operations such as instance lifecycle management, operating system patching, runtime updates, request routing, and automatic scaling. This approach gives your teams greater control over compute characteristics, EC2 pricing model and reduces operational overhead of managing servers or clusters.

In this post, you will learn how to configure AWS Lambda Managed Instances by creating a Capacity Provider that defines your compute infrastructure, associating your Lambda function with that provider, and publishing a function version to provision the execution environments. We will conclude with production best practices including scaling strategies, thread safety, and observability for reliable performance.

Figure 1. Creating Function on LMI

Creating Capacity Providers

A Capacity Provider defines the infrastructure blueprint for running LMI functions on Amazon EC2. It specifies instance types, network placement, and scaling behavior. To create a Capacity Provider, you need two parameters: an IAM role (Capacity Provider Operator Role) granting Lambda permissions to launch and manage instances and your VPC configuration with subnets and security groups. Create this role in your account with the AWSLambdaManagedEC2ResourceOperator managed policy following the Principle of Least Privilege (granting only the minimum permissions necessary).

This command creates a Capacity Provider with instance types and scaling configuration:

aws lambda create-capacity-provider \
  --capacity-provider-name my-lmi-capacity \
  --vpc-config SubnetIds=subnet-abc123,subnet-def456,SecurityGroupIds=sg-xyz789 \
  --permissions-config CapacityProviderOperatorRoleArn=arn:aws:iam::123456789012:role/LMIOperatorRole \
  --instance-requirements Architectures=x86_64,AllowedInstanceTypes=c5.2xlarge,r5.4xlarge \
  --capacity-provider-scaling-config MaxVCpuCount=50,ScalingMode=Auto \
  --region us-east-1

This command returns a Capacity Provider ARN that you’ll use to create your LMI function. Your functions behavior depends on four main configurations in the capacity provider:

Instance selection

Lambda currently supports three Amazon EC2 instance families (.large and up): C (compute optimized) for CPU-heavy work, M (general purpose) for balanced workloads, and R (memory optimized) for large datasets. Choose x86 (Intel/AMD) or ARM (Graviton) architectures. If you don’t specify instance types, Lambda defaults to appropriate instances based on your function’s memory and CPU configuration. This is the recommended starting point unless you have specific performance requirements. When you need more control, use AllowedInstanceTypes to specify only the instance types that Lambda can use or use ExcludedInstanceTypes to exclude specific types while allowing all other instance types. You can’t use both parameters together.

VPC and networking

Configure multiple subnets across Availability Zones. Lambda creates a minimum Amazon EC2 fleet of three instances distributed across your configured Availability Zones to maintain availability and resiliency. Egress traffic from functions, including Amazon CloudWatch Logs, transits through the Amazon EC2 instance’s network interface in your Amazon Virtual Private Cloud (Amazon VPC). As functions send logs and metrics to CloudWatch, you will need internet access through a NAT Gateway or VPC endpoints with AWS PrivateLink for Amazon CloudWatch. This only affects egress traffic; function invoke requests don’t flow through your VPC. Security groups attached to your instances should allow only the traffic your function code needs. With LMI, configure VPC once at the Capacity Provider level instead of per function, simplifying management for multiple LMI functions. Standard Lambda functions continue to use their own VPC configurations. This Capacity Provider VPC configuration applies only to LMI functions.

Figure 2. LMI Networking

Scaling configuration

Set MaxVCpuCount to cap compute capacity and control costs. New invocations throttle when you reach this limit until capacity frees up. Lambda monitors CPU utilization and scales instances automatically. Choose automatic scaling mode where Lambda tunes thresholds based on load patterns, or manual mode where you set a target CPU utilization percentage. Multiple functions can share the same Capacity Provider to reduce costs through better resource utilization, though you might want separate providers for functions with different performance or isolation requirements.

Security

Lambda encrypts Amazon Elastic Block Store (Amazon EBS) volumes attached to EC2 instances with a service-managed key by default. You can provide your own AWS Key Management Service (AWS KMS) key for encryption. Place instances in private subnets with restrictive security groups for enhanced security.

Creating Lambda Managed Instance Functions

You create an LMI function similarly to creating a standard Lambda function. You package your code, set your runtime, assign an execution role, and configure memory. The difference is specifying a CapacityProviderConfig to tell Lambda which Capacity Provider to use and how to size each execution environment. Specify CapacityProviderConfig during function creation with the Capacity Provider ARN and configure two execution environment settings. ExecutionEnvironmentMemoryGiBPerVCpu sets the memory-to-vCPU ratio (2:1, 4:1, or 8:1) based on your workload type and PerExecutionEnvironmentMaxConcurrency defines how many concurrent requests share each execution environment. This table shows how memory and vCPU allocation maps across supported execution environment ratio.

2:1 Ratio(Compute optimized)		4:1 Ratio(General purpose)		8:1 Ratio(Memory optimized)
Memory (GB)	vCPU(s)	Memory (GB)	vCPU(s)	Memory (GB)	vCPU(s)
2	1	4	1	8	1
4	2	8	2	16	2
6	3	12	3	24	3
8	4	16	4	32	4
10	5	20	5
12	6	24	6
14	7	28	7
16	8	32	8
…	…
32	16

Function Memory-to-CPU configuration

Set the function’s memory size (up to 32 GB for LMI) and ExecutionEnvironmentMemoryGiBPerVCpu ratio. The default ratio is 2:1. A 2:1 ratio map to compute optimized instances for CPU-intensive tasks like video encoding, 4:1 map to a general purpose for balanced workloads, and 8:1 maps to a memory optimized instances for large in-memory datasets or caching. You must set memory in multiples of the ratio. LMI requires a 2 GB minimum as execution environments need sufficient memory to handle multiple concurrent requests. LMI supports up to 32 GB memory per execution environment.

Multi-Concurrency settings

LMI supports multiple concurrent invocations sharing the same execution environment, reducing cost per invocation by maximizing vCPU utilization. This is particularly effective for I/O-bound workloads, where invocations waiting on database queries or API calls yield vCPU usage to other invocations during idle periods. Lambda defaults to max concurrency per execution environment based on your runtime: Node.js (64 per vCPU), Java, and .NET (32 per vCPU), Python (16 per vCPU). Use PerExecutionEnvironmentMaxConcurrency to set a lower limit based on your workload’s resource needs. Decrease it if you’re experiencing memory pressure or CPU contention. When environments reach their configured max concurrency, new invocations throttle until capacity frees up at the execution environment level. This table captures the maximum concurrency per vCPU for each supported programming language.

Language	Default Max Concurrency
Node.js	64 per vCPU
Java	32 per vCPU
.NET	32 per vCPU
Python	16 per vCPU

This command creates a Lambda function and associates it with your Capacity Provider:

aws lambda create-function \
  --function-name my-lmi-function \
  --runtime python3.13 \
  --role arn:aws:iam::123456789012:role/LambdaExecutionRole \
  --handler app.lambda_handler \
  --zip-file fileb://function.zip \
  --memory-size 4096 \
  --capacity-provider-config '{
    "LambdaManagedInstancesCapacityProviderConfig": {
      "CapacityProviderArn": "arn:aws:lambda:us-east-1:123456789012:capacity-provider:my-lmi-capacity",
      "ExecutionEnvironmentMemoryGiBPerVCpu": 4.0,
      "PerExecutionEnvironmentMaxConcurrency": 10
    }
  }' \
  --region us-east-1

Publishing Lambda Managed Instance Functions

Important: publish a function version before invoking an LMI function. Publishing triggers Lambda to provision Amazon EC2 instances and initialize execution environments, so that the configured baseline capacity is ready before you start invoking. Expect a brief delay before your code goes live as Lambda provisions and launches Amazon EC2 instances. With LMI, execution environments pre-warm after publishing and remain invoke-ready, without cold starts for published versions. Standard Lambda environments initialize on first invoke (cold starts).

This command publishes a Lambda function version and provisions capacity:

aws lambda publish-version --function-name my-lmi-function \
--region us-east-1

After publishing, the function works with standard invocation methods including direct invokes, event source mappings, and service integrations with Amazon API Gateway, Amazon Simple Storage Service (Amazon S3), Amazon DynamoDB Streams, and Amazon EventBridge.

Figure 3. LMI Invocation from event sources

Scaling LMI Functions

Lambda monitors CPU utilization at Capacity Provider level. When CPU utilization reaches the target threshold, Lambda automatically provisions additional EC2 instances, and creates more execution environments on those instances, up to the MaxVCpuCount limit you configured for your capacity provider. As demand decreases, Lambda consolidates workloads onto fewer EC2 instances. You can choose automatic scaling mode (Lambda adjusts thresholds based on your patterns) or manual mode (you set a target CPU percentage). Automatic mode works for variable traffic patterns or when getting started. Manual mode fits when you have predictable patterns and want precise control over scaling thresholds for cost optimization.

Min and max execution environments

Control scaling at the function level with min and max execution environments. The default minimum is 3 execution environments to maintain high availability across Availability Zones. Your total function concurrency equals the number of execution environments multiplied by PerExecutionEnvironmentMaxConcurrency. For example, with min set to 3 and PerExecutionEnvironmentMaxConcurrency of 10, you have provided capacity for 30 concurrent invocations. With max set to 20, you can scale up to 200 concurrent invocations with incoming traffic, based on CPU utilization or concurrency saturation per execution environment. Set max to cap total concurrency and prevent noisy neighbor issues when multiple functions share a Capacity Provider. LMI maintains a minimum number of execution environments with a minimum Amazon EC2 fleet, while standard Lambda scales to zero when idle. Set both min and max to 0 to deactivate a function without deleting it.

Figure 4. LMI Scaling

This command updates the minimum and maximum execution environments for your function:

aws lambda put-function-scaling-config \
  --function-name my-lmi-function \
  --qualifier $LATEST \
  --function-scaling-config MinExecutionEnvironments=5,MaxExecutionEnvironments=20 \
  --region us-east-1

We’ll cover scaling patterns and throughput optimization strategies in depth in a separate blog post.

Best Practices and Production Considerations

Thread Safety

Since LMI supports multiple invocations sharing execution environments, your code must be thread-safe. Code that isn’t thread-safe causes data corruption, security issues, or unpredictable behavior under concurrent load.

Thread safety essentials

Avoid mutating shared objects or global variables. Use thread-local storage for request-specific data. Initialize shared clients (AWS SDK, database connections) outside the function handler and verify that configurations remain immutable during invocations. Write to /tmp using request-specific file names to prevent concurrent writes.

Runtime-specific guidance

Java applications should use immutable objects, thread-safe collections, and proper synchronization. Node.js applications should use async context for request isolation. Python applications run separate processes per execution environment. So, focus on interprocess coordination and file locking for /tmp access.

Workload Optimization

I/O-bound workloads perform better with higher concurrency per environment. Use asynchronous patterns and non-blocking I/O to maximize efficiency. CPU-bound workloads get no benefit from concurrency greater than one per vCPU. Instead, configure more vCPUs per function for true parallelism for compute-heavy tasks like data transformation or image processing.

Testing

Validate your code under concurrent execution. Test with multiple simultaneous invocations to detect race conditions and shared state issues before production deployment. You can use LocalStack for local emulation of LMI. Learn more about LocalStack’s LMI support in their announcement blog.

Compatibility

Tools like Powertools for AWS work with LMI without code changes. However, if you’re reusing existing Lambda function code, layers, or packaged dependencies on LMI, test for thread safety and compatibility with the multi-concurrent execution model before production deployment.

Observability

LMI automatically publishes CloudWatch metrics at two levels: capacity provider (CPU, memory, network, and disk utilization across your Amazon EC2 fleet) and execution environment (concurrency, CPU, and memory per function). Monitor CPUUtilization to understand scaling headroom and right-size your MaxVCpuCount. Track ExecutionEnvironmentConcurrency against ExecutionEnvironmentConcurrencyLimit to catch throttling before it impacts users. Lambda publishes metrics at 5-minute intervals. Use CloudWatch alarms to stay ahead of capacity limits in production.

Conclusion

AWS Lambda Managed Instances combines serverless simplicity with compute flexibility, helping you run high-performance workloads with reduced operational complexity. You maintain the familiar programming model of Lambda while accessing the diverse instance types of Amazon EC2 and predictable pricing, making it well-suited for data processing pipelines, compute intensive operations and cost-sensitive steady-state applications.

Ready to get started with LMI? Deploy our Monte Carlo risk simulation example from GitHub to see LMI in action with a real compute-intensive workload. The sample includes complete infrastructure code and walks you through capacity provider configuration, function setup, and performance optimization.

We want to hear from you. Share your feedback, questions, and use cases on re:Post.

Enhancing auto scaling resilience by tracking worker utilization metrics

Brian Moore — Tue, 24 Mar 2026 16:17:58 +0000

A resilient auto scaling policy requires metrics that correlate with application utilization, which may not be tied to system resources. Traditionally, auto scaling policies track system resource such as CPU utilization. These metrics are easily available, but they only work when resource consumption correlates with worker capacity. Factors such as high variance in request processing time, mixed instance types, or natural changes in application behavior over time can break this assumption.

Worker utilization tracking offers an alternative approach. Using a combination of total worker slots, work in flight, and work waiting in the backlog, a utilization value can be calculated for use in an auto scaling policy. This approach remains accurate across fleets with mixed instance types, applications with variable latencies, and requires no changes as your application evolves.

The limitations of resource-based scaling

Traditional auto scaling policies track system resource metrics like CPU utilization, assuming a direct correlation between resource consumption and available application capacity. Consider an application that reads messages from Amazon Simple Queue Service (SQS), processes them, and writes results to Amazon DynamoDB. If this application uses a fixed-size thread pool to process messages, such as 10 worker threads, the application reaches maximum capacity when all threads are busy, regardless of CPU utilization.

In our example, each worker spends most of its time waiting for DynamoDB responses rather than consuming CPU. All 10 threads become occupied handling requests, but CPU utilization stays low. From the perspective of the auto scaling policy, the fleet looks like it has enough capacity because plenty of CPU headroom remains. Meanwhile, new messages accumulate in the SQS queue because no workers are available to process them.

For queue-based workloads, AWS provides guidance to scale based on an acceptable backlog per worker. This is a calculated target based on your application’s average processing latency (queue delay). This works well when processing times are consistent, but breaks down if an application has variable latency characteristics.

Consider an image processing application that initially handles thumbnails taking 500 ms each. Using the traditional guidance with a target latency of 5 seconds you calculate an acceptable backlog of 10 messages per worker and deploy your scaling policy. Over time, the application evolves to also process 4K photos which take 2 seconds each. Eventually 4K photos are 50% of your traffic and total latency for queued messages has increased to 12.5 seconds, 2.5x more than your initial target.

The scaling policy is no longer fit for its intended purpose because your original latency assumptions no longer reflect reality. To keep this type of scaling effective you must also remember to update your scaling policies as your application behavior evolves.

A shift to using mixed instance types in your application can lead to additional complexity when using traditional resource-based scaling policies. Different instance types may handle the same workload at different CPU levels leading to an unbalanced average that misrepresents your actual application health. By changing your mental model to consider how much work your application can accept instead of how much of a system resource is available you can improve your scaling rules and better model your application’s capacity.

Understanding worker utilization

Worker utilization measures the ratio of active work to available processing capacity. To calculate it, divide total work by total workers.

We use an SQS-based processing application as an example to demonstrate how worker utilization operates, but this approach can also be applied to other applications where work units and worker capacity are measurable. In our example application total work consists of messages waiting to be processed plus messages currently being processed. Amazon CloudWatch provides these values through the ApproximateNumberOfMessagesVisible metric (messages waiting in the queue) and the ApproximateNumberOfMessagesNotVisible metric (messages currently being processed or in flight). Each host in your application should publish the number of available workers as a custom CloudWatch metric with at least a 1-minute period. For Java thread pools or Python multiprocessing pools, this represents the pool or process count. The formula works regardless of the metric period. Using the shortest period possible allows more responsive target tracking and enables Fast Target Tracking if your application has sub-minute data points.

To derive the formula, we can use the following CloudWatch Metric Math expressions:

totalWork = FILL(backlog, REPEAT) + FILL(inFlight, REPEAT)
utilizationRatio = totalWork / workers

Where:

backlog = ApproximateNumberOfMessagesVisible with the Maximum statistic.
inFlight = ApproximateNumberOfMessagesNotVisible with the Maximum statistic.
workers = Your custom TotalWorkers metric with the Sum statistic.

Putting the components together the final expression for your target tracking scaling policy uses the following formula:

IF(FILL(workers, 0) > 0, utilizationRatio, IF(totalWork > 0, 1, 0))

The FILL function uses last known values if SQS metrics are delayed, and the IF statement handles the case where you have no traffic and your fleet scales to zero instances. When there are no available workers, the formula metric reports 1 to indicate that the workers are fully saturated. This prevents the application from getting stuck at zero capacity and not being able to respond to any requests.

In this formula, a value of 1 or higher represents full or over saturation, where all workers are busy with no spare capacity, like running at 100% CPU. Values below 1 indicate available capacity for your application to process more work.

For applications without a measurable backlog metric, you can track worker utilization using only the in-flight work. This approach works for APIs or other synchronous workloads where work arrives and is immediately assigned to workers rather than queuing. In these cases, the formula becomes:

IF(FILL (workers, 0) > 0, utilizationRatio, IF(FILL(inFlight, 0) > 0, 1, 0))

In this scenario the utilization ratio is calculated as follows:

utilizationRatio = FILL(inFlight, REPEAT) / workers

The definitions of workers and inFlight remain the same for this formula. The primary difference is that the ratio directly tracks workers available and does not consider the backlog as an option.

How worker utilization prevents outages

Worker utilization-based scaling works for any application that can define available workers and total work. When the ratio of total work to available workers exceeds your threshold, the system scales out. This approach measures whether workers are available to handle the workload and treats application bottlenecks consistently. Whether workers are waiting on network I/O, performing CPU-intensive calculations, or experiencing another bottleneck doesn’t matter; the only question is whether total work exceeds available worker capacity. Any situation causing messages to accumulate on the queue increases the utilization ratio and triggers scale-out.

Implementing worker utilization scaling

To set up worker utilization-based auto scaling, identify metrics to use in the formula discussed earlier. First, identify a metric to track the amount of work being worked on. For SQS-based processing, AWS provides this metric. Second, implement a custom metric from your application representing the total workers. Optionally you can also identify a metric to track the available backlog of work.

Using CloudWatch metric math, you calculate the utilization metric and use it in a target tracking scaling policy. Here is an example AWS CloudFormation snippet showing the metric math configuration for a Amazon EC2 Auto Scaling group. This snippet shows only the scaling policy configuration and is only an example, before using in production fully test with your application. Your complete template also needs IAM roles with appropriate permissions for SQS, DynamoDB, and CloudWatch access.

ScalingPolicy: 
  Type: AWS::AutoScaling::ScalingPolicy 
  Properties: 
    AutoScalingGroupName: !Ref AutoScalingGroup 
    PolicyType: TargetTrackingScaling 
    TargetTrackingConfiguration: 
      TargetValue: 0.7 
      CustomizedMetricSpecification: 
        Metrics: 
          - Id: backlog 
            MetricStat: 
            Metric: 
              Namespace: AWS/SQS 
              MetricName: ApproximateNumberOfMessagesVisible 
              Dimensions: 
                - Name: QueueName 
                  Value: !GetAtt ProcessingQueue.QueueName 
              Stat: Maximum 
          - Id: inFlight 
            MetricStat: 
            Metric: 
              Namespace: AWS/SQS 
              MetricName: ApproximateNumberOfMessagesNotVisible 
              Dimensions: 
                - Name: QueueName 
                  Value: !GetAtt ProcessingQueue.QueueName 
              Stat: Maximum 
          - Id: workers 
            MetricStat: 
            Metric: 
              Namespace: YourApp 
              MetricName: TotalWorkers 
            Stat: Sum 
          - Id: totalWork 
            Expression: FILL(backlog, REPEAT) + FILL(inFlight, REPEAT) 
          - Id: utilizationRatio 
            Expression: totalWork / workers 
          - Id: utilization 
            Expression: IF(FILL(workers, 0) > 0, utilizationRatio, IF(totalWork > 0, 1, 0)) 
            ReturnData: true

This approach also works for Amazon ECS services using AWS Application Auto Scaling. The metric math configuration remains the same, but you create an AWS::ApplicationAutoScaling::ScalingPolicy resource instead, adapting the parameters accordingly.

Choosing a target utilization

Since the worker utilization metric directly tracks the available capacity of your application, the target utilization value you choose reflects your organization’s balance between cost efficiency and availability. Lower target values provide more headroom for traffic spikes and faster response to load changes but result in higher infrastructure costs due to lower utilization. Higher target values maximize cost efficiency by keeping workers busy but leave less headroom for sudden traffic increases.

When choosing a target consider traffic patterns, acceptable latency during scale-out events, and cost sensitivity. Applications with unpredictable traffic spikes may benefit from lower targets, while an application with predictable load can safely use higher targets. Start with a moderate value like 0.7 and adjust based on observed behavior and your business requirements. If you previously tracked a resource utilization metric such as CPU, consider starting with the same target.

Monitoring resource utilization for cost optimization

While worker utilization drives scaling decisions, CPU and latency should be regularly evaluated to ensure cost-effective operations. Resource-based metrics can identify host resizing opportunities to better match your application requirements. If no scale-in happens when CPU utilization is consistently low, you are likely running instances that are too large for your workload. By using worker utilization in an auto scaling policy, you can switch to a different instance type without adjusting the auto scaling policy. The formula automatically adapts as you add different instance types or update the capacity per worker.

Conversely, if CPU utilization is consistently high while worker utilization remains at your target, your instances might be undersized. Upgrading to larger instance types can improve per-worker throughput, allowing each worker to process tasks faster. Changes to your auto scaling policy are not needed in this situation either. As messages are processed faster, they spend less time in the in-flight state, and the utilization ratio naturally adjusts.

This approach manages application availability independent of instance size, while resource utilization guides cost optimization. Each can be optimized independently without complex coordination.

Conclusion

Worker utilization-based auto scaling reduces the operational burden of continuously validating your scaling rules as application requirements and infrastructure change. By tracking the ratio of work to workers, your auto scaling policies automatically respond to capacity constraints based on available work. The approach works across workloads with discrete processing units and remains effective when you modify instance configurations or application worker pool sizes.

Implementation requires identifying a metric for available work, publishing a custom metric representing total workers, and using CloudWatch metric math in a target tracking scaling policy. This setup provides resilience that scaling based solely on resource metrics cannot achieve, while maintaining the flexibility to optimize costs and change your instance size without impacting system availability.

To get started:

Identify an application in your environment that uses a worker pool.
Instrument the application to publish worker count metrics.
Configure a scaling policy tracking worker utilization.
Monitor how the system responds to traffic changes and capacity events.

Learn more

To learn more about auto scaling and monitoring, see the following resources:

Best practices for Lambda durable functions using a fraud detection example

Debasis Rath — Mon, 23 Mar 2026 22:04:39 +0000

AWS Lambda durable functions extend the Lambda programming model to build fault-tolerant multi-step applications and AI workflows using familiar programming languages. They preserve progress despite interruptions and execution can suspend for up to one year, for human approvals, scheduled delays, or other external events, without incurring compute charges for on-demand functions.

This post walks through a fraud detection system built with durable functions. It also highlights the best practices that you can apply to your own production workflows, from approval processes to data pipelines to AI agent orchestration. You will learn how to handle concurrent notifications, wait for customer responses, and recover from failures without losing progress. If you are new to durable functions, check out the Introduction to Durable Functions blog post first.

Fraud detection with human-in-the-loop

Consider a credit card fraud detection system, which uses an AI agent to analyze incoming transactions and assign risk scores. For ambiguous cases (medium-risk scores), the system needs human approval before authorizing a transaction. The workflow branches based on risk:

Low risk (score < 3): Authorize immediately
High risk (score ≥ 5): Send to the fraud department immediately
Medium risk (score 3–4): Suspend transaction, send SMS and email to cardholder, wait up to 24 hours for confirmation (wait time is customizable)

Figure 1. Agentic Fraud Detection with durable Lambda functions

With human-in-the-loop workflows, response times can vary from minutes to hours. These delays introduce the need to durably preserve the state without consuming compute resources while waiting. With financial systems, we must also implement idempotency to guard against duplicate messages (invocations) and recover from failures without reprocessing completed work. To address these requirements, developers implement polling patterns with external state stores like Amazon DynamoDB or Amazon Simple Storage Service (Amazon S3) to manage idempotency, pay for idle compute while waiting for callbacks, introduce external orchestration components, or build asynchronous message-driven systems to handle long-processing tasks.

Lambda durable functions provide a new alternative to address these challenges through durable execution, a pattern that uses checkpoints (saved state snapshots) to preserve progress and replays from saved state to recover from failures or resume after waiting. With checkpointing capabilities, you no longer need to pay Lambda compute charges while waiting, whether for callbacks, scheduled delays, or external events. Learn how to implement durable functions using the complete fraud detection implementation at this GitHub repository. You can deploy it to your AWS account and experiment with the code as you read. The repository includes deployment instructions, sample data, and helper functions for testing.

As we walk through the code, we’ll focus on best practices for designing workflows with durable execution and how to apply these patterns correctly in production workflows.

Design steps to be idempotent

Durable execution is designed to preserve progress through checkpoints and replay, but that reliability model means step logic can execute more than once. When steps retry, how do you prevent duplicate actions like charges to the credit card or repeated customer SMS or email notifications?

Durable functions use at-least-once execution by default, executing each step at least one time, potentially more if failures occur. When a step fails, it retries. There are two strategies to design idempotent steps that prevent duplicate side effects: using external API idempotency keys and using the at-most-once step semantics built into durable functions.

Strategy A: External API Idempotency Keys

// Strategy A: Use external API idempotency keys
await context.step(`authorize-${tx.id}`, async () => {
  return payment.charges.create({
    amount: tx.amount,
    currency: 'usd',
    idempotency_key: `tx-${tx.id}`, // Prevents duplicate charges
    description: `Transaction ${tx.id}`
  });
});

Notice the configuration:

idempotency_key in API call: If the step retries, the payment processor recognizes it’s a duplicate request and returns the original result
Defense in depth: Two layers of protection: Lambda checkpointing and external API idempotency

Each layer provides independent protection. If Lambda’s checkpoint fails, the external API prevents duplicate charges. For legacy systems without idempotency support, where it’s critical that an operation is not executed more than once, use at-most-once semantics:

Strategy B: Use At-Most-Once Semantics

For legacy systems without idempotency support, use at-most-once execution, a delivery feature that executes each step zero or one time, never more:

// Strategy B: At-most-once step semantics
await context.step("charge-legacy-system", async () => {
  return await legacyPaymentSystem.charge(tx.amount);
}, {
  semantics: StepSemantics.AtMostOncePerRetry,
  retryStrategy: createRetryStrategy({ maxAttempts: 0 })
});

This checkpoints before step execution, preventing the step from re-execution on retries. The tradeoff? If the step fails, you must decide whether to retry (risking duplicates) or fail the entire workflow.

Use idempotency for critical side effects like payment processing, database writes, external API calls, state transitions, and resource provisioning. Read more about idempotency here.

Prevent duplicate executions with DurableExecutionName

Idempotent steps prevent duplicate side effects within a single execution, but what about duplicate workflow executions running concurrently? For example, duplicate messages in the queue, users clicking “Submit” multiple times in the UI, or the same event arriving via multiple channels like webhook and API. Without protection, each invocation creates a separate durable execution, potentially running the fraud check multiple times, sending duplicate notifications, and creating confusion about which execution is authoritative. Durable functions provide DurableExecutionName to help ensure only one concurrent execution per unique name.

// Invoke fraud detection function with execution name
await lambda.invoke({
  FunctionName: 'fraud-detection',
  InvocationType: 'Event',
  DurableExecutionName: `tx-${transactionId}`,
  Payload: JSON.stringify({
    id: transactionId,
    amount: 6500,
    location: 'New York, NY',
    vendor: 'Amazon.com'
  })
});

Notice the configuration:

DurableExecutionName: tx-${transactionId}: Uses the transaction ID as a unique execution identifier
InvocationType: ‘Event’: Asynchronous invocation supports long-running workflows beyond 15 minutes
One execution per transaction: If three invocations arrive with the same transaction ID, only the first creates an execution. Subsequent requests with the same execution name and payload receive an idempotent response returning the existing execution’s ARN, rather than creating a new execution.

Lambda durable functions work with Lambda event sources, including event source mappings (ESM) such as Amazon Simple Queue Service (Amazon SQS), Amazon Kinesis, and DynamoDB Streams. ESMs invoke durable functions synchronously and inherit Lambda’s 15-minute invocation limit. Therefore, like direct Request/Response invocations, durable functions executions using event source mappings cannot exceed 15 minutes.

For workflows exceeding 15 minutes, use an intermediary Lambda function between the event source mapping and durable function:

// Intermediary function for SQS -> Durable function
export const handler = async (event) => {
  for (const record of event.Records) {
    const transaction = JSON.parse(record.body);
    await lambda.invoke({
      FunctionName: process.env.FRAUD_DETECTION_FUNCTION,
      InvocationType: 'Event',
      DurableExecutionName: `tx-${transaction.id}`,
      Payload: JSON.stringify(transaction)
    });
  }
};

This removes the 15-minute limit, allows executions up to one year, and enables custom execution name parameters for idempotency. Use Powertools for AWS Lambda to prevent duplicate invocations of the durable function when the event source mapping retries the intermediary function. Additionally, configure failure handling for your event source to capture failed invocations for future redrive or replay. For example, dead letter queues for SQS, or on-failure destinations for other event sources.

Match timeouts to invocation type

One important configuration detail ties these patterns together: matching your timeout settings to your invocation type. Lambda synchronous invocations (RequestResponse) have a hard 15-minute timeout limit. If you configure a durable execution to run for 24 hours but invoke it synchronously, the synchronous invocation fails immediately with an exception. Durable functions support workflows up to one year when invoked asynchronously.

// Lambda function configuration
{
  FunctionName: 'fraud-detection',
  Timeout: 300,
  MemorySize: 512,
  DurableConfig: {
    ExecutionTimeout: 90000
  }
}

And invoke asynchronously:

// Async invocation for long-running workflow
await lambda.invoke({
  FunctionName: 'fraud-detection',
  InvocationType: 'Event',
  DurableExecutionName: `tx-${transactionId}`,
  Payload: JSON.stringify(transaction)
});

Notice the configuration:

Timeout: 300: Lambda function timeout (5 minutes in this example, up to a maximum of 15 minutes). This defines the maximum duration for each active execution phase, including the initial invocation and any subsequent replays. Set this to cover the longest expected active processing time in your workflow.
ExecutionTimeout: { hours: 25 }: Durable execution timeout covers the workflow’s expected total duration including suspension periods. Set this slightly above the longest wait timeout to avoid edge cases.
InvocationType: ‘Event’: Asynchronous invocation removes the 15-minute limit and enables executions up to one year.

The Lambda function timeout applies to active execution phases (AI calls, notification sending). During suspension (waiting for callbacks), the function isn’t running, so this timeout doesn’t apply. Setting the durable execution timeout to a meaningful boundary prevents workflows from running longer than expected. Without an explicit timeout, executions can run up to the maximum lifetime of one year.

	Synchronous (RequestResponse)	Asynchronous (Event)
Total duration	Under 15 minutes	Up to 1 year
Caller needs result	Yes	No
Idempotency support	Yes	Yes
Waits with suspension	Yes	Yes

Execute Concurrent Operations with context.parallel()

In the fraud detection workflow, the system notifies the cardholder through multiple channels such as SMS and email. Preserving business logic when executing parallel workflows introduces code complexities such as managing execution state across branches, handling synchronization, and coordinating branch completion. Durable functions simplify parallel workflow implementation using context.parallel(), which executes branches concurrently while maintaining durable checkpoints for each branch and provides configurable options to handle partial completions. By checkpointing and managing the state internally, durable functions help make sure that the state is preserved even if there are retries or failures. Note that context.parallel() manages the internal execution state for each branch. If your branches interact with a shared external state (such as a database), you’re responsible for managing concurrent access to that external state.

// Human-in-the-loop: verify via email AND SMS (first response wins)
let verified = await context.parallel("human-verification", [
  (ctx) => ctx.waitForCallback("SendVerificationEmail",
    async (callbackId) => sendCustomerNotification(callbackId, 'email', tx)
  ),
  (ctx) => ctx.waitForCallback("SendVerificationSMS",
    async (callbackId) => sendCustomerNotification(callbackId, 'sms', tx)
  )
], {
  maxConcurrency: 2,
  completionConfig: {
    minSuccessful: 1 // Continue after 1 success
  }
});

Notice the configuration:

maxConcurrency: 2: Both notifications sent at the same time
minSuccessful: 1: We only need one channel to succeed, whichever responds first wins

Each parallel branch waits for its callback independently, and the durable execution checkpoints each branch as part of the execution state. Using the minSuccessful parameter, you control the minimum number of successful branch executions required for the parallel operation to complete. In this example, only one of the two branches needs to succeed. Verifications through SMS or email are both valid, and the workflow resumes as soon as either channel completes successfully. We call this the first-response-wins pattern. This pattern works well when you only need a single successful result from any parallel branch and want the remaining branches to stop blocking progress.

But what happens if neither channel responds? Without timeouts, this workflow could remain suspended for up to the configured execution lifetime.

Always configure callback timeouts

Let’s add timeout protection to the parallel verification from the previous section. context.waitForCallback() accepts a timeout option that bounds how long each branch waits before throwing an exception. By wrapping the parallel call in a try/catch, you can implement fallback logic when users don’t respond in time.

// Enhanced: parallel verification with timeout and error handling
let verified;
try {
  verified = await context.parallel("human-verification", [
    (ctx) => ctx.waitForCallback("SendVerificationEmail",
      async (callbackId) => sendCustomerNotification(callbackId, 'email', tx),
      { timeout: { days: 1 } }  // Wait up to 1 day for email response
    ),
    (ctx) => ctx.waitForCallback("SendVerificationSMS",
      async (callbackId) => sendCustomerNotification(callbackId, 'sms', tx),
      { timeout: { days: 1 } }  // Wait up to 1 day for SMS response
    )
  ], {
    maxConcurrency: 2,
    completionConfig: {
      minSuccessful: 1
    }
  });
} catch (error) {
  const isTimeout = error.message?.includes("timeout");
  if (isTimeout) {
    context.logger.warn("Customer verification timeout", { error, txId: tx.id });
    // Fallback: escalate to fraud department
    return await context.step("sendToFraudDepartment", async () =>
      sendToFraudDepartment(tx, true)
    );
  }
  throw error; // Re-throw non-timeout errors
}

Notice what changed from the previous section:

timeout: { days: 1 }: Each callback branch now has a maximum wait time of 1 day. If neither the email nor SMS callback arrives within that window, a timeout exception is thrown.
try/catch with timeout detection: The catch block distinguishes between timeout errors and other exceptions. When a timeout occurs, the workflow implements fallback logic by escalating the transaction to the fraud department, while non-timeout errors are re-thrown to be handled by the durable execution retry mechanism.

Without this error handling, the entire execution fails unhandled. The timeout also works with the minSuccessful configuration: if one branch times out but the other succeeds, the parallel operation still completes successfully since only one successful result is required.

For advanced use cases where the callback handler performs long-running work, you can also configure a heartbeatTimeout to detect stalled callbacks before the main timeout expires. See the Lambda Developer Guide for details.

Use callback timeouts for human approvals, external API callbacks, asynchronous processing, and third-party integrations.

Putting it all together: complete fraud detection implementation

Now let’s see how all the best practices work together in the complete fraud detection workflow:

import { withDurableExecution } from "@aws/durable-execution-sdk-js";
import { BedrockAgentCoreClient, InvokeAgentRuntimeCommand } from "@aws-sdk/client-bedrock-agentcore";

const agentRuntimeArn = process.env.AGENT_RUNTIME_ARN;
const agentRegion = process.env.AGENT_REGION || 'us-east-1';
const client = new BedrockAgentCoreClient({ region: agentRegion });

export const handler = withDurableExecution(async (event, context) => {
  const tx = {
    id: event.id,
    amount: event.amount,
    location: event.location,
    vendor: event.vendor
  };

  // AI fraud assessment with error handling
  tx.score = await context.step("fraudCheck", async () => {
    try {
      const payloadJson = JSON.stringify({ input: { amount: tx.amount } });
      const command = new InvokeAgentRuntimeCommand({
        agentRuntimeArn: agentRuntimeArn,
        qualifier: 'DEFAULT',
        payload: Buffer.from(payloadJson, 'utf-8'),
        contentType: 'application/json',
        accept: 'application/json'
      });
      const response = await client.send(command);
      const responseText = await response.response.transformToString();
      const result = JSON.parse(responseText);
      return result?.output?.risk_score ?? 5;  // Default to high-risk if score unavailable
    } catch (error) {
      context.logger.error("Fraud check failed", { error, txId: tx.id });
      return 5;
    }
  });

  // Route based on AI decision
  if (tx.score < 3) {
    // Best Practice: Idempotent authorization
    return await context.step(`authorize-${tx.id}`, async () =>
    authorizeTransaction(tx, { idempotency_key: `tx-${tx.id}` })
    );
  }

  if (tx.score >= 5) {
    return await context.step(`sendToFraudDepartment-${tx.id}`, async () =>
      sendToFraudDepartment(tx)
    );
  }

  // Medium risk: need human verification
  await context.step(`suspend-${tx.id}`, async () => suspendTransaction(tx));

  // Best Practice: Concurrent operations with timeout configuration
  let verified;
  try {
    verified = await context.parallel("human-verification", [
      (ctx) => ctx.waitForCallback("SendVerificationEmail",
        async (callbackId) => sendCustomerNotification(callbackId, 'email', tx),
        { timeout: { days: 1 } }
      ),
      (ctx) => ctx.waitForCallback("SendVerificationSMS",
        async (callbackId) => sendCustomerNotification(callbackId, 'sms', tx),
        { timeout: { days: 1 } }
      )
    ], {
      maxConcurrency: 2,
      completionConfig: {
        minSuccessful: 1
      }
    });
  } catch (error) {
    const isTimeout = error.message?.includes("timeout");
    context.logger.warn(
      isTimeout ? "Customer verification timeout" : "Customer verification failed",
      { error, txId: tx.id }
    );
    return await context.step(`timeout-escalate-${tx.id}`, async () =>
      sendToFraudDepartment(tx, true)
    );
  }

  // Idempotent final step with idempotency key
  return await context.step(`finalize-${tx.id}`, async () => {
    const action = !verified.hasFailure && verified.successCount > 0
      ? "authorize"
      : "escalate";
    if (action === "authorize") {
      return authorizeTransaction(tx, true, { idempotency_key: `finalize-${tx.id}` });
    }
    return sendToFraudDepartment(tx, true);
  });
});

Notice how the best practices work together: context.parallel() sends SMS and email concurrently, resuming when either channel responds. Both callbacks configure 1-day timeouts with try/catch handling that escalates on timeout. The DurableExecutionName: tx-${transactionId} parameter (specified at invocation time, shown in the following CLI example) provides execution-level deduplication, while idempotency keys in the authorization steps prevent duplicate charges at the application layer. Asynchronous invocation (InvocationType: 'Event') enables the 24-hour wait period.

Once deployed, invoke the function asynchronously with a sample transaction to see it in action:

transactionId="123456789"
aws lambda invoke \
  --function-name "fraud-detection:$LATEST" \
  --invocation-type Event \
  --durable-execution-name "tx-${transactionId}" \
  --cli-binary-format raw-in-base64-out \
  --payload "{\"id\": \"${transactionId} \", \"amount\": 6500, \"location\": \"New York, NY\", \"vendor\": \"Amazon.com\"}" \
  --region us-east-2 \
  response.json

Upon successful invocation, you can view the execution state in the Lambda console’s durable operations view. The execution shows a suspended state, waiting for customer response:

Figure 2: Suspended execution state

Notice the fraudCheck and suspendTransaction steps show as succeeded with checkpointed results. The human-verification parallel operation shows that both SMS and email branches started. The timeline shows the function in a suspended state. Simulate a customer response by sending a callback success through the console, AWS Command Line Interface (AWS CLI) or Lambda API:

aws lambda send-durable-execution-callback-success \
	--callback-id <CALLBACK_ID_FROM_EMAIL_OR_SMS> \
	--result '{"status":"approved","channel":"email"}' \
	--cli-binary-format raw-in-base64-out

Figure 3: Completed execution with customer approval

After receiving the customer’s approval, the durable execution resumes from its checkpoint, authorizes the transaction, and completes. The execution spanned hours but consumed only seconds of compute time.

Conclusion

With durable functions, Lambda extends beyond single-event processing to power core business processes and long-running workflows, while retaining the operational simplicity, reliability, and scale that define Lambda. You can build applications that run for days or months, survive failures, and resume where they left off, all within the familiar event-driven programming model.

Deploy the fraud detection workflow from our GitHub repository and experiment with human-in-the-loop patterns in your own account. For core concepts, see Introduction to AWS Lambda Durable Functions. For comprehensive documentation, see the Lambda Developer Guide. Browse Serverless Land for reference architectures and discover where durable execution fits in your designs.

Share your feedback, questions, and use cases in the SDK repositories or on re:Post.

Testing Step Functions workflows: a guide to the enhanced TestState API

D Surya Sai — Sun, 22 Mar 2026 17:06:38 +0000

AWS Step Functions recently announced new enhancements to local testing capabilities for Step Functions, introducing API-based testing that developers can use to validate workflows before deploying to AWS. As detailed in our Announcement blog post, the TestState API transforms Step Functions development by enabling individual state testing in isolation or as complete workflows. This supports mocked responses and actual AWS service integrations, and provides advanced capabilities. These capabilities include Map/Parallel states, error simulation with retry mechanisms, context object validation, and detailed inspection metadata for comprehensive local testing of your serverless application.

The TestState API can be accessed through multiple interfaces such as AWS Command Line Interface (AWS CLI), AWS SDK, LocalStack. By default, TestState API in AWS CLI and SDK runs against the remote AWS endpoint, providing validation against the actual Step Functions service infrastructure. We’ve partnered with LocalStack to offer an additional testing endpoint for the TestState API. Developers can use LocalStack for unit testing their workflows by changing the AWS SDK client endpoint configuration to point to LocalStack: http://localhost.localstack.cloud:4566/ instead of AWS endpoint. This approach provides complete network isolation when needed. For a streamlined development experience, you can also use the LocalStack VSCode extension to automatically configure your environment to point to the LocalStack endpoint. This approach is detailed in the AWS blog post.

This blog post demonstrates building test suites to unit test your Step Functions workflows using the AWS SDK for Python using the pytest framework. The complete implementation is available in the GitHub repository.

Building test cases using the TestState API

This example workflow implements a real-world ecommerce order processing system using JSONata for advanced data transformations. It incorporates complex Step Functions patterns including distributed Map states, Parallel execution, and waitForTaskToken callback mechanisms. The process validates orders through AWS Lambda functions, distributes order item processing with configurable failure tolerance, runs parallel payment and inventory updates, handles human approval workflows using task tokens, then persists orders in Amazon DynamoDB with notification delivery. This workflow demonstrates advanced error handling with multiple Catchers and Retriers, exponential backoff for Lambda throttling and DynamoDB limits, and sophisticated state transitions that were previously challenging to test locally. This makes it the recommended choice for demonstrating the use of enhanced TestState API’s local testing features.

The complete workflow is available in the GitHub repository, where you can examine the full state machine definition and see how JSONata expressions handle data transformation throughout the execution flow.

Figure 1: State machine workflow that demonstrates a real-world ecommerce order processing system.

Effective Step Functions testing requires a systematic approach to TestState API integration that provides state validation, error simulation, and assertion capabilities. The testing framework is built using Python’s pytest framework, using fixtures to automatically provide pre-configured runner instances that handle TestState API client initialization and state machine definition loading. This eliminates repetitive setup code and provides consistent test environments. The enhanced TestState API supports both mock integrations and actual integrations with AWS services, providing flexibility in testing strategies. For this demonstration, you use mock integrations to showcase how a complete local testing can be achieved without having any resources deployed to AWS accounts.

This framework is built for demonstration purposes, and you can similarly build your own testing frameworks using other programming languages like Java, Node.js. The testing framework uses method chaining patterns to create readable test cases with comprehensive assertion methods, automatic output chaining between state executions, and error simulation for testing retry mechanisms, backoff intervals, and catch blocks across AWS service error conditions.

The following test implementations demonstrate the testing capabilities that are achievable with the enhanced TestState API in local development environments. The test cases are run against the preceding Statemachine.

Test Case 1: Lambda throttling and retry mechanism testing

Service integrations with Statemachines like AWS Lambda, Amazon DynamoDB may face throttling depending on their usage. A key capability of the enhanced TestState API is its ability to simulate retry mechanisms with control over retry counts and backoff intervals. This test demonstrates the enhanced TestState API’s retry testing capabilities through the stateConfiguration.retrierRetryCount parameter and inspectionData.errorDetails response fields. This response field provides retryBackoffIntervalSeconds for validating exponential backoff calculations, retryIndex for tracking retry attempt sequences, and catchIndex for identifying which error handler processed the exception. These enhanced inspection capabilities enable validation of retry logic, backoff strategies, and error propagation patterns across complex state machine workflows.

def test_lambda_throttling_retry_mechanism(self, runner):
"""Test retry mechanism for Lambda.TooManyRequestsException"""
throttling_error = {
"Error": "Lambda.TooManyRequestsException",
"Cause": "Request rate exceeded"
}

# Test first retry attempt
(runner
.with_input({"orderId": "order-retry-test"})
.with_mock_error(throttling_error)
.with_retrier_retry_count(0)
.execute("ValidateOrder")
.assert_retriable()
.assert_error("Lambda.TooManyRequestsException"))

# Verify exponential backoff calculation
response = runner.get_response()
error_details = response['inspectionData']['errorDetails']
assert error_details['retryBackoffIntervalSeconds'] == 2

# Test retry exhaustion
(runner
.with_retrier_retry_count(3)
.execute("ValidateOrder")
.assert_caught_error()
.assert_next_state("ValidationFailed"))

Test Case 2: Map state testing with tolerance thresholds

Distributed Map states present unique testing challenges due to their parallel processing nature and failure tolerance capabilities. The enhanced TestState API provides specialized configuration options for testing these complex scenarios.

def test_map_state_tolerated_failure_threshold(self, runner):
"""Test Map state with tolerated failure threshold"""
test_input = {
"orderId": "order-map-test",
"orderItems": [
{"itemId": "item-1"}, {"itemId": "item-2"}, 
{"itemId": "item-3"}, {"itemId": "item-4"}
]
}

# Test normal Map state execution
map_success_result = [
{"itemId": "item-1", "processed": True},
{"itemId": "item-2", "processed": True}
]

(runner
.with_input(test_input)
.with_mock_result(map_success_result)
.execute("ProcessOrderItems")
.assert_succeeded()
.assert_next_state("ParallelProcessing"))

# Test tolerance threshold exceeded scenario
tolerance_error = {
"Error": "States.ExceedToleratedFailureThreshold",
"Cause": "Map state exceeded tolerated failure threshold"
}

(runner
.with_input(test_input)
.with_mock_error(tolerance_error)
.execute("ProcessOrderItems")
.assert_caught_error()
.assert_next_state("ValidationFailed"))

This test demonstrates the enhanced TestState API’s Map state testing capabilities through the stateConfiguration.mapIterationFailureCount parameter for simulating iteration failures. The API provides comprehensive inspection data including inspectionData.afterItemSelector for validating ItemSelector transformations, inspectionData.afterItemBatcher for batch processing validation, inspectionData.toleratedFailureCount and inspectionData.toleratedFailurePercentage for threshold verification. When the specified failure count exceeds the configured tolerance, the API correctly returns States.ExceedToleratedFailureThreshold, enabling testing of Map state resilience patterns.

Test Case 3: WaitForCallback pattern testing

The waitForCallback integration requires context object construction to simulate realistic execution environments, particularly for human approval workflows.

def test_context_object_usage_in_jsonata_expressions(self, runner):
"""Test Context object usage in waitForTaskToken scenarios"""
test_input = {
"orderId": "order-context-test",
"amount": 125.0
}

context_data = {
"Task": {"Token": "ahbdgftgehbdcndsjnwjkhas327yr4hendc73yehdb723y"},
"Execution": {
"Id": "arn:aws:states:us-east-1:123456789012:execution:test:exec-123"
},
"State": {
"Name": "WaitForApproval",
"EnteredTime": "2025-01-15T10:45:00Z"
}
}

mock_result = {
"approved": True,
"taskToken": "ahbdgftgehbdcndsjnwjkhas327yr4hendc73yehdb723y"
}

(runner
.with_input(test_input)
.with_context(context_data)
.with_mock_result(mock_result)
.execute("WaitForApproval")
.assert_succeeded()
.assert_next_state("CheckApproval"))

# Verify JSONata expressions processed context correctly
response = runner.get_response()
after_args = json.loads(response['inspectionData']['afterArguments'])
assert after_args['Payload']['taskToken'] == context_data['Task']['Token']

This test demonstrates the enhanced TestState API’s support for waitForCallback integrations through the `context` parameter for realistic Context object simulation. The API enables comprehensive testing of JSONata expressions that reference $states.context.Task.Token, $states.context.Execution.Id, and other context fields. The inspectionData.afterArguments response field validates that JSONata expressions correctly processed the context data, while the API automatically handles the complexity of task token embedding in service integration payloads for waitForCallback testing scenarios.

Test Case 4: Happy path testing – complete workflow validation

Happy path testing validates that workflows execute correctly under normal operating conditions. The enhanced TestState API allows you to chain state executions together, automatically passing outputs between states to simulate a complete workflow execution.

def test_complete_order_processing_workflow(self, runner):
"""Integration test: Complete happy path workflow using method chaining"""
test_input = {
"orderId": "order-12345",
"amount": 150.75,
"customerEmail": "customer@example.com",
"orderItems": [
{"itemId": "item-1", "quantity": 2, "price": 50.25}
]
}

# Test ValidateOrder state
(runner
.with_input(test_input)
.with_mock_result({"statusCode": 200, "isValid": True})
.execute("ValidateOrder")
.assert_succeeded()
.assert_next_state("CheckValidation"))

# Test CheckValidation choice state (no mock needed)
validation_output = runner.get_output()
(runner
.with_input(validation_output)
.clear_mocks()
.execute("CheckValidation")
.assert_succeeded()
.assert_next_state("ProcessOrderItems"))

This test demonstrates how the TestState API maintains state context between executions, enabling realistic workflow simulation. The get_output() method retrieves the processed output from one state to use as input for the next, mimicking actual Step Functions execution behavior.

Note: The code snippet above shows only the first two states of the complete workflow test for brevity. The full test code with all states (ProcessOrderItems, ParallelProcessing, WaitForApproval, CheckApproval, SaveOrderDetails, and SendNotification) can be viewed in the complete GitHub repository, demonstrating end-to-end workflow validation using the same method chaining pattern.

Integration with modern CI/CD pipelines

In this section, we will explore how to integrate the previous unit tests in a CI CD pipeline to enable local testing.

The sample repository includes a GitHub Actions workflow that demonstrates how TestState API testing integrates into continuous integration and continuous delivery (CI/CD) pipelines. The workflow (.github/workflows/test-and-deploy.yml) provides a two-step process that validates before any AWS resources are deployed using AWS Serverless Application Model (AWS SAM).

The CI/CD pipeline follows the following pattern:

Unit Tests: Executes the complete TestState API test suite using pytest tests/unit_test.py -v
SAM Deploy: Deploys AWS resources using sam build and sam deploy

To enable the GitHub Actions workflow to deploy resources to your AWS account, configure these AWS credentials in your GitHub repository settings. For detailed setup instructions, see the AWS blog post.

Following are the required secrets to be configured in GitHub repository settings:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_REGION

In production environments, you can typically extend this basic pipeline to include additional stages. The enhanced pipeline often begins with deploying to a development account first, followed by integration testing against deployed resources. The final stage involves moving to production with proper approval gates and security scanning compliance checks.

Conclusion

The enhanced TestState API enables testing Step Functions workflows locally without requiring AWS deployments that accelerated development cycles, and reduce testing times. This post demonstrates how to implement testing for state types including Map states with tolerance thresholds, retry mechanisms with exponential backoff, and waitForTaskToken patterns with context object simulation using mock integrations for isolated testing.

By integrating TestState API testing into CI/CD pipelines, you can validate workflow logic before deployment, reducing the risk of production issues. The GitHub Actions workflow example demonstrates an implementation that runs tests and deploys resources in a controlled sequence. The complete code examples and testing framework are available in the GitHub repository to implement similar testing practices for Step Functions workflows.

Enabling high availability of Amazon EC2 instances on AWS Outposts servers (Part 3)

Brianna Rosentrater — Fri, 06 Mar 2026 23:11:22 +0000

This post is part 3 of the three-part series ‘Enabling high availability of Amazon EC2 instances on AWS Outposts servers’. We provide you with code samples and considerations for implementing custom logic to automate Amazon Elastic Compute Cloud (EC2) relaunch on Outposts servers. This post focuses on guidance for using Outposts servers with third party storage for boot and data volumes, whereas part 1 and part 2 focus on automating EC2 relaunch between standalone servers. Outposts servers support integration with Dell PowerStore, HPE Alletra Storage MP B10000 systems, NetApp on-premises enterprise storage arrays, and Pure Storage FlashArray.

Outposts servers provide compute and networking services that are designed for low-latency, local data processing needs for on-premises locations such as retail stores, branch offices, healthcare provider locations, or environments that are space-constrained. Outposts servers use EC2 instance store storage to provide non-durable block-level storage to the instances running stateless workloads. For applications that require persistent storage, you can create a three-tier architecture by connecting your Outposts servers to a third-party storage appliance. In this post, you will learn how to implement custom logic to provide high availability (HA) for your applications running on Outposts servers using two or more servers for N+1 fault tolerance. The code provided is meant to help you get started, and can be modified further for your unique workload needs.

Overview

In the following sections we will show how custom logic can be used to automate EC2 instance relaunch between two or more Outposts servers using boot and data volumes on third party storage. If your EC2 instance fails while using this solution, an Amazon CloudWatch alarm monitoring the EC2 StatusCheckFailed_Instance metric of your source EC2 instance will be triggered, and you will receive an Amazon Simple Notification Service (Amazon SNS) notification. An AWS Lambda function will then relaunch your EC2 instance onto the destination Outposts server that you’ve set up for resiliency. This is done using a launch template created during setup, and the script will connect your relaunched instance to the existing boot and data volumes on your third party storage appliance. This storage device provides shared storage for your Outposts servers. If a single server fails, new instances can connect to existing volumes on the array. This allows for a zero data loss Recovery Point Objective (RPO) and a Recovery Time Objective (RTO) equaling the time it takes to launch your EC2 instance. Take advantage of the features on your storage appliance for configuring data durability and resiliency to hardware failures, and make sure that you are regularly backing up your SAN volumes.

Figure 1 – Solution Architecture for automated EC2 Relaunch

Prerequisites

The following prerequisites are required to complete the walkthrough:

Two Outposts servers that can be set up as an active-active or active-passive resilient pair.
For workloads with a low threshold for downtime, ensure that your secondary Outpost server that’s used for recovery has a unique service link connection.
Outposts servers must be colocated within the same Layer 2 (L2) network.
Network latency between the Outposts servers must not exceed 5ms round trip time (RTT).
A storage appliance that supports the iSCSI protocol. Credentials to manage the storage appliance initiator/target mappings. See Simplifying the use of third-party block storage with AWS Outposts for more information.
If you’re setting this up from an Outposts consumer account, you must configure Amazon CloudWatch cross-account observability between the consumer account and the Outposts owning account to view Outposts metrics in your consumer account.
Create launch templates for the EC2 instances that you want to protect, the launch wizard will help you create these.
Credentials with permissions for AWS CloudFormation, Amazon EC2, and (optional) AWS Secrets Manager if authentication is required. IAM Permission Examples.md is provided in the repository.
A Windows or Linux host that can access the storage appliance and your AWS account (management computer).
AWS Outposts iPXE Amazon Machine Image (AMI) from the AWS Marketplace.
Python 3.8 or later (recommended) is used to run the init.py script that dynamically creates a CloudFormation stack in the account specified as an input parameter.
AWS SDK for Python (Boto3) version 1.26.0 or later recommended.
Operating system with iSCSI boot support (Windows Server 2022 and Red Hat Enterprise Linux 9 AMIs are provided).
Internet access to AWS service endpoints for the private subnet hosting the recovery Lambda function.
Download the repository ec2-outposts-autorestart_3Pstorage.

Walkthrough

The first step is to deploy an EC2 instance configured to boot from a volume on the third-party storage that is prepared with an OS boot image. This step uses the launch wizard portion of the solution.

Download and extract the OutpostServer_Recovery_3Pstorage repository to the management computer that has the AWS SDK for Python (Boto3) and Python installed.
Run launch_wizard from the sample-outposts-third-party-storage-integration directory. You can run interactively or provide arguments for region, subnet, iPXE AMI, storage vendor, storage management ip, and credentials.

Figure 2 – Running launch wizard

When prompted for a feature name, enter sanboot.
For Guest OS type, enter in Linux or Windows.
When prompted “Do you want to continue with this unverified AMI?”, select Y.
The launch wizard will provide a list of instance types available on the Outpost server associated with the subnet you specified. Enter the instance type that you want to use.
The launch wizard will now prompt you for optional EC2 Key Pair, Security Group, and Instance Profile settings for the EC2 instance that you are launching.
Next, the launch wizard prompts you to specify an instance name. Note that specifying an instance name is required to set up automated instance recovery because the instance name is used as part of the recovery process.

Figure 3 – Taking user input for variable values

The launch wizard prompts for root volume size. This is the root volume that the iPXE AMI boots from. The default is a 1GB volume on the Outpost server instance storage.
Next, the launch wizard prompts you to select which third party storage controller you want to use based on the management ip that you specified. In this example, we are using NetApp, so I select a NetApp Storage Virtual Machine (SVM) named outpost_iscsi.
If the connection to the storage array is successful and the protocol is available (iSCSI or NVMe over TCP) you are provided additional storage options for initiator group and logical unit number (LUN).
In this example, we are using NetApp with iSCSI, so I can select an existing initiator group or create a new one.
You can specify an existing initiator qualified name (IQN), or the launch wizard can generate a new one. IMPORTANT: Make sure that IQNs are unique to each instance because duplicates can cause data corruption.
Next the launch wizard prompts which LUN’s you want to connect to this instance. For this example, I am going to use a Windows Server 2022 boot volume that I already created on the NetApp storage array.
You are now asked which storage array target interface you want to use for connecting to these LUNs.
The launch wizard provides the capability to specify guest OS scripts to customize the OS after sanboot. Combining this capability with storage array cloning provides a streamlined process for deploying new instances.
The launch wizard now displays the EC2 user data template that it generated for use with the iPXE AMI and asks if you want to proceed with launching the instance.
After the EC2 instance is launched, select yes to proceed with automated instance recovery setup.

Figure 4 – Running launch template creation script

Generating EC2 launch templates for recovery and failback

In the second step, we are generating EC2 launch templates for the EC2 instance launched in step 1. Launch templates can be generated for the primary and secondary Outpost servers. The launch template for the secondary Outpost server can be used for automated or manual recovery of the EC2 instance. Failback to the primary Outpost server is manual using the primary launch template.

Select the instance that you want automated recovery for and select the subnet that you launched the instance in. This subnet represents the primary Outpost server that the instance is running on.

Figure 5 – Selecting subnets for EC2 instance relaunch

When prompted to create a second launch template for Outpost server recovery, select yes, and then select to use the same instance (for recovery on different Outpost server).
When you get a list of available subnets, select the subnet that’s associated with your secondary Outpost server. This is the server that the EC2 instance will be launched on in the event of the EC2 StatusCheckFailed_Instance metric triggers the CloudWatch alarm.
You will see both launch templates created successfully.

Deploying automated EC2 instance recovery

The third step creates a CloudFormation template for monitoring, notifications, and automated recovery of the EC2 instance deployed in step 1. The CloudFormation template automatically captures the instance and secondary launch template information necessary for automatic recovery.

Select Y to set up automated recovery. This will create a CloudFormation stack.
Provide a name and description for the CloudFormation stack.
Select whether you want automated recovery or notification only. This provides flexibility to choose manual or automatic recovery based on whether you want to verify the primary Outpost server is down before initiating recovery.
In the AWS CloudFormation console, monitor the CloudFormation stack creation process.

Figure 6 – CloudFormation stack creation in progress

After the CloudFormation Stack is complete, you have successfully deployed an EC2 instance using third party storage for boot and data volumes on a primary Outpost server. You also created instance recovery capabilities by using the Amazon Outpost server automated recovery solution for third party storage.
You can verify whether the EC2 StatusCheckFailed_Instance is healthy under the Alarms section in the Amazon CloudWatch console.

Considerations

The logic discussed in this post relies on the secondary destination Outposts server having a connected service link. For more information about how to create a highly available service link connection for your Outpost servers, see the Networking section of AWS Outposts High Availability Design and Architecture Considerations whitepaper.

Clean up

Confirm whether it is safe to terminate the Amazon EC2 instance that you launched with this walkthrough. The operating system and data volumes are on the third party storage, so EC2 instance termination only removes the iPXE AMI from the Outposts server instance storage. To clean up, complete the following steps.

Terminate the Amazon EC2 instance. Then, verify that the Instance state is Terminated to ensure that the instance is not using Outposts server resources.
Delete the Amazon EC2 Launch Templates associated with the Amazon EC2 instance that you terminated. The names of the launch templates that were automatically generated will start with ‘lt-‘, followed by the instance name and the instance id. If you generated a recovery launch template, it will have a ‘-recovery’ suffix in the name.
Delete the AWS CloudFormation Stack. The Stack name will start with ‘autorestart-‘ followed by the Amazon EC2 instance name.
Clean up your initiators, initiator group, and LUNs on the third party storage array.

Conclusion

With the use of custom logic through AWS tools such as CloudFormation, CloudWatch, Amazon SNS, and AWS Lambda, you can architect for HA for stateful workloads on Outposts server. By implementing the custom logic in this post, you can automatically relaunch EC2 instances running on a source Outposts server to a secondary destination Outposts server if an instance fails, and connect to existing volumes on a shared storage appliance for recovery. This also reduces the downtime of your applications in the event of a hardware or service link failure. The code provided in this post can be further expanded upon to meet the unique needs of your workload.

While the use of infrastructure-as-code (IaC) can improve your application’s availability and be used to standardize deployments across multiple Outposts servers, it’s crucial to do regular failure drills to test the custom logic in place. This is to make sure that you understand your application’s expected behavior on relaunch in the event of a failure. To learn more about Outposts servers, visit the Outposts servers User Guide. Reach out to your AWS account team, or fill out this form to learn more about Outposts servers.

Optimizing Compute-Intensive Serverless Workloads with Multi-threaded Rust on AWS Lambda

Daniel Abib — Wed, 25 Feb 2026 12:49:44 +0000

Customers use AWS Lambda to build Serverless applications for a wide variety of use cases, from simple API backends to complex data processing pipelines. Lambda’s flexibility makes it an excellent choice for many workloads, and with support for up to 10,240 MB of memory, you can now tackle compute-intensive tasks that were previously challenging in a Serverless environment. When you configure a Lambda function’s memory size, you allocate RAM and Lambda automatically provides proportional CPU power. When you configure 10,240 MB, your Lambda function has access to up to 6 vCPUs.

However, there’s an important consideration that many developers discover: simply allocating more memory may not automatically make your function faster. If your code runs sequentially, it will only use one vCPU regardless of how many are available. The remaining vCPUs sit idle while you’re still paying for the full memory allocation.

To help benefit from Lambda’s multi-core capabilities, your code should explicitly implement concurrent processing through multi-threading or parallel execution. Without this, you’re paying for compute power you’re not using.

Rust provides excellent support for this pattern. The AWS Lambda Rust Runtime provides developers with a language that combines exceptional performance with built-in concurrency primitives. In this post, we show you how to implement multi-threading in Rust to achieve 4-6x performance improvements for CPU-intensive workloads.

Our Test Workload: Why Bcrypt Password Hashing?

For this analysis, we use bcrypt password hashing as our CPU-intensive workload to evaluate multi-core scaling behavior. This choice is deliberate for several reasons:

Real-world relevance: Bcrypt is commonly used in authentication systems, making our benchmarks practically relevant rather than synthetic.
Predictable CPU work: Bcrypt with cost factor 10 provides approximately 100ms of pure CPU work per operation on typical hardware, creating a consistent and measurable baseline.
Embarrassingly parallel: Each hash operation is completely independent, making it an ideal candidate for parallel processing without shared state or lock contention.
CPU-bound: Bcrypt is deterministic and CPU-bound (not memory or I/O bound), isolating the performance characteristics we want to measure.

Throughout this post, we process batches of passwords and measure how multi-threading improves throughput as we scale from 1 to 6 vCPUs.

Understanding Lambda’s vCPU Allocation

AWS Lambda allocates CPU resources proportionally to the configured memory. According to AWS Lambda function memory documentation, at 1,769 MB a function has the equivalent of one vCPU.

vCPU Allocation by Memory:

Memory (MB)	Approximate vCPUs
128 – 1,769	~1
1,770 – 3,538	~2
3,539 – 5,307	~3
5,308 – 7,076	~4
7,077 – 8,845	~5
8,846 – 10,240	~6

Note: The num_cpus crate returns the number of logical CPUs visible to the Lambda environment, which may differ from the allocated vCPU share. At lower memory configurations, you may see 2 CPUs reported even though only 1 vCPU worth of compute time is allocated.

Solution Overview

The solution consists of a Rust Lambda function that:

Receives a request specifying the number of items to process
Detects available vCPUs and configures a thread pool accordingly
Processes items in parallel using the Rayon library (a data parallelism library that allows you to convert sequential iterators into parallel ones with a .par_iter() call)
Returns performance metrics including duration and throughput

Architecture Diagram: Lambda receives request, initializes Rayon thread pool based on WORKER_COUNT environment variable, processes bcrypt hashes in parallel across multiple vCPUs, and returns results.

Creating a Multi-threaded Rust Lambda Function

Create a new Lambda project using Cargo Lambda:

cargo lambda new rust-multithread-demo
cd rust-multithread-demo

Dependencies

Update Cargo.toml with the necessary dependencies:

[package]
name = "rust-multithread-lambda"
version = "0.1.0"
edition = "2021"

[dependencies]
lambda_runtime = "1.0.0"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
bcrypt = "0.15"
rayon = "1.7"
num_cpus = "1.16"

[profile.release]
opt-level = 3
lto = true
codegen-units = 1
strip = true

The optimization flags in [profile.release] reduce binary size and improve performance:

opt-level = 3: Maximum optimization
lto = true: Link-time optimization for smaller binaries
strip = true: Remove debug symbols

Implementing the Lambda Entry Point

First, let’s look at how we initialize the thread pool during cold start:

src/main.rs:

use lambda_runtime::{run, service_fn, Error, LambdaEvent};
mod handler;
use handler::{function_handler, get_worker_count, init_thread_pool, ProcessRequest};

#[tokio::main]
async fn main() -> Result<(), Error> {
    // Initialize Rayon thread pool at cold start (once per container lifecycle)
    init_thread_pool(get_worker_count());

    run(service_fn(|event: LambdaEvent<ProcessRequest>| async move {
        function_handler(event.payload).await
    }))
    .await
}

Why initialize in main() and not in the handler?

Deterministic Configuration: The thread pool is configured once per container, before any requests arrive. This prevents race conditions if multiple requests try to initialize concurrently.
Container Reuse: Lambda containers can serve multiple requests. Initializing in main() ensures the configuration is set during the cold start and persists for all subsequent warm invocations.
Performance: Thread pool setup happens during cold start (already counted as initialization time), not during request processing.

Implementing the Request Handler

src/handler.rs:

use serde::{Deserialize, Serialize};
use std::env;
use std::sync::Once;
use std::time::Instant;
use std::collections::HashSet;
use std::sync::Mutex;
use rayon::prelude::*;

static INIT: Once = Once::new();

#[derive(Deserialize)]
pub struct ProcessRequest {
    count: usize,
    mode: String,
}

#[derive(Serialize)]
pub struct ProcessResponse {
    processed: usize,
    duration_ms: u128,
    mode: String,
    workers: usize,
    detected_cpus: usize,
    avg_ms_per_item: f64,
    memory_used_kb: u64,
    threads_used: usize, // Actual threads that processed items (proves multi-threading)
}

// CPU-intensive bcrypt hashing with cost factor 10
fn hash_password(password: &str) -> Result<String, bcrypt::BcryptError> {
    bcrypt::hash(password, 10)
}

// Process items one at a time (baseline for comparison)
fn process_sequential(items: Vec<String>) -> Result<(Vec<String>, usize), Box<dyn std::error::Error + Send + Sync>> {
    let results: Result<Vec<String>, _> = items
        .iter()
        .map(|item| hash_password(item))
        .collect();
    results
        .map(|r| (r, 1))
        .map_err(|e| Box::new(e) as Box<dyn std::error::Error + Send + Sync>)
}

// Process items in parallel using Rayon's work-stealing scheduler
// Thread pool size is configured once at cold start via init_thread_pool()
fn process_parallel(items: Vec<String>) -> Result<(Vec<String>, usize), Box<dyn std::error::Error + Send + Sync>> {
    let thread_ids: Mutex<HashSet<std::thread::ThreadId>> = Mutex::new(HashSet::new());

    let results: Result<Vec<String>, _> = items
        .par_iter()
        .map(|item| {
            thread_ids.lock().unwrap().insert(std::thread::current().id());
            hash_password(item)
        })
        .collect();

    let threads_used = thread_ids.lock().unwrap().len();
    results
        .map(|r| (r, threads_used))
        .map_err(|e| Box::new(e) as Box<dyn std::error::Error + Send + Sync>)
}

// Get worker count from env var or detect CPUs, clamped to 1-6
pub fn get_worker_count() -> usize {
    if let Ok(count_str) = env::var("WORKER_COUNT") {
        if let Ok(count) = count_str.parse::<usize>() {
            return count.clamp(1, 6);
        }
    }
    num_cpus::get().clamp(1, 6)
}

// Initialize Rayon global thread pool (only once per Lambda container)
pub fn init_thread_pool(workers: usize) {
    INIT.call_once(|| {
        let _ = rayon::ThreadPoolBuilder::new()
            .num_threads(workers)
            .build_global();
    });
}

// Read RSS memory from /proc/self/statm (Linux only)
fn get_memory_usage_kb() -> u64 {
    std::fs::read_to_string("/proc/self/statm")
        .ok()
        .and_then(|s| s.split_whitespace().nth(1)?.parse::<u64>().ok())
        .map(|pages| pages * 4)
        .unwrap_or(0)
}

// Main Lambda handler - processes items sequentially or in parallel
pub async fn function_handler(request: ProcessRequest) -> Result<ProcessResponse, Box<dyn std::error::Error + Send + Sync>> {
    if request.count == 0 { return Err("count must be greater than 0".into()); }
    if request.count > 1000 { return Err("count exceeds maximum of 1000 items".into()); }

    let items: Vec<String> = (0..request.count)
        .map(|i| format!("password_{:06}", i))
        .collect();

    let workers = get_worker_count();
    let mode = match request.mode.as_str() {
        "sequential" => "sequential",
        "parallel"   => "parallel",
        _            => if workers > 1 { "parallel" } else { "sequential" },
    };

    let start = Instant::now();
    let (results, threads_used) = match mode {
        "sequential" => process_sequential(items)?,
        _            => process_parallel(items)?,
    };
    let duration_ms = start.elapsed().as_millis();

    Ok(ProcessResponse {
        processed: results.len(),
        duration_ms,
        mode: mode.to_string(),
        workers: if mode == "parallel" { workers } else { 1 },
        detected_cpus: num_cpus::get(),
        avg_ms_per_item: duration_ms as f64 / request.count as f64,
        memory_used_kb: get_memory_usage_kb(),
        threads_used,
    })
}

Key Implementation Details

Thread Pool Initialization at Cold Start: The code initializes the thread pool in main() before the Lambda runtime starts, not during request processing. This approach is designed to eliminate race conditions and provide deterministic behavior across all invocations.

Important Note: Lambda initializes the thread pool once per container. The thread pool configuration retains its original value even if you change the WORKER_COUNT environment variable between invocations within the same container. For production deployments, keep WORKER_COUNT consistent for the function’s lifecycle.

Input Validation: The handler validates that count is between 1 and 1000 to prevent resource exhaustion.

Thread Tracking: The threads_used field proves multi-threading is working by tracking unique thread IDs during parallel processing. This provides empirical validation that work is distributed across multiple threads.

Memory Tracking: The memory_used_kb field reports RSS memory usage by reading /proc/self/statm, providing visibility into actual memory consumption.

Mode Selection: The function supports three modes:

sequential: Single-threaded processing
parallel: Multi-threaded processing using Rayon
auto: Automatically selects based on available workers

Building and Deploying

With the implementation complete, let’s compile the function for Lambda’s environment and deploy it to AWS.

# Build for ARM64 (Graviton2) - recommended for cost efficiency
cargo lambda build --release --arm64

# Or build for x86_64
cargo lambda build --release --x86-64

The build process produces a binary of approximately 1.7 MB (uncompressed) or 0.8 MB (zipped).

Deploy to AWS

Use Cargo Lambda to deploy the function with your desired memory configuration and worker count.

# Deploy with 6144 MB memory (4 vCPUs) and 4 workers
cargo lambda deploy rust-multithread-lambda \
    --memory 6144 \
    --timeout 30 \
    --env-var WORKER_COUNT=4

Note: To test different configurations, repeat the build and deploy commands with different --memory values and WORKER_COUNT settings for each configuration you want to benchmark. For comprehensive testing across architectures, build with --arm64, deploy all memory configurations, then rebuild with --x86-64 and deploy again.

Required IAM Permissions

The Lambda execution role needs the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
}

Test the Function

After deployment, verify the function works correctly by invoking it with a test payload.

aws lambda invoke \
    --function-name rust-multithread-lambda \
    --payload '{"count":20,"mode":"parallel"}' \
    --cli-binary-format raw-in-base64-out \
    response.json

Performance Benchmarks

We tested multiple configurations on ARM64 (Graviton2) to measure the impact of multi-threading.

Test workload: Processing 20 bcrypt password hashes (cost factor 10)

Note: Benchmark results may vary between runs due to factors such as Lambda placement, underlying hardware differences, and AWS infrastructure conditions. The numbers presented here are representative of typical performance observed across multiple test runs.

Performance Results: ARM64 (Graviton2)

Memory	vCPUs	Workers	Avg (ms)	P50 (ms)	P95 (ms)	P99 (ms)	Min	Max	Speedup
1536 MB	~1	1	1,885	1,882	1,898	1,898	1,877	1,907	1.00x
2048 MB	~2	2	1,334	1,331	1,341	1,341	1,324	1,356	1.41x
4096 MB	~3	3	685	683	699	699	669	704	2.75x
6144 MB	~4	4	463	464	467	467	453	469	4.07x
8192 MB	~5	5	338	343	345	345	325	346	5.57x
10240 MB	~6	6	280	278	292	292	271	293	6.73x

Performance Results: x86_64

Memory	vCPUs	Workers	Avg (ms)	P50 (ms)	P95 (ms)	P99 (ms)	Min	Max	Speedup
1536 MB	~1	1	1,671	1,675	1,681	1,681	1,659	1,684	1.00x
2048 MB	~2	2	1,253	1,249	1,265	1,265	1,241	1,294	1.33x
4096 MB	~3	3	892	891	899	899	888	900	1.87x
6144 MB	~4	4	429	425	443	443	417	449	3.89x
8192 MB	~5	5	330	323	349	349	317	358	5.06x
10240 MB	~6	6	292	292	298	298	291	298	5.72x

Architecture Comparison

Memory	Workers	ARM64 Avg	x86_64 Avg	Diff %	Faster Arch
1536 MB	1	1,885 ms	1,671 ms	-12.8%	x86_64
2048 MB	2	1,334 ms	1,253 ms	-6.4%	x86_64
4096 MB	3	685 ms	892 ms	+23.2%	ARM64
6144 MB	4	463 ms	429 ms	-7.9%	x86_64
8192 MB	5	338 ms	330 ms	-2.4%	x86_64
10240 MB	6	280 ms	292 ms	+4.1%	ARM64

Key Observations

Cold Start Performance: Rust’s cold start initialization times are consistently between 19-28 ms across all memory configurations and architectures. ARM64 (Graviton2) shows slightly faster cold starts (19-23 ms) compared to x86_64 (26-29 ms). Both are significantly faster than interpreted runtimes because the binary is pre-compiled.

Near-Linear Scaling: Both architectures achieve impressive speedups:

ARM64: 6.73x speedup with 6 workers (exceeds theoretical 6x!)
x86_64: 5.72x speedup with 6 workers

Latency Consistency: The P95 and P99 metrics show excellent consistency:

ARM64 at 6 vCPUs: P50=278ms, P95=292ms, P99=292ms (low variance)
x86_64 at 6 vCPUs: P50=292ms, P95=298ms, P99=298ms

Both architectures show consistent latency at maximum parallelization.

Cost Analysis

Let’s analyze the cost implications of different configurations for processing 20 bcrypt hashes.

Cost Comparison: ARM64 vs x86_64 (us-east-1, as of January 2026):

Config	Memory	Workers	ARM64 Duration	ARM64 Cost/1M	x86_64 Duration	x86_64 Cost/1M	Cheaper Arch
1 vCPU	1536 MB	1	1,885 ms	$38.60	1,671 ms	$42.78	ARM64
2 vCPU	2048 MB	2	1,334 ms	$36.46	1,253 ms	$42.77	ARM64 *
3 vCPU	4096 MB	3	685 ms	$37.47	892 ms	$60.80	ARM64
4 vCPU	6144 MB	4	463 ms	$37.97	429 ms	$44.00	ARM64
5 vCPU	8192 MB	5	338 ms	$36.94	330 ms	$45.10	ARM64
6 vCPU	10240 MB	6	280 ms	$38.27	292 ms	$49.87	ARM64

*Cheaper Arch

Cost Formulas:

ARM64: (Memory in GB) × (Duration in seconds) × $0.0000133334
x86_64: (Memory in GB) × (Duration in seconds) × $0.0000166667 (25% higher rate)

Key Insight: The 2 vCPU ARM64 configuration provides the lowest cost at $36.46 per million invocations while achieving 1.41x speedup. All ARM64 configurations remain cost-competitive ($36-$39 range) despite significant performance differences, demonstrating how increased throughput can offset higher memory costs.

Choosing the Right Configuration:

Priority	Recommended Config	Rationale
Lowest Cost	ARM64, 2048 MB, 2 workers	$36.46/1M invocations, 1.41x speedup
Balanced	ARM64, 4096 MB, 3 workers	$37.47/1M invocations, 2.75x speedup
Low Latency	ARM64, 10240 MB, 6 workers	280ms avg, 6.73x speedup

When to Use Multi-threaded Rust on Lambda

Recommended Use Cases

Batch data processing: Transform, validate, or enrich large datasets
Cryptographic operations: Hashing, encryption, digital signatures
Image/video processing: Resize, transcode, analyze media files
Scientific computing: Simulations, data analysis, machine learning inference
High-volume workloads: Functions invoked >100,000 times per day benefit from optimization

When to Consider Alternatives

I/O-bound operations: Use async Rust instead of multi-threading for database queries or API calls
Simple transformations: Functions completing in <100ms rarely benefit from parallelization
Low-volume workloads: Development overhead may not be justified for <10,000 invocations per day
Rapid prototyping: Python or Node.js may be more appropriate when iteration speed is critical

Cleanup

To delete the resources created in this post:

# Delete the Lambda function
aws lambda delete-function --function-name rust-multithread-lambda

# Delete the CloudWatch log group
aws logs delete-log-group --log-group-name /aws/lambda/rust-multithread-lambda

Note: If you deployed multiple configurations for testing, you’ll need to delete each function individually by repeating the delete command with each function name, or use the SAM template for bulk cleanup:

aws cloudformation delete-stack --stack-name rust-multithread-benchmark

Conclusion

When you allocate more memory to your Lambda function, AWS provides proportionally more vCPUs—up to 6 vCPUs at 10,240 MB. However, sequential code only uses one vCPU, leaving the additional compute power idle while you pay for the full allocation. Multi-threaded Rust with Rayon enables you to harness all available vCPUs for CPU-intensive workloads, transforming unused capacity into real performance gains.

Our benchmarks demonstrate this clearly:

Near-linear scaling: ARM64 achieved 6.73x speedup with 6 workers—you get proportional returns on your vCPU investment
Fast cold starts: 19-28 ms initialization across all configurations, eliminating the cold start concerns often associated with compiled languages
Consistent latency: ARM64 at 6 vCPUs shows only 1ms variance between P50 and P99, critical for predictable response times
Cost efficiency: ARM64 is 15-20% cheaper than x86_64 with better scaling at maximum parallelization

The key takeaway: If your Lambda function performs CPU-intensive work and you’re allocating more than 1,769 MB of memory, you likely have multiple vCPUs available. Without multi-threading, those vCPUs sit idle. Rayon’s parallel iterators allow you to switch from sequential to parallel processing by changing .iter() to .par_iter() in your code.

Recommended starting point: ARM64 with 4096 MB (3 workers) offers an excellent balance of cost and performance for most workloads. Scale up to 6 vCPUs for latency-critical applications, or down to 2 vCPUs for maximum cost savings.

Additional Resources

The complete sample code, SAM template, and test scripts from this post are available at Github Repository.

Amazon SageMaker AI now hosts NVIDIA Evo-2 NIM microservices

Malvika Viswanathan — Tue, 24 Feb 2026 18:48:08 +0000

This post is co-written with Neel Patel, Abdullahi Olaoye, Kristopher Kersten, Aniket Deshpande from NVIDIA.

Today, we’re excited to announce that the NVIDIA Evo-2 NVIDIA NIM microservice are now listed in Amazon SageMaker JumpStart. You can use this launch to deploy accelerated and specialized NIM microservices to build, experiment, and responsibly scale your drug discovery workflows on Amazon Web Services (AWS).

In this post, we demonstrate how to get started with these models using Amazon SageMaker Studio.

NVIDIA NIM microservices on AWS

NVIDIA NIM integrates closely with AWS managed services, such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Kubernetes Service (Amazon EKS), and Amazon SageMaker AI, to support deployment of generative AI models at scale. As part of NVIDIA AI Enterprise, which is available in the AWS Marketplace, NVIDIA NIM is a set of microservices designed to accelerate the deployment of generative AI. These prebuilt containers support a broad spectrum of generative AI models, from open source community models, to NVIDIA Nemotron and custom models. NIM microservices are deployed with just a few lines of code, or with a few actions in the SageMaker Studio console. Engineered to facilitate seamless generative AI inferencing at scale, NIM ensures that generative AI applications can be deployed on various AWS services.

NVIDIA BioNeMo Evo 2 overview

NVIDIA BioNeMo is a platform of NIM microservices, developer tools, and AI models that accelerate building, adapting, and deploying biomolecular AI models for drug discovery. It packages curated training recipes, data loaders, and domain-optimized pretrained models for DNA, RNA, and proteins, alongside NVIDIA CUDA-X libraries such as NVIDIA cuEquivariance. These components power tasks such as 3D structure prediction, de novo design, virtual screening, docking, and property prediction with GPU-accelerated performance.

NVIDIA NIM microservices provide optimized, API-first inference that integrates directly into enterprise pipelines across on-premises and the cloud, providing scalable and secure deployment with faster time-to-market and lower Total Cost of Ownership (TCO). The Evo 2 NIM delivers a 40-billion parameter foundation model (FM) trained on a vast dataset of genomes that can be used to predict protein function, identify mutations, and accelerate bioengineering research. Furthermore, the Evo 2 NIM can be chained with other NIM microservices such as ESMFold to create end-to-end, containerized workflows that cut time-to-insight while streamlining deployment through consistent APIs.

SageMaker Studio overview

SageMaker Studio is a web-based integrated development environment (IDE) for machine learning (ML) that provides a unified visual interface for all of the tools that you need to complete each step of the ML development lifecycle. SageMaker Studio provides complete access, control, and visibility into each step of the ML workflow, from data preparation to model building, training, and deployment.

The key features of SageMaker Studio include:

Unified interface: Access all SageMaker capabilities through a single, web-based visual interface
Jupyter notebooks: Fully managed Jupyter notebooks with pre-configured kernels for popular ML frameworks
Model management: Browse, deploy, and manage models from AWS Marketplace and other sources through an intuitive interface
Collaboration: Share notebooks, experiments, and models with your team members
Built-in security: Integrated with AWS Identity and Access Management (IAM) for secure access control
Cost management: Monitor and control costs with built-in usage tracking and resource management tools

Amazon SageMaker JumpStart overview

SageMaker JumpStart is a fully managed service that offers state-of-the-art foundation models for various use cases such as content writing, code generation, question answering, copywriting, summarization, classification, and information retrieval. It provides a collection of pre-trained models that you can deploy quickly, accelerating the development and deployment of ML applications. One of the key components of SageMaker JumpStart is model hubs, which offer a vast catalog of pre-trained models, such as Mistral, for a variety of tasks. You can now discover and deploy Evo 2 NIM in Amazon SageMaker Studio or programmatically through the SageMaker Python SDK, so you can derive model performance and MLOps controls with Amazon SageMaker AI features such as Amazon SageMaker Pipelines, Amazon SageMaker Debugger, or container logs. The model is deployed in a secure AWS environment and in your VPC, helping to support data security for enterprise security needs.

Prerequisites

Before getting started with deployment, make sure that your IAM service role for SageMaker AI has the SageMakerFullAccess permission policy attached. To deploy the NVIDIA NIM microservices successfully, confirm one of the following:

Make sure that your IAM role has the following permissions, and that you have the authority to make AWS Marketplace subscriptions in the AWS account used:

aws-marketplace:ViewSubscriptions
aws-marketplace:Unsubscribe
aws-marketplace:Subscribe

If your account is already subscribed to the model, then you can skip to the following Deploy section. Otherwise, start by subscribing to the model package and move to the Deploy section after.

Subscribe to the model package

To subscribe to the model package, complete the following steps:

Open the SageMaker Jumpstart portal from the SageMaker AI page.
Search for Evo 2 NIM.
Choose View model, and on the Model details page choose Subscribe. This will take you to the AWS Marketplace listing for the Evo 2 NIM.
On the AWS Marketplace listing page, choose View purchase options, review the purchase terms and choose the Subscribe button if you and your organization agree with EULA, pricing, and support terms.
Choose Continue to with the configuration and choose an AWS Region where you have the service quota for the desired instance type.

A product Amazon Resource Name (ARN) is displayed. This is the model package ARN that you need to specify while creating a deployable model using the SageMaker SDK.

Option 1: Deploy the Evo 2 NIM using SageMaker Studio

The following section outlines how to deploy the EVO 2 NIM using SageMaker Studio.

Getting started with SageMaker Studio

Begin by accessing the AWS Management Console and navigating to the SageMaker AI service. When you’re in the SageMaker AI console, locate Studio in the left navigation panel and choose Open Studio next to your user profile. If you haven’t set up a SageMaker Studio domain yet, then you must create a new domain and user profile first. This launches the web-based SageMaker Studio interface where you can manage all aspects of your ML workflow.

Navigating to model packages

Within SageMaker Studio, look for Models in the left sidebar and choose JumpStart base models tab within the Models interface. This section contains all available model packages in SageMaker JumpStart, including those from the AWS Marketplace

Locating the Evo-2 NIM model

Use the search functionality to find the NVIDIA Evo-2 NIM model by searching for terms such as “Evo-2” or “NVIDIA”. When you locate the model package in the filtered results, choose it to view the Model overview page. This page provides an overview of the model and can have a Notebooks tab that will show a sample notebook that contains an example showing how to use the NIM. You can choose Open in JupyterLab to open the notebook in JupyterLab and use it as a starting point for using the NIM.

Configuring the model deployment

On the model package overview page, choose the Deploy button on the top right to begin the deployment process. You must configure several important settings: provide a unique endpoint name (such as “Evo-2-nim-endpoint”), choose an appropriate instance type (ml.g6e.12xlarge is recommended for optimal performance), set the initial instance count (typically 1 for initial testing), and specify an endpoint configuration name. Review all of these settings carefully before proceeding.

Initiating and monitoring the deployment

After verifying your configuration settings, choose Deploy to start the deployment process for creating a Real-time inferance endpoint. Navigate to the Deployments section and then the Endpoints section in the left sidebar to monitor the deployment progress. The endpoint status initially shows Creating and typically takes 5–10 minutes to complete. You can track the progress and should see the status change to InService once the deployment is successful.

Testing and validation

When your endpoint is deployed and shows the In Service status, you can optionally test it directly through the SageMaker Studio interface. Choose your deployed endpoint from the endpoints list to access the Endpoint summary page. Scroll down and select the Playground tab. If available, you will see two options: Test the sample request and Use Python SDK example code. You can use either option to validate the deployment by using a sample protein sequence. This validates the endpoint is working correctly before integrating it into your applications.

Option 2: Deploy Evo 2 using the SageMaker SDK

In this section we walk through deploying the Evo-2 NIM through the SageMaker SDK. Make sure that you have the account-level service limit for using ml.g6e.12xlarge for endpoint usage as one or more instances. Furthermore, NVIDIA provides a list of supported instance types that support deployment. Refer to the AWS Marketplace listing for the model to see the supported instance types. To request a service quota increase, go to the AWS service quotas.

import sagemaker
import boto3
from sagemaker import ModelPackage, get_execution_role
import json
# Initialize SageMaker session and role
role = get_execution_role()
sagemaker_session = sagemaker.Session()
# Model Package ARN from your AWS Marketplace subscription
# Replace this with your actual Model Package ARN after subscription
model_package_arn = "arn:aws:sagemaker:<region>:<account-id>:model-package/Evo-2-nim-model"
# Create model from AWS Marketplace Model Package
model = ModelPackage(
    role=role, 
    model_package_arn=model_package_arn,
    sagemaker_session=sagemaker_session
)
# Deploy the model to an endpoint
predictor = model.deploy(
    initial_instance_count=1,
    instance_type="ml.g6e.12xlarge",  # Using recommended NVIDIA GPU instance
    endpoint_name="Evo-2-endpoint",
    wait=True
)

Run Inference with Evo 2 SageMaker endpoint

When you have the model, you can use a sample text to do an inference request. NIM on SageMaker supports the OpenAI API inference protocol inference request format. For an explanation of the supported parameters, go to the Evo-2 API documentation.

Real-time inference example

sm_runtime = boto3.client("sagemaker-runtime", region_name=region)

generate_payload = {

 "sequence": "ACGTACGTACGT",

 "num_tokens": 100,

 "temperature": 0.7,

 "top_k": 3,

}

response = sm_runtime.invoke_endpoint(

EndpointName='Evo2-40b-2-1-0',

ContentType="application/json",

Body=json.dumps(generate_payload),

)

result = json.loads(response["Body"].read())

print("Generated DNA:", result["sequence"])
print("Elapsed (ms):", result.get("elapsed_ms"))

Example output:

Generated DNA: ACGTACATATGTTCGTACATTCGCACAGACGCCATTTTGAAAAATGCTTTAAATGGATTCAGAATTGGTCAAAATGCATAAATCCATCAAAATTTTTTTC
Elapsed (ms): 10770

Cleaning up

To avoid unwanted charges, complete the steps in this section to clean up your resources.

Deleting the endpoint from SageMaker Studio

In SageMaker Studio, navigate to the Endpoints section in the left sidebar under Inference to view all your active endpoints. Locate your Evo-2 NIM endpoint in the list and select it to open the endpoint details page. On this page, there is a Delete button. Choose Delete and confirm the deletion when prompted. The endpoint status changes to Deleting and disappears from your endpoints list when the deletion is complete. This process typically takes a few minutes, and when it’s deleted the endpoint stops incurring charges immediately.

Delete the SageMaker endpoint

The SageMaker endpoint that you deployed incurs costs if you leave it running. Use the following code to delete the endpoint if you want to stop incurring charges. For more details, go to Delete endpoints and resources.

# Delete endpoint when done (important for cost management)
predictor.delete_endpoint()

Conclusion

The availability of NVIDIA Evo-2 NIM microservices on Amazon SageMaker Jumpstart represents a significant advancement for researchers and organizations working in drug discovery. This solution provides GPU-accelerated multiple sequence alignments and dramatically speeds up structure prediction pipelines that are critical for protein design and antibody research. Users can implement the flexible deployment options—through SageMaker Studio, or SageMaker SDK—to choose the approach that best fits their workflow and technical expertise. The optimized performance of these NIM microservices, combined with the scalability and security of SageMaker, enables faster time-to-insight while streamlining the deployment of complex biomolecular AI models. We encourage you to try the Evo-2 NIM today and look out for future release of MSA-search and Boltz-2 NIMs to accelerate your drug discovery workflows and use the power of NVIDIA’s specialized microservices on AWS infrastructure.

Building fault-tolerant applications with AWS Lambda durable functions

Rahul Pisal — Fri, 06 Feb 2026 16:54:39 +0000

Business applications often coordinate multiple steps that need to run reliably or wait for extended periods, such as customer onboarding, payment processing, or orchestrating large language model inference. These critical processes require completion despite temporary disruptions or system failures. Developers currently spend significant time implementing mechanisms to track progress, handle failures, and manage resources when waiting for external events, shifting focus from business logic to undifferentiated tasks.

At re:Invent 2025, Amazon Web Services (AWS) launched AWS Lambda durable functions, a new capability extending Lambda’s event-driven programming model with built-in capabilities to build fault-tolerant multi-step applications and AI workflows using familiar programming languages. At its core, durable functions are regular Lambda functions, so your development and operational processes for Lambda continue to apply. However, when you create a Lambda function you can now enable durable execution, so that you can checkpoint progress, automatically recover from failures, and suspend execution for up to one year when waiting on long-running tasks, such as human-in-the-loop processes.

How Lambda durable functions work

When working with standard Lambda functions, your code runs from start to finish in a single invocation. If a failure occurs at any point during the execution, the entire function must be retried by the invoking event source. Any state that needs to be preserved between executions must be explicitly saved and retrieved. This is typically done by using external storage services such as Amazon DynamoDB or Amazon Simple Storage Service (Amazon S3). Furthermore, you must typically guard against duplicate (concurrent) invocations of the same event and have a strategy to safely deploy updates while continuing to process events.

In contrast, with Lambda durable functions, developers use durable operations such as “Steps” and “Waits” in the event handler to checkpoint progress, handle failures, and suspend execution during wait periods without incurring compute charges for on-demand functions. These durable operations and any optional state returned from them are automatically persisted by Lambda in a fully-managed durable execution backend. If failures occur during the execution, or if your function resumes its execution after being paused, Lambda invokes your function again, restoring (replaying) the previous state by executing the event handler from the start, but skipping over completed durable operations. To streamline this checkpoint/replay mechanism for developers, you can use the Lambda durable execution SDK to wrap or annotate your event handler, which enhances the existing Lambda context with several new methods like context.step() and context.wait(). Furthermore, you can use methods such as context.waitForCallback() to wait on external jobs or asynchronous processes, such as “human-in-the-loop” scenarios. The execution is paused until a SendDurableExecutionCallbackSuccess or SendDurableExecutionCallbackFailure response is sent to the Lambda API.

Getting started

Use the AWS Serverless Application Model (AWS SAM) to create a new durable function with sam init with an AWS Quick Start Template. Lambda durable functions are also supported by the AWS Cloud Development Kit (AWS CDK), AWS Command Line Interface (AWS CLI), AWS CloudFormation and other infrastructure as code (IaC) frameworks such as Terraform.

Consider the following function, which performs user onboarding. First, it creates a user profile based on some data, then it sends out an email for verification and waits until the user either confirms the email address, or a 24-hour timeout is reached. Finally, it sends out a confirmation.

import {
  DurableContext,
  withDurableExecution,
} from '@aws/durable-execution-sdk-js';
export const handler = withDurableExecution(
  async (event: OnboardingEvent, context: DurableContext) => {
    try {    
      // Create user profile
      const profile = await context.step("create-profile", async () =>
        createUserProfile(event.email, event.name)
      );
      // Wait for email verification via callback
      const verification = await context.waitForCallback(
        "wait-for-email-verification",
        async (callbackId) => {
          // Send email to user and pass callbackId
          await sendVerificationEmail(profile, callbackId);
        },
        {
          timeout: { hours: 24 } 
        }
      );
      // Send confirmation and welcome email
      const result = await context.step("complete-onboarding", async () => {
        if (!verification || !verification.verified) 
     return { ...profile, status: 'failed' };
        await sendWelcomeEmail(profile.email, profile.name);
        return { ...profile, status: 'active' };
      });
      return result;
    } catch (error) {
      // omitted 
    }
  }
);

Durable functions have built-in and fully customizable error handling for steps. For example, if the profile was successfully created and verified, but a temporary error occurred when sending out the confirmation, then the step is retried. The retry skips over any previously completed checkpoints, such as the profile creation and callback. Only the code within the send confirmation step is run again.

Next, you update the AWS SAM template to include your durable function. You create a Lambda durable function by including the DurableConfig setting for your function. Note that you currently cannot add a durable configuration to a function that was originally created without it. The ExecutionTimeout defines after which time the durable execution times out to protect against runaway or deadlock application bugs. This setting is separate from the invocation timeout, which defines for how long a single invocation can run. The maximum invocation timeout for a single function invocations remains unchanged at 15 minutes. With Lambda durable functions, you will typically see multiple invocations per durable execution, such as when using the wait capabilities in the SDK or automatic retries. You can set the ExecutionTimeout for up to one year when using asynchronous invocations.

The RetentionPeriodInDays defines how long the execution data of a durable execution is available to you after executions complete.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
 
Resources:
  UserOnboardingFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: UserOnboardingFunction
      CodeUri: ./src
      Handler: index.handler
      Runtime: nodejs24.x
      Architectures:
        - x86_64
      MemorySize: 256
      Timeout: 60		   // Timeout for an individual invocation
      DurableConfig:		   // This makes the function a durable function
        ExecutionTimeout: 90000 // 25h timeout for the durable execution overall
        RetentionPeriodInDays: 7 
UserOnboardingFunctionRole:
    Type: AWS::IAM::Role
    // omitted for brevity

You must include the necessary permissions for your function. For example, the AWSLambdaBasicDurableExecutionRole managed policy only allows the minimal AWS Identity and Access Management (IAM) actions to create/retrieve checkpoints and logs to increase security. Therefore, it does not include permissions to invoke other (durable) functions or manage callbacks. Refer to the documentation for more details.

Testing locally

Before deploying your function, you can test it locally using AWS SAM local invoke.

AWS SAM locally invokes your function and runs the event handler until it reaches the context.waitForCallback(). To complete callbacks, AWS SAM offers new commands to interact with your durable functions. In this example, you send a Success response to complete the callback. You can also include relevant data in the response. You can send the response directly using the on-screen guide or using another AWS SAM CLI command from another process.

sam local callback succeed <your-callback-id> --result '<your data>'

To inspect an execution, you can use AWS SAM to retrieve the durable execution history of your function, which includes details about steps, callbacks, and wait durations, as shown in the following example code.

sam local execution history <execution-arn>

Depending on your use case, you can instead send a Failure response to a callback and handle those errors in your code. For example, by performing compensation logic in a subsequent step:

sam local callback fail <your-callback-id> --error-data '<your data>'

Now that you have verified that your function works as intended, deploy it to AWS using sam deploy command.

Best practices and considerations

Invoking a Lambda durable function requires a qualified Amazon Resource Name (ARN), such as an alias or version. We recommend that you don’t use the $LATEST qualifier except for rapid prototyping or local testing. Using explicit versions ensures that replays always happen with the same code with which the execution was started. This is to ensure deterministic execution and prevent inconsistencies when updating your function code during executions.

We recommend bundling the durable execution SDK with your function code using your preferred package manager. The SDKs are fast-moving, so you can update dependencies as new features become available.

There are other durable operations in the Lambda durable functions SDK that you can use to build your application:

waitForCondition(): Pauses the execution of your function until a condition is met. For example, the status of a job polled with an API. For this to work, you provide the waitStrategy and a check function to poll the status.
parallel(): Runs multiple durable operations in parallel within the same function, with configurable options such as the maximum number of concurrent branches and desired failure behavior. This streamlines managing durability and checkpointing for simultaneous asynchronous actions.
map(): Creates a durable operation and checkpoint for each item of an array, based on the provided mapping function. The items are processed concurrently.
invoke(): Invokes another Lambda function and waits for its result. The SDK creates a checkpoint, invokes the target function, and resumes your function when the invocation completes. This enables function composition and workflow decomposition.

Refer to the developer guide for more details.

Lambda compute charges apply to all invocations, including any replays. When using wait operations, the function suspends execution and, for on-demand functions, doesn’t incur duration charges until execution resumes. You’re also charged for durable operations, data written, and data retention. To learn more about Lambda durable functions pricing, refer to the Lambda pricing page.

For the latest Region availability, visit the AWS Capabilities by Region page.

Conclusion

AWS Lambda durable functions extends the Lambda programming model to streamline building fault-tolerant, long-running applications using familiar programming patterns. You can use Lambda durable functions to write multi-step workflows in your preferred programming language, using built-in methods that automatically handle progress checkpointing and error recovery. This streamlines your architectures so that you can focus on your business logic, and optimize cost by charging only for active compute time.

You can build durable functions for Python or Node.js based Lambda functions using the Lambda API, AWS Management Console, AWS CLI, AWS CloudFormation, AWS SAM, AWS SDK, and AWS CDK.

To get started, visit the Lambda Developer Guide or watch the re:Invent breakout session.

Serverless ICYMI Q4 2025

Julian Wood — Fri, 30 Jan 2026 15:23:57 +0000

Stay current with the latest serverless innovations that can transform your applications. In this 31st quarterly recap, discover the most impactful AWS serverless launches, features, and resources from Q4 2025 that you might have missed.

In case you missed our last ICYMI, check out what happened in Q3 2025.

2025 Q4 calendar

Serverless at re:Invent 2025

This post covers the biggest serverless announcements from re:Invent 2025, highlighting key feature updates that can improve your applications, and shares valuable resources to keep you informed.

AWS re:Invent 2025 had more than 60,000 in-person attendees and more than 2 million online viewers for the keynotes. The event featured 3,500 sessions from 3,000 speakers, which included information on 530 AWS service and feature announcements.

Keynote Igniting the serverless movement

The serverless content consisted of two tracks: Containers and Serverless (CNS) and Application Integration (API). These tracks included 150 unique sessions watched in-person by more than 16,000 attendees. There were developer-focused experiences including a Road to re:Invent Hackathon, AWS Builder Loft, and Builders Arena. Serverlesspresso, the coffee shop powered by serverless technology, operated in two locations during the event: the Expo Hall and the certification lounge.

Serverless and developer community photo

Find a curated list of serverless videos on Serverless Land YouTube.

AWS Lambda durable functions

Managing state across multi-step serverless workflows has traditionally required complex external orchestration tools. AWS Lambda durable functions expand how developers can use Lambda. You can now build reliable multi-step applications and AI workflows directly within Lambda.

AWS Lambda durable functions code

Durable functions automatically checkpoint progress by saving the current state and completed steps at key points during execution. This allows them to suspend execution for up to one year during long-running tasks and recover from failures by resuming from the last checkpoint rather than restarting from the beginning, all without requiring additional infrastructure management.

Developers can now build in Python or TypeScript, wrap calls in steps with automatic retries and checkpointing. You can use waits to suspend execution for minutes, hours, or even up to a year without paying for idle compute. Durable functions use a replay mechanism to maintain state and handle failures gracefully. The replay mechanism works by re-executing your function code from checkpoints when recovering from failures, ensuring state consistency without data loss. This also means you don’t need complex external orchestration tools for many use cases. This can be helpful for AI workflows and multi-step applications where you need reliable state management without managing external infrastructure.

For more information, read the launch blog post and watch the re:Invent Breakout Session video: Deep Dive on AWS Lambda durable functions (CNS380)

AWS Lambda Managed Instances

Lambda now offers Lambda Managed Instances, a new compute option that combines Amazon EC2 flexibility with fully managed infrastructure. AWS automatically handles instance provisioning, scaling, and maintenance while allowing access to the full range of EC2 capabilities, including Graviton4, network-optimized instances, and other specialized compute options.

AWS Lambda Managed Instances configuration

Your functions run on dedicated EC2 capacity from your account, in your own Amazon Virtual Private Cloud (Amazon VPC). AWS still manages the operational overhead, including OS patching, load balancing, and auto-scaling. This gives you access to specialized hardware options while maintaining the serverless operational model. You can further improve costs by using EC2 pricing models, including Compute Savings Plans and Reserved Instances for Lambda workloads. Each instance can handle multiple concurrent requests, making this particularly valuable for high-volume, steady-state workloads where predictable pricing and specific hardware requirements matter.

For more information, read the launch blog post and watch the re:Invent Breakout Session video: Lambda Managed Instances: EC2 Power with Serverless Simplicity (CNS382).

Other Lambda announcements

Multi-tenant SaaS applications face challenges like data leakage between tenants and noisy neighbor effects where one tenant’s workload impacts others. They also struggle with implementing custom isolation mechanisms. Tenant isolation mode addresses these by processing function invocations in separate execution environments for each tenant. This manages tenant-level compute environment isolation automatically.

AWS Lambda tenant isolation

Lambda adds Provisioned Mode for Amazon SQS event-source mappings, providing predictable performance and reduced cold starts for high-throughput SQS processing workloads.

You can now send up to 1 MB of data in asynchronous Lambda invocations, increased from 256 KB, helping you build more complex data processing scenarios.

Lambda functions now support IPv6 networking, so you don’t need NAT Gateways when accessing the internet or other AWS services from VPC-connected functions.

Lambda internet connectivity through a NAT Gateway (IPv4) and Lambda internet connectivity through an egress-only internet gateway (IPv6).

Lambda Rust support is now generally available, moving from experimental status. This is backed by AWS Support and the Lambda availability SLA.

Lambda has expanded its runtime support by adding Python 3.14, Node.js 24, and Java 25 as both managed runtimes and container base images, providing access to the latest language features and ensuring long-term support.

Amazon ECS

Amazon Elastic Container Service (Amazon ECS) Express Mode streamlines the deployment and management of containerized applications by automating the infrastructure setup that traditionally slows down developers.

Amazon ECS Express Mode deployment

This means you can focus on building applications while deploying with confidence using AWS best practices. Express Mode lets you deploy production-ready containerized web applications and APIs with a single command. This automatically handles domains, networking, load balancing, AWS Identity and Access Management (IAM) roles, and auto-scaling through simplified APIs. When your applications evolve and require advanced features, you can seamlessly configure and access the full capabilities of the resources, including Amazon ECS. Learn more from the launch blog post.

Amazon ECS announced a public preview of a fully managed MCP server, enabling AI-powered experiences for development and operations. The Model Context Protocol (MCP) server provides enterprise-grade capabilities like automatic updates and patching, centralized security through AWS IAM integration, comprehensive audit logging via AWS CloudTrail, and the proven scalability, reliability, and support of AWS.

Amazon Elastic Container Registry (ECR) managed container image signing enhances your security posture and eliminates the operational overhead of setting up signing. Container image signing allows you to verify that images are from trusted sources. ECR automatically signs images as they are pushed using the identity of the entity pushing the image. Signing operations are logged through CloudTrail for full auditability.

Amazon API Gateway

Amazon API Gateway allows you to improve the responsiveness of your REST APIs by progressively streaming response payloads back to the client. With this new capability, you can use streamed responses to enhance user experience when building LLM-driven applications (such as AI agents and chatbots), improve time-to-first-byte (TTFB) performance for web and mobile applications, stream large files, and perform long-running operations while reporting incremental progress using protocols such as server-sent events (SSE).

Amazon API Gateway streaming

API Gateway introduces private integration with Application Load Balancers (ALBs). You can use this to expose your VPC-based applications securely through REST APIs without exposing your ALBs to the public internet.

You can also now configure enhanced TLS security policies on API endpoints and custom domain names, providing you with greater control over the security posture of your APIs.

Amazon EventBridge

Amazon EventBridge introduced an enhanced visual rule builder that helps developers discover and subscribe to events from custom applications and over 200 AWS services. The console-based interface integrates the EventBridge schema registry with a comprehensive event catalog and intuitive drag-and-drop canvas that simplifies building event-driven applications. Developers can browse and search through events with readily available sample payloads and schemas without having to hunt through individual service documentation. The schema-aware visual builder guides developers through creating event filter patterns and rules, reducing syntax errors and accelerating development time.

EventBridge also allows targeting SQS fair queues.

AWS Step Functions

AWS Step Functions allows for enhanced local testing through the TestState API, providing programmatic access to comprehensive testing capabilities without deploying to AWS. This helps you build automated test suites that validate your workflow definitions locally on your development machines. Test error handling patterns, data transformations, and mock service integrations using your preferred testing frameworks.

There is also a new metrics dashboard, giving you visibility into your workflow operations at both the account and state machine levels.

Other announcements

Savings Plans flexible pricing model extends to AWS managed database services with the launch of Database Savings Plans. This helps reduce database costs by up to 35% when committing to a consistent amount of usage ($/hour) over a 1-year term. Savings automatically apply each hour to eligible usage across supported database services, and additional usage beyond the commitment is billed at on-demand rates.

Amazon DynamoDB now supports multi-attribute composite keys in global secondary indexes. You no longer need to concatenate values into synthetic keys manually, which sometimes results in the need to backfill data before adding new indexes. Instead, you can create primary keys using up to eight existing attributes, making it easier to model diverse access patterns and adapt to new query requirements.

Amazon Bedrock introduced AgentCore with quality evaluations and policy controls for deploying trusted AI agents at scale.

Bedrock also added 18 fully managed open weight models, expanding AI model options for developers.

The Strands Agents SDK is an open source framework that takes a model-driven approach to building and running AI agents in just a few lines of code. TypeScript support is now available in preview so you can choose between Python and TypeScript for building Strands Agents.

Amazon S3 Vectors became generally available. S3 Vectors delivers purpose-built, cost-optimized vector storage for AI agents, inference, Retrieval Augmented Generation (RAG), and semantic search at billion-vector scale.

Serverless blog posts

October

November

Serverless Office Hours

Join our livestream every Tuesday at 11 AM PT for live discussions, Q&A sessions, and deep dives into serverless technologies. Episodes are available on-demand at serverlessland.com/office-hours.

October

Oct 7 – Amazon API Gateway Routing Rules
Oct 14 – Amazon DynamoDB Global Tables
Oct 21 – Building agents with Amazon Bedrock AgentCore
Oct 28 – “What’s new with Observability

November

December

Dec 9 – AWS Lambda Managed Instances
Dec 16 – AWS Lambda durable functions

Still looking for more?

You can also follow the Serverless Developer Advocacy team to see the latest news, follow conversations, and interact with the team.

Julian Wood: @julian_wood, https://www.linkedin.com/in/julianrwood/
Eric Johnson: @edjgeek, https://www.linkedin.com/in/singledigit/
Gunnar Grosch: @GunnarGrosch, https://se.linkedin.com/in/gunnargrosch
Erik Hanchet: @ErikCH, https://www.linkedin.com/in/erikhanchett/
Salih Gueler: @salihgueler, https://www.linkedin.com/in/salihgueler/
Marcia Villalba: @mavi888uy, https://www.linkedin.com/in/marciavillalba

And finally, visit Serverless Land for all your serverless needs.

More room to build: serverless services now support payloads up to 1 MB

Anton Aleksandrov — Thu, 29 Jan 2026 22:16:14 +0000

To support cloud applications that increasingly depend on rich contextual data, AWS has raised the maximum payload size from 256 KB to 1 MB for asynchronous AWS Lambda function invocations, Amazon Simple Queue Service (Amazon SQS), and Amazon EventBridge. Developers can use this enhancement to build and maintain context-rich event-driven systems and reduce the need for complex workarounds such as data chunking or external large object storage.

Overview

Modern cloud applications rely on context-rich, structured data to drive intelligent behavior. Large language model (LLM) prompts, telemetry signals, personalization data, machine learning (ML) outputs, and user interaction logs are no longer simple strings. Instead, they’re typically complex, nested JSON or YAML objects carrying meaningful context. Previously, developers working with serverless services such as Amazon SQS, Lambda (asynchronous invocations and Amazon SQS event-source mapping), or EventBridge had to carefully manage their data to fit within the 256 KB payload size limit. This commonly meant chunking larger payloads, externalizing payloads to object stores such as Amazon S3, or using data compression. These workarounds added complexity and latency, creating edge cases that were difficult to monitor and debug.

With the recent launches, you can now transmit payloads up to 1 MB, significantly reducing the need for complex data chunking and architectural workarounds. This increased capacity streamlines design patterns, reduces operational overhead, and makes event-driven systems more intuitive to build and maintain. Developers can now include richer data in single payloads—from detailed LLM prompts and full system states to comprehensive context and complete transaction histories.

The new 1 MB payload size limit applies to asynchronous Lambda function invocations, whether you trigger them using either SQS event-source mapping, AWS Command Line Interface (AWS CLI), AWS SDKs, Lambda Invoke API, or AWS services such as EventBridge. The increased limit also extends to all messages and events flowing through Amazon SQS queues and EventBridge Event Buses.

Getting started

There’s nothing you need to do to get started. This enhancement is automatically applied to all new and existing Lambda functions, SQS queues, and EventBridge Event Buses.

If you were previously chunking data at 256KB (or lower) threshold, then you might need to make changes to your service configurations or business logic code to start using the new limit. For example, if you’ve explicitly set Amazon SQS MaximumMessageSize attribute, then you might need to adjust it to a new desired value. Larger payloads might also result in higher costs, as described in the following section.

Real-world example: rich event context in agentic event-driven architectures

Event-driven architectures allow services to operate independently without centralized coordination. In these systems, comprehensive event context is essential. With the increased 1 MB payload limit, events can now carry more comprehensive data—from user profiles and order details to historical interactions. This enables services such as inventory, shipping, and notifications to act autonomously.

Consider the following example. In hospitality and quick-service industries, customer satisfaction depends on timely, thoughtful service recovery. When a guest submits negative feedback through a survey, review, or complaint form, service teams must gather context, interpret the issue, and craft a response. Traditionally, this meant manually piecing together visit logs, loyalty data, and prior complaints. Now, this can be fully automated using an AI agent powered by AWS serverless services and Amazon Bedrock, as shown in the following figure.

Figure 1: Customer feedback processing pipeline

The workflow:

Receive: A new review is submitted through the Review application and emitted as an event to EventBridge Event Bus.
Detect: Event Bus delivers the event to downstream Feedback analysis agent. The agent running in a Lambda function recognizes the review as low-rating or complaint.
Enrich: The agent collects the guest’s visit metadata, booking details, loyalty activity, and complaint history using attached MCP tools into a single structured JSON payload (up to 1 MB).
Queue: The payload is sent to an SQS queue for further asynchronous processing by downstream components.
Generate: A separate Lambda function polls messages from Amazon SQS and invokes an Amazon Bedrock model to analyze the full complaint context, draft a personalized response, suggest a gesture (such as a refund or credit), and classify issue severity.
Deliver: The message is logged and sent to the customer, and to the service team for further analysis.

This use case demonstrates the importance of having a rich context: current and previous visits details, loyalty tier, prior interactions, and feedback history. Previously, teams had to offload pieces of context to Amazon S3 and reference them externally, adding latency and architectural complexity. The new 1 MB payload size means that all this information can be transported together, improving the serverless agentic workflow efficiency and streamlining maintenance.

Best practices when using large payloads

The following sections outline best practices that you should apply when using larger payloads.

Performance considerations

Monitor Lambda function memory usage carefully when working with larger payloads, because parsing and processing complex JSON objects can increase memory usage and execution duration. Test your systems thoroughly under load, especially for high-throughput applications, by benchmarking with realistic payload sizes and traffic patterns. Although the payload limit has increased to 1 MB, the Lambda 15-minute timeout and memory limits remain unchanged. When applicable, you can use compression to process even larger datasets efficiently, but remember to account for the added CPU overhead of compression and decompression in your performance calculations. Read the Monitoring best practices for event delivery with Amazon EventBridge post for more best practices to tune your event-driven architectures performances.

Operational guidelines

Configure dead-letter-queues (DLQ) to make sure that failed messages are retained for inspection and troubleshooting. This becomes especially important with larger payloads, because debugging complex data structures necessitates access to the complete message context. Implement robust error handling and retries to manage transient failures, particularly when processing rich payload content that may contain nested structures or complex relationships.

To further optimize throughput, you can batch similar smaller events together into a single payload. However, avoid mixing unrelated events and maintain clear boundaries between different business domains and processes.

Always make sure that your downstream dependencies are capable of handling larger payloads.

When to use external storage

Even with the increased 1 MB payload limit, there are scenarios where patterns such as claim check remain a sound architectural choice. These patterns involve storing a full payload in an external system, such as Amazon S3, and passing a lightweight reference through your event stream. This approach continues to provide value when payloads exceed the new limit, when data needs to be reused by multiple consumers, or when strict governance, traceability, and security requirements are involved. For example, audit logs, image metadata, or large ML inference inputs may still surpass the 1 MB boundary, even when compressed. Instead of risking truncation or fragmentation, a claim check enables consistent, scalable access to the complete data set.

You can use open source libraries such as the Kafka sink connector for EventBridge and Amazon SQS Extended Client Library (available for Python and Java) that abstract complexities of storing large objects in external storage.

Cost management

Although larger payloads enable richer context in your applications, logging full payloads can increase storage and processing costs. Services such as CloudWatch Logs charge based on data volume, thus implementing selective logging, payload truncation, or sampling becomes crucial for high-volume events. Consider logging only essential fields or implementing smart sampling strategies based on business importance.

For full payload archival and retention, evaluate cost-effective storage solutions such as Amazon S3 with appropriate lifecycle policies. This can include moving older logs to cheaper storage tiers or implementing automated cleanup procedures for non-critical data. Balance your retention needs with cost optimization by defining clear policies for what data needs to be kept and for how long.

Review the pricing pages for AWS Lambda, Amazon EventBridge, and Amazon SQS to learn about the costs of delivering and processing events and messages.

Conclusion

The increase in maximum payload size from 256 KB to 1 MB enables developers to build more efficient distributed architectures. You can use this enhancement to transport richer context in event and message payloads, reducing the need for complex workarounds that previously added architectural complexity and operational overhead. This added room to transmit rich context means that you can streamline your workflows, improve observability, and reduce architectural complexity whether using choreography or orchestration patterns.

Go to the developer guides for AWS Lambda, Amazon EventBridge, and Amazon SQS, to learn more about how to take advantage of this update.

To learn more about serverless architectures, visit Serverless Land.

Simplify network segmentation for AWS Outposts racks with multiple local gateway routing domains

Brianna Rosentrater — Fri, 16 Jan 2026 18:49:35 +0000

AWS now supports multiple local gateway (LGW) routing domains on AWS Outposts racks to simplify network segmentation. Network segmentation is the practice of splitting a computer network into isolated subnetworks, or network segments. This reduces the attack surface so that if a host on one network segment is compromised, the hosts on the other network segments are not affected. Many customers in regulated industries such as manufacturing, health care and life sciences, banking, and others implement network segmentation as part of their on-premises network security standards to reduce the impact of a breach and help address compliance requirements. Some AWS services also have network requirements that specify certain IP ranges to be used for endpoints, and may or may not support customers bringing their own IP pool (also called CoIP routing, see How to choose between CoIP and Direct VPC routing (DVR) modes on AWS Outposts rack for more information). Customers want the flexibility to use both routing modes (CoIP and DVR) on the same logical Outpost. With this new feature, AWS Outposts racks now support multiple LGW routing domains to meet subnetwork isolation and cloud service network requirements in an on-premises environment. For example, a leading automotive company deploys latency-sensitive manufacturing workloads on Outposts racks in a multi-AZ architecture for resiliency. This feature provides traffic separation between routing domains and enables both customer-owned IP (CoIP) and direct VPC routing (DVR) modes on the same logical Outpost.

In this post you will learn how to use multiple LGW routing domains on Outposts racks and considerations for implementation.

Overview

With the introduction of multiple LGW routing domains on Outposts, you can now create multiple routing domains and associate one or more VLANs with each routing domain. This allows you to integrate your Outposts rack into your existing on-premises network schema. Each LGW routing domain will have a unique LGW Virtual Interface (VIF) Group and an LGW Route Table, enabling logical network traffic isolation. You can have a mix of up to 10 active routing domains with route tables using either DVR or CoIP routing mode, and you can make changes to these routing domains as needed in a self-service fashion allowing for network flexibility as architectures are updated over time. These settings can be found in the AWS Outposts console under the Networking tab in the menu.

The following diagram shows an example of 3 VPCs, each with at least 1 subnet on the Outpost rack, and each VPC corresponds to its own routing domain. Each routing domain can then be associated with one or more VLANs, and one or more VPCs. You can only associate a VPC to one LGW routing domain per Outpost.

Figure 1 – Architecture diagram showing 3 routing domains

Walkthrough

Before creating a LGW routing domain, first you’ll need to create an LGW VIF group and an LGW route table. A local gateway routing domain is the association of a local gateway route table and local gateway VIF group. Each VIF group can be associated with one or more VLANs, but a route table can only be associated with one VIF group.

To create a LGW VIF Group, navigate to the AWS Outposts console, go to LGW virtual interfaces groups, and select Create VIF group. Enter your VIF details which include BGP and VLAN routing information, you must create 4 LGW VIFs per VIF group.

Figure 2 – Creating VIF group for RD1 routing domain

After creating your VIF group, create a LGW route table. You’ll have the option to use Direct VPC Routing (DVR) or Customer-owned IP address pool (CoIP) routing. If CoIP routing is selected, you’ll have the option to enter your CIDR before creating. A LGW route table’s routing mode cannot be changed after creating. However, you can disassociate a LGW route table from a VIF group and attach a new route table if you need to change the routing mode of a VIF group.

Figure 3 – Creating LGW route table for RD1 routing domain

After you’ve created your LGW route table and VIF group, you can proceed to the final step which is to create your LGW routing domain where you will associate the LGW route table and VIF group.

Figure 4 – Creating LGW routing domain for RD1

You can view and create up to 10 active routing domains through the AWS Outposts console under the Networking tab.

Figure 5 – Local Gateway (LGW) routing domains

Considerations

Multiple LGW routing domains feature is only available on second-generation Outposts racks.
Avoid overlapping IP addresses across subnetworks and local routing domains as those can create IP routing conflicts.
A VIF group can only be associated to one LGW route table/routing domain at a time. A routing domain is the association of a VIF group and LGW route table.
LGW routing domain will allow for logical local network traffic isolation, however all traffic will still travel across your local gateway Link Aggregation Control Protocol (LACP) Link Aggregation Group (LAG) to uplink into your on-premises network.
Additional network isolation can be achieved through Virtual Routing and Forwarding (VRF) on Cisco platforms or Routing Instances on Juniper equipment, providing logical separation of routing tables and enabling secure multi-tenancy within the same physical infrastructure.
You can only associate a VPC to one LGW routing domain per Outpost. You can self-serve to change VPC association as needed. Multiple on-premises VLANs can be connected to a single routing domain.

Conclusion

This post demonstrated how to configure multiple local routing domains on Outposts racks to integrate into your on-premises network. For more information see LGW routing domains section in the AWS Outposts user guide. Reach out to your AWS account team to learn more about Outposts racks network configuration options.

In addition to multiple LGW routing domains, we have also announced several updates to Outposts in the past week to help you meet digital sovereignty and local data processing needs. To learn more, read the following announcements:

To discuss Outposts with an expert on any of these topics, submit this form.

Optimizing storage performance for Amazon EKS on AWS Outposts

Arun Kumar — Tue, 13 Jan 2026 18:57:12 +0000

Amazon Elastic Kubernetes Service (Amazon EKS) on AWS Outposts brings the power of managed Kubernetes to your on-premises infrastructure. Use Amazon EKS on Outposts rack to create hybrid cloud deployments that maintain consistent AWS experiences across environments. As organizations increasingly adopt edge computing and hybrid architectures, storage optimization and performance tuning become critical for successful workload deployment.

Outposts extend AWS infrastructure, services, APIs, and tools to virtually any datacenter, co-location space, or on-premises facility. In this blog post you will learn about your storage options and their performance characteristics which is essential for building resilient, high-performing applications using Amazon EKS on Outposts.

Amazon EKS on Outposts deployment options

The following two sections outline the differences between Amazon EKS extended and local cluster deployment options available on Outposts.

Amazon EKS extended cluster architecture

Amazon EKS extended clusters on Outposts provide a powerful solution for organizations seeking to use the benefits of Kubernetes while maintaining certain workloads on-premises, as shown in the following figure. This hybrid architecture allows businesses to extend their EKS clusters from the AWS Cloud to their own data centers or edge locations using Outposts. The Kubernetes control plane remains in the AWS Region, providing centralized management and benefiting from the AWS infrastructure in the cloud and on the Outpost.

Outposts is designed to be a connected service, and needs reliable network connectivity to the AWS Region using the Outposts service link.

Figure 1 : Extended cluster

Amazon EKS local cluster architecture

Amazon EKS local clusters deploy the Kubernetes control plane on your Outpost, as shown in the following figure. This provides greater network resilience against outages as cluster operations run entirely on the Outposts and reduces the dependency on network connectivity to the AWS Region. Having the Kubernetes control plane hosted on your Outpost also reduces latency for cluster operations.

Figure 2: Local cluster

Storage options for Amazon EKS extended clusters on Outposts

Persistent Volumes (PV) and Persistent Volume Claims (PVC) serve as a critical abstraction layer in Kubernetes, separating the storage consumption details from storage provisioning, and allowing administrators to manage storage resources independently from how applications consume them. PVs and PVCs make sure of data persistence across pod restarts and rescheduling events, making them essential for applications that need to maintain state, such as databases, file storage systems, and other data-intensive workloads. The abstraction provided by PV and PVC enables platform-agnostic storage management, where applications can request storage through PVCs without needing to know the underlying storage implementation details. PVs and PVCs support dynamic provisioning through Storage Classes, allowing for automated storage allocation based on application demands, while also providing features such as access modes, capacity management, and reclaim policies to effectively manage the storage lifecycle in a Kubernetes cluster.

Integrating Amazon EBS with Amazon EKS

Amazon Elastic Block Store (Amazon EBS) provides high-performance block storage that’s ideal for low-latency applications providing consistent performance. When deployed on Outposts racks, EBS volumes are stored on the Outposts hardware, providing significant performance advantages over network-attached storage solutions, as shown in the following figure.

Figure 3 : Integrating Amazon EBS with Amazon EKS on Outposts

Benefits and use cases

Storage: EBS volumes on Outposts racks provide data access without dependency on external connectivity.
Performance: Local storage delivers consistent latency and high IOPS/throughput.
Cost: On-premises storage eliminates data transfer costs and reduces bandwidth needs, lowering the total cost of ownership.

Implementation considerations

Consider the following when using EBS on Outposts rack:

EBS volumes on Outposts are tied to a single rack and the availability zone the Outpost is homed to, needing applications to address single-point-of-failure risks.
Protect data using EBS snapshots in the parent Region and schedule regular backups.
Capacity on Outposts is finite, monitor Outposts storage usage and plan expansions proactively to avoid insufficient capacity errors.

Refer to Dynamic Volume Provisioning to learn more about deploying pod with the EBS volume attached.

Amazon EFS with Amazon EKS

Amazon Elastic File System (Amazon EFS) provides scalable, shared file storage that can be accessed across multiple AWS Availability Zones (AZs) and on-premises environments. Although Amazon EFS with Amazon EKS on Outposts maintains the same setup procedures as standard cloud deployments, there is a critical dependency on the service link connection between your Outposts and the AWS Region. Amazon EFS is not a locally supported service on Outposts, so connectivity to the AWS Region is required to use this service with your Outpost.

Amazon EFS allows multiple pods to concurrently access shared file systems. It is well-suited for applications that need collaborative data access, content management, and distributed processing workloads.

Amazon EFS as a persistent storage solution for Amazon EKS extended cluster instances

Amazon EFS as a PV for your Amazon EKS extended cluster operates through a hybrid architecture where the Amazon EFS file system resides in the Region, but mount points can be created on the worker nodes running on Outposts subnets through the service link as shown in the following figure.

Figure 4 : Amazon EFS as a persistent storage solution for extended clusters

Benefits and use cases

Shared storage capabilities: multiple pods can access a centralized file system, enabling shared data, code, and assets across instances.
Scalability: storage capacity and performance automatically scale with usage, eliminating manual provisioning and upfront planning.
Compliance: Amazon EFS provides full file system features and compatibility for traditional applications, such as locking, permissions, and directory structure.

Challenges and limitations

Consider the following when using Amazon EFS with Outposts:

Network latency: file access involves network traversal to Amazon EFS in the Region, adding more latency and making small or metadata operations potentially slow for latency-sensitive applications.
Throughput: aggregate throughput is restricted by the available bandwidth on the service link between the Outposts and AWS Region. This impacts concurrent access and large file transfers during peak usage.
Dependency on AWS Region connectivity: Amazon EFS needs continuous connectivity to the parent Region. Disruptions may affect file system availability, operations, and disaster recovery processes.
Data Transfer charges: Since EFS is in AWS Parent region and EKS worker nodes and pods are in Outpost additional charges are applicable.

You can refer to Amazon EFS Features and When to Choose Amazon EFS for more detailed insights into its capabilities and use cases.

Deploying pods on extended clusters using Amazon EFS as PV

Refer to Use Elastic File System Storage with Amazon EFS for deployment guidance. Note, Create Amazon EFS mount targets in subnets that are in the same Availability Zone (AZ) as the Outposts subnets.

Amazon S3 with Amazon EKS extended cluster

Amazon Simple Storage Service (Amazon S3) on Outposts delivers local object storage on your Outposts, allowing applications to use Amazon S3 APIs for storing and retrieving data while keeping it onsite. It is ideal for workloads that need Amazon S3 compatibility, low latency access to object data, and local data residency.

You should use Amazon S3 access point Amazon Resource Names (ARNs) and not bucket ARNs for proper integration with Amazon EKS workloads.

Learn more about Amazon S3 on Outposts.

Figure 5 : Amazon S3 with Amazon EKS extended cluster on Outposts

Benefits and use cases

Data archiving and compliance: Enables cost-effective, locally retained storage for logs, audit trails, regulatory compliance, backups, and sensitive healthcare data with strict residency requirements.
Content distribution and media: Provides ultra-low latency local storage for serving static content, media streaming, digital asset management, and gaming asset delivery.
Data lake and analytics: Supports local data processing for analytics, ETL, machine learning (ML), real-time Internet of Things (IoT) data handling, and business intelligence with reduced latency and transfer costs.
Application integration: Seamlessly integrates with Amazon S3 compatible apps for backup, synchronization, microservices storage, API-driven workflows, and container image management on-premises.

Refer to How is Amazon S3 on Outposts different from Amazon S3 and the Amazon S3 on Outposts documentation to learn more.

Deploying pods on extended clusters using Amazon S3 as PV

Step 1: Create Amazon S3 on Outposts bucket
Step 2: Create Amazon S3 Access Point (necessary for Amazon EKS integration)
Step 3: Configure IAM roles and policies
Step 4: Install Amazon S3 CSI driver
Step 5: Deploying your pod with Amazon S3 volume attached
Step 6: Complete Amazon S3 configuration with Kubernetes

Refer to the documentation Static Provisioning on Outposts bucket for more details on Step 5.

Best practices for optimizing performance

Optimizing performance starts with selecting the right storage type for your workload: Amazon EBS for low-latency, high-throughput block storage; Amazon EFS for shared POSIX-compliant file systems; and Amazon S3 for scalable object storage with API compatibility. Ensure proper volume sizing, monitor usage proactively, and configure CPU and memory requests accurately to balance performance and efficiency—auto scaling and QoS classes can further optimize resource management. Improve data locality by using local storage, apply caching with intelligent eviction, and design for efficient, asynchronous, and compressed data access patterns.

Monitoring and observability

Monitoring key performance metrics is essential to maintain storage efficiency and application reliability. For Amazon EBS, track IOPS, throughput, latency, burst balance, queue depth, and snapshot performance to avoid degradation—see the Amazon CloudWatch metrics for Amazon EBS for the full list. For Amazon EFS, monitor total I/O, throughput, client connections, metadata operations, burst credits, and Regional data transfers to support effective capacity planning—refer to CloudWatch metrics for Amazon EFS. For Amazon S3, observe request and error rates, data transfer, storage usage, latency, multipart upload efficiency, and access patterns to optimize performance and cost—see Metrics and dimensions.

Security considerations

Strong security practices are critical for Amazon EKS on Outposts. Use AWS Key Management Service (AWS KMS) for Amazon EBS encryption, encrypt Amazon EFS data at rest and in transit, and enable server- or client-side encryption for Amazon S3. Enforce TLS for all data transfers and apply key rotation with compliance controls. Implement least privilege IAM policies, scoped roles, and Kubernetes Role-Based Access Control (RBAC) for granular pod access. Secure traffic with security groups and NACLs, and maintain audit logs for all storage operations.

Cost optimization strategies

Manage storage costs by right-sizing volumes, automating lifecycle policies, selecting appropriate storage classes, monitoring data transfer, and using de-duplication and compression where applicable. Lower operational expenses through automated backups, infrastructure as code (IaC), monitoring automation, leveraging managed services, applying cost allocation tags, and conducting regular usage reviews.

Conclusion

Amazon EKS on Outposts empowers organizations to build hybrid applications with storage options that align to performance, compliance, and data residency needs. By selecting the right storage solution for each workload and leveraging Outposts’ local infrastructure, you can reduce latency, minimize network dependencies, and maintain consistency across environments. As Outposts capabilities continue to evolve, they offer a strong foundation for modern, resilient, and cost-efficient hybrid cloud architectures.

Reach out to your AWS account team, or fill out this form to learn more about running containarized applications on Outposts.