A Bedrock spend spike and an unauthorized InvokeModel call are the same event. One shows up in AWS Cost Anomaly Detection and routes to your FinOps team via Amazon Simple Notification Service (SNS). The other shows up in CloudTrail and, if you’ve written the detection logic, routes to your Security Operations Center (SOC). In most organizations I work with, these two signals never meet. The fastest indicator that someone is abusing an AI service in your AWS org is sitting in a billing dashboard your security team doesn’t watch.

Two Alerts, No Correlation

AWS Cost Anomaly Detection uses machine learning to flag spend deviations by service, account, or cost allocation tag. When it detects something, it sends a notification through SNS or email to whoever owns the cost monitor. That’s typically a FinOps engineer or a finance team lead.

Separately, CloudTrail logs every bedrock:InvokeModel, bedrock:Converse, and bedrock:InvokeModelWithResponseStream call as a management event. GuardDuty can flag suspicious patterns like guardrail removal or anomalous access from unfamiliar principals. If you’ve enabled model invocation logging (which requires explicit opt-in and routes to CloudWatch Logs or S3), you can also capture the request and response payloads.

These are two views of the same activity. One tells you what it cost. The other tells you who did it, from where, and with what permissions. There is no native AWS integration that cross-correlates a cost anomaly alert with the CloudTrail events that generated the spend. Neither team sees both views by default.

FinOps knows spend spiked but can’t assess whether the access was authorized. Security has the access data but didn’t know to look because nobody flagged the cost event to them. I wrote about this organizational dynamic in the first piece in this series. This article is about what to do about it at the detection layer.

 

The CloudTrail Gaps Worth Knowing

CloudTrail gives you the basics for Bedrock: the principal Amazon Resource Name (ARN), the model ID, the source IP, the timestamp. That’s enough to start an investigation. But there are real gaps that affect detection quality.

In mid-2024, Sysdig’s Threat Research Team found that failed Bedrock API calls logged in the same format as successful ones, without distinct error codes in the CloudTrail record. AWS resolved that specific issue by August 2024 after Sysdig’s disclosure, but the episode is instructive: security teams should validate CloudTrail log fidelity for newer AI services rather than assuming error codes behave as expected. Splunk has published specific detections for Bedrock access-denied events and for DeleteModelInvocationLoggingConfiguration, which is an anti-forensics indicator worth monitoring. Someone disabling model invocation logging should always generate an alert.

Model invocation logging itself is not on by default. Without it, you know a model was invoked but not what data was sent to it. For a security team trying to assess data exfiltration risk through a foundation model, that’s a significant blind spot.

Most SOCs I’ve assessed have CloudTrail feeding their Security Information and Event Management (SIEM) platform. Very few have written detection rules specifically for Bedrock API patterns. The rules exist in published form, but adoption is low because Bedrock wasn’t on the threat model when the detection engineering backlog was last prioritized.

 

Cost Anomaly Detection Is Fast but Shallow

Here’s what FinOps tooling does well in this context: it catches spend deviations relatively fast. AWS Cost Anomaly Detection runs approximately three times per day against cost data that can lag up to 24 hours, so detection typically occurs within a day of the anomalous spend. For AI services where a single compromised credential can generate thousands of dollars in model invocations in a short window, that’s still potentially faster than a CloudTrail-based detection in an environment where nobody has written rules for Bedrock API patterns.

Earlier this year, Sysdig’s Threat Research Team documented an AWS breach where attackers went from an exposed credential in a public S3 bucket to full administrative control in under ten minutes. The privilege escalation phase, credential theft to successful

Lambda execution, took eight minutes. The compromised IAM user had read/write permissions on Lambda and restricted permissions on Bedrock. In a scenario like that, a cost anomaly alert firing on unexpected Bedrock spend could be an early indicator of compromise, particularly in environments where the SOC hasn’t written Bedrock-specific detection rules.

But Cost Anomaly Detection can tell you that Bedrock spend increased 400% in a specific account. It can’t tell you whether the IAM role generating that spend should exist. It can’t tell you the role was attached to a Lambda function reachable from outside the VPC. It can’t tell you that model invocations included customer data in the prompt. It’s a financial signal. By itself, it triggers a cost investigation. Paired with CloudTrail context, it triggers a security investigation.

 

Building the Bridge

The integration doesn’t require a new product. It requires an EventBridge rule and a routing decision.

When AWS Cost Anomaly Detection flags a spend deviation on AI services (Bedrock, SageMaker, or whatever your org has classified as AI-adjacent), that alert should route to two destinations: the FinOps team’s existing workflow and a security triage queue. Not as an escalation. As a standing enrichment. The security team’s first step on receiving that alert is straightforward: pull the CloudTrail events for the flagged service and account over the anomaly window, identify the principals involved, assess whether the access was expected.

Going the other direction, detection rules for InvokeModel volume anomalies in CloudTrail should include cost context. If a principal is generating model invocations at a rate that would produce a meaningful cost event, that should be visible in the detection output. Correlating the two signals turns an ambiguous CloudTrail alert into one with financial impact attached, which changes how it gets prioritized.

At the account level, this means AI workloads need dual-owner alerting from provisioning. When an account is vended for AI experimentation, both a cost monitor (Cost Anomaly Detection, scoped to the account) and a security detection (CloudTrail rules for model invocation APIs, scoped to the account) should be provisioned at the same time. Bolting on one after the other creates the gap this entire series is about.

 

What Changes Operationally

When these two signal streams connect, triage changes. A FinOps engineer who flags a Bedrock cost spike doesn’t just tag it to a cost center and close the ticket. They route it for an access review, because that’s the standing workflow. A security analyst investigating anomalous InvokeModel calls doesn’t start from zero trying to assess blast radius. They already have the cost data showing financial exposure.

The organizations getting this right aren’t running exotic tooling. They’ve made a routing decision: AI service alerts go to both teams, from the moment the account is provisioned. If your cost anomaly alerts and your CloudTrail detections for AI services currently land in different queues with no shared context, that’s the gap to close first.

Here’s a scenario that’s becoming more common than most teams want to admit.

A cost anomaly alert fires late on a Tuesday. AWS Bedrock spend spiked 340% in a single account over 72 hours. The FinOps engineer traces it to a Lambda function attached to an IAM role with broad model invocation permissions. Nobody on the team recognizes the role. They escalate to Security. Security asks who provisioned it. Nobody knows. The account was inherited eight months ago during an acquisition and never fully inventoried.

That’s a FinOps problem, a security incident, and an AI governance failure all at once. Most organizations aren’t structured to treat it as all three simultaneously, and that’s where the risk compounds.

 

The Org Chart Didn’t Anticipate AI

The separation of FinOps and Security made sense when the infrastructure was predictable. FinOps owned cost visibility, tagging hygiene, and rightsizing. Security owned IAM, detective controls, and compliance posture. For EC2, RDS, and S3 workloads, these lanes rarely collided in meaningful ways.

AI services broke that assumption quietly.

An over-permissioned IAM role attached to a Bedrock endpoint is both a security finding and an uncapped cost exposure. An untagged SageMaker experiment is a workload running outside your threat model that nobody is tracking spend on. A foundation model accessible to an externally-invocable function is a data egress vector and a budget liability that nobody explicitly approved.

Every AI service misconfiguration has two blast radii. One is measured in dollars. The other is measured in data. Most organizations have a team watching each, but nobody watching both at the same time.

 

Shadow AI Is Already Inside Your AWS Org

This isn’t a future problem. It’s already happening in environments I’ve worked in across PE-backed and enterprise AWS organizations.

Developers are experimenting with Bedrock, SageMaker, and third-party model APIs directly in non-production accounts. Those accounts frequently carry permissive Service Control Policies (SCPs) inherited from early-stage growth or a pre-acquisition baseline that nobody revisited. AI services were never included in the tagging strategy because nobody anticipated needing to. FinOps dashboards aren’t surfacing model invocations with the same visibility as compute spend. Security teams aren’t writing detection logic for bedrock:InvokeModel the way they’d write rules for anomalous S3 access.

The assumptions quietly baked into this: AI spend is too small to govern yet. Bedrock is a managed service so the security surface is Amazon’s problem. Developers aren’t using foundation models in production because nobody submitted a budget request.

All three of those assumptions are wrong in most environments I’ve assessed.

 

Where Convergence Actually Matters

A shared accountability model, applied to a specific class of infrastructure that neither team currently owns cleanly. Not a new tool. Not a reorganization.

Visibility has to be symmetric. Cost anomaly alerts on AI services should automatically trigger a security review threshold alongside the budget notification. If FinOps sees a spike in Bedrock invocations, Security should be in that conversation by default, as a standing workflow rather than an escalation. Tag enforcement for AI services needs joint authorship. Right now it’s typically a FinOps initiative that Security never signed off on.

IAM is a cost control. Least-privilege access to AI services directly limits cost exposure. An IAM principal that can only invoke one specific model in one specific account can’t generate runaway spend across your org. Unused model access permissions are simultaneously a security finding and a FinOps finding. Treating them as only one or the other means half the organization never sees them.

The account boundary is your best governance primitive. AI workloads should live in dedicated accounts with an explicit cost owner and an explicit security owner, two people who have both agreed they’re responsible. Account vending for AI experimentation needs guardrails at provisioning time: budget thresholds, SCPs scoped to approved model access, egress controls active from day one. Retrofitting governance onto an account after spend has already appeared is governance that consistently arrives too late.

 

What Leadership Usually Gets Wrong

“We’ll govern AI workloads once they reach production.”

By that point the IAM roles are already months old, the tagging debt is structural, and the cost baseline was never established cleanly. The window to set governance boundaries is at provisioning, not at promotion.

“Our FinOps tool will catch runaway AI spend.”

FinOps tooling tells you that spend happened. It doesn’t tell you whether the principal that generated it should have had access at all. Cost anomaly detection and access governance are different signals. They need different owners. The problem is that right now, in most organizations, neither team is watching both signals simultaneously.

 

The Real Ask

The enterprises handling this well aren’t doing anything exotic. They’ve extended a principle that works everywhere else in mature cloud governance: the account boundary is the unit of ownership, and ownership means someone is accountable for both what it costs and what it can access.

AI just makes the cost of skipping that step visible faster than anything that came before it.

If your FinOps and Security teams aren’t in the same room when AI services get provisioned, they probably need to be. Not because of a compliance mandate. Because the blast radius when they aren’t is bigger than most organizations have priced in.

ARC-AMPE’s Second Deadline

The Centers for Medicare & Medicaid Services (CMS) published ARC-AMPE (Acceptable Risk Controls for ACA, Medicaid, and Partner Entities) a year ago as a replacement for MARS-E, which is now deprecated. Administering Entities (AEs), state-based health insurance marketplaces and Medicaid agencies had until March 4, yet industry assessors were flagging widespread readiness gaps leading up to the deadline. Direct Enrollment Entities (DEEs), insurers and web brokers, have until the end of June to comply with 308 controls. The NIST revision jump and restructuring of control families broke the old mapping. If your organization is treating ARC-AMPE’s second deadline as a documentation migration, you’re underestimating what changed under the hood.

What Changed and Why It Matters for Cloud Environments

ARC-AMPE replaced MARS-E v2.2. The AE baseline is 402 controls. The DEE baseline is 308. Both are derived from NIST SP 800-53 Revision 5. Four changes have real architectural implications for organizations running Affordable Care Act (ACA) or Medicaid workloads in AWS, GCP, or hybrid environments.

Data residency is now explicit

All data processing and storage must happen within U.S. legal jurisdiction. For cloud environments, this means region-locking, verifying that managed services don’t replicate data outside U.S. regions, and documenting data residency controls. SA-9(8), Processing and Storage Location, is explicitly in the ARC-AMPE baseline. Organizations using multi-region or global cloud configurations need to prove containment.

Cloud-specific controls are gone

Previously, MARS-E maintained separate control expectations for cloud vs. on-premises environments. ARC-AMPE applies uniformly. If you were relying on a lighter control set for your cloud workloads, that distinction no longer exists.

The PT control family is entirely new

Ten Personally Identifiable Information Processing and Transparency (PT) controls covering consent, privacy notices, PII processing purposes, Social Security Number handling, and revocation rights. This family didn’t exist in MARS-E because it didn’t exist in NIST 800-53 Rev 4. These are enforceable privacy controls that need to be implemented at the infrastructure and application layer, not just documented in a policy binder.

Supply Chain Risk Management is now required

SR controls mandate vendor oversight for cloud service providers, managed services, and SaaS tooling used in ACA and Medicaid environments. Documented risk assessments for every vendor in the chain, with contractual security requirements and ongoing monitoring.

The Reciprocity Advantage Most Organizations Are Missing

ARC-AMPE is 402 controls from NIST 800-53 Rev 5. FedRAMP High is 410. The overlap is substantial, and most organizations aren’t mapping it.

We ran the analysis. An organization with an existing FedRAMP Moderate authorization already satisfies 292 of the 402 ARC-AMPE AE controls. That’s 73% coverage before doing any ARC-AMPE-specific work. FedRAMP High covers 302 of 402, or 75%. The numbers for GovRAMP are nearly identical: GovRAMP Moderate maps to 292 ARC-AMPE controls, GovRAMP High maps to 302.

The remaining gap from FedRAMP High is 100 controls, and it concentrates in specific areas. Twenty-eight are Program Management (PM) family controls: organizational governance like risk management strategy, insider threat programs, security workforce planning, and system inventory. These are documentation-heavy but not technically complex. Ten are the entirely new PT family, none of which appear in any FedRAMP baseline. Roughly a dozen more are PII-related controls scattered across other families (SI, SC, AU, SA, CM, AT), covering data minimization, de-identification, disposal, and quality operations. Seven are SA-8 security engineering principle enhancements. Six are maintenance controls.

The practical implication: an organization with FedRAMP Moderate posture doesn’t have a 402-control problem. It has a 110-control problem, concentrated in privacy governance and program management documentation. That’s a fundamentally different compliance project than starting from zero, and it should be scoped accordingly.

State Medicaid agencies and health insurance marketplaces that have done any 800-53-based compliance work, whether for FedRAMP, GovRAMP, or other federal requirements, are sitting on evidence and control implementations that directly satisfy ARC-AMPE. The organizations that treat ARC-AMPE as a standalone assessment are duplicating efforts they’ve already completed.

How Organizations Can Prepare Now

Regardless of where you are in the timeline: run a reciprocity analysis against any existing 800-53 compliance work before scoping net-new effort. Map your data residency controls early, since region-locked cloud environments close multiple ARC-AMPE requirements at once. Build your System Security and Privacy Plan (SSPP) in the new ARC-AMPE Excel template from the start. CMS replaced the old Word format, and converting later breaks every documentation workflow.

Risks of Missing ARC-AMPE’s Second Deadline

File a realistic Plan of Action and Milestones (POA&M) with CMS. Prioritize the PT and SR control families first, since those have no MARS-E predecessor to map from and represent the largest net-new implementation effort. If you have existing 800-53 compliance work from FedRAMP, GovRAMP, or other federal frameworks, you likely already satisfy 70-75% of the baseline. Don’t start from scratch.

DEEs facing the June deadline

Three months is tight for 308 controls. Start with a gap assessment against the DEE baseline specifically. Identify which controls you already meet through existing HIPAA or SOC 2 work.

Conclusion

RKON helps organizations running ACA and Medicaid workloads in AWS and GCP close the ARC-AMPE gap. We maintain cross-framework compliance tooling that maps existing control implementations to ARC-AMPE requirements, so you’re building on evidence you already have rather than starting over. If your team is facing the June deadline or remediating after March, reach out.

The most frequently cited downside of using Kubernetes is almost always the operational overhead. Node management, upgrade cycles, scaling configurations, and the endless YAML manifests quickly begin to chip away at a team’s ability to operate efficiently. Google Kubernetes Engine (GKE) has been working on solving some of these operational overhead problems for more than a decade and has matured to the point where we have real solutions for most of those complaints. However, it’s important to note, that a “managed” or even “fully managed” platform does not mean “zero decisions.” This post will help frame how you should think about GKE today, especially if you’re supporting workloads with requirements that fall under FedRAMP or other highly regulated industries.

Autopilot Mode in GKE

GKE offers two primary modes of operation, Autopilot and Standard mode. Google now recommends Autopilot mode as the default. In which, Google will handle the node provisioning, scaling, upgrades, and security hardening of the nodes. Your responsibility is to define what your workloads will need via your pod specs. GKE figures out the infrastructure. Billing on Autopilot is done primarily on a per-pod basis by charging for the vCPU, memory, and ephemeral storage resources requested by the running pods rather than billing for the underlying node Compute Engine instance based on the machine type. This helps to avoid paying for idle resources. However, if your spec selects specific hardware then those pods will use a node-based billing model. Autopilot clusters come preconfigured with Workload Identity, shielded nodes, and secure boot features enabled by default.

Standard Mode for Greater Infrastructure Control

Standard mode abstracts away less of the cluster management responsibilities from the user. It will be useful when you need more granular control over node pools, specific images, custom instance types, or the ability to run workloads in the kube-system namespace. You manage the nodes and pay for the underlying Compute Engine VMs regardless of the utilization.

Hybrid Workloads with Autopilot ComputeClasses

There is also now a new way of running workloads in GKE that is worth noting. A new “hybrid” option allows you to run Autopilot workloads inside of a Standard cluster. This is achieved by offering ComputeClasses. You can specify an Autopilot ComputeClass for the workload to run on. This means that you can run some workloads fully managed by Google while keeping others under manual control. All within the same cluster. This offers a practical solution for teams with mixed requirements between stateless microservices and those workloads that require specific hardware.

FedRAMP Compliance Support in GKE

For teams that are supporting FedRAMP workloads, GKE has a strong compliance posture. GKE is included in GCP’s FedRAMP High provisional Authority to Operate (ATO). This is the highest bar for FedRAMP compliance and means that GKE can be used within the authorization boundary of lower level FedRAMP ATO designations as well.

Implementing FedRAMP Guardrails with Assured Workloads

The setup for FedRAMP workloads involves using Assured Workloads in GCP. This enforces guardrails within a GCP project such as US only data residency, restricted support access to the appropriate personnel, and applies FIPS 140-2 compliant encryption by default. You will not need to manage any OS-level encryption to satisfy the FedRAMP requirements for data protection at rest and in transit since it is baked into the platform by using Assured Workloads.

The Maturity of GKE for Regulated Cloud Workloads

GKE has reached a level of maturity where the “build vs buy” question for managed Kubernetes clusters becomes clearer for most teams. The hybrid approach helps reduce operational burden without sacrificing control where you need it. You no longer have to pick one mode for the entire cluster. These operational improvements combined with Assured Workloads in GCP make it easier to run regulated workloads in the cloud by inheriting a significant portion of the control baseline from the platform itself. Leading to less documentation and fewer audit findings.

Your HIPAA Deadline Is Coming Faster Than You Think. A Story Every Hospital Should Hear.

“Tell me this is overblown,” Jane, the Chair, said as she nervously fidgeted with her glasses.

A slide showing the Federal Register notice lit up the boardroom screen, and it looked a lot like a subpoena. Only three days had passed since the first email landed in the board’s inboxes. Now, no one at Little Valley Medical Center could pretend this was just another regulatory update. The subject line alone had already made a few board physicians check their blood pressure:

“Final HIPAA Security Rule Published. Effective in 60 Days. Full Compliance Required in 240. HIPAA Penalties Will Apply.”

“Nope, not a joke,” Daniel, the CEO, leaned back in his chair and exhaled loudly, clearly working to control his concern and embarrassment. “We let ourselves down here. We knew this was coming, we did not get in front of it, and our security partner did not really push us on it either.”

About five weeks earlier, the CEO and team learned about the upcoming deadline during an emergency briefing with a security partner. Since then, they had been scrambling to catch up.

The room fell quiet. Their three‑week‑tenured CISO, Alexis Ramirez, stood, and every eye turned to her.

“Short answer, we are not ready,” she said. “But we could be, if we move fast. Multifactor authentication everywhere. Encryption. Asset inventories. Pen tests. Incident response. Vendor crackdowns. All of it. We also need to do all this with no impact on providing excellent care to our patients”

From the NPRM
“The proposal would remove the distinction between ‘required’ and ‘addressable’ .. and make all implementation specifications required, subject only to specific, limited exceptions.”

Chapter 1: A Slow‑Motion Crash

Before Alexis joined, Little Valley did what many mid-size hospitals do: just enough to pass audits on a good day, but not enough for anyone who really understood the risks to feel comfortable.

IT begged for investments. Clinical leaders fought to avoid implementing anything they perceived as slowing them down. In fairness, they were right about one thing: fast and effective patient care was their number one priority.

Over time, this compromise led to a mix of half-finished technology projects and good intentions that never fully came together.

In a corner of the ER admitting area, tucked under a desk, was an old server with a Post‑it so worn it had been picked up and taped back on several times. It read “Important.” Nobody was sure why.

An account used by certain technicians was shared. The username “xray_technician” was used by over a dozen people and had never had its password changed. “We would lock ourselves out,” they explained.

Try as she might, Alexis could not find a risk assessment performed within the last three years. A new EHR system was installed two years ago. A risk assessment was discussed at the time, but never carried out.

There were policies, but hardly anyone knew where to find them. The SharePoint site was so neglected it might as well have been covered in dust.

From the NPRM
“Regulated entities would be required to maintain written documentation of all Security Rule policies, procedures, plans, and analyses..”

Little Valley had avoided major incidents mostly through luck. A ransomware campaign had swept through neighboring hospitals the previous year, but Little Valley had somehow escaped. Leadership took this as proof that they were “too small to bother with” and that their existing controls were sufficient.

Alexis did not believe in that kind of luck, and she knew better about “sufficiency.”

During her first walk through the server room, she noticed overloaded racks, mismatched labels, and a switch with a Post-it note that read, “Do not touch! Breaks lab!” It seemed like IT used Post-it notes as their main way to communicate.

“Do we know where all of our systems that hold patient data are?” she asked the IT manager, Priya.

Priya hesitated. “We know where most of them are.”

“That is not going to be good enough anymore,” Alexis said.

Chapter 2: The Outsider

Six months earlier, a resident had plugged a personal laptop into the network to print a research paper. Within a few minutes, suspicious outbound traffic set off alerts at Little Valley’s outsourced SOC. A misconfigured system allowed the device onto a sensitive network segment. The MSSP stepped in, contained it, and found no privacy impacts, but not before a tense weekend of log review and emergency calls.

The hospital was lucky. The incident easily could have become a headline moment.

The board decided they needed “a real cybersecurity leader.” That was Alexis. She had faced more upset executives than she could count, carried the scars of prior breaches, and had a valuable habit of listening longer than she spoke.

Even though she was new, Alexis already understood the politics. It was nothing out of the ordinary.

• Finance saw security as an expense,
• Clinicians saw security as friction,
• IT saw security as another demand on their time, with no extra staff.

And then there was Dr. Noah Pike.

Chapter 3: Noah

Every good story needs an antagonist. At Little Valley, this was Noah Pike.

In security terms, Dr. Pike was a walking threat.

Noah was not a cartoon villain with nefarious objectives. He was worse. He was a smart, charismatic director of a lucrative specialty clinic whose incentives were based on throughput and revenue, not security and compliance. “We have anti‑malware,” he told Alexis. “We are covered.”

From the NPRM
“The proposal would require encryption of ePHI at rest and in transit, with limited exceptions, and would require the use of multi‑factor authentication, with limited exceptions, for access to specified electronic information systems…”

Worse still, he had power. He ran one of the service lines that kept the hospital profitable. Patients loved him, and the board valued him.

He pushed back on access controls, insisted his staff needed exceptions to every policy, and had quietly signed his department up for a cloud‑based scheduling platform without involving IT.

When Alexis’ team discovered a stream of patient data flowing into that platform without a Business Associate Agreement in place, Noah brushed it off.

“We are helping patients,” he said. “The vendor says they are secure. Stop the bureaucracy. You are slowing me down, you hurt care.” Then came the knockout punch. “Do you want that on your conscience?”

It was a clever argument, and it had worked for years.

But the new rules changed that.

Chapter 4: The Clock Starts ….

Author: Gerard Onorato, Chief Information Security Officer at RKON
Gerard Onorato I LinkedIn

Imagine a world where adding or removing a user’s SSH access was as simple as a single button in AWS.

I’m not sure about you, but the problem of provisioning and de-provisioning SSH users on a fleet of nodes is a giant pain in the ass. The conventional wisdom is have a file called ~/.ssh/authorized_keys in the home folder of the user on the server you want to log into. That file will contain the SSH public key of authorized users. When a new engineer joins, you’ll have to add her to the system including adding her SSH public key.

If your company is old-school and doesn’t use a configuration management system, you know this pain. If you want to provision or deprovision a user, you’ll have to execute an SSH script and execute it on every node, then validate that that person is actually out of the system. Typically what you’ll see with a configuration management solution like Chef, as an example, is one where each user has their own databag with their public key in it, or there’s a flat file of users and public keys. The problem with this approach is that you have a explicitly do a Chef deploy to be able to add or remove users. From a security perspective this is a really bad thing, because in the event of someone’s laptop and SSH key being compromised, every second counts.

I’d like to propose a better way of handling SSH keys with any generic public key store. The key to this is the AuthorizedKeysCommand in the sshd_config. From the main page: