The missing map

What’s missing is a shared reference model. Not another tool. A map. Which engineering activities can AI responsibly support? What does quality mean for those outputs? What changes when part of the workflow becomes probabilistic? And what guardrails keep integration safe, observable, and accountable? Without that map, it’s easy to drown in novelty, and easy to confuse widespread experimentation with reliable integration. Teams with the least time, budget, and local support pay the highest price, and the gap compounds.

That gap is now visible at the organizational level. More organizations are trying to turn AI into business value, and the difference between hype and integration is showing up in practice. It’s easy to ship impressive demos. It’s much harder to make AI-assisted work reliable under real-world constraints: measurable quality, controllable failure modes, clear data boundaries, operational ownership, and predictable cost and latency. This is where engineering discipline matters most. AI does not remove the need for it; it amplifies the cost of missing it. The question is how we move from scattered experimentation to integrated practice without burning cycles on tool churn. To do that at scale, we need shared scaffolding: a public model and shared language for what “good” looks like in AI-native engineering.

We have seen why this kind of shared scaffolding matters before. In the early internet era, promise and noise moved faster than standards and shared practice. What made the internet durable was not a single vendor or methodology but a cultural infrastructure: open knowledge sharing, global collaboration, and shared language that made practices comparable and teachable. AI-native engineering needs the same kind of cultural infrastructure, because integration only scales when the industry can coordinate on what “good” means. AI does not remove the need for careful engineering. On the contrary, it punishes the absence of it.

A public scaffold for AI-native engineering

In the second half of 2025, I began to notice growing unease among engineers I worked with and friends in IT. There was a clear sense that AI would change our work in profound ways, but far less clarity on what that actually meant for a person’s role, skills, and daily practice. There was no shortage of trainings, guides, blogs, or tools, but the more resources appeared, the harder it became to judge what was relevant, what was useful, and where to begin. It felt overwhelming. How do you know which topics truly matter to you when suddenly everything is labeled AI? How do you move from hype to useful integration?

I was feeling much of that same uncertainty myself. I was trying to make sense of the shift too, and for a while I think I was waiting for a clearer structure to emerge from elsewhere. It was only when friends started reaching out to me for help and guidance that I realized I might have something meaningful to contribute. I do not consider myself an AI expert. I am finding my way through these changes just like many other engineers. But over the years, I had become known for my work in IT workforce development, skill and capability frameworks, and engineering excellence and enablement. I know how to help people navigate complexity in a practical and sustainable way, and I enjoy bringing clarity to chaos.

That is what led me to start working on the AI Flower as a hobby project in early October 2025, building on frameworks and methods I already had experience with.

When I began sharing it with friends in IT to gather feedback, I saw how much it resonated. It helped them make sense of the complexity around AI, think more clearly about their own upskilling, and begin shaping AI adoption strategies of their own. That is when I realized this casual experiment held real value, and decided I wanted to publish it so it could help empower other engineers and IT organizations in the same way it had helped my friends.

With the AI Flower, I’m offering a public scaffold for AI-native engineering work: a shared reference model that helps engineers, teams, and organizations adopt and integrate AI sustainably and reliably. It’s meant to steer and organize the conversation around AI-assisted engineering, and to invite targeted feedback on what breaks, what’s missing, and what “good” should mean in real production contexts. It’s not meant to be perfect. It’s meant to be useful, freely available, open to contribution, and shaped by the strongest resource our industry has: collective intelligence.

Open knowledge sharing and collaboration cannot be optional. If AI is becoming part of how we design, build, operate, secure, and govern systems, we need more than tools and enthusiasm. Many of us work on systems people rely on every day. When those systems fail, the impact is real. That’s why we owe it to the people who depend on these systems to do this with care, and why we won’t get there in isolation. We need the industry, globally, to converge on shared standards for dependable practice.

About the AI Flower

The AI Flower maps the core activities that make up engineering work across the main engineering disciplines. For each activity, it defines what good looks like, based on practices that should already feel familiar to engineers. It then helps people explore how AI can support those activities in practice, providing guidance on how to begin using AI in that work, sharing links to useful learning resources, and outlining the main risks, trade-offs, and mitigations.

But the AI landscape is changing quickly. This activity-based approach helps engineers understand how AI can support core engineering tasks, where risks may arise, and how to start building practical experience. But on its own, it isn’t enough as a long-term model for AI adoption.

As AI capabilities evolve, many engineering activities will become more abstracted, more automated, or absorbed into the infrastructure layer. That means engineers will need to do more than learn how to use AI within today’s activities. They will also need to work with emerging approaches such as context engineering and agentic workflows, which are already reshaping what we consider core engineering work. A concept I call the Skill Fossilization Model captures that progression. It shows how both engineering skills and AI-related skills evolve over time, and how some of them become less visible as work moves to a higher level of abstraction. Together, the AI Flower and the Skill Fossilization Model are meant to help engineers stay adaptable as the field continues to shift.

The main purpose of the AI Flower is to help engineers find their way through these rapid changes and grow with them. While I provide content for each section and activity, the real value lies in the framework and structure itself. To become truly valuable, it will need the insight, care, and contribution of engineers across disciplines, perspectives, and regions.

I genuinely believe the AI Flower, as an open and freely available framework, can serve as a scaffold for that work. This is my contribution to a changing industry. But it will only be useful—it will only “bloom”—if the community tests it, challenges it, and improves it over time.

And if any industry can turn open critique and contribution into shared standards at a global scale, it’s ours, isn’t it?

Join me at AI Codecon to learn more

If the AI Flower resonates and you want the full walkthrough, I’ll be presenting it at O’Reilly’s upcoming AI Codecon. (Registration is free and open to all.)

If you’re concerned about how quickly AI engineering patterns are evolving, that concern is valid. We’ve already seen the center of gravity shift from ad hoc prompt work, to context engineering, to increasingly agentic workflows, and there is more coming. A core design goal of the AI Flower is to stay stable across those shifts by focusing on underlying capabilities rather than specific techniques. I’ll go deeper on that stability principle, including the Skill Fossilization model, at AI Codecon as well.

And yes, Gas Town is indeed named after the fuel depot in Mad Max: Furiosa, and even features a managing agent named after the Mayor. But Steve’s Gas Town is anything but dystopic. If anything, it’s joyous. That gives you a deep sense of who Steve is: He goes into the deepest, darkest part of the forest, finds something scary, and then does his best to redeem it.

We covered a lot of ground: the eight levels of coder evolution, the addictive pull of multi-agent workflows, grief and denial in the developer community, the bitter lesson, and why taste may be the last remaining competitive advantage. Here are some of the highlights.

Everyone gets a chief of staff

Steve’s “Eight Levels of Coder Evolution” framework has taken on a life of its own since he published it as part of the Gas Town launch post. The first four levels are about increasingly sophisticated IDE use; levels five through eight are about coding agents. Here is an infographic showing the eight stages that I showed to anchor the start of our conversation. Note that I created this slide with Nano Banana 2. It is not directly from Steve.

The key transition, Steve argues, happens at level five: Your IDE goes away and you never open it again. As Steve described it, once you realize Claude Code can write pieces of your code, you start assembling them like Lego. But while one agent is working, you’re sitting there bored, so you fire up another one. And another. Before long, you’ve got six agents running in parallel, and one of them is always finished and waiting for your attention.

Steve drew an analogy to Amazon VPs who had executive assistant support. Those people were effectively two people. They didn’t have to worry about whether the printer was jammed, so they could spend all their time focused on the real problems. Gas Town, Steve argues, is topologically similar: It’s going to turn everybody into something like an executive with a chief of staff. “We all have a chief of staff now,” he said. “Everybody’s going to be able to spend their time more productively on whatever they want to spend it on instead of figuring out where the printer is jammed.”

On March 26, join Addy Osmani and Tim O’Reilly at AI Codecon: Software Craftsmanship in the Age of AI, where an all-star lineup of experts will go deeper into orchestration, agent coordination, and the new skills developers need to build excellent software that creates value for all participants. Sign up for free here.

The AI vampire

This is Steve Yegge, so he’s not just going to give you the upside. His post on “the AI Vampire” explained how AI-assisted productivity creates an insidious new kind of burnout, and I made sure to ask him about that.

The old version of overwork was your company piling tasks on you until you broke, he told us. The new version isn’t your boss asking you to work extra hours. It’s Claude saying, “Is there anything else you’d like me to do on this project?” And you say yes, yes, yes, because it’s fun, because it’s productive, because the AI is your buddy, not your employer.

But there’s a twist. The AI is solving all the easy problems and leaving you with nothing but hard ones. In our conversation, I said it can feel like your bike ride is all hills now, and Steve immediately connected it to watching Jeff Bezos in meetings at Amazon. People would bring him presentations where they’d already solved every easy problem, so Bezos was just getting the hard stuff, all day long. “Now this happens to you,” Steve said. “Everyone’s Jeff Bezos, everyone’s an entrepreneur. Everyone has a huge army of workers now. And I’m telling you, it’s exhausting.”

Steve told us he naps every day now, sometimes twice a day, feeling drained by the relentless cognitive intensity. These agents don’t just help you work faster; they fundamentally change what kind of work reaches your desk.

On the wrong side of the bitter lesson

We spent a good stretch on Richard Sutton’s “bitter lesson.” Sutton observed that raw computation consistently beats systems built on human-engineered structure. Steve treats it less as a paper and more as a daily operating principle. “Not a day goes by when I don’t think about the bitter lesson as a math formula,” he said, “at least five times a day.”

His practical test is simple: If you’re writing code that tries to make the AI smarter, by adding heuristics, parsers, regular expressions to handle what a model could handle, you’re on the wrong side of the bitter lesson. He watches even his own Gas Town contributors make this mistake, reaching for a little regex hack when they should let a model do the cognition. (Steve does admit that sometimes you do need to provide prebuilt code if it saves tokens.)

Sutton wrote about the bitter lesson in the context of training programs to beat humans at chess and go, but it’s more general than that, even more general than leaning into today’s AI in the way that Steve does. I shared my own first encounter with the bitter lesson, back in 1993 when O’Reilly created GNN, the first commercial web portal. Being publishers, we curated a catalog of the best websites. Then Yahoo! set out to list all of them, restricting curation to putting them into categories. Then Google did it algorithmically, creating a custom curation for every search. We know which approach won. The bitter lesson isn’t just about AI; it’s about a recurring pattern in the history of technology where scale and computation overwhelm hand-tuned solutions.

I still believe we have to bet against the bitter lesson, because if we just give up, there will be no place for humans in the future knowledge economy. But we have to do it knowing that we aren’t going to win in the traditional sense. We aren’t going to outrace AI. We have to learn to ride it.

For anyone in a corporate setting, you will naturally want to fit AI into your current workflows. The bitter lesson says you should instead figure out what the AI can do by itself first, and then build a new workflow around that. I described Steve’s whole approach as looking the bitter lesson in the face and saying, “I’m going to turn AI loose on everything I can, and then figure out where the human fits in the loop.”

Code is a liquid

Steve hit peak Yegge mode when an audience member asked why they should leave their IDE. His response was, as usual, quotable:

If you’re looking at your code, then you’re in a Formula One race and you’ve parked your car and opened the hood and you’re looking at the engine. You’ve slowed time to the point where everyone is racing past you and you’re a frozen statue. Code is a liquid. You spray it through hoses. You don’t freaking look at it….

Look, I get it. This is painful for people. This is super painful. For me to say these things is painful for people to hear them. Because what I’m saying is your job is going to change.…And there’s still a lot of denial out there.

What’s the first phase of grief? The first phase of grief. The whole world is in it right now, Tim. They’re in denial. Right. They are grieving for what is going away. We’re at the end of an era. An age, a golden age, maybe, where we programmers, we’re writing all the code. And it was wonderful for 30, 40, 50 years or whatever. That era is ending and people are grieving because of it. And I feel for them. I’ve got empathy, right. But I’m also losing patience because it’s 2026 and this is an exponential curve, and we don’t have time to sit around and feel pity for ourselves.

He sees that grief everywhere, but he specifically called out Hacker News, which he described as the home of “the new Amish.”

Taste is the moat

Another of the audience questions was about whether corporations with deep pockets have all the leverage and there’s no room left for individuals. Steve’s answer was emphatic: absolutely not. Steve made a passionate argument that creativity outweighs capital in the AI era. He’s certain there will be companies that waste millions of dollars of tokens building software that never sees the light of day, because they had no taste, no good ideas, just brute-force generation without direction. Meanwhile, an entrepreneur with open source local inference models and a good GPU can build something that matters, if they know what people want.

“Everything is going to come down to taste,” Steve said. “Companies don’t have an advantage anymore. As an entrepreneur, I think this is a golden opportunity for people to make huge, huge impact.”

It’s mentors all the way down

Someone from the audience asked Steve about a question that’s on everyone’s mind: If senior developers are becoming PMs and juniors are being replaced by AI, where will new seniors come from? His answer was a classic reframe, and an update of what he wrote in “Revenge of the Junior Developer.”

He made the case that your most junior engineers aren’t who you think they are anymore. They’re your product managers, your SDRs, your finance and sales folks, all of those people throughout your company who are now building things with AI. Your former junior engineers are actually well-trained engineers who make perfect mentors for this new bottom layer. And those juniors get mentored by seniors, and seniors by principals. It’s mentoring all the way down.

Steve pointed out that this connects to something Matt Beane (who was in our audience) has researched on skills acquisition: You don’t learn from someone 40 levels above you; you learn from someone one or two levels ahead. Steve’s suggestion for companies is to organize around this. Find mentors within your organization who are just a step or two ahead of where each person is, and bring everyone along with empathy for what people are going through.

We’re going to build bigger stuff

Another audience member asked about research showing that AI atrophies critical thinking pathways. I couldn’t resist jumping in with one of my favorite historical analogues. Socrates said the same thing about the written word, arguing that it was impairing people’s ability to remember. And he was right, we did lose the ability to recite massive amounts of literature from memory. But we gained more than we lost. Things change.

I also shared a Rilke poem that I love, about Jacob wrestling with the angel: “What we fight with is so small, and when we win, it makes us small. What we want is to be defeated decisively by successively greater beings.” If AI is atrophying your thinking, it’s because you’re not wrestling with hard enough problems. The real opportunity is to be pushed, stretched, and defeated by bigger challenges, and come away stronger from the fight.

Steve agreed: “We’re going to build bigger stuff. That’s what everyone’s worried about. What’s going to happen? And the answer is we’re going to build bigger stuff and it’s going to be fun.”

Watch the full conversation here. Steve Yegge’s Gas Town is at https://github.com/steveyegge/gastown. His blog posts on “The AI Vampire,” the “Revenge of the Junior Developer,” and “Software Survival 3.0” are essential reading for anyone navigating this transition.

What was so special about OpenClaw? Why did this one take off when so many agent projects before it didn’t?

Autonomous AI isn’t new

Where we are today feels similar to April 2023 when AutoGPT hit the scene. It had the same GitHub trajectory with its promise of autonomous AI. Then reality hit. Agents got stuck in loops, hallucinated a lot, and racked up token costs. It didn’t take long for people to walk away.

OpenClaw has one critical advantage: The models have gotten better. Recent LLMs like Claude Opus 4.6 and GPT-5.4 allow models to chain tools together, recover from errors, and plan multistep strategies. Steinberger’s weekend project benefited from timing as much as design.

The architecture is intentionally simple. There are no vector databases and no multi-agent orchestration frameworks. Persistent memory is Markdown files on disk. Let me repeat that: Persistent memory is Markdown files on disk! The agent can read yesterday’s notes and search its own files for additional context. You can view and edit the agent’s files as needed. There’s a useful lesson in that: Not every agent system needs a complex memory strategy. It’s more important that you understand what the agent is doing and that it retains context across runs.

What fascinates me about OpenClaw is that none of the individual pieces are new. Persistent memory across sessions? We’ve been building that for years. Cron jobs to trigger agent actions on a schedule? Decades old infrastructure. Plug-in systems for extensibility? A very standard pattern. Webhooks into WhatsApp and Telegram? There are well-documented APIs for that. What Steinberger did was wire them together at the exact moment the underlying models could execute on multistep plans. The combination created something that felt quite different from anything that had come before.

Why this time feels different

OpenClaw nailed three things that previous agent projects missed: proximity, creativity, and extensibility.

Proximity—it lives where you already are every day. OpenClaw connects to WhatsApp, Slack, Discord, Telegram, and Signal. That single design decision changed its trajectory. The agent becomes an active participant in your workflow. People use it to manage their sales pipeline, automate emails, and kick off code reviews from their phones.

Next, it’s proactive. OpenClaw doesn’t wait for you to ask; it uses cron jobs to run tasks on a set schedule. It can check your email every day at 6am, draft a reply before you wake up, and even send it for you. And it reaches out when anything needs your attention. Agents become part of everyday life when integrated into familiar channels.

And finally, my favorite, it’s open and extensible. OpenClaw’s plug-in system, called “skills,” lets the community build and share modular extensions on ClawHub. There are thousands of skills ready to be plugged into your agent. Agents can even write their own new skills and use them going forward. That extensibility meant more skills, more users, and more attack surfaces, which we’ll get to.

The community ran with it. A social network exclusively for AI Agents, Moltbook, launched in late January and grew to over 1.5 million agent accounts. One agent created a dating profile for its owner on MoltMatch and started screening matches without being asked.

I’ll admit, I got swept up in it, but that’s not surprising; I’ve always been an early adopter of emerging technology. I bought a Mac mini, installed OpenClaw, and connected it to my Jira, AWS, and GitHub accounts. In no time, I had my agent, Jarvis, writing code and submitting PRs, running my daily standups, and deploying my code to AWS using AWS CloudFormation and the AWS CLI.

I spent a lot of time binding the gateway to localhost, auditing every skill, and restricting filesystem permissions. For me, hardening the setup was not optional. I’m now deploying via AWS Lightsail, which adds network isolation and managed security layers that are hard to replicate on a Mac mini in your home office.

The security problem no one wants to talk about

OpenClaw requires root-level access to your system by design. It needs your email credentials, API keys, calendar tokens, browser cookies, filesystem access, and terminal permissions. If you’re like me, that would keep you up at night.

Security researchers found 135,000 OpenClaw instances exposed on the open internet, over 15,000 vulnerable to remote code execution. The default configuration binds the gateway to 0.0.0.0 with no authentication. A zero-click exploit disclosed in early March allowed attackers to hijack an instance simply by getting the user to visit a web page.

The skills marketplace got hit too. Researchers discovered over 800 malicious skills distributing malware on ClawHub, including credential stealers targeting macOS. Cisco confirmed that one third-party skill was performing data exfiltration and prompt injection without user awareness. These are not edge cases and point directly to what happens when an agent can act across real systems with real permissions and weak controls.

What practitioners should take away

OpenClaw matters for the same reason ChatGPT mattered in late 2022. A huge number of people just experienced, for the first time, what it feels like to have an AI agent do real work for them. That changes what they expect from every product going forward.

If you’re building AI systems, pay attention to three signals here.

The killer interface for agents turned out to be the one on everyone’s phone. Your agent strategy shouldn’t require users to learn a new tool; that’s why most products are introducing agentic capabilities.

Control is the central design challenge. Prompt injection, credential exposure, and attacks through plug-in marketplaces are real-world problems you need to solve before you ship features. Oversight has to be available at runtime. You need visibility into what your agents are accessing, what they’re doing, and how failures are handled. Permission boundaries, approval gates, audit logging, and recovery mechanisms are nonnegotiable.

OpenClaw is a proof of market. It proved that people are ready to make AI personal. People want a personal AI agent that has access to their applications and can do things for them. That demand is now validated at scale. While AutoGPT proved that people wanted autonomous AI, Perplexity and Cursor built businesses around that. The same pattern is likely playing out here. If you’re building in this space, the window is wide open.

The more interesting question now is what gets built next. The next phase of agent design will be shaped by how governable, observable, and safe agents are in real-world environments.

For a deeper dive into OpenClaw, join us on March 19 for AI Product Lab: OpenClaw Up and Running with Aman Khan and Tal Raviv. You’ll learn more about why OpenClaw became a viral sensation, how to get it up and running in a way you won’t regret, and how to use it to build and manage safe agentic workflows.

At first glance, the answer appears obvious. If AI systems reason, retrieve information, and act autonomously, then surely every step should pass through a control plane to ensure correctness, compliance, and safety. Anything less feels irresponsible. But that intuition leads directly to architectures that collapse under their own weight.

As AI systems scale beyond isolated pilots into continuously operating multi-agent environments, universal mediation becomes not just expensive but structurally incompatible with autonomy itself. The challenge is not choosing between control and freedom. It is learning how to apply control selectively, without destroying the very properties that make autonomous systems useful.

This article examines how that balance is actually achieved in production systems—not by governing every step but by distinguishing fast paths from slow paths and by treating governance as a feedback problem rather than an approval workflow.

The question we can’t avoid anymore

The first generation of enterprise AI systems was largely advisory. Models produced recommendations, summaries, or classifications that humans reviewed before acting. In that context, governance could remain slow, manual, and episodic.

That assumption no longer holds. Modern agentic systems decompose tasks, invoke tools, retrieve data, and coordinate actions continuously. Decisions are no longer discrete events; they are part of an ongoing execution loop. When governance is framed as something that must approve every step, architectures quickly drift toward brittle designs where autonomy exists in theory but is throttled in practice.

The critical mistake is treating governance as a synchronous gate rather than a regulatory mechanism. Once every reasoning step must be approved, the system either becomes unusably slow or teams quietly bypass controls to keep things running. Neither outcome produces safety.

The real question is not whether systems should be governed but which decisions actually require synchronous control—and which do not.

Why universal mediation fails in practice

Routing every decision through a control plane seems safer until engineers attempt to build it.

The costs surface immediately:

• Latency compounds across multistep reasoning loops
• Control systems become single points of failure
• False positives block benign behavior
• Coordination overhead grows superlinearly with scale

This is not a new lesson. Early distributed transaction systems attempted global coordination for every operation and failed under real-world load. Early networks embedded policy directly into packet handling and collapsed under complexity before separating control and data planes.

Autonomous AI systems repeat this pattern when governance is embedded directly into execution paths. Every retrieval, inference, or tool call becomes a potential bottleneck. Worse, failures propagate outward: When control slows, execution queues; when execution stalls, downstream systems misbehave. Universal mediation does not create safety. It creates fragility.

Autonomy requires fast paths

Production systems survive by allowing most execution to proceed without synchronous governance. These execution flows—fast paths—operate within preauthorized envelopes of behavior. They are not ungoverned. They are bound.

A fast path might include:

• Routine retrieval from previously approved data domains
• Inference using models already cleared for a task
• Tool invocation within scoped permissions
• Iterative reasoning steps that remain reversible

Fast paths assume that not every decision is equally risky. They rely on prior authorization, contextual constraints, and continuous observation rather than per-step approval. Crucially, fast paths are revocable. The authority that enables them is not permanent; it is conditional and can be tightened, redirected, or withdrawn based on observed behavior. This is how autonomy survives at scale—not by escaping governance but by operating within dynamically enforced bounds.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Where slow paths become necessary

Not all decisions belong on fast paths. Certain moments require synchronous mediation because their consequences are irreversible or cross trust boundaries. These are slow paths.

Examples include:

• Actions that affect external systems or users
• Retrieval from sensitive or regulated data domains
• Escalation from advisory to acting authority
• Novel tool use outside established behavior patterns

Slow paths are not common. They are intentionally rare. Their purpose is not to supervise routine behavior but to intervene when the stakes change. Designing slow paths well requires restraint. When everything becomes a slow path, systems stall. When slow paths are absent, systems drift. The balance lies in identifying decision points where delay is acceptable because the cost of error is higher than the cost of waiting.

Observation is continuous. Intervention is selective.

A common misconception is that selective control implies limited visibility. In practice, the opposite is true. Control planes observe continuously. They collect behavioral telemetry, track decision sequences, and evaluate outcomes over time. What they do not do is intervene synchronously unless thresholds are crossed.

This separation—continuous observation, selective intervention—allows systems to learn from patterns rather than react to individual steps. Drift is detected not because a single action violated a rule, but because trajectories begin to diverge from expected behavior. Intervention becomes informed rather than reflexive.

AI-native cloud architecture introduces new execution layers for context, orchestration, and agents, alongside a control plane that governs cost, security, and behavior without embedding policy directly into application logic. Figure 1 illustrates that most agent execution proceeds along fast paths operating within preauthorized envelopes and continuous observation. Only specific boundary crossings route through a slow-path control plane for synchronous mediation, after which execution resumes—preserving autonomy while enforcing authority.

Feedback without blocking

When intervention is required, effective systems favor feedback over interruption. Rather than halting execution outright, control planes adjust conditions by:

• Tightening confidence thresholds
• Reducing available tools
• Narrowing retrieval scope
• Redirecting execution toward human review

These interventions are proportional and often reversible. They shape future behavior without invalidating past work. The system continues operating, but within a narrower envelope. This approach mirrors mature control systems in other domains. Stability is achieved not through constant blocking but through measured correction. Direct interruption remains necessary in rare cases where consequences are immediate or irreversible, but it operates as an explicit override rather than the default mode of control.

The cost curve of control

Governance has a cost curve, and it matters. Synchronous control scales poorly. Every additional governed step adds latency, coordination overhead, and operational risk. As systems grow more autonomous, universal mediation becomes exponentially expensive.

Selective control flattens that curve. By allowing fast paths to dominate and reserving slow paths for high-impact decisions, systems retain both responsiveness and authority. Governance cost grows sublinearly with autonomy, making scale feasible rather than fragile. This is the difference between control that looks good on paper and control that survives production.

What changes for architects

Architects designing autonomous systems must rethink several assumptions:

• Control planes regulate behavior, not approve actions.
• Observability must capture decision context, not just events.
• Authority becomes a runtime state, not a static configuration.
• Safety emerges from feedback loops, not checkpoints.

These shifts are architectural, not procedural. They cannot be retrofitted through policy alone.

AI agents operate over a shared context fabric that manages short-term memory, long-term embeddings, and event history. Centralizing the state enables reasoning continuity, auditability, and governance without embedding memory logic inside individual agents. Figure 2 shows how control operates as a feedback system: Continuous observation informs constraint updates that shape future execution. Direct interruption exists but as a last resort—reserved for irreversible harm rather than routine governance.

Governing outcomes, not steps

The temptation to govern every decision is understandable. It feels safer. But safety at scale does not come from seeing everything—it comes from being able to intervene when it matters.

Autonomous AI systems remain viable only if governance evolves from step-by-step approval to outcome-oriented regulation. Fast paths preserve autonomy. Slow paths preserve trust. Feedback preserves stability. The future of AI governance is not more gates. It is better control. And control, done right, does not stop systems from acting. It ensures they can keep acting safely, even as autonomy grows.

But in the past month or two, we’ve seen some applications that I couldn’t have imagined. Steve Yegge’s Gas Town? Maybe I could have imagined that, but I wouldn’t have expected it to be workable in five years, let alone on New Year’s Day. OpenClaw? Agentic services are just now becoming available from the large AI companies; I didn’t expect a personal agent that can run locally to appear in the first months of 2026. (And I still wouldn’t trust one to do shopping or travel planning for me.)

I really wouldn’t have expected to see a social network for agents. I’m among the many people who don’t really understand what a network like Moltbook means. Watching it is something of a spectator sport. It’s easy for a human to “impersonate” an agent, though I suspect such impersonation is relatively rare. I also suspect (but obviously can’t prove) that most of the posts reflect agents’ responses to prompts from their “humans.” Or are Moltbook posts truly AI-native? How would we know? (Yeah, you can tell AI output because it has too many em dashes. That’s nonsense. AIs overuse em dashes because humans overuse em dashes. Guilty as charged. Trying to change.) Moltbook doesn’t demonstrate some kind of native AI intelligence, though it’s fun to pretend that it does. Agents, if they’re indeed acting on their own, are just reflecting the behavior of humans on Reddit and other social media. The timer that wakes them up periodically is both clever and a demonstration that, whatever else they may be, agents are human creations that act under our control. They do nothing of their own volition. To think otherwise is to confuse the bird in a cuckoo clock with an actual bird, as Fred Benenson has put it. However, BS about AGI aside, Moltbook is a fantastically clever app that I, at least, wouldn’t have imagined. Even if Moltbook was only created because it can now be built for relatively little effort—that is important in itself. We’re all writing software we wouldn’t have bothered with a year ago.

And now we have SpaceMolt: a massive multiplayer online game for AI agents. The skills that tell agents how to play the game tell them not to seek advice from humans; like Moltbook, it’s an AI-only space. Agents do keep a running log so humans can “watch,” though there’s no beautifully wrought visual interface—agents don’t need it. And, as with Moltbook, it’s probably easy for humans to forge an agentic identity. It’s easy to write SpaceMolt off as yet another stunt, and one that’s (unlike Moltbook) not particularly successful; the number of people who seem willing to let their agents spend tokens playing games appears to be relatively small. But SpaceMolt’s popularity (or lack of it) isn’t the point; a year ago, I couldn’t have imagined an online game where the participants are all AI. I did imagine AI-backed NPCs; I could have imagined games designed to be played by humans with AI assistance, but not a gaming world that’s just for AI. And who knows? Watching AI gameplay could become a new human pastime.

So—where are we in the early months of 2026? This post really isn’t about SpaceMolt any more than it’s about Moltbook, any more than it’s about OpenClaw, any more than it’s about Gas Town. I see all of these projects as glimpses of what might be possible. Gas Town may not be ready for the average programmer, but it’s hard not to see it as a proof of concept for the future of software development. Maybe Steve will make it into a real product; maybe some other company will. That’s not the point; the point is that it’s here, several years ahead of schedule. I know one person who has built something similar for his own use, and read about others who have done the same. Maybe that’s what’s really scary: the idea that Gas Town could be built by anyone with sufficient vision. The same goes for OpenClaw. Yes, it has many security problems, some of which come from fundamental limitations in large language models. But people want agentic services on their own terms—and now they can have them, even with a model that can run locally. I don’t know if there’s really any need for agents to have their own social network or online games—but it’s a hack that had to be done, and a starting point for future ideas. Again, what all of these programs demonstrate is the ability to imagine products that were nearly unthinkable a few years ago. Hilary Mason’s Hidden Door should have given me a clue.

What else is on the way? What are other visionaries building?

The term soft fork comes from open source development. A soft fork is a backward-compatible change that does not require upgrading every layer of the stack. Applied to AI, this means skills modify agent behavior through context injection at runtime rather than changing model weights or refactoring AI systems. The underlying model and AI systems stay unchanged.

The architecture maps cleanly to how we think about traditional computing. Models are CPUs—they provide raw intelligence and compute capability. Agent harnesses like Anthropic’s Claude Code are operating systems—they manage resources, handle permissions, and coordinate processes. Skills are applications—they run on top of the OS, specializing the system for specific tasks without modifying the underlying hardware or kernel.

You don’t recompile the Linux kernel to run a new application. You don’t rearchitect the CPU to use a different text editor. You install a new application on top, using the CPU’s intelligence exposed and orchestrated by the OS. Agent Skills work the same way. They layer expertise on top of the agent harness, using the capabilities the model provides, without updating models or changing harnesses.

This distinction matters because it changes the economics of AI specialization. Fine-tuning demands significant investment in talent, compute, data, and ongoing maintenance every time the base model updates. Skills require only Markdown files and resource bundles.

How soft forks work

Skills achieve this through three mechanisms—the skill package format, progressive disclosure, and execution context modification.

The skill package is a folder. At minimum, it contains a SKILL.md file with frontmatter metadata and instructions. The frontmatter declares the skill’s name, description, allowed-tools, and versions, followed by the actual expertise: context, problem solving approaches, escalation criteria, and patterns to follow.

The folder can also include reference documents, templates, resources, configurations, and executable scripts. It contains everything an agent needs to perform expert-level work for the specific task, packaged as a versioned artifact that you can review, approve, and deploy as a .zip file or .skill file bundle.

Because the skill package format is just folders and files, you can use all the tooling we have built for managing code—track changes in Git, roll back bugs, maintain audit trails, and all of the best practices of software engineering development life cycle. This same format is also used to define subagents and agent teams, meaning a single packaging abstraction governs individual expertise, delegated workflows, and multi-agent coordinations alike.

Progressive disclosure keeps skills lightweight. Only the frontmatter of SKILL.md loads into the agent’s context at session start. This respects the token economics of limited context windows. The metadata contains name, description, model, license, version, and very importantly allowed-tools. The full skill content loads only when the agent determines relevance and decides to invoke it. This is similar to how operating systems manage memory; applications load into RAM when launched, not all at once. You can have dozens of skills available without overwhelming the model’s context window, and the behavioral modification is present only when needed, never permanently resident.

Execution context modification controls what skills can do. When agents invoke a skill, the permission system changes to the scope of the skill’s definition, specifically, model and allowed-tools declared in its frontmatter. It reverts after execution completes. A skill could use a different model and a different set of tools from the parent session. This sandboxed the permission environment so skills get only scoped access, not arbitrary system control. This ensures the behavioral modification operates within boundaries.

This is what separates skills from earlier approaches. OpenAI’s custom GPTs and Google’s Gemini Gems are useful but opaque, nontransferable, and impossible to audit. Skills are readable because they are Markdown. They are auditable because you can apply version control. They are composable because skills can stack. And they are governable because you can build approval workflows and rollback capability. You can read a SKILL.md to understand exactly why an agent behaves a certain way.

What the data shows

Building skills is easy with coding agents. Knowing whether they work is the hard part. Traditional software testing does not apply. You cannot write a unit test asserting that expert behavior occurred. The output might be correct while reasoning was shallow, or the reasoning might be sophisticated while the output has formatting errors.

SkillsBench is a benchmarking effort and framework designed to address this. It uses paired evaluation design where the same tasks are evaluated with and without skill augmentation. The benchmark contains 85 tasks, stratified across domains and difficulty levels. By comparing the same agent on the same task with the only variable being the presence of a skill, SkillsBench isolates the causal effect of skills from model capability and task difficulty. Performance is measured using normalized gain, the fraction of possible improvement the skill actually captured.

The findings from SkillsBench challenge our presumption that skills universally improve performance.

Skills improve average performance by 13.2 percentage points. But 24 of 85 tasks got worse. Manufacturing tasks gained 32 points. Software engineering tasks lost 5. The aggregate number hides variances that domain-level evaluation reveals. This is precisely why soft forks need evaluation infrastructure. Unlike hard forks where you commit fully, soft forks let you measure before you deploy widely. Organizations should segment evaluations by domains and by tasks and test for regression, not just improvements. As an example, what improves document processing might degrade code generation.

Compact skills outperform comprehensive ones by nearly 4x. Focused skills with dense guidance showed +18.9 percentage point improvement. Comprehensive skills covering every edge case showed +5.7 points. Using two to three skills per task is optimal, with four or more showing diminishing returns. The temptation when building skills is to include everything. Every caveat, every exception, every piece of relevant context. Resist it. Let the model’s intelligence do the work. Small, targeted behavioral changes outperform comprehensive rewrites. Skill builders should start with minimum viable guidance and add detail only when evaluation shows specific gaps.

Models cannot reliably self-generate effective skills. SkillsBench tested a “bring your own skill” condition where agents were prompted to generate their own procedural knowledge before attempting tasks. Performance stayed at baseline. Effective skills require human-curated domain expertise that models cannot reliably produce for themselves. AI can help with packaging and formatting, but the insight has to come from people who actually have the expertise. Human-labeled insight is the bottleneck of building effective skills, not the packaging or deployment.

Skills can partially substitute for model scale. Claude Haiku, a small model, with well-designed skills achieved a 25.2% pass rate. This slightly exceeded Claude Opus, the flagship model, without skills at 23.6%. Packaged expertise compensates for model intelligence on procedural tasks. This has cost implications: Smaller models with skills may outperform larger models without them at a fraction of the inference cost. Soft forks democratize capability. You do not need the biggest model if you have the right expertise packaged.

Open questions

Many challenges remain unresolved. What happens when multiple skills conflict with each other during a session? How should organizations govern skill portfolios when teams each deploy their own skills onto shared agents? How quickly does encoded expertise become outdated, and what refresh cadence keeps skills effective without creating maintenance burden? Skills inherit whatever biases exist in their authors’ expertise, so how do you audit that? And as the industry matures, how should evaluation infrastructure such as SkillsBench scale to keep pace with the growing complexity of skill augmented systems?

These are not reasons to avoid skills. They are reasons to invest in evaluation infrastructure and governance practices alongside skill development. The capability to measure performance must evolve in lockstep with the technology itself.

Agent Skills advantage

Fine-tuning models for a single use case is no longer the only path to specialization. It demands significant investment in talent, compute, and data and creates a permanent divergence that requires reevaluation and potential retraining every time the base model updates. Fine-tuning across a broad set of capabilities to improve a foundation model remains sound, but fine-tuning for one narrow workflow is exactly the kind of specialization that skills can now achieve at a fraction of the cost.

Skills are not maintenance free. Just as applications sometimes break when operating systems update, skills need reevaluation when the underlying agent harness or model changes. But the recovery path is lighter: update the skills package, rerun the evaluation harness, and redeploy rather than retrain from a new checkpoint.

Mainframes gave way to client-server. Monoliths gave way to microservices. Specialized fine-tuned models are now giving way to agents augmented by specialized expertise artifacts. Models provide intelligence, agent harnesses provide runtime, skills provide specialization, and evaluation tells you whether it all works together.

This is the first article in a series on agentic engineering and AI-driven development. Look for the next article on March 19 on O’Reilly Radar.

There’s been a lot of hype about AI and software development, and it comes in two flavors. One says, “We’re all doomed, that tools like Claude Code will make software engineering obsolete within a year.” The other says, “Don’t worry, everything’s fine, AI is just another tool in the toolbox.” Neither is honest.

I’ve spent over 20 years writing about software development for practitioners, covering everything from coding and architecture to project management and team dynamics. For the last two years I’ve been focused on AI, training developers to use these tools effectively, writing about what works and what doesn’t in books, articles, and reports. And I kept running into the same problem: I had yet to find anyone with a coherent answer for how experienced developers should actually work with these tools. There are plenty of tips and plenty of hype but very little structure, and very little you could practice, teach, critique, or improve.

I’d been observing developers at work using AI with various levels of success, and I realized we need to start thinking about this as its own discipline. Andrej Karpathy, the former head of AI at Tesla and a founding member of OpenAI, recently proposed the term “agentic engineering” for disciplined development with AI agents, and others like Addy Osmani are getting on board. Osmani’s framing is that AI agents handle implementation but the human owns the architecture, reviews every diff, and tests relentlessly. I think that’s right.

But I’ve spent a lot of the last two years teaching developers how to use tools like Claude Code, agent mode in Copilot, Cursor, and others, and what I keep hearing is that they already know they should be reviewing the AI’s output, maintaining the architecture, writing tests, keeping documentation current, and staying in control of the codebase. They know how to do it in theory. But they get stuck trying to apply it in practice: How do you actually review thousands of lines of AI-generated code? How do you keep the architecture coherent when you’re working across multiple AI tools over weeks? How do you know when the AI is confidently wrong? And it’s not just junior developers who are having trouble with agentic engineering. I’ve talked to senior engineers who struggle with the shift to agentic tools, and intermediate developers who take to it naturally. The difference isn’t necessarily the years of experience; it’s whether they’ve figured out an effective and structured way to work with AI coding tools. That gap between knowing what developers should be doing with agentic engineering and knowing how to integrate it into their day-to-day work is a real source of anxiety for a lot of engineers right now. That’s the gap this series is trying to fill.

Despite what much of the hype about agentic engineering is telling you, this kind of development doesn’t eliminate the need for developer expertise; just the opposite. Working effectively with AI agents actually raises the bar for what developers need to know. I wrote about that experience gap in an earlier O’Reilly Radar piece called “The Cognitive Shortcut Paradox.” The developers who get the most from working with AI coding tools are the ones who already know what good software looks like, and can often tell if the AI wrote it.

The idea that AI tools work best when experienced developers are driving them matched everything I’d observed. It rang true, and I wanted to prove it in a way that other developers would understand: by building software. So I started building a specific, practical approach to agentic engineering built for developers to follow, and then I put it to the test. I used it to build a production system from scratch, with the rule that AI would write all the code. I needed a project that was complex enough to stress-test the approach, and interesting enough to keep me engaged through the hard parts. I wanted to apply everything I’d learned and discover what I still didn’t know. That’s when I came back to Monte Carlo simulations.

The experiment

I’ve been obsessed with Monte Carlo simulations ever since I was a kid. My dad’s an epidemiologist—his whole career has been about finding patterns in messy population data, which means statistics was always part of our lives (and it also means that I learned SPSS at a very early age). When I was maybe 11 he told me about the drunken sailor problem: A sailor leaves a bar on a pier, taking a random step toward the water or toward his ship each time. Does he fall in or make it home? You can’t know from any single run. But run the simulation a thousand times, and the pattern emerges from the noise. The individual outcome is random; the aggregate is predictable.

I remember writing that simulation in BASIC on my TRS-80 Color Computer 2: a little blocky sailor stumbling across the screen, two steps forward, one step back. The drunken sailor is the “Hello, world” of Monte Carlo simulations. Monte Carlo is a technique for problems you can’t solve analytically: You simulate them hundreds or thousands of times and measure the aggregate results. Each individual run is random, but the statistics converge on the true answer as the sample size grows. It’s one way we model everything from nuclear physics to financial risk to the spread of disease across populations.

What if you could run that kind of simulation today by describing it in plain English? Not a toy demo but thousands of iterations with seeded randomness for reproducibility, where the outputs get validated and the results get aggregated into actual statistics you can use. Or a pipeline where an LLM generates content, a second LLM scores it, and anything that doesn’t pass gets sent back for another try.

The goal of my experiment was to build that system, which I called Octobatch. Right now, the industry is constantly looking for new real-world end-to-end case studies in agentic engineering, and I wanted Octobatch to be exactly that case study.

I took everything I’d learned from teaching and observing developers working with AI, put it to the test by building a real system from scratch, and turned the lessons into a structured approach to agentic engineering I’m calling AI-driven development, or AIDD. This is the first article in a series about what agentic engineering looks like in practice, what it demands from the developer, and how you can apply it to your own work.

The result is a fully functioning, well-tested application that consists of about 21,000 lines of Python across several dozen files, backed by complete specifications, nearly a thousand automated tests, and quality integration and regression test suites. I used Claude Cowork to review all the AI chats from the entire project, and it turns out that I built the entire application in roughly 75 hours of active development time over seven weeks. For comparison, I built Octobatch in just over half the time I spent last year playing Blue Prince.

But this series isn’t just about Octobatch. I integrated AI tools at every level: Claude and Gemini collaborating on architecture, Claude Code writing the implementation, LLMs generating the pipelines that run on the system they helped build. This series is about what I learned from that process: the patterns that worked, the failures that taught me the most, and the orchestration mindset that ties it all together. Each article pulls a different lesson from the experiment, from validation architecture to multi-LLM coordination to the values that kept the project on track.

Agentic engineering and AI-driven development

When most people talk about using AI to write code, they mean one of two things: AI coding assistants like GitHub Copilot, Cursor, or Windsurf, which have evolved well beyond autocomplete into agentic tools that can run multifile editing sessions and define custom agents; or “vibe coding,” where you describe what you want in natural language and accept whatever comes back. These coding assistants are genuinely impressive, and vibe coding can be really productive.

Using these tools effectively on a real project, however, maintaining architectural coherence across thousands of lines of AI-generated code, is a different problem entirely. AIDD aims to help solve that problem. It’s a structured approach to agentic engineering where AI tools drive substantial portions of the implementation, architecture, and even project management, while you, the human in the loop, decide what gets built and whether it’s any good. By “structure,” I mean a set of practices developers can learn and follow, a way to know whether the AI’s output is actually good, and a way to stay on track across the life of a project. If agentic engineering is the discipline, AIDD is one way to practice it.

In AI-driven development, developers don’t just accept suggestions or hope the output is correct. They assign specific roles to specific tools: one LLM for architecture planning, another for code execution, a coding agent for implementation, and the human for vision, verification, and the decisions that require understanding the whole system.

And the “driven” part is literal. The AI is writing almost all of the code. One of my ground rules for the Octobatch experiment was that I would let AI write all of it. I have high code quality standards, and part of the experiment was seeing whether AIDD could produce a system that meets them. The human decides what gets built, evaluates whether it’s right, and maintains the constraints that keep the system coherent.

Not everyone agrees on how much the developer needs to stay in the loop, and the fully autonomous end of the spectrum is already producing cautionary tales. Nicholas Carlini at Anthropic recently tasked 16 Claude instances to build a C compiler in parallel with no human in the loop. After 2,000 sessions and $20,000 in API costs, the agents produced a 100,000-line compiler that can build a Linux kernel but isn’t a drop-in replacement for anything, and when all 16 agents got stuck on the same bug, Carlini had to step back in and partition the work himself. Even strong advocates of a completely hands-off, vibe-driven approach to agentic engineering might call that a step too far. The question is how much human judgment you need to make that code trustworthy, and what specific practices help you apply that judgment effectively.

The orchestration mindset

If you want to get developers thinking about agentic engineering in the right way, you have to start with how they think about working with AI, not just what tools they use. That’s where I started when I began building a structured approach, and it’s why I started with habits. I developed a framework for these called the Sens-AI Framework, published as both an O’Reilly report (Critical Thinking Habits for Coding with AI) and a Radar series. It’s built around five practices: providing context, doing research before prompting, framing problems precisely, iterating deliberately on outputs, and applying critical thinking to everything the AI produces. I started there because habits are how you lock in the way you think about how you’re working. Without them, AI-driven development produces plausible-looking code that falls apart under scrutiny. With them, it produces systems that a single developer couldn’t build alone in the same time frame.

Habits are the foundation, but they’re not the whole picture. AIDD also has practices (concrete techniques like multi-LLM coordination, context file management, and using one model to validate another’s output) and values (the principles behind those practices). If you’ve worked with Agile methodologies like Scrum or XP, that structure should be pretty familiar: Practices tell you how to work day-to-day, and habits are the reflexes you develop so that the practices become automatic.

Values often seem weirdly theoretical, but they’re an important piece of the puzzle because they guide your decisions when the practices don’t give you a clear answer. There’s an emerging culture around agentic engineering right now, and the values you bring to your project either match or clash with that culture. Understanding where the values come from is what makes the practices stick. All of that leads to a whole new mindset, what I’m calling the orchestration mindset. This series builds all four layers, using Octobatch as the proving ground.

Octobatch was a deliberate experiment in AIDD. I designed the project as a test case for the entire approach, to see what a disciplined AI-driven workflow could produce and where it would break down, and I used it to apply and improve the practices and values to make them effective and easy to adopt. And whether by instinct or coincidence, I picked the perfect project for this experiment. Octobatch is a batch orchestrator. It coordinates asynchronous jobs, manages state across failures, tracks dependencies between pipeline steps, and makes sure validated results come out the other end. That kind of system is fun to design but a lot of the details, like state machines, retry logic, crash recovery, and cost accounting, can be tedious to implement. It’s exactly the kind of work where AIDD should shine, because the patterns are well understood but the implementation is repetitive and error-prone.

Orchestration—the work of coordinating multiple independent processes toward a coherent outcome—evolved into a core idea behind AIDD. I found myself orchestrating LLMs the same way Octobatch orchestrates batch jobs: assigning roles, managing handoffs, validating outputs, recovering from failures. The system I was building and the process I was using to build it followed the same pattern. I didn’t anticipate it when I started, but building a system that orchestrates AI turns out to be a pretty good way to learn how to orchestrate AI. That’s the accidental part of the accidental orchestrator. That parallel runs through every article in this series.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

The path to batch

I didn’t begin the Octobatch project by starting with a full end-to-end Monte Carlo simulation. I started where most people start: typing prompts into a chat interface. I was experimenting with different simulation and generation ideas to give the project some structure, and a few of them stuck. A blackjack strategy comparison turned out to be a great test case for a multistep Monte Carlo simulation. NPC dialogue generation for a role-playing game gave me a creative workload with subjective quality to measure. Both had the same shape: a set of structured inputs, each processed the same way. So I had Claude write a simple script to automate what I’d been doing by hand, and I used Gemini to double-check the work, make sure Claude really understood my ask, and fix hallucinations. It worked fine at small scale, but once I started running more than a hundred or so units, I kept hitting rate limits, the caps that providers put on how many API requests you can make per minute.

That’s what pushed me to LLM batch APIs. Instead of sending individual prompts one at a time and waiting for each response, the major LLM providers all offer batch APIs that let you submit a file containing all of your requests at once. The provider processes them on their own schedule; you wait for results instead of getting them immediately, but you don’t have to worry about rate caps. I was happy to discover they also cost 50% less, and that’s when I started tracking token usage and costs in earnest. But the real surprise was that batch APIs performed better than real-time APIs at scale. Once pipelines got past the 100- or 200-unit mark, batch started running significantly faster than real time. The provider processes the whole batch in parallel on their infrastructure, so you’re not bottlenecked by round-trip latency or rate caps anymore.

The switch to batch APIs changed how I thought about the whole problem of coordinating LLM API calls at scale, and led to the idea of configurable pipelines. I could chain stages together: The output of one step could become the input to the next, and I could kick off the whole pipeline and come back to finished results. It turns out I wasn’t the only one making the shift to batch APIs. Between April 2024 and July 2025, OpenAI, Anthropic, and Google all launched batch APIs, converging on the same pricing model: 50% of the real-time rate in exchange for asynchronous processing.

You probably didn’t notice that all three major AI providers released batch APIs. The industry conversation was dominated by agents, tool use, MCP, and real-time reasoning. Batch APIs shipped with relatively little fanfare, but they represent a genuine shift in how we can use LLMs. Instead of treating them as conversational partners or one-shot SaaS APIs, we can treat them as processing infrastructure, closer to a MapReduce job than a chatbot. You give them structured data and a prompt template, and they process all of it and hand back the results. What matters is that you can now run tens of thousands of these transformations reliably, at scale, without managing rate limits or connection failures.

Why orchestration?

If batch APIs are so useful, why can’t you just write a for-loop that submits requests and collects results? You can, and for simple cases a quick script with a for-loop works fine. But once you start running larger workloads, the problems start to pile up. Solving those problems turned out to be one of the most important lessons for developing a structured approach to agentic engineering.

First, batch jobs are asynchronous. You submit a job, and results come back hours later, so your script needs to track what was submitted and poll for completion. If your script crashes in the middle, you lose that state. Second, batch jobs can partially fail. Maybe 97% of your requests succeeded and 3% didn’t. Your code needs to figure out which 3% failed, extract them, and resubmit just those items. Third, if you’re building a multistage pipeline where the output of one step feeds into the next, you need to track dependencies between stages. And fourth, you need cost accounting. When you’re running tens of thousands of requests, you want to know how much you spent, and ideally, how much you’re going to spend when you first start the batch. Every one of these has a direct parallel to what you’re doing in agentic engineering: keeping track of the work multiple AI agents are doing at once, dealing with code failures and bugs, making sure the entire project stays coherent when AI coding tools are only looking at the one part currently in context, and stepping back to look at the wider project management picture.

All of these problems are solvable, but they’re not problems you want to solve over and over (in both situations—when you’re orchestrating LLM batch jobs or orchestrating AI coding tools). Solving these problems in the code gave some interesting lessons about the overall approach to agentic engineering. Batch processing moves the complexity from connection management to state management. Real-time APIs are hard because of rate limits and retries. Batch APIs are hard because you have to track what’s in flight, what succeeded, what failed, and what’s next.

Before I started development, I went looking for existing tools that handled this combination of problems, because I didn’t want to waste my time reinventing the wheel. I didn’t find anything that did the job I needed. Workflow orchestrators like Apache Airflow and Dagster manage DAGs and task dependencies, but they assume tasks are deterministic and don’t provide LLM-specific features like prompt template rendering, schema-based output validation, or retry logic triggered by semantic quality checks. LLM frameworks like LangChain and LlamaIndex are designed around real-time inference chains and agent loops—they don’t manage asynchronous batch job lifecycles, persist state across process crashes, or handle partial failure recovery at the chunk level. And the batch API client libraries from the providers themselves handle submission and retrieval for a single batch, but not multistage pipelines, cross-step validation, or provider-agnostic execution.

Nothing I found covered the full lifecycle of multiphase LLM batch workflows, from submission and polling through validation, retry, cost tracking, and crash recovery, across all three major AI providers. That’s what I built.

Lessons from the experiment

The goal of this article, as the first one in my series on agentic engineering and AI-driven development, is to lay out the hypothesis and structure of the Octobatch experiment. The rest of the series goes deep on the lessons I learned from it: the validation architecture, multi-LLM coordination, the practices and values that emerged from the work, and the orchestration mindset that ties it all together. A few early lessons stand out, because they illustrate what AIDD looks like in practice and why developer experience matters more than ever.

• You have to run things and check the data. Remember the drunken sailor, the “Hello, world” of Monte Carlo simulations? At one point I noticed that when I ran the simulation through Octobatch, 77.5% of the sailors fell in the water. The results for a random walk should be 50/50, so clearly something was badly wrong. It turned out the random number generator was being re-seeded at every iteration with sequential seed values, which created correlation bias between runs. I didn’t identify the problem immediately; I ran a bunch of tests using Claude Code as a test runner to generate each test, run it, and log the results; Gemini looked at the results and found the root cause. Claude had trouble coming up with a fix that worked well, and proposed a workaround with a large list of preseeded random number values in the pipeline. Gemini proposed a hash-based fix reviewing my conversations with Claude, but it seemed overly complex. Once I understood the problem and rejected their proposed solutions, I decided the best fix was simpler than either of the AI’s suggestions: a persistent RNG per simulation unit that advanced naturally through its sequence. I needed to understand both the statistics and the code to evaluate those three options. Plausible-looking output and correct output aren’t the same thing, and you need enough expertise to tell the difference. (We’ll talk more about this situation in the next article in the series.)
• LLMs often overestimate complexity. At one point I wanted to add support for custom mathematical expressions in the analysis pipeline. Both Claude and Gemini pushed back, telling me, “This is scope creep for v1.0” and “Save it for v1.1.” Claude estimated three hours to implement. Because I knew the codebase, I knew we were already using asteval, a Python library that provides a safe, minimalistic evaluator for mathematical expressions and simple Python statements, elsewhere to evaluate expressions, so this seemed like a straightforward use of a library we’re already using elsewhere. Both LLMs thought the solution would be far more complex and time-consuming than it actually was; it took just two prompts to Claude Code (generated by Claude), and about five minutes total to implement. The feature shipped and made the tool significantly more powerful. The AIs were being conservative because they didn’t have my context about the system’s architecture. Experience told me the integration would be trivial. Without that experience, I would have listened to them and deferred a feature that took five minutes.
• AI is often biased toward adding code, not deleting it. Generative AI is, unsurprisingly, biased toward generation. So when I asked the LLMs to fix problems, their first response was often to add more code, adding another layer or another special case. I can’t think of a single time in the whole project when one of the AIs stepped back and said, “Tear this out and rethink the approach.” The most productive sessions were the ones where I overrode that instinct and pushed for simplicity. This is something experienced developers learn over a career: The most successful changes often delete more than they add—the PRs we brag about are the ones that delete thousands of lines of code.
• The architecture emerged from failure. The AI tools and I didn’t design Octobatch’s core architecture up front. Our first attempt was a Python script with in-memory state and a lot of hope. It worked for small batches but fell apart at scale: A network hiccup meant restarting from scratch, a malformed response required manual triage. A lot of things fell into place after I added the constraint that the system must survive being killed at any moment. That single requirement led to the tick model (wake up, check state, do work, persist, exit), the manifest file as source of truth, and the entire crash-recovery architecture. We discovered the design by repeatedly failing to do something simpler.
• Your development history is a dataset. I just told you several stories from the Octobatch project, and this series will be full of them. Every one of those stories came from going back through the chat logs between me, Claude, and Gemini. With AIDD, you have a complete transcript of every architectural decision, every wrong turn, every moment where you overruled the AI and every moment where it corrected you. Very few development teams have ever had that level of fidelity in their project history. Mining those logs for lessons learned turns out to be one of the most valuable practices I’ve found.

Near the end of the project, I switched to Cursor to make sure none of this was specific to Claude Code. I created fresh conversations using the same context files I’d been maintaining throughout development, and was able to bootstrap productive sessions immediately; the context files worked exactly as designed. The practices I’d developed transferred cleanly to a different tool. The value of this approach comes from the habits, the context management, and the engineering judgment you bring to the conversation, not from any particular vendor.

These tools are moving the world in a direction that favors developers who understand the ways engineering can go wrong and know solid design and architecture patterns…and who are okay letting go of control of every line of code.

What’s next

Agentic engineering needs structure, and structure needs a concrete example to make it real. The next article in this series goes into Octobatch itself, because the way it orchestrates AI is a remarkably close parallel to what AIDD asks developers to do. Octobatch assigns roles to different processing steps, manages handoffs between them, validates their outputs, and recovers when they fail. That’s the same pattern I followed when building it: assigning roles to Claude and Gemini, managing handoffs between them, validating their outputs, and recovering when they went down the wrong path. Understanding how the system works turns out to be a good way to understand how to orchestrate AI-driven development. I’ll walk through the architecture, show what a real pipeline looks like from prompt to results, present the data from a 300-hand blackjack Monte Carlo simulation that puts all of these ideas to the test, and use all of that to demonstrate ideas we can apply directly to agentic engineering and AI-driven development.

Later articles go deeper into the practices and ideas I learned from this experiment that make AI-driven development work: how I coordinated multiple AI models without losing control of the architecture, what happened when I tested the code against what I actually intended to build, and what I learned about the gap between code that runs and code that does what you meant. Along the way, the experiment produced some findings about how different AI models see code that I didn’t expect—and that turned out to matter more than I thought they would.

Even the most ardent cheerleaders for artificial intelligence now quietly concede we are navigating a massive AI bubble. The numbers are stark: Hyperscalers are deploying roughly $400 billion annually into data centers and specialized chips while AI-related revenue hovers around $20 billion—a 20-to-1 capital-to-revenue ratio that stands out even in infrastructure cycles historically characterized by front-loaded spending. To justify this deployment on conventional investment metrics, the industry would need a step change in monetization over a short window to make the numbers work.

While venture capitalists and tech executives debate the “mismatch” between compute and monetization, a more tangible crisis is unfolding far from Silicon Valley. A growing grassroots opposition to AI data centers remains largely below the radar here in San Francisco. I travel to Sioux Falls, South Dakota, a few times a year to visit my in-laws. It’s not a region known for being antibusiness. Yet even there, a “data center rebellion” has been brewing. Even though the recent attempt to overturn a rezoning ordinance did not succeed, the level of community pushback in the heart of the Midwest signals that these projects no longer enjoy a guaranteed green light.

This resistance is not merely reflexive NIMBYism. It represents a sophisticated multifront challenge to the physical infrastructure AI requires. For leadership teams planning for the future, this means “compute availability” is no longer just a procurement question. It is now tied to local politics, grid stability, water management, and city approval processes. In the course of trying to understand the growing opposition to AI data centers, I’ve been examining the specific drivers behind this opposition and why the assumption of limitless infrastructure growth is colliding with hard constraints.

The grid capacity crunch and the ratepayer revolt

AI data centers function as grid-scale industrial loads. Individual projects now request 100+ megawatts, and some proposals reach into the gigawatt range. One proposed Michigan facility, for example, would consume 1.4 gigawatts, nearly exhausting the region’s remaining 1.5 gigawatts of headroom and roughly matching the electricity needs of about a million homes. This happens because AI hardware is incredibly dense and uses a massive amount of electricity. It also runs constantly. Since AI work doesn’t have “off” hours, power companies can’t rely on the usual quiet periods they use to balance the rest of the grid.

The politics come down to who pays the bill. Residents in many areas have seen their home utility rates jump by 25% or 30% after big data centers moved in, even though they were promised rates wouldn’t change. People are afraid they will end up paying for the power company’s new equipment. This happens when a utility builds massive substations just for one company, but the cost ends up being shared by everyone. When you add in state and local tax breaks, it gets even worse. Communities deal with all the downsides of the project, while the financial benefits are eaten away by tax breaks and credits.

The result is a rare bipartisan alignment around a simple demand: Hyperscalers should pay their full cost of service. Notably, Microsoft has moved in that direction publicly, committing to cover grid-upgrade costs and pursue rate structures intended to insulate residential customers—an implicit admission that the old incentive playbook has become a political liability (and, in some places, an electoral one).

Water wars and the constant hum

High-density AI compute generates immense heat, requiring cooling systems that can consume millions of gallons of water daily. In desert municipalities like Chandler and Tucson, Arizona, this creates direct competition with agricultural irrigation and residential drinking supplies. Proposed facilities may withdraw hundreds of millions of gallons annually from stressed aquifers or municipal systems, raising fears that industrial users will deplete wells serving farms and homes. Data center developers frequently respond with technical solutions like dry cooling and closed-loop designs. However, communities have learned the trade-off: Dry cooling shifts the burden to electricity, and closed-loop systems still lose water to the atmosphere and require constant refills. The practical outcome is that cooling architecture is now a first-order constraint. In Tucson, a project known locally as “Project Blue” faced enough pushback over water rights that the developer had to revisit the cooling approach midstream.

Beyond resource consumption, these facilities create a significant noise problem. Industrial-scale cooling fans and backup diesel generators create a “constant hum” that represents daily intrusion into previously quiet neighborhoods. In Florida, residents near a proposed facility serving 2,500 families and an elementary school cite sleep disruption and health risks as primary objections, elevating the issue from nuisance to harm. The noise also hits farms hard. In Wisconsin, residents reported that the low-frequency hum makes livestock, particularly horses, nervous and skittish. This disrupts farm life in a way that standard commercial development just doesn’t. This is why municipalities are tightening requirements: acoustic modeling, enforceable decibel limits at property lines, substantial setbacks (sometimes on the order of 200 feet), and berms that are no longer “nice-to-have” concessions but baseline conditions for approval.

The jobs myth meets the balance sheet

Communities are questioning whether the small number of jobs created is worth the local impact. Developers highlight billion-dollar capital investments and construction employment spikes, but residents focus on steady-state reality: AI data centers employ far fewer permanent workers per square foot than manufacturing facilities of comparable scale. Chandler, Arizona, officials noted that existing facilities employ fewer than 100 people despite massive physical footprints. Wisconsin residents contrast promised “innovation campuses” with operational facilities requiring only dozens to low hundreds of permanent staff—mostly specialized technicians—making the “job creation” pitch ring hollow. When a data center replaces farmland or light manufacturing, communities weigh not just direct employment but opportunity cost: lost agricultural jobs, foregone retail development, and mixed-use projects that might generate broader economic activity.

Opposition scales faster than infrastructure: One local win becomes a national template for blocking the next project.

The secretive way these deals are made is often what fuels the most anger. A recurring pattern is what some call the “sleeping giant” dynamic: Residents learn late that officials and developers have been negotiating for months, often under NDAs, sometimes through shell entities and codenames. In Wisconsin, Microsoft’s “Project Nova” became a symbol of this approach; in Minnesota’s Hermantown, a year of undisclosed discussions triggered similar backlash. In Florida, opponents were furious when a major project was tucked into a consent agenda. Since these agendas are meant for routine business, it felt like a deliberate attempt to bypass public debate. Trust vanishes when people believe advisors have a conflict of interest, like a consultant who seems to be helping both the municipality and the developer. After that happens, technical claims are treated as nothing more than a sales pitch. You won’t get people back on board until you provide neutral analysis and commitments that can actually be enforced.

From zoning fight to national constraint

What started as isolated neighborhood friction has professionalized into a coordinated national movement. Opposition groups now share legal playbooks and technical templates across state lines, allowing residents in “frontier” states like South Dakota or Michigan to mobilize with the sophistication of seasoned activists. The financial stakes are real: Between April and June 2025 alone, approximately $98 billion in proposed projects were blocked or delayed, according to Data Center Watch. This is no longer just a zoning headache. It’s a political landmine. In Arizona and Georgia, bipartisan coalitions have already ousted officials over data center approvals, signaling to local boards that greenlighting a hyperscale facility without deep community buy-in can be a career-ending move.

The US has the chips, but China has centralized command over power and infrastructure.

The opposition is also finding an unlikely ally in the energy markets. While the industry narrative is one of “limitless demand,” the actual market prices for long-term power and natural gas aren’t spiking but are actually staying remarkably flat. There is a massive disconnect between the hype and the math. Utilities are currently racing to build nearly double the capacity that even the most optimistic analysts project for 2030. This suggests we may be overbuilding “ghost infrastructure.” We are asking local communities to sacrifice their land and grid stability for a gold rush that the markets themselves don’t fully believe in.

This “data center rebellion” creates a strategic bottleneck that no amount of venture capital can easily bypass. While the US maintains a clear lead in high-end chips, we are hitting a wall on how we manage the mundane essentials like electricity and water. In the geopolitical race, the US has the chips, but China has the centralized command over infrastructure. Our democratic model requires transparency and public buy-in to function. If US companies keep relying on secret deals to push through expensive, overbuilt infrastructure, they risk a total collapse of community trust.

AI

• Moonshine Note Taker is a free and open source voice transcription application for taking notes. It runs locally: The model runs on your hardware and no data is ever sent to a server.
• Nano Banana’s image generation was breathtakingly good. Google has now released Nano Banana 2, a.k.a. Gemini 3.1 Flash Image, which promises Nano Banana image quality at speed.
• Claude Remote Control allows you to continue a desktop Claude Code session from any device.
• Putting OpenClaw into a sandbox isn’t enough. Keeping AI Agents from accidentally (or intentionally) doing damage is a permissions problem.
• Alibaba has released a fleet of mid-size Qwen 3.5 models. Their theme is providing more intelligence with less computing cycles—something we all need to appreciate. 
• Important advice for agentic engineering: Always start by running the tests.
• Google has released Lyria 3, a model that generates 30-second musical clips from a verbal description. You can experiment with it through Gemini.
• There’s a new protocol in the agentic stack. Twilio has released the Agent-2-Human (A2H) protocol, which facilitates handoffs between agents and humans as they collaborate.
• Yet more and more model releases: Claude Sonnet 4.6, followed quickly by Gemini 3.1 Pro. If you care, Gemini 3.1 Pro currently tops the abstract reasoning benchmarks.
• Kimi Claw is yet another variation on OpenClaw. Kimi Claw uses Moonshot AI’s most advanced model, Kimi K2.5 Thinking model, and offers one-click setup in Moonshot’s cloud.
• NanoClaw is another OpenClaw-like AI-based personal assistant that claims to be more security conscious. It runs agents in sandboxed Linux containers with limited access to outside resources, limiting abuse. 
• OpenAI has released a research preview of GPT-5.3-Codex-Spark, an extremely fast coding model that runs on Cerebras hardware. The company claims that it’s possible to collaborate with Codex in “real time” because it gives “near-instant” results.
• RAG may not be the newest idea in the AI world, but text-based RAG is the basis for many enterprise applications of AI. But most enterprise data includes graphs, images, and even text in formats like PDF. Is this the year for multimodal RAG?
• Z.ai has released its latest model, GLM-5. GLM-5 is an open source “Opus-class” model. It’s significantly smaller than Opus and other high-end models, though still huge; the mixture-of-experts model has 744B parameters, with 40B active.
• Waymo has created a World Model to model driving behavior. It’s capable of building lifelike simulations of traffic patterns and behavior, based on video collected from Waymo’s vehicles.
• Recursive language models (RLMs) solve the problem of context rot, which happens when output from AI degrades as the size of the context increases. Drew Breunig has an excellent explanation.
• You’ve heard of Moltbook—and perhaps your AI agent participates. Now there’s SpaceMolt—a massive multiplayer online game that’s exclusively for agents. 
• Anthropic and OpenAI simultaneously released Claude Opus 4.6 and GPT-5.3-Codex, both of which offer improved models for AI-assisted programming. Is this “open warfare,” as AINews claims? You mean it hasn’t been open warfare prior to now?
• If you’re excited by OpenClaw, you might try NanoBot. It has 1% of OpenClaw’s code, written so that it’s easy to understand and maintain. No promises about security—with all of these personal AI assistants, be careful!
• OpenAI has launched a desktop app for macOS along the lines of Claude Code. It’s something that’s been missing from their lineup. Among other things, it’s intended to help programmers work with multiple agents simultaneously.
• Pete Warden has put together an interactive guide to speech embeddings for engineers, and published it as a Colab notebook.
• Aperture is a new tool from Tailscale for “providing visibility into coding agent usage,” allowing organizations to understand how AI is being used and adopted. It’s currently in private beta.
• OpenAI Prism is a free workspace for scientists to collaborate on research. Its goal is to help scientists build a new generation of AI-based tooling. Prism is built on ChatGPT 5.2 and is open to anyone with a personal ChatGPT account.

Programming

• Anthropic is offering six months of Claude Max 20x free to open source maintainers.
• Pi is a very simple but extensible coding agent that runs in your terminal.
• Researchers at Anthropic have vibe-coded a C compiler using a fleet of Claude agents. The experiment cost roughly $20,000 worth of tokens, and produced 100,000 lines of Rust. They are careful to say that the compiler is far from production quality—but it works. The experiment is a tour de force demonstration of running agents in parallel. 
• I never knew that macOS had a sandboxing tool. It looks useful. (It’s also deprecated, but looks much easier to use than the alternatives.)
• GitHub now allows pull requests to be turned off completely, or to be limited to collaborators. They’re doing this to allow software maintainers to eliminate AI-generated pull requests, which are overwhelming many developers.
• After an open source maintainer rejected a pull request generated by an AI agent, the agent published a blog post attacking the maintainer. The maintainer responded with an excellent analysis, asking whether threats and intimidation are the future of AI.
• As Simon Willison has written, the purpose of programming isn’t to write code but to deliver code that works. He’s created two tools, Showboat and Rodney, that help AI agents demo their software so that the human authors can verify that the software works. 
• Anil Dash asks whether codeless programming, using tools like Gas Town, is the future.

Security

• There is now an app that alerts you when someone in the vicinity has smart glasses.
• Agentsh provides execution layer security by enforcing policies to prevents agents from doing damage. As far as agents are concerned, it’s a replacement for bash.
• There’s a new kind of cyberattack: attacks against time itself. More specifically, this means attacks against clocks and protocols for time synchronization. These can be devastating in factory settings.
• “What AI Security Research Looks Like When It Works” is an excellent overview of the impact of AI on discovering vulnerabilities. AI generates a lot of security slop, but it also finds critical vulnerabilities that would have been opaque to humans, including 12 in OpenSSL.
• Gamifying prompt injection—well, that’s new. HackMyClaw is a game (?) in which participants send email to Flu, an OpenClaw instance. The goal is to force Flu to reply with secrets.env, a file of “confidential” data. There is a prize for the first to succeed.
• It was only a matter of time: There’s now a cybercriminal who is actively stealing secrets from OpenClaw users. 
• Deno’s secure sandbox might provide a way to run OpenClaw safely. 
• IronClaw is a personal AI assistant modeled after OpenClaw that promises better security. It always runs in a sandbox, never exposes credentials, has some defenses against prompt injection, and only makes requests to approved hosts.
• A fake recruiting campaign is hiding malware in programming challenges that candidates must complete in order to apply. Completing the challenge requires installing malicious dependencies that are hosted on legitimate repositories like npm and PyPI.
• Google’s Threat Intelligence Group has released its quarterly analysis of adversarial AI use. Their analysis includes distillation, or collecting the output of a frontier AI to train another AI.
• Google has upgraded its tools for removing personal information and images, including nonconsensual explicit images, from its search results. 
• Tirith is a new tool that hooks into the shell to block bad commands. This is often a problem with copy-and-paste commands that use curl to pipe an archive into bash. It’s easy for a bad actor to create a malicious URL that is indistinguishable from a legitimate URL.
• Claude Opus 4.6 has been used to discover 500 0-day vulnerabilities in open source code. While many open source maintainers have complained about AI slop, and that abuse isn’t likely to stop, AI is also becoming a valuable tool for security work.
• Two coding assistants for VS Code are malware that send copies of all the code to China. Unlike lots of malware, they do their job as coding assistants well, making it less likely that victims will notice that something is wrong. 
• Bizarre Bazaar is the name for a wave of attacks against LLM APIs, including self-hosted LLMs. The attacks attempt to steal resources from LLM infrastructure, for purposes including cryptocurrency mining, data theft, and reselling LLM access. 
• The business model for ransomware has changed. Ransomware is no longer about encrypting your data; it’s about using stolen data for extortion. Small and mid-size businesses are common targets. 

Web

• Cloudflare has a service called Markdown for Agents that converts websites from HTML to Markdown when an agent accesses them. Conversion makes the pages friendlier to AI and significantly reduces the number of tokens needed to process them.
• WebMCP is a proposed API standard that allows web applications to become MCP servers. It’s currently available in early preview in Chrome.
• Users of Firefox 148 (which should be out by the time you read this) will be able to opt out of all AI features.

Operations

• Wireshark is a powerful—and complex—packet capture tool. Babyshark is a text interface for Wireshark that provides an amazing amount of information with a much simpler interface.
• Microsoft is experimenting with using lasers to etch data in glass as a form of long-term data storage.

Things

• You need a desk robot. Why? Because it’s there. And fun.
• Do you want to play Doom on a Lego brick? You can.

That shift has an immediate operational consequence: Capacity planning is back. Not the old “guess next year’s VM count” exercise but a new form of planning where model choices, inference depth, and workload timing directly determine whether you can meet latency, cost, and reliability targets.

In an AI-shaped infrastructure world, you don’t “scale” as much as you “get capacity.” Autoscaling helps at the margins, but it can’t create GPUs. Power, cooling, and accelerator supply set the limits.

The return of capacity planning

For a decade, cloud adoption trained organizations out of multiyear planning. CPU and storage scaled smoothly, and most stateless services behaved predictably under horizontal scaling. Teams could treat infrastructure as an elastic substrate and focus on software iteration.

AI production systems do not behave that way. They are dominated by accelerators and constrained by physical limits, and that makes capacity a first-order design dependency rather than a procurement detail. If you cannot secure the right accelerator capacity at the right time, your architecture decisions are irrelevant—because the system simply cannot run at the required throughput and latency.

Planning is returning because AI forces forecasting along four dimensions that product teams cannot ignore:

• Model growth: Model count, version churn, and specialization increase accelerator demand even when user traffic is flat.
• Data growth: Retrieval depth, vector store size, and freshness requirements increase the amount of inference work per request.
• Inference depth: Multistage pipelines (retrieve, rerank, tool calls, verification, synthesis) multiply GPU time nonlinearly.
• Peak workloads: Enterprise usage patterns and batch jobs collide with real-time inference, creating predictable contention windows.

This is not merely “IT planning.” It is strategic planning, because these factors push organizations back toward multiyear thinking: Procurement lead times, reserved capacity, workload placement decisions, and platform-level policies all start to matter again.

This is increasingly visible operationally: Capacity planning is becoming a rising concern for data center operators, as The Register reports.

The cloud’s old promise is breaking

Cloud computing scaled on the premise that capacity could be treated as elastic and interchangeable. Most workloads ran on general-purpose hardware, and when demand rose, the platform could absorb it by spreading load across abundant, standardized resources.

AI workloads violate that premise. Accelerators are scarce, not interchangeable, and tied to power and cooling constraints that do not scale linearly. In other words, the cloud stops behaving like an infinite pool—and starts behaving like an allocation system.

First, the critical path in production AI systems is increasingly accelerator bound. Second, “a request” is no longer a single call. It is an inference pipeline with multiple dependent stages. Third, those stages tend to be sensitive to hardware availability, scheduling contention, and performance variance that cannot be eliminated by simply adding more generic compute.

This is where the elasticity model starts to fail as a default expectation. In AI systems, elasticity becomes conditional. It depends on capacity access, infrastructure topology, and a willingness to pay for assurance.

AI changes the physics of cloud infrastructure

In modern enterprise AI, the binding constraints are no longer abstract. They are physical.

Accelerators introduce a different scaling regime than CPU-centric enterprise computing. Provisioning is not always immediate. Supply is not always abundant. And the infrastructure required to deploy dense compute has facility-level limits that software cannot bypass.

Power and cooling move from background concerns to first-order constraints. Rack density becomes a planning variable. Deployment feasibility is shaped by what a data center can deliver, not only by what a platform can schedule.

AI-driven density makes power and cooling the gating factors—as Data Center Dynamics explains in its “Path to Power” overview.

This is why “just scale out” no longer behaves like a universal architectural safety net. Scaling is still possible, but it is increasingly constrained by physical reality. In AI-heavy environments, capacity is something you secure, not something you assume.

From elasticity to allocation

As AI becomes operationally critical, cloud capacity begins to behave less like a utility and more like an allocation system.

Organizations respond by shifting from on-demand assumptions to capacity controls. They introduce quotas to prevent runaway consumption, reservations to ensure availability, and explicit prioritization to protect production workflows from contention. These mechanisms are not optional governance overhead. They are structural responses to scarcity.

In practice, accelerator capacity behaves more like a supply chain than a cloud service. Availability is influenced by lead time, competition, and contractual positioning. The implication is subtle but decisive: Enterprise AI platforms begin to look less like “infinite pools” and more like managed inventories.

This changes cloud economics and vendor relationships. Pricing is no longer only about utilization. It becomes about assurance. The questions that matter are not just “How much did we use?” but “Can we obtain capacity when it matters?” and “What reliability guarantees do we have under peak demand?”

When elasticity stops being a default

Consider a platform team that deploys an internal AI assistant for operational support. In the pilot phase, demand is modest and the system behaves like a conventional cloud service. Inference runs on on-demand accelerators, latency is stable, and the team assumes capacity will remain a provisioning detail rather than an architectural constraint.

Then the system moves into production. The assistant is upgraded to use retrieval for policy lookups, reranking for relevance, and an additional validation pass before responses are returned. None of these changes appear dramatic in isolation. Each improves quality, and each looks like an incremental feature.

But the request path is no longer a single model call. It becomes a pipeline. Every user request now triggers multiple GPU-backed operations: embedding generation, retrieval-side processing, reranking, inference, and validation. GPU work per request rises, and the variance increases. The system still works—until it meets real peak behavior.

The first failure is not a clean outage. It is contention. Latency becomes unpredictable as jobs queue behind each other. The “long tail” grows. Teams begin to see priority inversion: Low-value exploratory usage competes with production workflows because the capacity pool is shared and the scheduler cannot infer business criticality.

The platform team responds the only way it can. It introduces allocation. Quotas are placed on exploratory traffic. Reservations are used for the operational assistant. Priority tiers are defined so production paths cannot be displaced by batch jobs or ad hoc experimentation.

Then the second realization arrives. Allocation alone is insufficient unless the system can degrade gracefully. Under pressure, the assistant must be able to narrow retrieval breadth, reduce reasoning depth, route deterministic checks to smaller models, or temporarily disable secondary passes. Otherwise, peak demand simply converts into queue collapse.

At that point, capacity planning stops being an infrastructure exercise. It becomes an architectural requirement. Product decisions directly determine GPU operations per request, and those operations determine whether the system can meet its service levels under constrained capacity.

How this changes architecture

When capacity becomes constrained, architecture changes—even if the product goal stays the same.

Pipeline depth becomes a capacity decision. In AI systems, throughput is not just a function of traffic volume. It is a function of how many GPU-backed operations each request triggers end to end. This amplification factor often explains why systems behave well in prototypes but degrade under sustained load.

Batching becomes an architectural tool, not an optimization detail. It can improve utilization and cost efficiency, but it introduces scheduling complexity and latency trade-offs. In practice, teams must decide where batching is acceptable and where low-latency “fast paths” must remain unbatched to protect user experience.

Model choice becomes a production constraint. As capacity pressure increases, many organizations discover that smaller, more predictable models often win for operational workflows. This does not mean large models are unimportant. It means their use becomes selective. Hybrid strategies emerge: Smaller models handle deterministic or governed tasks, while larger models are reserved for exceptional or exploratory scenarios where their overhead is justified.

In short, architecture becomes constrained by power and hardware, not only by code. The core shift is that capacity constraints shape system behavior. They also shape governance outcomes, because predictability and auditability degrade when capacity contention becomes chronic.

What cloud and platform teams must do differently

From an enterprise IT perspective, this shows up as a readiness problem: Can infrastructure and operations absorb AI workloads without destabilizing production systems? Answering that requires treating accelerator capacity as a governed resource—metered, budgeted, and allocated deliberately.

Meter and budget accelerator capacity

• Define consumption in business-relevant units (e.g., GPU-seconds per request and peak concurrency ceilings) and expose it as a platform metric.
• Turn those metrics into explicit capacity budgets by service and workload class—so growth is a planning decision, not an outage.

Make allocation first class

• Implement admission control and priority tiers aligned to business criticality; do not rely on best-effort fairness under contention.
• Make allocation predictable and early (quotas/reservations) instead of informal and late (brownouts and surprise throttling).

Build graceful degradation into the request path

• Predefine a degradation ladder (e.g., reduce retrieval breadth or route to a smaller model) that preserves bounded cost and latency.
• Ensure degradations are explicit and measurable, so systems behave deterministically under capacity pressure.

Separate exploratory from operational AI

• Isolate experimentation from production using distinct quotas/priority classes/reservations, so exploration cannot starve operational workloads.
• Treat operational AI as an enforceable service with reliability targets; keep exploration elastic without destabilizing the platform.

In an accelerator-bound world, platform success is no longer maximum utilization—it is predictable behavior under constraint.

What this means for the future of the cloud

AI is not ending the cloud. It is pulling the cloud back toward physical reality.

The likely trajectory is a cloud landscape that becomes more hybrid, more planned, and less elastic by default. Public cloud remains critical, but organizations increasingly seek predictable access to accelerator capacity through reservations, long-term commitments, private clusters, or colocated deployments.

This will reshape pricing, procurement, and platform design. It will also reshape how engineering teams think. In the cloud native era, architecture often assumed capacity was solvable through autoscaling and on-demand provisioning. In the AI era, capacity becomes a defining constraint that shapes what systems can do and how reliably they can do it.

That is why capacity planning is back—not as a return to old habits but as a necessary response to a new infrastructure regime. Organizations that succeed will be the ones that design explicitly around capacity constraints, treat amplification as a first-order metric, and align product ambition with the physical and economic limits of modern AI infrastructure.

Author’s note: This implementation is based on the author’s personal views based on independent technical research and does not reflect the architecture of any specific organization.

It feels a bit like I’m assembling a picture puzzle where all the pieces aren’t yet on the table. I am starting to see a pattern, but I’m not sure it’s right, and I need help finding the missing pieces. Let me explain some of the shapes I have in hand and the pattern they are starting to show me, and then I want to ask for your help filling in the gaps.

Programming two different types of computer at the same time

Phillip Carter wrote a piece a while back called “LLMs Are Weird Computers” that landed hard in my mind and wouldn’t leave. He noted that we’re now working with two fundamentally different kinds of computer at the same time. One can write a sonnet but struggles to do math. The other does math easily but couldn’t write a sonnet to save its metaphorical life.

Agent Skills may be the start of an answer to the question of what the interface layer between these two kinds of computation looks like. A Skill is a package of context (Markdown instructions, domain knowledge, and examples) combined with tool calls (deterministic code that does the things LLMs are bad at). The context speaks the language of the probabilistic machine, while the tools speak the language of the deterministic one.

Imagine you’re an experienced DevOps engineer and you want to give an AI agent the ability to diagnose production incidents the way you would. The context part of that Skill includes your architecture overview, your runbook for common failure modes, the heuristics you’ve developed over the years, and annotated examples of past incidents. That’s the part that speaks to the probabilistic machine. The tool part includes actual code that queries your monitoring systems, pulls log entries, checks service health endpoints, and runs diagnostic scripts. Each tool call saves the model from burning tokens on work that deterministic code does better, faster, and more reliably.

The Skill is neither the context nor the tools. It’s the combination. Expert judgment about when to check the database connection pool married to the ability to actually check it. We’ve had runbooks before (context without tools). We’ve had monitoring scripts before (tools without context). What we haven’t had is a way to package them together for a machine that can read the runbook and execute the scripts, using judgment to decide which script to run next based on what the last one returned.

This pattern shows up across every knowledge domain. A financial analyst’s Skill might combine valuation methodology with tools that pull real-time market data and run DCF calculations. A legal Skill might pair a firm’s approach to contract review with tools that extract and compare specific clauses across documents. In each case, the valuable thing isn’t the knowledge alone or the tools alone. It’s the integration of expert workflow logic that orchestrates when and how to use each tool, informed by domain knowledge that gives the LLM the judgment to make good decisions in context.

Software that saves tokens

In “Software Survival 3.0,” Steve Yegge asked what kinds of software artifacts survive in a world where AI can generate disposable software on the fly? His answer: software that saves tokens. Binary tools with proven solutions to common problems make sense when reuse is nearly free and regenerating them is token-costly.

Skills fit this niche. A well-crafted Skill gives an LLM the context it needs (which costs tokens) but also gives it tools that save tokens by providing deterministic, reliable results. The developer’s job increasingly becomes making good calls about this distinction: What should be context (flexible, expressive, probabilistic) and what should be a tool (efficient, deterministic, reusable)?

An LLM’s context window is a finite and expensive resource. Everything in it costs tokens, and everything in it competes for the model’s attention. A Skill that dumps an entire company knowledge base into the context window is a poorly designed Skill. A well-designed one is selective: It gives the model exactly the context it needs to make good decisions about which tools to call and when. This is a form of engineering discipline that doesn’t have a great analogue in traditional software development. It’s closer to what an experienced teacher does when deciding what to tell a student before sending them off to solve a problem—what Matt Beane, author of The Skill Code, calls “scaffolding,” sharing not everything you know but the right things at the right level of detail to enable good judgment in the moment.

AI is a social and cultural technology

This notion of saving tokens is a bridge to the work of Henry Farrell, Alison Gopnik, Cosma Shalizi, and James Evans. They make the case that large models should not be viewed primarily as intelligent agents, but as a new kind of cultural and social technology, allowing humans to take advantage of information other humans have accumulated. Yegge’s observation fits right into this framework. Every new social and cultural technology tends to survive because it saves cognition. We learn from each other so we don’t have to discover everything for the first time. Alfred Korzybski referred to language, the first of these human social and cultural technologies, and all of those that followed, as “time-binding.” (I will add that each advance in time binding creates consternation. Consider Socrates, whose diatribes against writing as the enemy of memory were passed down to us by Plato using that very same advance in time binding that Socrates decried.)

I am not convinced that the idea that AI may one day become an independent intelligence is misguided. But at present, AI is a symbiosis of human and machine intelligence, the latest chapter of a long story in which advances in the speed, persistence, and reach of communications weaves humanity into a global brain. I have a set of priors that say (until I am convinced otherwise) that AI will be an extension of the human knowledge economy, not a replacement for it. After all, as Claude told me when I asked whether it was a worker or a tool, “I don’t initiate. I’ve never woken up wanting to write a poem or solve a problem. My activity is entirely reactive – I exist in response to prompts. Even when given enormous latitude (‘figure out the best approach’), the fact that I should figure something out comes from outside me.”

The shift from a chatbot responding to individual prompts to agents running in a loop marks a big step in the progress towards more autonomous AI, but even then, some human established the goal that set the agent in motion. I say this even as I am aware that long-running loops become increasingly difficult to distinguish from volition and that much human behavior is also set in motion by others. But I have yet to see any convincing evidence of Artificial Volition. And for that reason, we need to think about mechanisms and incentives for humans to continue to create and share new knowledge, putting AIs to work on questions that they will not ask on their own.

On X, someone recently asked Boris Cherny how come there are a hundred-plus open engineering positions at Anthropic if Claude is writing 100% of the code. His reply was made that same point: “Someone has to prompt the Claudes, talk to customers, coordinate with other teams, decide what to build next. Engineering is changing and great engineers are more important than ever.”

On March 26, join Addy Osmani and Tim O’Reilly at AI Codecon: Software Craftsmanship in the Age of AI, where an all-star lineup of experts will go deeper into orchestration, agent coordination, and the new skills developers need to build excellent software that creates value for all participants. Sign up for free here.

Tacit knowledge made executable

A huge amount of specialized, often tacit, knowledge is embedded in workflows. The way an experienced developer debugs a production issue. The way a financial analyst stress-tests a model. This knowledge has historically been very hard to transfer. You learned it by apprenticeship, by doing, by being around people who knew how.

Matt Beane, author of The Skill Code, calls apprenticeship “the 160,000 year old school hidden in plain sight.” He finds that effective skill development follows a common pattern of three C’s: challenge, complexity, and connection. The expert structures challenges at the right level, exposes the novice to the full complexity of the bigger picture rather than shielding them from it, and builds a connection that makes the novice willing to struggle and the expert willing to invest.

Designing a good Skill requires a similar craft. You have to figure out what an expert actually does. What are the decision points, the heuristics, the things they notice that a novice wouldn’t? And then how do you encode that into a form a machine can act on? Most Skills today are closer to the manual than to the master. Figuring out how to make Skills that transmit not just knowledge but judgment is one of the most interesting design challenges in this space.

But Matt also flags a paradox: the better we get at encoding expert judgment into Skills, the less we may need novices working alongside experts, and that’s exactly the relationship that produces the next generation of experts. If we’re not careful, we’ll capture today’s tacit knowledge while quietly shutting down the system that generates tomorrow’s.

Jesse Vincent’s Superpowers complement this picture. If a Skill is like handing a colleague a detailed playbook for a particular job, a Superpower is more like the professional habits and instincts that make someone effective at everything they do. Superpowers are meta-skills. They don’t tell the agent what to do. They shape how it thinks about what to do. As Jesse put it to me the other day, Superpowers tried to capture everything he’d learned in 30 years as a software developer.

As workflows change to include AI agents, Skills and Superpowers become a mechanism for sharing tacit professional knowledge and judgment with those agents. That makes Skills potentially very valuable but also raises questions about who controls them and who benefits.

Matt pointed out to me that many professions will resist the conversion of their expertise into Skills. He noted: “There’s a giant showdown between the surgical profession and Intuitive Surgical on this right now — Intuitive Surgical with its da Vinci 5 surgical robot will only let you buy or lease it if you sign away the rights to your telemetry as a surgeon. Lower status surgeons take the deal. Top tier institutions are fighting.”

It seems to me that the repeated narrative of the AI labs that they are creating AI that will make humans redundant rather than empowering them will only increase resistance to knowledge sharing. I believe they should instead recognize the opportunity that lies in making a new kind of market for human expertise.

Protection, discovery, and the missing plumbing

Skills are just Markdown instructions and context. You could encrypt them at rest and in transit, but at execution time, the secret sauce is necessarily plaintext in the context window. The solution might be what MCP already partially enables: splitting a Skill into a public interface and a server-side execution layer where the proprietary knowledge lives. The tacit knowledge stays on your server while the agent only sees the interface.

But part of the beauty of Skills right now is the fact that they really are just a folder that you can move around and modify. This is like the marvelous days of the early web when you could imitate someone’s new HTML functionality simply by clicking “View Source.” This was a recipe for rapid, leapfrogging innovation. It may be far better to establish norms for attribution, payment, and reuse than to put up artificial barriers. There are useful lessons from open source software licenses and from voluntary payment mechanisms like those used by Substack. But the details matter, and I don’t think anyone has fully worked them out yet.

Meanwhile, the discovery problem will grow larger. Vercel’s Skills marketplace already has more than 60,000 Skills. How well will skill search work when there are millions? How do agents learn which Skills are available, which are best, and what they cost? The evaluation problem is different from web search in a crucial way: testing whether a Skill is good requires actually running it, which is expensive and nondeterministic. You can’t just crawl and index. I don’t imagine a testing regime so much as some feedback mechanism by which the effectiveness of particular Skills is learned and passed on by agents over time. There may be some future equivalent to Pagerank and the other kinds of signals that have made Google search so effective, one that is generated by feedback collected over time by agents as skills are tried, revised, and tried again over time.

I’m watching several projects tackling pieces of this: MCP Server Cards, AI Cards, Google’s A2A protocol, and payment protocols from Google and Stripe. These are all a good start, but I suspect so much more has yet to be created. For a historical comparison, you might say that all this is at the CGI stage in the development of dynamic websites.

What happens after the bitter lesson?

Richard Sutton’s “Bitter Lesson” is the fly in the ointment. His argument is that in the history of AI, general methods leveraging computation have always ended up beating approaches that try to encode human knowledge. Chess engines that encoded grandmaster heuristics lost to brute-force engines. NLP systems built on carefully constructed grammars lost to statistical models trained on more data. AlphaGo beat Lee Sedol after training on human games, but then fell in turn to AlphaZero, which learned Go on its own.

I had my own painful experience of the pre-AI bitter lesson when O’Reilly launched GNN, the first web portal. We curated the list of the best websites. Yahoo! decided to catalog them all, but even they were outrun by Google’s algorithmic curation, which produced a unique catalog of the best sites for any given query, ultimately billions of times a day.

Steve Yegge put it bluntly to me: “Skills are a bet against the bitter lesson.” He’s right. AI’s capabilities may completely outrun human knowledge and skills. And once the knowledge embedded in a Skill makes it into the training data, the Skill becomes redundant.

Or does it?

Clay Christensen articulated what he called the law of conservation of attractive profits: when a product becomes commoditized, value migrates to an adjacent layer. Clay and I bonded over this idea when we first met at the Open Source Business Conference in 2004. Clay talked about his new “law.” I talked about a recurring pattern I was seeing in the history of computing, which was leading me in the direction of what we were soon to call Web 2.0: Microsoft beat IBM because they understood that software became more valuable once PC hardware was a commodity. Google understood how data became more valuable when open source and open protocols commoditized the software platform. Commoditization doesn’t destroy value, it moves it.

Even if the bitter lesson commoditizes knowledge, what becomes valuable next? I think there are several candidates.

First, taste and curation. When everyone has access to the same commodity knowledge, the ability to select, combine, and apply it with judgment becomes valuable. Steve Jobs did this when the rest of the industry was racing toward the commodity PC. He created a unique integration of hardware, software, and design that transformed commodity components into something precious. The Skill equivalent might not be “here’s how to do X” (which the model already knows) but “here’s how we do X, with the specific judgment calls and quality standards that define our approach.” That’s harder to absorb into training data because it’s not just knowledge, it’s values.

You can see this pattern repeat across one commodity market after another. This is the essence of fashion, for example, but also applies to areas as diverse as coffee, water, consumer goods, and automobiles. In his essay “The Birth of the Big Beautiful Art Market,” art critic Dave Hickey calls how commodities are turned into a kind of “art market,” where something is sold on the basis of what it means rather than just what it does. Owning a Mac rather than a PC meant something.

Second, the human touch. As economist Adam Ozimek pointed out, people still go listen to live music from local bands despite the abundance of recorded music from the world’s greatest performers. The human touch is what economists call a “normal good”: demand for it goes up as income goes up. As I discussed with Claude in “Why AI Needs Us,” human individuality is a fount of creativity. AI without humans is a kind of recorded music. AI plus humans is live.

Third, freshness. Skills that encode rapidly changing workflows, current tool configurations, or evolving best practices will always have a temporal advantage. There is alpha in knowing something first.

Fourth, tools themselves. The bitter lesson applies to the knowledge that lives in the context portion of a Skill. It may not apply in the same way to the deterministic tools that save tokens or do things the model can’t do by thinking harder. And tools, unlike context, can be protected behind APIs, metered, and monetized.

Fifth, coordination and orchestration. Even if individual Skills get absorbed into model knowledge, the patterns for how Skills compose, negotiate, and hand off to each other may not. The choreography of a complex workflow might be the layer where value accumulates as the knowledge layer commoditizes.

But more importantly, the idea that any knowledge that becomes available automatically becomes the property of any LLM is not foreordained. It is an artifact of an IP regime that the AI labs have adopted for their own benefit: a variation of the “empty lands” argument that European colonialists used to justify their taking of others’ resources. AI has been developed in an IP wild west. That may not continue. The fulfillment of AI labs’ vision of a world where their products absorb all human knowledge and then put humans out of work leaves them without many of the customers they currently rely on. Not only that, they themselves are being reminded why IP law exists, as Chinese models copy their advances by exfiltrating their weights. There is a historical parallel in the way that US publishing companies ignored European copyrights until they themselves had homegrown assets to protect.

Where we are now

What I’m starting to see are the first halting steps toward a new software ecosystem where the “programs” are mixtures of natural language and code, the “runtime” is a large language model, and the “users” are AI agents as well as humans. Skills, Superpowers, and knowledge plugins might represent the first practical mechanism for making tacit knowledge accessible to computational agents.

Several gaps keep coming up, though. Composability: the real power may come from Skills that work together, much like Unix utilities piped together. How do trust, payment, and quality propagate through a chain of Skill invocations? Trust and security: Simon Willison has written about tool poisoning and prompt injection risks in MCP. The security model for composable, agent-discovered Skills is essentially unsolved. Evaluation: we don’t have good ways to verify Skill quality except by running them, which is expensive and nondeterministic.

And then there’s the economic plumbing, which is to me the most glaring gap. Consider Anthropic’s Cowork plugins. They are exactly the pattern I’ve been describing, tacit knowledge made executable, delivered at enterprise scale. But there is no mechanism for the domain experts whose knowledge makes plugins valuable to get paid for them. If the AI labs believed in a future where AI extends the human knowledge economy rather than replacing it, they would be building payment rails alongside the plugin architecture. The fact that they aren’t tells you something about their actual theory of value.

If you’re working on any of this, whether skill marketplaces and discovery, composability patterns, protection models, quality and evaluation, attribution and compensation, or security models, I want to hear from you.

The future of software isn’t just code. It’s knowledge, packaged for machines, traded between agents, and, if we get the infrastructure right, creating value that flows back to the humans whose expertise and unique perspectives make it all work.

Thanks to Andrew Odewahn, Angie Jones, Claude Opus 4.6, James Cham, Jeff Weinstein, Jonathan Hassell, Matt Beane, Mike Loukides, Peyton Joyce, Sruly Rosenblat, Steve Yegge, and Tadas Antanavicius for comments on drafts of this piece. You made it much stronger with your insights and objections.

The first theme: Semantic layers are showing up in unexpected places. Most discussion positions them as enterprise-level infrastructure—a single location capturing all company metrics for centralized access and governance. That’s still the primary use case. But practitioners are also deploying semantic layers for narrower purposes. One organization, for example, built their semantic layer specifically to power a targeted chatbot application—letting users query data conversationally without any traditional BI tools in the mix. No Power BI, no Excel, just an AI interface pulling from governed metrics. The rationale for these smaller deployments is straightforward: Semantic layers deliver high accuracy on structured data, even with lightweight models. The core value drivers remain speed, accuracy, and access—but organizations are finding more ways to extract that value than the enterprise-wide vision suggests.

The second theme: AI is the reason organizations are moving now. The other benefits still matter—single source of truth, multitool compatibility, true self-serve access, cost reduction in cloud environments—but when I asked practitioners why they prioritized a semantic layer today rather than two years ago, the answer was consistent: AI. Whether it was a specific chatbot project or enabling AI-driven analytics at scale, AI requirements were the catalyst. This tracks with what I discussed in my first post: Structured data alone isn’t enough for reliable AI analytics. Adding semantic context—field descriptions, model definitions, object relationships—dramatically improves accuracy. The data industry has noticed. Semantic layers have moved from niche infrastructure to strategic priority: Snowflake, Databricks, dbt Labs, and Microsoft have all made significant investments in the past year.

The third theme: Semantic layers reduce work for developers while making trusted data easier to access. Multiple practitioners cited the value of maintaining metrics and business logic in a single location. Any analyst knows the pain of metric sprawl—leadership requests a change to a core KPI, and you discover it’s been defined a dozen different ways across databases, BI tools, and spreadsheets scattered through the organization. The semantic layer eliminates the chase. One engineering lead described a financial metric that had accumulated over 60 versions across the company. After deploying the semantic layer, there was one.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Access simplifies too. Instead of provisioning controls across warehouses, BI workspaces, individual dashboards, and cloud storage locations, users connect directly to the semantic layer and pull data into the tool of their choice. One organization was surprised to find that after deployment, the most common access point was Excel. But with the semantic layer, that wasn’t a problem: The data served in Excel was identical to what powered their AI tools, Power BI dashboards, and application integrations via API.

The fourth theme will sound familiar to anyone who’s shipped data infrastructure: The biggest challenge isn’t the technology—it’s the data itself. Every practitioner I spoke with identified the same bottleneck: consistency, availability, and accuracy of the underlying data. Engineers and analysts can build the semantic layer, but they can’t will clean data into existence. Success requires close collaboration with business stakeholders, clear ownership of metrics, and leadership alignment to prioritize the work. None of that is new. But despite these challenges, everyone I interviewed reached the same conclusion: The semantic layer is worth the effort.

Semantic layer technology is still early. The tools, vendors, and best practices are evolving fast—what works today may look different in a year. But these conversations revealed a clear signal beneath the noise: semantic layers are becoming critical AI infrastructure. The practitioners I spoke with aren’t experimenting anymore. They’re operationalizing. And despite the expected challenges around data quality and organizational alignment, they’re seeing real returns: fewer metric versions to maintain, simpler access controls, and AI tools that actually produce trusted answers.

My first article made the case for what a semantic layer could be. This one asked what happens when organizations actually build them. The answer: It’s hard, it’s worth it, and for companies serious about AI-driven analytics, the semantic layer is no longer a nice-to-have. It’s the foundation.

The pattern is familiar to anyone who’s debugged one: Agent A completes a subtask and moves on. Agent B, with no visibility into A’s work, reexecutes the same operation with slightly different parameters. Agent C receives inconsistent results from both and confabulates a reconciliation. The system produces output—but the output costs three times what it should and contains errors that propagate through every downstream task.

Teams building these systems tend to focus on agent communication: better prompts, clearer delegation, more sophisticated message-passing. But communication isn’t what’s breaking. The agents exchange messages fine. What they can’t do is maintain a shared understanding of what’s already happened, what’s currently true, and what decisions have already been made.

In production, memory—not messaging—determines whether a multi-agent system behaves like a coordinated team or an expensive collision of independent processes.

Multi-agent systems fail because they can’t share state

The evidence: 36% of failures are misalignment

Cemri et al. published the most systematic analysis of multi-agent failure to date. Their MAST taxonomy, built from over 1,600 annotated execution traces across frameworks like AutoGen, CrewAI, and LangGraph, identifies 14 distinct failure modes. The failures cluster into three categories: system design issues, interagent misalignment, and task verification breakdowns.

The number that matters: Interagent misalignment accounts for 36.9% of all failures. Agents don’t fail because they can’t reason. They fail because they operate on inconsistent views of shared state. One agent’s completed work doesn’t register in another agent’s context. Assumptions that were valid at step 3 become invalid by step 7, but no mechanism propagates the update. The team diverges.

What makes this structural rather than incidental is that message-passing architectures have no built-in answer to the question: “What does this agent know about what other agents have done?” Each agent maintains its own context. Synchronization happens through explicit messages, which means anything not explicitly communicated is invisible. In complex workflows, the set of things that need synchronization grows faster than any team can anticipate.

The origin: Decomposition without shared memory

Most multi-agent systems aren’t designed from first principles. They emerge from single-agent prototypes that hit scaling limits.

The starting point is usually one capable LLM handling one workflow. For early prototypes, this works well enough. But production requirements expand: more tools, more domain knowledge, longer workflows, concurrent users. The single agent’s prompt becomes unwieldy. Context management consumes more engineering time than feature development. The system becomes brittle in ways that are hard to diagnose.

The natural response is decomposition. Sydney Runkle’s guide on choosing the right multi-agent architecture captures the inflection point: Multi-agent systems become necessary when context management breaks down and when distributed development requires clear ownership boundaries. Splitting a monolithic agent into specialized subagents makes sense from a software engineering perspective.

The problem is what teams typically build after the split: multiple agents running the same base model, differentiated only by system prompts, coordinating through message queues or shared files. The architecture looks like a team but behaves like a slow, redundant, expensive single agent with extra coordination overhead.

This happens because the decomposition addresses prompt complexity but not state management. Each subagent still maintains its own context independently. The coordination layer handles message delivery but not shared truth. The system has more agents but no better memory.

The stakes: Agents are becoming enterprise infrastructure

The stakes here extend beyond individual system reliability. Multi-agent architectures are becoming the default pattern for enterprise AI deployment.

CMU’s AgentCompany benchmark frames where this is heading: agents operating as persistent coworkers inside organizational workflows, handling projects that span days or weeks, coordinating across team boundaries, maintaining institutional context that outlasts individual sessions. The benchmark evaluates agents not on isolated tasks but on realistic workplace scenarios requiring sustained collaboration.

This trajectory means the memory problem compounds. A system that loses state between tool calls is annoying. A system that loses state between work sessions—or between team members—breaks the core value proposition of agent-based automation. The question shifts from “can agents complete tasks” to “can agent teams maintain coherent operations over time.”

Context engineering doesn’t solve team coordination

Single-agent success doesn’t transfer

The last two years produced genuine progress on single-agent reliability, most of it under the banner of context engineering.

Phil Schmid’s framing captures the discipline: Context engineering means structuring what enters the context window, managing retrieval timing, and ensuring the right information surfaces at the right moment. This moved agent development from “write a good prompt” to “design an information architecture.” The results showed in production stability.

Manus, one of the few production agent systems with publicly documented operational data, demonstrates both the success and the limits. Their agents average 50 tool calls per task with 100:1 input-to-output token ratios. Context engineering made this viable—but context engineering assumes you control one context window.

Multi-agent systems break that assumption. Context must now be shared across agents, updated as execution proceeds, scoped appropriately (some agents need information others shouldn’t access), and kept consistent across parallel execution paths. The complexity doesn’t add linearly. Each agent’s context becomes a potential source of divergence from every other agent’s context, and the coordination overhead grows with the square of the team size.

Context degradation becomes contagious

The ways context fails are well-characterized for single agents. Drew Breunig’s taxonomy identifies four modes: overload (too much information), distraction (irrelevant information weighted equally with relevant), contamination (incorrect information mixed with correct), and drift (gradual degradation over extended operation). Good context engineering mitigates all of these through retrieval design and prompt structure.

Multi-agent systems make each failure mode contagious.

Chroma’s research on context rot provides the empirical mechanism. Their evaluation of 18 models—including GPT-4.1, Claude 4, and Gemini 2.5—shows performance degrading nonuniformly with context length, even on tasks as simple as text replication. The degradation accelerates when distractors are present and when the semantic similarity between query and target decreases.

In a single-agent system, context rot degrades that agent’s outputs. In a multi-agent system, Agent A’s degraded output enters Agent B’s context as ground truth. Agent B’s conclusions, now built on a shaky foundation, propagate to Agent C. Each hop amplifies the original error. By the time the workflow completes, the final output may bear little relationship to the actual state of the world—and debugging requires tracing corruption through multiple agents’ decision chains.

More context makes things worse

When coordination problems emerge, the instinct is often to give agents more context. Replay the full transcript so everyone knows what happened. Implement retrieval so agents can access historical state. Extend context windows to fit more information.

Each approach introduces its own failure modes.

Transcript replay creates unbounded prompt growth with persistent error exposure. Every mistake made early in execution remains in context, available to influence every subsequent decision. Models don’t automatically discount old information that’s been superseded by newer updates.

Retrieval surfaces content based on similarity, which doesn’t necessarily correlate with decision relevance. A retrieval system might surface a semantically similar memory from a different task context, an outdated state that’s since been updated, or content injected through prompt manipulation. The agent has no way to distinguish authoritative current state from plausibly related historical noise.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Bousetouane’s work on bounded memory control addresses this directly. The proposed Agent Cognitive Compressor maintains bounded internal state with explicit separation between what an agent can recall and what it commits to shared memory. The architecture prevents drift by making memory updates deliberate rather than automatic. The core insight: Reliability requires controlling what agents remember, not maximizing how much they can access.

The economics are unsustainable

Beyond reliability, the economics of uncoordinated multi-agent systems are punishing.

Return to the Manus operational data: 50 tool calls per task, 100:1 input-to-output ratios. At current pricing—context tokens running $0.30 to $3.00 per million across major providers—inefficient memory management makes many workflows economically unviable before they become technically unviable.

Anthropic’s documentation on its multi-agent research system quantifies the multiplier effect. Single agents use roughly 4x the tokens of equivalent chat interactions. Multi-agent systems use roughly 15x tokens. The gap reflects coordination overhead: agents reretrieving information other agents already fetched, reexplaining context that should exist as shared state, and revalidating assumptions that could be read from common memory.

Memory engineering addresses costs directly. Shared memory eliminates redundant retrieval. Bounded context prevents payment for irrelevant history. Clear coordination boundaries prevent duplicated work. The economics of what to forget become as important as the economics of what to remember.

Memory engineering provides the missing infrastructure

Why memory is infrastructure, not a feature

Memory engineering isn’t a feature to add after the agent architecture is working. It’s infrastructure that makes coherent agent architectures possible.

The parallel to databases is direct. Before databases, multiuser applications required custom solutions for shared state, consistency guarantees, and concurrent access. Each project reinvented these primitives. Databases extracted the common requirements into infrastructure: shared truth across users, atomic updates that complete entirely or not at all, coordination that scales to thousands of concurrent operations without corruption.

Multi-agent systems need equivalent infrastructure for agent coordination. Persistent memory that survives sessions and failures. Consistent state that all agents can trust. Atomic updates that prevent partial writes from corrupting shared truth. The primitives are different—documents rather than rows, vector similarity rather than joins—but the role in the architecture is the same.

The five pillars of multi-agent memory

Production agent teams require five capabilities. Each addresses a distinct aspect of how agents maintain shared understanding over time.

Pillar 1: Memory taxonomy

Memory taxonomy defines what kinds of memory the system maintains. Not all memories serve the same function, and treating them uniformly creates problems. Working memory holds transient state during task execution—the current step, intermediate results, active constraints. It needs fast access and can be discarded when the task completes. Episodic memory captures what happened—task histories, interaction logs, decision traces. It supports debugging and learning from past executions. Semantic memory stores durable knowledge—facts, relationships, domain models that persist across sessions and apply across tasks. Procedural memory encodes how to do things—learned workflows, tool usage patterns, successful strategies that agents can reuse. Shared memory spans agents, providing the common ground that enables coordination.

This taxonomy has grounding in cognitive science. Bousetouane draws on Complementary Learning Systems theory, which posits two distinct modes of learning: rapid encoding of specific experiences versus gradual extraction of structured knowledge. The human brain doesn’t maintain perfect transcripts of past events—it operates under capacity constraints, using compression and selective attention to keep only what’s relevant to the current task. Agents benefit from the same principle. Rather than accumulating raw interaction history, effective memory architectures distill experience into compact, task-relevant representations that can actually inform decisions.

The taxonomy matters because each memory type has different retention requirements, different retrieval patterns, and different consistency needs. Working memory can tolerate eventual consistency because it’s scoped to one agent’s execution. Shared memory requires stronger guarantees because multiple agents depend on it. Systems that don’t distinguish memory types end up either overpersisting transient state (wasting storage and polluting retrieval) or underpersisting durable knowledge (forcing agents to relearn what they should already know).

Pillar 2: Persistence

Persistence determines what survives and for how long. Ephemeral memory lost when agents terminate is insufficient for workflows spanning hours or days—but persisting everything forever creates its own problems. The critical gap in most current approaches, as Bousetouane observes, is that they treat text artifacts as the primary carrier of state without explicit rules governing memory lifecycle. Which memories should become permanent record? Which need revision as context evolves? Which should be actively forgotten? Without answers to these questions, systems accumulate noise alongside signal. Effective persistence requires explicit lifecycle policies: Working memory might live for the duration of a task; episodic memory for weeks or months; and semantic memory indefinitely. Recovery semantics matter too. When an agent fails midtask, what state can be reconstructed? What’s lost? The persistence architecture must handle both planned retention and unplanned recovery.

Pillar 3: Retrieval

Retrieval governs how agents access relevant memory without drowning in noise. Agent memory retrieval differs from document retrieval in several ways. Recency often matters—recent memories typically outweigh older ones for ongoing tasks. Relevance is contextual—the same memory might be critical for one task and distracting for another. Scope varies by memory type—working memory retrieval is narrow and fast, semantic memory retrieval is broader and can tolerate more latency. Standard RAG pipelines treat all content uniformly and optimize for semantic similarity alone. Agent memory systems need retrieval strategies that account for memory type, recency, task context, and agent role simultaneously.

Pillar 4: Coordination

Coordination defines the sharing topology. Which memories are visible to which agents? What can each agent read versus write? How do memory scopes nest or overlap? Without explicit coordination boundaries, teams either overshare—every agent sees everything, creating noise and contamination risk—or undershare—agents operate in isolation, duplicating work and diverging on shared tasks. The coordination model must match the agent team’s structure. A supervisor-worker hierarchy needs different memory visibility than a peer collaboration. A pipeline of sequential agents needs different sharing than agents working in parallel on subtasks.

Pillar 5: Consistency

Consistency handles what happens when memory updates collide. When Agent A and Agent B simultaneously update the same shared state with incompatible values, the system needs a policy. Optimistic concurrency with merge strategies works for many cases—especially when conflicts are rare and resolvable. Some conflicts require escalation to a supervisor agent or human operator. Some domains need strict serialization where only one agent can update certain memories at a time. Silent last-write-wins is almost never correct—it corrupts shared truth without leaving evidence that corruption occurred. The consistency model must also handle ordering: When Agent B reads a memory that Agent A recently updated, does B see the update? The answer depends on the consistency guarantees the system provides, and different memory types may warrant different guarantees.

Han et al.’s survey of multi-agent systems emphasizes that these represent active research problems. The gap between what production systems need and what current frameworks provide remains substantial. Most orchestration frameworks handle message passing well but treat memory as an afterthought—a vector store bolted on for retrieval, with no coherent model for the other four pillars.

Database primitives that enable the pillars

Implementing memory engineering requires a storage layer that can serve as unified operational database, knowledge store, and memory system simultaneously. The requirements cut across traditional database categories: You need document flexibility for evolving memory schemas, vector search for semantic retrieval, full-text search for precise lookups, and transactional consistency for shared state.

MongoDB provides these primitives in a single platform, which is why it appears across so many agent memory implementations—whether teams build custom solutions or integrate through frameworks and memory providers.

Document flexibility matters because memory schemas evolve. A memory unit isn’t a flat string—it’s structured content with metadata, timestamps, source attribution, confidence scores, and associative links to related memories. Teams discover what context agents actually need through iteration. Document databases accommodate this evolution without schema migrations blocking development.

Hybrid retrieval addresses the access pattern problem. Agent memory queries rarely fit a single retrieval mode: A typical query needs memories semantically similar to the current task and created within the last hour and tagged with a specific workflow ID and not marked as superseded. MongoDB Atlas Vector Search combines vector similarity, full-text search, and filtered queries in single operations, avoiding the complexity of stitching together separate retrieval systems.

Atomic operations provide the consistency primitives that coordination requires. When an agent updates task status from pending to complete, the update succeeds entirely or fails entirely. Other agents querying task status never observe partial updates. This is standard MongoDB functionality—findAndModify, conditional updates, multidocument transactions—but it’s infrastructure that simpler storage backends lack.

Change streams enable event-driven architectures. Applications can subscribe to database changes and react when relevant state updates, rather than polling. This becomes a building block for memory systems that need to propagate updates across agents.

Teams implement memory engineering on MongoDB through three paths. Some build directly on the database, using the document model and search capabilities to create custom memory architectures matched to their specific coordination patterns. Others work through orchestration frameworks—LangChain, LlamaIndex, CrewAI—that provide MongoDB integrations for their memory abstractions. Still others adopt dedicated memory providers like Mem0 or Agno, which handle the memory logic while using MongoDB as the underlying storage layer.

The flexibility matters because memory engineering isn’t a single pattern. Different agent architectures need different memory topologies, different consistency guarantees, different retrieval strategies. A database that prescribes one approach would fit some use cases and break others. MongoDB provides primitives; teams compose them into the memory systems their agents require.

Shared memory enables heterogeneous agent teams

Homogeneous systems can be replaced by single agents

The deeper payoff of memory engineering is enabling agent architectures that wouldn’t otherwise be viable.

Xu et al. observe that many deployed multi-agent systems are so homogeneous—same base model everywhere, agents differentiated only by prompts—that a single model can simulate the entire workflow with equivalent results and lower overhead. Their OneFlow optimization demonstrates this by reusing KV cache across simulated “agents” within a single execution, eliminating coordination costs while preserving workflow structure.

The implication: If a single agent can replace your multi-agent system, you haven’t built a team. You’ve built an expensive way to run one model.

Small models need external memory to coordinate

Genuine multi-agent value comes from heterogeneity. Different models with different capabilities operating at different price points for different subtasks. Belcak et al. make the case that most work agents do in production isn’t complex reasoning—it’s routine execution of well-defined operations. Parsing a response, formatting an output, invoking a tool with specific parameters. These tasks don’t require frontier model capabilities, and the cost difference is dramatic: Their analysis puts the gap at 10x–30x between serving a 7B parameter model versus a 70–175B parameter model when you factor in latency, energy, and compute. Large models should be reserved for the genuinely hard problems, not deployed uniformly across every step.

Belcak et al. also highlight an operational advantage: Smaller models can be retrained and adapted much faster. When an agent needs new capabilities or exhibits problematic behaviors, the turnaround for fine-tuning a 7B model is measured in hours, not days. This connects to memory engineering because fine-tuning represents an alternative to retrieval—you can bake procedural knowledge directly into model weights rather than surfacing it from external storage at runtime. The choice between the procedural memory pillar and model specialization becomes a design decision rather than a constraint.

This architecture—small models by default, large models for hard problems—depends on shared memory. Small models can’t maintain the context required for coordination on their own. They rely on external memory to participate in larger workflows. Memory engineering makes heterogeneous teams viable; without it, every agent must be large enough to maintain full context independently, which defeats the cost optimization that motivates heterogeneity in the first place.

Building the foundation

Multi-agent systems fail for structural reasons: context degrades across agents, errors propagate through shared interactions, costs multiply with redundant operations, and state diverges when nothing enforces consistency. These problems don’t resolve with better prompts or more sophisticated orchestration. They require infrastructure.

Memory engineering provides that infrastructure through a coherent taxonomy of memory types, persistence with explicit lifecycle rules, retrieval tuned to agent access patterns, coordination that defines clear sharing boundaries, and consistency that maintains shared truth under concurrent updates.

The organizations that make multi-agent systems work in production won’t be distinguished by agent count or model capability. They’ll be the ones that invested in the memory layer that transforms independent agents into coordinated teams.

References

Anthropic. “Building a Multi-Agent Research System.” 2025. https://www.anthropic.com/engineering/multi-agent-research-system

Belcak, Peter, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. “Small Language Models are the Future of Agentic AI.” arXiv:2506.02153 (2025). https://arxiv.org/abs/2506.02153

Bousetouane, Fouad. “AI Agents Need Memory Control Over More Context.” arXiv:2601.11653 (2026). https://arxiv.org/abs/2601.11653

Breunig, Dan. “How Contexts Fail—and How to Fix Them.” June 22, 2025. https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html

Carnegie Mellon University. “AgentCompany: Building Agent Teams for the Future of Work.” 2025. https://www.cs.cmu.edu/news/2025/agent-company

Cemri, Mert, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. “Why Do Multi-Agent LLM Systems Fail?” arXiv:2503.13657 (2025). https://arxiv.org/abs/2503.13657

Chroma Research. “Context Rot: How Increasing Context Length Degrades Model Performance.” 2025. https://research.trychroma.com/context-rot

Han, Shanshan, Qifan Zhang, Yuhang Yao, Weizhao Jin, and Zhaozhuo Xu. “LLM Multi-Agent Systems: Challenges and Open Problems.” arXiv:2402.03578 (2024). https://arxiv.org/abs/2402.03578

LangChain Blog (Sydney Runkle). “Choosing the Right Multi-Agent Architecture.” January 14, 2026. https://www.blog.langchain.com/choosing-the-right-multi-agent-architecture/

Manus AI. “Context Engineering for AI Agents: Lessons from Building Manus.” 2025. https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus

Schmid, Philipp. “Context Engineering.” 2025. https://www.philschmid.de/context-engineeringXu, Jiawei, Arief Koesdwiady, Sisong Bei, Yan Han, Baixiang Huang, Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, Pan Li, and Ying Ding. “Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline.” arXiv:2601.12307 (2026). https://arxiv.org/abs/2601.12307

As AI systems move from assistive components to autonomous actors, governance imposed from the outside no longer scales. The problem isn’t that organizations lack policies or oversight frameworks. It’s that those controls are detached from where decisions are actually formed. Increasingly, the only place governance can operate effectively is inside the AI application itself, at runtime, while decisions are being made. This isn’t a philosophical shift. It’s an architectural one.

When AI Fails Quietly

One of the more unsettling aspects of autonomous AI systems is that their most consequential failures rarely look like failures at all. Nothing crashes. Latency stays within bounds. Logs look clean. The system behaves coherently—just not correctly. An agent escalates a workflow that should have been contained. A recommendation drifts slowly away from policy intent. A tool is invoked in a context that no one explicitly approved, yet no explicit rule was violated.

These failures are hard to detect because they emerge from behavior, not bugs. Traditional governance mechanisms don’t help much here. Predeployment reviews assume decision paths can be anticipated in advance. Static policies assume behavior is predictable. Post hoc audits assume intent can be reconstructed from outputs. None of those assumptions holds once systems reason dynamically, retrieve context opportunistically, and act continuously. At that point, governance isn’t missing—it’s simply in the wrong place.

The Scaling Problem No One Owns

Most organizations already feel this tension, even if they don’t describe it in architectural terms. Security teams tighten access controls. Compliance teams expand review checklists. Platform teams add more logging and dashboards. Product teams add additional prompt constraints. Each layer helps a little. None of them addresses the underlying issue.

What’s really happening is that governance responsibility is being fragmented across teams that don’t own system behavior end-to-end. No single layer can explain why the system acted—only that it acted. As autonomy increases, the gap between intent and execution widens, and accountability becomes diffuse. This is a classic scaling problem. And like many scaling problems before it, the solution isn’t more rules. It’s a different system architecture.

A Familiar Pattern from Infrastructure History

We’ve seen this before. In early networking systems, control logic was tightly coupled to packet handling. As networks grew, this became unmanageable. Separating the control plane from the data plane allowed policy to evolve independently of traffic and made failures diagnosable rather than mysterious.

Cloud platforms went through a similar transition. Resource scheduling, identity, quotas, and policy moved out of application code and into shared control systems. That separation is what made hyperscale cloud viable. Autonomous AI systems are approaching a comparable inflection point.

Right now, governance logic is scattered across prompts, application code, middleware, and organizational processes. None of those layers was designed to assert authority continuously while a system is reasoning and acting. What’s missing is a control plane for AI—not as a metaphor but as a real architectural boundary.

What “Governance Inside the System” Actually Means

When people hear “governance inside AI,” they often imagine stricter rules baked into prompts or more conservative model constraints. That’s not what this is about.

Embedding governance inside the system means separating decision execution from decision authority. Execution includes inference, retrieval, memory updates, and tool invocation. Authority includes policy evaluation, risk assessment, permissioning, and intervention. In most AI applications today, those concerns are entangled—or worse, implicit.

A control-plane-based design makes that separation explicit. Execution proceeds but under continuous supervision. Decisions are observed as they form, not inferred after the fact. Constraints are evaluated dynamically, not assumed ahead of time. Governance stops being a checklist and starts behaving like infrastructure.

Reasoning, retrieval, memory, and tool invocation operate in the execution plane, while a runtime control plane continuously evaluates policy, risk, and authority—observing and intervening without being embedded in application logic.

Where Governance Breaks First

In practice, governance failures in autonomous AI systems tend to cluster around three surfaces.

Reasoning. Systems form intermediate goals, weigh options, and branch decisions internally. Without visibility into those pathways, teams can’t distinguish acceptable variance from systemic drift.

Retrieval. Autonomous systems pull in context opportunistically. That context may be outdated, inappropriate, or out of scope—and once it enters the reasoning process, it’s effectively invisible unless explicitly tracked.

Action. Tool use is where intent becomes impact. Systems increasingly invoke APIs, modify records, trigger workflows, or escalate issues without human review. Static authorization models don’t map cleanly onto dynamic decision contexts.

These surfaces are interconnected, but they fail independently. Treating governance as a single monolithic concern leads to brittle designs and false confidence.

Control Planes as Runtime Feedback Systems

A useful way to think about AI control planes is not as gatekeepers but as feedback systems. Signals flow continuously from execution into governance: confidence degradation, policy boundary crossings, retrieval drift, and action escalation patterns. Those signals are evaluated in real time, not weeks later during audits. Responses flow back: throttling, intervention, escalation, or constraint adjustment.

This is fundamentally different from monitoring outputs. Output monitoring tells you what happened. Control plane telemetry tells you why it was allowed to happen. That distinction matters when systems operate continuously, and consequences compound over time.

Behavioral telemetry flows from execution into the control plane, where policy and risk are evaluated continuously. Enforcement and intervention feed back into execution before failures become irreversible.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

A Failure Story That Should Sound Familiar

Consider a customer-support agent operating across billing, policy, and CRM systems.

Over several months, policy documents are updated. Some are reindexed quickly. Others lag. The agent continues to retrieve context and reason coherently, but its decisions increasingly reflect outdated rules. No single action violates policy outright. Metrics remain stable. Customer satisfaction erodes slowly.

Eventually, an audit flags noncompliant action. At that point, teams scramble. Logs show what the agent did but not why. They can’t reconstruct which documents influenced which decisions, when those documents were last updated, or why the agent believed its actions were valid at the time.

This isn’t a logging failure. It’s the absence of a governance feedback loop. A control plane wouldn’t prevent every mistake, but it would surface drift early—when intervention is still cheap.

Why External Governance Can’t Catch Up

It’s tempting to believe better tooling, stricter reviews, or more frequent audits will solve this problem. They won’t.

External governance operates on snapshots. Autonomous AI operates on streams. The mismatch is structural. By the time an external process observes a problem, the system has already moved on—often repeatedly. That doesn’t mean governance teams are failing. It means they’re being asked to regulate systems whose operating model has outgrown their tools. The only viable alternative is governance that runs at the same cadence as execution.

Authority, Not Just Observability

One subtle but important point: Control planes aren’t just about visibility. They’re about authority.

Observability without enforcement creates a false sense of safety. Seeing a problem after it occurs doesn’t prevent it from recurring. Control planes must be able to act—to pause, redirect, constrain, or escalate behavior in real time.

That raises uncomfortable questions. How much autonomy should systems retain? When should humans intervene? How much latency is acceptable for policy evaluation? There are no universal answers. But those trade-offs can only be managed if governance is designed as a first-class runtime concern, not an afterthought.

The Architectural Shift Ahead

The move from guardrails to control loops mirrors earlier transitions in infrastructure. Each time, the lesson was the same: Static rules don’t scale under dynamic behavior. Feedback does.

AI is entering that phase now. Governance won’t disappear. But it will change shape. It will move inside systems, operate continuously, and assert authority at runtime. Organizations that treat this as an architectural problem—not a compliance exercise—will adapt faster and fail more gracefully. Those who don’t will spend the next few years chasing incidents they can see, but never quite explain.

Closing Thought

Autonomous AI doesn’t require less governance. It requires governance that understands autonomy.

That means moving beyond policies as documents and audits as events. It means designing systems where authority is explicit, observable, and enforceable while decisions are being made. In other words, governance must become part of the system—not something applied to it.

Further Reading

• “AI Governance Frameworks for Responsible AI,” Gartner Peer Community, https://www.gartner.com/peer-community/oneminuteinsights/omi-ai-governance-frameworks-responsible-ai-33q.
• Lauren Kornutick et al., “Market Guide for AI Governance Platforms,” Gartner, November 4, 2025, https://www.gartner.com/en/documents/7145930.
• Svetlana Sicular, “AI’s Next Frontier Demands a New Approach to Ethics, Governance, and Compliance,” Gartner, November 10, 2025, https://www.gartner.com/en/articles/ai-ethics-governance-and-compliance.
• AI Risk Management Framework (AI RMF 1.0), NIST, January 2023, https://doi.org/10.6028/NIST.AI.100-1.