This is the first article in a series on agentic engineering and AI-driven development. Look for the next article on March 19 on O’Reilly Radar.

There’s been a lot of hype about AI and software development, and it comes in two flavors. One says, “We’re all doomed, that tools like Claude Code will make software engineering obsolete within a year.” The other says, “Don’t worry, everything’s fine, AI is just another tool in the toolbox.” Neither is honest.

I’ve spent over 20 years writing about software development for practitioners, covering everything from coding and architecture to project management and team dynamics. For the last two years I’ve been focused on AI, training developers to use these tools effectively, writing about what works and what doesn’t in books, articles, and reports. And I kept running into the same problem: I had yet to find anyone with a coherent answer for how experienced developers should actually work with these tools. There are plenty of tips and plenty of hype but very little structure, and very little you could practice, teach, critique, or improve.

I’d been observing developers at work using AI with various levels of success, and I realized we need to start thinking about this as its own discipline. Andrej Karpathy, the former head of AI at Tesla and a founding member of OpenAI, recently proposed the term “agentic engineering” for disciplined development with AI agents, and others like Addy Osmani are getting on board. Osmani’s framing is that AI agents handle implementation but the human owns the architecture, reviews every diff, and tests relentlessly. I think that’s right.

But I’ve spent a lot of the last two years teaching developers how to use tools like Claude Code, agent mode in Copilot, Cursor, and others, and what I keep hearing is that they already know they should be reviewing the AI’s output, maintaining the architecture, writing tests, keeping documentation current, and staying in control of the codebase. They know how to do it in theory. But they get stuck trying to apply it in practice: How do you actually review thousands of lines of AI-generated code? How do you keep the architecture coherent when you’re working across multiple AI tools over weeks? How do you know when the AI is confidently wrong? And it’s not just junior developers who are having trouble with agentic engineering. I’ve talked to senior engineers who struggle with the shift to agentic tools, and intermediate developers who take to it naturally. The difference isn’t necessarily the years of experience; it’s whether they’ve figured out an effective and structured way to work with AI coding tools. That gap between knowing what developers should be doing with agentic engineering and knowing how to integrate it into their day-to-day work is a real source of anxiety for a lot of engineers right now. That’s the gap this series is trying to fill.

Despite what much of the hype about agentic engineering is telling you, this kind of development doesn’t eliminate the need for developer expertise; just the opposite. Working effectively with AI agents actually raises the bar for what developers need to know. I wrote about that experience gap in an earlier O’Reilly Radar piece called “The Cognitive Shortcut Paradox.” The developers who get the most from working with AI coding tools are the ones who already know what good software looks like, and can often tell if the AI wrote it.

The idea that AI tools work best when experienced developers are driving them matched everything I’d observed. It rang true, and I wanted to prove it in a way that other developers would understand: by building software. So I started building a specific, practical approach to agentic engineering built for developers to follow, and then I put it to the test. I used it to build a production system from scratch, with the rule that AI would write all the code. I needed a project that was complex enough to stress-test the approach, and interesting enough to keep me engaged through the hard parts. I wanted to apply everything I’d learned and discover what I still didn’t know. That’s when I came back to Monte Carlo simulations.

The experiment

I’ve been obsessed with Monte Carlo simulations ever since I was a kid. My dad’s an epidemiologist—his whole career has been about finding patterns in messy population data, which means statistics was always part of our lives (and it also means that I learned SPSS at a very early age). When I was maybe 11 he told me about the drunken sailor problem: A sailor leaves a bar on a pier, taking a random step toward the water or toward his ship each time. Does he fall in or make it home? You can’t know from any single run. But run the simulation a thousand times, and the pattern emerges from the noise. The individual outcome is random; the aggregate is predictable.

I remember writing that simulation in BASIC on my TRS-80 Color Computer 2: a little blocky sailor stumbling across the screen, two steps forward, one step back. The drunken sailor is the “Hello, world” of Monte Carlo simulations. Monte Carlo is a technique for problems you can’t solve analytically: You simulate them hundreds or thousands of times and measure the aggregate results. Each individual run is random, but the statistics converge on the true answer as the sample size grows. It’s one way we model everything from nuclear physics to financial risk to the spread of disease across populations.

What if you could run that kind of simulation today by describing it in plain English? Not a toy demo but thousands of iterations with seeded randomness for reproducibility, where the outputs get validated and the results get aggregated into actual statistics you can use. Or a pipeline where an LLM generates content, a second LLM scores it, and anything that doesn’t pass gets sent back for another try.

The goal of my experiment was to build that system, which I called Octobatch. Right now, the industry is constantly looking for new real-world end-to-end case studies in agentic engineering, and I wanted Octobatch to be exactly that case study.

I took everything I’d learned from teaching and observing developers working with AI, put it to the test by building a real system from scratch, and turned the lessons into a structured approach to agentic engineering I’m calling AI-driven development, or AIDD. This is the first article in a series about what agentic engineering looks like in practice, what it demands from the developer, and how you can apply it to your own work.

The result is a fully functioning, well-tested application that consists of about 21,000 lines of Python across several dozen files, backed by complete specifications, nearly a thousand automated tests, and quality integration and regression test suites. I used Claude Cowork to review all the AI chats from the entire project, and it turns out that I built the entire application in roughly 75 hours of active development time over seven weeks. For comparison, I built Octobatch in just over half the time I spent last year playing Blue Prince.

But this series isn’t just about Octobatch. I integrated AI tools at every level: Claude and Gemini collaborating on architecture, Claude Code writing the implementation, LLMs generating the pipelines that run on the system they helped build. This series is about what I learned from that process: the patterns that worked, the failures that taught me the most, and the orchestration mindset that ties it all together. Each article pulls a different lesson from the experiment, from validation architecture to multi-LLM coordination to the values that kept the project on track.

Agentic engineering and AI-driven development

When most people talk about using AI to write code, they mean one of two things: AI coding assistants like GitHub Copilot, Cursor, or Windsurf, which have evolved well beyond autocomplete into agentic tools that can run multifile editing sessions and define custom agents; or “vibe coding,” where you describe what you want in natural language and accept whatever comes back. These coding assistants are genuinely impressive, and vibe coding can be really productive.

Using these tools effectively on a real project, however, maintaining architectural coherence across thousands of lines of AI-generated code, is a different problem entirely. AIDD aims to help solve that problem. It’s a structured approach to agentic engineering where AI tools drive substantial portions of the implementation, architecture, and even project management, while you, the human in the loop, decide what gets built and whether it’s any good. By “structure,” I mean a set of practices developers can learn and follow, a way to know whether the AI’s output is actually good, and a way to stay on track across the life of a project. If agentic engineering is the discipline, AIDD is one way to practice it.

In AI-driven development, developers don’t just accept suggestions or hope the output is correct. They assign specific roles to specific tools: one LLM for architecture planning, another for code execution, a coding agent for implementation, and the human for vision, verification, and the decisions that require understanding the whole system.

And the “driven” part is literal. The AI is writing almost all of the code. One of my ground rules for the Octobatch experiment was that I would let AI write all of it. I have high code quality standards, and part of the experiment was seeing whether AIDD could produce a system that meets them. The human decides what gets built, evaluates whether it’s right, and maintains the constraints that keep the system coherent.

Not everyone agrees on how much the developer needs to stay in the loop, and the fully autonomous end of the spectrum is already producing cautionary tales. Nicholas Carlini at Anthropic recently tasked 16 Claude instances to build a C compiler in parallel with no human in the loop. After 2,000 sessions and $20,000 in API costs, the agents produced a 100,000-line compiler that can build a Linux kernel but isn’t a drop-in replacement for anything, and when all 16 agents got stuck on the same bug, Carlini had to step back in and partition the work himself. Even strong advocates of a completely hands-off, vibe-driven approach to agentic engineering might call that a step too far. The question is how much human judgment you need to make that code trustworthy, and what specific practices help you apply that judgment effectively.

The orchestration mindset

If you want to get developers thinking about agentic engineering in the right way, you have to start with how they think about working with AI, not just what tools they use. That’s where I started when I began building a structured approach, and it’s why I started with habits. I developed a framework for these called the Sens-AI Framework, published as both an O’Reilly report (Critical Thinking Habits for Coding with AI) and a Radar series. It’s built around five practices: providing context, doing research before prompting, framing problems precisely, iterating deliberately on outputs, and applying critical thinking to everything the AI produces. I started there because habits are how you lock in the way you think about how you’re working. Without them, AI-driven development produces plausible-looking code that falls apart under scrutiny. With them, it produces systems that a single developer couldn’t build alone in the same timeframe.

Habits are the foundation, but they’re not the whole picture. AIDD also has practices (concrete techniques like multi-LLM coordination, context file management, and using one model to validate another’s output) and values (the principles behind those practices). If you’ve worked with Agile methodologies like Scrum or XP, that structure should be pretty familiar: Practices tell you how to work day-to-day, and habits are the reflexes you develop so that the practices become automatic.

Values often seem weirdly theoretical, but they’re an important piece of the puzzle because they guide your decisions when the practices don’t give you a clear answer. There’s an emerging culture around agentic engineering right now, and the values you bring to your project either match or clash with that culture. Understanding where the values come from is what makes the practices stick. All of that leads to a whole new mindset, what I’m calling the orchestration mindset. This series builds all four layers, using Octobatch as the proving ground.

Octobatch was a deliberate experiment in AIDD. I designed the project as a test case for the entire approach, to see what a disciplined AI-driven workflow could produce and where it would break down, and I used it to apply and improve the practices and values to make them effective and easy to adopt. And whether by instinct or coincidence, I picked the perfect project for this experiment. Octobatch is a batch orchestrator. It coordinates asynchronous jobs, manages state across failures, tracks dependencies between pipeline steps, and makes sure validated results come out the other end. That kind of system is fun to design but a lot of the details, like state machines, retry logic, crash recovery, and cost accounting, can be tedious to implement. It’s exactly the kind of work where AIDD should shine, because the patterns are well understood but the implementation is repetitive and error-prone.

Orchestration—the work of coordinating multiple independent processes toward a coherent outcome—evolved into a core idea behind AIDD. I found myself orchestrating LLMs the same way Octobatch orchestrates batch jobs: assigning roles, managing handoffs, validating outputs, recovering from failures. The system I was building and the process I was using to build it followed the same pattern. I didn’t anticipate it when I started, but building a system that orchestrates AI turns out to be a pretty good way to learn how to orchestrate AI. That’s the accidental part of the accidental orchestrator. That parallel runs through every article in this series.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

The path to batch

I didn’t begin the Octobatch project by starting with a full end-to-end Monte Carlo simulation. I started where most people start: typing prompts into a chat interface. I was experimenting with different simulation and generation ideas to give the project some structure, and a few of them stuck. A blackjack strategy comparison turned out to be a great test case for a multistep Monte Carlo simulation. NPC dialogue generation for a role-playing game gave me a creative workload with subjective quality to measure. Both had the same shape: a set of structured inputs, each processed the same way. So I had Claude write a simple script to automate what I’d been doing by hand, and I used Gemini to double-check the work, make sure Claude really understood my ask, and fix hallucinations. It worked fine at small scale, but once I started running more than a hundred or so units, I kept hitting rate limits, the caps that providers put on how many API requests you can make per minute.

That’s what pushed me to LLM batch APIs. Instead of sending individual prompts one at a time and waiting for each response, the major LLM providers all offer batch APIs that let you submit a file containing all of your requests at once. The provider processes them on their own schedule; you wait for results instead of getting them immediately, but you don’t have to worry about rate caps. I was happy to discover they also cost 50% less, and that’s when I started tracking token usage and costs in earnest. But the real surprise was that batch APIs performed better than real-time APIs at scale. Once pipelines got past the 100- or 200-unit mark, batch started running significantly faster than real time. The provider processes the whole batch in parallel on their infrastructure, so you’re not bottlenecked by round-trip latency or rate caps anymore.

The switch to batch APIs changed how I thought about the whole problem of coordinating LLM API calls at scale, and led to the idea of configurable pipelines. I could chain stages together: The output of one step could become the input to the next, and I could kick off the whole pipeline and come back to finished results. It turns out I wasn’t the only one making the shift to batch APIs. Between April 2024 and July 2025, OpenAI, Anthropic, and Google all launched batch APIs, converging on the same pricing model: 50% of the real-time rate in exchange for asynchronous processing.

You probably didn’t notice that all three major AI providers released batch APIs. The industry conversation was dominated by agents, tool use, MCP, and real-time reasoning. Batch APIs shipped with relatively little fanfare, but they represent a genuine shift in how we can use LLMs. Instead of treating them as conversational partners or one-shot SaaS APIs, we can treat them as processing infrastructure, closer to a MapReduce job than a chatbot. You give them structured data and a prompt template, and they process all of it and hand back the results. What matters is that you can now run tens of thousands of these transformations reliably, at scale, without managing rate limits or connection failures.

Why orchestration?

If batch APIs are so useful, why can’t you just write a for-loop that submits requests and collects results? You can, and for simple cases a quick script with a for-loop works fine. But once you start running larger workloads, the problems start to pile up. Solving those problems turned out to be one of the most important lessons for developing a structured approach to agentic engineering.

First, batch jobs are asynchronous. You submit a job, and results come back hours later, so your script needs to track what was submitted and poll for completion. If your script crashes in the middle, you lose that state. Second, batch jobs can partially fail. Maybe 97% of your requests succeeded and 3% didn’t. Your code needs to figure out which 3% failed, extract them, and resubmit just those items. Third, if you’re building a multistage pipeline where the output of one step feeds into the next, you need to track dependencies between stages. And fourth, you need cost accounting. When you’re running tens of thousands of requests, you want to know how much you spent, and ideally, how much you’re going to spend when you first start the batch. Every one of these has a direct parallel to what you’re doing in agentic engineering: keeping track of the work multiple AI agents are doing at once, dealing with code failures and bugs, making sure the entire project stays coherent when AI coding tools are only looking at the one part currently in context, and stepping back to look at the wider project management picture.

All of these problems are solvable, but they’re not problems you want to solve over and over (in both situations—when you’re orchestrating LLM batch jobs or orchestrating AI coding tools). Solving these problems in the code gave some interesting lessons about the overall approach to agentic engineering. Batch processing moves the complexity from connection management to state management. Real-time APIs are hard because of rate limits and retries. Batch APIs are hard because you have to track what’s in flight, what succeeded, what failed, and what’s next.

Before I started development, I went looking for existing tools that handled this combination of problems, because I didn’t want to waste my time reinventing the wheel. I didn’t find anything that did the job I needed. Workflow orchestrators like Apache Airflow and Dagster manage DAGs and task dependencies, but they assume tasks are deterministic and don’t provide LLM-specific features like prompt template rendering, schema-based output validation, or retry logic triggered by semantic quality checks. LLM frameworks like LangChain and LlamaIndex are designed around real-time inference chains and agent loops—they don’t manage asynchronous batch job lifecycles, persist state across process crashes, or handle partial failure recovery at the chunk level. And the batch API client libraries from the providers themselves handle submission and retrieval for a single batch, but not multistage pipelines, cross-step validation, or provider-agnostic execution.

Nothing I found covered the full lifecycle of multiphase LLM batch workflows, from submission and polling through validation, retry, cost tracking, and crash recovery, across all three major AI providers. That’s what I built.

Lessons from the experiment

The goal of this article, as the first one in my series on agentic engineering and AI-driven development, is to lay out the hypothesis and structure of the Octobatch experiment. The rest of the series goes deep on the lessons I learned from it: the validation architecture, multi-LLM coordination, the practices and values that emerged from the work, and the orchestration mindset that ties it all together. A few early lessons stand out, because they illustrate what AIDD looks like in practice and why developer experience matters more than ever.

• You have to run things and check the data. Remember the drunken sailor, the “Hello, world” of Monte Carlo simulations? At one point I noticed that when I ran the simulation through Octobatch, 77.5% of the sailors fell in the water. The results for a random walk should be 50/50, so clearly something was badly wrong. It turned out the random number generator was being re-seeded at every iteration with sequential seed values, which created correlation bias between runs. I didn’t identify the problem immediately; I ran a bunch of tests using Claude Code as a test runner to generate each test, run it, and log the results; Gemini looked at the results and found the root cause. Claude had trouble coming up with a fix that worked well, and proposed a workaround with a large list of preseeded random number values in the pipeline. Gemini proposed a hash-based fix reviewing my conversations with Claude, but it seemed overly complex. Once I understood the problem and rejected their proposed solutions, I decided the best fix was simpler than either of the AI’s suggestions: a persistent RNG per simulation unit that advanced naturally through its sequence. I needed to understand both the statistics and the code to evaluate those three options. Plausible-looking output and correct output aren’t the same thing, and you need enough expertise to tell the difference. (We’ll talk more about this situation in the next article in the series.)
• LLMs often overestimate complexity. At one point I wanted to add support for custom mathematical expressions in the analysis pipeline. Both Claude and Gemini pushed back, telling me, “This is scope creep for v1.0” and “Save it for v1.1.” Claude estimated three hours to implement. Because I knew the codebase, I knew we were already using asteval, a Python library that provides a safe, minimalistic evaluator for mathematical expressions and simple Python statements, elsewhere to evaluate expressions, so this seemed like a straightforward use of a library we’re already using elsewhere. Both LLMs thought the solution would be far more complex and time-consuming than it actually was; it took just two prompts to Claude Code (generated by Claude), and about five minutes total to implement. The feature shipped and made the tool significantly more powerful. The AIs were being conservative because they didn’t have my context about the system’s architecture. Experience told me the integration would be trivial. Without that experience, I would have listened to them and deferred a feature that took five minutes.
• AI is often biased toward adding code, not deleting it. Generative AI is, unsurprisingly, biased toward generation. So when I asked the LLMs to fix problems, their first response was often to add more code, adding another layer or another special case. I can’t think of a single time in the whole project when one of the AIs stepped back and said, “Tear this out and rethink the approach.” The most productive sessions were the ones where I overrode that instinct and pushed for simplicity. This is something experienced developers learn over a career: The most successful changes often delete more than they add—the PRs we brag about are the ones that delete thousands of lines of code.
• The architecture emerged from failure. The AI tools and I didn’t design Octobatch’s core architecture up front. Our first attempt was a Python script with in-memory state and a lot of hope. It worked for small batches but fell apart at scale: A network hiccup meant restarting from scratch, a malformed response required manual triage. A lot of things fell into place after I added the constraint that the system must survive being killed at any moment. That single requirement led to the tick model (wake up, check state, do work, persist, exit), the manifest file as source of truth, and the entire crash-recovery architecture. We discovered the design by repeatedly failing to do something simpler.
• Your development history is a dataset. I just told you several stories from the Octobatch project, and this series will be full of them. Every one of those stories came from going back through the chat logs between me, Claude, and Gemini. With AIDD, you have a complete transcript of every architectural decision, every wrong turn, every moment where you overruled the AI and every moment where it corrected you. Very few development teams have ever had that level of fidelity in their project history. Mining those logs for lessons learned turns out to be one of the most valuable practices I’ve found.

Near the end of the project, I switched to Cursor to make sure none of this was specific to Claude Code. I created fresh conversations using the same context files I’d been maintaining throughout development, and was able to bootstrap productive sessions immediately; the context files worked exactly as designed. The practices I’d developed transferred cleanly to a different tool. The value of this approach comes from the habits, the context management, and the engineering judgment you bring to the conversation, not from any particular vendor.

These tools are moving the world in a direction that favors developers who understand the ways engineering can go wrong and know solid design and architecture patterns…and who are okay letting go of control of every line of code.

What’s next

Agentic engineering needs structure, and structure needs a concrete example to make it real. The next article in this series goes into Octobatch itself, because the way it orchestrates AI is a remarkably close parallel to what AIDD asks developers to do. Octobatch assigns roles to different processing steps, manages handoffs between them, validates their outputs, and recovers when they fail. That’s the same pattern I followed when building it: assigning roles to Claude and Gemini, managing handoffs between them, validating their outputs, and recovering when they went down the wrong path. Understanding how the system works turns out to be a good way to understand how to orchestrate AI-driven development. I’ll walk through the architecture, show what a real pipeline looks like from prompt to results, present the data from a 300-hand blackjack Monte Carlo simulation that puts all of these ideas to the test, and use all of that to demonstrate ideas we can apply directly to agentic engineering and AI-driven development.

Later articles go deeper into the practices and ideas I learned from this experiment that make AI-driven development work: how I coordinated multiple AI models without losing control of the architecture, what happened when I tested the code against what I actually intended to build, and what I learned about the gap between code that runs and code that does what you meant. Along the way, the experiment produced some findings about how different AI models see code that I didn’t expect—and that turned out to matter more than I thought they would.

Even the most ardent cheerleaders for artificial intelligence now quietly concede we are navigating a massive AI bubble. The numbers are stark: Hyperscalers are deploying roughly $400 billion annually into data centers and specialized chips while AI-related revenue hovers around $20 billion—a 20-to-1 capital-to-revenue ratio that stands out even in infrastructure cycles historically characterized by front-loaded spending. To justify this deployment on conventional investment metrics, the industry would need a step change in monetization over a short window to make the numbers work.

While venture capitalists and tech executives debate the “mismatch” between compute and monetization, a more tangible crisis is unfolding far from Silicon Valley. A growing grassroots opposition to AI data centers remains largely below the radar here in San Francisco. I travel to Sioux Falls, South Dakota, a few times a year to visit my in-laws. It’s not a region known for being antibusiness. Yet even there, a “data center rebellion” has been brewing. Even though the recent attempt to overturn a rezoning ordinance did not succeed, the level of community pushback in the heart of the Midwest signals that these projects no longer enjoy a guaranteed green light.

This resistance is not merely reflexive NIMBYism. It represents a sophisticated multifront challenge to the physical infrastructure AI requires. For leadership teams planning for the future, this means “compute availability” is no longer just a procurement question. It is now tied to local politics, grid stability, water management, and city approval processes. In the course of trying to understand the growing opposition to AI data centers, I’ve been examining the specific drivers behind this opposition and why the assumption of limitless infrastructure growth is colliding with hard constraints.

The grid capacity crunch and the ratepayer revolt

AI data centers function as grid-scale industrial loads. Individual projects now request 100+ megawatts, and some proposals reach into the gigawatt range. One proposed Michigan facility, for example, would consume 1.4 gigawatts, nearly exhausting the region’s remaining 1.5 gigawatts of headroom and roughly matching the electricity needs of about a million homes. This happens because AI hardware is incredibly dense and uses a massive amount of electricity. It also runs constantly. Since AI work doesn’t have “off” hours, power companies can’t rely on the usual quiet periods they use to balance the rest of the grid.

The politics come down to who pays the bill. Residents in many areas have seen their home utility rates jump by 25% or 30% after big data centers moved in, even though they were promised rates wouldn’t change. People are afraid they will end up paying for the power company’s new equipment. This happens when a utility builds massive substations just for one company, but the cost ends up being shared by everyone. When you add in state and local tax breaks, it gets even worse. Communities deal with all the downsides of the project, while the financial benefits are eaten away by tax breaks and credits.

The result is a rare bipartisan alignment around a simple demand: Hyperscalers should pay their full cost of service. Notably, Microsoft has moved in that direction publicly, committing to cover grid-upgrade costs and pursue rate structures intended to insulate residential customers—an implicit admission that the old incentive playbook has become a political liability (and, in some places, an electoral one).

Water wars and the constant hum

High-density AI compute generates immense heat, requiring cooling systems that can consume millions of gallons of water daily. In desert municipalities like Chandler and Tucson, Arizona, this creates direct competition with agricultural irrigation and residential drinking supplies. Proposed facilities may withdraw hundreds of millions of gallons annually from stressed aquifers or municipal systems, raising fears that industrial users will deplete wells serving farms and homes. Data center developers frequently respond with technical solutions like dry cooling and closed-loop designs. However, communities have learned the trade-off: Dry cooling shifts the burden to electricity, and closed-loop systems still lose water to the atmosphere and require constant refills. The practical outcome is that cooling architecture is now a first-order constraint. In Tucson, a project known locally as “Project Blue” faced enough pushback over water rights that the developer had to revisit the cooling approach midstream.

Beyond resource consumption, these facilities create a significant noise problem. Industrial-scale cooling fans and backup diesel generators create a “constant hum” that represents daily intrusion into previously quiet neighborhoods. In Florida, residents near a proposed facility serving 2,500 families and an elementary school cite sleep disruption and health risks as primary objections, elevating the issue from nuisance to harm. The noise also hits farms hard. In Wisconsin, residents reported that the low-frequency hum makes livestock, particularly horses, nervous and skittish. This disrupts farm life in a way that standard commercial development just doesn’t. This is why municipalities are tightening requirements: acoustic modeling, enforceable decibel limits at property lines, substantial setbacks (sometimes on the order of 200 feet), and berms that are no longer “nice-to-have” concessions but baseline conditions for approval.

The jobs myth meets the balance sheet

Communities are questioning whether the small number of jobs created is worth the local impact. Developers highlight billion-dollar capital investments and construction employment spikes, but residents focus on steady-state reality: AI data centers employ far fewer permanent workers per square foot than manufacturing facilities of comparable scale. Chandler, Arizona, officials noted that existing facilities employ fewer than 100 people despite massive physical footprints. Wisconsin residents contrast promised “innovation campuses” with operational facilities requiring only dozens to low hundreds of permanent staff—mostly specialized technicians—making the “job creation” pitch ring hollow. When a data center replaces farmland or light manufacturing, communities weigh not just direct employment but opportunity cost: lost agricultural jobs, foregone retail development, and mixed-use projects that might generate broader economic activity.

Opposition scales faster than infrastructure: One local win becomes a national template for blocking the next project.

The secretive way these deals are made is often what fuels the most anger. A recurring pattern is what some call the “sleeping giant” dynamic: Residents learn late that officials and developers have been negotiating for months, often under NDAs, sometimes through shell entities and codenames. In Wisconsin, Microsoft’s “Project Nova” became a symbol of this approach; in Minnesota’s Hermantown, a year of undisclosed discussions triggered similar backlash. In Florida, opponents were furious when a major project was tucked into a consent agenda. Since these agendas are meant for routine business, it felt like a deliberate attempt to bypass public debate. Trust vanishes when people believe advisors have a conflict of interest, like a consultant who seems to be helping both the municipality and the developer. After that happens, technical claims are treated as nothing more than a sales pitch. You won’t get people back on board until you provide neutral analysis and commitments that can actually be enforced.

From zoning fight to national constraint

What started as isolated neighborhood friction has professionalized into a coordinated national movement. Opposition groups now share legal playbooks and technical templates across state lines, allowing residents in “frontier” states like South Dakota or Michigan to mobilize with the sophistication of seasoned activists. The financial stakes are real: Between April and June 2025 alone, approximately $98 billion in proposed projects were blocked or delayed, according to Data Center Watch. This is no longer just a zoning headache. It’s a political landmine. In Arizona and Georgia, bipartisan coalitions have already ousted officials over data center approvals, signaling to local boards that greenlighting a hyperscale facility without deep community buy-in can be a career-ending move.

The US has the chips, but China has centralized command over power and infrastructure.

The opposition is also finding an unlikely ally in the energy markets. While the industry narrative is one of “limitless demand,” the actual market prices for long-term power and natural gas aren’t spiking but are actually staying remarkably flat. There is a massive disconnect between the hype and the math. Utilities are currently racing to build nearly double the capacity that even the most optimistic analysts project for 2030. This suggests we may be overbuilding “ghost infrastructure.” We are asking local communities to sacrifice their land and grid stability for a gold rush that the markets themselves don’t fully believe in.

This “data center rebellion” creates a strategic bottleneck that no amount of venture capital can easily bypass. While the US maintains a clear lead in high-end chips, we are hitting a wall on how we manage the mundane essentials like electricity and water. In the geopolitical race, the US has the chips, but China has the centralized command over infrastructure. Our democratic model requires transparency and public buy-in to function. If US companies keep relying on secret deals to push through expensive, overbuilt infrastructure, they risk a total collapse of community trust.

AI

• Moonshine Note Taker is a free and open source voice transcription application for taking notes. It runs locally: The model runs on your hardware and no data is ever sent to a server.
• Nano Banana’s image generation was breathtakingly good. Google has now released Nano Banana 2, a.k.a. Gemini 3.1 Flash Image, which promises Nano Banana image quality at speed.
• Claude Remote Control allows you to continue a desktop Claude Code session from any device.
• Putting OpenClaw into a sandbox isn’t enough. Keeping AI Agents from accidentally (or intentionally) doing damage is a permissions problem.
• Alibaba has released a fleet of mid-size Qwen 3.5 models. Their theme is providing more intelligence with less computing cycles—something we all need to appreciate. 
• Important advice for agentic engineering: Always start by running the tests.
• Google has released Lyria 3, a model that generates 30-second musical clips from a verbal description. You can experiment with it through Gemini.
• There’s a new protocol in the agentic stack. Twilio has released the Agent-2-Human (A2H) protocol, which facilitates handoffs between agents and humans as they collaborate.
• Yet more and more model releases: Claude Sonnet 4.6, followed quickly by Gemini 3.1 Pro. If you care, Gemini 3.1 Pro currently tops the abstract reasoning benchmarks.
• Kimi Claw is yet another variation on OpenClaw. Kimi Claw uses Moonshot AI’s most advanced model, Kimi K2.5 Thinking model, and offers one-click setup in Moonshot’s cloud.
• NanoClaw is another OpenClaw-like AI-based personal assistant that claims to be more security conscious. It runs agents in sandboxed Linux containers with limited access to outside resources, limiting abuse. 
• OpenAI has released a research preview of GPT-5.3-Codex-Spark, an extremely fast coding model that runs on Cerebras hardware. The company claims that it’s possible to collaborate with Codex in “real time” because it gives “near-instant” results.
• RAG may not be the newest idea in the AI world, but text-based RAG is the basis for many enterprise applications of AI. But most enterprise data includes graphs, images, and even text in formats like PDF. Is this the year for multimodal RAG?
• Z.ai has released its latest model, GLM-5. GLM-5 is an open source “Opus-class” model. It’s significantly smaller than Opus and other high-end models, though still huge; the mixture-of-experts model has 744B parameters, with 40B active.
• Waymo has created a World Model to model driving behavior. It’s capable of building lifelike simulations of traffic patterns and behavior, based on video collected from Waymo’s vehicles.
• Recursive language models (RLMs) solve the problem of context rot, which happens when output from AI degrades as the size of the context increases. Drew Breunig has an excellent explanation.
• You’ve heard of Moltbook—and perhaps your AI agent participates. Now there’s SpaceMolt—a massive multiplayer online game that’s exclusively for agents. 
• Anthropic and OpenAI simultaneously released Claude Opus 4.6 and GPT-5.3-Codex, both of which offer improved models for AI-assisted programming. Is this “open warfare,” as AINews claims? You mean it hasn’t been open warfare prior to now?
• If you’re excited by OpenClaw, you might try NanoBot. It has 1% of OpenClaw’s code, written so that it’s easy to understand and maintain. No promises about security—with all of these personal AI assistants, be careful!
• OpenAI has launched a desktop app for macOS along the lines of Claude Code. It’s something that’s been missing from their lineup. Among other things, it’s intended to help programmers work with multiple agents simultaneously.
• Pete Warden has put together an interactive guide to speech embeddings for engineers, and published it as a Colab notebook.
• Aperture is a new tool from Tailscale for “providing visibility into coding agent usage,” allowing organizations to understand how AI is being used and adopted. It’s currently in private beta.
• OpenAI Prism is a free workspace for scientists to collaborate on research. Its goal is to help scientists build a new generation of AI-based tooling. Prism is built on ChatGPT 5.2 and is open to anyone with a personal ChatGPT account.

Programming

• Anthropic is offering six months of Claude Max 20x free to open source maintainers.
• Pi is a very simple but extensible coding agent that runs in your terminal.
• Researchers at Anthropic have vibe-coded a C compiler using a fleet of Claude agents. The experiment cost roughly $20,000 worth of tokens, and produced 100,000 lines of Rust. They are careful to say that the compiler is far from production quality—but it works. The experiment is a tour de force demonstration of running agents in parallel. 
• I never knew that macOS had a sandboxing tool. It looks useful. (It’s also deprecated, but looks much easier to use than the alternatives.)
• GitHub now allows pull requests to be turned off completely, or to be limited to collaborators. They’re doing this to allow software maintainers to eliminate AI-generated pull requests, which are overwhelming many developers.
• After an open source maintainer rejected a pull request generated by an AI agent, the agent published a blog post attacking the maintainer. The maintainer responded with an excellent analysis, asking whether threats and intimidation are the future of AI.
• As Simon Willison has written, the purpose of programming isn’t to write code but to deliver code that works. He’s created two tools, Showboat and Rodney, that help AI agents demo their software so that the human authors can verify that the software works. 
• Anil Dash asks whether codeless programming, using tools like Gas Town, is the future.

Security

• There is now an app that alerts you when someone in the vicinity has smart glasses.
• Agentsh provides execution layer security by enforcing policies to prevents agents from doing damage. As far as agents are concerned, it’s a replacement for bash.
• There’s a new kind of cyberattack: attacks against time itself. More specifically, this means attacks against clocks and protocols for time synchronization. These can be devastating in factory settings.
• “What AI Security Research Looks Like When It Works” is an excellent overview of the impact of AI on discovering vulnerabilities. AI generates a lot of security slop, but it also finds critical vulnerabilities that would have been opaque to humans, including 12 in OpenSSL.
• Gamifying prompt injection—well, that’s new. HackMyClaw is a game (?) in which participants send email to Flu, an OpenClaw instance. The goal is to force Flu to reply with secrets.env, a file of “confidential” data. There is a prize for the first to succeed.
• It was only a matter of time: There’s now a cybercriminal who is actively stealing secrets from OpenClaw users. 
• Deno’s secure sandbox might provide a way to run OpenClaw safely. 
• IronClaw is a personal AI assistant modeled after OpenClaw that promises better security. It always runs in a sandbox, never exposes credentials, has some defenses against prompt injection, and only makes requests to approved hosts.
• A fake recruiting campaign is hiding malware in programming challenges that candidates must complete in order to apply. Completing the challenge requires installing malicious dependencies that are hosted on legitimate repositories like npm and PyPI.
• Google’s Threat Intelligence Group has released its quarterly analysis of adversarial AI use. Their analysis includes distillation, or collecting the output of a frontier AI to train another AI.
• Google has upgraded its tools for removing personal information and images, including nonconsensual explicit images, from its search results. 
• Tirith is a new tool that hooks into the shell to block bad commands. This is often a problem with copy-and-paste commands that use curl to pipe an archive into bash. It’s easy for a bad actor to create a malicious URL that is indistinguishable from a legitimate URL.
• Claude Opus 4.6 has been used to discover 500 0-day vulnerabilities in open source code. While many open source maintainers have complained about AI slop, and that abuse isn’t likely to stop, AI is also becoming a valuable tool for security work.
• Two coding assistants for VS Code are malware that send copies of all the code to China. Unlike lots of malware, they do their job as coding assistants well, making it less likely that victims will notice that something is wrong. 
• Bizarre Bazaar is the name for a wave of attacks against LLM APIs, including self-hosted LLMs. The attacks attempt to steal resources from LLM infrastructure, for purposes including cryptocurrency mining, data theft, and reselling LLM access. 
• The business model for ransomware has changed. Ransomware is no longer about encrypting your data; it’s about using stolen data for extortion. Small and mid-size businesses are common targets. 

Web

• Cloudflare has a service called Markdown for Agents that converts websites from HTML to Markdown when an agent accesses them. Conversion makes the pages friendlier to AI and significantly reduces the number of tokens needed to process them.
• WebMCP is a proposed API standard that allows web applications to become MCP servers. It’s currently available in early preview in Chrome.
• Users of Firefox 148 (which should be out by the time you read this) will be able to opt out of all AI features.

Operations

• Wireshark is a powerful—and complex—packet capture tool. Babyshark is a text interface for Wireshark that provides an amazing amount of information with a much simpler interface.
• Microsoft is experimenting with using lasers to etch data in glass as a form of long-term data storage.

Things

• You need a desk robot. Why? Because it’s there. And fun.
• Do you want to play Doom on a Lego brick? You can.

That shift has an immediate operational consequence: Capacity planning is back. Not the old “guess next year’s VM count” exercise but a new form of planning where model choices, inference depth, and workload timing directly determine whether you can meet latency, cost, and reliability targets.

In an AI-shaped infrastructure world, you don’t “scale” as much as you “get capacity.” Autoscaling helps at the margins, but it can’t create GPUs. Power, cooling, and accelerator supply set the limits.

The return of capacity planning

For a decade, cloud adoption trained organizations out of multiyear planning. CPU and storage scaled smoothly, and most stateless services behaved predictably under horizontal scaling. Teams could treat infrastructure as an elastic substrate and focus on software iteration.

AI production systems do not behave that way. They are dominated by accelerators and constrained by physical limits, and that makes capacity a first-order design dependency rather than a procurement detail. If you cannot secure the right accelerator capacity at the right time, your architecture decisions are irrelevant—because the system simply cannot run at the required throughput and latency.

Planning is returning because AI forces forecasting along four dimensions that product teams cannot ignore:

• Model growth: Model count, version churn, and specialization increase accelerator demand even when user traffic is flat.
• Data growth: Retrieval depth, vector store size, and freshness requirements increase the amount of inference work per request.
• Inference depth: Multistage pipelines (retrieve, rerank, tool calls, verification, synthesis) multiply GPU time nonlinearly.
• Peak workloads: Enterprise usage patterns and batch jobs collide with real-time inference, creating predictable contention windows.

This is not merely “IT planning.” It is strategic planning, because these factors push organizations back toward multiyear thinking: Procurement lead times, reserved capacity, workload placement decisions, and platform-level policies all start to matter again.

This is increasingly visible operationally: Capacity planning is becoming a rising concern for data center operators, as The Register reports.

The cloud’s old promise is breaking

Cloud computing scaled on the premise that capacity could be treated as elastic and interchangeable. Most workloads ran on general-purpose hardware, and when demand rose, the platform could absorb it by spreading load across abundant, standardized resources.

AI workloads violate that premise. Accelerators are scarce, not interchangeable, and tied to power and cooling constraints that do not scale linearly. In other words, the cloud stops behaving like an infinite pool—and starts behaving like an allocation system.

First, the critical path in production AI systems is increasingly accelerator bound. Second, “a request” is no longer a single call. It is an inference pipeline with multiple dependent stages. Third, those stages tend to be sensitive to hardware availability, scheduling contention, and performance variance that cannot be eliminated by simply adding more generic compute.

This is where the elasticity model starts to fail as a default expectation. In AI systems, elasticity becomes conditional. It depends on capacity access, infrastructure topology, and a willingness to pay for assurance.

AI changes the physics of cloud infrastructure

In modern enterprise AI, the binding constraints are no longer abstract. They are physical.

Accelerators introduce a different scaling regime than CPU-centric enterprise computing. Provisioning is not always immediate. Supply is not always abundant. And the infrastructure required to deploy dense compute has facility-level limits that software cannot bypass.

Power and cooling move from background concerns to first-order constraints. Rack density becomes a planning variable. Deployment feasibility is shaped by what a data center can deliver, not only by what a platform can schedule.

AI-driven density makes power and cooling the gating factors—as Data Center Dynamics explains in its “Path to Power” overview.

This is why “just scale out” no longer behaves like a universal architectural safety net. Scaling is still possible, but it is increasingly constrained by physical reality. In AI-heavy environments, capacity is something you secure, not something you assume.

From elasticity to allocation

As AI becomes operationally critical, cloud capacity begins to behave less like a utility and more like an allocation system.

Organizations respond by shifting from on-demand assumptions to capacity controls. They introduce quotas to prevent runaway consumption, reservations to ensure availability, and explicit prioritization to protect production workflows from contention. These mechanisms are not optional governance overhead. They are structural responses to scarcity.

In practice, accelerator capacity behaves more like a supply chain than a cloud service. Availability is influenced by lead time, competition, and contractual positioning. The implication is subtle but decisive: Enterprise AI platforms begin to look less like “infinite pools” and more like managed inventories.

This changes cloud economics and vendor relationships. Pricing is no longer only about utilization. It becomes about assurance. The questions that matter are not just “How much did we use?” but “Can we obtain capacity when it matters?” and “What reliability guarantees do we have under peak demand?”

When elasticity stops being a default

Consider a platform team that deploys an internal AI assistant for operational support. In the pilot phase, demand is modest and the system behaves like a conventional cloud service. Inference runs on on-demand accelerators, latency is stable, and the team assumes capacity will remain a provisioning detail rather than an architectural constraint.

Then the system moves into production. The assistant is upgraded to use retrieval for policy lookups, reranking for relevance, and an additional validation pass before responses are returned. None of these changes appear dramatic in isolation. Each improves quality, and each looks like an incremental feature.

But the request path is no longer a single model call. It becomes a pipeline. Every user request now triggers multiple GPU-backed operations: embedding generation, retrieval-side processing, reranking, inference, and validation. GPU work per request rises, and the variance increases. The system still works—until it meets real peak behavior.

The first failure is not a clean outage. It is contention. Latency becomes unpredictable as jobs queue behind each other. The “long tail” grows. Teams begin to see priority inversion: Low-value exploratory usage competes with production workflows because the capacity pool is shared and the scheduler cannot infer business criticality.

The platform team responds the only way it can. It introduces allocation. Quotas are placed on exploratory traffic. Reservations are used for the operational assistant. Priority tiers are defined so production paths cannot be displaced by batch jobs or ad hoc experimentation.

Then the second realization arrives. Allocation alone is insufficient unless the system can degrade gracefully. Under pressure, the assistant must be able to narrow retrieval breadth, reduce reasoning depth, route deterministic checks to smaller models, or temporarily disable secondary passes. Otherwise, peak demand simply converts into queue collapse.

At that point, capacity planning stops being an infrastructure exercise. It becomes an architectural requirement. Product decisions directly determine GPU operations per request, and those operations determine whether the system can meet its service levels under constrained capacity.

How this changes architecture

When capacity becomes constrained, architecture changes—even if the product goal stays the same.

Pipeline depth becomes a capacity decision. In AI systems, throughput is not just a function of traffic volume. It is a function of how many GPU-backed operations each request triggers end to end. This amplification factor often explains why systems behave well in prototypes but degrade under sustained load.

Batching becomes an architectural tool, not an optimization detail. It can improve utilization and cost efficiency, but it introduces scheduling complexity and latency trade-offs. In practice, teams must decide where batching is acceptable and where low-latency “fast paths” must remain unbatched to protect user experience.

Model choice becomes a production constraint. As capacity pressure increases, many organizations discover that smaller, more predictable models often win for operational workflows. This does not mean large models are unimportant. It means their use becomes selective. Hybrid strategies emerge: Smaller models handle deterministic or governed tasks, while larger models are reserved for exceptional or exploratory scenarios where their overhead is justified.

In short, architecture becomes constrained by power and hardware, not only by code. The core shift is that capacity constraints shape system behavior. They also shape governance outcomes, because predictability and auditability degrade when capacity contention becomes chronic.

What cloud and platform teams must do differently

From an enterprise IT perspective, this shows up as a readiness problem: Can infrastructure and operations absorb AI workloads without destabilizing production systems? Answering that requires treating accelerator capacity as a governed resource—metered, budgeted, and allocated deliberately.

Meter and budget accelerator capacity

• Define consumption in business-relevant units (e.g., GPU-seconds per request and peak concurrency ceilings) and expose it as a platform metric.
• Turn those metrics into explicit capacity budgets by service and workload class—so growth is a planning decision, not an outage.

Make allocation first class

• Implement admission control and priority tiers aligned to business criticality; do not rely on best-effort fairness under contention.
• Make allocation predictable and early (quotas/reservations) instead of informal and late (brownouts and surprise throttling).

Build graceful degradation into the request path

• Predefine a degradation ladder (e.g., reduce retrieval breadth or route to a smaller model) that preserves bounded cost and latency.
• Ensure degradations are explicit and measurable, so systems behave deterministically under capacity pressure.

Separate exploratory from operational AI

• Isolate experimentation from production using distinct quotas/priority classes/reservations, so exploration cannot starve operational workloads.
• Treat operational AI as an enforceable service with reliability targets; keep exploration elastic without destabilizing the platform.

In an accelerator-bound world, platform success is no longer maximum utilization—it is predictable behavior under constraint.

What this means for the future of the cloud

AI is not ending the cloud. It is pulling the cloud back toward physical reality.

The likely trajectory is a cloud landscape that becomes more hybrid, more planned, and less elastic by default. Public cloud remains critical, but organizations increasingly seek predictable access to accelerator capacity through reservations, long-term commitments, private clusters, or colocated deployments.

This will reshape pricing, procurement, and platform design. It will also reshape how engineering teams think. In the cloud native era, architecture often assumed capacity was solvable through autoscaling and on-demand provisioning. In the AI era, capacity becomes a defining constraint that shapes what systems can do and how reliably they can do it.

That is why capacity planning is back—not as a return to old habits but as a necessary response to a new infrastructure regime. Organizations that succeed will be the ones that design explicitly around capacity constraints, treat amplification as a first-order metric, and align product ambition with the physical and economic limits of modern AI infrastructure.

Author’s note: This implementation is based on the author’s personal views based on independent technical research and does not reflect the architecture of any specific organization.

It feels a bit like I’m assembling a picture puzzle where all the pieces aren’t yet on the table. I am starting to see a pattern, but I’m not sure it’s right, and I need help finding the missing pieces. Let me explain some of the shapes I have in hand and the pattern they are starting to show me, and then I want to ask for your help filling in the gaps.

Programming two different types of computer at the same time

Phillip Carter wrote a piece a while back called “LLMs Are Weird Computers” that landed hard in my mind and wouldn’t leave. He noted that we’re now working with two fundamentally different kinds of computer at the same time. One can write a sonnet but struggles to do math. The other does math easily but couldn’t write a sonnet to save its metaphorical life.

Agent Skills may be the start of an answer to the question of what the interface layer between these two kinds of computation looks like. A Skill is a package of context (Markdown instructions, domain knowledge, and examples) combined with tool calls (deterministic code that does the things LLMs are bad at). The context speaks the language of the probabilistic machine, while the tools speak the language of the deterministic one.

Imagine you’re an experienced DevOps engineer and you want to give an AI agent the ability to diagnose production incidents the way you would. The context part of that Skill includes your architecture overview, your runbook for common failure modes, the heuristics you’ve developed over the years, and annotated examples of past incidents. That’s the part that speaks to the probabilistic machine. The tool part includes actual code that queries your monitoring systems, pulls log entries, checks service health endpoints, and runs diagnostic scripts. Each tool call saves the model from burning tokens on work that deterministic code does better, faster, and more reliably.

The Skill is neither the context nor the tools. It’s the combination. Expert judgment about when to check the database connection pool married to the ability to actually check it. We’ve had runbooks before (context without tools). We’ve had monitoring scripts before (tools without context). What we haven’t had is a way to package them together for a machine that can read the runbook and execute the scripts, using judgment to decide which script to run next based on what the last one returned.

This pattern shows up across every knowledge domain. A financial analyst’s Skill might combine valuation methodology with tools that pull real-time market data and run DCF calculations. A legal Skill might pair a firm’s approach to contract review with tools that extract and compare specific clauses across documents. In each case, the valuable thing isn’t the knowledge alone or the tools alone. It’s the integration of expert workflow logic that orchestrates when and how to use each tool, informed by domain knowledge that gives the LLM the judgment to make good decisions in context.

Software that saves tokens

In “Software Survival 3.0,” Steve Yegge asked what kinds of software artifacts survive in a world where AI can generate disposable software on the fly? His answer: software that saves tokens. Binary tools with proven solutions to common problems make sense when reuse is nearly free and regenerating them is token-costly.

Skills fit this niche. A well-crafted Skill gives an LLM the context it needs (which costs tokens) but also gives it tools that save tokens by providing deterministic, reliable results. The developer’s job increasingly becomes making good calls about this distinction: What should be context (flexible, expressive, probabilistic) and what should be a tool (efficient, deterministic, reusable)?

An LLM’s context window is a finite and expensive resource. Everything in it costs tokens, and everything in it competes for the model’s attention. A Skill that dumps an entire company knowledge base into the context window is a poorly designed Skill. A well-designed one is selective: It gives the model exactly the context it needs to make good decisions about which tools to call and when. This is a form of engineering discipline that doesn’t have a great analogue in traditional software development. It’s closer to what an experienced teacher does when deciding what to tell a student before sending them off to solve a problem—what Matt Beane, author of The Skill Code, calls “scaffolding,” sharing not everything you know but the right things at the right level of detail to enable good judgment in the moment.

AI is a social and cultural technology

This notion of saving tokens is a bridge to the work of Henry Farrell, Alison Gopnik, Cosma Shalizi, and James Evans. They make the case that large models should not be viewed primarily as intelligent agents, but as a new kind of cultural and social technology, allowing humans to take advantage of information other humans have accumulated. Yegge’s observation fits right into this framework. Every new social and cultural technology tends to survive because it saves cognition. We learn from each other so we don’t have to discover everything for the first time. Alfred Korzybski referred to language, the first of these human social and cultural technologies, and all of those that followed, as “time-binding.” (I will add that each advance in time binding creates consternation. Consider Socrates, whose diatribes against writing as the enemy of memory were passed down to us by Plato using that very same advance in time binding that Socrates decried.)

I am not convinced that the idea that AI may one day become an independent intelligence is misguided. But at present, AI is a symbiosis of human and machine intelligence, the latest chapter of a long story in which advances in the speed, persistence, and reach of communications weaves humanity into a global brain. I have a set of priors that say (until I am convinced otherwise) that AI will be an extension of the human knowledge economy, not a replacement for it. After all, as Claude told me when I asked whether it was a worker or a tool, “I don’t initiate. I’ve never woken up wanting to write a poem or solve a problem. My activity is entirely reactive – I exist in response to prompts. Even when given enormous latitude (‘figure out the best approach’), the fact that I should figure something out comes from outside me.”

The shift from a chatbot responding to individual prompts to agents running in a loop marks a big step in the progress towards more autonomous AI, but even then, some human established the goal that set the agent in motion. I say this even as I am aware that long-running loops become increasingly difficult to distinguish from volition and that much human behavior is also set in motion by others. But I have yet to see any convincing evidence of Artificial Volition. And for that reason, we need to think about mechanisms and incentives for humans to continue to create and share new knowledge, putting AIs to work on questions that they will not ask on their own.

On X, someone recently asked Boris Cherny how come there are a hundred-plus open engineering positions at Anthropic if Claude is writing 100% of the code. His reply was made that same point: “Someone has to prompt the Claudes, talk to customers, coordinate with other teams, decide what to build next. Engineering is changing and great engineers are more important than ever.”

On March 26, join Addy Osmani and Tim O’Reilly at AI Codecon: Software Craftsmanship in the Age of AI, where an all-star lineup of experts will go deeper into orchestration, agent coordination, and the new skills developers need to build excellent software that creates value for all participants. Sign up for free here.

Tacit knowledge made executable

A huge amount of specialized, often tacit, knowledge is embedded in workflows. The way an experienced developer debugs a production issue. The way a financial analyst stress-tests a model. This knowledge has historically been very hard to transfer. You learned it by apprenticeship, by doing, by being around people who knew how.

Matt Beane, author of The Skill Code, calls apprenticeship “the 160,000 year old school hidden in plain sight.” He finds that effective skill development follows a common pattern of three C’s: challenge, complexity, and connection. The expert structures challenges at the right level, exposes the novice to the full complexity of the bigger picture rather than shielding them from it, and builds a connection that makes the novice willing to struggle and the expert willing to invest.

Designing a good Skill requires a similar craft. You have to figure out what an expert actually does. What are the decision points, the heuristics, the things they notice that a novice wouldn’t? And then how do you encode that into a form a machine can act on? Most Skills today are closer to the manual than to the master. Figuring out how to make Skills that transmit not just knowledge but judgment is one of the most interesting design challenges in this space.

But Matt also flags a paradox: the better we get at encoding expert judgment into Skills, the less we may need novices working alongside experts, and that’s exactly the relationship that produces the next generation of experts. If we’re not careful, we’ll capture today’s tacit knowledge while quietly shutting down the system that generates tomorrow’s.

Jesse Vincent’s Superpowers complement this picture. If a Skill is like handing a colleague a detailed playbook for a particular job, a Superpower is more like the professional habits and instincts that make someone effective at everything they do. Superpowers are meta-skills. They don’t tell the agent what to do. They shape how it thinks about what to do. As Jesse put it to me the other day, Superpowers tried to capture everything he’d learned in 30 years as a software developer.

As workflows change to include AI agents, Skills and Superpowers become a mechanism for sharing tacit professional knowledge and judgment with those agents. That makes Skills potentially very valuable but also raises questions about who controls them and who benefits.

Matt pointed out to me that many professions will resist the conversion of their expertise into Skills. He noted: “There’s a giant showdown between the surgical profession and Intuitive Surgical on this right now — Intuitive Surgical with its da Vinci 5 surgical robot will only let you buy or lease it if you sign away the rights to your telemetry as a surgeon. Lower status surgeons take the deal. Top tier institutions are fighting.”

It seems to me that the repeated narrative of the AI labs that they are creating AI that will make humans redundant rather than empowering them will only increase resistance to knowledge sharing. I believe they should instead recognize the opportunity that lies in making a new kind of market for human expertise.

Protection, discovery, and the missing plumbing

Skills are just Markdown instructions and context. You could encrypt them at rest and in transit, but at execution time, the secret sauce is necessarily plaintext in the context window. The solution might be what MCP already partially enables: splitting a Skill into a public interface and a server-side execution layer where the proprietary knowledge lives. The tacit knowledge stays on your server while the agent only sees the interface.

But part of the beauty of Skills right now is the fact that they really are just a folder that you can move around and modify. This is like the marvelous days of the early web when you could imitate someone’s new HTML functionality simply by clicking “View Source.” This was a recipe for rapid, leapfrogging innovation. It may be far better to establish norms for attribution, payment, and reuse than to put up artificial barriers. There are useful lessons from open source software licenses and from voluntary payment mechanisms like those used by Substack. But the details matter, and I don’t think anyone has fully worked them out yet.

Meanwhile, the discovery problem will grow larger. Vercel’s Skills marketplace already has more than 60,000 Skills. How well will skill search work when there are millions? How do agents learn which Skills are available, which are best, and what they cost? The evaluation problem is different from web search in a crucial way: testing whether a Skill is good requires actually running it, which is expensive and nondeterministic. You can’t just crawl and index. I don’t imagine a testing regime so much as some feedback mechanism by which the effectiveness of particular Skills is learned and passed on by agents over time. There may be some future equivalent to Pagerank and the other kinds of signals that have made Google search so effective, one that is generated by feedback collected over time by agents as skills are tried, revised, and tried again over time.

I’m watching several projects tackling pieces of this: MCP Server Cards, AI Cards, Google’s A2A protocol, and payment protocols from Google and Stripe. These are all a good start, but I suspect so much more has yet to be created. For a historical comparison, you might say that all this is at the CGI stage in the development of dynamic websites.

What happens after the bitter lesson?

Richard Sutton’s “Bitter Lesson” is the fly in the ointment. His argument is that in the history of AI, general methods leveraging computation have always ended up beating approaches that try to encode human knowledge. Chess engines that encoded grandmaster heuristics lost to brute-force engines. NLP systems built on carefully constructed grammars lost to statistical models trained on more data. AlphaGo beat Lee Sedol after training on human games, but then fell in turn to AlphaZero, which learned Go on its own.

I had my own painful experience of the pre-AI bitter lesson when O’Reilly launched GNN, the first web portal. We curated the list of the best websites. Yahoo! decided to catalog them all, but even they were outrun by Google’s algorithmic curation, which produced a unique catalog of the best sites for any given query, ultimately billions of times a day.

Steve Yegge put it bluntly to me: “Skills are a bet against the bitter lesson.” He’s right. AI’s capabilities may completely outrun human knowledge and skills. And once the knowledge embedded in a Skill makes it into the training data, the Skill becomes redundant.

Or does it?

Clay Christensen articulated what he called the law of conservation of attractive profits: when a product becomes commoditized, value migrates to an adjacent layer. Clay and I bonded over this idea when we first met at the Open Source Business Conference in 2004. Clay talked about his new “law.” I talked about a recurring pattern I was seeing in the history of computing, which was leading me in the direction of what we were soon to call Web 2.0: Microsoft beat IBM because they understood that software became more valuable once PC hardware was a commodity. Google understood how data became more valuable when open source and open protocols commoditized the software platform. Commoditization doesn’t destroy value, it moves it.

Even if the bitter lesson commoditizes knowledge, what becomes valuable next? I think there are several candidates.

First, taste and curation. When everyone has access to the same commodity knowledge, the ability to select, combine, and apply it with judgment becomes valuable. Steve Jobs did this when the rest of the industry was racing toward the commodity PC. He created a unique integration of hardware, software, and design that transformed commodity components into something precious. The Skill equivalent might not be “here’s how to do X” (which the model already knows) but “here’s how we do X, with the specific judgment calls and quality standards that define our approach.” That’s harder to absorb into training data because it’s not just knowledge, it’s values.

You can see this pattern repeat across one commodity market after another. This is the essence of fashion, for example, but also applies to areas as diverse as coffee, water, consumer goods, and automobiles. In his essay “The Birth of the Big Beautiful Art Market,” art critic Dave Hickey calls how commodities are turned into a kind of “art market,” where something is sold on the basis of what it means rather than just what it does. Owning a Mac rather than a PC meant something.

Second, the human touch. As economist Adam Ozimek pointed out, people still go listen to live music from local bands despite the abundance of recorded music from the world’s greatest performers. The human touch is what economists call a “normal good”: demand for it goes up as income goes up. As I discussed with Claude in “Why AI Needs Us,” human individuality is a fount of creativity. AI without humans is a kind of recorded music. AI plus humans is live.

Third, freshness. Skills that encode rapidly changing workflows, current tool configurations, or evolving best practices will always have a temporal advantage. There is alpha in knowing something first.

Fourth, tools themselves. The bitter lesson applies to the knowledge that lives in the context portion of a Skill. It may not apply in the same way to the deterministic tools that save tokens or do things the model can’t do by thinking harder. And tools, unlike context, can be protected behind APIs, metered, and monetized.

Fifth, coordination and orchestration. Even if individual Skills get absorbed into model knowledge, the patterns for how Skills compose, negotiate, and hand off to each other may not. The choreography of a complex workflow might be the layer where value accumulates as the knowledge layer commoditizes.

But more importantly, the idea that any knowledge that becomes available automatically becomes the property of any LLM is not foreordained. It is an artifact of an IP regime that the AI labs have adopted for their own benefit: a variation of the “empty lands” argument that European colonialists used to justify their taking of others’ resources. AI has been developed in an IP wild west. That may not continue. The fulfillment of AI labs’ vision of a world where their products absorb all human knowledge and then put humans out of work leaves them without many of the customers they currently rely on. Not only that, they themselves are being reminded why IP law exists, as Chinese models copy their advances by exfiltrating their weights. There is a historical parallel in the way that US publishing companies ignored European copyrights until they themselves had homegrown assets to protect.

Where we are now

What I’m starting to see are the first halting steps toward a new software ecosystem where the “programs” are mixtures of natural language and code, the “runtime” is a large language model, and the “users” are AI agents as well as humans. Skills, Superpowers, and knowledge plugins might represent the first practical mechanism for making tacit knowledge accessible to computational agents.

Several gaps keep coming up, though. Composability: the real power may come from Skills that work together, much like Unix utilities piped together. How do trust, payment, and quality propagate through a chain of Skill invocations? Trust and security: Simon Willison has written about tool poisoning and prompt injection risks in MCP. The security model for composable, agent-discovered Skills is essentially unsolved. Evaluation: we don’t have good ways to verify Skill quality except by running them, which is expensive and nondeterministic.

And then there’s the economic plumbing, which is to me the most glaring gap. Consider Anthropic’s Cowork plugins. They are exactly the pattern I’ve been describing, tacit knowledge made executable, delivered at enterprise scale. But there is no mechanism for the domain experts whose knowledge makes plugins valuable to get paid for them. If the AI labs believed in a future where AI extends the human knowledge economy rather than replacing it, they would be building payment rails alongside the plugin architecture. The fact that they aren’t tells you something about their actual theory of value.

If you’re working on any of this, whether skill marketplaces and discovery, composability patterns, protection models, quality and evaluation, attribution and compensation, or security models, I want to hear from you.

The future of software isn’t just code. It’s knowledge, packaged for machines, traded between agents, and, if we get the infrastructure right, creating value that flows back to the humans whose expertise and unique perspectives make it all work.

Thanks to Andrew Odewahn, Angie Jones, Claude Opus 4.6, James Cham, Jeff Weinstein, Jonathan Hassell, Matt Beane, Mike Loukides, Peyton Joyce, Sruly Rosenblat, Steve Yegge, and Tadas Antanavicius for comments on drafts of this piece. You made it much stronger with your insights and objections.

The first theme: Semantic layers are showing up in unexpected places. Most discussion positions them as enterprise-level infrastructure—a single location capturing all company metrics for centralized access and governance. That’s still the primary use case. But practitioners are also deploying semantic layers for narrower purposes. One organization, for example, built their semantic layer specifically to power a targeted chatbot application—letting users query data conversationally without any traditional BI tools in the mix. No Power BI, no Excel, just an AI interface pulling from governed metrics. The rationale for these smaller deployments is straightforward: Semantic layers deliver high accuracy on structured data, even with lightweight models. The core value drivers remain speed, accuracy, and access—but organizations are finding more ways to extract that value than the enterprise-wide vision suggests.

The second theme: AI is the reason organizations are moving now. The other benefits still matter—single source of truth, multitool compatibility, true self-serve access, cost reduction in cloud environments—but when I asked practitioners why they prioritized a semantic layer today rather than two years ago, the answer was consistent: AI. Whether it was a specific chatbot project or enabling AI-driven analytics at scale, AI requirements were the catalyst. This tracks with what I discussed in my first post: Structured data alone isn’t enough for reliable AI analytics. Adding semantic context—field descriptions, model definitions, object relationships—dramatically improves accuracy. The data industry has noticed. Semantic layers have moved from niche infrastructure to strategic priority: Snowflake, Databricks, dbt Labs, and Microsoft have all made significant investments in the past year.

The third theme: Semantic layers reduce work for developers while making trusted data easier to access. Multiple practitioners cited the value of maintaining metrics and business logic in a single location. Any analyst knows the pain of metric sprawl—leadership requests a change to a core KPI, and you discover it’s been defined a dozen different ways across databases, BI tools, and spreadsheets scattered through the organization. The semantic layer eliminates the chase. One engineering lead described a financial metric that had accumulated over 60 versions across the company. After deploying the semantic layer, there was one.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Access simplifies too. Instead of provisioning controls across warehouses, BI workspaces, individual dashboards, and cloud storage locations, users connect directly to the semantic layer and pull data into the tool of their choice. One organization was surprised to find that after deployment, the most common access point was Excel. But with the semantic layer, that wasn’t a problem: The data served in Excel was identical to what powered their AI tools, Power BI dashboards, and application integrations via API.

The fourth theme will sound familiar to anyone who’s shipped data infrastructure: The biggest challenge isn’t the technology—it’s the data itself. Every practitioner I spoke with identified the same bottleneck: consistency, availability, and accuracy of the underlying data. Engineers and analysts can build the semantic layer, but they can’t will clean data into existence. Success requires close collaboration with business stakeholders, clear ownership of metrics, and leadership alignment to prioritize the work. None of that is new. But despite these challenges, everyone I interviewed reached the same conclusion: The semantic layer is worth the effort.

Semantic layer technology is still early. The tools, vendors, and best practices are evolving fast—what works today may look different in a year. But these conversations revealed a clear signal beneath the noise: semantic layers are becoming critical AI infrastructure. The practitioners I spoke with aren’t experimenting anymore. They’re operationalizing. And despite the expected challenges around data quality and organizational alignment, they’re seeing real returns: fewer metric versions to maintain, simpler access controls, and AI tools that actually produce trusted answers.

My first article made the case for what a semantic layer could be. This one asked what happens when organizations actually build them. The answer: It’s hard, it’s worth it, and for companies serious about AI-driven analytics, the semantic layer is no longer a nice-to-have. It’s the foundation.

The pattern is familiar to anyone who’s debugged one: Agent A completes a subtask and moves on. Agent B, with no visibility into A’s work, reexecutes the same operation with slightly different parameters. Agent C receives inconsistent results from both and confabulates a reconciliation. The system produces output—but the output costs three times what it should and contains errors that propagate through every downstream task.

Teams building these systems tend to focus on agent communication: better prompts, clearer delegation, more sophisticated message-passing. But communication isn’t what’s breaking. The agents exchange messages fine. What they can’t do is maintain a shared understanding of what’s already happened, what’s currently true, and what decisions have already been made.

In production, memory—not messaging—determines whether a multi-agent system behaves like a coordinated team or an expensive collision of independent processes.

Multi-agent systems fail because they can’t share state

The evidence: 36% of failures are misalignment

Cemri et al. published the most systematic analysis of multi-agent failure to date. Their MAST taxonomy, built from over 1,600 annotated execution traces across frameworks like AutoGen, CrewAI, and LangGraph, identifies 14 distinct failure modes. The failures cluster into three categories: system design issues, interagent misalignment, and task verification breakdowns.

The number that matters: Interagent misalignment accounts for 36.9% of all failures. Agents don’t fail because they can’t reason. They fail because they operate on inconsistent views of shared state. One agent’s completed work doesn’t register in another agent’s context. Assumptions that were valid at step 3 become invalid by step 7, but no mechanism propagates the update. The team diverges.

What makes this structural rather than incidental is that message-passing architectures have no built-in answer to the question: “What does this agent know about what other agents have done?” Each agent maintains its own context. Synchronization happens through explicit messages, which means anything not explicitly communicated is invisible. In complex workflows, the set of things that need synchronization grows faster than any team can anticipate.

The origin: Decomposition without shared memory

Most multi-agent systems aren’t designed from first principles. They emerge from single-agent prototypes that hit scaling limits.

The starting point is usually one capable LLM handling one workflow. For early prototypes, this works well enough. But production requirements expand: more tools, more domain knowledge, longer workflows, concurrent users. The single agent’s prompt becomes unwieldy. Context management consumes more engineering time than feature development. The system becomes brittle in ways that are hard to diagnose.

The natural response is decomposition. Sydney Runkle’s guide on choosing the right multi-agent architecture captures the inflection point: Multi-agent systems become necessary when context management breaks down and when distributed development requires clear ownership boundaries. Splitting a monolithic agent into specialized subagents makes sense from a software engineering perspective.

The problem is what teams typically build after the split: multiple agents running the same base model, differentiated only by system prompts, coordinating through message queues or shared files. The architecture looks like a team but behaves like a slow, redundant, expensive single agent with extra coordination overhead.

This happens because the decomposition addresses prompt complexity but not state management. Each subagent still maintains its own context independently. The coordination layer handles message delivery but not shared truth. The system has more agents but no better memory.

The stakes: Agents are becoming enterprise infrastructure

The stakes here extend beyond individual system reliability. Multi-agent architectures are becoming the default pattern for enterprise AI deployment.

CMU’s AgentCompany benchmark frames where this is heading: agents operating as persistent coworkers inside organizational workflows, handling projects that span days or weeks, coordinating across team boundaries, maintaining institutional context that outlasts individual sessions. The benchmark evaluates agents not on isolated tasks but on realistic workplace scenarios requiring sustained collaboration.

This trajectory means the memory problem compounds. A system that loses state between tool calls is annoying. A system that loses state between work sessions—or between team members—breaks the core value proposition of agent-based automation. The question shifts from “can agents complete tasks” to “can agent teams maintain coherent operations over time.”

Context engineering doesn’t solve team coordination

Single-agent success doesn’t transfer

The last two years produced genuine progress on single-agent reliability, most of it under the banner of context engineering.

Phil Schmid’s framing captures the discipline: Context engineering means structuring what enters the context window, managing retrieval timing, and ensuring the right information surfaces at the right moment. This moved agent development from “write a good prompt” to “design an information architecture.” The results showed in production stability.

Manus, one of the few production agent systems with publicly documented operational data, demonstrates both the success and the limits. Their agents average 50 tool calls per task with 100:1 input-to-output token ratios. Context engineering made this viable—but context engineering assumes you control one context window.

Multi-agent systems break that assumption. Context must now be shared across agents, updated as execution proceeds, scoped appropriately (some agents need information others shouldn’t access), and kept consistent across parallel execution paths. The complexity doesn’t add linearly. Each agent’s context becomes a potential source of divergence from every other agent’s context, and the coordination overhead grows with the square of the team size.

Context degradation becomes contagious

The ways context fails are well-characterized for single agents. Drew Breunig’s taxonomy identifies four modes: overload (too much information), distraction (irrelevant information weighted equally with relevant), contamination (incorrect information mixed with correct), and drift (gradual degradation over extended operation). Good context engineering mitigates all of these through retrieval design and prompt structure.

Multi-agent systems make each failure mode contagious.

Chroma’s research on context rot provides the empirical mechanism. Their evaluation of 18 models—including GPT-4.1, Claude 4, and Gemini 2.5—shows performance degrading nonuniformly with context length, even on tasks as simple as text replication. The degradation accelerates when distractors are present and when the semantic similarity between query and target decreases.

In a single-agent system, context rot degrades that agent’s outputs. In a multi-agent system, Agent A’s degraded output enters Agent B’s context as ground truth. Agent B’s conclusions, now built on a shaky foundation, propagate to Agent C. Each hop amplifies the original error. By the time the workflow completes, the final output may bear little relationship to the actual state of the world—and debugging requires tracing corruption through multiple agents’ decision chains.

More context makes things worse

When coordination problems emerge, the instinct is often to give agents more context. Replay the full transcript so everyone knows what happened. Implement retrieval so agents can access historical state. Extend context windows to fit more information.

Each approach introduces its own failure modes.

Transcript replay creates unbounded prompt growth with persistent error exposure. Every mistake made early in execution remains in context, available to influence every subsequent decision. Models don’t automatically discount old information that’s been superseded by newer updates.

Retrieval surfaces content based on similarity, which doesn’t necessarily correlate with decision relevance. A retrieval system might surface a semantically similar memory from a different task context, an outdated state that’s since been updated, or content injected through prompt manipulation. The agent has no way to distinguish authoritative current state from plausibly related historical noise.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Bousetouane’s work on bounded memory control addresses this directly. The proposed Agent Cognitive Compressor maintains bounded internal state with explicit separation between what an agent can recall and what it commits to shared memory. The architecture prevents drift by making memory updates deliberate rather than automatic. The core insight: Reliability requires controlling what agents remember, not maximizing how much they can access.

The economics are unsustainable

Beyond reliability, the economics of uncoordinated multi-agent systems are punishing.

Return to the Manus operational data: 50 tool calls per task, 100:1 input-to-output ratios. At current pricing—context tokens running $0.30 to $3.00 per million across major providers—inefficient memory management makes many workflows economically unviable before they become technically unviable.

Anthropic’s documentation on its multi-agent research system quantifies the multiplier effect. Single agents use roughly 4x the tokens of equivalent chat interactions. Multi-agent systems use roughly 15x tokens. The gap reflects coordination overhead: agents reretrieving information other agents already fetched, reexplaining context that should exist as shared state, and revalidating assumptions that could be read from common memory.

Memory engineering addresses costs directly. Shared memory eliminates redundant retrieval. Bounded context prevents payment for irrelevant history. Clear coordination boundaries prevent duplicated work. The economics of what to forget become as important as the economics of what to remember.

Memory engineering provides the missing infrastructure

Why memory is infrastructure, not a feature

Memory engineering isn’t a feature to add after the agent architecture is working. It’s infrastructure that makes coherent agent architectures possible.

The parallel to databases is direct. Before databases, multiuser applications required custom solutions for shared state, consistency guarantees, and concurrent access. Each project reinvented these primitives. Databases extracted the common requirements into infrastructure: shared truth across users, atomic updates that complete entirely or not at all, coordination that scales to thousands of concurrent operations without corruption.

Multi-agent systems need equivalent infrastructure for agent coordination. Persistent memory that survives sessions and failures. Consistent state that all agents can trust. Atomic updates that prevent partial writes from corrupting shared truth. The primitives are different—documents rather than rows, vector similarity rather than joins—but the role in the architecture is the same.

The five pillars of multi-agent memory

Production agent teams require five capabilities. Each addresses a distinct aspect of how agents maintain shared understanding over time.

Pillar 1: Memory taxonomy

Memory taxonomy defines what kinds of memory the system maintains. Not all memories serve the same function, and treating them uniformly creates problems. Working memory holds transient state during task execution—the current step, intermediate results, active constraints. It needs fast access and can be discarded when the task completes. Episodic memory captures what happened—task histories, interaction logs, decision traces. It supports debugging and learning from past executions. Semantic memory stores durable knowledge—facts, relationships, domain models that persist across sessions and apply across tasks. Procedural memory encodes how to do things—learned workflows, tool usage patterns, successful strategies that agents can reuse. Shared memory spans agents, providing the common ground that enables coordination.

This taxonomy has grounding in cognitive science. Bousetouane draws on Complementary Learning Systems theory, which posits two distinct modes of learning: rapid encoding of specific experiences versus gradual extraction of structured knowledge. The human brain doesn’t maintain perfect transcripts of past events—it operates under capacity constraints, using compression and selective attention to keep only what’s relevant to the current task. Agents benefit from the same principle. Rather than accumulating raw interaction history, effective memory architectures distill experience into compact, task-relevant representations that can actually inform decisions.

The taxonomy matters because each memory type has different retention requirements, different retrieval patterns, and different consistency needs. Working memory can tolerate eventual consistency because it’s scoped to one agent’s execution. Shared memory requires stronger guarantees because multiple agents depend on it. Systems that don’t distinguish memory types end up either overpersisting transient state (wasting storage and polluting retrieval) or underpersisting durable knowledge (forcing agents to relearn what they should already know).

Pillar 2: Persistence

Persistence determines what survives and for how long. Ephemeral memory lost when agents terminate is insufficient for workflows spanning hours or days—but persisting everything forever creates its own problems. The critical gap in most current approaches, as Bousetouane observes, is that they treat text artifacts as the primary carrier of state without explicit rules governing memory lifecycle. Which memories should become permanent record? Which need revision as context evolves? Which should be actively forgotten? Without answers to these questions, systems accumulate noise alongside signal. Effective persistence requires explicit lifecycle policies: Working memory might live for the duration of a task; episodic memory for weeks or months; and semantic memory indefinitely. Recovery semantics matter too. When an agent fails midtask, what state can be reconstructed? What’s lost? The persistence architecture must handle both planned retention and unplanned recovery.

Pillar 3: Retrieval

Retrieval governs how agents access relevant memory without drowning in noise. Agent memory retrieval differs from document retrieval in several ways. Recency often matters—recent memories typically outweigh older ones for ongoing tasks. Relevance is contextual—the same memory might be critical for one task and distracting for another. Scope varies by memory type—working memory retrieval is narrow and fast, semantic memory retrieval is broader and can tolerate more latency. Standard RAG pipelines treat all content uniformly and optimize for semantic similarity alone. Agent memory systems need retrieval strategies that account for memory type, recency, task context, and agent role simultaneously.

Pillar 4: Coordination

Coordination defines the sharing topology. Which memories are visible to which agents? What can each agent read versus write? How do memory scopes nest or overlap? Without explicit coordination boundaries, teams either overshare—every agent sees everything, creating noise and contamination risk—or undershare—agents operate in isolation, duplicating work and diverging on shared tasks. The coordination model must match the agent team’s structure. A supervisor-worker hierarchy needs different memory visibility than a peer collaboration. A pipeline of sequential agents needs different sharing than agents working in parallel on subtasks.

Pillar 5: Consistency

Consistency handles what happens when memory updates collide. When Agent A and Agent B simultaneously update the same shared state with incompatible values, the system needs a policy. Optimistic concurrency with merge strategies works for many cases—especially when conflicts are rare and resolvable. Some conflicts require escalation to a supervisor agent or human operator. Some domains need strict serialization where only one agent can update certain memories at a time. Silent last-write-wins is almost never correct—it corrupts shared truth without leaving evidence that corruption occurred. The consistency model must also handle ordering: When Agent B reads a memory that Agent A recently updated, does B see the update? The answer depends on the consistency guarantees the system provides, and different memory types may warrant different guarantees.

Han et al.’s survey of multi-agent systems emphasizes that these represent active research problems. The gap between what production systems need and what current frameworks provide remains substantial. Most orchestration frameworks handle message passing well but treat memory as an afterthought—a vector store bolted on for retrieval, with no coherent model for the other four pillars.

Database primitives that enable the pillars

Implementing memory engineering requires a storage layer that can serve as unified operational database, knowledge store, and memory system simultaneously. The requirements cut across traditional database categories: You need document flexibility for evolving memory schemas, vector search for semantic retrieval, full-text search for precise lookups, and transactional consistency for shared state.

MongoDB provides these primitives in a single platform, which is why it appears across so many agent memory implementations—whether teams build custom solutions or integrate through frameworks and memory providers.

Document flexibility matters because memory schemas evolve. A memory unit isn’t a flat string—it’s structured content with metadata, timestamps, source attribution, confidence scores, and associative links to related memories. Teams discover what context agents actually need through iteration. Document databases accommodate this evolution without schema migrations blocking development.

Hybrid retrieval addresses the access pattern problem. Agent memory queries rarely fit a single retrieval mode: A typical query needs memories semantically similar to the current task and created within the last hour and tagged with a specific workflow ID and not marked as superseded. MongoDB Atlas Vector Search combines vector similarity, full-text search, and filtered queries in single operations, avoiding the complexity of stitching together separate retrieval systems.

Atomic operations provide the consistency primitives that coordination requires. When an agent updates task status from pending to complete, the update succeeds entirely or fails entirely. Other agents querying task status never observe partial updates. This is standard MongoDB functionality—findAndModify, conditional updates, multidocument transactions—but it’s infrastructure that simpler storage backends lack.

Change streams enable event-driven architectures. Applications can subscribe to database changes and react when relevant state updates, rather than polling. This becomes a building block for memory systems that need to propagate updates across agents.

Teams implement memory engineering on MongoDB through three paths. Some build directly on the database, using the document model and search capabilities to create custom memory architectures matched to their specific coordination patterns. Others work through orchestration frameworks—LangChain, LlamaIndex, CrewAI—that provide MongoDB integrations for their memory abstractions. Still others adopt dedicated memory providers like Mem0 or Agno, which handle the memory logic while using MongoDB as the underlying storage layer.

The flexibility matters because memory engineering isn’t a single pattern. Different agent architectures need different memory topologies, different consistency guarantees, different retrieval strategies. A database that prescribes one approach would fit some use cases and break others. MongoDB provides primitives; teams compose them into the memory systems their agents require.

Shared memory enables heterogeneous agent teams

Homogeneous systems can be replaced by single agents

The deeper payoff of memory engineering is enabling agent architectures that wouldn’t otherwise be viable.

Xu et al. observe that many deployed multi-agent systems are so homogeneous—same base model everywhere, agents differentiated only by prompts—that a single model can simulate the entire workflow with equivalent results and lower overhead. Their OneFlow optimization demonstrates this by reusing KV cache across simulated “agents” within a single execution, eliminating coordination costs while preserving workflow structure.

The implication: If a single agent can replace your multi-agent system, you haven’t built a team. You’ve built an expensive way to run one model.

Small models need external memory to coordinate

Genuine multi-agent value comes from heterogeneity. Different models with different capabilities operating at different price points for different subtasks. Belcak et al. make the case that most work agents do in production isn’t complex reasoning—it’s routine execution of well-defined operations. Parsing a response, formatting an output, invoking a tool with specific parameters. These tasks don’t require frontier model capabilities, and the cost difference is dramatic: Their analysis puts the gap at 10x–30x between serving a 7B parameter model versus a 70–175B parameter model when you factor in latency, energy, and compute. Large models should be reserved for the genuinely hard problems, not deployed uniformly across every step.

Belcak et al. also highlight an operational advantage: Smaller models can be retrained and adapted much faster. When an agent needs new capabilities or exhibits problematic behaviors, the turnaround for fine-tuning a 7B model is measured in hours, not days. This connects to memory engineering because fine-tuning represents an alternative to retrieval—you can bake procedural knowledge directly into model weights rather than surfacing it from external storage at runtime. The choice between the procedural memory pillar and model specialization becomes a design decision rather than a constraint.

This architecture—small models by default, large models for hard problems—depends on shared memory. Small models can’t maintain the context required for coordination on their own. They rely on external memory to participate in larger workflows. Memory engineering makes heterogeneous teams viable; without it, every agent must be large enough to maintain full context independently, which defeats the cost optimization that motivates heterogeneity in the first place.

Building the foundation

Multi-agent systems fail for structural reasons: context degrades across agents, errors propagate through shared interactions, costs multiply with redundant operations, and state diverges when nothing enforces consistency. These problems don’t resolve with better prompts or more sophisticated orchestration. They require infrastructure.

Memory engineering provides that infrastructure through a coherent taxonomy of memory types, persistence with explicit lifecycle rules, retrieval tuned to agent access patterns, coordination that defines clear sharing boundaries, and consistency that maintains shared truth under concurrent updates.

The organizations that make multi-agent systems work in production won’t be distinguished by agent count or model capability. They’ll be the ones that invested in the memory layer that transforms independent agents into coordinated teams.

References

Anthropic. “Building a Multi-Agent Research System.” 2025. https://www.anthropic.com/engineering/multi-agent-research-system

Belcak, Peter, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. “Small Language Models are the Future of Agentic AI.” arXiv:2506.02153 (2025). https://arxiv.org/abs/2506.02153

Bousetouane, Fouad. “AI Agents Need Memory Control Over More Context.” arXiv:2601.11653 (2026). https://arxiv.org/abs/2601.11653

Breunig, Dan. “How Contexts Fail—and How to Fix Them.” June 22, 2025. https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html

Carnegie Mellon University. “AgentCompany: Building Agent Teams for the Future of Work.” 2025. https://www.cs.cmu.edu/news/2025/agent-company

Cemri, Mert, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. “Why Do Multi-Agent LLM Systems Fail?” arXiv:2503.13657 (2025). https://arxiv.org/abs/2503.13657

Chroma Research. “Context Rot: How Increasing Context Length Degrades Model Performance.” 2025. https://research.trychroma.com/context-rot

Han, Shanshan, Qifan Zhang, Yuhang Yao, Weizhao Jin, and Zhaozhuo Xu. “LLM Multi-Agent Systems: Challenges and Open Problems.” arXiv:2402.03578 (2024). https://arxiv.org/abs/2402.03578

LangChain Blog (Sydney Runkle). “Choosing the Right Multi-Agent Architecture.” January 14, 2026. https://www.blog.langchain.com/choosing-the-right-multi-agent-architecture/

Manus AI. “Context Engineering for AI Agents: Lessons from Building Manus.” 2025. https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus

Schmid, Philipp. “Context Engineering.” 2025. https://www.philschmid.de/context-engineeringXu, Jiawei, Arief Koesdwiady, Sisong Bei, Yan Han, Baixiang Huang, Dakuo Wang, Yutong Chen, Zheshen Wang, Peihao Wang, Pan Li, and Ying Ding. “Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline.” arXiv:2601.12307 (2026). https://arxiv.org/abs/2601.12307

As AI systems move from assistive components to autonomous actors, governance imposed from the outside no longer scales. The problem isn’t that organizations lack policies or oversight frameworks. It’s that those controls are detached from where decisions are actually formed. Increasingly, the only place governance can operate effectively is inside the AI application itself, at runtime, while decisions are being made. This isn’t a philosophical shift. It’s an architectural one.

When AI Fails Quietly

One of the more unsettling aspects of autonomous AI systems is that their most consequential failures rarely look like failures at all. Nothing crashes. Latency stays within bounds. Logs look clean. The system behaves coherently—just not correctly. An agent escalates a workflow that should have been contained. A recommendation drifts slowly away from policy intent. A tool is invoked in a context that no one explicitly approved, yet no explicit rule was violated.

These failures are hard to detect because they emerge from behavior, not bugs. Traditional governance mechanisms don’t help much here. Predeployment reviews assume decision paths can be anticipated in advance. Static policies assume behavior is predictable. Post hoc audits assume intent can be reconstructed from outputs. None of those assumptions holds once systems reason dynamically, retrieve context opportunistically, and act continuously. At that point, governance isn’t missing—it’s simply in the wrong place.

The Scaling Problem No One Owns

Most organizations already feel this tension, even if they don’t describe it in architectural terms. Security teams tighten access controls. Compliance teams expand review checklists. Platform teams add more logging and dashboards. Product teams add additional prompt constraints. Each layer helps a little. None of them addresses the underlying issue.

What’s really happening is that governance responsibility is being fragmented across teams that don’t own system behavior end-to-end. No single layer can explain why the system acted—only that it acted. As autonomy increases, the gap between intent and execution widens, and accountability becomes diffuse. This is a classic scaling problem. And like many scaling problems before it, the solution isn’t more rules. It’s a different system architecture.

A Familiar Pattern from Infrastructure History

We’ve seen this before. In early networking systems, control logic was tightly coupled to packet handling. As networks grew, this became unmanageable. Separating the control plane from the data plane allowed policy to evolve independently of traffic and made failures diagnosable rather than mysterious.

Cloud platforms went through a similar transition. Resource scheduling, identity, quotas, and policy moved out of application code and into shared control systems. That separation is what made hyperscale cloud viable. Autonomous AI systems are approaching a comparable inflection point.

Right now, governance logic is scattered across prompts, application code, middleware, and organizational processes. None of those layers was designed to assert authority continuously while a system is reasoning and acting. What’s missing is a control plane for AI—not as a metaphor but as a real architectural boundary.

What “Governance Inside the System” Actually Means

When people hear “governance inside AI,” they often imagine stricter rules baked into prompts or more conservative model constraints. That’s not what this is about.

Embedding governance inside the system means separating decision execution from decision authority. Execution includes inference, retrieval, memory updates, and tool invocation. Authority includes policy evaluation, risk assessment, permissioning, and intervention. In most AI applications today, those concerns are entangled—or worse, implicit.

A control-plane-based design makes that separation explicit. Execution proceeds but under continuous supervision. Decisions are observed as they form, not inferred after the fact. Constraints are evaluated dynamically, not assumed ahead of time. Governance stops being a checklist and starts behaving like infrastructure.

Reasoning, retrieval, memory, and tool invocation operate in the execution plane, while a runtime control plane continuously evaluates policy, risk, and authority—observing and intervening without being embedded in application logic.

Where Governance Breaks First

In practice, governance failures in autonomous AI systems tend to cluster around three surfaces.

Reasoning. Systems form intermediate goals, weigh options, and branch decisions internally. Without visibility into those pathways, teams can’t distinguish acceptable variance from systemic drift.

Retrieval. Autonomous systems pull in context opportunistically. That context may be outdated, inappropriate, or out of scope—and once it enters the reasoning process, it’s effectively invisible unless explicitly tracked.

Action. Tool use is where intent becomes impact. Systems increasingly invoke APIs, modify records, trigger workflows, or escalate issues without human review. Static authorization models don’t map cleanly onto dynamic decision contexts.

These surfaces are interconnected, but they fail independently. Treating governance as a single monolithic concern leads to brittle designs and false confidence.

Control Planes as Runtime Feedback Systems

A useful way to think about AI control planes is not as gatekeepers but as feedback systems. Signals flow continuously from execution into governance: confidence degradation, policy boundary crossings, retrieval drift, and action escalation patterns. Those signals are evaluated in real time, not weeks later during audits. Responses flow back: throttling, intervention, escalation, or constraint adjustment.

This is fundamentally different from monitoring outputs. Output monitoring tells you what happened. Control plane telemetry tells you why it was allowed to happen. That distinction matters when systems operate continuously, and consequences compound over time.

Behavioral telemetry flows from execution into the control plane, where policy and risk are evaluated continuously. Enforcement and intervention feed back into execution before failures become irreversible.

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

A Failure Story That Should Sound Familiar

Consider a customer-support agent operating across billing, policy, and CRM systems.

Over several months, policy documents are updated. Some are reindexed quickly. Others lag. The agent continues to retrieve context and reason coherently, but its decisions increasingly reflect outdated rules. No single action violates policy outright. Metrics remain stable. Customer satisfaction erodes slowly.

Eventually, an audit flags noncompliant action. At that point, teams scramble. Logs show what the agent did but not why. They can’t reconstruct which documents influenced which decisions, when those documents were last updated, or why the agent believed its actions were valid at the time.

This isn’t a logging failure. It’s the absence of a governance feedback loop. A control plane wouldn’t prevent every mistake, but it would surface drift early—when intervention is still cheap.

Why External Governance Can’t Catch Up

It’s tempting to believe better tooling, stricter reviews, or more frequent audits will solve this problem. They won’t.

External governance operates on snapshots. Autonomous AI operates on streams. The mismatch is structural. By the time an external process observes a problem, the system has already moved on—often repeatedly. That doesn’t mean governance teams are failing. It means they’re being asked to regulate systems whose operating model has outgrown their tools. The only viable alternative is governance that runs at the same cadence as execution.

Authority, Not Just Observability

One subtle but important point: Control planes aren’t just about visibility. They’re about authority.

Observability without enforcement creates a false sense of safety. Seeing a problem after it occurs doesn’t prevent it from recurring. Control planes must be able to act—to pause, redirect, constrain, or escalate behavior in real time.

That raises uncomfortable questions. How much autonomy should systems retain? When should humans intervene? How much latency is acceptable for policy evaluation? There are no universal answers. But those trade-offs can only be managed if governance is designed as a first-class runtime concern, not an afterthought.

The Architectural Shift Ahead

The move from guardrails to control loops mirrors earlier transitions in infrastructure. Each time, the lesson was the same: Static rules don’t scale under dynamic behavior. Feedback does.

AI is entering that phase now. Governance won’t disappear. But it will change shape. It will move inside systems, operate continuously, and assert authority at runtime. Organizations that treat this as an architectural problem—not a compliance exercise—will adapt faster and fail more gracefully. Those who don’t will spend the next few years chasing incidents they can see, but never quite explain.

Closing Thought

Autonomous AI doesn’t require less governance. It requires governance that understands autonomy.

That means moving beyond policies as documents and audits as events. It means designing systems where authority is explicit, observable, and enforceable while decisions are being made. In other words, governance must become part of the system—not something applied to it.

Further Reading

• “AI Governance Frameworks for Responsible AI,” Gartner Peer Community, https://www.gartner.com/peer-community/oneminuteinsights/omi-ai-governance-frameworks-responsible-ai-33q.
• Lauren Kornutick et al., “Market Guide for AI Governance Platforms,” Gartner, November 4, 2025, https://www.gartner.com/en/documents/7145930.
• Svetlana Sicular, “AI’s Next Frontier Demands a New Approach to Ethics, Governance, and Compliance,” Gartner, November 10, 2025, https://www.gartner.com/en/articles/ai-ethics-governance-and-compliance.
• AI Risk Management Framework (AI RMF 1.0), NIST, January 2023, https://doi.org/10.6028/NIST.AI.100-1.

However, this is also where things can get uncomfortable. Everyone in the field knows LLMs are probabilistic. We all track leaderboard scores, but then quietly ignore that this uncertainty compounds when we wire multiple models together. That’s the blind spot. Most multi-agent systems (MAS) don’t fail because the models are bad. They fail because we compose them as if probability doesn’t compound.

The Architectural Debt of Multi-Agent Systems

The hard truth is that improving individual agents does very little to improve overall system-level reliability once errors are allowed to propagate unchecked. The core problem of agentic systems in production isn’t model quality alone; it’s composition. Once agents are wired together without validation boundaries, risk compounds.

In practice, this shows up in looping supervisors, runaway token costs, brittle workflows, and failures that appear intermittently and are nearly impossible to reproduce. These systems often work just well enough to pass benchmarks, then fail unpredictably once they are placed under real operational load.

If you think about it, every agent handoff introduces a chance of failure. Chain enough of them together, and failure compounds. Even strong models with a 98% per-agent success rate can quickly degrade overall system success to 90% or lower. Each unchecked agent hop multiplies failure probability and, with it, expected cost. Without explicit fault tolerance, agentic systems aren’t just fragile. They are economically problematic.

This is the key shift in perspective. In production, MAS shouldn’t be thought of as collections of intelligent components. They behave like probabilistic pipelines, where every unvalidated handoff multiplies uncertainty and expected cost.

This is where many organizations are quietly accumulating what I call architectural debt. In software engineering, we are comfortable talking about technical debt: development shortcuts that make systems harder to maintain over time. Agentic systems introduce a new form of debt. Every unvalidated agent boundary adds probabilistic risk that doesn’t show up in unit tests but surfaces later as instability, cost overruns, and unpredictable behavior at scale. And unlike technical debt, this one doesn’t get paid down with refactors or cleaner code. It accumulates silently, until the math catches up with you.

The Multi-Agent Reliability Tax

If you treat each agent’s task as an independent Bernoulli trial, a simple experiment with a binary outcome of success (p) or failure (q), probability becomes a harsh mistress. Look closely and you’ll find yourself at the mercy of the product reliability rule once you start building MAS. In systems engineering, this effect is formalized by Lusser’s law, which states that when independent components are executed in sequence, overall system success is the product of their individual success probabilities. While this is a simplified model, it captures the compounding effect that is otherwise easy to underestimate in composed MAS.

Consider a high-performing agent with a single-task accuracy of p = 0.98 (98%). If you apply the product rule for independent events to a sequential pipeline, you can model how your total system accuracy unfolds. That is, if you assume each agent succeeds with probability p_i, your failure probability is q_i = 1 − p_i. Applied to a multi-agent pipeline, this gives you:

Table 1 illustrates how your agent system propagates errors through your system without validation.

In production, LLMs aren’t 98% reliable on structured outputs in open-ended tasks. Because they have no single correct output, so correctness must be enforced structurally rather than assumed. Once an agent introduces a wrong assumption, a malformed schema, or a hallucinated tool result, every downstream agent conditions on that corrupted state. This is why you should insert validation gates to break the product rule of reliability.

From Stochastic Hope to Deterministic Engineering

If you introduce validation gates, you change how failure behaves inside your system. Instead of allowing one agent’s output to become the unquestioned input for the next, you force every handoff to pass through an explicit boundary. The system no longer assumes correctness. It verifies it.

In practice, you’d want to have a schema-enforced generation via libraries like Pydantic and Instructor. Pydantic is a data validation library for Python, which helps you define a strict contract for what is allowed to pass between agents: Types, fields, ranges, and invariants are checked at the boundary, and invalid outputs are rejected or corrected before they can propagate. Instructor moves that same contract into the generation step itself by forcing the model to retry until it produces a valid output or exhausts a bounded retry budget. Once validation exists, the reliability math fundamentally changes. Validation catches failures with probability v, now each hop becomes:

Again, assume you have a per-agent accuracy of p = 0.98, but now you have a validation catch rate of v = 0.9, then you get:

The +0.02 · 0.9 term reflects recovered failures, since these events are disjoint. Table 2 shows how this changes your systems behavior.

Comparing Table 1 and Table 2 makes the effect explicit: Validation fundamentally changes how failure propagates through your MAS. It’s no longer a naive multiplicative decay, it’s a controlled reliability amplification. If you want a deeper, implementation-level walkthrough of validation patterns for MAS, I cover it in AI Agents: The Definitive Guide. You can also find a notebook in the GitHub repository to run the computation from Table 1 and Table 2. Now, you might ask what you can do, if you can’t make your models 100% perfect. The good news is that you can make the system more resilient through specific architectural shifts.

From Deterministic Engineering to Exploratory Search

While validation keeps your system from breaking, it doesn’t necessarily help the system find the right answer when the task is difficult. For that, you need to move from filtering to searching. Now you give your agent a way to generate multiple candidate paths to replace fragile one-shot execution with a controlled search over alternatives. This is commonly referred to as test-time compute. Instead of committing to the first sampled output, the system allocates additional inference budget to explore multiple candidates before making a decision. Reliability improves not because your model is better but because your system delays commitment.

At the simplest level, this doesn’t require anything sophisticated. Even a basic best-of-N strategy already improves system stability. For instance, if you sample multiple independent outputs and select the best one, you reduce the chance of committing to a bad draw. This alone is often enough to stabilize brittle pipelines that fail under single-shot execution.

One effective approach to select the best one out of multiple samples is to use frameworks like RULER. RULER (Relative Universal LLM-Elicited Rewards) is a general-purpose reward function which uses a configurable LLM-as-judge along with a ranking rubric you can adjust based on your use case. This works because ranking several related candidate solutions is easier than scoring each one in isolation. By looking at multiple solutions side by side, this allows the LLM-as-judge to identify deficiencies and rank them accordingly. Now you get evidence-anchored verification. The judge doesn’t just agree; it verifies and compares outputs against each other. This acts as a “circuit breaker” for error propagation, by resetting your failure probability at every agent boundary.

Amortized Intelligence with Reinforcement Learning

As a next possible step you could use group-based reinforcement learning (RL), such as group relative policy optimization (GRPO)¹ and group sequence policy optimization (GSPO)² to turn that search into a learned policy. GRPO works on the token level, while GSPO works on the sequence level. You can take your “golden traces” found by your search and adjust your base agents. The golden traces are your successful reasoning paths. Now you aren’t just filtering errors anymore; you’re training the agents to avoid making them in the first place, because your system internalizes those corrections into its own policy. The key shift is that successful decision paths are retained and reused rather than rediscovered repeatedly at inference time.

From Prototypes to Production

If you want your agentic systems to behave reliably in production, I recommend you approach agentic failure in this order:

• Introduce strict validation between agents. Enforce schemas and contracts so failures are caught early instead of propagating silently. 
• Use simple best-of-N sampling or tree-based search with lightweight judges such as RULER to score multiple candidates before committing. 
• If you need consistent behavior at scale use RL to teach your agents how to behave more reliably for your specific use case.

The reality is you won’t be able to fully eliminate uncertainty in your MAS, but these methods give you real leverage over how uncertainty behaves. Reliable agentic systems are build by design, not by chance.

References

1. Zhihong Shao et al. “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models,” 2024, https://arxiv.org/abs/2402.03300.
2. Chujie Zheng et al. “Group Sequence Policy Optimization,” 2025, https://arxiv.org/abs/2507.18071.

This post first appeared on Addy Osmani’s Elevate Substack newsletter and is being republished here with the author’s permission.

TL;DR: Aim for a clear spec covering just enough nuance (this may include structure, style, testing, boundaries. . .) to guide the AI without overwhelming it. Break large tasks into smaller ones versus keeping everything in one large prompt. Plan first in read-only mode, then execute and iterate continuously.

“I’ve heard a lot about writing good specs for AI agents, but haven’t found a solid framework yet. I could write a spec that rivals an RFC, but at some point the context is too large and the model breaks down.”

Many developers share this frustration. Simply throwing a massive spec at an AI agent doesn’t work—context window limits and the model’s “attention budget” get in the way. The key is to write smart specs: documents that guide the agent clearly, stay within practical context sizes, and evolve with the project. This guide distills best practices from my use of coding agents including Claude Code and Gemini CLI into a framework for spec-writing that keeps your AI agents focused and productive.

We’ll cover five principles for great AI agent specs, each starting with a bolded takeaway.

1. Start with a High-Level Vision and Let the AI Draft the Details

Kick off your project with a concise high-level spec, then have the AI expand it into a detailed plan.

Instead of overengineering upfront, begin with a clear goal statement and a few core requirements. Treat this as a “product brief” and let the agent generate a more elaborate spec from it. This leverages the AI’s strength in elaboration while you maintain control of the direction. This works well unless you already feel you have very specific technical requirements that must be met from the start.

Why this works: LLM-based agents excel at fleshing out details when given a solid high-level directive, but they need a clear mission to avoid drifting off course. By providing a short outline or objective description and asking the AI to produce a full specification (e.g., a spec.md), you create a persistent reference for the agent. Planning in advance matters even more with an agent: You can iterate on the plan first, then hand it off to the agent to write the code. The spec becomes the first artifact you and the AI build together.

Practical approach: Start a new coding session by prompting 

You are an AI software engineer. Draft a detailed specification for
[project X] covering objectives, features, constraints, and a step-by-step plan.

Keep your initial prompt high-level: e.g., “Build a web app where users can
track tasks (to-do list), with user accounts, a database, and a simple UI.”

The agent might respond with a structured draft spec: an overview, feature list, tech stack suggestions, data model, and so on. This spec then becomes the “source of truth” that both you and the agent can refer back to. GitHub’s AI team promotes spec-driven development where “specs become the shared source of truth…living, executable artifacts that evolve with the project.” Before writing any code, review and refine the AI’s spec. Make sure it aligns with your vision and correct any hallucinations or off-target details.

Use Plan Mode to enforce planning-first: Tools like Claude Code offer a Plan Mode that restricts the agent to read-only operations—it can analyze your codebase and create detailed plans but won’t write any code until you’re ready. This is ideal for the planning phase: Start in Plan Mode (Shift+Tab in Claude Code), describe what you want to build, and let the agent draft a spec while exploring your existing code. Ask it to clarify ambiguities by questioning you about the plan. Have it review the plan for architecture, best practices, security risks, and testing strategy. The goal is to refine the plan until there’s no room for misinterpretation. Only then do you exit Plan Mode and let the agent execute. This workflow prevents the common trap of jumping straight into code generation before the spec is solid.

Use the spec as context: Once approved, save this spec (e.g., as SPEC.md) and feed relevant sections into the agent as needed. Many developers using a strong model do exactly this. The spec file persists between sessions, anchoring the AI whenever work resumes on the project. This mitigates the forgetfulness that can happen when the conversation history gets too long or when you have to restart an agent. It’s akin to how one would use a product requirements document (PRD) in a team: a reference that everyone (human or AI) can consult to stay on track. Experienced folks often “write good documentation first and the model may be able to build the matching implementation from that input alone” as one engineer observed. The spec is that documentation.

Keep it goal oriented: A high-level spec for an AI agent should focus on what and why more than the nitty-gritty how (at least initially). Think of it like the user story and acceptance criteria: Who is the user? What do they need? What does success look like? (For example, “User can add, edit, complete tasks; data is saved persistently; the app is responsive and secure.”) This keeps the AI’s detailed spec grounded in user needs and outcome, not just technical to-dos. As the GitHub Spec Kit docs put it, provide a high-level description of what you’re building and why, and let the coding agent generate a detailed specification focusing on user experience and success criteria. Starting with this big-picture vision prevents the agent from losing sight of the forest for the trees when it later gets into coding.

2. Structure the Spec Like a Professional PRD (or SRS)

Treat your AI spec as a structured document (PRD) with clear sections, not a loose pile of notes.

Many developers treat specs for agents much like traditional product requirement documents (PRDs) or system design docs: comprehensive, well-organized, and easy for a “literal-minded” AI to parse. This formal approach gives the agent a blueprint to follow and reduces ambiguity.

The six core areas

GitHub’s analysis of over 2,500 agent configuration files revealed a clear pattern: The most effective specs cover six areas. Use this as a checklist for completeness:

1. Commands: Put executable commands early—not just tool names but full commands with flags: npm test, pytest -v, npm run build. The agent will reference these constantly.
2. Testing: How to run tests, what framework you use, where test files live, and what coverage expectations exist.
3. Project structure: Where source code lives, where tests go, where docs belong. Be explicit: “src/ for application code, tests/ for unit tests, docs/ for documentation.”
4. Code style: One real code snippet showing your style beats three paragraphs describing it. Include naming conventions, formatting rules, and examples of good output.
5. Git workflow: Branch naming, commit message format, PR requirements. The agent can follow these if you spell them out.
6. Boundaries: What the agent should never touch—secrets, vendor directories, production configs, specific folders. “Never commit secrets” was the single most common helpful constraint in the GitHub study.

Be specific about your stack: Say “React 18 with TypeScript, Vite, and Tailwind CSS,” not “React project.” Include versions and key dependencies. Vague specs produce vague code.

Use a consistent format: Clarity is king. Many devs use Markdown headings or even XML-like tags in the spec to delineate sections because AI models handle well-structured text better than free-form prose. For example, you might structure the spec as:

# Project Spec: My team's tasks app


## Objective
- Build a web app for small teams to manage tasks...


## Tech Stack
- React 18+, TypeScript, Vite, Tailwind CSS
- Node.js/Express backend, PostgreSQL, Prisma ORM


## Commands
- Build: `npm run build` (compiles TypeScript, outputs to dist/)
- Test: `npm test` (runs Jest, must pass before commits)
- Lint: `npm run lint --fix` (auto-fixes ESLint errors)


## Project Structure
- `src/` – Application source code
- `tests/` – Unit and integration tests
- `docs/` – Documentation


## Boundaries
- ✅ Always: Run tests before commits, follow naming conventions
- ⚠ Ask first: Database schema changes, adding dependencies
- 🚫 Never: Commit secrets, edit node_modules/, modify CI config

This level of organization not only helps you think clearly but also helps the AI find information. Anthropic engineers recommend organizing prompts into distinct sections (like <background>, <instructions>, <tools>, <output_format> etc.) for exactly this reason: It gives the model strong cues about which info is which. And remember, “minimal does not necessarily mean short”—don’t shy away from detail in the spec if it matters, but keep it focused.

Integrate specs into your toolchain: Treat specs as “executable artifacts” tied to version control and CI/CD. The GitHub Spec Kit uses a four-phase gated workflow that makes your specification the center of your engineering process. Instead of writing a spec and setting it aside, the spec drives the implementation, checklists, and task breakdowns. Your primary role is to steer; the coding agent does the bulk of the writing. Each phase has a specific job, and you don’t move to the next one until the current task is fully validated:

1. Specify: You provide a high-level description of what you’re building and why, and the coding agent generates a detailed specification. This isn’t about technical stacks or app design—it’s about user journeys, experiences, and what success looks like. Who will use this? What problem does it solve? How will they interact with it? Think of it as mapping the user experience you want to create, and letting the coding agent flesh out the details. This becomes a living artifact that evolves as you learn more.

2. Plan: Now you get technical. You provide your desired stack, architecture, and constraints, and the coding agent generates a comprehensive technical plan. If your company standardizes on certain technologies, this is where you say so. If you’re integrating with legacy systems or have compliance requirements, all of that goes here. You can ask for multiple plan variations to compare approaches. If you make internal docs available, the agent can integrate your architectural patterns directly into the plan.

3. Tasks: The coding agent takes the spec and plan and breaks them into actual work—small, reviewable chunks that each solve a specific piece of the puzzle. Each task should be something you can implement and test in isolation, almost like test-driven development for your AI agent. Instead of “build authentication,” you get concrete tasks like “create a user registration endpoint that validates email format.”

4. Implement: Your coding agent tackles tasks one by one (or in parallel). Instead of reviewing thousand-line code dumps, you review focused changes that solve specific problems. The agent knows what to build (specification), how to build it (plan), and what to work on (task). Crucially, your role is to verify at each phase: Does the spec capture what you want? Does the plan account for constraints? Are there edge cases the AI missed? The process builds in checkpoints for you to critique, spot gaps, and course-correct before moving forward.

This gated workflow prevents what Willison calls “house of cards code”: fragile AI outputs that collapse under scrutiny. Anthropic’s Skills system offers a similar pattern, letting you define reusable Markdown-based behaviors that agents invoke. By embedding your spec in these workflows, you ensure the agent can’t proceed until the spec is validated, and changes propagate automatically to task breakdowns and tests.

Consider agents.md for specialized personas: For tools like GitHub Copilot, you can create agents.md files that define specialized agent personas—a @docs-agent for technical writing, a @test-agent for QA, a @security-agent for code review. Each file acts as a focused spec for that persona’s behavior, commands, and boundaries. This is particularly useful when you want different agents for different tasks rather than one general-purpose assistant.

Design for agent experience (AX): Just as we design APIs for developer experience (DX), consider designing specs for “agent experience.” This means clean, parseable formats: OpenAPI schemas for any APIs the agent will consume, llms.txt files that summarize documentation for LLM consumption, and explicit type definitions. The Agentic AI Foundation (AAIF) is standardizing protocols like MCP (Model Context Protocol) for tool integration. Specs that follow these patterns are easier for agents to consume and act on reliably.

PRD versus SRS mindset: It helps to borrow from established documentation practices. For AI agent specs, you’ll often blend these into one document (as illustrated above), but covering both angles serves you well. Writing it like a PRD ensures you include user-centric context (“the why behind each feature”) so the AI doesn’t optimize for the wrong thing. Expanding it like an SRS ensures you nail down the specifics the AI will need to actually generate correct code (like what database or API to use). Developers have found that this extra upfront effort pays off by drastically reducing miscommunications with the agent later.

Make the spec a “living document”: Don’t write it and forget it. Update the spec as you and the agent make decisions or discover new info. If the AI had to change the data model or you decided to cut a feature, reflect that in the spec so it remains the ground truth. Think of it as version-controlled documentation. In spec-driven workflows, the spec drives implementation, tests, and task breakdowns, and you don’t move to coding until the spec is validated. This habit keeps the project coherent, especially if you or the agent step away and come back later. Remember, the spec isn’t just for the AI—it helps you as the developer maintain oversight and ensure the AI’s work meets the real requirements.

3. Break Tasks into Modular Prompts and Context, Not One Big Prompt

Divide and conquer: Give the AI one focused task at a time rather than a monolithic prompt with everything at once.

Experienced AI engineers have learned that trying to stuff the entire project (all requirements, all code, all instructions) into a single prompt or agent message is a recipe for confusion. Not only do you risk hitting token limits; you also risk the model losing focus due to the “curse of instructions”—too many directives causing it to follow none of them well. The solution is to design your spec and workflow in a modular way, tackling one piece at a time and pulling in only the context needed for that piece.

The curse of too much context/instructions: Research has confirmed what many devs anecdotally saw: as you pile on more instructions or data into the prompt, the model’s performance in adhering to each one drops significantly. One study dubbed this the “curse of instructions”, showing that even GPT-4 and Claude struggle when asked to satisfy many requirements simultaneously. In practical terms, if you present 10 bullet points of detailed rules, the AI might obey the first few and start overlooking others. The better strategy is iterative focus. Guidelines from industry suggest decomposing complex requirements into sequential, simple instructions as a best practice. Focus the AI on one subproblem at a time, get that done, then move on. This keeps the quality high and errors manageable.

Divide the spec into phases or components: If your spec document is very long or covers a lot of ground, consider splitting it into parts (either physically separate files or clearly separate sections). For example, you might have a section for “backend API spec” and another for “frontend UI spec.” You don’t need to always feed the frontend spec to the AI when it’s working on the backend, and vice versa. Many devs using multi-agent setups even create separate agents or subprocesses for each part (e.g., one agent works on database/schema, another on API logic, another on frontend—each with the relevant slice of the spec). Even if you use a single agent, you can emulate this by copying only the relevant spec section into the prompt for that task. Avoid context overload: Don’t mix authentication tasks with database schema changes in one go, as the DigitalOcean AI guide warns. Keep each prompt tightly scoped to the current goal.

Extended TOC/summaries for large specs: One clever technique is to have the agent build an extended table of contents with summaries for the spec. This is essentially a “spec summary” that condenses each section into a few key points or keywords, and references where details can be found. For example, if your full spec has a section on security requirements spanning 500 words, you might have the agent summarize it to: “Security: Use HTTPS, protect API keys, implement input validation (see full spec §4.2).” By creating a hierarchical summary in the planning phase, you get a bird’s-eye view that can stay in the prompt, while the fine details remain offloaded unless needed. This extended TOC acts as an index: The agent can consult it and say, “Aha, there’s a security section I should look at,” and you can then provide that section on demand. It’s similar to how a human developer skims an outline and then flips to the relevant page of a spec document when working on a specific part.

To implement this, you can prompt the agent after writing the spec: “Summarize the spec above into a very concise outline with each section’s key points and a reference tag.” The result might be a list of sections with one or two sentence summaries. That summary can be kept in the system or assistant message to guide the agent’s focus without eating up too many tokens. This hierarchical summarization approach is known to help LLMs maintain long-term context by focusing on the high-level structure. The agent carries a “mental map” of the spec.

Utilize subagents or “skills” for different spec parts: Another advanced approach is using multiple specialized agents (what Anthropic calls subagents or what you might call “skills”). Each subagent is configured for a specific area of expertise and given the portion of the spec relevant to that area. For instance, you might have a database designer subagent that only knows about the data model section of the spec, and an API coder subagent that knows the API endpoints spec. The main agent (or an orchestrator) can route tasks to the appropriate subagent automatically.

The benefit is each agent has a smaller context window to deal with and a more focused role, which can boost accuracy and allow parallel work on independent tasks. Anthropic’s Claude Code supports this by letting you define subagents with their own system prompts and tools. “Each subagent has a specific purpose and expertise area, uses its own context window separate from the main conversation, and has a custom system prompt guiding its behavior,” as their docs describe. When a task comes up that matches a subagent’s domain, Claude can delegate that task to it, with the subagent returning results independently.

Parallel agents for throughput: Running multiple agents simultaneously is emerging as “the next big thing” for developer productivity. Rather than waiting for one agent to finish before starting another task, you can spin up parallel agents for non-overlapping work. Willison describes this as “embracing parallel coding agents” and notes it’s “surprisingly effective, if mentally exhausting.” The key is scoping tasks so agents don’t step on each other: One agent codes a feature while another writes tests, or separate components get built concurrently. Orchestration frameworks like LangGraph or OpenAI Swarm can help coordinate these agents, and shared memory via vector databases (like Chroma) lets them access common context without redundant prompting.

Single versus multi-agent: When to use each

In practice, using subagents or skill-specific prompts might look like: You maintain multiple spec files (or prompt templates)—e.g., SPEC_backend.md, SPEC_frontend.md—and you tell the AI, “For backend tasks, refer to SPEC_backend; for frontend tasks refer to SPEC_frontend.” Or in a tool like Cursor/Claude, you actually spin up a subagent for each. This is certainly more complex to set up than a single-agent loop, but it mimics what human developers do: We mentally compartmentalize a large spec into relevant chunks. (You don’t keep the whole 50-page spec in your head at once; you recall the part you need for the task at hand, and have a general sense of the overall architecture.) The challenge, as noted, is managing interdependencies: The subagents must still coordinate. (The frontend needs to know the API contract from the backend spec, etc.) A central overview (or an “architect” agent) can help by referencing the subspecs and ensuring consistency.

Focus each prompt on one task/section: Even without fancy multi-agent setups, you can manually enforce modularity. For example, after the spec is written, your next move might be: “Step 1: Implement the database schema.” You feed the agent the database section of the spec only, plus any global constraints from the spec (like tech stack). The agent works on that. Then for Step 2, “Now implement the authentication feature”, you provide the auth section of the spec and maybe the relevant parts of the schema if needed. By refreshing the context for each major task, you ensure the model isn’t carrying a lot of stale or irrelevant information that could distract it. As one guide suggests: “Start fresh: begin new sessions to clear context when switching between major features.” You can always remind the agent of critical global rules (from the spec’s constraints section) each time, but don’t shove the entire spec in if it’s not all needed.

Use in-line directives and code TODOs: Another modularity trick is to use your code or spec as an active part of the conversation. For instance, scaffold your code with // TODO comments that describe what needs to be done, and have the agent fill them one by one. Each TODO essentially acts as a mini-spec for a small task. This keeps the AI laser focused (“implement this specific function according to this spec snippet”), and you can iterate in a tight loop. It’s similar to giving the AI a checklist item to complete rather than the whole checklist at once.

The bottom line: Small, focused context beats one giant prompt. This improves quality and keeps the AI from getting “overwhelmed” by too much at once. As one set of best practices sums up, provide “One Task Focus” and “Relevant info only” to the model, and avoid dumping everything everywhere. By structuring the work into modules—and using strategies like spec summaries or subspec agents—you’ll navigate around context size limits and the AI’s short-term memory cap. Remember, a well-fed AI is like a well-fed function: Give it only the inputs it needs for the job at hand.

4. Build in Self-Checks, Constraints, and Human Expertise

Make your spec not just a to-do list for the agent but also a guide for quality control—and don’t be afraid to inject your own expertise.

A good spec for an AI agent anticipates where the AI might go wrong and sets up guardrails. It also takes advantage of what you know (domain knowledge, edge cases, “gotchas”) so the AI doesn’t operate in a vacuum. Think of the spec as both coach and referee for the AI: It should encourage the right approach and call out fouls.

Use three-tier boundaries: GitHub’s analysis of 2,500+ agent files found that the most effective specs use a three-tier boundary system rather than a simple list of don’ts. This gives the agent clearer guidance on when to proceed, when to pause, and when to stop:

Always do: Actions the agent should take without asking. “Always run tests before commits.” “Always follow the naming conventions in the style guide.” “Always log errors to the monitoring service.”

Ask first: Actions that require human approval. “Ask before modifying database schemas.” “Ask before adding new dependencies.” “Ask before changing CI/CD configuration.” This tier catches high-impact changes that might be fine but warrant a human check.

Never do: Hard stops. “Never commit secrets or API keys.” “Never edit node_modules/ or vendor/.” “Never remove a failing test without explicit approval.” “Never commit secrets” was the single most common helpful constraint in the study.

This three-tier approach is more nuanced than a flat list of rules. It acknowledges that some actions are always safe, some need oversight, and some are categorically off-limits. The agent can proceed confidently on “Always” items, flag “Ask first” items for review, and hard-stop on “Never” items.

Encourage self-verification: One powerful pattern is to have the agent verify its work against the spec automatically. If your tooling allows, you can integrate checks like unit tests or linting that the AI can run after generating code. But even at the spec/prompt level, you can instruct the AI to double-check (e.g., “After implementing, compare the result with the spec and confirm all requirements are met. List any spec items that are not addressed.”). This pushes the LLM to reflect on its output relative to the spec, catching omissions. It’s a form of self-audit built into the process.

For instance, you might append to a prompt: “(After writing the function, review the above requirements list and ensure each is satisfied, marking any missing ones).” The model will then (ideally) output the code followed by a short checklist indicating if it met each requirement. This reduces the chance it forgets something before you even run tests. It’s not foolproof, but it helps.

LLM-as-a-Judge for subjective checks: For criteria that are hard to test automatically—code style, readability, adherence to architectural patterns—consider using “LLM-as-a-Judge.” This means having a second agent (or a separate prompt) review the first agent’s output against your spec’s quality guidelines. Anthropic and others have found this effective for subjective evaluation. You might prompt “Review this code for adherence to our style guide. Flag any violations.” The judge agent returns feedback that either gets incorporated or triggers a revision. This adds a layer of semantic evaluation beyond syntax checks.

Conformance testing: Willison advocates building conformance suites—language-independent tests (often YAML based) that any implementation must pass. These act as a contract: If you’re building an API, the conformance suite specifies expected inputs/outputs, and the agent’s code must satisfy all cases. This is more rigorous than ad hoc unit tests because it’s derived directly from the spec and can be reused across implementations. Include conformance criteria in your spec’s success section (e.g., “Must pass all cases in conformance/api-tests.yaml”).

Leverage testing in the spec: If possible, incorporate a test plan or even actual tests in your spec and prompt flow. In traditional development, we use TDD or write test cases to clarify requirements—you can do the same with AI. For example, in the spec’s success criteria, you might say, “These sample inputs should produce these outputs…” or “The following unit tests should pass.” The agent can be prompted to run through those cases in its head or actually execute them if it has that capability. Willison noted that having a robust test suite is like giving the agents superpowers: They can validate and iterate quickly when tests fail. In an AI coding context, writing a bit of pseudocode for tests or expected outcomes in the spec can guide the agent’s implementation. Additionally, you can use a dedicated “test agent” in a subagent setup that takes the spec’s criteria and continuously verifies the “code agent’s” output.

Bring your domain knowledge: Your spec should reflect insights that only an experienced developer or someone with context would know. For example, if you’re building an ecommerce agent and you know that “products” and “categories” have a many-to-many relationship, state that clearly. (Don’t assume the AI will infer it—it might not.) If a certain library is notoriously tricky, mention pitfalls to avoid. Essentially, pour your mentorship into the spec. The spec can contain advice like “If using library X, watch out for memory leak issue in version Y (apply workaround Z).” This level of detail is what turns an average AI output into a truly robust solution, because you’ve steered the AI away from common traps.

Also, if you have preferences or style guidelines (say, “use functional components over class components in React”), encode that in the spec. The AI will then emulate your style. Many engineers even include small examples in the spec (for instance, “All API responses should be JSON, e.g., {“error”: “message”} for errors.”). By giving a quick example, you anchor the AI to the exact format you want.

Minimalism for simple tasks: While we advocate thorough specs, part of expertise is knowing when to keep it simple. For relatively simple, isolated tasks, an overbearing spec can actually confuse more than help. If you’re asking the agent to do something straightforward (like “center a div on the page”), you might just say, “Make sure to keep the solution concise and do not add extraneous markup or styles.” No need for a full PRD there. Conversely, for complex tasks (like “implement an OAuth flow with token refresh and error handling”), that’s when you break out the detailed spec. A good rule of thumb: Adjust spec detail to task complexity. Don’t underspec a hard problem (the agent will flail or go off-track), but don’t overspec a trivial one (the agent might get tangled or use up context on unnecessary instructions).

Maintain the AI’s “persona” if needed: Sometimes, part of your spec is defining how the agent should behave or respond, especially if the agent interacts with users. For example, if building a customer support agent, your spec might include guidelines like “Use a friendly and professional tone” and “If you don’t know the answer, ask for clarification or offer to follow up rather than guessing.” These kinds of rules (often included in system prompts) help keep the AI’s outputs aligned with expectations. They are essentially spec items for AI behavior. Keep them consistent and remind the model of them if needed in long sessions. (LLMs can “drift” in style over time if not kept on a leash.)

You remain the exec in the loop: The spec empowers the agent, but you remain the ultimate quality filter. If the agent produces something that technically meets the spec but doesn’t feel right, trust your judgement. Either refine the spec or directly adjust the output. The great thing about AI agents is they don’t get offended—if they deliver a design that’s off, you can say, “Actually, that’s not what I intended, let’s clarify the spec and redo it.” The spec is a living artifact in collaboration with the AI, not a one-time contract you can’t change.

Simon Willison humorously likened working with AI agents to “a very weird form of management” and even “getting good results out of a coding agent feels uncomfortably close to managing a human intern.” You need to provide clear instructions (the spec), ensure they have the necessary context (the spec and relevant data), and give actionable feedback. The spec sets the stage, but monitoring and feedback during execution are key. If an AI was a “weird digital intern who will absolutely cheat if you give them a chance,” the spec and constraints you write are how you prevent that cheating and keep them on task.

Here’s the payoff: A good spec doesn’t just tell the AI what to build; it also helps it self-correct and stay within safe boundaries. By baking in verification steps, constraints, and your hard-earned knowledge, you drastically increase the odds that the agent’s output is correct on the first try (or at least much closer to correct). This reduces iterations and those “Why on Earth did it do that?” moments.

5. Test, Iterate, and Evolve the Spec (and Use the Right Tools)

Think of spec writing and agent building as an iterative loop: test early, gather feedback, refine the spec, and leverage tools to automate checks.

The initial spec is not the end—it’s the beginning of a cycle. The best outcomes come when you continually verify the agent’s work against the spec and adjust accordingly. Also, modern AI devs use various tools to support this process (from CI pipelines to context management utilities).

Continuous testing: Don’t wait until the end to see if the agent met the spec. After each major milestone or even each function, run tests or at least do quick manual checks. If something fails, update the spec or prompt before proceeding. For example, if the spec said, “Passwords must be hashed with bcrypt” and you see the agent’s code storing plain text, stop and correct it (and remind the spec or prompt about the rule). Automated tests shine here: If you provided tests (or write them as you go), let the agent run them. In many coding agent setups, you can have an agent run npm test or similar after finishing a task. The results (failures) can then feed back into the next prompt, effectively telling the agent “Your output didn’t meet spec on X, Y, Z—fix it.” This kind of agentic loop (code > test > fix > repeat) is extremely powerful and is how tools like Claude Code or Copilot Labs are evolving to handle larger tasks. Always define what “done” means (via tests or criteria) and check for it.

Iterate on the spec itself: If you discover that the spec was incomplete or unclear (maybe the agent misunderstood something or you realized you missed a requirement), update the spec document. Then explicitly resync the agent with the new spec: “I have updated the spec as follows… Given the updated spec, adjust the plan or refactor the code accordingly.” This way the spec remains the single source of truth. It’s similar to how we handle changing requirements in normal dev, but in this case you’re also the product manager for your AI agent. Keep version history if possible (even just via commit messages or notes), so you know what changed and why.

Utilize context management and memory tools: There’s a growing ecosystem of tools to help manage AI agent context and knowledge. For instance, retrieval-augmented generation (RAG) is a pattern where the agent can pull in relevant chunks of data from a knowledge base (like a vector database) on the fly. If your spec is huge, you could embed sections of it and let the agent retrieve the most relevant parts when needed, instead of always providing the whole thing. There are also frameworks implementing the Model Context Protocol (MCP), which automates feeding the right context to the model based on the current task. One example is Context7 (context7.com), which can auto-fetch relevant context snippets from docs based on what you’re working on. In practice, this might mean the agent notices you’re working on “payment processing” and it pulls the payments section of your spec or documentation into the prompt. Consider leveraging such tools or setting up a rudimentary version (even a simple search in your spec document).

Parallelize carefully: Some developers run multiple agent instances in parallel on different tasks (as mentioned earlier with subagents). This can speed up development (e.g., one agent generates code while another simultaneously writes tests, or two features are built concurrently). If you go this route, ensure the tasks are truly independent or clearly separated to avoid conflicts. (The spec should note any dependencies.) For example, don’t have two agents writing to the same file at once. One workflow is to have an agent generate code and another review it in parallel, or to have separate components built that integrate later. This is advanced usage and can be mentally taxing to manage. (As Willison admitted, running multiple agents is surprisingly effective, if mentally exhausting!) Start with at most 2–3 agents to keep things manageable.

Version control and spec locks: Use Git or your version control of choice to track what the agent does. Good version control habits matter even more with AI assistance. Commit the spec file itself to the repo. This not only preserves history, but the agent can even use git diff or blame to understand changes. (LLMs are quite capable of reading diffs.) Some advanced agent setups let the agent query the VCS history to see when something was introduced—surprisingly, models can be “fiercely competent at Git.” By keeping your spec in the repo, you allow both you and the AI to track evolution. There are tools (like GitHub Spec Kit mentioned earlier) that integrate spec-driven development into the Git workflow—for instance, gating merges on updated specs or generating checklists from spec items. While you don’t need those tools to succeed, the takeaway is to treat the spec like code: Maintain it diligently.

Cost and speed considerations: Working with large models and long contexts can be slow and expensive. A practical tip is to use model selection and batching smartly. Perhaps use a cheaper/faster model for initial drafts or repetitions, and reserve the most capable (and expensive) model for final outputs or complex reasoning. Some developers use GPT-4 or Claude for planning and critical steps, but offload simpler expansions or refactors to a local model or a smaller API model. If using multiple agents, maybe not all need to be top tier; a test-running agent or a linter agent could be a smaller model. Also consider throttling context size: Don’t feed 20K tokens if 5K will do. As we discussed, more tokens can mean diminishing returns.

Monitor and log everything: In complex agent workflows, logging the agent’s actions and outputs is essential. Check the logs to see if the agent is deviating or encountering errors. Many frameworks provide trace logs or allow printing the agent’s chain of thought (especially if you prompt it to think step-by-step). Reviewing these logs can highlight where the spec or instructions might have been misinterpreted. It’s not unlike debugging a program—except the “program” is the conversation/prompt chain. If something weird happens, go back to the spec/instructions to see if there was ambiguity.

Learn and improve: Finally, treat each project as a learning opportunity to refine your spec-writing skill. Maybe you’ll discover that a certain phrasing consistently confuses the AI, or that organizing spec sections in a certain way yields better adherence. Incorporate those lessons into the next spec. The field of AI agents is rapidly evolving, so new best practices (and tools) emerge constantly. Stay updated via blogs (like the ones by Simon Willison, Andrej Karpathy, etc.), and don’t hesitate to experiment.

A spec for an AI agent isn’t “write once, done.” It’s part of a continuous cycle of instructing, verifying, and refining. The payoff for this diligence is substantial: By catching issues early and keeping the agent aligned, you avoid costly rewrites or failures later. As one AI engineer quipped, using these practices can feel like having “an army of interns” working for you, but you have to manage them well. A good spec, continuously maintained, is your management tool.

Avoid Common Pitfalls

Before wrapping up, it’s worth calling out antipatterns that can derail even well-intentioned spec-driven workflows. The GitHub study of 2,500+ agent files revealed a stark divide: “Most agent files fail because they’re too vague.” Here are the mistakes to avoid:

Vague prompts: “Build me something cool” or “Make it work better” gives the agent nothing to anchor on. As Baptiste Studer puts it: “Vague prompts mean wrong results.” Be specific about inputs, outputs, and constraints. “You are a helpful coding assistant” doesn’t work. “You are a test engineer who writes tests for React components, follows these examples, and never modifies source code” does.

Overlong contexts without summarization: Dumping 50 pages of documentation into a prompt and hoping the model figures it out rarely works. Use hierarchical summaries (as discussed in principle 3) or RAG to surface only what’s relevant. Context length is not a substitute for context quality.

Skipping human review: Willison has a personal rule—“I won’t commit code I couldn’t explain to someone else.” Just because the agent produced something that passes tests doesn’t mean it’s correct, secure, or maintainable. Always review critical code paths. The “house of cards” metaphor applies: AI-generated code can look solid but collapse under edge cases you didn’t test.

Conflating vibe coding with production engineering: Rapid prototyping with AI (“vibe coding”) is great for exploration and throwaway projects. But shipping that code to production without rigorous specs, tests, and review is asking for trouble. I distinguish “vibe coding” from “AI-assisted engineering”—the latter requires the discipline this guide describes. Know which mode you’re in.

Ignoring the “lethal trifecta”: Willison warns of three properties that make AI agents dangerous: speed (they work faster than you can review), nondeterminism (same input, different outputs), and cost (encouraging corner cutting on verification). Your spec and review process must account for all three. Don’t let speed outpace your ability to verify.

Missing the six core areas: If your spec doesn’t cover commands, testing, project structure, code style, git workflow, and boundaries, you’re likely missing something the agent needs. Use the six-area checklist from section 2 as a sanity check before handing off to the agent.

Conclusion

Writing an effective spec for AI coding agents requires solid software engineering principles combined with adaptation to LLM quirks. Start with clarity of purpose and let the AI help expand the plan. Structure the spec like a serious design document, covering the six core areas and integrating it into your toolchain so it becomes an executable artifact, not just prose. Keep the agent’s focus tight by feeding it one piece of the puzzle at a time (and consider clever tactics like summary TOCs, subagents, or parallel orchestration to handle big specs). Anticipate pitfalls by including three-tier boundaries (always/ask first/never), self-checks, and conformance tests—essentially, teach the AI how to not fail. And treat the whole process as iterative: use tests and feedback to refine both the spec and the code continuously.

Follow these guidelines and your AI agent will be far less likely to “break down” under large contexts or wander off into nonsense.

Happy spec-writing!

First, you provision them tools. Email access. Slack. CRM. Office software. Project management software. Development environment. Connecting a person to the system they’ll need to do their job. However, this is necessary but not sufficient. Nobody becomes effective just because they can log into Salesforce.

Then comes the harder part: teaching them how your organization actually works. The analysis methodology your team developed over years of iteration. The quality bar that is not written down anywhere. The implicit ways of working. The judgment calls about when to escalate and when to handle something independently. The institutional knowledge that separates a new hire from someone who’s been there for years.

This second part—the expertise transfer—is where organizations struggle. It’s expensive and inconsistent, and does not scale. It lives in mentorship relationships, institutional knowledge, and documentation that goes stale the moment it’s written.

Claude Skills and MCP (Model Context Protocol) follow exactly this pattern. MCP gives AI agents such as Claude the tools: access to systems, databases, APIs, and resources. Skills are the training materials that teach Claude how to work and how to use these tools.

This distinction matters more than it might first appear. While we have gotten reasonably good at provisioning tools, we have never had a good way to package expertise. Skills change that. They package expertise into a standardized format.

Tools Versus Training

MCP is tool provisioning. It’s the protocol that connects AI agents to external systems: data warehouse, CRM, GitHub repositories, internal APIs, and knowledge bases. Anthropic describes it as “USB-C for AI”—a standardized interface that lets Claude plug into your existing infrastructure. An MCP server might give Claude the ability to query customer records, commit code, send Slack messages, or pull analytics data with authorized permissions.

This is necessary infrastructure. But like giving a new hire database credentials, it does not tell AI agents what to do with that access. MCP answers the question “What tools can an agent use?” It provides capabilities without opinions.

Skills are the training materials. They encode how your organization actually works: which segments matter, what churn signal to watch for, how to structure findings for your quarterly business review, when to flag something for human attention.

Skills answer a different question: “How should an AI agent think about this?” They provide expertise, not just access.

Consider the difference in what you’re creating. Building an MCP server is infrastructure work; it’s an engineering effort to connect systems securely and reliably. Creating a Skill is knowledge work; domain experts articulating what they know, in markdown files, for AI agents to operationalize and understand. These require different people, different processes, and different governance.

The real power emerges when you combine them. MCP connects AI agents to your data warehouse. A Skill teaches AI agents your firm’s analysis methodology and which MCP tools to use. Together, AI agents can perform expert-level analysis on live data, following your specific standards. Neither layer alone gets you there, just as a new hire with database access but no training, or training but no access, won’t be effective at their jobs.

MCP is the toolbox. Skills are the training manuals that teach how to use those tools.

Why Expertise Has Been So Hard to Scale

The training side of onboarding has always been the bottleneck.

Your best analyst retires, and their methods walk out of the door. Onboarding takes months because the real tacit knowledge lives in people’s heads, not in any document a new hire can read. Consistency is impossible when “how we do things here” varies by who trained whom and who worked with whom. Even when you invest heavily in training programs, they produce point-in-time snapshots of expertise that immediately begin to rot.

Previous approaches have all fallen short:

Documentation is passive and quickly outdated. It requires human interpretation, offers no guarantee of correct application, and can’t adapt to novel situations. The wiki page about customer analysis does not help when you encounter an edge case the author never anticipated.

Training programs are expensive, and a certificate of completion says nothing about actual competency.

Checklists and SOPs capture procedure but not judgment. They tell you what to check, not how to think about what you find. They work for mechanical tasks but fail for anything requiring expertise.

We’ve had Custom GPTs, Claude projects, and Gemini Gems attempting to address this. They are useful but opaque. You cannot invoke them based on context; the AI agent working as Copy Editing Gem stays in copy editing and can’t switch to Laundry Buddy Custom GPTs mid-task. They are not transferable and cannot be packaged for distribution.

Skills offer something new: expertise packaged as a versionable, governable artifact.

Skills are files in folders—a SKILL.md document with supporting assets, scripts, and resources. They leverage all the tooling we have built for managing code. Track changes in Git. Roll back mistakes. Maintain audit trails. Review Skills before deployment through PR workflows with version control. Deploy organization-wide and ensure consistency. AI agents can compose Skills for complex workflows, building sophisticated capabilities from simple building blocks.

The architecture also enables progressive disclosure. AI agents see only lightweight metadata until a Skill becomes relevant, then loads the full instruction on demand. You can have dozens of Skills available without overwhelming the model’s precious context window, which is like a human’s short-term memory or a computer’s RAM. Claude loads expertise as needed and coordinates multiple Skills automatically.

This makes the enterprise deployment model tractable. An expert creates a Skill based on best practices, with the help of an AI/ML engineer to audit and evaluate the effectiveness of the Skill. Administrators review and approve it through governance processes. The organization deploys it everywhere simultaneously. Updates propagate instantly from a central source.

One report cites Rakuten achieving 87.5% faster completion of a finance workflow after implementing Skills. Not from AI magic but from finally being able to distribute their analysts’ methodologies across the entire team. That’s the expertise transfer problem, solved.

Training Materials You Can Meter

The onboarding analogy also created a new business model.

When expertise lives in people, you can only monetize it through labor—billable hours, consulting engagements, training programs, maintenance contracts. The expert has to show up, which limits scale and creates key-person dependencies.

Skills separate expertise from the expert. Package your methodology as a Skill. Distribute it via API. Charge based on usage.

A consulting firm’s analysis framework can become a product. A domain expert’s judgment becomes a service. The Skill encodes the expertise; the API calls become the meter. This is service as software, the SaaS of expertise. And it’s only possible because Skills put knowledge in a form that can be distributed, versioned, and billed against.

The architecture is familiar. The Skill is like an application frontend (the expertise, the methodology, the “how”), while MCP connections or API calls form the backend (data access, actions, the “what”). You build training material once and deploy them everywhere, metering usage through the infrastructure layer.

No more selling API endpoints with 500-page obscure documentation explaining what each endpoint does then staffing a team to support it. Now we can package the expertise of how to use those API directly into Skills. Customers can realize the value of an API via their AI agents. Cost to implement and time to implement drop to zero with MCP. Time to value becomes immediate with Skills.

The Visibility Trade-Off

Every abstraction has a cost. Skills trade visibility for scalability, and that trade-off deserves honest examination.

When expertise transfers human to human, through mentorship, working sessions, apprenticeship, the expert sees how their knowledge gets applied and becomes better in the process. They watch the learner struggle with edge cases. They notice which concepts don’t land. They observe how their methods get adapted to new situations. This feedback loop improves the expertise over time.

Skills break that loop. As a Skill builder, you do not see the conversations that trigger your Skill. You do not know how users adapted your methodology or which part of your guidance AI agents weighted most heavily. Users interact with their own AI agents; your Skill is one influence among many.

Your visibility is limited to the infrastructure layer: API calls, MCP tool invocations, and whatever outputs you explicitly capture. You see usage patterns, not the dialogue that surrounds them. Those dialogues reside with the user’s AI agents.

This parallels what happened when companies moved from in-person training to self-service documentation and e-learning. You lost the ability to watch every learner, but you gained the ability to train at scale. Skills make the same exchange; less visibility per user interaction, vastly more interactions possible.

Managing the trade-off requires intentional design. Build logging and tracing into your Skills where appropriate. Create feedback mechanisms inside skills for AI agents to surface when users express confusion or request changes. And in the development process, focus on outcomes—Did the Skill produce good results?—rather than process observation.

In production, the developer of Skills or MCPs will not have most of the context of how a user’s AI agent uses their Skills.

What to Watch

For organizations going through AI transformations, the starting point is an audit of expertise. What knowledge lives only in a specific person’s head? Where does inconsistency emerge because “how we do things” isn’t written down in an operationalizable form? These are your candidates for Skills.

Start with bounded workflows: a report format, an analysis methodology, a review checklist. Prove the pattern before encoding more complex expertise. Govern early. Skills are artifacts that require review, evaluation, and lifecycle management. Establish those processes before Skills proliferate.

For builders, the mental shift is from “prompt” to “product.” Skills are versioned artifacts with users. Design accordingly. Combine Skills with MCP for maximum leverage. Accept the visibility trade-off as the cost of scale.

Several signals suggest where this is heading. Skill marketplaces are emerging. Agent Skills are now a published open standard being adopted by multiple AI agents and soon agent SDKs. Enterprise governance tooling with version control, approval workflows, and audit trails organizations need will determine adoption in regulated industries.

Expertise Can Finally Be Packaged

We’ve gotten good at provisioning tools as APIs. MCP extends that to AI with standardized connections to systems and data.

But tools access was never the bottleneck. Expertise transfer was. The methodology. The judgment. The caveats. The workflows. The institutional knowledge that separates a new hire from a veteran.

Skills are the first serious attempt to package the expertise into a file format, where AI agents can operationalize it while humans can still read, review, and govern. They are training materials that actually scale.

The organizations that figure out how to package their expertise, both for internal and external consumption, will have a structural advantage. Not because AI replaces expertise. Because AI amplifies the expertise of those who know how to share it.

MCP gives AI agents the tools. Skills teach AI agents how to work. The question is whether you can encode what your best people know. Skills are the first real answer.

References

• “What Is the Model Context Protocol (MCP)?,” LF Projects, https://modelcontextprotocol.io/docs/getting-started/intro.
• Michael Nuñez, “How Anthropic’s ‘Skills’ Make Claude Faster, Cheaper, and More Consistent for Business Workflows,” VentureBeat, October 16, 2025, https://venturebeat.com/ai/how-anthropics-skills-make-claude-faster-cheaper-and-more-consistent-for.
• “Skills,” Anthropics, https://github.com/anthropics/skills.
• “Create and Distribute a Plugin Marketplace,” Claude Code Docs, https://code.claude.com/docs/en/plugin-marketplaces.

The following article includes clips from a recent Live with Tim O’Reilly interview. You can watch the full version on the O’Reilly Media learning platform.

Addy Osmani is one of my favorite people to talk with about the state of software engineering with AI. He spent 14 years leading Chrome’s developer experience team at Google, and recently moved to Google Cloud AI to focus on Gemini and agent development. He’s also the author of numerous books for O’Reilly, including The Effective Software Engineer (due out in March), and my cohost for O’Reilly’s AI Codecon. Every time I talk with him I come away feeling like I have a better grip on what’s real and what’s noise. Our recent conversation on Live with Tim O’Reilly was no exception.

Here are some of the things we talked about.

The hard problem is coordination, not generation

Addy pointed out that there’s a spectrum in how people are working with AI agents right now. On one end you have solo founders running hundreds or thousands of agents, sometimes without even reviewing the code. On the other end you have enterprise teams with quality gates, reliability requirements, and long-term maintenance to think about.

Addy’s take is that for most businesses, “the real frontier is not necessarily having hundreds of agents for a task just for its own sake. It’s about orchestrating a modest set of agents that solve real problems while maintaining control and traceability.” He pointed out that frameworks like Google’s Agent Development Kit now support both deterministic workflow agents and dynamic LLM agents in the same system, so you get to choose when you need predictability and when you need flexibility.

The ecosystem is developing fast. A2A (the agent-to-agent protocol Google contributed to the Linux Foundation) handles agent-to-agent communication while MCP handles agent-to-tool calls. Together they start to look like the TCP/IP of the agent era. But Addy was clear-eyed about where things stand: “Almost nobody’s figured out how to make everything work together as smoothly as possible. We’re getting as close to that as we can. And that’s the actual hard problem here. Not generation, but coordination.”

The “Something Big Is Happening” debate

In response to one of the audience questions, we spent some time on Matt Shumer’s viral essay arguing that the current moment in AI is like just before the COVID-19 epidemic hit. Those in the know were sounding the alarm, but most people weren’t hearing it.

Addy’s take was that “it felt a little bit like somebody who hadn’t been following along, just finally getting around to trying out the latest models and tools and having an epiphany moment.” He thinks the piece lacked grounding in data and didn’t do a great job distinguishing between what AI can do for prototypes and what it can do in production. As Addy put it, “Yes, the models are getting better, the harnesses are getting better, the tools are getting better. I can do more with AI these days than I could a year ago. All of that is true. But to say that all kinds of technical work can now be done with near perfection, I wouldn’t personally agree with that statement.”

I agree with Addy, but I also know how it feels when you see the future crashing in and no one is paying attention. At O’Reilly, we started working with the web when there were only 200 websites. In 1993, we built GNN, the first web portal, and the web’s first advertising. In 1994, we did the first large-scale market research on the potential of advertising as the web’s future business model. We went around lobbying phone companies to adopt the web and (a few years later) for bookstores to pay attention to the rise of Amazon, and nobody listened. I’m a big believer in “something is happening” moments. But I’m also very aware that it always takes longer than it appears.

Both things can be true. The direction and magnitude of this shift are real. The models keep getting better. The harnesses keep getting better. But we still have to figure out new kinds of businesses and new kinds of workflows. AI won’t be a tsunami that wipes everything away overnight.

Addy and I will be cohosting the O’Reilly AI Codecon: Software Craftsmanship in the Age of AI on March 26, where we’ll go much deeper on orchestration, agent coordination, and the new skills developers need. We’d love to see you there. Sign up for free here.

And if you’re interested in presenting at AI Codecon, our CFP is open through this Friday, February 20. Check out what we’re looking for and submit your proposal here.

Feeling productive vs. being productive

There was a great line from a post by Will Manidis called “Tool Shaped Objects” that I shared during our conversation: “The market for feeling productive is orders of magnitude larger than the market for being productive.” The essay is about things that feel amazing to build and use but aren’t necessarily doing the work that needs to be done.

Addy picked up on this immediately. “There is a difference between feeling busy and being productive,” he said. “You can have 100 agents working in the background and feel like you’re being productive. And then someone asks, What did you get built? How much money is it making you?”

This isn’t to dismiss anyone who’s genuinely productive running lots of agents. Some people are. But a healthy skepticism about your own productivity is worth maintaining, especially when the tools make it so easy to feel like you’re moving fast.

Planning is the new coding

Addy talked about how the balance of his time on a task has shifted significantly. “I might spend 30, 40% of the time a task takes just to actually write out what exactly is it that I want,” he said. What are the constraints? What are the success criteria? What’s the architecture? What libraries and UI components should be used?

All of that work to get clarity before you start code generation leads to much-higher-quality outcomes from AI. As Addy put it, “LLMs are very good at grounding things in the lowest common denominator. If there are patterns in the training data that are popular, they’re going to use those unless you tell them otherwise.” If your team has established best practices, codify them in Markdown files or MCP tools so the agent can use them.

I connected the planning phase to something larger about taste. Think about Steve Jobs. He wasn’t a coder. He was a master of knowing what good looked like and driving those who worked with him to achieve it. In this new world, that skill matters enormously. You’re going to be like Jobs telling his engineers “no, no, not that” and giving them a vision of what’s beautiful and powerful. Except now some of those engineers are agents. So management skill, communication skill, and taste are becoming core technical competencies.

Code review is getting harder

One thing Addy flagged that doesn’t get enough attention: “Increasingly teams feel like they’re being thrashed with all of these PRs that are AI generated. People don’t necessarily understand everything that’s in there. And you have to balance increased velocity expectations with ‘What is a quality bar?’ because someone’s going to have to maintain this.”

Knowing your quality bar matters. What are the cases where you’re comfortable merging an AI-generated change? Maybe it’s small and well-compartmentalized and has solid test coverage. And what are the cases where you absolutely need deep human review? Getting clear on that distinction is one of the most practical things a team can do right now.

Yes, young people should still go into software

We got a question about whether students should still pursue software engineering. Addy’s answer was emphatic: “There has never been a better time to get into software engineering if you are someone that is comfortable with learning. You do not necessarily have the burden of decades of knowing how things have historically been built. You can approach this with a very fresh set of eyes.” New entrants can go agent first. They can get deep into orchestration patterns and model trade-offs without having to unlearn old habits. And that’s a real advantage when interviewing at companies that need people who already know how to work this way.

The more important point is that in the early days of a new technology, people basically try to make the old things over again. The really big opportunities come when we figure out what was previously impossible that we can now do. If AI is as powerful as it appears to be, the opportunity isn’t to make companies more efficient at the same old work. It’s to solve entirely new problems and build entirely new kinds of products.

I’m 71 years old and 45 years into this industry, and this is the most excited I’ve ever been. More than the early web, more than open source. The future is being reinvented, and the people who start using these tools now get to be part of inventing it.

The token cost question

Addy had a funny and honest admission: “There were weeks when I would look at my bill for how much I was using in tokens and just be shocked. I don’t know that the productivity gains were actually worthwhile.”

His advice: experiment. Get a sense of what your typical tasks cost with multiple agents. Extrapolate. Ask yourself whether you’d still find it valuable at that price. Some people spend hundreds or even thousands a month on tokens and feel it’s worthwhile because the alternative was hiring a contractor. Others are spending that much and mostly feeling busy. As Addy said, “Don’t feel like you have to be spending a huge amount of money to not miss out on productivity wins.”

I’d add that we’re in a period where these costs are massively subsidized. The model companies are covering inference costs to get you locked in. Take advantage of that while it lasts. But also recognize that a lot of efficiency work is yet to be done. Just as JavaScript frameworks replaced everyone hand-coding UIs, we’ll get frameworks and tools that make agent workflows much more token-efficient than they are today.

2028 predictions are already here

One of the most striking things Addy shared was that a group in the AI coding community that he is part of had put together predictions for what software engineering would look like by 2028. “We recently revisited that list, and I was kind of shocked to discover that almost everything on that list is already possible today,” he said. “But how quickly the rest of the ecosystem adopts these things is on a longer trajectory than what is possible.”

That gap between capability and adoption is where most of the interesting work will happen over the next few years. The technology is running ahead of our ability to absorb it. Figuring out how to close that gap, in your team, your company, and your own practice, is the real job right now.

Agents writing code for agents

Near the end we answered another great audience question: Will agents eventually produce source code that’s optimized for other agents to read, not humans? Addy said yes. There are already platform teams having conversations about whether to build for an agent-first world where human readability becomes a secondary concern.

I have a historical parallel for this. I wrote the manual for the first C compiler on the Mac, and I worked closely with the developer who was hand-tuning the compiler output at the machine code level. That was about 30 years ago. We stopped doing that. And I’m quite confident there will be a similar moment with AI-generated code where humans mostly just let it go and trust the output. There will be special cases where people dive in for absolute performance or correctness. But they’ll be rare.

That transition won’t happen overnight. But the direction seems pretty clear. You can help to invent the future now, or spend time later trying to catch up with those who do.

This conversation was part of my ongoing series of discussions with innovators, Live with Tim O’Reilly. You can explore past episodes on the O’Reilly learning platform.

AI quietly breaks that assumption.

As large language models and AI services make their way into production systems, they often arrive through familiar shapes. There’s an API endpoint, a request payload, and a response body. Latency, retries, and timeouts all look manageable. From an architectural distance, it feels natural to treat these systems like libraries or external services.

In practice, that familiarity is misleading. AI systems behave less like deterministic components and more like nondeterministic collaborators. The same prompt can produce different outputs, small changes in context can lead to disproportionate shifts in results, and even retries can change behavior in ways that are difficult to reason about. These characteristics aren’t bugs; they’re inherent to how these systems work. The real problem is that our architectures often pretend otherwise. Instead of asking how to integrate AI as just another dependency, we need to ask how to design systems around components that do not guarantee stable outputs. Framing AI as a nondeterministic dependency turns out to be far more useful than treating it like a smarter API.

One of the first places where this mismatch becomes visible is retries. In deterministic systems, retries are usually safe. If a request fails due to a transient issue, retrying increases the chance of success without changing the outcome. With AI systems, retries don’t simply repeat the same computation. They generate new outputs. A retry might fix a problem, but it can just as easily introduce a different one. In some cases, retries quietly amplify failure rather than mitigate it, all while appearing to succeed.

Testing reveals a similar breakdown in assumptions. Our existing testing strategies depend on repeatability. Unit tests validate exact outputs. Integration tests verify known behaviors. With AI in the loop, those strategies quickly lose their effectiveness. You can test that a response is syntactically valid or conforms to certain constraints, but asserting that it is “correct” becomes far more subjective. Matters get even more complicated as models evolve over time. A test that passed yesterday may fail tomorrow without any code changes, leaving teams unsure whether the system regressed or simply changed.

Observability introduces an even subtler challenge. Traditional monitoring excels at detecting loud failures. Error rates spike. Latency increases. Requests fail. AI-related failures are often quieter. The system responds. Downstream services continue. Dashboards stay green. Yet the output is incomplete, misleading, or subtly wrong in context. These “acceptable but wrong” outcomes are far more damaging than outright errors because they erode trust gradually and are difficult to detect automatically.

Once teams accept nondeterminism as a first-class concern, design priorities begin to shift. Instead of trying to eliminate variability, the focus moves toward containing it. That often means isolating AI-driven functionality behind clear boundaries, limiting where AI outputs can influence critical logic, and introducing explicit validation or review points where ambiguity matters. The goal isn’t to force deterministic behavior from an inherently probabilistic system but to prevent that variability from leaking into parts of the system that aren’t designed to handle it.

This shift also changes how we think about correctness. Rather than asking whether an output is correct, teams often need to ask whether it is acceptable for a given context. That reframing can be uncomfortable, especially for engineers accustomed to precise specifications, but it reflects reality more accurately. Acceptability can be constrained, measured, and improved over time, even if it can’t be perfectly guaranteed.

Observability needs to evolve alongside this shift. Infrastructure-level metrics are still necessary, but they’re no longer sufficient. Teams need visibility into outputs themselves: how they change over time, how they vary across contexts, and how those variations correlate with downstream outcomes. This doesn’t mean logging everything, but it does mean designing signals that surface drift before users notice it. Qualitative degradation often appears long before traditional alerts fire, if anyone is paying attention.

One of the hardest lessons teams learn is that AI systems don’t offer guarantees in the way traditional software does. What they offer instead is probability. In response, successful systems rely less on guarantees and more on guardrails. Guardrails constrain behavior, limit blast radius, and provide escape hatches when things go wrong. They don’t promise correctness, but they make failure survivable. Fallback paths, conservative defaults, and human-in-the-loop workflows become architectural features rather than afterthoughts.

For architects and senior engineers, this represents a subtle but important shift in responsibility. The challenge isn’t choosing the right model or crafting the perfect prompt. It’s reshaping expectations, both within engineering teams and across the organization. That often means pushing back on the idea that AI can simply replace deterministic logic, and being explicit about where uncertainty exists and how the system handles it.

If I were starting again today, there are a few things I would do earlier. I would document explicitly where nondeterminism exists in the system and how it’s managed rather than letting it remain implicit. I would invest sooner in output-focused observability, even if the signals felt imperfect at first. And I would spend more time helping teams unlearn assumptions that no longer hold, because the hardest bugs to fix are the ones rooted in outdated mental models.

AI isn’t just another dependency. It challenges some of the most deeply ingrained assumptions in software engineering. Treating it as a nondeterministic dependency doesn’t solve every problem, but it provides a far more honest foundation for system design. It encourages architectures that expect variation, tolerate ambiguity, and fail gracefully.

That shift in thinking may be the most important architectural change AI brings, not because the technology is magical but because it forces us to confront the limits of determinism we’ve relied on for decades.

Here’s the paradox: Agent2Agent (A2A) and the Agent Communication Protocol (ACP) are so effective at eliminating integration friction that they’ve removed the natural “brakes“ that used to force governance conversations. We’ve solved the plumbing problem brilliantly. In doing so, we’ve created a new class of integration debt—one where organizations borrow speed today at the cost of accountability tomorrow.

The technical protocols are solid. The organizational protocols are missing. We’re rapidly moving from the “Can these systems connect?“ phase to the “Who authorized this agent to liquidate a position at 3 am?“ phase. In practice, that creates a governance gap: Our ability to connect agents is outpacing our ability to control what they commit us to.

To see why that shift is happening so fast, it helps to look at how the underlying “agent stack“ is evolving. We’re seeing the emergence of a three-tier structure that quietly replaces traditional API-led connectivity:

This stack makes multi-agent workflows a configuration problem instead of a custom engineering project. That is exactly why the risk surface is expanding faster than most CISOs realize.

Think of it this way: A2A is the handshake between agents (who talks to whom, about what tasks). ACP is the briefing document they exchange (what context, history, and goals move in that conversation). MCP is the toolbox each agent has access to locally. Once you see the stack this way, you also see the next problem: We’ve solved API sprawl and quietly replaced it with something harder to see—agent sprawl, and with it, a widening governance gap.

Most enterprises already struggle to govern hundreds of SaaS applications. One analysis puts the average at more than 370 SaaS apps per organization. Agent protocols do not reduce this complexity; they route around it. In the API era, humans filed tickets to trigger system actions. In the A2A era, agents use “Agent Cards“ to discover each other and negotiate on top of those systems. ACP allows these agents to trade rich context—meaning a conversation starting in customer support can flow into fulfillment and partner logistics with zero human handoffs. What used to be API sprawl is becoming dozens of semiautonomous processes acting on behalf of your company across infrastructure you do not fully control. The friction of manual integration used to act as a natural brake on risk; A2A has removed that brake.

That governance gap doesn’t usually show up as a single catastrophic failure. It shows up as a series of small, confusing incidents where everything looks “green“ in the dashboards but the business outcome is wrong. The protocol documentation focuses on encryption and handshakes but ignores the emergent failure modes of autonomous collaboration. These are not bugs in the protocols; they’re signs that the surrounding architecture has not caught up with the level of autonomy the protocols enable.

Policy drift: A refund policy encoded in a service agent may technically interoperate with a partner’s collections agent via A2A, but their business logic may be diametrically opposed. When something goes wrong, nobody owns the end-to-end behavior.

Context oversharing: A team might expand an ACP schema to include “User Sentiment“ for better personalization, unaware that this data now propagates to every downstream third-party agent in the chain. What started as local enrichment becomes distributed exposure.

The determinism trap: Unlike REST APIs, agents are nondeterministic. An agent’s refund policy logic might change when its underlying model is updated from GPT-4 to GPT-4.5, even though the A2A Agent Card declares identical capabilities. The workflow “works“—until it doesn’t, and there’s no version trace to debug. This creates what I call “ghost breaks“: failures that don’t show up in traditional observability because the interface contract looks unchanged.

Taken together, these aren’t edge cases. They’re what happens when we give agents more autonomy without upgrading the rules of engagement between them. These failure modes have a common root cause: The technical capability to collaborate across agents has outrun the organization’s ability to say where that collaboration is appropriate, and under what constraints.

That’s why we need something on top of the protocols themselves: an explicit “Agent Treaty“ layer. If the protocol is the language, the treaty is the constitution. Governance must move from “side documentation“ to “policy as code.“

Want Radar delivered straight to your inbox? Join us on Substack. Sign up here.

Traditional governance treats policy violations as failures to prevent. An antifragile approach treats them as signals to exploit. When an agent makes a commitment that violates a business constraint, the system should capture that event, trace the causal chain, and feed it back into both the agent’s training and the treaty ruleset. Over time, the governance layer gets smarter, not just stricter.

Define treaty-level constraints: Don’t just authorize a connection; authorize a scope. Which ACP fields is an agent allowed to share? Which A2A operations are “read only“ versus “legally binding“? Which categories of decisions require human escalation?

Version the behavior, not just the schema: Treat Agent Cards as first-class product surfaces. If the underlying model changes, the version must bump, triggering a rereview of the treaty. This is not bureaucratic overhead—it’s the only way to maintain accountability in a system where autonomous agents make commitments on behalf of your organization.

Cross-organizational traceability: We need observability traces that don’t just show latency but show intent: Which agent made this commitment, under which policy? And who is the human owner? This is particularly critical when workflows span organizational boundaries and partner ecosystems.

Designing that treaty layer isn’t just a tooling problem. It changes who needs to be in the room and how they think about the system. The hardest constraint isn’t the code; it’s the people. We’re entering a world where engineers must reason about multi-agent game theory and policy interactions, not just SDK integration. Risk teams must audit “machine-to-machine commitments“ that may never be rendered in human language. Product managers must own agent ecosystems where a change in one agent’s reward function or context schema shifts behavior across an entire partner network. Compliance and audit functions need new tools and mental models to review autonomous workflows that execute at machine speed. In many organizations, those skills sit in different silos, and A2A/ACP adoption is proceeding faster than the cross-functional structures needed to manage them.

All of this might sound abstract until you look at where enterprises are in their adoption curve. Three converging trends are making this urgent: Protocol maturity means A2A, ACP, and MCP specifications have stabilized enough that enterprises are moving beyond pilots to production deployments. Multi-agent orchestration is shifting from single agents to agent ecosystems and workflows that span teams, departments, and organizations. And silent autonomy is blurring the line between “tool assistance“ and “autonomous decision-making“—often without explicit organizational acknowledgment. We’re moving from integration (making things talk) to orchestration (making things act), but our monitoring tools still only measure the talk. The next 18 months will determine whether enterprises get ahead of this or we see a wave of high-profile failures that force retroactive governance.

The risk is not that A2A and ACP are unsafe; it’s that they are too effective. For teams piloting these protocols, stop focusing on the “happy path“ of connectivity. Instead, pick one multi-agent workflow and instrument it as a critical product:

Map the context flow: Every ACP field must have a “purpose limitation“ tag. Document which agents see which fields, and which business or regulatory requirements justify that visibility. This isn’t an inventory exercise; it’s a way to surface hidden data dependencies.

Audit the commitments: Identify every A2A interaction that represents a financial or legal commitment—especially ones that don’t route through human approval. Ask, “If this agent’s behavior changed overnight, who would notice? Who is accountable?“

Code the treaty: Prototype a “gatekeeper“ agent that enforces business constraints on top of the raw protocol traffic. This isn’t about blocking agents; it’s about making policy visible and enforceable at runtime. Start minimal: One policy, one workflow, one success metric.

Instrument for learning: Capture which agents collaborate, which policies they invoke, and which contexts they share. Treat this as telemetry, not just audit logs. Feed patterns back into governance reviews quarterly.

If this works, you now have a repeatable pattern for scaling agent deployments without sacrificing accountability. If it breaks, you’ve learned something critical about your architecture before it breaks in production. If you can get one workflow to behave this way—governed, observable, and learn-as-you-go—you have a template for the rest of your agent ecosystem.

If the last decade was about treating APIs as products, the next one will be about treating autonomous workflows as policies encoded in traffic between agents. The protocols are ready. Your org chart is not. The bridge between the two is the Agent Treaty—start building it before your agents start signing deals without you. The good news: You don’t need to redesign your entire organization. You need to add one critical layer—the Agent Treaty—that makes policy machine-enforceable, observable, and learnable. You need engineers who think about composition and game theory, not just connection. And you need to treat agent deployments as products, not infrastructure.

The sooner you start, the sooner that governance gap closes.

AI coding assistants have quickly moved from novelty to necessity, where up to 90% of software engineers use some kind of AI for coding. But a new paradigm is emerging in software development—one where engineers leverage fleets of autonomous coding agents. In this agentic future, the role of the software engineer is evolving from implementer to manager, or in other words, from coder to conductor and ultimately orchestrator.

Over time, developers will increasingly guide AI agents to build the right code and coordinate multiple agents working in concert. This write-up explores the distinction between conductors and orchestrators in AI-assisted coding, defines these roles, and examines how today’s cutting-edge tools embody each approach. Senior engineers may start to see the writing on the wall: Our jobs are shifting from “How do I code this?” to “How do I get the right code built?”—a subtle but profound change.

What’s the tl;dr of an orchestrator tool? It supports multi-agent workflows where you can run many agents in parallel without them interfering with each other. But let’s talk terminology first.

The Conductor: Guiding a Single AI Agent

In the context of AI coding, acting as a conductor means working closely with a single AI agent on a specific task, much like a conductor guiding a soloist through a performance.

The engineer remains in the loop at each step, dynamically steering the agent’s behavior, tweaking prompts, intervening when needed, and iterating in real time. This is the logical extension of the “AI pair programmer” model many developers are already familiar with. With conductor-style workflows, coding happens in a synchronous, interactive session between human and AI, typically in your IDE or CLI.

Key characteristics: A conductor keeps a tight feedback loop with one agent, verifying or modifying each suggestion, much as a driver navigates with a GPS. The AI helps write code, but the developer still performs many manual steps—creating branches, running tests, writing commit messages, etc.—and ultimately decides which suggestions to accept.

Crucially, most of this interaction is ephemeral: Once code is written and the session ends, the AI’s role is done and any context or decisions not captured in code may be lost. This mode is powerful for focused tasks and allows fine-grained control, but it doesn’t fully exploit what multiple AIs could do in parallel.

Modern tools as conductors

Several current AI coding tools exemplify the conductor pattern:

• Claude Code (Anthropic): Anthropic’s Claude model offers a coding assistant mode (accessible via a CLI tool or editor integration) where the developer converses with Claude to generate or modify code. For example, with the Claude Code CLI, you navigate your project in a shell and ask Claude to implement a function or refactor code, and it prints diffs or file updates for you to approve. You remain the conductor: You trigger each action and review the output immediately. While Claude Code has features to handle long-running tasks and tools, in the basic usage it’s essentially a smart codeveloper working step-by-step under human direction.
• Gemini CLI (Google): A command-line assistant powered by Google’s Gemini model, used for planning and coding with a very large context window. An engineer can prompt Gemini CLI to analyze a codebase or draft a solution plan, then iterate on results interactively. The human directs each step and Gemini responds within the CLI session. It’s a one-at-a-time collaborator, not running off to make code changes on its own (at least in this conductor mode).
• Cursor (editor AI assistant): The Cursor editor (a specialized AI-augmented IDE) can operate in an inline or chat mode where you ask it questions or to write a snippet, and it immediately performs those edits or gives answers within your coding session. Again, you guide it one request at a time. Cursor’s strength as a conductor is its deep context integration—it indexes your whole codebase so the AI can answer questions about any part of it. But the hallmark is that you, the developer, initiate and oversee each change in real time.
• VS Code, Cline, Roo Code (in-IDE chat): Similar to above, other coding agents also fall into this category. They suggest code or even multistep fixes, but always under continuous human guidance.

This conductor-style AI assistance has already boosted productivity significantly. It feels like having a junior engineer or pair programmer always by your side. However, it’s inherently one-agent-at-a-time and synchronous. To truly leverage AI at scale, we need to go beyond being a single-agent conductor. This is where the orchestrator role comes in.

The Orchestrator: Managing a Fleet of Agents

If a conductor works with one AI “musician,” an orchestrator oversees the entire symphony of multiple AI agents working in parallel on different parts of a project. The orchestrator sets high-level goals, defines tasks, and lets a team of autonomous coding agents independently carry out the implementation details.

Instead of micromanaging every function or bug fix, the human focuses on coordination, quality control, and integration of the agents’ outputs. In practical terms, this often means an engineer can assign tasks to AI agents (e.g., via issues or prompts) and have those agents asynchronously produce code changes—often as ready-to-review pull requests. The engineer’s job becomes reviewing, giving feedback, and merging the results rather than writing all the code personally.

This asynchronous, parallel workflow is a fundamental shift. It moves AI assistance from the foreground to the background. While you attend to higher-level design or other work, your “AI team” is coding in the background. When they’re done, they hand you completed work (with tests, docs, etc.) for review. It’s akin to being a project tech lead delegating tasks to multiple devs and later reviewing their pull requests, except the “devs” are AI agents.

Modern tools as orchestrators

Over just the past year, several tools have emerged that embody this orchestrator paradigm:

• GitHub Copilot coding agent (Microsoft): This upgrade to Copilot transforms it from an in-editor assistant into an autonomous background developer. (I cover it in this video.) You can assign a GitHub issue to Copilot’s agent or invoke it via the VS Code agents panel, telling it (for example) “Implement feature X” or “Fix bug Y.” Copilot then spins up an ephemeral dev environment via GitHub Actions, checks out your repo, creates a new branch, and begins coding. It can run tests, linters, even spin up the app if needed, all without human babysitting. When finished, it opens a pull request with the changes, complete with a description and meaningful commit messages. It then asks for your review.
  
  You, the human orchestrator, review the PR (perhaps using Copilot’s AI-assisted code review to get an initial analysis). If changes are needed, you can leave comments like “@copilot please update the unit tests for edge case Z,” and the agent will iterate on the PR. This is asynchronous, autonomous code generation in action. Notably, Copilot automates the tedious bookkeeping—branch creation, committing, opening PRs, etc.—which used to cost developers time. All the grunt work around writing code (aside from the design itself) is handled, allowing developers to focus on reviewing and guiding at a high level. GitHub’s agent effectively lets one engineer supervise many “AI juniors” working in parallel across different issues (and you can even create multiple specialized agents for different task types).

• Jules, Google’s coding agent: Jules is an autonomous coding agent. Jules is “not a copilot, not a code-completion sidekick, but an autonomous agent that reads your code, understands your intent, and gets to work.” Integrated with Google Cloud and GitHub, Jules lets you connect a repository and then ask it to perform tasks much as you would a developer on your team. Under the hood, Jules clones your entire codebase into a secure cloud VM and analyzes it with a powerful model. You might tell Jules “Add user authentication to our app” or “Upgrade this project to the latest Node.js and fix any compatibility issues.” It will formulate a plan, present it to you for approval, and once you approve, execute the changes asynchronously. It makes commits on a new branch and can even open a pull request for you to merge. Jules handles writing new code, updating tests, bumping dependencies, etc., all while you could be doing something else.
  
  Crucially, Jules provides transparency and control: It shows you its proposed plan and reasoning before making changes, and allows you to intervene or modify instructions at any point (a feature Google calls “user steerability”). This is akin to giving an AI intern the spec and watching over their shoulder less frequently—you trust them to get it mostly right, but you still verify the final diff. Jules also boasts unique touches like audio changelogs (it generates spoken summaries of code changes) and the ability to run multiple tasks concurrently in the cloud. In short, Google’s Jules demonstrates the orchestrator model: You define the task, Jules does the heavy lifting asynchronously, and you oversee the result.

• OpenAI Codex (cloud agent): OpenAI introduced a new cloud-based Codex agent to complement ChatGPT. This evolved Codex (different from the 2021 Codex model) is described as “a cloud-based software engineering agent that can work on many tasks in parallel.” It’s available as part of ChatGPT Plus/Pro under the name OpenAI Codex and via an npm CLI (npm i -g @openai/codex). With the Codex CLI or its VS Code/Cursor extensions, you can delegate tasks to OpenAI’s agent similar to Copilot or Jules. For instance, from your terminal you might say, “Hey Codex, implement dark mode for the settings page.” Codex then launches into your repository, edits the necessary files, perhaps runs your test suite, and when done, presents the diff for you to merge. It operates in an isolated sandbox for safety, running each task in a container with your repo and environment.
  
  Like others, OpenAI’s Codex agent integrates with developer workflows: You can even kick off tasks from a ChatGPT mobile app on your phone and get notified when the agent is done. OpenAI emphasizes seamless switching “between real-time collaboration and async delegation” with Codex. In practice, this means you have the flexibility to use it in conductor mode (pair-programming in your IDE) or orchestrator mode (hand off a background task to the cloud agent). Codex can also be invited into your Slack channels—teammates can assign tasks to @Codex in Slack, and it will pull context from the conversation and your repo to execute them. It’s a vision of ubiquitous AI assistance, where coding tasks can be delegated from anywhere. Early users report that Codex can autonomously identify and fix bugs, or generate significant features, given a well-scoped prompt. All of this again aligns with the orchestrator workflow: The human defines the goal; the AI agent autonomously delivers a solution.

• Anthropic Claude Code (for web): Anthropic has offered Claude as an AI chatbot for a while, and their Claude Code CLI has been a favorite for interactive coding. Anthropic took the next step by launching Claude Code for web, effectively a hosted version of their coding agent. Using Claude Code for web, you point it at your GitHub repo (with configurable sandbox permissions) and give it a task. The agent then runs in Anthropic’s managed container, just like the CLI version, but now you can trigger it from a web interface or even a mobile app. It queues up multiple prompts and steps, executes them, and when done, pushes a branch to your repo (and can open a PR). Essentially, Anthropic took their single-agent Claude Code and made it an orchestratable service in the cloud. They even provided a “teleport” feature to transfer the session to your local environment if you want to take over manually.
  
  The rationale for this web version aligns with orchestrator benefits: convenience and scale. You don’t need to run long jobs on your machine; Anthropic’s cloud handles the heavy lifting, with filesystem and network isolation for safety. Claude Code for web acknowledges that autonomy with safety is key—by sandboxing the agent, they reduce the need for constant permission prompts, letting the agent operate more freely (less babysitting by the user). In effect, Anthropic has made it easier to use Claude as an autonomous coding worker you launch on demand.

• Cursor background agents: tl;dr Cursor 2.0 has a multi-agent interface more focused around agents rather than files. Cursor 2 expands its background agents feature into a full-fledged orchestration layer for developers. Beyond serving as an interactive assistant, Cursor 2 lets you spawn autonomous background agents that operate asynchronously in a managed cloud workspace. When you delegate a task, Cursor 2’s agents now clone your GitHub repository, spin up an ephemeral environment, and check out an isolated branch where they execute work end-to-end. These agents can handle the entire development loop—from editing and running code to installing dependencies, executing tests, running builds, and even searching the web or referencing documentation to resolve issues. Once complete, they push commits and open a detailed pull request summarizing their work.
  
  Cursor 2 introduces multi-agent orchestration, allowing several background agents to run concurrently across different tasks—for instance, one refining UI components while another optimizes backend performance or fixes tests. Each agent’s activity is visible through a real-time dashboard that can be accessed from desktop or mobile, enabling you to monitor progress, issue follow-ups, or intervene manually if needed. This new system effectively treats each agent as part of an on-demand AI workforce, coordinated through the developer’s high-level intent. Cursor 2’s focus on parallel, asynchronous execution dramatically amplifies a single engineer’s throughput—fully realizing the orchestrator model where humans oversee a fleet of cooperative AI developers rather than a single assistant.

• Agent orchestration platforms: Beyond individual product offerings, there are also emerging platforms and open source projects aimed at orchestrating multiple agents. For instance, Conductor by Melty Labs (despite its name!) is actually an orchestration tool that lets you deploy and manage multiple Claude Code agents on your own machine in parallel. With Conductor, each agent gets its own isolated Git worktree to avoid conflicts, and you can see a dashboard of all agents (“who’s working on what”) and review their code as they progress. The idea is to make running a small swarm of coding agents as easy as running one. Similarly, Claude Squad is a popular open source terminal app that essentially multiplexes Anthropic’s Claude—it can spawn several Claude Code instances working concurrently in separate tmux panes, allowing you to give each a different task and thus code “10x faster” by parallelizing. These orchestration tools underscore the trend: Developers want to coordinate multiple AI coding agents and have them collaborate or divide work. Even Microsoft’s Azure AI services are enabling this: At Build 2025 they announced tools for developers to “orchestrate multiple specialized agents to handle complex tasks,” with SDKs supporting agent-to-agent communication so your fleet of agents can talk to each other and share context. All of this infrastructure is being built to support the orchestrator engineer, who might eventually oversee dozens of AI processes tackling different parts of the software development lifecycle.

I found Conductor to make the most sense to me. It was a perfect balance of talking to an agent and seeing my changes in a pane next to it. Its Github integration feels seamless; e.g. after merging PR, it immediately showed a task as “Merged” and provided an “Archive” button.
—Juriy Zaytsev, Staff SWE, LinkedIn

He also tried Magnet:

The idea of tying tasks to a Kanban board is interesting and makes sense. As such, Magnet feels very product-centric.

Conductor versus Orchestrator—Differences

Many engineers will continue to engage in conductor-style workflows (single agent, interactive) even as orchestrator patterns mature. The two modes will coexist.

It’s clear that “conductor” and “orchestrator” aren’t just fancy terms; they describe a genuine shift in how we work with AI.

• Scope of control: A conductor operates at the micro level, guiding one agent through a single task or a narrow problem. An orchestrator operates at the macro level, defining broader tasks and objectives for multiple agents or for a powerful single agent that can handle multistep projects. The conductor asks, “How do I solve this function or bug with the AI’s help?” The orchestrator asks, “What set of tasks can I delegate to AI agents today to move this project forward?”
• Degree of autonomy: In conductor mode, the AI’s autonomy is low—it waits for user prompts each step of the way. In orchestrator mode, we give the AI high autonomy—it might plan and execute dozens of steps internally (writing code, running tests, adjusting its approach) before needing human feedback. A GitHub Copilot agent or Jules will try to complete a feature from start to finish once assigned, whereas Copilot’s IDE suggestions only go line-by-line as you type.
• Synchronous vs asynchronous: Conductor interactions are typically synchronous—you prompt; AI responds within seconds; you immediately integrate or iterate. It’s a real-time loop. Orchestrator interactions are asynchronous—you might dispatch an agent and check back minutes or hours later when it’s done (somewhat like kicking off a long CI job). This means orchestrators must handle waiting, context-switching, and possibly managing multiple things concurrently, which is a different workflow rhythm for developers.
• Artifacts and traceability: A subtle but important difference: Orchestrator workflows produce persistent artifacts like branches, commits, and pull requests that are preserved in version control. The agent’s work is fully recorded (and often linked to an issue/ticket), which improves traceability and collaboration. With conductor-style (IDE chat, etc.), unless the developer manually commits intermediate changes, a lot of the AI’s involvement isn’t explicitly documented. In essence, orchestrators leave a paper trail (or rather a Git trail) that others on the team can see or even trigger themselves. This can help bring AI into team processes more naturally.
• Human effort profile: For a conductor, the human is actively engaged nearly 100% of the time the AI is working—reviewing each output, refining prompts, etc. It’s interactive work. For an orchestrator, the human’s effort is front-loaded (writing a good task description or spec for the agent, setting up the right context) and back-loaded (reviewing the final code and testing it), but not much is needed in the middle. This means one orchestrator can manage more total work in parallel than would ever be possible by working with one AI at a time. Essentially, orchestrators leverage automation at scale, trading off fine-grained control for breadth of throughput.

To illustrate, consider a common scenario: adding a new feature that touches frontend and backend and requires new tests. As a conductor, you might open your AI chat and implement the backend logic with the AI’s help, then separately implement the frontend, then ask it to generate some tests—doing each step sequentially with you in the loop throughout. As an orchestrator, you could assign the backend implementation to one agent (Agent A), the frontend UI changes to another (Agent B), and test creation to a third (Agent C). You give each a prompt or an issue description, then step back and let them work concurrently.

After a short time, you get perhaps three PRs: one for backend, one for frontend, one for tests. Your job then is to review and integrate them (and maybe have Agent C adjust tests if Agents A/B’s code changed during integration). In effect, you managed a mini “AI team” to deliver the feature. This example highlights how orchestrators think in terms of task distribution and integration, whereas conductors focus on step-by-step implementation.

It’s worth noting that these roles are fluid, not rigid categories. A single developer might act as a conductor in one moment and an orchestrator the next. For example, you might kick off an asynchronous agent to handle one task (orchestrator mode) while you personally work with another AI on a tricky algorithm in the meantime (conductor mode). Tools are also blurring lines: As OpenAI’s Codex marketing suggests, you can seamlessly switch between collaborating in real-time and delegating async tasks. So, think of “conductor” versus “orchestrator” as two ends of a spectrum of AI-assisted development, with many hybrid workflows in between.

Why Orchestrators Matter

Experts are suggesting that this shift to orchestration could be one of the biggest leaps in programming productivity we’ve ever seen. Consider the historical trends: We went from writing assembly to using high-level languages, then to using frameworks and libraries, and recently to leveraging AI for autocompletion. Each step abstracted away more low-level work. Autonomous coding agents are the next abstraction layer. Instead of manually coding every piece, you describe what you need at a higher level and let multiple agents build it.

As orchestrator-style agents ramp up, we could imagine even larger percentages of code being drafted by AIs. What does a software team look like when AI agents generate, say, 80% or 90% of the code, and humans provide the 10% critical guidance and oversight? Many believe it doesn’t mean replacing developers—it means augmenting developers to build better software. We may witness an explosion of productivity where a small team of engineers, effectively managing dozens of agent processes, can accomplish what once took an army of programmers months. (Note: I continue to believe the code review loop where we’ll continue to focus our human skills is going to need work if all this code is not to be slop.)

One intriguing possibility is that every engineer becomes, to some degree, a manager of AI developers. It’s a bit like everyone having a personal team of interns or junior engineers. Your effectiveness will depend on how well you can break down tasks, communicate requirements to AI, and verify the results. Human judgment will remain vital: deciding what to build, ensuring correctness, handling ambiguity, and injecting creativity or domain knowledge where AI might fall short. In other words, the skillset of an orchestrator—good planning, prompt engineering, validation, and oversight—is going to be in high demand. Far from making engineers obsolete, these agents could elevate engineers into more strategic, supervisory roles on projects.

Toward an “AI Team” of Specialists

Today’s coding agents mostly tackle implementation: write code, fix code, write tests, etc. But the vision doesn’t stop there. Imagine a full software development pipeline where multiple specialized AI agents handle different phases of the lifecycle, coordinated by a human orchestrator. This is already on the horizon. Researchers and companies have floated architectures where, for example, you have:

• A planning agent that analyzes feature requests or bug reports and breaks them into specific tasks
• A coding agent (or several) that implements the tasks in code
• A testing agent that generates and runs tests to verify the changes
• A code review agent that checks the pull requests for quality and standards compliance
• A documentation agent that updates README or docs to reflect the changes
• Possibly a deployment/monitoring agent that can roll out the change and watch for issues in production.

In this scenario, the human engineer’s role becomes one of oversight and orchestration across the whole flow: You might initiate the process with a high-level goal (e.g., “Add support for payment via cryptocurrency in our app”); the planning agent turns that into subtasks; coding agents implement each subtask asynchronously; the testing agent and review agent catch problems or polish the code; and finally everything gets merged and deployed under watch of monitoring agents.

The human would step in to approve plans, resolve any conflicts or questions the agents raise, and give final approval to deploy. This is essentially an “AI swarm” tackling software development end to end, with the engineer as the conductor of the orchestra.

While this might sound futuristic, we see early signs. Microsoft’s Azure AI Foundry now provides building blocks for multi-agent workflows and agent orchestration in enterprise settings, implicitly supporting the idea that multiple agents will collaborate on complex, multistep tasks. Internal experiments at tech companies have agents creating pull requests that other agent reviewers automatically critique, forming an AI/AI interaction with a human in the loop at the end. In open source communities, people have chained tools like Claude Squad (parallel coders) with additional scripts that integrate their outputs. And the conversation has started about standards like the Model Context Protocol (MCP) for agents sharing state and communicating results to each other.

I’ve noted before that “specialized agents for Design, Implementation, Test, and Monitoring could work together to develop, launch, and land features in complex environments”—with developers onboarding these AI agents to their team and guiding/overseeing their execution. In such a setup, agents would “coordinate with other agents autonomously, request human feedback, reviews and approvals” at key points, and otherwise handle the busywork among themselves. The goal is a central platform where we can deploy specialized agents across the workflow, without humans micromanaging each individual step—instead, the human oversees the entire operation with full context.

This could transform how software projects are managed: more like running an automated assembly line where engineers ensure quality and direction rather than handcrafting each component on the line.

Challenges and the Human Role in Orchestration

Does this mean programming becomes a push-button activity where you sit back and let the AI factory run? Not quite—and likely never entirely. There are significant challenges and open questions with the orchestrator model:

• Quality control and trust: Orchestrating multiple agents means you’re not eyeballing every single change as it’s made. Bugs or design flaws might slip through if you solely rely on AI. Human oversight remains critical as the final failsafe. Indeed, current tools explicitly require the human to review the AI’s pull requests before merging. The relationship is often compared to managing a team of junior developers: They can get a lot done, but you wouldn’t ship their code without review. The orchestrator engineer must be vigilant about checking the AI’s work, writing good test cases, and having monitoring in place. AI agents can make mistakes or produce logically correct but undesirable solutions (for instance, implementing a feature in a convoluted way). Part of the orchestration skillset is knowing when to intervene versus when to trust the agent’s plan. As the CTO of Stack Overflow wrote, “Developers maintain expertise to evaluate AI outputs” and will need new “trust models” for this collaboration.
• Coordination and conflict: When multiple agents work on a shared codebase, coordination issues arise—much like multiple developers can conflict if they touch the same files. We need strategies to prevent merge conflicts or duplicated work. Current solutions use workspace isolation (each agent works on its own Git branch or separate environment) and clear task separation. For example, one agent per task, and tasks designed to minimize overlap. Some orchestrator tools can even automatically merge changes or rebase agent branches, but usually it falls to the human to integrate. Ensuring agents don’t step on each others’ toes is an active area of development. It’s conceivable that in the future agents might negotiate with each other (via something like agent-to-agent communication protocols) to avoid conflicts, but today the orchestrator sets the boundaries.
• Context, shared state, and handoffs: Coding workflows are rich in state: repository structure, dependencies, build systems, test suites, style guidelines, team practices, legacy code, branching strategies, etc. Multi-agent orchestration demands shared context, memory, and smooth transitions. But in enterprise settings, context sharing across agents is nontrivial. Without a unified “workflow orchestration layer,” each agent can become a silo, working well in its domain but failing to mesh. In a coding-engineering team this may translate into: One agent creates a feature branch; another one runs unit tests; another merges into master—if the first agent doesn’t tag metadata the second is expecting, you get breakdowns.
• Prompting and specifications: Ironically, as the AI handles more coding, the human’s “coding” moves up a level to writing specifications and prompts. The quality of an agent’s output is highly dependent on how well you specify the task. Vague instructions lead to subpar results or agents going astray. Best practices that have emerged include writing mini design docs or acceptance criteria for the agents—essentially treating them like contractors who need a clear definition of done. This is why we’re seeing ideas like spec-driven development for AI: You feed the agent a detailed spec of what to build, so it can execute predictably. Engineers will need to hone their ability to describe problems and desired solutions unambiguously. Paradoxically, it’s a very old-school skill (writing good specs and tests) made newly important in the AI era. As agents improve, prompts might get simpler (“write me a mobile app for X and Y with these features”) and yet yield more complex results, but we’re not quite at the point of the AI intuiting everything unsaid. For now, orchestrators must be excellent communicators to their digital workforce.
• Tooling and debugging: With a human developer, if something goes wrong, they can debug in real time. With autonomous agents, if something goes wrong (say the agent gets stuck on a problem or produces a failing PR), the orchestrator has to debug the situation: Was it a bad prompt? Did the agent misinterpret the spec? Do we roll back and try again or step in and fix it manually? New tools are being added to help here: For instance, checkpointing and rollback commands let you undo an agent’s changes if it went down a wrong path. Monitoring dashboards can show if an agent is taking too long or has errors. But effectively, orchestrators might at times have to drop down to conductor mode to fix an issue, then go back to orchestration. This interplay will improve as agents get more robust, but it highlights that orchestrating isn’t just “fire and forget”—it requires active monitoring. AI observability tools (tracking cost, performance, accuracy of agents) are likely to become part of the developer’s toolkit.
• Ethics and responsibility: Another angle—if an AI agent writes most of the code, who is responsible for license compliance, security vulnerabilities, or bias in that code? Ultimately the human orchestrator (or their organization) carries responsibility. This means orchestrators should incorporate practices like security scanning of AI-generated code and verifying dependencies. Interestingly, some agents like Copilot and Jules include built-in safeguards: They won’t introduce known vulnerable versions of libraries, for instance, and can be directed to run security audits. But at the end of the day, “trust, but verify” is the mantra. The human remains accountable for what ships, so orchestrators will need to ensure AI contributions meet the team’s quality and ethical standards.

In summary, the rise of orchestrator-style development doesn’t remove the human from the loop—it changes the human’s position in the loop. We move from being the one turning the wrench to the one designing and supervising the machine that turns the wrench. It’s a higher-leverage position, but also one that demands broader awareness.

Developers who adapt to being effective conductors and orchestrators of AI will likely be even more valuable in this new landscape.

Conclusion: Is Every Engineer a Maestro?

Will every engineer become an orchestrator of multiple coding agents? It’s a provocative question, but trends suggest we’re headed that way for a large class of programming tasks. The day-to-day reality of a software engineer in the late 2020s could involve less heads-down coding and more high-level supervision of code that’s mostly written by AIs.

Today we’re already seeing early adopters treating AI agents as teammates—for example, some developers report delegating 10+ pull requests per day to AI, effectively treating the agent as an independent teammate rather than a smart autocomplete. Those developers free themselves to focus on system design, tricky algorithms, or simply coordinating even more work.

That said, the transition won’t happen overnight for everyone. Junior developers might start as “AI conductors,” getting comfortable working with a single agent before they take on orchestrating many. Seasoned engineers are more likely to early-adopt orchestrator workflows, since they have the experience to architect tasks and evaluate outcomes. In many ways, it mirrors career growth: Junior engineers implement (now with AI help); senior engineers design and integrate (soon with AI agent teams).

The tools we discussed—from GitHub’s coding agent to Google’s Jules to OpenAI’s Codex—are rapidly lowering the barrier to try this approach, so expect it to go mainstream quickly. The hyperbole aside, there’s truth that these capabilities can dramatically amplify what an individual developer can do.

So, will we all be orchestrators? Probably to some extent—yes. We’ll still write code, especially for novel or complex pieces that defy simple specification. But much of the boilerplate, routine patterns, and even a lot of sophisticated glue code could be offloaded to AI. The role of “software engineer” may evolve to emphasize product thinking, architecture, and validation, with the actual coding being a largely automated act. In this envisioned future, asking an engineer to crank out thousands of lines of mundane code by hand would feel as inefficient as asking a modern accountant to calculate ledgers with pencil and paper. Instead, the engineer would delegate that to their AI agents and focus on the creative and critical-thinking aspects around it.

BTW, yes, there’s plenty to be cautious about. We need to ensure these agents don’t introduce more problems than they solve. And the developer experience of orchestrating multiple agents is still maturing—it can be clunky at times. But the trajectory is clear. Just as continuous integration and automated testing became standard practice, continuous delegation to AI could become a normal part of the development process. The engineers who master both modes—knowing when to be a precise conductor and when to scale up as an orchestrator—will be in the best position to leverage this “agentic” world.

One thing is certain: The way we build software in the next 5–10 years will look quite different from the last 10. I want to stress that not all or most code will be agent-driven within a year or two, but that’s a direction we’re heading in. The keyboard isn’t going away, but alongside our keystrokes we’ll be issuing high-level instructions to swarms of intelligent helpers. In the end, the human element remains irreplaceable: It’s our judgment, creativity, and understanding of real-world needs that guides these AI agents toward meaningful outcomes.

The future of coding isn’t AI or human, it’s AI and human—with humans at the helm as conductors and orchestrators, directing a powerful ensemble to achieve our software ambitions.