This post was originally published in Vanishing Gradients on September 11, 2026. It’s been updated to address the subsequent declaration by 25 Fields Medalists and the debate about AI, mathematical progress, and research incentives.

Science without understanding?

“For seven and a half million years, Deep Thought computed and calculated, and in the end announced that the answer was in fact 42—and so another, even bigger, computer had to be built to find out what the actual question was.”

―Douglas Adams, The Restaurant at the End of the Universe

I recently went back to Dresden for the 25th birthday of the Max Planck Institute (MPI) of Cell Biology and Genetics, where I did part of my postdoc. The MPI was founded to research the physical and biological mechanisms of cells to bridge the gap between the molecular and tissue scales. At the anniversary conference, Michael Bronstein (DeepMind Professor of AI, University of Oxford) delivered the keynote, “Biological Black-Box Data in the Age of AI.” His argument went something along these lines: Biological experiments should generate data optimized for machine learning, even when those measurements aren’t directly interpretable by humans. He argued for prioritizing scale over the quality of individual measurements, producing vast amounts of cheap, noisy data from which noninterpretable models can extract signal.

When asked whether such systems could produce the understanding offered by Newton’s theory of gravitation in a single equation (bridging the scales of an apple falling on your head to that of the moon and the tides), Bronstein responded that this wasn’t the goal: Black-box data and models would, if anything, produce equations with tens, hundreds, thousands, or more noninterpretable parameters. Outcome prioritized at the expense of insight and understanding. He suggested we could gain that understanding by interpreting the black-box models afterward.¹ I was startled to see Bronstein bring such a worldview to an institute founded to understand molecular and cellular mechanisms and the emergent properties at the tissue level.

The MPI was unusual within the Max Planck Society for its collaborative structure, with directors leading relatively small groups alongside independent research groups. At the anniversary’s opening, founding director Marino Zerial explained how they had collaborated so effectively from the start. He said they shared a taste for mechanistic science. This made me think of how often we talk about “taste” and “judgment” when describing the human role in the age of AI.

The worldview that we don’t need understanding or insight isn’t new. In his 2008 essay “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete,” Chris Anderson argues that big data allows us to skip hypotheses, models, and testing. Bronstein invoked Anderson’s vision of post-theory science in his MPI keynote, as he does here also, presenting DeepMind’s AlphaFold as an example of experimentally testable predictions without a human-understandable theory of protein folding. Part of Anderson’s project is to champion big tech, and the future of science becomes a vehicle for doing so. His essay ends: “What can science learn from Google?”

AI gives this worldview a new form: Machines can produce results that withstand verification while the understanding needed to explain them remains out of reach. Developing that understanding takes time, access, and collaboration. Whoever controls those conditions gains power over what people can understand and pursue.

An abundance of proofs

Mathematics makes this possibility particularly stark. I’m excited by AI’s potential to expand what we can discover. Fields Medalist Terence Tao has organized collaborative research combining mathematicians, AI tools, and formal proof verification. His questions about mathematics in the age of AI come from engaging with that potential and asking what we want it to serve.

Tao has noted that we’re producing more verified mathematical proofs that no individual human understands. A world of an abundance of verified mathematical proofs! Tao points out that our peer review, academic incentives, and journals weren’t designed for this abundance. The existing system is already broken, tying careers to publication counts, relying on researchers’ unpaid reviewing labor, and locking much publicly funded knowledge behind commercial paywalls. Reviewers already struggle to keep up with the volume of submissions. AI will multiply that volume far beyond what this system can handle.

Tao also describes fruitful open problems as nonrenewable resources: problems whose pursuit can generate new techniques, collaborations, and understanding that extend far beyond the original question. Once the answer is known, the incentive to explore those paths can disappear. For example, 10,000 OpenAI agents working concurrently may have solved the Navier–Stokes Millennium Prize problem. (The announcement has also sparked a dispute over credit and competition, bringing the question of who controls mathematical discovery into sharp focus, which I’ll get to.) A common conceit in science and mathematics is that solutions open up new questions and fields of inquiry. Tao’s point is that the search for a solution does too. Tao argues that proposing a solution, discovering precisely why it fails, and revising it can reveal new insights into fluid mechanics. Knowing the final answer beforehand can discourage that exploration:

“The process of starting with one ansatz, discovering the precise obstruction preventing it from working. . .would almost certainly reveal important new insights about fluid mechanics.”

—Terence Tao, Mastodon, September 3

Late last month, probabilist Hugo Duminil-Copin gave another example: Unsuccessful attempts at a percolation conjecture led to collaborations and revived techniques that subsequently solved other problems. Both acknowledge AI’s capabilities while asking what the pursuit of mathematics should produce.

This brings me back to Bronstein’s proposal to recover understanding after building the model. Would interpreting that model give us Maxwell’s equations, and the understanding that connects electricity, magnetism and light? The promise feels a little like plugging Neo into a computer: “I know kung fu.” In the Matrix, downloading the knowledge gives him the ability. Receiving a machine’s result doesn’t do that for us. As Tao and Duminil-Copin describe, understanding why an approach fails changes what researchers try next, generating new questions, techniques, and collaborations. Recovering an explanation afterward may teach us something, but it can’t recreate the paths that understanding would have opened during the search.

A timeline of mathematical results

These questions are becoming pressing as results accumulate. Over the past year, AI systems have produced new mathematical constructions, tackled unpublished research problems and formalized existing proofs. Since July, announcements have arrived in quick succession:

These achievements involve different kinds of work. Formalizing Fermat’s Last Theorem means making an existing proof checkable by a computer; finding a counterexample establishes something new. A system can produce a verified result while the work of explaining it remains to be done.

Some of that work is happening through wonderfully strange exchanges on X, where researchers post new results, check one another’s constructions, and develop explanations. It’s reminiscent of when science in Europe was people passing notes and sending letters on horseback:

• A wall of plus and minus signs: Levent Alpöge posted a newly constructed Hadamard matrix. Ion Nechita checked it on his phone while queuing for eclipse glasses.
• A formula overturning a conjecture: Alpöge posted a counterexample to the Jacobian conjecture, and Terence Tao subsequently explained its geometry.
• An AI proof followed by a simpler human proof: After Claude advanced a result about the zeros of the Riemann zeta function, number theorist Youness Lamzouri found a shorter argument. Thomas Wolf shared the development.
• A cryptography breakthrough announced as a number: Eric Lu posted a factor of RSA-260, letting anyone check the factorization.

Tao’s geometric explanation and Lamzouri’s shorter proof help turn verified results into mathematics people can understand and build on. Responding to an early draft in our Discord community, Carol Willing, a Python core developer, former Python Software Foundation director, and longtime leader of Project Jupyter, asked:

While I believe these tools have value for advancing science/math, do they have more value than a human scientist or group of scientists who can view and challenge open results?

If we judge value by who produces a result first, we miss what Lamzouri and Tao contribute by simplifying a proof or explaining its geometry. An answer can close off some paths of inquiry while creating others. I want much more of this: machines producing results that people can explore, explain and build on together. These exchanges depend on results being available to examine, researchers having time to understand them, and people being able to share what they discover. Those conditions deserve as much attention as the systems producing the proofs.

Why is this happening now?

Why the explosion in AI-generated mathematical results now? As Sebastian Raschka explains, reinforcement learning with verifiable rewards (RLVR) became a major technique in model post-training in 2025. The premise is straightforward: If you can computationally check an output, you can reward correct answers and update the model accordingly. Code can be run against tests; mathematical answers can be checked, and formal proofs verified by tools such as Lean, a proof assistant that checks each logical step against specified axioms and previously established results (recently used by Anthropic to formalize the proof of Fermat’s Last Theorem!). That provides feedback without a human grading every attempt. These checks also guide agents during problem-solving: An agent can propose a proof, use Lean to check it, and use the resulting errors to revise its attempt, repeating the process without a person checking every step.

You may ask, Why did coding agents become useful before we saw this explosion in mathematical results? Well, the labs had an immediate incentive to improve the tools they use themselves. Engineers building AI systems want better coding agents to help build those systems. Improve the machine that improves the machine. Mathematics benefits from the resulting capabilities too: agents that can write programs, run experiments, and work with automated checks.

Cost, competition, and credit

On September 11, 25 Fields Medalists issued a declaration warning that the race to solve benchmark problems was undermining mathematics. Some responses on X treated this as professional protectionism; others assumed that understanding would follow the proofs. That brings us back to Bronstein’s proposal, and to who gets to decide that producing results comes first while other researchers supply the explanations afterward.

Many assume that the goal of pure mathematics is to produce results. Tao’s point is that pursuing those results also develops methods, understanding, and people capable of asking better questions. Solved problems have served as a proxy for that broader progress. Goodhart’s law describes the danger of turning the proxy into the target. AI mirrors our incentive systems and is exceptionally good at pursuing what they reward. If schools reward the essay over learning, students will generate essays. If mathematical prestige attaches primarily to solved problems, labs have every incentive to produce them.

Producing results and developing understanding aren’t mutually exclusive, but the current system makes pursuing both prohibitively difficult. Frontier labs have strong incentives for outcomes rather than insight. (See, for example, Anthropic’s incentives for solving Millennium Prize problems with Claude pre-IPO, discussed in Gavin Baker’s commentary on Anthropic’s pre-IPO positioning; Samuel Kerr makes a related argument about OpenAI’s mathematical results and its IPO narrative.) OpenAI’s run involved 10,000 agents working concurrently for 88 hours. Abhishek Nagaraj, associate professor at UC Berkeley, calculated this would cost a regular user $20–$30 million in tokens.

NYU mathematician Tristan Buckmaster says OpenAI pressured him to publish without his collaborator Levent Alpöge, who works at Anthropic. OpenAI’s Sébastien Bubeck disputes his account. Buckmaster also describes how the pressure affected the mathematics: He and Alpöge had verified their proofs but wanted more time to understand them and produce readable explanations. Instead, they rushed to publish work they considered inadequately explained. If understanding is deferred until after the result, what ensures that anyone gets the time, resources, and access to develop it?

What’s worse is that we’re not even sure whether using OpenAI agents could result in them scooping you. It looks like they’re not sure either:

While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.

In “The End of Mathematics,” mathematician Daniel Litt imagines researchers withholding unfinished ideas for fear of being scooped. The collaborations Duminil-Copin describes depend on people being willing to share work before it succeeds.

What happens to mathematicians, and who controls mathematics?

If researchers stop sharing promising ideas for fear of being scooped, companies with the most computation gain greater control over what others can learn. A published proof may be available to everyone while the failed approaches and intermediate insights remain private. Threats to public research funding in the US compound that dependence: Companies supplying the resources gain greater influence over what science gets done. This brings us to Shoshana Zuboff’s questions about knowledge and power: “Who knows? Who decides who knows? Who decides who decides?” Who gets to pursue a fruitful question, and who determines whether the work behind its answer becomes shared knowledge?

The movement of AI researchers from academia into industry concentrates expertise alongside those resources. And I get it: If I wanted to return to doing research in depth, frontier labs would be among the most attractive places to work. Access to capital, data, computation, and incredibly talented colleagues can make research possible that would be difficult to pursue in academia. The attraction for individual researchers is clear, even as their collective movement gives companies greater influence over research priorities and leaves universities with fewer people to teach the next generation. Thinking about this brain drain, it isn’t lost on me that Bronstein is the “DeepMind Professor of AI” at Oxford. Corporate influence reaches into the universities themselves.

Students also need opportunities to develop the judgment we keep asking humans to exercise. Po-Ling Loh describes the difficulty of advising students and postdocs as AI changes research expectations. Choosing a fruitful problem, recognizing why an approach failed, and deciding what to try next are abilities developed through doing mathematics. If students delegate that work before developing those abilities, where will their judgment come from? AI could also help them explore more approaches and work through unfamiliar ideas, provided their understanding remains an explicit purpose of the process. That requires mentors with time to teach, and institutions willing to support work whose value includes what the researcher learns, even when a machine could produce the result faster.

When careers depend on producing papers, time spent explaining a result, simplifying a proof, or helping others understand it can compete with the pressure to publish the next one. Martin Hairer argues that authors should understand their arguments, trace ideas to their sources, and explain AI’s contributions. Those responsibilities become harder to fulfil when results arrive faster than researchers can absorb them. Universities, funders, and journals will help determine whether mathematicians can afford to do that work. If we value shared understanding, then developing explanations, teaching difficult ideas, and making proofs useful to other researchers need to count toward careers as well. Otherwise, the institutions asking people to exercise judgment may reward them for spending less time developing it.

Mathematics as the canary

After Bronstein’s keynote, we sat in a Dresden beer garden eating currywurst and drinking radlers. It was late summer, and the conversations were wild. Cell biologists, biochemists, mathematicians, and engineers were asking what this future meant for them. Some were scared. Others thought it was inevitable and would turn scientists into something like artists. Because I now work in AI, people asked me, “Do you think this is where things are going?” They wanted to know what the human’s role would be and how scientific knowledge would be passed down. I started telling them about mathematics. The prospect of abundant results without shared understanding was already raising the questions we were asking over our beers.

In biology, a proposed result still has to meet the physical world: Someone has to prepare samples, run experiments, and measure what happens. Robotics and laboratory automation will let agents carry out more of that work, giving individual scientists the capacity to direct experiments that once required an entire group. Perhaps more scientists become PIs of automated labs, choosing questions and supervising agents and instruments. But the work being automated is also how students, postdocs, and technicians learn. Handling a sample, noticing something unexpected, and figuring out why an experiment failed develop judgment that directing a system may not teach. Who gets to acquire that experience before they’re expected to lead?

Researching a policy brief, building a financial model, or developing a product strategy helps people learn the territory in which they’ll make decisions. In my work with agentic data science, I encourage people to explore data cell by cell with an agent, because working through the analysis develops the understanding needed to decide what to ask next. Across knowledge work, these tasks are also how junior colleagues develop expertise. If we automate their production, how do we preserve the learning and judgment developed through doing them? We could increasingly depend on models to hold and transmit expertise, with knowledge passing from model to model, then to humans who consult them as oracles. Whoever controls those systems gains power over what we can investigate and learn. Human understanding has to be part of what we’re trying to produce.

What comes next?

Mathematician Jared Duker Lichtman has proposed a Mathematics Atlas Project to formalize the existing mathematical literature, arguing that sufficient funding and computation could make this possible within a year. A library of computer-checkable mathematics could let researchers build on established results with greater confidence, while agents help find connections and assemble arguments across fields. It could also become a resource for learning, if people can connect formal proofs to explanations they understand. Achieving that would require deliberate work on access, exposition and teaching alongside formalization. We have an opportunity to build tools that help people explore mathematics more deeply, provided we make that part of the project.

The MPI in Dresden was founded to understand how cells work, how molecular mechanisms give rise to the behavior of living tissue. I want AI to help us pursue that ambition, including through approaches we could never have attempted before. But human understanding belongs among the things we ask this work to produce, with time and resources devoted to developing it. So does the ability to share what we learn and choose what to investigate next. If we leave those decisions to the companies supplying the machines, we also leave them to decide what scientific progress is for.

Want to understand how AI agents actually work? In Build AI Agents from First Principles, we’ll build an agent ourselves, then rebuild it with a modern SDK and MCP. You’ll leave with a working agent, code you can adapt, and the understanding to diagnose failures and decide what your system actually needs.

Support Vanishing Gradients

Vanishing Gradients is independent, and most of the podcasts, workshops, articles, skills, and workflows I publish are free.

If you’d like to help keep it going:

• Become a paid subscriber. Your subscription supports the podcast, newsletter, and open resources.
• Share this post with a friend or colleague who’d find it useful.
• Subscribe to the events calendar for upcoming livestreams, workshops, and meetups.
• Subscribe on YouTube for full episodes, live builds, and recordings.
• Join us on Discord. Come discuss this piece, challenge the ideas, and compare notes on what you’re building with AI.
• Work with me. I help teams build and improve AI-powered products.

Footnote

1. Bronstein and Naef propose an inversion: From “understand, encode, and then simulate” to “encode, simulate, understand,” recovering human understanding post hoc through mechanistic interpretability of black-box models. Useful scientific models may require enormous numbers of parameters. But predictive success alone does not tell us whether interpreting those models will give humans an understanding of the phenomena they describe. They offer negligible evidence that this will yield the kinds of physical and biological understanding we gain through relativity, quantum theory, or the double-helical structure of DNA. And even if it does, understanding developed afterward may not replace the understanding that guides inquiry, generating new questions and approaches along the way. ︎

Is cybersecurity part of your job in any way? If so, we’d like to know what you think for a report we’re writing. Just answer these quick 11 questions. Thanks in advance! Take the survey >

When AI engineer Sajal Sharma ran into this problem in his own work, he solved it by building a personal knowledge base to act as a shared brain for every agent he runs. On this week’s episode of Zero to Agent in 30 Minutes, Sajal showed how to set up that shared workspace yourself so that a task added on one tool shows up for all the others.

How to set up a knowledge base for your agents

Here’s how Sajal’s setup breaks down:

1. Create a workspace map. Set up an AGENTS.md file that lists where everything in your knowledge base lives, from current tasks to project notes to decision logs. This will help each of your agents navigate your workspace without guessing.
2. Layer daily notes into summaries. Keep the most detailed notes at the daily level, then roll several days into a weekly summary and several weeks into a monthly one. An agent can then work from the summarized view instead of reading through months of individual files, which keeps token use manageable as the knowledge base grows.
3. Bridge AGENTS.md with CLAUDE.md. Claude Code reads CLAUDE.md, not AGENTS.md, so add a short pointer in CLAUDE.md that redirects to the AGENTS.md or link the two files directly. Sajal uses this pattern to avoid maintaining two files separately and having them drift out of sync.
4. Package repeatable tasks as skills. Turn routines you do often, like producing a daily briefing or turning a saved article into a note, into skill files stored in the shared workspace. Any agent that can read the workspace can then run the task the same way, rather than working out the steps on its own each time.
5. Sync the workspace across machines. Use a file-sync tool, Git, or a shared server to keep your local copy of the knowledge base and your server copy aligned. That way, you ensure that an agent running on a laptop and one running on a home server, through a gateway like OpenClaw, are working from the same files.
6. Have agents reread the state before every write. Add an instruction in AGENTS.md telling every agent to check the current version of the knowledge base before making a change. When you have several agents writing to the same files, this step keeps one agent from acting on information another has already updated.

Sajal closed by pointing to two projects as evidence that this “shared brain” pattern is spreading beyond his own setup. LangChain recently released OpenWiki, a tool that generates and maintains repository documentation that both people and coding agents can use. And Y Combinator president Garry Tan built and open-sourced GBrain, a memory layer for agents built on the same principle.

Sajal’s starter repo is available on GitHub if you want to set up your own version, and you can reach out to him on LinkedIn to discuss the topic further.

Coming up next

On September 16, data science educator and AI consultant Chester Ismay joins Zero to Agent in 30 Minutes to build a personal sports concierge agent that will read the schedules for every sport he follows, decide what’s worth his time, and send a single weekly update to his phone. Viewers can take the pattern home to plan their own week.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

Is cybersecurity part of your job in any way? If so, we’d like to know what you think for a report we’re writing. Just answer these quick 11 questions. Thanks in advance! Take the survey >

For decades we tried to solve this by consolidating data. We built larger warehouses, semantic layers, APIs, and dashboards. Each solved part of the problem, but none solved the fundamental one: orchestrating reasoning across heterogeneous sources in response to an arbitrary business question. Earlier systems supported limited federation and semantic querying, yet they struggled to reason across those sources at enterprise scale without significant custom engineering.

Modern LLMs change this. Instead of replacing databases, they facilitate a new architectural primitive: an intelligent orchestration layer that dynamically reasons across specialized systems. Rather than consolidating the data into a single store, this layer consolidates the access pattern to data that stays where it lives.

This article presents a reference architecture for LLM-powered enterprise analytics agents that coordinate purpose-built, heterogeneous data stores through intelligent orchestration while preserving security, performance, and auditability.

What specifically changed with GenAI

BI tools have always been constrained to predefined reports and dashboards. Before GenAI, building a cross-system query engine meant hardcoding every possible query pattern, data source combination, and synthesis path. And because the number of possible questions grows exponentially with the number of data sources, exhaustive coverage is impossible through traditional engineering. GenAI changes this in three specific ways.

Intent understanding replaces query templates: An LLM parses natural language and determines which data sources are relevant based on semantic understanding rather than keyword matching. Unlike a keyword search, an LLM understands that “Why did retention drop in Asia last quarter?” and “What is driving churn in Asian markets?” are the same question expressed differently. More importantly, it infers that answering the question requires customer relationship data, revenue metrics, and possibly support ticket sentiment, even though none of those systems are named.

Dynamic query decomposition replaces static pipelines: A question like “What are the biggest risk factors in our supply chain?” might require relationship data from a graph database, metrics from a key-value store, contract details from a document repository, and market intelligence from an API. The agent decomposes it into specialized subqueries on the fly, each optimized for the target store’s access pattern. There’s no prebuilt pipeline and no engineering ticket to wire up a new combination, because the decomposition happens at inference time. The system handles novel questions without code changes.

Semantic synthesis replaces manual consolidation: Before GenAI, making sense of the data together was the real work. An analyst would pull numbers from the warehouse, check relationships in a CRM, read through documents, and mentally synthesize an answer. That took hours or days and was bounded by one person’s ability to hold context. I’ve watched senior analysts spend entire Mondays answering a single leadership question. An LLM reasons about how metrics relate to the relationship patterns in a knowledge graph and the strategic context in unstructured documents, and it does so in seconds with full source attribution. A dashboard shows numbers; an analytics agent explains what those numbers mean in the context of everything else it knows.

The architecture: Consolidate the access pattern, not the data

Rather than consolidating the data into a single store, consolidate the access pattern through an intelligent orchestration layer. If your instinct is to get everything into one place, you aren’t alone, but every time we did that we lost something. Graph relationships flattened into join tables, hierarchical documents shredded into rows, and real-time signals turned stale in batch loads. The warehouse was always a compromise.

The better approach is to keep each data store optimized for its specific query pattern:

• Graph database for relationship traversal and multihop reasoning
• Key-value store for instant metric lookups with sub-millisecond latency
• Vector store for semantic document search and similarity matching
• External APIs for market intelligence and real-time signals
• Data Warehouse for large-scale historical aggregation and ad hoc SQL

The LLM-powered agent coordinates across all of them through a unified orchestration layer. This follows the same principle that makes microservices work: specialized services with well-defined interfaces, coordinated by an orchestrator. The difference is that the orchestrator now understands natural language, reasons about which services to call based on intent rather than explicit routing rules, and synthesizes results semantically rather than through programmatic joins. Think of it as a data mesh for inference, where each node keeps its operational independence while an intelligent layer federates queries across them.

The orchestration protocol

The agent follows a multiphase protocol for every query. The full reasoning loop with security enforcement and parallel execution goes well beyond a simple RAG pattern.

Let’s trace a business question through each phase:

“Why did Q2 revenue fall short of forecast in the enterprise segment?”

This question requires revenue metrics (metrics store), account relationships and sales coverage (graph), deal commentary and executive notes (vector store), and market benchmarks (external APIs). No single system holds the answer.

Phase 1: Intent analysis. The LLM determines what the user is asking and which data sources are relevant.

“Why did Q2 revenue fall short of forecast in the enterprise segment?”
                                                   ↓
Intent: Revenue variance root cause analysis
Entities: Enterprise segment
Timeframe: Q2
Metric: Revenue vs. forecast
Required stores: Metrics + Graph + Vector + External API

Not every query needs every store. “What is our current ARR?” might need to hit the metrics store only. This revenue variance question requires all four.

Phase 2: Query decomposition. The original question is broken into specialized subqueries optimized for each target store:

• Metrics store: “Q2 revenue actuals vs. forecast for enterprise, by region and product line”
• Graph store: “Enterprise accounts with closed-lost or slipped deals in Q2; common patterns in sales coverage, partner relationships, deal stage progression”
• Vector store: “Deal notes, QBR summaries, and executive correspondence referencing enterprise deal delays or losses in Q2”
• External API: “Industry benchmark data for enterprise software spending in Q2”

Each is tailored to the target system’s access pattern, not forced through a common query language.

Phase 3: Parallel execution. Tools execute concurrently. This is critical for latency. Sequential execution across four stores would blow past any reasonable response time. With parallel execution, the total data retrieval time equals the slowest individual store, not the sum of all stores.

Phase 4: Deterministic security enforcement. A security layer enforces user permissions at the data query level before any retrieval happens. This is structural rather than application-level: It routes queries to the appropriate data partitions, applies row-level filters at the database query level, and gates the input rather than filtering the output. If our user is a regional VP without access to EMEA deal data, the graph returns no EMEA accounts, the metrics lookup excludes EMEA revenue, and the vector search filters out EMEA deal notes. The LLM can’t leak what it never received. (More on why this must be deterministic below.)

Phase 5: Result synthesis. The LLM reasons about how results from different stores relate, identifies patterns across sources, resolves contradictions, and generates a unified answer with source attribution.

For our revenue question, synthesis might produce:

Q2 enterprise revenue missed forecast by $4.2M (8% variance). Three root causes account for 85% of the gap: (1) Four deals totaling $2.8M slipped to Q3 due to extended legal review cycles—all four involved customers who adopted a new procurement platform in Q1 (graph: common vendor relationship). (2) Two large renewals downsized by $1.1M; deal notes cite budget reallocation toward AI initiatives, a pattern appearing across 6 QBR summaries (vector store). (3) Enterprise software spending contracted 3% QoQ industry-wide (external benchmark), suggesting partial macro headwind. Additionally, all four slipped deals lacked executive sponsor engagement in the final 30 days (graph: engagement signal absent).

No single store contains that answer. On a dashboard, the miss shows up as a red number. The orchestrated synthesis explains why it happened, surfaces the structural patterns behind it, and points to what needs to change.

The core orchestration pattern

The orchestration loop itself is straightforward. Here’s the core pattern:

from concurrent.futures import ThreadPoolExecutor, as_completed


def run_agent(question, tools, execute_fn, model="gpt-4o"):
    # Phase 1-2: LLM analyzes intent and decides which tools to call
    response = client.chat.completions.create(
        model=model, messages=[{"role": "user", "content": question}],
        tools=tools, tool_choice="auto"
    )
    tool_calls = response.choices[0].message.tool_calls


    # Phase 3: Execute tool calls in parallel
    with ThreadPoolExecutor(max_workers=len(tool_calls)) as executor:
        futures = {
            executor.submit(execute_fn, tc.function.name,
                            json.loads(tc.function.arguments)): tc
            for tc in tool_calls
        }
        results = {futures[f].id: f.result() for f in as_completed(futures)}


    # Phase 5: Synthesize results into unified answer
    messages = [response.choices[0].message]
    for tc_id, result in results.items():
        messages.append({"role": "tool", "tool_call_id": tc_id,
                         "content": json.dumps(result)})
    return client.chat.completions.create(model=model, messages=messages)

The tool definitions tell the LLM what each store is optimized for. The LLM decides which to invoke based on the question’s intent. With parallel execution, data retrieval completes in milliseconds even when hitting multiple stores simultaneously, making LLM inference the dominant latency factor, not the data layer.

Why the knowledge graph is the highest-leverage component

Knowledge graphs have existed for decades and have always been powerful. They’ve also stayed on the exotic end of the enterprise stack, and the reason is human rather than technical. The last-mile problem was translating between natural language and graph traversals. A graph database can answer extraordinarily complex relationship questions, such as “Which accounts have overlapping stakeholders with our churned customers from last quarter who also evaluated competitor products?” but asking that question required an engineer fluent in both the graph schema and the business domain. That combination of skills is rare and expensive, which is exactly why graph databases have never quite gone mainstream.

GenAI removes this bottleneck, and it does so precisely where the barrier was highest: the translation step that used to require a specialist. With an LLM as the translation layer, the graph becomes accessible to anyone who can type a question in plain language. The LLM generates graph queries, traverses multihop relationship paths, and explains results in business context. In our revenue variance example, the graph reveals that all four slipped deals share a common pattern of customers who adopted a new procurement platform in Q1 and lacked executive sponsor engagement in the final 30 days. That pattern is invisible in revenue metrics alone, because it requires relationship traversal across account nodes, vendor relationships, and engagement signals.

Designing the graph schema

The critical design decision is aligning the graph schema with your business ontology. The temptation is to model the graph around your data model (tables, columns, foreign keys). The correct approach is to model it around how your organization actually thinks about its domain:

• Entity nodes with lifecycle stages, engagement indicators, and health scores
• Product nodes connected to use cases, adoption patterns, and competitive alternatives
• Relationship edges that capture not just connections but context: strength, recency, business driver, and directionality

When the data model matches the business mental model, the agent’s responses feel natural rather than technically correct but practically useless.

GraphRAG: Deterministic reasoning chains

A powerful extension is GraphRAG (graph retrieval-augmented generation), where the agent constructs deterministic inference paths by traversing the graph rather than relying on the LLM’s parametric knowledge. The LLM isn’t remembering something from training; it’s following an explicit path through verified data. The result is auditable reasoning chains: “Account A connects to Partner B through implementation relationship X, and Partner B appears in three other churned accounts, suggesting a systemic delivery issue.” Each step is verifiable against source data, which is critical in enterprise environments where decisions need justification beyond model confidence scores.

GraphRAG also reduces hallucination risk. When the LLM follows graph edges rather than generating from parametric memory, the actual data constrains the space of possible outputs. The graph acts as a factual guardrail on the reasoning process.

Production considerations

Moving from prototype to production exposes a set of challenges that don’t appear in demos.

Model flexibility and cost management

Decouple the orchestration layer from any specific LLM provider. I can’t stress this enough. If it’s tightly coupled to one provider’s API, you’ll end up rewriting it within a few months, when pricing changes or a better model drops. Implement model fallback for throttling resilience. The landscape moves fast, and you don’t want architectural lock-in baked into your data infrastructure.

Multihop model selection is also worth considering. Use a smaller, faster model for intent classification and query decomposition, where the task is well-defined, and a larger model for synthesis, where reasoning quality matters. Intent classification with a small model takes around 200 ms and costs a fraction of a full reasoning pass, so reserve the expensive inference for synthesis, where quality directly impacts user experience.

Security at the data layer

Row-level security must be enforced deterministically before data reaches the agent, as described in Phase 4 above. This is a nonnegotiable architectural constraint. LLMs are probabilistic systems, and security enforcement can’t be. Don’t rely on the model to filter sensitive information after the fact; the data should never enter the context window in the first place.

Data freshness and serving architecture

Separate data construction from data serving. An offline batch pipeline refreshes the stores from source systems on a scheduled cadence, and the real-time agent only reads preprocessed data. This keeps query latency low while letting computationally intensive transformations such as graph construction, embedding generation, and metric aggregation happen asynchronously. The pipeline should be idempotent and observable, with freshness monitoring per store. When the graph is six hours stale but the metrics store is real-time, the agent should know this and communicate its confidence accordingly. Add freshness metadata to every tool response rather than waiting for a user to catch a stale number and lose trust for weeks; that metadata becomes part of the agent’s context for answer generation.

Evaluation and observability

Agent responses need evaluation at both the tool level (“Did the graph query return the right entities?”) and the synthesis level (“Did the final answer correctly combine tool outputs?”). These are different failure modes that require different approaches.

Log every tool call and result, every query decomposition decision, and every synthesis step. You’ll need these traces when something goes wrong, and in a multistore system that can mean a bad graph query, a stale metric, a poorly matched document, or a synthesis error. Without traces, debugging is guesswork. Build automated evaluation pipelines that test known questions against expected answers and track accuracy over time. Degradation usually signals a data quality issue in one of the stores rather than an LLM regression.

Design that observability around the failure modes that actually occur, because production deployments rarely fail because the LLM is inaccurate. They fail because supporting systems drift. Typical examples include:

• Stale graph data causing the agent to reference relationships that no longer exist
• Poorly generated embeddings returning irrelevant documents from the vector store
• Inconsistent business definitions across stores (e.g., “active customer” means different things in the graph and the metrics store)
• Conflicting metrics from different source systems with no reconciliation logic
• Missing source attribution making answers unverifiable

Handling these well matters just as much as prompt engineering. Most debugging sessions trace back to data quality, not model quality.

Implementation guidance

Start with the questions your BI tool can’t answer today (and beyond). The trap with a question inventory is that it captures only what people already know how to ask. The most valuable questions are usually the ones missing from every existing report. They require stitching together three systems, so users either answer them by hand in a spreadsheet or quietly give up. So catalog what your users actually care about, and pay special attention to the questions they route around. Interview analysts about the analysis they abandon halfway, the recurring spreadsheet they dread, and the follow-up question they never bother to ask because the current system makes it too expensive. If most of what you find can still be answered from a single store, you need a better dashboard, not an agent. The pattern earns its complexity only when synthesis across sources is the bottleneck rather than the data access itself.

Pick one use case and go deep. Build for the case where the manual synthesis burden is highest, where an analyst currently spends four hours pulling data from three systems to answer a leadership question. Prove value in that narrow corridor, then expand; adding a new store is incremental once the orchestration layer exists.

Invest in the knowledge graph early. It’s the hardest component to build and the highest-leverage one to have. The schema will be wrong on the first attempt and less wrong on the third. It evolves with your understanding of the domain, and that understanding deepens only through iteration with real users asking real questions.

Design for the analyst, not the engineer. The success metric isn’t technical elegance. It’s whether the finance analyst stops building the same three-system Excel mashup every Monday morning. Talk to your users, watch them work, and build for their actual workflow rather than your ideal architecture.

Measure what matters. Track response accuracy, latency (P50 and P99), user adoption, and reduction in manual synthesis time. Track the questions the agent can’t answer, because those gaps are your roadmap for which stores to add or which schemas to extend.

What this enables at scale

This pattern does more than make existing workflows faster. It enables workflows that weren’t possible before, no matter how many analysts or engineers you threw at the problem. An agent querying a graph, a metrics store, and a document repository at once can surface patterns no human would find by checking each system manually. A finding like “entities in segment X who adopted product Y and had a support escalation in the last 90 days are 3x more likely to churn” requires reasoning across three data sources in a single inference; no dashboard surfaces that, and no analyst checks that specific combination unprompted. In the same motion it democratizes access, opening information that was previously reachable only by engineers who could write Cypher or SQL to anyone who can ask a question in plain language. The analyst’s role shifts from answering routine questions toward building the ontology, curating the graph, and tackling problems that require genuine human judgment.

It also delivers auditable reasoning at enterprise scale. GraphRAG provides deterministic inference paths that are verifiable against source data, so every conclusion traces back through explicit edges and nodes. This builds trust where decisions carry financial, regulatory, or strategic weight. “The AI said so” becomes “the data shows that A connects to B through X, B exhibits property Y, and historical pattern Z suggests the following.” That traceability changes the conversation from “Can we trust AI?” to a review of the work the AI actually did.

Conclusion

The future of enterprise analytics is unlikely to be a larger warehouse or a smarter dashboard. It is an orchestration layer capable of reasoning across specialized systems while preserving each system’s strengths. Purpose-built data stores remain exactly where they are; what changes is how we access them.

The architectural pattern described here doesn’t replace warehouses, graphs, vector stores, or APIs. It coordinates them. That distinction is subtle, but it fundamentally changes what enterprise analytics systems can deliver. The next generation of analytics platforms will do more than answer questions faster. They’ll reason across enterprise knowledge in ways that previously required experienced human analysts. The data already existed. The orchestration layer did not.

Disclaimer: The views and architectural perspectives in this article are entirely my own and do not represent my employer or any affiliated organization. References to patterns and technologies are based on publicly available information and personal experience. No proprietary or internal information was used.

For three years, the agent story has centered on capability: Models got better at planning, tool use, and long tasks. Integration improved more slowly. Every agent was still wired to its tools, data sources, and host application with custom glue, leaving even successful systems difficult to move or reuse. Standard interfaces change the economics of that work. Networking became an ecosystem when machines could agree on interfaces; programming tools followed the same path with the Language Server Protocol. In my judgment, the standardization now underway will influence what engineering teams ship over the next three years as much as further gains in model capability.

The Model Context Protocol (MCP) standardizes how an AI application connects to tools and context. It uses JSON-RPC messages between hosts, clients, and servers. Servers can expose tools that a model invokes, resources containing context and data, and prompts that describe templated workflows. The spec credits the Language Server Protocol as an inspiration. Language servers replaced a separate integration for every editor-language pair with one interface on each side; MCP applies the same idea to AI applications and integrations. The current spec is dated July 28, 2026, the fifth dated protocol revision since the project launched in November 2024. That pace reflects active governance and warns implementers to expect movement.

Adoption has spread across competing hosts. The MCP maintainers’ December 2025 announcement cited more than 97 million monthly SDK downloads, over 10,000 active servers, and first-class client support across ChatGPT, Claude, Cursor, Gemini, Microsoft Copilot, and Visual Studio Code. Any individual count can be debated. Cross-vendor implementation is harder to dismiss, because a shared interface becomes infrastructure when rivals support it independently.

MCP covers the boundary between an application and its tools. The Agent2Agent protocol (A2A) addresses the next boundary: agents discovering and communicating with one another across vendors. It reached v1.0 in April 2026 with more than 150 supporting organizations, signed Agent Cards for verifiable identity, SDKs in five languages, and general availability in Microsoft Copilot Studio, Azure AI Foundry, and Amazon Bedrock AgentCore. A2A is younger and less proven in production than MCP, but its governance and adoption are moving agent-to-agent communication beyond the confines of a single platform.

The emerging stack extends beyond those two protocols. AGENTS.md, OpenAI’s convention for giving coding agents repository-level instructions, joined MCP as a founding project of the Agentic AI Foundation. The OpenTelemetry generative AI semantic conventions are developing a shared vocabulary for traces and metrics from model and tool calls, though that work has yet to reach stable status. Connection, cooperation, instruction, and telemetry are beginning to acquire common interfaces.

For an engineering organization, the immediate consequence is reuse. An MCP server for an internal ticketing system can serve every compatible IDE, chat application, and agent. Models and hosts will keep changing; the interface contract, server implementation, schemas, and authorization wiring can endure. The integration becomes an asset shared across applications. Language servers followed the same path as editors came and went around them. Agent integrations deserve the same treatment as libraries and services, with owners, versions, tests, and upgrade policies.

Reuse also concentrates attention on a common security boundary. Custom integrations tend to receive separate reviews, when they receive them at all. A shared protocol creates a recognizable seam where teams can specify identity, permissions, data flow, and audit behavior, then apply those controls across many tools and hosts. The MCP authorization specification builds that seam from established IETF work, including OAuth 2.1, protected resource metadata, authorization server metadata, and resource indicators. It requires Proof Key for Code Exchange (PKCE) to protect authorization codes from interception. MCP servers must also reject tokens issued for another audience and must not forward them downstream. The value comes from familiarity: Teams can draw on years of OAuth deployment experience instead of rediscovering its failure modes inside a new protocol.

The spec is equally useful when it states what metadata cannot prove. Tools can carry behavioral annotations describing them as read-only, destructive, idempotent, or open-world. Those terms give hosts a vocabulary for policy. The tools specification still requires clients to treat annotations as untrusted unless they come from a trusted server. Self-description can inform a security decision; it cannot establish the trust on which that decision rests.

Once servers share an interface, they also become discoverable. The MCP Registry is an open catalog and API for public servers, designed to feed downstream marketplaces and private catalogs. Its role resembles npm or PyPI as a discovery mechanism, with an important difference: It stores standardized server metadata and leaves package distribution elsewhere. The registry remains in preview, with possible breaking changes and no durability guarantee.

Discovery brings familiar supply-chain risks to components with unusually powerful access. Typosquatting, abandoned packages, malicious updates, and uncertain provenance now concern software that may hold live credentials and act on production systems. The registry provides namespace verification and moderation, and its downstream model allows organizations to build curated catalogs. An internal subregistry or allowlist is therefore a sensible first control. The standard interface makes that curation practical across multiple hosts.

The same contract improves testing. MCP tools declare a JSON Schema for their inputs and may declare one for structured outputs. Under the current tools specification, servers that declare an output schema must return conforming structured results, and clients should validate them. Teams can test a server without putting a model in the loop, mock it with recorded or synthetic behavior, and contract-test both sides as they would a REST or gRPC boundary.

This separates two kinds of uncertainty that agent evaluations often mix together. Protocol tests can determine whether an integration exchanged valid messages and enforced its contract. Model evaluation can focus on whether the agent chose the right tool and interpreted the result well. OpenTelemetry’s emerging conventions extend this approach to runtime evidence by giving systems a common language for tool calls. Teams will still have to absorb changes as those conventions mature.

These benefits stop at the edge of what the interfaces describe. A schema captures the shape of a tool’s arguments, while its meaning still lives largely in a free-form description that a model must interpret. A schema cannot tell an agent when a tool is appropriate, how its effects interact with other tools, or whether two similarly named operations have equivalent semantics. Portability makes a server available across hosts; behavior can still vary across models and contexts.

The semantic gap leads to a trust gap. MCP can carry a server’s claims, and an A2A Agent Card can carry a digital signature, but a signature only ties a statement to an identity. Engineering organizations still need a basis for deciding which identities, publishers, code, and claims deserve authority. The protocol can carry that decision through a system. It does not make the decision for the organization.

Delegation makes the problem harder. MCP’s authorization model handles a client calling a server with an audience-bound token. Production systems increasingly involve an agent calling another agent, which calls a tool or a third agent. Preventing token passthrough closes a serious hole, yet each downstream hop still needs a narrower grant derived from the user’s original authority. No common mechanism defines how those rights should attenuate across an arbitrary chain. Platforms currently solve this locally or leave too much authority in place.

All of this work is unfolding on young infrastructure. The registry is in preview, the telemetry conventions are unstable, and MCP has produced five dated protocol revisions in less than two years. Revision is how standards mature, so teams should version-pin, keep protocol code behind thin internal adapters, and budget for migrations. Some abstractions will prove wrong because the standards and the underlying practice are developing at the same time.

A sensible response begins with ownership and containment. Each internal server needs a durable owner, and its protocol surface should be treated as an architecture decision. Schema validation, conformance tests, and protocol mocks put that seam under contract. Third-party servers belong behind an allowlist or private catalog, with provenance requirements and corroboration for their annotations. An internal abstraction around the emerging telemetry vocabulary can limit exposure to draft revisions.

Organizations with a large stake in agents should also participate in the standards work. The Agentic AI Foundation and the A2A project are young enough that engineering teams outside the founding companies can still influence what becomes portable, observable, and enforceable. The rules written now will become assumptions embedded in future products.

Model improvements will continue to generate the headlines. Standards determine whether those models can participate in an ecosystem. Ethernet mattered because it became an interface that many implementers could agree on, and language servers mattered because editors and language tools no longer needed to be designed in pairs. Agent systems now have their first interfaces that competitors jointly govern and independently implement. Engineering teams should build on them with two expectations: The interfaces are likely to last, and their current forms will change.

The emergency route itself was fine. Production systems need emergency routes, and a human approved this one. The trouble is that the route now runs without a live incident, an owner, or an expiry condition. A controlled exception has quietly settled into the platform’s standard runtime behavior.

This article is about that execution layer: the temporary operating state that lingers in runtime behavior. Temporary authority is easy to grant under pressure and hard to retire once the pressure lifts. An enterprise agent shouldn’t have to guess from prompts or conversation history whether it’s running under normal, incident, or recovery conditions. That state should be handed to it as authoritative runtime input, the way platforms already hand over identity, tenant, environment, and permissions.

An exception is safe while the incident that justified it is live. It becomes dangerous the moment the platform can no longer show the incident has ended.

The problem is unmanaged exception state

Exception drift is what happens when temporary exception behavior outlives its authorized scope, authority, or duration, and emergency accommodations settle into normal execution. The drift is usually quiet: a routing rule that stays reachable, an approval shortcut that survives closure, a tool permission that keeps shaping execution after the triggering condition has passed.

Enterprises already have the human machinery for this. Incident management defines abnormal conditions, change control governs deviations from standard practice, and postincident reviews confirm that temporary measures have closed.¹,² The gap is architectural. Most agent platforms still treat organizational operating state as something outside the runtime rather than an input to it.

Once an accommodation proves useful, it fades into an invisible operating state. The routing rule stays enabled, the shortened approval path stays reachable, and the temporary queue keeps taking work. No dramatic model failure is required, only a platform with no reliable way to close runtime state.

The practical question to ask is “Which operating mode is active for this workflow, this user, this segment, at this moment?” When the answer is left implicit, every agent, workflow, and tool gateway invents its own. Discussions of agent architecture tend to dwell on capabilities such as models, tools, and orchestration, yet production behavior depends just as much on runtime context: identity, tenant, environment, permissions, session state, policy, and tool access. One piece of that context is usually absent. The runtime knows who is acting and what they may do but not whether the organization is under normal conditions, incident response, recovery review, or a declared exception. Exception drift begins in that blind spot.

The exception lifecycle

Organizations move through a predictable sequence: normal operations, incident declaration, a temporary exception window, closure review, and return to normal. Each stage answers a question, from why the exception exists and who authorized it to what evidence shows it’s no longer shaping execution. Most enterprises handle the front of that sequence well; they’re practiced at declaring incidents and authorizing workarounds. The hard part is the retirement: proving that the exception behavior actually disappeared.

Declaring an exception is loud. Retiring one is quiet, especially when the workaround improved throughput or helped the team recover faster. That asymmetry is where drift lives, because an incident can be closed on paper while emergency routing, override policies, or alternate workflows keep influencing execution. From the platform’s point of view, the lifecycle should close only when it can show that every exception path has been retired or formally adopted through change governance.

Why this matters more for AI agents

Agents raise the stakes because they act. Rather than sitting in a config file, they select tools, trigger workflows, coordinate with other agents, and adapt their paths at runtime. An accommodation introduced during an incident can spread through routing, tool use, approval paths, and downstream agents. A traditional exception stays legible in a runbook or workflow definition; an agent can carry the same exception along many paths at once, which makes it harder to find and retire.

For engineers, the missing layer is an authoritative operational-state context that arrives with the request rather than being inferred from it. Historical traces and retained memory can explain why an accommodation once existed. They should never decide whether it’s still authorized. Memory informs execution; operating mode governs it. And when the two disagree, authoritative runtime state wins.

A small example shows the shape of that context. It carries the minimum needed to bind an exception to a mode, a scope, an authority, an expiry, and a status:

{
  "mode": "incident",
  "exception_id": "INC-4721",
  "scope": {
    "segment": "premium-customers",
    "region": "us-east",
    "workflow": "customer-remediation"
  },
  "authority": "service-owner",
  "expires": "incident-close",
  "status": "active"
}

The pattern sits near familiar ones without matching them. Feature-flag platforms such as LaunchDarkly target behavior by context, RBAC governs what a principal may do, and tenancy metadata tells a service where a request belongs.³ Operating mode serves a different purpose. It doesn’t replace policy, permissions, or memory: Permissions determine who may act, and policies determine how they may act. Operating mode determines whether exception behavior is authorized at all. As a result, it acts as a higher-order governance constraint on agents, workflows, approvals, tools, and escalation paths. Operating mode draws its authority from the organization’s incident and change process and represents a governed state of the enterprise, consumed by the runtime.

A design pattern for exception-aware agents

Operating mode becomes actionable once the platform treats it as a first-class runtime construct. Most agent architectures already inject identity, permissions, tenant context, and policy into every request, and operating mode belongs in that set. Agents consume it as authoritative state after the organization declares the exception, scopes it, assigns authority, and sets an expiry, instead of reconstructing it from prompts or accumulated context.

That reframes the architectural question as whether the platform can guarantee that exception behavior is impossible outside an authorized operating mode. Emergency behavior exists because the platform enables it, and for no other reason. Table 1 shows the minimum contract that makes the boundary testable.

A workflow invocation receives that state alongside the user request, and orchestration, routing, and tool gateways read the same state. Figure 1 shows the shape.

Building an exception-aware control plane

The natural home for operating mode is an external control plane. Incident management platforms, maintenance window services, and change management workflows already hold authoritative operational state, and exception-aware architectures extend those signals into execution with explicit scope, authority, expiry, and closure semantics.⁴,⁵ Implementation will vary across organizations, but the principle holds: Exception state should be authoritative, observable, and externally managed, kept out of prompts and workflow definitions and away from agent memory.

Exception-aware execution

With explicit operating state, behavior shifts when system state shifts, and prompt wording stops being the lever. Under normal operations, agents run standard workflows, routing, and approvals. Under incident mode, the same workflows can expose scoped accommodations that the authorized mode makes available, such as expedited approvals, alternate routing, deferred reviews, or emergency runbooks.

Table 2’s payoff is testability. A workflow in normal mode should never reach an emergency path, and a workflow in incident mode should reach only the accommodations its scope, authority, and expiry allow. Governance becomes an enforceable runtime property the platform can check at execution time.

Return to the opening incident. With operating mode as runtime state, the emergency route opens only inside a declared exception: scoped to the affected segment and workflow, owned by the service owner, and stamped with an expiry tied to incident closure. When the incident closes, the mode returns to normal and the routing gate stops handing that route to the segment, ahead of any manual cleanup. Closure then runs as a check. The platform replays the exception’s scope against live routing, approval, tool, and queue configuration, and confirms that no path still resolves to the emergency behavior. No one has to remember to retire the route; it was bounded by state, and the platform can show it is gone.

A common operating model for multi-agent systems

The problem compounds across collaborating agents. Customer-facing, orchestration, and execution agents may share a workflow while disagreeing about state, so one keeps applying emergency routing after another has returned to standard controls. A shared operating state gives them a single governance boundary: The exception is represented once and read consistently everywhere. As agent ecosystems grow more autonomous, shared operational state matters as much as shared identity and authorization. Fragmented state produces fragmented accountability.

Making exception drift observable

The payoff is observability. These conditions have historically been hard to see because accommodations scatter across workflow definitions, approval policies, routing configurations, and tool permissions. Explicit operating state makes them measurable at runtime. Closure can trigger automated validation, and the platform can watch for residual exception behavior between closures. The revealing checks are direct: Is the exception’s routing path still reachable, do its temporary approvals or elevated permissions still resolve, and does any expired exception still touch behavior? Tracking how many exceptions are open, how long they stay open, and how often they harden into permanent change turns drift from an audit finding into a monitored signal.

From adaptive agents to governed systems

Table 3 captures the shift. Traditional governance documents exceptions and trusts that they are retired. An exception-aware platform represents them as runtime state that can be propagated, validated, monitored, and closed.

The test is whether the platform can demonstrate that the exception stayed bounded to its purpose and duration.

None of this demands a new governance model. Organizations already know how to govern identity, permissions, change, and operational risk, and the recommendation is to extend those disciplines to operating state. Once operating mode is authoritative, observable, and testable, drift becomes detectable, closure verifiable, and recovery an engineering result. The aim is a platform that can make its own operating conditions explicit, hold every agent to them, and demonstrate a clean return to normal. In that model, temporary accommodations stay temporary, governance lives in the architecture, and enterprise agents earn trust at scale.

References

1. ISO, “ISO 22301:2019: Security and resilience—Business continuity management systems—Requirements,” International Organization for Standardization, 2019. https://www.iso.org/standard/75106.html ︎
2. Axelos, “ITIL 4: Change enablement,” IT service management practice, 2019. https://www.axelos.com ︎
3. LaunchDarkly, “Target with Flags,” LaunchDarkly Documentation. https://launchdarkly.com/docs/home/flags/target ︎
4. ServiceNow, “Change Management,” ServiceNow Product Documentation. https://docs.servicenow.com ︎
5. PagerDuty, “Maintenance Windows,” PagerDuty Support Documentation. https://support.pagerduty.com/main/docs/maintenance-windows ︎

For most of the history of technical publishing, editing and verification were the same process, and that process was slow and expensive. Getting a book out took years. We found an author, vetted them, had them work with an editor, and checked their claims with technical reviewers. A lot of that time went into separating what was correct and useful from what was confusing or only sounded right. It was laborious, but it meant a reader could depend on the claims on the page. The credibility of the book, and of the publisher behind it, mattered as much as the information itself.

When the cost of production drops to zero, that credibility becomes worth more, not less. Content is easier to make than ever, but without a transparent process behind it, readers have no idea where the knowledge came from or whether it holds up. As Jasmine Sun puts it in “The Independent Writer’s Advantage in the Age of AI,” “Trust is not about information and its quality alone. It’s about the messenger. It’s about who says it and their track record and what they’ve told me before.” A practitioner has confidence in a source because someone she respects has put their reputation on the line for it. They believe what the author is saying because the publisher has a history of being right and of correcting itself when it isn’t, and because the work is attributed and verifiable.

The corpus matters, but it’s the assurances around it that are hard to replicate, and that comes not just from the people who produce the content but from the people whose judgment vouches for it. Sometimes a creator brings their own credibility with them. Other times, the publisher spots someone unknown and lends them its own. The art critic Dave Hickey said this about gallery owners in Air Guitar: They gain status from the famous artists they represent and share it with emerging talent who have something to offer but who haven’t had the chance to earn a reputation. This is what O’Reilly has done for nearly half a century, build a network of experts who vouch for what’s worth knowing.

Expertise is alive, and it compounds

Expertise is a living thing, continuously expanding. Content starts to decay the moment it’s published, because frameworks evolve, libraries deprecate, and yesterday’s best practice becomes today’s security incident. Keeping expertise alive requires a pipeline of people who stay current and an editorial layer that notices when something has gone stale, and either retires it or calls for a fix.

That pipeline isn’t something you switch on when an author has a book to ship. At O’Reilly, we’ve always prided ourselves on living at the bleeding edge, finding what Tim O’Reilly calls “the alpha geeks” and spreading what they know to everyone else. Content sits at the center of our platform, but we think about it in pace layers. Some advice is timeless, some moves but has a long shelf life (some of our books are still in print after nearly 50 years!), and some changes weekly. We work with experts at each pace layer, capturing what lasts while doing our best to keep pace with an industry that seems to have changed every time we wake up. We have relationships with hundreds of the best practitioners in the world, and our job is to keep them engaged continuously, with quick takes when something breaks, structured responses when major research drops, and live sessions on emerging topics while they’re still emerging.

An institution doesn’t stamp trust onto content. In a technical community, trust is conferred in both directions. A practitioner earns standing because people who already have standing engage with her work, cite it, argue with it, and build on it. That insight was the whole idea behind PageRank, Google’s first great innovation. A page mattered because other pages that mattered linked to it. Reputation works the same way.

The audience isn’t just consuming reputation signals; it’s generating them. When a senior engineer whose judgment others respect says out loud that something is worth reading, she spends a little of her own credibility; the author gains a little; and everyone watching recalibrates whom to trust next time. O’Reilly plugs into that existing economy of reputation. When we put our mark on someone’s work, we aren’t the sole source of its credibility. We’re amplifying a judgment the community is already making and adding our own track record to it. The reader who finds it reliable hands status back to the source.

When the readers are machines

Human practitioners aren’t the only ones who need trusted engineering knowledge. The AI systems now sitting in every workflow, the coding and debugging agents and architecture advisors, need it just as badly since most of them are built on scraped web data and documentation that was stale before it was ever indexed. They’re fluent, but they’re wrong often enough that you can’t just take their word for it.

The stakes grow with AI increasingly being used to generate not just provably correct types of content like code, which either works or it doesn’t, but persuasive documents in fuzzier areas like hiring, strategy, and so on. Like everyone else leaning on these tools, we at O’Reilly are reckoning with the consequences of the ability to talk to a model and get back something that looks smart at a glance. A few rounds in, the slop is still there. In the last few months, maybe 10 times as many documents have crossed our desks, from new product ideas to strategic plans and proposals. But the ease of generating the text hides the fact that either the model or the person prompting it doesn’t actually know what they’re talking about. Knowledge workers need ways to ground their work in insights from human experts, particularly when that work is AI-assisted. So we’re building tools that let agents draw on our repository of expertise to support their proposed decisions.

Credible sources are particularly important when thinking through and justifying important choices. Our CTO, Andrew Odewahn, describes the shift this way: “18 months ago, it was all about how to get engineers to be more productive, but now it’s about how to get organizations to make better decisions. The engineering tasks are moving away from coding output to planning.” For planning tasks like comparing implementation approaches, you need expert-over-your-shoulder guidance for contextual decision-making. You can’t just rely on an LLM’s best guess to solve your problem, which is why we see great opportunity for new products like O’Reilly’s Expert Intelligence offering that delivers grounded knowledge embedded in your AI tools and your workflows to help navigate what you do. Trust is foundational because the expertise behind it stays genuine, practical, and human.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Takeaways

01.26 The usability gap is smaller than the marketing gap. The capabilities of local models are only a few months behind those from the big commercial companies, but because there’s no subscription revenue model behind local models, they often go unpromoted. “It’s very hard to make money off local models,” Pete explains, so the big companies aren’t focused on selling them. “Every company is going to go for the [product] that has an easy subscription revenue model. And that means you have a massive ton of marketing around all of these tools that are kind of like, ‘Oh, let’s have a little text box on a website.’ And so it means mostly that people have never heard of these local models.”

04.20 Local models are already good enough for most use cases. Pete compares the moment to the early web, when free alternatives like Apache eventually overtook expensive commercial servers. “All of these alternatives, once people actually had time to look around and they had a little bit of time to improve, they just wiped the floor with the commercial [offerings],” he points out. “I don’t know if we’re going to quite get there, but that’s the kind of pattern that I’m seeing.”

07.26 “The hardware barriers are a lot lower than people think.” Ben and Pete discuss what hardware you actually need to get up and running, from parameter counts, quantization (Q4, 8-bit), and VRAM requirements to the new Apple M5 Studio’s unified memory as a way to run very large models locally at usable speed. “The key thing is whether you can fit [your model] into your graphics card’s memory,” Pete says. “So with weight quantization, 9 billion [parameters] if it was 8 bits is like 9 GB. A lot of mid-end decent laptops that are shipping now have more than that.”

18.33 “It’s not that people don’t like voice interfaces. It’s that people don’t like bad voice interfaces.” We’ve solved most of the big problems, like dealing with background noise, phrasing, and speech in a range of accents—or at least have improved tools’ capabilities. However, “there’s no commercial incentive to kind of pull them all together,” Pete says. Most tools feel like they haven’t caught up to the LLM era, but “open source can be a really strong lever” to updating them, argues Pete.

28.26 We’re navigating the split between “LLM maximalist” end-to-end models (favored by big AI companies with the most capital) and the “compound AI” approach of chaining together specialized models from different sources. “If the future is end-to-end models, then only the people with the most money can actually build and train them,” Pete notes. Compound AI lets you “actually train all of the models independently” to accomplish your particular goals. While the performance of end-to-end models continues to improve, especially for multimodal models like Qwen or Gemma, using one can be a bit like choosing a Swiss Army knife over a tool specially designed to accomplish a single specific task, to use Pete’s metaphor. It may get the job done, but it’s probably not the most effective way to do it.

36.10 Voice capabilities in the browser could be a game changer. Embedding a model directly in the browser—Chrome has a built-in ~4B parameter model that’s accessible from any website via JavaScript, for instance—makes it part of the operating system. “Once you are able to transcribe fast and accurately in the browser, it’s a way for people to easily start experimenting with this stuff,” Pete explains. Could this be an iPhone moment for LLMs?

39:58 The “gravitational pull” is toward on-prem. Unlike most recent technological advances that depend on the cloud to function, LLMs are well-suited to running locally, even with no internet connectivity. Enterprises are grappling with concerns about cost, privacy, capabilities changing with no notice, or even the models they depend on disappearing. Hosting your own model, whether on your laptop or in your corporate infrastructure, gives you the stability to plan for the long term.

44:21 GPUs are fantastic for training but “complete overkill for inference.” Pete likens it to “trying to use an oil tanker to go and do your shopping.” Memory bandwidth is the real limiting factor, and it’s a problem that companies like Apple, with its new chip designs and unified memory bandwidth, are working on solving. “Even if you’re running on the CPU, if you have something that’s got high-enough bandwidth to pull 27 billion weights in a fraction of a second, then the rest of it is fairly easy in terms of actually doing the processing,” Pete says. “I think we’re going to see a lot of really imaginative solutions now that people understand what the workload looks like.”

The following article originally appeared on Addy Osmani’s blog and is being republished here with the author’s permission.

In the past year, the conversation around agentic engineering has moved to harnesses and loops, fleets and software factories. My 2 cents is engineers need to own the outer loop—the accountability for these systems. This only gets more true as powerful models like Fable and GPT-5.6 become available.

Agents have leverage, and leverage creates obligations. Someone must be able to explain exactly what changed, why it was safe, and what will happen if they’re wrong. Otherwise, their actions can’t be justified. Which makes it unlikely their organization will ask for them in the first place.

And so I want to talk about three terms. The first, Quality, refers to all the checks we install before we let the system loose. Those checks produce evidence, and from that evidence we derive a Verdict.

The second, Verdict, refers to the final decision we make before work enters our dependent system: I’m the line-producer of this content. I run the team whose work is shipped under my name. The model may write the line, but the Verdict is mine. The work of my team will not enter our dependent systems without my decision. A Verdict is the production decision: Should we ship, block, redirect, narrow the response, add a guardrail, or reject outright?

The third, Answerability, refers to the guarantee that if someone asks, I can explain why.

To say this another way: Our agent (which I define as a model plus a harness of files, tools, memory, skills, sandboxes, permissions, observability, and recovery) is what runs our loop (which I define as investigation, implementation, verification, and repeat). And it’s what creates our software factory.

The model is just the engine. The harness—tools, memory, permissions, sandboxes, tests—is the car you build around it so it can do real work safely.

The loop is how one good run becomes a process you can trust to run again. Wrap that harness in a repeatable cycle—investigate, implement, verify, repeat—where an independent check, not the model’s own say-so, decides when the work is done.

Now run many loops at once. A factory is loops at scale: The agents ship the work inside, while humans own the decisions at the boundary.

And at the heart of that factory is a careful boundary between what’s inside the system and what’s outside it. Inside the system we collect inputs (from the product team’s intent, or knowledge of previously shipped work, or of recent incidents, or of specific feedback from users). The agent loop investigates the task, implements a plan, and verifies the result. Then, evidence crosses that boundary. A human, who owns the dependent system, sees the evidence and decides whether to proceed.

And that, friends, is the shift we’re trying to make. Before, our agents were doing the inner loop of the execution loop. Now they run the inner execution loop. Engineers own the outer loop.

Inside the system, there’s really just one kind of thing our agents are doing: capability. The capability to investigate tasks, implement plans, test their results, and report back. That’s the capability of a model. And as we’ve said, that future is already here.

Outside the system, there’s a single kind of thing: agency. The agency to decide, verify, approve, and own.

We’re still talking about code, you see. It just needs to live in a place and be performed by people who know what they’re doing.

The potential for AI code is no longer marginal. In a Sonar 2026 survey, we asked teams about the share of their commits that were AI-assisted. It was small but nontrivial. And several of the respondents said they expect the share of AI-assisted commits to grow substantially.

Sonar’s 2026 State of Code report found that 42% of committed code was AI-generated or significantly AI-assisted, with expectations for that share to keep growing rather than plateauing.

Creation, in other words, is getting cheaper. Scarcer resources are review, validation, understanding, and maintenance.

We moved the speed of generation faster than we moved the speed of control, and so we have a trust-verification gap. A lot of people we talk to still express some degree of distrust in AI code. Yet fewer of them seem to consistently build that distrust into their verification processes.

That’s a dangerous place to be. We’re going to need cheaper, clearer ways to verify the trustworthiness of AI code.

If you look at the GitLab June 2026 report, you’ll see that governance questions have shifted.

GitLab’s June 2026 AI accountability research shows that review and validation are the current bottlenecks when using AI and, more worryingly, that governance usually happens after code creation, after we’ve accepted the risk and lost control over ownership. Today, it’s not just about control. It’s about what constraints we set on the system. It’s about how we’ll check the work with evidence, and how we’ll hold teams accountable. It’s about who will own what part of the AI lifecycle.

So the final distinction in this series is between process and quality. Quality is the concept of backpressure. We mean it literally. We don’t want to grant our agents as much autonomy as they can possibly exercise. We want to grant them just enough autonomy that we have enough backpressure to stop them, regulate them, check their work, and ensure our humanity.

Ordinary engineering holds up a lot of signals that indicate that the work being done is doing the right thing. Type checks, tests, hooks, sandbox limits, audit logs, monitors. Our engineering systems are full of these kinds of signals, and they’re designed to provide enough backpressure to keep the system honest.

And so as long as our agents are emitting these same signals, we can trust our ordinary engineering to provide appropriate backpressure.

Trusting our systems doesn’t mean we don’t want a human in the loop. It just means that the human doesn’t need to be in the inner loop. We want them in the constraints loop (What inputs, architectures, instructions, or invariants should we set?), the sampling loop (How much output should we sample and review?), the audit loop (What evidence should we keep, and how do we make sure our audit log is effective?), and the ownership loop (What part of the production boundary should we own?).

But the human doesn’t need to be in the inner loop.

The agent can ship more than you can review.

And the scarce resource is your own core human judgment, informed by quality signals like logs or tests.

The AI June 2026 report shows that, in the experimental setting, agentic delegation along hour-scale time horizons is essentially here. The work by OpenAI this year on agents and the future of work was a great source for these ideas. So we need to start thinking about how to establish this ownership boundary, as our systems start shipping more than we can review.

And that’s where the answerability comes in.

Because with long-horizon agents, the decisions made over hour-scale time horizons are just that—decisions. And not all the decisions are going to be recorded. You can’t trace them all back to input tokens. If all you’re doing is trusting that the output you get is the correct choice for the problem at hand, the hundreds or even thousands of human hours of work you’re going to need to reconstruct the chain of decisions that lead to it become impossible. And so, again, answerability becomes something that must be at the core of our system design.

Three hidden costs

And there are three hidden costs:

Cognitive surrender ~ blindly accepting what AI gives you. When you delegate work to an agent, the work itself may appear to be the work of the agent. But it’s actually your work. It’s your reputation. It’s your responsibility. And it’s your software that suffered the defects in the output. And it’s your software that needs to be changed to reflect that output. So the agent’s output becomes your answer. And with it comes all the accountability. The Wharton study that put this together is reassuring when the AI is right. But when it’s wrong, the news isn’t great. When the AI was wrong, nearly three-quarters of people accepted it anyway, and felt more confident than they would have without the AI.

Cognitive debt ~ erosion of your understanding and memory of how to solve problems. When you delegate work to an agent, you’re offloading all the thought work to the agent. And while thinking it all out yourself takes time and energy, thinking it out on a massive codebase takes resources that aren’t available when you’re trying to run up the learning curve. So the output you get is often unattainable by you. And the longer the time horizon of the agentic planning, the bigger the gap between the code the agent produces and your understanding of it becomes. The gap compounds. The debt accumulates. And the cost of climbing the learning curve grows almost exponentially.

There’s a randomized controlled trial from Anthropic looking at whether engineers who lean on AI to write code understand it as well as engineers who write it themselves. The conclusion was gloomy: On a comprehension quiz, the engineers who worked through AI scored 17 percentage points lower than those who didn’t, 50% versus 67%.

And then there’s the orchestration tax: It’s easy to spin up lots of agents now, but your cognitive bandwidth doesn’t parallelize in the same way. Steering your agent away from the worst behaviors, sorting the work the agent produces to identify the ones that need your attention, directing it to focus on the work you care about first, verifying your most important constraints and your most dangerous assumptions before you let it run. . .

All of that takes work, and it can’t be automated. There’s no substitute for human judgment.

Brownfield systems are especially dangerous here, because the system behavior you have to audit doesn’t live in the code. It lives in the scars.

Fixes? Make attention the priority in your architectural decisions. Use worktrees, scopes, and evidence to reduce the coupling between your initial plan and the work that emerges from it. Time-box the effort to resolve unactionable steps. And make change in your software strictly an opt-in permission.

Alpha, decay, and taste: These are the three core patterns that shape careers and performances across domains.

Alpha is the lead part taken up by the highest achiever in the competition, when you’re playing your highest-value game move. Decays are established patterns that everyone learns through repetition and watching others (plateaus, if you like). Taste is the earliest we can sense the lead in an alpha or the change in a decay. It’s our judgment of what’s coming before we have any evidence that anything is happening.

Paul Graham’s point is that when anyone can make anything, choosing what to make matters more, and Mitchell Hashimoto’s definition is the operational one: making high-quality qualitative judgments where no objective metric exists yet. From now on, taste drives everything. Alpha shifts are taste changes. And decays fade out because we start to taste something different.

Next step? Operationalize your taste. How? Give it a name that reflects what you’re trying to move from limbic to conscious. Practice it in critique and examples. Make its rationale explicit.

And keep making the move that delivers the most durable competitive advantage in your industry. What’s that? Keep moving the edge up from just doing the task to teaching it, systematizing it, deciding when it should be done, and owning the result.

Everyone is a developer, but not everyone is an engineer. Engineering is what a developer turns into when they embrace a work discipline that is more strict: thorough and logically sound reasoning, consideration of constraints and tradeoffs, recognition of risk and exposure, and practical accountability.

In the future, people will leave the administrative work of engineering and embrace new roles that emerge as engineering becomes more demanding. Roles that are unbundled from the spirit of craft but make clear what each person does. There will be those who prototype. Those who build. Those who sweep. Those who grow. Those who maintain.

The humans hold the edge of the system in the other direction too. Increasing the alpha: choosing what is worth doing, defining the constraints within which it should be done, deciding if the evidence is sufficient to proceed, and caring for the result. Whether it’s a single team or a hundred teams, this is the edge that only humans can hold.

Accountability will scale the factory. Like attention and taste, accountability is also one of the three dualities that makes everything work. Without accountability, there are no rules. No wrangling with questioners. No trade-offs. No risks. No safety nets. If nobody owns the consequence of a decision, then high agency can only bring chaos.

The half-life of an edge is one release, but the half-life of a signature is a career. A signature is your name on the work, such that you feel you can stand behind what was shipped. Skills get you leverage; accountability turns leverage into trust.

Only people can choose. Only people inherit consequence. Agents can be asked to choose, route, merge, and escalate safely inside a policy, but they cannot inherit the consequences.

Every codebase should perhaps come with some kind of accountability contract that explicitly states the checklist that was understood when the change was accepted, the evidence that went into the decision, who was accountable for the change, and the system status after the change was blocked. Just like:

• Your attention and taste
• Your evidence, verdict, and ownership
• Your alpha, decay, and taste

High agency

In a typical agentic workflow, high agency is the art of knowing when to delegate, when to inspect, when to stop, and when to own the result of a process. The ladder of agency runs from low to high: flag a potential problem, investigate it, execute against it, diagnose it, propose solutions, recommend fixes, and resolve the issue. A high rung on the agency ladder is discernment: found it, it’s not worth fixing, moving on.

The 12 pillars that hold up the software factory

Brownfield is the frontier for factories that hope to scale. All those clever little innovations may not feel like much yet, but the production environment is a lot. When building an entirely new system, it’s much easier to plan and implement sufficient backpressure mechanisms because you have full control. When you’re adding intelligent agents to a legacy system, however, it’s another matter entirely.

Legacy systems include the entirety of production behavior, future expectations from customers, migration histories, release and budget cycle durations, unspoken assumptions, edge cases, data weirdness, runbook procedurals, and all the scars that accumulated without the will to care for the system.

To be a steward of brownfield requires a form of durable engineering. Work has to be done to turn implicit knowledge into explicit constraints, keep it coherent across teams and through generations, formalize that knowledge into test procedures and functional specifications, and tie that knowledge to objective evidence. All while ratcheting failure into more learning. Because if the system doesn’t get the care it has always received, everything will come crashing down.

New work is real work

The work will get more interesting as you scale. Because when everything else is built, people will want to build new things. They’ll want to employ the alpha and taste they have developed through their craft to design new loops that can be grafted onto the software factory. Or they’ll want to build greenfield systems that employ all the knowledge of the software factory to one elegant, well-meaning, principled effort. They’ll want to design and implement new forms of evidence that will rise to the level of verification for the new systems. They’ll want to take care of brownfield systems that are now so complex they need dedicated attention. They’ll want to design and manage new backpressure mechanisms. They’ll want to design new agents. And they’ll want to build agency.

And, as they do, they’ll come to see that all this is real work. That’s a good thing.

Automation creates bottlenecks. Bottlenecks in production that are worth owning. Because automation gives us control over industrial scale. But there’s also new bottlenecks that arise from industrial scale. The bottleneck moves from “Can we build this?” to “Should this exist? Can we answer for it?”

What I’m suggesting is a practical operating model for scaling agentic engineering. There’s inner and outer loops. The inner loop is where the work is done. Loops are designed to be as independent as possible. Put all quality assurances and verification inside the loop. Once you’ve designed and validated the loop itself, the only thing you have left to do is to grant autonomy by putting in place a back-pressure mechanism that acts to control the rate at which the loop is run and its scope of operation. And put humans in their rightful place, on the right decisions. Don’t treat understanding as a hand-off or a release gate but rather as a point of decision where humans are primed to provide their insight. And then for every artifact that exists and is fed back into production and into new teams and engineers, leave behind better artifacts.

Build the factory; keep the lights on; make work legible, verifiable, owned.

An agent can write it. But before it reaches users, someone must explain why it should exist, why it’s safe enough to be part of production, and what they will do when it is wrong.

That’s agentic engineering at the outer loop—that’s the work now.

The following article originally appeared on Sean Goedecke’s blog and is being republished here with the author’s permission.

In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.

Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver—PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing—you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.

This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.

A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently discovered counterexample to the Jacobian conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.

There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:

• Tao’s messages are very short and to the point. He doesn’t respond point by point to the model, just to the gist.
• The model outputs are much more concise than when I try to talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode.
• Tao pushes back when the model’s responses look wrong, but he doesn’t directly contradict; instead, he says things like “This looks more complex than I was hoping for.”
• Tao makes several leaps and suggestions himself. He almost never takes the model’s advice about where to go next.

However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multiparagraph response, suggesting alternate approaches or formulations, and identifying what “looks weird.”

Terence Tao is a better mathematician than I am a programmer. But the idea here—that domain knowledge makes you better at using LLMs—is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say, “No, I think it could be simpler here” or “But don’t we already do X?” or “Can we express this problem in these familiar terms?”

This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “Does X work here?” or “Given Y and Z, why A?” I can’t ask those questions about the Jacobian conjecture, but I can ask them about the systems I own at GitHub.

If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.

This post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI does have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.

In this episode of Zero to Agent in 30 Minutes, Microsoft senior solution engineer Maxim Salnikov walked through the Agent Package Manager (APM), a terminal-driven open source product from Microsoft that treats agent context the way modern software already treats its dependencies: versioning it, pinning it, and checking it before it ships. It’s a technical session, but rather than building an AI agent, you’ll discover how to manage all the customizations you’ve installed for your agents and make them portable, secure, and governed by policies you or your company define.

How to build a supply chain for agent context

Maxim demoed the process of setting up and using APM step-by-step. Here’s how it works.

1. Keep every customization in your codebase. Agent skills, custom instructions, and hooks tend to spread across a developer’s machine, a harness’s install folder, and enterprise-level defaults. Maxim recommends storing them in the codebase instead so that anyone on a new machine or using a new harness ends up with the same setup as everyone else. Yes, your codebase will be bigger, but the trade-off for consistency is worth it. But be sure that skills and other customizations are secure before adding them to your codebase. Consider building your very own enterprise repository of the customizations you’ve identified as safe to use. (See step 5.)
2. Install APM and initialize a project. APM works like npm for AI agent customizations. Running apm init sets up an apm.yaml file targeting one or more harnesses, such as GitHub Copilot, Claude Code, or Cursor, and apm install pulls a skill from a repository into the right location for each one. When you install a skill, APM also creates a log file documenting the entire resolution history.
3. Pin to a version or a commit hash. Once a skill is installed, pin it the way you’d pin a library dependency. Doing so protects against a scenario where a legitimate skill later gets compromised and a bad actor slips malicious content into a new version under the same name. APM also scans files for hidden Unicode characters that may contain malicious instructions.
4. Reproduce the exact setup with a lock file. APM logs every file and hash it installs. Running apm install --frozen rebuilds that exact environment from the log instead of reresolving everything from apm.yaml, so a teammate’s machine ends up with precisely the same setup as yours.
5. Source customizations from a vetted registry, and set a policy for it. Public repositories work for experimentation, but Maxim suggests enterprises build a private, gated registry of reviewed skills. APM policy files enforce which sources are approved, so a customization from an unvetted location gets flagged.
6. Audit for drift, then enforce it in CI. The apm audit command checks installed customizations against policy and catches unauthorized sources or content that has changed since installation. Run that same audit as a gate in a CI/CD pipeline to protect the entire organization against skill drift and bad actors.

The software supply chain already has decades of tooling behind it. That discipline hasn’t caught up with agentic AI, but APM is attempting to close that gap. Explore the project GitHub repo and get started.

Coming up next

On September 9, Menyala’s Sajal Sharma joins Zero to Agent in 30 Minutes to build a shared knowledge base that acts as a common brain across agents. He’ll show how a single repository of research, daily logs, and notes can give Claude Code, Codex, OpenClaw, and Hermes access to the same accumulated information instead of starting from zero with every new session.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

Dashboards don’t interpret themselves

AI and real-time data give organizations more information than ever. Dashboards. Live metrics. Instant customer feedback. All of it moving fast. But data doesn’t make decisions. People do. A dashboard can tell you that cart abandonment jumped 12% this week. It can’t tell you why, and it definitely can’t tell your marketing, product, and support teams what to do about it together. That takes a conversation. It takes people in a room—virtual or real—looking at the same thing, arguing about what it means and what to do next.

The map freezes a moment so you can talk about it

Experience mapping is a broad field of visualizing human experiences. You’re probably familiar with things like journey maps, service blueprints, and other similar diagrams of the experiences. But none of them hand you an answer. What they do is take a fast-moving, chaotic situation and freeze it for a moment. It gives a team something to point at, argue about, and align around.

Picture a typical working session. People from different parts of the business sit down around a map of the customer experience. Each of them already knows a piece of the picture. None of them has the whole picture, together, at the same time. That’s what the map provides. That’s usually the moment something surprising surfaces. Not because the map contains secret information but because it puts scattered knowledge in one place, in front of the people who each hold a piece of it. The visual aspect of maps is critical. Laying out an abstract concept like a “customer experience” allows teams to engage with it in new ways and reach new conclusions that are hard to get from a spreadsheet or data alone. Grasping cause and effect in one visual overview helps teams find the patterns of behavior that matter the most and to conceive of viable interventions.

AI can surface these kinds of patterns in seconds. But it cannot create the moment when a cross-functional team collectively recognizes how its silos are hurting customers. Only people, looking at the same picture, can do that.

The map isn’t the point

Some claim that journey mapping is dead. That static maps can’t keep up with real-time data and AI-driven personalization. This confuses the artifact with the activity. A map that gets built, presented once, and filed away never helps anyone. It fails for the same reason a report fails: Nobody’s talking about it anymore. The value was never in the diagram. It’s in the conversation the diagram makes possible.

Take how I got that team to reach their own conclusions about the invoice problem rather than just telling them about it. After scoping the customer type and situation we wanted to understand better, I interviewed a dozen or so customers about their billing experience. Nothing unusual came up at first. People described the routine steps: get an invoice, check it, pay it. But a few mentioned something in passing. They’d disputed a charge and kept getting late payment warnings anyway, even while the dispute was still open.

From those interviews, I built a draft map of the invoicing journey. I called it a draft on purpose. I didn’t want to hand stakeholders a finished diagram and ask them to approve it. I wanted them to lean into it, question it, and add to it. Then I scheduled a working session. The room included people who’d never worked together before, despite being at the same company for years: billing, support, and product.

We didn’t rush through the map. We slowed down, section by section, and used structured exercises to pinpoint the moments that mattered most to customers. That’s when someone in the room realized: A customer who’s actively disputing an invoice can still get a warning notice for that same invoice. Nobody had designed it that way on purpose. It fell through the gap between two systems that never talked to each other. But once it was visible, laid out in front of the people who owned each part of the process, it became impossible to ignore. The room got quiet, then loud. People were genuinely upset, not at each other, but at what customers were going through.

Of course, I had uncovered this already in my research. And sure, it was also visible on the map. But my diagram wasn’t about giving a magic answer. The process of learning together is the point. That reaction didn’t come from a dashboard. It came from people confronting the evidence together, in the same room, at the same time.

What actually changed

Before the workshop, this problem was invisible in a specific way. Support knew customers complained about warning notices. Billing knew disputes existed. Product knew the systems didn’t sync. But no one held all three pieces at once. The map put all three in the same field of view. That’s the mechanism. Mapping doesn’t create new information. It puts existing, scattered information into one shared picture, at the same time, in front of the people who each hold a piece of it.

What changed after that: Billing and product agreed to flag disputed invoices so no warning could go out. Support got a way to check dispute status before responding to a complaint. And the three teams kept meeting monthly, something none of them had done before. The map didn’t do any of that. The conversation the map created did.

What good collaboration looks like

We started with customer evidence and a deliberately unfinished map. We included people who owned different parts of the experience and asked them to question what the map showed, identify what they knew and what they were assuming, and examine the gaps between their systems. The session ended with specific commitments, and the teams continued meeting as they learned more.

That is what getting collaboration right requires: the right people, shared evidence, visible disagreement, clear ownership of the next decision, and a cadence for revisiting what the team thinks it knows. Without those conditions, mapping can easily become another workshop that produces an attractive artifact but little change.

What this means for your team

As AI speeds up execution, don’t cut the time you spend aligning as a team. Protect it. Expand it. AI won’t give you an edge. Your competitors have access to the same models you do, trained on much of the same data, producing much of the same output. If everyone moves at the same speed, speed stops being an advantage. It becomes the minimum bar for staying in the game. AI also works like a spotlight, amplifying whatever’s already happening in your organization. If your teams collaborate well, AI makes that strength visible fast. If they’re siloed, AI exposes it just as fast.  Now is the time to get collaboration right, while staying focused on the customer. Waiting until AI forces the issue is waiting too long.

In the end, AI can help with customer discovery and accelerate insights. But it doesn’t replace human judgment and decision making. Rallying around a map—a visual depiction of customer experiences—provides a natural forum for discussion, debate, and shared understanding to align before acting. The tools will keep getting faster. The organizations that win won’t be the ones with the best dashboards. They’ll be the ones who are best at coming together, again and again, to make sense of what those dashboards show them.

If you want to dive deeper into mapping, join Jim on October 9 for his Beyond the Book conversation about the latest edition of Mapping Experiences. He and host Vicki Reyzelman will chat about how experience mapping has evolved from a UX technique into a strategic capability for organizations, how AI is transforming the way we create and analyze maps, and how you can use mapping to align business goals with customer needs, facilitate collaboration across teams, and drive transformation at scale. It’s free to attend. Register now.

Host Christina Stathopoulos, founder of Dare to Data and a former data scientist at Google and Waze, returned to This Week in AI with developments that stretched from Claude testing ways to improve model safety to Chinese open weight models gaining developer traffic and new systems learning to model physics. She also examined what Anthropic and OpenAI’s business moves, workforce forecasts, and debates over access reveal about how quickly the AI landscape is broadening.

Claude is taking on more of the research process

Anthropic provided an early example of AI helping improve future AI systems. In research Christina highlighted, Claude searched existing work, proposed methods, generated training data, and repeatedly tested and refined its approaches to reduce unwanted model behaviors. The experiments covered 10 such behaviors, including deception, hallucination, prompt injection, privacy violations, and reward hacking. Anthropic reported improvements across all 10 without degrading the model’s broader capabilities.

For deception, Claude tested more than 150 methods and eventually closed 85% of the measured safety gap. Human safety researchers closed only 20% in Anthropic’s comparison. Christina emphasized that this wasn’t a direct contest because Claude could run and refine experiments much faster and at a much greater scale. She also cautioned that the work didn’t amount to full recursive self-improvement.

This showed how AI could increasingly handle experimentation in model development, changing the pace and scale of research while humans still set the goals and evaluate the results.

Model performance is only one part of the frontier race

Competition among AI labs increasingly involves business performance, infrastructure, and deployment options alongside model quality. Anthropic estimates that the market for its systems could eventually reach $30 trillion, a long-term estimate that Christina treated skeptically because it approaches the size of the entire US economy. She also highlighted more concrete evidence of momentum in how Anthropic’s annualized revenue run rate rose from less than half of OpenAI’s at the start of the year to surpassing it within several months. Both companies are preparing for possible public offerings.

OpenAI faces a different set of pressures, and Christina highlighted its 14 executive departures this year. That sustained leadership turnover could raise questions about the company’s ability to execute consistently. OpenAI is also trying to gain more control over its infrastructure. Its Jalapeño inference chip, developed with Broadcom, delivered up to 1.9 times more AI work per watt and up to 3.6 times lower latency than comparable NVIDIA systems in OpenAI’s own testing.

Chinese open weight models are widening the field further. Christina cited an AI gateway where open weight models recently reached as much as 62% of developer traffic on a single day, compared with an average of roughly 10% in April. The competition now spans benchmark performance, capital, infrastructure, cost, deployment flexibility, and organizational execution.

Physics models could extend AI beyond language and images

Christina closed with research aimed at helping AI systems model physics. Researchers from MIT and Tsinghua University developed a pretraining approach that learned from more than one million synthetic interactions between moving particles and complex 3D objects, then applied those patterns to simulations involving wind, water, collisions, and light. The researchers described physics as a potential “third modality” for AI alongside language and pixels.

She also covered Accelerated Understanding, a startup that recently emerged from stealth with an architecture based on neural operators rather than transformers. The company is targeting problems involving enormous physical datasets, including chip design, robotics, extreme-weather forecasting, and geological exploration.

By learning directly from physical systems, these models could become valuable for simulation, engineering, robotics, forecasting, and other work that depends on understanding complex real-world environments.

What’s next

Christina also examined who could benefit from these advances. She discussed Bill Gates’s argument that access, deployment, policy, and distribution will shape AI’s social impact, and brought in new US Bureau of Labor Statistics projections showing job growth in areas including technical services and healthcare, while office and administrative roles face greater pressure from automation.

Her larger point was that access, workforce preparation, and public policy will determine how AI’s benefits and disruptions are distributed.

Due to the Labor Day holiday, This Week in AI will return on Monday, September 14, when we’ll dive into more of the news, issues, and key developments shaping the AI era. And check back each Friday for the latest episode, or watch on YouTube, Spotify, Apple, or wherever you get your podcasts.

That’s why, three months ago, I built my own software factory, Squid, to ship all of Decoding AI’s small and medium projects with minimal human intervention. The first version was so overbuilt I stopped using it.

Meanwhile, I kept seeing people obsess over the next “_____ engineering” label, instead of focusing on actionable outcomes. Prompt engineering, then context engineering, then harness engineering. So far, so good. But in the last few weeks (July 2026 as I write this), things got off track with loop engineering and graph engineering, which already read more like marketing talk than like anything that solves real problems. Graph engineering overtheorizes how teams have built AI applications since the LangGraph era kicked off in ~2024. Don’t get me wrong. The terms aren’t wrong (Boris Cherny, who leads Claude Code at Anthropic, says, “My job is to write loops”), but we’re overexplaining intuitive things we started doing years ago.

While you’re defining what counts as a loop, you’re not thinking about the processes that actually deliver software.

The right frame is the software factory, which was one of the core themes at AI Engineer World’s Fair 2026, where Tereza Tížková (growth at Factory.ai) defined one as “the whole loop, the whole lifecycle of developing software with autonomy.”

I bet you already have an intuitive sense of what a software factory is. In this article, I want to further formalize it and map it onto the software development lifecycle (SDLC). We’ll explore how big your software factory should be, and when to stop automating before it adds more friction than value. Most importantly, I want to highlight where the human belongs in this process, and where I believe they’ll still belong even in a world where all the code is generated by AI.

So…what’s worth automating? Where does the human bring the most value? What’s worth building, and what’s worth buying?

The design of a software factory

Like a physical factory, a software factory automates software creation with minimal human input. Raw work (bug reports, feature ideas, incidents) goes in. Shipped software comes out. It needs a few highly qualified people making high-leverage decisions, and defined gates that work can’t pass through without them.

Factory.ai pitches “a self-improving system for your Software Development Life Cycle (SDLC).” Addy Osmani frames the stack as loop, harness, factory: “The loop is the atom”; a factory is “an org chart made of loops.” Warp’s CEO, Zach Lloyd, states that “software engineering will become factory engineering.”

The factory is made up of eight stages that can be divided into three buckets:

What to build. Triage/intake classifies, deduplicates, and routes incoming work. Brainstorming finds high-impact features through market analysis, user data, and technical research. Planning, the most important stage, turns that research into a polished plan, refines it by letting the agent grill you, and tracks decisions in an ADR (Architecture Decision Record) log plus a glossary. The outputs of this stage are tickets backed by documentation that a team of agents can implement, which can be tracked in plain files or a project management tool, such as GitHub Issues, Linear, or Notion.

At this stage, the agent plans in read-only mode, going through the code, the AGENTS.md file, and most importantly the context layer.

Actual building and checking. Implementing is a software engineer and QA agent loop that goes through the tasks and supporting documentation. Review checks the PR diff against product, architecture, and code standards. Review-CI runs the test suite, and failures trigger a fixing agent. Release handles CD to staging/production with human deployment checks.

Self-improving. Monitor/incident response feeds production signals (alerts, errors, incidents) back into triage as fresh input on what to build next, closing the loop.

Orthogonal to the eight stages, we have the context layer. The layer is especially important at the front of the line. Brainstorming is limited to the data it sees: user analytics, competitor analysis, research, transcripts, and documentation. At this stage, a poor context layer directly limits the space of possibilities you can explore. It has a similar impact on planning, where transforming the raw idea into technical specs and tasks depends heavily on how good the examples in the context layer are. If you want to implement a new product-recommendation feature, and you have zero examples, the LLM will just predict the most common thing to do, which often isn’t the best solution for your product.

The context layer can take many forms. One strategy that is becoming increasingly popular is the LLM Wiki, a term coined by Andrej Karpathy. It’s basically a strategy to transform your data into a structured knowledge base, just by using files, instead of a database. Factory, via its AutoWiki feature, transforms popular codebases into a structured knowledge base that agents can query instead of parsing the codebase itself. LangChain recently released OpenWiki, a CLI tool to manage wikis for agent memory. If you’re curious, in this article I detail how I turn my data from Obsidian, Readwise, and Google Drive into agent memory via LLM wikis.

Where the human belongs

To see where the human belongs, let’s walk through the factory with an end-to-end example. We’ll build a feature for a shopping-assistant agent on an ecommerce platform similar to Amazon’s. The scenario is that usage data says users aren’t engaging with its recommendations, and we have to ship an improvement.

Brainstorm is where taste lives. Agents do the grunt work: They analyze user activity, scan competitors’ assistants, and pull research into the knowledge base. Then a member of the technical staff starts looking at the data, understands why people are not engaging with the recommendations, explores how the competition implements their solution, and proposes a fix as a feature spec. At this stage, the spec solves a business problem. It doesn’t need to prescribe a technical solution yet.

Plan is where a human, with the help of the software factory, transforms the feature spec into an implementation plan. Let’s assume that we want to make a change to the recommender engine algorithm. The human chats with the knowledge base, figures out whether it’s feasible, and thinks through architecture, interfaces, data flow, cost, and latency. They then let the agent scan the codebase and grill them until the plan is properly refined into something that fits the codebase. The output is a bunch of tickets plus an ADR explaining the algorithm change and an update to the glossary.

The agent can help within these two stages by quickly scanning through a ton of data and improving the plan, but the human is still central.

Use the strongest model (Fable) for brainstorming and planning. These stages burn fewer tokens than implementation itself, but everything downstream depends on them. A well-written plan lets cheaper models (Opus, Sonnet) execute without reasoning their way out of dead ends. A weak plan makes them retry until the extra tokens erase the price gap.

With a weak plan, I’ve watched Sonnet on high reasoning out-cost Opus on the same task: The smaller model needs more attempts to reach the same goal. Total cost is tokens × price, not model tier. So more failures equals more reasoning, more tokens, and more cost.

From here on, we move into “loop” and “graph” engineering territory.

Implement runs a software engineer agent that picks up every ticket that’s ready to go. As the loop is scoped to a feature, it takes only the associated tickets. After each ticket is implemented, a QA agent tries to find bugs by stress-testing the application. As agents tend to have a positive bias towards their own work, the split between the software engineer and QA agents matters. As Addy Osmani puts it, the model that wrote the code is “way too nice grading its own homework.” This loop, at solo scale, can be as simple as a bunch of terminals pulling tickets. At larger scale, it runs on remote agents working 24/7.

The loop only works if agents can interact with the app. The QA agent needs one command that starts the whole stack reproducibly. From there, it drives the app in the browser, calls the data or fine-tuning pipelines, or hits your server’s API. Whatever your app’s interface is, the agent needs access to it, the same way a human user would.

The key idea is to integrate feedback loops as natively as possible into your software factory. Ideally, you want multiple levels, depending on how expensive it is to run them: linting, unit tests, integration tests, and end-to-end tests. When the loop keeps failing, the root cause is almost always missing plumbing, not the agents.

Review has three steps. Step one checks the product and architecture requirements against the ticket and the ADR. Any discrepancy becomes a new ticket passed back to the implement loop. Step two ensures code quality (modularity, naming) and guards against AI slop such as verbose comments or cryptic function names. Step three looks at the CI/CD pipeline. At every step, any failure auto-creates a task picked up by the software agent.

Not every project needs all three steps. The “factory” ends with a PR that you as a human need to review and merge. But in reality, if you spend enough time creating a strong plan, the PR that reaches you is usually ready to ship as-is.

So where does the human belong? You’re indispensable during brainstorming and planning, and you return for the final check. Agents own everything in between. OpenAI took this to the extreme: ~1M lines and ~1,500 merged PRs over five months with zero hand-written lines. Their framing is “Humans steer. Agents execute.”

Don’t overbuild the factory

With my first Squid version (my own software factory), I got greedy and chased full autonomy: big remote workflows, parallel agents, and one grand pipeline running end to end. It worked, until something went offscript. Which it usually does. I couldn’t debug it, couldn’t halt it mid-run, and couldn’t redirect it without throwing the run away. It was a big monolith that took me too far out of the loop, and I couldn’t control it.

I realized you need two options. The first is granular commands that let you grill your plan, implement a specific task, or review one particular step. The second, for when you’re comfortable giving the agent 24/7 autonomy, is an end-to-end command that chains all the smaller ones into a fully autonomous graph, such as one big /plan and /implement-review-all command. Basically, each step is a “loop,” while the whole pipeline is the “graph” of your software factory. Still, note how planning and the rest are split into two different commands, as planning is, and always will be, human-driven (at least if you want the result to stay aligned with what you actually want).

Bottom line. You need to be able to step in, halt it, redirect it, and interrupt it, while still having the option to go fully autonomous.

The bottleneck is me, and that’s by design. To be honest, I’ve worked mostly solo since the AI coding agent boom, and I don’t understand who the people shipping 100 features in parallel are. Most of my features (per project) build on one another, which makes them impossible to parallelize. As the project grows, you can find more and more independent features that can be implemented in parallel, but I still believe that number is limited.

That’s why, when I parallelize, I only use local agents, each running in an isolated codebase via worktrees. So far, I’ve never felt the need for 24/7 remote agents, or wanted the overhead of managing them.

A big team can justify more automation, but it has to earn it. So as with any other software product, start small, start by automating the most time-consuming bottlenecks, and add complexity gradually as people get comfortable with the system. Don’t be like me, with my Squid experiment.

Build vs. buy

In all scenarios, you’ll start with a prebuilt coding harness. The most popular vendor-locked ones are Claude Code and Codex. Or go open source with OpenCode or Pi, which took off thanks to its minimalist, extensible architecture that lets you easily build on top of it.

But picking a harness isn’t the same as knowing how to configure it and wire it into your software factory. That’s why everyone needs to know, at least intuitively, how a coding agent works under the hood: the agent loop running in your terminal, what changes when it runs remotely, how you evaluate it, and which context engineering strategies keep it cheap without making it dumber. If you want to learn more about building a coding agent from scratch, consider exploring my open source course on GitHub. Even if you never plan to build your own harness, that intuition is what lets you become a power user.

For a small team, you’ll get extremely far just by defining a set of skills and agents that encode your process on top of the coding harness (a.k.a. your software factory). To keep it simple, this is what I did with Squid, which I use to implement all my projects.

There are other off-the-shelf “software factories” powered just by skills and agents defined in .md files, such as Matt Pocock’s skills repository or the BMad method.

But remember that the factory is mostly about processes, not tools: A factory that doesn’t fit how your team already works adds friction, never gets adopted, and ends up useless.

You cross the buy line the moment engineers you don’t personally supervise run agents. Observability, tracing, cost tracking, and pay-per-token billing stop being optional and become someone’s full-time job. Agent swarms wired into Linear, Slack, and CI across distributed infrastructure are a logistical hell that isn’t your product. That’s when it makes sense to look into off-the-shelf solutions such as Factory.ai (that comes with the Droid agent), or Warp’s Oz. In Warp CEO Zach Lloyd’s words, “Most of the factory is not necessarily a new interface. It is an integration into people’s existing workflows.”

At the other end of the spectrum, you cross back to building when the platform’s constraints cost more than the team it would take to replace it, as OpenAI’s report on its Codex-built product shows.

The smallest builds, the middle buys, and the largest builds again.

What’s next

Someone is already coining next quarter’s “_____ engineering” term as we speak. But the software engineering processes you use to output real code won’t change that often. That’s why you should be open-minded but at the same time focus on actionable outcomes, not on overthinking how to label things.

As Zach Lloyd suggests: Find one “annoying part of your job” and build the smallest loop that handles it.

The harsh reality is that software factories are just at the beginning. They’re far from perfect, and especially far from being fully “autonomous.” Usually, when someone claims they’ve cracked the software factory problem, they either haven’t tested the idea enough or are trying to sell it to you. I’m confident we’ll reach the point where almost the entire software development lifecycle is automated (with the exception of brainstorming and planning), but at the moment we’re still figuring things out.

But here’s what I’m wondering:

Which stage of your factory still needs you the most? I keep automating mine, and the bottleneck stubbornly stays at planning.

Explore next

1. Osmani, A. (2025). “Loop Engineering.” X.
   https://x.com/addyosmani/status/2064127981161959567
2. MacManus, R. (2026). “AIEWF Daily Dispatch: Loops, Software Factories & Forward Deployed Engineers.” Latent Space.
   https://www.latent.space/p/aiewf-daily-dispatch-loops
3. Factory.ai. (n.d.). Agent-Native Software Development Platform. https://factory.ai
4. Osmani, A. (2025). “Software Factories, Light and Dark.” X.
   https://x.com/addyosmani/status/2079442194449232227
5. Karpathy, A. (n.d.). LLM-Wiki. GitHub.
   https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
6. Abboud, M. (n.d.). “How Coding Agents Actually Work: Inside OpenCode.”
   https://cefboud.com/posts/coding-agents-internals-opencode-deepdive/
7. Kapoor, S. (n.d.). “Building and Evaluating AI Agents.” AI Engineer.
   https://youtube.com/watch?v=d5EltXhbcfA
8. OpenAI. (n.d.). “Harness Engineering: Leveraging Codex in an Agent-First World.”
   https://openai.com/index/harness-engineering/
9. Parsons, C. (n.d.). “Ralph Loops: Build Dumb AI Loops That Ship.” AI Engineer.
   https://www.youtube.com/watch?v=2TLXsxkz0zI
10. Pocock, M. (n.d.). “Software Fundamentals Matter More Than Ever.” AI Engineer.
    https://www.youtube.com/watch?v=v4F1gFy-hqg
11. MacManus, R. (2026). “Warp CEO Zach Lloyd on Why Software Factories Are the Next Phase of Coding.” Latent Space.
    https://www.latent.space/p/software-factories
12. Iusztin, P. (2026). “Building a Coding Agent From Scratch: Harness Architecture.” Decoding AI.
    https://www.decodingai.com/p/building-a-coding-agent-from-scratch-system-design
13. Iusztin, P. (2026). Building a Coding Agent from Scratch Course. GitHub.
    https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course
14. Iusztin, P., & Bouchard, L.-F. (2026). “LLM Wikis as Living Memory for AI Agents.” Decoding AI.
    https://www.decodingai.com/p/llm-wiki-agent-memory

Join 44,000+ engineers eager to learn how to build their own software factories by subscribing to Decoding AI Magazine!

How to build a content creation agent

1. Give the system the context it needs to generate relevant content. Max started with a small knowledge base containing information about himself and his business, details about his audience, sample scripts, and notes on his writing voice. These files gave Claude Code reference material for evaluating topics and generating scripts that reflected his brand and style.
2. Explain the process in plain English and let AI take it from there. As Max pointed out, “Vibe coding is describing what you want clearly and let[ting] the model handle the entire building process for you.” However, you’re still ultimately responsible for what’s built. Claude Code may have generated the implementation, but Max set the context, approved permissions, reviewed results, and answered clarifying questions along the way.
3. Build the individual stages of your pipeline. Max wrote a list of prompts to take Claude Code through the process of researching topics (stage 1), scoring them and selecting the most relevant one (stage 2), then generating three hooks and expanding the strongest option into a full script based on the knowledge base and sample content (stage 3). He ran them through Claude Code and ended up with a working three-part pipeline, with each stage triggered manually.
4. Connect the stages into a single workflow you can run with one command. Max had Claude Code combine research, scoring, hook generation, and script writing into a unified content engine that produces a structured JSON file he uses in the next stage. “This is the actual moment that it stops being a few separate prompts and starts becoming an agent,” Max says.
5. Add a browser-based interface. Max built a local dashboard that displays the generated topics, scores, hooks, and scripts. This lets him inspect the results and start new runs from a browser instead of returning to the terminal each time. Here’s a tip from Max on creating a dashboard you’ll actually want to use as your mission control: Be as creative as you want, but “make it look like something designed on purpose, not a default template.”
6. Test the workflow and extend it. The first version Max created had two limitations: It produced scripts only for the top-ranked topic, and starting a new research run still required the terminal. Max solved this by asking Claude Code to generate hooks and scripts for all five topics and add a button that could start a new run from the dashboard. The revised system produced three scripts for each topic, giving him 15 script options per run.

If you’re building something similar, Max recommends starting with a repetitive task you can describe clearly. Build a working version with your preferred coding tool, use it, and then extend it in response to what you learn.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

I don’t especially care about the check. (Okay, that’s not true, nine grand is a lot of money, but hopefully you’ll get my point.) Jenny and I didn’t write those books to get rich, and nobody who’s met a working author would mistake the job for a path to wealth. But one thing I very much care about, and I suspect almost all authors would agree, is getting credit for the work I’ve done. If you ask ChatGPT, Claude, or Gemini (or especially Google AI overviews) a question that it partly learned to answer from one of my books, I would love for it to be able to say so. Something like “Some of this comes from Andrew Stellman’s work, and if you want to go deeper, that’s where to look.” In other words, what I’m asking for is a hat tip.

All of this sits inside a much bigger question, one that usually goes by the name attribution. In the broadest terms, attribution means knowing where a piece of work or knowledge came from, and being able to trace it back to the person who made it. That might sound like a narrow, technical matter, but right now it’s one of the biggest live questions in AI. At Foo Camp recently, it came up constantly, quite possibly the most talked-about topic of the whole weekend, and everyone there seemed to have something to say about it. It’s also a tangled subject, part technical and part legal, and more than either of those, deeply emotional. I want to take a little time to pull those threads apart.

Why so many people are angry

Before any of the technical or legal questions, there’s a simpler reason attribution is such a live wire right now: People are angry, and to be perfectly honest, they have good reason to be. Artists have watched models learn to imitate their style from work that got scraped without anyone asking. Writers have found their books sitting in training sets we never agreed to (even though that’s not the reason Anthropic has to pay us). A lot of people are simply afraid that the work they do for a living is about to be done by a machine that learned part of the job from them. Their work got taken, which may be fair use but almost never included asking the author for permission, and often without any way to even find out it happened. And that for many of us feels really uncomfortable.

A lot of that anger ends up funneled into legalistic arguments about copyright, which is notoriously misunderstood and misapplied. People reach for it for understandable reasons. It feels tangible in a way the rest of this doesn’t, and while many of us feel like it was written to protect authors and artists and creators, it’s actually much more nuanced, especially when it comes to fair use. But I don’t think copyright is the right move for anyone who actually wants attribution, and I’ll go further: I think creators will get a lot more mileage working with the AI labs than fighting them in court.

Underneath all of it, what most of us want is simple enough: We want the work to still be ours, and we want that to be visible. That’s an attribution problem. And before anyone can argue about whether AI can solve it, it helps to be clear about what attribution even is, because it’s a slipperier word than it looks.

What attribution actually is

At its simplest, attribution is the link between something and where it came from. When you read a fact in a book, there’s a chain running from that sentence back to the author, and back again to whatever they drew on. That chain is how credit works, and how trust works, and how you know what to go read next when something grabs you. For as long as people have made things, you could usually follow it back to a person.

You see attribution everywhere once you start noticing it. Bibliographies and references are attribution. So is every footnote on a Wikipedia page, and so is every time one news story credits another, the way you’ll read that “Business Insider is reporting” something before another site passes it along (that is, when we remember to do it). For authors it runs deeper than that: Attribution underpins almost everything we do, and the whole system of academic publishing rests on it, because the entire point is to show exactly whose work each new piece is built on.

AI is the first technology that takes in essentially all of it, every book and article and repository it can reach, and hands back answers with the chain erased. And while that may be technically allowed under fair use, it shouldn’t be the end of the discussion. The knowledge comes out; the line back to whoever produced it does not. That erasure, underneath the lawsuits, is what people are really reacting to. The model learned from their work and gives no sign of it, and the trail that used to lead back to them is gone. Attribution is just the name for putting that thread back.

The catch is that people use the word for at least three pretty different things. The cheap version, the one people reach for first, is resemblance: Does the answer look like my work? That’s easy to check and mostly beside the point, because two people can write nearly the same sentence about a for-loop without either one copying the other. What actual products ship is citation, the little source links under a chatbot’s answer, which credit the page the system fetched while it was answering, not the books it learned from years earlier. Those links are the result of retrieval, not memory, where the system went out and fetched a live page mid-answer. There aren’t any technical challenges in adding an attribution for those live retrieval results, and the fact that the AI labs are fine with that attribution gives them a template to follow if model-based attribution becomes technically feasible.

We’ve got a really good real-world example of how this can work. O’Reilly’s learning platform has an AI engine that answers questions out of the books on the platform, tells you which ones it drew on, and pays the authors and publishers behind them. It works because the corpus is small and everything in it is licensed, which is precisely the condition the frontier models don’t have. Because the version that actually matters is the third one, causal: Did my work shape the part of the model that produced this answer? That’s the honest meaning of the word, and by far the hardest to compute.

Notice that none of that is about money. Paying me would mean working out what a given answer is worth and what share of it is mine, which is a hard allocation problem nobody has really solved (and, to be honest, probably works out to a tiny fraction of a cent in any given answer). Attribution asks a smaller question: Did this come from my work, or didn’t it? A hat tip is only that identification, not necessarily a required payment, and it’s still the thing everyone tells you is impossible.

Trying it on a tiny model

I may be an author, but I’m a developer too, and when someone tells us something is impossible, our first urge is always to build that thing. A lot of the time “impossible” just means nobody has worked out how yet, and every so often it means it would be inconvenient for someone if they did. In this case, some really smart people actually have worked out how, at least some important parts of it, and it’s worth understanding both why the problem is hard and how they’re tackling it.

Start with why it’s hard. When a model trains on your book, it doesn’t file the book away somewhere you can later point to, because models are not databases of books and other source material. What it learns gets spread across billions of numbers, tangled up with everything else it ever read, and no single weight says, “This part came from Stellman.” There’s no receipt anywhere that records which source contributed what. The obvious way to check whether your book mattered would be to pull it out, train the model over again, and see what changed, but nobody is going to retrain a frontier model from scratch a billion times. The upshot is that the knowledge is still in there, it’s just smeared across the whole model with no index back to where it came from, and that’s why a lot of people have called attribution impossible.

It may not be quite as impossible as it seems from that particular approach, though.

Researchers have been chipping away at exactly this, and the approaches run from cheap and rough to expensive and precise. At the precise end is leave-one-out: remove a source, retrain, and measure how far the answer falls. It’s about as close to ground truth as you can get, and hopeless at scale for the reason I just gave. The interesting work is on cheaper stand-ins that approximate that result without all that retraining. The one I find most compelling is TracIn, described by four Google researchers in 2020 in “Estimating Training Data Influence by Tracing Gradient Descent”: As the model trains, it saves snapshots of itself, and TracIn measures how much each training example pushed the model toward a given answer by comparing gradients at those snapshots, with no retraining required.

It belongs to a broader family. Influence functions, which Pang Wei Koh and Percy Liang introduced in 2017 in “Understanding Black-Box Predictions via Influence Functions,” are the older, heavier cousin, and Anthropic scaled them up to models with tens of billions of parameters in a 2023 paper, “Studying Large Language Model Generalization with Influence Functions.” A newer method, TRAK from MIT’s Madry Lab, takes on a weakness they share, where near-duplicate sources muddy the accounting. None of it is settled, and TracIn in particular is contested at frontier scale, but the direction is real and the people pushing it have far bigger budgets than mine.

So I wanted to see it work with my own eyes. I asked an AI to build me the smallest real language model that could still run one of these methods, and about 20 minutes later I had a working one, which I called tiny-provenance: a language model small enough, around 37,000 parameters, that I can retrain it from scratch in a few seconds. That size is the whole point, because it let me run the expensive leave-one-out check for real, as ground truth, and set the cheap TracIn approximation next to it to see whether they agreed.

They did. I gave it a trick question, “Who designed the Analytical Engine?” with a look-alike line about Babbage and the Difference Engine sitting right next to the correct one in the training data. The cheap resemblance check, the kind of thing real products lean on, took the bait and confidently credited the wrong line. Leave-one-out and TracIn both ignored the look-alike and pointed straight at the source the model actually used. The lazy method everyone reaches for was confident and wrong, the honest ones were right, and the whole thing ran in about two minutes on a laptop.

I proved it at a size where I can retrain the model at will, which is exactly what nobody can do at frontier scale, so I won’t pretend I showed it holds up there. Attribution works best at the extremes, where a model has nearly memorized a passage or leaned hard on a single source, and it stays hard in the muddy middle where an almost unimaginable number of books may each have added a tiny little bit to the model. The fair summary is that attribution isn’t impossible. It’s just currently expensive, and more importantly, really impractical with today’s technology.

The whole repo is public, and since this is an article about showing where things came from, it seemed only right to be transparent about where the demo came from too. The README walks through exactly how I built it, prompts and all.

The legal hurdle

I’ll put my own cards on the table: I think copyright is the wrong tool for the fight we’re having about AI. My first job out of college was at EMI Records, working on the system that tracked their music contracts, so I learned early how tangled copyright and trademark and mechanical royalties and the rest of it can get.

Here’s the analogy I keep coming back to. If a student copies a paragraph out of Wikipedia word for word, that’s plagiarism. If the student reads it, understands it, and rewrites it in their own words, usually it isn’t, though that depends on what got rewritten, because plagiarism is about the ideas and not only the words. A physics student writing “F=ma” in a paper isn’t plagiarizing, even if they copied the formula straight out of a textbook. On the other hand, paraphrasing a specific result from somebody’s research paper without saying where it came from is still plagiarism, even if you technically changed the words. And it gets even more complicated. If you copy text word for word but credit the source, you’re not plagiarizing, but you could still have a copyright problem. We throw around the term “fair use” a lot, but that’s actually a very thorny legal area. More importantly, reducing these ideas to a legal argument doesn’t really get to the core of the problem, because laws are often imperfect and dictated by decades of (sometimes conflicting) precedent, so what’s legal doesn’t always match up with what’s the right thing to do. What a model does is a step further from copying than the rewrite: It turns the text into an enormous pile of numbers that stand in for the concepts, and there’s no paragraph left anywhere to find. So far, the courts that have looked at this, including the one behind the Anthropic settlement, have called that use “exceedingly transformative” (those are the Anthropic judge’s words, and lawyers on both sides recognized that as quite a statement). What they mean is that it changes the work into something new enough, and for a different enough purpose, that it isn’t simply a copy of the original. I think they basically got it right.

Basically, I’m not getting a check from Anthropic because they used my work without permission; I’m getting a check because they literally used a stolen book downloaded from a pirated website, instead of paying O’Reilly for a copy or getting it from another legitimate source.

But I don’t get to wave copyright away entirely, because there’s a real case on the other side, and it’s one I feel personally. Someone who wants to learn C# or study for a project management exam can now ask an AI and get an answer that came partly from books I wrote, without ever buying the book. The model learned from my books (and many others on the same topics I write about), and now it competes with them, in the exact market they were written for, using what it took from them to do it.

The worry that AI tools compete with the very material they trained on isn’t theoretical. Stack Overflow, the question-and-answer site where a generation of programmers went for help, has lost roughly three-quarters of its question volume since ChatGPT launched, as developers ask the AI instead. And the AI answers them well in part because it trained on Stack Overflow’s answers in the first place. And even that whole issue is highly nuanced, especially since the material on Stack Overflow itself is written by its users and released to the public under a Creative Commons license.

That goes to the center of what copyright is meant to protect. One of the things courts weigh in a fair-use case is the effect on the market for the original, and a free substitute trained on the original is about as direct a market effect as there is. A judge has already pointed right at it. In the case a group of authors including Sarah Silverman brought against Meta, the court handed Meta a win on fair use. But the judge then went out of his way to hint that the the authors may have lost because they’d argued it wrong, and that this market-dilution theory, the flood of cheap substitutes, is exactly the one that could win on a better record. The judge also said there was “no serious question” that the use of the technology was “highly transformative” in his ruling, and again, lawyers consider that very strong language. I’m not a lawyer, and this is only how the case reads to me as a layman, but I think it’s the strongest argument the other side has. Between that and the piracy, the bigger question is a long way from settled.

This is where it comes back to the hat tip, and the solution I’d like to see for all of these complex, nuanced problems. In a calm world, credit would be a simple courtesy. But we aren’t in a calm world. Copyright sets statutory damages of up to $150,000 per work for willful infringement, and a lot of lawyers see that number and essentially see a bounty, and their whole case turns on escalating litigation and trying to increase a damage award (of which they receive a substantial cut) by trying to sweep in as many works as they possibly can. In other words, they have a huge financial incentive to show that one of the AI labs creating models knew whose work it was using, knew that it was infringing, and did it anyway. Now look at what a hat tip is. The moment an AI lab prints “This answer draws on Andrew Stellman’s book,” it has stated, in writing, “We knew we were using Andrew Stellman’s book,” and a good plaintiff’s attorney could easily turn that around and make it exhibit A. Would a judge see that as willful infringement? I have no idea. But I’m not sure I would bet the entire future of a company like Anthropic or OpenAI.

If credit that would cost a lab almost nothing to give carries a legal risk that dwarfs the cost of crediting no one, then the rational move, the one any lawyer would advise, is to say nothing and admit nothing. The threat of a potential lawsuit could be enough to convince a lab that they can’t safely open the conversation about voluntary attribution at all, because anything it offers in good faith can be turned into ammunition. That’s the worst outcome for everybody, authors included, because the one technology that might be able to finally tell you where an idea came from ends up legally better off staying silent about it.

If you build with these tools, or just lean on them all day the way I do, this is a key reason the answers you get will almost never tell you where they came from, even if the technology to trace them keeps getting better. The people who build these tools could add that little “Here’s who this came from” line tomorrow. What likely stops them is a legal system that makes giving it to you too dangerous to try.

What I actually want

There’s a smaller version of the hat tip that doesn’t run into any of the legal or technical issues I’ve been talking about. Even when a model can’t say which book an answer came from, when it honestly can’t be traced, it can still point you to the authoritative sources on the subject, the ones worth reading to go deeper. That isn’t attribution in the strict sense; nobody has proven those books shaped that answer. But a model trained on a topic was almost certainly trained on the standard works for it, so the correlation is strong. For the reader it does the useful thing anyway by saying where to go next, and it’s something the AI labs can do today.

Which brings me back to that check from Anthropic I may or may not be about to cash. I meant it when I said I’d rather have the credit. What my writing has done for my career matters more to me than what it’s done for my bank account, and I would happily cut that settlement check in half if it meant twice as many people found their way to the work. A hat tip does exactly that, and it’s what I actually want, if it opens up new exposure to my work and new opportunities for authors like me.

The part I keep turning over is that the hat tip is more possible than the people who say “impossible” want to admit. I made a toy version, and researchers at AI labs are pushing these ideas a great deal further than I can. A key part of what stands in the way now is a legal question: whether good faith can survive contact with $150,000 a work. I don’t know how that one comes out. But I’m fairly sure it isn’t the math, and the math was supposed to be the hard part.