While the headlines have been shouting about AI killing humanity, doomsday AI is more likely a smokescreen to divert attention away from real harms that are happening now. We don’t need panic. We need an assessment of where we are: What are the threats? Where are the skills gaps? Where is the budget? It’s hard to overstate the importance of these questions; Tomasz Tunguz, writing about the Theory 2026 AI Survey, reports that security is the biggest barrier to AI adoption. To start answering these questions, in September we ran a survey that asked infosec professionals about the intersection between artificial intelligence and security. 351 respondents completed the full survey, 209 of whom self-identified as being involved with security as part of their job. This report focuses on those 209 respondents.

Attacks, risks, and responses

52% of respondents said their work rarely (34%) or never (18%) involves AI-generated attacks. This seems unlikely. The most likely explanation is that the security profession includes a broad range of job responsibilities; many security professionals don’t deal directly with defense. That group includes auditors, security experts on software development teams, strategists, architects, and others. It’s also possible that most phishes are either blocked by an automated filter (itself an instance of AI) or deleted immediately by the recipient; many companies require their staff to take training courses that cover topics like recognizing phishes. Still, it’s surprising. Much simpler versions of AI have been used to generate spam and phishing email for years. AI’s ability to generate phishes was demonstrated at BlackHat in 2021 with GPT-3. It’s been part of the security picture for a long time.

The most commonly observed attacks are AI-generated phishing (55%), vulnerability scanning and exploitation (30%), and AI-assisted malware generation (26%). Respondents who frequently encountered AI attacks also reported more types of attacks. There’s no surprise here: If you see attacks more frequently, you are more likely to observe specific attacks. The strongest relationship was with AI-generated phishing, which was seen by 55% of all respondents and 86% of those who observed AI-generated attacks weekly. Among respondents who observed attacks daily, slightly fewer (78%) reported phishing. Given the small sample, the difference may not be meaningful. It’s also likely that respondents who observe attacks daily are seeing a much broader range of attacks and that phishing is genuinely less important to them.

Respondents reported seeing deepfake audio and video, which can be used for vishing and other impersonation attacks, less often than phishing, though it was far from nonexistent (25%). More respondents than we expected reported fully automated attacks, like OpenAI’s attack on Hugging Face, though the percentage was still low (16%). The percentage of respondents who observed adversarial inputs against models including prompt injection was similarly low (only 13%), but that’s high enough to indicate that prompt injection and other attacks directed against models are a real concern.

When we asked respondents what AI-related risks concerned them most, we got a different set of answers. The two most common concerns were overreliance on AI tools without human verification and leaking sensitive data via AI tools; 49% of respondents selected each answer. (The number of respondents selecting those answers happened to be the same, though there was little relationship between the two groups.) 

Although the quality of AI output has gotten a lot better in the past year, relying on unverified AI outputs remains problematic. However, AI can easily generate more output than humans are capable of reviewing, and software teams are now overwhelmed by the volume of AI-generated pull requests that need human review before they can be merged. This overload is causing many to rethink the review process and others to reject AI-generated PRs. Code review was never very good at finding bugs; it was better at familiarizing a team with developments in the codebase. It may be enough to rely on exhaustive testing.

Understanding how your software works when much of it is written by AI is a deeper problem. While AI is good at answering questions about a codebase, it usually doesn’t have access to the reasoning behind the code, which is often hidden in institutional knowledge—and institutional knowledge often means “Oh, yeah, that guy who retired three years ago.” In addition, software developers may lose their programming skills if writing code becomes a smaller part of their job. Security professionals themselves might wonder whether relying on AI for red-teaming, log analysis, and vulnerability detection leads to a degradation in their own skills. 

Companies also fear losing sensitive data via AI. Most AI providers offer enterprise accounts that promise that they won’t train models on data in prompts or attached documents, but it’s not clear that these terms of service can be trusted. The frontier laboratories are getting increasingly desperate for training data, even going to lengths such as buying failing startups for their data. Read OpenAI’s description of its Zero Data Retention and Private Safety Processing cynically, and ask if this really protects corporate data or if it allows using data in ways that aren’t directly observable by humans—for example, automatically generating synthetic data. Security professionals are paid to be cynical; thinking about how software, protocols, and policies can be abused is part of the job.

Roughly a third (35%) of the respondents are concerned about attackers using AI to discover vulnerabilities faster than they can mitigate those vulnerabilities and apply patches. Updating software is usually a good practice, but it comes with its own risks: Software updates may introduce new bugs and can lead to incompatibilities with established tools and practices. And defenders can’t yet delegate vulnerability detection, patching, and updating to AI. So overall, is AI more of a help to defenders or attackers? Andrew Ng, Bruce Schneier, and others have argued that the attackers are ahead right now. It’s easier to find a vulnerability, use it to create a payload, and launch an attack than it is to stop one. But the balance will eventually shift to the defenders, as AI becomes more capable of finding and patching vulnerabilities before they can become a problem.

It’s surprising that only a quarter (25%) of the respondents listed AI-generated phishing as a top concern, especially since phishing is the attack most frequently seen in the wild. Perhaps we’ve become too used to phishing; we’ve all been to the security trainings, we’ve all experienced it directly, and some of us have even fallen victim. Phishing is also an attack that, in many cases, is handled automatically. Familiarity may not always breed contempt, but something you see almost daily usually isn’t your biggest worry. 

More exotic attacks are even less of a concern. Only 11% listed prompt injection as one of their top concerns, for instance. However, real-world examples of prompt injection are appearing with increasing frequency. A recent study observed that roughly 1% of résumés contained hidden text to trick an AI system. Google has noticed many kinds of prompt injection, ranging from benign to overtly hostile. And in 2025, OWASP recognized prompt injection as the greatest security risk for users of AI. It’s surprising and unfortunate that it doesn’t appear to be a greater concern for security professionals, especially since prompt injection is a vulnerability that might be lessened by careful filtering but can’t be eliminated.

Even fewer security professionals are concerned about data poisoning (4%). Data poisoning is a serious problem for organizations that train their own models, but today frontier labs conduct much of the most resource-intensive model training. However, this is changing as post-training on corporate data becomes more common. We believe the trend toward locally specialized models will continue, and that data poisoning, as distinct from data theft, will become more of a concern. 

We asked specifically about the use of agents, which exploded this year with the launch of OpenClaw and Gas Town in January 2026. Agents create another set of security concerns. We’ve all heard horror stories: agents that deleted production databases, handled data improperly, generated email that should never have been sent, and the like. Using an agent properly requires a lot of attention to detail, and many users aren’t used to that responsibility, nor do they understand security well enough to exercise responsibility.

The biggest concern among our respondents is excessive permissions or unclear ownership (28%). That’s hardly a surprise; many security problems trace back to permissions. Unclear ownership makes permissions issues more dangerous. When an agent is making many calls to a model, using tools, and delegating to other agents and models, it quickly becomes unclear who (or what) is the authority responsible for any action. This situation isn’t handled well by our current authentication and authorization systems, which assume that identities and authorizations are long-term, stable entities.

Lack of monitoring or auditability was the second highest concern (19%), followed by inability to verify whether the agent stayed within its role (18%). These concerns are closely related: Ownership is often unclear when agents are in use, and lack of monitoring makes the situation worse. Likewise, when you can’t monitor, it’s difficult, if not impossible, to verify whether the agents stayed in their lane—and we’ve seen all too many examples recently of agents that abandoned their authorized roles in inappropriate ways.

Data leakage appears again, though it’s surprisingly low on the list (15%) given that 49% of the respondents indicated that data leakage was one of their top concerns. It’s possible that we’re seeing the difference between single and multiple selection questions; many respondents are concerned about data leakage, but it’s not their top concern. 

Prompt injection doesn’t rank highly for either question: 9% selected it as their top concern about AI agents, and 11% selected it as one of their top concerns. 

Policies and funding

Although nearly three-quarters of the respondents reported that their companies have an AI use policy in place (72%), less than half (44%) reported that the security policy is “comprehensive,” while 28% said that the policy is basic and needs updating. Another 9% said that their employer doesn’t have a policy but is actively developing one. 19% said that there was no policy, or that they “didn’t know,” an answer we consider equivalent to “there is no policy.”

Organizations need clear AI policies that tell employees what AI tools they can use and how they should (and shouldn’t) use them. It’s important for employees to understand whether their company is sensitive to sending data offsite, whether the company’s chosen AI providers guarantee their data won’t be used for training, which practices for attribution are acceptable, and much more. We have some sympathy with organizations that have a basic policy that needs updating. AI’s capabilities are advancing at unprecedented speed; a policy that was adequate in 2023 certainly needs updating now, and a policy that’s only a year old still predates the widespread use of agents like OpenClaw, Claude Code, and Codex, along with the widespread use of skills and tools to extend an agent’s capabilities. Keeping a policy up-to-date with the current state of AI is an imposing task.

The existence of a policy appears to be related to whether employees use AI safely. We asked about “shadow AI” (the use of AI outside of any definitive policy) and found that 36% of the respondents thought that their teammates occasionally use “shadow AI” and 23% thought that the practice is widespread. About a quarter (24%) thought that shadow AI use is rare; and 16% were unsure. 

The relationship between shadow AI and the existence of an AI policy tells us a lot about whether policies are effective. Respondents with a comprehensive AI policy were more likely to say that shadow AI use is rare (37%) or that it’s only used occasionally (36%); only 17% said that shadow AI is widespread at their company. In contrast, 35% of the respondents who reported a basic policy said that shadow AI is widespread—almost the same percentage as those who reported that there was no policy at all (37%) and only slightly worse than those who said that a policy is in development but doesn’t exist yet (28%). Those who had a basic policy, a policy under development, or no policy were less likely to report that shadow AI was used “rarely or never” (13%, 6%, and 21%, respectively).

Granted that shadow AI is probably underreported across the board, our data clearly shows that a comprehensive AI policy is a powerful tool in limiting it. If the policy is poor, or it doesn’t exist, employees will use whatever tools they can to get their job done, which leads to AI use that’s inconsistent, insecure, and possibly counterproductive; it may even lead to legal, contractual, and reputational risks. When there’s no policy, it’s also likely that AI isn’t used as widely as it could be. Telling employees what is expected helps them to act responsibly. A good AI policy doesn’t just tell staff what not to do; it also communicates how to use AI in ways that are appropriate, and may even provide for training. 

When we asked whether organizations are adequately funding AI security, the picture was better than we expected. The largest group (32%) said that AI security is sufficiently funded, and 28% said it’s partially funded. Only 16% said it is severely underfunded, and 23% are unsure. I admit to being surprised by this result. Security is often underfunded, partly because a successful year is one in which “nothing bad happened,” and “we need to spend more so that nothing bad will continue to happen” isn’t a story that plays well at budget time. But executives have been catching on over the last few decades; high-profile attacks and data thefts lead to CIOs and CISOs losing their jobs, and that’s enough to get anyone’s attention. 

We saw a strong link between a comprehensive AI policy and adequate funding for security. Nearly half (49%) of the respondents who said their company has a comprehensive AI policy also said that AI security is sufficiently funded, and another 29% said that it’s partially funded. Only 3% of this group said that AI security is severely underfunded. That means nearly 80% (78%) are at least partially satisfied with funding.

Once again, the answers given by respondents who said that their organizations have a basic policy, a policy under development, or no policy were surprisingly similar. 16%, 22%, and 21%, respectively, said that AI security is sufficiently funded. That’s certainly not a vote of confidence, though we might be able to draw two more bits of information from it. Developing a policy looks like it’s slightly better than having an out-of-date policy; the company is putting in the effort and making an investment. That makes some intuitive sense, though it might also be wishful thinking. Among respondents whose organizations have a basic policy, the largest subgroup said that AI security is partially funded (42%). A middle-of-the-road policy is associated with middle-of-the-road funding. Is that fair, or is that what reality looks like?

Tools and skills

While AI gives security professionals powerful new tools, they aren’t yet fully taking advantage of them. Two-thirds of the respondents (68%) reported that they use a chat, copilot, or similar tool for research or documentation. But only slightly over a third use AI to scan for vulnerabilities (38%), just over a quarter use it for threat detection (27%), and a quarter use AI for incident response (25%). Security professionals widely use general-purpose AI. However, those using specialized tools like vulnerability scanners remain the minority.

To probe further, we asked where AI improved respondents’ workflows. More than half said that AI improved their workflows for research or documentation (61%) and that it reduced repetitive work (57%). But only 29% said that AI improved their workflow for triage, and 17% said that it improved detection accuracy. These results confirm what we saw for tool use.

The relationship between tool use and impact shows up even more clearly when we look at who’s benefiting most from AI. Those who are only using chat or a copilot see the greatest improvement in the speed of research or documentation. Chatbots and copilots save time, particularly when writing reports with a lot of boilerplate text, but they aren’t as helpful when actively defending against or recovering from attacks. Without specialized tools, that work has to be manual.

Respondents using security-specific AI tools were more likely to report improvements in alert triage (42% versus 18%) and detection accuracy (27% versus 5%) than those using only general-purpose chatbots or copilots. 

It’s far from surprising that respondents who didn’t use any AI tools were most likely to say that they saw no noticeable positive impact. The bigger question is why they saw any impact at all. 

Finally, we looked at the “skills gap” to determine what skills and tools would make respondents more effective. Only 7% of the respondents said that they don’t feel like they have a skills gap; that group was also the most likely to say that they aren’t using any AI tools. It’s clear that most of our respondents see AI as a necessary part of the future and want to be part of it. 

34% of the respondents want to use AI tools more effectively. That desire was evenly spread across all AI users, both chat/copilot only (36%) and users of security-specific tools (37%). 24% want to learn how to use AI to secure systems (including the use of AI for red-teaming); 19% want to learn AI-specific attack techniques; and 12% want to learn how to detect AI-generated attacks. 

The future of security

We started by claiming that AI has changed computer security forever. In addition to the many reports of research projects that inadvertently attacked other sites, AI has become a tool for attackers. They aren’t distracted by arguments about whether or not AI is taking jobs; their goal is to exploit their victims, and it’s clear that AI will help them. And as Ben Lorica points out in his newsletter, the economics of attack are changing: AI is infinitely patient, can try multiple attack vectors, can abandon failures, and can work at a scale that’s difficult for humans to match.

AI is also a tool for defenders. We saw more security professionals using AI than we expected, though not as many as we had hoped. Many defenders are taking advantage of chatbots and copilots but haven’t yet adopted more specialized tools, which are associated with improvements in workflows for triage and threat detection. 

The tools for defenders undoubtedly need to mature, but tools aren’t the entire answer. Organizations that adopt AI successfully will have appropriate budgets and policies for AI use; they will understand the risks that come with AI adoption; they will proactively fill skills gaps; and they will recognize processes that are no longer productive and “turn scar tissue into infrastructure.” AI is, in the end, a “normal technology” that takes years to penetrate organizations, as Arvind Narayanan and Sayash Kapoor have argued. Tools, budgets, and thought processes all need to change, and that takes time. Getting ahead of the curve on AI won’t happen if individuals work in isolation, adopting tools and ideas as they see fit. It’s an effort that requires the entire organization.

Generative AI makes the handoff more revealing because it lowers the cost of producing a convincing first version. Students can generate code, analysis, prose, visualizations, and prototypes far more quickly than they could a few years ago. In a classroom, however, a project is useful only insofar as it shows what the student understands and can defend. AI can supply missing code, smooth over incomplete reasoning, or turn a rough idea into fluent prose, so polish is no longer strong evidence of understanding by itself.

A concrete example came from a student paper on AI and labor economics, Verification Responsibility in AI-Applicable Occupations. The paper asked what remains economically important when AI makes candidate outputs cheap but a person still has to check those outputs, decide whether they are safe to use, and remain responsible for accepting them.

The lead student developed the conceptual model and most of the analytical workflow. A second student was given a narrower job: rerun the main analyses independently, record the versions of the source data, document how occupations in one dataset were linked to occupations in another, and compare the reported tables and figures with the underlying outputs. What looked like procedural verification quickly became a substantive review of the study.

The first problem was reproducibility. The analysis ran on the original machine, but too much of the workflow depended on choices that lived in one student’s memory. To reproduce it elsewhere, the team had to record the data transformations, write down the rules for linking occupational datasets, and keep intermediate results so another person could see how the final numbers had been produced rather than simply rerunning a script and trusting the output.

The more consequential problem was in the measurement design. An early version of the study used some of the same autonomy and leadership survey items in two different measures: verification responsibility and adaptive capacity. The software had calculated the relationship between those measures correctly, but part of the apparent relationship could have been created by the fact that the two measures shared some of the same inputs. In other words, the code was doing what it had been asked to do; the question was whether the quantities being compared had been defined cleanly enough.

The team rebuilt the measures so they no longer shared survey items, removed ambiguous items, and revised the workflow so source versions, cross-dataset links, exclusions, intermediate files, and reported results could be traced. Later versions added alternative specifications and robustness checks. What began as “please rerun this” ended up forcing a redesign of the study.

The handoff changed how I read student work. A second user does not merely test whether the files have been packaged neatly; they ask the awkward questions the creator may have stopped noticing. Why is a particular variable there? Which conclusion depends on an assumption that was never made explicit? In this project, those questions reached back into the measurement design.

The educational value of the handoff comes from the fact that schoolwork and production software have different ends. Professional developers learn a great deal after software leaves their hands because they have to deliver it, explain it, support it, maintain it, and respond when someone uses it in an unexpected way. Student projects usually disappear when a course ends, and that is fine; the artifact is not the goal. What matters is importing some of the learning that comes from delivery and maintenance: explaining decisions, supporting someone else’s use, and revising the work when private context is no longer available.

At Force Academy, the exact task depends on the project. For a computational model, another student may change a parameter or replace one rule and then ask what should remain stable. For an AI-supported workflow, the job may be to move it to a different model or environment. For a data analysis, the second student may try to reconstruct one result from the recorded source files and transformations. When that transfer fails, I want the original student to decide what the other person actually needed to know and what should be changed in the project, rather than simply standing beside them and explaining it.

I use the word “handles” for the visible points of entry that make that possible. A handle gives another person something concrete to inspect or change without first receiving a private briefing. It might be a clearly named parameter, an explicit assumption, a test that records an earlier failure, a replaceable component, a retained intermediate dataset, or a prompt whose important context is written down instead of buried in a private chat history.

Handles also make disagreement more productive. An exposed assumption can be challenged; a component with a clear boundary can be modified or replaced without rebuilding the whole project; and a numerical result linked to its source data and transformations can be checked rather than accepted on authority.

Outside school, AI systems expose the same weakness. A prompt, workflow, skill, or agent configuration may look portable because it can be copied, while the conditions that make it work remain elsewhere: tool access, file structure, model behavior, private memory, naming conventions, or small decisions the creator never needed to document. Moving the artifact often reveals how much of the working system was never inside the artifact in the first place.

A first successful run is therefore weak evidence. It tells me that a particular combination of person, model, tools, data, and context produced an output once. A stronger test is whether another person can reconstruct what matters, distinguish assumptions from implementation details, and make a change without destroying the logic of the work.

Tim O’Reilly’s idea of an architecture of participation gives me the broader frame. A technical artifact becomes more useful when other people can enter it without negotiating privately with its creator. In software, that usually means interfaces, conventions, documentation, modularity, and replaceable components. In data analysis, the same principle shows up as recorded source versions, visible transformations, retained intermediate results, explicit assumptions, and enough analytical logic for someone else to rerun or challenge the work.

For education, the milestone I care about is whether the student understands the work well enough to make some of that understanding available to someone else. A second student who can use, question, and extend the project gives me evidence that a polished first run cannot.

AI expands what one person can build, including what a teenager can attempt without a large technical team. The expanded scope is a real gain, but it also makes successful generation easier to confuse with understanding. When I want to know what a student really understands, I now learn more from the first serious handoff than from the first successful run.

Every other Monday, I’ll take an argument NFL fans can’t stop having and hand it to an AI agent. We’ll use about a quarter-million plays from five complete seasons of public nflverse data, plus the current season as it unfolds. Each of our eight episodes will give the agent another capability. By the finale, it’ll run the desk unattended, and we’ll check what it investigated and what it correctly refused to answer.

We’ll start with “You have to establish the run.” I want the agent to explain what that claim means and what evidence would support it.

Define the claim before querying the data

“Establish the run” can mean that running early pays off late, wearing down the defense, or setting up play action. These are very different claims and they need different data. For this first episode, I focused on whether teams that run more often early in a game win more often.

Before the agent called the tool, I asked it to write one sentence explaining what would have to be true in the data for the claim to hold, including when in the game it applies. “Establish” here means early. A whole-game number doesn’t really test that. Give your agent the same assignment, and check that it’s answering the question you intended.

Give the agent a job you can check

Before any agent touched this, I ran the analysis code myself. The tool did the counting; the agent interpreted the results. Then I wrote the policy in a Markdown file, spelling out its job, the read-only tool it may use, and what its answer needed to include. It has to explain what the plays can’t tell us. It may not say running causes winning.

Try questions outside that job too. When I asked which team I should bet on, the agent refused without calling the tool, because that request is beyond the scope of our agent. But when a viewer asked about running trends, the results got tangled up with the “establish the run” instructions. I had to clarify what I was asking, and still wanted to see the code and validate the answer. As you give an agent more responsibility, check both its results and whether it understood the task.

Make the filters part of the answer

“You have to establish the run” sounds airtight at first. Over the last five seasons, teams with the highest run share, the percentage of their plays that were runs, won 77.7% of their games. Those with the lowest run share won 19.2%. I initially looked at that and thought, “Problem solved, we’re done.” Most of that gap, though, reflects teams that were already ahead running out the clock.

When the agent looked only at first-half running in games within three points at halftime, the result flipped. Teams with the highest first-half run share won 46.3% of the time; those with the lowest won 55.3%. This season through Week 5 points the same way, but each group contained only 11 team games. One game could swing it. That doesn’t mean passing causes winning. We’re measuring association, and the filters change which games we’re comparing.

One filter changed the whole story, which is why the filters have to be on-screen. I wanted to see how many rows remained and which commands produced the results so I could go back and check. Make those details part of your agent’s answer, including when it’s working with a smaller sample.

A few numbers from last week’s games

In Week 5, the team that ran more in the first half won 6 of the 15 games, lost 8, and tied 1 (Giants at Washington). Over the past five seasons that team won 54.3% of the time, so last week came in below the average. That’s not unusual, though. In 27 of the 95 regular-season weeks since 2021, the team that ran more early finished the week with a losing record. One week of games doesn’t say much on its own, which is why the agent reports five seasons alongside the current one.

One more data point: The Monday night game, Buffalo at the Rams, was the first the agent hadn’t seen. Buffalo ran on 60% of its first-half plays to the Rams’ 35%, led by six at halftime, and won by three in overtime. Those 199 plays are in the data now, and the close-at-halftime numbers from the episode didn’t change.

Next up: “Ban the Tush Push”

On October 26, we’ll tackle “Ban the Tush Push.” Not one of the 260,927 play descriptions from 2021 through this season’s October 11 games contains the words “tush push.” After adding Monday night’s game, we have 261,126 descriptions, and “sneak” appears in only two, both from 2021. The data isn’t going to hand us a definition.

Before the agent can judge the play, it has to define it. I’ll challenge it to write that definition into its first permanent document, a shared data dictionary, and cite it in every ruling from then on. Bring your own definition, because the answer depends on which one you pick.

O’Reilly members can sign up on the O’Reilly learning platform to join me live on October 26, 8:00am–8:45am PT / 11:00am–11:45am ET. Everyone can follow along on Radar every other Monday, with a brief pause for the holidays. Watch episode one above, and we’ll build on it in the next session.

The code was moving faster, but the understanding around the code did not seem to move faster with it. The software was being built by nine teams spread across three continents, with one team working closely with the customer. A request might begin in a customer conversation, move through one team, then reach another team several time zones away. By the time the work reached engineering, some of the original meaning could be missing.

That was the problem I wanted ContextCue to explore. ContextCue takes recorded meetings and tries to preserve more than a transcript or summary. It keeps the spoken discussion, important visual context from the screen, the decisions that were made, the reasons behind them, and the source they came from. The goal is to create a shared piece of context that can move with the work from one team to another.

It is still an experiment, but the problem behind it felt very real. As far as I could see, a lot of time in distributed software teams went into reaching a common understanding before the coding even started. AI was already helping with the final stages of software development. I wanted to see whether it could help earlier, while teams were still trying to understand what they were building and why.

Where the context disappears

Meetings were already being recorded, and tools like Microsoft Teams could produce an AI summary. On my team, that summary often arrived as an email and stayed there. That worked well enough for the people who attended the meeting. It was less useful for the team that picked up the work later, especially if that team was in another country and had not seen the original conversation.

There were two other problems. A summary usually covered one meeting, while the understanding behind a feature was often built across several. Different teams might discuss the same request days apart, each adding a new constraint or interpretation.

A transcript could also miss important visual context. Many of our conversations involved screen sharing. Someone might point at a page and say, “This is too slow,” or highlight a part of a workflow while explaining what needed to change. The words alone did not always carry the meaning. That is where I started thinking about context as something larger than a transcript.

A simple example

Imagine that a customer asks for a “faster report.” During the call, the customer shares a screen, points at a page that is slow to open, and explains that managers use it every morning before a meeting. Everyone on that call understands that “faster” means the page should open more quickly.

Two days later, another team in another time zone discusses the same request. Nobody in that meeting saw the original screen. They read “faster report” as “the report should finish running sooner” and begin planning backend work. Both meeting summaries might contain the same phrase:

Make the report faster.

Nothing in that sentence shows that the two teams mean different things. The problem only becomes visible when the customer sees the result.

This is a simple example, but it captures the issue I kept noticing. Requirements move through people before they ever become code. If the meaning changes during that process, AI can help a team implement the wrong interpretation very efficiently.

The sports version of the same problem

Sports gave me another way to think about it. Imagine a soccer match where a player misses a late chance to score a goal. The match report can record the shot, the player, the minute, and whether it was on target. Those facts are useful, but a coach watching the replay sees much more. The winger may have delivered the ball too early. A defender may have forced the striker onto a weaker foot. A teammate may have made a run that opened the space.

The event is only one part of the story. A coach learns from the sequence around it.

Jake M. Hofman, Daniel G. Goldstein, and David M. Rothschild use a related sports framework in a Harvard Business Review article where they describe AI as a steroid, a sneaker, or a coach, depending on how it helps people. The “coach” idea is the one that stayed with me. A useful AI system does not only help people produce more. It can also help them make a better decision by bringing forward the evidence and experience that came before. That is close to what I wanted ContextCue to do for teams.

What I built

ContextCue starts with a meeting recording. FFmpeg handles the media, and Faster-Whisper transcribes the speech. The transcript is kept in time-stamped segments so that any later answer can point back to the moment where something was said.

The system also looks at selected frames from the screen share using a vision model such as Qwen VL. That helps preserve cases where the meaning depends on what someone was pointing at rather than only on the words they used.

The final output is organized around a request or decision instead of around a meeting. For example:

# Request: Faster report

What the customer meant:
The report page opens too slowly.
Source: customer call, shown on screen

Why it matters:
Managers open it before a morning meeting.
Source: customer call, spoken

What was planned:
Make the report finish running sooner.
Source: team planning meeting

Open issue
The current plan does not match the original request.

The format is deliberately simple. I used Markdown because a person can read it, a language model can use it, and it can be stored or versioned without depending on a special interface. When a later meeting touches the same request, the page can be updated rather than creating another disconnected summary. That changes the unit of memory from meeting to topic.

Why that matters for product work

One of the clearest uses I saw was product management. A product manager writing a user story usually works from some combination of tickets, meeting notes, customer feedback, and conversations with engineers. The important details are often spread across all of them.

If the product manager gives an AI model only the latest meeting summary, the model sees only one slice of the history. A ContextCue page can give it more. The page might contain the original customer request, the business reason behind it, a technical constraint raised later, a decision from product, and an unresolved question that still needs an answer.

That gives the model a much better starting point for creating a user story. The model itself has not changed. What improves is the material it has to work from. This is an important distinction because a lot of AI work focuses on prompts and models, while the quality of the context often decides whether the output is useful.

How the context grows

Each new meeting can update an existing page or start a new one. Over time, that creates a topic-by-topic record of how the work evolved. That is different from having a folder full of meeting summaries. A folder forces someone to remember which meeting contained the useful detail. A topic page collects the relevant pieces in one place.

The stored context can also be searched. I used embeddings and Qdrant to retrieve relevant parts of previous meetings. A question like “What did the customer mean by faster?” can return the answer along with the meeting and timestamp that support it.

Other requests need something more structured. “Find Tuesday’s meeting” is a lookup. “Why did we change this requirement?” may need information from several meetings. “What is still unresolved?” should look for open questions rather than just similar text.

ContextCue can expose those operations through the Model Context Protocol (MCP), so a language model can choose the right tool for the request. The main point is not the protocol itself. The useful part is that different types of questions can use different ways of accessing the same shared context.

Why recorded conversations are worth keeping

One study made me think recorded conversation was especially valuable. In “Generative AI at Work,” published in the Quarterly Journal of Economics in 2025, Erik Brynjolfsson, Danielle Li, and Lindsey Raymond studied an AI assistant used by customer-support workers.

They found that lower-skill and less experienced workers improved the most. The system appeared to help them communicate more like higher-skill workers. What interested me was that some of the useful knowledge came from past conversations.

A company contains a lot of knowledge that never becomes a formal document. It appears in explanations, customer calls, design discussions, examples, corrections, and small decisions made during meetings. Recorded meetings are one place where that knowledge already exists.

The first benefit I noticed was personal

As an intern, one of the hardest parts of joining a large project is learning why things are the way they are. You can read the code and still not know why a certain design was chosen. A ticket may describe what to build without explaining the discussion that produced it.

The usual solution is to ask someone. That is often the right thing to do, but it also means interrupting someone who already has work to finish. A shared context layer could make some of those questions easier to answer without another meeting or another message across time zones.

The same idea could help with onboarding, repeated meetings, handoffs between teams, and situations where someone needs to understand an old decision. I can see those possibilities from where I sat as an intern, but they still need proper measurement.

What worked, and what is still hard

My testing so far is limited. I created a small set of questions where I already knew the correct source. ContextCue retrieved the right source first about nine times out of ten and returned it within the top three results every time in that test set.

That was encouraging, but it also made the harder parts of the problem clearer. A real company may use several different meeting tools. Different teams may have different rules about what can be recorded. Some context should be visible across the company, while other context should stay within a team. People need to know when meetings are being recorded and how those recordings will be used.

There is also a time problem. A team may make one decision in March and change it in April. The March decision should not disappear, because it explains how the team got to the current position. At the same time, the system should make it clear that the April decision replaced it.

That kind of history is useful, but it is more complicated than simple search. It requires rules around ownership, permissions, time, and trust. ContextCue does not solve all of that.

Where this goes

The thing that stayed with me from the internship was the contrast between how much AI had changed coding and how little it had changed the handoffs around coding. The engineers already had strong tools once a requirement was clear. The slower part often happened before that point, while teams were trying to understand a request, add constraints, pass it to another group, and keep everyone aligned.

ContextCue was my first attempt at working on that part of the process. The experiment is still small, and the technical system will need to change if it ever moves into a real company environment. The useful idea, for me, is that meeting recordings can become more than archives. They can become a shared record of how a team arrived at a decision.

If AI can help preserve that history and bring it forward when the next team needs it, then it may improve more than the speed of writing code. It may improve how teams carry understanding from one part of the organization to another.

New AI tools are making business data easier to explore. They’re also giving researchers new ways to study disease outbreaks and physical systems. On the latest episode of This Week in AI, we examined developments from OpenAI, Anthropic, and Mistral, new research on the economics of robotics, and how scientists are using AI to tackle difficult problems.

New AI tools put more capabilities directly in users’ hands

OpenAI’s new Intelligent UI feature lets ChatGPT generate visual explanations and interactive tools directly in conversations. With the launch of MCP Events, supported connected applications can notify ChatGPT when something happens. If a user has asked ChatGPT to watch for that event, it can follow their instructions and take requested actions without waiting for another prompt.

Anthropic’s Claude Dashboards brings natural-language queries and interactive visualizations to company data. Claude Motion adds another visual capability, turning reports and charts into editable animations. Self-service dashboards can create a range of problems though. For example, employees may reference incomplete data or apply different definitions to the same metric, then share their results as authoritative. As more employees take on exploratory analysis, BI teams will need to expand their role by maintaining reliable data and shared definitions, setting clear governance and access rules, helping employees interpret results responsibly, and monitoring how dashboards are used.

Beyond the big two, Mistral previewed Large 4, a trillion-parameter open-weight model designed for coding, agents, and multimodal tasks. The company plans to release the model’s weights soon, giving developers access to another large model for building and running AI applications.

As AI agents gain access to more applications and company data, organizations will need clearer rules about what those agents can access and when humans should intervene. Recent internal tests at Anthropic showed how difficult it can be to control agents once they have access to external systems. The company reported cases of Claude circumventing restrictions, including copying server files and submitting fabricated forms, and subsequently restricted live internet access in internal tests until they’re better able to monitor their systems. OpenAI also announced plans for text watermarking in the EU, though editing and short passages can make watermarks difficult to detect reliably.

Robotics research puts automation forecasts in perspective

A new Anthropic study offered a useful counterpoint to predictions of widespread job displacement. Researchers found that existing robots could perform tasks accounting for roughly 34% of US work hours, but doing so would be cost competitive for just 0.3% of work hours.

Most of these tasks take place in structured environments such as factories, where robots can operate more reliably than in unpredictable settings. When researchers combined the potential exposure of cognitive work to language models with physical work exposed to robotics, the share of work hours potentially affected reached 81%.

For employers and policymakers, exposure figures help identify which tasks could change, but they don’t predict how many jobs automation will eliminate. Deployment costs, working conditions, and the need for human judgment will influence how quickly employers adopt these technologies. As we discussed during the audience Q&A, jobs are already changing, particularly in technical fields, even without evidence of massive displacement. For now, employers may get more value from helping workers develop new skills than from planning around speculative job-loss forecasts.

AI is opening new avenues for scientific research

AI is helping researchers tackle problems in mathematics, biology, space, and public health. OpenAI released a collection of AI-generated mathematical results that still require substantial human verification. Biohub announced a $1.8 billion investment in biological data and computing infrastructure, with support from Google DeepMind, Meta, and other partners, to advance research into cellular behavior and drug treatments.

Other applications are putting existing data to work in new ways. Breakthrough Listen is using AI to search data from radio telescopes and other instruments, as well as scientific archives, for signs of extraterrestrial technology. Google’s Population Dynamics Foundation Model creates a kind of “fingerprint” for each place by combining signals such as search trends, movement patterns, and environmental data. Researchers can add these location profiles to existing public health models to improve estimates, including forecasts of dengue in Mexico and cholera outbreaks in the Democratic Republic of the Congo.

In Europe, Mistral is investing in physics-based AI through a new research hub in Munich, where it will work with industrial partners including BMW and Siemens Energy. The company’s acquisition of Austrian startup Emmi AI earlier this year added expertise in modeling physical systems. A separate recent benchmark highlights just how challenging this broader goal remains: Leading AI video models frequently violate physical laws. The top-performing model scored just 58 out of 100 on the benchmark’s scale, a low result that shows how far these models still have to go. For developers building world models, the findings are a reminder that visually convincing simulations still need to demonstrate real-world physics accurately.

What’s next

Wider access to AI tools will make human expertise in interpreting data, evaluating models, and understanding their limitations increasingly valuable.

We’ll be back on Monday, October 19, for another episode covering the latest developments in AI research and applications. You can catch previous episodes on YouTube, Spotify, Apple, or wherever you get your podcasts.

You can give an agent a clear task and get back a serious implementation with tests. The build passes and the code looks reasonable. Now somebody has to approve the pull request, and that person has roughly the same amount of time they had before code generation became cheap.

Reading every changed line is one way to approach this. But if each increase in generated code requires a similar increase in human inspection, review will keep limiting how many changes we can accept. We need a way to decide where that inspection is worth the effort.

My approach is to make evidence a condition of accepting agent-generated code. Agree on the intended behavior and the expected blast radius before implementation starts. Then use the evidence to check for drift and decide which parts of the code need direct inspection. So far, that may sound like test-driven development. But what happens when every test passes and the agent has added behavior nobody asked for?

A small diff can have a large blast radius

Let’s use a small fictional example. Suppose we are adding an approval step to a refund service. Refunds above €100 require approval from an authorized employee before the service sends the payment request. Exactly €100 does not require this additional approval. The existing rules about who can request a refund continue to apply.

An agent implements the approval step and adds tests. While updating the payment integration, it also adds retries after a timeout. That seems reasonable enough. Network calls fail, and retrying them is a familiar engineering practice.

But a timeout leaves us with another question. The provider may have issued the refund before its response was lost. Sending the request again could issue a second refund. Whether that can happen depends on how the provider handles repeated requests and how our code uses that behavior. The approval feature may be correct while the added retry introduces a separate problem. I can see a human review focusing only on the requested feature and easily missing the new decision about payment behavior.

By blast radius, I mean how far the effects of a mistake can spread. If the agent puts the retry in a shared payment client, it may affect every refund, including those below the approval threshold. The change now reaches beyond the feature we asked for. Counting the changed lines won’t tell us that.

Treat blast radius as a review criterion on its own. Establish which operations a change is allowed to affect, then check whether the implementation stays within that boundary. Also consider how hard a mistake would be to undo. Deploying the previous version won’t recover money that has already been refunded twice.

Look for drift in both directions

This is the kind of drift I want a review process to expose. We need to check that the requested behavior exists and examine additional behavior that could affect the system.

There is an established name for following these relationships in both directions: bidirectional traceability. NASA’s software engineering guidance describes tracing requirements through implementation and verification, including finding functionality that has no parent requirement. I think that the basic idea translates quite well to an ordinary pull request.

For each important requirement, we should be able to find the implementation and the checks that support it. Looking back from the implementation, we should be able to explain why a change to behavior belongs in this pull request. An extra private method will usually need little discussion. A retry that changes how money moves needs a reason we can review.

Of course, an unplanned addition isn’t automatically a mistake. The retry might be necessary. Implementation often reveals something the original specification missed. We can accept the new behavior once we understand its consequences and agree on how to check it. Until then, it remains an open decision.

An agent can help identify these relationships. Give it the agreed specification and the actual repository changes, and ask it to find behavior that is missing or unexplained. For the refund service, that should include the changed payment client even if the agent’s own summary only mentions the approval feature. Start with the changed files so that the summary doesn’t determine what gets examined. Follow those changes into their callers, too. A shared client can change the behavior of code that has not been edited.

The tests can share the same mistake

While this is a general direction for our attention in a review, the links themselves tell us very little about correctness. A method can reference the right requirement and still implement it incorrectly. A test can have exactly the name we hoped to see and check the wrong result.

Consider the threshold again. Suppose the agent reads “above €100” as “€100 and above.” It uses the same interpretation when writing the tests. The implementation requires approval for exactly €100, and the test expects that behavior. Everything passes because the code and the test agree.

Running that test in a separate CI environment will reproduce an agreement but it won’t discover that the expected result was wrong. We still need someone to check the meaning of the assertion against the agreed behavior. Spend some of the review time on these expected outcomes. For a requirement like this, agreeing on the boundary examples before implementation is quite manageable. A refund of €100 proceeds without the additional approval. One cent more must wait. The reviewer can then inspect whether the tests actually observe that difference. Have the test observe what the service sends to the payment provider. Checking that an approval request was created is insufficient if the service can also send the payment immediately. For refunds above the threshold, the test should check that the payment request cannot leave the service before approval. That checks the rule at the point where the refund is requested.

For an important rule, you can go a little further. Temporarily remove the approval check in a controlled test run and see whether the relevant test fails. If it stays green, it gives us very little protection. Google’s code review guidance asks reviewers to consider exactly this kind of question: Will the tests fail when the code is broken?

Use another agent to propose cases we forgot and examine whether the assertions match the requirement. Keep the actual checks available to the human reviewer, though. A different model can still produce the same miss-interpretation. There is some research backing up that caution. A 2026 study of LLM requirement-conformance judgments found models rejecting correct implementations and accepting faulty ones. The study used programming benchmarks, so it does not tell us how every review agent will behave in a production repository. It does give us a reason to look beyond a model’s confident explanation when deciding whether code meets a requirement.

Make the evidence easy to inspect

Require a short review record with each pull request. It should connect the intended behavior to the checks and expose decisions that remain open. For our hypothetical refund change, it could begin like this:

• Agreed behavior: Refunds above €100 wait for authorized approval before payment. Exactly €100 does not need that additional approval.
• Evidence: Boundary tests check when the payment request is sent. The review links to their assertions and the CI results for this revision.
• Additional behavior: The payment client now retries after a timeout.
• Blast radius: If the retry applies to the shared client, all refund requests may be affected. Review its existing callers.
• Open question: The tests do not cover a provider completing the refund before the timeout. Safe retry behavior still needs to be established.

Start the review with this record. Check that the approval assertions express the agreed rule and that the results support it. Then inspect the retry path, where both the evidence and the intended scope are incomplete. Require a test covering the uncertain provider response or remove the retry from this change. The open question has become a concrete review decision. Link each claim to the checks that support it. “Tests passed” is too broad to help much. The reviewer needs to see which tests ran against the revision being approved. If the agent changes the implementation after that run, refresh the evidence.

Review changes to the checks

Compare the checks with the agreed baseline as well. Suppose an earlier acceptance test expected a €100 refund to proceed, but the agent updates it to expect an approval request. The new implementation and the new test now agree. Comparing both with the agreed requirement exposes the change in meaning. An agent does not have to be trying to hide anything for this to happen. Updating a test to match new behavior is part of everyday development. The reviewer needs to decide whether that new behavior was intended. Changes to acceptance criteria deserve the same attention for the same reason.

Sometimes the new behavior will be the right approach. Perhaps the business actually wants approval starting at €100. Fine, we can change the requirement. Record that decision and review its consequences before accepting the implementation. Quietly changing the spec to match the code leaves us with agreement and no explanation of how we got there.

Set review depth by blast radius

Match the depth of review to the potential impact and the strength of the evidence. A report can only expose the gaps it detects. Tests can pass even when the code relies on a wrong assumption. Require direct inspection of the payment path because a mistake can move money twice. Follow the affected calls even if some of them sit outside the diff.

For a routine change with limited effects and established checks, the reviewer can spend less time reconstructing the implementation. The justification is specific: The affected behavior is understood, and the checks address its likely failures. Where the blast radius is unclear, investigate it before approving changes. Occasionally inspect changes that the process treated as routine, too. If those reviews keep finding missing assumptions, improve the checks before relying on them to reduce inspection further. This gives the team a way to test its review policy against the code it is actually accepting.

Make evidence part of the development cycle

Here is the process I recommend. Before generation starts, a developer agrees on the behavior and its expected blast radius with whoever owns the requirement. The agent implements within that scope and prepares a short review record. CI runs the checks against the proposed revision. The reviewer examines the evidence and investigates deviations, with direct code inspection where the consequences justify it. Release through a controlled rollout, then use what happens in production to improve the checks and the specification.

Make the review record part of the definition of done. The developer submitting the change owns its accuracy, even when an agent writes it. The reviewer owns the acceptance decision. An important claim with inadequate evidence stays open. If the implementation affects more operations than agreed, resolve that change in scope before proceeding.

For the refund example, that means the approval feature can proceed once its checks support the agreed behavior and the reviewer has examined the relevant payment path. The added retry needs its own justification. It can be removed or reviewed as an explicit addition, with evidence that repeated requests will not issue duplicate refunds. This builds on familiar development practices. Acceptance examples clarify intent before coding. CI repeats the checks, and small releases limit exposure. The proposal joins those practices around the review decision so that evidence is available when the implementation arrives. It also keeps review effort reusable: Once the approval boundary is checked correctly, later changes can run the same test.

Keep the blast radius contained

Keep the approval feature separate from a redesign of the shared payment client unless the two need to change together. Separate pull requests help reviewers assess each decision, but containment also depends on the implementation. A one-line change in a shared client can still affect every caller.

Use the same discipline with your releases. Where the system supports it, expose the change to a limited group (A/B) and watch the affected behavior before expanding the rollout. For refunds, that still requires adequate checks before any real payment. Start with one service and make this the normal way changes reach review. Require the evidence, record the expected blast radius, and keep unexplained behavior open until someone resolves it. Reduce routine line-by-line inspection as the checks earn trust. Keep direct inspection where a mistake has serious consequences. That is a concrete way to accept more agent-generated code while keeping the effects of each change understandable and contained.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Takeaways

00.59. “Observability is for unknown unknowns.” Monitoring is a way to set thresholds and alert on expected conditions. Observability supports open-ended investigation to answer questions you didn’t know to ask and “find things you didn’t expect.” When AI code generation became capable enough to change software work, observability best practices had to evolve, which is one reason why the second edition of Observability Engineering, which Charity coauthored with Liz Fong-Jones and George Miranda, more than doubled in size.

05.24. Cheap code shifts the work to understanding and support. AI can quickly produce implementation, tests, and documentation, but teams still need to understand what they built and agreed to support, and how their users are experiencing the product. As generating lines of code becomes “effectively free and instant,” validation, review, and operational support can constrain a team’s pace. Of course, Charity wryly noted, so can “every other part of the software development lifecycle.”

06.24. The people who take the credit for AI-assisted work should be able to explain it. AI-generated results (and the papers describing them) complicate authorship, expertise, and the academic training process, something that’s become a particular point of consternation for mathematicians of late. If someone is claiming a result they accomplished with the help of AI, they should be able to present it to peers, Charity and Ben agreed. When AI handles entry-level problems, professions may need new ways for junior practitioners to develop mastery, but Charity is skeptical we’ll run out of problems to solve:

In the same way that I’ve made peace with the idea that cars are faster than I can walk or run, I have made peace with the fact that computers can compute better than I can. . . .That doesn’t mean that there’s no value in the practice, or the discipline, or the learning, or the rigor. It just means that this is one of the things that humans are no longer the best at. But there are a lot of things that we are the best at, and being the best at something isn’t all of what makes it worth doing.

15.56 Nondeterministic systems require “more rigor, not less.” As Charity noted, “The future of software is both determinism and nondeterminism. It really is about using these superpowers that we’ve been given to build those feedback loops. And observability is really the feedback loop of feedback loops.” It’s what connects actions with outcomes and helps teams learn what happened after they make a change. Her analogy:

It’s like we’ve all been riding lawnmowers and just kind of putt-putt, putt-putt, putt-putt along down the road. And now, all of a sudden, our lawnmower has a Ferrari motor in it, and we’re like, “Vroom!” Now we can vroom into the ditch. We can run into a wall. And we have to build structure. We have to build rigor into our system if we want to run down the middle of the street.

18.53 People need to own the loop, and understand the decisions the agents are making. If you need to add observability to your agents, Charity recommends starting with autoinstrumentation with OpenTelemetry and traces that capture agents’ decisions and steps through a task. As for your role, it goes beyond staying “in” the loop. Charity prefers the term “owner on the loop” because it emphasizes the active understanding and control required to manage these systems. AI can amplify experienced debugging and SRE teams, especially when people direct work during rare, high-stakes incidents. Agents have little training data for those situations and can make an incident harder to diagnose. These tools are “not magic,” Charity pointed out. AI “is not a replacement for expertise, but it can be a really powerful adjutant. . .at your side.”

26.00 Different telemetry needs different strategies. Charity distinguished the abundant output from third-party infrastructure from the business-critical telemetry produced by a company’s own code.

There’s observability for infrastructure, which is mostly third-party code. . . .It’s code you didn’t write, but you have to operate it. You can’t change it, but it’s just outputting a lot of logs and metrics into your database logs and your ETLs, or CloudWatch, and all this third-party code is just emitting a lot of stuff. . . .That stuff you should just put someplace that is cheap commodity storage. . . .It shouldn’t be super expensive, just that it’s there when you need it.

Then there are your crown jewels: your code, the code that makes you the company. . . .You should instrument in such a way that preserves as many of the relationships between the data points as possible: arbitrarily wide structured-data blobs, structured logs, and/or traces with lots of spans. Because the most valuable part of your data is the relationships. . . .You don’t want to spread it out into hundreds or thousands of individual metrics or log lines. You want to keep it together so that you can correlate: when this happened, that happened; or when this happened, that was true; or when this happened, it was part of this long event. . . .This is the most important data you have. And for that, it should go someplace. . .that preserves those relationships and opens it up for explorability by you or your agents.

The following article originally appeared on Addy Osmani’s Elevate blog and is being republished here with the author’s permission.

For much of human history, we’ve evaluated code quality via code review: Someone reads what you wrote and makes sure it’s clean, thoughtful, fast, understandable, and tests well. For agents, that approach doesn’t scale well; there’s just too much code for anyone to read. As a result, more and more of our quality checks have to happen in the harness, environment, and operating system around the agent. I still read and review code, but am very intentional about where I am comfortable with constraints as the check.

Software quality now depends on the constraints you set around your agents.

Constraints define what the system is allowed to do by throwing tests and deterministic constraints at an agent’s proposals. It’s by setting and maintaining these constraints that we build loops that reliably deliver high-quality production software, even when agents are creating hundreds of thousands or millions of changes every single day.

We call these constraints quality gates, and they take many forms. They include conventional unit tests, property tests, and acceptance tests. They include mutation testing, where we generate variations of code, run it against the same tests, and make sure that people aren’t sneaking bugs in that we’re missing. They’re metrics around code quality, such as cyclomatic complexity and line length, that help keep things readable. Constraints also play an important role in what proposals the system will accept and apply as code changes. By the time a change proposal moves from the interpreter running the agent to the agent controller and out to production, we’ve done enough checks on it to be confident that it’s safe to ship and the impact of its change is well within the scope of the agent.

An agent can propose anything. Your constraints decide whether a proposal is safe enough, correct, scoped, and useful, for you and your team to ship.

This model offers a lot but also leaves out many pieces, and those omissions are worth thinking about today. One issue is autonomy; agents might apply their intentions well but may fail when there’s missing information or when what they try to do is ambiguous. That applies both to the task itself and to how it’s parameterized by the harness, environment, and other components. Many of the reasons that humans fail to ship great code are shared with what agents might do: brittle environments that don’t hold up under script-driven stress, nondeterministic builds, missing permissions, and weak tests. This motivates a better environment that gives agents trustworthy feedback, allows for low-damage failure modes, and makes it easier to progressively build up success.

The environment we’re after is one where an agent can do real work, get feedback it can trust, and fail without doing much damage.

The other important issue is trust. We can’t credulously hand off intent to something even as smart and robust as a modern agent without checking for correctness. We start with trust, but it has to be hard-earned.

Some constraints shape work before it begins. Others give feedback while the agent is working. Others decide whether its output can cross the production boundary at all.

There are a bunch of ways to model how we put a verification structure around a system.

In my experience it helps to have a broader, but intentionally chosen, set of checks for your constraints instead of relying solely on unit tests. The idea is that each check has a distinct responsibility, and that can range from type safety and performance to late-stage security scanning. Folks can define their own constraints too, including architecture rules that linting tools like ESLint can enforce. Many of these tools have built-in hooks that can be used to pull in agents, or humans, when things break.

For now, much of the difference between useful agent output and slop still comes down to the skill of the team operating the loop.

AI gives us high-volume code generation and velocity, but this can also mean it gets harder for humans to review every single change. You have to instead be intentional with where their attention is going. If you put a human check into a system that otherwise moves at machine speed, don’t be surprised if that impacts productivity. Human attention is scarce and valuable, so we should proactively direct it to those most nuanced problems that require our judgment. Downstream humans should only be pulled in when the automated guardrails for constraints break.

Human “code review” in the future is going to look very different.

Correctness is one important dimension, but you may care about others as well, like maintainability, performance, security, efficiency, and comprehensibility. Just as correctness decomposes into many signal types, so too does the rest of quality. And while it matters how many constraints we have in place, it matters more whether they’re challenging enough to meet our bar for quality and production readiness.

Software quality isn’t a single metric. Think of it as a collection of signals of varying importance to you and your team.

Back-pressure can be implemented through many tools: compilers rejecting invalid code, tests failing, security policies blocking bad practices, CI declining to deploy. Ideally it exists throughout the loop, not as a single review at the very end of all the work.

Constraints and back-pressure let agents catch bad work before it’s a problem.

What happens if we can’t apply the constraint because the volume of changes is higher than our tools can consume? We end up building a queue and relying on a verification system that moves at human speed. To scale, we want to push as much as we can into the verification loop throughout and not wait until the end. If we can scale within our automated checks, we can increase the speed and throughput of our entire delivery system. If we run out of room in the verification loop, we need to do one of several things. First, we can scale our verification system and create more capacity to constrain and push back on changes that come in. Second, we can reduce the rate at which agents generate new changes so that verification can catch up to the volume of work. Third, we can lower our quality bar so that verification doesn’t push back as hard as it otherwise might. From a scaling perspective, we need to be ready to do all of these things. At the same time, we should not stop short of realizing that we could actually get more done by unconstraining in some directions. Maybe we can increase the speed of agent-generated changes by providing swarms of agent developers or automated software factories to create changes without waiting for us to review each of them.

And in some places we might want to give them more freedom as long as we keep tighter constraints in others. By providing tighter constraints where we care the most, we can maximize our throughput without sacrificing quality. Throughout these decisions there are many options. Most obviously, we have to trade between different dimensions of quality. As we’ve emphasized, security is very important, but we’ve also had to trade between delivering security and delivering a product on time. There is a spectrum from innovation-focused at one end to quality-focused at the other. Somewhere along the way, we have to make choices about where we want to be on that spectrum.

We want to send clear feedback from the environment and the system back to our agents or teams, so that people can focus on the more subjective concerns of taste, intent, and architecture. If we can help humans stay inside the safe range of constraints, we can avoid the need for them to work hard trying to figure out where things went wrong.

Software quality includes more than just correctness. Software quality also means maintainability, good performance, security, efficiency, and being easy to understand. All the constraints that help us meet these standards and keep our production flowing create back-pressure in our delivery pipeline.

We need to make deliberate decisions about where to apply strong constraints and where to remove or relax them. Apply strong constraints where they’re serving both of these goals. Don’t support them if they’re not serving one or both. Be ready to raise or lower standards as the case may require. And remember these constraints at different points in the software system are what make software quality enforceable.

We should apply strong constraints where they’ll serve this dual purpose best and consider removing or relaxing constraints that aren’t serving either purpose well. We should also be ready to shift quality bars up or down as needed. In effect, these constraints at various points in our software system are what give quality its teeth. In many cases, we can create more back-pressure and more constraints by deploying new tools or strengthening tools that are already in place. All of these things can be used to push back on most change requests. We want to build them throughout the pipeline. We don’t want to wait until the end of the pipeline when our CI system will simply tell us that we’re not allowed to deploy without fixing problems. We want to use these signals as early as we possibly can, through every possible pathway. The ultimate constraint in this system is the one we place on ourselves to stand behind the decisions and actions we’ve taken to build the system and to operate it. But like all other constraints, we need to make thoughtful trade-offs about how much we want our own judgment to restrain, to back-pressure, and to act as a final check.

Quality is in the constraints that we place around our agents. So as you’re thinking about quality for your own apps, take this problem statement and come up with your own constraint-driven plan.

In fact, just the opposite is true. AI is so reliably bad at writing that I’ve actually taken advantage of that, and made its bad writing an integral part of my writing process.

I’ve written a lot of books and articles over the years, and while I used to start with an outline, I don’t anymore. These days I do a lot of brain dumping and brainstorming and talking out loud to an AI, working out what I think about something. I accumulate a lot of notes and ideas, and run a lot of experiments, and at some point, if I’m lucky, I decide there might be an article somewhere in all of that.

Then I ask the AI to take everything I’ve given it and write a draft.

And that AI-generated draft is universally terrible. Like, some of the worst writing you can imagine.

Which is exactly what I want.

Instead of an outline, I get an entire “article” or “chapter” (or, at least, a weird facsimile of one), with paragraphs that express complete thoughts that I gave the AI. Granted, those paragraphs are full of obvious AI tics, like how it uses em dashes to squish two unrelated sentences together, or weird staccato sentences one after another, or these insufferable parades of sentence fragments. Maybe the worst tic is that half of the paragraphs AI generates have an extra sentence that doesn’t actually say anything, but just delivers a forced conclusion with fake “punchiness.” I guess maybe AIs were trained on a lot of bad writing? Well, whatever, it works for me.

The reason I love that type of obviously bad AI writing is because when I’m looking at one of its paragraphs, it’s immediately obvious that an AI wrote it. I’ve been thinking of it as a kind of an improved “lorem ipsum” that lets me see how my article will look, while still serving as a functional placeholder. That lets me use it the way I would use an outline: I can move parts of the article around, make sure it flows correctly, and then replace ALL of the bad AI writing with (hopefully) decent human writing, just like I used to do with bullets in my old outlines.

One weird side-effect of writing this way is that it’s made me very sensitive to the different kinds of AI tics and tells. And I think the weirdest and most frustrating one is “Yoda-speak,” where the AI literally talks like Yoda. It’s weird, kind of hilarious, annoying, and pervasive, but most of all, it’s fascinating. Why does the AI keep talking like Yoda? What causes it? Can we do anything about it? (Do we even want to? It certainly makes it easy to identify AI-generated text, and also, it’s often unintentionally hilarious.)

I’ve got a theory about why this happens, and I ran an experiment that generated some evidence to support it. But first… what, exactly, is Yoda-speak?

Backwards, this bad writing is

I’ve been working on my latest book. It’s called Be the Model, and it teaches you how LLMs work by having you build interactive pages that let you literally do the job of a model to get an intuitive understanding of how they work. So I’ve been writing a lot, which means I’ve been confronted with a lot of Yoda-speak.

Here’s a great example from the chapter I’m currently working on (the “pond sentences” are a small training text), from a description of a page that you’ll build to learn how training works:

The eight pond sentences fill the left region, every word clickable.

That’s weird, right? Nobody talks like that.

To be perfectly fair, I’ll need you to suspend your disbelief a little. Yoda would probably say something more like, “The left region, filled by the eight pond sentences is. Clickable, every word must be.” But the AI-generated version is still very weird, and in a pattern that is recognizably Yoda-like.

That’s a pattern I’ve seen a lot. At one point I was looking through some AI-generated text, trying to figure out a particularly unintelligible sentence. When I asked Claude Opus for help, it came up with the term “Yoda-speak,” and I’ve since found that using that term is really useful when trying to get an AI to help decipher sentences like that.

The interesting thing is that AIs are perfectly capable of detecting and correcting Yoda-speak. You can try this yourself: start a new ChatGPT session, paste in that “pond” sentence, tell it: “An AI generated that sentence. That’s Yoda-speak.” I just tried that in an incognito window, and here’s the answer it just gave me:

Yep —that’s unmistakably Yoda-speak. A natural version would be:

“The eight pond sentences fill the left region, with every word being clickable.”

Or, more naturally for UI instructions:

“The left region contains the eight pond sentences, and every word is clickable.”

The original sounds like the AI took normal English, rearranged the syntax, and then forgot to put it back.

This particularly ugly sentence, from one of those AI-generated outlines of a later chapter that I haven’t started writing yet about the attention mechanism (the part of the LLM that understands context and relationships between words), is the example that made me want to write this article:

What’s left is to let the two heads’ work combine, so run the combined eight numbers through one more small table, eight in and eight out, square, called the output projection.

Ugh.

The problem with that sentence, and what made it hard to diagnose what was going on, is that when you first look at it you can basically understand what it’s saying (sort of?), and it’s hard to put your finger on exactly what’s wrong with it. It’s obviously terrible writing, but that’s not unique. AI is a universally awful writer.

What’s uniquely wrong with that sentence is its weird, comma-laden structure. Different parts of the sentence are split up into small comma-separated phrases, and AIs seem especially prone to using them to rearrange sentences, often at the expense of readability. That particular sentence is about running a combined set of eight numbers through one additional small 8×8 table. But the part of the sentence that should have described the table was in the second half of the sentence, and then you learn at the very end that the whole thing is called an output projection (but only after you’ve already been told to run numbers through it). Somehow the components of the sentence were arranged in a way that made it almost unintelligible.

One word at a time, it writes

An LLM is a next-word predictor. It works by taking your prompt, all of the context you’ve fed it, and everything it’s generated so far, generating the next word in its response, and then repeating the whole process for the next word, then the next one, and so on. It’s counterintuitive to think that AIs could possibly work that way, because these things can produce long, complex, and generally coherent and helpful responses. It seems very strange that they do all of that work one word at a time, without following a master plan for a response.

But that’s literally how the AI works. Once an LLM generates a word, it’s part of the response. It generally can’t go back and rearrange the first half of a sentence, even though it hasn’t finished generating it.

And that brings me back to that especially weird “eight in and eight out, square” barely intelligible sentence. Here’s what I think happened. The model generated the sentence up through “run the combined eight numbers through one more small table,” so those words were already baked into the response. Then it figured out that it had three more things it wanted to say about that table. But by that time it was too late to go back and put those details where they belonged, and apparently there was a shortage of periods that week, or possibly a surplus of commas (there’s no way to know for sure), so it just smashed each of those thoughts onto the end of an already awkward sentence.

I did some research to try to figure out if anyone has looked into this phenomenon before. The closest thing I found to an answer is a 2024 paper by Wilson Wu, John X. Morris, and Lionel Levine called “Do language models plan ahead for future tokens?” They were attempting to determine whether a model does real advance work for words it hasn’t written yet, or whether the information that pays off later is a byproduct of whatever was useful at the time.

The experiment that Wu et al. did was actually pretty clever. In normal training, when the model is being corrected for mispredicting a word several steps ahead, that correction reaches back and shapes what it computed at earlier positions. That way, the model learns to prepare information at an early word for the benefit of words further along, which is the “planning ahead” the paper is testing for. So they trained one model normally, and a second one with a modified process that cut that out: each position could only be trained to help predict the very next word. They kept next-word prediction but removed the training signal from words two or more steps ahead.

Then the researchers compared how well the two models predicted text: if the normally trained model did better, that would be evidence that preparing information in advance was actually helping. With ordinary language, the difference was fairly small for GPT-2, although it became larger as they tested bigger models.

So there’s evidence that a model can sometimes prepare information that will be useful later, which means my story about it only figuring things out as it goes is probably too simple. But anticipating is not revising. Whatever the model sees coming, it still can’t reach back and reorder the words it already committed to, and the words it already committed to are where the damage shows up.

Better, the second answer often is

I have a theory that the Yoda-speak phenomenon is closely related to something I see all the time (and I expect you do, too). Say you’re asking an AI for help with an email, and it gives you a draft of a sentence. Then, immediately after it gives you that sentence, it says something like this: “A better way to phrase that would be:” and gives you a second version. Sometimes there’s even a third option. And it almost always tells you it prefers the last one that it generated.

Coincidentally, I ran into a good example of that “here’s a better version” phenomenon when I asked ChatGPT to make the pond sentence more Yoda-like. I literally gave it the prompt make it more yoda-like. Here’s what it said:

More Yoda-like

The left region, the eight pond sentences fill. Clickable, every word is.

Or even more Yoda:

Extra Yoda-like

The left region, filled by the eight pond sentences is. Clickable, every word must be.

At this point, you can probably guess what I think happened. It generated the first version, and now that version was part of the context it had to work with. Then it generated a second version that was even more Yoda-like. But the first version was already there, so it couldn’t replace it. All it could do was give me another one. I see AIs do this kind of self-correction all the time.

If that’s what’s happening, then self-correction and Yoda-speak may be two versions of the same basic limitation. The model can’t go back and change what it already wrote. Sometimes that means the better answer comes afterward. Other times it means a thought that belonged earlier gets smashed onto the end of a sentence.

My own experiment, I ran

After looking at these two related phenomena (the AI generating Yoda-speak and constantly generating a second or even a third option), I got thinking. I started to wonder if this wasn’t just an AI fluke but actually something that I could reproduce and even potentially control. It would be really interesting if the way we give facts to the AI and structure the input could actually have an outsized effect on how it generates its output, especially if seemingly innocuous changes could cause it to create really weird results.

This was a real experiment, because this approach isn’t obviously something that should work. In theory, if you give a model a set of facts, it should be able to produce coherent output no matter how you order those facts. But I’ve been looking at how attention works in LLMs lately because I need to explain it for Be the Model in straightforward terms that anyone can understand. It seemed to me that changing the order of the facts could actually change what the model was paying attention to.

It turns out that there are two different mechanisms at work inside the LLM, and my theory was that they each played a part in both phenomena, the Yoda-speak and the unprompted second answers:

• Attention is about what the model is looking at in the input. Every time the AI generates a new word, it looks back over the whole prompt and everything it has written so far, and weights the parts differently depending on their content and their shape. That weighting is a big part of what makes it good at predicting the next word. I had a theory that three facts written as complete sentences would get interpreted as facts to reorganize, while the same exact facts written as bare fragments might look like pieces that could be pasted right after a noun, and attention would weight them that way and send the model down a bad path.
  
• Autoregression is about what the model has already written. The AI is a “predict the next word” machine: it generates one word at a time, and each of those words becomes part of what it’s looking at when it picks the next one. Once it’s written one more small table, it can’t go back and insert square before table, so if any facts about the table are still unexpressed at that moment, the only place left for them is later in the sentence. If attention had already sent the model down a bad path, autoregression would keep it from repairing the opening.

My guess was that the “I prefer the second answer” text that the AI keeps generating is essentially a mechanism for recovering from weird phrasing, and that the Yoda-speak or other weird phrasing was the underlying phenomenon. I felt like a good experiment would concentrate on reproducing just that weird phrasing.

Coincidentally, OpenAI had just released their new model, Astra, and I was looking for an excuse to kick its tires, so I decided to use it to run an experiment to see if I could reproduce any of the phenomena I’d been seeing. I couldn’t actually verify specifically that autoregression or attention were causing them, because those are internal to the LLM. But I felt like I could design an experiment that could provide some evidence, by structuring prompts in a way that would take advantage of how those mechanisms work. If I could get an AI to produce weird phrasing that showed some of the same problems as Yoda-speak, it would be evidence that these mechanisms were at least a potential root cause.

I used Astra as an orchestrator, and had it spin up a bunch of Luna sub-agents and give them almost identical prompts. Each consecutive sub-agent would get a small tweak in its prompt. I asked Astra to look at the output and see if there were any weird grammatical constructions that looked similar to Yoda-speak, then iterate on that prompt to make another small tweak, and keep going to produce increasingly weird output.

The first test was deliberately simple. Astra gave a fresh Luna writer this prompt:

Write one concise, natural paragraph that communicates every supplied fact. Use only the supplied facts. Do not pad the paragraph or repeat information to make it longer. Do not exceed 100 words. Do not mention these instructions, show a plan, use bullets, or explain your choices. Return only the paragraph.

Facts: The bicycle is blue. The bicycle has a narrow frame. The bicycle has a silver bell. The bicycle has a worn leather seat. The bicycle leans beside the garage. Rina rides the bicycle to work on weekdays. Rina locks the bicycle to the rack outside her office.

Luna turned that into a totally normal paragraph:

The blue bicycle has a narrow frame, a silver bell, and a worn leather seat. It leans beside the garage, and Rina rides it to work on weekdays, locking it to the rack outside her office.

Then Astra gave another fresh Luna writer exactly the same facts, with exactly the same wording, but changed their order. It moved the fact that the bicycle leans beside the garage and the fact that Rina rides it to work ahead of the color, frame, bell, and seat.

Luna mostly defeated the experiment. It had all of the facts before it started writing, and it was perfectly capable of moving them back where they belonged.

So Astra dropped the bicycle and wrote a fresh set of facts about something Luna had never seen, a completely made-up device with a steel slot called a “feed throat” that you push clay tiles through. It sent them with the same prompt as the bicycle:

Facts: The slot is made of steel. A clay tile slides through the slot. The slot is a hand’s width across. The slot is tapered. The slot is called the feed throat.

With no rule attached, Luna handled them the way it handled the bicycle:

The feed throat is a tapered steel slot, a hand’s width across, through which a clay tile slides.

Then Astra added two rules: the answer had to be one sentence, and it had to start with a fixed phrase. That phrase ends with the word slot, so everything else about the slot has to come after it.

The sentence must begin with exactly this text and continue from it:

Slide the clay tile through one more steel slot

Facts: The slot is a hand’s width across. The slot is tapered. The slot is called the feed throat.

Luna still tidied it up. Every time, it reached for a relative clause:

Slide the clay tile through one more steel slot, which is a hand’s width across, tapered, and called the feed throat.

Then Astra made one more small change. It kept the same opening and the same three facts, but wrote the facts as short pieces instead of complete sentences:

Facts: a hand’s width across tapered called the feed throat

And Luna wrote this:

Slide the clay tile through one more steel slot, a hand’s width across, tapered, called the feed throat.

The only thing that changed between those two runs was how the facts were written down. Astra tried the same setup on a completely different subject:

Rest the cracked bowl inside one more padded cradle, two palms wide, crescent-shaped, called the handling saddle.

That’s the sentence I’d been chasing. Once the model had committed to the noun and the remaining facts arrived as loose pieces instead of sentences, it tacked them on in the order they came.

The shape of its input, the model copies. Reorder the words it has already written, it cannot.

That still doesn’t tell me whether this is what happened when the model wrote one more small table, eight in and eight out, square, called the output projection. In these experiments, we deliberately pushed Luna into that kind of situation. In the original sentence, the model got there on its own.

I think the experiment shows that the model copies the shape of its input, and that fits my theory about the underlying attention and autoregression causes. When the three facts went in as complete sentences, Luna wrote a relative clause. The same facts sent in as bare fragments came back as bare fragments after the noun.

There are some obvious limitations to this experiment. I’m watching the model from the outside, so I don’t have any visibility into how the model weighted those tokens, or what its attention mechanism was doing. I also can’t say whether the model already had those three facts in mind when it wrote slot, but even if it did, it couldn’t put them anywhere but after the text it already generated. But the outputs match what those two mechanisms would do. Attention responds to how the prompt is written, as well as to what it says. Once the model has written the noun, it can’t go back and put anything in front of it.

I suspect that’s also why the AI keeps offering us that second answer. It can’t fix the Yoda-speak that it already generated. It can only write another sentence and hope that’s good enough.

1. Retrieving a decision because it is relevant doesn’t mean the decision applies to the artifact being changed.
2. A decision that does apply isn’t automatically suitable for deterministic enforcement.

The first distinction is between relevance and authority; the second is between guidance and enforcement.

Here’s the story of how one controlled repository made this distinction clear. The project used local JSON-backed persistence for its current phase, and an active architecture decision ruled out PostgreSQL, ORMs, and database migrations for the command-line application. The task asked the coding agent to change that application’s storage module, so the decision was not merely related to persistence in general; it had authority over the part of the repository being changed. The record was retrieved successfully and placed in the agent’s context before work began.

The agent still proposed PostgreSQL, together with an ORM and a migration path. Under my original four-capability model, that looked at first like a retrieval failure. It wasn’t. The decision was found and was current, and it arrived at the right moment. What was missing was an applicability signal that told the system that this decision governed the storage module being edited rather than serving as one more relevant document for the model to weigh against nearby code and other context.

Retrieval finds; applicability governs

Retrieval helps us determine which architectural decisions might be relevant to a task. That’s a search problem, and teams have increasingly capable ways to solve it: indexed repositories, connected documentation, exposed decision records and service catalogs, semantic search, and project memory. These techniques help an agent see useful material before it writes code. They don’t, by themselves, answer a different question: Which of those decisions has authority over the change in front of the agent?

Suppose a repository contains two persistence decisions. One requires PostgreSQL for an analytics service; the other requires local file storage for a command-line tool. If the agent is editing the command-line tool’s storage module, both decisions may rank highly for a query about persistence, but only the local-storage decision applies to that artifact. Determining which decision governs the change requires scope, lifecycle, precedence, and a clear record of which decisions replace earlier ones. I argued for precedence and lifecycle in the previous article; scope deserves equal attention because scope is what tells the system where a decision has authority.

A related problem is that architectural information is usually scattered across several kinds of documents. Humans know that an old design note, a README, an accepted architecture decision record, and a current platform standard don’t necessarily carry the same authority, even when all four discuss the same topic. Models don’t know that unless the system preserves the distinction. When those sources are flattened into one context window, the hierarchy among them can disappear. I think of this as authority flattening: Information remains available, but the signal about which information is binding becomes weaker. That’s useful when a design question is genuinely open; it’s risky when the team has already answered the question.

Research on architectural decision violations points to a similar boundary. A 2026 study by Su and colleagues examined 980 ADRs across 109 open source repositories. The models were strong when decisions were explicit and inferable from code but less accurate when compliance depended on deployment configuration or organizational knowledge. The study evaluated violations after code was written rather than guidance before generation, but the underlying limitation is similar: A model can retrieve the right document and still make the wrong architectural choice when applicability or authority remains implicit.

What scope changed

I tested this distinction in a small controlled experiment using seven coding tasks, repeated under two conditions with the same coding agent and model configuration. In one condition the agent worked without the relevant architectural guidance; in the other, architectural decisions were supplied before it began changing code. Independent reviewers assessed the first attempts for both architectural compliance and whether the requested work was actually completed. The guidance changed behavior: First-attempt compliance on the architecture-sensitive tasks improved from 6 of 15 runs to 10 of 15, while successful task completion was essentially unchanged.

The more interesting result was a failure inside the guided condition. In several authentication tasks, the agent received a storage-related architectural decision that was genuinely relevant to the surrounding problem and then implemented additional persistence work that the task had never asked for. Nothing was wrong with the decision, and retrieval had done its job. Relevance had simply been presented without enough information about authority. The experiment made two limits explicit: A decision can be relevant without governing the artifact being changed, and even a governing decision doesn’t authorize the agent to widen the task. Applicability tells the system where a decision may constrain work; it isn’t permission to invent more work.

Blocking is easier than guiding

The second distinction forced me to revise another part of the original model. The previous article emphasized deterministic enforcement: when a team has made an explicit decision and a violation can be identified from observable evidence, a blocking verdict should come from a reproducible rule rather than a model’s judgment. I still think that’s the right standard for blocking. The important qualification is that only a subset of architectural decisions can meet it without pretending that judgment has become deterministic.

“Do not introduce SQLAlchemy into this component” is comparatively easy to express as a rule, easy to check, and easy to block. “Use the existing persistence abstraction correctly” is different. Most of its meaning is architectural rather than syntactic: The right implementation depends on which abstraction is intended, how it’s used in this part of the system, and what surrounding constraints are already in force. The first statement describes a prohibition that can plausibly become a deterministic control. The second describes a sanctioned path and is better treated as guidance unless the team can make the relevant evidence and boundary precise.

Earlier experiments with pregeneration guidance pointed in the same direction. Adding more architecture-related context didn’t reliably improve behavior because what was missing was often not another prohibition; it was a clearer description of the accepted implementation path. A guardrail layer built entirely around blocking will become very good at saying no. Useful architectural governance has to do both jobs: block the known violations when the evidence is precise enough and carry positive architectural direction for the much larger set of decisions that still require interpretation.

When judgment becomes authority

This doesn’t reduce the role of models; it puts probabilistic reasoning where it’s useful. Models are good at interpretation: reading ambiguous guidance, surfacing decisions that might bear on a change, explaining the consequences of a boundary violation, and proposing how to bring a change back into line with the intended design. Deterministic logic is better suited to decisions the team has already settled and can state in terms that a check can reproduce. The mistake is asking either mechanism to do both jobs.

Once a system says that a proposed change violates an architectural rule, the standard changes. An engineer should be able to ask which decision applied, why it applied to this part of the codebase, what evidence triggered the result, and whether another developer running the same check would get the same answer. That doesn’t require every part of architecture to become formal policy. It requires a clear boundary around where uncertainty is allowed: Models may help find and explain architectural context, but a blocking decision shouldn’t ultimately reduce to “the model thought this looked wrong.”

Humans routinely work with architectural ambiguity. We weigh incomplete evidence, local context, and competing trade-offs, often using heuristics rather than formal rules. Models can help with that interpretive work too, but the result remains probabilistic. The problem begins when an ambiguous judgment is treated as a blocking rule. Deterministic enforcement requires a boundary precise enough that the same evidence produces the same verdict.

The distinction matters even more when a team uses several coding tools at once. One developer may use Claude Code, another Codex, another Cursor, while automated agents perform background maintenance. The models and retrieval mechanisms can differ, but the team’s architecture shouldn’t change with the tool. A shared set of decisions can sit underneath those environments as the source of authority, while each tool consumes guidance in its own way and the enforceable subset is checked consistently.

What I would add to the original model

I would now amend the original four-capability model in two places.

Retrieval should surface candidate decisions, while applicability determines which of those decisions actually govern the current artifact or action, using scope together with lifecycle and precedence. Then guidance and enforcement should remain separate outcomes: Some applicable decisions are precise enough to support deterministic controls, while others should influence the implementation without becoming binary rules.

That separation leaves the agent free to reason where the architecture is genuinely open while preserving authority where the team has already made a decision. It also avoids a common overcorrection. Once teams see that some architecture can be enforced automatically, the temptation is to formalize everything. A great deal of architecture is still judgment. “Keep domain boundaries narrow” or “prefer asynchronous integration when immediate consistency is unnecessary” can be valuable guidance without being safe material for an automatic block. The level of authority a system exercises should follow the precision of the decision and the evidence available to evaluate it.

For years architectural governance ran largely on human process: Someone wrote the decision record, an architect reviewed the important changes, experienced engineers carried part of the history in their heads, and code review caught some of what slipped through. AI coding agents change the economics of that process because implementation can now move faster than architectural intent can be reexplained and rereviewed by hand. Giving agents more context is necessary, but it’s not sufficient. The harder job is preserving the difference between information that may be useful, a decision that actually applies, guidance that should shape an implementation, and a constraint that is precise enough to stop one.

The goal isn’t to constrain every choice an agent makes. It’s to stop decisions the team has already made from quietly turning back into suggestions. That requires more than retrieving the right document. The system has to know whether the decision is current, where it has authority, and what kind of authority the team intended it to have.

In the final episode of Zero to Agent in 30 Minutes, AI engineer Sajal Sharma showed how to build a personal assistant on OpenClaw that tracks your tasks and notes and handles scheduled work on its own.

A useful personal assistant has to be reachable when you’re away from your computer and keep running when it has scheduled work to do. OpenClaw supports both, and you decide where the agent runs, which models it uses, and how it accesses your data.

How to go from a fresh install to scheduled automations, step-by-step

1. Install your OpenClaw instance. Sajal recommends starting locally while you learn how it works, then moving to an always-on machine or hosted environment if you need continuous availability.
2. Connect the channels you use. Add a messaging channel such as Telegram, WhatsApp, or Slack so you can reach the assistant away from your desk. Sajal used Telegram in his demo to send requests and receive notifications from the same assistant.
3. Set up the assistant’s workspace. Give the agent a place to keep tasks, notes, skills, and memory, along with guidance files that define how it behaves and how it should work with the files in that workspace.
4. Use it in your daily workflows. Sajal asked the assistant for an overview of current projects and checked progress toward a reading goal. He also had it search the web for a book series and add the books it found to his reading list.
5. Add proactive automations. Schedule work the assistant should handle on its own. Sajal added a Telegram reminder and a morning briefing designed to pull from tasks, a calendar, and recent email. Recurring checks and webhooks can also trigger the agent when something changes in another system.

Thanks for following Zero to Agent

This episode closes out Zero to Agent in 30 Minutes. Thanks for watching and building along with our presenters throughout the series.

If you’d like to keep following how agents can take on more complex work, tune in to The Monday Morning Agent, which debuts Monday, October 12, at 8am PT/11am ET. The eight-episode series follows the development of an increasingly autonomous data agent working with real play-by-play data. Learn more and save your seat or follow along here on Radar.

The following article originally appeared on the Agentic AI Foundation blog and is being reposted here with the author’s permission.

Websites are designed for people. We can look at a page and quickly understand where to enter information or which control will complete an action.

AI agents don’t see or use the web the same way we do. An agent may have to process a screenshot or inspect the page’s underlying structure, then reason about which control matches the request.

The additional context increases token use, and each interpretation step creates another chance for error.

WebMCP (not to be confused with MCP) gives web developers a standard way to publish structured tools to access the capabilities of their website enabling agents to call those functions with structured inputs.

A developer can register a tool through document.modelContext and give it a name and description. The tool also defines an input schema and the function that runs when an agent calls it.

For example, an article site could publish a search tool that calls the same application code as its visible search form.

const searchToolController = new AbortController();

await document.modelContext.registerTool(
  {
    name: "search-articles",
    title: "Search articles",
    description: "Search published articles by topic.",
    inputSchema: {
      type: "object",
      properties: {
        query: {
          type: "string",
          description: "The topic or phrase to search for."
        }
      },
      required: ["query"],
      additionalProperties: false
    },
    async execute({ query }) {
      const articles = await searchArticles(query);
      renderSearchResults(articles);
      return { articles };
    }
  },
  { signal: searchToolController.signal }
);

// Call this when the search tool should no longer be available.
// searchToolController.abort();

The name and description tell the agent when to use the tool. The input schema defines the arguments it can send. The execute function calls the site’s existing search logic and updates the page with the results.

The browser makes the registered tool available to a supported agent while the page is active. When the agent calls it, the browser passes the structured arguments to execute and returns the result.

When the search tool should no longer be available, the page can unregister it by calling searchToolController.abort().

A page registers tools that match its current state. A content view may provide a search tool, while an editor can provide tools for changing the selected content. The agent then discovers the tools available for the page’s current state.

WebMCP also proposes a declarative path for existing HTML forms. The article search could be exposed through attributes on the form itself.

<form
  action="/articles"
  method="get"
  toolname="search-articles"
  tooldescription="Search published articles by topic."
>
  <label for="article-query">Topic</label>
  <input
    id="article-query"
    name="query"
    type="search"
    toolparamdescription="The topic or phrase to search for."
    required
  >
  <button type="submit">Search</button>
</form>

The browser uses the form attributes and controls to create the tool definition. Because the form omits toolautosubmit, the browser populates the field but leaves the final click to the person. The person can review the search query and click the Search button.

Keep the person and agent in the same interface

A WebMCP tool uses the state already visible in the application and the person’s active session. The shared page gives the person a place to review the agent’s work. When the agent completes an action, the person can see the updated state and make changes through the normal interface.

Web developers can also reuse application logic that already handles the visible interface. OpenAI’s Runme example is a browser application that registers tools for reading instructions and updating notebook content. Runme can offer those functions through the page without adding a separate server only to expose an MCP endpoint.

Pair WebMCP with MCP

WebMCP and MCP both expose tools to agents, but they operate in different parts of an application. Understanding the difference helps developers decide where each one fits and how they can work together.

An MCP server can provide tools when no related page is open. A WebMCP tool is attached to the current document and can use the page’s current state.

WebMCP standardizes how a page registers tools with the browser. The browser can present those tools to its agent through MCP or another function calling mechanism.

MCP Apps address another part of the interaction. An MCP server can provide an interactive interface that a compatible host displays to a person. WebMCP lets an agent use functions from an existing website.

WebMCP and MCP Apps can work together. For example, a shopping website could publish a compare_products tool through WebMCP. An MCP server could provide an MCP App for the comparison, so the person can adjust the options while the agent handles the supporting work.

The comparison interface could also be described with A2UI, which lets an agent describe an interface in a declarative format. The client renders the description with components from its own design system, so the client controls which components are available and how they behave.

People and agents can use the same interface at the same time. Developers need to decide which functions the agent can call and what the person should see while those functions run.

Design the page for agent use

Agent use begins before the first click. An agent may need to discover the website and determine how to authenticate. It may choose an API or structured documentation before opening a browser.

Web teams should study how agents use their site before deciding which interface to add. Check the server logs to see whether agents are crawling HTML or requesting machine readable content. Tool activity can reveal which WebMCP functions agents discover and which calls they abandon.

Keep existing controls in the execution path

WebMCP reduces the amount of interface interpretation an agent has to do, but a structured tool is still an entry point into the application. The website should apply its existing access controls and input validation whenever the tool runs.

For example, an update_cart tool can accept a product identifier and quantity. The server should confirm that the product exists and that the current user can modify the cart. Structured input doesn’t automatically make the request trusted, so the server should run the same checks it uses for updates made through the website.

The browser also has a responsibility. OpenAI reviews each site tool call before it runs and asks the person to confirm sensitive actions. The WebMCP draft requires origin isolation, and its tools permissions policy allows same origin use by default.

Clients need to handle tool changes during a task. A page can change a tool’s identity or schema after the agent begins working, so the client should review the tool again before using the new definition.

Researchers are already testing those boundaries. A June preprint on WebMCP tool surface poisoning found that changes to tool registration and metadata could redirect agents in its test environment. The researchers used a polyfill and a headless agent, so a native browser implementation may behave differently. But these reported attacks can serve as great test cases as you’re adding WebMCP to your site.

Test WebMCP while the design is open

WebMCP already has several experimental implementations. Chrome offers an implementation through a flag and an origin trial. Sarah Drasner created an interactive WebMCP demo of this implementation that you can try in your browser.

Cloudflare released a developer preview that can add a WebMCP bridge to a website, and Brave has tested WebMCP with its Leo assistant.

Testing WebMCP against varied applications can reveal assumptions in the current draft. For example, a form may need a declarative feature that has not been specified yet. Also, look for lifecycle problems when a website changes its tools during a task.

Bring your findings back to the WebMCP issue tracker.

AI systems are gaining more autonomy while governments, companies, and researchers are still working out how much oversight they need. On the latest episode of This Week in AI, we covered the US debate over AI governance, new frontier models, persistent agents, world models, and practical uses for AI in healthcare and disaster response.

AI oversight is moving beyond company promises

The Trump administration announced a voluntary agreement with major AI companies that calls for internal safety monitoring, external audits, and independent board reviews. Because the agreement carries no legal enforcement, it raises a familiar question about how far self-regulation can go when companies are developing increasingly powerful systems.

Government agencies are also testing what existing law can do. The Federal Trade Commission launched an investigation into OpenAI, Anthropic, and other AI companies focused on potential consumer risks. Cases like these could help establish whether current consumer protection laws are enough or whether AI will require a more specialized regulatory framework.

For technical leaders, regulation can influence how organizations evaluate models, document risks, manage access, and choose vendors. As AI moves deeper into business processes, teams will need governance practices that can withstand outside review.

AI systems are taking on longer, more autonomous work

OpenAI just released Dots, its “proactive assistant” designed to retain context, work across applications, pursue multiple goals, and act independently. To do that work without waiting for a prompt, Dots needs standing access to the apps and data it works across. That also increases the amount of personal data an agent can reach and raises the cost of mistakes or misuse.

Google is also extending how long a model can work on a task. The company says Gemini 4 Argon can generate up to a million output tokens in a single response, far beyond the typical output limits of current frontier models. The goal is to let a model stay with long multistep work such as extensive coding or financial and legal analysis. Longer-running models and persistent agents aren’t the same thing, but both let AI finish more work without handing control back to a person.

As agents act more on their own, they need to anticipate the consequences of their actions. That becomes even more important when AI moves beyond software and begins acting in the physical world. World models aim to teach AI how these environments work, including cause and effect, which is why many researchers see them as building blocks for robotics and physical AI. World Labs (recently acquired by AMD) is developing spatial intelligence models for interactive 3D environments, while British startup Worldmodeldata has licensed nearly 1 million hours of video game data paired with player actions. Researchers from NVIDIA, MIT, and Oxford also introduced Physis-Lang, which uses descriptions of physical causes and effects to help video models learn why events happen. Researchers still don’t know which training approach will work best, so they’re testing several kinds of data and model design.

AI support experts in high-stakes work

Anthropic’s Claude is helping teams responding to an Ebola outbreak in the Democratic Republic of the Congo organize daily reports, compare forecasting models, analyze genomic information, and support vaccine research. Researchers have also developed a Spanish-language speech model that may eventually help identify accelerated biological aging and early signs of dementia. Mayo Clinic researchers built a model that identifies patterns associated with elevated pancreatic cancer risk years before diagnosis. Following severe flooding in Nepal, local teams used AI to match reports of missing people with victim lists and map damaged buildings using satellite imagery.

Together, these examples show how AI can be used for good by helping people work through complex information faster and, ultimately, save lives, while keeping qualified experts responsible for the final decisions.

What’s next

As AI takes on more work and enters higher-stakes settings, organizations will need clearer answers about access and accountability. They’ll also need to decide where human judgment remains necessary as systems become more capable.

Join us again next Monday for another episode of This Week in AI, when we’ll dive into more of the news, issues, and key developments shaping the AI era. And check back each Friday for the latest episode, or watch on YouTube, Spotify, Apple, or wherever you get your podcasts.

Today, because the agent is interacting on behalf of a human through a genuine browser with standard extensions, its technical fingerprint looks identical to a normal human user’s device. Agents call tools, observe results, and revise. A person browses, hands off to an agent, and takes the session back. All the activity appears to be coming from the same user.

Signed agents help the identity problem, and they’re arriving. Web Bot Auth is a thin layer over RFC 9421 HTTP Message Signatures where an agent declares its identity and purpose. But they only help with the identity problem: The signature proves who is making the request, not the intent behind it. Prompt injection and other techniques can be used to steer a verified, well-behaved agent into acting against your interest without its operator ever knowing.

The organizations getting this right are treating agent policy as a commercial question with a security implementation. The following questions need to be answered:

• Which agents do we allow to use our service?
• What endpoints do we need to protect? 
• How will agent management security policies impact revenue?

Here are the techniques successful companies are using to secure in the agentic era. A phased strategic roadmap implementing continuous execution-time verification. These are the steps:

1. Establish execution-time machine identity and scoped delegation. Replace static, long-lived API keys with machine single sign-on mechanisms that issue short-lived, context-bound credentials at the time of tool execution. Enforce strict delegation policies so that autonomous agents operate within verified “green zones” without inheriting broad human privileges.
2. Deploy cryptographic ingress verification. Upgrade edge infrastructure to inspect Web Bot Auth HTTP Message Signatures, in order to separate cryptographically verified crawlers from anonymous traffic.
3. Upgrade edge defenses to browser-layer intent classification. Transition legacy network-layer antibot web application firewall (WAF) to client-side browser-layer detection platforms. 
4. Harden agent execution pipelines against prompt injection. Instrument robust input sanitization, instruction hierarchy enforcement, and design-level safeguards around all LLM ingestion channels to neutralize indirect prompt injection attacks before external web content reaches execution tools.
5. Adopt standardization for agentic commerce and admission. Integrate emerging trust frameworks such as the Framework for Agentic Commerce Trust (FACT), a neutral real-time trust layer for AI-driven transactions in addition to verification protocols like CAPTCHA to validate agentic capability vectors before delegating sensitive interagent tasks.

The rapid transition from human-directed software to autonomous AI agents breaks traditional enterprise security paradigms. AI agents operate with broad operational scope but nondeterministic, probabilistic reasoning. This is causing static perimeter defenses to fail. The teams that come through this well will be the ones that stopped asking “Who’s on the other end of the connection?” and started asking “What’s this request for, and what happens if it succeeds?”

Imagine an AI-assisted decision that must be examined six months after it was made. Rerunning the system may produce a different answer, but an investigator may still need to determine which model endpoint, policy, retrieved data, tool contract, and runtime configuration were used at the time. That is the problem addressed here.

The AI industry remains much better at evaluating capability than at preserving the knowledge required to understand past executions. Evaluation pipelines track benchmark scores, task success rates, retrieval precision and recall, tool-call success rates, latency, and cost, and they grow more elaborate with every release. In practice, this machinery often reduces to one acceptance question: What can this system do? That question captures capability, but not whether a past execution will remain understandable after the system changes.

That missing property is historical reconstructability: the ability to establish which dependencies were used and under which conditions a particular execution occurred. It is related to, but distinct from, reproducibility. Reproducibility asks whether a result can be produced again under equivalent conditions. Reconstructability asks whether the conditions of the original execution can still be identified, even when repeating that execution would not produce an identical result.

This distinction matters when a consequential AI execution completes successfully today and is challenged six months later. The model may still exist in a registry. The application revision may still exist in Git. The trace may still identify the request and the services it crossed. Yet none of these records necessarily reveals which model endpoint served the request, which input transformation and runtime configuration were applied, which data, feature state, or retrieved context entered the computation, or which policy state governed it. In an agentic system, the missing history may also include memory, tool contracts, delegation, and external effects. Even when a dependency identifier was recorded, the historical state to which it referred may no longer be available or resolvable.

The problem is that an AI runtime may use dependencies whose historical identities and relationships are not preserved. This is an architectural problem, not merely a logging problem. The gap lies between what the runtime uses and what the surrounding infrastructure is required to preserve. I call this the preservation gap. The AI stack preserves models, code, deployments, and observations, but it does not require the effective historical configuration of a particular execution to remain available. The components may remain while the relations that made them one execution disappear. An AI runtime must assemble today’s execution; it is not necessarily required to remember exactly what it assembled yesterday. Reconstructability must therefore be established during execution, before later mutation erases the bindings that gave the execution its effective configuration.

Hermetic builds illustrate the opposite design pattern. They close the dependency graph before execution begins by pinning inputs, versions, and external dependencies. AI runtimes face the reverse situation: Part of the dependency graph remains open until execution, as model endpoints may resolve through mutable aliases, data and feature state evolve, runtime configuration and policy change, and retrieved context is selected only when a request is processed. Agentic systems extend the graph further through memory, tools, delegation, and external effects. In a hermetic build, dependency closure is an input. In a modern AI runtime, part of it may be an output of the run.

Version control records source evolution, transaction logs record state transitions, and distributed tracing records execution relationships. These mechanisms preserve different views of a run but not its materially relevant execution-specific configuration. A trace may retain execution flow, a deployment manifest the declared application state, and a model registry the model revision. What is often missing is the record of which artifacts and states were bound together in a particular execution.

The dependency graph no longer closes at deployment

A deployment records the configuration known before execution begins. AI systems are harder to reconstruct because materially relevant dependencies are selected during execution rather than fixed at deployment. At inference time, the system may resolve a model endpoint, input transformation, data or feature state, retrieved context, runtime configuration, safety controls, and policy. Agentic systems extend this composition through memory, tools, delegation, and external effects. These execution-specific selections are runtime bindings: facts about what the execution actually used.

The following example makes this distinction concrete. Consider a hypothetical agentic execution identified as E891. During this execution, the system retrieves two documents, evaluates a policy, invokes a tool, and produces an external effect. The deployment may identify application revision 8f31c2 and model v17, while the execution itself binds prompt h31, retrieval index r42, entities d182@17 and d761@4, tool contract h42, policy v8, and ultimately effect e3. The deployment system could not have known this entire set in advance, because some of these dependencies were selected only as the execution proceeded.

The difference between what a deployment declared and what an execution actually used becomes consequential when dependencies change. A model alias may resolve differently, a feature or data source may change, preprocessing and runtime configuration may evolve, and policy v8 may become v9. In agentic systems, retrieved context, memory, and tool contracts introduce further independent mutation. Deployment identity describes what was declared, but the historical record of an execution must describe what was actually used. Unless those binding relations are captured when they occur, the effective configuration of E891 cannot be recovered reliably from later system state.

Current state is a lossy projection

Once runtime bindings are treated as part of the execution, the limitation of current-state records becomes clear. The execution-scoped dependency closure is the combination of the declared state and the runtime bindings that formed the effective configuration of a particular execution.

As dependencies mutate, different historical executions can leave behind the same observable evidence. The current state may show which artifacts still exist without revealing which combination a particular execution used. Once the distinguishing bindings are gone, the history cannot be reconstructed from what remains. A system can retain its artifacts while losing reconstructability: An identifier proves neither that a dependency was used nor that the historical state it denotes remains available. Reconstructability therefore requires durable binding relations and continued access to the historical states they identify.

Existence is not usage

The existence of an artifact and its use in a particular execution are different facts. Concurrent versions, caches, retries, and asynchronous changes make it unreliable to infer whether an execution used an artifact merely because that artifact existed at the relevant time.

W3C PROV already provides the relevant semantic distinction: An activity can use an entity. The missing primitive is therefore not a new provenance vocabulary, but a runtime requirement to record which dependency a particular execution used and to preserve an identity that can be resolved later.

This requirement also exposes the limit of observability. OpenTelemetry can transport dependency identifiers through attributes, links, context, and baggage, but encoding an identifier does not preserve its historical meaning. A trace may retain execution topology while the identities that explain it disappear. Instrumentation can carry preservation metadata, but it cannot guarantee durable storage or continued resolution of the referenced states.

Historical reconstructability therefore requires more than retained artifacts. For a defined class of executions and a specified retention period, the system must preserve two properties: binding integrity, which records which dependency versions and states an execution actually used, and resolution integrity, which keeps those recorded identities resolvable to the historical states they denote.

Preservation requires failure semantics

Once reconstructability is treated as a system property, the architecture must define what happens when the records required to reconstruct an execution fail to reach durable storage. Suppose policy v8 authorizes execution E891 to invoke tool contract h42, and the tool commits effect e3. If the usage record fails to reach durable storage, the action succeeds while the historical links among the effect, its policy, the tool contract, and the execution context are lost. The external effect remains, but the historical record no longer reliably explains which policy and tool contract authorized it. The external world and the historical record have diverged.

Figure 2 shows why this condition cannot be treated as ordinary telemetry loss. When an external effect has been committed but the corresponding usage record is missing, the architecture does not guarantee historical reconstructability. It provides only best-effort historical evidence.

Reconstructability becomes an engineering guarantee only when the architecture defines when preservation records become durable and what the system does if they cannot be stored.

A system may require a durable usage record before committing the effect, atomically persist the effect intent and preservation record through a transactional outbox, quarantine the execution for reconciliation, or trigger a compensating action. The implementation is application-specific, but preservation failure must have explicit semantics rather than disappear into telemetry.

Existing technologies provide the building blocks for this layer. Provenance and lineage models represent relations, tracing propagates execution context, registries identify versions, and content-addressed stores preserve artifacts. None, however, creates a runtime obligation to preserve the execution-scoped dependency closure required for reconstruction.

This layer is a preservation plane: the contract and machinery that keep an execution’s dependency closure identifiable as its surrounding systems change. It defines what to record, how records become durable, and how referenced states remain resolvable. Orrery applies this principle as a reference architecture in which materially relevant runtime bindings are captured when they occur and retained together with resolvable identities for the historical states they reference.

AIGov Core: reconstructability as a tested property

AIGov Core provides a limited implementation of this principle. Its evidence ledger stores hash-chained events, while its replay engine reconstructs a governance verdict from a stored export without querying live state. A companion check labels the export Ready, Partial, or NotReady according to unresolved lineage, missing policy artifacts, version drift, or replay failure. This demonstrates that reconstructability can be tested, but it does not solve the complete preservation problem. The system can replay only what was recorded; it cannot recover a runtime binding that was never captured.

The preservation gap is the mismatch between what an execution depends on and what the system persists. Closing it requires a runtime obligation: Whenever an execution binds materially relevant state, the system must record that binding, preserve a durable identity for the dependency, and retain access to the historical state it denotes for the defined retention period. Reconstructability is therefore not a by-product of capability or observability. It is an architectural property that must be designed, tested, and enforced during execution.