For three years, the agent story has centered on capability: Models got better at planning, tool use, and long tasks. Integration improved more slowly. Every agent was still wired to its tools, data sources, and host application with custom glue, leaving even successful systems difficult to move or reuse. Standard interfaces change the economics of that work. Networking became an ecosystem when machines could agree on interfaces; programming tools followed the same path with the Language Server Protocol. In my judgment, the standardization now underway will influence what engineering teams ship over the next three years as much as further gains in model capability.

The Model Context Protocol (MCP) standardizes how an AI application connects to tools and context. It uses JSON-RPC messages between hosts, clients, and servers. Servers can expose tools that a model invokes, resources containing context and data, and prompts that describe templated workflows. The spec credits the Language Server Protocol as an inspiration. Language servers replaced a separate integration for every editor-language pair with one interface on each side; MCP applies the same idea to AI applications and integrations. The current spec is dated July 28, 2026, the fifth dated protocol revision since the project launched in November 2024. That pace reflects active governance and warns implementers to expect movement.

Adoption has spread across competing hosts. The MCP maintainers’ December 2025 announcement cited more than 97 million monthly SDK downloads, over 10,000 active servers, and first-class client support across ChatGPT, Claude, Cursor, Gemini, Microsoft Copilot, and Visual Studio Code. Any individual count can be debated. Cross-vendor implementation is harder to dismiss, because a shared interface becomes infrastructure when rivals support it independently.

MCP covers the boundary between an application and its tools. The Agent2Agent protocol (A2A) addresses the next boundary: agents discovering and communicating with one another across vendors. It reached v1.0 in April 2026 with more than 150 supporting organizations, signed Agent Cards for verifiable identity, SDKs in five languages, and general availability in Microsoft Copilot Studio, Azure AI Foundry, and Amazon Bedrock AgentCore. A2A is younger and less proven in production than MCP, but its governance and adoption are moving agent-to-agent communication beyond the confines of a single platform.

The emerging stack extends beyond those two protocols. AGENTS.md, OpenAI’s convention for giving coding agents repository-level instructions, joined MCP as a founding project of the Agentic AI Foundation. The OpenTelemetry generative AI semantic conventions are developing a shared vocabulary for traces and metrics from model and tool calls, though that work has yet to reach stable status. Connection, cooperation, instruction, and telemetry are beginning to acquire common interfaces.

For an engineering organization, the immediate consequence is reuse. An MCP server for an internal ticketing system can serve every compatible IDE, chat application, and agent. Models and hosts will keep changing; the interface contract, server implementation, schemas, and authorization wiring can endure. The integration becomes an asset shared across applications. Language servers followed the same path as editors came and went around them. Agent integrations deserve the same treatment as libraries and services, with owners, versions, tests, and upgrade policies.

Reuse also concentrates attention on a common security boundary. Custom integrations tend to receive separate reviews, when they receive them at all. A shared protocol creates a recognizable seam where teams can specify identity, permissions, data flow, and audit behavior, then apply those controls across many tools and hosts. The MCP authorization specification builds that seam from established IETF work, including OAuth 2.1, protected resource metadata, authorization server metadata, and resource indicators. It requires Proof Key for Code Exchange (PKCE) to protect authorization codes from interception. MCP servers must also reject tokens issued for another audience and must not forward them downstream. The value comes from familiarity: Teams can draw on years of OAuth deployment experience instead of rediscovering its failure modes inside a new protocol.

The spec is equally useful when it states what metadata cannot prove. Tools can carry behavioral annotations describing them as read-only, destructive, idempotent, or open-world. Those terms give hosts a vocabulary for policy. The tools specification still requires clients to treat annotations as untrusted unless they come from a trusted server. Self-description can inform a security decision; it cannot establish the trust on which that decision rests.

Once servers share an interface, they also become discoverable. The MCP Registry is an open catalog and API for public servers, designed to feed downstream marketplaces and private catalogs. Its role resembles npm or PyPI as a discovery mechanism, with an important difference: It stores standardized server metadata and leaves package distribution elsewhere. The registry remains in preview, with possible breaking changes and no durability guarantee.

Discovery brings familiar supply-chain risks to components with unusually powerful access. Typosquatting, abandoned packages, malicious updates, and uncertain provenance now concern software that may hold live credentials and act on production systems. The registry provides namespace verification and moderation, and its downstream model allows organizations to build curated catalogs. An internal subregistry or allowlist is therefore a sensible first control. The standard interface makes that curation practical across multiple hosts.

The same contract improves testing. MCP tools declare a JSON Schema for their inputs and may declare one for structured outputs. Under the current tools specification, servers that declare an output schema must return conforming structured results, and clients should validate them. Teams can test a server without putting a model in the loop, mock it with recorded or synthetic behavior, and contract-test both sides as they would a REST or gRPC boundary.

This separates two kinds of uncertainty that agent evaluations often mix together. Protocol tests can determine whether an integration exchanged valid messages and enforced its contract. Model evaluation can focus on whether the agent chose the right tool and interpreted the result well. OpenTelemetry’s emerging conventions extend this approach to runtime evidence by giving systems a common language for tool calls. Teams will still have to absorb changes as those conventions mature.

These benefits stop at the edge of what the interfaces describe. A schema captures the shape of a tool’s arguments, while its meaning still lives largely in a free-form description that a model must interpret. A schema cannot tell an agent when a tool is appropriate, how its effects interact with other tools, or whether two similarly named operations have equivalent semantics. Portability makes a server available across hosts; behavior can still vary across models and contexts.

The semantic gap leads to a trust gap. MCP can carry a server’s claims, and an A2A Agent Card can carry a digital signature, but a signature only ties a statement to an identity. Engineering organizations still need a basis for deciding which identities, publishers, code, and claims deserve authority. The protocol can carry that decision through a system. It does not make the decision for the organization.

Delegation makes the problem harder. MCP’s authorization model handles a client calling a server with an audience-bound token. Production systems increasingly involve an agent calling another agent, which calls a tool or a third agent. Preventing token passthrough closes a serious hole, yet each downstream hop still needs a narrower grant derived from the user’s original authority. No common mechanism defines how those rights should attenuate across an arbitrary chain. Platforms currently solve this locally or leave too much authority in place.

All of this work is unfolding on young infrastructure. The registry is in preview, the telemetry conventions are unstable, and MCP has produced five dated protocol revisions in less than two years. Revision is how standards mature, so teams should version-pin, keep protocol code behind thin internal adapters, and budget for migrations. Some abstractions will prove wrong because the standards and the underlying practice are developing at the same time.

A sensible response begins with ownership and containment. Each internal server needs a durable owner, and its protocol surface should be treated as an architecture decision. Schema validation, conformance tests, and protocol mocks put that seam under contract. Third-party servers belong behind an allowlist or private catalog, with provenance requirements and corroboration for their annotations. An internal abstraction around the emerging telemetry vocabulary can limit exposure to draft revisions.

Organizations with a large stake in agents should also participate in the standards work. The Agentic AI Foundation and the A2A project are young enough that engineering teams outside the founding companies can still influence what becomes portable, observable, and enforceable. The rules written now will become assumptions embedded in future products.

Model improvements will continue to generate the headlines. Standards determine whether those models can participate in an ecosystem. Ethernet mattered because it became an interface that many implementers could agree on, and language servers mattered because editors and language tools no longer needed to be designed in pairs. Agent systems now have their first interfaces that competitors jointly govern and independently implement. Engineering teams should build on them with two expectations: The interfaces are likely to last, and their current forms will change.

The emergency route itself was fine. Production systems need emergency routes, and a human approved this one. The trouble is that the route now runs without a live incident, an owner, or an expiry condition. A controlled exception has quietly settled into the platform’s standard runtime behavior.

This article is about that execution layer: the temporary operating state that lingers in runtime behavior. Temporary authority is easy to grant under pressure and hard to retire once the pressure lifts. An enterprise agent shouldn’t have to guess from prompts or conversation history whether it’s running under normal, incident, or recovery conditions. That state should be handed to it as authoritative runtime input, the way platforms already hand over identity, tenant, environment, and permissions.

An exception is safe while the incident that justified it is live. It becomes dangerous the moment the platform can no longer show the incident has ended.

The problem is unmanaged exception state

Exception drift is what happens when temporary exception behavior outlives its authorized scope, authority, or duration, and emergency accommodations settle into normal execution. The drift is usually quiet: a routing rule that stays reachable, an approval shortcut that survives closure, a tool permission that keeps shaping execution after the triggering condition has passed.

Enterprises already have the human machinery for this. Incident management defines abnormal conditions, change control governs deviations from standard practice, and postincident reviews confirm that temporary measures have closed.¹,² The gap is architectural. Most agent platforms still treat organizational operating state as something outside the runtime rather than an input to it.

Once an accommodation proves useful, it fades into an invisible operating state. The routing rule stays enabled, the shortened approval path stays reachable, and the temporary queue keeps taking work. No dramatic model failure is required, only a platform with no reliable way to close runtime state.

The practical question to ask is “Which operating mode is active for this workflow, this user, this segment, at this moment?” When the answer is left implicit, every agent, workflow, and tool gateway invents its own. Discussions of agent architecture tend to dwell on capabilities such as models, tools, and orchestration, yet production behavior depends just as much on runtime context: identity, tenant, environment, permissions, session state, policy, and tool access. One piece of that context is usually absent. The runtime knows who is acting and what they may do but not whether the organization is under normal conditions, incident response, recovery review, or a declared exception. Exception drift begins in that blind spot.

The exception lifecycle

Organizations move through a predictable sequence: normal operations, incident declaration, a temporary exception window, closure review, and return to normal. Each stage answers a question, from why the exception exists and who authorized it to what evidence shows it’s no longer shaping execution. Most enterprises handle the front of that sequence well; they’re practiced at declaring incidents and authorizing workarounds. The hard part is the retirement: proving that the exception behavior actually disappeared.

Declaring an exception is loud. Retiring one is quiet, especially when the workaround improved throughput or helped the team recover faster. That asymmetry is where drift lives, because an incident can be closed on paper while emergency routing, override policies, or alternate workflows keep influencing execution. From the platform’s point of view, the lifecycle should close only when it can show that every exception path has been retired or formally adopted through change governance.

Why this matters more for AI agents

Agents raise the stakes because they act. Rather than sitting in a config file, they select tools, trigger workflows, coordinate with other agents, and adapt their paths at runtime. An accommodation introduced during an incident can spread through routing, tool use, approval paths, and downstream agents. A traditional exception stays legible in a runbook or workflow definition; an agent can carry the same exception along many paths at once, which makes it harder to find and retire.

For engineers, the missing layer is an authoritative operational-state context that arrives with the request rather than being inferred from it. Historical traces and retained memory can explain why an accommodation once existed. They should never decide whether it’s still authorized. Memory informs execution; operating mode governs it. And when the two disagree, authoritative runtime state wins.

A small example shows the shape of that context. It carries the minimum needed to bind an exception to a mode, a scope, an authority, an expiry, and a status:

{
  "mode": "incident",
  "exception_id": "INC-4721",
  "scope": {
    "segment": "premium-customers",
    "region": "us-east",
    "workflow": "customer-remediation"
  },
  "authority": "service-owner",
  "expires": "incident-close",
  "status": "active"
}

The pattern sits near familiar ones without matching them. Feature-flag platforms such as LaunchDarkly target behavior by context, RBAC governs what a principal may do, and tenancy metadata tells a service where a request belongs.³ Operating mode serves a different purpose. It doesn’t replace policy, permissions, or memory: Permissions determine who may act, and policies determine how they may act. Operating mode determines whether exception behavior is authorized at all. As a result, it acts as a higher-order governance constraint on agents, workflows, approvals, tools, and escalation paths. Operating mode draws its authority from the organization’s incident and change process and represents a governed state of the enterprise, consumed by the runtime.

A design pattern for exception-aware agents

Operating mode becomes actionable once the platform treats it as a first-class runtime construct. Most agent architectures already inject identity, permissions, tenant context, and policy into every request, and operating mode belongs in that set. Agents consume it as authoritative state after the organization declares the exception, scopes it, assigns authority, and sets an expiry, instead of reconstructing it from prompts or accumulated context.

That reframes the architectural question as whether the platform can guarantee that exception behavior is impossible outside an authorized operating mode. Emergency behavior exists because the platform enables it, and for no other reason. Table 1 shows the minimum contract that makes the boundary testable.

A workflow invocation receives that state alongside the user request, and orchestration, routing, and tool gateways read the same state. Figure 1 shows the shape.

Building an exception-aware control plane

The natural home for operating mode is an external control plane. Incident management platforms, maintenance window services, and change management workflows already hold authoritative operational state, and exception-aware architectures extend those signals into execution with explicit scope, authority, expiry, and closure semantics.⁴,⁵ Implementation will vary across organizations, but the principle holds: Exception state should be authoritative, observable, and externally managed, kept out of prompts and workflow definitions and away from agent memory.

Exception-aware execution

With explicit operating state, behavior shifts when system state shifts, and prompt wording stops being the lever. Under normal operations, agents run standard workflows, routing, and approvals. Under incident mode, the same workflows can expose scoped accommodations that the authorized mode makes available, such as expedited approvals, alternate routing, deferred reviews, or emergency runbooks.

Table 2’s payoff is testability. A workflow in normal mode should never reach an emergency path, and a workflow in incident mode should reach only the accommodations its scope, authority, and expiry allow. Governance becomes an enforceable runtime property the platform can check at execution time.

Return to the opening incident. With operating mode as runtime state, the emergency route opens only inside a declared exception: scoped to the affected segment and workflow, owned by the service owner, and stamped with an expiry tied to incident closure. When the incident closes, the mode returns to normal and the routing gate stops handing that route to the segment, ahead of any manual cleanup. Closure then runs as a check. The platform replays the exception’s scope against live routing, approval, tool, and queue configuration, and confirms that no path still resolves to the emergency behavior. No one has to remember to retire the route; it was bounded by state, and the platform can show it is gone.

A common operating model for multi-agent systems

The problem compounds across collaborating agents. Customer-facing, orchestration, and execution agents may share a workflow while disagreeing about state, so one keeps applying emergency routing after another has returned to standard controls. A shared operating state gives them a single governance boundary: The exception is represented once and read consistently everywhere. As agent ecosystems grow more autonomous, shared operational state matters as much as shared identity and authorization. Fragmented state produces fragmented accountability.

Making exception drift observable

The payoff is observability. These conditions have historically been hard to see because accommodations scatter across workflow definitions, approval policies, routing configurations, and tool permissions. Explicit operating state makes them measurable at runtime. Closure can trigger automated validation, and the platform can watch for residual exception behavior between closures. The revealing checks are direct: Is the exception’s routing path still reachable, do its temporary approvals or elevated permissions still resolve, and does any expired exception still touch behavior? Tracking how many exceptions are open, how long they stay open, and how often they harden into permanent change turns drift from an audit finding into a monitored signal.

From adaptive agents to governed systems

Table 3 captures the shift. Traditional governance documents exceptions and trusts that they are retired. An exception-aware platform represents them as runtime state that can be propagated, validated, monitored, and closed.

The test is whether the platform can demonstrate that the exception stayed bounded to its purpose and duration.

None of this demands a new governance model. Organizations already know how to govern identity, permissions, change, and operational risk, and the recommendation is to extend those disciplines to operating state. Once operating mode is authoritative, observable, and testable, drift becomes detectable, closure verifiable, and recovery an engineering result. The aim is a platform that can make its own operating conditions explicit, hold every agent to them, and demonstrate a clean return to normal. In that model, temporary accommodations stay temporary, governance lives in the architecture, and enterprise agents earn trust at scale.

References

1. ISO, “ISO 22301:2019: Security and resilience—Business continuity management systems—Requirements,” International Organization for Standardization, 2019. https://www.iso.org/standard/75106.html ︎
2. Axelos, “ITIL 4: Change enablement,” IT service management practice, 2019. https://www.axelos.com ︎
3. LaunchDarkly, “Target with Flags,” LaunchDarkly Documentation. https://launchdarkly.com/docs/home/flags/target ︎
4. ServiceNow, “Change Management,” ServiceNow Product Documentation. https://docs.servicenow.com ︎
5. PagerDuty, “Maintenance Windows,” PagerDuty Support Documentation. https://support.pagerduty.com/main/docs/maintenance-windows ︎

For most of the history of technical publishing, editing and verification were the same process, and that process was slow and expensive. Getting a book out took years. We found an author, vetted them, had them work with an editor, and checked their claims with technical reviewers. A lot of that time went into separating what was correct and useful from what was confusing or only sounded right. It was laborious, but it meant a reader could depend on the claims on the page. The credibility of the book, and of the publisher behind it, mattered as much as the information itself.

When the cost of production drops to zero, that credibility becomes worth more, not less. Content is easier to make than ever, but without a transparent process behind it, readers have no idea where the knowledge came from or whether it holds up. As Jasmine Sun puts it in “The Independent Writer’s Advantage in the Age of AI,” “Trust is not about information and its quality alone. It’s about the messenger. It’s about who says it and their track record and what they’ve told me before.” A practitioner has confidence in a source because someone she respects has put their reputation on the line for it. They believe what the author is saying because the publisher has a history of being right and of correcting itself when it isn’t, and because the work is attributed and verifiable.

The corpus matters, but it’s the assurances around it that are hard to replicate, and that comes not just from the people who produce the content but from the people whose judgment vouches for it. Sometimes a creator brings their own credibility with them. Other times, the publisher spots someone unknown and lends them its own. The art critic Dave Hickey said this about gallery owners in Air Guitar: They gain status from the famous artists they represent and share it with emerging talent who have something to offer but who haven’t had the chance to earn a reputation. This is what O’Reilly has done for nearly half a century, build a network of experts who vouch for what’s worth knowing.

Expertise is alive, and it compounds

Expertise is a living thing, continuously expanding. Content starts to decay the moment it’s published, because frameworks evolve, libraries deprecate, and yesterday’s best practice becomes today’s security incident. Keeping expertise alive requires a pipeline of people who stay current and an editorial layer that notices when something has gone stale, and either retires it or calls for a fix.

That pipeline isn’t something you switch on when an author has a book to ship. At O’Reilly, we’ve always prided ourselves on living at the bleeding edge, finding what Tim O’Reilly calls “the alpha geeks” and spreading what they know to everyone else. Content sits at the center of our platform, but we think about it in pace layers. Some advice is timeless, some moves but has a long shelf life (some of our books are still in print after nearly 50 years!), and some changes weekly. We work with experts at each pace layer, capturing what lasts while doing our best to keep pace with an industry that seems to have changed every time we wake up. We have relationships with hundreds of the best practitioners in the world, and our job is to keep them engaged continuously, with quick takes when something breaks, structured responses when major research drops, and live sessions on emerging topics while they’re still emerging.

An institution doesn’t stamp trust onto content. In a technical community, trust is conferred in both directions. A practitioner earns standing because people who already have standing engage with her work, cite it, argue with it, and build on it. That insight was the whole idea behind PageRank, Google’s first great innovation. A page mattered because other pages that mattered linked to it. Reputation works the same way.

The audience isn’t just consuming reputation signals; it’s generating them. When a senior engineer whose judgment others respect says out loud that something is worth reading, she spends a little of her own credibility; the author gains a little; and everyone watching recalibrates whom to trust next time. O’Reilly plugs into that existing economy of reputation. When we put our mark on someone’s work, we aren’t the sole source of its credibility. We’re amplifying a judgment the community is already making and adding our own track record to it. The reader who finds it reliable hands status back to the source.

When the readers are machines

Human practitioners aren’t the only ones who need trusted engineering knowledge. The AI systems now sitting in every workflow, the coding and debugging agents and architecture advisors, need it just as badly since most of them are built on scraped web data and documentation that was stale before it was ever indexed. They’re fluent, but they’re wrong often enough that you can’t just take their word for it.

The stakes grow with AI increasingly being used to generate not just provably correct types of content like code, which either works or it doesn’t, but persuasive documents in fuzzier areas like hiring, strategy, and so on. Like everyone else leaning on these tools, we at O’Reilly are reckoning with the consequences of the ability to talk to a model and get back something that looks smart at a glance. A few rounds in, the slop is still there. In the last few months, maybe 10 times as many documents have crossed our desks, from new product ideas to strategic plans and proposals. But the ease of generating the text hides the fact that either the model or the person prompting it doesn’t actually know what they’re talking about. Knowledge workers need ways to ground their work in insights from human experts, particularly when that work is AI-assisted. So we’re building tools that let agents draw on our repository of expertise to support their proposed decisions.

Credible sources are particularly important when thinking through and justifying important choices. Our CTO, Andrew Odewahn, describes the shift this way: “18 months ago, it was all about how to get engineers to be more productive, but now it’s about how to get organizations to make better decisions. The engineering tasks are moving away from coding output to planning.” For planning tasks like comparing implementation approaches, you need expert-over-your-shoulder guidance for contextual decision-making. You can’t just rely on an LLM’s best guess to solve your problem, which is why we see great opportunity for new products like O’Reilly’s Expert Intelligence offering that delivers grounded knowledge embedded in your AI tools and your workflows to help navigate what you do. Trust is foundational because the expertise behind it stays genuine, practical, and human.

About the Generative AI in the Real World podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In Generative AI in the Real World, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.

Check out other episodes of this podcast on the O’Reilly learning platform or follow us on YouTube, Spotify, Apple, or wherever you get your podcasts.

Takeaways

01.26 The usability gap is smaller than the marketing gap. The capabilities of local models are only a few months behind those from the big commercial companies, but because there’s no subscription revenue model behind local models, they often go unpromoted. “It’s very hard to make money off local models,” Pete explains, so the big companies aren’t focused on selling them. “Every company is going to go for the [product] that has an easy subscription revenue model. And that means you have a massive ton of marketing around all of these tools that are kind of like, ‘Oh, let’s have a little text box on a website.’ And so it means mostly that people have never heard of these local models.”

04.20 Local models are already good enough for most use cases. Pete compares the moment to the early web, when free alternatives like Apache eventually overtook expensive commercial servers. “All of these alternatives, once people actually had time to look around and they had a little bit of time to improve, they just wiped the floor with the commercial [offerings],” he points out. “I don’t know if we’re going to quite get there, but that’s the kind of pattern that I’m seeing.”

07.26 “The hardware barriers are a lot lower than people think.” Ben and Pete discuss what hardware you actually need to get up and running, from parameter counts, quantization (Q4, 8-bit), and VRAM requirements to the new Apple M5 Studio’s unified memory as a way to run very large models locally at usable speed. “The key thing is whether you can fit [your model] into your graphics card’s memory,” Pete says. “So with weight quantization, 9 billion [parameters] if it was 8 bits is like 9 GB. A lot of mid-end decent laptops that are shipping now have more than that.”

18.33 “It’s not that people don’t like voice interfaces. It’s that people don’t like bad voice interfaces.” We’ve solved most of the big problems, like dealing with background noise, phrasing, and speech in a range of accents—or at least have improved tools’ capabilities. However, “there’s no commercial incentive to kind of pull them all together,” Pete says. Most tools feel like they haven’t caught up to the LLM era, but “open source can be a really strong lever” to updating them, argues Pete.

28.26 We’re navigating the split between “LLM maximalist” end-to-end models (favored by big AI companies with the most capital) and the “compound AI” approach of chaining together specialized models from different sources. “If the future is end-to-end models, then only the people with the most money can actually build and train them,” Pete notes. Compound AI lets you “actually train all of the models independently” to accomplish your particular goals. While the performance of end-to-end models continues to improve, especially for multimodal models like Qwen or Gemma, using one can be a bit like choosing a Swiss Army knife over a tool specially designed to accomplish a single specific task, to use Pete’s metaphor. It may get the job done, but it’s probably not the most effective way to do it.

36.10 Voice capabilities in the browser could be a game changer. Embedding a model directly in the browser—Chrome has a built-in ~4B parameter model that’s accessible from any website via JavaScript, for instance—makes it part of the operating system. “Once you are able to transcribe fast and accurately in the browser, it’s a way for people to easily start experimenting with this stuff,” Pete explains. Could this be an iPhone moment for LLMs?

39:58 The “gravitational pull” is toward on-prem. Unlike most recent technological advances that depend on the cloud to function, LLMs are well-suited to running locally, even with no internet connectivity. Enterprises are grappling with concerns about cost, privacy, capabilities changing with no notice, or even the models they depend on disappearing. Hosting your own model, whether on your laptop or in your corporate infrastructure, gives you the stability to plan for the long term.

44:21 GPUs are fantastic for training but “complete overkill for inference.” Pete likens it to “trying to use an oil tanker to go and do your shopping.” Memory bandwidth is the real limiting factor, and it’s a problem that companies like Apple, with its new chip designs and unified memory bandwidth, are working on solving. “Even if you’re running on the CPU, if you have something that’s got high-enough bandwidth to pull 27 billion weights in a fraction of a second, then the rest of it is fairly easy in terms of actually doing the processing,” Pete says. “I think we’re going to see a lot of really imaginative solutions now that people understand what the workload looks like.”

The following article originally appeared on Addy Osmani’s blog and is being republished here with the author’s permission.

In the past year, the conversation around agentic engineering has moved to harnesses and loops, fleets and software factories. My 2 cents is engineers need to own the outer loop—the accountability for these systems. This only gets more true as powerful models like Fable and GPT-5.6 become available.

Agents have leverage, and leverage creates obligations. Someone must be able to explain exactly what changed, why it was safe, and what will happen if they’re wrong. Otherwise, their actions can’t be justified. Which makes it unlikely their organization will ask for them in the first place.

And so I want to talk about three terms. The first, Quality, refers to all the checks we install before we let the system loose. Those checks produce evidence, and from that evidence we derive a Verdict.

The second, Verdict, refers to the final decision we make before work enters our dependent system: I’m the line-producer of this content. I run the team whose work is shipped under my name. The model may write the line, but the Verdict is mine. The work of my team will not enter our dependent systems without my decision. A Verdict is the production decision: Should we ship, block, redirect, narrow the response, add a guardrail, or reject outright?

The third, Answerability, refers to the guarantee that if someone asks, I can explain why.

To say this another way: Our agent (which I define as a model plus a harness of files, tools, memory, skills, sandboxes, permissions, observability, and recovery) is what runs our loop (which I define as investigation, implementation, verification, and repeat). And it’s what creates our software factory.

The model is just the engine. The harness—tools, memory, permissions, sandboxes, tests—is the car you build around it so it can do real work safely.

The loop is how one good run becomes a process you can trust to run again. Wrap that harness in a repeatable cycle—investigate, implement, verify, repeat—where an independent check, not the model’s own say-so, decides when the work is done.

Now run many loops at once. A factory is loops at scale: The agents ship the work inside, while humans own the decisions at the boundary.

And at the heart of that factory is a careful boundary between what’s inside the system and what’s outside it. Inside the system we collect inputs (from the product team’s intent, or knowledge of previously shipped work, or of recent incidents, or of specific feedback from users). The agent loop investigates the task, implements a plan, and verifies the result. Then, evidence crosses that boundary. A human, who owns the dependent system, sees the evidence and decides whether to proceed.

And that, friends, is the shift we’re trying to make. Before, our agents were doing the inner loop of the execution loop. Now they run the inner execution loop. Engineers own the outer loop.

Inside the system, there’s really just one kind of thing our agents are doing: capability. The capability to investigate tasks, implement plans, test their results, and report back. That’s the capability of a model. And as we’ve said, that future is already here.

Outside the system, there’s a single kind of thing: agency. The agency to decide, verify, approve, and own.

We’re still talking about code, you see. It just needs to live in a place and be performed by people who know what they’re doing.

The potential for AI code is no longer marginal. In a Sonar 2026 survey, we asked teams about the share of their commits that were AI-assisted. It was small but nontrivial. And several of the respondents said they expect the share of AI-assisted commits to grow substantially.

Sonar’s 2026 State of Code report found that 42% of committed code was AI-generated or significantly AI-assisted, with expectations for that share to keep growing rather than plateauing.

Creation, in other words, is getting cheaper. Scarcer resources are review, validation, understanding, and maintenance.

We moved the speed of generation faster than we moved the speed of control, and so we have a trust-verification gap. A lot of people we talk to still express some degree of distrust in AI code. Yet fewer of them seem to consistently build that distrust into their verification processes.

That’s a dangerous place to be. We’re going to need cheaper, clearer ways to verify the trustworthiness of AI code.

If you look at the GitLab June 2026 report, you’ll see that governance questions have shifted.

GitLab’s June 2026 AI accountability research shows that review and validation are the current bottlenecks when using AI and, more worryingly, that governance usually happens after code creation, after we’ve accepted the risk and lost control over ownership. Today, it’s not just about control. It’s about what constraints we set on the system. It’s about how we’ll check the work with evidence, and how we’ll hold teams accountable. It’s about who will own what part of the AI lifecycle.

So the final distinction in this series is between process and quality. Quality is the concept of backpressure. We mean it literally. We don’t want to grant our agents as much autonomy as they can possibly exercise. We want to grant them just enough autonomy that we have enough backpressure to stop them, regulate them, check their work, and ensure our humanity.

Ordinary engineering holds up a lot of signals that indicate that the work being done is doing the right thing. Type checks, tests, hooks, sandbox limits, audit logs, monitors. Our engineering systems are full of these kinds of signals, and they’re designed to provide enough backpressure to keep the system honest.

And so as long as our agents are emitting these same signals, we can trust our ordinary engineering to provide appropriate backpressure.

Trusting our systems doesn’t mean we don’t want a human in the loop. It just means that the human doesn’t need to be in the inner loop. We want them in the constraints loop (What inputs, architectures, instructions, or invariants should we set?), the sampling loop (How much output should we sample and review?), the audit loop (What evidence should we keep, and how do we make sure our audit log is effective?), and the ownership loop (What part of the production boundary should we own?).

But the human doesn’t need to be in the inner loop.

The agent can ship more than you can review.

And the scarce resource is your own core human judgment, informed by quality signals like logs or tests.

The AI June 2026 report shows that, in the experimental setting, agentic delegation along hour-scale time horizons is essentially here. The work by OpenAI this year on agents and the future of work was a great source for these ideas. So we need to start thinking about how to establish this ownership boundary, as our systems start shipping more than we can review.

And that’s where the answerability comes in.

Because with long-horizon agents, the decisions made over hour-scale time horizons are just that—decisions. And not all the decisions are going to be recorded. You can’t trace them all back to input tokens. If all you’re doing is trusting that the output you get is the correct choice for the problem at hand, the hundreds or even thousands of human hours of work you’re going to need to reconstruct the chain of decisions that lead to it become impossible. And so, again, answerability becomes something that must be at the core of our system design.

Three hidden costs

And there are three hidden costs:

Cognitive surrender ~ blindly accepting what AI gives you. When you delegate work to an agent, the work itself may appear to be the work of the agent. But it’s actually your work. It’s your reputation. It’s your responsibility. And it’s your software that suffered the defects in the output. And it’s your software that needs to be changed to reflect that output. So the agent’s output becomes your answer. And with it comes all the accountability. The Wharton study that put this together is reassuring when the AI is right. But when it’s wrong, the news isn’t great. When the AI was wrong, nearly three-quarters of people accepted it anyway, and felt more confident than they would have without the AI.

Cognitive debt ~ erosion of your understanding and memory of how to solve problems. When you delegate work to an agent, you’re offloading all the thought work to the agent. And while thinking it all out yourself takes time and energy, thinking it out on a massive codebase takes resources that aren’t available when you’re trying to run up the learning curve. So the output you get is often unattainable by you. And the longer the time horizon of the agentic planning, the bigger the gap between the code the agent produces and your understanding of it becomes. The gap compounds. The debt accumulates. And the cost of climbing the learning curve grows almost exponentially.

There’s a randomized controlled trial from Anthropic looking at whether engineers who lean on AI to write code understand it as well as engineers who write it themselves. The conclusion was gloomy: On a comprehension quiz, the engineers who worked through AI scored 17 percentage points lower than those who didn’t, 50% versus 67%.

And then there’s the orchestration tax: It’s easy to spin up lots of agents now, but your cognitive bandwidth doesn’t parallelize in the same way. Steering your agent away from the worst behaviors, sorting the work the agent produces to identify the ones that need your attention, directing it to focus on the work you care about first, verifying your most important constraints and your most dangerous assumptions before you let it run. . .

All of that takes work, and it can’t be automated. There’s no substitute for human judgment.

Brownfield systems are especially dangerous here, because the system behavior you have to audit doesn’t live in the code. It lives in the scars.

Fixes? Make attention the priority in your architectural decisions. Use worktrees, scopes, and evidence to reduce the coupling between your initial plan and the work that emerges from it. Time-box the effort to resolve unactionable steps. And make change in your software strictly an opt-in permission.

Alpha, decay, and taste: These are the three core patterns that shape careers and performances across domains.

Alpha is the lead part taken up by the highest achiever in the competition, when you’re playing your highest-value game move. Decays are established patterns that everyone learns through repetition and watching others (plateaus, if you like). Taste is the earliest we can sense the lead in an alpha or the change in a decay. It’s our judgment of what’s coming before we have any evidence that anything is happening.

Paul Graham’s point is that when anyone can make anything, choosing what to make matters more, and Mitchell Hashimoto’s definition is the operational one: making high-quality qualitative judgments where no objective metric exists yet. From now on, taste drives everything. Alpha shifts are taste changes. And decays fade out because we start to taste something different.

Next step? Operationalize your taste. How? Give it a name that reflects what you’re trying to move from limbic to conscious. Practice it in critique and examples. Make its rationale explicit.

And keep making the move that delivers the most durable competitive advantage in your industry. What’s that? Keep moving the edge up from just doing the task to teaching it, systematizing it, deciding when it should be done, and owning the result.

Everyone is a developer, but not everyone is an engineer. Engineering is what a developer turns into when they embrace a work discipline that is more strict: thorough and logically sound reasoning, consideration of constraints and tradeoffs, recognition of risk and exposure, and practical accountability.

In the future, people will leave the administrative work of engineering and embrace new roles that emerge as engineering becomes more demanding. Roles that are unbundled from the spirit of craft but make clear what each person does. There will be those who prototype. Those who build. Those who sweep. Those who grow. Those who maintain.

The humans hold the edge of the system in the other direction too. Increasing the alpha: choosing what is worth doing, defining the constraints within which it should be done, deciding if the evidence is sufficient to proceed, and caring for the result. Whether it’s a single team or a hundred teams, this is the edge that only humans can hold.

Accountability will scale the factory. Like attention and taste, accountability is also one of the three dualities that makes everything work. Without accountability, there are no rules. No wrangling with questioners. No trade-offs. No risks. No safety nets. If nobody owns the consequence of a decision, then high agency can only bring chaos.

The half-life of an edge is one release, but the half-life of a signature is a career. A signature is your name on the work, such that you feel you can stand behind what was shipped. Skills get you leverage; accountability turns leverage into trust.

Only people can choose. Only people inherit consequence. Agents can be asked to choose, route, merge, and escalate safely inside a policy, but they cannot inherit the consequences.

Every codebase should perhaps come with some kind of accountability contract that explicitly states the checklist that was understood when the change was accepted, the evidence that went into the decision, who was accountable for the change, and the system status after the change was blocked. Just like:

• Your attention and taste
• Your evidence, verdict, and ownership
• Your alpha, decay, and taste

High agency

In a typical agentic workflow, high agency is the art of knowing when to delegate, when to inspect, when to stop, and when to own the result of a process. The ladder of agency runs from low to high: flag a potential problem, investigate it, execute against it, diagnose it, propose solutions, recommend fixes, and resolve the issue. A high rung on the agency ladder is discernment: found it, it’s not worth fixing, moving on.

The 12 pillars that hold up the software factory

Brownfield is the frontier for factories that hope to scale. All those clever little innovations may not feel like much yet, but the production environment is a lot. When building an entirely new system, it’s much easier to plan and implement sufficient backpressure mechanisms because you have full control. When you’re adding intelligent agents to a legacy system, however, it’s another matter entirely.

Legacy systems include the entirety of production behavior, future expectations from customers, migration histories, release and budget cycle durations, unspoken assumptions, edge cases, data weirdness, runbook procedurals, and all the scars that accumulated without the will to care for the system.

To be a steward of brownfield requires a form of durable engineering. Work has to be done to turn implicit knowledge into explicit constraints, keep it coherent across teams and through generations, formalize that knowledge into test procedures and functional specifications, and tie that knowledge to objective evidence. All while ratcheting failure into more learning. Because if the system doesn’t get the care it has always received, everything will come crashing down.

New work is real work

The work will get more interesting as you scale. Because when everything else is built, people will want to build new things. They’ll want to employ the alpha and taste they have developed through their craft to design new loops that can be grafted onto the software factory. Or they’ll want to build greenfield systems that employ all the knowledge of the software factory to one elegant, well-meaning, principled effort. They’ll want to design and implement new forms of evidence that will rise to the level of verification for the new systems. They’ll want to take care of brownfield systems that are now so complex they need dedicated attention. They’ll want to design and manage new backpressure mechanisms. They’ll want to design new agents. And they’ll want to build agency.

And, as they do, they’ll come to see that all this is real work. That’s a good thing.

Automation creates bottlenecks. Bottlenecks in production that are worth owning. Because automation gives us control over industrial scale. But there’s also new bottlenecks that arise from industrial scale. The bottleneck moves from “Can we build this?” to “Should this exist? Can we answer for it?”

What I’m suggesting is a practical operating model for scaling agentic engineering. There’s inner and outer loops. The inner loop is where the work is done. Loops are designed to be as independent as possible. Put all quality assurances and verification inside the loop. Once you’ve designed and validated the loop itself, the only thing you have left to do is to grant autonomy by putting in place a back-pressure mechanism that acts to control the rate at which the loop is run and its scope of operation. And put humans in their rightful place, on the right decisions. Don’t treat understanding as a hand-off or a release gate but rather as a point of decision where humans are primed to provide their insight. And then for every artifact that exists and is fed back into production and into new teams and engineers, leave behind better artifacts.

Build the factory; keep the lights on; make work legible, verifiable, owned.

An agent can write it. But before it reaches users, someone must explain why it should exist, why it’s safe enough to be part of production, and what they will do when it is wrong.

That’s agentic engineering at the outer loop—that’s the work now.

The following article originally appeared on Sean Goedecke’s blog and is being republished here with the author’s permission.

In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet. Today, everyone can write sort-of-okay CSS by delegating the task to an LLM. LLMs make everybody into a generalist.

Because of this, lots of people don’t think there’s any skill involved in working with LLMs. If you want the product that LLMs can deliver—PhD-level mathematics, pretty good but sometimes tasteless computer code, or awkward LinkedIn-style writing—you can simply ask for it. Since everyone is talking to the same models, “skilled prompters” are getting the same results as people touching LLMs for the first time.

This is wrong. The most important skill in prompting is expertise in the domain you’re prompting for.

A good illustration of this is Terence Tao’s conversation with ChatGPT about the recently discovered counterexample to the Jacobian conjecture. This is not the same ChatGPT I talk to! I couldn’t get to where Tao gets, even with unlimited tokens to burn.

There’s a lot to learn about good prompting from Tao’s conversation. Here are a few observations:

• Tao’s messages are very short and to the point. He doesn’t respond point by point to the model, just to the gist.
• The model outputs are much more concise than when I try to talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode.
• Tao pushes back when the model’s responses look wrong, but he doesn’t directly contradict; instead, he says things like “This looks more complex than I was hoping for.”
• Tao makes several leaps and suggestions himself. He almost never takes the model’s advice about where to go next.

However, you can’t prompt like Tao on mathematical questions just by following these tips. The key to his technique is actually understanding the mathematics: pulling the relevant idea out of ChatGPT’s multiparagraph response, suggesting alternate approaches or formulations, and identifying what “looks weird.”

Terence Tao is a better mathematician than I am a programmer. But the idea here—that domain knowledge makes you better at using LLMs—is something I’ve also experienced in my own work. If you have a good theory of your codebase, you can push the LLM much harder than if you have no familiarity. Because you have your own sense of what a good solution might look like, you can say, “No, I think it could be simpler here” or “But don’t we already do X?” or “Can we express this problem in these familiar terms?”

This touches on an idea I’ve written about before: that system design problems are dominated by concrete specifics, not generic principles. Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In his conversation, Terence Tao asks a lot of specific questions like “Does X work here?” or “Given Y and Z, why A?” I can’t ask those questions about the Jacobian conjecture, but I can ask them about the systems I own at GitHub.

If you have no domain knowledge, you can cling onto the LLM to at least get something. That’s not bad! But if you have domain knowledge, you can wring far more value out of the same LLM by steering it hard in the direction you want. Most of us will have to do a mix of both these approaches, since we have domain knowledge in some areas but not others.

The usefulness of domain knowledge suggests that human expertise will continue to be useful even as models get stronger. For many tasks, the human is the bottleneck, not the model, because the difficult part is in communicating to the model exactly what kind of solution the human wants. The information is “in the model” already, but it takes a very smart human to pull it out.

This post got many comments on Hacker News. Some commenters share their anecdotes about how expertise has helped and lack of expertise has hurt. Other commenters say it’s plausible, but they have a sensible suspicion of a view that’s reassuring them about how they’re still valuable. I agree with that, though I suspect by the time we get around to studying this, the landscape will have changed under our feet again. Some commenters point out that OpenAI’s math prompts were inexpert, and so expertise isn’t required. Here I’d respond that OpenAI does have a team of expert mathematicians that checked and filtered the model’s suggested discoveries, and that you cannot currently skip that step.

In this episode of Zero to Agent in 30 Minutes, Microsoft senior solution engineer Maxim Salnikov walked through the Agent Package Manager (APM), a terminal-driven open source product from Microsoft that treats agent context the way modern software already treats its dependencies: versioning it, pinning it, and checking it before it ships. It’s a technical session, but rather than building an AI agent, you’ll discover how to manage all the customizations you’ve installed for your agents and make them portable, secure, and governed by policies you or your company define.

How to build a supply chain for agent context

Maxim demoed the process of setting up and using APM step-by-step. Here’s how it works.

1. Keep every customization in your codebase. Agent skills, custom instructions, and hooks tend to spread across a developer’s machine, a harness’s install folder, and enterprise-level defaults. Maxim recommends storing them in the codebase instead so that anyone on a new machine or using a new harness ends up with the same setup as everyone else. Yes, your codebase will be bigger, but the trade-off for consistency is worth it. But be sure that skills and other customizations are secure before adding them to your codebase. Consider building your very own enterprise repository of the customizations you’ve identified as safe to use. (See step 5.)
2. Install APM and initialize a project. APM works like npm for AI agent customizations. Running apm init sets up an apm.yaml file targeting one or more harnesses, such as GitHub Copilot, Claude Code, or Cursor, and apm install pulls a skill from a repository into the right location for each one. When you install a skill, APM also creates a log file documenting the entire resolution history.
3. Pin to a version or a commit hash. Once a skill is installed, pin it the way you’d pin a library dependency. Doing so protects against a scenario where a legitimate skill later gets compromised and a bad actor slips malicious content into a new version under the same name. APM also scans files for hidden Unicode characters that may contain malicious instructions.
4. Reproduce the exact setup with a lock file. APM logs every file and hash it installs. Running apm install --frozen rebuilds that exact environment from the log instead of reresolving everything from apm.yaml, so a teammate’s machine ends up with precisely the same setup as yours.
5. Source customizations from a vetted registry, and set a policy for it. Public repositories work for experimentation, but Maxim suggests enterprises build a private, gated registry of reviewed skills. APM policy files enforce which sources are approved, so a customization from an unvetted location gets flagged.
6. Audit for drift, then enforce it in CI. The apm audit command checks installed customizations against policy and catches unauthorized sources or content that has changed since installation. Run that same audit as a gate in a CI/CD pipeline to protect the entire organization against skill drift and bad actors.

The software supply chain already has decades of tooling behind it. That discipline hasn’t caught up with agentic AI, but APM is attempting to close that gap. Explore the project GitHub repo and get started.

Coming up next

On September 9, Menyala’s Sajal Sharma joins Zero to Agent in 30 Minutes to build a shared knowledge base that acts as a common brain across agents. He’ll show how a single repository of research, daily logs, and notes can give Claude Code, Codex, OpenClaw, and Hermes access to the same accumulated information instead of starting from zero with every new session.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

Dashboards don’t interpret themselves

AI and real-time data give organizations more information than ever. Dashboards. Live metrics. Instant customer feedback. All of it moving fast. But data doesn’t make decisions. People do. A dashboard can tell you that cart abandonment jumped 12% this week. It can’t tell you why, and it definitely can’t tell your marketing, product, and support teams what to do about it together. That takes a conversation. It takes people in a room—virtual or real—looking at the same thing, arguing about what it means and what to do next.

The map freezes a moment so you can talk about it

Experience mapping is a broad field of visualizing human experiences. You’re probably familiar with things like journey maps, service blueprints, and other similar diagrams of the experiences. But none of them hand you an answer. What they do is take a fast-moving, chaotic situation and freeze it for a moment. It gives a team something to point at, argue about, and align around.

Picture a typical working session. People from different parts of the business sit down around a map of the customer experience. Each of them already knows a piece of the picture. None of them has the whole picture, together, at the same time. That’s what the map provides. That’s usually the moment something surprising surfaces. Not because the map contains secret information but because it puts scattered knowledge in one place, in front of the people who each hold a piece of it. The visual aspect of maps is critical. Laying out an abstract concept like a “customer experience” allows teams to engage with it in new ways and reach new conclusions that are hard to get from a spreadsheet or data alone. Grasping cause and effect in one visual overview helps teams find the patterns of behavior that matter the most and to conceive of viable interventions.

AI can surface these kinds of patterns in seconds. But it cannot create the moment when a cross-functional team collectively recognizes how its silos are hurting customers. Only people, looking at the same picture, can do that.

The map isn’t the point

Some claim that journey mapping is dead. That static maps can’t keep up with real-time data and AI-driven personalization. This confuses the artifact with the activity. A map that gets built, presented once, and filed away never helps anyone. It fails for the same reason a report fails: Nobody’s talking about it anymore. The value was never in the diagram. It’s in the conversation the diagram makes possible.

Take how I got that team to reach their own conclusions about the invoice problem rather than just telling them about it. After scoping the customer type and situation we wanted to understand better, I interviewed a dozen or so customers about their billing experience. Nothing unusual came up at first. People described the routine steps: get an invoice, check it, pay it. But a few mentioned something in passing. They’d disputed a charge and kept getting late payment warnings anyway, even while the dispute was still open.

From those interviews, I built a draft map of the invoicing journey. I called it a draft on purpose. I didn’t want to hand stakeholders a finished diagram and ask them to approve it. I wanted them to lean into it, question it, and add to it. Then I scheduled a working session. The room included people who’d never worked together before, despite being at the same company for years: billing, support, and product.

We didn’t rush through the map. We slowed down, section by section, and used structured exercises to pinpoint the moments that mattered most to customers. That’s when someone in the room realized: A customer who’s actively disputing an invoice can still get a warning notice for that same invoice. Nobody had designed it that way on purpose. It fell through the gap between two systems that never talked to each other. But once it was visible, laid out in front of the people who owned each part of the process, it became impossible to ignore. The room got quiet, then loud. People were genuinely upset, not at each other, but at what customers were going through.

Of course, I had uncovered this already in my research. And sure, it was also visible on the map. But my diagram wasn’t about giving a magic answer. The process of learning together is the point. That reaction didn’t come from a dashboard. It came from people confronting the evidence together, in the same room, at the same time.

What actually changed

Before the workshop, this problem was invisible in a specific way. Support knew customers complained about warning notices. Billing knew disputes existed. Product knew the systems didn’t sync. But no one held all three pieces at once. The map put all three in the same field of view. That’s the mechanism. Mapping doesn’t create new information. It puts existing, scattered information into one shared picture, at the same time, in front of the people who each hold a piece of it.

What changed after that: Billing and product agreed to flag disputed invoices so no warning could go out. Support got a way to check dispute status before responding to a complaint. And the three teams kept meeting monthly, something none of them had done before. The map didn’t do any of that. The conversation the map created did.

What good collaboration looks like

We started with customer evidence and a deliberately unfinished map. We included people who owned different parts of the experience and asked them to question what the map showed, identify what they knew and what they were assuming, and examine the gaps between their systems. The session ended with specific commitments, and the teams continued meeting as they learned more.

That is what getting collaboration right requires: the right people, shared evidence, visible disagreement, clear ownership of the next decision, and a cadence for revisiting what the team thinks it knows. Without those conditions, mapping can easily become another workshop that produces an attractive artifact but little change.

What this means for your team

As AI speeds up execution, don’t cut the time you spend aligning as a team. Protect it. Expand it. AI won’t give you an edge. Your competitors have access to the same models you do, trained on much of the same data, producing much of the same output. If everyone moves at the same speed, speed stops being an advantage. It becomes the minimum bar for staying in the game. AI also works like a spotlight, amplifying whatever’s already happening in your organization. If your teams collaborate well, AI makes that strength visible fast. If they’re siloed, AI exposes it just as fast.  Now is the time to get collaboration right, while staying focused on the customer. Waiting until AI forces the issue is waiting too long.

In the end, AI can help with customer discovery and accelerate insights. But it doesn’t replace human judgment and decision making. Rallying around a map—a visual depiction of customer experiences—provides a natural forum for discussion, debate, and shared understanding to align before acting. The tools will keep getting faster. The organizations that win won’t be the ones with the best dashboards. They’ll be the ones who are best at coming together, again and again, to make sense of what those dashboards show them.

If you want to dive deeper into mapping, join Jim on October 9 for his Beyond the Book conversation about the latest edition of Mapping Experiences. He and host Vicki Reyzelman will chat about how experience mapping has evolved from a UX technique into a strategic capability for organizations, how AI is transforming the way we create and analyze maps, and how you can use mapping to align business goals with customer needs, facilitate collaboration across teams, and drive transformation at scale. It’s free to attend. Register now.

Host Christina Stathopoulos, founder of Dare to Data and a former data scientist at Google and Waze, returned to This Week in AI with developments that stretched from Claude testing ways to improve model safety to Chinese open weight models gaining developer traffic and new systems learning to model physics. She also examined what Anthropic and OpenAI’s business moves, workforce forecasts, and debates over access reveal about how quickly the AI landscape is broadening.

Claude is taking on more of the research process

Anthropic provided an early example of AI helping improve future AI systems. In research Christina highlighted, Claude searched existing work, proposed methods, generated training data, and repeatedly tested and refined its approaches to reduce unwanted model behaviors. The experiments covered 10 such behaviors, including deception, hallucination, prompt injection, privacy violations, and reward hacking. Anthropic reported improvements across all 10 without degrading the model’s broader capabilities.

For deception, Claude tested more than 150 methods and eventually closed 85% of the measured safety gap. Human safety researchers closed only 20% in Anthropic’s comparison. Christina emphasized that this wasn’t a direct contest because Claude could run and refine experiments much faster and at a much greater scale. She also cautioned that the work didn’t amount to full recursive self-improvement.

This showed how AI could increasingly handle experimentation in model development, changing the pace and scale of research while humans still set the goals and evaluate the results.

Model performance is only one part of the frontier race

Competition among AI labs increasingly involves business performance, infrastructure, and deployment options alongside model quality. Anthropic estimates that the market for its systems could eventually reach $30 trillion, a long-term estimate that Christina treated skeptically because it approaches the size of the entire US economy. She also highlighted more concrete evidence of momentum in how Anthropic’s annualized revenue run rate rose from less than half of OpenAI’s at the start of the year to surpassing it within several months. Both companies are preparing for possible public offerings.

OpenAI faces a different set of pressures, and Christina highlighted its 14 executive departures this year. That sustained leadership turnover could raise questions about the company’s ability to execute consistently. OpenAI is also trying to gain more control over its infrastructure. Its Jalapeño inference chip, developed with Broadcom, delivered up to 1.9 times more AI work per watt and up to 3.6 times lower latency than comparable NVIDIA systems in OpenAI’s own testing.

Chinese open weight models are widening the field further. Christina cited an AI gateway where open weight models recently reached as much as 62% of developer traffic on a single day, compared with an average of roughly 10% in April. The competition now spans benchmark performance, capital, infrastructure, cost, deployment flexibility, and organizational execution.

Physics models could extend AI beyond language and images

Christina closed with research aimed at helping AI systems model physics. Researchers from MIT and Tsinghua University developed a pretraining approach that learned from more than one million synthetic interactions between moving particles and complex 3D objects, then applied those patterns to simulations involving wind, water, collisions, and light. The researchers described physics as a potential “third modality” for AI alongside language and pixels.

She also covered Accelerated Understanding, a startup that recently emerged from stealth with an architecture based on neural operators rather than transformers. The company is targeting problems involving enormous physical datasets, including chip design, robotics, extreme-weather forecasting, and geological exploration.

By learning directly from physical systems, these models could become valuable for simulation, engineering, robotics, forecasting, and other work that depends on understanding complex real-world environments.

What’s next

Christina also examined who could benefit from these advances. She discussed Bill Gates’s argument that access, deployment, policy, and distribution will shape AI’s social impact, and brought in new US Bureau of Labor Statistics projections showing job growth in areas including technical services and healthcare, while office and administrative roles face greater pressure from automation.

Her larger point was that access, workforce preparation, and public policy will determine how AI’s benefits and disruptions are distributed.

Due to the Labor Day holiday, This Week in AI will return on Monday, September 14, when we’ll dive into more of the news, issues, and key developments shaping the AI era. And check back each Friday for the latest episode, or watch on YouTube, Spotify, Apple, or wherever you get your podcasts.

That’s why, three months ago, I built my own software factory, Squid, to ship all of Decoding AI’s small and medium projects with minimal human intervention. The first version was so overbuilt I stopped using it.

Meanwhile, I kept seeing people obsess over the next “_____ engineering” label, instead of focusing on actionable outcomes. Prompt engineering, then context engineering, then harness engineering. So far, so good. But in the last few weeks (July 2026 as I write this), things got off track with loop engineering and graph engineering, which already read more like marketing talk than like anything that solves real problems. Graph engineering overtheorizes how teams have built AI applications since the LangGraph era kicked off in ~2024. Don’t get me wrong. The terms aren’t wrong (Boris Cherny, who leads Claude Code at Anthropic, says, “My job is to write loops”), but we’re overexplaining intuitive things we started doing years ago.

While you’re defining what counts as a loop, you’re not thinking about the processes that actually deliver software.

The right frame is the software factory, which was one of the core themes at AI Engineer World’s Fair 2026, where Tereza Tížková (growth at Factory.ai) defined one as “the whole loop, the whole lifecycle of developing software with autonomy.”

I bet you already have an intuitive sense of what a software factory is. In this article, I want to further formalize it and map it onto the software development lifecycle (SDLC). We’ll explore how big your software factory should be, and when to stop automating before it adds more friction than value. Most importantly, I want to highlight where the human belongs in this process, and where I believe they’ll still belong even in a world where all the code is generated by AI.

So…what’s worth automating? Where does the human bring the most value? What’s worth building, and what’s worth buying?

The design of a software factory

Like a physical factory, a software factory automates software creation with minimal human input. Raw work (bug reports, feature ideas, incidents) goes in. Shipped software comes out. It needs a few highly qualified people making high-leverage decisions, and defined gates that work can’t pass through without them.

Factory.ai pitches “a self-improving system for your Software Development Life Cycle (SDLC).” Addy Osmani frames the stack as loop, harness, factory: “The loop is the atom”; a factory is “an org chart made of loops.” Warp’s CEO, Zach Lloyd, states that “software engineering will become factory engineering.”

The factory is made up of eight stages that can be divided into three buckets:

What to build. Triage/intake classifies, deduplicates, and routes incoming work. Brainstorming finds high-impact features through market analysis, user data, and technical research. Planning, the most important stage, turns that research into a polished plan, refines it by letting the agent grill you, and tracks decisions in an ADR (Architecture Decision Record) log plus a glossary. The outputs of this stage are tickets backed by documentation that a team of agents can implement, which can be tracked in plain files or a project management tool, such as GitHub Issues, Linear, or Notion.

At this stage, the agent plans in read-only mode, going through the code, the AGENTS.md file, and most importantly the context layer.

Actual building and checking. Implementing is a software engineer and QA agent loop that goes through the tasks and supporting documentation. Review checks the PR diff against product, architecture, and code standards. Review-CI runs the test suite, and failures trigger a fixing agent. Release handles CD to staging/production with human deployment checks.

Self-improving. Monitor/incident response feeds production signals (alerts, errors, incidents) back into triage as fresh input on what to build next, closing the loop.

Orthogonal to the eight stages, we have the context layer. The layer is especially important at the front of the line. Brainstorming is limited to the data it sees: user analytics, competitor analysis, research, transcripts, and documentation. At this stage, a poor context layer directly limits the space of possibilities you can explore. It has a similar impact on planning, where transforming the raw idea into technical specs and tasks depends heavily on how good the examples in the context layer are. If you want to implement a new product-recommendation feature, and you have zero examples, the LLM will just predict the most common thing to do, which often isn’t the best solution for your product.

The context layer can take many forms. One strategy that is becoming increasingly popular is the LLM Wiki, a term coined by Andrej Karpathy. It’s basically a strategy to transform your data into a structured knowledge base, just by using files, instead of a database. Factory, via its AutoWiki feature, transforms popular codebases into a structured knowledge base that agents can query instead of parsing the codebase itself. LangChain recently released OpenWiki, a CLI tool to manage wikis for agent memory. If you’re curious, in this article I detail how I turn my data from Obsidian, Readwise, and Google Drive into agent memory via LLM wikis.

Where the human belongs

To see where the human belongs, let’s walk through the factory with an end-to-end example. We’ll build a feature for a shopping-assistant agent on an ecommerce platform similar to Amazon’s. The scenario is that usage data says users aren’t engaging with its recommendations, and we have to ship an improvement.

Brainstorm is where taste lives. Agents do the grunt work: They analyze user activity, scan competitors’ assistants, and pull research into the knowledge base. Then a member of the technical staff starts looking at the data, understands why people are not engaging with the recommendations, explores how the competition implements their solution, and proposes a fix as a feature spec. At this stage, the spec solves a business problem. It doesn’t need to prescribe a technical solution yet.

Plan is where a human, with the help of the software factory, transforms the feature spec into an implementation plan. Let’s assume that we want to make a change to the recommender engine algorithm. The human chats with the knowledge base, figures out whether it’s feasible, and thinks through architecture, interfaces, data flow, cost, and latency. They then let the agent scan the codebase and grill them until the plan is properly refined into something that fits the codebase. The output is a bunch of tickets plus an ADR explaining the algorithm change and an update to the glossary.

The agent can help within these two stages by quickly scanning through a ton of data and improving the plan, but the human is still central.

Use the strongest model (Fable) for brainstorming and planning. These stages burn fewer tokens than implementation itself, but everything downstream depends on them. A well-written plan lets cheaper models (Opus, Sonnet) execute without reasoning their way out of dead ends. A weak plan makes them retry until the extra tokens erase the price gap.

With a weak plan, I’ve watched Sonnet on high reasoning out-cost Opus on the same task: The smaller model needs more attempts to reach the same goal. Total cost is tokens × price, not model tier. So more failures equals more reasoning, more tokens, and more cost.

From here on, we move into “loop” and “graph” engineering territory.

Implement runs a software engineer agent that picks up every ticket that’s ready to go. As the loop is scoped to a feature, it takes only the associated tickets. After each ticket is implemented, a QA agent tries to find bugs by stress-testing the application. As agents tend to have a positive bias towards their own work, the split between the software engineer and QA agents matters. As Addy Osmani puts it, the model that wrote the code is “way too nice grading its own homework.” This loop, at solo scale, can be as simple as a bunch of terminals pulling tickets. At larger scale, it runs on remote agents working 24/7.

The loop only works if agents can interact with the app. The QA agent needs one command that starts the whole stack reproducibly. From there, it drives the app in the browser, calls the data or fine-tuning pipelines, or hits your server’s API. Whatever your app’s interface is, the agent needs access to it, the same way a human user would.

The key idea is to integrate feedback loops as natively as possible into your software factory. Ideally, you want multiple levels, depending on how expensive it is to run them: linting, unit tests, integration tests, and end-to-end tests. When the loop keeps failing, the root cause is almost always missing plumbing, not the agents.

Review has three steps. Step one checks the product and architecture requirements against the ticket and the ADR. Any discrepancy becomes a new ticket passed back to the implement loop. Step two ensures code quality (modularity, naming) and guards against AI slop such as verbose comments or cryptic function names. Step three looks at the CI/CD pipeline. At every step, any failure auto-creates a task picked up by the software agent.

Not every project needs all three steps. The “factory” ends with a PR that you as a human need to review and merge. But in reality, if you spend enough time creating a strong plan, the PR that reaches you is usually ready to ship as-is.

So where does the human belong? You’re indispensable during brainstorming and planning, and you return for the final check. Agents own everything in between. OpenAI took this to the extreme: ~1M lines and ~1,500 merged PRs over five months with zero hand-written lines. Their framing is “Humans steer. Agents execute.”

Don’t overbuild the factory

With my first Squid version (my own software factory), I got greedy and chased full autonomy: big remote workflows, parallel agents, and one grand pipeline running end to end. It worked, until something went offscript. Which it usually does. I couldn’t debug it, couldn’t halt it mid-run, and couldn’t redirect it without throwing the run away. It was a big monolith that took me too far out of the loop, and I couldn’t control it.

I realized you need two options. The first is granular commands that let you grill your plan, implement a specific task, or review one particular step. The second, for when you’re comfortable giving the agent 24/7 autonomy, is an end-to-end command that chains all the smaller ones into a fully autonomous graph, such as one big /plan and /implement-review-all command. Basically, each step is a “loop,” while the whole pipeline is the “graph” of your software factory. Still, note how planning and the rest are split into two different commands, as planning is, and always will be, human-driven (at least if you want the result to stay aligned with what you actually want).

Bottom line. You need to be able to step in, halt it, redirect it, and interrupt it, while still having the option to go fully autonomous.

The bottleneck is me, and that’s by design. To be honest, I’ve worked mostly solo since the AI coding agent boom, and I don’t understand who the people shipping 100 features in parallel are. Most of my features (per project) build on one another, which makes them impossible to parallelize. As the project grows, you can find more and more independent features that can be implemented in parallel, but I still believe that number is limited.

That’s why, when I parallelize, I only use local agents, each running in an isolated codebase via worktrees. So far, I’ve never felt the need for 24/7 remote agents, or wanted the overhead of managing them.

A big team can justify more automation, but it has to earn it. So as with any other software product, start small, start by automating the most time-consuming bottlenecks, and add complexity gradually as people get comfortable with the system. Don’t be like me, with my Squid experiment.

Build vs. buy

In all scenarios, you’ll start with a prebuilt coding harness. The most popular vendor-locked ones are Claude Code and Codex. Or go open source with OpenCode or Pi, which took off thanks to its minimalist, extensible architecture that lets you easily build on top of it.

But picking a harness isn’t the same as knowing how to configure it and wire it into your software factory. That’s why everyone needs to know, at least intuitively, how a coding agent works under the hood: the agent loop running in your terminal, what changes when it runs remotely, how you evaluate it, and which context engineering strategies keep it cheap without making it dumber. If you want to learn more about building a coding agent from scratch, consider exploring my open source course on GitHub. Even if you never plan to build your own harness, that intuition is what lets you become a power user.

For a small team, you’ll get extremely far just by defining a set of skills and agents that encode your process on top of the coding harness (a.k.a. your software factory). To keep it simple, this is what I did with Squid, which I use to implement all my projects.

There are other off-the-shelf “software factories” powered just by skills and agents defined in .md files, such as Matt Pocock’s skills repository or the BMad method.

But remember that the factory is mostly about processes, not tools: A factory that doesn’t fit how your team already works adds friction, never gets adopted, and ends up useless.

You cross the buy line the moment engineers you don’t personally supervise run agents. Observability, tracing, cost tracking, and pay-per-token billing stop being optional and become someone’s full-time job. Agent swarms wired into Linear, Slack, and CI across distributed infrastructure are a logistical hell that isn’t your product. That’s when it makes sense to look into off-the-shelf solutions such as Factory.ai (that comes with the Droid agent), or Warp’s Oz. In Warp CEO Zach Lloyd’s words, “Most of the factory is not necessarily a new interface. It is an integration into people’s existing workflows.”

At the other end of the spectrum, you cross back to building when the platform’s constraints cost more than the team it would take to replace it, as OpenAI’s report on its Codex-built product shows.

The smallest builds, the middle buys, and the largest builds again.

What’s next

Someone is already coining next quarter’s “_____ engineering” term as we speak. But the software engineering processes you use to output real code won’t change that often. That’s why you should be open-minded but at the same time focus on actionable outcomes, not on overthinking how to label things.

As Zach Lloyd suggests: Find one “annoying part of your job” and build the smallest loop that handles it.

The harsh reality is that software factories are just at the beginning. They’re far from perfect, and especially far from being fully “autonomous.” Usually, when someone claims they’ve cracked the software factory problem, they either haven’t tested the idea enough or are trying to sell it to you. I’m confident we’ll reach the point where almost the entire software development lifecycle is automated (with the exception of brainstorming and planning), but at the moment we’re still figuring things out.

But here’s what I’m wondering:

Which stage of your factory still needs you the most? I keep automating mine, and the bottleneck stubbornly stays at planning.

Explore next

1. Osmani, A. (2025). “Loop Engineering.” X.
   https://x.com/addyosmani/status/2064127981161959567
2. MacManus, R. (2026). “AIEWF Daily Dispatch: Loops, Software Factories & Forward Deployed Engineers.” Latent Space.
   https://www.latent.space/p/aiewf-daily-dispatch-loops
3. Factory.ai. (n.d.). Agent-Native Software Development Platform. https://factory.ai
4. Osmani, A. (2025). “Software Factories, Light and Dark.” X.
   https://x.com/addyosmani/status/2079442194449232227
5. Karpathy, A. (n.d.). LLM-Wiki. GitHub.
   https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
6. Abboud, M. (n.d.). “How Coding Agents Actually Work: Inside OpenCode.”
   https://cefboud.com/posts/coding-agents-internals-opencode-deepdive/
7. Kapoor, S. (n.d.). “Building and Evaluating AI Agents.” AI Engineer.
   https://youtube.com/watch?v=d5EltXhbcfA
8. OpenAI. (n.d.). “Harness Engineering: Leveraging Codex in an Agent-First World.”
   https://openai.com/index/harness-engineering/
9. Parsons, C. (n.d.). “Ralph Loops: Build Dumb AI Loops That Ship.” AI Engineer.
   https://www.youtube.com/watch?v=2TLXsxkz0zI
10. Pocock, M. (n.d.). “Software Fundamentals Matter More Than Ever.” AI Engineer.
    https://www.youtube.com/watch?v=v4F1gFy-hqg
11. MacManus, R. (2026). “Warp CEO Zach Lloyd on Why Software Factories Are the Next Phase of Coding.” Latent Space.
    https://www.latent.space/p/software-factories
12. Iusztin, P. (2026). “Building a Coding Agent From Scratch: Harness Architecture.” Decoding AI.
    https://www.decodingai.com/p/building-a-coding-agent-from-scratch-system-design
13. Iusztin, P. (2026). Building a Coding Agent from Scratch Course. GitHub.
    https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course
14. Iusztin, P., & Bouchard, L.-F. (2026). “LLM Wikis as Living Memory for AI Agents.” Decoding AI.
    https://www.decodingai.com/p/llm-wiki-agent-memory

Join 44,000+ engineers eager to learn how to build their own software factories by subscribing to Decoding AI Magazine!

How to build a content creation agent

1. Give the system the context it needs to generate relevant content. Max started with a small knowledge base containing information about himself and his business, details about his audience, sample scripts, and notes on his writing voice. These files gave Claude Code reference material for evaluating topics and generating scripts that reflected his brand and style.
2. Explain the process in plain English and let AI take it from there. As Max pointed out, “Vibe coding is describing what you want clearly and let[ting] the model handle the entire building process for you.” However, you’re still ultimately responsible for what’s built. Claude Code may have generated the implementation, but Max set the context, approved permissions, reviewed results, and answered clarifying questions along the way.
3. Build the individual stages of your pipeline. Max wrote a list of prompts to take Claude Code through the process of researching topics (stage 1), scoring them and selecting the most relevant one (stage 2), then generating three hooks and expanding the strongest option into a full script based on the knowledge base and sample content (stage 3). He ran them through Claude Code and ended up with a working three-part pipeline, with each stage triggered manually.
4. Connect the stages into a single workflow you can run with one command. Max had Claude Code combine research, scoring, hook generation, and script writing into a unified content engine that produces a structured JSON file he uses in the next stage. “This is the actual moment that it stops being a few separate prompts and starts becoming an agent,” Max says.
5. Add a browser-based interface. Max built a local dashboard that displays the generated topics, scores, hooks, and scripts. This lets him inspect the results and start new runs from a browser instead of returning to the terminal each time. Here’s a tip from Max on creating a dashboard you’ll actually want to use as your mission control: Be as creative as you want, but “make it look like something designed on purpose, not a default template.”
6. Test the workflow and extend it. The first version Max created had two limitations: It produced scripts only for the top-ranked topic, and starting a new research run still required the terminal. Max solved this by asking Claude Code to generate hooks and scripts for all five topics and add a button that could start a new run from the dashboard. The revised system produced three scripts for each topic, giving him 15 script options per run.

If you’re building something similar, Max recommends starting with a repetitive task you can describe clearly. Build a working version with your preferred coding tool, use it, and then extend it in response to what you learn.

Follow along with Zero to Agent in 30 Minutes on Radar, or watch the latest episode on YouTube, Spotify, Apple, or wherever you get your podcasts. If you’re an O’Reilly member, you can watch live. Save your seat.

I don’t especially care about the check. (Okay, that’s not true, nine grand is a lot of money, but hopefully you’ll get my point.) Jenny and I didn’t write those books to get rich, and nobody who’s met a working author would mistake the job for a path to wealth. But one thing I very much care about, and I suspect almost all authors would agree, is getting credit for the work I’ve done. If you ask ChatGPT, Claude, or Gemini (or especially Google AI overviews) a question that it partly learned to answer from one of my books, I would love for it to be able to say so. Something like “Some of this comes from Andrew Stellman’s work, and if you want to go deeper, that’s where to look.” In other words, what I’m asking for is a hat tip.

All of this sits inside a much bigger question, one that usually goes by the name attribution. In the broadest terms, attribution means knowing where a piece of work or knowledge came from, and being able to trace it back to the person who made it. That might sound like a narrow, technical matter, but right now it’s one of the biggest live questions in AI. At Foo Camp recently, it came up constantly, quite possibly the most talked-about topic of the whole weekend, and everyone there seemed to have something to say about it. It’s also a tangled subject, part technical and part legal, and more than either of those, deeply emotional. I want to take a little time to pull those threads apart.

Why so many people are angry

Before any of the technical or legal questions, there’s a simpler reason attribution is such a live wire right now: People are angry, and to be perfectly honest, they have good reason to be. Artists have watched models learn to imitate their style from work that got scraped without anyone asking. Writers have found their books sitting in training sets we never agreed to (even though that’s not the reason Anthropic has to pay us). A lot of people are simply afraid that the work they do for a living is about to be done by a machine that learned part of the job from them. Their work got taken, which may be fair use but almost never included asking the author for permission, and often without any way to even find out it happened. And that for many of us feels really uncomfortable.

A lot of that anger ends up funneled into legalistic arguments about copyright, which is notoriously misunderstood and misapplied. People reach for it for understandable reasons. It feels tangible in a way the rest of this doesn’t, and while many of us feel like it was written to protect authors and artists and creators, it’s actually much more nuanced, especially when it comes to fair use. But I don’t think copyright is the right move for anyone who actually wants attribution, and I’ll go further: I think creators will get a lot more mileage working with the AI labs than fighting them in court.

Underneath all of it, what most of us want is simple enough: We want the work to still be ours, and we want that to be visible. That’s an attribution problem. And before anyone can argue about whether AI can solve it, it helps to be clear about what attribution even is, because it’s a slipperier word than it looks.

What attribution actually is

At its simplest, attribution is the link between something and where it came from. When you read a fact in a book, there’s a chain running from that sentence back to the author, and back again to whatever they drew on. That chain is how credit works, and how trust works, and how you know what to go read next when something grabs you. For as long as people have made things, you could usually follow it back to a person.

You see attribution everywhere once you start noticing it. Bibliographies and references are attribution. So is every footnote on a Wikipedia page, and so is every time one news story credits another, the way you’ll read that “Business Insider is reporting” something before another site passes it along (that is, when we remember to do it). For authors it runs deeper than that: Attribution underpins almost everything we do, and the whole system of academic publishing rests on it, because the entire point is to show exactly whose work each new piece is built on.

AI is the first technology that takes in essentially all of it, every book and article and repository it can reach, and hands back answers with the chain erased. And while that may be technically allowed under fair use, it shouldn’t be the end of the discussion. The knowledge comes out; the line back to whoever produced it does not. That erasure, underneath the lawsuits, is what people are really reacting to. The model learned from their work and gives no sign of it, and the trail that used to lead back to them is gone. Attribution is just the name for putting that thread back.

The catch is that people use the word for at least three pretty different things. The cheap version, the one people reach for first, is resemblance: Does the answer look like my work? That’s easy to check and mostly beside the point, because two people can write nearly the same sentence about a for-loop without either one copying the other. What actual products ship is citation, the little source links under a chatbot’s answer, which credit the page the system fetched while it was answering, not the books it learned from years earlier. Those links are the result of retrieval, not memory, where the system went out and fetched a live page mid-answer. There aren’t any technical challenges in adding an attribution for those live retrieval results, and the fact that the AI labs are fine with that attribution gives them a template to follow if model-based attribution becomes technically feasible.

We’ve got a really good real-world example of how this can work. O’Reilly’s learning platform has an AI engine that answers questions out of the books on the platform, tells you which ones it drew on, and pays the authors and publishers behind them. It works because the corpus is small and everything in it is licensed, which is precisely the condition the frontier models don’t have. Because the version that actually matters is the third one, causal: Did my work shape the part of the model that produced this answer? That’s the honest meaning of the word, and by far the hardest to compute.

Notice that none of that is about money. Paying me would mean working out what a given answer is worth and what share of it is mine, which is a hard allocation problem nobody has really solved (and, to be honest, probably works out to a tiny fraction of a cent in any given answer). Attribution asks a smaller question: Did this come from my work, or didn’t it? A hat tip is only that identification, not necessarily a required payment, and it’s still the thing everyone tells you is impossible.

Trying it on a tiny model

I may be an author, but I’m a developer too, and when someone tells us something is impossible, our first urge is always to build that thing. A lot of the time “impossible” just means nobody has worked out how yet, and every so often it means it would be inconvenient for someone if they did. In this case, some really smart people actually have worked out how, at least some important parts of it, and it’s worth understanding both why the problem is hard and how they’re tackling it.

Start with why it’s hard. When a model trains on your book, it doesn’t file the book away somewhere you can later point to, because models are not databases of books and other source material. What it learns gets spread across billions of numbers, tangled up with everything else it ever read, and no single weight says, “This part came from Stellman.” There’s no receipt anywhere that records which source contributed what. The obvious way to check whether your book mattered would be to pull it out, train the model over again, and see what changed, but nobody is going to retrain a frontier model from scratch a billion times. The upshot is that the knowledge is still in there, it’s just smeared across the whole model with no index back to where it came from, and that’s why a lot of people have called attribution impossible.

It may not be quite as impossible as it seems from that particular approach, though.

Researchers have been chipping away at exactly this, and the approaches run from cheap and rough to expensive and precise. At the precise end is leave-one-out: remove a source, retrain, and measure how far the answer falls. It’s about as close to ground truth as you can get, and hopeless at scale for the reason I just gave. The interesting work is on cheaper stand-ins that approximate that result without all that retraining. The one I find most compelling is TracIn, described by four Google researchers in 2020 in “Estimating Training Data Influence by Tracing Gradient Descent”: As the model trains, it saves snapshots of itself, and TracIn measures how much each training example pushed the model toward a given answer by comparing gradients at those snapshots, with no retraining required.

It belongs to a broader family. Influence functions, which Pang Wei Koh and Percy Liang introduced in 2017 in “Understanding Black-Box Predictions via Influence Functions,” are the older, heavier cousin, and Anthropic scaled them up to models with tens of billions of parameters in a 2023 paper, “Studying Large Language Model Generalization with Influence Functions.” A newer method, TRAK from MIT’s Madry Lab, takes on a weakness they share, where near-duplicate sources muddy the accounting. None of it is settled, and TracIn in particular is contested at frontier scale, but the direction is real and the people pushing it have far bigger budgets than mine.

So I wanted to see it work with my own eyes. I asked an AI to build me the smallest real language model that could still run one of these methods, and about 20 minutes later I had a working one, which I called tiny-provenance: a language model small enough, around 37,000 parameters, that I can retrain it from scratch in a few seconds. That size is the whole point, because it let me run the expensive leave-one-out check for real, as ground truth, and set the cheap TracIn approximation next to it to see whether they agreed.

They did. I gave it a trick question, “Who designed the Analytical Engine?” with a look-alike line about Babbage and the Difference Engine sitting right next to the correct one in the training data. The cheap resemblance check, the kind of thing real products lean on, took the bait and confidently credited the wrong line. Leave-one-out and TracIn both ignored the look-alike and pointed straight at the source the model actually used. The lazy method everyone reaches for was confident and wrong, the honest ones were right, and the whole thing ran in about two minutes on a laptop.

I proved it at a size where I can retrain the model at will, which is exactly what nobody can do at frontier scale, so I won’t pretend I showed it holds up there. Attribution works best at the extremes, where a model has nearly memorized a passage or leaned hard on a single source, and it stays hard in the muddy middle where an almost unimaginable number of books may each have added a tiny little bit to the model. The fair summary is that attribution isn’t impossible. It’s just currently expensive, and more importantly, really impractical with today’s technology.

The whole repo is public, and since this is an article about showing where things came from, it seemed only right to be transparent about where the demo came from too. The README walks through exactly how I built it, prompts and all.

The legal hurdle

I’ll put my own cards on the table: I think copyright is the wrong tool for the fight we’re having about AI. My first job out of college was at EMI Records, working on the system that tracked their music contracts, so I learned early how tangled copyright and trademark and mechanical royalties and the rest of it can get.

Here’s the analogy I keep coming back to. If a student copies a paragraph out of Wikipedia word for word, that’s plagiarism. If the student reads it, understands it, and rewrites it in their own words, usually it isn’t, though that depends on what got rewritten, because plagiarism is about the ideas and not only the words. A physics student writing “F=ma” in a paper isn’t plagiarizing, even if they copied the formula straight out of a textbook. On the other hand, paraphrasing a specific result from somebody’s research paper without saying where it came from is still plagiarism, even if you technically changed the words. And it gets even more complicated. If you copy text word for word but credit the source, you’re not plagiarizing, but you could still have a copyright problem. We throw around the term “fair use” a lot, but that’s actually a very thorny legal area. More importantly, reducing these ideas to a legal argument doesn’t really get to the core of the problem, because laws are often imperfect and dictated by decades of (sometimes conflicting) precedent, so what’s legal doesn’t always match up with what’s the right thing to do. What a model does is a step further from copying than the rewrite: It turns the text into an enormous pile of numbers that stand in for the concepts, and there’s no paragraph left anywhere to find. So far, the courts that have looked at this, including the one behind the Anthropic settlement, have called that use “exceedingly transformative” (those are the Anthropic judge’s words, and lawyers on both sides recognized that as quite a statement). What they mean is that it changes the work into something new enough, and for a different enough purpose, that it isn’t simply a copy of the original. I think they basically got it right.

Basically, I’m not getting a check from Anthropic because they used my work without permission; I’m getting a check because they literally used a stolen book downloaded from a pirated website, instead of paying O’Reilly for a copy or getting it from another legitimate source.

But I don’t get to wave copyright away entirely, because there’s a real case on the other side, and it’s one I feel personally. Someone who wants to learn C# or study for a project management exam can now ask an AI and get an answer that came partly from books I wrote, without ever buying the book. The model learned from my books (and many others on the same topics I write about), and now it competes with them, in the exact market they were written for, using what it took from them to do it.

The worry that AI tools compete with the very material they trained on isn’t theoretical. Stack Overflow, the question-and-answer site where a generation of programmers went for help, has lost roughly three-quarters of its question volume since ChatGPT launched, as developers ask the AI instead. And the AI answers them well in part because it trained on Stack Overflow’s answers in the first place. And even that whole issue is highly nuanced, especially since the material on Stack Overflow itself is written by its users and released to the public under a Creative Commons license.

That goes to the center of what copyright is meant to protect. One of the things courts weigh in a fair-use case is the effect on the market for the original, and a free substitute trained on the original is about as direct a market effect as there is. A judge has already pointed right at it. In the case a group of authors including Sarah Silverman brought against Meta, the court handed Meta a win on fair use. But the judge then went out of his way to hint that the the authors may have lost because they’d argued it wrong, and that this market-dilution theory, the flood of cheap substitutes, is exactly the one that could win on a better record. The judge also said there was “no serious question” that the use of the technology was “highly transformative” in his ruling, and again, lawyers consider that very strong language. I’m not a lawyer, and this is only how the case reads to me as a layman, but I think it’s the strongest argument the other side has. Between that and the piracy, the bigger question is a long way from settled.

This is where it comes back to the hat tip, and the solution I’d like to see for all of these complex, nuanced problems. In a calm world, credit would be a simple courtesy. But we aren’t in a calm world. Copyright sets statutory damages of up to $150,000 per work for willful infringement, and a lot of lawyers see that number and essentially see a bounty, and their whole case turns on escalating litigation and trying to increase a damage award (of which they receive a substantial cut) by trying to sweep in as many works as they possibly can. In other words, they have a huge financial incentive to show that one of the AI labs creating models knew whose work it was using, knew that it was infringing, and did it anyway. Now look at what a hat tip is. The moment an AI lab prints “This answer draws on Andrew Stellman’s book,” it has stated, in writing, “We knew we were using Andrew Stellman’s book,” and a good plaintiff’s attorney could easily turn that around and make it exhibit A. Would a judge see that as willful infringement? I have no idea. But I’m not sure I would bet the entire future of a company like Anthropic or OpenAI.

If credit that would cost a lab almost nothing to give carries a legal risk that dwarfs the cost of crediting no one, then the rational move, the one any lawyer would advise, is to say nothing and admit nothing. The threat of a potential lawsuit could be enough to convince a lab that they can’t safely open the conversation about voluntary attribution at all, because anything it offers in good faith can be turned into ammunition. That’s the worst outcome for everybody, authors included, because the one technology that might be able to finally tell you where an idea came from ends up legally better off staying silent about it.

If you build with these tools, or just lean on them all day the way I do, this is a key reason the answers you get will almost never tell you where they came from, even if the technology to trace them keeps getting better. The people who build these tools could add that little “Here’s who this came from” line tomorrow. What likely stops them is a legal system that makes giving it to you too dangerous to try.

What I actually want

There’s a smaller version of the hat tip that doesn’t run into any of the legal or technical issues I’ve been talking about. Even when a model can’t say which book an answer came from, when it honestly can’t be traced, it can still point you to the authoritative sources on the subject, the ones worth reading to go deeper. That isn’t attribution in the strict sense; nobody has proven those books shaped that answer. But a model trained on a topic was almost certainly trained on the standard works for it, so the correlation is strong. For the reader it does the useful thing anyway by saying where to go next, and it’s something the AI labs can do today.

Which brings me back to that check from Anthropic I may or may not be about to cash. I meant it when I said I’d rather have the credit. What my writing has done for my career matters more to me than what it’s done for my bank account, and I would happily cut that settlement check in half if it meant twice as many people found their way to the work. A hat tip does exactly that, and it’s what I actually want, if it opens up new exposure to my work and new opportunities for authors like me.

The part I keep turning over is that the hat tip is more possible than the people who say “impossible” want to admit. I made a toy version, and researchers at AI labs are pushing these ideas a great deal further than I can. A key part of what stands in the way now is a legal question: whether good faith can survive contact with $150,000 a work. I don’t know how that one comes out. But I’m fairly sure it isn’t the math, and the math was supposed to be the hard part.

The first version of an autonomous SDLC framework driving Claude Code did the work. That generation didn’t record usage, and Claude Code’s default 30-day transcript retention erased the only other record. The bill isn’t approximately known because it’s gone. If measurement isn’t part of the pipeline, it doesn’t exist.

The instrumented factory

The second generation of the factory persists its own bill as it works. Every stage attempt writes its tokens (input, output, cache read, cache write), its cost, its model, and its failure category to a ledger. That first-generation application left no records because its framework kept none; this one cannot run without keeping them.

I should define the unit before pricing anything. In this pipeline, a story is the agile artifact we know. It’s a small requirement decomposed from an epic with acceptance criteria that give the agent its stopping condition. The Definition of Done is how the machine knows it’s finished. We humans batched stories into sprints to manage the delivery. In this pipeline, a story-build is just a story going through its own full delivery cycle. This includes writing tests first, the build, a coverage gate, review by a dedicated reviewer agent, and the merge. It also includes bugfixes and repeat requests when an agent’s reply is malformed. Up to five stories are in flight at once in isolated git worktrees. The dataset for this article covers every story the factory built in one repository between June 25 and July 18, 2026. This includes 17 runs, 193 story-builds, 374 stage attempts, and 336 session logs. The June runs used Claude Opus 4.8 and the July runs used Claude Fable 5, while Claude Haiku 4.5 handled smaller parts.

Usage lives in the ledger and the raw session logs, but they disagree. The logs are the ground truth for a reason I’ll explain later. The factory (claude-code-config) and the repository it built (local-code-bench) are both public. The methodology section includes the CSVs and extraction script so you can check every number. The production application from the opening remains private, so only its ghost appears here.

What a story costs

The factory consumed 595.7 million tokens to ship 77 stories, 7.7 million tokens per delivered story: at list prices for those models, $837.53, or $10.88 per story. The numerator includes every token thrown away, the five stories that ended FAILED, the 22 failed stage attempts, the bugfix and re-ask loops, and the retries; the denominator counts only stories that shipped.

In a previous essay I estimated my factory’s stories at “a few dollars to a few tens of dollars.” The meter’s answer is $3.02 to $43.24 with a median of $9.56, so my estimates were valid. Two findings in the table surprised me. Story points barely predict cost because the medium and large bands are only 8% apart at the median. The most expensive story was $43.24 for a 3-pointer that hit a review retry and a bugfix loop. The wall-clock mean is roughly two and a half times the median because the overnight run hit the subscription plan’s rate-limit window twice and stalled for hours. This was a billing artifact rather than an agent one.

Prices reflect Anthropic’s list rates as of the run dates.

696 stories of the ghost application at this rate is roughly 5.4 billion tokens. We’ll never know.

The factory is a reading machine

Here’s where my estimates were off. In that essay’s worked example, I priced a story as if cache writes were free. They aren’t, and they aren’t even small.

An agent resends the same instructions and repository context on every turn. The API caches that stable context, which makes a cached reread cost a tenth of fresh input, though it charges a premium to write new content into the cache.

95.4% of all tokens are cache reads. The factory rereads about 73 cached tokens for every new token it writes or receives. A dark code factory is mostly a reading machine that occasionally types.

The cost side of Table 2 shows where my estimate broke. Cache writes are only 3.3% of tokens but 31.2% of the bill. Cache traffic overall is 77% of the cost. Fresh input is merely a rounding error at 1.6% of the cost.

This shape is not a quirk of one pipeline. The cache-read share is 96.4% in my interactive framework-development sessions and 91.9% in the ghost’s surviving scraps: three independent samples, two framework generations, two working modes, same shape. It looks like a property of how agentic development consumes compute.

The practical consequence surprised me most: Cost optimization in an agentic pipeline is cache management, not prompt shortening. Context discipline, cache-tier awareness, and orchestrators that don’t stuff their own windows move the bill. Trimming your prompt wording does not.

The honest denominator

There are two ways to read the failure number. The narrow reading, attempts marked FAILED, is 5.0% of tokens. The honest reading, all rework, retries, bugfix and re-ask loops, plus the crashed sessions that streamed tokens and died, is about 13%. Public cost claims rarely say which reading they use.

Only 34 of 76 stories were clean first-pass, but rework stays cheap because retries are small relative to builds. I count the 13% as a quality bill because the gates catch problems.

I found a bug while dissecting the raw data that showed my meter lied. The ledger missed a sixth of real consumption, recording $694.65 against the logs’ $837.53. When a result envelope failed validation, the controller’s re-ask overwrote the original stage row’s usage. This erased the expensive failed session from the books, and crashed sessions never wrote back at all. 57 attempts were affected, which is why the session logs are the ground truth.

The measurement system needed auditing just like the code it measures. So I filed the bug against my own factory and let its fix pipeline handle it. It decomposed the report into three defects and repaired the overwrite and the model recording in one merged PR (issue #480, PR #482, 3,200 tests passing). The factory audited its own meter and fixed most of it, while the work to recover spend from crashed sessions is queued as open work.

Who actually pays

The marginal bill for all of this was zero. I run a $200-per-month Max 20x subscription, which is why every dollar in this piece is labeled API-equivalent.

The subscription’s real currency is quota rather than money. The overnight run stalled twice on the 5-hour rate-limit window, and ten dispatches waited 3.3 to 4.2 hours before auto-resuming. On a flat monthly fee, time is the fence.

One rolling month of measured work across all three codebases totals about $1,088 API-equivalent against the $200 fee, more than five to one, and that’s a floor, because older transcripts are purged. This proves a pricing asymmetry against list rates, not a subsidy: List price isn’t Anthropic’s cost; it includes their margin.

Can a professional, or a small firm, legitimately run on these flat fees? Nothing in the plan terms stops them. There’s no revenue test and no company-size cap. The line Anthropic draws is contractual, not financial. Individual seats run under consumer terms; a Team premium seat at $125 buys business terms and central administration, but roughly half the quota per dollar. Climbing the subscription ladder buys governance, not tokens.

This flat-fee window won’t stay open forever; quotas tighten and tiers reprice. A factory that meters itself will notice the day the trade turns. One that doesn’t will simply feel slower and poorer, without knowing why.

What the meter changes

I discovered while analyzing the data for this article that every number was produced with model routing switched off. Mechanical merges burned premium-model prices on Haiku-grade work, which accounted for 12.3% of all tokens. This means 7.7 million tokens per delivered story is the unoptimized rate. The article you’re reading found the bug, and the fix is already in the factory’s backlog.

A second find came from pointing the meter at myself. Writing the factory’s specifications—its epics and stories, in interactive sessions—consumed about 190 million tokens, which is roughly 25 stories’ worth of consumption (about $160 in converted terms). When implementation is this cheap, the code is no longer the expensive artifact. The difference between the $10.88 story and the unknowable 861,601 lines is that one pipeline wrote its bill down.

Midway through each month, I think “The next Trends is going to be small. Not much is happening.” This is the first time that I’ve been right. Was everyone on vacation in August? Am I becoming jaded? There were many model releases, though few of them seemed significant. Then again, it may be time to get over the one-upmanship by the frontier vendors and spend more time thinking about the myriad small and open-weight models. Every month, the best laptop-scale models (30B and smaller) seem closer to the leading frontier models. And every month, we’re seeing organizations realize that paying premium per-token prices for the latest frontier models gives at best a small advantage over the best open-weight models.

AI models

Capability and model size are decoupling. Several models here run comfortably on a laptop or a single accelerator while claiming performance close to much larger frontier systems. While it can be hard to work with a smaller model without thinking that you’re choosing “second best,” the biggest model isn’t always the right choice. Major releases aside, the most important news from August might be Anthropic’s deployment of watermarks for text. If the watermarking scheme works, it will be possible to tell which parts of an article like this were written by AI.

• OpenAI has announced that, beginning November 12, 2026, Cursor will no longer have access to their models.
• A mysterious model named Ox Alpha quickly became the most heavily used model on OpenRouter. Z.ai recently confirmed that Ox Alpha was GLM-5.3-Flash, a 320B open weight model that claims performance similar to Opus 4.8 and that has been deployed running entirely on Chinese chips.
• IBM’s Granite 4.2 is a small open-weight reasoning model that has been tuned for multistep tasks. It comes in 3B, 8B, and 30B sizes. It’s another model making the argument that small local models can be competitive with frontier models. 
• The team that developed Ornith-1.5 claims that they have made a major step toward self-improvement. The model supports a self-improvement loop in which it proposes new tasks, generates solutions, and uses reinforcement learning to apply the results to itself.
• DeepSeek-V4-Flash-Vision adds vision to DeepSeek V4’s capabilities. Images can be mixed with text; the model can describe images, extract text from images, and do other things that we expect from a leading LLM.
• Anthropic is now embedding watermarks into all of the text that its models generate or edit. The watermarks are apparently based on word choice; the algorithm “changes the source of randomness used to pick words.” We don’t (yet) know of any tools to detect the presence of a watermark, but there are already tools that claim to remove them. It isn’t clear that these tools work.
• A new benchmark, SWE-Bench ProMax, tests the ability of LLMs to do large-scale refactoring. It’s a multilingual benchmark based on real-world code in seven languages.
• Qwen3.8-27B is a small open-weight model that claims performance similar to Opus 4.6 max. It runs easily on a reasonably well-equipped laptop.
• Google has released Gemini 3.7 Flash, claiming improved coding and debugging.
• Z.ai has released GLM-5.3. It’s very similar to GLM-5.2, differing only in that it has received additional post-training. Z.ai claims that it’s better at code generation and long-running tasks.
• NVIDIA has released Nemotron 3.5 Lightning, an open-weight mixture-of-experts model with 30B parameters and 3B active parameters. Like many recent models, it’s optimized for long-running agents such as OpenClaw.
• Cactus Compute has released Needle 2, another small model that’s worth a look. It’s a 45B-parameter model that has been designed for “tool calling, device use, and structured extraction.” Needle requires only 28 MB of RAM, so it will run on many laptops and small devices and microcontrollers.
• Meta open-sourced Muse Glimmer, a 30B model designed for agentic applications. It can run on consumer hardware. Meta also released Muse Code and Muse Spark 1.2. Muse Code is a model designed for code generation. It implements an agent loop and a local event log that allows exact replays and restarts. Spark is a general-purpose model with near-frontier performance—Meta describes it as “a step towards the frontier.”

Software Development

Features that we associate with agents or harnesses, such as the ability to spawn subagents and delegate tasks to less-expensive models, are continuing to find their way into the models themselves. There’s also a countertrend: Individuals and organizations are building their own agents that are closely integrated into their working environment. Are we headed for walled gardens controlled by the leading providers? Or will a thousand flowers bloom, each reflecting an idiosyncratic way of working with AI? Don’t avoid tools from the major AI labs, like Claude Code and Codex, but don’t lock yourself into thinking that they’re the only option.

• DeepSeek has open-sourced Harness, its agent harness. What makes Harness unique is that almost everything is a plugin, so it’s extremely flexible. It can be used with many models, and can delegate work to Claude Code and Codex.
• TrueForge is an open source agent harness that can be used with any model. It includes tools to debug and govern agents in production.
• Computer History is a new feature of ChatGPT Work and Codex that records how you use your computer. It’s similar to Microsoft’s controversial Windows Recall, but it’s based on key clicks and other actions rather than screenshots. Data is stored locally rather than sent to OpenAI. It’s off by default.
• Zed’s Delta is a “multiplayer environment for coding with agents and reviewing what they build.” It’s a new take on Git and GitHub, designed specifically for the AI world. The company’s big insight is that the conversation about the code is as important as the code itself, and must be captured along with the source.
• Companies are now building their own agents (a.k.a. harnesses). While they’re still using AI services from Anthropic, OpenAI, and other providers, many organizations are finding that custom agents are a useful way to incorporate their own workflows into an AI-driven development process.
• Anthropic has added cross-session messaging to Claude Code. Messaging allows one agent to inform others about actions it has taken that might affect another agent’s work, reducing the need for a programmer to act as a communications medium.
• Agent Plugins is a standard for extending agents with plugins built from reusable components. It’s supported by OpenAI, Microsoft, Cursor, and AWS, though not by Google or Anthropic.
• OpenAI now has a hardware product. Codex Micro is a small terminal (certainly the wrong word) for remote AI work; it has 13 keys, a rotary encoder, a touch sensor, a joystick, and some status lights, and it hints at voice control (though I see no mention of a microphone). Its purpose is to allow you to control Codex workflows remotely.
• “Just because a feature is easy to build doesn’t mean that it is worth shipping”: Good advice on using AI effectively for software development.
• An update to the Model Context Protocol (MCP) addresses one of the most significant barriers to adoption by making it stateless.
• Software developers who didn’t grow up with Linux frequently haven’t discovered the art of the command line. Atomic Object recommends four terminal tools: Ghostty, tmux, lazygit, and lazydocker. Try one of them—or all.

Infrastructure and operations

Optimizing AI usage has become its own discipline, sometimes called “tokenomics.” Tokenomics can’t be separated from safety, which has also been much in the news. Disposable containers built for agents, GPU scheduling that treats accelerators as a heterogeneous pool, and infrastructure providers publishing how they actually serve open models at scale all match workloads to hardware without waste or risk. AI performance isn’t just about models; it’s about infrastructure. Understanding how the model is run will prove more important than the model’s specs and benchmarks.

• Taalas has built a chip that incorporates Llama 3.1 8B. All the weights are on the chip, which can’t be used for any other models. It’s extremely fast. Whether single-model chips make sense when new models are released almost daily is a good question.
• Docker Sandboxes are isolated disposable containers that are designed for running AI agents safely.
• Kubernetes’s Device Resource Allocation (DRA) makes it much easier to schedule jobs on heterogeneous clusters of GPUs.
• Cloudflare has published a description about how it runs the Kimi and GLM models at scale. It’s worth reading.
• WARP (formerly Waste) is an inference engine with one purpose: run Kimi K3 on a laptop. K3 is a 2.8T parameter model with 104B active parameters, typically requiring a small fleet of GPUs. WARP requires a 64 GB Macbook Pro with a few TB of disk. It’s slow (about 0.5 tokens/second), but it runs.

Security

Security work is inseparable from AI development, not a layer added afterward—but security professionals have been saying that about traditional software for years. Artificial intelligence is spawning new attacks as well as new defenses. While it’s always fascinating to look at new attacks, the most significant shift is in defense: rethinking security in terms of actions and resources rather than user identities, a change we’ve also covered on the Radar blog.

• Anthropic, OpenAI, Google and many other AI companies have signed an open letter saying that defense against cyberattacks has to become a priority for governments, and that governments and organizations need to act collectively to build defenses. 
• The Chrome browser has adopted device-bound service credentials (DBSC) to prevent session cookie theft, a critical step in account takeovers. DBSC stores an encryption key in a secure enclave or other trusted storage.
• There is now a Python library that supports ML-KEM and ML-DSA, NIST-standard key encapsulation and digital signature algorithms for postquantum cryptography.
• Simon Willison has published a timeline of OpenAI’s inadvertent attack against HuggingFace. His timeline is based on a postmortem that OpenAI presented at Black Hat. OpenAI has published a full incident report.
• The ChainDrop credential stealing malware has compromised over 1,300 packages on npm, the Node package manager. The malware is self-propagating, and compromised packages appear to have legitimate provenance.
• OpenAI has open-sourced Codex Security, a command-line tool and API that uses ChatGPT to analyze code for vulnerabilities. Their documentation says that the CLI and API are both in “limited beta,” possibly because of the model used to do the analysis.
• Context Collapse is a three-part series that discusses context poisoning attacks against Copilot, culminating with self-propagating attacks against Word. Microsoft collaborated on the analysis and mitigations.
• Google has introduced Beyond Zero, a new security model that takes zero trust a step further. Beyond Zero makes decisions on the basis of specific actions and resources, not just users or applications. Decisions are governed by both static policies and dynamic controls that can respond to changes in the environment.

People and Organizations

How do people use AI? Does AI use lead to greater productivity? We know surprisingly little about either question. We’re still learning how to use AI effectively; the best metric isn’t a simple measure of productivity but whether you can do things you couldn’t do before.

• The AI Observatory collects data about how people use AI. What we know about the ways people use AI is surprisingly limited. We know that usage patterns vary from model to model, but model providers only publish the data they want to see; we still don’t understand the big picture.
• How do you measure AI productivity? “Why AI Productivity Is a Faulty Metric” has some good ideas. Develop metrics around code quality and whether AI-generated code survives review, rather than counting lines of code.

Web

There’s now a specialized version of ChatGPT for teens; a site that serves different content to scrapers and humans; and an AI-generated animation of the start of The Lord of the Rings. The web is proving that it can adapt to anything that’s thrown at it. It’s where we learn and play, and AI isn’t changing that.

• OpenAI has launched ChatGPT for Teens, a specialized mode for users between 13 and 17 years old. This new product stresses learning and studying rather than using AI to get answers, has stronger content safeguards, and tries not to become a surrogate for human interaction.
• A theremin in the browser is something you don’t see every day! Use your mouse or your webcam to control it.
• TIME magazine has started giving AI scrapers a minimal Markdown version of articles with additional advertisements. The site’s behavior depends on the User-Agent HTTP header. Some user agents are denied access, while humans are given HTML with graphics and layout.
• Tired of pelicans on bicycles? Andrej Karpathy had Claude Opus animate the first paragraph of The Lord of the Rings with Three.js. The result isn’t great, but it’s certainly fun and points to some areas where the best current models aren’t yet strong enough.

Biology

• The National University of Singapore’s Life Sciences Institute now has a server rack where the computational power comes from 16 million lab-grown human neurons. Life support is a problem, but power consumption is a small fraction of the power required by GPUs.
• Claude has successfully run a complete protein design workflow, generating new designs for proteins that have been synthesized and tested in labs.
• There could be a fly on your desktop. This one is driven by a simulation of over 23,000 neurons from a fly’s connectome. It behaves like the real thing (macOS only).

How to build a source-backed briefing agent, step by step

1. Define the scope. Start by setting the analyst’s focus area, research questions, and time window, whether that’s the last one day, seven days, or 30 days. Narrowing the scope up front, including a list of preferred sources, keeps the agent’s web searches directional instead of generic.
2. Plan the coverage. A coverage planner agent breaks the research question into discrete sections, such as market backdrop or company catalysts, so the search that follows can run in parallel rather than one long sequential query.
3. Search and gather. A news researcher agent runs multiple queries against the preferred sources first, then falls back to a general search if the preferred sources don’t return enough results.
4. Validate the sources. A validation agent checks each link for a working, clickable URL and a correct publication date, removes duplicate stories covering the same news, and filters out paywalled pages that won’t return usable content.
5. Generate the briefing. A briefing writer agent assembles the validated developments into a set structure, including an executive summary, key bullet points, and a section-by-section breakdown of what each development means for the analyst and what to watch next.
6. Capture feedback into memory. A feedback agent logs corrections such as formatting or terminology preferences and saves them to a memory database, so the next briefing run applies those preferences automatically instead of requiring the analyst to re-prompt.

Jayeeta built the entire stack on open source models so newcomers can run it without an API key, and she recommends starting with a smaller model before scaling up. The takeaway extends well beyond finance. Building single-purpose agents rather than one large agent that handles every task means a failure at one stage doesn’t force a restart of the whole pipeline, and each agent’s output stays easier to trace and debug.

The full code base, including the sample data and the UI shown in the demo, is available in Jayeeta’s GitHub repo, so readers can clone it and run the briefing agent on their own systems.

Coming this week

This week, Maxim Salnikov joins Zero to Agent in 30 Minutes to build a supply chain for agent context. He’ll show how to source approved packages from a trusted registry, pin and hash-verify them on any harness, and enforce org policy with a CI gate that can’t be bypassed.