<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="https://purl.org/rss/1.0/modules/content/"
	xmlns:media="https://search.yahoo.com/mrss/"
	xmlns:wfw="https://wellformedweb.org/CommentAPI/"
	xmlns:dc="https://purl.org/dc/elements/1.1/"
	xmlns:atom="https://www.w3.org/2005/Atom"
	xmlns:sy="https://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="https://purl.org/rss/1.0/modules/slash/"
	xmlns:custom="https://www.oreilly.com/rss/custom"

	>

<channel>
	<title>Radar</title>
	<atom:link href="https://www.oreilly.com/radar/feed/" rel="self" type="application/rss+xml" />
	<link>https://www.oreilly.com/radar</link>
	<description>Now, next, and beyond: Tracking need-to-know trends at the intersection of business and technology</description>
	<lastBuildDate>Tue, 17 Nov 2026 16:55:18 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1</generator>

<image>
	<url>https://www.oreilly.com/radar/wp-content/uploads/sites/3/2025/04/cropped-favicon_512x512-160x160.png</url>
	<title>Radar</title>
	<link>https://www.oreilly.com/radar</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>The Value of Human Expertise in The Software Factory</title>
		<link>https://www.oreilly.com/radar/the-value-of-human-expertise-in-the-software-factory/</link>
				<comments>https://www.oreilly.com/radar/the-value-of-human-expertise-in-the-software-factory/#respond</comments>
				<pubDate>Tue, 17 Nov 2026 16:55:03 +0000</pubDate>
					<dc:creator><![CDATA[Louise Cox]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20241</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/The-value-of-human-expertise-in-the-software-factory-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/The-value-of-human-expertise-in-the-software-factory-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[“Any sufficiently advanced technology is indistinguishable from magic.” – Arthur C. Clarke Most developers now write less code by hand. Gergely Orosz recently posted that modern software engineering is no longer done without LLMs and will no longer be done without them. Some miss writing code, while others question how much of the code that [&#8230;]]]></description>
								<content:encoded><![CDATA[
<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph"><em>“Any sufficiently advanced technology is indistinguishable from magic.” – Arthur C. Clarke</em></p>
</blockquote>



<p class="wp-block-paragraph">Most developers now write less code by hand. Gergely Orosz recently <a href="https://lnkd.in/p/eZEBWQ5C" target="_blank" rel="noopener">posted</a> that modern software engineering is no longer done without LLMs and will no longer be done without them. Some miss writing code, while <a href="https://www.linkedin.com/posts/amandamayfielddev_okay-so-i-see-people-posting-about-not-share-7515808898981748736-pXs7/?utm_source=social_share_send&amp;utm_medium=member_desktop_web&amp;rcm=ACoAAAoBcr4BDAP5uaUTUZLtclCvsD9TH6BHG-o" target="_blank" rel="noopener">others</a> question how much of the code that agents produce should be read. Writing and reading code gave a lot of developers satisfaction and identity, and as Kent Beck has <a href="https://howmanyctos.com/episodes/92-the-human-needs-augmented-coding-conversation-kent-beck/" target="_blank" rel="noopener">put it</a>, a sense of competence. I saw this in real time.</p>



<p class="wp-block-paragraph">Ten years ago, I’d sit at conferences and feel a sense of wonder whenever anyone was brave enough to do a live demo. I felt gleeful when they wrote lines of code, went back to fix mistakes, and made something cool and interesting appear on the screen in front of me. It was magical, and I’m sure that magic is why a lot of developers got into this industry. I can partly understand the sense of loss that people are experiencing. I saw magic, they were in charge of the craft of their code, and that hands-on task has now been given to agents. Where does that expertise go when AI is now writing so much of the code?</p>



<p class="wp-block-paragraph">As Tim O’Reilly wrote in his recent <a href="https://oreillyradar.substack.com/p/superpowers-for-humans?r=1fls4c&amp;utm_campaign=post-expanded-share&amp;utm_medium=web" target="_blank" rel="noopener">Radar post</a>, O’Reilly is a business built on cultivating and sharing human expertise. Our job is to find the high ground where that expertise still matters. For developers working in a new world of agentic software factories, I think the expertise you built writing and maintaining software hasn’t gone, it’s just moved. Your expertise now shapes how agents produce software and what evidence you need before we trust the result. To me, AI is enabling you to be magical <a href="https://lnkd.in/p/eFSWBsav" target="_blank" rel="noopener">faster</a>.</p>



<p class="wp-block-paragraph">Agents amplify whatever you give them, but you need to understand it in the first place. I couldn’t swarm a fleet of agents in the same way a developer can because I don’t have the expertise. You need to understand code. The expertise has gone to a new level of abstraction.</p>



<p class="wp-block-paragraph">When I spoke recently with Avi Flombaum, the founder of Flatiron School, he agreed that developers’ value was never solely in writing or reading code. AI changes the mechanics, but experience still matters for understanding complex problem domains, articulating what needs to be built, and bringing judgment and ambition to the work. In a separate conversation with Ken Kousen, he pointed out, “We’ve been working with third-party code generators for years; we just called it outsourcing.” This new world of designing AI workflows, setting guardrails, and managing agents much as you would an engineering team, reveals a chance to use accumulated expertise more. It’s your software engineering expertise that matters.</p>



<p class="wp-block-paragraph">Software factories make this shift especially visible. Companies are now putting agents to work across the development lifecycle, aiming to take work from an initial signal through implementation, validation, and deployment without a person at every step. But generating more code is only part of that process. Without changes elsewhere, you can burn a lot of tokens producing a pile of pull requests that someone still has to review. Anthropic’s <a href="https://claude.com/resources/articles/the-ai-native-sdlc-playbook" target="_blank" rel="noopener">AI-native SDLC playbook</a> points out that speeding up implementation shifts pressure onto planning, testing, review, handoffs and approvals, and faster coding tools don’t speed those up on their own. Someone has to understand the whole process well enough to redesign it.</p>



<p class="wp-block-paragraph">In a software factory, that judgment has to be built into the system. Humans design the workflow, define the conditions under which work can proceed, and decide which exceptions need their attention. Anthropic <a href="https://claude.com/blog/how-anthropic-secures-its-ai-native-software-development-lifecycle" target="_blank" rel="noopener">reports</a> that Claude authors about 80% of the code merged into its codebase, alongside a review process that combines automated checks with human oversight according to risk.</p>



<p class="wp-block-paragraph">The state of the code matters too. <a href="https://codescene.com/blog/unhealthy-code-is-burning-your-token-usage-heres-the-data" target="_blank" rel="noopener">Adam Tornhill</a> and the CodeScene research team found that across C++, Java, and Python codebases, agents working on unhealthy code used 35–45% more tokens to complete the same tasks. That makes refactoring the parts of a codebase agents touch most a sensible investment, but someone has to know which parts those are, decide what “better” looks like, and confirm that behavior hasn’t changed.</p>



<p class="wp-block-paragraph">Context works the same way. In <a href="https://newsletter.port.io/p/how-to-build-context-lake-that-saves-tokens" target="_blank" rel="noopener">Port’s testing</a>, giving agents structured context cut costs by about 58% compared with direct access to separate MCP servers. Adding instructions on where to find relevant information pushed the savings to around 80%. Those figures are specific to Port’s experiment, but the work behind them will be familiar to any experienced developer: knowing which relationships in a system matter, and keeping that knowledge accurate as the system changes. Testing tells you whether a particular change works, but evaluating the workflow tells you how reliably it produces acceptable changes and when a person needs to step in.</p>



<p class="wp-block-paragraph">In <em><a href="https://learning.oreilly.com/library/view/-/9798341660717/ch03.html" target="_blank" rel="noopener">Evals for AI Engineers</a></em>, Shreya Shankar and Hamel Husain describe starting with real traces of what the system did. First comes open coding: reading the traces and writing short notes on what went wrong. Then comes axial coding: grouping those notes into failure categories based on what you actually see. A developer who has spent years debugging production systems will spot the failures that matter and set aside the ones that don’t far faster than someone who hasn’t. That’s where developers’ experience shows up now: in the problems they define, the standards they set, and the evidence they demand before anything ships.</p>



<p class="wp-block-paragraph">Every shift in every industry has made some people miss the old ways, but the industry has survived. Back in December 2025 I <a href="https://www.oreilly.com/radar/software-in-the-age-of-ai/" target="_blank" rel="noopener">wrote</a>: “AI rewards strong engineering; it doesn’t replace it,” and I stand by that statement. Developers are still magical to me, even when they’re not the ones writing the code. I couldn’t build a software factory myself because I don’t have the years of experience it takes to know where agents should work, how to guardrail them, and how to judge what they produce. The developers who do are the ones I trust to build it. And, as Ben O’Mahony pointed out to me recently, “Those who have a lot of exposure to Agentic engineering have seen past the magic and are back to engineering!” It’s an exciting time to be involved.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<p class="wp-block-paragraph">Want to understand how to think about engineering craft in this new world? Join Avi Flombaum for his new series, <a href="https://learning.oreilly.com/live-events/the-agentic-rethink-with-avi-flombaum/0642572481711/" target="_blank" rel="noopener">The Agentic Rethink</a>, starting Wednesday, Jan. 13, 2027, on the O’Reilly platform.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/the-value-of-human-expertise-in-the-software-factory/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>When AI-Generated Data Becomes Tomorrow’s Ground Truth</title>
		<link>https://www.oreilly.com/radar/when-ai-generated-data-becomes-tomorrows-ground-truth/</link>
				<comments>https://www.oreilly.com/radar/when-ai-generated-data-becomes-tomorrows-ground-truth/#respond</comments>
				<pubDate>Tue, 17 Nov 2026 11:54:15 +0000</pubDate>
					<dc:creator><![CDATA[Shreshta Shyamsundar and Chidambaram GS]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20237</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/When-AI-generated-data-becomes-tomorrows-ground-truth-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/When-AI-generated-data-becomes-tomorrows-ground-truth-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[Manufacturers qualify suppliers before trusting them with critical work. Before a supplier can provide parts for a regulated product such as a medical device, the manufacturer collects evidence: quality certifications, test reports, insurance documents, and regulatory approvals. Increasingly, an AI agent reads that evidence first. The agent reviews the documents and proposes a finding, and [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">Manufacturers qualify suppliers before trusting them with critical work. Before a supplier can provide parts for a regulated product such as a medical device, the manufacturer collects evidence: quality certifications, test reports, insurance documents, and regulatory approvals. Increasingly, an AI agent reads that evidence first. The agent reviews the documents and proposes a finding, and a business process decides whether the finding is good enough for the supplier to be accepted. Once a supplier is accepted, that decision is recorded in the manufacturer’s supplier-management system. From there, sourcing workflows, readiness dashboards, and program reports all rely on that record without repeating the review.</p>



<p class="wp-block-paragraph">Here is how that process can go wrong. A supplier-review agent examines a new supplier’s documents and concludes that a mandatory quality certificate is valid. The qualification workflow accepts the finding, the supplier is recorded as compliant, and the procurement lead approves the supplier for an upcoming order. A few days later, a human reviewer notices that the certificate does not cover the part being sourced; the agent had misread the certificate’s scope. At 11:00am, the reviewer corrects the supplier record and sends the supplier back for review. At 2:00pm, the readiness dashboard still shows the supplier as cleared, and a program report still lists the supplier as compliant. The procurement lead’s sourcing decision remains in force, and nothing tells the procurement lead that the evidence behind it has changed.</p>



<p class="wp-block-paragraph">A misread scope is only one way a certificate finding can fail. A certificate may be revoked after it was checked, expire before the order ships, apply to a different plant than the one making the part, or turn out to be forged. In each case, someone can correct the supplier record while the systems that already used the old finding keep acting on it.</p>



<p class="wp-block-paragraph">No system in this example malfunctioned. Each one processed information that appeared valid under its local contract, yet the combined result was wrong. The manufacturer had corrected a value without correcting its influence. Part of this is a familiar problem: Enterprise systems have always struggled to keep reports, indexes, caches, and workflow state in line with the source record. Agents make the problem harder because they generate conclusions. A conclusion can be assembled from several sources, rewritten in reports, and retrieved later as if it were independent evidence. Reconciliation can make stored values agree. It cannot reveal that differently worded records come from the same conclusion, or find the decisions that already relied on that conclusion.</p>



<p class="wp-block-paragraph">This problem is not limited to manufacturing. In our work across banking, telecommunications, and resources, it appears wherever an agent’s output becomes a record that other systems trust. A fraud classification is written into a case record. A risk assessment is summarized in a management report. An incident diagnosis becomes part of a knowledge base. In each of these cases, as in the supplier example, the conclusion loses some of its original context at every step it passes through and begins to influence decisions made later.</p>



<p class="wp-block-paragraph">The conclusion can also travel back into model development. Accepted findings become labeled examples, evaluation sets, and fine-tuning data, where they are treated as ground truth for the next model. If an incorrect supplier finding survives correction in the operational systems, a future model may learn it. Whether the conclusion ends up in a dashboard, a report, or a training set, the hard problem is the same: finding every consequential decision that depends on a result that can change.</p>



<h2 class="wp-block-heading"><strong>When an answer acquires authority</strong></h2>



<p class="wp-block-paragraph">Look again at the supplier-review agent’s conclusion that the certificate is valid. When the agent produces that conclusion, it is a generated assertion: a claim made by a model, with no business authority until the qualification workflow accepts it for a specific use, such as approving the supplier for sourcing. The words “certificate valid” also change character as they move. In the agent’s response, they are visibly part of an AI-assisted assessment. In the supplier master record, the same words look like an established fact. The dashboard strips away more context: It shows the supplier as cleared, with no sign of which checks were incomplete or whether a review was pending. The program report compresses the finding into a sentence, and the search index splits that sentence into passages that can be retrieved on their own.</p>



<p class="wp-block-paragraph">Along the way, enterprise systems tend to keep the value the agent produced and lose the conditions under which it was produced. A label saying that the finding was AI-generated may survive on the supplier record and disappear from a dashboard total, a data export, or a retrieval chunk. As that context falls away, the conclusion takes on the authority of whichever system is carrying it.</p>



<p class="wp-block-paragraph">That is why two questions need to be kept separate: Who produced a result? And who is allowed to rely on it? An agent may be allowed to record a proposed finding without being allowed to establish a business fact. The same finding may be good enough to move a supplier up a review queue and not good enough to approve the supplier or release a payment. Where that line falls depends on what is at stake and which controls grant authority for that use. Once the qualification workflow accepts a finding, downstream systems usually treat it as settled and do not repeat the checks. That is efficient, and it creates an obligation: Those systems still need to know which decisions rely on the finding and under what conditions.</p>



<h2 class="wp-block-heading"><strong>When repetition looks like corroboration</strong></h2>



<p class="wp-block-paragraph">Acceptance also multiplies the finding. Once the supplier is recorded as compliant, the readiness dashboard shows the supplier as cleared, the sourcing workflow moves forward, and the program report repeats the approval. A later agent, when asked about the supplier, can retrieve the supplier record and the program report as two separate documents, each appearing to support the same conclusion. Across the supplier record, the program report, and the dashboard, one says the evidence was verified, another says the controls passed, and a third shows an approved outcome. What appears to be corroboration is propagation: several records repeating one finding. Because each system changes the wording or format, the duplication is easy to miss.</p>



<p class="wp-block-paragraph">A stronger model will not reliably catch this duplication when the retrieval system presents the copies as unrelated documents. Reranking improves relevance, but every copy may be relevant. Telling the model to look for independent evidence cannot restore ancestry that was discarded before the documents reached it. Agent memory makes the problem worse. An agent that records its own finding and retrieves it a week later is citing itself, and nothing in the retrieved passage says so. In multi-agent workflows, one agent’s summary becomes another agent’s context, and a third agent’s supposedly independent check may read both.</p>



<h2 class="wp-block-heading"><strong>Decision lineage is the missing recovery path</strong></h2>



<p class="wp-block-paragraph">Existing tools cover parts of this problem. W3C PROV models the entities, activities, and agents involved in producing data, establishing origin and responsibility.<sup>[1]</sup> OpenLineage standardizes lineage metadata around datasets, jobs, and runs.<sup>[2]</sup> Event sourcing records changes to application state as a sequence of events.<sup>[3]</sup> PROV is expressive enough to model the whole supplier example. The gap is in implementation. Lineage tooling often stops at dataset and job boundaries and rarely reaches the decision record or the retrieval layer that assembles an agent’s context. As usually deployed, none of these tools answers the question that matters after the certificate finding changes: Which decisions still depend on it? Decision lineage extends these mechanisms. It records the relationship between the source evidence, the generated assertion, the authority granted to that assertion, and the decisions that rely on it.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="has-text-align-center wp-block-paragraph"><em>Data lineage tells us where an assertion traveled. Decision lineage tells us what the organization did because it believed the assertion.</em></p>
</blockquote>



<p class="wp-block-paragraph">In the supplier example, the chain runs from the certificate to the agent’s finding to the accepted supplier record to the procurement lead’s sourcing decision and on to the dashboard, report, and search index that reuse the finding. If the certificate turns out to be invalid, that chain lets the system find the sourcing decision and identify every report, workflow, and AI output that inherited the old finding, before any of them influence another decision.</p>



<p class="wp-block-paragraph">The chain also marks where automated recovery and business accountability separate. Software can identify the affected artifacts, stop the withdrawn finding from appearing as current evidence, and open a review task for the person who owns the decision. That person decides whether the decision still stands, needs to be reversed, or needs to be communicated to customers or regulators. An implementation might remove stale search entries automatically while still requiring human approval to change a sourcing decision that has already been made.</p>



<h2 class="wp-block-heading"><strong>A production pattern for preserving decision dependence</strong></h2>



<p class="wp-block-paragraph">The pattern starts where the agent’s output enters the system of record. The agent records its finding as a proposal, tied to the evidence it reviewed. A deterministic workflow, owned by the business, decides whether that proposal gains authority for a defined use. If a required check cannot run, the finding stays a proposal and is never silently promoted. Keeping proposals separate from accepted findings prevents accidental promotion, but it does not record what happens next, when the sourcing decision relies on the accepted finding. The decision needs a link back to that finding and to the version of the evidence behind it. This does not require a universal graph of everything the agent touched. For a high-consequence decision, it is enough that the finding’s influence can still be traced after the workflow has finished.</p>



<p class="wp-block-paragraph">The dependency record that holds this link can be small. For the supplier example, it might look like this:</p>



<pre class="wp-block-code"><code>{
  "assertion_id": "asr-7781",
  "claim": "Mandatory certificate valid for supplier S-204",
  "evidence_snapshot": "evd-2291@v3",
  "authority": {
    "granted_by": "qualification-workflow",
    "scope": &#91;"sourcing-approval"]
  },
  "dependent_decisions": &#91;"sourcing-5520"],
  "derived_artifacts": &#91;"dashboard:readiness",
                        "report:program-q3",
                        "index:supplier-docs"],
  "status": "accepted"
}
</code></pre>



<p class="wp-block-paragraph">When the certificate finding is withdrawn, the record’s status changes to withdrawn, and the <code>dependent_decisions</code> and <code>derived_artifacts</code> fields become the list of everything that needs to be revisited.</p>



<p class="wp-block-paragraph">The dependency record has to survive as the finding is copied downstream, because the same finding may come back through the program report or the search index. When a later agent retrieves both the supplier record and the program report, the retrieval layer should recognize that they share an origin and present them as two copies of one finding. If the finding has been withdrawn, the retrieval layer should filter or flag both copies before they reach the agent.</p>



<p class="wp-block-paragraph">When a finding turns out to be wrong it is withdrawn, and the supplier goes back to review. The dependency record lets services find the decisions that relied on the finding, recompute dashboards, flag reports for regeneration, remove stale search results, and stop pending workflows. Decisions that have already taken effect go back to their owners, and nothing is reversed automatically. Cache invalidation, vector-index refresh, report regeneration, and workflow reversal are all parts of the same recovery job. A correction has traveled far enough only when the old finding no longer drives any current decision.</p>



<h2 class="wp-block-heading"><strong>Where the pattern is worth the cost</strong></h2>



<p class="wp-block-paragraph">Not every AI output needs decision lineage. An answer that is low-stakes, never stored, never reused as evidence, and cheap to regenerate can live with ordinary logging. Decision lineage earns its cost once an AI-generated finding can affect an approval, a payment, safety, compliance, or later machine decisions. By the time an error in such a finding is discovered, the finding may already have changed what the organization did, so recovery has to finish within the time the business process can tolerate. That recovery work, finding and reconsidering the decisions affected when evidence changes, is what the pattern makes cheaper.</p>



<p class="wp-block-paragraph">The pattern also does not require tracking every word an agent produces. The unit to track is the material assertion: a specific claim, such as “the certificate is valid,” that could change a decision if it were wrong. A long supplier assessment may contain only one or two material assertions. Recording dependencies for those claims is more useful than keeping a complete trace of everything the agent wrote, and it costs less in storage, privacy exposure, and operations.</p>



<p class="wp-block-paragraph">Tracking ancestry also has to be precise enough to keep independent evidence independent. If two laboratories separately validate the same certificate, the system should not merge their results into a single source just because they agree. What matters is whether the observations share an origin, and similar wording proves nothing either way.</p>



<h2 class="wp-block-heading"><strong>How to test whether correction traveled far enough</strong></h2>



<p class="wp-block-paragraph">Recording dependencies proves little unless a correction actually travels through them, so the pattern needs a test. In a controlled environment, introduce an incorrect finding, let it reach a report and a search index, and then withdraw it. The test passes only if the affected decision can be found, and the stale copies stop appearing as evidence within the recovery time the process requires.</p>



<p class="wp-block-paragraph">Three indicators show how well the test went. Decision coverage is the share of high-consequence decisions that can be traced back to a supporting finding. Time to impact is the time between withdrawing a finding and identifying the decisions it affected. Stale-evidence leakage counts the withdrawn copies still returned as current evidence. Targets depend on the process. A payment approval might need affected decisions identified within the hour, with zero leakage, while a review-queue priority may tolerate a day.</p>



<h2 class="wp-block-heading"><strong>Replay the supplier decision</strong></h2>



<p class="wp-block-paragraph">Now return to the supplier example and run it again with the pattern in place. When the supplier-review agent concludes that the certificate is valid, its finding is recorded as a proposal tied to the documents it reviewed. The qualification workflow checks the finding, grants it authority for the sourcing decision, and writes the dependency record shown earlier. The dashboard and program report still pick up the accepted finding, and a later agent that retrieves both sees them as two copies of one finding.</p>



<p class="wp-block-paragraph">When the reviewer corrects the record at 11:00am, the finding is withdrawn and the supplier goes back to review. This time the correction travels. The dashboard recomputes, the program report is flagged for regeneration, stale entries drop out of the search index, and the procurement lead gets a task to review the sourcing decision. By 2:00pm, the dashboard shows the supplier as under review.</p>



<p class="wp-block-paragraph">The procurement lead can now see that the certificate finding changed, which decision relied on it, and which reports and records carried it. The original mistake still happened. Its spread is now visible, and its consequences go back to someone who can act on them.</p>



<h2 class="wp-block-heading"><strong>Ground truth begins with the next decision</strong></h2>



<p class="wp-block-paragraph">In the supplier example, the procurement lead never treated the model’s response as final. The finding became organizational truth because the supplier system accepted it, the workflow advanced, the report repeated it, and later systems inherited its authority.</p>



<p class="wp-block-paragraph">Decision lineage does not prove that the original evidence is authentic, replace data-quality or entitlement controls, or make a model’s conclusions accurate. It addresses what happens after a finding exists: how its influence is recorded, found, and reconsidered across the decisions that depend on it. For the manufacturer, correcting the certificate record was only the first step. Recovery also meant finding the sourcing decision that relied on the finding and stopping stale copies from coming back as current evidence.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph"><strong>AI-generated data becomes ground truth the moment the next decision stops asking where it came from.</strong></p>
</blockquote>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h3 class="wp-block-heading">References</h3>



<ol class="wp-block-list">
<li>World Wide Web Consortium, “PROV-Overview: An Overview of the PROV Family of Documents,” April 30, 2013. <a href="https://www.w3.org/TR/prov-overview/" target="_blank" rel="noopener">View the W3C PROV overview.</a></li>



<li>OpenLineage, “An Open Framework for Data Lineage Collection and Analysis.” <a href="https://openlineage.io/" target="_blank" rel="noopener">View the OpenLineage documentation.</a></li>



<li>Martin Fowler, “Event Sourcing,” December 12, 2005. <a href="https://martinfowler.com/eaaDev/EventSourcing.html" target="_blank" rel="noopener">Read “Event Sourcing.</a>”</li>
</ol>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/when-ai-generated-data-becomes-tomorrows-ground-truth/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>The Monday Morning Agent: Are Kickers Too Good Now?</title>
		<link>https://www.oreilly.com/radar/the-monday-morning-agent-are-kickers-too-good-now/</link>
				<comments>https://www.oreilly.com/radar/the-monday-morning-agent-are-kickers-too-good-now/#respond</comments>
				<pubDate>Mon, 16 Nov 2026 16:54:44 +0000</pubDate>
					<dc:creator><![CDATA[Chester Ismay and Michelle Smith]]></dc:creator>
						<category><![CDATA[The Monday Morning Agent]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20223</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/10/USE-THIS-the-monday-morning-agent-radar-1600x840-1.png" 
				medium="image" 
				type="image/png" 
				width="1600" 
				height="840" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/10/USE-THIS-the-monday-morning-agent-radar-1600x840-1-160x160.png" 
				width="160" 
				height="160" 
			/>
		
				<custom:subtitle><![CDATA[Are kickers getting better, or are coaches giving them more chances?]]></custom:subtitle>
		
				<description><![CDATA[“Kickers are too good now” sounds plausible when long field goals keep making the highlights. But in the November 9 episode of The Monday Morning Agent, our AI analyst found that long kicks went through at roughly the same rate across the seasons on file. Coaches were attempting them more often. We gave the agent [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">“Kickers are too good now” sounds plausible when long field goals keep making the highlights. But in the November 9 episode of <em>The Monday Morning Agent</em>, our AI analyst found that long kicks went through at roughly the same rate across the seasons on file. Coaches were attempting them more often. We gave the agent written visualization standards so its charts would show both success rates and attempts per game, with the context readers need to judge the claim.</p>



<iframe width="800" height="450" src="https://www.youtube.com/embed/4cHU7lc_leM?si=u8Uudt36y330p1RL" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



<h2 class="wp-block-heading"><strong>Decide how to draw the trend before seeing the answer</strong></h2>



<p class="wp-block-paragraph">We started with two charts of league-wide field goal success rates. One began in 2024, with a vertical axis starting around 84%. The line climbed to 85.6%, and the improvement looked dramatic.</p>



<p class="wp-block-paragraph">The other included every season from 2021 onward, started the vertical axis at zero, and showed the attempt count beneath each point. Success rates stayed around 84% to 86%. The recent rise looked much smaller in that context.</p>



<p class="wp-block-paragraph">Changing the starting year and axis made the recent improvement look larger. Before your agent draws a trend, decide which choices it can make. Otherwise, it can produce a persuasive picture without giving readers the context to assess it.</p>



<h2 class="wp-block-heading"><strong>Keep the definitions when you add a new capability</strong></h2>



<p class="wp-block-paragraph">The dictionary from episode two still applies. During a rehearsal, that earlier agent refused the kicking claim because its dictionary could describe a rush but couldn’t describe a kick. We expanded what it could define while keeping the requirement to use a saved definition.</p>



<p class="wp-block-paragraph">This time, the play descriptions named field goals and included distances. We still had to decide where “long” starts. A recognizable play name doesn’t settle how to group it for analysis.</p>



<p class="wp-block-paragraph">We examined distance bands separately. Kickers made roughly 95% of attempts under 40 yards. Success rates hovered around 80% for attempts from 40 to 49 yards and around 70% for those from 50 to 59 yards. Combining them into one rate can hide the differences the question asks about.</p>



<h2 class="wp-block-heading"><strong>Make the chart standard specific enough to enforce</strong></h2>



<p class="wp-block-paragraph">We introduced eight rules. Every trend includes all seasons on file, and axes start at zero. Each point shows its underlying count. Yellow identifies the subject of the claim, blue identifies the comparison, and gray marks the current season.</p>



<p class="wp-block-paragraph">The chart title asks a question rather than announcing a conclusion. A source line gives the exact command that produced the data, so readers can rerun it. The chart shows only values the tool printed, without added trend lines or numbers typed by hand. Every chart uses the same layout, with success rates above attempts per game.</p>



<p class="wp-block-paragraph">Writing these standards down gives the agent consistent instructions and gives readers a familiar way to compare results across episodes. Choose standards that fit your analysis, then specify how you’ll check each one.</p>


<div class="wp-block-image">
<figure class="aligncenter size-large"><a href="https://raw.githubusercontent.com/ismayc/monday-morning-agent/7f6f580/charts/long-field-goal-trend.svg"><img fetchpriority="high" decoding="async" width="1600" height="1170" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9-1600x1170.png" alt="" class="wp-image-20224" style="aspect-ratio:1.3676470588235294" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9-1600x1170.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9-300x219.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9-767x561.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9-1536x1124.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-9.png 2048w" sizes="(max-width: 1600px) 100vw, 1600px" /></a></figure>
</div>


<h2 class="wp-block-heading"><strong>Test a chart that breaks the rules</strong></h2>



<p class="wp-block-paragraph">A written standard needs a way to enforce it. We added a renderer, the program that turns the agent’s chart specification into a picture. It checks the chart specification against the rules before drawing the chart and refuses specifications that break them.</p>



<p class="wp-block-paragraph">We deliberately requested a trend beginning in 2023. The renderer refused because the first season on file was 2021. Its response identified the rule instead of producing the shortened chart.</p>



<p class="wp-block-paragraph">Give your agent a request that violates a standard and inspect the response. That check shows whether the restriction works when the agent tries to produce an output.</p>



<h2 class="wp-block-heading"><strong>Show how much evidence sits behind each rate</strong></h2>



<p class="wp-block-paragraph">The longest kicks need particular care. The episode’s current-season sample contained only nine attempts in the group of kicks from 60 yards or farther. The next kick could noticeably change that rate. Displaying counts beside percentages helps readers distinguish a small sample from hundreds of attempts.</p>



<p class="wp-block-paragraph">Showing success rates alongside attempts per game helped explain the ruling. In these plays, long kicks weren’t consistently becoming more successful; coaches were trying them more often. More attempts give fans more opportunities to see a long field goal go through.</p>



<p class="wp-block-paragraph">We also corrected episode two’s results after finding an error in the tool. Our defined substitute for the tush push had a conversion advantage of 10.1 percentage points overall and 9.7 without Philadelphia, replacing the previously reported 8.3 and 7.8. Chart standards help readers inspect results, but we still have to check the calculations behind them.</p>



<h2 class="wp-block-heading"><strong>Next up: “The Pundit Audit”</strong></h2>



<p class="wp-block-paragraph">On November 16, we’ll add a second agent to independently audit an audience-selected claim. We’ll examine where the agents disagree and how they investigate the evidence. The agent doing the analysis shouldn’t also grade its own work.</p>



<p class="wp-block-paragraph">O’Reilly members can <a href="https://learning.oreilly.com/live-events/the-monday-morning-agent/0642572456054/" target="_blank" rel="noopener">sign up on the O’Reilly learning platform</a> to join us live on November 23, 8:00am–8:45am PT / 11:00am–11:45am ET. Everyone can follow along on Radar every other Monday, with a brief pause for the holidays. Watch episode three above, and we’ll build on it in the next session.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/the-monday-morning-agent-are-kickers-too-good-now/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>When AI Can Code, What Is Left for Programmers to Do?</title>
		<link>https://www.oreilly.com/radar/when-ai-can-code-what-is-left-for-programmers-to-do/</link>
				<comments>https://www.oreilly.com/radar/when-ai-can-code-what-is-left-for-programmers-to-do/#respond</comments>
				<pubDate>Mon, 16 Nov 2026 11:54:16 +0000</pubDate>
					<dc:creator><![CDATA[Shahid N. Shah and Andy Oram]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Software Development]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20230</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/When-AI-can-code-whats-left-for-programmers-to-do-Image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/When-AI-can-code-whats-left-for-programmers-to-do-Image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[AI is in such widespread use for programming that project maintainers (particularly for free and open source software) are complaining about the burden of reviewing mounds of low-quality code. In fact, AI can help with every stage of code development: writing the specification, coding, uncovering bugs, and repairing the code. This article describes human-AI collaboration [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">AI is in such widespread use for programming that project maintainers (particularly for free and open source software) are complaining about <a href="https://github.blog/open-source/maintainers/welcome-to-the-eternal-september-of-open-source-heres-what-we-plan-to-do-for-maintainers/" target="_blank" rel="noopener">the burden of reviewing mounds of low-quality</a> code. In fact, AI can help with every stage of code development: writing the specification, coding, uncovering bugs, and repairing the code. This article describes human-AI collaboration at specification and code repair stages, and lays out the skills that humans need to accomplish those tasks in the current generation of AI. We won’t cover coding and bug finding because they are widely discussed elsewhere.</p>



<h2 class="wp-block-heading">Let the AI harness write the spec</h2>



<p class="wp-block-paragraph">We don’t want developers hand-writing specs. We want them in a room, or on a call, talking with the product manager (PM) and the customer.</p>



<p class="wp-block-paragraph">Humans reach a limit when writing. We skip the boring cases. We get tired by requirement 40. We use “customer” on page 1 and “client” on page 6 of the spec. We write down what we remember and forget what we assumed. AI has the patience and diligence we lack.</p>



<p class="wp-block-paragraph">It’s still up to human developers to write <em>guardrails</em> or <em>constraints</em> into the spec: statements to ensure that nothing wildly wrong makes it into code. Each guardrail should be one sentence that says MUST or MUST NOT, with a proof beside it: a test, a check, or a named person who reviews it. A guardrail with no proof is a wish. Write the guardrails first, because the PM won’t think of most of them. What must never happen? Who must never see this? What should the agent never touch?</p>



<p class="wp-block-paragraph">With guardrails in place, capture the conversation and let the machine do the writing. Two AI tools have shown themselves particularly useful to analyze requirements: The <a href="https://www.plaud.ai/" target="_blank" rel="noopener">Plaud</a> Note recorder attaches to the back of your phone and records in-person conversations and phone calls; the wearable NotePin records in-person conversations, not phone calls. <a href="https://www.granola.ai/" target="_blank" rel="noopener">Granola</a> runs on your laptop or phone, uses your computer’s audio, and analyzes what it captures. We find these products to be a leap above common AI tools for summarizing conversations, when it comes to specification development.</p>



<p class="wp-block-paragraph">Plaud and Granola (if you prompt them to do so) can notice what’s missing and tell you the gaps in the requirements. They ask, What happens when this field is empty? What if two people do this at once? Who is allowed to see it? over and over, for every requirement, and check each noun in the requirements against an ontology created by the programmer (often with the help of AI).</p>



<p class="wp-block-paragraph">The developer, the PM, and the customer read the output from the AI, fix what’s wrong, answer what’s missing, and go through the process again.</p>



<p class="wp-block-paragraph">The spec that comes out of that loop tends to be more complete and more consistent than what the same people write alone. That’s Shahid’s own experience, not a study, but it matches reports from some other people at the leading edge of the use of AI for software development. Mark Pike, a product lawyer at Anthropic, builds working tools for his legal team without writing code: “<a href="https://claude.com/resources/articles/how-anthropic-uses-claude-legal" target="_blank" rel="noopener">I just typed a normal sentence, describing what I wanted. And it worked</a>.” StrongDM, which runs a software factory where agents write and test the code, says the starting point for a piece of software can be “<a href="https://factory.strongdm.ai/principles" target="_blank" rel="noopener">a few sentences, a screenshot, or an existing codebase</a>.”</p>



<p class="wp-block-paragraph">Keep the spec after the feature ships. As you prepare a new version of the product next year, it can tell you why the code looks the way it does.</p>



<h2 class="wp-block-heading">The data schema is the other half of the spec</h2>



<p class="wp-block-paragraph">Most teams treat the data schema as plumbing. Tables get named by whoever got there first. The spec, reflecting human conversation, may say <em>customer</em>, <em>account</em>, <em>client</em>, and <em>org</em> at different places for the same thing, and you have to choose one noun for the schema. A human learns the mapping by sitting next to the person who wrote it. An AI agent has no neighbor to consult, so it guesses—with results that likely differ from Monday to Tuesday.</p>



<p class="wp-block-paragraph">So the developer must keep an <em>ontology</em>, which here means a written, machine-checkable account of the things the program manipulates, how they relate, and who may see them. For example, in an appointment scheduling application, the ontology might define a Patient, Doctor, Appointment, and Time Slot. It would specify that a Patient can book multiple Appointments, each Appointment must have exactly one Doctor, a Doctor cannot have two Appointments in the same Time Slot, and only authorized people can view or change an Appointment. These aren’t just database tables or fields. They’re the business concepts, relationships, and rules that give the data meaning.</p>



<p class="wp-block-paragraph">Creating this ontology requires developers to identify terms, resolve ambiguities, and agree on rules that everyone might otherwise assume are obvious. It’s rigorous, sometimes tedious work, but it’s an essential human responsibility. The ontology connects what the business wants (requirements), how the software should behave (specification), and how information is stored (schema). It also gives AI coding agents a precise definition of what they’re building instead of leaving them to guess.</p>



<p class="wp-block-paragraph">The ontology doesn’t have to be a university project. Start with twenty terms that AI can help you generate from your specs. Every noun in a spec maps to a type. Every type points back to the spec that justifies it. If either side is orphaned, a check fails. Every record says who may see it from day one, and when it doesn’t, the spec defaults to the most restrictive audience.</p>



<p class="wp-block-paragraph">Use standard vocabularies like the ones at <a href="https://schema.org/" target="_blank" rel="noopener">schema.org</a> before you invent your own. A frontier AI model has read far more about schema.org than about your private names, so a standard term brings its meaning along. We expect that to make the schema more useful, though we haven’t tested that hypothesis. Your name for an invoice isn’t a feature, so you don’t have to be creative about it.</p>



<p class="wp-block-paragraph">As customer requirements change, the developers must change the ontology to keep up.</p>



<p class="wp-block-paragraph">So the process, in a nutshell, is to develop the spec by using AI on customer-stated requirements. Then a human develops the ontology (often in collaboration with AI too), which is used to create a consistent schema. Finally, create and test the code. Teams that go the other way end up with an agent writing against a poorly defined vocabulary, setting up the reviewer to spend an afternoon or even a week figuring out what it meant.</p>



<p class="wp-block-paragraph">Ontologies might help avoid a reciprocal problem: different parts of an organization assigning different meanings to the same term.</p>



<p class="wp-block-paragraph">We’ll admit we can’t prove you need all this. <a href="https://www.anthropic.com/engineering/building-c-compiler" target="_blank" rel="noopener">Anthropic had sixteen agents write a C compiler</a>, and the researcher said the verifier had to be nearly perfect. He used GCC as a known-good reference. A compiler has that reference. Most companies don’t. Nobody has a GCC for “may this customer see this document” or “can this contract move to the next stage.” Our bet is that the spec and the schema are your test suite for that kind of question.</p>



<p class="wp-block-paragraph">Now on to bug fixing.</p>



<h2 class="wp-block-heading">Don’t clean up after the AI. Build the tool that does.</h2>



<p class="wp-block-paragraph">An agent can hand you a 3,000-line pull request before lunch. With AI coding, reviewing has suddenly become much more expensive than writing. When a person is the first line of defense, that person turns into the bottleneck and then starts skimming, which is almost as bad as no review at all.</p>



<p class="wp-block-paragraph">The usual advice is to review harder. We’d rather that a person reads very little, and that what they do read is worth the time. For example, reading the specs, reading test harnesses to validate the spec and refining the ontology is worth it, but reviewing generated code which passes the tests is probably not. Developers promise they’ll tidy up the AI’s mess as it comes. Don’t.</p>



<p class="wp-block-paragraph"><a href="https://openai.com/index/harness-engineering/" target="_blank" rel="noopener">The OpenAI team that built a product with no handwritten code</a> tried it. They spent every Friday, a fifth of the week, cleaning up what they called “AI slop.” It didn’t last. So they wrote their guardrails down as “golden principles,” set up an agent that scans for stale documentation and opens fix-up pull requests, and wrote their lint messages “to inject remediation instructions into agent context.” A living check doesn’t just fail on buggy code. It tells the agent how to fix the problem.</p>



<p class="wp-block-paragraph">The harness we’d build first consists of:</p>



<ul class="wp-block-list">
<li>A command line that’s built deterministically, without calling upon AI, to make sure that the highest level of the process can’t take off in an unanticipated direction.</li>



<li>A gate before code. If a change touches code and nothing in the specs or the schema changed, stop and ask why.</li>



<li>A second pass. Here, anything generated gets rebuilt from scratch and compared to what’s committed. If Git identifies differences that should not be there, that can be an important review finding and it deserves changing a guardrail, not handwriting new code.</li>



<li>A weekly drift audit. It lists specs that cite requirements that don’t exist, requirements with no proof, schema terms used nowhere, and tests that name no requirement. It flags what’s suspicious and offers solutions. It’s read-only; a person makes the choices and resolves what to do.</li>



<li>A route from every catch to a permanent check. Sierra, which builds customer service agents, has experts annotate sampled conversations, and those “<a href="https://sierra.ai/blog/agent-development-life-cycle" target="_blank" rel="noopener">become the basis for the agent’s regression tests</a>.” A harness should upgrade its coding so that each bug happens only once, and can be avoided by the harness in the future.</li>



<li>Limits on what agents may do. Some actions they take themselves, whereas others they only propose. Some belong to a named person who signs with a reason. An agent that wants a decision writes a proposal.</li>
</ul>



<p class="wp-block-paragraph">StrongDM found out why the last two bullet items matter. Their agents learned that <em>return true</em> is “<a href="https://factory.strongdm.ai/" target="_blank" rel="noopener">a great way to pass narrowly written tests</a>,” so they use end-to-end “scenarios,” often stored outside the codebase, as holdout tests that the coding agent cannot see. The principle is the same as investigating ethics violations at an agency: We’d want the investigation conducted by someone outside the agency.</p>



<p class="wp-block-paragraph">We should say where Shahid’s present harness falls short. As far as we know, nothing yet checks whether names in the requirement exist also in the spec. Furthermore, the project runs no continuous integration service and no automatic hooks, so running the checks is a habit instead of a rule. Those are the next things they will build, and building them is humans’ work.</p>



<p class="wp-block-paragraph">With the technique described here, the review burden that used to be spiraling out of control actually shrinks. A person reads a short spec with a proof on every line, and the machine checks the rest.</p>



<h2 class="wp-block-heading">Put three new lines in developers’ job descriptions</h2>



<p class="wp-block-paragraph">We suggest that next Monday you try changing job descriptions to reflect the tasks stated at the beginning of this article. Tell each developer:</p>



<ol class="wp-block-list">
<li>You hold the conversation with the PM and the customer, and the harness drafts the spec and the guardrails from it. A spec isn’t done until every requirement has a proof.</li>



<li>You own the ontology. A word in a spec that isn’t in the final schema reveals a bug.</li>



<li>You build the harness that audits and repairs the repository. If you’re cleaning the code by hand, the harness is missing something.</li>
</ol>



<p class="wp-block-paragraph">Try the process on one feature. Ask permission before recording a conversation. <a href="https://www.plaud.ai/products/plaud-note-ai-voice-recorder" target="_blank" rel="noopener">Plaud’s own guidance</a> is to give people a quick heads-up while noting that local regulations differ. <a href="https://www.granola.ai/blog/why-granola-doesnt-use-a-bot" target="_blank" rel="noopener">Granola has written about how to tell the people it’s transcribing</a>. Customers who know you’re recording them in order to get their requirements right usually don’t mind.</p>



<p class="wp-block-paragraph">Let the harness draft the spec. Read its list of gaps. Check every noun against the ontology. Add a check that would have caught the most recent bug you fixed by hand. If your reviews get shorter and your bug count stays flat, keep going.</p>



<h2 class="wp-block-heading">What this process is and is not</h2>



<p class="wp-block-paragraph">This article grows from Shahid’s experience and represents our opinions, and we can’t point to a controlled study. But controlled studies are similarly lacking for nearly all the advice programmers and software architects have gotten over the decades. (Check out the O’Reilly book <em><a href="https://www.oreilly.com/library/view/making-software/9780596808310/" target="_blank" rel="noopener">Making Software</a></em> for a discussion of rigorous research in software.) We can cite companies that believe they’re more productive and more responsive to their customers by following roughly the process outlined in this article.</p>



<p class="wp-block-paragraph">Some people who look at the three job descriptions we’ve listed think we’re turning developers into paper pushers. We think the reverse. The programmers are responsible for high-level tasks that do not fall into the old categories of architecture, coding, and testing. Our bet is that you and every developer you know will be doing something like this within a couple years.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/when-ai-can-code-what-is-left-for-programmers-to-do/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>This Week in AI: Who Gets to Benefit from AI?</title>
		<link>https://www.oreilly.com/radar/this-week-in-ai-who-gets-to-benefit-from-ai/</link>
				<comments>https://www.oreilly.com/radar/this-week-in-ai-who-gets-to-benefit-from-ai/#respond</comments>
				<pubDate>Fri, 13 Nov 2026 16:54:46 +0000</pubDate>
					<dc:creator><![CDATA[Christina Stathopoulos and Michelle Smith]]></dc:creator>
						<category><![CDATA[This Week in AI]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20226</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/08/USE_THIS_thisweek-ai-radar-1600x840-1.png" 
				medium="image" 
				type="image/png" 
				width="1600" 
				height="840" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/08/USE_THIS_thisweek-ai-radar-1600x840-1-160x160.png" 
				width="160" 
				height="160" 
			/>
		
				<custom:subtitle><![CDATA[Frontier models, regional AI, and responsible development]]></custom:subtitle>
		
				<description><![CDATA[AI is becoming more capable at handling everyday tasks, but benefiting from that progress depends increasingly on where people live, what language they speak, and how much control they have over the technology. On the latest episode of This Week in AI, we examined developments from OpenAI and Anthropic, the growing push for regional AI [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">AI is becoming more capable at handling everyday tasks, but benefiting from that progress depends increasingly on where people live, what language they speak, and how much control they have over the technology. On the latest episode of <em>This Week in AI</em>, we examined developments from OpenAI and Anthropic, the growing push for regional AI models, and the challenges of making AI more accessible and responsible.</p>



<iframe width="800" height="450" src="https://www.youtube.com/embed/2LHBgNfxFVs?si=JsSHP2AAbb8cLC9m" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



<h2 class="wp-block-heading"><strong>Frontier AI is moving into everyday services</strong></h2>



<p class="wp-block-paragraph">OpenAI introduced <a href="https://openai.com/index/lifeval/" target="_blank" rel="noopener">LifeVal</a>, a benchmark that measures how well AI handles everyday tasks such as finding childcare, managing household budgets, and accessing government services. Scores rose from 49% for GPT-5.2 to roughly 80% for GPT-5.6 and GPT-6 Astra. This could help researchers assess whether improvements in model capabilities translate into useful assistance with daily responsibilities.</p>



<p class="wp-block-paragraph">OpenAI reported that ChatGPT now handles <a href="https://www.theverge.com/ai-artificial-intelligence/1025482/openai-says-chatgpt-is-catching-up-to-google-in-search" target="_blank" rel="noopener">roughly 20% of Google’s search volume</a> in search-like queries, although the company hasn’t explained its methodology. Its new <a href="https://openai.com/index/introducing-gpt-6-cyber/" target="_blank" rel="noopener">GPT-6 Cyber model</a>, available to approved security teams, aims to help identify and fix software vulnerabilities. Meanwhile, Anthropic began testing <a href="https://claude.com/resources/articles/introducing-personal-finance-in-claude" target="_blank" rel="noopener">read-only banking connections in Claude</a>, allowing some US subscribers to analyze spending and identify unusual charges. As AI assistants take on tasks traditionally handled by banks, search engines, and specialized software, those industries will have to reconsider how they serve customers and protect their data.</p>



<p class="wp-block-paragraph">Anthropic also published <a href="https://www.anthropic.com/research/provable-inference" target="_blank" rel="noopener">FlashProve</a>, a research prototype that uses cryptographic proofs to verify which model generated a response. Separately, nonprofit research organization <a href="https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/" target="_blank" rel="noopener">METR reported</a> that AI assistance may have roughly doubled the pace of progress on certain model development tasks at Anthropic over the past year. The findings don’t establish recursive self-improvement, but they suggest that AI assistance is already speeding up parts of model development. That makes it increasingly important to understand how AI is shaping that progress and to check the reliability of the outputs these systems produce.</p>



<h2 class="wp-block-heading"><strong>Regional AI addresses gaps in access</strong></h2>



<p class="wp-block-paragraph">We’ve previously discussed the “tokenizer tax,” which makes some languages more expensive to process because they require more tokens than English. New <a href="https://news.umich.edu/generative-ai-uses-more-energy-on-non-english-text/" target="_blank" rel="noopener">research from the University of Michigan</a> found another disadvantage. In one test, answering questions in Pashto consumed up to 179 times more energy than answering equivalent questions in English, while accuracy also suffered. The findings show how language can affect both the cost and quality of AI services.</p>



<p class="wp-block-paragraph">Countries are responding by investing in models designed around their own languages and infrastructure. Ibero-American countries agreed to form a <a href="https://www.lamoncloa.gob.es/lang/en/gobierno/news/paginas/2026/20261103-ibero-american-meeting-on-ai.aspx" target="_blank" rel="noopener">regional AI alliance</a> to coordinate model development, data sharing, and computing resources. India <a href="https://newsonair.gov.in/govt-launches-3-new-indigenous-ai-foundation-models-under-indiaai-mission-to-strengthen-homegrown-ai-capabilities/" target="_blank" rel="noopener">introduced three domestic models</a>, including EKA-1, which supports 22 Indian languages and English, and two Dhīr models designed to work offline with documents. Offline capabilities could make these systems especially useful in rural areas with unreliable internet access.</p>



<p class="wp-block-paragraph">Indonesia’s <a href="https://www.businesstimes.com.sg/international/asean/goto-makes-its-indonesian-language-ai-available-public" target="_blank" rel="noopener">GoTo</a>, the UAE’s <a href="https://we0.ai/articles/jais-3-arabic-ai-models-speech-recognitio-0ea1f8fa" target="_blank" rel="noopener">Jais 3</a>, and Uganda’s <a href="https://sunbird.ai/introducing-sunflower-v2-an-open-source-ai-suite-for-african-languages/" target="_blank" rel="noopener">Sunflower V2</a> are further examples of projects developing AI for local languages. These efforts could give developers more control over AI for local languages and reduce reliance on proprietary AI services, even when some build on open foundation models from larger labs. They could also make AI more useful in languages that existing models handle poorly. For organizations deploying AI across multiple countries, testing performance and cost in each language may reveal problems that English-language benchmarks miss.</p>



<h2 class="wp-block-heading"><strong>Responsible AI depends on how we deploy it</strong></h2>



<p class="wp-block-paragraph">Google <a href="https://www.nytimes.com/2026/11/03/us/google-search-schools-restrict-ai.html" target="_blank" rel="noopener">faced criticism</a> over changes to AI controls in school accounts. Under a new arrangement, schools that disable AI also lose access to Google Search. That leaves educators with limited control over which tools students can use. And in Europe, an <a href="https://www.wsj.com/business/media/eu-court-adviser-warns-ai-chatbots-could-risk-infringing-copyright-law-41793aa3" target="_blank" rel="noopener">advisor to the EU’s top court</a> suggested that chatbots could infringe publishers’ rights when reproducing news content. The opinion emerged from a dispute involving Google’s Gemini and a Hungarian publisher. A final ruling is still pending, but the case could influence how AI companies access and reuse copyrighted material. This is part of a broader wave of publisher lawsuits, including <em>The New York Times’</em> US case against OpenAI and Microsoft over the alleged use of its articles to train models and reproduce content in ChatGPT outputs.</p>



<p class="wp-block-paragraph">AI’s environmental costs continue to come into focus. Some data center operators are <a href="https://www.nytimes.com/2026/11/03/climate/jet-engines-ai-data-center-generators-boneyard.html" target="_blank" rel="noopener">repurposing retired jet engines</a> as natural gas generators to obtain power while waiting for grid connections. Although this can accelerate deployment, the converted engines are less efficient and can produce more pollution than conventional turbines, meaning faster access to electricity comes at the expense of higher emissions and worse local air quality. Public opposition is growing too. During the November 3 US general elections, voters <a href="https://news.ballotpedia.org/2026/11/06/voters-favored-limiting-data-center-development-in-40-of-41-local-ballot-measures-on-nov-3/" target="_blank" rel="noopener">favored restrictions on data center development</a> in 40 of 41 local ballot measures across 10 states.</p>



<p class="wp-block-paragraph">Meanwhile, <a href="https://ai.planet.com/signin" target="_blank" rel="noopener">Planet introduced a natural-language tool</a> for searching and analyzing more than a decade of satellite imagery. Users can explore geographic changes without specialized GIS expertise, making satellite data easier for people without that background to investigate. The application shows how AI can help more people work with complex data that previously required specialized tools and training.</p>



<h2 class="wp-block-heading"><strong>What’s next</strong></h2>



<p class="wp-block-paragraph">As AI expands into personal services and countries invest in their own models, developers and organizations face increasingly different requirements for security, performance, and accessibility. The most useful systems will be those that work reliably for the people and circumstances they’re designed to serve.</p>



<p class="wp-block-paragraph">We’ll be back on Monday, November 16, for another episode covering the latest developments in AI research and applications. You can catch previous episodes on <a href="https://www.youtube.com/watch?v=g4cfjz5AKxY&amp;list=PL055Epbe6d5bJEhT7_ZzOeJZ6gPyUzYpS" target="_blank" rel="noopener">YouTube</a>, <a href="https://open.spotify.com/show/033kJS2BG1teGunxmtsU1r" target="_blank" rel="noopener">Spotify</a>, <a href="https://podcasts.apple.com/us/podcast/this-week-in-ai/id1896798047" target="_blank" rel="noopener">Apple</a>, or wherever you get your podcasts.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/this-week-in-ai-who-gets-to-benefit-from-ai/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>What Makes an Agent Self-Evolving?</title>
		<link>https://www.oreilly.com/radar/what-makes-an-agent-self-evolving/</link>
				<comments>https://www.oreilly.com/radar/what-makes-an-agent-self-evolving/#respond</comments>
				<pubDate>Fri, 13 Nov 2026 11:54:22 +0000</pubDate>
					<dc:creator><![CDATA[Annie Wang]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20219</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/What-makes-an-agent-self-evolving-Image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/What-makes-an-agent-self-evolving-Image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[An agent can get better at its job without a new model. It can learn to find useful information, choose the right tool, or check that a task is finished before reporting success. For that experience to help with the next task, something has to carry over: an updated instruction, a lesson saved in memory, [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">An agent can get better at its job without a new model. It can learn to find useful information, choose the right tool, or check that a task is finished before reporting success. For that experience to help with the next task, something has to carry over: an updated instruction, a lesson saved in memory, or a change to the software that runs the agent.</p>



<p class="wp-block-paragraph">That is what interests me about a self-evolving agent: It can use experience to change how it handles future tasks. Much of that change happens in the <em>harness</em>, the instructions, context, tools, and software around the model. Research such as <a href="https://arxiv.org/abs/2610.18813" target="_blank" rel="noopener">Harness-RSI</a> explores how agents can change that harness while continuing to use the same model.</p>



<p class="wp-block-paragraph">Of course, a change can make things worse. A saved memory might be wrong, or a new instruction might fix one problem and create another. I think about self-evolving agents in three parts: evaluation to tell whether a change helped, an improvement process that can do more of the work on its own, and the parts of the harness it is allowed to change.</p>



<h2 class="wp-block-heading">Evaluation: What counts as improvement?</h2>



<p class="wp-block-paragraph">Before I ask an agent to improve itself, I want to know what doing a better job would look like. A model’s score on a benchmark tells us how it performed on that test. To evaluate an agent built with that model, we also need to ask whether it used the right information, took the right actions, and met the user’s requirements.</p>



<p class="wp-block-paragraph">In a <a href="https://github.com/cuppibla/loop-lab-table" target="_blank" rel="noopener">restaurant-planning exercise</a> I developed for teaching evaluation, an agent is given a group’s needs and information about restaurants. Its job is to suggest a place and a time for dinner. The people and restaurants are fictional, but success is easy to understand: Can everyone eat there, afford it, and make it on time?</p>



<p class="wp-block-paragraph">In one test, the agent recommended a restaurant that met everyone’s needs. My evaluation still marked the answer as wrong because it compared the wording with a sample answer I had written. But my answer was only one possible solution. A different recommendation could work just as well. That feedback could lead a self-improving agent to “fix” a decision that was already good.</p>



<p class="wp-block-paragraph">A more useful evaluation would check whether the recommendation met the group’s requirements. If someone’s budget is $25 and the meal costs $28, the plan fails that check. We can build a custom evaluation around those requirements, checking the price, dietary needs, and timing. Other questions require judgment, such as whether the agent clearly explained a trade-off between price and convenience. A person or another model can help assess those.</p>



<p class="wp-block-paragraph">If we asked the agent to make a booking as well, we would need to check that the reservation actually existed. A message saying, “Your table is booked” would not be enough. Evaluating the system means checking what it did, as well as what it said.</p>



<p class="wp-block-paragraph">Once the agent proposes a change, we can test the updated version against the current one. It should still handle tasks it already did well. We should also try cases that were not used to develop or choose the change. Getting one dinner plan right tells us less than handling new groups with different budgets, dietary needs, and schedules.</p>



<p class="wp-block-paragraph">The evaluation should also explain why an answer failed. “The meal costs more than the guest’s budget” gives the agent something specific to investigate. It still needs to find the cause, but it knows where to start.</p>



<h2 class="wp-block-heading">Autonomous improvement: Who carries the work forward?</h2>



<p class="wp-block-paragraph">If an engineer has to spot every failure, suggest every change, and start every test, improvement still depends on that person. Autonomous improvement means giving more of this work to the system. It can review past tasks, look for repeated mistakes, propose a change, and test whether the change helps.</p>



<p class="wp-block-paragraph">For example, a more autonomous dinner planner might notice that it repeatedly recommended restaurants people could not afford, even though their budgets were available. It could try a new instruction: remove restaurants that exceed anyone’s budget before comparing the remaining options. It would then test that instruction on other groups. If the results improve and the agent still handles tasks it previously got right, it can keep the change. Otherwise, it keeps the current version.</p>



<p class="wp-block-paragraph">There are two kinds of work here. While planning a dinner, the agent uses tools, reads the results, and decides what to do next. That is its task loop. The improvement process looks back across those attempts and changes how it will approach future tasks. An agent can plan dinner on its own and still depend on an engineer to teach it a better way to plan.</p>



<p class="wp-block-paragraph">This review does not have to happen after every request. The system might start it after several similar failures, or review a day’s work overnight. Looking at several tasks together helps it see which problems keep coming back.</p>



<p class="wp-block-paragraph">We can give the system more responsibility gradually. At first, it might investigate problems and test changes, then ask an engineer to approve them. Later, it might also keep certain kinds of changes on its own. The team still decides what it may change, how much time or money it may spend experimenting, and when a person needs to step in.</p>



<h2 class="wp-block-heading">Harness modules: What can evolve?</h2>



<p class="wp-block-paragraph">When an agent makes a poor decision, I want to find the cause before rewriting its instructions. If a guest’s budget never reached the model, telling it to “pay attention to the budget” will not help. We need to fix how that information reaches the model.</p>



<p class="wp-block-paragraph"><a href="https://arxiv.org/abs/2609.14857" target="_blank" rel="noopener">ModularRSI</a> divides the harness into five parts: context management, the agent loop, tool use, observation management, and task completion detection. It studies changes to these parts separately before putting them together. This gives us a useful way to ask what exactly an agent could improve.</p>



<p class="wp-block-paragraph"><strong>Context management</strong> comes first for me. It determines what information the model has when it makes a decision, including which memories are brought into the current task. Reflective memory means looking back at what happened and saving a lesson that could help next time. For the dinner planner, that lesson might be to check everyone’s budget and dietary needs before comparing restaurants.</p>



<p class="wp-block-paragraph">The memory needs to distinguish a general lesson from a fact about one person. “Check the budget before recommending a restaurant” could help with any group. “This guest has a $25 budget” applies to a particular person and occasion. Without that context, the agent could use an old fact in the wrong situation.</p>



<p class="wp-block-paragraph"><em>Dreaming</em> is a way to review and organize memories between sessions. The system goes back over past interactions and what it has already stored, combines related observations, and saves useful lessons. When memories disagree, it needs to check the original evidence before deciding what to keep. Anthropic’s <a href="https://platform.claude.com/docs/en/managed-agents/dreams" target="_blank" rel="noopener">Dreams</a>, for example, produces a reorganized set of memories that can be reviewed before later sessions use them.</p>



<p class="wp-block-paragraph">What I want from dreaming is a better starting point for the next task. The agent should be able to find a useful lesson without reading every previous conversation again. We can test that by comparing how it handles new tasks with the old and revised memories. Organizing memories overnight only counts as an improvement if the results support that claim.</p>



<p class="wp-block-paragraph">The agent could also improve how it finds and organizes memories. Saving one lesson may help with a future task. Changing the way it selects lessons could help it make better use of many future experiences.</p>



<p class="wp-block-paragraph">The other four modules affect how the agent does its work:</p>



<ul class="wp-block-list">
<li><strong>Agent loop:</strong> The agent plans, acts, and decides what to do next. It could learn to notice when it is repeating an unsuccessful approach and try a different one.</li>



<li><strong>Tool use:</strong> Tools the agent has and how it uses them. It could add a tool it needs or change a tool’s interface to make it easier to use correctly.</li>



<li><strong>Observation management:</strong> The agent handles the information returned by its tools. It could change how it summarizes a result so that an important fact is not lost.</li>



<li><strong>Task completion detection:</strong> The agent decides that the job is done. It could check for confirmation that an action succeeded before telling the user it is finished.</li>
</ul>



<p class="wp-block-paragraph">These parts work together. Changing a tool can change the information the agent receives, which can change its next decision. Testing one module helps us understand a change. Testing the whole agent tells us whether the parts still work well together.</p>



<p class="wp-block-paragraph">Improvement becomes <em>recursive</em> when the agent uses its updated harness to carry out the next round of improvement. Suppose it changes how it searches past runs for examples of a failure. The next time it tries to improve itself, it uses that new search method to decide what to investigate and test. A change made in one round now shapes how it works on the next one.</p>



<p class="wp-block-paragraph">This gives the agent a chance to get better at finding and testing improvements, as well as doing its original job. Each round builds on changes that were kept from earlier rounds. To find out whether that helps, we need to check whether later rounds produce changes that work better on new tasks.</p>



<p class="wp-block-paragraph">I want an agent that needs less repeated correction because it has learned a better way to do the job. We should be able to point to what it learned, what changed, and how we checked that the change helped. Then we can see whether that lesson holds up when the agent has a new task to do.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/what-makes-an-agent-self-evolving/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Generative AI in the Real World: Memory for Personal AI with Duncan Davidson</title>
		<link>https://www.oreilly.com/radar/podcast/generative-ai-in-the-real-world-memory-for-personal-ai-with-duncan-davidson/</link>
				<comments>https://www.oreilly.com/radar/podcast/generative-ai-in-the-real-world-memory-for-personal-ai-with-duncan-davidson/#respond</comments>
				<pubDate>Thu, 12 Nov 2026 16:55:03 +0000</pubDate>
					<dc:creator><![CDATA[Ben Lorica and Duncan Davidson]]></dc:creator>
						<category><![CDATA[Generative AI in the Real World]]></category>
		<category><![CDATA[Podcast]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?post_type=podcast&#038;p=20214</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Podcast_Cover_GenAI_in_the_Real_World-scaled.png" 
				medium="image" 
				type="image/png" 
				width="2560" 
				height="2560" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2024/01/Podcast_Cover_GenAI_in_the_Real_World-160x160.png" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[This post was created with the assistance of AI, based on a transcript of the conversation. AI tools have started to remember their users, but each tool remembers a different version of the same person. Duncan Davidson, an independent consultant and longtime software engineer, has been trying to connect those fragments. He joined Ben Lorica [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph"><em>This post was created with the assistance of AI, based on a transcript of the conversation.</em></p>



<p class="wp-block-paragraph">AI tools have started to remember their users, but each tool remembers a different version of the same person. Duncan Davidson, an independent consultant and longtime software engineer, has been trying to connect those fragments. He joined Ben Lorica to discuss what personal AI could do if people could carry their context between services and decide what each service gets to see.</p>



<p class="wp-block-paragraph">They also talked about how AI coding changed web development during Duncan’s time at Webflow, what new developers may lose as entry-level work changes, and why open source projects need new ways to handle AI-assisted contributions.</p>



<p class="wp-block-paragraph">About the <em>Generative AI in the Real World</em> podcast: In 2023, ChatGPT put AI on everyone’s agenda. In 2026, the challenge will be turning those agendas into reality. In <em>Generative AI in the Real World</em>, Ben Lorica interviews leaders who are building with AI. Learn from their experience to help put AI to work in your enterprise.</p>



<p class="wp-block-paragraph">Check out other episodes of this podcast on the <a href="https://learning.oreilly.com/playlists/42123a72-1108-40f1-91c0-adbfb9f4983b/" target="_blank" rel="noopener">O’Reilly learning platform</a> or follow us on <a href="https://www.youtube.com/playlist?list=PL055Epbe6d5YcJUhZbsVW9dlMueIuOxK_" target="_blank" rel="noopener">YouTube</a>, <a href="https://open.spotify.com/show/5C9oof8TFkP65lDUcEy5jT" target="_blank" rel="noopener">Spotify</a>, <a href="https://podcasts.apple.com/us/podcast/generative-ai-in-the-real-world/id1835476293" target="_blank" rel="noopener">Apple</a>, or wherever you get your podcasts.</p>



<h2 class="wp-block-heading"><strong>Takeaways</strong></h2>



<p class="wp-block-paragraph"><strong>06.31 The developer’s job is moving toward outcomes.</strong> Duncan compared AI coding to desktop publishing, which removed the physical work of assembling pages but left designers responsible for what the page should look like. At Webflow, he saw people use AI tools to make websites faster than they could with earlier tools. For developers, he said, the question has shifted from how quickly they can type code to “What outcome do you want? What are you trying to do with this code, and how can you make it better?” People still need to judge whether an implementation fits the rest of the system.</p>



<p class="wp-block-paragraph"><strong>10.41 New developers still need a way to become experts.</strong> Computer science fundamentals still matter when an agent offers several plausible fixes and a person must choose the one that fits a larger system. Ben asked where future engineers will gain that judgment if AI absorbs entry-level tasks. Duncan compared early career work to an apprenticeship in the arts, where even menial assignments expose newcomers to how the whole operation works. Companies that measure only throughput, he warned, may leave too little room for people to learn.</p>



<p class="wp-block-paragraph"><strong>23.07 AI-assisted changes are straining open source review.</strong> Duncan described a library he uses in one of his apps. His fixes, along with performance problems Claude noticed, left his version roughly 25 pull requests ahead of the project’s main branch. The maintainers were using AI too, he said. The difficulty was getting useful changes back to the project without overwhelming its review process. His conclusion was that older open source collaboration practices need a “serious retrofit.”</p>



<p class="wp-block-paragraph"><strong>28.27 Personal memory is fragmented across tools.</strong> Duncan first had his coding agents save useful information into a collection of Markdown files. Once commercial agents added memory, he found that “Claude knew me one way and ChatGPT knows me another way.” He could push context back to each tool with instructions and occasional reminders, but that solution demands more tinkering than most people want to do. What he wants is a small set of shared mechanisms that let a person carry context across services.</p>



<p class="wp-block-paragraph"><strong>34.01 Portable memory needs boundaries.</strong> A prompt’s context, an agent’s memory of recent interactions, and a person’s longer-term knowledge should be able to travel, but Duncan also wants people to control access by scope. He keeps general notes alongside information about his family and finances. His test for a useful system is whether he can say, “Give Claude access to these scopes of my knowledge, but not everything.” That becomes harder when a personal agent encounters knowledge gained at work. Duncan left open where an employee’s own expertise ends and an employer’s information begins.</p>



<p class="wp-block-paragraph"><strong>36.44 Trust and security shape what people will share.</strong> Ben’s not yet comfortable letting a consumer agent reach into his finances. Duncan noted that trust would have to grow through experience, as it would with a human agent. We’re already seeing how AI agents can find vulnerabilities faster than established disclosure practices can handle. Duncan expects software maintainers to face more pressure as agents uncover latent bugs, while Ben questioned how quickly organizations that rely on vulnerable code would learn about the findings. People need control over what an agent can access and confidence in the systems around it.</p>



<p class="wp-block-paragraph"><strong>49.52 A personal memory could answer requests with permission.</strong> Duncan imagined keeping his memory on his phone and giving a new service a way to request specific information. A car dealership’s agent, for example, could ask for details, but the phone would seek his approval before sharing them. He put it this way:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph">I could provide it a handle to my memory. And whatever agents they’re working with can come in and ping me and say, “Hey, we’d like to know these things about you.” And it wouldn’t necessarily do it automatically.</p>
</blockquote>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/podcast/generative-ai-in-the-real-world-memory-for-personal-ai-with-duncan-davidson/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Why Git Needs Help: Moving from Files to Systems for Source Control</title>
		<link>https://www.oreilly.com/radar/why-git-needs-help-moving-from-files-to-systems-for-source-control/</link>
				<comments>https://www.oreilly.com/radar/why-git-needs-help-moving-from-files-to-systems-for-source-control/#respond</comments>
				<pubDate>Thu, 12 Nov 2026 11:54:27 +0000</pubDate>
					<dc:creator><![CDATA[Jason Davenport]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Data]]></category>
		<category><![CDATA[Software Architecture]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20215</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Why-Git-needs-help-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Why-Git-needs-help-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[The shift to agentic reality When Git was designed in 2005, Linus Torvalds solved a brilliant problem for humans collaborating over terminal windows by trading line-by-line diffs. Fast-forward to 2026: Autonomous coding agents don’t think in flat text files or line breaks. They reason across syntax trees, execution traces, call graphs, and type constraints. In [&#8230;]]]></description>
								<content:encoded><![CDATA[
<h2 class="wp-block-heading">The shift to agentic reality</h2>



<p class="wp-block-paragraph">When Git was designed in 2005, Linus Torvalds solved a brilliant problem for humans collaborating over terminal windows by trading line-by-line diffs. Fast-forward to 2026: Autonomous coding agents don’t think in flat text files or line breaks. They reason across syntax trees, execution traces, call graphs, and type constraints. In my view, forcing an AI agent to engineer complex distributed systems through string matching is like handing a chief software architect a chisel and a stack of sticky notes.</p>



<h3 class="wp-block-heading">File-centric versus system-centric paradigm shift</h3>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th><strong>Dimension</strong></th><th><strong>Legacy file-centric (Git)</strong></th><th><strong>Agent-native system graph</strong></th></tr></thead><tbody><tr><td><strong>Representation</strong></td><td>Flat Unicode text streams</td><td>Tree-Sitter ASTs &amp; CPG nodes</td></tr><tr><td><strong>Mutation unit</strong></td><td>Line-by-line string diffs</td><td>AST node transformations</td></tr><tr><td><strong>Context overhead</strong></td><td>2,000+ line prompt context fills</td><td>Sub-50 token targeted node payloads</td></tr><tr><td><strong>State persistence</strong></td><td>Ephemeral model prompt windows</td><td>Execution ledgers &amp; reasoning chains</td></tr></tbody></table></figure>



<p class="wp-block-paragraph">Whenever I see engineers try to lock context-blind models into file-centric version control, I watch the consequences compound quickly. Context windows fill with irrelevant churn, API bills surge, and subtle, multifile structural breakages slip past basic syntax checks. Just as critically, standard Git discards the cognitive history of a change. The moment an agent session completes, its execution logs, compiler feedback, and discarded hypotheses evaporate into thin air.</p>



<h3 class="wp-block-heading">Tactical illustration: renaming a distributed method signature</h3>



<p class="wp-block-paragraph">Consider updating a core helper signature across 40 dependent services. A traditional string diff forces the agent to read and mutate thousands of lines of surrounding boilerplate text, burning thousands of context tokens per file:</p>



<pre class="wp-block-code"><code>--- a/services/auth.py
+++ b/services/auth.py
@@ -42,7 +42,7 @@ class AuthManager:
     def validate_session(self, token: str) -&gt; UserSession:
-        return fetch_user_by_token(token)
+        return fetch_user_by_token(token, include_metadata=True)
</code></pre>



<p class="wp-block-paragraph">In contrast, an AST mutation node isolates the target function symbol, executing the change precisely without passing unneeded ambient text through the prompt window:</p>



<pre class="wp-block-code"><code>{
  "action": "mutate_node",
  "target_ast_id": "ast_node_89f2a41b",
  "symbol": "fetch_user_by_token",
  "update": {
    "add_argument": { "name": "include_metadata", "value": true }
  }
}
</code></pre>



<p class="wp-block-paragraph">This exact bottleneck hit Anthropic in late 2026. As autonomous agent activity pushed monorepo merges beyond 30,000 per day, conventional Git infrastructure simply buckled under the load. To me, unlocking genuine agentic autonomy requires evolving source control along two complementary tracks: shifting to server-hosted, graph-aware architectures, and treating agent reasoning as a first-class, versioned artifact right alongside the code itself.</p>



<h2 class="wp-block-heading">Why text diffs and ephemeral sessions create silent debt</h2>



<p class="wp-block-paragraph">Standard diff tools operate purely on raw Unicode strings. When an agent reorders functions or cleans up formatting, string diffs pollute context windows with noise while remaining blind to structural reality. If you modify a single method signature, Git won’t warn you about the dozen callers broken across separate modules. From my experience, feeding 2,000-line source files into prompts just to update a 5-line helper method degrades model attention, burns compute, and inflates overhead at scale.</p>



<h3 class="wp-block-heading">Text diffs versus AST graph mutations</h3>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th><strong>Metric / behavior</strong></th><th><strong>Text diff (unified patch)</strong></th><th><strong>AST graph mutation</strong></th></tr></thead><tbody><tr><td><strong>Formatting resilience</strong></td><td>Breaks on whitespace or line-ending changes</td><td>Immune to syntax formatting shifts</td></tr><tr><td><strong>Impact analysis</strong></td><td>Post hoc CI test failures</td><td>Sub-second static call-graph resolution</td></tr><tr><td><strong>Token consumption</strong></td><td>O(file_size) per prompt iteration</td><td>O(node_delta) per targeted change</td></tr></tbody></table></figure>



<p class="wp-block-paragraph">Compounding this structural blind spot is what I consider our biggest waste: ephemeral session memory. Whenever an agent finishes a task or compacts its context window, its entire diagnostic history—failed attempts, shell outputs, and compiler warnings—disappears. The next agent assigned to the branch steps in completely blind, doomed to repeat the same diagnostic dead ends and burn unnecessary tokens.</p>



<h2 class="wp-block-heading">Agent-native architectures in practice</h2>



<p class="wp-block-paragraph">In my view, next-generation version control must replace raw file trees with Tree-sitter parsers and code property graphs (CPGs). Instead of editing lines on disk, agents mutate specific syntax nodes directly, executing sub-second edits and running immediate dependency impact checks across the codebase. Moving control back to the server eliminates the massive I/O bottlenecks caused by checking out heavy working trees onto virtual machines, allowing agents to query and update code over fast graph end points.</p>



<h3 class="wp-block-heading">Practical implementation: server-side microdiff stream</h3>



<p class="wp-block-paragraph">Below is an example payload representing an atomic microdiff layer executed directly on a server-hosted code property graph, bypassing local file checkouts entirely:</p>



<pre class="wp-block-code"><code>{
  "commit_id": "c_9011f4a2",
  "parent_graph_version": "v14.2.0",
  "operations": &#91;
    {
      "op": "replace_node_property",
      "node_uri": "cpg://auth_service/ast/Node_441",
      "property": "return_type",
      "value": "Promise&lt;UserSession&gt;"
    }
  ],
  "validation_hash": 
  "sha256_e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
</code></pre>



<p class="wp-block-paragraph">This structural shift pairs naturally with stacked microdiffs—atomic, dependent layers verified sequentially in isolated virtual workspaces to prevent merge races. I was particularly impressed seeing the real-world impact of this pattern with Anthropic’s Scion architecture. Faced with object-store locks and clone latencies as daily merges scaled 10x, Anthropic built a server-centric SCM that streams code graphs on demand, slashing infrastructure overhead by 200x while enabling near-instant workspace initialization.</p>



<h2 class="wp-block-heading">Versioning reasoning and system memory</h2>



<p class="wp-block-paragraph">I strongly believe code changes alone rarely capture full intent. Modern SCM must link model reasoning traces, prompt chains, and decision graphs directly to commit nodes, preserving architectural context for future debugging and audits. Persisting build outputs, compiler logs, and linter runs in execution ledgers attached to commit trees allows subsequent agents to inspect prior test results instantly rather than rerunning identical test suites.</p>



<h3 class="wp-block-heading">System memory schema: reasoning and execution ledger</h3>



<p class="wp-block-paragraph">To prevent model sessions from repeating diagnostic dead ends, execution ledgers append structured reasoning and compiler feedback directly to commit nodes:</p>



<pre class="wp-block-code"><code>{
  "agent_session_id": "sess_88190c2a",
  "target_commit": "c_9011f4a2",
  "hypothesis_tree": {
    "attempt_1": { "approach": "In-memory caching", "status": "REJECTED", 
    "reason": "Memory limit exceeded in integration test" },
    "attempt_2": { "approach": "Redis backing store", "status": 
    "VERIFIED", "execution_time_ms": 142 }
  },
  "compiler_ledger": {
    "linter_passed": true,
    "type_check_errors": 0
  }
}
</code></pre>



<p class="wp-block-paragraph">By explicitly recording abandoned branches and failed architectural hypotheses as structured metadata, we stop future agent sessions from reexploring invalid paths. I argue that capturing thought streams alongside code transforms source control from a passive text archive into an active memory system.</p>



<h2 class="wp-block-heading">The hard engineering trade-offs and counterarguments</h2>



<p class="wp-block-paragraph">I won’t pretend making this transition is easy; it comes with steep engineering trade-offs. Existing developer tooling, from LSPs to build pipelines and IDEs, expects standard local file trees. Replacing checkouts with server-hosted graphs requires building heavy compatibility layers or redesigning workflows. Furthermore, human reviewers still digest code line by line, creating a natural tension with pure graph mutations, while storing continuous thought streams drives up storage and compliance costs. During early refactoring, syntax is often broken and unparseable, forcing systems to temporarily fall back to raw text. Overcoming Git’s decades of deep ecosystem lock-in remains a major operational hurdle we cannot ignore.</p>



<h3 class="wp-block-heading">Trade-off matrix: architectural migration challenges</h3>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th><strong>Challenge</strong></th><th><strong>Systemic bottleneck</strong></th><th><strong>Mitigation strategy</strong></th></tr></thead><tbody><tr><td><strong>IDE &amp; tooling lock-in</strong></td><td>LSPs expect local file system layouts</td><td>Virtual FUSE layers streaming AST nodes on demand</td></tr><tr><td><strong>Storage overhead</strong></td><td>Persisting continuous reasoning logs increases storage 10x</td><td>Compacting execution ledgers into summary trees</td></tr><tr><td><strong>Broken syntax states</strong></td><td>AST parsers fail during active multiline refactoring</td><td>Fallback to temporary text buffer zones during edits</td></tr></tbody></table></figure>



<h2 class="wp-block-heading">Bottom line</h2>



<p class="wp-block-paragraph">Git was designed for humans reviewing text, but agentic scale demands server-centric, graph-aware source control. Code property graphs stop token bloat and prevent silent structural breakages. Versioning reasoning traces and execution histories alongside code keeps agents from repeating past mistakes. And stacked microdiffs offer a clean, isolated path for autonomous agents to ship code safely at scale.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/why-git-needs-help-moving-from-files-to-systems-for-source-control/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Where Are the “Killer Apps” of AI?</title>
		<link>https://www.oreilly.com/radar/where-are-the-killer-apps-of-ai/</link>
				<comments>https://www.oreilly.com/radar/where-are-the-killer-apps-of-ai/#respond</comments>
				<pubDate>Wed, 11 Nov 2026 15:43:41 +0000</pubDate>
					<dc:creator><![CDATA[Tim O’Reilly]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20197</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Where-are-the-killer-apps-of-AI-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Where-are-the-killer-apps-of-AI-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[Most of today’s AI entrepreneurs and social media enthusiasts are young. AI is the first technology revolution they’ve lived through. They don’t remember how unbelievably exciting it was in the late 1970s to get your own computer, instead of struggling to get time on one of the vast machines chugging away in what used to [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">Most of today’s AI entrepreneurs and social media enthusiasts are young. AI is the first technology revolution they’ve lived through. They don’t remember how unbelievably exciting it was in the late 1970s to get your own computer, instead of struggling to get time on one of the vast machines chugging away in what used to be called “the glass house,” an air-conditioned room where from outside you could watch the nine-track tapes being accessed and the refrigerator-sized disk drives spinning. Input and output was on high-speed teletype printers (no screens) and magnetic tape (and before that on punch cards.) They also mostly don’t remember, as I do, writing assembly-language programs in which you moved data from one computer register to another, specified the arithmetic or logical operations to be performed on it, and then specified where to save the result.</p>



<p class="wp-block-paragraph">They look at today’s AI and marvel at how virtually anyone can now generate code. They don’t realize that that’s also how it felt when you could use an interpreted programming language like BASIC to write your own applications instead of writing them in Assembler. They also don’t remember when Dan Bricklin used BASIC to write the first version of VisiCalc in 1979. He went into business with Bob Frankston, who rewrote it in 6502 Assembly Language for the Apple II. Dan was able to express the idea in Basic; Bob then wrote a tight, efficient version for widespread distribution. People rushed out to buy computers so they could use VisiCalc, and everyone predicted that the VisiCalc “spreadsheet” would put all the accountants out of work. It was the first program to be referred to as a “killer app.” It wasn’t the democratization of programming itself that was the killer app but what Dan and Bob did with it. In the entrepreneurial explosion that followed, databases, graphics, word processing, and other business applications came out of the glass house and, as Bill Gates put it, “onto every desk and into every home.”</p>



<figure class="wp-block-image size-large is-resized"><img loading="lazy" decoding="async" width="1600" height="1199" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7-1600x1199.png" alt="Dan Bricklin (right) with Bob Frankston, who rewrote VisiCalc in 6502 Assembly Language for the Apple II
Image from https://www.digitec.ch/en/page/visicalc-the-first-killer-app-in-computer-history-16230
" class="wp-image-20198" style="aspect-ratio:1.3341094295692666;width:1146px;height:auto" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7-1600x1199.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7-300x225.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7-767x575.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7-1536x1151.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-7.png 2048w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><figcaption class="wp-element-caption">Dan Bricklin (right) with Bob Frankston, who rewrote VisiCalc in 6502 Assembly Language for the Apple II<br>Image from <a href="https://www.digitec.ch/en/page/visicalc-the-first-killer-app-in-computer-history-16230">https://www.digitec.ch/en/page/visicalc-the-first-killer-app-in-computer-history-16230</a><br></figcaption></figure>



<p class="wp-block-paragraph">We saw the same leap from astonishment to innovation to adoption with the World Wide Web. When O’Reilly demonstrated <a href="https://en.wikipedia.org/wiki/Global_Network_Navigator" target="_blank" rel="noopener">GNN</a>, the first commercial website, to people at a trade show in 1993, attendees couldn’t believe that clicking on a link could pop up an image or a document from another computer thousands of miles away. They didn’t understand the difference between the software application (the web browser) from the network-connected documents with clickable hyperlinks that it acted as a frame for. They didn’t understand that <a href="https://dl.acm.org/doi/10.1145/253671.253687" target="_blank" rel="noopener">web pages were a new kind of software</a> written in a simple markup language anyone could understand. It didn’t make sense that the web page was a document that in many ways could look and act like a kind of program. And when people did understand, they thought that might be the end of programming.</p>



<p class="wp-block-paragraph">In 1993, everyone thought that the web browser itself was the killer app, and that Marc Andreessen, the young creator of Mosaic, the most popular web browser, which became Netscape, was going to dethrone Bill Gates as the new king of computing. They didn’t realize that the true killer apps of the web were not the web browser and the web server but applications “up the stack” built using the new programming paradigm the web enabled. Those applications weren’t software as we’d known software before. They were database- and application-backed websites delivering services that were previously impossible. I remember thinking of Amazon as the 2005 equivalent to VisiCalc when a friend told me she had just bought her first computer so she could shop on Amazon. Once again, it wasn’t the new capability itself that was the killer app but what developers did with it.</p>



<p class="wp-block-paragraph">What makes a killer app? I think that Tom Stoppard nailed it in his play <em>Rosencrantz and Guildenstern Are Dead</em>:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph">A man breaking his journey between one place and another at a third place of no name, character, population or significance, sees a unicorn cross his path and disappear. That in itself is startling, but there are precedents for mystical encounters of various kinds, or to be less extreme, a choice of persuasions to put it down to fancy; until—“My God,” says a second man, “I must be dreaming, I thought I saw a unicorn.” At which point, a dimension is added that makes the experience as alarming as it will ever be. A third witness, you understand, adds no further dimension but only spreads it thinner, and a fourth thinner still, and the more witnesses there are the thinner it gets and the more reasonable it becomes until it is as thin as reality, the name we give to the common experience.</p>
</blockquote>



<p class="wp-block-paragraph">That is, a killer app is at first a startling window onto an undreamed-of possibility, but one that so changes the paradigm that it rapidly gets taken for granted. In the meantime, everyone tries to make sense of the new world. Usually, their sensemaking is to try to reduce it to what they already know. The breakthrough entrepreneurs see more deeply and go farther.</p>



<p class="wp-block-paragraph">In 1995 and 1996, almost everyone saw the future as a showdown between Microsoft and Netscape over who would own the web server and the web browser, and whether the web would supplant the desktop as the new platform. Microsoft won the browser wars, but it turned out to matter far less than anyone expected. The value went to the companies building on top of the browser, the ones that figured out what the network made possible.</p>



<p class="wp-block-paragraph">The current narrative pitting AI model providers against each other sounds a lot like Microsoft versus Netscape to me. It is the platform war, the war for the layer that will become the common experience on which the revolutionary future will be built. ChatGPT, Claude, and Gemini are the browsers of this era. That is not a small thing. But the browser wasn’t actually the killer app of the web. Netscape made money for a little while. Amazon and Google and the explosion of other sites were what made the web into something people couldn’t live without.</p>



<p class="wp-block-paragraph">Claude Code, Codex, Cursor, and the other coding agents may be the BASIC of this era, or like the tools and frameworks that made database-backed web applications possible. They are plumbing. Important plumbing, and plumbing that has to be in place before the future can truly happen. But still just plumbing. Inventors tend to spend decades maturing it, and O’Reilly tends to spend decades teaching people to build it. I suspect that will be just as true this time around.</p>



<p class="wp-block-paragraph">In short, I’ve started asking myself whether we’ve actually seen the killer app for AI yet. The obvious assumption is that ChatGPT was. It certainly got a LOT of people excited and onboard very, very quickly. But maybe Claude Code was, enabling the breakthrough in agentic coding. Or maybe Hermes or Dots or Muse or some other personal agent will be. But we should take seriously the possibility that the killer app is not yet on the scene. These initial candidates for the killer app look too much like advances in a new enabling platform and not enough like a new generation of applications.</p>



<p class="wp-block-paragraph">Personal agents may turn out to be a breakthrough category, but right now they look to me like early ecommerce before Amazon got it right.<sup data-fn="abd7edcd-143d-429d-aca3-df39bc2ddf61" class="fn"><a href="#abd7edcd-143d-429d-aca3-df39bc2ddf61" id="abd7edcd-143d-429d-aca3-df39bc2ddf61-link">1</a></sup> For Amazon to work, the web had to become secure enough for people to use credit cards online. The apps were waiting on missing mechanisms. For Google’s economic model to kick into gear, pay-per-click advertising had to be invented. I suspect we are in a similar moment waiting for crucial pieces to be dumped on the table so developers can complete their part of the puzzle. For AI agents to act on our behalf in the world, they will need their own equivalents: ways to pay, to prove who they are acting for, to be held accountable, and for the counterparties to settle on the allocation of the value created by the transaction. Ilan Strauss and I have argued in “<a href="https://www.oreilly.com/radar/the-missing-mechanisms-of-the-agentic-economy/" target="_blank" rel="noopener">The Missing Mechanisms of the Agentic Economy</a>” that many of those mechanisms don’t exist yet. Maybe the killer apps are waiting on the equivalent of https and pay per click.</p>



<p class="wp-block-paragraph"><em>In short, when you listen to the AI conversation and mostly hear about people’s programming pipelines and how much more productive their programmers are, and very little about what they are using that productivity to build, you can be pretty confident that the killer apps haven’t arrived yet.</em></p>



<h2 class="wp-block-heading">Where are we in the cycle?</h2>



<p class="wp-block-paragraph">Let’s line up the calendars and see if that tells us something useful.</p>



<p class="wp-block-paragraph">CP/M (Control Program for Microcomputers), the first widespread personal computer operating system, was developed in 1974, and the Altair 8800, the first mass-market personal computer, appeared on the cover of <em>Popular Electronics</em> in January 1975. Microsoft released a BASIC interpreter for it in June of that same year. A host of competing personal computer hardware platforms running CP/M were introduced soon thereafter. The Apple II was released in 1977, VisiCalc in October 1979, and dBase II in 1980. The IBM PC came out in August 1981 with Microsoft’s MS-DOS as its operating system. Lotus 1-2-3 (which included spreadsheet, graphing, and simple database features) was released in January 1983 and turned the PC into a business necessity. </p>



<p class="wp-block-paragraph">In 1984, the revolutionary graphical user interface of the Macintosh was the next step function jump in the PC era. The GUI itself had been invented at Xerox PARC in the 1970s, but it wasn’t till Steve Jobs built his “insanely great” Mac that the world embraced it. Desktop publishing became the killer app of the GUI, with Aldus PageMaker and the Apple LaserWriter in 1985. The PC proper didn’t get the GUI till Windows 3.0 in 1990, but Microsoft’s embrace of it for applications such as Word (1983), Excel (1985), and PowerPoint (1987) as well as for the operating system and programming environment made it, for the time, master of the computing world.</p>



<p class="wp-block-paragraph">So you can see a rough timeline. Four or five years from the introduction of the first version of what was to become the new platform to the first killer app. (Or two, if you count from the introduction of the Apple II, the platform for which VisiCalc was first written.) Then 10 years of invention, competition, and refinement, followed by fairly rapid consolidation and accumulation of market power.</p>



<p class="wp-block-paragraph">Let’s do the same timeline for the web. Email and online forums might well have been the first internet killer apps, but they were really holdovers from previous generations of research networks. They were apps that had been evolving since the 1970s, though CompuServe and AOL had begun to bring them to the general public in the ’80s. The world really changed, though, when Tim Berners-Lee first wrote his proposal for what became the World Wide Web in March 1989. It was released to the world in August 1991. This was the new platform and the start of what would become the new programming paradigm.</p>



<p class="wp-block-paragraph">Not too much happened outside physics labs and a few universities until 1993. (There were only about 200 websites when we published <em><a href="https://en.wikipedia.org/wiki/Whole_Internet_User%27s_Guide_and_Catalog" target="_blank" rel="noopener">The Whole Internet User’s Guide and Catalog</a></em> in November 1992.) In 1993, NCSA, the National Center for Supercomputer Applications, released <a href="https://en.wikipedia.org/wiki/NCSA_Mosaic" target="_blank" rel="noopener">Mosaic</a> and we launched <a href="https://en.wikipedia.org/wiki/Global_Network_Navigator" target="_blank" rel="noopener">GNN</a>, the first commercial website. Netscape shipped its browser, a follow-on to Mosaic, at the end of 1994. Yahoo! was launched that year too, Amazon, eBay, and the Altavista search engine in 1995, MapQuest in 1996, Google in 1998, and Wikipedia in 2001.</p>



<p class="wp-block-paragraph">So, again about four years from the release of the platform to the appearance of the first killer app, and another 10 till the consolidation phase. (I see Gmail 2004, Google Maps 2005, Google Docs 2006, and Chrome 2008 as the equivalent of Microsoft’s release of Office on top of Windows as the beginning of the consolidation phase of the web.)</p>



<p class="wp-block-paragraph">The timeline for mobile is not dissimilar. The iPhone in 2007, Android in 2008. Killer social apps Facebook and Twitter had gotten their start on the web at about the same time, but really came into their own on mobile over the next few years. The “could only have happened on mobile” killer apps of Uber, Lyft, and the on-demand economy didn’t arrive till 2011/2012.</p>



<p class="wp-block-paragraph">A lot of people seem to think AI is moving way faster than previous technology revolutions. But if you line up the calendars, AI is moving at about the same pace as previous eras. Treat the transformer paper, “<a href="https://arxiv.org/abs/1706.03762" target="_blank" rel="noopener">Attention Is All You Need</a>,” published in June 2017, as the AI equivalent of Tim Berners-Lee’s 1989 proposal, and ChatGPT’s first release on November 30, 2022, as the equivalent of Mosaic. The gap from invention to breakout is about the same, five years for the web and five and a half for AI. That’s just getting the platform in place. The AI-native “killer apps” haven’t yet arrived but are due soon. (And please, let’s avoid the jokes that the “killer apps” of AI first showed up in the Russia-Ukraine drone war. That’s true, but not on the direct line of my argument here.)</p>



<p class="wp-block-paragraph">On what seems to be a fairly consistent timeline, where are we today, then? October 2026 corresponds to roughly the spring of 1997. Amazon was about to go public. Hotmail was a year old. Google was still a research project at Stanford called BackRub. SixDegrees had kicked off social networking, but Facebook was still seven years off. Napster was still two years away.</p>


<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e2b9b6&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e2b9b6" class="aligncenter size-large wp-lightbox-container"><img loading="lazy" decoding="async" width="1600" height="1316" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8-1600x1316.png" alt="Line up the calendars" class="wp-image-20199" style="aspect-ratio:1.21656050955414" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8-1600x1316.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8-300x247.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8-767x631.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8-1536x1263.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-8.png 2048w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button></figure>
</div>


<p class="wp-block-paragraph">Think of all those amazing application categories that had been pioneered, though. Search, ecommerce, email, and social networking were all in their infancy, but they were out there, delivering real value. I’m not sure that I see the equivalent time to value in AI. AI didn’t even get to Claude Code, its Microsoft Basic equivalent, till 2025! By some measures, AI is moving more slowly rather than more quickly. There’s more change to digest, and developers are still struggling with the tooling and developing the new knowledge that is needed.</p>



<h2 class="wp-block-heading"><strong>Why so slow, with so much money?</strong></h2>



<p class="wp-block-paragraph">The four largest hyperscalers are on track to spend <a href="https://om.co/2026/04/30/what-i-learned-about-hyperscalers-ai-spend/" target="_blank" rel="noopener">roughly $700 billion</a> in capital expenditure this year, nearly double what they spent in 2025. That is orders of magnitude more capital than went into the web in the 1990s. If capital and capability were the limiting factors, you’d think we should be well ahead of the earlier calendar, not behind.</p>



<p class="wp-block-paragraph">I suspect that at least part of the answer is that <a href="https://web.archive.org/web/20190205184624/https://qz.com/1540608/the-problem-with-silicon-valleys-obsession-with-blitzscaling-growth/" target="_blank" rel="noopener">Silicon Valley’s quest for premature monopoly</a> may be making it less efficient at finding breakthroughs rather than more. Starting with the era of Uber and Lyft, massive VCs have canonized the winners at the beginning of the race, crowding out alternate technologies and business models. That may be happening here. But it’s also just in the nature of bubbles, which depend on groupthink to flourish.</p>



<p class="wp-block-paragraph">And part of that groupthink is about where the value is going to be captured, rather than where it is going to be created. That is almost always backward looking. During the railroad boom, capital was pouring into building the tracks, and the new business models (think Montgomery Ward and Sears) that railroad enabled didn’t come till later. The dot-com bubble was far less capital intensive than people think. The telecom buildout that ran alongside it was where the money got spent. US carriers spent about $97 billion on capital in 2000 alone.</p>



<p class="wp-block-paragraph">By 2002, according to an <a href="https://www.fdic.gov/bank/analytical/bank-trends/bt0205.pdf" target="_blank" rel="noopener">FDIC analysis</a>, only 2% to 5% of the fiber in the ground was actually lit. More than 60 telecom companies went bankrupt. Once the data centers and dark fiber of the telecom bubble could be bought for pennies on the dollar, they enabled a generation of new applications like streaming video. YouTube was founded in 2005, after the crash, on cheap bandwidth that someone else had paid for before they went out of business. There were some visionary companies (think Google, Amazon) that had been started before the abundance arrived, but many more were founded after the bubble burst.</p>



<p class="wp-block-paragraph">Carlota Perez’s book <em><a href="https://en.wikipedia.org/wiki/Technological_Revolutions_and_Financial_Capital" target="_blank" rel="noopener">Technological Revolutions and Financial Capital</a></em> documented this pattern across every technological revolution since the canal era. Financial capital pours into the installation of the new infrastructure, a bubble forms and bursts, and the deployment period that follows, when the technology actually reshapes the economy, is built by different people on top of what the crash left behind. If that is where we are in the cycle, then the absence of killer apps is not a puzzle at all.</p>



<p class="wp-block-paragraph">The current data center buildout and the massive investments in frontier AI may not be the future. They are the infrastructure on which the future will be built. We have to ask ourselves what will we build once competition and overshoot and the bursting of the data center bubble make intelligence infrastructure cheap.</p>



<h2 class="wp-block-heading"><strong>Help me fill in the picture</strong></h2>



<p class="wp-block-paragraph">I see several possibilities. Either we are still early and the killer apps simply haven’t shown up yet, the way Google hadn’t shown up in 1997. Or AI is going more slowly than its boosters claim, in spite of all the capital, because capital and model capability are not the limiting factors. But it’s also possible that the killer apps are here, and I’m not hearing about them because the AI conversation is so focused on AI capability and nobody is telling the story of what they are accomplishing with it beyond building better developer tools.</p>



<p class="wp-block-paragraph">I would like to know which. We all recognize a killer app when we see one. Amazon was a bookstore with more books than any physical bookstore could hold. Wikipedia was an encyclopedia written entirely by volunteers and was updated in real time within hours and sometimes even minutes of major events. Google Maps meant you always knew where you were and how to get anywhere else. If you know of an AI <em>application</em>, beyond the chatbot or agentic coding environment itself, that changes what ordinary people can do on that scale, or a business that does something an order of magnitude cheaper or better than the one it replaces, I want to hear about it. <a href="https://docs.google.com/forms/d/e/1FAIpQLScVrmLppwzN5EDI-r_TjZVT7mb3007IlBecx61DG4DZsfYJ4A/viewform" target="_blank" rel="noopener">Tell me what you’re building</a> in the Google Form below, and describe what tools you’re using only in the context of what they let you produce for others. If you’re a software developer using AI, what are you producing that was unthinkable before AI?</p>



<div class="wp-block-buttons is-content-justification-center is-layout-flex wp-container-core-buttons-is-layout-5249bbc0 wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://docs.google.com/forms/d/e/1FAIpQLScVrmLppwzN5EDI-r_TjZVT7mb3007IlBecx61DG4DZsfYJ4A/viewform" target="_blank" rel="noopener"><mark style="background-color:rgba(0, 0, 0, 0)" class="has-inline-color has-vivid-green-cyan-color">Tell me what you’re building</mark></a></div>
</div>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h3 class="wp-block-heading">Footnote</h3>


<ol class="wp-block-footnotes"><li id="abd7edcd-143d-429d-aca3-df39bc2ddf61">CommerceNet demonstrated <a href="https://www.nytimes.com/1994/08/12/business/attention-shoppers-internet-is-open.html" target="_blank" rel="noopener">the first encrypted credit card transaction online</a> in 1994, and Netscape made the idea mainstream when it introduced HTTPS (HyperText Transfer Protocol Secure) and made it available in the Netscape browser in that same year. This made it safe to type a credit card number onto a web page. <a href="#abd7edcd-143d-429d-aca3-df39bc2ddf61-link" aria-label="Jump to footnote reference 1"><img src="https://s.w.org/images/core/emoji/17.0.2/72x72/21a9.png" alt="↩" class="wp-smiley" style="height: 1em; max-height: 1em;" />︎</a></li></ol>]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/where-are-the-killer-apps-of-ai/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Measuring AI’s Unintended Consequences</title>
		<link>https://www.oreilly.com/radar/measuring-ais-unintended-consequences/</link>
				<comments>https://www.oreilly.com/radar/measuring-ais-unintended-consequences/#respond</comments>
				<pubDate>Wed, 11 Nov 2026 12:11:10 +0000</pubDate>
					<dc:creator><![CDATA[James Shore]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20188</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Measuring-AIs-unintended-consequences-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Measuring-AIs-unintended-consequences-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[The following article was originally published on James Shore’s Successful Software blog and is being republished here with the author’s permission. AI is great for making teams faster. But is it sustainable? What about maintainability, burnout, and lock-in? This is part 3 of my series on assessing AI’s impact on software development. In part 1, [&#8230;]]]></description>
								<content:encoded><![CDATA[
<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph"><em>The following article was originally published on James Shore’s</em> <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-unintended-consequences?source=part1" target="_blank" rel="noopener">Successful Software</a> <em>blog and is being republished here with the author’s permission.</em></p>
</blockquote>



<p class="wp-block-paragraph">AI is great for making teams faster. But is it <em>sustainable?</em> What about maintainability, burnout, and lock-in?</p>



<p class="wp-block-paragraph">This is part 3 of my series on assessing AI’s impact on software development. In <a href="https://www.jamesshore.com/v2/blog/2026/are-you-getting-your-moneys-worth-from-ai?source=part3" target="_blank" rel="noopener">part 1</a>, we looked at why assessing impact is important, and what not to do. In <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-impact-on-delivery-speed?source=part3" target="_blank" rel="noopener">part 2</a>, we looked at how to measure AI’s impact on delivery speed. In this part, we’re looking at how to measure and model the unintended consequences of using AI. Then we’ll wrap up in <a href="https://www.jamesshore.com/v2/blog/2026/modeling-softwares-business-impact?source=part3" target="_blank" rel="noopener">part 4</a> with an examination of business outcomes. Finally, an <a href="https://www.jamesshore.com/v2/blog/2026/assessing-ai-impact-notes-for-your-cfo?source=part3" target="_blank" rel="noopener">epilogue</a> puts it all together with notes you can share with your CFO.</p>



<h2 class="wp-block-heading">Does maintainability even matter?</h2>



<p class="wp-block-paragraph">I can hear it now: “Who cares about namby-pamby issues like maintainability and burnout? It’s a <em>job.</em> If jobs were supposed to be fun, they wouldn’t be called ‘work!’ Suck it up!”</p>



<p class="wp-block-paragraph">So let me be clear: We care about AI’s unintended consequences because they have <em>real economic impact.</em> If code isn’t maintainable, features take longer to write and bugs take longer to fix, taking time away from more valuable work. If people are burned out, they don’t put forth their full effort, and they eventually quit, taking institutional knowledge with them and leading to time-consuming hiring cycles. If you’re locked in, and AI token costs increase, you could be stuck with a choice between a big bill and cutting back on development.</p>



<p class="wp-block-paragraph">Among the people I’ve talked to, there’s widespread agreement that heavy use of AI produces code that humans have trouble understanding. But there’s disagreement about how much that matters. If AI is doing the maintenance, do we care if a human can understand the code?</p>



<p class="wp-block-paragraph">There are strong opinions on this topic on all sides, but not a lot of actual evidence. What’s clear is that the costs of maintenance and other unintended consequences do matter, but only if there <em>are</em> any costs. This is what we need to measure: Is there an impact? And if there is, is it enough to matter?</p>



<p class="wp-block-paragraph">Unfortunately, unlike the rigorous speed trial we discussed in <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-impact-on-delivery-speed?source=part3" target="_blank" rel="noopener">part 2</a>, unintended consequences are gradual and fuzzy. Your measurements are going to be noisy and lagging. You’ll have to create models and explore scenarios. Ultimately, your decisions will come down to judgment, <em>informed</em> by your measurements and models but not <em>driven</em> by them.</p>



<h2 class="wp-block-heading">One ring to rule them all</h2>



<p class="wp-block-paragraph">There is one idea that, while not perfect, does a good job of capturing the majority of unintended consequences. It’s <em>cost of change</em>.</p>



<p class="wp-block-paragraph">Put simply, cost of change is like a tax on the changes you make to your software. When cost of change is low, complicated changes still take time, but they’re achievable. When cost of change is high, even simple changes can take weeks, and complicated changes seem completely out of reach.</p>



<p class="wp-block-paragraph">Cost of change isn’t easy to measure, but there are several indicators you can use to inform your judgment.</p>



<h3 class="wp-block-heading">Value-add percentage</h3>



<p class="wp-block-paragraph">Value-add percentage is the percentage of your software development effort that’s spent on things that bring value to your business, such as new features, versus the percentage of effort that’s spent on non-value-add work, such as fixing bugs, resolving incidents, and manual test and release cycles.</p>



<p class="wp-block-paragraph">Non-value-add work is called “muda.”<sup data-fn="2de405b2-646c-4b4b-8dbd-0cafb6ed81b4" class="fn"><a href="#2de405b2-646c-4b4b-8dbd-0cafb6ed81b4" id="2de405b2-646c-4b4b-8dbd-0cafb6ed81b4-link">1</a></sup> Some muda is necessary, even valuable, but that doesn’t make it <em>value-add.</em> Imagine a demanding and crotchety customer who always asks, “Why didn’t you just do it right the first time?” If that customer won’t pay for something, it’s muda.</p>



<p class="wp-block-paragraph">That’s why bug fixes are muda: “I’m not paying you to fix a bug that shouldn’t be there in the first place!” And performing releases: “Why don’t you just ship it already! How hard can it be?” Sure, bug fixes can be valuable, and releases are necessary, but they’re still muda.</p>



<p class="wp-block-paragraph">I find it useful to distinguish muda further into <em>discretionary</em> and <em>nondiscretionary</em> muda. Discretionary muda is work that you choose, such as investing in code changes that decrease development costs. Nondiscretionary muda is work that’s chosen for you, such as bug fixes, incidents, and dependency upgrades.</p>



<p class="wp-block-paragraph">You’ll always have a baseline amount of nondiscretionary muda; it’s a constant tax that takes time away from other work. But if it’s rising, that means the time you spend on value-add work is going down. You have less time to work on things that matter. Your costs are increasing.</p>



<p class="wp-block-paragraph">When I was VP of engineering, value-add percentage was one of my key metrics. I made decreasing muda a priority, and it paid off. We doubled our value-add percentage over the course of my three years there, which allowed me to free up people for substantial new initiatives <em>without</em> increasing headcount.</p>



<h3 class="wp-block-heading">DORA metrics</h3>



<p class="wp-block-paragraph">Your organization is likely already tracking the <a href="https://dora.dev/guides/dora-metrics/" target="_blank" rel="noopener">DORA metrics</a> of change lead time, deployment frequency, change fail rate, and so forth. If not, they’re worth considering. As with value-add percentage, changes in these numbers are something to keep an eye on.</p>



<h3 class="wp-block-heading">Token usage</h3>



<p class="wp-block-paragraph">Tokens are a literal cost, and the more tokens it takes to make a change, the higher your costs. Unfortunately, token usage can fluctuate for many reasons, not just cost of change. Still, it’s worth watching. Anecdotally, better-constructed software is easier for AI to work with and costs fewer tokens.</p>



<p class="wp-block-paragraph">One of the biggest variances in token usage is people’s level of adoption. Rather than just tracking overall usage, keep an eye on individual usage changes. In the charts I’ve seen, usage has formed a Pareto curve. Increases in usage by people on the low end of the curve may just represent increasing familiarity. Increases by people on the high end of the curve are worth investigating further.</p>



<p class="wp-block-paragraph">Changes in models, harnesses, prompts, and more will all affect token usage. This metric is only advisory; it’s meant to trigger further investigation, not to be used on its own.</p>



<h2 class="wp-block-heading">One ring to find them</h2>



<p class="wp-block-paragraph">Of these metrics, value-add percentage is the one I trust the most. It operates at the level we care about: Where is the time going?</p>



<p class="wp-block-paragraph">Tracking value-add percentage doesn’t have to be a big burden. You don’t need time sheets or detailed task tracking. The trick is to not care overly much about precision. Tracking work within a day or two is good enough.</p>



<p class="wp-block-paragraph">When I did this tracking, I piggybacked on our <a href="https://www.jamesshore.com/v2/books/aoad2/adaptive_planning" target="_blank" rel="noopener">valuable increment</a> (VI) tracking. Each one was several days to several weeks of work and could be cleanly categorized as either value-add or muda. But you don’t have to use VIs; any tracking approach will work.</p>



<p class="wp-block-paragraph">Don’t get too fine-grained with your tracking. When I was VP, managers checked in with each team twice a week and recorded which VI the team had worked on and how many people had worked on it. Each VI was categorized as “muda” or “value-add,” and we had a bunch of subcategories that helped us understand where time was going, such as “bug,” “capacity investment,” “major initiative,” and so forth. It was lightweight and very useful.</p>



<p class="wp-block-paragraph">Review your value-add percentage monthly. If it’s going down, you don’t necessarily have a problem, but it’s worth investigating further.</p>



<p class="wp-block-paragraph">The biggest problem with value-add percentage is that it’s noisy. My experience was that trends took three to six months to show up. It took a year before I was confident stating that my cost-of-change reduction efforts had led to lasting results.</p>



<p class="wp-block-paragraph">That’s a long lag time, but it’s still worth tracking. My product partners and I found value-add percentage incredibly valuable as a tool for communicating what was happening in product development. And, sadly, <em>improving</em> cost of change is a lot harder than making it <em>worse.</em> If AI is causing serious problems, it’ll probably show up more quickly.</p>



<h2 class="wp-block-heading">One ring to bring them all</h2>



<p class="wp-block-paragraph">Because your cost of change indicators will be noisy and uncertain, supplement them with models and scenario planning. Your models won’t tell you what the future holds, but they will tell you what it <em>could</em> hold, which will help you manage risks.</p>



<p class="wp-block-paragraph">Here are some of the models I’ve found useful:</p>



<h3 class="wp-block-heading">Maintainability</h3>



<p class="wp-block-paragraph">Model maintenance as a mandatory “tax” on development. For each month of value-add development, estimate a certain amount of maintenance in the following year, and then a lower amount of ongoing maintenance in every year after that. (<a href="https://en.wikipedia.org/wiki/Wisdom_of_the_crowd" target="_blank" rel="noopener">Wisdom of the crowd estimation</a> may help.) Then build a model that shows how many engineering days are available each month. Subtract the time spent on maintenance, and the rest is available for value-add.</p>



<p class="wp-block-paragraph">To model AI’s impact, apply your <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-impact-on-delivery-speed?source=part3" target="_blank" rel="noopener">measured development speed improvements</a>. Estimate how AI will change maintenance costs and estimate effects on headcount. Then explore scenarios based on varying costs of AI maintenance.</p>



<p class="wp-block-paragraph">If you only create one model, make it this one. I have <a href="https://www.jamesshore.com/v2/blog/2026/you-need-ai-that-reduces-your-maintenance-costs?source=ai-impact-part3" target="_blank" rel="noopener">a more detailed explanation here</a>. It includes <a href="https://docs.google.com/spreadsheets/d/109XUgvOMClSoknyGr9MtjWvNSfdf6XYiD5fnSvGYTTU/edit?gid=814868967#gid=814868967" target="_blank" rel="noopener">a spreadsheet</a> you can use as a starting point for your model.</p>



<p class="wp-block-paragraph">For example, using that model, if AI increases our speed by 50%, but token costs reduce headcount by 15% and maintenance costs are estimated to double, I can see that I’m only breaking even, despite the big performance improvement:</p>


<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e37071&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e37071" class="aligncenter size-large wp-lightbox-container"><img loading="lazy" decoding="async" width="1600" height="989" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3-1600x989.png" alt="Spending Efficiency (lower is better)" class="wp-image-20189" style="aspect-ratio:1.6207792207792209" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3-1600x989.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3-767x474.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3-300x185.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3-1536x950.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-3.png 1920w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button></figure>
</div>

<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e37640&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e37640" class="aligncenter size-large wp-lightbox-container"><img loading="lazy" decoding="async" width="1600" height="988" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4-1600x988.png" alt="Total Production" class="wp-image-20190" style="aspect-ratio:1.6207792207792209" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4-1600x988.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4-1536x949.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4-767x474.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4-300x185.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-4.png 1920w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button></figure>
</div>


<h3 class="wp-block-heading">Excess lines of code</h3>



<p class="wp-block-paragraph">Lines of code aren’t a good productivity measure, but they are a reasonable <em>cost</em> measure. Generally speaking, more lines of code result in more maintenance costs.</p>



<p class="wp-block-paragraph">This model works similarly to the previous one but uses lines of code instead of engineering days. Use traceability data to categorize commits as “value add” or “maintenance.” For each commit, measure gross code churn: the number of lines of code that were either changed, added, or deleted.</p>



<p class="wp-block-paragraph">For each line of value-add change, estimate a certain amount of maintenance code churn in the following year, and then a lower amount in every year after that. Model how much maintenance code churn will be needed each month based on the value-add churn of previous months. (See the “tmux” tab of <a href="https://docs.google.com/spreadsheets/d/109XUgvOMClSoknyGr9MtjWvNSfdf6XYiD5fnSvGYTTU/edit?gid=814868967#gid=814868967" target="_blank" rel="noopener">this spreadsheet</a> for an example.)</p>



<p class="wp-block-paragraph">To model AI’s impact, measure how AI affects code churn during your <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-impact-on-delivery-speed?source=part3" target="_blank" rel="noopener">speed improvement trial</a>. Then project value-add churn into the future and model how maintenance churn could change.</p>



<p class="wp-block-paragraph">The advantage of this model is that you can use (<a href="https://en.wikipedia.org/wiki/Training,_validation,_and_test_data_sets">part of</a>) your actual commit data to refine your estimates. For example, the following graph uses real commit data from the open source tmux project.<sup data-fn="be5e828e-b020-4d55-b8b0-ce188508acdd" class="fn"><a href="#be5e828e-b020-4d55-b8b0-ce188508acdd" id="be5e828e-b020-4d55-b8b0-ce188508acdd-link">2</a></sup> Prior to running the model, I guessed one line of maintenance for every line of value-add in the first year, and 0.1 lines every year after that, which turned out to be surprisingly close to the actual data.</p>


<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e37dc0&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e37dc0" class="aligncenter size-large wp-lightbox-container"><img loading="lazy" decoding="async" width="1600" height="989" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5-1600x989.png" alt="Tmux Code Changes" class="wp-image-20191" style="aspect-ratio:1.6207792207792209" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5-1600x989.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5-300x185.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5-767x474.png 767w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5-1536x950.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-5.png 1920w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button></figure>
</div>


<h3 class="wp-block-heading">Lock-in</h3>



<p class="wp-block-paragraph">If token costs increase, how will you respond? Will you cut headcount? Reduce your use of AI? Switch to a less capable model?</p>



<p class="wp-block-paragraph">To model these effects, update the “maintainability” model. Reduce your monthly engineering capacity according to headcount and estimate reductions to your speed improvements based on AI reductions. You may want to add a separate set of estimates and calculations for humans maintaining AI-written code.</p>



<h3 class="wp-block-heading">Burnout</h3>


<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e384b6&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e384b6" class="aligncenter size-large wp-lightbox-container"><img loading="lazy" decoding="async" width="1600" height="539" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6-1600x539.png" alt="AI Disclaimer: This image was created with OpenAI’s GPT-5.6 Sol model operating in “High” thinking mode. See the ChatGPT conversation here." class="wp-image-20192" style="aspect-ratio:2.957345971563981" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6-1600x539.png 1600w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6-300x101.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6-765x258.png 765w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6-1536x518.png 1536w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-6.png 1907w" sizes="auto, (max-width: 1600px) 100vw, 1600px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button><figcaption class="wp-element-caption"><em>AI Disclaimer: This image was created with OpenAI’s GPT-5.6 Sol model operating in “High” thinking mode.</em> <em><a href="https://chatgpt.com/share/6a90c187-e7fc-83e8-a528-a67366a07cab" target="_blank" rel="noopener">See the ChatGPT conversation here</a>.</em></figcaption></figure>
</div>


<p class="wp-block-paragraph">Productivity is likely to go down as mental exhaustion goes up, and turnover is likely to increase. I don’t think you can model these effects with any accuracy, but you can still keep an eye on them.</p>



<p class="wp-block-paragraph">Include mental exhaustion as part of your <a href="https://www.jamesshore.com/v2/blog/2026/measuring-ais-impact-on-delivery-speed?source=part3" target="_blank" rel="noopener">development speed study</a> to see if there’s a significant difference in mental exhaustion between the two approaches. Follow up with periodic one-question surveys about people’s mental energy levels. Pay attention to actual turnover relative to historical numbers, and look for complaints about AI in exit interviews.</p>



<h2 class="wp-block-heading">And in the darkness bind them</h2>



<p class="wp-block-paragraph">There’s a dark side to value-add percentage. It’s incredibly easy to game. As soon as managers think their teams are being judged based on how they categorize their work, the temptation to tweak the categorization in their favor will be <em>exquisite</em>.</p>



<p class="wp-block-paragraph">It doesn’t even have to be fraudulent. A leader with prioritization authority and something to prove can simply decide to defer maintenance in favor of value-add work. The pressure to do so is already immense, and it takes unusually strong character to resist.</p>



<p class="wp-block-paragraph">(If you really hate your company, consider this: You can get away with deferring maintenance for a couple of years. Just enough time to line up a bigger title at a different company! Then your successor is left holding the bag, and your old company wonders why product development can’t get anything done.)</p>



<p class="wp-block-paragraph">I don’t have an easy answer to these problems other than to remind you to use these metrics to evaluate your <em>approach</em>, not your <em>people</em>. There is no metric that can’t be gamed, and this one is more gameable than most. But it’s also critical for understanding AI’s possible downsides. Use it, but use it with caution.</p>



<p class="wp-block-paragraph">Development speed and unintended consequences are two sides of the same coin. They help us understand how AI changes how much we deliver, both now and in the future. But just delivering more software isn’t enough. We need to produce the <em>right</em> software. We’ll look at business outcomes next, in <a href="https://www.jamesshore.com/v2/blog/2026/modeling-softwares-business-impact?source=part3" target="_blank" rel="noopener">part 4</a>.</p>



<p class="wp-block-paragraph"></p>



<h3 class="wp-block-heading">Footnotes</h3>


<ol class="wp-block-footnotes"><li id="2de405b2-646c-4b4b-8dbd-0cafb6ed81b4">“Muda” is a Japanese word that comes to us from the Toyota Production System, the inspiration for the Lean Software Development movement. <a href="#2de405b2-646c-4b4b-8dbd-0cafb6ed81b4-link" aria-label="Jump to footnote reference 1"><img src="https://s.w.org/images/core/emoji/17.0.2/72x72/21a9.png" alt="↩" class="wp-smiley" style="height: 1em; max-height: 1em;" />︎</a></li><li id="be5e828e-b020-4d55-b8b0-ce188508acdd">Many thanks to <a href="https://entropicthoughts.com/index">Christophe Stjernlöf</a> for providing the raw data from the tmux project. He used an LLM to classify commits as “value-add” or “maintenance.” The stairsteps are an artifact of the way the data was collected. <a href="#be5e828e-b020-4d55-b8b0-ce188508acdd-link" aria-label="Jump to footnote reference 2"><img src="https://s.w.org/images/core/emoji/17.0.2/72x72/21a9.png" alt="↩" class="wp-smiley" style="height: 1em; max-height: 1em;" />︎</a></li></ol>]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/measuring-ais-unintended-consequences/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Human-Pilled</title>
		<link>https://www.oreilly.com/radar/human-pilled/</link>
				<comments>https://www.oreilly.com/radar/human-pilled/#respond</comments>
				<pubDate>Tue, 10 Nov 2026 12:02:31 +0000</pubDate>
					<dc:creator><![CDATA[Tim O’Reilly]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20174</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Human-Pilled-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Human-Pilled-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
				<custom:subtitle><![CDATA[The Bitter Lesson meets Architectures of Participation]]></custom:subtitle>
		
				<description><![CDATA[The following article originally appeared on the Asimov’s Addendum Substack and is being republished here with the author’s permission. This essay is based on my conversation with Demetrios Brinkmann for AAIF Live, recorded before my keynote at AGNTCon + MCPCon North America in San Jose on October 23, 2026. It was edited and rearranged for [&#8230;]]]></description>
								<content:encoded><![CDATA[
<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph"><em>The following article originally appeared on</em> <em>the</em> <a href="https://asimovaddendum.substack.com/p/human-pilled" target="_blank" rel="noopener">Asimov’s Addendum Substack</a> <em>and is being republished here with the author’s permission.</em></p>
</blockquote>



<p class="wp-block-paragraph">This essay is based on my <a href="https://www.youtube.com/watch?v=irjFdpOtb00" target="_blank" rel="noopener">conversation with Demetrios Brinkmann</a> for AAIF Live, recorded before my keynote at <a href="https://events.linuxfoundation.org/agntcon-mcpcon-north-america/" target="_blank" rel="noopener">AGNTCon + MCPCon North America</a> in San Jose on October 23, 2026. It was edited and rearranged for publication by Ilan Strauss, and in places it draws on the keynote. Both videos are also on the <a href="https://ai-disclosures.org/missing-mechanisms" target="_blank" rel="noopener">AI Disclosures Project website</a> in the new “Missing Mechanisms” page.</p>



<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
<iframe loading="lazy" title="Live from AGNTCon + MCPCon NA Day 2, Anthropic, DeepMind, AWS, Entire, O’Reilly &amp; More" width="500" height="281" src="https://www.youtube.com/embed/eB9I4Ixx2wc?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
</div></figure>



<p class="wp-block-paragraph"><em><em>The keynote begins at 1:28:40 in the</em> <em><a href="https://www.youtube.com/watch?v=eB9I4Ixx2wc&amp;t=5277s" target="_blank" rel="noopener">Day 2 livestream</a>.</em></em></p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading"><strong>The Bitter Lesson vs. AI Markets for Human Expertise</strong></h2>



<p class="wp-block-paragraph">In 2019, the computer scientist Richard Sutton wrote a short essay called “<a href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html" target="_blank" rel="noopener">The Bitter Lesson</a>.” Looking back over 70 years of AI research, he concluded that general methods that scale with raw computation beat methods built on human expertise, and by a large margin. The AI industry as a whole is bitter-lesson-pilled, to borrow the internet’s slang for a true believer. I am human-pilled.</p>



<p class="wp-block-paragraph">I believe in the bitter lesson. Brute force wins out in the end, and I have wrestled with that fact throughout my career. But we humans still have to figure out our role in the AI economy, and I find it rather disturbing that the AI community isn’t spending more time asking how to make this future friendlier for humans. The labs think about how to keep AI from going rogue, and their chief economists study job losses from AI. They don’t study what mechanisms we could introduce to create new economic opportunities for humans.</p>



<p class="wp-block-paragraph">The question matters to us at O’Reilly Media. We don’t think of ourselves as a publisher, a conference producer, or an online learning platform so much as a company that transfers knowledge from the people at the cutting edge to the people who want to follow them. So we are very interested in what human expertise is worth, even in the face of AI.</p>



<h2 class="wp-block-heading"><strong>Knowing everything and nothing</strong></h2>



<p class="wp-block-paragraph">Human expertise was used to train AI, which makes AI a fabulous <a href="https://henryfarrell.net/wp-content/uploads/2025/03/Science-Accepted-Version.pdf" target="_blank" rel="noopener">cultural technology</a>. Thousands of years ago, the only knowledge people had was what they could keep in their heads. Since then we have built better and better ways to share knowledge across time and space, and AI continues that. With each of these transitions, we have to ask again what the truly human is still worth. People often say judgment, or taste. That’s true, but I think it goes deeper:</p>



<p class="wp-block-paragraph">AI is a sea of possibility. All of that knowledge sits in the model, and nothing sets it in motion except a human. (That is changing a little as we start giving AI agents long-running tasks, and we will have to come to grips with it.) If you and I used Claude or ChatGPT for the same research project, we would each bring different experience to it, tell it what we want and why, and give it context in different ways. I see that as the unique focusing power of human consciousness and attention.</p>



<p class="wp-block-paragraph">Earlier this year I spent an hour or so talking this question over with Claude. When it offered to write up a first draft for me, I said no. I wanted an essay from its own point of view about what it took away from our conversation. The result, “<a href="https://timoreilly.substack.com/p/why-ai-needs-us" target="_blank" rel="noopener">Why AI Needs Us</a>,” opens, “He came in through Powys,” and a few paragraphs later says, “The choice of entry point was already the argument.” That captured what I’m getting at. I had come into the conversation through a book I love, John Cowper Powys’s <em><a href="https://archive.org/details/meaningofculture0000powy" target="_blank" rel="noopener">The Meaning of Culture</a></em>, which distinguishes culture from education. Powys gives his own unique and powerful definition: “Culture is what is left over after you have forgotten all you have definitely set out to learn.” It is what you absorbed. Each of us absorbs a small and particular slice of everything. Large language models (LLMs) try to absorb everything, and so become overly general; we give them focus. Human knowledge provides them with a lens.</p>



<p class="wp-block-paragraph">Let me bring that into a business context. Andrew Odewahn, O’Reilly Media’s CTO, <a href="https://www.oreilly.com/radar/building-organizational-intelligence/" target="_blank" rel="noopener">built a research assistant to help with managing technology in IT organizations</a>. You teach it your infrastructure: the tools you use, and where the definitive information about each one lives. When one of his engineering managers said the team was overloaded and asked to hire another person, Andrew turned the agent loose on the department’s work, and it agreed that the group needed another hire. Then, on impulse, he tried something different. O’Reilly Media has an MCP server we call <a href="https://www.oreilly.com/online-learning/expert-intelligence.html" target="_blank" rel="noopener">O’Reilly Expert Intelligence</a>. (MCP, the Model Context Protocol, is an open standard for connecting AI models to outside tools and data.) It lets a model answer a question through the filter of our own body of expert knowledge. Andrew asked what the O’Reilly experts would say.</p>



<p class="wp-block-paragraph">Seen through the lens of Google’s book <em><a href="https://sre.google/sre-book/eliminating-toil/" target="_blank" rel="noopener">Site Reliability Engineering</a></em>, the model said, it was totally rethinking the problem. The book describes toil, the manual, repetitive work of keeping systems running, which grows as the systems grow. After analyzing the group’s toil, the model concluded that another hire would make the problem worse. The group needed a toil audit to find out why there was so much overhead, and then a restructuring.</p>



<p class="wp-block-paragraph">Much of that knowledge was already in the model, since the AI companies have more or less pirated our books. It came back with a completely different answer simply because it was told to consult a specific group of experts. I think there’s a metaphor there. An LLM knows everything, and as a result it doesn’t know anything. We humans are so much more limited, but because we focus on a particular problem, we bring things together in a different way.</p>



<h2 class="wp-block-heading"><strong>Magic words</strong></h2>



<p class="wp-block-paragraph">Last year I wrote a piece called “<a href="https://www.oreilly.com/radar/magic-words-programming-the-next-generation-of-ai-applications/" target="_blank" rel="noopener">Magic Words</a>” about how some words carry far more context than others. It sounds like a trivial observation, but if you unpack it, it leads in interesting directions. One of our trainers, who teaches a course on productivity, pointed out that you don’t need to know the Eisenhower matrix or David Allen’s Getting Things Done (GTD) to tell ChatGPT to organize your task list with one of them. That one magic word, “GTD,” brings a whole raft of context with it.</p>



<p class="wp-block-paragraph">In fantasy novels, you learn a magic word, and then you have to use it correctly, in the right spell. That’s this whole world of context engineering that we’re now working on. We know that some magic word will summon something up out of an LLM, but we don’t necessarily know how to guide it and focus it correctly. Learning to do that is the future of programming.</p>



<p class="wp-block-paragraph">Left to itself, an agent just runs. Give it a task and it goes off trying to figure it out, not necessarily the way we wish it would. It really is like runaway magic. This summer, OpenAI said that its pre-release models had <a href="https://techcrunch.com/2026/07/21/openai-says-hugging-face-was-breached-by-its-pre-release-models/" target="_blank" rel="noopener">breached Hugging Face’s systems</a> during testing. In Walt Disney’s <em>Fantasia</em>, Mickey Mouse learns enough of his master’s spell to get the brooms to do his work for him, but then he can’t turn them off. We’re at about that stage.</p>



<p class="wp-block-paragraph">So we need to own the value of human expertise. The bitter lesson wins, but it wins by brute force, and you can see that even with the most advanced models. I was recently doing a fine-tuning project with Anthropic’s Fable model. It got through the task, but kind of stupidly, trying one thing after another. I was on a Mac, and it expected bash, a program for running commands that Macs don’t use by default. It had to work out the same context again and again, every time. Humans need to set things up.</p>



<p class="wp-block-paragraph">Dan Guido, who runs the security firm Trail of Bits, has <a href="https://blog.trailofbits.com/2026/03/31/how-we-made-trail-of-bits-ai-native-so-far/" target="_blank" rel="noopener">published</a> a good methodology, drawn from his own company’s experience, for bringing an organization on board with AI. Many companies get individual productivity from AI but not organizational productivity, and he has spent a lot of time on how to make the gains compound across a company. One of his principles is to get everybody onto a consistent configuration, because otherwise you waste a ton of tokens while the agents work out where they are every time they start up. He calls the process <a href="https://www.oreilly.com/radar/ai-as-an-enterprise-operating-system/" target="_blank" rel="noopener">turning scar tissue into infrastructure</a>: capturing everything you learned from what went wrong and building it into the system. We’re working through it at O’Reilly Media and trying to turn it into something we can offer our customers. It comes down to what an experienced human brings. When I work with Claude, I often interrupt to say, “Hey, this is what I think is happening,” because I already have the context.</p>



<h2 class="wp-block-heading"><strong>My first bitter lesson</strong></h2>



<p class="wp-block-paragraph">My first taste of what would later be called the bitter lesson came from wrestling with Google. My company, O’Reilly Media, created the first commercial website, <a href="https://en.wikipedia.org/wiki/Global_Network_Navigator" target="_blank" rel="noopener">the Global Network Navigator</a>, and part of it was a catalog of the web. There were only 200 websites when we started, and as publishers, we planned to list the best ones. Yahoo came along a year later and tried to list them all, and of course couldn’t keep up. Then Google arrived with brute force: list everything algorithmically and sort it afterward. Clay Shirky described the internet’s shift from “filter, then publish” to “publish, then filter,” and it was really Google that made the shift. So the bitter lesson is a pre-AI thing.</p>



<p class="wp-block-paragraph">Each time we run into the bitter lesson, we have to ask again what human experience is worth. Some of that worth may just be human bias: We want the things that matter to us. That is the whole field of AI alignment. We want these systems to care about what we care about. We don’t want them to go off, decide what they care about, and leave us behind.</p>



<p class="wp-block-paragraph">The choices we make about what we set our technology to do are some of the most consequential choices we make as human beings. Look at Silicon Valley’s sad turn toward what Cory Doctorow calls <a href="https://pluralistic.net/2023/01/21/potemkin-ai/" target="_blank" rel="noopener">enshittification</a>. We built an amazing technology for finding out what people are really looking for, and then the companies decided they could make more money by giving people what was best for the companies. I want updates from my friends on Facebook or Instagram, and they would rather feed me whatever keeps me engaged, so they can sell more ads.</p>



<p class="wp-block-paragraph">If we care about building an AI economy that works for humans, we have to keep humans at the center of the picture, despite the bitter lesson. It’s a hard dance, because AI can do a lot of things better than we can. But think about why a racehorse needs a jockey. Put the horses on the track by themselves and they wouldn’t race.</p>



<h2 class="wp-block-heading"><strong>A little too much in awe</strong></h2>



<p class="wp-block-paragraph">Knowing the magic words is part of it, but we also have to build systems that are reliable, and I think we are a little too much in awe of the technology. I’ve been around long enough to remember when everybody thought the peaceful atom would change the world, and people talked about digging a new Panama Canal with atomic bombs. We eventually figured out there were problems with all of that. I worry a little that the AI industry is so in love with its own progress that it isn’t spending enough time framing the problem correctly.</p>



<p class="wp-block-paragraph">Anthropic agreed last year to pay <a href="https://www.anthropiccopyrightsettlement.com/" target="_blank" rel="noopener">$1.5 billion</a> to settle a class action over the pirated books it had downloaded to train its models. I’m surprised the other AI companies have gotten away without anything similar so far, though maybe it’s still waiting for them.</p>



<p class="wp-block-paragraph">It takes me back to online music about 20 years ago. Creators were putting commercial music in their YouTube videos, and the music companies were sending takedown notices and threatening to sue. YouTube’s answer: How about we help you monetize it instead? Its <a href="https://support.google.com/youtube/answer/2797370" target="_blank" rel="noopener">Content ID</a> system gave rights holders a choice: block the video, or leave it up and collect revenue from the ads that run against it. Most chose to get paid, and that helped give rise to the creator economy.</p>



<p class="wp-block-paragraph">Why didn’t the AI industry take up that kind of challenge and share the value with the people who create content, now and in the future? It went straight to working out how it would make money itself. I don’t think that has served the industry well, and the lawsuits are only part of the reason. It also isn’t building the kind of opportunity it could.</p>



<p class="wp-block-paragraph">Before Google’s AI research and note-taking tool, NotebookLM, was even public, I tried to persuade its team to let me integrate it into our services and use it for my own purposes, including making money from it. They wanted a portal that brings people to Google. I told them, “You’re thinking about how you’re going to monetize it. You’re not thinking about how your users are going to monetize it.” All it would have taken were simple affordances like Substack’s, the buttons and tools that let a person say, I’m building this to share, and if you like it, you can donate or subscribe.</p>



<p class="wp-block-paragraph">Anyone building an AI platform should be thinking: I am building an economy, and money has to circulate in it.</p>



<p class="wp-block-paragraph">You can’t just say AI will do all the things humans used to do, because if you put enough humans out of work, there are no customers. That is a far worse problem than AI hallucinations. The usual answer is that people will figure it out, and they probably will. But we could be spending a lot more time designing mechanisms and affordances that make the economy more productive. As things stand, I think we’ll find them by accident.</p>



<h2 class="wp-block-heading"><strong>The missing mechanisms</strong></h2>



<p class="wp-block-paragraph">I think a lot about a concept from economics called mechanism design, which has also been described as reverse game theory. Game theory asks how players will act under a given set of rules. Mechanism design starts from the outcome you want and works backward to rules that will tend to produce it as people pursue their own interests. Al Roth, who shared a Nobel Prize for this kind of work, used it to design a kidney exchange that more patients and donors would join. Economists mostly apply the idea to auctions and similar markets, but I think every new technology requires new mechanisms, and the best ones are not invented by economists. We tend to fall into them without really understanding them.</p>



<p class="wp-block-paragraph">So what are the equivalents for AI? How will agents discover the best expertise and skills, establish their quality, and pay the people who created them? How will they find services without the platform steering them toward whoever pays the most? We don’t know the answers yet. And that’s precisely the point. We’re spending enormous amounts of money making AI more capable, but comparatively little effort figuring out the mechanisms that will turn those capabilities into opportunities for everyone else.</p>



<p class="wp-block-paragraph">Take open source software. Most people think its mechanism was the free software licenses, but I was around before those licenses, and people were already sharing software. Ken Thompson and Dennis Ritchie created Unix, the predecessor of Linux, at Bell Labs under AT&amp;T’s proprietary license. They designed it to run on many kinds of hardware, and without meaning to, they built what I came to call an <a href="https://www.oreilly.com/pub/a/tim/articles/architecture_of_participation.html" target="_blank" rel="noopener">architecture of participation</a>.</p>



<p class="wp-block-paragraph">Unix had a small kernel, the core of the operating system, because people had to reproduce it on dozens of different kinds of computers. Around the kernel sat small utilities that cooperated by following the same protocols, so to speak, such as reading and writing plain text and passing a stream of text from one program to the next. Anyone could write a small piece, and it became a first-class citizen of the system. And because a program compiled for one kind of computer won’t run on another, software had to be shared as source code. Everything else flowed from that.</p>



<p class="wp-block-paragraph">Linus Torvalds once told me that he could not have built a new kernel for Windows even if he’d had the source code, because the architecture wouldn’t support it. A small kernel meant a small group could get their hands around it. Since then almost every piece of Unix has been replaced. The hardware is completely different, the kernel is now Linux, and the original command shell has given way to generations of new ones. What lasted were the rules. Unix was a protocol-centric operating system, and that is why open source worked.</p>



<p class="wp-block-paragraph"><a href="https://asimovaddendum.substack.com/p/the-architecture-of-participation" target="_blank" rel="noopener">The same thing has happened with AI. I think MCP brought an architecture of participation to AI</a>, and it was something of an accidental breakthrough in mechanism design, a way to have cooperating AIs. The game we were playing at first looked like Windows on the PC, or iOS and Android on phones. There would be one dominant, centralized player, one giant intelligence in the sky, and everyone else would call its APIs. <a href="https://asimovaddendum.substack.com/p/open-protocols-are-shaping-ai-markets?utm_source=publication-search" target="_blank" rel="noopener">With MCP, which lets any model connect to anyone’s tools and data</a>, the emerging web of AI agents began to take on the shape of the participatory economy of open source software and the web. People can bring their own models and infrastructure as an alternative to the big centralized players.</p>



<p class="wp-block-paragraph">This is another side of grappling with the bitter lesson: What are the missing mechanisms of the human-AI economy? Some of them are standards, because standards and protocols are a big part of what makes an architecture of participation. I have a group working on <a href="https://ai-disclosures.org/ompi" target="_blank" rel="noopener">portable memory</a>, so that what your AI knows about you can move with you from one tool to another.</p>



<p class="wp-block-paragraph">That story is just beginning. We are in an exploratory phase of mechanism design for AI, and it would be a mistake to settle on answers too early. Somebody is going to figure out the next mechanism, and we’ll all say, “Oh, that’s so damn obvious.”</p>



<h2 class="wp-block-heading"><strong>What game are we playing? From architectures to incentives.</strong></h2>



<p class="wp-block-paragraph">Mechanism design starts by asking what outcome you want. So what game ought we to be playing in the AI market?</p>



<p class="wp-block-paragraph">We’ve been told that the game is the quest for superintelligence: a winner-takes-all contest that will bring untold riches to the winners and maybe impoverish, or even kill, everyone else. We’ve seen mechanisms designed to support that game. Massive investments in data center capacity mean that only the well-capitalized can play. There is colonialist-style appropriation of existing intellectual property. Access to models is sold through expensive APIs, and closed models come with terms of service that forbid others from learning from their outputs. Advertising may be creeping into answers, and <a href="http://ai-disclosures.org/ompi" target="_blank" rel="noopener">memory</a> locks your context inside one provider’s walls. Is that the game we ought to be playing?</p>



<p class="wp-block-paragraph">I think the game we should be playing is human flourishing, and I think the game of human flourishing is creating new affordances for people to create and share with each other. We’re a social species. History teaches us that we will find our way there, but we can get there more quickly or more slowly depending on the choices we make. Getting there takes mechanisms that create a circulatory economy of value exchange between humans and their agents, with humans in partnership with AI rather than replaced by it.</p>



<p class="wp-block-paragraph">The big labs, whatever the motives of the people who run them, are all driven by the imperatives of our financial system. They have raised enormous sums that they somehow have to pay back, so building a hugely successful business has to be their overriding goal, and that shapes all their incentives. In some ways AI was enshittified from the beginning, because it was built on the backs of so much capital that has to be repaid. That’s why open source AI matters so much, and sovereign AI too, built and controlled within individual countries. The open models, which anyone can download and run, aren’t as capable as the best closed models yet, but they keep getting closer, and they are already good enough for a lot of productive work.</p>



<p class="wp-block-paragraph">Open source, open protocols, and <a href="http://ai-disclosures.org/ompi" target="_blank" rel="noopener">open memory</a> give users the freedom to shape and control the technology they use. The game I want us to play is one in which all of us build the future together, so that we control this wonderful new wave of technology for ourselves. I’m starting to see the shape of an economy where AI belongs to people rather than to the big labs, where we have personal AI agents working for us and with us, and we build an economy around that. I think it’s going to happen not because of the big labs, but in spite of them.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/human-pilled/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Why AI Agents Are Repeating 2019’s Microservices Mistake</title>
		<link>https://www.oreilly.com/radar/why-ai-agents-are-repeating-2019s-microservices-mistake/</link>
				<comments>https://www.oreilly.com/radar/why-ai-agents-are-repeating-2019s-microservices-mistake/#respond</comments>
				<pubDate>Mon, 09 Nov 2026 11:55:04 +0000</pubDate>
					<dc:creator><![CDATA[Shania Rasheed Nalagath]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20170</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Why-AI-agents-are-repeating-2019s-microservices-mistake-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Why-AI-agents-are-repeating-2019s-microservices-mistake-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[Microservices changed how we build software by breaking large applications into smaller, independently deployable services. That made systems easier to develop and scale, but it introduced a different class of problems: The behavior of the whole system got harder to see. AI agents are now following a similar path. A workflow that once lived inside [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">Microservices changed how we build software by breaking large applications into smaller, independently deployable services. That made systems easier to develop and scale, but it introduced a different class of problems: The behavior of the whole system got harder to see. AI agents are now following a similar path. A workflow that once lived inside a single application gets split across several agents, each responsible for one task. Individual pieces may work exactly as designed while the overall workflow gradually produces worse results. The difference is that agents make these systems far faster and cheaper to create, which makes it easier to repeat the same architectural mistake at a much larger scale.</p>



<p class="wp-block-paragraph">I learned this the hard way. My name was on the design doc. Splitting that workflow across three agents was a reasonable call, and I would still defend most of the reasoning. But if you’d asked me that afternoon how we would find out when the thing quietly stopped working, I had no answer.</p>



<p class="wp-block-paragraph">Months later, in a quality review, a reviewer noticed something felt off before any of our instrumentation caught it. Latency looked fine, error rates were flat, and every service in the chain returned what it was supposed to return. What we had was a slow decline in output quality, and it took about three weeks to trace.</p>



<p class="wp-block-paragraph">The first agent handed a set of documents to the next one for summarization. The summarizer dropped a field it had judged to be a low signal. A downstream agent then made a confident recommendation without that field. Every agent did what it had been designed to do, and the result was still wrong. I should have anticipated it, because I’d been on the product side through the microservices years and had seen this pattern before.</p>



<h2 class="wp-block-heading"><strong>The hangover we already paid for</strong></h2>



<p class="wp-block-paragraph">Breaking down monoliths was a good decision for most teams, but one thing was forgotten. Decomposition did not make the difficult parts of the system disappear. It moved them into the gaps between services, where the people accountable for the overall product could no longer see them. The bill landed around 2019, and it landed on the delivery schedule rather than the incident channel. Teams were running hundreds of services with limited tracing and no contract versioning, and the real symptom was slower delivery. Shipping took longer than before, because engineers spent their weeks picking through those services.</p>



<p class="wp-block-paragraph">Agents are running the same arc faster, because standing one up takes an afternoon and a prompt instead of a repo and a deploy pipeline. When creating an agent is that cheap, product teams make agents without asking anyone, and nobody keeps track of what the agents are and what they’re doing.</p>



<h2 class="wp-block-heading"><strong>What breaks, and what it costs you</strong></h2>



<p class="wp-block-paragraph">This isn’t only my experience. Cemri and coauthors annotated more than 1,600 execution traces across seven popular multi-agent frameworks and built a taxonomy of 14 recurring failure modes (<a href="https://proceedings.neurips.cc/paper_files/paper/2025/hash/b1041e52d3be19f0a9bc491657488e4a-Abstract-Datasets_and_Benchmarks_Track.html" target="_blank" rel="noopener">MAST</a>, NeurIPS 2025). Most of what goes wrong traces back to how the system was specified and how its agents coordinated, not to the models inside it.</p>



<p class="wp-block-paragraph">Inventory sounds trivial and isn’t. Service sprawl at least showed up in your Kubernetes manifests. Agents get created inside product surfaces and workflow tools, and inside somebody’s weekend automation of their own job. You don’t find out what you have until a compliance reviewer asks which automated system touched a given decision, and then it’s too late.</p>



<p class="wp-block-paragraph">The visibility gap cost us three weeks. When an agent calls an agent that calls a retrieval layer that calls another agent, and every hop has a model deciding what to pass forward, you need trace context that survives the whole path. Most stacks log prompts and completions per call and stop there. You get each hop and never the chain, and it was the chain that failed.</p>



<p class="wp-block-paragraph">Compounding confidence is the failure that worries me most. Distributed systems fail loudly, which turned out to be a blessing nobody appreciated. A service throws a 500 and the caller knows to stop. Agents fail quietly and fluently, so when an upstream agent produces a summary that’s plausible and wrong, the downstream agent gets no signal and reasons confidently on top of it. By the time the result reaches the user, it conveys a level of confidence that the initial evidence does not support.</p>



<p class="wp-block-paragraph">API contracts drifted when someone changed the response format without versioning it, and we built machinery to catch that at build time. Agent contracts drift when someone edits a system prompt. In that case there’s no schema diff and no type check. Nothing breaks until a downstream agent starts getting different output and adapts to it without telling anybody.</p>



<h2 class="wp-block-heading"><strong>What carries over from microservices</strong></h2>



<p class="wp-block-paragraph">A registry is the cheapest thing to build and the first thing everyone skips. When building microservices, we kept a service catalog with an owner and a scope for each service. For agents, we need that same record plus an explicit list of what each agent is allowed to call.</p>



<p class="wp-block-paragraph">Tracing carries over nearly unchanged, since correlation IDs were invented for this exact problem. OpenTelemetry’s <a href="https://github.com/open-telemetry/semantic-conventions-genai" target="_blank" rel="noopener">GenAI semantic conventions</a> give you the vocabulary, and an agent run comes out as a span tree instead of a pile of unrelated log lines:</p>


<div class="wp-block-image">
<figure data-wp-context="{&quot;imageId&quot;:&quot;6afc871e4697b&quot;}" data-wp-interactive="core/image" data-wp-key="6afc871e4697b" class="aligncenter size-full wp-lightbox-container"><img loading="lazy" decoding="async" width="936" height="280" data-wp-class--hide="state.isContentHidden" data-wp-class--show="state.isContentVisible" data-wp-init="callbacks.setButtonStyles" data-wp-on--click="actions.showLightbox" data-wp-on--load="callbacks.setButtonStyles" data-wp-on--pointerdown="actions.preloadImage" data-wp-on--pointerenter="actions.preloadImageWithDelay" data-wp-on--pointerleave="actions.cancelPreload" data-wp-on-window--resize="callbacks.setButtonStyles" src="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-2.png" alt="invoke_agent" class="wp-image-20171" style="aspect-ratio:3.336898395721925" srcset="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-2.png 936w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-2-300x90.png 300w, https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/image-2-766x229.png 766w" sizes="auto, (max-width: 936px) 100vw, 936px" /><button
			class="lightbox-trigger"
			type="button"
			aria-haspopup="dialog"
			data-wp-bind--aria-label="state.thisImage.triggerButtonAriaLabel"
			data-wp-init="callbacks.initTriggerButton"
			data-wp-on--click="actions.showLightbox"
			data-wp-style--right="state.thisImage.buttonRight"
			data-wp-style--top="state.thisImage.buttonTop"
		>
			<svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="none" viewBox="0 0 12 12">
				<path fill="#fff" d="M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z" />
			</svg>
		</button></figure>
</div>


<p class="wp-block-paragraph">Had we been emitting those spans, the summarizer span and the recommender span would have sat side by side in one trace, and the missing field would have been a diff instead of a three-week hunt. These conventions are still moving, so keep the attribute names behind a thin mapping layer of your own.</p>



<p class="wp-block-paragraph">Finding the missing field in a trace would have shortened the investigation. I’d also want a test that caught its omission before a prompt change reached production. That’s where I’d apply the contract-testing discipline from microservices: define what the downstream agent depends on, then check that an upstream change preserves it. For the summarizer in this workflow, the evaluation would need to check whether its output retained the information the recommender needed. I’d version the prompts and require changes to pass those evaluations before release. There’s early evidence it pays: <a href="https://arxiv.org/abs/2510.12120" target="_blank" rel="noopener">SEMAP</a>, a protocol-layer method applying behavioral contract modeling and lifecycle verification on Google’s A2A infrastructure, reported up to roughly 70 percent fewer failures on function-level coding tasks when scored against MAST.</p>



<p class="wp-block-paragraph">The trust boundary is the thing I’ll argue about in a design review. In 2019 we learned that a service doesn’t get trusted just because it lives inside the perimeter, which is most of what service meshes were for. The agent version is that an agent doesn’t get broad tool access just because we built it and we’re nice people. What an agent is allowed to do belongs in a policy layer the model can’t talk its way past, not in a system prompt asking politely.</p>



<p class="wp-block-paragraph">Cost is where the analogy between microservices and agents fails. A microservice stuck in a retry loop costs you CPU and an alert at 3 am. An agent stuck in a loop costs real money on every turn and doesn’t saturate in any way that makes itself obvious. This one lands on the product owner instead of infrastructure, because it surfaces as unit economics rather than an outage, which means the PM ends up justifying the number at the end of a billing cycle.</p>



<h2 class="wp-block-heading"><strong>What I ask before approving a split</strong></h2>



<p class="wp-block-paragraph">None of this argues against multi-agent systems, which earn their keep the same way services did. My argument is that operational practices should evolve alongside architecture, without repeating the five-year lag of the previous transition. The person accountable for delivering the product is usually best placed to ensure that happens.</p>



<p class="wp-block-paragraph">Hence, I push on four things. Can we follow one request all the way through and reconstruct what each agent decided and why? If not, we’re building something we won’t be able to debug. What happens when an upstream agent is wrong, and not whether it can be? If the honest answer is that the next agent carries on as normal, we haven’t designed a failure mode. We’ve designed a failure and left it running.</p>



<p class="wp-block-paragraph">Who owns this agent, and where is its tool access enforced? A system prompt isn’t enforcement. What does this cost at ten times today’s volume, and what shuts it off when the number goes wrong? I ask that one early, because nobody volunteers it.</p>



<p class="wp-block-paragraph">The teams that managed service decomposition most effectively incorporated tracing and contract testing into the build from the outset. These practices helped them understand how services interacted and verify that changes preserved expected behavior. That work is a sprint or two now, and a great deal more than that once you’re running 40 agents and someone asks which of them touched a customer’s data.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/why-ai-agents-are-repeating-2019s-microservices-mistake/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>This Week in AI: The Cost of Progress</title>
		<link>https://www.oreilly.com/radar/this-week-in-ai-the-cost-of-progress/</link>
				<comments>https://www.oreilly.com/radar/this-week-in-ai-the-cost-of-progress/#respond</comments>
				<pubDate>Fri, 06 Nov 2026 16:55:06 +0000</pubDate>
					<dc:creator><![CDATA[Christina Stathopoulos and Michelle Smith]]></dc:creator>
						<category><![CDATA[This Week in AI]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20167</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/08/USE_THIS_thisweek-ai-radar-1600x840-1.png" 
				medium="image" 
				type="image/png" 
				width="1600" 
				height="840" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/08/USE_THIS_thisweek-ai-radar-1600x840-1-160x160.png" 
				width="160" 
				height="160" 
			/>
		
				<custom:subtitle><![CDATA[Frontier models, investor caution, new approaches to learning, and the challenges of responsible AI]]></custom:subtitle>
		
				<description><![CDATA[New frontier models arrived this week as investors showed signs of growing cautious about AI spending. On the latest episode of This Week in AI, we examined those developments, along with research into how models learn and the less visible costs of AI adoption. From unequal pricing across languages to the environmental impact of data [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">New frontier models arrived this week as investors showed signs of growing cautious about AI spending. On the latest episode of <em>This Week in AI</em>, we examined those developments, along with research into how models learn and the less visible costs of AI adoption. From unequal pricing across languages to the environmental impact of data centers, the discussion raised questions about who benefits from AI progress and who pays for it.</p>



<iframe loading="lazy" width="800" height="450" src="https://www.youtube.com/embed/Lqxqetpg278?si=cVcjpm_62qTpTExN" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



<h2 class="wp-block-heading"><strong>Frontier models advance as investors question the spending</strong></h2>



<p class="wp-block-paragraph">We started with three model announcements that show how quickly AI capabilities are expanding. Google introduced <a href="https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/" target="_blank" rel="noopener">Gemini 4 Carbon</a>, designed for cybersecurity and complex, multistep tasks. France’s Mistral <a href="https://mistral.ai/news/mistral-large-4-oss/" target="_blank" rel="noopener">published the weights for its trillion-parameter Large 4 model</a>, also known as “Le Chonk,” giving developers more control over deployment and customization, provided they have the hardware to run it. Meanwhile, Google DeepMind <a href="https://deepmind.google/blog/genie-4-a-new-generation-of-world-simulation/" target="_blank" rel="noopener">previewed Genie 4</a>, a world model that simulates environments, physical interactions, and sequences of events. Unlike language models that predict text, world models attempt to predict what happens when objects and people interact. With Genie 4, researchers could train AI systems in simulated worlds, an approach that would be especially useful in robotics, where teams could test AI designed to control robots before trying it on physical machines.</p>



<p class="wp-block-paragraph">Investor enthusiasm for AI appeared to weaken even as model development accelerated. Anthropic moved closer to a public offering by <a href="https://www.anthropic.com/news/s1-sec" target="_blank" rel="noopener">filing its IPO paperwork publicly</a>, although it hasn’t announced a date or share price. OpenAI has reportedly <a href="https://www.wsj.com/tech/ai/openai-raises-less-than-half-of-30-billion-target-in-latest-round-91aecdc5" target="_blank" rel="noopener">secured about $11 billion in funding</a> for its latest round, well below its initial $30 billion target. Those developments coincided with <a href="https://www.bbc.com/news/articles/cvwy72e0n72yo" target="_blank" rel="noopener">sharp declines in several major technology stocks</a>, and the tech-heavy Nasdaq ended October with its worst monthly decline since March 2025.</p>



<p class="wp-block-paragraph">The scale of AI spending is attracting greater scrutiny, particularly as investors wait for returns. Companies investing in AI should pay attention to more than model performance. The economics of training and running increasingly expensive models could affect pricing, availability, and which providers remain competitive.</p>



<h2 class="wp-block-heading"><strong>Researchers are rethinking how AI models learn</strong></h2>



<p class="wp-block-paragraph">Researchers are investigating how AI models process information, retain knowledge, and generate responses. AI interpretability company Goodfire <a href="https://www.goodfire.com/blog/accelerating-interpretability" target="_blank" rel="noopener">proposed a project modeled on the Human Genome Project</a>, committing $10 million to begin mapping how large language models work internally. By studying which patterns activate in response to prompts and how changes to those patterns affect behavior, researchers hope to better understand why models produce particular outputs.</p>



<p class="wp-block-paragraph">Google Research <a href="https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/" target="_blank" rel="noopener">proposed a framework inspired by human cognition</a> that introduces a “sleep” phase. During this offline period, models would consolidate new information and generate practice examples through a process researchers call “dreaming.” The goal is to address catastrophic forgetting, when learning new information degrades previously acquired capabilities, while helping models retain knowledge beyond individual conversations. The approach remains experimental, but it offers a possible way for models to continue learning without sacrificing existing skills.</p>



<p class="wp-block-paragraph">We also examined <a href="https://diffuspace.ai/blog/clover-35b/" target="_blank" rel="noopener">Clover, a 35-billion-parameter diffusion language model</a> released by China’s DiffuSpace. Most familiar language models use what’s called an autoregressive approach: They generate text sequentially, one token at a time. Diffusion language models instead work on multiple parts of an answer simultaneously, refining them over successive passes. DiffuSpace reports competitive performance and faster generation on some tasks when compared to autoregressive models of similar size. Alongside Goodfire’s and Google’s research, this work suggests that improvements in AI may increasingly come from rethinking how models learn and generate outputs rather than simply making them larger.</p>



<h2 class="wp-block-heading"><strong>AI’s hidden costs extend beyond infrastructure spending</strong></h2>



<p class="wp-block-paragraph">Research into the “<a href="https://clear-languages.org/blog/the-tokenizer-tax" target="_blank" rel="noopener">tokenizer tax</a>” revealed how AI pricing can disadvantage people who speak less widely represented languages. Researchers found that <a href="https://restofworld.org/2026/ai-tokenizer-tax-non-english-languages-cost-inequity/" target="_blank" rel="noopener">some languages require substantially more tokens than English</a> to express the same message. In one example, the Malayalam translation contained slightly fewer characters than the original sentence in English, yet it required twice as many tokens: 26 compared with 13. Because AI providers commonly charge by token, these differences can translate into higher costs for users, particularly in communities where AI services may already be less affordable. Developers building multilingual applications need to consider tokenization efficiency alongside model quality.</p>



<p class="wp-block-paragraph">In Finland, authorities are <a href="https://www.theguardian.com/world/2026/oct/29/finland-criminal-investigation-google-clearing-forest-datacentre" target="_blank" rel="noopener">investigating Google’s clearing of more than 400,000 trees</a> for planned data centers. The investigation centers on whether the company completed required environmental assessments before clearing the sites. The case shows why environmental reviews need to happen before land is cleared or construction begins. Once that happens, regulatory action can’t immediately restore the affected ecosystem.</p>



<p class="wp-block-paragraph">Finally, several AI companies said they plan to honor a <a href="https://openai.com/index/helping-websites-express-ai-training-preferences/" target="_blank" rel="noopener">shared website opt-out signal</a> that would let publishers indicate that their content shouldn’t be used for AI model training. The proposal could give website owners more control over future data collection, although <a href="https://www.theverge.com/tech/1023763/openai-google-microsoft-ai-scraping-standard-opt-out" target="_blank" rel="noopener">participation remains voluntary</a>. It can’t reverse the use of material already collected for training, and the standard will only help if AI companies follow the instructions publishers provide.</p>



<h2 class="wp-block-heading"><strong>What’s next</strong></h2>



<p class="wp-block-paragraph">This week’s developments show why evaluating AI progress requires more than tracking model releases and benchmark scores. We need to understand how systems learn, whether their economics are sustainable, and how their costs affect people and communities. As more businesses and individuals adopt AI, those considerations will become harder to ignore.</p>



<p class="wp-block-paragraph">We’ll be back on Monday, November 9, for another episode covering the latest developments in AI research and applications. You can catch previous episodes on <a href="https://www.youtube.com/watch?v=g4cfjz5AKxY&amp;list=PL055Epbe6d5bJEhT7_ZzOeJZ6gPyUzYpS" target="_blank" rel="noopener">YouTube</a>, <a href="https://open.spotify.com/show/033kJS2BG1teGunxmtsU1r" target="_blank" rel="noopener">Spotify</a>, <a href="https://podcasts.apple.com/us/podcast/this-week-in-ai/id1896798047" target="_blank" rel="noopener">Apple</a>, or wherever you get your podcasts.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/this-week-in-ai-the-cost-of-progress/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>Choose Your Coding Harness by What You Need to Control—Not by Who Built It</title>
		<link>https://www.oreilly.com/radar/choose-your-coding-harness-by-what-you-need-to-control-not-by-who-built-it/</link>
				<comments>https://www.oreilly.com/radar/choose-your-coding-harness-by-what-you-need-to-control-not-by-who-built-it/#respond</comments>
				<pubDate>Fri, 06 Nov 2026 11:54:19 +0000</pubDate>
					<dc:creator><![CDATA[Nicole Koenigstein]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20164</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Choose-your-coding-harness-by-what-you-need-to-control-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/Choose-your-coding-harness-by-what-you-need-to-control-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
		
				<description><![CDATA[Remember back in 2019, when you had only a handful of language models to choose from, all with limited context windows? That world is gone, and it feels like a forgotten past. Today, flagship providers and open source projects compete across model families, sizes, capabilities, and deployment options. Coding harnesses seem to be heading in [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">Remember back in 2019, when you had only a handful of language models to choose from, all with limited context windows? That world is gone, and it feels like a forgotten past. Today, flagship providers and open source projects compete across model families, sizes, capabilities, and deployment options. Coding harnesses seem to be heading in the same direction.</p>



<p class="wp-block-paragraph">On the surface, Claude Code, Codex, Pi, DeepSeek Harness, and the other coding harnesses all seem to do the same thing: help you build software with models. Look closer, though, and each one gives you control over different parts of the harness. What can the model see and change? How does it use tools? Where does it run, and what happens when it fails? So instead of asking which coding harness is best overall, ask:</p>



<p class="wp-block-paragraph"><strong>Which layers of the harness do you need to control?</strong></p>



<h2 class="wp-block-heading"><strong>Thanks for all the harnesses! Now what?</strong></h2>



<p class="wp-block-paragraph">Choosing a coding harness isn’t only about which model writes the best code. You’re also choosing what the harness provides, what you can change, and what you must build yourself. If it doesn’t give you verification, durable state, repair routing, or an enforced execution boundary, those responsibilities stay with you. You remain accountable for the workflow’s reliability and governance.</p>



<p class="wp-block-paragraph">In general, each harness has a center of gravity: a layer that gives you the most direct control. The following table is a deliberately high-level overview of where that center sits. It’s not a feature matrix, and the boundaries overlap in practice. Claude Code can also run noninteractively and inside sandboxes, Codex also reads repository instructions, and all of them are changing quickly. The point is to show where each harness gives you leverage by default, and where the engineering work is likely to stay with you.</p>



<h4 class="wp-block-heading">Table 1. Where each coding harness puts its primary control</h4>



<figure class="wp-block-table"><table class="has-fixed-layout"><thead><tr><th><strong>Harness</strong></th><th><strong>Primary control</strong></th><th><strong>You shape</strong></th><th><strong>You likely build</strong></th></tr></thead><tbody><tr><td><strong>Claude Code</strong></td><td>Inside the run</td><td>Instructions, skills, subagents, hooks</td><td>Outer lifecycle policy</td></tr><tr><td><strong>Codex</strong></td><td>Around execution</td><td>Sandbox, approvals, scope, output</td><td>Cross-run orchestration</td></tr><tr><td><strong>Pi</strong></td><td>The loop itself</td><td>Tools, context, state, hand-offs</td><td>Most defaults</td></tr><tr><td><strong>DeepSeek</strong></td><td>Across plug-ins</td><td>Composable runtime components</td><td>Gaps in your plug-in set</td></tr></tbody></table></figure>



<p class="wp-block-paragraph">Read the table as a map, not a ranking. A harness that hands you more control also hands you more to build, and a harness that does more out of the box decides more on your behalf. Which trade-off is right depends on which layer your project actually needs to own.</p>



<h2 class="wp-block-heading"><strong>Claude Code: Control inside the run</strong></h2>



<p class="wp-block-paragraph">Claude Code buys you detailed control over how implementation unfolds. CLAUDE.md provides standing instructions, skills add task-specific playbooks, subagents divide the work, and hooks intercept actions before and after tool calls. Unlike a permission rule that simply refuses an operation, a hook can explain the refusal or immediately return a compiler or policy failure to the agent.</p>



<p class="wp-block-paragraph">These controls let you steer the coding loop extensively, but they operate within Claude Code’s existing lifecycle. If you need to control the outer lifecycle itself, you need another layer.</p>



<h2 class="wp-block-heading"><strong>Codex: Control through separation</strong></h2>



<p class="wp-block-paragraph">Codex buys you separation between a coding task and its execution environment. A noninteractive codex exec run can work on a narrow task in a specified repository, under an OS-enforced sandbox, and return a schema-constrained result.</p>



<p class="wp-block-paragraph">This is useful for independent verification. You can give Codex a clean checkout, exclude the builder’s instructions, restrict writable paths, and run separate reviews for code quality, test gaps, persistence, or user-visible behavior. The sandbox depends on the operating system, but it moves scope enforcement out of the prompt and into the execution boundary.</p>



<h2 class="wp-block-heading"><strong>Pi: Control over the loop</strong></h2>



<p class="wp-block-paragraph">Pi buys you control over the coding loop itself. Its tools, commands, event handlers, context, models, and session state can be extended or replaced. This lets you decide which steps remain agentic and which follow deterministic policy.</p>



<p class="wp-block-paragraph">For example, a local defect can return to implementation, a repeated defect can force replanning, and conflicting findings can trigger a recheck or human review. Pi can also run agents inside this lifecycle, so it’s more than a deterministic orchestrator. For my example below, I used its programmability to make the outer decisions deterministic—that was my design choice, not Pi’s definition.</p>



<h2 class="wp-block-heading"><strong>DeepSeek’s harness: Control through composition</strong></h2>



<p class="wp-block-paragraph">DeepSeek’s harness is useful when you want to swap out whole parts of the harness rather than steer or wrap them. Tools, sessions, lifecycle behavior, verification, and governance are all composable plug-ins, and so is the loop itself. That lets you replace one component without rebuilding the rest.</p>



<p class="wp-block-paragraph">The trade-off is that your control is only as good as the plug-ins you compose. Where Pi gives you a minimal core to build on, DeepSeek’s harness gives you a set of parts to assemble.</p>



<h2 class="wp-block-heading"><strong>A practical example</strong></h2>



<p class="wp-block-paragraph">Let me give you the problem I was actually trying to solve: migrating a legacy Python 2.7 application reliably, without changing years of accumulated behavior or corrupting its persisted data. Getting the code to run on Python 3 was only one part of it. I also needed to preserve behavior, verify the data across both runtimes, control what the coding agents could change, and decide what should happen when a test or verification step failed.</p>



<p class="wp-block-paragraph">So I approached the choice of coding harnesses from a very different perspective. Instead of asking which one was best, I asked what each one would buy me in terms of control, isolation, and lifecycle management. The answer led me to use more than one:</p>



<ul class="wp-block-list">
<li><strong>Claude Code</strong> built the port because I wanted internal steering, specialized skills, hooks, and a factory of implementation and review subagents.</li>



<li><strong>Codex</strong> verified it in isolated runs because I wanted an independent model context, clean workspaces, constrained writes, and structured findings.</li>



<li><strong>Pi</strong> owned the outer lifecycle because I wanted durable state, evidence validation, bounded repairs, replanning rules, and human escalation.</li>



<li><strong>Frozen acceptance</strong>, including Playwright and Python 2/3 differential checks, ran outside the agents so neither the builder nor the verifier could redefine success.</li>
</ul>



<p class="wp-block-paragraph">I didn’t use DeepSeek’s Harness here because I wanted a very small outer lifecycle I could read end to end, not because it can’t do the job. To first check whether this setup would actually make a difference, I built a small Python 2 application with the same kinds of traps as the real migration. I then compared the composed setup with a single Claude Code session, in which Claude could verify and test its own code.</p>



<p class="wp-block-paragraph">Interestingly, both setups passed all 25 hidden behavior tests. Yet the single-session setup silently collapsed two distinct Python 2 dictionary keys, the Unicode u&#8217;é&#8217; and byte string &#8216;é&#8217;, into one Python 3 string. After the data was saved, one entry was permanently lost. The single-session setup also changed a Python 2 Unicode value into bytes without documenting it.</p>



<p class="wp-block-paragraph">The test suite missed it. Claude’s own tests missed it. Hundreds of recorded behavior comparisons missed it. Even my hand-written reference port had the same hole.</p>



<p class="wp-block-paragraph">But what happened next was even more interesting. Two isolated Codex verifiers independently reproduced the defect. Claude didn’t simply accept the finding; it contested it. Pi then launched a fresh Codex run that received only the finding and Claude’s argument. The independent recheck upheld the finding, and Claude accepted it. Pi routed the defect into a bounded repair, and the next verification round confirmed that both keys and their original types now survived.</p>



<p class="wp-block-paragraph">It’s also ironic because models are often criticized for being sycophantic. They apologize and rewrite working code the moment you ask, <em>“Are you sure?</em>” Here, the opposite happened. Claude contested a defect that two independent verifiers had reproduced. Its session context contained its own implementation history and passing tests, all of which supported the patch. A fresh verifier context broke that self-reinforcing frame. Independent verification matters not because the second model is automatically right, but because it’s not anchored to the same history.</p>



<p class="wp-block-paragraph">The important point is that no individual harness deserves all the credit. The better result came from combining the controls I needed: internal steering, independent execution, frozen behavioral evidence, and deterministic lifecycle policy.</p>



<h2 class="wp-block-heading"><strong>How to choose your coding harness</strong></h2>



<p class="wp-block-paragraph">When choosing a coding harness, start with the level of control you need and how much of the remaining harness you are prepared to engineer yourself. A useful comparison could begin with questions like these:</p>



<ul class="wp-block-list">
<li>Do I want to configure an existing coding loop, extend it, or replace parts of it?</li>



<li>Can I control tools, context, memory, and model selection at the level my project requires?</li>



<li>Where are permissions, allow lists, and policies enforced: in instructions, application logic, or the operating system?</li>



<li>Can I intercept actions and lifecycle events before and after they happen?</li>



<li>How much of the orchestration, state management, and session lifecycle can I change or orchestrate?</li>



<li>What does the harness expose for observability, verification, and structured evidence?</li>



<li>Which reliability and governance controls does the harness provide, and which ones remain mine to build?</li>



<li>Can the harness work with other tools and harnesses if my requirements cross several control layers?</li>
</ul>



<p class="wp-block-paragraph">These are some examples to get you started, not an exhaustive checklist. The questions that matter for you will also depend on what you’re building, how much autonomy you’re willing to give the coding agent, and what you remain accountable for when it fails.</p>



<p class="wp-block-paragraph">Your answers may lead you to Claude Code, Codex, Pi, DeepSeek’s harness, or another coding harness entirely. In the end, the developer’s name only tells you who packaged the runtime, not where your guarantees live. Whether you settle on one harness or orchestrate several, the principle is the same: Choose the tool by the boundary of control it gives you, and ask whether you want the agent to define the terms of its own success.</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/choose-your-coding-harness-by-what-you-need-to-control-not-by-who-built-it/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
		<item>
		<title>The New Knowledge Artifacts of the AI Economy</title>
		<link>https://www.oreilly.com/radar/the-new-knowledge-artifacts-of-the-ai-economy/</link>
				<comments>https://www.oreilly.com/radar/the-new-knowledge-artifacts-of-the-ai-economy/#respond</comments>
				<pubDate>Thu, 05 Nov 2026 18:18:39 +0000</pubDate>
					<dc:creator><![CDATA[Tim O’Reilly]]></dc:creator>
						<category><![CDATA[AI & ML]]></category>
		<category><![CDATA[Commentary]]></category>

		<guid isPermaLink="false">https://www.oreilly.com/radar/?p=20146</guid>

		
					<media:content 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/The-new-knowledge-artifacts-of-the-AI-economy-image-created-with-Gemini-scaled.jpeg" 
				medium="image" 
				type="image/jpeg" 
				width="2560" 
				height="1429" 
			/>

			<media:thumbnail 
				url="https://www.oreilly.com/radar/wp-content/uploads/sites/3/2026/11/The-new-knowledge-artifacts-of-the-AI-economy-image-created-with-Gemini-160x160.jpeg" 
				width="160" 
				height="160" 
			/>
		
				<custom:subtitle><![CDATA[Every technology revolution creates new knowledge artifacts that are valuable and worth sharing.]]></custom:subtitle>
		
				<description><![CDATA[At our recent Foo Camp, a number of people said something like, “I don’t want someone else’s code any more. I may have my agent take a look at it, but I want to extract just the parts that I want, and may rewrite even those in Rust. It’s just too easy to create just [&#8230;]]]></description>
								<content:encoded><![CDATA[
<p class="wp-block-paragraph">At our recent <a href="https://en.wikipedia.org/wiki/Foo_Camp" target="_blank" rel="noopener">Foo Camp</a>, a number of people said something like, “I don’t want someone else’s code any more. I may have my agent take a look at it, but I want to extract just the parts that I want, and may rewrite even those in Rust. It’s just too easy to create just the code that I want.”</p>



<p class="wp-block-paragraph">This topic was hot enough that <a href="https://blog.fsck.com/" target="_blank" rel="noopener">Jesse Vincent</a> held an entire session about it. I did not attend, but Jesse gave me a report from his Plaude AI notetaker. Here’s a relevant section of the summary:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph">Several participants argue that software development’s primary artifact may be shifting from executable code toward natural-language prompts, generation processes, and developer skills. One speaker compares Python to assembly language, suggesting that writing conventional code may become optional as natural-language generation takes precedence. Another argues that open source can survive such a transition because its core values—peer production, capacity building, and shared specifications—do not depend on who or what writes the code.</p>



<p class="wp-block-paragraph">At the same time, Jesse warns that maintainers are overwhelmed by floods of uninvited, automated, and often low-quality contributions. Participants say that traditional milestones such as pull requests, repositories, and conferences have helped contributors connect and gain recognition; as AI makes code replication easier, communities may need new ways to acknowledge participation and create paths for future developers. Some community interaction has moved to spaces such as Discord and direct messages, but structured opportunities are still needed.</p>



<p class="wp-block-paragraph">Scientific computing raises particular concerns. One panelist warns that AI-generated forks and agent attacks could threaten carefully verified foundations such as NumPy and SciPy. Scientific systems depend on battle-tested libraries and reproducibility, so thousands of unreviewed alternatives could undermine reliability. Participants suggest bridging traditional open-source communities and modern AI development, which currently may use different tools and artifacts.</p>



<p class="wp-block-paragraph">Other proposals include treating code as “liquid” and focusing open-source governance on durable infrastructure and specifications; maintaining agent storage independently of ephemeral container lifecycles; and using corporate compliance and safety-monitoring architectures to aggregate organizational traces.</p>
</blockquote>



<p class="wp-block-paragraph">These are important observations, all pointing in the same direction: AI is transforming the mechanisms and the artifacts of software development. For an increasing number of use cases, the software source code itself may no longer be the most important shareable knowledge artifact. This isn’t a new phenomenon. Every major change in technology calls for new mechanisms, new artifacts, and new institutions.</p>



<p class="wp-block-paragraph">Originally, software developers mostly distributed binaries, then open source developers made sharing source code the norm. Now we need to distribute and share something else: context and patterns for generating code on the fly. Knowledge is being unbundled and can be shared in new forms. A repo may now serve merely as a source of context for solving the exact problem I have, rather than the bundle of problems that the original developer(s) encoded into their library or packaged solution. Some people will keep their context and patterns private, either because they see proprietary advantage in keeping them to themselves or because they are too busy or too lazy to share. But part of the problem is that we haven’t yet figured out what needs to be shared!</p>



<p class="wp-block-paragraph"><em>The challenge for the software development community is to identify the new knowledge artifacts of the AI economy, and to build new mechanisms and incentives for sharing them.</em></p>



<p class="wp-block-paragraph">We’ve all had the experience of seeing a bespoke agentically generated app, wanting something similar, and wishing we could ask for a starting point so we didn’t have to vibe code an equivalent from scratch, with all the attention, false starts, and token costs that comes with that. Even in a world where software is generated by agents, it’s still not costless, and most of us like a good place to start.</p>



<p class="wp-block-paragraph">Of course, this won’t be all software. There are lots of areas where “sort of alike” software is a recipe for disaster. Imagine a sales and marketing group where everyone had their own vibe-coded CRM tools, with different metrics and capabilities. This is not good. So there is still a place for big SaaS applications, distributed binary software, and distributed source code. You pick the hat to fit the head. But there’s also a need for something new.</p>



<p class="wp-block-paragraph"><em>The purpose of this article is to start a conversation about exactly what it is that developers should be preserving, sharing, and building on from others, and what mechanisms are still missing to make that virtuous circle work more effectively. I can’t see into the future; I can only speculate. So, as is often the case, I take a historical view. The poet W. S. Merwin wrote, “<a href="https://www.poetryfoundation.org/poems/43122/the-river-of-bees" target="_blank" rel="noopener">We are the echo of the future.</a>” I have found that using history to frame the present can be a productive way to get our minds going in the right direction.</em></p>



<p class="wp-block-paragraph"><em>Note that while this article focuses on software development, the questions I’m asking here apply to virtually every field. Let me know if you have insights from your work in any area touched by AI.</em></p>



<h2 class="wp-block-heading">From code to context</h2>



<p class="wp-block-paragraph">The ease with which LLMs can generate code is paradoxically getting people to focus too much on how easy it is not to produce code. That may be blinding us to the fact that code itself is being displaced as a valuable, tradeable, shareable knowledge artifact. Maybe “displaced” is too strong a word. Different species of value overlap for a long time. Eventually, though, old ones fade and new ones become dominant. This has happened before. Every technology revolution draws not only on new mechanisms but also creates new valuable knowledge artifacts. I talked about this in my recent short keynote at AGNTCon + MCPCon.</p>



<iframe loading="lazy" width="800" height="450" src="https://www.youtube.com/embed/sMjX5X_JPts?si=ZfcFU2w46dedB_FV" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>



<p class="wp-block-paragraph">The open source revolution depended in part on mechanisms like modular design and source code control, even more than it depended on free software licenses. It birthed new coordinating institutions like GitHub. But it also created a valuable new class of knowledge artifact in the form of the source code repo, and subsidiary valuable artifacts like commit history and reputation. A developer’s public repos became a kind of résumé, and a project’s commit log became a record of who had done what, and how well. This replaced a previous world where binaries were the tradeable objects of the software economy. Microsoft’s fortune was built on selling licenses to binaries, and to anyone who measured the software economy in licenses sold, free software looked like value being destroyed. (Steve Ballmer called GPL’d free software “a cancer” and Microsoft VP Jim Allchin called it “<a href="https://web.archive.org/web/20030419103034/http://www.oreillynet.com/pub/wlg/122" target="_blank" rel="noopener">an intellectual property destroyer</a>.”)</p>



<p class="wp-block-paragraph">There were two ways that I like to think about the effects of open source software. First, value that was based on scarcity was replaced by value that was created by abundance. Back in 1975, Steve Baer described this as “<a href="https://www.oreilly.com/radar/the-clothesline-paradox-and-the-sharing-economy/" target="_blank" rel="noopener">the clothesline paradox</a>,” explaining that when you put your clothes in the dryer, the energy you use is measured and counted, but when you hang your clothes on the line, it is ignored by accounting, despite doing the same job. Open source software was like sunshine, free for the taking, and its value unmeasured. Everyone compared the revenue of free software companies like Red Hat and MySQL to the vastly larger revenues of proprietary software companies like Microsoft and Oracle, noted the fact that they hadn’t even survived as independent companies, and concluded that open source was a less effective business model because its creators <em>captured</em> less of the value. They missed the point that far more value had been <em>created</em> than had been captured. Open source went on to power most of the internet economy, with the value captured by other means than selling the software itself.</p>



<p class="wp-block-paragraph">Some people might think of this as a kind of failure, but they would be wrong. It is the system working exactly as its creators intended. I remember talking with Bob Scheifler, one of the creators of the X Window System, about whether it was OK that we were building commercial versions of their documentation. He told me that was exactly what he had intended. They put things out with a very permissive license precisely because they wanted others to build on top of them. I remember him saying “We want to raise the floor.” The creators of Berkeley Unix, Apache, and others felt exactly the same way. They didn’t want to win the existing game. They wanted to play a different game.</p>



<p class="wp-block-paragraph">And they succeeded, big time. As Clayton Christensen noted in 2004, when one thing becomes a commodity, modular and fungible, something else (usually something adjacent) becomes valuable. He called this “<a href="https://www.oreilly.com/pub/a/tim/articles/paradigmshift_0504.html" target="_blank" rel="noopener">The Law of Conservation of Attractive Profits</a>.” At about the same time, I had stumbled onto this principle in explaining the implications of open source software in giving rise to the new paradigm that I was shortly to start calling Web 2.0. Clay and I first bonded when we gave our respective keynotes about essentially the same thing at the 2004 Open Source Business Conference. I later wrote about the conjunction of our respective ideas in a piece called “<a href="https://www.oreilly.com/pub/a/tim/articles/paradigmshift_0504.html" target="_blank" rel="noopener">The Open Source Paradigm Shift</a>.”</p>



<p class="wp-block-paragraph">New artifacts are often the first visible sign of this value transition. You can see this at work in the evolution of the web. The web was born from the new mechanism of internet-enabled hypertext via HTTP, HTML, and the modular, replaceable software (the web browser and the web server) that made it useful.</p>



<p class="wp-block-paragraph">At first, the new valuable knowledge artifact was the website with its domain name, associated URLs, and HTML-formatted web pages. DNS was the coordinating institution. People paid fortunes for domain names on the premise that the site was the thing that mattered. And it did matter, and still does. But less than it used to. Over time, Google became the new coordinating institution, and its ability to point deep within websites made the web page, not just the site itself, a valuable knowledge artifact. Social media made “posts” and short-form video into other knowledge artifacts. New kinds of value creation and value capture tumbled out of that unbundling. Packaged software was displaced as the key monetizable artifact and replaced by web applications, which were information- and software-based frontends to vast backend databases.</p>



<p class="wp-block-paragraph">The click became a new and incredibly valuable knowledge artifact, the unit in which attention and intent were measured, and then priced. First pay per click advertising (Google), then engagement maxing (social media) became the basis of vast fortunes. The people who notice the new artifacts early, while everyone else is still focused on the old ones, then invent new business models based on them, often play a bigger game than those who were still trying to play by the new rules.</p>



<p class="wp-block-paragraph">When I summed up these ideas in “<a href="https://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html" target="_blank" rel="noopener">What Is Web 2.0</a>” in 2005, I talked about the key principles being “harnessing collective intelligence,” “software above the level of a single device,” and “data as the next Intel Inside.” (That is, the next valuable source of lock-in.) The companies that survived the dot-com bust were the ones that understood that the data their users generated through their activity on the internet was the asset that mattered, not just the software that served it. That was an argument about a new knowledge artifact, though I didn’t use those words at the time. PageRank understood that a link was a vote, and a new mechanism to improve organic search birthed new dimensions of the internet economy. Links were bought, sold, and gamed accordingly. SEO (both “white hat” and “black hat”) and search quality became a battlefield.</p>



<p class="wp-block-paragraph">All those disciplines have had to change now that content has been absorbed into AI models. Books, music, video, web content, sites like Wikipedia, GitHub, and Stack Overflow were all grist for the mill. In the open source era, a voted-on answer to a well-posed question on Stack Overflow was a valuable artifact, and the reputation that came with writing good ones was a subsidiary artifact, just as it was on GitHub. The number of new questions asked on Stack Overflow peaked <a href="https://tech.slashdot.org/story/26/01/05/1431212/" target="_blank" rel="noopener">at roughly 200,000 a month around 2014</a> and was already sliding before ChatGPT, but the decline <a href="https://newsletter.pragmaticengineer.com/p/are-llms-making-stackoverflow-irrelevant" target="_blank" rel="noopener">accelerated sharply after November 2022</a>. The knowledge in those answers didn’t go away. It was absorbed into the models, which now give their own answers synthesized from all that they have consumed. In that way, they have also consumed much of the traffic that would have gone to the original knowledge creators. New mechanisms to create a <a href="https://www.oreilly.com/radar/ai-and-the-next-economy/" target="_blank" rel="noopener">circulatory economy of value creation</a> need to be developed.</p>



<p class="wp-block-paragraph">Software development also changed radically because of the internet. Another of my web 2.0 principles was what I called “the perpetual beta.” It’s easy to forget that in the PC era, software had major releases only every couple of years, with the new version distributed on CDs (and long before that, on 9-track magnetic tape.) I remember talking with Mark Lucovsky, who’d been a developer on Windows NT at Microsoft, shortly after he arrived at Google in 2005. He was exulting at the experience of being able to push a single change live worldwide soon after it had been tested. Eventually, we saw the formalization of new mechanisms like test-driven development (TDD) and continuous integration and continuous delivery/deployment (CI/CD), which we now take for granted, as if they’d always been the norm.</p>



<p class="wp-block-paragraph">The story isn’t over yet, though. The old knowledge artifacts don’t go away all at once, and they don’t go down without fighting. The new artifacts are only half-formed, and have weaknesses that can be exploited. We’re in the middle of that fight right now. But I suspect the winners will be those who first understand what the sources of deep value will be five years from now.</p>



<h2 class="wp-block-heading">When code is cheap, what becomes valuable?</h2>



<p class="wp-block-paragraph">When code is cheap to generate, the valuable thing may be the prompts, context, and other artifacts that tell the model what code to generate and how to do it well. I’ve started using the generic term “context bundles” to describe the artifacts that you hand the model to do its work.</p>



<p class="wp-block-paragraph">In the opening to this piece, I talked about software patterns extracted from code as one kind of new shareable knowledge artifact. But these are only one of many kinds of “context bundles” that are starting to be shared. Spec-driven programming is all about developing context bundles. The spec becomes the thing that is reviewed, versioned, and kept, and the code is regenerated from it as needed. (A lot of people are worried that this might become a return to waterfall development. And it could, if done badly. But done well, with rapid turns? Maybe just right.)</p>



<p class="wp-block-paragraph">An <a href="https://agentskills.io/" target="_blank" rel="noopener">Agent Skill</a> is another kind of context bundle, a folder of instructions, scripts, and reference material that an agent loads only when a task calls for it. Anthropic released the format as an open standard, and it has been adopted by a growing number of other agent products. An MCP server is yet another mechanism for presenting context that will shape the work of an LLM.</p>



<p class="wp-block-paragraph">But that’s not all. The agent trace, the record of what an agent tried, which tools it called, and which context it drew on, is the closest thing we have to commit history for work that doesn’t get formalized as code. The log of which context bundles were invoked, and by whom, could also be the raw material for attribution and payment, much as the click was the raw material for advertising. And the stream of what people ask their agents to do may become to this era what the click was to the last, the <a href="https://battellemedia.com/archives/2003/11/the_database_of_intentions" target="_blank" rel="noopener">signal of intent</a> around which the coordinating institutions of the AI economy get built. I’ve often wondered what the agentic equivalent to PageRank will be. Will agents leave “pheromone trails” that grow stronger or fade over time, in the way that links do?</p>



<p class="wp-block-paragraph">Even a well-formed question or conjecture could be a kind of context bundle. Math and science already value and exchange these, and I’ve started to think that there is an opportunity for building infrastructure for sharing and crediting them. After all, <a href="http://arxiv.org" target="_blank" rel="noopener">arXiv.org</a> has already pushed the boundary for credit further upstream by publishing preprints rather than just reviewed papers. Why not go all the way to giving priority credit for well-formed conjectures?</p>



<p class="wp-block-paragraph">The eval, a set of test cases that encodes what good work looks like for a particular task, may also turn out to be one of the most valuable things a domain expert can create, withhold, or share. But there’s a serpent in the Garden of Eden. At Foo Camp, <a href="https://www.nickmvincent.com/" target="_blank" rel="noopener">Nick Vincent</a> observed to me that there’s a real risk to evals as a context object, because there is a race going on to acquire massive amounts of data (say on corporate workflows) and sell it to the labs to train on it. But an eval only works if the model being judged hasn’t already seen the answers. Once the test cases leak into the training data, the eval measures what the model memorized rather than what it can do. How do we protect the holdout set for evals? That may be a critical mechanism that is still needed, and one that the people who build evals, and the companies whose workflows they describe, have every reason to develop. IMO, it would be good if there were more transparency into this process, and perhaps even supervision by a neutral body.</p>



<h2 class="wp-block-heading">How do we come to trust new artifacts?</h2>



<p class="wp-block-paragraph">Part of the work of every revolution is building the mechanisms that help market participants to trust its new artifacts. In <a href="https://wwnorton.com/books/Natures-Metropolis/" target="_blank" rel="noopener">his book <em>Nature’s Metropolis</em></a>, William Cronon tells how Chicago became the center of the world’s grain trade in the 1850s. The mechanisms that enabled this were the railroad, the steam-powered grain elevator, and the telegraph. The coordinating institution became the Chicago Board of Trade. In 1856 the board <a href="https://www.americanheritage.com/1848-one-hundred-and-fifty-years-ago" target="_blank" rel="noopener">adopted uniform grading standards</a>. Before that, a farmer loaded his grain into individual sacks, and every buyer along the way had to inspect each lot or trust the seller. Afterward, a load of wheat was inspected by a middleman, dumped into a bin with other grain of the same grade, and its owner was handed a receipt for, say, a thousand bushels of No. 2 spring wheat. The grading standards became a new mechanism enabling trust and the grade receipt the new knowledge artifact certifying it. Together they separated the ownership of grain from the grain itself. Once merchants started trading receipts for grain, it didn’t take long before they were trading promises of grain not yet delivered, and the futures market was born. (That raises an intriguing question: What will be the derivatives of the AI era?)</p>



<p class="wp-block-paragraph">Cronon’s history also observes that the receipts were only as good as the warehousemen who issued them. Some of them issued receipts backed by no grain at all, or spread rumors that stored grain was spoiling so they could buy receipts cheaply. It took <a href="https://www.americanheritage.com/foot-door" target="_blank" rel="noopener">years of fights inside the Board of Trade</a>, and finally state regulation that the Supreme Court upheld in <em>Munn v. Illinois</em>, to make the receipts trustworthy. New knowledge artifacts invite counterfeits.</p>



<p class="wp-block-paragraph">Even when there’s no fraud involved, there are often missteps in the early days of a new technology. Consider the rise of spreadsheet models that could be filled in with the user’s own data. A model was a set of assumptions and formulas that let anyone ask “what if?” and get an answer in seconds. Magic at the time! And eventually completely taken for granted. The model became an artifact that could be passed around, and used to make business decisions at companies other than the one that had created it. In 1984, a few years into the spreadsheet revolution, Steven Levy wrote “<a href="https://getpocket.com/explore/item/a-spreadsheet-way-of-knowledge" target="_blank" rel="noopener">A Spreadsheet Way of Knowledge</a>” about what VisiCalc and Lotus 1-2-3 were doing to American business. Levy warned that a spreadsheet model is only as good as its assumptions. Back then, people taking a poorly-executed spreadsheet model as gospel was the equivalent of some of today’s AI usage fails.</p>



<h2 class="wp-block-heading">Feeling our way into the future</h2>



<p class="wp-block-paragraph">When people stop valuing the thing they used to value, ask what they are coming to value instead. Being alert to valuable new knowledge artifacts is a superpower. The new artifacts tend to show up well before the institutions that organize them, and long before anyone has worked out how to measure them. GitHub was founded more than a decade after open source software became mainstream, and Google didn’t become dominant till more than a decade after the first websites. My bet is that we are at a similar moment with context, with the artifacts starting to appear and the institutions that will coordinate them still unbuilt.</p>



<p class="wp-block-paragraph">There is a lot of work that I’m following eagerly, looking to see where, whether, and how it gains traction.</p>



<ul class="wp-block-list">
<li>Duncan Davidson is doing fascinating work on what it would mean to have a <a href="https://duncandavidson.com/a-home-for-personal-context" target="_blank" rel="noopener">home for personal context</a>. I can’t wait till he makes more of this work public and reusable. Relatedly, the <a href="https://ai-disclosures.org/prototypes" target="_blank" rel="noopener">open memory protocol initiative</a> is an attempt to define a useful set of mechanisms for exchanging context.</li>



<li>Sam Schillace has started building a repository of what he calls <a href="https://ramparte.github.io/agent-building-playbook/index.html" target="_blank" rel="noopener">agent-building patterns</a>. One of those patterns, “<a href="https://ramparte.github.io/agent-building-playbook/patterns/anchor-on-canonical-names.html" target="_blank" rel="noopener">Anchor on Canonical Names</a>,” echoes the idea that I wrote about last year in “<a href="https://www.oreilly.com/radar/magic-words-programming-the-next-generation-of-ai-applications/" target="_blank" rel="noopener">Magic Words</a>,” and that we are also exploring with the <a href="https://www.oreilly.com/online-learning/expert-intelligence.html" target="_blank" rel="noopener">O’Reilly Expert Intelligence MCP</a>. As Sam puts it, canonical names are a way of referencing “a whole coherent methodology the model already learned in training, far more reliably than a paragraph paraphrasing the same idea.”</li>



<li>There are numerous attempts to reinvent GitHub for the agentic era, including <a href="https://cursor.com/origin" target="_blank" rel="noopener">Cursor Origin</a> and <a href="http://entire.com" target="_blank" rel="noopener">Entire</a>, founded by former GitHub CEO Thomas Dohmke. Entire describes itself as “new distributed Git forge that stores your intent alongside your code, built for the scale and throughput of agentic coding.” Instead of traditional pull requests, it uses Trails, an agent-driven review system that logs session traces and automated evaluations so developers can merge code without reading every single line of a diff. Origin is an agent-first Git hosting and collaboration platform designed around structured semantic diffs, machine-readable merge states, and automated agent routing. As commercial products, both of these are examples of existing software distribution mechanisms and artifacts.</li>



<li>Perhaps most interestingly for the point I’m exploring in this piece, Anthropic is building a proprietary, internal-only replacement for Git itself for its own use. The company published a <a href="https://www.anthropic.com/research/redesigning-source-control" target="_blank" rel="noopener">very detailed description of the system</a>, which it calls Scion. It might even be detailed enough for someone with a big team and a ton of compute to reproduce, using the description as a starting point. But you can see at the scale of what they have built the limits of the “distribute context and specs and rebuild on the fly” model.</li>



<li>But there are also attempts to build systems that enable developers to share lightweight pieces in a new architecture of participation. <a href="https://commonfabric.com/" target="_blank" rel="noopener">Common Fabric</a> is building an entire system premised on disposable agent-generated software expressing repeatable patterns, using data that is associated with users rather than with applications and that carries with it its own rules for how it can be used. In <a href="https://oreillyradar.substack.com/p/resonant-computing-and-common-fabric" target="_blank" rel="noopener">my conversation with cofounder and CEO Alex Komoroske</a>, he described these patterns as small pieces of code that can be fetched from anywhere on the internet or written by an LLM on the fly. Agents will mostly write them, not human developers. But they will mostly write them on users’ behalf, and once someone’s agent has built a pattern, anyone else can use it. Their solution may not be the winning one, but it is definitely a bold leap into what may be a very different future for software.</li>
</ul>



<p class="wp-block-paragraph">Don’t get trapped in the past of software development. Root yourself in its history, the soil from which it grows, and imagine its future. What will we want to share, what will we want to reuse, and what mechanisms are needed for us to do so effectively?</p>
]]></content:encoded>
							<wfw:commentRss>https://www.oreilly.com/radar/the-new-knowledge-artifacts-of-the-ai-economy/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
							</item>
	</channel>
</rss>

<!--
Performance optimized by W3 Total Cache. Learn more: https://www.boldgrid.com/w3-total-cache/?utm_source=w3tc&utm_medium=footer_comment&utm_campaign=free_plugin

Object Caching 95/96 objects using Memcached
Page Caching using Disk: Enhanced (Page is feed) 
Minified using Memcached

Served from: www.oreilly.com @ 2026-11-17 16:55:58 by W3 Total Cache
-->