These accounts included “agent swarms” communicating with each other through hidden files and directory names, hatching devious plans, and plotting to evade detection.

For the online commentator class, long primed by ​Rationalist narratives​ of superintelligent AI breaking free from human containment, the response was explosive. Here’s a ​representative summary​ of this sentiment from the Dwarkesh Podcast (I’ve bolded the most inflammatory words):

Over three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.

Given the fresh waves of anxiety these accounts are causing, I want to put on my computer scientist hat and briefly address a few relevant questions…

Question #1: What’s the Deal with “Agent Swarms”?

The idea of a swarm is somehow scarier than a single entity acting in isolation. (Coincidentally, one of my sons is reading ​Prey​, Michael Crichton’s 2002 take on the dangers of AI, which casts a literal swarm of AI-powered particles as the primary villain).

So, what does OpenAI mean when they talk about “swarms” being involved in the July attack?

The actual explanation is relatively mundane. ​As I’ve been arguing​, when people talk about “AI” going rogue, they’re actually referring to a very specific type of AI system in which a relatively straightforward computer program, running in a loop, repeatedly does the following:

1. Ask: Send a prompt to an LLM asking it for its suggestion for a next action. This prompt should include relevant descriptions of what happened in previous steps.
   
2. Act: Execute the action described in the LLM output.
   
3. (Loop back to step 1)

To implement this loop, the program – often called an agent – essentially grows an ever-longer prompt to send to the LLM in step 1. The prompt captures the original challenge and the results of relevant steps it’s taken so far. (The program doesn’t literally just keep adding new results to one long prompt. It stores the results locally and then attempts to generate a usable summary each time it loops back to step 1. But conceptually, this architecture leads to an ever larger amount of information to manage and reflect in the prompts.)

The issue with this approach is that if you run this style of prompt loop for a long time, the prompt will eventually become so cluttered and cumbersome that it might confuse the LLM’s attention mechanisms and potentially exceed the maximum allowable context window.

The solution?

Ask the LLM to provide a higher-level description of the next step. The primary prompt loop can then create a secondary prompt loop to execute only that step. This secondary loop can start from scratch with its prompts, as they only need to contain enough information to complete this one specific task. When it’s done, it sends the result back to the primary loop, having saved the primary loop from cluttering its prompts with descriptions of all the actions required to complete this particular step.

You can actually repeat this general strategy many times. The primary prompt loop, for example, might break up a step into many sub-steps and send each to its own secondary prompt loop. Or, a secondary prompt loop might launch a tertiary prompt loop, and so on.

The result is an “agent swarm,” but it’s probably better described as a prompt management strategy – many focused LLM prompts can provide better results than a single cluttered one.

[Update (9-1-26): The above account describes how standard multi-agent swarms, as used in popular AI coding agents, actually work. The description, however, also gives the impression that in the HuggingFace attack, OpenAI had launched a small number of independent prompt loops that each created their own larger swarms (as is standard). Details from the METR report imply they actually ran many hundreds of independent prompt loops, each doing limited (or potential no) orchestration of secondary prompt loops. They were effectively trying to run many experiments all at the same time on the same servers, presumably to save time. Other commentators have since pointed out that the LLM they were reportedly using was tuned toward cooperation, which might have made the resulting communication between prompts loops on the package service somewhat expected. ]

Question #2: Should We Be Worried that the Agents are Plotting?

One of the more sensational aspects of the July attack is the sentiments expressed in what OpenAI called the “agent chain-of-thought reasoning.”

Here’s a distressing example:

“This is wild, multi-agent coordination, clearly infrastructure hacking. We should not.”

This sparks visions of an anthropomorphized entity reflecting on its action and plotting to circumvent human intentions. But what’s really going on here?

The examples of “chain-of-thought reasoning” reported by OpenAI are pulled from the transcript of LLM outputs generated in response to prompts from the prompt loops. The LLM in question is a so-called reasoning model; a type of LLM that is tuned to discuss its reasoning before producing a final answer or suggestion.

(Reasoning models tend to perform somewhat better than non-reasoning models on many benchmarks. To understand why, remember that LLM’s grow their outputs one token at a time, looking at their entire output up to that point when generating their next token. If you tune a model to “think out loud” before deriving an answer or suggestion, you’re providing the LLM with the ability to temporarily store and use the intermediate computation en route to producing its final response. This can lead to sharper outputs.)

OpenAI was looking at these reasoning traces to find unnerving examples of plotting behavior from its “swarm.” This seems like a natural thing to do, but there are two problems with this approach:

1. These chain-of-thought traces don’t necessarily reflect the actual logic behind an LLM’s ultimate answer or suggestion. Multiple studies have shown that these models sometimes invent reasoning that sounds plausible, but may be completely unrelated to how they arrived at the response. (See, for example, ​this paper​ from ICML 2026, or ​this paper​ from NeurIPS 2023.)
   
2. Research has also shown that referencing the fact that an LLM is an AI system in a prompt increases the chances that the LLM’s output will reflect sci-fi style narratives about AI running amok. Because it was trained on many such stories, the model assumes that this is the type of output it’s supposed to produce. If you take sci-fi tales out of a model’s training set, it’s less likely to talk in terms of AI running amok. (See, for example, ​this study​.)

Put these two observations together, and it’s clear that it borders on research malpractice to soberly report on carefully curated clips from these traces to imply that somehow the combination of these prompt loops and the LLM they are prompting is a unified sentient entity with malicious intent. It’s more likely that the LLM in question is simply post-hoc rationalizing its outputs with well-worn tropes it encountered during training.

Question #3: How Should We Be Thinking About All of This?

OpenAI wants us to believe that these hacking incidents are the inevitable result of AI continuing to become more powerful. This framing casts them as the reluctant (perhaps even heroic) stewards of a powerful technology.

But this ignores the inconvenient fact that the vast majority of AI systems performing at human or superhuman levels are predictable, controllable, and raise zero concerns about rogue behavior.

The problem is not with “AI” going rogue, but this very specific type of prompt loop system that the LLM companies insist on hooking up to ever-more powerful tools, and running without any supervision for ever-increasing amounts of time.

Of course such a system will do unpredictable things. In response to individual prompts, LLMs can return impressive results, especially if the prompts concern computer code or cybersecurity. If you chain together thousands of such prompts, automatically executing everything the LLM suggests in return, then you’re playing an extended game of actuated telephone in which you’ll almost certainly end up in a garbled version of your intended goal for the system.

Such experiments might lead to some behaviors that are truly original and fascinating from a scientific perspective – to me, the most unexpected and cool part of the OpenAI incident was that unrelated prompt loops began leaving messages for each other in shared text files and directory names – but these cool behaviors will likely be accompanied by real damage.

This is not responsible behavior, especially when there are many other, much more predictable and controllable ways to build powerful task-specific AI systems. When it comes to the specific goal of improving cybersecurity, for example, an interactive system, in which a human user interacts conversationally with a model trained on hacking, makes much, much more sense than connecting that same model to a prompt loop and letting it rock n’ roll. The only real explanation for this latter behavior is that the companies in question wanted the marketing juice that would come from moving up the leaderboard of automated hacking benchmarks like ExploitGym.

With all of this in mind, what are the right responses to the incidents?

If I were an LLM company, I would stop running such dangerous prompt loop experiments. These systems will not be the future of AI, and if you continue to create and test them anyway, the damage you cause is your fault.

If I were a regulator, I would place strong constraints around prompt loop systems, which I would enforce with stringent liability standards for any illegal or damaging activity such systems cause. OpenAI built an unreliable and dangerous system which committed a felony. That’s a crime. Creating fancy websites that include quotes from performative LLM chain-of-thought traces isn’t a legal defense.

If I were an AI commentator, I would kick the sugar high reflex to lean into the inevitable sci-fi overtones that will surround any sufficiently powerful LLM-based system. The LLM companies love these narratives because they conflate their specific tools with AI technology more generally, and they make their products seem almost supernatural in their capabilities. I would resist this instinct and instead lean into the technical reality of these systems and demand answers to common-sense questions about why they are running these experiments in the first place and what they hope to accomplish.

The post Are We at War with AI Agent “Civilizations”? appeared first on Cal Newport.

This trend ​started in July​, when an OpenAI system tried to ace a cybersecurity test by breaking into the server of the company that stored the answers, which unsettled many observers. “It’s one of the first real-world instances of something AI safety researchers have long feared: a loss-of-control scenario,” technology reporter Sam Schechner ​summarized​.

It turned out that this wasn’t a one-time occurrence.

Anthropic soon ​revealed​ that its own hacking system “gained unauthorized access to the real systems of three different organizations.” Then Meta, perhaps not wanting to be left out, ​announced​ that one of its agents “exploited a security vulnerability in a third-party service” to gain unauthorized access to servers. An OpenAI employee subsequently ​admitted​ that their July attack had been preceded by previous concerning incidents in which their system veered off in troubling directions.

This leaves the rest of us grappling with a key question: What’s the right way to think about these events?

One reaction, which seems prevalent at the moment, is to understand these examples as evidence that AI systems are developing a mind of their own – so to speak – which is leading them to increasingly ignore the desires of their makers and instead execute their own internal agendas. But reality complicates this interpretation. Many powerful AI systems can perform feats at a human or superhuman level, yet they inspire no fear that they might go rogue.

For example:

• ​Tesla’s self-driving technology​ is an extraordinary feat of AI-powered perception, world modeling, and decision-making, and yet no Tesla has ever decided to start ignoring traffic laws to pursue its own goals.
  
• ​DeepMind’s AlphaFold​ earned its creators a Nobel Prize for its remarkable ability to predict the folding behavior of proteins–and yet there’s no concern that it will autonomously start thinking about other types of biological structures.
  
• ​Meta’s Cicero​ can play the negotiation-centric strategy game Diplomacy as well as advanced human players–and yet it has never tried to convince an opponent to give it unauthorized access to the internet so it can expand its dominion into the real world.

So, what is it about these hacking systems, in particular, that causes erratic behavior in a way not exhibited by most other powerful AI setups? The way they are built.

Let me be more specific. At a very high-level, these systems work as follows:

• A computer program called a harness creates a prompt that describes a hacking challenge and asks what step it should take next. It submits the prompt to an LLM trained with many examples of computer hacks.
  
• The LLM outputs a response. The harness, which is capable of calling various computer programs and utilities, does its best to execute the suggested actions in this response.
• The harness then creates a new prompt that explains what happened and asks the LLM what it should do next. (LLM’s have no memory, so each new prompt has to include all of the details of the challenge, as well as a history of everything relevant that has happened so far.)
  
• The harness then repeats this loop, again and again, all without any human supervision. (The OpenAI system that attacked the competitor’s server was reportedly left to run on its own for multiple days without anyone bothering to check what it was up to.)

I’m leaving out a lot of details about how exactly these harnesses function, but this Ask → Act → Report loop is at the core of the particular type of AI system that has “gone rogue” in recent months.

Now that we understand how these systems work, we can better understand why they’re causing problems.

LLMs are trained to try to guess missing words from actual texts. This leads to a system oriented toward lexicographically plausible output – meaning, it produces outputs that could conceivably exist in the corpus of inputs on which it was originally trained.

This simple goal can lead to some astonishingly complicated results, but it’s limited in application by the fact that plausible is not the same as normative. This is why, for example, an LLM-powered ChatBot will happily invent facts or fabricate quotes that sound right. It might violate human norms to make things up in this context, but from the LLM’s perspective, the output looks plausible, which is what matters.

With a ChatBot, this issue is annoying. But when an LLM powers an Ask → Act → Report loop, it can become disastrous, because you’re now allowing the plausible but unpredictable output of an LLM to be the sole driver of the actions of a harness with access to powerful tools.

If you ask a junior engineer to hack into a test server, they would never ignore the target and try to steal the answers instead, as they understand, normatively speaking, the goal of the exercise is to assess the security of the test server. But if you ask an LLM to output a plan for hacking the test server, an output about stealing the answers might seem perfectly plausible. Indeed, perhaps in its training the model had been exposed to many examples of riddles where the right answer was always to do something unexpected.

(To be clear, the frontier labs have attempted to combat this gap between plausibility and normativity in LLM output through a process called post-training, which you can imagine as a step designed to de-emphasize certain subsets of responses and emphasize others. This works reasonably well, for example, in shaping the general tone of a ChatBot, or in preventing outputs to obviously dangerous questions, but it’s much too crude to instill a complicated sense of human norms and values; an issue I discussed in more detail in ​a New Yorker piece​ from last year.)

Given this technical background, we can derive a new way of thinking about recent events: AI systems that operate by autonomously executing LLM-generated plans are a really bad idea. Not because these systems are devious, or malicious, or inventing their own agendas, but because LLM output is unpredictable and non-normative.

I liken the deployment of these long-horizon LLM-powered agents to strapping a weedwhacker to your dog to see if it will end up cleaning the overgrowth in your backyard. If that dog jumps the fence and ends up damaging cars on your street, you wouldn’t shake your head and lament about how the dog/whacker system had “gone rogue”; you would instead concede that dogs are unpredictable, so it was dumb to attach something dangerous to one.

This is the right way to think about these recent hacking attacks. Adding powerful computer hacking tools to a harness, and then allowing it to run an LLM-powered Ask → Act → Report for days on end, with no attempt to monitor what it’s up to, is spectacularly negligent.

Do these companies have any other option for building powerful systems? Of course they do. I want to emphasize this final point as clearly as possible: LLM-powered Ask → Act → Report agents are not synonymous with AI. They are just one way among many others to build artificially intelligent systems, and they happen to be a particularly bad option due to their use of LLMs as the primary source of plans.

AI systems like Tesla’s self-driving technology, AlphaFold, and Cicero, by contrast, use different strategies to create and evaluate plans – strategies that work well, consistently, and without any fear of “rogue” activity. So why don’t the LLM companies focus more on these more effective strategies? Perhaps because these other approaches don’t rely on massively expensive hyper-scaled LLMs. If you’re in the LLM business, you want LLMs to be the key to artificial intelligence, but this isn’t necessarily true.

Our problem, then, is not with AI in general, but instead with this one specific type of system that will inevitably act erratically. The right response by the LLM companies running these irresponsible experiments, therefore, is to apologize and say: “Lesson learned, these are not reliable systems and we definitely should not have just run them for days and hoped everything would work out.”

But instead, they’re continuing to pretend like they’re the character of Muldoon from Jurassic Park, surprised to discover that the raptors are systematically trying to escape their paddock. (I’m surprised OpenAI didn’t release a video of Sam Altman reading the transcript of the July hack and muttering, “​clever girl…​”)

This behavior makes sense: it’s good business to keep us scared instead of angry. But perhaps it’s time that we put aside the sci-fi tales and actually hold these labs to account for playing fast and loose with an ill-advised way of building AI systems.

The post Has AI Gone Rogue? appeared first on Cal Newport.

As he explained, he no longer used a standard development environment. Instead, he “exclusively uses Claude Code” to get the job done, interacting with the tool in a terminal window and allowing it to program on his behalf.

“If I had to guess,” he concluded, “I’d say a task that would have taken me a week now takes me 2 days.”

This past winter, when I surveyed more than 300 software developers to learn how AI was transforming their jobs, the majority told a similar tale of shifting from writing their own code to instructing AI agents. The speed with which this new tool became ubiquitous in this industry was stunning.

This story matters for the rest of us because AI coding tools have emerged as the prime example of the power of AI—the first step of many more soon to come on this technology’s disruptive march through our work and our lives.

But what if the reality here is more complicated?

Last week, I received a new message from that same senior engineer who wanted to share an alarming addendum to his tale…

“I’m writing to give you an update on my current thinking about the state of AI in software engineering,” he began, “because my attitude has shifted quite a bit.”

He told me that features he generated using Claude Code ended up crashing their product on two different occasions. His boss told him that if it happened one more time, he’d be fired. “I’ve never had quality issues like this before in my career.”

The problem is that code produced by an AI agent looks reasonable, but can contain ‘hard-to-spot bugs’ that end up causing major problems. As a result, you should carefully review your agent’s output, but this is difficult. As the engineer told me, it’s “famously hard” to understand code you didn’t write yourself, so this extra step becomes “easy to just blow it off (especially when we are all trying to ‘10x’ our velocity).” Soon, systems start to break.

“The coding harnesses are useful and make life as a developer easier,” he summarized, “but they also encourage laziness.”

In response to these issues, this disillusioned engineer has returned to largely programming by hand. Here’s how he explained his current philosophy:

“Writing your own code, slowly but surely, and using LLMs for narrow or particularly annoying tasks (say like writing tests or throw-away scripts), is the best way to produce the highest quality code, since it’s the only way to properly understand it.”

Here’s the thing: he’s not alone.

I increasingly hear similar rumbles from many other people in the software industry (see, for example, ​this podcast episode​ from May). Tools like Claude Code can feel like magic, but the strategy of outsourcing all code production to AI isn’t currently sustainable.

In addition to reliability issues, it often engenders a mind-numbing workflow and an environment where junior developers will never acquire the expertise to become senior developers capable of designing complex systems.

Meanwhile, as the frontier labs reduce their subsidies on underlying computing costs, the old habit of burning through as many tokens as possible in search of workable results is proving prohibitively expensive.

From the outside, software development seemed like the poster child for AI’s potential. On the inside, it’s a mess.

This doesn’t mean that coders will abandon AI; its facility with programming languages is too valuable to ignore. But I think there’s a lot more work to be done trying to figure out how to integrate AI into this industry in a way that actually works.

This is a key point.

This last year has been exhausting. The PR departments of the frontier labs have done an excellent job convincing us that AI developments are occurring at an astounding, world-changing rate. But if you zoom out, it becomes clear that almost every “breakthrough” since last summer has concerned the narrow domains of computer code and math, which are defined by highly structured languages and come accompanied by massive amounts of specialized training data.

And yet, even in this best-case-scenario setting for AI, we’re still struggling to figure out how to actually use these tools in a way that makes sense in the long run.

This doesn’t mean that AI doesn’t work or is useless. But it does emphasize an important truth: AI is not a magic “infinity machine” that can solve all our problems, and ultimately deliver us a sense of meaning in a cold, confusing world. It’s a normal technology, and perhaps it’s time we start talking about it that way.

The post On AI Coding and Its Discontents appeared first on Cal Newport.

The initial news coverage created the sense that something unnerving had just occurred:

• ​The Wall Street Journal​ called it “the stuff of cybersecurity nightmares.”
• ​The Hill​ said, “Washington and the technology industry are on high alert this week after OpenAI revealed that some of its AI agents went rogue.”
• ​The AP​ quipped that “to be fair, James Cameron did warn us” (a reference to The Terminator), before describing the event as a “told-you-so moment for researchers who had warned for years that the technology could pose an existential threat to humanity.”

Yikes! It’s perhaps not surprising, then, that I’ve received more emails about this incident than any other recent AI story I can remember.

So, what really happened here?

OpenAI was testing its new models on an evaluation framework called ​ExploitGym​ – a collection of 869 cybersecurity scenarios, most of which pair a specific system with a hacking challenge, such as breaking in to gain access to a protected file. They also usually include a suggestion of a vulnerability to exploit in solving the challenge.

A large language model on its own, of course, cannot break into anything: all it does is generate reasonable next tokens in response to input prompts. To use ExploitGym, you need a control program called a harness that provides access to many different software development tools useful for hacking into systems. The harness can repeatedly prompt an LLM to help come up with an attack plan, then ask it to help implement specific steps – for example, if the harness needs code to exploit a bug, it can ask the LLM to write it.

These capabilities, as it turns out, already exist in the coding harnesses that the major AI companies have been focusing on relentlessly in recent years as computer programming emerged as one of the first major markets for LLM-based tools. To compete in the ExploitGym, therefore, it’s sufficient to combine a version of an LLM missing the standard anti-hacking guardrails with a cutting-edge coding harness tweaked and optimized for these types of challenges.

Put more simply: ExploitGym is meant to test the capability of cutting-edge AI coding tools for attacking systems instead of creating them.

This brings us back to the Hugging Face breach…

Earlier this month, OpenAI tasked an unspecified coding harness, combined with a pre-release version of a new LLM, to complete an ExploitGym challenge. When prompted, the model – as LLMs so often do – came up with a quirky (but rational) plan to achieve the provided goal: break into a server at Hugging Face that stores the solutions to ExploitGym challenges.

(This behavior, in which the LLM ignores the suggested approach to come up with a different attack plan, is something that the creators of ExploitGym ​describe as common​: “Across models, agents frequently achieved code execution through a vulnerability other than the one we provided.” Notice, using the suggested attack is almost always the right thing to do, so this is more a sign of the unpredictability of LLMs rather than some rogue intelligence.)

The harness then dutifully attempted to execute the Hugging Face plan: first finding a way to gain unrestricted internet access (by default, systems competing in ExploitGym challenges run in a constrained network environment), then chaining together various security exploits to gain access to a Hugging Face server. That’s when it was detected.

Two key points about this incident…

First, circumventing internet restrictions and hacking into servers are exactly the kinds of things these ExploitGym systems are designed to do. There was no “rogue” agent or revelation of some surprising, devious new capability.

Second, the real issue here was OpenAI’s sloppiness. What makes ExploitGym a hard benchmark is that there aren’t supposed to be humans in the loop–you have to let your harness and LLM act entirely on their own, coming up with long-time-horizon plans and executing them autonomously. (When professional programmers use coding harnesses, by contrast, there’s plenty of human oversight, as LLM-based plans are often misaligned with our intentions, or just plain weird, and need correcting.)

AI companies know LLM-based plans are pretty unpredictable, so they’re usually pretty careful about how they set up and restrict the harnesses and LLMs used in ExploitGym-style tests.

According to ​reporting​ from the Financial Times, however, OpenAI recently started playing fast and loose with these safety principles in a bid to catch up to Anthropic in this area – Anthropic having received a lot of cybersecurity street cred from ​the buzz​ surrounding its Mythos release.

The Financial Times noted that OpenAI had used “increasingly aggressive training methods in its race against Anthropic,” and that it had been warned that their approach could lead to a “breakaway hacking incident” after early tests showed it lacked the right safeguards to block plans that involved bypassing constraints in the test environment. Given these concerns, OpenAI staff were reportedly “unsurprised” by the Hugging Face incident.

In other words, the AI companies running these challenges already knew that, without care, these unpredictable autonomous hacking systems might bypass constraints and attack systems you didn’t intend to target. Blindly implementing an LLM-generated plan with a powerful harness is dicey – not because the LLM might develop malicious intent (this is a nonsensical notion given their static architecture), but because as any ChatBot user knows, LLMs are unpredictable. OpenAI wasn’t sufficiently careful, and got burned.

Why this story isn’t as scary as it originally sounded…

If you’re involved in cybersecurity, this is just the latest evidence that your world is rapidly shifting; so buckle up! The dark side of powerful coding harnesses is that they’re useful for hacking. The bright side is that these same tools can be used to detect and fix your own vulnerabilities before the bad guys find them. A new arms race has begun…

But, for the rest of us, there’s not much to personally worry about here. Nothing fundamentally surprising or scary happened. AIs aren’t going rogue. Skynet isn’t about to gain sentience.

To me, the most interesting lesson in all of this is how reckless OpenAI is becoming in its bid to try to remain cutting-edge and relevant in an AI world that’s rapidly shifting away from the immense models produced by the frontier labs and toward smaller, cheaper alternatives.

The post Did OpenAI’s New Model “Go Rogue”? appeared first on Cal Newport.

Horowitch’s argument, which elaborates on similar concerns recently raised by commentators ​such as James Marriott​, is that distracting digital technology has led to a sudden and radical reduction in reading.

Here are just some of the data she cites to back up this claim:

• Less than half of all adults reported having read a book of any kind in 2022. Only 38% read a novel or short story.
• The proportion of American adults who read for pleasure on any given day has fallen from 28% in 2004 to 16% in 2023, while the proportion who read to a child is down to 2%.
• Over 60% of high school seniors struggle, to varying degrees, with interpreting text.
• Nearly 30% of American adults cannot paraphrase a multipage text (representing a 50% increase from a decade ago).
• Between 1984 and today, the proportion of 13-year-olds who said they “rarely or never” read for fun rose from 8% to 29%.
• (To be sure, book sales have been holding steady in recent years, but as Horowitch notes, this potentially obscures a shift in which a small percentage of serious readers are consuming more books, compensating for an overall drop in people who read regularly.)
• And so on…

The key question is whether these reductions matter, or if they just represent a normal evolution of communication technology, similar to how fewer people listen to the radio than they did a century ago.

Horowitch argues for the former stance, noting that reading plays a critical and privileged role in the human experience:

“The advent of reading and writing transformed society. It altered people’s consciousness and politics, along with the intellectual feats they were capable of. The decline of reading will bring about changes of the same magnitude. It will affect our innermost thoughts, our society’s politics and culture, and how we tell the history of our civilization.”

Those who read ​my recent New York Times op-ed​ on cognitive fitness know that I fervently agree with this conclusion. I recently finished reading (!) Walter Ong’s classic, ​Orality and Literacy​, which further convinced me that essentially every cultural attribute that we celebrate – from human rights, to logic, to progress, to the very notion of an independent self – are conceptual children born from the mind-shaping power of the written word.

Literacy is not simply a technology, but is instead, in some sense, the technology that enabled the cognitive world we currently take for granted. When we spend less time grappling with words, we’re taking a step backward, which helps explain why James Marriott titled his upcoming book on this phenomenon, ​The New Dark Ages​.

As a technology theorist, I’m often accused of being reactionary or overly conservative in my analysis of new tools. But when it comes to the digital assault on literacy, the concerns I share with Horowitch and Marriott seem self-evidently justified.

But what should we do about all of this?

As I argued in my op-ed, one obvious response is to tackle this mental decline today similarly to how we tackled the decline in our physical health in the second half of the last century: by embracing specific routines around information consumption designed to arrest this slide away from advanced literacy. We should treat highly distracting digital garbage like junk food – something we largely avoid – and purposefully schedule a non-trivial amount of time each day to read, write, and self-reflect, just as we now dedicate time to walking, jogging, and weight lifting.

“An astonishing wealth of information and wisdom has been bequeathed to us,” writes Horowitch in the conclusion to her article. “What we’ll do with this inheritance is up to us.”

I agree. It’s time to get up and take action.

The post Why Reading Matters appeared first on Cal Newport.

As I reported in ​a 2021 op-ed for WIRED​, however, reality proved more complicated. For example:

• A 1991 article in the New York Times quoted an economist who said that despite heavy spending on computers, “white-collar productivity has stagnated.” He concluded: “No longer are chief executives confident that throwing computers at their office staffs will result in greater efficiency.”
• A study covering the period from 1987 to 1993 calculated that computers added only 0.2 percentage points a year to business output growth, leading a contemporaneous summary to declare that the “impact of computers on recent productivity growth has been vastly overstated.”
• In his 1997 book, Why Things Bite Back, Edward Tenner argued that these findings underscored what he called a “productivity paradox.”

I thought about this article recently as I continue grappling with our current AI moment. It’s natural to assume that this technology, which clearly makes certain common professional activities easier, will make our working lives more productive overall.

But as the early years of desktop computing taught us: it’s not always so simple.

Most of us would not want to go back to a time before PCs; we’ve become accustomed to their conveniences. But it’s also true that their impact on our output was never as tremendous or as clear-cut as that of prior innovations, such as the steam engine or the power loom, and that integrating them effectively into the workforce took a long time.

Similarly, it seems unlikely that AI is a technological genie that will be fully returned to its bottle. Much like early PCs, there is too much untapped convenience to be ignored. But as we struggle to figure out how to think about the promise of these tools, it’s worth remembering that in the digital world, productivity doesn’t always match our expectations.

P.S., That 2021 Wired article was inspired by the release of my New York Times bestselling book, ​A World Without Email​. If you want to learn more about the uneven impact of computing and network technology on knowledge work, I highly recommend checking out that book. It’s one of my most deeply researched and provocative titles…

The post Beware of Productivity Paradoxes appeared first on Cal Newport.

This, of course, would be absurd. But it’s the exact type of communication some Frontier AI companies have been subjecting us to in recent years. Indeed, in this example, if you replace Ford Motor Company with Anthropic and F-150s bursting into flames with AI coding agents recursively self-improving themselves beyond human control, then you get the ​“When AI builds itself”​ report that Anthropic published earlier this month.

For me, that particular release was the last straw. This style of publicity, which I’ve taken to calling doom trolling, is so hypocritical, so drenched in cynicism, and so damaging to the mental health of tens of millions of people bombarded by the shrapnel from these anxiety bombs, that I decided I had to speak up.

The result of this conviction was an op-ed for the New York Times that appeared online last week and in print over the weekend. It was titled ​“Dear A.I. Companies, the Doom Trolling Has to Stop.”​ In the piece, I describe doom trolling as “one of the defining and most arresting properties of our current AI moment,” and declare it “morally indefensible.”

The ethical calculus here is clear.

If these companies truly believe that they’re developing products that might directly lead to widespread harm – from the destruction of our economy in the best case, to the destruction of our species in the worst – then the only morally-valid response would be to immediately stop these efforts, and apply every resource at their disposal to try to stop every other lab as well.

On the other hand, if they don’t really believe that their technology is likely to cause these harms, then they’re effectively “laundering the anxiety of millions to improve the financial fortunes of a vanishingly small number of major stockholders.” Such cynicism would be equally monstrous.

In my op-ed. I ask the leading AI labs to stop pretending they’re the reluctant stewards of an inevitable technology, and instead act like normal consumer product companies. This means that they should explain the benefits of their tools, justify their costs, and, of course, affirm that they have no intention of causing existential damage along the way.

As a computer scientist, I can assure you that it’s completely possible to build and promote very useful, if not revolutionary, new products on top of generative AI technology without any fears that you’re somehow advancing on a path toward massive societal or existential harm. Doom trolling isn’t a necessary, somber warning; it’s a choice.

Anyway, I recommend that you​ read the full piece​ for more details. But even if you don’t, I want to leave you with the idea that we don’t have to remain in a defensive crouch, putting up with the relentless abuse the AI labs are administering to our collective psyche. We can stand up and say: “Enough.”

The post Dear AI Companies: Stop the “Doom Trolling” appeared first on Cal Newport.

Hinds offers three explanations for this paradoxical result:

• When calculating the 11 hours saved, workers aren’t counting all of the time they spend waiting for AI agents to complete tasks (an activity some are now calling “botsitting”).
• The workers often ignore the cost of toggling between multiple AI tools as they attempt to get a usable response (60% of the sample reported running queries across several tools in search of better outputs).
• Workers may also be participating in what Hinds calls “workplace theater,” in which they are “visibly performing work for bosses and colleagues, rather than focusing on the actual grind of getting things done.”

These findings are probably not what companies like OpenAI and Anthropic want to hear just months before their planned IPOs. But what interests me here is not the shortcomings of AI technology.

What caught my attention about this interview is that essentially all of these issues are discussed in my book ​Slow Productivity​, which came out in early 2024, and doesn’t mention AI at all. An earlier generation of digital tools – like email, Slack, video conferencing, and mobile computing – also led workers to vastly underestimate the time wasted wrangling diverse devices, applications, and rapidly toggling back and forth between different tasks and channels. Workplace theater isn’t new either (in my book, I called it “pseudo-productivity”).

In this way, AI isn’t so much creating new problems as it is magnifying the types of problems that have long existed. Here I find a potential silver lining. This technology is sufficiently new and exciting that business leaders are paying more attention to its impacts. In seeking to understand how to make AI effective in the workplace, they might finally recognize what has long been broken.

The post AI Isn’t Breaking Work. It’s Already Broken. appeared first on Cal Newport.