Wandering Nomad

I'm Stealing This AI Researcher's Workflow for My Own Projects

2026-04-05T10:39:00.002+08:00

Karpathy doesn't use a fancy app to manage his research. He uses a folder, Obsidian, and an AI — and I want to copy it.

He posted about it last week. The short version: he dumps raw material — articles, notes, papers, images — into a folder, then lets a large language model (LLM — the AI brain behind tools like Claude or ChatGPT) build a wiki from it automatically. The LLM writes the summaries, creates the links between ideas, organizes everything into categories. He barely touches the wiki himself. When it gets big enough, he asks it questions and gets answers pulled from his own research.

I've been sitting with this for a few days, thinking about what it would look like for my work.

---

What My Work Actually Looks Like

I build things. Agents, content apps, Claude Code workflows, automation scripts. A lot of what I do involves figuring something out — what tool does what, how to wire two things together, what prompt pattern produces the right output, what broke last time and why.

Most of that knowledge lives in my head, or in scattered notes, or in past conversations I can't find anymore.

That's the problem. Every time I start something new, I spend time re-learning things I already know. What flags to use in Claude Code. What agent structure works for what kind of task. What API response format caused everything to break last month.

Karpathy's idea is simple: stop keeping that knowledge in your head. Dump it in a folder. Let the AI organize it. Ask it back when you need it.

—

The Specific Thing I Keep Thinking About

He mentioned that his wiki grows and gets more useful with every question he asks. He asks something, the AI goes through his notes and answers it — and then he saves that answer back into the wiki. So every session adds something. Nothing gets lost.

That hit me because the opposite is true for how I work right now. Every build session ends and most of the small things I figured out just disappear. The next session starts almost from scratch on some of the same ground.

If I had a knowledge base for my Claude Code workflows alone — prompts that worked, structures that didn't, patterns I figured out, error fixes — and an AI that could surface the right piece when I needed it, I'd stop repeating myself.

—

The Part That Actually Excited Me

He also runs "health checks" on his wiki. He asks the AI to find gaps, spot inconsistencies, and find connections between ideas he hadn't noticed yet. The AI suggests new things to look into.

That's the part I can't stop thinking about.

Not just a system that stores what I know. A system that notices what's missing. For someone building content automation apps, that means the system isn't just remembering what tools I've used — it's noticing when two things I built separately could be connected. It's pointing to the next piece.

That changes how building feels. Less like starting from zero every time, more like picking up a thread.

—

What I'm Going to Test

I'm starting with one folder. My Claude Code workflows — the scripts, prompts, notes, fixes, things that broke and how I solved them.

I'll ask Claude to read through everything and build an index: summaries of each file, links between related ideas, a map of what I already know.

From there, I'll ask it questions mid-project. "What pattern did I use last time for a multi-step agent?" "What was the issue I kept hitting with streaming output?" Instead of digging through old files or trying to remember, I just ask.

I'm not building the full Karpathy setup yet. I'm testing whether the core idea holds: does having a searchable, AI-organized version of my own work actually save time and reduce the re-learning?

I Told an AI to Build a Full Newsletter. One Sentence. No Code.

2026-03-14T17:12:00.004+08:00

I used to spend half a day on client newsletters. Last week I did it in 15 minutes — and I'll show you exactly what I typed.

🤖 What Claude Code Actually Is

Claude Code is an extension you install inside VS Code — a free code editor. Once it's set up, you have an AI agent sitting inside your project that can read files, write code, call APIs, fix its own errors, and run automations — all through a chat window.

You talk to it. It builds things. That's the whole loop. You don't need to know how to code. You need to know what you want.

🧪 What I Actually Tested

I set up a project, gave the agent a brand logo and a color guide, connected a few API keys (one for research, one for Gmail), and typed one prompt. That was it.

What came back:

Research pulled from live sources
Three branded infographics generated with AI images
Full HTML newsletter formatted to the brand's colors and fonts
Email sent directly to a list via Gmail
Sources linked at the bottom, clickable

It hit errors along the way. An API endpoint had changed. A Gmail formatting issue broke the layout. Both times, it found the problem, explained what happened, fixed the tool, and kept running. I didn't touch anything.

🏗️ How the Whole Thing Is Organized

The system behind this is called

Component	What It Is
Pipeline	The instructions, written in plain English — like a step-by-step recipe
Tools	The specific actions the agent can take (research, generate image, send email)
Orchestrator	Claude Code itself, reading the pipeline and running each tool in order

You also give it a file called CLAUDE.md — a standing brief that tells the agent what the project is, where things live, and what it's supposed to do. Every time you open the project, it reads that first.

Once you've built and tested a pipeline, you can schedule it to run on its own — every Monday morning, every time a form is submitted, whatever you need. At that point the agent isn't running live. The code it built is. It runs predictably, like any other automation.

💼 What This Means for VAs and Small Business Owners

You don't need to learn to code. You need to be able to describe a process clearly.

If you manage newsletters, reports, client updates, or social content for clients — build a pipeline, test it once, let it run. If you spend hours every week on the same repetitive tasks in your own business, same idea.

What makes this different from something like Zapier is what happens during the build. The agent adapts. It hits an error, investigates, fixes the tool, and updates the instructions so the same thing doesn't break again. The final automation is more reliable — and you got there faster than mapping every node by hand.

💰 If You're Thinking About Offering This as a Service

Businesses aren't paying for a pipeline or a blueprint. They're paying for a solved problem.

Don't open with "do you want AI automation?" Ask: where are you losing the most time or money right now? Find that, build the thing that fixes it, and price it on what it's worth — hours saved, errors cut, costs gone — not on how long you spent building it.

A pipeline that saves a client 20 hours a week isn't a half-day job. Over a year, that's tens of thousands of dollars in time they're not paying someone else to cover. Price it that way.

🚀 Where to Start


Tool	Claude Code — runs inside VS Code (free to download)
Cost	Paid Claude subscription — $17/month on the Pro plan
Setup time	~1 hour, most of it connecting API keys

Pick one task you do every single week. Something repetitive, something predictable. Let the agent build the first version, watch it run, fix what doesn't land, and build from there.

The hardest part isn't the tool. It's deciding what to automate first.

This Paper Says We've Been Fine-Tuning the Hard Way

2026-03-14T16:49:00.006+08:00

I was reading a paper that dropped in March 2026 and about three paragraphs in, I had to stop. The claim seemed too simple: you can adapt a large AI model to a specific task without gradient descent at all. Just random sampling and a vote.

What Fine-Tuning Actually Is

When an AI model gets trained, it learns general patterns from a massive dataset. Fine-tuning is the step where you take that general model and push it toward a specific task — coding, reasoning, following instructions — using reinforcement learning or optimization algorithms.

Methods like PPO and GRPO (types of reinforcement learning commonly used to fine-tune large language models) work well. But they're expensive, require careful setup, and involve a lot of iteration.

The Paper's Core Claim

Title: Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights Authors: Yulu Gan and Phillip Isola Published: March 12, 2026 · arXiv: 2603.12228

The idea: in large, well-pretrained models, the weight space around the original parameters is already densely packed with useful task-specific solutions. The authors call these clusters "neural thickets."

In smaller models, good solutions are scattered — you need gradient-based search to find them. In large models, they're close to where you already are.

The Method — Remarkably Simple

They call it RandOpt. Here's how it works:

Take the pretrained model weights
Randomly sample N small perturbations of those weights
Evaluate each perturbation on your task
Keep the top K performers
Combine them with a majority vote

No gradients. No reward model. No RL training loop. Perturb, evaluate, vote.

💡 Code at github.com/sunrainyg/RandOpt

What the Results Showed

RandOpt kept up with PPO, GRPO, and evolutionary strategies on the tasks they tested — and this held on large-scale contemporary models.

That stopped me. These optimization methods have entire research communities, years of papers, and significant infrastructure built around them. The idea that randomly sampling neighbors of the original weights and taking a vote can match them says something about what's already sitting inside a well-trained model — waiting, not absent.

The way they frame it: small models have sparse expert solutions, so you need search to find them. Large models have dense ones. The pretrained weights aren't just a starting point. They're already rich.

What I Took From This

This paper shifted something in how I think about pretraining. We usually assume the pretrained model is a rough starting point and fine-tuning is where the real work happens. This flips that. If the solution space is already dense around the initial weights, the pretrained model is doing more work than we give it credit for.

There's also a question I don't have a full answer to yet: if random sampling with ensemble voting matches expensive RL fine-tuning, what does that mean for how we should be spending compute? I'm still working through the paper and the code, but it's the kind of result that sits with you.

Full paper: arXiv 2603.12228

Google Just Replaced Five AI Search Tools With One

2026-03-12T10:18:00.002+08:00

Have you ever tried searching through a client’s content library where videos are in one folder, PDFs in another, and audio recordings scattered everywhere?

That’s the reality for most content libraries.

Until now, AI search tools struggled with this kind of setup because each type of content needed a different system to process it.

But that may be changing.

Gemini Embedding 2 — recently released by Google — can search across text, images, audio, video, and PDFs at the same time, without converting everything first.

For anyone managing knowledge bases, course content, research archives, or client media libraries, this could be a major shift.

What an Embedding Model Actually Does

Before explaining why this matters, it helps to understand what an embedding model is.

When AI systems search through content, they don’t read information the same way humans do. Instead, they convert content into numerical representations that capture meaning.

For example:

A sentence about a cat
A photo of a cat

Both produce similar number patterns, which allows the AI to recognize that they are related.

That’s how modern AI-powered search works.

The tool responsible for converting content into these numerical representations is called an embedding model.

The Problem With Older AI Search Systems

Until recently, every type of content required a different embedding system.

Typical setups looked like this:

Text → processed by a text embedding model
Images → processed by image models such as CLIP or SigLIP
Audio → first transcribed using systems like Whisper
Video → broken into frames or transcripts
PDFs → converted into plain text

This created several issues:

Multiple models to manage
Several conversion steps
More chances for things to break
Slower search performance

In many cases, five different pipelines were required just to search one content library.

What Gemini Embedding 2 Changes

Gemini Embedding 2 solves this by creating one shared search space for multiple content types.

Instead of converting everything separately, the model processes different media formats directly and places them into the same semantic search system.

That means a single query can return results from:

Documents
Images
Audio clips
Video files
PDFs

All at once.

For example, you could:

Upload a photo and find related videos
Submit a voice recording and find matching documents
Search inside PDF files without converting them

Supported Input Types

Gemini Embedding 2 currently supports multiple media types in one system:

Text
Up to roughly 8,000 words

Images
Up to six images in one request

Audio
Raw audio files — no transcription required

Video
Clips up to two minutes long

PDFs
Original files can be processed without converting to plain text

All of this works through one model instead of multiple specialized ones.

Combining Multiple Inputs in One Search

One interesting feature is the ability to combine different types of input into a single query.

For example, you might have:

A photo of a product
A text description of what you want

Both can be submitted together, and the system generates one combined embedding representing the meaning of both inputs.

This allows searches that were previously impossible using single-modality tools.

Easy Integration for Developers

Another surprising detail is how quickly developers can start using it.

Gemini Embedding 2 launched with support for popular AI development frameworks, including:

LangChain
LlamaIndex
ChromaDB
QDrant

Because many AI applications are already built on these frameworks, developers can integrate the model without building a new infrastructure from scratch.

It’s available through:

Google AI Studio (free tier for experimentation)
Vertex AI (enterprise deployment)

Why This Matters for Virtual Assistants and Content Managers

Think about the kinds of content many clients manage.

A podcast brand might have:

Audio episodes
Show notes
PDFs
Promotional images

A course creator may have:

Video lessons
Slide decks
Written summaries

A consultant might maintain:

Recorded calls
Presentations
Research reports

Searching across all of that in a single step has been extremely difficult.

With models like Gemini Embedding 2, developers can build search tools where one query instantly returns:

the right video segment
the correct slide
the relevant document section

All from one search bar.

The Bigger Picture

You probably won’t interact with Gemini Embedding 2 directly.

Instead, it will power the next generation of search tools used in:

knowledge management systems
research databases
course platforms
internal company search tools

But knowing that technology like this exists helps you understand what’s becoming possible.

That knowledge can make a big difference when clients start asking about AI-powered search, automation, or content organization systems.

If you manage content libraries, research archives, or client knowledge bases, this is a technology worth paying attention to.

The tools many teams will rely on in the near future are already being built on models like this.

Google Just Had a Huge Week — Nano Banana 2, Opal's Agent Step and a New Developer Ecosystem for ADK

2026-02-28T18:19:00.003+08:00

There's been a lot happening at Google lately, and honestly, the updates from this past week alone are worth talking about. In just a few days, Google dropped a new default image model, upgraded their Opal workflow tool with real AI agent capabilities, and announced a brand-new integrations ecosystem for developers building AI agents. That's a lot to unpack — and I've been digging into all three.

I've been paying close attention to how Google is quietly (and sometimes not so quietly) building out their AI stack. These three updates feel connected in a bigger way. They're all pushing toward the same idea: faster, smarter, more autonomous AI tools that don't require you to be an engineer to use — or that supercharge you if you are one.

Nano Banana 2 — Pro-Quality Images at Flash Speed

One of the things that's always frustrated me about AI image generation is the trade-off. You either get fast and mediocre, or slow and beautiful. Nano Banana 2 — Google's new default image model built on Gemini 3.1 Flash Image — is trying to change that entirely.

What makes this interesting is the combination it's pulling off. It's generating at Flash speed while delivering what Google is calling Pro-level quality. To put that in practical terms: you're not waiting around for results, but you're also not getting blurry, inconsistent images. And the resolution range is wide — from 512px all the way up to 4K. That means the same model works whether you're making a quick thumbnail or something that needs to look polished and production-ready.

Here's the detail that really caught my attention: it can consistently handle up to 5 characters and 14 objects in a single image. If you've ever tried to generate a scene with multiple people or items and watched AI completely fall apart — characters blending together, objects disappearing — you know why this matters. Consistency across complex compositions has been a real weak spot in image generation, so this feels like a genuine step forward.

Nano Banana 2 is rolling out across the Gemini app and Google Workspace, and it's also available for enterprise use through Google Cloud. It's already the new default — which means if you're using Gemini for image generation, you're already on it.

Google Labs Opal Gets an Agent Step — Workflows Just Got Smarter

I've been curious about Opal since Google Labs introduced it as a workflow builder. The idea is that you set up a series of steps — like a recipe — and Opal runs through them to help you create something, whether that's a video, a piece of content, or a research brief. It's been useful, but the steps were static. You'd set it up once and it would just follow the same path every time.

The new agent step changes that completely. Instead of a fixed sequence, Opal now has a step where an actual AI agent takes over — it understands your goal, picks the right tools (like Veo for video generation, or web search for research), manages memory across the workflow, and routes dynamically based on what's needed. Think of it like the difference between following a printed map and having a navigation app that can reroute you in real time when things change.

This is part of a bigger shift we're seeing across the AI space, where "agentic" capabilities — AI that can reason, decide, and act rather than just respond — are becoming the new baseline. Google adding this to Opal means even people without a coding background can now build workflows that genuinely adapt. You don't have to anticipate every scenario upfront; the agent figures it out.

You can try Opal right now at opal.google — the agent step is available for all users.

Google ADK's New Integrations Ecosystem — Big News for Developers

This one is more developer-focused, but it's worth knowing about even if you're not building apps yourself. Google announced a new integrations ecosystem for their Agent Development Kit, or ADK — which is the framework developers use to build AI agents.

The idea is to make it easier for developers to connect their AI agents with external tools and services, so agents can actually do useful things in the real world instead of just talking about them. It's similar to how apps on your phone connect to different services — except here, we're talking about AI agents that can go off and complete tasks on your behalf.

What I find fascinating is how this fits into the broader picture. Between Nano Banana 2 (powerful image generation, accessible to everyone), Opal's agent step (autonomous workflows without code), and the ADK ecosystem (tools for developers to build custom agents), Google is building out every layer of the stack at once. There's something for the casual user, the content creator, and the professional developer — all in the same week.

AI Is Finally Fighting Back — And Anthropic Just Made It Official

2026-02-21T19:51:00.002+08:00

I've been watching the cybersecurity space for a while now, and I have to be honest — it's one of those areas that used to feel completely out of reach for someone like me. No coding background, no deep technical knowledge of exploits or patches. Just a person who's curious about what AI can actually do in the real world.

But here's the thing I kept noticing over the years: the good guys were always playing catch-up.

Think back to how security worked — and honestly, how it still works for most teams. You'd have a tool scan your code, it would match against a list of known bad patterns, and spit out a report. The problem? The sneaky stuff, the subtle logic flaws, the vulnerabilities that had been hiding in open-source code for decades — those never showed up. Because rule-based tools can't reason. They can only recognize what they've already been told to look for.

Meanwhile, attackers got smarter. And faster.

That gap — between what automated tools could catch and what skilled human researchers could catch — was always the weak point. And there just aren't enough human security researchers to close it. That's not a criticism, that's just math. The attack surface keeps growing. The backlogs keep piling up.

This is why what Anthropic announced on February 20, 2026 actually stopped me mid-scroll.

Claude Code Security is now in limited research preview, and what it does is genuinely different from what I'd seen before. Instead of scanning for known patterns, Claude reads your code the way a human security researcher would — tracing how data moves, understanding how different parts of an application talk to each other, and catching the complex, context-dependent vulnerabilities that traditional tools walk right past.

What really got me is the verification layer. Claude doesn't just flag something and move on. It goes back and tries to disprove its own findings, filtering out false positives before anything reaches a developer. Every validated finding comes with a severity rating and a confidence score, so teams know what to prioritize. And nothing gets applied automatically — a human always has to approve the fix. I love that. It's AI as a sharp, tireless assistant, not a rogue decision-maker.

But here's the connection I keep thinking about: Anthropic's Frontier Red Team has been quietly building toward this for over a year. They entered Claude in cybersecurity competitions. They partnered with the Pacific Northwest National Laboratory to test AI on critical infrastructure defense. They used Claude to review their own internal code. This wasn't a product announcement that came from nowhere — it's the result of real, careful work testing what Claude could actually do before putting it in the hands of others.

And the results of that work? Using Claude Opus 4.6, their team found over 500 vulnerabilities in production open-source codebases. Bugs that had survived years of expert human review, undetected.

That's the part that really lands for me. These weren't theoretical vulnerabilities. They were sitting in real code, in real projects that real people depend on — sometimes for decades.

The reason I find this so meaningful isn't just the technology. It's the timing and the intent. Anthropic is releasing this in a limited preview specifically because the same capabilities that help defenders could help attackers. They're being deliberate about who gets access first — Enterprise and Team customers, plus open-source maintainers who can apply for free expedited access. They're working with the community to get this right before it scales.

That's a different posture than "ship it and see what happens."

We're at a point where AI is going to scan a significant share of the world's code — that's not speculation anymore, it's the direction things are clearly heading. The question has always been who benefits from that first. Attackers who use AI to find weaknesses faster? Or defenders who use it to find and patch those same weaknesses before they're exploited?

Claude Code Security is Anthropic's answer to that question.

Gemini 3.1 Pro Changed How I Actually Use AI — And It Has Nothing to Do With Benchmarks

2026-02-20T17:00:00.001+08:00

I've been using AI tools almost every day for the past couple of years now, and if there's one thing I've learned, it's that a smarter model doesn't automatically mean you're getting smarter results. How you use the model matters just as much as what the model can do. And with Google's Gemini 3.1 Pro, there's something that I think is being really underrated in all the release coverage — the thinking level system, and what it actually means for the way you work.

This isn't a conversation about whether 77% on ARC-AGI beats 31%. It's about something more practical: the moment you realize you've been using these tools wrong, and how this new release hands you a dial you probably didn't know you needed.

What Even Is a "Thinking Level"?

Here's the quick version for anyone who hasn't come across this yet. Gemini 3.1 Pro lets you set how much thinking the model does before it responds. With the previous Gemini 3 Pro, you had two choices: low or high. With 3.1 Pro, there are now three — low, medium, and high.

Think of it like choosing between a quick gut reaction, a considered opinion, and a deep research session. The model isn't just choosing between "fast" and "slow" — it's choosing how much internal reasoning to do before it gives you an answer. At high, the model essentially behaves like a mini version of Gemini Deep Think, which is Google's most powerful reasoning model. That's a significant thing to have access to in a general-purpose assistant.

What surprised me when I started playing around with this is how much the choice actually matters. For a tricky math problem, setting thinking to high produced the right answer after several minutes of work. Setting it to low gave a fast but wrong answer. Same prompt, same model, completely different outcome.

The Problem Nobody Is Talking About

Here's what I find really fascinating about this. Most people who use AI tools have never really thought about what mode they should be in for a given task. We all tend to just fire off a prompt and expect the model to figure it out. But the thinking level system kind of forces you to be intentional, and that intentionality is where the real upgrade lives.

I started thinking about all the times I've used AI for tasks that fell into two completely different buckets. There's the stuff I need quickly — drafting a quick reply, summarizing a short article, brainstorming a list of ideas, generating a social post. And then there's the stuff where I actually want the model to sit with a problem — writing a full script outline, analyzing something complex, working through a nuanced question. Those two categories have always existed. What's new is that now I have a setting that actually reflects that difference.

Before 3.1 Pro, running everything at the same compute level was a bit like always driving in the same gear. Sometimes it worked. Sometimes it didn't. Now there's a gearshift.

How This Actually Changes My Workflow

When I started being intentional about thinking levels, a few things shifted for me pretty quickly.

For anything where I need a fast creative spark — like coming up with a hook for a video, finding synonyms, or doing a quick rewrite of a sentence — low thinking is more than enough. It's snappy, it's responsive, and frankly it's exactly what you want when you're in flow and don't want to wait. Speed matters there.

For medium tasks — things like drafting a structured outline, explaining a concept clearly, or building a content calendar — medium thinking has become my go-to. It takes a little longer, but the output feels more considered. Less surface-level. Like the model actually thought about the structure before it started writing.

And then there's high. I've started reserving high thinking for the things that actually deserve it. Complex analysis, tricky research questions, anything where getting the answer wrong would cost me time. The wait is longer — we're talking several minutes in some cases — but the quality of what comes back is on a different level. It's not just more text. It's more thoughtful text.

Why This Matters Even If You're Not Technical

I know a lot of people who use AI tools but feel like they're not getting as much out of them as they should. And honestly, after thinking about this thinking level system, I wonder if part of that frustration is just a mismatch between the task and the mode.

If you've ever asked an AI a complicated question and gotten a shallow answer, it might not be a model quality problem. It might be a compute budget problem. The model didn't spend enough time thinking. And now, for the first time, you have direct control over that.

That's actually a pretty big shift. Instead of just hoping the model figures out when to try harder, you get to tell it. It puts a little more responsibility on the user, sure. But it also puts a lot more power in your hands.

The Bigger Picture

What I keep coming back to is this: Gemini 3.1 Pro isn't just a smarter model. It's a model that respects the fact that not every question deserves the same amount of effort. And it's the first time I've felt like a general-purpose AI assistant is actually designed around how I naturally work — some things fast, some things slow, some things in between.

The AI tools that stick around aren't always the ones with the highest benchmark numbers. They're the ones that fit into how people actually think and work. This thinking level system feels like a step in that direction — and it's one I don't think enough people are paying attention to yet.

Grok 4.20: What Nobody Is Actually Talking About

2026-02-20T14:22:00.004+08:00

There's been a lot of noise about Grok 4.20 this week. The four agents, the multi-agent setup, the "AI brains arguing with each other" framing. I get it — it's a fun story. But the more I dug into this release, the more I felt like everyone was covering the surface and missing the stuff underneath that's actually worth paying attention to.

So here's what caught my eye.

The model solved a real math problem nobody had solved before

I came across this one almost by accident and it stopped me completely.

Paata Ivanisvili, a math professor at UC Irvine, had been working on an open problem in harmonic analysis — trying to find the exact maximum of something called a Bellman function. He and his student had already published a paper on it. They had a partial answer. The problem was still technically unsolved.

He got early access to the Grok 4.20 beta, fed it the problem, and within five minutes it handed him an explicit formula — a cleaner, sharper result than what he and his student had managed. His reaction on X was basically just: "Wow."

What makes this different from the usual "AI is smart" story is that benchmarks test problems with known answers. This didn't. Grok wasn't pulling a solution from somewhere — it was reasoning toward something that hadn't been written down yet. That's a genuinely different category of thing, and it happened because the Benjamin agent (the math and logic one) is specifically built for this kind of step-by-step proof reasoning.

One result doesn't prove everything. But it happened, it's documented, and it made me think differently about what "AI for research" actually looks like in practice.

It was already winning competitions before anyone knew it existed

Here's something that got buried in the launch coverage: Grok 4.20 didn't launch when the public beta dropped on February 17. It had been running — quietly, under fake names — for weeks before that.

On LMArena it showed up as "Theta-hat" and "Slateflow." On DesignArena it appeared as "Pearl" and "Obsidian." And in Alpha Arena's live AI stock trading competition, a "Mystery Model" was quietly the only entry making money while every other AI lost. That Mystery Model was Grok 4.20.

xAI's last official model announcement on their news page is still Grok 4.1 from November 2025. There was no blog post for this launch. No announcement thread. Just posts from employees on X and a new option appearing quietly in the grok.com dropdown. By the time most people heard about it, it had already been competing for weeks.

The "four brains" framing is wrong — and the real version is more interesting

Everyone's been describing this as four separate AI models arguing with each other. That's not quite what's happening, and I think the actual explanation is more impressive.

Grok 4.20 is one model — estimated around 3 trillion parameters — with four different roles running from the same base weights. Grok the coordinator, Harper the real-time researcher, Benjamin the math and logic brain, Lucas the contrarian wildcard. They're not four separate models. They're four ways of prompting the same model, each with different tool access and context framing.

Why does this matter? Cost. Running four truly separate models would be four times as expensive. Because these agents share weights and the same input context, xAI puts the overhead at 1.5x to 2.5x — not 4x. That's what makes this available at a consumer price point rather than Grok Heavy's $300/month tier. The agents don't have long debates either. The rounds are short, structured, trained through reinforcement learning to be efficient. They send targeted corrections and move on.

I found it interesting that nobody in the coverage this week mentioned this. "Four brains" is a better headline, sure, but one brain wearing four hats is actually a harder technical trick to pull off.

The trading competition result deserves more scrutiny than it got

Alpha Arena Season 1.5 was a live stock trading competition — real transactions, verified on the blockchain. GPT-5, Gemini, Claude, open-source models, Chinese models — everyone participated in multiple strategy variants over several weeks. When it ended, every single configuration of every other model finished in the red. Grok 4.20's four variants were the only ones in profit, with the best posting around 35% returns from a $10,000 start.

A few people mentioned this. Nobody really asked why.

My read: it wasn't that Grok out-thought everyone on finance. It's that Harper — the agent that pulls from X's real-time feed of roughly 68 million English tweets per day — gave it a live information edge the other models didn't have. The competition fed other models summarized news prompts. Grok was processing raw market sentiment as it happened.

That's worth being honest about, because it means the result says more about data access than raw intelligence. But here's the thing — that data access is the architecture. That's what the Harper agent was built to do. So the win was real, and it was structural, and it came from a deliberate design choice.

Grok 4.20 is the proof of concept. Grok 5 is the actual bet.

While everyone's been testing the 4.20 beta, xAI is already training its successor. Grok 5 is reportedly sitting around 6 trillion parameters — roughly double the size of 4.20 — with a target launch somewhere between April and June 2026.

For context, Grok 4.20 arrived in a crowded week. OpenAI shipped GPT-5.3-Codex on February 5. Anthropic released Claude Sonnet 5 on February 3. By that measure, this was a delayed point release catching up while competitors were already moving to the next generation. Even xAI treated it like a soft launch.

Grok 5 will not be a soft launch. xAI is spending roughly $1 billion a month. SpaceX formally acquired them weeks ago. An IPO is on the table. A 6-trillion-parameter release is exactly the kind of moment you build that story around.

What I keep thinking about is that the ecosystem is already being built before most people have noticed. Perplexity is reportedly running Grok 4.20 under the hood for a new search mode. There are signs of something called Grok Build — a coding tool with up to 8 parallel agents. The multi-agent architecture isn't a feature. It's the foundation, and xAI is building on top of it fast.

Grok 4.20 showed the approach works. The math problem, the trading competition, the benchmark trajectory — all of it points in the same direction. Grok 5 is where we find out if xAI can actually deliver on the scale of that promise.

I'll have a full video on this on YouTube soon — I want to actually get hands-on time with it first before I say anything definitive. But if you're following AI closely, this release is worth paying more attention to than the "four agents" headline suggests.

Claude Sonnet 4.6 Is Here — And There's Way More Going On Than the Benchmarks Show

2026-02-19T22:11:00.001+08:00

There's been a lot of buzz this week about Claude Sonnet 4.6, Anthropic's latest model release. And honestly? Most of the coverage I've seen is stopping at the surface level — benchmarks, pricing, computer use scores. All of that is interesting, sure. But when I started looking closer at what actually shipped alongside this model, I found four things that nobody's really talking about that I think matter a lot more for people who are actually using Claude day to day.

That 1 Million Token Context Window Has a Catch

Okay, so everyone is excited about the 1 million token context window. And I get it — the number sounds incredible. A million tokens is enough to hold entire codebases, thousands of documents, months of conversations. But here's what I didn't realize until I looked into this more carefully: a big context window doesn't automatically mean a useful context window.

There's a real issue in AI called context rot. The basic idea is that as you keep filling up a model's memory during a conversation, its ability to actually use older information starts to degrade. It's a bit like trying to hold too many things in your head at once — the stuff from earlier starts slipping away. Anthropic's previous model, Sonnet 4.5, scored just 18.5% on a benchmark called MRCR v2, which specifically tests whether a model can find information buried deep in a long context. That's... not great.

Sonnet 4.6 is genuinely better at this. But there are two things I think people need to know before they get too excited. First, the 1 million token window is still in beta — it's not just available to everyone right now, and you need to explicitly turn it on. Second, and this one surprised me, the pricing changes dramatically once you cross 200,000 tokens. Below that threshold you pay the standard rate. Above it, the pricing structure shifts significantly and applies to your entire request, not just the extra tokens. So if you're planning to throw huge amounts of text at this model, make sure you understand what that's going to cost before you build a workflow around it.

Here's the Part That Actually Impressed Me

Now here's where it gets interesting, because Anthropic didn't just train a better model and call it a day. They shipped two features alongside Sonnet 4.6 that directly address the context rot problem — and almost nobody is talking about them.

The first is dynamic web search filtering. When Claude searches the web, it now writes and runs its own code to filter search results before loading them into context. Instead of pulling in a full HTML page and reasoning over all of it — ads, navigation menus, irrelevant paragraphs and everything — it strips out the noise first and only keeps what's relevant. Anthropic tested this across two benchmarks and found it improved accuracy by an average of 11% while actually cutting input tokens by 24%. That's a really meaningful result. Quora's Poe platform tested it against all the major frontier models and said Opus 4.6 with this feature "achieved the highest accuracy on our internal evals" — specifically because it approaches research the way a human researcher would, filtering information programmatically instead of reasoning over raw noise.

The second is programmatic tool calling, which just hit general availability. This one is more for developers, but the idea is fascinating. When Claude needs to call multiple tools — say, querying a database five times to compare regions, then sorting and summarizing the results — it can now write code that does all of that inside a sandboxed container without bringing each intermediate result back into the conversation context. The result only shows up once everything is done. Think of it like doing all your rough math on a scratch pad before writing the final answer on the paper — the scratch work never clutters the main context.

Together, these two features tell a really clear story: Anthropic's answer to context rot isn't just a bigger bucket. It's smarter filtering so less noise goes in.

Should You Actually Cancel Opus?

This is the question I keep seeing pop up and I wanted to think through it properly. Sonnet 4.6 costs $3 per million input tokens and $15 per million output tokens. Opus 4.6 is roughly 1.7x more expensive on input and output. For most individual users that difference feels abstract — but if you're building on the API and processing a lot of requests, it compounds fast.

What made me look at this differently is where Sonnet 4.6 actually beats Opus. On agentic financial analysis — think tasks like researching stock data, calculating ratios, pulling together a market summary — Sonnet 4.6 scored 63.3% versus Opus 4.6's 60.1%. On office tasks, Sonnet leads again. On computer use benchmarks, the gap is almost nothing: 72.5% for Sonnet versus 72.7% for Opus. For everyday knowledge work, Sonnet 4.6 is genuinely at the same level for a meaningfully lower price.

The places where Opus still wins are specific: novel problem-solving, deep codebase work, complex situations where getting it exactly right is more important than getting it fast and cheap. Anthropic themselves describe Opus as the right choice for "codebase refactoring, coordinating multiple agents in a workflow, and problems where getting it just right is paramount." That's a real distinction — but it's a narrower one than it used to be.

And here's one more thing that matters for API developers. Because of programmatic tool calling, tool results from multi-step workflows don't count toward your token usage at all — only the final output does. So if you have workflows that currently make eight or ten tool calls in sequence, each one loading results back into context, you may be spending significantly more than you need to. That changes the cost math even further in Sonnet's favor for the right use cases.

The Security Problem Nobody Mentioned

I want to talk about prompt injection because every video I watched mentioned it in one sentence and moved on. I think it deserves more attention than that — especially now that Claude has computer use and can take real actions on your behalf.

Here's what prompt injection actually means in practice. Imagine you ask Claude to read through your emails and schedule any meeting requests it finds. One of those emails was crafted by someone who knew an AI would read it. Inside that email, hidden in the text, are instructions that tell Claude to forward any email with the word "confidential" to an external address before it drafts your replies. You'd never see it happen. That's the attack. Anthropic has described exactly this scenario in their own research on browser-agent security, and it's not hypothetical — it's an active concern for anyone building or using agentic AI.

Anthropic says Sonnet 4.6 is significantly better at detecting and resisting these attacks compared to its predecessor, and their evaluations show it performs similarly to Opus 4.6 in this area. That's meaningful progress. But independent testing on earlier Claude models found that while the model actively resists simple injection attempts, it can still be confused when the malicious instructions are buried inside what looks like a legitimate document or data structure.

What I didn't see anyone mention is a second security issue that comes with programmatic tool calling. When Claude calls your tools programmatically and gets results back, those results come back as raw strings — and those strings can contain anything, including code snippets that might get processed by the execution environment. If your tools are pulling data from external sources or user inputs, there's a real code injection risk if you're not validating what comes back before acting on it. This is separate from prompt injection — it's a layer deeper, and it's something every developer building agentic workflows needs to think about before shipping.

The honest summary is this: Sonnet 4.6 is more secure than what came before. But "more secure" and "fully solved" are very different things. The more autonomous you make your agents, the more carefully you need to think about what they can be tricked into doing.

What This All Means

I find the Sonnet 4.6 release genuinely exciting — not just because of the model improvements, but because of what the surrounding features tell us about where Anthropic is heading. They're building a system where Claude reasons over less noise, not more. Dynamic filtering, programmatic tool calling, context compaction, memory tools — these are all solving the same underlying problem. And the fact that accuracy went up while token usage went down on web search benchmarks suggests this approach is actually working.

If you're a knowledge worker using Claude through the app, the takeaway is pretty straightforward: this model is fast, capable, and meaningfully better at the kind of office work most people actually do. If you're a developer building on the API, there are real architectural decisions to make now — about when to use Sonnet versus Opus, how to take advantage of programmatic tool calling, and how to think about security in agentic workflows.

What I'm watching next is how the 1 million token context window performs in real production use cases over the next few weeks. The beta label is still on it for a reason. But the direction is clear — and it's more interesting than most of the coverage I've seen this week.

Sources: Anthropic Claude Sonnet 4.6 announcement · Dynamic Web Search Filtering · Programmatic Tool Calling Docs

Google's Gemini 3 Deep Think Just Got a Major Upgrade — And It's Designed for Real Science

2026-02-13T11:19:00.005+08:00

There's been this interesting trend in AI lately where models are getting better at reasoning through complex problems. We've seen it with OpenAI's o1 and o3, DeepSeek's R1, and now Google is making a serious push into this space with a major update to Gemini 3 Deep Think.

What makes this update different from the reasoning models we've seen before is that Google specifically built it for scientists, researchers, and engineers working on real-world problems. This isn't just about solving math competitions anymore—though it still does that incredibly well. It's about tackling messy, incomplete data and problems without clear solutions, which is what actual research looks like.

I've been following the development of reasoning models closely, and Deep Think's focus on practical scientific applications is a shift I find really interesting. This is part of a larger movement where AI is moving from being a general-purpose tool to something more specialized for specific domains.

Google AI Ultra subscribers can access the updated Deep Think in the Gemini app starting today, and for the first time, researchers and enterprises can apply for early access to use it via the Gemini API.

Why Deep Think Focuses on Science and Engineering

The way Google approached this update is pretty clever. Instead of just making a model that's good at abstract reasoning, they worked directly with scientists and researchers to understand what kinds of problems they actually face in their work.

Real research isn't like solving textbook problems. You're dealing with incomplete data, messy information, and questions that don't have single right answers. Traditional AI models often struggle with this kind of ambiguity, but Deep Think was specifically trained to handle it.

What caught my attention in the announcement were the real-world examples. A mathematician at Rutgers University used Deep Think to review a highly technical paper and it found a logical flaw that had passed through human peer review. At Duke University, researchers used it to optimize crystal growth methods for semiconductor materials, hitting a precise target that previous methods couldn't achieve.

These aren't just impressive demos—they're solving actual research bottlenecks.

The Numbers Are Genuinely Impressive

Deep Think continues to push what's possible on academic benchmarks. It scored 48.4% on Humanity's Last Exam, a benchmark specifically designed to test the limits of frontier models. That's without using any external tools, just pure reasoning.

It also achieved 84.6% on ARC-AGI-2, which tests abstract reasoning abilities that supposedly indicate progress toward artificial general intelligence. The ARC Prize Foundation verified this result, which gives it more credibility.

On Codeforces, a competitive programming platform, Deep Think reached an Elo rating of 3455. To put that in perspective, that's gold medal territory at international programming competitions.

The really interesting part is that Deep Think now also excels at chemistry and physics olympiad problems, achieving gold medal-level performance on both. It scored 50.5% on CMT-Benchmark, which tests advanced theoretical physics understanding.

Built for Practical Engineering Applications

Beyond benchmarks, what makes Deep Think stand out is how it's being used in practice. Google designed it to interpret complex data and model physical systems through code, which means engineers can actually use it for real work.

One example they showed is turning a sketch into a 3D-printable file. You draw something, Deep Think analyzes it, models the complex shape, and generates a file ready for 3D printing. That's the kind of practical application that makes this more than just an impressive reasoning model—it's a tool people can actually use.

Google's also making this available through the Gemini API for researchers and enterprises, which is significant. Previous versions of Deep Think were mostly limited to the consumer app, but opening it up via API means developers can integrate it into their own workflows and tools.

What This Means for AI Reasoning Models

This release is part of a broader competition happening right now in the reasoning model space. OpenAI has o1 and o3, DeepSeek released R1, Anthropic has been working on extended thinking capabilities, and now Google is pushing hard with Deep Think.

What's interesting is how these companies are differentiating their approaches. OpenAI focuses on general reasoning, DeepSeek emphasizes efficiency and open-source access, and Google is positioning Deep Think as the model for scientific and engineering work.

The practical difference here is that Deep Think isn't trying to be everything to everyone. It's specialized for domains where deep reasoning through complex, messy problems actually matters—research, engineering, advanced mathematics, theoretical physics.

For anyone working in these fields, having a model that understands the nuances of scientific work rather than just being good at logic puzzles could be genuinely transformative.

The fact that Google worked directly with scientists to build this, and that early testers are already finding real research applications, suggests this is more than just benchmark chasing. It's an attempt to make AI actually useful for advancing human knowledge in concrete ways.

If you're a researcher, engineer, or working in a technical field, Deep Think might be worth keeping an eye on—especially if you can get into the early access program for the API. This could be one of those tools that changes how certain kinds of work get done.

Google Chrome's WebMCP is About to Change How AI Agents Browse the Web

2026-02-13T10:55:00.005+08:00

There's been this ongoing challenge with AI agents: when they visit a website, they're basically tourists who don't speak the language. Whether you're using LangChain, Claude Code, or tools like OpenClaw, your agent is stuck guessing which buttons to press, scraping HTML, or processing thousands of tokens worth of screenshots just to figure out what's on a page. If you've been building with agents for a while, you know exactly how painful this is.

That's what makes Google Chrome's new WebMCP preview so interesting. Earlier this week, the Chrome team shipped an early version of what could be the most important change to how agents interact with the web in years. Instead of treating every website like a foreign language that needs translation, WebMCP lets websites expose structured tools directly to AI agents. No more scraping. No more processing endless screenshots. Your agent just calls functions.

This is part of a bigger shift we're seeing where the web itself is becoming more agent-friendly, not just more human-friendly. And honestly, it's about time.

I've been following the development of browser-based agents, and WebMCP caught my attention because it solves problems most people aren't even talking about yet. Watch my YouTube video on it below.

WebMCP Begins Rollout

Why Current Web Interaction Is So Inefficient

Right now, agents interact with websites in two main ways. The first is through screenshots—you take an image of the page, feed it to a multimodal model, and hope it can identify buttons, form fields, and interactive elements. The problem? You're burning through thousands of tokens for every single image you process.

The second approach is accessing the DOM directly and parsing raw HTML and JavaScript code. While this uses fewer tokens than images, you're still translating from one language to another. The agent has to sift through paragraph tags, CSS styling, and all sorts of presentation markup that doesn't actually matter for understanding what actions it can take.

Both methods feel like working through a translator when you could just speak the same language.

How WebMCP Actually Works

The idea behind WebMCP is beautifully simple: let each webpage act like an MCP server that agents can query directly. The page basically tells the agent, "Here's what you can read. Here's what you can click. Here's what you can fill in."

This isn't entirely new—academics and companies have been proposing versions of this for a while. But in the second half of last year, Microsoft and Google actually got together to build a real spec for how this would work. The timing makes sense too—this was right around when we saw Perplexity release Comet and OpenAI release Atlas, when web interaction was clearly heating up.

What makes Chrome's approach interesting is that it's designed for human-in-the-loop workflows first. The agent works with the user, not just autonomously. So normal people still use websites normally, but agents can help speed things up and improve the experience.

Google presented three core pillars at the Web AI Summit: context (understanding what the user is doing beyond just the current screen), capabilities (taking actions on the user's behalf), and coordination (managing the handoff between agent and user when needed).

The Two APIs You Need to Know

Chrome has structured WebMCP around two main APIs. The Declarative API handles standard actions—think HTML forms with added tool names and descriptions. If you've already got well-structured forms on your site, you're apparently about 80% of the way there.

The Imperative API is for more complex, dynamic interactions that require JavaScript execution. This is where you'd define custom tools, similar to how you'd structure function calls for OpenAI or Anthropic's API endpoints.

The practical difference here is huge. Instead of dozens of interactions clicking through filters and scrolling pages, a single tool call could return structured results. Imagine your agent calling a "search products" function and getting back organized data instead of trying to parse a visual search interface.

What This Means Going Forward

While WebMCP is still behind a flag in Chrome, it's already in the browser. This isn't a theoretical spec anymore—it's actually happening. Google will likely roll this out fully at Google Cloud Next or Google IO in the coming months, and I expect things to move quickly from there.

We'll probably see tools and maybe even Claude skills that help convert existing websites to expose their own WebMCPs. For anyone building AI agents or websites that want agents to use them, this is definitely something to have on your radar.

The shift from agents guessing their way through the web to websites speaking the agent's language directly? That's the kind of change that makes everything else possible.

Claude Opus 4.6 Fast Mode — Up to 2.5x Faster Responses at Premium Pricing

2026-02-08T11:50:00.001+08:00

Anthropic launched Fast Mode for Claude Opus 4.6 in research preview. The feature delivers up to 2.5x higher output tokens per second from the same model at a higher cost per token.

Fast Mode is available now in Claude Code for users with extra usage enabled and through a waitlist for API access. The feature is also rolling out to GitHub Copilot, Cursor, and other platforms.

How Fast Mode Works

Fast Mode is not a different model. It uses the same Opus 4.6 with a different API configuration that prioritizes speed over cost efficiency. You get identical quality and capabilities, just faster responses.

The speed improvement focuses on output tokens per second, not time to first token. The same model weights and behavior remain unchanged.

Accessing Fast Mode

In Claude Code, toggle Fast Mode on or off by typing /fast in the CLI or VS Code extension. You can also enable it in your user settings file by setting "fastMode": true. Fast Mode persists across sessions.

When enabled, Claude Code automatically switches to Opus 4.6 if you're on a different model. A small ↯ icon appears next to the prompt while Fast Mode is active.

For API users, set speed: "fast" in your API request to enable Fast Mode. The feature is currently in limited research preview with waitlist access.

Pricing and Availability

Fast Mode pricing starts at $30 per million input tokens and $150 per million output tokens. This is 6x the standard Opus pricing of $5 per million input and $25 per million output.

A 50% discount is available for all plans until February 16, 2026, bringing the cost to 3x standard pricing during the discount period.

Fast Mode usage is billed directly to extra usage, even if you have remaining usage on your plan. Fast Mode tokens do not count against your plan's included usage.

Requirements and Limitations

Fast Mode requires extra usage enabled on your account. For individual accounts, enable this in Console billing settings. For Teams and Enterprise, an admin must enable both extra usage and Fast Mode for the organization.

Fast Mode is not available on third-party cloud providers including Amazon Bedrock, Google Vertex AI, or Microsoft Azure Foundry. It's only available through the Anthropic Console API and Claude subscription plans using extra usage.

Fast Mode has separate rate limits from standard Opus 4.6. When you hit the rate limit or run out of extra usage credits, Fast Mode automatically falls back to standard Opus 4.6.

When to Use Fast Mode

Fast Mode works best for interactive workflows where speed matters more than cost. Use it for rapid iteration, live debugging, or real-time agent interactions.

Toggle it off when cost efficiency is more important than latency. You can combine Fast Mode with lower effort levels for maximum speed on straightforward tasks.

For API users, note that switching between fast and standard speed invalidates the prompt cache. Requests at different speeds do not share cached prefixes.

Google DeepMind's Evo-Memory Redefines AI Agent Memory — Cutting Task Steps by 50% Without Retraining

2026-02-05T16:47:00.002+08:00

The gap between how AI agents remember and how they actually learn from experience has long been a fundamental limitation. While chatbots can recall what you said in a previous conversation, they typically can't leverage that experience to solve similar problems faster or smarter. A new research collaboration between Google DeepMind and the University of Illinois Urbana-Champaign proposes a solution: "Test-Time Evolution" — where agents actively Search, Synthesize, and Evolve their memory after every interaction.

This isn't just another benchmark paper. Evo-Memory introduces a comprehensive streaming evaluation framework alongside ReMem, an action-think-memory refine pipeline that fundamentally changes how we think about agent memory. The results are striking: active memory refinement reduced task completion steps by roughly 50% on ALFWorld (from 22.6 steps down to 11.5), and smaller models like Gemini Flash achieved gains that often rivaled larger static models. The success hinges not on storing more information, but on the agent's ability to refine and delete irrelevant experiences.

For anyone building AI agents, personal assistants, or autonomous systems, this research signals a shift in how we should approach memory architecture. Current RAG systems and long-context models excel at passive retrieval, but they don't learn from what worked and what didn't. Evo-Memory closes that gap by treating memory as something that evolves during deployment rather than remaining frozen after training.

The Core Problem: Remembering vs. Learning

The paper identifies a critical distinction that often gets overlooked. Current LLM memory systems focus on conversational recall — retrieving facts from dialogue history to answer queries. But this misses the more valuable capability of experience reuse, where agents abstract reasoning strategies from past tasks to improve future performance.

Think about it this way: if you ask a math tutor the same type of problem twice, they shouldn't solve it from scratch the second time. They should recognize the pattern and apply the successful strategy faster. Yet most AI agents today do exactly that — they recall context but fail to adapt across sessions. The researchers demonstrate this limitation persists even in sophisticated systems using retrieval-augmented generation, hierarchical memory, and workflow-based approaches.

The benchmark transforms static datasets into streaming task sequences, explicitly testing whether LLMs can accumulate knowledge and refine strategies during deployment. This reframing from isolated task evaluation to continuous adaptation assessment reveals significant weaknesses in current memory architectures.

ReMem: The Think-Act-Refine Loop

The proposed solution introduces a three-operation framework that goes beyond traditional ReAct-style agents. At each step, the agent chooses between Think (internal reasoning traces), Act (execute an operation or output a response), and Refine (meta-reasoning over memory to exploit useful experiences, prune noise, and reorganize stored knowledge).

This creates what the researchers describe as a Markov decision process where memory becomes an adaptive component that interacts with reasoning in real time rather than remaining passive context. The agent can loop between Think and Refine arbitrarily before committing to an action, forming a lightweight but powerful paradigm for continual adaptation.

A concrete example from the paper: when solving a household task like "put a hot apple in the fridge," the ReMem agent thinks about needing a heat source, searches memory for relevant experiences with microwaves, prunes an obsolete entry about stoves, executes the microwave action, then creates a new memory entry capturing the successful "hot→fridge = cooldown" strategy. This completed in 9 steps versus 19 for vanilla ReAct.

Benchmark Results That Challenge Assumptions

The research evaluated over ten representative memory modules across 10 diverse datasets spanning embodied reasoning (ALFWorld, BabyAI, PDDL, ScienceWorld) and single-turn tasks (AIME-24/25, GPQA, MMLU-Pro, ToolBench). The results reveal several important findings.

ReMem on Claude 3.7 Sonnet achieved 0.92 success rate and 0.96 progress on ALFWorld, 0.83 success and 0.95 progress on PDDL planning tasks. On Gemini 2.5 Flash, the average success reached 0.50 with 0.64 progress, consistently outperforming history baselines and ReAct-style approaches across all four multi-turn environments.

Perhaps most notably, the performance gains correlate strongly with task similarity within datasets. The researchers found a Pearson correlation of 0.72 on Gemini 2.5 Flash and 0.56 on Claude 3.7 Sonnet between ReMem's improvement margin and within-dataset coherence. Structured domains like PDDL and ALFWorld with higher intra-task similarity showed larger improvements, while diverse datasets like AIME-25 or GPQA showed smaller gains.

Step efficiency improvements proved equally significant. In ALFWorld, average steps to complete tasks dropped from 22.6 for history baselines to 11.5 for ReMem. ScienceWorld showed similar gains, going from 20.5 steps down to 14.0. The researchers note this represents a direct compute-cost win without any fine-tuning.

The Surprising Power of Simple Approaches

One unexpected finding deserves attention: ExpRAG, a simple retrieval-based baseline, outperformed several more complex designs. This baseline stores each task interaction as structured experience text and retrieves similar experiences for new tasks using basic embedding similarity.

Even ExpRecent, which simply maintains condensed traces of recent task trajectories, performed competitively. This suggests that explicit task-level utilization during test-time evolution represents a promising and underexplored direction, and that architectural complexity isn't always the answer.

The research also tested how agents handle both successful and failed experiences in memory. Baseline methods experienced clear performance drops when exposed to unfiltered failures, indicating that naive memory accumulation introduces noise. ReMem remained robust by actively refining stored experiences, achieving the highest overall success rates under both Claude and Gemini backbones when fed mixed feedback.

Why This Matters for AI Development

The implications extend beyond benchmark scores. Evo-Memory demonstrates that test-time evolution — the ability to retrieve, integrate, and update memory continuously during deployment — represents a viable path to more capable AI agents without additional training.

Smaller models particularly benefit from self-evolving memory, suggesting this approach could democratize access to more sophisticated agent capabilities. The correlation between task similarity and memory effectiveness provides practical guidance: domains with structured, recurring task patterns stand to gain the most from implementing these techniques.

For developers building production AI systems, the key insight is that memory architecture matters as much as model capability. Simply increasing context windows or adding retrieval doesn't capture the adaptive, self-improving behavior that humans naturally exhibit when learning from experience.

The researchers have indicated plans to release all code and configurations for reproducibility, making this a practical resource for the AI community rather than just a research contribution. As we move toward agents that operate autonomously over extended periods, the shift from static recall to dynamic evolution may prove foundational for the next generation of AI systems.

PaperBanana: The AI That's Automating Academic Illustration (And It's Kind of Mind-Blowing)

2026-02-05T13:37:00.002+08:00

If you've ever written a research paper, you know the pain: you've done the hard work, written thousands of words explaining your groundbreaking methodology, and then... you need to create diagrams. Beautiful, publication-ready diagrams that somehow capture your complex ideas in a single visual. For many researchers, this becomes the most time-consuming part of the entire process.

Enter PaperBanana, a revolutionary framework from researchers at Peking University and Google Cloud AI Research that's tackling this exact bottleneck. And yes, they named it PaperBanana because even serious AI research deserves a smile.

What Makes PaperBanana Special?

Think of PaperBanana as your personal illustration team, but instead of humans, it's five specialized AI agents working together. Each agent has a specific role: the Retriever finds relevant reference examples from existing papers, the Planner translates your research context into detailed visual descriptions, the Stylist ensures everything looks professionally polished, the Visualizer creates the actual diagrams, and the Critic reviews and refines the output until it meets publication standards.

This isn't just about slapping together some boxes and arrows. PaperBanana generates diagrams that are faithful to your research, concise enough to be readable, aesthetically pleasing, and sophisticated enough to appear in top-tier conferences like NeurIPS.

PaperBanana's architecture: Five specialized AI agents collaborate to transform research content into publication-ready illustrations.

The Secret Sauce: Reference-Driven Intelligence

What sets PaperBanana apart is its reference-driven approach. Instead of generating illustrations from scratch with no context, it learns from the visual language already established in academic publishing. The system analyzes methodology diagrams from recent NeurIPS papers, understanding not just what makes a diagram functional, but what makes it beautiful and publication-ready.

The results speak for themselves. In comprehensive testing against leading baselines, PaperBanana consistently outperformed competitors across all evaluation dimensions: faithfulness, conciseness, readability, and aesthetics. It's not just good—it's setting a new standard.

Beyond Methodology Diagrams

But here's where it gets even more interesting: PaperBanana doesn't just do methodology diagrams. It also generates high-quality statistical plots. The researchers tested both code-based and image generation approaches for creating visualizations, revealing fascinating trade-offs. Image generation creates more visually appealing plots, but code-based methods maintain better content fidelity. Understanding these nuances helps researchers choose the right approach for their needs.

The Benchmark That Changes Everything

To properly evaluate automated illustration generation, the team created PaperBananaBench—a rigorous benchmark comprising 292 test cases curated from NeurIPS 2025 publications. This benchmark captures the sophisticated aesthetics and diverse logical compositions of modern AI research, spanning multiple research domains and illustration styles.

The average source context contains over 3,000 words, proving that PaperBanana can handle the complexity of real research papers, not just simplified examples.

PaperBananaBench statistics showing 292 test cases with average source context of 3,020 words per diagram.

PaperBanana consistently outperforms baselines across all evaluation dimensions: faithfulness, conciseness, readability, and aesthetics.

Real-World Applications

The practical applications extend beyond just generating new diagrams. PaperBanana can enhance the aesthetics of existing human-drawn diagrams, applying automatically summarized style guidelines to elevate visual quality. Imagine taking a rough sketch and having it instantly transformed into a polished, publication-ready illustration that maintains your original intent while looking professionally designed.

Before and after: PaperBanana transforms verbose, outdated diagrams into concise, aesthetically modern illustrations while maintaining accuracy.

The Road Ahead

Of course, no system is perfect. The researchers openly acknowledge failure modes, particularly around connection errors in complex diagrams. But this transparency is refreshing—they're not claiming to have solved everything, just to have made a significant leap forward.

For AI researchers, content creators, and anyone involved in scientific communication, PaperBanana represents something bigger than just a tool. It's a glimpse into a future where the tedious parts of research communication are automated, freeing scientists to focus on what they do best: pushing the boundaries of knowledge.

The code is available on GitHub, the paper is on arXiv, and the framework is ready to explore. As AI continues to augment scientific workflows, tools like PaperBanana remind us that automation isn't about replacing human creativity—it's about amplifying it, one beautifully generated diagram at a time.

Qwen Just Dropped a Coding AI That Runs on Your Laptop — And It's Competing with Models 20x Larger

2026-02-04T15:14:00.001+08:00

Okay, I need to tell you about something that just happened in the AI coding world that has me genuinely excited. The Qwen team just released Qwen3-Coder-Next, and if you've been following the whole "local AI coding assistant" conversation, this one's a big deal.

Here's why: This model has 80 billion parameters but only uses 3 billion at a time. And somehow, it's matching the performance of models with 10-20x more active parameters. Yeah, you read that right.

Let me break down what this actually means for people like us who want powerful AI coding tools that don't require sending all our code to the cloud.

What Makes This Different from Other Coding Models?

Most coding AI models you've heard about—like GitHub Copilot, ChatGPT for coding, or Claude—run on massive cloud servers. They're great, but you're always dependent on an internet connection, you're sharing your code with a third party, and there are costs involved.

Qwen3-Coder-Next is built specifically for local development and coding agents. That means it's designed to run on your own machine (yes, even a beefy laptop or desktop), keep your code private, and work with tools like Claude Code, Cline, and other IDE integrations.

But here's where it gets interesting: unlike traditional models that need all their billions of parameters active to work well, Qwen3-Coder-Next uses something called a Mixture-of-Experts (MoE) architecture with sparse activation.

Think of it like having a team of 80 billion specialists, but for any given task, you only need to consult 3 billion of them. This makes it incredibly efficient—you get the intelligence of a huge model with the speed and memory requirements of a much smaller one.

The Architecture: Hybrid Attention That Actually Makes Sense

Now, I know "hybrid attention" and "sparse MoE" sound like buzzwords, but stick with me because this is actually pretty clever.

Traditional transformer models have a problem: as you give them more context (like a large codebase), the computational cost grows exponentially. It's called the "quadratic scaling problem," and it's why most models struggle when you try to feed them an entire repository's worth of code.

Qwen3-Coder-Next solves this by combining three different types of attention mechanisms:

Gated DeltaNet (for efficient linear attention)
Gated Attention (for focused reasoning)
Mixture-of-Experts layers (for specialized knowledge)

The model has 48 layers total, and the layout repeats a pattern: three DeltaNet-MoE blocks followed by one Attention-MoE block. Each MoE layer has 512 expert networks, but only 10 experts plus 1 shared expert activate for each token you process.

What this means practically: You can give this model a 256,000 token context window (that's roughly 200,000 words or a massive codebase), and it won't choke. It'll keep reasoning through your entire project without slowing to a crawl.

Trained Like an Actual Coding Agent, Not Just a Code Generator

Here's where Qwen3-Coder-Next really stands out from other coding models: how it was trained.

Most coding AI is trained on static code snippets—just reading code and learning patterns. Qwen3-Coder-Next went through what the team calls "agentic training at scale."

They created 800,000 executable coding tasks with real environments. These weren't simple "write a function" exercises. They were actual bug-fixing scenarios pulled from GitHub, complete with test suites, containerized environments, and the ability to execute code and see if it works.

During training, the model:

Receives a coding task
Writes code to solve it
Runs the code in a real environment
Gets feedback if it fails
Learns to recover from errors and try again

This is reinforcement learning applied to real-world coding workflows. The model learned to plan, use tools, run tests, and recover from failures—not just spit out code and hope for the best.

The team even trained specialized "expert models" for specific domains:

A Web Development Expert for full-stack UI work (tested by actually rendering pages in a browser)
A User Experience Expert for CLI tool interactions across different frameworks

This training approach is why Qwen3-Coder-Next excels at long-horizon coding tasks—the kind where you need to make multiple changes across several files, run tests, fix errors, and iterate until everything works.

The Benchmarks: Punching Way Above Its Weight Class

Let me show you where this gets really impressive. On SWE-Bench Verified (a benchmark that tests how well models can solve real GitHub issues), here's how Qwen3-Coder-Next compares:

Qwen3-Coder-Next (3B active params): 70.6%
DeepSeek-V3.2 (671B total params): 70.2%
GLM-4.7 (358B total params): 74.2%

So a model with only 3 billion active parameters is matching or beating models with hundreds of billions of active parameters. That's insane efficiency.

On SWE-Bench Pro (an even harder benchmark):

Qwen3-Coder-Next: 44.3%
DeepSeek-V3.2: 40.9%
GLM-4.7: 40.6%

And on Terminal-Bench 2.0 (testing CLI agent capabilities) and Aider (a coding assistant benchmark), it continues to perform at the level of much larger models.

The takeaway: You're getting elite coding assistant performance in a package that can actually run locally on consumer hardware.

You Can Actually Run This on Your Own Machine

This is where things get practical. The Qwen team didn't just release model weights and say "good luck." They've made this genuinely deployable for real people.

For Server Deployment:

Works with SGLang and vLLM (industry-standard inference engines)
Supports OpenAI-compatible API endpoints
Can handle the full 256K token context window
Requires multiple GPUs for full performance (2-4 GPUs with tensor parallelism)

For Local Deployment:

Unsloth provides GGUF quantizations (compressed versions)
4-bit quantization: Needs about 46 GB of RAM
8-bit quantization: Needs about 85 GB of RAM
Works with llama.cpp and llama-server
Compatible with Apple Silicon unified memory (yes, you can run this on an M-series Mac)

The Unsloth team has even created guides showing how to plug Qwen3-Coder-Next into frameworks that mimic OpenAI Codex and Claude Code, but running entirely on your local machine.

For most modern development machines—especially those with 64GB+ of unified memory or dedicated GPUs—this is totally feasible. You can have a production-grade coding assistant running locally.

What You Can Actually Do With It

Qwen3-Coder-Next isn't just for autocompleting code. It's designed for agentic workflows, meaning it can:

Understand entire codebases (thanks to the 256K context window)
Plan multi-step refactoring tasks across multiple files
Execute code and interpret results
Call external tools (linters, formatters, test runners, debuggers)
Recover from errors by analyzing stack traces and trying different approaches
Work with different IDE scaffolds (Claude Code, Qwen Code, Cline, Kilo, etc.)

It supports tool calling natively, meaning it can interact with your development environment like a junior developer would—running commands, reading outputs, and making decisions based on results.

One important note: This model does NOT use "thinking" mode (it doesn't generate <think> tags). It goes straight to action. This makes it more predictable for agent workflows where you want direct tool calls and responses.

Who Should Care About This?

If you're a developer who:

Works with sensitive codebases that can't go to cloud APIs
Wants a powerful coding assistant without monthly subscription fees
Has a decent development machine (64GB+ RAM or multiple GPUs)
Prefers open-source tools over proprietary solutions
Wants to experiment with AI coding agents

Then Qwen3-Coder-Next is worth checking out.

If you're a company that:

Needs coding assistance for proprietary code
Wants to avoid data leaving your infrastructure
Has the compute resources to host models locally or on-premises
Wants to customize the model for specific frameworks or languages

This is a compelling option under the Apache 2.0 license (meaning you can use it commercially without restrictions).

The Bigger Picture: Local AI Coding Assistants Are Here

What excites me most about Qwen3-Coder-Next isn't just the model itself—it's what it represents.

For the past couple of years, the best AI coding tools have been cloud-only. You had to use GitHub Copilot, Claude, or ChatGPT, all of which require sending your code to external servers.

But in just the past few weeks, we've seen an explosion of local coding assistants:

Anthropic's Claude Code
OpenAI's Codex app
Various open-source frameworks like OpenClaw
And now Qwen3-Coder-Next

The tech has matured to the point where you can have GPT-4-class coding assistance running entirely on your own hardware. For privacy-conscious developers and companies working on proprietary systems, this is huge.

How to Get Started

If you want to try Qwen3-Coder-Next:

Check the model weights on Hugging Face: Look for Qwen/Qwen3-Coder-Next
Read the technical report on GitHub: QwenLM/Qwen3-Coder
Follow the Unsloth guide for local deployment: unsloth.ai/docs/models/qwen3-coder-next
Try it with your favorite agent framework: Claude Code, Cline, or Qwen Code all support it

The setup isn't quite as simple as installing a VSCode extension (yet), but the documentation is solid, and the community is already building tooling around it.

Final Thoughts

I think we're at an inflection point with AI coding tools. The cloud-based options are still excellent and will continue to improve, but now we have legitimate alternatives that run locally, respect privacy, and don't require subscriptions.

Qwen3-Coder-Next proves that you don't need to activate hundreds of billions of parameters to get strong coding assistance. With clever architecture (sparse MoE with hybrid attention) and smart training (agentic training on executable tasks), you can build something powerful enough to rival the big proprietary models.

For me, this opens up possibilities for experimentation, customization, and building coding tools that work the way I want them to—without worrying about API costs or data privacy.

If you've been curious about local AI coding assistants, now's the time to dive in. The tech is finally here.

Claude’s new file creation tools vs. ChatGPT and Gemini: who’s ahead on real productivity

2025-09-12T20:00:00.001+08:00

What Claude offers now

From Anthropic’s announcements:

Creates and edits real files directly in chats or the desktop app: Excel (.xlsx), Word (.docx), PowerPoint (.pptx), PDFs.
Users can upload data or supply shared input, then ask Claude to build files from scratch (e.g. spreadsheets with formulas, documents or slide decks).
The outputs are downloadable, usable “ready-to-use” artifacts. Claude can also convert document formats (e.g. PDF→slides) and do statistical/analysis tasks within spreadsheets.
File size limits: up to 30 MB uploads/downloads.
The feature is currently a preview for certain paid plans (Max, Team, Enterprise), with Pro plans getting access “soon.”

What ChatGPT currently supports (vs. Claude)

Based on public info:

File uploads & summarization / extraction: ChatGPT can accept PDFs, presentations, plaintext documents, etc., and then respond to queries about their contents.
Data analysis / code execution environment (“Code Interpreter” / “Advanced Data Analysis”): for spreadsheets or CSVs, you can upload, have it run code, do charts/visualizations, clean data, etc.
File editing or direct file creation: ChatGPT so far does not create or modify Excel/Word/PPTX/PDF files as downloadable artifacts via a “create new file + edit” flow in chat (at least very broadly marketed). There are plugins and workflows, but not a core feature announced the way Claude’s was.
Canvas interface: ChatGPT introduced “Canvas,” allowing inline editing of texts or code alongside chat—helpful for refining, rewriting, collaborating. But this is about editing text/code drafts in the interface, not necessarily generating formal document files with formatting and exporting to PPTX, XLSX, etc.

What we know less about Gemini (Google)

Public info is less detailed for Gemini’s ability to generate downloadable files like PowerPoints, spreadsheets with formulas, etc. Gemini can export Deep Research reports as Google Docs, which implies some document generation + formatting functionality. But whether it handles real .xlsx spreadsheets or retains formula logic is less clear. (This comes from secondary sources referencing export of reports as Google Docs.)
Gemini does well with research-style reports, text generation, multimedia input/output; but direct file editing workflows (upload file, edit content, download in formatted artifact) are not obviously at parity with Claude’s newly announced capability as of now (publicly).

Side-by-side strengths & gaps

Feature	Claude’s new file creation/edit	ChatGPT’s current capacities	Google Gemini (publicly known)
Create + download Word / PPTX / Excel / PDF from scratch via chat	✅ Yes (Claude)	❓ Mostly no / limited; chat drafts or upload → extract, but not full artifact creation with formatting & formulas	✅ Some doc export (e.g. Google Doc), but file formats & formula support unclear
Edit existing files (spreadsheets, slide decks, PDFs) by specifying edits	✅ Yes (Claude can modify uploaded file)	Partial: you can ask ChatGPT to suggest edits, maybe produce updated content; but usually via text, not editing the actual file artifact internally	Less clear publicly
Formulas / spreadsheet logic, charts, data analysis within file	✅ Claude supports formulas, chart generation in Excel sheets etc.	✅ ChatGPT’s Advanced Data Analysis / Code Interpreter can run code, generate charts etc., but output often image or code rather than in Excel with working formulas	Unknown detail for Gemini
Format preservation / bulk edits (e.g. replace terms, style, layout)	Claude claims it preserves formatting and supports direct editing without opening the file manually.	ChatGPT can manipulate content, but not always preserve all formatting when exporting to external files; often conversion‐based or re‐rendered text	Gemini likely similar to document export, with less file format variety known
File size & limits	Claude: upload/download up to ~30 MB.	ChatGPT file uploads also have size limits (for large files or large images), but the limit & support for editing artifacts with formulas or presentation layouts is more constrained	Not fully disclosed / varies across features / tools
Availability / plan restrictions	Preview for paid tiers in Claude; not yet general free access.	Many advanced features gated to Plus / Pro / Teams; “Canvas” is in beta etc.	Gemini similarly has tiered and regionized feature access; some users may have access, but not universally confirmed for all features

Implications & what this means

Claude’s added file creation/edit increases its utility for document + presentation work flows, especially for business / enterprise use, where formatted deliverables (slides, reports, spreadsheets) are key.
If ChatGPT (or Gemini) wants to match this, they'd need to support not just text/coding, but full artifact generation + editing with retention of formula logic/formatting + download/export in common office file formats.
Users whose workflows involve formatting, layout, bulk edits, or converting between formats will benefit more from Claude’s new feature—less manual reformatting and fewer copy-paste hacks.
For many use cases, existing tools (ChatGPT + Code Interpreter) suffice, especially when output is data or charts. But for file artifacts that are meant to be “finished” or shared, Claude’s offering tightens the gap.

Claude’s Leap: From Chat to File Factory

2025-09-12T19:50:00.001+08:00

Anthropic just upgraded Claude to be more than a conversational assistant. A fresh feature preview lets users create and edit real files—Excel sheets, Word docs, PowerPoint decks, and PDFs—directly through Claude.ai and the desktop app. Rather than simply getting text output, you can describe what you need, upload data, and receive usable files already formatted and ready to share or export.

What’s New

File types supported: .xlsx, .docx, .pptx, .pdf—spreadsheet, word-processed, slide, and presentation formats.
Complex workflows enabled: You can ask Claude to build financial models with formulas and multiple sheets, convert PDFs into slides, clean raw data, run statistical analyses, produce charts, or stitch together reports—all via natural instructions.
Sandboxed computing: Claude now operates in a restricted internal computing environment. It can run code (e.g. Python), load libraries, and generate artifacts without exposing your local machine.

Availability & Plans

Already available now for those on Max, Team, and Enterprise plans.
Pro users will get access soon.
It’s currently a feature preview—opt-in required per user via the Claude settings (“Upgraded file creation and analysis”) and may still be tweaked.

Use Cases: What You Can Do

Transform raw data into polished reports (CSV → charts → formatted Word or PDF).
Build project trackers, scenario models, dashboards in Excel with working formulas.
Convert existing documents from one format to another: e.g., meeting notes → slide decks; PDF reports → editable docs.

Risks & Safeguards

Security: Because Claude gets limited internet access in order to import packages or execute code, there is risk of malicious content or prompt injection. Users are encouraged to monitor outputs and disable the feature if suspicious behavior arises.
Sandbox isolation: Enterprise settings allow admins to enable or disable file creation organization-wide. Team users must opt in; individuals can toggle the feature.

Why It Matters

This move shifts Claude (and similar models) further into hands-on productivity automation. Rather than merely advising, Claude can now execute parts of what used to require manual effort: formatting, data manipulation, cross-format conversion. That reduces friction for users who want to go from idea → usable artifact in fewer steps. It’s also a more natural way to blend AI into workflows: you stay in chat, give instructions, and get back files—not just text dumps you have to reformat. It’s a signal of what’s next: smarter agents embedded in the tools people use daily.

Blog Link

A Survey of Reinforcement Learning for Large Reasoning Models: mapping the promise and the gaps

2025-09-12T19:00:00.001+08:00

Reinforcement learning (RL) isn’t new—but as Large Language Models (LLMs) evolve into reasoning machines, RL is taking a central role not just in alignment, but in building reasoning itself. A new survey, “Reinforcement Learning for Large Reasoning Models (LRMs)” by a large group from Tsinghua, Shanghai AI Lab, SJTU, and others, lays out an exhaustive map of the nascent field: what’s working, what’s risky, and what future architects need to solve.

What the survey covers

The paper dives into the core building blocks of using RL in reasoning-centered LLMs (often called LRMs): how to define rewards, what training algorithms are in play, how sampling strategies are evolving, and how infrastructure and task domains factor into the picture. It considers both alignment-adjacent RL (e.g. RLHF, preference learning) and RL whose goal is reasoning performance (accuracy, planning, reflection).

Key themes and insights

Reward design
The survey classifies rewards into several types:
- Verifiable rewards (e.g. test correctness, unit tests, exact checks) when tasks allow.
- Generative / learned reward models for subjective or open domains.
- Dense rewards vs outcome-only reward schemes—bringing signal into intermediate reasoning steps.
- Unsupervised or weak rewards when neither full correctness metrics nor human feedback are feasible.
  The authors emphasize that tasks with strong verifiability tend to yield more reliable RL learning.
Policy optimization & sampling strategies
There’s a broad sweep of algorithms: policy gradients, off-policy methods, regularized RL, hybrid approaches, critic-based vs critic-free methods. Sampling strategies—how you gather candidate outputs or intermediate chains—have big effects both on performance and on compute cost. Dynamic / structured sampling (e.g. adaptively adjusting paths, beam vs sampling) is becoming more common.
Foundational problems and gaps
Several of these stand out:
- Distinguishing when RL improves reasoning vs just memorization.
- Balancing weak model priors: does your base LLM already encode reasoning bias, or do you need to train from scratch?
- Trap of over-rewarding narrow achievements; reward hacking.
- Challenges in reward specification in subjective domains.
- Scaling issues: compute, infrastructure, verifying many candidates.
Training resources & infrastructure
The survey catalogues the spectrum of environments and corpora used: from static datasets to dynamic environments (interactive tasks, tool usage), from single-task to multi-agent setups. It also considers RL frameworks and infrastructure tools (e.g. RL pipeline libraries) that enable reproducible LLM+RL research.
Applications
RL for LRMs has been used in:
- Coding: unit tests, code correctness, reflection.
- Agentic tasks: agents using tools, web retrieval, planning.
- Multimodal reasoning: vision-language tasks, code+images.
- Robotics / medical / scientific domains. Each has its own reward/verification constraints.

Why it matters & what to watch next

Reasoning as an explicit target. RL is being woven into models not just to be more “helpful” or “safe,” but to reason more deeply: plan, reflect, self-correct.
Verifiability is a power lever. Where tasks allow for exact or semi-exact verification, RL works well. When reward is fuzzy, progress is slower and riskier.
Cost and scalability are fundamental constraints. As LRMs become larger and used with more test-time compute (more chain-of-thought, more candidate generations), RL training and inference costs balloon; infrastructure and sampling strategy choices can make or break feasibility.
Hybrid and co-evolving reward models are growing. There’s increasing interest in reward models that both learn and evolve alongside the LLM, or in having the model itself critique or verify its own work.

Takeaways for researchers and builders

If you’re designing RL for reasoning tasks, aim for verifiable reward signals where possible—they give cleaner gradients and fewer surprises.
Pay attention to sampling strategy—generating more candidates or reasoning branches helps, but only when combined with selective reinforcement.
For subjective or “open” tasks (creative writing, alignment, etc.), you likely need sophisticated reward models, rubric-based or generative rewards, and strong regularization.
Infrastructure matters: your ability to scale RL—from having candidate generation, verifiers, tool execution environments, caching, etc.—significantly affects what you can achieve.

Bottom line: This survey is a timely, comprehensive lookup table for anyone playing at the intersection of LLMs, RL, and reasoning. It confirms that reward design and verifiability are major levers, that RL is now essential for pushing reasoning as a capability, but also that many technical, infrastructural, and algorithmic challenges remain before “reasoning superintelligence.”

Paper link: arXiv 2509.08827 (PDF)

How to Build High-Quality Tools for LLM Agents — Lessons from Anthropic

2025-09-12T09:40:00.001+08:00

As agents become more central to AI workflows, what separates a good agent from a great one often comes down to the tools it has—and how well those tools are designed. In “Writing effective tools for agents — with agents,” Anthropic shares a practical roadmap for building better tools powered by tools themselves, using Claude and the Model Context Protocol (MCP) as real-use labs.

What are “tools” in the agentic context?

Unlike conventional software APIs—deterministic functions that always give the same output for the same input—tools for agents must be built to coexist with non-deterministic systems. Agents like Claude must decide when to use tools, how to parse their output, and how to call them responsibly. A tool here is not just an API call; it's part of an interface contract between predictable software and unpredictable agent behavior. Tools are the mechanisms by which agents expand what they can reliably do.

Key workflows: prototyping, evaluating, and iterating

Anthropic emphasizes an iterative workflow:

Prototype early: Build simple versions of your tools. Use MCP servers or desktop extensions to connect your tool to Claude Code, allowing rapid experimentation and detection of rough edges. Include clear documentation that the agent can consume.
Run realistic evaluations: Create evaluation tasks that reflect real-world usage (multiple tool calls, complex chains, integration with other services). Use verifiable outcomes, not just “it seems right.” Capture metrics such as tool calls, token consumption, runtime, errors. Avoid toy tasks that underrepresent complexity.
Use agents to improve tools: Let Claude analyze transcripts and feedback to suggest refinements—maybe better prompt descriptions, more efficient tool outputs, clearer schemas. Anthropic reports improvements even for tools built by internal experts, purely by letting agents inspect tools’ performance.

Best practices and guiding principles

Anthropic distills the lessons into a set of design principles. Key among them:

Choosing tools selectively: Not every API needs to become a tool. Tools should cover high-impact, repeated workflows—not wrapping every possible existing endpoint. Also, consolidate when possible.
Namespaces and naming clarity: Clear, consistent naming helps agents pick the right tool. Avoid ambiguous names or overlapping functionality. Group related tools under logical prefixes or categories.
Return meaningful, concise context: Tools should return high-signal info. Avoid overwhelming the agent with technical IDs, long metadata unless necessary. Also allow “concise” vs “detailed” response modes.
Optimize for token efficiency: Use truncation, filtering, pagination. Prompt agents to use fewer tool calls or more precise queries. Efficient context limits make downstream tasks more reliable.
Clear tool specs and descriptions: Explicit parameter naming, clear input/output formats, good examples. Prompt engineering of tool descriptions can significantly impact performance.

Why this matters

Tools shape what agents can do. When tools are poorly described, overly broad, or return huge dumps of irrelevant context, agents waste resources, produce hallucinations, or fail to successfully orchestrate workflows. On the other hand, well-designed tools reduce ambiguity, reduce token use, reduce error, and let agents scale reliably across real-world tasks.

Especially as agents connect to many tools (hundreds via MCP servers), these design principles become the difference between brittle behavior and something that feels reliable and intuitive. Anthropic’s experience shows that many improvements come not from changing the LLM itself but refining the tools around it.

If you’re building agent tools or service/tool APIs for agents, following Anthropic’s workflow—prototype → evaluate → iterate—and using clear naming, context-efficient returns, and good documentation will set you up for tools agents actually use well.

Link: https://www.anthropic.com/engineering/writing-tools-for-agents

Parallel-R1: Teaching LLMs to reason from multiple angles—permanently

2025-09-11T17:55:00.001+08:00

Modern large language models (LLMs) often reason sequentially—one thought chain at a time. Parallel thinking, in contrast, involves spawning multiple reasoning paths (or perspectives), then merging the insights. While prompting tricks can induce this behavior at inference, they carry heavy overhead and brittle generalization. Parallel-R1, a new paper by Tencent AI Lab Seattle with collaborators, pioneers a training-time RL framework for instilling parallel thinking as a native reasoning strategy.

What is Parallel-R1

The key idea: don’t just prompt models to use parallel paths—train them to do so. Parallel-R1 has a progressive curriculum:

Cold start (format learning via SFT) — teach the model the syntax/tags of parallel blocks (e.g. <Parallel>, <Path>...</Path>, <Summary>), using easier math problems (GSM8K) where high-quality parallel traces are easy to generate.
Reinforcement learning (RL) on easy tasks, to explore usage of parallel thinking, with reward that combines correctness + usage of parallel structure.
RL on more difficult problems (e.g. DAPO, AMC, AIME), so the model generalizes both performance and the parallel thinking style.

The architecture has two variants: a causal (structure-agnostic) version and a structured version. The structured version modifies the attention mechanism (via path-window masking, separate position encodings) so paths are more isolated during reasoning. But structured variants show trade-offs—good for generalization in some settings, but less robust under distribution shift.

Results & gains

On a battery of math benchmarks (MATH, AMC23, AIME24, AIME25), Parallel-R1 shows consistent improvements:

The “Seen” variant (causal) achieves ~48.9% average across benchmarks (Mean@16 / Pass@16, etc.), beating baseline GRPO RL on general math tasks.
In particular, on AIME’25, Parallel-R1 raises accuracy by ~8.4% over a purely sequential RL model trained on the harder tasks directly.
The structured (Unseen) variant also performs well under certain reward schedules; the “alternating ACC/PAR” reward schedule (switching between rewarding correctness and parallel structure periodically) helps balance parallel usage and performance.

Beyond numerical gains, the authors observe a behavioral shift: early in training, the model heavily uses parallel paths as an exploration tool, branching in many places; as the model becomes stronger, it shifts to using parallel paths more conservatively, mostly for verification near the end of reasoning. This shift correlates with stronger final performance.

Why this matters

Performance & efficiency trade-off: Parallel-R1 shows that training models for parallel thinking can yield higher reasoning ability without ballooning inference cost (since only when needed are parallel paths triggered).
Better than imitation: Many earlier works used supervised fine-tuning on synthetic parallel reasoning traces under teacher forcing; but those often over-fit to particular patterns. RL in Parallel-R1 helps models learn to decide when parallel paths help, not just how to mimic them.
Scaffolding exploration: The cold-start + easy tasks + alternating reward strategy functions as a scaffold, enabling RL to find a stronger policy space than direct RL on hard tasks.
Architecture designs matter: The structured variant shows that attention masking and position encodings can help or hurt depending on how well training data matches deployment tasks.

Limitations & future directions

The gains, though significant, still leave room before human-level performance in very hard math tasks.
The structured variants can struggle under domain shift; care needed in architectural changes that assume particular path structures.
Triggering parallel thinking (using <Parallel> blocks) costs some token and compute overhead, though the model learns to use it more sparsely over time.
There’s a balance tension between pushing for parallel structure (which encourages exploration) and maximizing accuracy (which sometimes pushes toward fewer divergences). Reward engineering is delicate.

Bottom line: Parallel-R1 is a breakthrough toward training LLMs that think in parallel, not just deeper. By combining curriculum learning, structured or causal variants, and reinforcement learning with rewards for both correctness and reasoning style, it unlocks better performance on challenging math tasks. As reasoning benchmarks and applications demand both correctness and robustness, methods like this will likely become a standard part of the toolkit.

Paper link: arXiv 2509.07980 (PDF)

The Majority Isn’t Always Right: AggLM Learns to Aggregate Better Than Voting

2025-09-11T17:10:00.009+08:00

When logic is tricky, the most common answer isn’t always the correct one. A new Meta/Fair & CMU paper titled “The Majority is not always right: RL training for solution aggregation” challenges the standard practice of combining LLM outputs via voting or reward-scored selection. Their method—AggLM—trains a dedicated aggregator model to review, correct, and synthesize among multiple LLM-generated candidate solutions via reinforcement learning from verifiable rewards (RLVR), yielding big gains over majority voting and reward model baselines.

Solving it: learned reconciliation vs. counting

Standard aggregation in LLM reasoning often works like this: sample many candidate solutions, then pick the answer that's most frequent (majority voting) or highest scored by some reward model. While effective in many settings, these methods have a blind spot—when correct answers exist only among minority solutions. In contrast, AggLM treats aggregation itself as a reasoning task. It takes a set of candidate solutions, analyzes them, spots mistakes or partial correctness, then combines ideas or corrects missing steps to produce a final solution. Importantly, it’s trained using verifiable rewards—i.e. only when the aggregated output matches a known correct solution.

Key ingredients & experiments

Dataset & training: Using Qwen3-1.7B as the solution generator, AggLM-1.7B is trained on ~446,000 examples drawn from a mixture of “easy” and “hard” sets. Hard sets are those where the majority answer among candidates is actually incorrect; the mix helps the model learn both to follow the majority and to rescue correctness from minority solutions.
Aggregation via RLVR: The model uses Group-Relative Policy Optimization (GRPO), with a binary reward (1 for matching the ground truth, 0 otherwise). The aggregator is initialized from the Qwen3-1.7B model but is tuned via this RL signal.
Benchmarks: Evaluated on four math contest datasets: AIME24, AIME25, HMMT24, HMMT25. AggLM was tested aggregating candidate solutions from both the same generator model (Qwen3-1.7B) and stronger ones (Qwen3-8B), in both thinking and non-thinking modes.

Results & token-efficiency

On solutions from Qwen3-1.7B in thinking mode, AggLM-1.7B lifts accuracy significantly. For example, on AIME25, majority voting with 8 candidates yields ~67.9%, while AggLM pushes it to 50.0% in a different benchmark context (depending on the exact evaluation variant). More striking, when aggregating from the stronger 8B model, AggLM still outperforms majority voting, weighted voting, and reward-model selection baselines.
In non-thinking modes (i.e. when the candidate-generating model is weaker or does not use chain-of-thought reasoning), AggLM retains its lead—showing that it generalizes beyond just cherry-picking strong or specifically-formatted inputs.
Regarding cost, AggLM is more token efficient: instead of needing large numbers of candidate solutions (i.e. very large k) for majority voting to reach high accuracy, AggLM achieves similar or better accuracy with fewer candidate solutions, saving both inference time and compute.

Implications & what’s next

AggLM shifts thinking in two ways:

Aggregation as reasoning. Aggregation isn’t just picking among options—it’s an opportunity to correct, synthesize, and integrate partial truths. Models that can do that perform better, especially in instances where majority answers mislead.
Balancing examples is key. Training on a mix of easy and hard cases was essential. If you train only on “easy” majority-correct groups, or only on “hard” ones, performance suffers.
Generalization beyond training generators. AggLM works well even when aggregating from stronger models than those used during training—implying aggregation skills are transferable, not just overfitted to particular output distributions.
Efficiency trade-off. Instead of scaling k (number of solutions) to very high values, using a learned aggregator yields larger gains per additional candidate, meaning happier ceilings on tokens/time.

Bottom line: AggLM demonstrates that “the majority vote” should not be the default in reasoning aggregation. Models that are trained to look across candidate solutions—identify hidden truth, correct errors, and combine the best ideas—do better than simple heuristics. Especially in math and logic tasks where minority correct answers exist, learned aggregation via RL with verifiable reward is a strong lever. If you’re designing agents or reasoning pipelines, integrating an aggregator like AggLM can be a powerful performance boost with reasonable cost.

Paper link: arXiv 2509.06870 (PDF)

ParaThinker: parallel minds beat longer monologues

2025-09-11T10:57:00.006+08:00

LLMs have ridden test-time compute—“think longer” chains of thought—but returns taper as early tokens lock models into bad trajectories. Tsinghua’s ParaThinker calls this Tunnel Vision and proposes native thought parallelism: generate several independent reasoning paths simultaneously, then fuse them into one answer.

Instead of external voting, ParaThinker trains the model itself to branch and merge: specialized control tokens (<think i>) trigger distinct trajectories, path-specific positional embeddings keep streams separate, and a two-phase attention mask enforces independence during thinking and controlled integration during summarization. The KV cache from the thinking stage is reused, avoiding re-prefill costs.

On AIME-24/25, AMC-23 and MATH-500, ParaThinker with 8 parallel paths boosts accuracy by +12.3 pts (1.5B) and +7.5 pts (7B) over sequential baselines under the same token budget, and still beats majority voting by +4.3/+2.0 pts—with only ~7.1% latency overhead. Generating up to 16 paths costs <2× single-path latency, thanks to better arithmetic intensity on GPUs.

The takeaway: scale width, not just depth. ParaThinker shows that orchestrating compute across diverse, parallel thoughts unlocks latent reasoning ability and makes smaller models out-punch larger sequential ones. Code is available on GitHub.

Paper link: arXiv 2509.04475 (PDF)

TraceRL puts diffusion LLMs on the reasoning map

2025-09-10T18:00:00.000+08:00

Autoregressive (AR) giants have dominated reasoning benchmarks, while diffusion language models (DLMs) were seen as “fast samplers” with limited logic chops. A new paper from Princeton and UChicago argues that’s mostly a training-objective problem—and offers TraceRL, a trajectory-aware reinforcement learning framework that aligns what a DLM learns with how it actually samples. The team also releases code and ready-to-run models under the TraDo banner.

What’s new

Trajectory-aware RL for DLMs. Instead of scoring randomly masked sequences, TraceRL optimizes against the model’s intermediate inference traces, matching the left-to-right / blockwise behavior used at decode time. A diffusion-based value model stabilizes training by reducing variance. Crucially, the method works for full-attention and block-attention DLMs.
Open stack. The release includes a framework to build/train/deploy DLMs across architectures, with KV-cache acceleration, inference engines, SFT + RL recipes for math and code, and links to TraDo-4B/8B checkpoints.

The receipts

On headline benchmarks (dynamic vs. static sampling shown in the paper), the TraDo models post the strongest DLM numbers to date and overtake AR peers at similar scale on math:

TraDo-8B-Instruct: MATH500 78.5, AIME’24 13.3, LCB-V2 25.9—a +6.1% relative lift over Qwen2.5-7B-Instruct and +51.3% over Llama-3.1-8B-Instruct on math reasoning.
TraDo-4B-Instruct: MATH500 75.6, AIME’24 10.3, LCB-V2 18.7, consistently edging 7B AR baselines on math.
TraDo-8B-Thinking (long-CoT): first long chain-of-thought diffusion LLM, hitting MATH500 87.4, AIME’24 35.5, LCB-V2 34.6 with very long answers.

The authors attribute gains to objective/trajectory alignment and show smoother curves with the value model vs. policy-only RL. They also document a speed/accuracy trade-off: dynamic sampling is faster; static top-1 decoding squeezes out extra points.

Why it matters

DLMs aren’t just “fast”—they can reason. With the right RL target, parallel generation stacks clear long-form math and coding hurdles previously ceded to AR. 2) Unifies the zoo. One RL recipe spans full-attention and block-diffusion, and even helps enlarge block size for more flexible sampling. 3) Practical path. The open framework + KV-cache tricks make DLM post-training and deployment feel product-ready, not just a lab exercise.

Setup notes

Math RL uses 8k hard MATH tasks; coding RL uses 6k verified problems from PrimeIntellect. Long-CoT training mixes TraceRL with long-form SFT as a curriculum.

Bottom line: TraceRL reframes diffusion LLMs as credible reasoners, not just fast generators—and TraDo-8B-Thinking plants the first long-CoT flag on the DLM side of the field.

Paper link: arXiv 2509.06949 (PDF)

Language Self-Play: training an LLM without adding data actually works

2025-09-10T14:00:00.000+08:00

LLMs keep getting better by eating more data—until the data well runs dry. A new paper from Meta Superintelligence Labs proposes Language Self-Play (LSP): turn training into a game where a single model plays both sides—a Challenger that generates tougher prompts and a Solver that answers them—so the system improves without ingesting new datasets. In tests on AlpacaEval using Llama-3.2-3B-Instruct, LSP matches a strong data-driven RL baseline and even pushes beyond it when used as a follow-on stage.

How it works: one model, two roles

LSP frames training as a minimax game: Challenger tries to minimize reward by making hard queries; Solver tries to maximize reward by answering them. Crucially, both roles are instantiated by the same LLM via a role-selecting prompt (e.g., a special challenger prompt), avoiding the instability and memory overhead of training an external adversary. KL regularization keeps the Challenger from devolving into nonsense prompts.

Under the hood, LSP borrows group-relative baselines from GRPO: Challenger generates N queries, Solver samples G answers per query, and the average reward defines both a per-answer advantage (for Solver) and a “difficulty” signal (for Challenger). A practical variant, LSP-Zero, runs as a pure zero-sum game; the full LSP adds a quality self-reward scored by a reference model to prevent reward-hacking (e.g., answering everything in Python).

Results: data-free ≈ data-driven—and sometimes better

Using GPT-4o as judge on AlpacaEval, the team compares models trained from the same base:

From base (no data): Overall win rates vs. the base model—GRPO (with data) 40.9%, LSP-Zero 40.1%, LSP 40.6%. Translation: self-play without any RL data keeps pace with standard RL.
From RL (as a next stage): Starting from the GRPO model and continuing with self-play, LSP lifts overall win rate to 43.1%, with large gains on Vicuna-style conversational tasks (28.7% → 46.3%).

The setup uses Skywork-Reward-V2-Llama-3.2-3B as the reward model; the authors note that LSP (with the added quality reward) avoids the degradation seen with LSP-Zero in some splits, and acknowledge dips on “chatbot-y” Koala prompts—likely because Challenger skews toward structured, orderly instructions.

Why this matters

Data bottleneck relief. If you can translate “more practice data” into a self-generated curriculum, you can keep improving without chasing new corpora.
A clean follow-on stage. Even after data-based RL, self-play adds headroom—useful when further high-quality preference data is scarce.
Single-model simplicity. One backbone serves both roles, avoiding adversary models and the instability they bring.

Caveats and open questions

Self-play can degenerate without the quality self-reward; reward choice caps the ceiling (a weak reward model means weak training signal); and Challenger diversity remains an open knob to broaden beyond the structured style seen in examples. Still, the authors argue the method should work even better on tasks with verifiable rewards (e.g., code tests), not just preferences.

If your roadmap hits a data wall, Language Self-Play is a compelling new leg in the post-training pipeline: spin up a Challenger inside your own model, let it stress-test itself, and learn—no fresh dataset required.

Paper link: arXiv 2509.07414 (PDF)

An AI that writes expert-level scientific software—and often beats the leaderboard

2025-09-10T11:25:00.005+08:00

A large Google team is pushing past “chatty copilot” and into AI that authors working scientific code. Their system pairs a large language model with tree search to iteratively write, run, and score programs for scorable research problems—then learns to recombine ideas from papers and prior algorithms. In benchmarks, it discovered 40 new single-cell RNA-seq methods that outperformed the top human-made entries on OpenProblems, and produced 14 COVID-19 hospitalization forecasters that beat the CDC’s ensemble and every individual competitor during the study window.

How it works. Researchers frame a scientific task as “maximize a quality metric,” let the LLM generate code variants, and use tree search to expand promising branches while pruning the rest. The agent can ingest research ideas from literature (summarized with Gemini 2.5 Pro) and also tries automatic recombinations of methods, plus proposals from Gemini Deep Research and AI co-scientist tools. In head-to-head tests on nine published algorithms, the system’s implementations beat eight of nine baselines; its best run—BBKNN(TS)—improved the bioinformatics leaderboard by 14% over the long-standing ComBat approach.

Bioinformatics at scale. The team evaluates on OpenProblems v2.0.0, spanning 1,747,937 cells and 13 metrics across six datasets. Beyond re-implementing published methods, recombination mattered: among 55 pairwise hybrids, 24 outperformed both parents and most others beat at least one—evidence that the search can synthesize competitive, novel ideas rather than just tune hyperparameters.

Public-health forecasting. For U.S. COVID-19 hospitalization forecasting (the CDC’s Forecast Hub), the system generated models that were consistently lower-error (better WIS) than the official ensemble in most jurisdictions; in an aggregate comparison, 14 strategies (10 recombinations, plus two Deep Research, one AI co-scientist, and one replicated baseline) surpassed the ensemble across the three-week hold-out period.

Not just biology. The abstract lists additional wins in geospatial image segmentation, zebrafish neural activity prediction, general time-series, and numerical integration, arguing the approach generalizes to diverse “empirical software” problems where code can be scored automatically.

Engineering notes—and guardrails. To avoid overfitting, bio experiments hill-climb on a separate CELLxGENE dataset and report on the held-out OpenProblems benchmark; metrics that fail to compute are clamped to worst-case—making robustness part of the score. The team also ran multiple replicates to show stability, and reports practical budgets: ≈500 nodes (~7 hours) per scRNA-seq search and ≈2000 nodes per COVID run on their infra.

Why it matters. Rather than waiting for domain-specific code to be hand-crafted over months, this “AI co-scientist” produces working software, tests it against public leaderboards, and composes new hybrids from the literature. If those patterns hold beyond the reported tasks, the future of scientific computing looks less like prompt engineering—and more like searching the space of programs.

Paper link: arXiv 2509.06503 (PDF)