Share this article | Listen to this article ($)

My initial reaction to the recent package hacks was fear. But I quickly learned I can mitigate much of the risk with a handful of configuration settings.

If you haven't been following along, a large number of packages (think of packages as bundles of reusable code) have been compromised by malicious actors. Software developers have long known to guard against these types of attacks. But as more of us turn to coding agents to write code on our local devices, we are finding that we also need to understand and mitigate this risk.

Last week, we dove deep on how these malicious attacks work, how we might defend against them, and explored whether Cowork was a safer option for most of us. The verdict was that it can be, but only if we understand how to use it safely. If you are a fan of Cowork and you missed that article, be sure to give it a read.

Cowork, however, isn't always the best solution for me. I have too many use cases where I want Claude to access many folders across my file system.

I use Claude to search, create, and edit the markdown files that make up my task management system. These files reference source material across my drive. I created a research system where Claude automatically creates notes from PDFs that I download and those notes get sorted across a variety of topic folders. I have a comprehensive network of context files that I want Claude to reference on as-needed basis.

I don't want to have to grant Claude access to one folder at a time. But I also don't want to throw caution to the wind. Recent weeks have taught me that I need to do more to keep my devices and code safe. Thankfully, I'm learning this is very doable. And for my Cowork fans, most of the strategies that I use on my local device can also be used to make Cowork safer.

If you missed last week's article, let's do a quick recap of what we covered.

How Package Hacks Typically Work

An agent scoured over 230,000 malicious code incidents and found that most follow a similar pattern. Malicious code needs an entry point onto your computer. Once installed it scours your device for sensitive data, and then it uses your network connection to send that data to its own servers.

The good news is this pattern also tells us exactly what we need to guard against. We can block the most common entry points, we can limit what our code (and coding agents) have access to, and we can (sometimes) limit our outgoing network traffic.

Before we dive into how to defend against these attacks, let's make sure we are all up to speed on the basics.

Understanding Key Concepts: Packages, Installers and Registries

You've already learned that a package is a bundle of reusable code. When you are creating software, you don't have to create everything from scratch.

For example, I use a package called luxon to help me with time zone math. I use another package called jest to help me run my unit and integration tests. I use a package called pandas to ingest and manipulate CSV data.

Before my code can use these packages, I have to download and install them. I can download them directly from GitHub, but more commonly I grab them through a package registry. Think of a registry like the App Store. Package creators publish their packages to registries, making them available for other people to download. npm is the Node.js registry and PyPI is the Python registry.

To access a registry, you need an installation tool. npm (confusingly) is both a registry and an installation tool. So you use the npm command-line tool to download and install packages from the npm registry. pip is the primary installation tool for Python and you use it to access the PyPI registry.

A coding project will often maintain a manifest of dependent packages—package.json for Node.js projects and pyproject.toml or requirements.txt for Python projects. These manifests tell a developer new to a project what packages the project depends on. Build tools use these manifests to automatically download and install all of the required dependencies.

When you download a package, it might have its own dependencies. If it includes a manifest (package.json, pyproject.toml, requirements.txt), your install tool will automatically install those dependencies as well. These dependencies are often called transitive packages. And each transitive package can also have a manifest.

Every time you install a package, you might be installing an entire dependency tree. For example, I just installed jest and it included 266 packages.

Developers are often aware of the packages they choose to use. But they aren't always aware of all of the transitive packages that come along with those packages. Malicious software often spreads through these transitive packages. The risk compounds when we let coding agents install packages for us.

With this foundation, let's look at how I defend against malicious code when installing packages.

How I Configure My Devices to Guard Against Malicious Software

My local strategy is primarily designed to limit malicious software from getting on my device in the first place. My goal is to block as many entry points as I can.

A quick note for Cowork fans: We saw in last week's article that Cowork provides protection through isolation, but does little to keep malicious code out. It turns out you can use almost all of the following strategies in Cowork to plug this gap and make Cowork significantly safer.

Today, I'll cover how and why:

• I'm only installing packages that are at least seven days old.
• I'm not auto-running install scripts before vetting them.
• I'm using provenance signals to identify red flags.
• I'm blocking exotic dependencies.
• I'm using hooks to configure Claude so that it has to follow these same rules.

That's not all I'm doing. In future posts, I'll cover how and why:

• I'm pinning all package versions (including transitive packages), so that they don't auto-update.
• I'm doing weekly audit sweeps to make sure nothing seeped through the cracks.
• And despite all of this, I'm still moving most of my development off of my local machine.

We'll look at each in turn. This is going to get technical. If much of this is new to you, just take it step by step. I'll walk you through all of it.

Product Talk is a reader-supported publication. The rest of this article is for paid subscribers. If you haven't already, be sure to upgrade to continue reading.

Listen to this episode on: Spotify | Apple Podcasts

What's the best way to invest in your team's professional development — train everyone at once, let people self-direct, or something in between?

Petra Wille and Teresa Torres explore why the sweet spot might not be what most organizations default to. They share real examples from their work with product teams and leaders — from book clubs to group coaching — and make a case for small, intentional groups as a powerful (and underused) learning model.

Three Models of Team Learning

• Train everyone at once — builds shared language, but not everyone is ready at the same time
• Self-directed learning — works for highly motivated individuals, but lacks accountability
• Small-group learning — the sweet spot: peer accountability, shared momentum, and just-in-time relevance

Why Learning Together Works

• Creates natural accountability and deadlines
• Helps people apply concepts to their own real work
• Especially valuable for product leaders, who rarely have built-in peers to learn alongside

Group Coaching vs. One-on-One Coaching

• Individual: sounding board, holding space, powerful questions
• Group/team: real work in the room, peer learning, bridges between leaders who rarely collaborate
• Keep participants as close colleagues — trust and vulnerability go up when people already know each other

Key Takeaways

1. Start a book club — debriefing together beats reading alone
2. Train pilot teams before rolling out org-wide
3. Encourage duos or trios to take courses together
4. Match your learning format to your actual goal
5. Keep coaching groups tight for more honest, productive sessions

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode:

• Communities of Practice
• Petra Wille's book Strong Product Communities - The Essential Guide to Product
• Become a Better Product Leader: A 52-Week Transformation Journey - Petra's email course with quarterly live Q&A
• Teresa Torres’ book Continuous Discovery Habits
• Continuous Discovery Habits (CDH) Book Club
• Petra’s STRONG Product People Corporate book clubs
• Teresa's Product Discovery Fundamentals course
• Work with Petra
• Learning together at a conference like Product at Heart
• Teresa & Hope Gurion's group leadership coaching program through Product Talk Train Your Team

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Share this article | Listen to this article ($)

I love being a builder. I feel like I have a new superpower and I can't get enough of it.

I enjoy tinkering with my Claude Code workflows to make my days more effortless. I'm having a blast building AI-generated interview snapshots and opportunity solution trees for Vistaly. I thoroughly enjoy digging into my traces and iterating on my AI coaches for our discovery courses.

But some of the shine of that superpower started to wear off over the past couple of weeks. Recently, malicious software started spreading through the open-source community. It affected companies big and small—many that you've heard of, like OpenAI, PostHog, and Zapier.

As I started to investigate what I needed to do to defend against this hack, it looked like the solution was simple—move to some better, properly configured tools. But as I learned more, I realized what most cybersecurity experts have long known—this is a rabbit hole that just keeps getting deeper. I'm learning that to build responsibly, I need to get far better at keeping my devices and my code safe. And this doesn't just affect me; it probably affects you, too.

Let me explain why. We are all using open-source software. Most software isn't built from scratch. It mixes and matches components from the broader software development community. When you need to parse a PDF, you don't have to build your own PDF parser. Someone else has already built that and made it available for you to use. If you need to handle dates across time zones, visualize data from a spreadsheet, or connect to an API, someone has already written software that you can simply reuse.

Most third-party code is perfectly safe. Almost every app on your computer relies on it. Open-source software has flourished because it helps all of us build faster. More recently, the popularity of agent skills and MCP servers (another form of third-party code) is helping us all learn how to get more value out of models. This is good. I don't want you to walk away from this article thinking third-party software is bad. It's not.

But malicious actors do take advantage of this ecosystem to propagate harmful code. We need to be aware of the mechanisms that they use and we need to defend against those mechanisms.

What Happened: How This Ended Up on My Radar

On May 11th, I started seeing tweets about a TanStack hack. At that time, I didn't know what TanStack was. But apparently, it's a popular set of JavaScript libraries that are used by a lot of React sites.

At first, this wasn't notable. I was aware of package hacks and I didn't use TanStack, so I continued on with my day. But then, I started to see reports of more packages being hacked. It turns out the TanStack packages were compromised by a worm—malicious software that has a mechanism to self-replicate. We'll cover exactly how this works in a bit, but for now, the key point is that it spread quickly.

Within a couple of hours, I was seeing reports of dozens of packages being compromised and by the end of the day, the numbers were in the hundreds. This scared me. I realized I needed to start paying attention.

If you read my "How to Use Claude Code Safely" article, you've already been introduced to package hacking. A package is a bundle of reusable code that is shared through package registries.

Almost every piece of software that you use relies on third-party packages. This is normal. But because there are malicious actors out there, not all packages are safe. Before we install a package on our local device, we need to make sure it's safe. My safety article included some basic tips for how to evaluate if a package is safe.

Unfortunately, this hack (known as the Mini Shai-Hulud worm) is showing that this guidance wasn't adequate. We can't just rely on popular or even trusted sources. We have to do more to protect ourselves. And that realization is why I am introducing a new series.

Today, I'll cover:

• how malicious software typically works,
• a framework for how we can guard against it,
• the risk of using Cowork to write and run code,
• and how to mitigate some of that risk.

In subsequent articles, we'll go even deeper. Next week, I'll dive into how to use coding agents like Claude Code and Codex safely on your local device. I'll share:

• the exact tools I'm using to defend against malicious actors,
• how I've configured Claude Code to ensure it always uses the right tools,
• and a public repo where you can grab my config files and utility scripts that make it all work.

And in the following week, I'll share why I'm moving most of my development off of my local device and how I'm doing that. We'll explore the differences between containers and virtual machines, whether you should run them locally or in the cloud, and what resources already exist that can help you get set up quickly.

My goal is to help you keep building—despite the risks—in a way that keeps you and your data safe.

But before we dive in, I have to share a disclaimer. I'm not a security expert. I'm sharing my personal journey and what I learned through a lot of research. Please use your best judgment when applying the content of this article.

With that caveat, let's look at the anatomy of malicious software.

How Malicious Software Typically Works

An agent recently scoured over 230,000 malicious software incidents and found that most malicious software follows a similar pattern. First, it needs an entry point onto your computer. Once installed, it scours your device for sensitive data, and then it uses your network connection to send that data to its own servers.

The Mini Shai-Hulud worm spreads using this exact pattern. It gains entry into new devices via a malicious package install script. When a developer downloads a package from a package registry, it can include install scripts that automatically run at time of download. Once the install script runs, the malicious code gains access to the developer's device. The code searches the device for credentials, including credentials to publish other packages. It uses those credentials to poison any additional packages the developer has publishing rights to. This is how the worm spread so quickly. It then uses three different channels—including the victim's own GitHub public repos—to distribute the secrets.

So an attack typically involves three steps:

1. It finds an entry point to your device.
2. It searches your device for sensitive data.
3. It sends that data to its own server.

The good news is this pattern also tells us exactly what we need to guard against. We can block the most common entry points, we can limit what our code (and coding agents) have access to, and we can (sometimes) limit our outgoing network traffic.

The Mini Shai-Hulud worm used package install scripts to gain access to new devices. But this isn't the only entry point. Any code that has permission to run on your computer can include malicious code.

So we need to be mindful not just of what packages we install, but also what agent skills we download, what MCP servers we use, and even what third-party app extensions we add.

It used to be that only engineers had to worry about this problem. But as more of us use agent skills, MCP servers, vibe coding tools, and coding agents, more of us are being exposed to this risk. We need to understand how to mitigate it.

How To Determine If You Might Be at Risk

Before we dive into how to defend against malicious actors, let's talk about what opens you up to this type of risk. Here are several common scenarios that rely on using third-party code:

• You download and use third-party skills or MCP servers.
• You let Claude Code, Codex, or any other coding agent write scripts that run directly on your local device and those scripts use third-party packages.
• You use an integrated development environment (e.g. VS Code, Cursor) and use third-party extensions.
• You use Obsidian and install third-party extensions.

This list isn't exhaustive. But if you are doing any of these things, please keep reading. We'll cover the full list over the course of the series.

How to Defend Against Malicious Code: The Safe-ish Option

The most reliable way to defend against malicious actors is to never install third-party software on your local device. But that's not realistic. We all use countless apps every day. I know many of you are already using third-party skills and MCP servers, and some of you are even using coding agents to write and execute code that needs third-party packages.

Today, we'll start with one of the most common ways non-engineers are writing and running code—with Cowork.

Evaluating Cowork's Safety When It Comes to Malicious Code

I was surprised to learn Cowork isn't as safe as I had thought. It does offer some protection and it is much safer than Claude Code, but it has a significant safety gap that you need to be aware of.

Understanding Cowork's Security Defenses

Cowork addresses two out of the three steps in our malicious code pattern. It provides protection through isolation—it runs code in a virtual machine. You can think of a virtual machine like a quarantine room. In a quarantine room, you want to keep what's inside from getting out so that the germs can't spread.

Cowork does little to protect what can enter the room. It's up to you to manage that. However, if a germ (malicious code) does enter the room, it can't get out. Anything that sits outside the room (most of your files) can't get infected.

Cowork also limits network traffic that leaves the virtual machine. So not only does it isolate code, it limits what can exit. So in the event that malicious code does get access to the virtual machine, Anthropic makes it harder for hackers to send data home (our third step in the malicious code pattern). But we'll see this isn't 100% secure.

Claude can install packages inside Cowork. That means it is susceptible to malicious code like the Mini Shai-Hulud worm. And while it does limit outgoing network traffic, GitHub is on the allow list. This allows Cowork to read and write to your GitHub repos.

The problem is the Mini Shai-Hulud worm uses GitHub to publish secrets. So Cowork does not fully protect you from this type of attack. But, remember, if you never give Cowork access to sensitive data, then there is no data to expose to the attackers.

Understanding Your Responsibility When You Use Cowork

So the important thing to keep in mind is: Your data is only safe if it lives outside of the virtual machine. If you move your data inside the quarantine room, then any malicious code that can get inside can also access your data.

Cowork does allow you to put data inside the isolation room. When you do this, those folders become accessible from within the virtual machine. So any malicious code that runs inside the virtual machine will be able to access those folders.

There are two things to consider when you give Cowork access to folders on your device.

1. Do those folders contain any credentials or secrets?
2. Do those folders contain any proprietary data that would be harmful for a malicious actor to access?

It's not unusual for code to need access to credentials. We need API keys to use APIs. But once those credentials enter the virtual machine, they are now exposed to any malicious code that might get access.

This is why Cowork has built-in connectors to third-party sources (e.g. Google Drive, Slack). This allows you to give Claude access to data that otherwise might require an API key. The credentials that you configure for connectors never get exposed to the virtual machine. They stay outside the quarantine room. So you don't have to worry about malicious actors getting access to these credentials. But if your code requires other credentials to do its task, those credentials would be exposed inside the virtual machine. Make sure these credentials are tightly scoped.

You can also use custom MCP servers (ones that you create yourself) with Cowork. And when you do this, those credentials also stay outside of the quarantine room. But these MCP servers have to be remote—meaning you have to host them on a web server somewhere. It's not as easy as downloading an MCP server on your local device and asking Claude to configure it for you. But if you do want to use your own MCP servers with Cowork, this is the way to go—it keeps your credentials safe.

But it's not just credentials that you have to worry about. You also need to consider what's in the folders that you explicitly share—including the content that is coming via the connectors. This is where most of the risk lies. You don't want to expose any of your sensitive or proprietary data to bad actors. A good rule of thumb is to give Cowork only as much access as it needs to do the task at hand and no more.

Are Skills Safe in Cowork?

Cowork does support skills. You can add third-party skills inside the quarantine room, and as long as you aren't putting any of your own data in the room, then it's perfectly fine to take some risks here. But as soon as you add your own data (which is typically what makes skills useful), then you'll want to be mindful about which skills you are using. Skills can (and often do) include third-party code. Bad actors are using skills directories to distribute malicious code.

I never use third-party skills. If there's a third-party skill that I'm interested in, I simply read through the files to see what it's doing and ask Claude to recreate it for me. This allows me to learn from how other skills work and ensures that my data stays safe. But if I were to use third-party skills, I would use them in Cowork and limit what data they had access to.

The Limitations of Cowork's Safety

Generally, if you are mindful about what you share with Cowork, it's a pretty safe option. However, it can be hard to limit what you put in the room. Giving agents access to our data is exactly what makes them useful. Next week, we'll look at strategies for how to keep malicious code out. These strategies are designed for your local device—but they can also work in Cowork with a little bit of effort. So if you want to get more out of Cowork, stay tuned for next week's article.

Don't Confuse Cowork with the Code Tab in the Desktop App

One more thing to keep in mind: The Claude Desktop app includes both a Cowork tab and a Code tab. Cowork runs code inside a virtual machine, Code does not. If you ask Claude to write code for you from within the Code tab, you'll need to manage the security of that code yourself. This code runs on your local device. There is one exception to this—Code in the Desktop app does allow you to run code in Anthropic's cloud. We'll cover this option in the third article in this series.

Some Final Thoughts

I suspect we covered a lot of new territory for many of you. In my Claude Code safety article, I tried to strike the right balance between encouraging you to take safety seriously and not scaring you off completely. I'm trying to strike that same balance here.

I still love building with AI—I'm just doing it a bit more cautiously these days. Today, we only scratched the surface. We covered how to use Cowork safely. But this approach has limitations. I personally still prefer Claude Code and I'll be sharing next week how I configured it to be safer. But safer doesn't mean safe, which is why I'm also moving to a strategy where most of my development will happen off of my device. Stay tuned for more on both.

In the meantime, let me know in the comments if this series is helpful. This is a pretty significant departure from my discovery content. But if I'm going to encourage you to wade into this new AI builder world, I want to make sure that I'm also helping you do it safely.

Until next week. Stay safe out there.

Audio Version

The audio version is only available for paid subscribers.

Listen to this episode on: Spotify | Apple Podcasts

Teresa is cranky — and honestly? She has every right to be.

In this episode, Teresa and Petra go on a deeply relatable rant about the absurdity of modern corporate procurement processes. Teresa is simultaneously juggling seven speaking engagement contracts, and six of them have become a part-time job in themselves — think 80-page ethics policies, 800-question security forms, and Multi-Factor Authentication (MFA) questions asked 17 different times. Meanwhile, the one company that just put her fee on a credit card? Scheduled, confirmed, and done in two weeks.

Petra chimes in with her own procurement horror story — filling out 12 identical Word document forms — and the two draw a bigger point from the chaos: broken vendor processes don't just frustrate consultants, they prevent organizations from getting the expertise they actually need.

If you've ever wondered why that training got canceled, why that speaker backed out, or why your company can't seem to bring in outside experts — this episode might answer that question.

Topics Covered:

• Why Teresa is (half-jokingly) threatening to retire
• The stark contrast between vendor-friendly companies and bureaucratic nightmares — and why company size isn't the deciding factor
• The hidden cost of procurement theater: scope creep, repetitive forms, and the 800-question security review
• How a CEO's buy-in can compress a 4-month legal review process into 10 days — and what that reveals about organizational priorities
• The vendor experience as a brand experience — a lens product people should apply to their own organizations
• Teresa's new policy: her paperwork, credit card payment, no vendor setup — or no speaking engagement
• How bureaucracy compounds over time through adverse events — and why startups historically win by moving fast

Key Takeaways:

• Your procurement process is a reflection of your culture. If it's painful for vendors, it's probably painful internally too.
• Product thinkers should care about vendor experience. Every process your company delivers — including procurement — has brand impact.
• CEO-level buy-in is the fastest procurement tool there is. It reveals how much your organization actually values what it's trying to buy.
• Simple vendor terms attract the best experts. The more friction you create, the more likely top speakers and consultants will just say no.

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in the episode:

• Hire Teresa to Speak
• AWS DynamoDB & GuardDuty
• Claude Code
• The classic film Office Space (referenced for reasons that will be very clear)

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Continuous Discovery Habits turns five this year. And to celebrate we are reading the book together.

Each month, I am releasing an in-depth reading guide that includes:

• The chapters we will be reading
• A preview of the most important concepts we'll be learning about
• Short videos you can share with friends and colleagues to help spread the ideas
• Individual and team discussion questions to help you absorb and engage with the reading
• Team exercises to help you put the ideas into practice
• Additional reading to help you go deeper on the core ideas

We'll be discussing each month's reading in the comment section and we'll gather quarterly to discuss on a live call.

Joining late? No problem. I monitor the comments on each reading guide throughout the year. Start with the current month or go back to January—whatever works for you. You can ask for help, share what's working, and connect with other readers at any point.

If you want to participate, grab a copy of the book (or dig up your old copy), share the "Spread the Love" videos, reserve some time to do the team exercises, and register for the community sessions. Let's do this!

This Month's Reading

Chapter:

• Chapter 7: Prioritizing Opportunities, Not Solutions

Estimated reading time: ~16 minutes

This month's chapter will introduce you to:

• Why product strategy happens in the opportunity space, not the solution space
• How to focus on one target opportunity at a time to deliver value iteratively
• Using the tree structure to simplify prioritization decisions
• The four criteria for assessing opportunities: sizing, market factors, company factors, and customer factors
• Why treating prioritization as a messy, subjective decision leads to better outcomes than scoring formulas
• The concept of two-way door decisions and how they apply to opportunity prioritization

Need a copy? Grab the book

Share the Love with Friends and Colleagues

We learn best in community. Use the following short videos to share the key concepts from this chapter with friends and colleagues. Invite them to participate in the book club with you.

• Work on one small opportunity at a time - Reduce your batch size
• Getting started with compare and contrast decisions - Choose the right target opportunity
• Turn big intractable problems into smaller, more solvable problems - The power of decomposition

Reflect & Discuss What You Read

When we reflect and discuss what we read, we absorb more of the material. It helps us put what we learn into practice. Don't skip this step.

This chapter challenges a deeply ingrained habit: prioritizing solutions. Most product teams spend countless hours ranking features in spreadsheets, debating roadmaps, and arguing over what to build next. But this chapter makes the case that product strategy doesn't happen in the solution space—it happens in the opportunity space. The opportunities we choose to address define our competitive position far more than the features we ship.

Individual Reflection

1. Think about your team's current roadmap or backlog. How much of your time is spent prioritizing features versus understanding and prioritizing customer opportunities? What would change if you flipped that ratio?
2. Reflect on the last time you made a product decision. Did you treat it as a one-way door (irreversible) or a two-way door (reversible)? How did that framing affect your decision-making process and timeline?
3. Consider the four assessment criteria (opportunity sizing, market factors, company factors, customer factors). Which of these does your team currently emphasize most? Which do you tend to overlook or underweight?

Team Discussion

1. As a team, list the top 5-10 items on your current roadmap or backlog. For each one, try to identify the underlying customer opportunity it addresses. If you can't clearly articulate the opportunity, what does that tell you about how you're making decisions?
2. The chapter argues against scoring formulas (like RICE or ICE) for prioritization, calling them "made-up math." If your team uses a scoring system, discuss: What is it really measuring? Does it help you make better decisions, or does it just make subjective decisions feel more objective?
3. Walk through a recent prioritization decision. Did you assess options in isolation ("should we build this?") or compare and contrast them? How might your decision have been different with a compare-and-contrast approach?

Put It Into Practice

This month is all about shifting from solution-first to opportunity-first thinking. These exercises will help you practice prioritizing opportunities and making faster, better decisions about which customer needs to address.

Exercise: Map Your Roadmap to Opportunities

Time: 45 minutes
Do this: With your product trio

Take your current roadmap or backlog and work backwards. For each planned feature or solution:

• Identify the customer opportunity it's meant to address
• Write it as something a customer might say (e.g., "I can't find anything to watch" not "We need better search")
• Look for patterns: Are multiple solutions addressing the same opportunity? Are some solutions disconnected from any clear customer need?

This exercise often reveals that you're either:

1. Spreading yourself thin across too many opportunities
2. Over-investing in a single opportunity with multiple solutions
3. Building solutions with no clear opportunity attached

Use these insights to inform your next prioritization conversation.

Exercise: Practice Two-Way Door Thinking

Time: 30 minutes
Do this: With your product trio

Choose 3-5 recent or upcoming product decisions. For each one, discuss:

• Is this a one-way door decision (hard to reverse) or a two-way door decision (easy to reverse)?
• If it's a two-way door, what's the smallest step we could take to learn whether we're on the right track?
• What would we need to see to know we made the wrong choice?
• If we realize we're wrong, how quickly could we course-correct?

The goal is to calibrate your team's decision-making speed. Two-way door decisions should be made quickly with "just enough" evidence. One-way door decisions deserve more deliberation and data.

Go Deeper: Additional Reading

If you prefer an audio summary of this month's reading, including the book chapters and the following resources, I've included an audio version for paid subscribers at the bottom of this post.

Related In-Depth Guides

• Opportunity Solution Trees: Visualize Your Discovery to Stay Aligned and Drive Outcomes
• Customer Interviews: Uncover Hidden Insights from Every Conversation

Supplementary Reading

• Prioritize Opportunities, Not Solutions
• 7 Key Benefits of Using Opportunity Solution Trees
• Product in Practice: How 2-Way Door Decisions Helped Simply Business Learn Fast
• Product in Practice: Getting Started with Opportunity Solution Trees at SuperAwesome

Related Courses

• Product Discovery Fundamentals: Learn a structured and sustainable approach to continuous discovery.

Our Live Discussion Schedule

Our live discussion sessions are for paid subscribers. Sessions are not recorded. Invitations will go out to Supporting Members and CDH Members two weeks before the scheduled event. But reserve the time on your calendar now.

• Tuesday, June 16, 2026: 9am-10am PDT
• Thursday, September 17, 2026: 9am-10am PDT
• Wednesday, December 16, 2026: 9am-10am PST

Audio Summary

This summary was produced by NotebookLM. The sources supplied were the book chapters as well as all of the additional reading.

June Prioritizing Opportunities

0:00

/1394.242177

1×

This article is part of the CDH Book Club celebrating the 5-year anniversary of Continuous Discovery Habits. See all book club posts

Listen to this episode on: Spotify | Apple Podcasts

What does it take to build an AI customer support agent that actually knows when it can't help — and says so?

In this episode of Just Now Possible, Teresa Torres talks with Jamie Hall (Co-founder & CTO), Xharmagne Carandang (Product Engineer), and Rona Wang (Product Engineer) of Lorikeet, a startup building AI customer support concierge agents for businesses in regulated industries. Lorikeet's vision: an agent that responds like the best customer support you've ever had — one that knows you, gets things fixed, and hands off gracefully when it's out of its depth.

The team spent months exploring the wrong ideas — reflection tools, information dashboards — before a healthcare startup pulled them toward the real problem: just help us clear the inbox. Their earliest prototype was a command-line script delivering results via CSV. Today, Lorikeet runs two agents: a Concierge that handles customer tickets end-to-end, and a Coach that helps customers configure, test, and continuously improve it.

You'll hear how they built customer-configurable guardrails (and why a cannabis company's support tickets broke their first approach), designed a "resolution in the loop" pattern for human-AI collaboration, and are now flipping the configuration workflow so customers define what good looks like before they ever write a standard operating procedure.

Show Notes

Guests:

• Jamie Hall, Co-founder & CTO, Lorikeet
• Xharmagne Carandang, Product Engineer, Lorikeet
• Rona Wang, Product Engineer, Lorikeet

In this episode:

• How Lorikeet evolved from failed ops tools to a full AI customer support concierge
• The dual-agent architecture: Concierge for customer tickets, Coach for configuration and ongoing improvement
• Why "AI humility" — defaulting to human handoff when uncertain — is a core design principle
• How Lorikeet integrates with Zendesk and Intercom instead of replacing them
• The UX evolution from workflow builder to conversational interface — and why the blank chat box is still hard
• "Resolution in the loop": how human agents unblock the AI without taking over a ticket
• Why guardrails need to be domain-specific — the cannabis company story
• How customers define their own evals and guardrails through the Coach interface
• Using AI to diagnose failure modes in traces and automatically suggest fixes
• Lorikeet's product engineering culture: every engineer asks weekly what they learned from a customer

Resources & Links:

• Lorikeet — AI customer support concierge for enterprises in regulated industries
• Gradient Labs on Just Now Possible — another AI agent team in regulated financial services
• Neople on Just Now Possible — AI digital coworkers with a similar training-by-conversation approach
• Incident.io on Just Now Possible — AI SRE with multi-agent hypothesis investigation

Chapters

00:00 Meet the Team
01:05 What Lorikeet Builds
02:34 Origin Story and Early Missteps
06:42 Finding the Real Support Pain
07:37 Why AI Fits Support Work
11:16 First Prototype and Early Evals
14:42 Design Partners and Selling the CLI
16:30 Product Mindset and the Real Moat
19:47 Rona Joins and Scaling Up
21:02 Milestones Voice Actions Escalation
23:48 Integrations with Zendesk Intercom
25:59 How the Agent Works Today
28:30 Coach Agent and Configuration UX
32:58 SOPs to Test Cases
34:35 Refund Flow Setup
36:12 Coach Conversational UI
38:12 Hybrid UX Guidance
40:46 Resolution in Loop
43:17 Collaboration Middle Ground
49:40 Process Maturity Limits
53:30 Confidence and Guardrails
55:59 Customer Defined Guardrails
01:01:14 Trace Diagnosis Agent
01:03:14 Product Engineers Culture
01:07:46 Closing Thoughts

Full Transcript

Podcast transcripts are only available to paid subscribers.

Listen to this episode on: Spotify | Apple Podcasts

We built the internet. We championed social media. We're now building AI. But what if, looking back, we made things worse?

In this candid and wide-ranging conversation, Teresa and Petra wrestle with a question that's hard to ask from inside tech: is the industry we've dedicated our careers to actually net positive for the world? From a chance comment by a non-tech family member to jarring observations about human isolation in San Francisco, this episode doesn't shy away from the discomfort.

They explore the role of greed in how technology has evolved, what the "tech bro" narrative is costing us in human terms, the loneliness epidemic, and why speaking to people outside our industry echo chambers might be one of the most important things we can do. But they don't stop at critique — they also cast a genuinely hopeful vision for what responsible, community-centered technology could look like, including the idea of "mom and pop tech" that serves neighborhoods, not shareholders.

If you work in tech and you've ever had a nagging feeling that something's gone wrong — this one is for you.

Key Takeaways

• The honest self-reckoning: Both Petra and Teresa reflect on having been active contributors to technologies — social media, the internet, AI — that have had real negative consequences, even when built with good intentions.
• Greed as the root cause: The shift from "make people's lives better" to "extract maximum value at any human cost" is framing a lot of what feels broken about tech right now.
• The loneliness crisis is real: Technology has been used to replace human connection rather than supplement it — and the two hosts see this as one of the most pressing consequences of how the industry has operated.
• The echo chamber problem: Teresa's floored reaction when a family member called the internet "net negative" is a reminder of how insulated tech workers are from how the rest of the world experiences their work.
• "Mom and pop tech" as an antidote: Rather than massive horizontal platforms built to scale, there's a hopeful model emerging where AI enables hyper-local, community-specific software — bespoke tools built by and for neighborhoods, not VCs.
• Changing parent norms around phones: Petra shares a striking observation from her daughter's school in Germany — where just five years ago half of 7–8 year olds had phones, parents now unanimously agree to hold off until age 11 or 12.
• We still have agency: Technology is a tool. The future isn't written yet. Choosing who you work for, what you build, and what narratives you amplify all matter.

Topics Discussed

• The personal and professional reckoning of working in tech during a period of visible harm
• Social media's broken promise: democratization vs. polarization
• The San Francisco "isolation by design" observation — QR codes, Waymos, and the disappearance of human interaction
• The greed cycle in tech: comparing today's climate to the railroad tycoon era
• Tech layoffs as a symptom of a deeper value problem
• What it means to be a responsible citizen in the tech industry
• The "inch wide, mile deep" approach to community-focused tech
• Ride share as a case study in extractive platform capitalism — and what a local alternative could look like
• The role of product people as educators and narrative-spreaders
• Why human physical touch, care, and community can't be automated

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in the episode:

• Product at Heart - Petra's conference, mentioned in the context of sharing alternative narratives about technology's future
• Arne Kitller of Product at Heart
• How to Save Democracy — podcast episode Petra listened to on community spaces, climate resilience, and local tech models
• Don’t be evil - Google’s former motto, and a phrase used in Google’s corporate code of conduct
• Elternabend
• Bowling Alone: The Collapse and Revival of American Community — book referenced by Teresa on the loneliness epidemic in the US
• Ronnie Varghese

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Audio Version ($)

Every morning I wake up and Claude has already added several tasks to my to-do list.

If I have a podcast recording on the calendar, my podcast-manager agent (powered by Claude) prepares a podcast-interview-prep task with a summary of who I'm interviewing and what they are building. It also creates a transcript review document and sets the right share settings. After the recording, it adds a task to my to-do list to share the transcript with the podcast participants.

If I have a sales call, my sales-admin agent (also powered by Claude) will prepare a sales-meeting-prep task for me with notes on who I am meeting with, where they are in the sales process, and anything else I need to know to move the deal forward. After the call, the sales admin creates tasks with all my next steps.

Every week, I get a report added to my Monday to-do list. It's generated by my coding-manager agent (still powered by Claude). It scans through all my coding sessions from the prior week and gives me tips on how I can improve. The tips cover common mistakes or dead-ends that I keep running into and how to fix or avoid them and ways I can work better with Claude. It's my weekly retrospective.

Today, I'm going to walk through how I get Claude to do tasks for me while I'm away from the computer.

I was inspired to build this system because of the growth and popularity of OpenClaw. OpenClaw is an open-source agent harness that allows you to set up personalized agents that work on your behalf. But it has some downsides.

When OpenClaw first came out, it required careful configuration to use safely. People enamored with the promise of a personalized agent gave it broad access to their machines (browser, terminal, files, credentials), installed third-party skills, and ran up large usage bills.

After hearing one too many horror stories about hours wasted and surprise bills, I wanted a safer way to explore the advantages of OpenClaw while also managing the costs. That's what inspired my agent setup.

If you are new to my Claude Code series, I've covered a lot of ground.

• Claude Code: What It Is, How It's Different, And Why Non-Technical People Should Use It
• Stop Repeating Yourself: Give Claude Code a Memory
• How to Use Claude Code Safely: A Non-Technical Guide to Managing Risk
• How to Choose Which Tasks to Automate with AI (+50 Real Examples)
• How to Build AI Workflows with Claude Code (Even If You're Not Technical)
• How to Use Claude Code: A Guide to Slash Commands, Agents, Skills, and Plugins
• Context Rot: Why AI Gets Worse the Longer You Chat (And How to Fix It)
• How to Share Your AI Context and Skills Across Devices

An Overview of How My Agent Team Works

I currently have three agents who work for me: a podcast manager, a sales admin, and a coding manager. I suspect my team will grow as I have more time to invest in this strategy.

This system works based on four key components:

1. Agent identity - a markdown file that tells the agent who it is, where its task folder lives, and provides context for the types of tasks it will do.
2. Scheduler - I'm using MacOS's built-in scheduler (via LaunchAgents). This is like cron, but runs with all your user permissions on Mac. That means I can run all of this under my Claude Code Max subscription or my ChatGPT/Codex subscription.
3. Tasks - each agent has a folder of tasks. A task is a markdown file with frontmatter.
4. Scripts - each agent has its own scripts folder that includes utilities that it can either run (as needed) or that run on a schedule.

Agent identity, tasks, and scripts are saved in Obsidian—not Claude Code skills or agents. The scheduler runs on my always-on Mac Mini. The benefit of this is it just works across all of my devices and I can seamlessly switch between Claude Code, Codex—or any other coding CLI—as I need to. All it takes is updating my script that the scheduler uses.

In the rest of this article, I'll walk through each of these four components in detail. And you'll get step-by-step instructions for how to set this up yourself.

Product Talk is a reader-supported publication. The rest of this article is for paid subscribers. If you haven't already, subscribe to get full access.

Listen to this episode on: Spotify | Apple Podcasts

What happens when a product leader accidentally becomes an AI engineer? In this episode, Teresa Torres shares how she went from occasional tinkerer to spending 60% of her time doing real engineering work — building AI-powered tools for continuous discovery, forming a licensing partnership with Vistaly, and quietly constructing "Teresa Bot," an AI discovery coach trained on everything she's ever written.

Teresa and Petra dig into what AI engineering actually looks like in practice: context engineering, prompt writing, RAG, observability, evals, and why Teresa thinks product managers who want to do great discovery might secretly be data scientists at heart. They also bust the myth that you need a strong engineering background to build in this space — and make the case that the most important skill right now isn't coding. It's being willing to learn.

Topics Covered

• Teresa's accidental path into AI engineering
• Her partnership with Vistaly
• Building "Teresa Bot"
• How she learned — and how you can too
• Discovery skills transfer directly to AI engineering
• On her engineering background (and why it's not what you think)
• The real takeaway

Key Quotes

"I know anything that I don't know how to do, Claude will teach me how to do. And Claude is infinitely patient." — Teresa Torres

"I don't want to build all that stuff. I don't really want to be a software company. I'm almost set up like an AI researcher." — Teresa Torres

"The moment I learned more about data science, all of my discovery work became so different." — Petra Wille

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode:

• Behind the Scenes: Building the Product Talk Interview Coach blog by Teresa
• Claude Code
• Vistaly, an opportunity solution tree software partnering with Teresa
• Opportunity Solution Trees: Visualize Your Discovery to Stay Aligned and Drive Outcomes by Teresa
• Product Talk Academy by Teresa
• Just Now Possible Podcast by Teresa, which is a podcast on teams building AI products
• Vibe Coding Best Practices: Avoid the Doom Loop with Planning and Code Reviews blog by Teresa which includes the full Claude conversation used to build Teresa Bot
• AI Evals for Engineers and PMs course by Hamel Husain and Shreya Shankar (get 35% off through Teresa’s link) on Maven
• CDH Membership (includes Slack community access with Teresa and other continuous discovery practitioners)
• Previous All Things Product episode: Product Builder Myth

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Listen to this episode on: Spotify | Apple Podcasts

Only 10% of the plastic we manufacture gets recycled. We've been trying to solve this for a hundred years using the same mechanical and chemical tools that created the problem. What if biology—specifically, engineered enzymes—is the missing piece?

In this episode of Just Now Possible, Teresa Torres talks with Arzu Sandıkçı (co-founder and CEO) and Mert Topcu (co-founder) of Rhea's Factory, a startup using engineered enzymes and AI to achieve what mechanical recycling can't: breaking plastic all the way back to its original molecular building blocks.

Arzu brings a background in molecular biology and enzyme engineering. Mert brings 20 years in tech, including a decade at Google as a product manager. Together, they've built an AI platform that uses protein language models, multi-step agentic pipelines, and proprietary wet lab data to design novel enzymes that deconstruct plastic polymers into their original monomers—selectively, at low temperatures, and at industrial scale.

You'll hear how they evolved from a human-orchestrated pipeline to an agentic AI scientist, why they sometimes want the model to hallucinate, and what it means to explore an enzyme design space that makes everything nature has ever evolved look like a tiny dot.

Show Notes

Guests

• Arzu Sandıkçı, Co-founder & CEO, Rhea's Factory
• Mert Topcu, Co-founder, Rhea's Factory

In this episode:

• Why only 10% of plastic gets recycled—and why mechanical and chemical methods hit a ceiling
• How enzymatic recycling breaks plastic all the way back to its original monomers, unlike traditional methods that just shorten polymer chains
• Why enzymes are selective: they can target specific plastic types even in mixed waste streams
• The discovery of a plastic-eating bacteria in Japan that opened the door to enzymatic recycling
• How AlphaFold and the Nobel Prize in Chemistry transformed what's possible in enzyme engineering
• How Rhea's Factory uses protein language models (PLMs) and multi-step AI pipelines to design novel enzymes computationally
• The evolution from a human-orchestrated pipeline to an agentic AI scientist
• How guardrails at each pipeline step keep the AI pointed in the right direction without limiting exploration
• Why wet lab data—even just hundreds of proprietary data points—can be enough to train a powerful domain-specific prediction model
• Why Mert sometimes wants the model to hallucinate (and how high temperature settings help explore the full enzyme design space)
• The business constraint: enzymatic recycling must compete economically with cheap, oil-based plastic production
• What's next: a process agent, a 5,000-ton demo plant in California, and enzymes for new plastic types

Resources & Links

• Rhea's Factory — Enzymatic plastic recycling technology
• AlphaFold — DeepMind's AI system for protein structure prediction (inspiration for the Nobel Prize in Chemistry)
• Maven AI Evals Course — The course Teresa took to learn about evals (35% off with Teresa's affiliate link)

Chapters

00:00 Meet the Founders
01:50 Why Plastic Circularity
03:19 Mechanical vs True Recycling
04:52 Biology as the New Tool
07:20 Necklace and Pearls Analogy
13:22 Low Energy Reactor Process
17:33 Origin Story and PET Enzyme
22:52 Protein Folding and AlphaFold
28:32 AI Designed Enzymes
34:28 Protein Language Models Stack
37:14 Multi Step Protein Generation
39:00 Building on Foundation Models
40:50 Lab First Success Metrics
43:10 From Human to Agentic Orchestration
43:59 Problem Statements as Inputs
46:18 Guardrails at Every Stage
47:48 Prediction Models and Data Limits
50:03 Industrial Reality and Cost
52:30 Agentic Parallels and Orchestrators
57:45 Impact on Timelines and Diversity
01:03:23 When Hallucination Helps
01:04:09 Scaling Up and Process Agents
01:06:56 Enzyme Blends for Mixed Plastics
01:07:49 Why Clamshells Aren't Recyclable
01:09:34 Closing Thoughts and Thanks

Full Transcript

Podcast transcripts are only available to paid subscribers.

Audio Version ($)

I just finished an all-out engineering sprint. That sounds weird to me.

I've been writing code on and off for several years. But I wouldn't call myself an engineer. I'm a product manager. And I used to be a designer. It's been a long time since I've written code for a living.

But AI is changing what's just now possible. Not just for the products we build. But also for us. We can do more than we might have ever imagined. That's certainly been my experience.

Today, I want to share an engineering story with you. There will be some technical bits. Some of it might feel foreign to you. But I want you to know I had no idea how to do any of this a year ago. I learned it with the help of AI. My goal in sharing my engineering stories is to show you what's just now possible.

Creating AI-Generated Opportunity Solution Trees

In mid-February, I announced that I was building two services for Vistaly—AI-generated interview snapshots and AI-generated opportunity solution trees. We both made a call for alpha partners. We got over 100 applicants before we closed the round. We selected eight to be our initial design partners.

Each team uploaded three customer interviews. We identified key moments and opportunities from each, then generated an opportunity solution tree from those three snapshots. I'm providing the AI services. Vistaly is building the snazzy UI and workflows around it.

Our initial feedback was strong. Teams started to ask us to let them upload more interviews. But we both had work to do to support this.

Integrating New Interviews into an AI-Generated Opportunity Solution Tree

Updating an opportunity solution tree with new interview content is a much harder challenge than generating a new opportunity solution tree from scratch. I initially underestimated the complexity.

Vistaly and I both have a shared goal. We don't want to produce AI-generated opportunity solution trees and have you take them as fact. We want you to engage with them, correct them, collaborate with the AI. We want to scaffold cross-interview synthesis, not do it for you.

To support this, we needed a way to communicate how your tree would change after you uploaded new interviews. We took inspiration from git diff and decided we needed the equivalent for opportunity solution trees. If you aren't familiar with git diff, it's how git—the most common version control software for coding—shows you what has changed in a file. It walks you through each change one step at a time.

While this was absolutely the right decision to make, I had no idea how much work it would create for me. It wasn't enough for my service to generate an updated opportunity solution tree with the content from the new interviews. It also had to provide a step-by-step walkthrough of what changed.

The Prototype Only Represents 20% of the Work

One of the themes that I hear often on my podcast Just Now Possible is that it's fairly easy to get to an impressive prototype with AI, but that it's much harder to get to a real production-quality product. This was exactly my experience.

My opportunity solution tree service is comprised of two sub-services: 1) generate a new tree from scratch and 2) update an existing tree based on new interviews.

The first service was part of our initial alpha and was working well. The second service had to be built from scratch before any of our design partners could add a fourth interview.

On the surface, these don't seem that different, but they are. When you are updating a tree, you want to preserve as much of the existing structure as you can—unless the new content really warrants a structural change. You have to account for compound operations like merges, splits, and deletes. And you need to make sure you don't lose any data. On the data front, each node in the tree has both source opportunities—interview opportunities that provide supporting evidence for the tree opportunity—and children—tree sub-opportunities.

But like with many AI products, I was pleasantly surprised when I got something working reasonably well in just a few days. I shipped it and our design partners started playing with it.

One design partner hit our beta limits quickly and asked if they could convert to a paid subscription so that they could get full access. We were thrilled. They showed a willingness to pay, converted, and immediately started uploading more interviews.

However, once they hit the 14th, 15th, and 16th uploads, my initial service design started to show some cracks. We started to see some odd behavior.

The Vistaly team noticed that the change sets—the step-by-step instructions emitted by my service—weren't always generating the final tree emitted by my service. Remember, we wanted to be able to walk the customer step-by-step through what changed. So my service returned both the step-by-step instructions (the change sets) and the final tree.

The intent was that Vistaly could start with the current version of the team's tree, apply the change set in the order I provided, and end up with the updated tree that my service emitted. But sometimes the tree that the change set generated was different from the tree that the service outputted.

Vistaly flagged this as a bug on the day I was flying to New Orleans for Jazz Fest. Looking back, I'm glad I had no idea what was waiting for me when I got home. I had about 80% of the work still left to do to make updating an opportunity solution tree a truly rock-solid product. Thankfully, I got to enjoy Jazz Fest first.

Uncovering the Full Scope of the Problem

When I got home, I started to diagnose the problem and I was quickly overwhelmed. My service was a pipeline that combined LLM steps with deterministic code.

At a high level, here's how it worked: I had deconstructed updating an opportunity into a series of steps that could be done by an LLM. After those steps completed, deterministic code compared the output tree to the input tree and generated the change sets.

As I started to dig into the bugs, I realized this was not a good approach. Tree diffs, unlike linear document diffs, are ambiguous.

If I'm editing a document and I add a sentence, the diff is simple. It shows that a sentence was added. If I delete a paragraph and rewrite it, the diff shows the removal of the old paragraph and the addition of a new paragraph.

But if I split an opportunity A into two opportunities, A and B, and then later I merge B with C, the first move is hidden from the diff.

Let me walk you through why.

If the model decides to split an opportunity into two different opportunities, it has to figure out how to divide the original opportunity's source opportunities (the supporting evidence) and its children (the sub-opportunities) across the two resulting opportunities.

For example, if we have opportunity A and we want to split it into A and B, we have to figure out which source opportunities and children belong on A and which belong on B.

Suppose A has source opportunities 1, 2, 3 and children x, y, z. After the split, we might end up with A having source opportunities 1 and 2 and child x and B having source opportunity 3 and children y and z. When the model decides to split an opportunity it has to distribute the source opportunities and children appropriately.

Now suppose the model decides to merge B with C. When two opportunities are merged, we have to combine all of the source opportunities and children. Remember, B has source opportunity 3 and children y and z. If C has source opportunities 4 and 5 and children t, u, and v, then after the merge C will have source opportunities 3, 4, and 5 and children t, u, v, y, and z.

At the end, when we do a diff between our input tree and our output tree, we can't see both the split and the merge. Our input tree had A with source opportunities 1, 2, 3 and children x, y, and z and C with opportunities 4 and 5 with children t, u, and v.

Our output tree has A with source opportunities 1 and 2 and child x and C with source opportunities 3, 4, and 5 and children t, u, v, y, and z. It looks to our diff like some of A's sources and children moved from A to C. But we don't know why. The split and the merge conceptually explain what happened, but those moves are invisible to the diff.

This is when I realized tree diffs were much harder than linear text diffs. We weren't just trying to show the user what changed. We wanted to show the user why it changed so they could weigh in. I needed a way to reconstruct each move—step by step.

That meant I had to get the model to show its work. And that opened a whole new can of worms.

Asking the Model to Show Its Work

I decided to refactor all of my prompts to have the model generate both the final output and the step-by-step moves it took to get there.

My prompts already told the model what actions were possible: add, delete, reframe, merge, split, etc. But these were just guidelines. The model could do whatever it wanted to produce the final tree.

But now, I needed it to show its work. And I needed it to show its work in the language of our change sets—specific moves that we predefined that make sense to our end user. In other words, split A into A and B and then merge B into C has much more semantic meaning than simply saying move some of A's sources and children to C arbitrarily.

For each LLM step, the model had to output its recommendation and the change set it used to generate the output. This worked fairly well. But, of course, the model made mistakes.

After a lot of testing and error analysis, I uncovered two categories of errors:

1. The model tried to make a move that was not valid.
2. The model had a discrepancy between the change set and the recommendation. (In other words, the change set didn't generate the recommendation.)

Category 1 was interesting. It felt like I was designing a game and the model was a very creative player. It kept finding ways to break my game. For each of these errors, I had to decide if this move should be allowed and if so, how the model should represent that move using our actions. If the move shouldn't be allowed, I had to let the model know.

For example, the model tried to merge a parent with a child. This was a problem. Suppose opportunity A has children B, C, and D. If the model tries to merge A with B, what should happen?

A merge is directional. The model has to define which of the two to keep and which to delete.

If it defines the merge as keep A and delete B, the merge is allowed.

But if it defines the merge as keep B and delete A, what happens to B, C, and D? They no longer have a parent and are now orphaned.

These were fun errors to resolve. It was like solving little puzzles.

Category 2 was much harder. I had to go through several iterations of prompt design to nudge the model toward better output.

I was able to reduce these errors to about 1 in 40 instances. But each service run has 10–20 LLM calls, so that meant I was hitting this error in about 50% of service runs. Clearly, not good enough for a production product.

At some point, I hit a wall and couldn't reduce this error rate any further. I felt stuck. And worried. We had a paying customer waiting for my fix. And several more design partners eager to try it out.

Trying to Correct the Model's Mistakes in Code

I initially tried to correct the model's mistakes with deterministic code. I had committed to Vistaly that my change sets would generate the output tree, so I needed code that verified this.

After the model generated a change set, I ran a number of quality checks on it. Were there any conflicts in the change set? A conflict might be the model deleted a tree node in one step and then later tried to use it in a merge in a subsequent step. Was there any potential data loss—were all of the source opportunities and children accounted for? Did the model orphan any tree nodes? And so on.

I thought, if I can detect these problems, maybe I can correct them. But to do that, I had to guess at what the model intended.

If the model had the following sequence in its change set—delete A, then merge A with B—what did it mean? Should I delete A or should I merge its source opportunities and children with B?

There were dozens of cases like this with no easy answers.

At this point, I was 11 days into working on this full time from the moment I woke up to the moment I went to sleep (including weekends). I don't say that as a humble brag. I hate hustle culture and this is not how I designed my life. I share this because I want you to understand the context. I was stuck and I was exhausted. And then I had an insight.

Let the Model Correct Its Own Mistakes

I was walking my dog with my husband. He is also an engineer and he had been hearing about my ups and downs over the past several days. As I explained my stubborn failure mode to him, I had an insight. I realized I could have the LLM repair its own mistakes.

My data contract with Vistaly requires that my change set generates the output tree. This was the core bug. This wasn't happening 100% of the time. Over the previous week, I had written a lot of code to validate change sets.

I now knew exactly when a change set generated the wrong tree. But no matter how much prompt engineering I did, I couldn't reduce the error rate.

My core insight was that I could use this same validation code as a tool for the agent. The agent could generate the change set, call the tool, and then get the results. If the change set was invalid, the agent would get instructions on what needed to be fixed. The agent would continue to work until the validation passed.

This was a simple idea. But could I get it to work? And more importantly, could I get it to work without blowing through a ton of compute?

I started by prototyping a solution. I wrote a simple Node.js script that made one LLM call, got the result, ran it through my verifier, and then sent the feedback to the LLM for corrections.

Initially, it didn't work at all. The model just kept introducing more mistakes. The loop never converged to a valid solution. It just racked up compute.

But I kept trying. I experimented with how to communicate the errors to the LLM. I experimented with how much context to give it about the larger problem. And eventually, I got it to work. The model was fixing its own mistakes and returning a valid change set after one or two repairs.

I was thrilled. But I still had to turn it into something I could run in production. And that's when I remembered: I had already built an agent loop utility to support my AI interviewer.

Here's what my final production implementation looks like. I have a pretty standard agent loop. It starts with a model call. The model can output a tool request. The loop runs the tool and sends the tool result back to the agent. It iterates until the tool (my validation code) returns a success signal or until we hit the max turns that I set in my code.

The initial model call is the tree-building task. The tool call runs my validation code. If the change set passes validation, the tool emits a success signal and the loop exits. If the change set fails validation, the tool returns simple instructions for the model to fix the errors. The model attempts to fix the errors and calls the validation tool again. My insight led to a feasible solution.

Errors are still theoretically possible. The repair loop can fail and hit max turns. But I haven't seen it in all of my testing. The agent typically fixes the bad change set in one to two turns.

I had that insight last Thursday. I worked through the weekend to prototype it, test it, and validate that it could work. I then built a production-quality version and integrated it into my pipeline. I shipped my completely revamped service to Vistaly on Monday at noon. They are working on integrating it and it will be available to our design partners shortly.

I was relieved. And ready for a day off.

Some Reflections on the Whole Process

The last two weeks were intense. But I learned a lot and I mostly enjoyed the process. I want to share some key reflections with you.

• I have a lot of limiting beliefs around being an engineer. I shed a lot of those beliefs over the last two weeks. To make this work well and to make it work reliably, I had to solve some legitimately hard problems. That feels good.
• This work was genuinely fun. I loved the puzzle part of designing an action set and then seeing how the model pushed that design beyond its intended limits. Models are incredibly creative and it was fun to see what it would do with a fixed toolset.
• I learned a lot about when I can and can't trust Claude to write code for me. Since Opus 4.6 came out, I gave Claude a much longer leash. After the past two weeks, Claude is back on a short leash. I found a lot of gaps in my implementation in areas where I simply trusted that Claude got it right, when in fact it didn't. I know the foundation labs are claiming that the models write most of their code, but I'm learning if you don't have the right infrastructure in place, this can be disastrous. I'll be investing much more in my planning, testing, and code-review tools over the next couple of weeks. And, of course, I'll share here as I do.
• If this work was spread out over two months, it would have been thoroughly enjoyable. I'm learning I really enjoy being an AI engineer. I was ready for a new chapter and I am even more confident this is it. It's fun to combine my past work—opportunity solution trees—with my new interests—AI engineering.

I'm really excited to share the work Vistaly and I are doing together with more people. We'll be adding more design partners soon. You'll be the first to know when we do. In the meantime, get on the waiting list.

And finally, it's still surprising for me to look back at the last 15 months and see how much I've learned and progressed in a whole new field. I have fully embraced what is just now possible. I hope by sharing my story, I inspire you to take the first small step toward what's just now possible for you.

Audio Version

The audio version is only available for paid subscribers.

Listen to this episode on: Spotify | Apple Podcasts

Is the "product builder" trend the future of product management—or just the latest thing everyone thinks you should be doing?

In this episode, Petra Wille and Teresa Torres push back on the hype. They dig into whether product managers should be vibe coding, what actually determines if it works (spoiler: it's your org setup, not the tools), and why "just because you can doesn't mean you should." They also break down the three distinct layers where AI is changing product work—and why mixing them up causes chaos.

Whether you're a PM wondering if coding is now part of your job, or a product leader figuring out how to set your team up for success, this one cuts through the noise.

Show Notes:

• Why "should product managers code?" is the wrong question
• Is it fun for you? Do you have enough experience to do it well? Two better questions to start with
• What a well-set-up org looks like vs. one that's headed for chaos
• The design system trap: why AI makes bad decisions look good on the surface
• Output-oriented muscle memory: what teams do with time freed up by AI reveals where they really are
• The three layers of AI's impact: personal productivity, team process, and product strategy — and why they're different stacks
• Why discovery still requires talking to your customers (sorry)
• The real takeaway on product builders: not everyone has to build, but everyone can if they want to

Key Takeaways:

• The product builder trend isn't a mandate — it's a tool. Let enjoyment and skill guide who on your team leans into it.
• AI can make unskilled work look polished. That's a feature and a bug — executives see the shine, engineers inherit the mess.
• Organizational readiness determines whether AI empowers your team or creates chaos. That's a leadership problem, not a tooling problem.
• Don't conflate the three layers: personal efficiency, process change, and product impact require different responses.
• Discovery fundamentals haven't changed. AI helps you go deeper, not skip the work.

Quotes:

"Just because I can do it — is it something I enjoy doing? And do I have enough experience to really get into the flow?" — Petra

"It's a tool in our toolbox. We can decide who on our team has fun with it, wants to do it, wants to contribute." — Teresa

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode:

• Claude Code
• Vibe coding
• Ruby on Rails

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Listen to this episode on: Spotify | Apple Podcasts

Is "taste" the must-have skill of the AI era — or just the latest tech buzzword? In this episode, Petra Wille and Teresa Torres unpack the growing hype around taste as a differentiating human trait in a world where AI is eating through design, delivery, and discovery. Teresa pushes back hard: taste is rarely defined, can't be easily taught, and risks becoming a cover story for "my preference trumps yours." Petra adds nuance, acknowledging the real pattern-recognition that comes from years of product experience — while questioning whether it's actually worth investing in. Together they trace the idea back to its roots: product sense, founder mode, and the enduring myth of the lone visionary. What they land on instead? Discovery skills, customer understanding, and learning to collaborate with AI — because taste without evidence is just opinion.

Show Notes

• What Is "Taste" — and Why Is Everyone Suddenly Talking About It?
• The Problem with Defining Taste
• Taste vs. Preference
• Will AI Eat Taste Too?

So What Should You Actually Invest In?

• Discovery skills — understanding customers, matching solutions to real needs
• Human-to-human interaction skills
• Learning to collaborate with AI effectively
• Critical thinking and judgment grounded in evidence

Key Quotes

"It's just this month's flavor of founder mode." — Teresa Torres
"It's not about your taste. It's about your customer's taste." — Teresa Torres
"I am a strong believer that you develop product sense and taste over time. It's never finished." — Petra Wille

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode:

• Founder mode
• Marty Cagan: Founder-Style Leadership
• Vercel/v0 CEO Guillermo Rauch on building taste: from Lenny Rachitsky’s Linkedin post
• Continuous discovery (Read Teresa’s Everyone Can Do Continuous Discovery—Even You! Here’s How
• The "great man" theory
• Steve Jobs and the myth of the lone product visionary

Join the Conversation:

Have thoughts on this episode? Leave a comment below.

Full Transcript

Full transcripts are only available for paid subscribers.

Continuous Discovery Habits turns five this year. And to celebrate we are reading the book together.

Each month, I am releasing an in-depth reading guide that includes:

• The chapters we will be reading
• A preview of the most important concepts we'll be learning about
• Short videos you can share with friends and colleagues to help spread the ideas
• Individual and team discussion questions to help you absorb and engage with the reading
• Team exercises to help you put the ideas into practice
• Additional reading to help you go deeper on the core ideas

We'll be discussing each month's reading in the comment section and we'll gather quarterly to discuss on a live call.

Joining late? No problem. I monitor the comments on each reading guide throughout the year. Start with the current month or go back to January—whatever works for you. You can ask for help, share what's working, and connect with other readers at any point.

If you want to participate, grab a copy of the book (or dig up your old copy), share the "Spread the Love" videos, reserve some time to do the team exercises, and register for the community sessions. Let's do this!

This Month's Reading

Chapter: Chapter 6: Mapping the Opportunity Space

Estimated reading time: ~23 minutes

This month's chapter will introduce you to:

• Why opportunity mapping is critical for structuring the ill-structured problem of reaching your desired outcome
• How to move from overwhelming opportunity backlogs to well-structured opportunity spaces
• The power of tree structures for depicting parent-child and sibling relationships between opportunities
• How to identify distinct branches in your opportunity space using key moments in time
• Common anti-patterns to avoid when building your first opportunity solution tree
• Why structure "gets done, undone, and redone" as you continue to learn

Need a copy? Grab the book

Share the Love with Friends and Colleagues

We learn best in community. Use the following short videos to share the key concepts from this chapter with friends and colleagues. Invite them to participate in the book club with you.

• The need for opportunity mapping - You will never fully satisfy your customers' desires
• Understanding the structure of an opportunity solution tree - Depicting two types of relationships
• Turn big intractable problems into smaller, more solvable problems - The power of decomposition
• How to map an opportunity space - Getting started with opportunity solution trees
• A well-structured opportunity space has distinct branches - Identify key moments in time

Reflect & Discuss What You Read

When we reflect and discuss what we read, we absorb more of the material. It helps us put what we learn into practice. Don't skip this step.

This chapter challenges us to shift from reacting to every customer need we hear to being systematic about structuring the opportunity space. Most of us have felt overwhelmed by the sheer volume of customer opportunities we could address. This month, focus on how your team currently organizes (or doesn't organize) the opportunities you hear from customers.

Individual Reflection

1. Think about your current product backlog or opportunity list. Is it a flat list, or do you have some structure to it? If you were to group similar opportunities together, what patterns would emerge?
2. When was the last time you heard a customer need and immediately jumped to a solution without exploring whether there were related opportunities? What would change if you took the time to map how that opportunity connects to others?
3. Review the anti-patterns from the chapter (opportunities framed from your company's perspective, vertical opportunities, opportunities with multiple parents, etc.). Which of these do you recognize in how your team currently talks about opportunities?

Team Discussion

1. As a team, pick a top-level opportunity you're currently working on. Try breaking it down into sub-opportunities together. Where do you struggle? Where do you disagree about how to frame or group opportunities? What does that tell you about gaps in your shared understanding?
2. Look at your experience map (from Chapter 4) and identify 3-5 distinct moments in time during your customer's experience. Could these become the top-level branches of your opportunity solution tree? Where do you see overlap, and where are there clear distinctions?
3. Discuss the quote from Barbara Tversky: "Structure gets done, undone, and redone." How does your team currently respond when you discover new information that changes how you understand the opportunity space? Do you treat your opportunity map as fixed or as something that evolves?

Put It Into Practice

Understanding opportunity mapping is one thing. Actually building your first opportunity solution tree is where the real learning begins. These exercises will help you get hands-on experience with structuring the opportunity space.

Exercise: Build Your First Opportunity Solution Tree

Time: 60 minutes
Do this: With your product trio

Start by reviewing your interview snapshots from the past few weeks. For each opportunity you captured, ask the three questions from the chapter:

• Is this opportunity framed as a customer need, pain point, or desire (not a solution)?
• Is this opportunity unique to one customer, or have we seen it in more than one interview?
• If we address this opportunity, will it drive our desired outcome?

Then, using your experience map, identify 3-5 distinct moments in time to serve as your top-level opportunities. Group the opportunities from your interviews under these top-level branches.

Look for opportunities to add structure to each branch:

• Group similar opportunities together and identify a parent opportunity
• Look for vertical stacks (one parent, one child) and fill in missing siblings
• Reframe opportunities that are too broad or that could live in multiple branches

Don't aim for perfection. Just get something on paper (or on a digital canvas). You'll iterate on this tree with every interview you do.

Exercise: Practice Framing Opportunities from Your Customer's Perspective

Time: 30-45 minutes
Do this: With your product trio

Take 10-15 opportunities from your current backlog or list. For each one, ask: "Can I imagine a customer saying this?"

If the answer is no, reframe it from your customer's perspective. For example:

• "Increase subscription conversions" becomes "I want to know if this product is worth paying for"
• "Reduce support tickets" becomes "I can't figure out how to do X"
• "Improve onboarding completion" becomes "I'm not sure what to do next"

This exercise helps you spot business-centric opportunities disguised as customer opportunities. It also trains your team to listen for opportunities in interviews that are framed from the customer's point of view.

Go Deeper: Additional Reading

If you prefer an audio summary of this month's reading, including the book chapters and the following resources, I've included an audio version for paid subscribers at the bottom of this post.

Related In-Depth Guides

• Opportunity Solution Trees: Visualize Your Discovery to Stay Aligned and Drive Outcomes
• Customer Interviews: Uncover Hidden Insights from Every Conversation

Supplementary Reading

• Prioritize Opportunities, Not Solutions
• Product in Practice: Opportunity Mapping at Grailed
• Product in Practice: Opportunity Mapping at trivago
• 7 Key Benefits of Using Opportunity Solution Trees
• Getting Started with Opportunity Solution Trees at SuperAwesome
• Bringing Order to Chaos: Using Opportunity Solution Trees in Everyday Life

Other Voices

• Why Groups Struggle to Solve Problems Together by Al Pittampalli
• More PM Problem Areas by Marty Cagan
• Five Superpowers of Diagrams by Abby Covert
• Critical Thinking is Product Management by This Is Product Management

Our Live Discussion Schedule

Our live discussion sessions are for paid subscribers. Sessions are not recorded. Invitations will go out to Supporting Members and CDH Members two weeks before the scheduled event. But reserve the time on your calendar now.

• Tuesday, June 16, 2026: 9am-10am PDT
• Thursday, September 17, 2026: 9am-10am PDT
• Wednesday, December 16, 2026: 9am-10am PST

Audio Summary

This summary was produced by NotebookLM. The sources supplied were the book chapters as well as all of the additional reading.

May Solve complex problems with Opportunity Solution Trees

0:00

/969.688526

1×

This article is part of the CDH Book Club celebrating the 5-year anniversary of Continuous Discovery Habits. See all book club posts