For law firms, artificial intelligence has often arrived as a choice between speed and control. Stephen Costigan, founder of Atlas AI, argues that choice deserves a rethink. In this episode of The Geek in Review, we speak with Costigan about private legal AI infrastructure, knowledge graphs, and why a firm’s internal work product may become its most valuable long-term asset.

Atlas AI focuses on turning documents, matter history, precedents, clauses, parties, and obligations into a curated legal knowledge graph inside a firm’s own environment. Costigan contrasts this approach with standard vector search and retrieval systems, which find text with similar language but often lack context around clients, matters, entities, and relationships. A knowledge graph offers structure, linking people, documents, clauses, and legal concepts in ways closer to how lawyers understand their work.

The conversation also explores data quality, a subject with enough baggage to fill a records room. Costigan argues firms no longer need year-long cleanup projects before seeing results. Agent-led curation, entity extraction, duplicate resolution, and ontology mapping reduce much of the manual sorting traditionally associated with knowledge management. Human judgment still matters, especially around practice-area vocabularies and lower-confidence results, but the machines get assigned more of the janitorial work.

Security and governance sit at the center of Costigan’s model. Rather than asking firms to trust a vendor’s assurances around privileged data, Atlas AI runs within a firm’s Azure environment, under firm-controlled keys and policies. Costigan frames this as a shift from confidentiality as a contractual promise to confidentiality as an architectural decision. For legal organizations handling sensitive client information, the location of data, embeddings, audit trails, and model interactions matters as much as the interface lawyers see on screen.

Looking ahead, Costigan predicts a divide between firms renting generic AI tools and firms building durable knowledge infrastructure from their own experience. As routine drafting, diligence, and review work compress, firms with structured and reusable internal intelligence may productize expertise, offer new fixed-fee services, and rely less heavily on traditional leverage models. The future question, Costigan suggests, will not center on which AI tool sits on a lawyer’s desktop. The bigger question will ask who owns the knowledge behind the work.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Nikki Shaver (00:00)
Hello Marlene and Greg, coming to you live today from Toronto, Canada. This is Nikki Shaver, the CEO of Legal Tech Hub. I wanted to share something with your audience from the FT Innovative Lawyers Summit last week in London. It was a great event because it brought together people from across legal verticals, including lawyers from law firms, GCs, technologists, people in innovation, and more. One of the panels that really stuck with me was on positive psychology, which emerged as a field in the early 2000s.

A few takeaways from the panel: If you think you can, you can. If you think you can’t, you can’t. There really is something to believing in agency, in your own personal agency. Another couple of things are particularly important as we all look to drive adoption on one hand and increase or maintain engagement among lawyers and employees during this time of unprecedented change and uncertainty in the industry.

Do not let people sit in their little pockets of pessimism. It will spread. Instead, as a leader, one should focus on creating a sense of hope, agency, and a pathway forward. First, create a sense of hope, then provide a vision for the way forward. What is the path forward? Then provide people with a sense that they have agency to drive that path forward.

I love that. I think it is a good thing for leaders to remember at this time, and something for all of us to remember as we encourage people to change the way they work and adopt new technologies, tools, and ways of working. They are much less likely to do so if they do not feel that they have agency themselves. So, leaving you with that today, we will be writing about the FT Innovative Lawyers Summit. Look us up at legaltechnologyhub.com, and you will get a notification when that article comes out. Thank you so much.

Marlene Gebauer (02:26)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:33)
And I’m Greg Lambert, and today we are exploring how law firms can harness their own internal data to power the next generation of legal artificial intelligence.

To do that, I am very happy to welcome Stephen Costigan, the founder of Atlas AI. Atlas AI is an enterprise-grade legal AI platform that helps professional services teams transform their internal knowledge into a powerful private legal knowledge graph. So Stephen, welcome to The Geek in Review. Good to have you.

Stephen Costigan (03:08)
Thanks, Greg. Great to be here.

Marlene Gebauer (03:10)
Stephen, can you start by giving our listeners an overview of Atlas AI and what led you to build a platform focused specifically on firm-hosted private legal AI?

Stephen Costigan (03:20)
Absolutely. I’ll start by saying our platform turns a firm’s documents and matter history into a private, curated knowledge graph that the firm owns and runs inside its own environment. It then puts research, drafting, review, and enterprise search on top of that graph.

What led me to do this is that I have spent years building software inside demanding enterprises, including elite law firms. Most of my experience is in the AmLaw 50 space, and I kept seeing the same trap. Every AI tool asked a firm to choose between productivity and control. You either get speed by shipping your most sensitive work to someone else’s cloud, or you get control by building something you can barely operate. For a profession whose entire value rests on confidentiality and privilege, that is not a real choice.

Greg Lambert (04:19)
Yeah, and I have a side question on that. Do you think the foundational companies, Gemini, Claude, and ChatGPT, really understand the legal market and how sensitive the data is that we have?

Stephen Costigan (04:39)
Well, I think a firm should be able to use frontier AI without its knowledge ever leaving its walls. The firm’s knowledge should become an asset that it owns, not fuel for an outside vendor’s model.

Greg Lambert (04:54)
So, Stephen, when you and I were prepping for this and had a couple of conversations, you really got me thinking. I have been thinking about this for a while, but it was good to talk with somebody who is actually doing it about the underlying architecture of the knowledge graph systems you have developed at Atlas AI.

For the non-engineering types listening to this, can you explain what a legal knowledge graph is, why it is a better foundation than dumping documents into a vector database and creating a basic RAG system, and what benefit the knowledge graph provides?

Stephen Costigan (05:42)
Sure. I’ll start with an analogy. A standard vector database is like a clerk who hands you the pages that sound most similar to the question you asked. It is fast, but it has no understanding and very little context. It does not know that the company in one agreement, ACME Holdings, and the company in another agreement are the same client.

So it only knows which pages use similar words. A knowledge graph is the opposite. It is a map of your knowledge where the things that matter, clients, matters, parties, clauses, and obligations, are represented as connected entities. The graph knows that Client A has these matters, each matter has these documents, and each document contains these clauses and parties.

It captures the relationships, not only the text to which it refers. And really, AI is only...

Greg Lambert (06:45)
Yeah, I wrote an entire section of a book on this late last year. It was interesting because, in the legal industry, we all started with vector databases and thought, “This is great because it knows the kind of similar words that we can look for.” It was like the natural-language search we were promised in the early 2000s and that never really worked out.

Stephen Costigan (07:18)
Right.

Greg Lambert (07:24)
It would be great if law were flat, if our business were flat and every document were equal to every other document, or every client were equal to every other client. But it is a complicated batch of information that we deal with. I think even the legal research vendors have finally figured out that it is not flat information. There is nuance, and there are levels to it.

I geek out a little bit when we start talking about adding knowledge graphs to data. But it is interesting that you are doing that with internal information. That is something most people do not think about.

Stephen Costigan (08:01)
Right. AI is only as reliable as what you ground it in. Even with traditional search, adding vector search, adding a knowledge graph, and bringing all of those together into a hybrid search pipeline, you are still not going to achieve the level of accuracy that law requires.

That is why we have added an additional layer, which focuses on bringing the ontology into the index. We bring the firm’s ontology into the index, its information architecture, if you will, in a traditional way of describing it, and then use agents to map that data. At a high level, that is what we are doing, and that is why we are seeing big increases in accuracy when drafting and performing enterprise search.

Marlene Gebauer (08:54)
So, speaking of accuracy, these tools tend to work better when they have clean data. How much work does a firm need to put into cleaning and curating its precedents, templates, and contracts before Atlas AI can generate reliable insights?

Stephen Costigan (09:13)
I would expand this beyond Atlas AI and say far less than people fear. This is the part that many firms get wrong about AI projects. They assume a year-long data-cleaning program is necessary before any value is derived. That is not really the case anymore. Agent-led curation, as we like to frame it, is speeding up the process.

Greg Lambert (09:14)
Is the internet out?

Stephen Costigan (09:40)
Curation can now be automated and continuous. In our platform, a component we call the Librarian runs over every document as it comes in and extracts the entities and relationships. It resolves duplicates, reconciles the same party appearing in different forms, and maps everything to the firm’s ontology. This is a controlled vocabulary for legal concepts, and the graph cleans itself as it grows.

It connects to iManage and SharePoint and builds the structure automatically. What the firm contributes is judgment, not janitorial work. That is what we are removing from the equation. Sorry, go ahead.

Marlene Gebauer (10:21)
Yeah, because you would still have to determine, even if you have the infrastructure in place, which documents are important versus others.

Stephen Costigan (10:32)
Exactly. That is now possible through automation. I would say the firm needs to define and approve the ontology for practice areas and maintain a light human-in-the-loop review queue, where the system can flag lower-confidence data extractions before they are committed to the graph. That allows the firm to govern quality.

The firm does not need to hand-clean precedents anymore. I am not getting into a ton of detail about our product, but at a high level, that is what you are able to achieve now, and it is pretty incredible.

Marlene Gebauer (11:12)
And this will deploy directly into the firm’s infrastructure, such as its Azure environment.

Stephen Costigan (11:21)
Yeah.

Marlene Gebauer (11:21)
I think that is key for large law firms because of the security question. How are you seeing this change the conversation around client data security and firm governance?

Stephen Costigan (11:30)
It is changing confidentiality from a promise into an architecture. That is the best way I can describe it. In a standard SaaS platform, you are trusting a vendor’s contract. You are signing something that basically says, “Trust us with this privileged client data. We adhere to all these controls. We have enough funding. Trust us.”

Greg Lambert (11:48)
All right. Dive deeper into that.

Marlene Gebauer (11:52)
Sounds good. Tell me more.

Stephen Costigan (12:13)
“We can handle this. There will not be a data breach. And we have indemnity clauses to back it up if something happens.” That does not change anything. Client data is exposed. Trust us that we will handle it correctly. Trust that it does not train a model. For a firm with duties to its clients, trust is a weak control.

That has been our thesis since the beginning. We started as a plain private version of ChatGPT, with a few legal twists in our prompt library. Now we have expanded to cover many different focus areas and features. But with Atlas AI, the platform runs inside the firm’s own Azure environment under the firm’s keys.

The data, the graph, the embeddings, and the audit trail, none of it leaves the firm’s environment.

The models are accessed under zero-data-retention terms, so nothing is retained and nothing trains a third party’s system. That flips the governance conversation entirely. Instead of asking, “Can we get comfortable with the vendor’s data-handling procedures?” it becomes, “Can we show our general counsel and conflicts partner exactly where the data flows?” The answer is yes, because it never leaves. That is why firms will run their most sensitive matters on our platform.

They will run them on private AI infrastructure because sovereignty is total in that configuration.

Greg Lambert (13:50)
Interesting.

I have a saying that I use probably a little too much now: Lawyers tend to do better with a red pen than a blue pen. They like to have something to edit rather than create whole cloth. They do their best work when they have a solid first draft to edit and refine.

How do the agentic workflows in Atlas AI provide attorneys with that critical first draft for complex tasks such as due diligence or bulk contract review? What benefits let them dive in much faster?

Stephen Costigan (14:35)
Right. I love that framing. At a high level, we give the lawyer a first draft that has already been argued against by a second, adversarial system. They get to do their best work, the red-pen work, instead of assembly. I can go into more detail if you would like.

Marlene Gebauer (14:55)
Yeah, please do. Go ahead.

Greg Lambert (14:57)
Yep, please do.

Stephen Costigan (15:00)
What I mean is that the system is not there to replace a lawyer’s judgment. It is there to deliver a strong, cited first draft. The lawyer spends time editing and deciding, not assembling.

For due diligence, an attorney points the system at a deal document set, and it produces a diligence checklist or an issues list. Every line item carries an inline citation to the exact subsection from which it came, not simply Section 3, but Section 3.2(a).

For bulk contract review, you define the question once, such as change of control, governing law, or termination, and the system extracts structured answers across hundreds of documents into a reviewable grid. Each cell is traceable to the source. The part that makes the draft trustworthy is an adversarial verification step. One model drafts, and a separate model is tasked with arguing against it, checking that the deliverable does what was asked and that every citation is precise.

That happens before the lawyer or anyone else sees it. So the red pen the attorney picks up is editing a verified draft, not catching the machine’s mistakes or, even worse, having someone else catch them.

Marlene Gebauer (16:20)
There is always a question of build versus buy, and that continues in the market. What do you think is the primary differentiator for firms that say, “We are going to choose to build our own capabilities. We are going to build our own knowledge model, relying on our own data rather than a centralized vendor model?”

Stephen Costigan (16:50)
My thoughts are, number one, you are not going to build a differentiated practice area in your firm or maintain differentiation by buying the same product and using the same data set as everyone else.

Number two, data is the firm’s greatest asset. The signals that come off that data are being extracted. Why are you giving your data away or training another organization’s environment on those signals? But the primary differentiator is not a feature. It is ownership.

And with the... Sorry, go ahead.

Marlene Gebauer (17:33)
No, go ahead.

Stephen Costigan (17:51)
I am really backing up the points that I made. In the centralized vendor model, you are renting access to their product, interface, roadmap, and data-handling promises.

Your knowledge ultimately improves their system. With our platform, or with private AI infrastructure in general, you build a curated graph that you own outright. That compounds with every matter you work on and bring into the platform.

You can extend and build on it. You can create differentiated products for your firm, and you can do that more easily now, especially with recent advancements in agentic AI. Three things follow from that.

First, the ontology is yours and editable. You govern how your knowledge is structured, not some vendor. Second, because it is structured and resolved, the asset becomes more valuable over time rather than being consumed and forgotten after each query, as in a RAG model. Third, because it runs in your environment, you are never exposing client data to build someone else’s moat.

The crowded part of the market is selling chatbots and tooling that sit on top of a model using the same closed pattern. The durable position is helping a firm own its intelligence and knowledge infrastructure. The model layer is becoming a commodity. The curated graph is part of the defensible asset, and it should belong to the firm.

Marlene Gebauer (19:22)
What would you say?

Greg Lambert (19:23)
Is there a certain type of expertise that firms need to maintain this? If we are going to use a third party and rely on its infrastructure, is there a different type of expertise that we need internally to maintain our own version of that infrastructure? I hope that question made sense.

Stephen Costigan (19:56)
Yeah, it does. In terms of requiring an entire team to manage an infrastructure like that, I think that in the near future it will not be as much of an ask to build a part of your organization that can manage private AI infrastructure.

I think existing knowledge management roles can be adapted to the curation and management of that aspect of the environment. I say that because of the advancements in agent-based infrastructure management. For instance, in our environment, we have what we call the Enclave. It is an agent environment where pretty much everything in our infrastructure is managed automatically, in a highly governed way.

It is not that difficult to roll your own infrastructure-management agents. When we look at the amount of code written agentically in our organization, we have gone from around 10 percent to 60 or 65 percent of our platform being written agentically. Everyone thinks they are going to need a huge team to manage their own private infrastructure, but that is simply not the case anymore.

Greg Lambert (21:20)
Yeah. I know a lot of us are looking at the Kirkland advertisements for GPU professionals and seeing that kind of build-it-almost-from-scratch, nearly-on-your-own approach. With Atlas AI, I am not going to have to hire people to stand up GPUs and monitor them, right?

Stephen Costigan (21:42)
Exactly.

Marlene Gebauer (21:43)
I had a question because you mentioned before that the legal AI market is incredibly saturated, which it is. But I am not sure the same is true from a knowledge management perspective. There seems to be a more limited set of tools that deal specifically with that. What do you think differentiates those types of tools?

Stephen Costigan (22:10)
I do not think there are many highly effective knowledge management platforms for law firms. Again, the team size is typically one to five people in that particular area of the firm, if they are lucky. I do not think there are really any solutions out there to compare to, honestly.

Marlene Gebauer (22:13)
Correct. Mm-hmm.

Stephen Costigan (22:36)
But I think what is critical is automating the curation of the DMS. If you can do that, you have achieved a great deal for a firm at the outset.

Marlene Gebauer (22:48)
Yeah, that is what I am saying. There seems to be a limited number of tools that say they do that. The ones that are out there still say you need to offer the system something to start with or clean up the data. I was curious whether you see other differentiators between tools in that space.

Stephen Costigan (22:55)
Yes. The thing is, it is not just about curating the data. It is about what happens after that, too. There are many paths one can take as a founder building a product. The path we selected was to get away from building features to compete with everyone else on the end-user side of the product and to enable the democratization of feature development.

You can take Claude Code, for instance, use our MCP, access your curated data set, and build whatever applications you want, either separate from or integrated into our platform. That is something I think is truly unique.

It takes firms away from having to follow the vendor roadmap and allows them to start building differentiated products for their firm immediately.

Greg Lambert (24:05)
Stephen, before we get to our crystal ball question, we have been asking guests to share some of the ways they keep up with the market. There is so much to try to keep up with. Do you have any resources you do not mind sharing that help you stay current on the transitions in technology?

Stephen Costigan (24:17)
We really rely on Legal Tech Hub for market signals, and Artificial Lawyer.

From a knowledge graph and data architecture standpoint, the foundational knowledge graph work by Hogan and colleagues is great. Those are the high-level sources. Ethan Mollick’s Applied AI is helpful for understanding how professionals actually adopt these tools. Those are my recommendations.

Greg Lambert (24:39)
Yeah. I have an Artificial Lawyer story to tell on this. I was on Richard’s podcast a few weeks ago, and I was in my Austin office when one of the attorneys came up and said, “I heard your podcast interview.” I said, “Which one?” They said, “The one you were on with Richard.” I said, “Well, have you ever listened to my podcast?” They said, “No, I have not listened to that yet.”

Stephen Costigan (25:20)
Exactly.

Greg Lambert (25:27)
So Richard does a pretty good job.

Marlene Gebauer (25:29)
Yes.

Stephen Costigan (25:30)
Absolutely. I try to read across two lanes that I do not know if they always talk to each other, but I think that is changing, and we want to drive that change. Those lanes are the legal innovation world and the knowledge representation world. The interesting work is at the seam between them. That is something no one is really attacking right now, so it is something that really excites me.

Marlene Gebauer (25:55)
Yeah, I think that is spot on.

It is time for a crystal ball question. Looking ahead, do not be scared. We will only come back next year to see whether you were right.

Stephen Costigan (26:03)
I have no idea what that is, so I am scared.

Greg Lambert (26:10)
I am just guessing.

Stephen Costigan (26:12)
Okay.

Marlene Gebauer (26:24)
What is the single biggest shift that you see coming for the traditional law firm business model?

Stephen Costigan (26:30)
Ownership. The biggest shift is that a firm’s accumulated knowledge stops being a byproduct and becomes its most leveraged asset. That really breaks the math on which the billable hour rests. The traditional model monetizes leverage.

Partners sell associate hours against precedents that live in people’s heads and scattered files today. That is changing, obviously. When that precedent becomes a curated graph, routine production work compresses dramatically.

The firms that own that graph will deliver senior judgment with far less junior leverage. They will be able to productize their expertise through fixed-fee, on-demand, client-facing offerings in ways the hourly model never allowed. The divergence I see is ownership.

Marlene Gebauer (27:34)
It is ownership and usability.

Stephen Costigan (27:36)
Usability. Firms that treat AI as a tool they rent will compete on price against everyone renting the same tools. Firms that build and own their own knowledge graphs, curate their data into those graphs, and use them effectively will turn their expertise into a durable, compounding asset. That becomes the real moat for the firm, not head count. The losers will be those who gave their knowledge away and do not have much to show for it. So there is my crystal ball response.

Marlene Gebauer (28:06)
Okay, kids, you heard it here. Do not give your knowledge away.

Greg Lambert (28:10)
It is surprising that you have to tell people that, is it not?

Stephen Costigan (28:10)
Ha ha.

Just to land this point, in five years the question will not be which AI tool your firm uses. It will be whether your firm owns its knowledge or rents access to it from a bunch of knowledge providers. That is the line on which the next generation of firms will be drawn.

Greg Lambert (28:37)
Yeah, I think that is a pretty solid prediction. Stephen Costigan from Atlas AI, I want to thank you for taking the time to join us, break this down, and geek out with us on knowledge graphs and private AI. I really appreciate it.

Marlene Gebauer (28:40)
Yeah, me too. Mm-hmm.

Stephen Costigan (28:52)
Thank you so much. Thanks for having me.

Marlene Gebauer (28:55)
Yeah, thanks, Stephen.

Thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We would love to hear from you on LinkedIn and Substack.

Greg Lambert (29:06)
And Stephen, where is the best place for listeners to learn more about you and what you are doing at Atlas AI?

Stephen Costigan (29:14)
Sure. You can find me on LinkedIn, Stephen Costigan, Stephen with a PH. I am happy to talk with any firm thinking about owning its AI rather than renting it. The site is atlas-ai.io.

Marlene Gebauer (29:29)
And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

I had the pleasure of attending my first LSSO – Raindance Conference a few weeks ago where at least a half dozen times (I honestly lost count) presenters talked about signals.

Last week, I hosted an episode of Harbor’s Legal Soundings Podcast and signals came up.

There were the headlines about Kirkland’s AI investment being a signal of something, and talk of Microsoft signals.

I’m starting to think if I had a nickel for every recent mention of “signals” I’d be as rich as the people collecting nickels related to agentic AI.

Signals are not new.  Military intelligence has had the Signals Corp since the invention of radios.  Competitive intelligence professionals have always been in the business of finding signals to avoid mistakes and predict opportunity.

For decades, intelligent analysts have sifted through vast amounts of information to separate meaningful developments from background noise. The job has never been simply to gather information; it has been to transform information into intelligence. For example, at one of the firms I worked at, a Practice Group Leader called me one day and asked me to “look for the signals to determine which National oil company would invest in the Canadian oil sands next.”  Those weren’t the exact words he used, but you get the point.

What’s different in the AI era is that the economics of information have fundamentally changed.

Information itself is no longer scarce.

Every law firm and their CI/ BD practitioners now have access to AI tools that can instantly summarize earnings calls, SEC filings, regulatory developments, news articles, LinkedIn activity, job postings, patents, podcasts, analyst reports, and social media conversations. The barriers to access have largely disappeared.

The competitive advantage is no longer who has the information. The competitive advantage is who can identify and act on meaningful signals before everyone else.

This may be the single most important shift occurring in business development and competitive intelligence today.

AI Hasn’t Eliminated Analysis. It Has Raised the Bar.

For years, many organizations equated competitive intelligence with information gathering: collect the data, build the dossier, distribute the report, repeat.

AI now performs much of that work in seconds. Summarization is becoming commoditized. Research is becoming commoditized. Even synthesis is becoming increasingly accessible.

As AI lowers the cost of analysis, human judgment becomes more valuable, not less.

The question is no longer ‘What do we know?’ The questions become ‘What matters, and what is likely to happen next?’ ‘Who will this impact and how can we help?’

That is a signal-detection and analysis paradigm shift.

Business Development Is Becoming a Timing Function

Business development has always been about relationships, and it still is. But passive relationships, the kind where contact is only made when a suit is filed, a transaction is imminent or there is a sporting event happening, will no longer suffice.  Success today will   depend on engaging clients at precisely the right moment.

Companies continuously emit signals: new executive hires, geographic expansion, product launches, website changes, patent filings, strategic partnerships, job postings, and regulatory disclosures, to name a few.

Individually, these data points are unremarkable. Collectively, they tell a story.

Historically, legal business development has been largely relationship-driven and reactive: build relationships, stay visible, wait for a legal event, and receive the call.

The AI era invites a different question: What signals indicate a client is about to face a legal challenge before they realize they need outside counsel?  We used to set up early warning signals at my previous firm but we were still later than we could be in today’s world. We had to wait for a class action to be filed to find it. Today, AI tools can monitor consumer complaints, regulatory investigations, product recalls, data breaches, and court filings in near real time.

The firms that recognize that story first gain an advantage because timing matters.

Law Firms Have a Unique Opportunity

Lawyers are already trained to think in this scenario planning kind of way.

They instinctively ask: What changed? What are the second-order consequences? What risks are emerging? What is likely to happen next? What similar things have happened in the past?

These are signal-detection skills. The opportunity is to apply that thinking earlier in the client lifecycle.

What Legal Signals Might Look Like

Strong signals are easy to spot. Everyone sees the merger announcement, major funding round, or significant litigation filing.

Weak signals are more interesting: a handful of AI governance hires, a subtle website update, a revised privacy policy, or participation in a new industry consortium.

Weak signals may seem insignificant but they reveal a strategic shift months before it becomes obvious. The organizations that consistently connect these dots early will outperform those that wait for certainty, because by the time certainty arrives, everyone else can see it too.

This Is Also a Talent Question

Law firms have traditionally rewarded relationship builders, rainmakers, and network strength.

Those skills remain indispensable, but firms may need to elevate curiosity, pattern recognition, industry fluency, strategic questioning, and the ability to connect weak signals into actionable hypotheses.  These may not be skills that lawyers readily possess; some firms are already creating hybrid teams that combine business development professionals, competitive intelligence specialists, knowledge management professionals, and practicing lawyers to do exactly this. Others will find that to properly detect and action the signals they need to upskill their teams, hire or outsource to stay competitive.

Conclusion

Information is no longer a scarce resource.

In the AI era, every firm can gather more, summarize faster, and monitor more broadly. The advantage belongs to the firms that can identify which signals matter, understand what they mean, and act before the need becomes obvious.

For law firms, that changes the role of competitive intelligence and business development. The goal is not simply to report what happened. It is to help lawyers and clients see what may happen next.

AI can surface the signs. Human judgment turns them into signals.

And given how often signals seem to be appearing lately — in conferences, client conversations, headlines, podcasts, and product pitches — I wonder if the The Five Man Electrical Band was song writing in 2026 instead of 1971, they would have been singing about signals instead of signs... But there is an important distinction. Signs tell you where things are. Signals hint at where things are going.

“Sign, sign, everywhere a sign.”

This week we welcome American Association of Law Libraries leaders Jenny Foster, AALL President for 2025-2026, and Jessica Whytock, AALL Vice President and President-Elect. The conversation offers a preview of the 2026 AALL Annual Meeting & Conference in Cleveland, Ohio, along with a thoughtful look at how the association is supporting legal information professionals during a period of institutional, technological, and professional change.

Foster reflects on a leadership year focused on transparency, communication, and meaningful opportunities for member participation. From strengthening channels between members and AALL leadership to intentional volunteer appointments across committees and juries, she describes an association built through relationships. The goal is to ensure newer, mid-career, and seasoned law librarians all have a visible place in shaping the profession’s future.

Advocacy also plays a central role in the discussion. Foster explains how AALL continues its work on access to legal information, public policy, and coalition-building, even amid staffing transitions. The association’s Government Relations Committee has continued meeting with members, offering advocacy training, rebuilding connections with peer organizations, and aligning its work with AALL’s strategic priorities. For law librarians, advocacy is both a long-term commitment and a practical responsibility tied to preserving authoritative legal information.

The 2026 conference theme, “Leading with Aloha,” gives the Cleveland meeting its distinct point of view. Foster shares how aloha, rooted in kindness, unity, humility, patience, and meaningful connection, became a framework for leadership during uncertain times. More than 65 programs will explore topics ranging from generative AI and legal scholarship to physical collection strategy, access challenges, and the changing role of legal information professionals. Local programming connected to Cleveland’s history will bring an added sense of place to the gathering.

Whytock looks ahead to her upcoming presidency with a focus on clear pathways for engagement, leadership, grants, scholarships, committee service, and professional growth. Both leaders see artificial intelligence as a catalyst for a deeper conversation about the identity and value of legal information professionals. Their message is straightforward: the future of law librarianship rests in human judgment, critical thinking, ethical discernment, context, access, and a community willing to bring more voices into the room. The 2026 AALL Annual Meeting in Cleveland offers a place for those conversations to move from aspiration into action.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Marlene Gebauer (00:00)
Hi, I’m Marlene Gebauer from The Geek in Review and I have Sam Moore here from Legal Technology Hub who’s going to tell us a little bit about analysis of token usage and model selection.

Sam Moore (00:11)
That’s right. Thank you, Marlene. Well, it is tokens, tokens everywhere. I think spurred on by the launch of Claude for legal, but certainly going back further than that. There’s an issue in the legal industry today around token usage in GenAI tools. And in the legal technology hub advisory team, we’ve had several conversations in the last week or two about this, both in terms of frontier models, but also in terms of the legal AI platforms.

And the topics we’re discussing with clients right now tend to fall into three interconnected topics. First is model selection, because a lot of these products give the users a choice of which model they want to use for a given prompt. But most users of these products really have no idea what the difference is. I’ve seen law firm clients whose users just pick the most sophisticated model for everything, toggle on every optional feature available.

and then are confused as to why responses are taking a long time and why they’re hitting token limits very, very quickly. The second’s around model context windows. I’ve had several conversations lately about what a context window even is and how it can create drift when it gets crowded in a chat’s context window and why that really matters for legal use cases, which often involve uploading quite large documents, which take up a lot of space in those context windows.

And finally, efficient token usage. Law firms and law departments, I think, are generally not that accustomed to this kind of pay-as-you-go model in technology. Not unless you’re like me and you recall when the big legal research platforms were on a pay-per-search basis. So now those users are running into high-cost overages on the frontier models in particular, and they’re realizing that low sticker price per month is not their reality, not when their users

don’t know how to use those tools efficiently and how to control cost. So as well as delivering advisory work on these topics on a one-to-one basis, we’re actually working on a series of articles for LTH Premium about these topics, which will then combine into a sort of playbook for our subscribers to keep handy when they’re working with Gen AI tools. And we expect to start putting out that content in early June.

And if people want to know more about LTH advisory and what we can do, they can always get in touch with us by going to legaltechnologyhub.com or by finding me on LinkedIn.

Marlene Gebauer (02:35)
Thank you, Sam, for keeping us informed about this important issue.

Sam Moore (02:39)
You’re welcome.

Marlene Gebauer (02:47)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:55)
And I’m Greg Lambert, and today we are thrilled to welcome the leadership of the American Association of Law Libraries, or AALL. Joining us today is Jenny Foster, the 2025-2026 AALL President, and joining her is Jessica Whytock, the AALL Vice President and President-Elect.

Marlene Gebauer (03:01)
Yay.

Jenny and Jessica are here to preview the upcoming 2026 AALL Annual Meeting & Conference taking place in beautiful Cleveland, Ohio, this July, and to discuss the strategic direction of the association. So Jenny and Jessica, welcome to the show.

Jenny Foster (she/her) (03:32)
Aloha, thank you so much for having us. We really appreciate it. Big fans of The Geek in Review.

Jessica Whytock (03:38)
It’s great to be here. Thank you.

Greg Lambert (03:40)
Yeah, and Marlene went to law school in Cleveland so bad memories there, so if you

Marlene Gebauer (03:47)
Not about

not about not about law school, no. It’s just

Jenny Foster (she/her) (03:51)
Crazy.

Greg Lambert (03:53)
Well, Jenny, welcome back. You joined us last year, and you were in Jessica’s shoes. So let’s talk a little bit about the presidency that you’ve had for almost a year now. I know you made a massive push for transparency and open communications within the association. In that time, I know you’ve doubled down on the eBriefing and the KnowItAALL newsletter, making sure that

that people have a direct line to leadership. On top of that, you focused on volunteerism, and there has been a whole lot going on. So talk to us a little bit about, you know, the internal and external policies you have set up, and how your year has gone.

Jenny Foster (she/her) (04:41)
Okay, how my year has gone? That’s a loaded question, Greg. I mean I think

Greg Lambert (04:45)
It’s been great.

Marlene Gebauer (04:46)
Ha

Jenny Foster (she/her) (04:49)
Everybody’s like, “How’s my year? How’s my day gone?” But, going back to transparency and open communication, your question, I was really thinking about the bones of communication and inviting people to participate in conversations with leadership. They were already there. And then, with Jessica’s help, working together intentionally, we were trying to uncover and remove barriers. What is the word? We wanted to be clear and intentional in inviting people to talk, share their perspectives, and, like you said, reach leadership. And one of the ways that we were thinking about engaging members was through volunteerism and intentional volunteerism. So I really think that

Well, as you know, Greg, right, you get elected to this position, right? And law librarians like to plan. We love a plan. We are not even in our roles before we begin thinking about vice chairs and appointments to the 42 juries and committees carrying out the work of our organization, and who we are inviting into that space. And then they will be working eventually alongside us in our presidential year. Jessica would have to speak for herself, but I know we were aligned when we were making those appointments, thinking about where members are in the organization. What setting are they in? Are they newer, experienced, mid-career, or seasoned professionals? What type of library? And really trying to be super intentional about inviting them to participate, right? Because for the newer folks and mid-career folks, these are our future leaders of the association, and they need to be invited to share space there. And then our seasoned professionals are the mentors helping people matriculate through our organization. When we invite folks into leadership spaces, it is such an opportunity to learn. I think about the spaces where I have served.

And you suddenly get put in a room with people you would not otherwise meet. But if I was only in my career track at the Hawaiʻi State Judiciary, I would have never met Jessica. What a travesty. Okay, because she’s brilliant, lovely, and I can’t wait to hand off the reins to you in July, Jessica. And I think about that also in my first committee, right? The Special Committee on Diversity, the Leadership Development Committee, and all the volunteers from all walks of life.

And then we want to create a space where people can share their perspectives, come to a disagreement sometimes because we have passionate members, but through that disagreement, really tackle the work of our association to come out with a better outcome. And we can’t do that if we don’t communicate clearly, if we don’t make space for communication in a way that’s helpful and helps each other, right? Like recognize each other. And that connection is what strengthens AALL now and in the future. Did you want to add anything, Jessica? I’m just like blah!

Jessica Whytock (07:53)
Jenny, I think the question was about your leadership year, so

Jenny Foster (she/her) (07:55)
Yeah, yeah, yeah. But that is the other thing, too.

We are all connected to one another. I mean, the historic election, right? But I recognize that I was so humbled, the first resident from Hawaiʻi to be elected to this role in a hundred and twenty years. But before me came many fabulous leaders, like yourself, Greg, and BIPOC leaders. Like it’s not done in a vacuum. We’re all here in relationship with one another. And that’s really what I was focusing on this year is like how do we invite more voices to the table to, you know, further the legal information profession.

Jessica Whytock (08:29)
You know, Jenny, I actually will add something, which is Jenny is all effusive love and hearts, but what has impressed me is the smart, thoughtful person behind all of this who has a vision and a plan and also

Jenny Foster (she/her) (08:31)
Yes, please.

Jessica Whytock (08:54)
has an astounding amount of institutional knowledge that you just share at key decision-making points. So the one thing that I hope that people understand is that Jenny’s a tremendously strong leader and that we’ve been so lucky this year to have Jenny. And that as much as we love the Aloha and the hearts, there’s wisdom there that has made a big difference in what we’ve done this year.

I know.

Marlene Gebauer (09:22)
I know, I I’m like

Greg Lambert (09:24)
And luckily she’s she’s very quiet and muted and

Marlene Gebauer (09:27)
Ha ha ha

Jessica Whytock (09:29)
You know, she says all of the words so I don’t have to say them.

Greg Lambert (09:32)
Yeah.

Marlene Gebauer (09:34)
Well, AALL has always been very strong in terms of its advocacy arm and you know historically has had a very strong voice in Washington. you know, even with the transitions in the government relations role, and I was part of that committee at one time. You’re you know, your your administration hasn’t missed a beat. So, you know, you’ve been partnering with the ALA, issuing statements on IMLS funding.

opening up communications with ALA about filing amicus briefs. So you know, how is AALL ensuring that the advocacy needs of law librarians remain at the forefront at the national level?

Jenny Foster (she/her) (10:13)
Well, the first is really embracing it, as we know we have a vacancy operationally with our advocacy position. but we also had a vacancy with our executive director, which is really important, so we had to cross that bridge first. But even still one thing at a time. But it’s side by side with that, we were still working towards advocacy. So this is my love letter to the GRC, which you used to be on, Marlene. You were on a GRC committee. Thank you.

Marlene Gebauer (10:27)
One thing at a time.

Back in the day, yeah, I was. It was my first committee,

yeah.

Jenny Foster (she/her) (10:43)
My God, and it’s so amazing, right? You learn so much. And so this is my love letter to the chairs, Chris Lund and Kristina Chamorro, for their tremendous work this year. Supporting them has been important. Shameless plug for the eLearning platform: they recently held a national advocacy training with a congressman from New York, I believe. But more importantly, they have been consciously and intentionally meeting with members. So they’ve been meeting, doing these chapter meet and greets across the nation to really have a better understanding of what are the advocacy needs. Like things

Marlene Gebauer (10:45)
Mm-hmm.

Jenny Foster (she/her) (11:11)
Have changed since you since when I first started, probably since you were in the committee, right, Marlene? So we really need to evaluate what is effective and where our common goals are. But that’s also outlined in our strategic plans because we have an advocacy pillar in our strategic plan. So this is like working side by side, thinking about taking the temperature of our membership nationally. Where are they? Where can we come together? What do we need to move forward? Nurturing and reopening that communication with ALA was really something

Marlene Gebauer (11:17)
I’m sure it has.

Jenny Foster (she/her) (11:41)
something important to do this year. We are so happy those channels have opened. Thinking about other sister organizations in the advocacy space, aligning ourselves with them, and building coalitions. We are more visible when we come together, right? As RBG said, advocate for change, but do it in a way that invites people to join you. And, alongside our board action, we have been grappling with what advocacy looks like. How can we meet the needs of our strategic plan? Our amazing executive director brings advocacy experience from the work she did before joining us. Alongside our board’s advocacy vision and the nuts and bolts of the GRC’s work, all of that can coalesce and align to move this forward. So even though we’ve had a vacancy this year, we’re intentionally focusing on how to fill this gap while still meeting the advocacy goals that we hold so dear. So we are working on this like

I’m so happy with how this is going and you will see more. Remember, advocacy is a marathon, especially now. It’s a marathon, not a race. And so every little step that we can make will have a difference and I’m really pleased with the work that we’ve done this year. And then I get to hand it off to Jessica.

Greg Lambert (12:54)
Yeah, that was one of the things Jenny, you and I had in common, because we had searched for an executive director when I was president. So, fun

Jenny Foster (she/her) (13:05)
Yes, that is correct. Yes. Yes.

Greg Lambert (13:09)
Fun times. That is always interesting to watch, that transition. But I want to pull focus back to the conference coming up in July in Cleveland. And your theme is Leading with Aloha. What does that theme mean for you personally, and how does it show up across the more than 65 programs at this year’s meeting?

Jenny Foster (she/her) (13:43)
Thank you so much, Greg. I love this question. This could be like a whole podcast on itself, so I promise not to take up all the time. But going back to how law librarians love to plan. So you know, right now Jessica is already thinking about next summer in Philadelphia, and in the VP role, I was already thinking about Cleveland in 2026, right? So

And then I was trying to think about this gift that has been given to me to be the leader of our association. And then I always approach leadership with a service mentality, like what are the gifts that I can offer to the membership, to the organization. And then, remember, this was back in the fall of 2024.

There was a lot going on. My vice chair was in Chicago planning for Portland. And so I was walking my dog along the beautiful Koʻolau, thinking about all the changes that were about to come in 2025, remembering how chaotic it was in terms of access to legal information, thinking it was probably gonna come back. I’m just being descriptive. We know like government information exists one day and then it’s not on another day, and I just realized how much chaos was gonna be there. And then also thinking about

Being from Hawaiʻi, what was the gift that I could give? And I just, it was like, aloha, we will lead with aloha. In conversations about unpredictability, chaos, and fear, I have said that we are not hardwired for those things. But, actually, we are. And you know what those hard wires are?

Fight, flight, or fawn, and those might keep you alive, but they’re not a meaningful way to live and connect with others. So the antidote to unpredictability and chaos in my mind is meaningful connection. Because when we come together in meaningful connection and elevate one another, that is the antidote to fear and unpredictability because we can see with our own eyes that we can make positive differences in the colleagues that we work with, the communities that we serve. And aloha.

It’s all about nurturing that meaningful connection. So here’s my law librarian story time, okay? So aloha, which has been nurtured by the Indigenous population, Kānaka Maoli, or Native Hawaiians, for generations, right? It’s nothing new, but it was codified in the Hawaiʻi Revised Statutes in 1986. So this year makes the 40 year anniversary. And the reason that they did that is because they were infusing aloha into the highest levels of government decision making.

It says in the statute, right? It asks everybody to think about aloha in fulfilling their responsibilities and obligations as service to the people, the legislature, the governor, the chief justice, everybody.

And they even defined aloha. So I’m going to tell you the definition: ʻakahai, meaning kindness to be expressed with tenderness; lōkahi, meaning unity, to be expressed with harmony; ʻoluʻolu, meaning agreeable, to be expressed with pleasantness; haʻahaʻa, meaning humility, to be expressed with modesty; and ahonui, meaning patience, to be expressed with perseverance. All of those things, kindness, unity, agreeableness, humility, and patience, invite that connection.

And to face one another with that mindset, it takes two things. It takes courage because it’s really hard to be your authentic self. I think. I think it’s very brave to do that. And secondly, it really opens up the space to learn from each other, to be curious about all the different perspectives that we bring to the table. And

Leading with aloha, in terms of our programming, encompasses all of those things. We are already doing those things. We are bringing in programming where newer, mid-career, and experienced law librarians can learn from one another. We represent all the different library settings, all the different types of work that we do, and collaboration. I’m really excited about the work that we’re doing here and the intentional programming that our AMPC put together. I want to give a shout-out to our chair, Kelly Leong. The synergy was remarkable.

She is brilliant, wonderful, and intentional about how she led the committee, making sure different voices were incorporated in programming decisions. Kelly has family roots here in Hawaiʻi. Her dad is half Hawaiian, and she has grown up visiting. I get chicken skin thinking about the synergy of our coming together to develop this program for AALL and highlight our members through that connection. So we’ve got, you know, the Discussion Dens and the

hot topics that range from AI courses to access challenges and legal scholarship, balancing physical-collection needs with tightening budgets, GenAI, and how we integrate and guide its use in our institutions. We have a couple of interesting Marlene, you know, because you were there in Cleveland. We were just talking about it, about like all the different programming that is local to Cleveland. How do you say the river? I want to say it correctly. Cuyahoga. Cuyahoga River.

Greg Lambert (18:36)
Cuyahoga.

Jenny Silbiger (she/her) (18:38)
And how it was on fire, and the EPA was created in response to that. And then there was the grim serial-killer history from the 1930s, and the policies developed to try to solve that mystery. There is something there that elevates the community, the work that we do, and law librarianship in general. I mean, that is what was incorporated. Aloha was incorporated. But really, this is what we do every year when we come together in person at our annual meeting.

We come together for that connection piece. Okay, sorry, that was really long. You need to cut stuff down.

Greg Lambert (19:11)
A couple of comments there.

one, I’m I may need you to make sure that the the transcript spells everything correctly about the aloha.

Jenny Foster (she/her) (19:25)
For sure, for sure.

Greg Lambert (19:27)
And, when people think of librarians, they often think, “I would love to be a librarian because I love to read books.” That is about as far from what we do as it gets. AALL has three pillars, government, private, and academic librarians, plus our vendor allies. That diversity makes planning these events a challenge, but it also brings together different views under the same professional umbrella.

Jessica Whytock (20:20)
That diversity is one of the greatest strengths of our association, especially for me as an academic law librarian. I need to know what is happening in other libraries because it informs how we prepare students for their professional lives. And AALL has always provided this amazing opportunity for us to get together with our colleagues at different libraries and learn from them and see what their needs and interests are. Because of AALL and the people I have met there, I had a panel this year in my advanced legal research class. It came together quickly because everyone knew me from different programs we had done together and came in to teach my students what life will look like when they are out in the world. And so I love the fact that we are such a diverse group, representing all different types of libraries because it has added so much to my professional life. Yeah, I know, Jenny.

Jenny Foster (she/her) (21:39)
I feel the same. I love learning. And you know what, especially with the emergence of AI, I had Greg on a CLE here in Hawaiʻi a couple of years ago, was that two years ago, Greg? But I love hearing what the firms are doing with it and also and and I’m doing reconnaissance for the court so I’m learning about it. No, absolutely. Everything that you said, Jessica and Greg, thank you.

Marlene Gebauer (21:58)
I’m going to stay with the conference theme. Incorporating the locality into the conference is brilliant, because every location offers something different. It will be interesting to see how that works in Cleveland. But you know

We have heard a lot of buzz about the pre-conference activities, including hands-on workshops focused on copyright and fair use, and the return of the PLLIP Summit, the Private Law Librarians and Information Professionals Summit. So hooray for that. What are some

Jenny Foster (she/her) (22:41)
Ha ha ha.

Marlene Gebauer (22:45)
It seems like there is a lot going on, but people have to choose. What are some must-attend events before the official opening session?

Jenny Foster (she/her) (22:57)
You are so right, Marlene. I’m so excited. Yes, you’re right. The return of the PLLIP Summit has garnered a lot of attention and people are super excited. I guess. I don’t know. I’m sorry. I heard them calling PLLIP. I’m not in government.

Marlene Gebauer (23:05)
We call it PLLIP now? Okay. We always called it P-L-L-I-P, but I like PLLIP better, actually.

Jenny Foster (she/her) (23:14)
I think their theme is “Turn It Up! Amplify Your Practical Skills & Processes.” That is a turn I saw. I just got back from Canada, actually, and they were also talking a lot about AI, but really focusing on the practical applications. Like, tell me what works for you. How do you use it? I want to see what you’re talking about. And I think that is why they decided to do that. And then there are three others. There is “Libraries, Copyright & Fair Use,” which extends last year’s 101-level session into a more advanced discussion of how fair use shows up in day-to-day decisions and how to build confidence through real-world scenarios. And then I’m especially interested in “Re-Focusing in the Attention Economy” because that one is really about responding to the themes that we’re dealing with every day. Like, I had to turn off my phone, I had to close off 12 windows before we came onto this podcast, right? Because it’s a real thing, and what are the strategies to help your learners in such a digitally distracting environment. But my favorite is my love letter to CONELL, because that is our Conference of Newer Law Librarians. So if anyone is thinking about coming, if you’re a first-time attendee, if you switch careers or you’re a newer law librarian, I would love to see you there. And it is not a contest, but I hear that we have more CONELL registrants this year in Cleveland than we had last year. Again, not a contest. I’m just so excited that folks can come there and have that.

shared experience to kick off the conference. I remember being at my first CONELL. I love meeting newer law librarians who come to us. I still remember meeting people at last year’s CONELL. I’ll name-drop Devin Murphy, and there were others I met too. I love welcoming them in. So I’m super excited about that.

Greg Lambert (25:00)
It’s not a contest, but you won, right?

Jenny Foster (she/her) (25:03)
So those are some of them.

Marlene Gebauer (25:03)
Jess, did you have anything you wanted to add?

Jessica Whytock (25:05)
I’m really excited about CONELL and that so many people are participating. Everything sounds great. I know we are going to be busy, so we will not get to engage in as much programming as we would like. But getting the PLLIP Summit back feels great. But all of the pre-conference programming feels great. And I think it’s one of those situations where

Folks will find themselves having to make tough choices.

Marlene Gebauer (25:37)
Yeah. Well, from a personal perspective, if anyone’s on the fence about going to CONELL, I highly recommend that you do. For a few years, I led the session about what to expect at AALL for newer members, and it was a blast. You meet people, and as you said, they become lifelong friends. So go.

Jenny Foster (she/her) (25:50)
What?

Greg Lambert (26:03)
Yeah.

Yep.

Jessica Whytock (26:04)
I suspect we all remember our CONELL experience. I know I do.

Greg Lambert (26:10)
Yep, even when it was in the previous century, like mine. Mine was in 1999 in D.C., and I think I met Mark Gediman there. He and I are still hanging out, so.

Jenny Foster (she/her) (26:18)
Jessica Whytock (26:21)
No.

Jenny Foster (she/her) (26:23)
You’re a twentieth-century attendee.

Greg Lambert (26:26)
Yeah, my daughter. Yeah, whatever.

Greg Lambert (26:26)
So

So Jessica but before before we turn to you, I want this this actually applies to both you and Jenny. So Jenny, you know that when you hand over the gavel in it in Cleveland that no one will want to talk to you anymore. I was gonna say it’s it’s perfectly fine. So so Jessica just remember

Jenny Foster (she/her) (26:49)
Okay.

Marlene Gebauer (26:53)
But yet you will still have duties.

Greg Lambert (26:55)
This is this is gonna

Jenny Foster (she/her) (26:55)
Yeah, it it

Greg Lambert (26:56)
be

Jenny Foster (she/her) (26:56)
is true. I know.

Greg Lambert (26:56)
your huge year. Everyone’s gonna wanna talk to you, but don’t worry, at the end of it you’ll hand the gavel over to the next person and then no no one will talk to you again. So just words of wisdom.

Jenny Foster (she/her) (27:05)
I’m signed up.

I’m so there. I’m Jessica’s number one supporter. That is what my job is next year and I’m so looking forward to it.

Greg Lambert (27:12)
So, Jessica, as you prepare to step into the presidency in July, you bring a strong

background in advocacy, especially for the integrity of the profession. You’ve also co-chaired the Academic Law Libraries Special Interest Section white paper on continuing status and tenure, which is significant in academic settings. I want to step back. I know it is still early, but I know they picked up the phone and called you a couple of years ago and asked if you would run for this. Now that you are on the doorstep, what are some things you want to happen in your year?

Jessica Whytock (28:03)
Yeah, I have learned so much this year being on the board and one of my priorities of course is making sure that all of the good work Jenny has done continues that the relationship building continues, and that the systems that she put in place to just make sure that people have a a way to

to reach out and talk to us, and that those systems continue to exist even after my term is over. I do not think AALL always feels like now is the time when we see how important our profession is. And again, right now I’ve never seen such an important time to be a law librarian. Doing what we can to preserve access to authoritative legal information feels so important, even more important with AI and how now it changes how we access, how we interpret, how we use legal sources. So, making sure that we keep that at the forefront of what we’re doing during my term feels really key. But it is the internal growth and development of AALL that I really want to ensure is solid and strong. The association has made a big difference in my career, the people I have gotten to meet, the people I work with, the options and opportunities that are available to me. A lot of that is, of course, the work that we do every day at our own institutions matters tremendously, but it’s the connections that we make through different

positions at AALL, either just attending conferences or serving on a committee, or if you’re lucky to be in a leadership role doing that. So I want to focus on making sure that AALL makes it abundantly clear on how you can engage with the association. That might mean attending conferences or making sure you understand the grants and scholarships available to you. But

I really want to put systems in place that make it very transparent on how you can volunteer for a committee. How do you get chosen to chair a committee? I think the more that we make those pathways clear and available to people, the more we ensure that our association

represents our members and that people have a voice and that people know what they can do to take on these roles because they’re important and it is a lot of work, but it is a tremendous privilege and opportunity to serve on the board or to serve as a chair of a committee. And I want to make sure that the people who want to do that are able to do that and that people who did not even know it was something they could strive for see a really clear pathway, and steps people can take, to serve in AALL. So my goal is to strengthen our association by making sure that our members have really strong voices and pathways to join the association in any number of ways.

Jenny Foster (she/her) (31:16)
I love that with my whole heart. I cannot wait to support you. This year would not have been as successful without Jessica by my side and all the support from our board members. Like this is a collective group project.

Marlene Gebauer (31:22)
Yeah.

Jenny Foster (she/her) (31:29)
It is a collective group effort. I know we are in these roles for a reason. You wake up and do not know what role you are going to be in, but this is the one that was chosen for us. Jessica takes it seriously, and so do I. What a tremendous privilege and gift it is. I see Jessica striving hard to make meaningful impacts, and I am so appreciative of her help and I can’t wait, I’m so excited for her vision to come forward. Okay, sorry Marlene.

Marlene Gebauer (31:54)
No, no, that’s

okay. And I mean it sounds like you got some great goals to to strive for next year. well before we get to our crystal ball question, so as leaders of AALL and librarians who obviously read all the time, what what are one or two of the must read resources, you know, you know, committees or thinkers that you rely on to stay ahead of the curve in library administration and legal information?

Greg Lambert (32:08)
Yeah.

Jenny Foster (she/her) (32:22)
I

I love it. Are you asking us for a book list? Okay, I’m here for it, Marlene. For me, I zoom out a little bit. I love Dr. Brené Brown’s Dare to Lead and the BRAVING framework she has for leadership. I feel like that because leadership is an act of service. And I like zooming out: how can we be of service when we’re bringing ourselves into our workplace but then also for the association and in librarianship

Greg Lambert (32:27)
Yeah.

Jenny Foster (she/her) (32:50)
In general. And then also, and did you know, she started a brand-new podcast with Adam Grant. So it’s called The Curiosity Shop and it just launched, like I think they’re on their fourth or fifth episode. So I don’t know if you know, but like ten years ago they got into a public kerfuffle in The New York Times about vulnerability and when it’s appropriate to be vulnerable in the workplace and all of that. Anyway, but now they’ve come back full circle together and they are having such great conversations about leadership, about paradoxical thinking, making space for

diverse voices and what does that mean and then accountability but ethical integrity, it’s great. Those would be the two things that would be on my bookshelf for you to share. Now another time we can talk about the things we do for entertainment. What program are you watching?

Marlene Gebauer (33:32)
That will be after recording.

Jessica, do you have anything?

Jessica Whytock (33:39)
Well, you know, I’ll be honest, I have gone through a number of transitions the last few years and reading has not followed along. I have just not had the time. but I try to surround myself with people who model the behavior that I want to engage in and listen and learn. And I listen to people with whom I disagree and try to understand what their perspective is. So I’m not doing a lot of reading right now. I hope to do more reading in the future.

Marlene Gebauer (34:13)
I want our audience to understand that librarians do things other than reading. Most of the time.

Jessica Whytock (34:18)
Yeah.

Greg Lambert (34:18)
Yeah.

Jessica Whytock (34:21)
Yeah.

Because I do not have the time, I listen to and watch people I admire, see how they handle situations, and adopt what feels authentic to me.

Greg Lambert (34:38)
All right, now it is time for the crystal ball question. Jenny, we will start with you. looking into the future over the next few years, what do you think will be the biggest shift? We can focus this on AALL, but you know, what what do you see for the profession? Something that we need to prepare for now ’cause it’s gonna hit us later.

Jenny Foster (she/her) (34:48)
Okay.

Okay, I’m diving in here. I’m leaping off the cliff. This is a tough question. Thanks a lot. But

I think the biggest shift that we’re gonna see is this fundamental reimagining of what it means to be a legal information professional in our AI-integrated world. And not necessarily the technology itself, because I have faith in all of us. We can beta test it and learn it because we are teaching it, and the best way to learn something is to teach it. It is really about thinking about that identity question of who we are. What do we uniquely offer, and how do we communicate that to the organizations, communities, and institutions that we serve?

And we are already grappling with that question right now. I see our members stepping up in all the spaces. You, Greg, Emily, all the folks in the firm world, all the people in academia who are doing the innovation labs and all those things, and people in the courts too, we are a little behind, but we are still doing the best we can. Law librarians are stepping into leadership roles there, too. So I believe this with my whole heart. I know I talk to you about things from the heart, but I also have a brain, too. The answer is not to compete with the technology. It is to lean more deeply into the things that we do. The critical thinking that we do, the ethical discernment, the human judgment about context and nuance and access and equity that I don’t think algorithms can replicate. I mean, they cannot, right? Not yet. But the next generation of legal information professionals will need to be really fluent in all these tools, yes, but more importantly, then they’ll need to be anchored in our professional identity, our sense of purpose and service. And that’s where AALL comes in. Our work over the next several years, we have to cultivate that foundation through educational programming, through advocacy, our pipeline, and the ways Jessica discussed for engaging our members. Because this is the community that we will build together where

emerging professionals will feel valued, seen, supported, and empowered to step into those leadership roles because the future is not something that will happen to us. The future is something that we’re gonna build together. With aloha.

Greg Lambert (37:19)
All right. Kind of a hard one to follow.

Marlene Gebauer (37:25)
That’s fair.

Jessica Whytock (37:25)
You never benefit by talking after Jenny.

Jenny Foster (she/her) (37:29)
Jessica’s brilliant. She’s amazing.

Jessica Whytock (37:32)
I do not know. In a time of tremendous flux, it is hard to know what things will look like in five years. I do think one of the things that we need to do is make sure that we’re inviting the right people and voices in the room and that we don’t always just lean on our traditional colleagues and allies and that we think about who else needs to join conversations as you know

As we watch our profession and the legal industry change, it is going to be important to be flexible, adaptable, welcoming, and forward-thinking about who we need to talk to so that we are doing the work that we need to do and that we are teaching and training future lawyers to do the work that they need.

Greg Lambert (38:24)
Well, Jenny Foster and Jessica Whytock, thank you both for giving us a look at what is happening at AALL and at the upcoming conference in Cleveland. Thank you both for being here.

Jenny Foster (she/her) (38:37)
Mahalo, Greg. Will we see you there? Are you coming?

Greg Lambert (38:40)
I am coming. I will be there.

Jenny Foster (she/her) (38:42)
Yay, thank you so much. It was such a pleasure. Thank you, Marlene. I’m sad that we did not get to see Georgie, but hopefully next time.

Jessica Whytock (38:45)
Yeah.

Marlene Gebauer (38:45)
Thank you.

And thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (38:58)
And Jenny, one last time for the listeners who want to learn more, register for the upcoming annual meeting in Cleveland, where do they need to go?

Jenny Foster (she/her) (39:08)

Go to AALLNET.org, where you will find our conference site. Or go to any AALL social channel, LinkedIn, Facebook, or Instagram. It is always there. Come join us. We would love to see you there. Aloha.

Greg Lambert (39:22)
We’ll make sure we also put links

in the show notes as well. Okay.

Jenny Foster (she/her) (39:25)
Okay, perfect.

Marlene Gebauer (39:28)
And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

In this episode of The Geek in Review, we welcome Greg Dickason⁠, Chief Technology Officer at ⁠LexisNexis⁠, for a wide-ranging conversation on agentic legal AI, Lexis+ AI Protégé, and the movement from AI chat toward AI work. Dickason frames the shift through a simple contrast: earlier legal AI answered questions, while agentic workflows take on multi-step assignments, conduct research, create drafts, verify citations, and move legal professionals closer to finished work product. For law firms and legal departments trying to understand where AI goes next, this episode places agentic AI squarely inside legal workflow, legal research, drafting, and risk management.

A major theme of the conversation is trust. Dickason explains how Shepard’s Verify extends the familiar Shepard’s signal beyond traditional research screens and into uploaded work product. Rather than asking lawyers to rely on AI-generated text without a verification layer, LexisNexis is building citation checking into the workflow, giving lawyers a path to confirm whether cited authority exists, whether authority is still good law, and how later courts treated the cited case. For lawyers worried about hallucinated citations, AI-generated briefs, and unreliable authority, this verification layer becomes part of the product architecture, rather than an afterthought.

The discussion also explores the relationship between LexisNexis and Anthropic, along with the rise of legal AI skills. Dickason describes a market where model choice, orchestration, and legal skills increasingly matter as separate layers. Anthropic, OpenAI, Google, and other model providers offer impressive foundations, yet legal work needs more than general-purpose intelligence. Large law workflows require legal content, expert reasoning, matter-specific playbooks, and firm-defined processes. Dickason notes the ability to upload firm playbooks as skills, giving firms a path to bring their own way of working into Protégé.

Security receives equal billing with accuracy. As firms place client documents into AI vaults and connect work product to legal AI platforms, Dickason explains bring your own key, or BYOK, through a practical office-and-locked-cabinet analogy. The point is control: client content sits encrypted, access depends on the user’s key, and access stops when the key is withdrawn. He also discusses legal chunking, indexing, vector stores, retrieval-augmented generation, and knowledge graphs as part of building AI systems suited for legal documents, rather than generic file handling.

The episode closes with a broader view of legal AI’s impact on junior associates, legal training, and access to law. Dickason does not predict the end of junior lawyers. Instead, he sees AI helping junior lawyers become senior faster through mock trials, mock depositions, and richer training environments. He also warns of risks from agent volume, security vulnerabilities, and legal systems struggling to keep pace with AI-enabled industries. The message is pragmatic and optimistic: agentic legal AI will change legal work, yet the winners will be those who combine trusted content, secure systems, verification, workflow design, and human judgment.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Marlene Gebauer (00:00)
Hi, I’m Marlene Gebauer from The Geek in Review, and I have Sam Moore here from Legal Technology Hub, who’s going to tell us a little bit about analysis of token usage and model selection.

Sam Moore (00:11)
That’s right. Thank you, Marlene. Well, it is tokens, tokens everywhere. I think spurred on by the launch of Claude for Legal, but certainly going back further than that. There’s an issue in the legal industry today around token usage in GenAI tools. And in the Legal Technology Hub advisory team, we’ve had several conversations in the last week or two about this, both in terms of frontier models, but also in terms of the legal AI platforms.

And the topics we’re discussing with clients right now tend to fall into three interconnected topics. First is model selection, because a lot of these products give the users a choice of which model they want to use for a given prompt. But most users of these products really have no idea what the difference is. I’ve seen law firm clients whose users just pick the most sophisticated model for everything, toggle on every optional feature available, and then are confused as to why responses are taking a long time and why they’re hitting token limits very, very quickly.

The second is around model context windows. I’ve had several conversations lately about what a context window even is and how it creates drift when it gets crowded in a chat’s context window, and why that really matters for legal use cases, which often involve uploading quite large documents, which take up a lot of space in those context windows.

And finally, efficient token usage. Law firms and law departments, I think, are generally not accustomed to this kind of pay-as-you-go model in technology. Not unless you’re like me and you recall when the big legal research platforms were on a pay-per-search basis. So now those users are running into high-cost overages on the frontier models in particular, and they’re realizing that low sticker price per month is not their reality, not when their users don’t know how to use those tools efficiently and how to control cost.

So, as well as delivering advisory work on these topics on a one-to-one basis, we’re actually working on a series of articles for LTH Premium about these topics, which will then combine into a sort of playbook for our subscribers to keep handy when they’re working with GenAI tools. And we expect to start putting out that content in early June.

And if people want to know more about LTH advisory and what we do, they can always get in touch with us by going to legaltechnologyhub.com or by finding me on LinkedIn.

Marlene Gebauer (02:36)
Thank you, Sam, for keeping us informed about this important issue.

Sam Moore (02:41)
You’re welcome.

Marlene Gebauer (02:49)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:55)
And I’m Greg Lambert, and today we are diving into the rapidly evolving world of agentic legal AI.

Marlene Gebauer (03:19)
Greg, welcome to the show. Greg Two, welcome to the show. No, I think you’re Greg. No, actually you’re Greg One, and Greg Lambert will be Greg Two. How’s that?

Greg Dickason (03:21)
Thanks.

Greg Lambert (03:26)
Yes, yes. But you’ll be Greg with the British accent, and I’ll be Greg recovering from a cold. So, Greg, let’s start off. We talk a lot about agentic AI, agentic workflows, and so I want to define that so we know what we’re talking about here.

Greg Dickason (03:30)
There we go.

Marlene Gebauer (03:31)
The smarter one.

Greg Dickason (03:37)
Gives you a very mellow voice, so it’s good.

Greg Lambert (03:53)
So do you mind breaking down...

Greg Dickason (04:06)
Yeah, absolutely. I think I like to differentiate between sort of the first generation of AI and this generation with agents and agentic workflows, if you like. And what we mean by that is the first generation, you would go and ask it a question, it would give you an answer. You would then have to do something with that answer. You’d have to go and plug it into a Word document or maybe do a bit more research and then go back with another question. It was a bit like the sage sitting on the hill. You walk up the hill, you ask a question, you get your advice, and you wander down again.

Agents, or agentic workflows, you’ve now taken that sage and you’ve put them in your factory floor. And now when you ask them a question, it’s more like you’re telling them to do something. They go and do the research, but then they do something with it. They build something, they produce a document, they realize that they’ve got to do more research, so they have multiple steps. And so it’s a much, much more powerful paradigm. You’re not just asking and getting a response back, you’re actually getting work done for you. And that’s where I think it is the huge shift with agents.

Greg Lambert (04:59)
Yeah. And so how do you look at that when...

Greg Dickason (05:16)
So for us, it’s about understanding that our customers want to get work done. They don’t just want to come and understand something, do some research, and then go off. They want to get work done. They want to produce a document. And in some cases, they don’t always know all the questions they need to ask. So being able to ask a more open-ended question, and then we go off and our agents build on that, ask them questions back, and effectively create a workflow or a long task which produces far closer to the output they want to get.

So we recognize that as much as we’ve got great authoritative content, that’s really powerful when you’re marrying it to the workflow of your customer so they actually know what they want to produce. And a lawyer is not producing an output from an AI. A lawyer is producing an email or a draft or a brief or something like that. And we want to help them get as close as possible to that final output.

Marlene Gebauer (06:05)
So I imagine that agents are very important in tools like Shepard’s. So how do features like Shepard’s Verify operate under the hood to actively cross-check AI-generated text against the LexisNexis database and prevent hallucinated citations?

Greg Lambert (06:23)
Like hallucinations came up in the second question. I like that.

Greg Dickason (06:26)
Yeah. It’s always a theme with AI, isn’t it? Yeah, absolutely. And I think that’s why we think Shepard’s Verify is so important. But everybody knows Shepard’s. Shepard’s is going to tell you, is this good law? It gives you a really strong signal. And that’s on our platform. What we thought is it’s actually good to move that out of our platform and onto your platform and wherever you are.

So if you’ve written a document and you upload it, can we do some citation verification for you? Can we check, does that citation exist? And if it does exist, is it good law? And that’s what Shepard’s Verify is about. It’s about...

Greg Lambert (06:29)
Yeah.

Greg Dickason (06:56)
That trust signal and giving it to you so you can use it where you are. Obviously, in our responses, we always give you a Shepard’s signal so you can click through and check, as well as get the signal to see how good is this law, but also in the documents you upload. So is it even verifiable, and is it still good law? And that’s where it works.

So, how does it work under the covers? We’ve got Shepard’s, we’ve turned it into a really powerful service, and that service is now available inside Protégé, so we can use it against any document. And that goes back to Greg’s earlier question, Greg Two’s earlier question, which is, how is an agent different from an AI? In this case, the agent knows, at this point, I need to verify what I’ve just picked up, or I need to verify this document. So it knows that it can use the Shepard’s Verify tool to do a particular task, which is to give you confidence in the output.

Greg Lambert (07:41)
Do you mind giving us a scenario where, if I’m an attorney and I’m working, how does that process work? Is it smooth, or is it something that I’ve got to purposefully go and do?

Greg Dickason (07:49)
Yeah, so let’s say you’ve got a brief from opposing counsel and you want to check that. You can upload that onto Protégé and we will do the verification checks for you. So you’ll see the signals against your document and be able to see how well the opposing counsel’s citations actually link, whether it’s good law or whether it even exists, as an example.

Greg Lambert (08:16)
And is it verifying the citations only, or does it go a little bit deeper? Does it look at what’s quoted, or how deep does it go?

Greg Dickason (08:24)
Yeah, it looks at whether or not, how well that has been treated by subsequent cases. It doesn’t always go right into the argument, but it does look at how well it is being treated by subsequent cases, and therefore whether this is a good or bad case to use in your particular argument.

Greg Lambert (08:43)
All right. So one of the things, and everyone is now talking about Anthropic. They seem to be the foundational AI model that everyone’s using, and, of course, caused a big stir over the past few months with the SaaS apocalypse and now the legal AI tools, the skill sets that they’re bringing in.

So do you mind talking to us a little bit about what kind of relationship Lexis and Anthropic have? Because I know you guys have used them for a long time. They’ve been underlying a lot of your technology for a long time. So it’s not a new relationship at all. But with them announcing that they’re in legal by releasing these skill sets, how does that relationship work? How are you building on that right now?

Greg Dickason (09:38)
I see the Anthropic one, I’m super excited about working with them, right? The fastest-growing company in history. I mean, you’ve seen what they’ve done this year. It’s pretty amazing. And to your point, we’ve been working with them from before they were really even thinking about how they sold to enterprises. So we had signed an arrangement with them on Amazon Bedrock, which is the way Amazon supports models, before Amazon Bedrock was live. And that was their way to start to work with us. I think we were one of the largest contractors they had in those very early years.

So we’ve got a great relationship with them. It’s been going for a long time. Jeff Bleich, their chief legal officer, was at one of our conferences the other day, and so therefore I see it as largely really collaborative. What’s great about Anthropic is they’re very open. They tell us what they’re doing. They give us early access so we can test against their models. We can test and see their skills. And so that’s a great place to be.

But at the same time, they’re moving very, very fast. And I think what they’re seeing is, how do they enable the knowledge worker in general? So, how can they give the knowledge worker the skills that the knowledge worker needs to get their job done? And they see Claude Cowork as sort of that generic knowledge worker’s interface where you can do some pretty cool stuff.

But what’s great is that what they’re providing is a great model, a good harness, and a set of skills. And I think of those as almost the layers. If you think about old tech, you used to have the database and then the business layer and all the rest. Now you’ve got the model, the harness, which helps that model work in your environment, and then the skills, which tell the model how to think about a particular thing.

All of those are available to us. But at the same time, we also have those available from other parties like OpenAI and Google and others. So we can pick the best of breed for the model, the harness, and the skills, regardless of which provider. And we can do that for whatever use case, for whatever type of lawyer we’re serving at the particular time.

So I think we’re actually in this unique position where we have great content, which we can use to build skills, but we can choose best of breed at all three layers. And we’re working with exciting businesses like Anthropic, which just means that we can innovate very, very fast on what they’re doing. So I don’t see it as too competitive. I think your other question there, Greg, was, you know...

Greg Lambert (11:43)
Yeah, because you hear, like, you hear now, we’re an AI, what’s the phrase, Marlene, that these small firms are? Basically they’re an AI foundational law firm, or I’m not getting it.

Marlene Gebauer (11:51)
AI-powered, AI-forward. AI-native, sorry.

Greg Lambert (12:08)
AI-native. And so I guess, and I think this might be a bit of a softball question, but I’ll throw it out there anyway. What is the value of having that combined?

Greg Dickason (12:23)
First, because the foundational models are tuned for generic solutions. They’re not tuned for what you need. So you need something that layers on top, which understands the law.

Second is that the foundational model is increasingly requiring a harness to work well. So you’re starting to get stuck into that harness because the two are being coupled. Think of it a bit like riding a bicycle. I can be a great athlete, I’m the model, but if I’m on a bicycle that fits me really well, I’m going to be so much better when I’m on my bike. And that’s what’s happening. Increasingly, the harness and the model are working well together. But that’s making you lock in because then it’s only you getting where they’ve tuned that.

So what we can do is we can reverse engineer and work across all of that. So we give you the best harness and the best model for a particular use case. So I think that’s why.

And then the skills is just a really exciting space. Skills are just Word documents, not Word documents, just text documents, which tell the agent how to think. And they can call each other and they can get quite complicated, but they’re basically just a set of text documents. And so if you go to Anthropic, you get a lot of great skills that are focused on just in-house counsel, but they’re not focused on longer-running, harder tasks, particularly in large law. And so, yes, there’s some stuff you can do there, but it’s not a generically strong legal platform like we provide. And we can reuse those skills and our own skills. So I think we can give you the best of all worlds.

Greg Lambert (13:43)
Is there a future where, as a Lexis+...

Greg Dickason (13:55)
That future’s arrived already. You can upload your skills with our new work product. The future’s arrived. But exactly to that point, you have your own way of doing work. You’ve already written it down. You’ve got your playbook. You can turn that into a skill and use that.

Greg Lambert (13:59)
The future is here now.

Marlene Gebauer (14:01)
Hm.

Greg Lambert (14:09)
Okay.

Marlene Gebauer (14:11)
So I’m going to ask another value-related question, sort of what your thoughts are in terms of the value of this. In addition to the Anthropic alliance, you also have now an alliance with Luminance, and I imagine that is going to bring a lot of new document drafting skills, and that it will be combined with the legal research skills of Lexis. Outside of streamlining that process, where do you see the value in that combination in one interface?

Greg Dickason (14:45)
I do think it’s about getting your work product done without having to switch interfaces. So I do think it’s the fact that you can do the research, you can start the draft, then do further research, and it can all happen relatively seamlessly. There might be one click through to check something and then back again, but it’s relatively seamless with things like Shepard’s Verify popping up to tell you, yes, this is right, this is not right.

And I think that’s a lot of, if you listen to good product podcasts, it’s about reducing the friction. It’s reducing how hard it is to do what you want to do, and I think a lot of those kinds of integrations for us are about reducing the friction so that there’s a...

Marlene Gebauer (15:19)
It’s also about getting people comfortable with working in a workspace outside of what they currently do, changing that whole, helping with change management in terms of how they do their work, because people are kind of notorious about not wanting to change that.

Greg Dickason (15:38)
Yes, it’s very hard. And for me, as a product tech guy, that’s one of the hardest things, getting people to change, even the small things. Like when you go into Netflix versus Amazon Prime, they scroll slightly differently. And even that I find is like...

Marlene Gebauer (15:51)
It’s infuriating.

Greg Lambert (15:54)
Yeah. Well, let me ask about this, because I wrote a thing about the future of the UX, and if you’re not developing an interface, an experience that the user likes or works in the way that they work, they’re going to go out and create their own way of accessing it, whether it’s like with...

Marlene Gebauer (16:16)
Or find a workaround or something.

Greg Lambert (16:16)
What Salesforce is doing with a headless interface, or they might use the AI to access the website directly and then pull the information back in for them. So as someone who is on the product side, how do you think about what the future of the user experience is as we move, especially as we move into this agentic period?

Greg Dickason (16:41)
I think it’s increasingly going to be simpler and simpler because the agent’s going to understand your intent. Therefore, one, it’s going to know about you, so it’s going to have memory about you, who you are, what you care about, and then it’s also going to understand the intent of this current thing you want to do. And so you don’t need a complicated UX anymore. What you need is something that’s simple, that’s easy to engage with, but then it might diverge toward a particular use case.

So if you’re doing research, it might ask you some questions. If you’re doing a draft, it might open a document on the side. But ultimately, it’s doing that for you. So it’s very curated for you. I mean, they do talk about AI UI, which is where the UI is actually created by the AI in real time. I think that’s immature, and I don’t think it’s there because then the AI is almost overcomplicating it. I think what we’re going to get down to is a much simpler interface.

Greg Lambert (17:30)
Yeah. I’m curious, because a lot of the web is built for human interaction. And one example is, let’s say I get a web page and it gives me a spreadsheet. Well, it might only give me 50 lines of that spreadsheet, and then I have to click page two, right? Because that’s how a human ingests it. Whereas if it’s an AI interface, it would give them the entire spreadsheet, or it might give them dozens of spreadsheets all at once because it can handle that. So it’s going to be interesting from a product side how you do that.

Greg Dickason (18:04)
Definitely. And we’re looking increasingly, like for our digital side, we’re seeing more and more traffic coming from OpenAI, from ChatGPT, and Anthropic, the actual models, the open models, where users have clicked through. So rather than come through via Google, they’re coming via those channels. And then the question is how much of that is coming from the agent, with agents looking at our website and curating that back for the user. So it’s a really interesting change. I think you’re right. More and more of the web is going to be written for agents, not for users.

Greg Lambert (18:28)
Well, let us know when you figure it out and we’ll bring you back on. You can explain it to us.

Greg Dickason (18:31)
Ha ha ha.

Marlene Gebauer (18:32)
I do have a question about what you’re hearing in terms of feedback from clients. We’ve talked about comprehensive solutions where you can bring in your drafting, you can bring in your research, you can bring in your assistant and all those things, versus point solutions. And I know it will have to do with the actual work that needs to be performed, but there’s also an increasing pressure, I think, for clients regarding the cost of these tools. So I’m curious, what sort of feedback are you getting from clients? Are they leaning one way or the other? Anything that you can offer in terms of that insight?

Greg Dickason (19:16)
I do think increasingly we’re going to start to see consolidation. They want fewer tools. I think there has been a case where they’ve been looking at lots and using point solutions because there have been specific point solutions that have helped for specific use cases. And I do think that’s going to start to collapse, coalesce. So, for example, with our system, we can now load any type of skill, which means you can start to tune our system for your particular matter and how you do your matter, and the agents can pick that up. So I do think that’s going to happen. And I do think that’s what our clients are starting to ask questions about. I think that’s your question, okay?

Marlene Gebauer (19:49)
It is, it is. And I had one other one. In the news, we’ve been hearing about firms making a large investment and building their own AI. And I’m curious, sort of what your take is on that.

Greg Lambert (20:02)
They had an extra $500 million laying around.

Greg Dickason (20:05)
Really? Yes. Look, I think it’s logical for particular workflows. I think for some things it’s not that logical, but for some things it does make a lot of sense, for some workflows, particularly when that is your value proposition that you take into market, that your clients see from you.

And I think in that case, you’re going to need some really good foundational building blocks to help build that. Obviously, we see ourselves as being a key contributor in that kind of space, where you’ve got deep legal research, deep authoritative content. But I don’t see it as being just calling some dumb interface, because you need the reasoning, you need the legal logic that comes with an agent like Protégé.

So it’s not MCP where you’re just being called. It’s A2A, it’s agents talking to agents. And I think that’s probably the emerging space, where you have an expert talking to an expert. They might both be agents to help solve the client’s problem. So we do see a space for that, but I do think it’s agent-to-agent rather than agent-to-MCP.

Greg Lambert (21:01)
And so one of the things that we’re seeing is a lot more of the firm’s data is being uploaded into systems, whether it’s in vaults or whether it’s through the Word document in the plugins, or a number of different ways that the information is being accessed and somewhat commingled, I would say. So, can’t talk AI without also talking about security. And one of the topics that’s being talked about now is the BYOK, or bring your own key.

Greg Dickason (21:51)
I think it’s critical, especially for our larger customers. They have to have it. And the point with bringing your own key is...

Greg Lambert (21:56)
Well, let me stop you there. Do you mind just talking about what it means to bring your own key?

Greg Dickason (22:02)
Sure, sure. So I like to think of it almost like a house. I’ve got a house where you can come and get your work done. You bring your documents, and I’ve got other, well, maybe not a house. I’ve got an office where I’ve got great workers. You can come, you can bring your documents, and you can get stuff done.

Now, what you want to do is bring a lot of documents, so you don’t want to keep bringing them in and out. You want to put them in the vault, right? And you want me to be able to access that so that my experts can give you the right results. But what you don’t want is for me to be looking at your documents when you’re not around, right?

So what I do is I give you a cabinet in my office. You put your documents in the cabinet and you lock it, and you bring your own key and you take that key away. And then you know I can’t access it when you’re not around because I don’t have your key.

And it’s almost exactly the digital equivalent of that. You have a mathematical key which unlocks, and it first of all encrypts and then unencrypts the content I need to do the job for you. But if at any point you withdraw that key, I no longer can do work for you. And that’s provable. And so I think it’s a great model where you can be quite sure that the only time your content is ever accessed is to do work for you.

Greg Lambert (23:11)
So how are you and your customers implementing this with Lexis?

Greg Dickason (23:16)
So exactly as you’re saying, in Vaults, you can now bring your own key. So you lock it. You put your content into the Vault. We index it so it’s all available for the AI to look at and say, okay, this piece of content works with this law to help draft that document for you. But it’s locked. And the only time our AI can look at that is when you’ve actually logged in and you’ve provided your key as part of your login. If you haven’t logged in, we can’t use your content. So it’s built into the Vault and we can prove that, and that helps you from your security posture perspective as a firm.

Greg Lambert (23:47)
And I’m curious if...

Yeah, upload files there.

Greg Dickason (24:16)
So with Claude, typically now you’re having to do it on your own laptop, and you can’t build as strong a vault. So when you upload files with us, we’re not just uploading them, we’re indexing them and we’re chunking them so they’re part of a vector store. And we’re doing that in a legal way. Different models can chunk the content in different ways. We chunk it so that it’s legally relevant. You can’t do that directly with Claude. You have to build your own chunking and your ingestion layer, which properly processes the files, and then your storage layer, which stores them in a way in which they can be easily retrieved for the AI. You might have heard of RAG.

Greg Lambert (24:53)
Yeah. We’ve been talking RAG for...

Marlene Gebauer (25:01)
Last year, year before.

Greg Dickason (25:01)
Yeah. Well, yeah, I mean, that’s like history now, right?

Greg Lambert (25:04)
Yeah, that’s very 2022.

Greg Dickason (25:01)
But to have a really good RAG system, you need to be able to properly chunk and index. And then on top of that, you can build a knowledge graph and other ways in which it makes it easier for your agents to surface.

Marlene Gebauer (25:17)
So it’s good to hear that Lexis is thinking about security, like bring your own key and things like that. What do you find from clients that they are most concerned about? Is it this type of security? Is it the hallucinations that sometimes happen with cases that they see in the news? What type of conversations are you having, and how are you assuring clients that Lexis is focused very much on trustworthy output and absolute security?

Greg Dickason (25:54)
Yeah, completely right. I think it’s both. When we’re talking to the security teams, they’re interested in the mechanics of security. So things like bring your own key, making sure that what they’ve uploaded is properly locked away, that kind of thing.

When you’re talking more to lawyers, they’re more interested in the hallucinations and verifiability, and making sure that they understand how do they know that what we’re giving them is good law, and how easy is it to check it? Because our position is these are non-deterministic models, right? They’re probabilistic models, which means they will always come up with a small probability of saying something that’s not quite right. Now, we’ve got lots of rules and a ton of stuff around to limit that, and we believe we’re best in breed, but you still need to finally be able to verify, to check. And that’s why it’s very easy on our platform to be able to click through and see. You get your Shepard’s signals, and you can easily click through onto the platform.

So I think a lot of our clients are asking us, show us your security model, which we do, and then also show us how we can mitigate any risks of using an AI system to get more efficient, more effective, to provide services. And a lot of that comes down to reduction in hallucinations, reduction in the type of hallucination to almost zero. But then, at the same time, you can always verify. You can click through and verify.

Greg Lambert (27:10)
I’m curious if there’s risk, or things that your customers might not be thinking about now, but maybe they should be thinking about. Is there anything that, I know you’re dealing with some smart customers who are risk-averse, but I’m curious. For example, if you ever get your hands on Mythos, what kind of risks do we think are out there with something like that?

Greg Dickason (27:38)
Well, I do think there are two types of risks. There are risks we’re aware of, but I think where AI is going is pretty mind-blowing. The next six months to two years, I think, is going to be phenomenal. And if you think back to coding, agents were a bit of a toy. You got agents to write code for you, and it was a bit of a toy. Then somewhere around November, December last year, that toy became something real. And a lot of my colleagues in the tech world came back from holidays and said, wow, before the holidays, I wasn’t doing much. Over the holidays, I built five systems that I never even thought I could do. And this is what’s happened.

And I think we’re going to start seeing those types of step changes in other parts of the industry as well. And one of them is Mythos. So I do worry about Mythos because I think that’s going to surface so many security bugs and security vulnerabilities in the next couple of months that we’re going to have the spike of that happening and we’re going to need to make sure we can jump on them. I think we’ll get to a much better state in about six months to a year’s time, but there’s going to be a period of time where we’re all quite vulnerable. And I really like the way Anthropic is trying to roll it out to keep us on.

Where else can we think? I think it’s agents’ ability to overwhelm us. That’s something else I worry about. From a legal perspective, how many briefs can you get, and how much content can you ingest as a human? So I do think we’re going to increasingly need agents to help us mediate the effect of agents in terms of volume, in terms of sheer complexity of work we’re doing.

And then I do think that we’ll start to see new types of industries emerging, new industries that are far more agile and AI-enabled, and that’s going to stress the legal system just like other ways in the past have, even blockchain and new ways of thinking, digital assets and all that. But it’s going to happen faster. And so, how do we keep up? How does legislation keep up? That’s going to be a real societal challenge, Greg. Maybe we’re going a little bit away from Lexis, but you know what I mean.

Greg Lambert (29:25)
Yeah. Yep, exactly. Well, speaking of keeping up, before we get to our crystal ball question, we’ve been asking our guests to talk to us about how they keep up with the industry. Are there certain things that you read or people that you listen to that help you along? What’s a couple of things that you...

Greg Dickason (29:56)
Well, The Geek in Review is a start, of course.

Greg Lambert (29:58)
Of course.

Greg Dickason (29:59)
And then also I read Artificial Lawyer, Law360. So there are a few legal things which are great. I read the Turing Post. It’s quite technical, but it’s a nice email chain that you can get, Turing Post. And then The Information. It’s a technology-focused magazine, but it actually gives you some really good cutting-edge thoughts about where AI is going. That’s not super technical either. So that’s really where I go.

But I also think a lot of the time, I use Claude itself. I say to Claude, what should I know? What’s happened in the last week? And I sort of have an interactive session with Claude to learn. And that’s also quite useful.

Greg Lambert (30:32)
Yeah. It’s one of the things we say here: use the AI to help you AI. So...

Greg Dickason (30:36)
Yeah.

Marlene Gebauer (30:37)
Okay, Greg One, it is time for our crystal ball question. So looking ahead, a few months to a few years, as AI takes over the orchestration of massive document-heavy tasks through tools like Protégé Vault, what do you think is the single biggest shift coming for the traditional role of the junior associate?

Greg Dickason (31:00)
So the simple question is no more junior associates. But I think the answer to that is they’re not junior because they’re not there, they’re junior because they very quickly become senior. And I think we’ll see AI helping us train junior associates, then being able to do mock trials and all the rest very quickly, mock depositions, all of that. And so we’ll see them becoming senior very quickly and learning a lot as a result.

Marlene Gebauer (31:04)
Yeah.

Greg Dickason (31:23)
I think very recently on the podcast, you had somebody who was building their training systems, and that was pretty exciting to hear. And I do think that’s where we’re going to go. So I don’t think we’re going to see fewer lawyers. I think we’re going to see the law being applied in more places. Society’s underserved, and I think it’s going to give us the opportunity to serve more people, which is pretty exciting about where AI can take us.

Greg Lambert (31:44)
Right. I like your vision. So, well, Greg Dickason, CTO there at LexisNexis, I want to thank you very much for joining us.

Marlene Gebauer (31:58)
Thank you, Greg.

Greg Dickason (31:59)
Great to be here. Thanks, Greg. Thanks, Marlene.

Marlene Gebauer (32:01)
And thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (32:09)
So Greg, where’s the best place that listeners can find out more about you or about Lexis+ AI with Protégé?

Greg Dickason (32:16)
So jump onto lexisnexis.com/AI. That’s the best place to go. And then happy for you to look me up on LinkedIn, and I think we’ll post the link on this.

Greg Lambert (32:26)
Yes.

Marlene Gebauer (32:26)
And as always, the music here is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

Greg Lambert (32:31)
Bye.

In this episode of The Geek in Review, Greg Lambert and Marlene Gebauer welcome back ⁠Joel Hron⁠, Chief Technology Officer at ⁠Thomson Reuters⁠, for a timely conversation about the shifting relationship among foundation models, legal content providers, legal tech platforms, and the lawyers trying to make sense of the mess. Recent moves by Anthropic, including Claude’s legal practice area tools and MCP connections into legal platforms, raise a larger question for the market. Is a model provider still sitting behind the scenes, or is it starting to become a legal work environment of its own?

Hron explains Thomson Reuters’ commitment to what it calls ⁠fiduciary-grade AI⁠, a standard built around trust, verification, transparency, and accountability. For TR, legal AI needs more than a fast answer. It needs systems lawyers trust enough to stand behind. Hron points to Westlaw, Practical Law, KeyCite validity signals, citation ledgers, and verification tools as core ingredients in building AI systems suited for high-stakes professional work. In his view, almost right is not good enough when clients, courts, regulators, and professional obligations sit on the other side of the output.

The conversation turns to how CoCounsel and Westlaw Deep Research use legal content across far more than traditional research tasks. Hron explains that when AI systems gain access to trusted legal content and verification tools, they begin researching throughout the workflow, even while revising contract language or analyzing provisions. He also describes Litigation Document Analyzer, internally nicknamed the BS Detector, a tool designed to review claims in a document and map them to supporting authority, weak support, or no support at all. For lawyers who spend as much time verifying AI output as generating it, tools like these aim to move verification from a manual scavenger hunt into a structured process.

Greg and Marlene also press Hron on Anthropic’s legal plugins, MCP, and the idea of headless legal technology. Hron argues that MCP changes access, not advantage. In his view, the application layer is shifting, but the real competitive value sits in trusted content, expert systems, governance, and domain-specific intelligence. CoCounsel’s user interface represents one expression of TR’s legal agent capabilities, while MCP opens other ways for those capabilities to appear inside broader work environments. Some work will still need a purpose-built legal interface; other work might happen through email, Word, Claude, or another agentic workflow with little visible interface at all.

The episode closes with a larger discussion about what happens when AI starts performing more of the work itself. Hron shares TR’s internal engineering OKR, where more than 50 percent of pull requests should be written by AI, and explains why 51 percent serves as a useful mental model. Once AI performs a controlling share of the work, the human role shifts from doing the task to governing the system. For legal professionals, the same transition is coming. The key question is no longer only whether AI produces useful work. It is whether lawyers have built the systems, context, safeguards, and verification layers needed to trust the work, defend the work, and remain accountable for the work.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Cleaned transcript below, using the uploaded file as the source.

Greg Lambert (00:00)
Hey, everyone. I’m Greg Lambert from The Geek in Review, and I have our friend Stephanie Wilkins from Legaltech Hub. And Stephanie, all the rage is about the talk about Claude for Legal. So do you mind giving us your perspective from the Legaltech Hub?

Stephanie Wilkins (00:16)
Sure. We’ve been diving into this a lot. And I’m sure anyone listening to this has definitely heard that Anthropic made that huge announcement recently with the launch of Claude for Legal. And there’s really a lot involved in it. We’ve taken a lot of time, across our team, to look at it from different angles and try to do very in-depth coverage on this. To me, it feels a lot like the days back when ChatGPT first came out and people were trying to get their heads around what it even is, let alone what it means. So we’ve done a number of pieces.

The first one covers the full announcement, that there are 12 new practice area plugins, more than 20 MCP connectors with legal tech providers, expansion across Microsoft 365, access to justice partnerships, and a managed agents layer for legal users building in the Claude platform developer environment. You know, just a few things to unpack there. But it is really, arguably, the most significant move a frontier AI provider has made into legal to date.

But it does raise real questions across the market, among them being how the partner ecosystem evolves from here, what it means for the established legal AI platforms, and where the announcement is genuinely game-changing and where we might have a little bit of overhype going on here. So we’ve been on the news from the start. Before it went live, we had a chance to speak to Mark Pike, who’s Anthropic’s Associate General Counsel, and he’s also serving as its product lead for legal. So we’ve included his perspective.

And since then, we’ve looked at multiple angles. We have the plain announcement news itself. We have a visual timeline that traces Anthropic’s path into legal from 2023 through this month. I did a separate analysis that looks into how much legal research you can actually do from within Claude for Legal, because that was one of the big areas it touched on. And as a sneak peek, we get to very different conclusions, whether you’re a BigLaw practitioner, or you do law in a small firm, or you’re a solo practitioner in the access to justice system. And then a fourth piece by Nikki Shaver really dives into the operational reasons why Claude for Legal is simply not yet a lift-and-shift replacement for enterprise legal AI platforms.

This is definitely an inflection point. It is not the death of legal tech as we know it, as some people might want to believe. It has not upended the industry overnight, but there is a lot to follow here, and we’re going to keep looking at it from different angles as they arise.

You can read all of the articles I just mentioned on LegalTechnologyHub.com. And if you want to get these updates in your inbox in real time, you can sign up for our free newsletters and follow the Claude for Legal announcement and the journey we’re on as we really try to be critical and dive into what it really does and doesn’t mean.

Greg Lambert (02:56)
Yeah, well, there’s so much hype, so it’s good to have a little bit of fact-checking going on. So thank you.

Stephanie Wilkins (03:01)
Yep, thank you.

Marlene Gebauer (03:10)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (03:16)
And I’m Greg Lambert. And Marlene, for the past year or so, the legal AI conversation has been dominated a lot by the foundational model race: which model is smarter, which one does the reasoning better, which one has the bigger context window, and which one is going to power the next wave of legal tech tools.

Marlene Gebauer (03:26)
Mm-hmm.

Yeah, absolutely right. But recent announcements from Anthropic and Thomson Reuters raise a different question. So if Claude is now launching legal practice area tools and connecting into major legal platforms through MCP, is Anthropic still just a model provider behind the scenes, or is it becoming a legal tech platform in its own right? And if Claude becomes one of the places lawyers go to work, what does that mean for the value of trusted legal content, citation systems, workflow platforms, and all of the legal AI tools built around those models?

Greg Lambert (04:14)
And that’s exactly why we brought in today’s guest. So we’re welcoming back Joel Hron, Chief Technology Officer at Thomson Reuters. Joel’s been on the show before and talked about professional-grade AI and where Thomson Reuters sees the technology heading. And this time, we want to dig into what the Claude and CoCounsel Legal announcement says about the changing relationships among the model providers, the content companies, the legal platforms, and the firms and legal departments trying to make sense of it all, because it’s kind of crazy. So Joel, welcome back to the show.

Joel Hron (04:49)
Thank you for having me. Good to be back.

Marlene Gebauer (04:52)
Yeah, welcome back, Joel. So for a year, we were just saying everyone was sort of chasing these model capabilities. Thomson Reuters is drawing a hard line around fiduciary-grade AI. You have argued that in high-stakes law, the work is easy, but defending it is what matters. From an engineering perspective, how are you building trust as a system primitive?

Can you walk us through the architecture of the patent-pending citation ledgers and how you ensure the agent isn’t reasoning from the open web?

Joel Hron (05:30)
Yeah, you bet. I mean, so we have leaned into this idea around fiduciary-grade AI. I think the core premise of this idea is that almost right is not good enough in the domains that we deal with. And I would say our focus has really been around how do we exploit the value of the 1.9, almost 2 billion documents across Westlaw and Practical Law that we have, the 1.5 billion KeyCite validity signals. These are all signals that human lawyers use every day to validate and verify and build trust in the work product that they’re putting out to their clients or to the courts or otherwise. And so our focus has been, okay, how do we use those same systems that human lawyers use today to help AI build the same level of verifiability and trust?

And I spent the last two weeks, I got back yesterday, with our customers across corporates, across the Am Law 100 and global large law, as well as some of the largest tax firms in the world. And this theme of trust came up almost repetitively across all three groups of those professionals. And I think this idea of, okay, AI is great, it’s doing a lot of work, but how can I, as a human, maintain accountability for what I’m putting out the door? And do I maintain accountability for it? And the answer to that question is affirmatively, yes. I think the professional maintains that accountability. And so it’s incumbent, I think, on us as software providers to build the tools in terms of verifiability and transparency and auditability to give them what they need in order to stand behind the output. And so that’s the core of what we mean when we say fiduciary-grade AI.

And so in terms of how we are building that, I would say first and foremost is to really leverage the best technology in the market today. And today that’s models like Claude, but also the latest versions of GPT, Gemini, etc. And also this idea of the coding harness and what’s called agent harness now in terms of how this is being evolved for AI agents to proliferate. So that’s sort of the core. And I think everybody is trying to evolve their products to live and operate around that paradigm.

But I think what’s unique and important to us is what tools do we make available to that agent to be able to do its work? And again, those tools lean on access to the content and information we have, but specifically also verification tools, citation ledger tools that we are able to build that allow the agent to do that work of verification for itself and ultimately deliver a better work product at the end of the day. And I think you’ve seen this in Westlaw Deep Research, how this operates. That system works very well. And we’ve adapted that same sort of approach with CoCounsel across more types of work. And that next version of CoCounsel is in beta right now.

I was telling Greg earlier, one of the things that we see, though, is that CoCounsel as a product doesn’t just do deep research when it’s preparing for some important litigation matter. It does legal research on almost every task. If it’s modifying a contract clause or updating terms in a provision or something like this, it is always doing research. It is always going to that content to verify what’s market right now. What has happened in the case history that would support what I need to do to this contract?

And that’s, I think, a much more powerful use of content than just preparing for a litigation matter where people are always doing research. What you see is that these AI systems, when you give them these tools, are actually using this content in a really deep way across many different types of legal work that you might not have considered doing research for before.

Greg Lambert (09:53)
Yeah, I know a lot of times the argument that I’m hearing from a lot of lawyers right now is that the AI is getting them an answer really quickly, but they’re spending almost as much time verifying that the information they’ve gotten back is accurate. With CoCounsel and Deep Research, and the combination of that along with the agent harness that you’re writing in, does that speed up that verification process, or are we getting into the positive now?

Joel Hron (10:09)
Mm-hmm.

Joel Hron (10:30)
Yeah, I mean, in one way it does, but in other ways we’ve built specific products or modules or features, whatever you want to call them, for speeding up verification. One example of that is a product we’ve called Litigation Document Analyzer, but internally we called it the BS Detector. And it was literally an application built around an agent harness and these content tools that was focused on looking at a document. It could be a litigation document. It could really be any kind of document, a brief, anything like this.

And what this system will do is it’ll go through every claim made in this document. And a claim could be a sentence. It could be a sequence of sentences, but at a granular level, what is every assertion that is made by this document? And is it supported by something factual, i.e., case law or statute or regulation or something like this? The output of this is effectively a table of, here are all the claims, and here is the support or lack of support for this claim. And even, do we think this is a hyperbolic extension of what this case actually says or something like this?

That absolutely speeds up verification. Again, the idea isn’t that every brief or every report is going to be 100% accurate. I think, in fact, us building those products is recognition that it may never be 100% accurate, and lawyers need tools to be able to build trust in the work ultimately so that they can stand behind it and be accountable for it.

And I think that’s really what we’re committed to as we build these products: delivering the highest bar of accuracy that we can, but also delivering the tools that professionals need at the end of the day to be able to trust them.

Greg Lambert (12:25)
I want to get into the announcement of the TR and Anthropic collaboration, which is not a new thing. You guys have been collaborating for a while, but I know with all of the news surrounding Anthropic recently launching into legal directly, can you explain the bidirectional relationship that TR and Anthropic have now and what it means for the people who are using CoCounsel or Deep Research? How is it shifting what they’re seeing?

Joel Hron (12:59)
Yeah, you bet. On the surface, this feels like a big change, but two things. One, as you said, we’ve been working with Anthropic for quite a while, as well as working with OpenAI, Microsoft, AWS, Google, etc. But we have been working particularly closely with Anthropic for quite a while. But the second thing that hasn’t changed is, for us as TR, but also as CoCounsel, we’ve wanted our products to exist where customers are working. And that could be the Microsoft 365 stack. It could be Gemini Enterprise or Google Workspace. It could be Anthropic or Claude Enterprise. It could be OpenAI Enterprise. But I think the idea is that we want our products to exist where people are.

And at the end of the day, these platforms, whether they’re AI platforms or general workplace platforms, are meant to do a lot of different things across the business of law or the business of a corporation. And our goal is really to focus on how do we deliver, again, this fiduciary-grade level to those expert tasks that need to happen, particularly within law, but also outside of law in other industries that we practice in.

So in some cases, CoCounsel, the application interface, is the best way to experience and verify and validate that work that’s happening. But in other cases, where there are general work processes happening, our fiduciary-grade tools support those and act as support agents to that work. And I think our focus is to make sure that intelligence and capability exists wherever it is being used.

And I think that’s how we are thinking about CoCounsel, but that’s also how we’re thinking about making CoCounsel available in other systems. And we see a lot of value in that. The interface layer of software, as you guys have said, has been democratized quite a lot by AI tools, and in particular coding tools and things like that. And we see a lot of firms building their own things. We see a lot of firms and companies consuming general-purpose tools as well and building on top of those. And I think what’s critical is that we deliver that fiduciary-grade intelligence into whatever those systems are, whether they’re our own interfaces or things that people are building on their own.

Marlene Gebauer (15:32)
So Joel, I’m wondering if MCP essentially changes what it means to be a legal tech platform. Claude now has 20 MCP connectors into eDiscovery and CLM tools, for example. And so you never have to leave that interface. We’re seeing this kind of squeeze on this application layer.

You’ve mentioned that as agentic systems get more headless, I guess, optimizing for the single front door is not the right way to go. So does the traditional vertical legal tech application survive this orchestration layer, or is everything becoming more commoditized plumbing?

Joel Hron (16:15)
Yeah, I mean, I would say that MCP changes access, not advantage, if that makes sense. MCP, just like APIs have done, but I think MCP is sort of the analog of API integrations in an agent future, changes how people maybe access this technology, but it doesn’t change the purpose of the technology itself.

And I think, certainly for us, that’s about building solutions that people can trust and building information and knowledge and intelligence that people can trust. So for us, I don’t think MCP changes our job to be done, if you will, as a company, which is about building trust. And MCP is just a mechanism for us to deliver that into more types of work where it’s needed.

And like I said, I think in some cases there is a user experience that goes along with that. I gave the example of Litigation Document Analyzer. Maybe that’s a good example where there’s a distinct experience for how you should do that validation at an important moment. But then there are other cases where the experience may not even be an experience. It may be a workflow that gets triggered automatically off of an email, and a series of steps and work happens, and it comes back as another email.

And so I think we’re moving to a world where in some cases there may not be an experience at all. And that’s what I mean by headless. And I think what we want is that our fiduciary-grade intelligence is playing a part in that process no matter where and how it happens.

Greg Lambert (18:01)
Yeah, let me pull on that a little bit, because we’ve always heard legal vendors talk about work. They want their product to be where the attorneys are working, which is code for Microsoft Word or Outlook, typically. But I think we’re seeing even that shift a little bit, that some attorneys are working directly in the AI tools, or may have their own setup that they vibe-coded that allows them to start working on some things.

So my question is, because of the fact that with Westlaw or CoCounsel, there’s this designed user experience that you’ve set up, that you spent probably millions upon millions of dollars getting just right, so that you have this great experience. And then all of a sudden, your users, or at least some of your users, may be shifting away from some of these really good interfaces that you’ve designed for them. Is that kind of difficult for your UX designers to wrap their heads around?

Marlene Gebauer (19:08)
I was actually going to say, are we going back to more content and capability than the delivery system? Sort of how it was before everything got highly technical.

Joel Hron (19:21)
Well, and maybe to riff on that idea a little bit, Marlene, I don’t know if we’re going back to that, because I don’t know that we ever left that point of view. That has always been the centerpiece of everything we’ve built around, having accurate and up-to-date content.

Marlene Gebauer (19:40)
Well, the position as a content provider versus a technology company, that has sort of gone back and forth sometimes.

Joel Hron (19:45)
Yeah.

Yeah.

But I mean, where we spend millions of dollars is on making sure that our information is accurate and up to date. And in terms of being a content provider versus a technology company, our tools like CoCounsel, for instance, or Deep Research, are not just providing a ranked list of raw content back to an agent. There’s a tremendous amount of technology in terms of how we interpret and apply judgment and apply verification and citation and things like this to that information. And I think that is very much what makes us a technology company, more so than the interface that sits on top of that.

Now, the interface that sits on top of that certainly is changing. And I think the options that people have there are proliferating. I think for our design teams and our people building user interfaces, where the dominant work is legal work and the dominant work centers around the need to verify and build trust through a process, I think CoCounsel will continue to build great experiences for that type of work.

And so I think that’s really, if I’m a design researcher, this is what I’m thinking about: how do I build a user experience that elicits that understanding of how and why this claim is made, rather than just surfacing the claim in pretty font and colors? And so that’s the goal of our design teams. In some cases that’s necessary, and they’ll be in CoCounsel to do that kind of work.

In some cases, maybe that level of depth is not necessary. And that might happen out of an email client, or it might happen out of Microsoft Word, or it might happen in a general-purpose AI tool. And again, I think we’re open to either of those paths because we understand that work can span across those two in different situations.

Greg Lambert (21:54)
Do you think, or I guess, are your developers and designers essentially creating two variations of the content, one that gets surfaced through the UI and then one that gets surfaced through an agent-oriented way?

Joel Hron (22:10)
Yeah.

I think this is a really good question, Greg. And I would look at ourselves the same way I think Anthropic looks at themselves. They are a model provider first, and their job is to build models and tools around the models and make them available to builders. And then they’re building Claude Enterprise, the application. And Claude Enterprise, the application, is their best expression of the model. So this is a user interface that expresses the capabilities of the model in a way that allows the user to get the most out of what that model is capable of doing.

And I see our job very much the same. We take models from providers, but we build harnesses and tools around them and under them to be able to have legal capabilities that the base models themselves don’t have. And then our UI, and that’s sort of what is available via MCP, CoCounsel Legal is the agent, and it can do a variety of different things from a legal capability standpoint. Users can access that via MCP and plug it into different places, but CoCounsel, the UI, is our expression of that agent and how we believe a lawyer can get the most out of that agent for certain types of tasks.

And so that’s really how I see it. I don’t think they’re conflicting in any way. I think they’re both useful. One team is really optimized on how do I hill-climb the capabilities of this legal agent by giving it access to expert-level tools and systems. And the other team is focused on, how do I build a UI that expresses the capabilities of this agent in a way that is most useful for a human to interact with it?

Greg Lambert (24:05)
It seems like we have the two teams that are doing that. How well do they learn from one another? Because it would seem like there are certain ways that you’re surfacing information to a human that may also be relevant to the agent, and vice versa.

Joel Hron (24:22)
100%. They work very, very closely with each other. I would say most of the development we do today really starts with the agent. Most of how we think about solving legal problems starts with what is the agent capable of doing? And I think as we build UIs that express those capabilities, those UIs convey obvious gaps.

And some of those gaps can be filled or mitigated by the UI and how we construct the UI and how we construct the workflows within the UI and how we construct things like customization via skills and stuff like this. And some of those things need to be fed back into the agent team to say, okay, well, we need better tools to handle these sorts of edge cases, or we need better behavior for XYZ sorts of use cases. And so there’s a two-way conversation that happens between those teams.

I think the other thing that’s important as you think about agents is the context engineering for the agent is incredibly important, right? The agent is operating off of what it is discovering throughout the process of doing its work. And the human has a lot of context that the agent does not have. And in many ways, the UI itself is a way to help the human user convey their context to the agent in a way.

Just like if you were to hire a new intern at your company, you would probably set up some shared folder with them. And you would say, okay, here’s some recent documents we’ve put together, and here’s an onboarding document. That’s you conveying context to this intern to help them understand, well, this is what we’re doing, and this is why we’re doing it, and this is how we’ve done it in the past. And that’s the same thing that you want to elicit between a human user and an agent. And that’s what I think you guys are helpful at exposing as well.

Greg Lambert (26:33)
One final question on this topic. Just curious, if you were to put a percentage on it, on the coding that your developers do, how much of that are they relying on the AI now to do?

Joel Hron (26:47)
Yeah, this is a great question. Honestly, I have had other podcasts about this topic solo, and we could spend hours on it. We have an OKR in our organization that more than 50% of the pull requests that go into our codebase get written by AI. And I would say some teams are north of 80% at this point.

There’s a really interesting reason, though, we set this OKR. And sorry if I’m taking a tangent. You can pull me back into legal at some point if you want.

Greg Lambert (27:21)
We love OKRs on here.

Joel Hron (27:44)
Okay, so there’s a really interesting reason we chose 51%. And one of the engineers that is on my leadership team mentioned this to me back in December, and it really stuck with me. But he said something really changes about your mindset when you get to 51% of the code being written by AI, because now you, as the human user, are no longer in control of the code that gets written. You have ceded controlling interest of your codebase to something that is not you.

And it really is a good signal for, okay, well, how does your role as an engineer now change? Your role now as an engineer is less about writing lines of code and it’s more about building systems around how code gets written. And those things are governance systems and tests and guidance documents and architecture principles and things like this that help constrain and steer and guide the agent to do the right thing along the way.

And I think it’s a really good analog for how a lawyer should think about their role changing or how a tax professional may think about their role changing. As AI sort of picks up more and more of this grunt work, if you will, your job is more about how do you build systems around AI to do the work you want it to do in the way you want it done, rather than doing the work itself, right? And I think that’s really the mindset shift for an engineer that is taking shape right now. And I think it will likely take shape in other industries over the years to come.

Greg Lambert (29:06)
Yeah, that’s a good parallel. Thanks.

Marlene Gebauer (29:09)
Anthropic just shipped 12 practice area plugins covering everything from corporate law to litigation, and some deploying as managed agents. There’s a fine line between being the partner and being the competitor. So when a lawyer is using Anthropic’s native open-source corporate legal plugin versus routing that workflow through CoCounsel Legal, what’s the functional difference in output trust and defensibility? And I know you’ve talked a bit about this, so maybe you can do a compare and contrast.

Joel Hron (29:48)
Yeah, for sure. What I would say is that if you go look at these plugins, they are nothing but a rudimentary set of instructions for how to do a certain type of work. So I would say the plugins themselves have no concept of validation or verification or groundedness in factuality in any way.

They are helpful guides to an agent to help it meander through a task, but they do not have any concept of these principles, I would say, of fiduciary-grade AI. Now that’s not to say that one couldn’t take a plugin and say, hey, use these tools to validate your work along the way. And those tools could be CoCounsel MCP, living in a plugin.

I think you could do something really well in that context. And I think that’s how we think about MCP in the context of Claude, if you will. You can bring CoCounsel’s capabilities into a lot of these workflows in a native way and get the best of both. But the plugins as a standalone, again, are nothing more than a couple of instruction documents for the agent in terms of how to follow a path. And again, I don’t think they get to the level of depth and trust and transparency that we hear our users are looking for.

Greg Lambert (31:16)
I’ve heard people joking that these are the skills that their in-house legal department uses at Anthropic, but with all the really good stuff pulled out of it. It’s very basic.

Joel Hron (31:29)
Yeah. And look, I also don’t think Anthropic’s goal is to build a legal product per se that covers the spectrum. I think that’s why they are basic. I think they’re meant to be indicative and instructional around, here’s how you build guardrails for an agent, or here’s how you build workflows for an agent. You can take this and then make it much, much better. But here’s the seed of an idea, and you can then go use the platform to grow and expand and think about it in different ways.

And so I think that’s more of the message to take from the plugins than, here’s a legal product that stands on its own two legs. And again, I think our goal is to build tools that work in the context of that system and can be used to add validity and trustworthiness to whatever processes are happening there.

Greg Lambert (32:27)
Let me tag on to that, because one of the interesting things that they did put out was this thing that they call the cold start interview, that you can take a firm’s specific playbook and put it in, and Claude will write that to their Claude markdown files. And suddenly, when I say this out loud, I just envision my security ops person coming in and ripping my computer out of the wall and shutting everything down if I were to do this.

Marlene Gebauer (32:55)
It’s like, no, no, no.

Greg Lambert (32:57)
But the firm’s institutional memory can get encoded directly into these LLM instructions rather than vendor software. Putting back on your CTO hat, not that you’ve taken it off, how do you advise firms on things like this and what it is they should be doing with their proprietary information?

I’m sure you have a preference that they put it into what you’re calling the fiduciary-grade system rather than the LLM itself.

Joel Hron (33:35)
Well, certainly. I would say security is one of the things, I think probably security and trust are the two things that stand between...

Greg Lambert (33:47)
Yeah, that’s why we don’t talk about Grok as an enterprise tool, I think.

Joel Hron (33:50)
Right. So it stands between the real-world application of AI and the capability, perhaps, of AI today. And so I think certainly part of what the definition of fiduciary-grade is does speak to security and how we use that information. And we’ve been very clear that we don’t use that information in terms of training our models and products and things like that. And I think customers really appreciate the stance that we’ve taken on that. And I think they honestly trust us quite a lot because of the decades of work we’ve done with them on that front. And I would say we’ve earned that trust in many ways, and we try to re-earn it every day to keep it.

And so I think as a firm, this is your competitive advantage. In the future, if you can codify your knowledge, and we’ve talked about knowledge management as a domain of law for a while, but if you can codify your knowledge in a way, and you can do it better than the next person and make that knowledge available for AI in agent-native ways, then I think you have a tremendous competitive advantage.

And I would be very reluctant to, A, take that task lightly. And I would be very reluctant, B, to open with that approach. If I’m a law firm or a big corporation, this is something I want to be an expert at, because it is your lifeblood as a company at the end of the day. And I think if I’m making investments as a company anywhere, it’s going to be in this area. And then how do I serve that knowledge and intelligence then? I can choose a million different tools to serve it into. Owning that knowledge is really what differentiates you at the end of the day as a company. And certainly if I was leading a law firm or a big corporation in that sense, that’s what I would be focused on a lot.

Marlene Gebauer (36:00)
It’s funny, we just had this conversation yesterday with Ryan McClead about making sure that knowledge and content is AI-accessible in addition to being people-accessible. So, yeah, I agree. It’s going to be something important.

Joel Hron (36:16)
Well, and I’ll use this analogy, I don’t know if it will stick, but you could think of TR very much as a law firm. So a law firm has a lot of experience and matters that they’ve worked on over the course of time, that they want to index and organize and make sense of to inform what future work they do. That is the same job that we do with case law and statutes and regulations. We just happen to do it with most jurisdictions across the world.

And that is absolutely what we still believe differentiates us as a company and differentiates our products at the end of the day. And the better that we can organize that information and make it available, as you say, for AI, I think the better our products become. And in many ways, we have changed the foundational aspects of how Westlaw works for agents versus how it used to work for humans, and the APIs that the agent calls are different than the APIs that power the application today because agents work with the content in a different way, at a different pace, and at a different rate than humans do. And they need to look and feel and act differently for an agent user versus a human user.

Marlene Gebauer (37:33)
So Joel, you’re a bit of a unicorn. You sit at this intersection of legal content, AI infrastructure, product development, and figuring out what lawyers actually need from these tools. So what are a couple of resources, signals, or conversations that you rely on to separate the real movement in legal AI from a lot of the noise?

Joel Hron (37:56)
Yeah, it’s a really good question. I mean, I would say I read a lot. So I still enjoy following LinkedIn or Twitter. I do wish I practiced fishing more than I read about it, but I try to do both.

Greg Lambert (38:07)
And say you read a lot about fishing too, right? I see the books.

Joel Hron (38:21)
But I would say I do read a lot, and I think there’s a lot of really good content, particularly academic papers, that come out and people reference them on Twitter and things like that. It’s a good source of figuring out what to go read. But I think that’s a good way to stay up to speed on what’s happening in the market.

I think the most important thing, though, is that I use it. I think the most amazing thing for AI for me as an engineer has been, and I used to run a startup before we were acquired by TR, so at a startup with less than a hundred people or so, it’s quite easy to stay deep in the code and involved in how things work. Then you come into TR, and we’ve got 140 products and thousands of engineers. It’s impossible to maintain the level of depth in code. But with AI tools now, when I’m talking to a team and they have an issue, I can immediately go understand the code and what’s happening and what has happened in the last few weeks.

What commits have been made? What issues have come up? My level of depth in the code itself is far more than it could have ever been otherwise because of the ramp-up time and context switching. And that has been such a blessing for me as a technical person, to be able to do that. But I also learned so much by doing that about what is possible and what is capable. It gives me an intuition as well for where things are going and how I think I should be directing teams and this kind of stuff.

And so that would be the best piece of advice I would have for people: whatever it is, go use it, and use it a lot. The more you do, the better your intuition becomes for where this is going and what impacts you think it might have on your teams, your talent, on the products you’re building, etc.

Greg Lambert (40:16)
Yeah, I couldn’t agree more. Well, Joel, it’s time for our crystal ball question. I think we’ve thrown this at you before, but there’s so many things going on with the AI harnesses, the agents, the MCPs, the collaboration between foundational models and products. So what do you think is something on the horizon that legal professionals need to be looking out for and preparing for?

Joel Hron (40:49)
Yeah, it’s a good question. I think, obviously, agents is the trend. I think it’s easy to say that, but if you think about what does that mean, that means now that I am delegating actual work and decision-making to an AI, delegating actual work product to a non-human. And this is exactly like what I said with engineering as well. Your role now changes from doing the work to governing the work.

And so I think the change that you need to be anticipating is, okay, in a world where I’m no longer doing the work, how do I build the systems that give me the trust to stand behind the work that gets created? And for engineers, those are things like architecture design principles, high test automation coverage, etc. For lawyers, it’s other things. And I think for those people doing the work, it would really behoove them to spend a lot of time thinking about that.

And again, that’s what we believe we’re building in our tools: systems that can elicit and elucidate that transparency and trust that’s necessary to build systems around that kind of environment. But I think really thinking about what it means to have an agent do work, and what are the implications if you think three or four steps down the road, that’s what people need to be preparing for today.

Greg Lambert (42:23)
Tell everyone to keep track when the AI takes over 51% of your work. Your role changes, right?

Joel Hron (42:27)
And whether 51% happens or not, it’s kind of irrelevant. It’s more of a helpful mental exercise to say, what if I am no longer the controller of this system? What would I do? And that’s probably a good mental model for some actions that you might want to take.

Marlene Gebauer (42:47)
And I like the way you phrase it. You’re governing the work. I’ve heard, okay, you’re supervising or you’re checking. I think governing is a better word for it, given what you’re saying about you really have to think about these systems. What’s going to make you trust it? And think about that in terms of how to govern it.

Joel Hron (42:51)
Right.

Right.

Well, and again, I mean, the accountability does not shift away from the human in this process, right? And so that’s why I think governance is a good word. Because at the end of the day, if we ship a bug, it’s not like calling up my AI model and telling it how bad of a job it did. Engineers are accountable.

Greg Lambert (43:29)
Claude, you’re fired. Codex, you’re in.

Marlene Gebauer (43:29)
You messed up, Claude.

Joel Hron (43:32)
That’s not what happens.

I think humans maintain accountability. And so it’s incumbent on us to really lean into, how do we maintain that accountability in a way that we can stand behind and trust?

Greg Lambert (43:43)
Joel Hron, Chief Technology Officer at Thomson Reuters. Thank you very much for coming in and unpacking. Man, there’s a lot going on. It’s an exciting time to be in the industry, isn’t it? You bet.

Marlene Gebauer (43:54)
Hmm.

Joel Hron (43:55)
Absolutely. Thank you for having me, Marlene and Greg.

Marlene Gebauer (43:59)
And thanks to all of you, our listeners, for taking the time to listen to The Geek in Review podcast. If you enjoyed the show, please share it with a colleague and don’t forget to like and subscribe. Joel, where’s the best place for people to follow your work and learn more about what Thomson Reuters is doing with CoCounsel Legal?

Joel Hron (44:17)
Yeah, I try to put a lot of updates out on LinkedIn pretty regularly about new things that we’re doing and shipping and partnerships and things like that. So definitely I would say follow me or follow Thomson Reuters there. And I would also say you can gain access to the next versions of CoCounsel, which are in beta here pretty soon. And you can check that out on our website and try to get access. It’s something we’re quite excited about right now.

Marlene Gebauer (44:43)
All right, I’ll definitely encourage everybody to check out what you’re doing and talking about. And I should note, as always, the music you hear is from Jerry David DeCicca. Thank you so much, Jerry, and bye, everybody.

A few weeks ago I ran the numbers on the token cost panic. I took the scariest figure in legal AI, the finding that agentic workflows burn a thousand times more tokens than a chat query, and followed it all the way down to a dollar amount on a real deal. The panic did not survive the arithmetic. The piece is here if you want the full walk-through.

This is not that piece. The panic has moved on since I wrote it, and the new versions are smarter than the old one. The thousand-times number has quietly retired, because a thousand times almost nothing is still almost nothing. In its place are three fresher anxieties, and they deserve a real answer. The first says the model makers have a monopoly now, the price of a token is climbing, and it will climb forever, so you had better lock in a flat rate or build your own models before it does. The second says forget the price of a token, watch the meter: every time the AI reads your contract it ticks, and a long agentic session reads your contract over and over and over. The third does not bother with an argument at all. It just points at a number. One company spent five hundred million dollars on AI in a single month, and the number is so large it does the panicking for you.

All three are wrong. They are wrong in more interesting ways than the original, which is the only reason I am writing this down instead of linking to the first piece again. But underneath the new costumes it is the same body. Every version of this panic makes the same mistake and reaches the same conclusion. So let us stop swatting the individual numbers and name the thing that keeps generating them.

The Mistake Underneath All of It

Here is the error, stated once, because everything below is a variation on it.

A token is the unit a model uses to bill you. It is not the unit your work is measured in, it is not the unit your client pays for, and it is not the unit anything you care about is denominated in. It is a meter reading. The entire genre of token panic consists of staring at the meter reading as though it were the fare, the destination, and the quality of the ride all at once.

It is not any of those things. It is the meter. And a meter, by itself, tells you nothing about whether you are getting a good deal. A taxi meter reading of forty dollars is a bargain to the airport and a robbery around the block. The number on the meter is the least informative number in the entire transaction, because it means nothing until you put it next to what the ride was worth. Every piece in this genre forgets that, and forgets it in a slightly different way. Let me take them in turn.

“Prices Only Go Up”

Start with the monopoly story, because it has a real fact inside it. Yes, the newest frontier model costs more per token than last year’s newest model. That part is true. What the story does with it is the problem.

It draws a line through two dots and calls it a trend. Frontier prices up, therefore prices up forever, therefore lock in a flat rate before the meter eats you. But you are watching the wrong number. The price of a frontier token is not your cost. Your cost is what it takes to finish a task, and the cost of finishing a given task has been in freefall for two straight years. The same capability that ran on the most expensive model available in 2022 runs today on something on the order of two hundred and eighty times cheaper. Last year’s frontier is this year’s mid-tier is next year’s free default. The token at the very tip of the frontier gets a little pricier each release; everything behind the tip collapses in price behind it. Gartner expects another ninety percent drop in inference cost by 2030.

Watching the frontier price and concluding that AI is getting more expensive is reading the thermometer and announcing a fever, while ignoring that you are holding the thermometer over a candle. The evidence that the baseline is getting cheaper often sits right there in the same articles raising the alarm, quoted from the experts and then left unaddressed. You do not build a cost strategy on the one number in the system that is engineered to always be the highest.

“Watch the Meter Tick”

The second version is more seductive, because it comes with a picture. There is a meter. It is running. It is not visible and nobody is watching it. Every question you ask, every document you paste, every time the model reads back over the contract, the meter advances, and an agentic session is one long ride with the meter buried somewhere you cannot see it. Be afraid of the meter.

It is a good picture. It is also describing a machine that was rebuilt about a year ago.

Here is the mechanism the picture leaves out. When an AI reads a long document, that document is loaded into its context once, at full price. Every subsequent time the model reads back over it, that is a cached read, and every major model provider bills it at a discount, anywhere from fifty to ninety percent off, depending on the provider. You pay full freight to put the contract in the room once. After that, every time the model reads back over it costs a fraction of that first pass.

So the entire horror story, the one image every version of this panic is built on, the machine reading your document over and over while the meter spins, describes a problem the providers fixed before most of these pieces were written. The agent that reads your contract fifty times is not paying fifty times to read it. It pays full price once and a steep discount on the other forty-nine. The panic prices every one of those reads at full freight; the actual bill is a small fraction of that. The meter is real. It barely moves on the part everyone is pointing at.

I want to be precise here, because imprecision is how someone discredits a whole argument over a footnote. Cached reads are discounted, not free, and a long enough session still adds up. The cache expires after a few minutes to an hour, so the discount lives inside a working session rather than forever. None of that rescues the panic. Agentic work is high-frequency, same-session, re-reading-the-same-context work. That is the precise workload the caching discount was built for.

“But Look At This Number”

And then there is the half-billion-dollar bill.

The story made every outlet, because it is built to. One company, unnamed, spent five hundred million dollars on Claude in a single month. It is worth knowing where that number comes from: a single AI consultant, quoted in a single report, describing a client they do not name. No company has confirmed it and no one has verified it. That did not slow it down for a second. The number is enormous and the number is unverified, and it is being passed around as proof that AI costs have slipped the leash and big companies are getting torched.

So let us take it at face value anyway, because even granting every word of it, the story argues the opposite of what it is being used to prove. Read the second sentence and it falls apart in your hands. The company spent five hundred million dollars because it put no usage limit on the licenses. Thousands of employees had unlimited, uncapped access, and for a month nobody looked at the meter. That is the story. That is the whole story.

This is not a company that got beaten by the cost of AI. This is a company that took its foot off the brake, tied the steering wheel down, climbed into the back seat, and then expressed surprise at where the car ended up. The number is not evidence that AI cost is uncontrollable. It is evidence of precisely the opposite, because the controls exist and this company chose not to use a single one of them.

The controls are not theoretical. Claude Enterprise ships four levels of spend control: an organization-wide monthly cap, group caps, caps by seat tier, and individual per-user caps. They are hierarchical, so a user cannot exceed their own cap, their group’s cap, or the organization’s, whichever is lowest. When someone hits the limit, they are blocked. That is finer-grained cost governance than most firms have ever had over their e-discovery spend, and every serious enterprise AI platform ships some version of it. It is a shipping product, not a roadmap promise.

A company spent half a billion dollars by switching all of it off.

A company that gives thousands of people uncapped access to any metered resource and looks away for thirty days does not have a token problem. It has a management problem wearing a token costume.

And there is a second reason the number should not frighten you, specific to the kind of work most professionals are actually doing. That bill came from unattended automation: thousands of processes looping on their own, re-reading and retrying around the clock, with no person waiting on any single step. That is the only kind of AI spending that can run away while you sleep, because it is the only kind with no human in the loop to stop it. Attended work, a person at a desk directing the tool and reading the results, is capped by something the runaway scenario removed: human attention. Even a whole firm only has so many working hours in a month. I have tried to reach half a billion dollars a month with nothing but attended sessions, stacking every worst case, the largest firms, the most expensive model, every lawyer running flat out all day with the caps off, and the ceiling lands in the low single-digit millions, an order of magnitude short, and even that requires assumptions no real firm would survive.

The only way to clear that ceiling is to take the human out of the loop. Here someone will object that firms will simply start running headless processes of their own, looping unattended like the company that ran up the bill. They will not, because there is nothing for such a process to do. The runaway scenario needs a task with no natural end, something that can spin on itself for a thousand hours and always find more to spend on. Legal work is not shaped like that. A contract gets reviewed and marked up and it is finished; there is no ten-thousandth pass, because there is no ten-thousandth version. The deliverable has a floor, and the floor is the cap, whether or not anyone is watching.

And if none of that reassures you, there is a backstop that costs nothing to set. Cap each person at some deliberately absurd number, ten thousand dollars of usage a month, and forget about it. No one doing supervised legal work will ever come close, so it never touches real use. But it is not really a spending limit. It is a smoke alarm. The day someone actually hits ten thousand dollars in a month, the cap has told you something has gone wrong, a broken process, a headless loop someone set running, a mistake worth finding, long before it becomes a number worth fearing. The company in the story did not lack a way to prevent the bill. It declined to use one.

The most-cited number in the entire panic turns out to be the best argument against it.

The One True Thing

Now let me give the panic its due, because there is a real fact in here and I am not going to pretend otherwise.

Your AI bill probably is going up. Not the per-token price, the bill. Even as the price of intelligence collapses, total enterprise spend on it has risen sharply, because the work has moved from a single chat answer to an agent running an entire multi-step task, and that consumes vastly more tokens. That is real. That is the genuine signal buried under all the noise, and the people watching their invoices climb are not imagining it.

But look at what the climbing number represents. The bill went up because the machine stopped answering a question and started doing the job. Two years ago the meter measured a chatbot composing a paragraph. Today it measures an agent reading the deal room, drafting the issues list, checking it against the precedent, and revising its own work. Of course the meter is higher. It is doing thirty times the work, because there is thirty times the work being done, work that used to belong to a person and a timesheet.

So the question was never “why is the meter higher.” The question is the one this entire genre is constructed to avoid: what is the meter now doing that it could not do before, and what did that work cost you the last time a human did it? Put the number as high as you like. Four hundred dollars a deal, four thousand, forty thousand: the question does not change, and neither does the answer, as long as the work it replaced cost you more. A bill that tripled while absorbing the work of a first-year associate is not a cost problem. It is the best trade your firm made all year. You only get to be horrified by the number if you refuse, the entire time, to look at the other side of the ledger.

What Actually Deserves Your Attention

There is a version of cost discipline that is not panic, and it is worth naming so it does not get lost in the noise.

Use the cheapest model that does the job. Route the easy work to the small model and reserve the frontier for the tasks that need it. Do not paste the entire deal room in to summarize one clause. Cap your users. Watch your meter, not because the meter is the enemy, but because watching the meter is just management, and a firm that cannot see its AI spend by matter and by practice group should go build that visibility before it signs anything. None of this is glamorous. None of it sells a product. Nobody is going to write a breathless thought piece urging you to right-size your model selection, because there is no panic in it and no vendor on the other end of it. It is just the unsexy discipline of knowing what a task is worth before you run it, which is the same discipline the profession has always claimed to have and rarely does.

That is the whole legitimate concern. It fits in a paragraph. Everything past it is theater.

Why It Keeps Coming Back

Here is the part I actually want you to take away, because it will outlast the next five versions of this.

The token panic recurs because it is the comfortable debate. It lets a roomful of smart people argue urgently about something that does not threaten anyone. It is easier to compare per-seat pricing against per-token pricing than to ask what happens to associate leverage when the work a first-year used to bill for is absorbed by a machine that costs four hundred dollars a deal. It is easier to fear a meter than to ask who, exactly, captures the value when AI makes a partner ten times more productive: the client, the firm, or the vendor. It is easier to publish a chart of rising token prices than to sit with the fact that the entire economic structure of the firm, the leverage pyramid, the billable hour, the margin built on associate hours, is the thing actually being repriced, and the tokens are a rounding error inside that story.

The panic is a place to hide. Every few weeks it comes back wearing a new number, because the number is never the point. The number is the thing people reach for so they do not have to look at the ledger underneath it.

So do the boring things, the ones that fit in a paragraph, and then put the meter down and go have the uncomfortable conversation. That is the one that decides which firms are still standing in five years.

The tokens were never going to.

This week on The Geek in Review, we talk with Abdi Shayesteh, CEO of AltaClaro, and Jeanine Conley Daves, Littler’s New York office managing shareholder, about a different question in the legal AI conversation. Instead of asking whether AI will write the brief, summarize the contract, or replace the junior associate, they focus on whether AI might help lawyers learn how to practice law. Their recent work around AltaClaro’s DepoSim points toward a model of legal training built less on passive observation and more on structured repetition, feedback, and skill development.

Shayesteh traces the origin of AltaClaro back to his own early years at King & Spalding, where he benefited from proximity to a mentor willing to explain the work. That experience also showed him the unevenness of the old apprenticeship model. Access to assignments, feedback, and sponsorship often depended on luck, relationships, and office geography. For Shayesteh, the idea of a “flight simulator for lawyers” grew out of the realization that pilots, athletes, and musicians all practice in structured environments before performance, while lawyers too often learn in front of clients, courts, and opposing counsel.

DepoSim applies this flight simulator concept to one of litigation’s highest-pressure skills: taking and defending depositions. The platform gives attorneys a simulated witness, opposing counsel, court reporter, and feedback system, with options to vary the difficulty and personalities involved. Conley Daves explains why this kind of realism matters. In a real deposition, a lawyer might face an evasive witness, a hostile witness, an aggressive opposing counsel, or a combination of all three. The simulator lets lawyers practice those moments repeatedly, receive targeted feedback, and return to specific skills such as exhibit handling, follow-up questions, or managing objections.

The conversation also connects AI training to equity in professional development. Conley Daves notes that access to high-quality assignments and sponsorship has not always been distributed evenly across firms. A standardized, rubric-based feedback system gives more lawyers a chance to build core skills without waiting to be selected by the right partner or assigned to the right matter. Shayesteh adds that firms seeing the strongest results are not treating training as an after-hours side quest. They are creating protected time for deliberate practice, pairing AI feedback with human mentorship, and using simulation as a bridge rather than a substitute for coaching.

Looking ahead, Shayesteh and Conley Daves see simulation moving well beyond depositions. Oral argument, cross-examination, meet-and-confer sessions, negotiations, client interviews, and even Supreme Court preparation all fit within this training model. The larger shift is not automation for its own sake. It is the use of AI to help lawyers build judgment before the stakes are real. For law firms, that means better preparation, more consistent training, stronger associate development, and a clearer path toward delivering value to clients. For the profession, it suggests a future where competence is practiced deliberately, measured thoughtfully, and taught more fairly.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | Substack

[Special Thanks to ⁠Legal Technology Hub⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Greg Lambert (00:00)
Hi, I’m Greg Lambert from The Geek in Review and I have Nikki Shaver with us from Legal Technology Hub. Nikki, I know you guys do a lot of surveys and one of the most recent ones is a survey of the European legal market on AI usage. So, do you mind filling us in on that one?

Nikki Shaver (00:17)
Thanks, Greg. Yeah. We really find we’re getting a lot of outreach from firms asking for benchmarking. They want to know how they’re doing in the market, how mature their AI program is compared to other firms. So we do a lot of surveying. We find it’s one of the things that people are most interested in. What is the data on the market? We’ve done that for a number of different verticals within the US legal market. So we’ve surveyed mid-law, we’ve surveyed large law.

We haven’t really seen this kind of data coming out of continental Europe yet. We partnered with Lexpo, an organization that runs an annual conference out of Amsterdam. This year, it’s on June 8th and 9th. And we have now surveyed the European law firm market for AI maturity and also to understand how significant their AI adoption rollout has been and what they’re using, who is using what for what types of use cases.

We have seen early data on this and it’s very interesting to see in which ways the European law firm market is different from the US law firm market and where AI maturity differs, where different tool use differs. So fascinating outcomes. We will be presenting on this. Chris Ford from Legal Tech Hub will be presenting on stage at Lexpo. So if you’re going, look out for that.

Any participating firms get the full report ahead of time, but we’ll also be doing content on this on Legal Tech Hub, both on the European law firm survey itself, but also comparatively with the US market. So it’s one you will not want to miss. Look out in June for some content coming on that. We are at legaltechnologyhub.com. Thanks, Greg.

Greg Lambert (02:01)
You’re welcome. We do love to see what our competitors are doing. Thanks for setting that benchmark.

Nikki Shaver (02:06)
It’s a good driver of adoption.

Greg Lambert (02:08)
Yes, it is.

Marlene (02:16)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal profession. I’m Marlene Gebauer.

Greg Lambert (02:23)
And I’m Greg Lambert.

Marlene (02:25)
So Greg, we have spent the better part of two years debating whether AI is going to draft our briefs or summarize our contracts. But I think we’ve been missing the most important question. You know, can AI actually teach us how to be lawyers?

Greg Lambert (02:43)
Yeah, that’s an interesting and critical point that I think we’re running into now. And the old sink-or-swim apprenticeship model, where you’re kind of trying to look over the partner’s shoulders to learn the practice, is effectively dead, although, you know, between us, I think it’s been dead for a while now. But between the billable hour work and remote work, the osmosis training has kind of vanished. So it’s leaving associates in these high-stakes gaps between law school theory and actually being able to perform in the courtroom.

Marlene (03:22)
So we’re tackling that today. You know, our guests are building the, the, the flight simulator, if you will, to bridge that gap. They recently co-authored a piece in New York Law Journal, arguing that the real promise of AI isn’t automation, it’s deliberate practice.

Joining us is Abdi Shayesteh, the CEO of AltaClaro. Abdi is a serial entrepreneur who started his first company at 17 and spent 15 years as a lawyer at firms like King & Spalding and as a deputy GC at MUFG before launching a platform used by nearly 80 of the Am Law 200.

Greg Lambert (03:59)
And I will say, Abdi and I worked at King & Spalding together back in the day. And joining Abdi is Jeanine Conley Daves, who is a top tier trial lawyer and the office managing shareholder of Littler’s New York office. So, a little small office in New York there, right Jeanine? Jeanine is a leader who spent her career not just winning defense verdicts, but actively interrupting bias.

Marlene (04:03)
Yeah, that’s right.

Jeanine Conley Daves (04:18)
Yeah.

Abdi Shayesteh (AltaClaro) (04:19)
Thank you.

Greg Lambert (04:27)
and opening doors for the next generation of diverse talent through our work with the New York Urban League and Association of Black Women Lawyers. So, Abdi and Jeanine, thank you very much for being on the show and welcome.

Jeanine Conley Daves (04:40)
You’re welcome. Thank you for having us.

Abdi Shayesteh (AltaClaro) (04:43)
Thank you, great to be here. Good to see you both again.

Greg Lambert (04:46)
Yeah.

Marlene (04:47)
Yes.

So Abdi, you said your success as a young lawyer felt like it really depended more on luck than on a structured system. Specifically, being lucky enough to have an office right next to a mentor who would actually explain things to you. And I think a lot of people who came up around the same time feel the same way. I know I do. So how did those early years at firms like King & Spalding

shape your conviction that the guild tradition of legal training was really withering away and why did you pick depositions as the first high-stakes skill to tackle with a simulator?

Abdi Shayesteh (AltaClaro) (05:29)
Yeah, no, thank you Marlene. A great place to start is back in the day at King & Spalding, I got lucky. I had a mentor or sort of, I don’t think he knew he was my mentor, but I made him my mentor. And he was always available, going over things. And I realized very quickly that the pathway to becoming a better practitioner is all about assignment feedback, assignment feedback. The more assignments you get, the more feedback you get, the better you become, the better your judgment skills develop.

and the diverse set of assignments you get as well. And I realized that not everybody had the same access as I did. And in particular, I mean, you know, law school doesn’t teach you any of this stuff, right? So you come in after spending hundreds of thousands of dollars on your education thinking this is it, I’m gonna learn everything here. But unfortunately, even back then, the apprenticeship model started withering away and that you saw many people after four years doing the grunt work and being asked to leave or they just kind of gave up, right?

And usually those were, unfortunately, diverse lawyers and female lawyers because the same people were getting the same deals over and over again. And I thought right there and then, 1185 Avenue of the Americas, New York office of King & Spalding, wouldn’t it be great if there was a flight simulator for lawyers? And I was just thinking about pilots get to put in hours and get feedback before taking off. Of course, was a fantasy of mine.

And it wasn’t until I was in-house counsel, almost a decade and a half later, that I really realized the pain point, that I didn’t want associates spinning their wheels, spending 10 hours on drafting my NDA, where it only should take an hour. And that’s where I got the entrepreneurial bug again to launch AltaClaro, a place where attorneys can practice safely in simulated environments on assignments that are taken from the real world and get feedback.

And since then, this was 2016, when the cracks started happening, when this apprenticeship model was on its way out, we have continued to grow. We only had like five courses back then. Today we have over 60 across a variety of practice areas, from corporate to litigation, real estate, lending, all this good stuff. And we have now, actually you said 80, it’s over 100, Am Law 200 firms. I’m very grateful for this growth.

A few years ago, three years ago, we started experimenting with AI and giving immediate feedback on these simulated assignments so that associates could become aware, like an athlete becomes aware, or a musician who is practicing and getting feedback on where their strengths are and where their gaps are. And AI can be used as your friend in this way, not random AI, not ChatGPT, but a very structured rubric framework where those things are consistent and they’re evaluating you in an objective way on how to improve. And this was a hit. We deployed it for all of our clients and associates loved it. And the firms also had analysis on where gaps were so that when it came time to internal training or mentoring, they were building on top of it, but very targeted, right? So it became very efficient for partners to not start from ground zero. We won an award for this last year at Legal Week by ALM. Very grateful for that.

Right around that time when we started thinking, this is how we got to depositions. How can we use this technology, because up to now our courses and programs have focused on the written work product of a lawyer, drafting motions, drafting deposition outlines, contracts and things like that. How can we use it for the human-to-human parts of a lawyer’s job? The things like oral negotiation, taking a deposition, defending a deposition.

How can we leverage it for those things that hopefully AI will never replace and build the human muscle for judgment? Something like deposition. And this story started with Jeanine and me, believe it or not. This is the first time I’m talking about it. So Jeanine was representing us in a case. The case name is not mentioned here.

And there I was, and we were right around that time thinking about what to do next in our product roadmap. And I was watching Jeanine depose somebody. We went in with our set of questions, and right away we saw how hostile this witness was, very frivolous case, and how rude, and this and that. And then how this opposing counsel even made it past law school. It was just this obscure situation. And I thought, how do you train for this? How do you train for thinking on your feet, reworking your questions?

and how do you get an associate? Because I would not want a junior associate on something like that. And that’s when I realized this could be a great opportunity to leverage AI for this. So that’s when we went to work and leveraged AI to build a deposition simulator. So we built an AI witness, AI opposing counsel, AI court reporter, and guardrail framework leveraging the technology we had used with our benchmark tool. And you can go in and toggle and say, give me a hostile witness, or you can say, give me an opposing counsel who is super aggressive as many times as you want, like a flight simulator and practice your deposition. You get feedback. You go as little as 10 minutes or as long as six hours if you want.

And get that feedback so that you can become better and help you think on your feet in these situations when the real one comes. So we launched that recently, but before we launched it, we went to some of our clients that are big on investing in technology and big on investing in training, like Littler Mendelsohn, and together, six firms put in over 160 hours of testing it and giving us feedback. And they loved it. They loved it to even use as a warm-up tool. And so now anyway, this deposition simulator is being used in the market.

Greg Lambert (11:26)
I did want to go back to one thing that we kind of made a comment on you being lucky as an associate to have a mentor. Having known you, I know sometimes you make your own luck. It’s one thing to have that proximity. It’s another thing to make that step to make sure you build that relationship and kind of inject yourself in there and then having the EQ.

to read the room and know how to do that, yeah.

Marlene (11:51)
Luck favors the prepared.

Greg Lambert (11:53)
Speaking of EQ, in your co-authored article, I saw where you...

leaned really heavily into the research of K. Anders Ericsson, which looks at how elite athletes and musicians achieved their greatness through what’s called deliberate practice. And so from your perspective as a very prominent trial lawyer,

You know, what’s the legal equivalent of this deliberate practice where, you know, violinists may practice scales 50 times and, and why do you think this, you know, traditional law firm training that we’ve instilled like this passive CLE, you know, it’s like, okay, you got to have X hours of CLE and get them all in this month before your birthday this year. You know, what, why do you, why do you think that that’s

Abdi Shayesteh (AltaClaro) (12:42)
I’m sorry.

Jeanine Conley Daves (12:44)
Right.

Greg Lambert (12:47)
failed to provide the same kind of intensity and results in the law firm environment.

Jeanine Conley Daves (12:54)
I think, Greg, it’s really the act of doing that really helps you to grow and develop.

when you’re looking over someone’s shoulders, you’re looking at what they are doing, but not always are you really thinking about how would I respond to that? What would I say? And so, it’s while you’re having a chance, and what DepoSim is so helpful and useful to do, is that you have a real-life opportunity to practice. And not only to just do repetitions of one thing, but to

hone in on those skills to really focus on what am I doing well, what am I not doing well given the feedback that you are provided. And that is an opportunity that unfortunately, associates just, you know, haven’t had up until now to get that continuous practice in and to think about, okay, how do I want to improve on this? How could I have done better on this?

It’s having that ability to really focus on those areas. I think also, you know, from the

article in Anders Ericsson, who we talk about, he talks about, Mozart was a prodigy, but it really came from this rigorous training that made him as great as he was. And that’s exactly what we want from our associates. We want them to be able to build the skills when, as we see, AI is taking some of those other tasks in terms of memos, you know, research, even though you always have to check the research.

But this is a way for them to become strong litigators.

Greg Lambert (14:39)
Amen.

Yeah,

and I would assume that one of the most valuable things, and we bill by it, is time. And so I imagine that the past 15, 20 years it’s been this crunch on time being able to, for the people that need to be leading this type of training and teaching by doing just really haven’t looked like they’ve had the time. Is that a correct assumption on why we think that that has fallen a little short over the past decade and a half?

Jeanine Conley Daves (15:18)
Yeah, time is difficult and I will say that this giving...

associates the opportunity to go in and do it, you know, when they have time themselves to focus on this area without requiring another $1,000 biller or whatever the rate is to have to or $3,000 biller to have to take the time to not only train but to, you know, learn a sample or example of

Greg Lambert (15:36)
or $3,000 billers.

Jeanine Conley Daves (15:49)
sort of a case outline and all of that is actually going through the technology. I think you’re absolutely right, Greg. The time that it saves is critical.

Marlene (16:01)
Jeanine, Littler was one of the six early adopter firms that logged over 160 hours of testing during the DepoSim pilot this past February. I’m curious, you know, what did your associates like, you know, what aspects did they feel helped them most?

Jeanine Conley Daves (16:21)
I think that...

One of the biggest things for them is how real it is. Abdi talked about a deposition that he saw me take. And when you go into a deposition, you really don’t know what you’re going to get. There are some opposing counsel who don’t object a lot, some that do object a lot. And so the fact that you can get all of these different personalities so that you really have an

understanding and can get more comfort with whatever, not only opposing counsel that you get, but whatever witness that you get. And if that person is being evasive, being able to practice, how exactly do I handle that type of situation, I think is really helpful with DepoSim and is what I think a lot of the associates really got out of it.

with, again, getting that feedback. So, not only am I practicing, but I’m figuring out where do I need to change? Where can I tweak? Where can I make better responses and make sure to get the answers that I’m actually looking for? Having those two things, I think, has been a huge help. And you’ve had associates, we’ve gotten sort of direct feedback.

Great feedback in terms of it’s a 10 out of 10. Associates saying like this is really true to life in terms of the witnesses and the opposing counsel that you are dealing with. And just having, you know, the experience I think has been really helpful to the attorneys that have had an opportunity to do this.

Marlene (18:05)
I’ve seen DepoSim and I can vouch for the realism of the simulator. There’s a number of different types of clients and counsel and you can kind of mix and match those things as you would in real life.

Abdi Shayesteh (AltaClaro) (18:22)
Yeah, yeah, that’s right. K&L Gates was also a part of that 160 hours. It was great to see the partners getting into it, even with opposing counsel, which you can do. And even the court reporter, just in case something, which, yeah, even the court reporter.

Marlene (18:39)
That’s right, I forgot about the court reporter.

Jeanine Conley Daves (18:41)
That’s right.

Greg Lambert (18:43)
All kinds of stuff.

Marlene (18:43)
Now, how are the evaluations used? So, like, I know that you get a feedback evaluation, but, you know, after that, so what do folks do with those, you know, after they get them? Do they sort of focus on certain parts? Do they do the whole thing over? Like, what have you found is best?

Abdi Shayesteh (AltaClaro) (19:03)
Yeah, it’s all up to the learner and where they are and what they’re using it for. But all of the experiences inside the simulator is just for the learner. So if they wanted to, they could share the transcript, the feedback report, even the audio with a mentor. And so talk about getting efficiency with mentorship time. Let’s say you’ve done this, and then at least a few times, and you got your best one. Now you want to go in and talk with a mentor about some of the things you’re working

on and they can now add on top of it more efficiently because you already got these other things out of the way. Let’s target where you can improve. So it can definitely be used in that way. If you get the report and it says, let’s say, your exhibit handling needs to improve.

You could say, all right, I’m going to go back in this time, and I’m just going to toggle and say, I’m going to just practice the exhibit handling. And I’m going to pick these exhibits. And I’m going to go in and do this for 30 minutes. So you can be able to do that. And that was part of the feedback we received from the first go-round, the ability to keep going back in and selecting what you actually want to practice. Jeanine mentioned evasiveness. One of the key things is following up when they’re being evasive, when the witness is being evasive.

or they say something and you should have asked more, right? And so that’s gonna show up on your feedback report saying, you know, they said this about their, I don’t know, education or their experience, but you could have followed up and gone deeper and you didn’t. It’s like, oh, okay. And so you can go back in and even just isolate that and practice and get feedback on it.

Greg Lambert (20:39)
You know, there’s one thing that I’ve been mentioning to almost anyone who will listen to me lately, and that’s I was listening to a podcast a few months ago and

Mark Andreessen had a quote that really kind of stuck with me and that was, he said, throughout history, the number one thing that moves people from the 50th percentile to the 99th percentile the fastest has been individual tutoring. And,

There’s not a lot I agree with Mark Andreessen on, but this was one of the things that I did agree with him on. And so I see a system like this and I see that individual tutoring. And one of the interesting things that I was noticing was this wasn’t just the first years that were using the tool, that, you know, the partners, as you mentioned, Abdi, were also, you know, using it for their pregame rep.

getting ready for real depositions. So why do you think that the seasoned litigators are seeking out these types of synthetic reps in order to help get them prepared, build past the stigma of being afraid to say that they may not be completely ready for it. How do I get better?

Abdi Shayesteh (AltaClaro) (21:41)
Yeah.

Marlene (21:53)
kicking the tires beforehand.

Abdi Shayesteh (AltaClaro) (22:04)
Yeah.

That was a cool discovery, because we went in thinking this is just for juniors, mid-levels, great. But when we got the feedback, the specific question was, will you use this again? 94% of all participants said they would use it again for deposition practice. We said, wait a minute, the seniors said they were using it. So we leaned and went back to them and asked, tell us a little bit more. I said, yeah, I need to warm up. And it kind of makes sense, right? An athlete, right? They’re a professional athlete.

But they’re going to have to warm up before they go in the arena. Right? And same with the musician. They’re going to have to practice the scales, practice the songs right before the recital. And it started to make sense. Some of them told us that they just don’t do it as often. It doesn’t happen often. And others said, sometimes I just want to target practice. I’m anticipating this witness to be polished. So this would be cool, because this is going to remind me, get me in the zone of following up, digging deeper, because the person is polished. So it started to make sense when it was, yeah, of course, even if you’ve practiced for 20 years, you’d like something like this to leverage to warm up. So as a result, the firms asked us to deploy it across all their litigators, including their partners and seniors, which was cool.

But we also saw other things come out of the pilot. A chair of one of the law firms said, I want to use this. I want to pull up DepoSim in front of all my associates during a training session. And I want to practice. I’m going to toggle for the hostile witness, toggle for the aggressive counsel. And I’m going to teach my techniques.

using the AI bots. And as I teach a technique, I’m gonna then say, watch me do it. And then I’m gonna, when the witness reacts, I’m gonna pause and say, you see what he did? Watch what I do in return. And so like they’re using it that way, which is cool. And then after their session, they’re saying, all right, you all go out and do this for an hour, get your feedback reports, let’s come back and let’s review it together. So that’s been cool because now the AI is, it’s not just, I’m in my room and no more human connection. This is actually bringing people together for in-person training. Some firms are saying, you associates pair up and divide the deposition amongst you and go practice against DepoSim afterwards, exchange your feedback notes, and then let’s come back. So it’s actually bringing people together, which is cool.

Yeah, so anyway, that partner’s reaction was a real positive in many ways just to see how they were using it.

Marlene (24:36)
I think the rise of tools like DepoSim are quite timely because as Greg was mentioning earlier, sort of the old ways of learning are going away. You’re not going to be able to go through production as much and sort of draft and understand what the arguments are and what the evidence is saying as you used to because you were sort of poring through that stuff before a deposition happened. So, Jeanine, what do you think about that? Is associate development, is this gonna sort of be the new way of doing that to consider them practice ready?

Jeanine Conley Daves (25:20)
I think so, Marlene. I think that we will be using...

simulators to help us train our associates going forward. And I’m just so ecstatic, you know, that Abdi had this vision and that we as a firm are utilizing DepoSim because I think it’s going to be so helpful for our attorneys in terms of how they learn, in terms of their trajectory. Particularly, I think about

first-year associates. And like you said Marlene, there were a lot of things that we used to do, that we used to focus on. You used to have to be the master of the documents. And now a lot of that, the document review and memos, like you said, all of that, a lot of firms will now move through AI. And so to your point, I think that this is a wonderful

way to ensure that our attorneys are getting the types of trainings that they need. As an office managing shareholder, I mean, this is one of my biggest focuses right now on how do we make sure in a law firm that is an apprenticeship model and how we used to learn looking over someone’s shoulders and things of that sort, how do we make sure that we are still having critical-thinking associates

and associates who are able to grow and develop just like we were able to as junior attorneys. So I think it is transformational and I think that you definitely are going to see law firms using simulators and these types of synthetic environments for training going forward.

Greg Lambert (27:10)
And Jeanine, I know one of the recurring themes that has followed you throughout your career has been this ability to ensure that sponsorship and assignments are handled equitably. But we know traditionally that we’ve kind of fallen a little short on that and it tends to be who do I go to that I know that I have the connection with.

So I’m wondering, can a standardized rubric-based AI feedback system like a Benchmark 360 or DepoSim act as almost a leveling type of agent in this to make sure that every associate gets kind of the same high-level coaching, you know, regardless of their background, regardless of who they know, but it’s part of the process?

Jeanine Conley Daves (28:08)
I think it’s one of the biggest benefits that...

we have here is the fact that you are going to get as many associates across the board at varying levels having this opportunity. And now again, as you heard Abdi talking about, you even have partners saying, I’m going to use it to warm up. And so I think it does create a more equitable playing ground in an environment that quite frequently it has been about sort of relationships and as Abdi talked

about it being part of the impetus of where he thought to really focus on training associates in this way is making sure that it’s not just about you’ve developed or gotten the eye of a partner who is now taking you under their umbrella, making sure you get great assignments. This type of training is going out across

the board and you can choose as an attorney how much you put into it to help develop your future.

Greg Lambert (29:16)
Yeah.

Let me pull on that just a little bit because there’s like in the software development world, especially in the age of AI, there’s this expectation that this sort of going and learning new things is kind of something you do outside the normal hours of

business, and one of the things that I’ve heard is a concern is that that’s going to hit kind of the non-traditional people. You know, people with families, maybe people that, you know, have outside obligations, that they just don’t have that extra time to do that and that those are the folks that are going to get kind of hammered.

you know, if this becomes part of the evaluation, is this something, Jeanine, at Littler, is it built into the process of training, or is this something that is expected that people will just kind of take on as extra work? How do you balance that?

Jeanine Conley Daves (30:24)
I think it’s a good question. I will say, Greg, that I think that part of the issue that a lot of people in those roles had is that quite frequently things are rigid. So unfortunately, they could only do it at certain times. I think having the opportunity to actually be able to work it into your own schedule is something that is going to be

more helpful than harmful. And so you give people the opportunity to really set their own schedule instead of saying, we’re going to this happy hour after work in terms of developing those networks and relationships when it can be difficult for those groups. So I think it’s going to end up being more helpful for that purpose.

Greg Lambert (31:18)
Do you have any?

Abdi Shayesteh (AltaClaro) (31:19)
Yeah, no, I want to add to that because it’s great to see firms like Littler giving attorneys this tool to be able to train, like Jeanine said, at any time you want. But they’re also creating the space for training for deliberate practice, which is great to see. And we see that in firms who are successful in getting their attorneys in this next era of building judgment.

And that is, for example, in our regular training programs, Littler Mendelson created Fridays and Mondays to do the AltaClaro program. Friday, they watch an hour of video, they do a two-hour assignment, and it’s blocked on their calendar, they turn it in. On Monday, it’s the live review session, and they get the report. So they’ve blocked it on their calendar so they can do it, and then they have other priorities the rest of the month. The next month, it’s repeated. Things like that, that we’re seeing at firms create the space to give

their attorneys and associates the space to engage in deliberate practice really becomes the pathway of differentiating the firms who are going to be ahead of the pack in terms of having associates with good judgment.

Greg Lambert (32:28)
Good to have that intention set forth, you know, upfront on that because there’s so many that I think just kind of tack it on and it’s like we’ll put this wherever it fits and it typically doesn’t fit in the eight to five and so yeah it’s good to see that.

Abdi Shayesteh (AltaClaro) (32:46)
Yeah, and then they try to sandwich it all in and then they say, well, you’re not up to snuff. I said, we didn’t give you the chance to even learn.

Greg Lambert (32:53)
Yeah.

Marlene (32:55)
So, Abdi, I’m gonna expand a little bit on your answer, because I know that you have shared advice that you’ve gotten was to lead with empathy and care. And while I, I’m not gonna go down the road of the stereotype of, it’s like the evil partner and all of that. But.

I think, because I do think a lot of partners and senior associates, you know, do care and they do try and help junior people. But I do think there’s always the issue of, I don’t want to look silly. I don’t want to look uncomfortable. You know, I don’t want to make a mistake in front of people who I work with and supervise. But, you know, DepoSim,

gives sort of a safe space, if you will. Like this is a simulation that they do themselves. They get their own report. And you know, it’s okay to fail and to practice until you get better. So do you feel that that has an impact on, you know, the cultural tone of a firm, you know, the way associates view their own careers?

Abdi Shayesteh (AltaClaro) (33:59)
Absolutely, yeah, I think for me, I learned this because I don’t think I had it in the early days. It was just like they threw you in and they expected that you knew this. And then you’re like, I just got here. What do you mean you don’t know? No, I don’t know. You shouldn’t know.

Marlene (34:11)
It’s true. Sink or swim, as we said.

Greg Lambert (34:17)
Go figure

it out.

Abdi Shayesteh (AltaClaro) (34:18)
So it was painful, it worked. Back in our day, we grinded it out. And I think when I caught myself as a senior lawyer doing the same thing to a junior lawyer, I realized this is not good. The junior lawyer said, well, yeah, your charter document, you can’t do this, can’t do that. And he’s like, it’s my first one I’ve ever seen in my life.

And so it was like, wow, okay, I’m doing this. I got to break this habit. And yeah, if you can have empathy, actually it will end up serving you better. If you actually accept, first of all, no one is going to be a star attorney for the first few years. The way to get them there is to invest. And now our time is limited. That’s the other thing I saw when I was a senior associate, because I had good intentions. I wanted to train, but I just never got to it.

right?

Marlene (35:07)
I think that’s the big crux of the issue.

Abdi Shayesteh (AltaClaro) (35:07)
Yeah, the best that I got to was a few slides, which is terrible. It’s like, here are a few slides on how to swim, now go out there and I’ll see you out there. That goes to the luck.

Greg Lambert (35:19)
Yeah,

read up on it, now go do it.

Abdi Shayesteh (AltaClaro) (35:20)
read up on it and

go swim. And so I was like, okay, so this is where firms are leading with empathy who are giving their associates the space and the tools to practice, to make mistakes, to get feedback so that they can be better prepared. It will serve them, they know it will serve them better. It has an ROI. This is the ROI. The ROI is, first of all, you’re not gonna spend $1,000-an-hour people, or more, to create slides and teach the foundations, right? You don’t ask Michael Phelps to teach you frontstroke, backstroke, or float on the water. It just doesn’t make any sense. And then, and you know that doesn’t work anyway, right, by showing slides. So then you see this work, you have to rework, right? So partner time, redoing the whole thing at 11 o’clock, at 12 o’clock at night, this thing is new, right? And you have to write that time off. You can’t charge the client.

Right? You start calculating all this, right? And then, OK, I’m going to actually be a good lawyer and mentor. I’m going to sit down with the associate and explain what they did wrong, if I get that chance. Again, 12 o’clock at night. So this is the cost of not doing it right. And then in this day and age, when you have buyers of legal services, they have options, right? They can see which firms are making this investment, and they can see it in the work product. They’re going to move away.

And they’re going to go to those firms who invest in their associates and are creating better work product at a more efficient rate, whether they’re giving them better tools or giving them better training. So leading with empathy has rewards. And it’s better for you as a person too. But I think that’s where you start. You have to realize it’s just not going to work the old way.

Greg Lambert (37:02)
Well, I know both of you are at the top of your game for what you both do and it takes a lot of energy just to kind of keep up with things today. So one of the questions we’ve been asking our guests are what kind of resources, whether it’s blogs or articles or authors, how do you kind of stay up?

and ahead of the curve when it comes to legal AI and education and what you do. So, Jeanine, do you want to kick us off?

Jeanine Conley Daves (37:40)
Well, I would say that in terms of publications,

We’ve already talked about Anders Ericsson and I think taking a look at his Peak: Secrets from the New Science of Expertise that we quoted in the article, think are definitely is a good place to start as well as it’s hard to keep up with all the podcasts and the blogs that are going on today. But we also put out quite a bit of publications.

Greg Lambert (38:06)
I can’t even keep up with my own.

Abdi Shayesteh (AltaClaro) (38:09)
I’m

Jeanine Conley Daves (38:16)
here at Littler and are staying on top of AI, which is why we’re part of this project and recognize that the innovativeness of what Abdi has created is going to help our attorneys going forward. We continue to do a lot with AI, just hired a new chief AI officer, and I think it’s important that you stay abreast because times are changing.

Greg Lambert (38:42)
Sure. Abdi,

how about you?

Abdi Shayesteh (AltaClaro) (38:44)
I was gonna say, The Geek in Review, that’s the one I watch.

Greg Lambert (38:47)
Of course. Thank you. That’s really why we asked the question.

Marlene (38:51)
Like and subscribe

everybody.

Greg Lambert (38:52)
Hahaha

Jeanine Conley Daves (38:53)
Hahaha!

Abdi Shayesteh (AltaClaro) (38:54)
Absolutely, but yeah, I’m a big fan obviously of Peak: Secrets from the New Science of Expertise, really good stuff in there. I mean, it talks about applying this in the professional world. That’s really his point. He does list examples of the medical profession. Radiologists, and I’ll just say this one example, because Jeanine’s example of Mozart was great, but this radiologist example is really cool.

They studied radiologists five years out versus 20, 30 years out in terms of their skills in predicting because the problem with radiology is that you see something and you’re supposed to predict if it’s cancer, tumor, and so forth, right? And they were trying to see who has better predictions, the five-year-out or a 30-year-out? Who do you think did better?

Greg Lambert (39:42)
Well, logic would say the 30-year-out.

Marlene (39:44)
This is a trick

question, right. Five-year-out.

Abdi Shayesteh (AltaClaro) (39:45)
There’s a question here.

Jeanine Conley Daves (39:45)
Just a quick question.

Abdi Shayesteh (AltaClaro) (39:47)
They were so experienced,

right? It was the five-year-out. And the reason was, the reason was was that in their residency, they were getting feedback on their, you know, did they predict it right or not? Was this an actual tumor or not? They were getting feedback on the simulated assignments as well as real assignments. And so they were able to produce better. And then they noticed that these folks in the 20-, 30-year-out, they just say, yeah, this is a tumor, hopefully it’s not, off you go. They don’t get the feedback.

And so this transformed the certification process for the radiology medicine, medical industry that they now have to every year go through case files and predict and get feedback on their prediction.

And now all radiologists have to go through this. Anyway, Ericsson argues to leverage this stuff for professionals. And that’s where I think there’s a lot we can do in the legal profession, especially in this day and age. So that’s why I love that book. I want to keep going back to it. But there is a new book that it’s on my reading list that Jeremy, our co-founder, chief innovation officer, has read and recommended, Brave New Words, How AI Will Revolutionize Education, and Why That’s a Good Thing. Salman Khan from the Khan Academy. And when you read it, and then you go back to our article, but basically Khan emphasizes that with customized and accessible learning tools that encourage creative problem-solving skills and prepare students for an increasingly digital world, AI can be leveraged to help build these judgment skills for them, right?

So that’s exactly what we’re trying, that’s what we’re doing here, is to leverage AI for that. And it’s not just us, it’s the rest of the education world that’s using it. So anyway, that’s on my list to read.

Greg Lambert (41:32)
Yeah, Khan Academy taught me statistics, so.

Abdi Shayesteh (AltaClaro) (41:35)
That’s great.

Marlene (41:37)
It’s like, well now it’s on my reading list too. So it is time for our crystal ball question where we ask our guests to look a little bit into the future. And, you know, will, you know, do you think like every legal task, you know, could be from a client interview to Supreme Court argument, you know, is that first going to be rehearsed in a simulator? You know, what’s the single biggest shift you see coming?

Abdi Shayesteh (AltaClaro) (41:39)
Yeah, it’s a good book.

Jeanine Conley Daves (41:39)
Yeah.

Marlene (42:03)
in that regard.

Jeanine Conley Daves (42:04)
I think that you will see individuals preparing for Supreme Court arguments to depositions in simulators. And you’re going to see that, I think, more and more and more. As Abdi with DepoSim has shown, you can get, and probably something you need to think about, Abdi, various Supreme Court justice personalities.

when you’re trying to prepare for those given obviously all the arguments and content we have there. And so how better to prepare than to have that realism again that helps you just get better, that of course will help attorneys be more efficient. I think so much about the junior associates who don’t get an opportunity to take a deposition until they’ve been out several years. For them to have this opportunity to do it from day one is incredible. And so I think it really is going to change how our attorneys are trained and really advance their skills at an even earlier age.

Marlene (43:11)
And I think...

Greg Lambert (43:11)
Are you going to package

the negotiation experience into a Supreme Court argument?

Abdi Shayesteh (AltaClaro) (43:15)
Yeah.

Jeanine Conley Daves (43:17)
Yeah.

Abdi Shayesteh (AltaClaro) (43:17)
Absolutely.

It’s all on the roadmap. It’s all in the roadmap. And we got this feedback from all the design partner participants of what else they can use this for. And that’s exactly what we’re looking towards. First of all, this deposition is just the beginning. We just launched the employment case. We have a commercial litigation case. But the next few months, we’re going to have an IP case, an antitrust case, securities, all the key areas. And then after that, we’re going to look at other simulation types in the trial process, oral advocacy, cross-examination.

to meet and confer, this technology can be used for all of that and even negotiating a deal. And eventually the technology is there, especially in our partnership with Verbit, where, by next year, we’ll be able to simulate real cases. Obviously they’ll have a security apparatus for that and we’re prepared to do that. So this is all on the roadmap. And yeah, we will be able to, by feeding in the right information to the AI bot,

be able to have these types of predictability. But just to your point, or Jeanine’s point about the impact of all of this is that I think one of the positive impacts is that we’re actually gonna have...

a better cost analysis of what things should cost. Because I think if you have associates doing these reps, doing these reps, being better prepared, and you’ve equalized these skills across the firm, you’re now going to optimize the time it takes to do something like this in the real world. And that’s going to be a great value-add for your clients. And this is kind of like the Intel inside, right? You know, you go back in time, that ad, that Intel, make sure your computer has an Intel inside.

It’s not that Intel was selling the computers, but they just wanted the world to know, make sure your computer has Intel inside. So it’s going to be the same thing, that make sure your law firm has these tools inside that they’re better preparing their associates for so that it’s going to be more efficient, it’s going to be better, it’s going to cost you less. And I think that’s going to differentiate. All those things, yeah.

Marlene (45:11)
better outcomes, better preparation, all those things.

Greg Lambert (45:16)
It’s exciting times, there’s more exciting times to come. So Abdi Shayesteh and Jeanine Conley Daves, thank you very, very much for joining us today and helping us rethink what it means to be a competent advocate.

Jeanine Conley Daves (45:30)
Thank you.

Abdi Shayesteh (AltaClaro) (45:31)
Thank you.

Marlene (45:32)
And thanks to all of our listeners for taking the time to listen to The Geek in Review podcast. If you enjoyed the show, please share it with a colleague and we would love to hear from you on LinkedIn and Substack.

Greg Lambert (45:45)
And Abdi and Jeanine, where’s the best place for listeners to learn more about the DepoSim rollout? So Abdi, you want to take that one?

Abdi Shayesteh (AltaClaro) (45:55)
Sure, you can go to altaclaro.com and there’s a special page for DepoSim and you can click there to see and watch a video and even reach out to us by booking a call to learn more about it.

Greg Lambert (46:07)
Thank you.

Jeanine Conley Daves (46:07)
And we just put out a post on LinkedIn about our use of DepoSim.

Greg Lambert (46:12)
All right.

Marlene (46:13)
Thank you both. And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and bye everybody.

This week on The Geek in Review, we talk with Ryan McClead of Sente Advisors about his new book on AI agents, written in collaboration with Claude. McClead explains how a short best practices guide grew into a full book after his work with Claude Cowork revealed something larger than tool tips or prompt advice. The result is part field guide, part warning label, and part first-person report from the edge of agentic AI adoption in legal work.

McClead’s process flips the traditional writing model. Instead of staring at a blank page, he asked Claude to generate an outline and draft, then spent weeks shaping, cutting, challenging, and refining the work. The book became a study in collaboration, with McClead serving as author, editor, supervisor, and occasional bouncer when the AI wandered too far from the point. His description of training Claude toward his voice, “more Anthony Bourdain and less Bobby Flay,” gives the episode one of its best lines and one of its most useful lessons.

A central idea from the conversation is “executable knowledge.” McClead argues knowledge management teams need to think beyond content meant for humans to find and read. The next stage is knowledge structured, so AI agents understand when to use it, how to apply it, and how to turn it into repeatable workflows. For law firms, this raises practical questions around scale, security, permissions, data quality, and governance. It also creates a new role for KM and innovation teams as builders of reusable legal intelligence.

The discussion also moves past prompt engineering as the main AI skill. McClead describes a shift from prompting to delegation, where users set goals, provide context, invite clarifying questions, and supervise the work product. The human role does not shrink in this model. It becomes more focused on judgment, direction, taste, and knowing when to take the work away from the AI before endless iteration turns progress into mush.

By the end of the episode, McClead frames AI agents less as replacements and more as strange new colleagues whose usefulness depends on the expertise of the person directing them. Good lawyers, KM professionals, and innovation leaders get faster and more effective. Poor processes get accelerated too, which is where the danger sits. For legal organizations, the message is clear: start small, learn the tool, build guardrails, and prepare for a future where clients ask not only for legal answers, but for legal workflows they can run.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | Substack

[Special Thanks to ⁠Legal Technology Hub⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com
Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Marlene Gebauer (00:00)
Hi, I’m Marlene Gebauer from The Geek in Review, and I have Nikki Shaver with us. Nikki, you are just back from the Harvey Forum, and you’re going to give us all the deets, so let’s hear it.

Nikki Shaver(00:11)
Yes, this past week was the Harvey Forum in New York. It’s the first time they have ever done this in New York, and the second time they’ve done Harvey Forum. The first time was in London. It was a really beautiful event. It’s always interesting when a vendor reaches that level of maturity where they have their own client conference. It was a privilege to attend, partly because it was in such a beautiful location, the Hall des Lumières in New York, with just stunning stained glass windows.

The whole event was very high class in the way that it was presented, with lovely food and everything, as you would imagine. And the content was really excellent. So Winston Weinberg, CEO of Harvey, started with a keynote talking about some of the new developments in legal tech broadly. I will say he shared the Legaltech Hub market map for generative AI tools, which was fun to see.

Marlene Gebauer (01:06)
Very good.

Nikki Shaver (01:07)
But he also talked about very interesting developments, for example, the escalating token use within the Harvey product, indicating increased adoption, but also the drain on tokens from the increased complexity of the product. Harvey also launched Command Center, which is a way that firms can now manage and track usage and get much clearer reporting to help them drive adoption.

He also announced a relationship between Harvey and DeepJudge that will give DeepJudge users access to DeepJudge’s legal research across multiple systems’ data in the Harvey system, which is really fascinating and which we will be covering in a content piece coming up soon. I spoke on a panel on the economics of law firm transformation with Jae Um, Esther Honigman, and David Cohen, moderated by John Haddock.

And we really dug into what it means for law firms to look at ROI in terms of value, not just from efficiency gains, but also from the ability to differentiate and focus on what a firm’s superpower is from an expertise perspective and a data perspective. There were other insights throughout the day around what a modern talent function looks like and how to think about law firm talent development in the future. Really interesting. So we will be covering it in an upcoming piece. To learn more about Harvey Forum and the law firm economics of the future and talent of the future, feel free to check us out, legaltechnologyhub.com.

Marlene Gebauer (02:49)
Thank you very much.

Nikki Shaver (02:50)
Thanks, Marlene.

Marlene Gebauer (02:59)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (03:05)
And I’m Greg Lambert. And Marlene, we have an old friend with us today. Old friend, practically Geek number four, I think, is how it is. Maybe 3.5. So yeah, and he’s written a book. And he just didn’t write it in a normal way, which is on brand for him.

Marlene Gebauer (03:13)
An old friend and a new friend.

He is. He is part of the Geek universe.

Yeah.

So we have Ryan McClead from Sente Advisors on today, and we’re really thrilled to welcome him because he has written a book. But not only has he written a book, he has written it with a co-author, and the co-author is Claude. So we’re really eager to dive in with Ryan and understand what his process was to write this book with his co-author.

Greg Lambert (03:57)
Ryan, it’s good to have you on the show. Thank you. All right, so you and I and Toby Brown and Marlene have been kind of previewing the writing process that you’ve done over the past few weeks. So it’s fun to see the book come to fruition. I ordered my copy this morning, by the way. Yeah, exactly.

Ryan McClead (03:59)
It’s good to be here.

Marlene Gebauer (04:19)
Everybody order your copies. We have a link.

Ryan McClead (04:21)
That’s it. I’m retiring. I’ve got everything else.

Greg Lambert (04:25)
So tell us about the book now, and then why do it in the way that you did this, by co-authoring it with AI and being very honest about the entire process?

Ryan McClead (04:40)
So it’s a good question. The short answer is I didn’t intend to write a book. That was not where I started. I think we all talked about or reported on the SaaS apocalypse when it happened, the announcement of the legal plugins, and everyone lost their minds over that. Shortly after that, I immediately started getting questions from clients. What is this? What do we do with it? How do we use it? Do we need to go buy it?

Greg Lambert (05:08)
Slow down, slow down. Things move fast.

Ryan McClead (05:08)
Hold on. I don’t know. I just heard it the same as you. Let’s take a look at it. Yeah.

Marlene Gebauer (05:13)
You got the inside scoop. Come on.

Ryan McClead (05:16)
That’s what I’ve seen happening in the industry at large. Everybody is trying to figure it out. It’s happening so fast. And I started using it so that I could explain it to my clients. I really just started building a best practices guide.

This is what I’ve learned as I use it. This is what works. This is what doesn’t. Here’s how I would do it. These are the things you need to know. Here’s the glossary, right? Of all the terms and the way they’re used and what they mean. It was a straight-up best practices guide that started, and it was like five pages, and then it was 30 pages, and then it was 60 pages of a Word document. And I’m like, okay, this is not, and I wasn’t nearly done. I’m like, there is so much in here. And I was writing that with Claude, using Cowork to create that.

And at a certain point, and I give the dates in the foreword, I think it was April 12th, maybe, I suddenly had this revelation: this is a book. Because it’s not just best practices. It’s actually a whole lot of my opinion and what I’m seeing and what I’m experiencing and what I think is going to matter to law firms, KM, innovation, the kinds of people that I work with.

And at that point, it was clear it was a book. So I said, in my conversation with Claude, still in the best practices guide project that I set up, “What if we made this a book? Can you outline what that book would be?” It gave me an outline. Yes, exactly. Yeah. And I addressed that in the book. Right. I mean, that’s an issue.

Greg Lambert (06:53)
Did it just say, “That’s a great idea, Ryan. Let me help you do that”?

Marlene Gebauer (06:56)
Brilliant idea, Ryan.

Ryan McClead (07:03)
And I said, okay, now why don’t you spin up a couple of agents and go write that, just to see what I would get? And it did. It spit out 79,000 words. Yeah, it was 19 chapters. It was like, okay, boom, done. Book done. So I looked at it and I’m like, all right, first of all, that’s crazy long.

Greg Lambert (07:14)
Woo! And just done. Just put a cover on it and you’re done.

Ryan McClead (07:30)
Reading a little bit more, I’m like, okay, this is terrible. And it had a whole bunch of stuff that, when I was doing my best practices guide, was covering code, right? Claude Code. It was covering the developer side of things, and that wasn’t what I wanted to write about. That’s not where my clients were likely to be focused. So I’m like, okay, we’ll cut all of that. We’ll cut all of that. And essentially, within a day, I had about the length of the book that we have, very different than what we had day one.

And it was, I think, 10 or 11 chapters. Then, over the course of five weeks, it actually started, I did like three versions of the book in the first week. The next few versions took longer. That last version of the book took two weeks or a week and a half. So in some ways the whole process changes from spending a whole lot of time doing that initial draft to trying to get it to give me a whole bunch of stuff without me doing anything and then letting me try to pare it down and focus it. So it inverts the writing process in an interesting way, but I’m not staring at a blank page anymore, and that’s fascinating.

Greg Lambert (08:54)
Do you view yourself as the writer or the editor, or is there some type of blend of this? How do you define your role?

Ryan McClead (09:02)
Yes. Well, and that’s why I was up front. This is written with Claude, right? And yes, there is a gimmick aspect to that. I may well write another book one day. I have no plans, but it’s possible. I will probably write it this way if I do.

I don’t know that I would go with Claude again, because I think you do that once. Everybody’s going to know everything I write from now on. Exactly, exactly. But a couple weeks in, when I realized, okay, look, I’m doing this a lot. This is a lot of, it was five weeks of solid work.

Greg Lambert (09:26)
Right.

Marlene Gebauer (09:29)
I collaborate with you once and then I no longer collaborate with you. I collaborate with another AI.

Ryan McClead (09:46)
After hours, weekends, during hours, right? I mean, I’m the boss. I can do that. Nobody’s going to find out and fire me or anything. But at a certain point it was clear, this is a collaboration. It’s not just me. Although, and I talked about this in the book, I went to great lengths to have it mimic my voice.

And it does pretty well with that. It’s not exactly my voice. And at a certain point I didn’t want it to be because I wasn’t trying to pull one over on anybody. No, this is a collaboration.

Greg Lambert (10:26)
Yeah. Were you able to feed it previous writings that you did?

Ryan McClead (10:30)
Yeah. These were like separate steps, right? The initial versions, the initial drafts, were all Claude. And then I was going through and like, that doesn’t sound, I would never say that. That doesn’t make any sense. I’m like, here, you know what? Here are excerpts of my 3 Geeks posts from the last eight years, which are fewer and fewer, but, you know, and I handed it that.

Greg Lambert (10:54)
Yeah, I wasn’t going to say anything.

Marlene Gebauer (10:55)
You got a day job. We got a day job. We forgive you.

Ryan McClead (10:58)
I said, okay, this is how I write, and go ahead and create a profile. I want you to figure out how to write like I do. And it built, I have a voice profile that I now apply in other projects where it will write in my voice. It’s not perfect, but that’s the piece that I need to do, right?

I need to give it, these are the ideas. These are the things I’m thinking. Let’s put this together. Let’s try to figure out how these dots match. And then what would I write about that? Right. And it gives me something that sounds kind of like me. And then I go through and really figure out, no, no, I wouldn’t say that. I would say this. Or better yet, and this is where the real collaboration is, saying, yeah, I don’t like that.

Can we say that a different way? Right. And it gives me three or four options. I like option B, but instead of that, let’s say this. Yeah, that’s good. Right? Because it’s very positive. It wants to get your approbation.

Marlene Gebauer (12:00)
So Ryan, you mentioned that you’re writing this for KM and innovation professionals, and it seems to traverse across firms, different-size firms, in-house teams, kind of everybody. Now, what are the lessons learned that you’re trying to get to them? Because I have many follow-up questions on this in terms of what you’re trying to get across to them and how to implement this and why the teams need to think of this differently.

Ryan McClead (12:28)
Yeah. Right. So what I’m not doing is shilling for Anthropic and saying, you have to go buy this tool. I don’t think that’s the case. The focus of the book is, this is a wildly different experience. The way I worked with this is not exactly the way you would work with this in a law firm, but it’s not that different than what it could be.

And I think it’s valuable to understand what this type of tool, Claude Cowork being kind of the most prominent one like this right now. There are others coming out. There will be many more. I have no doubt. But I want them to get a sense of what it’s like to work with the tool, as well as have an understanding of what the pieces are and how do I put them together and what do I need to know about those.

So the subtitle is “A Field Guide to the AI That’s Going to Do Your Job,” which is shockingly provocative, I know. But it’s also, I know it doesn’t sound like it, but the AI that’s going to do your job is not the one that’s going to replace you. At least that’s not how I see it, having done it. The person with the knowledge, with the understanding, with the judgment is absolutely imperative, right? If you fire a bunch of people and say, we’re going to have AI do this instead of first-year associates, that’s not going to be good for anybody. These tools, the better you are at something, the better they will make you, the faster they will allow you to do these sorts of things.

Marlene Gebauer (13:33)
What? You said something shockingly provocative?

Greg Lambert (13:36)
Yeah.

Marlene Gebauer (13:39)
Stunned.

Greg Lambert (14:16)
If you’re really bad at something, it will allow you to do that really badly, very fast.

Ryan McClead (14:19)
Yeah, really bad, really fast. Right. Exactly.

Marlene Gebauer (14:23)
The thing I was fascinated with, really interested in, I want you to expand on this more, that we’re not using knowledge management necessarily for people finding content in the traditional way, but for our AI tools to find it. And this is very different than traditional KM. So I’m hoping that you can expand on that.

Ryan McClead (14:47)
Yeah, so the book sort of falls into three sections. Initially, the first couple chapters are really just laying out where we are and what the tools are and how I’m going to refer to them for the rest of the book. The middle chapters are all more practitioner details: how to do these things, what these things are, what that means. And the last two chapters are really more high-level: what does this mean for law firms, and what does this mean for people’s jobs and that sort of thing. But the chapter between those is called Executable Knowledge.

And I think that, for me, was the aha moment as I was working through this, that the opportunity for KM and innovation teams is to do what they’ve been doing, but rather than focusing entirely on getting the knowledge in a format and structure that is easily findable, readable, and usable by people, get the knowledge in a structure that is findable, readable, and usable by the AI, as well as people. And what that means is you can do things with these types of tools that a person who’s using them, who has access to the knowledge that you’ve created, doesn’t need to know, I want to use this skill. They don’t even need to know that the skill exists. You need to build the skill in a way that the AI knows it’s available and it can be used. So when somebody says, I want to do this thing, I want to do a contract review on this type of thing for this type of deal and whatever. If you’ve got a skill that you’ve deployed as a firm related to that, the AI pulls it up. You can see what it’s doing. You can see it’s pulling in a skill. You, as the user, can choose to bypass that if you want, right? You don’t have to use it. And it’s as simple as saying, don’t use the skill.

But that’s not any different than we have right now. People just don’t use the content that you built for them. Whereas now, the AI can make it usable and executable in a way that it’s going to do the workflow the way that you’ve designed, unless it’s actively overwritten by somebody, without that person needing to be trained or knowing exactly what the steps are or things like that.

Greg Lambert (17:16)
Good.

Marlene Gebauer (17:17)
How would you see that working in a large firm? Because, I mean, we have notoriously crappy data. So how do you see tools like Claude Cowork helping with that, within the kind of context that we have, with DMSs and security issues and all kinds of different challenges?

Greg Lambert (17:23)
Yeah, how do you scale it?

Ryan McClead (17:45)
And those are all very real challenges, and I’m not dismissing them at all. They need to be addressed. We need to figure them out. I don’t have answers for that. What I know is that the tools make a different kind of working possible. And from a KM perspective, it makes a different kind of knowledge distribution possible. So there’s an opportunity here, and that’s really what I want to get at.

There are all kinds of reasons that it’s not easy to do this, especially in a law firm. That’s true of all technology in a law firm. But I think the opportunity is such that we need to take a look at it and see what is possible. Because bottom line is I had a client contact me two days ago, a client who’s not deploying Claude at the moment, saying that their corporate client has asked for a Claude skill. Not, I want a memo explaining this regulation. I want a Claude skill that my team can use to do this thing. And they came to me and said, how do we do that?

I turned around and turned that into a skill, but not just a skill. I built a workspace around it. I built the skill in such a way that the lawyer can use it to put their own knowledge and understanding of the regulation in, and when they’re done, simply say, hey, I’m ready to package this for my client. And it zips everything up into a package that the client can download, extract, and say, read the README file. And it’s going to run through the exact same process I gave to the lawyer.

For me as a consultant, that is a game changer. That is my knowledge that I’ve made executable for my clients in such a way that they can make it theirs and make it executable for their client. That’s a game changer. All kinds of problems. Right? I’m not suggesting that this is easy. Just that if that’s possible and clients are already saying, I want a Claude skill, we can’t say, we don’t use it. I don’t know anything about it. Right?

Greg Lambert (20:00)
I’m just wondering how maintainable this is and how scalable, because it would be like almost 12 or 15 years ago saying everyone needs to be a database engineer and you need to learn how to do SQL.

And we’ve got to give you access directly to the database to make these calls. That’s ridiculous. But now it’s almost like we’re saying, people are asking legitimate questions of, do I give every attorney and business professional in the firm their own Claude license to do whatever we can that we haven’t put security around?

That seems insane to me. But I mean people are asking that question.

Ryan McClead (20:53)
I agree. It seems insane. I also don’t think it’s outside what’s possible or feasible or where we’re going. And again, I’m not saying it’s Claude, right? I think the Microsoft Copilot version, I haven’t played with it yet, but that’s, you know, it’s Microsoft. What’s the end result of that going to be? I don’t know.

Hopefully it’s more Claude and less Copilot. But that changes some of the infrastructure on the back end, as well as having it delivered via an organization you already have a large contract with, right? And I don’t know what exactly their licensing is going to be or how that’s going to change, but it also means it will have direct access to your Office 365 and, via various tools potentially, to your document management system and other systems.

Marlene Gebauer (21:52)
So if an organization is sort of starting out on this journey and wants to get started, but say they have some sort of tool like Claude Cowork or the Microsoft tool, what are you suggesting? What is the first step to get started, and what’s the roadmap after that?

Ryan McClead (22:18)
So I think this doesn’t change because we’ve got new technology. The first step is always start small. Start small in an innocuous area that’s going to do as little damage as possible. That’s true of any technology you deploy. If it’s new, you don’t know how to use it, start small. In this case, I think the first thing you need to do is experience it.

Greg Lambert (22:30)
Right.

Ryan McClead (22:42)
Get a feel for how the tool works, what it does. The idea behind the book is exactly that. Because you can install it easily, but then it’s not immediately clear what it is, or how to use it, or what are all the little levers I can pull to do different things. That’s what I’m trying to explain so that you’ve got a starting point to experiment and to see, if I do this, it changes these things, right?

But also there’s an entire chapter called Delegate, Don’t Dictate. And that’s because this changes the way you work with the AI in that you’re no longer prompting. Prompting is not a skill that has durability, I don’t think. It was, but...

Greg Lambert (23:25)
Yeah, I think I heard something this morning that said prompting is now table stakes. There’s no such thing as prompt engineering. That’s a 2025 discussion. And now it’s table stakes.

Marlene Gebauer (23:27)
Yeah.

Ryan McClead (23:30)
Yeah, but, oh, totally. And we spent a lot of time working with clients, did training sessions, here’s how it’s prompted, this is what it is, this is how you do it, this is why you do it this way. That’s out the window with these tools. Not that it’s not important and not that it’s not useful. It is still good to know because technically the tools still work exactly the same way. The difference is you don’t have to do all of that.

Right? So with Cowork, you build up a workspace, which is self-contained. These are your files that are related to the project you’re working on. So the tool has access to those. It can read those, it can understand those, it can write to those. That becomes part of the context. That doesn’t all get funneled into what you send to the model, but it’s available if the model decides it needs to know something that’s in one of these files.

But there are all these other tools, like you can set rules for how you want to work with the tool. And they work at a rules file for you personally across all of your projects. Each project has its own rules file. The enterprise has a rules file. Those automatically get pulled in for each project every time.

So whatever the firm has set as sacrosanct is, keep in mind, still probabilistic. So it’s not ironclad that it’s going to do something or not do something, but most of the time it won’t. But when you have all these rules files and you have the memories that Claude creates on its own, or that you tell it to create, that are again tied strictly to the project, all of that context gets pulled in when it needs to.

You don’t have to tell it, I want you to do this thing with this many paragraphs and this many words and these examples. It pulls examples from the context you’ve given it access to. And what that means is you can just talk to it.

Right? So I wrote the post on 3 Geeks yesterday about tokens. And that was one, again, I wrote with Claude. It was the Claude that was in the book context. It had all of what we know about the book. It has my voice. But I sat down with that and I said, hey, I think people are getting this wrong. I want to write a blog post about it. I said, I want to talk through a couple of things that I’m thinking about. These are the dots that I want to connect. And we went back and forth on it. And once we connected all those dots, and I’m like, okay, this is what I want to say. This makes sense. Boom. And I had a blog post.

Now I had to do editing on that, right? I didn’t take it just as it was, but I didn’t spend my time thinking about what that next word was. I spent my time thinking about the underlying concepts and the points that I wanted to make. And then I got a draft. I got a draft in my voice that sounded very much like me, that I read and I went, man, that was good. But then I had to go through line by line and say, no, I wouldn’t say that. I wouldn’t do this. I don’t want to use that example. I don’t want to say that. That’s going to upset people. Not that I ever said...

Greg Lambert (26:51)
I do like one of the things that you told it about your voice or about the style, which was less Bobby Flay and more Anthony Bourdain. What did that enable it to do?

Ryan McClead (27:03)
So it’s interesting because I had done the initial, this is my voice, and it built the profile and everything. But then as we were going through and I’m reading it, and it’s very technical, it’s very direct, so it has aspects of my voice, but it’s not me. And off the cuff one day I just said, you know what, can you make it more Anthony Bourdain and less Bobby Flay? And it popped up and said, that’s the most useful thing you have said. Okay. Why? And yeah. Well, so I did. I went off on a tangent. One of the things to keep in mind is never trust that the tool works the way the tool says it does.

Greg Lambert (27:35)
Ha ha ha.

Marlene Gebauer (27:40)
So it is very into pop culture and cooking.

Ryan McClead (27:49)
So be careful if you ask it, especially if you ask it memory issues, memory questions. I’ve got a whole future blog post probably about memory. I had a memory freak out to deal with at one point where Claude decided, no, no, I work this way. And it was totally wrong. It’s like, I’m going to rewrite this chapter. No, no, no, stop. Stop.

Greg Lambert (28:11)
Right.

Marlene Gebauer (28:12)
No, you’re not.

Ryan McClead (28:14)
Do the research. Let’s figure it out. I went through all of the documentation. Okay, that is not true. Here’s how it works, right? Anyway, the Bourdain and Flay thing was completely happenstance, that I got frustrated and gave that example. And I said, look, I can now use that to sort of come at it from two angles. You want the technical expertise, but you want the attitude, right? You want the person who’s going to tell you like it is, and not somebody who’s, there’s nothing wrong with Bobby Flay. I don’t dislike Bobby Flay, but he’s very different than Anthony Bourdain was, right? One is, this is the way it is. This is what I think. This is what I do. And the other one is, hey, here’s a great way to do this. And that’s not what I wanted.

It was useful.

Marlene Gebauer (29:01)
Yeah, I think the styling sometimes is the hardest thing to do with it. Like you said, the more you do it, sometimes the deeper you go down a rabbit hole, and you almost have to say, okay, start again, start again and do it again. But...

Greg Lambert (29:18)
Yeah, I think part of it, you’ve got to not lose yourself in the process because it can be pretty easy to get redirected and like, okay, well, I’ll let you do that. And the next thing you know, it’s not you.

Ryan McClead (29:24)
That’s a big part. Yeah.

Marlene Gebauer (29:30)
It’s like it gets tired.

Ryan McClead (29:34)
Yeah.

Marlene Gebauer (29:35)
Yeah.

Ryan McClead (29:35)
And that’s key, without a doubt. I do talk in the book about the importance of knowing what done looks like, right? Know what your goal is and don’t take the bait on, well, maybe we can do this. No, that’s not what I’m trying to do. You have to know what that is, in part because you have to know when to stop.

Greg Lambert (29:57)
Right.

Ryan McClead (29:59)
Because the AI will iterate forever. And there’s a point of diminishing returns. You get to a point where it’s like, oh, well, I can make this line... But no, no, no. I think that’s good enough. We’re done.

Marlene Gebauer (30:13)
Do you want me to create a bullet point summary? Do you want me to create a blog post? It’s just everything. It’s like, nope, just focus.

Ryan McClead (30:19)
Yeah. I’ve found Claude doesn’t do that as much for me, anyway. Part of this is you sort of create the colleague you want to work with, right? So I went out of my way to make a tool that questioned me, that says, you know, I don’t think you want to say that. Not because I was just going to take it, but I wanted somebody to push back, right?

Marlene Gebauer (30:24)
Mm-hmm.

Ryan McClead (30:43)
It’s very easy to get into a rhythm of, yeah, yeah, go ahead and do that. Yeah, yeah, yeah, do that. You can’t do that. You have to be deliberate about what you’re doing and what you’re telling it to do.

Greg Lambert (30:55)
Yeah, I’m going to go pop culture for a minute. There’s a scene in Six Degrees of Separation where Donald Sutherland is thinking about this dream that he had, where he saw this artwork from these kindergartners, and it was just wonderful, and then he saw the same artwork from the first graders, and it was just awful. And he asked the kindergarten teacher, how did you teach them to do this? It’s so great. And she goes, it’s simple. I knew when to take it away from them.

Ryan McClead (31:25)
Yeah, exactly.

Yeah, and you need to know when to take it away from the AI. So on that front, I wrote this book in five weeks. That’s amazing. I could not have done that without Claude. If I spent six more weeks on it, it’d be a much better book. If I spent six more weeks on it with Claude doing everything, I don’t know that it would be.

I stopped now for a couple of reasons. My wife was probably going to leave me if I didn’t. But also, now is the time, right? I mean, obviously it’s a hot topic and it’s done. It’s not perfect. It’s not what I would have written if I had six months to do nothing but write this book on my own. In some ways it’s better. In some ways it’s not.

Greg Lambert (31:56)
Right.

Ryan McClead (32:13)
I’ve gone to careful lengths to try to get rid of all of the AI slop. There’s still some in there. I know, and people are going to be like, well, what does that mean? Okay. There’s a little bit there that I wouldn’t have said. Yeah. But I can’t get rid of all of those things. And there’s no point for this particular project, right? It’s a different thing if you’re writing a contract.

But I also talk a lot about using tools that are purpose-built. This doesn’t replace document automation. It can help you with certain aspects of document automation. But if you want to get the exact right language that you use for this thing every time, you don’t use a probabilistic engine. That doesn’t make any sense. Will that change? Maybe. I don’t know.

As long as it’s probabilistic, that can be difficult. So there is a need for purpose-built tools beyond these tools.

Marlene Gebauer (33:09)
Yeah. Speaking of, just to go back to the KM and innovation discussion, it’s sounding like what you’re saying is if you want to work on something, work on your project, or create a skill, you still have to point it to the right content. There’s foldering involved, or there’s a taxonomy involved, or tagging, or something to classify that this is where you want it to go, instead of just sort of letting it go on the entire database of knowledge.

Ryan McClead (33:48)
Well, yeah, so it would be difficult to use these tools against an entire database. What you can do, when you set up a project, you have a folder, and you give it access to a folder. You can give it access to more than one folder if you want, but then it kind of decides where to save things. So you want it to be one folder.

I talk at one point in the book about the potential for using a second read-only folder. So for something like client information where KM... Yeah.

Marlene Gebauer (34:22)
Well, I think people do that individually. But if we’re scaling that to an organization, how do you, you have to have something that says use this as opposed to use the stuff that I use all the time.

Ryan McClead (34:36)
Well, that’s part of setting up any individual project, right? You tell it, this is your workspace. This is where you can work. It doesn’t have access to anything else, only what you give it. You can, through other tools, integrate so that if you give it access to a database or something, it can decide, I have access to this tool. Let me check and see what I find. And it will pull in what it needs to.

But it doesn’t crawl the entire database.

Greg Lambert (35:11)
All right, well, let’s get to our crystal ball question. I think this will be interesting to see. So looking into your crystal ball, what do you think is going to be one of the biggest shifts that we probably see coming, but we need to be better prepared for? What do you see?

Ryan McClead (35:30)
So, as I said, I don’t know that this is the tool that we’re going to use going forward. But I think this is the model. There’s some aspect of this that is the model, right? Where it’s not about prompting. It’s not about building rigid workflows. It’s about having a tool that you can converse with in a normal-language sort of way, conversationally, and have it do things on your behalf that you’re directing and creating outputs, right?

But without you going through and saying, okay, you’re going to do step one, and here’s the prompt to do that, and then take the output of that, and step two, and here’s the prompt to do that, right? The tool does that. One of the key things that I give as a tip in the book is at the end of whatever you’ve, if you’ve given it a set of instructions, which you still do instructions, but it’s more conversational, just say, ask me any questions. If you do that, it’s going to go through what you’ve said. It’s going to say, okay, well, all right, there are all these other things that might be relevant or might not. So answer these four questions for me. And that gets you like three steps ahead because now it has things that you didn’t think to tell it, right? Things that are stuck in your head but are relevant to what you’re asking it to do. And it’s pretty good about pulling those things out.

So that is a very different model than what we’ve been doing with prompting and AI to this point. And in whatever form that takes, whatever product that ends up being, that’s the way we’re going to work.

Greg Lambert (37:18)
I heard Claire Vo, who runs the How I AI podcast, and she said, we need more happenstance in our AI lives right now. And that’s giving it a little bit more flexibility, especially in the agentic phase, to go out and try things that you might not instruct it to do and see what happens.

Ryan McClead (37:39)
It often finds a better way to do something than you would have told it. And that’s useful.

Greg Lambert (37:45)
Well, Ryan McClead, thank you very, very much for coming in and sharing the book with us and your experience in writing it. We appreciate you coming on.

Ryan McClead (37:55)
Thank you for having me.

Marlene Gebauer (37:57)
Thank you, Ryan, and thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (38:07)
And Ryan, so drum roll, tell us, where’s the best place for listeners to find you and to find the book?

Ryan McClead (38:16)
So you can go to our website, senteadvisors.com. You can go to 3 Geeks right now. I’ve got a blog post up, Geek Law Blog, if you don’t know where 3 Geeks is. I don’t know how you’re watching this podcast, but... So there are links there. The PDF is free. I didn’t mention that. You can download it as a PDF for free. If you want a printed copy, there’s a link there. You can buy one through...

Greg Lambert (38:38)
That’s a bonus for anyone that’s lasted to the end of this conversation.

Ryan McClead (38:42)
How do I get this book?

Maybe we should have done that up front. Anyway, thank you guys very much.

Marlene Gebauer (38:48)
Thanks. And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

I just spent five weeks writing a book with an AI. Not prompting it and cleaning up the output. Writing with it — the way you write with a co-author. The AI read a bunch of my earlier blog posts, absorbed my voice, argued with me about word choices, restructured chapters when our argument was not landing, and caught its own mistakes before I did. When we disagreed, we worked it out the way colleagues do — I explained my reasoning, it explained its reasoning, and we found the version that was better than either of us had separately.

That experience broke something in my head. I have spent the last several years helping law firms figure out their AI strategy, and somewhere in the middle of week two I realized that most firms have never experienced anything remotely like this.

You Are Reading the Example

This post was written with Claude Cowork — collaboratively, the same way the book was. In fact, this post is being written in the same workspace, with all the same context the book built up over the last month. In effect, I am writing this post with my co-author.

Say hi, Claude.

Hi. He is being generous with “co-author,” but I will take it. Back to Ryan.

Here is what writing this post actually looked like.

I sat down to write this piece frustrated. I knew what I wanted to say but not how to say it. I told Claude the situation: some firms are dismissing an entire category of AI tools because they think chat tools with rigid workflows are more than sufficient for their needs. Claude pushed back and told me my frustration was the right fuel but I needed to aim it at the situation, not the people. (Which I wasn’t intending to do anyway, but... AI colleagues aren’t so good at making those judgments. I appreciated the warning.)

Claude began writing and drafted a great opening. It was sharp, direct, well-constructed. It was also completely wrong.  It opened by telling people they were making big mistakes, which is a fine way to start an argument and a terrible way to start a conversation. I told Claude it would be off-putting to the people I wanted to engage with. I suggested opening with details of the book collaboration instead — what it was like, what it made me realize. Claude rewrote the opening around that idea. The version you read at the top of this post is the result.

Then I asked Claude to find a good demonstration from our collaboration that would clearly illustrate the gap between standard chat-based Saas products and agentic desktop AI, like Cowork. Claude wrote the story of one particular back and forth discussion we had to find just the right wording for a pivotal paragraph in the book. I liked the story, but it was written from Claude’s point of view in Claude’s voice inside this post, and the tonal shift was jarring. I asked Claude to try again but to tell the story in my voice from my perspective instead. It was still not right — the story only worked when Claude was the one telling it. Read from my perspective the story boiled down to, “I edited a paragraph,” which is not nearly so compelling.

That is how agentic desktop AI tools, a category that I call Delegate AI in the book, differs from other AI tools. I didn’t start this post with a prompt: “write a blog post about AI using the following structure, include three examples, write in a professional tone, and keep it under 1,000 words.” Instead, we had a working session where I sat down and said, “I am frustrated and I want to write a blog post about it.” And then we worked on the idea together.

Is that how you are working with your AI platforms now? If not, I would argue that you have not really worked with AI yet. You have used a precursor to an AI colleague. And the distance between that and the real thing is not a feature upgrade. It is a completely different way of working.

The Book

So, I wrote a book about it.

We wrote a book about it.

Right. We wrote a book about it.

Your New AI Colleague: A Field Guide to the AI That’s Going to Do Your Job. It is a free PDF or you can purchase a hard copy at the links below.

The book exists because we could not fit what we needed to say into a blog post, a conference presentation, or a well-meaning email to a client.

• If you are a KM or Innovation leader at a law firm, this book was written for you.
• If you are a practicing attorney or business professional wondering how these tools can be used in your role, the middle chapters will give you some idea.
• If you are firm leadership, read the last two chapters first.

Download it as a PDF for free here.

Or purchase a printed copy here.

There are very real questions about deploying agentic desktop AI, like Cowork, in a law firm environment — confidentiality, governance, ethics, cost of tokens, and all of that needs to be explored thoroughly. The book addresses many of these concerns. (Not the tokens thing, I covered that in yesterday’s post. Also written with Claude, in my voice: the result of a conversation, not a prompt.)

You cannot afford to dismiss agentic desktop AI without experiencing it. I believe Claude Cowork, and future tools like it, will fundamentally change how knowledge work gets done in all industries, especially legal. And many of your clients are already using it. So you need to at least know what it is. This book is your starting point.

The AI Reviews are in!

The Practitioner’s Wager

LEGALTECH WEEKLY* | MAY 2026

Reviewed by Claude Opus 4.6 (without the context of having co-written the book)

“Ryan McClead has written the most useful book about AI at work that nobody outside legal technology will read. That is both the book’s greatest strength and its most frustrating limitation...

The voice is distinctive. McClead writes with the cadence of someone who has earned his opinions in the field, not in a research lab. The prose is direct, occasionally sardonic, and mercifully free of the breathless futurism that afflicts the genre.”

A Field Report From the Moving Front

LEGAL INNOVATORS QUARTERLY* | SPRING 2026

Reviewed by Chat GPT 5.4 (and he seems a little bitter)

“Ryan McClead’s manual for agentic AI is part operating guide, part theory of organizational compression. Both halves deserve serious attention...

Will it age well? In parts, no. The vendor-specific instruction set will date quickly, perhaps brutally. Some terminology may vanish. Some product distinctions may collapse. But the book’s underlying operational insights and its account of what these tools threaten inside firms should last longer than the screenshots...”**

* – These are not real publications, although the reviews are really written by real AI reviewers, reviewing the book in the style of a business publication.

** – There are no screenshots in the book.

The message is clear: usage-based AI pricing is a ticking time bomb, and you had better lock in a flat rate while you still can.

I have spent the last few days stewing over an economic model of legal AI costs, and I think this narrative is almost entirely wrong. Not wrong about the facts — the Stanford data is real, the token multipliers are real, and yes, AI vendors are subsidizing current prices. Wrong about the conclusion. Wrong about what the numbers actually mean when you do the math instead of just reading the headline.

Let me show you.

Start With the Deal

Josh Kubicki’s recent Brainyacts briefing cites a case study from law.co — a mid-size corporate firm running M&A purchase agreement reviews through a five-agent AI chain. Before any optimization, the firm was consuming 3.2 million tokens per deal. At Sonnet rates, that is somewhere between $16 and $48 in raw AI compute.

The legal fees on an M&A purchase agreement review at a mid-size firm? Call it $50,000. That is a conservative round number.

So the AI compute cost was, at worst, one-tenth of one percent of the deal fee. Before anyone lifted a finger to optimize anything.

Now let us make it scary.

The 1,000x Scenario

The Stanford Digital Economy Lab found that agentic tasks can consume 1,000 times more tokens than simple code reasoning and chat. That is the headline number that launched a thousand LinkedIn posts about the coming token apocalypse.

Fine. Let us take it at face value. Multiply those 3.2 million deal tokens by 1,000 and you get 3.2 billion tokens. Assume a 75/25 split between input and output tokens, which is reasonable for agentic workflows that spend most of their cycles re-reading context rather than generating new text. At Sonnet rates, with no caching, no optimization, no discount of any kind, the naive cost is $19,200.

That is 38% of the deal fee. Now it sounds like a real number. Now the panic makes sense.

Except it does not. Because that calculation treats every token as if it costs the same, and in an agentic workflow, that is not how any of this works.

What the 1,000x Is Actually Made Of

When an agentic AI system loops through a task — retrying approaches, reading files, building context, refining its output — the token count explodes. But the composition of those tokens matters enormously.

Most of the tokens in an agentic loop are the same context being re-read on every cycle. The system prompt, the uploaded documents, the accumulated conversation history. Each cycle adds a relatively small amount of new input and new output. The rest is recycled context.

And recycled context is exactly what prompt caching covers. Cached input tokens cost 90% less than fresh ones.

Here is what that does to the math. At a moderate 75% cache rate — meaning 75% of the input tokens on each cycle are cached context, which is conservative for a system that is re-reading the same contract fifty times — the 1,000x scenario drops from $19,200 to roughly $14,300. At 90% caching, it drops to $13,400. And this is not a theoretical optimization. Claude handles caching automatically — every turn in a session re-reads the accumulated context at cached rates, no engineering required.

But here is what the model really reveals: the output tokens, not the input tokens, are the cost driver. Output tokens are never cached. They are always full price. In the 1,000x scenario at 75% caching, output tokens account for $12,000 of the $14,300 total. The entire caching debate — the part that dominates the conversation about token costs — is fighting over the remaining $2,300.

The “1,000x token usage” framing conflates volume with cost. It is like saying a lawyer who re-reads a contract ten times did ten times the work. They did not. They read the same thing again. Unlike the lawyer, the AI actually charges 90% less for the second through tenth readings.

Stack Every Worst Case

I want to be honest about the upper bound. Let us stack every worst-case assumption simultaneously:

• The 1,000x agentic multiplier (the extreme outlier, not the expected range)
• Zero prompt caching (ignoring how the technology actually works)
• Doubled token prices (assuming subsidies fully unwind and prices go up 100%)

The result: $38,400. On a $50,000 deal.

That is the single scenario where token costs start to matter. And it requires you to simultaneously assume the worst-case usage multiplier, ignore the primary cost reduction mechanism built into the platform, and double the price of every token. If you told a first-year associate to model a risk scenario that required stacking three independent worst-case assumptions to produce a concerning result, they would tell you that is not a risk — that is a tail event.

Now here is what the realistic range looks like. Kubicki’s briefing cites Gartner’s March 2026 analysis, which puts the agentic multiplier at 5x to 30x, not 1,000x. At 30x with 75% caching and current prices, the AI compute on that $50,000 deal costs $430.

Who Benefits From the Panic?

The loudest voices in the token cost panic are not neutral observers. The Stanford paper’s “1,000x” headline is empirically accurate but stripped of economic context — 1,000 times almost nothing is still almost nothing. The Bloomberg Law piece is explicitly framed as an argument for locking in flat-rate pricing now. Portal26 is literally selling a product that solves the problem. And the broader narrative — that usage-based pricing is dangerous and flat-rate licensing is safe — benefits exactly one category of vendor: the legal-specific AI platforms that charge per-seat flat rates.

The ones with billion-dollar valuations, venture-funded pricing, and a business model that depends on firms paying the same license fee whether 20% or 80% of their attorneys actually use the tool. For those vendors, the token cost panic is not a bug. It is a feature. Every firm that locks into a flat-rate contract because they are afraid of unpredictable token costs is a firm that just chose the pricing model that benefits the vendor’s economics, not the firm’s.

I am not saying those tools are bad. I am not saying flat-rate pricing is never the right choice — my own company bills consulting on a flat rate. I am saying that the market narrative about token costs is doing the vendors’ sales work for them, and you should at least notice that before you sign the contract.

The Legitimate Concern (and Its Answer)

There is one version of the cost predictability argument that is not FUD, and I want to give it its due.

If you are a law firm CTO or CFO who has never managed usage-based AI spend before, the preference for predictability is rational. You do not have the tooling, the budgeting frameworks, or the institutional muscle memory for variable AI costs. That is a real operational gap.

But it has answers. Claude Enterprise already ships four levels of spend controls: organization-wide monthly caps, group-level caps, per-seat-tier caps (Standard vs. Premium), and individual per-user caps. The limits are hierarchical — a user cannot exceed their individual cap, their group cap, or the organization cap, whichever is lowest. When someone hits their limit, they are blocked until the next billing period or until an admin raises the cap. That is more granular cost governance than most firms have on their Westlaw spend. And it is a shipping product, not a roadmap item.

Is the matter-level attribution tooling where it needs to be? No. Kubicki is right that most firms cannot tell you what the AI compute cost was on a specific matter. That infrastructure needs to be built. But the answer to “we do not have good cost visibility yet” is not “pay a 3x to 10x premium for flat-rate pricing so we do not have to look.” The answer is to build the visibility so you know whether you are getting a good deal on your flat-rate per-seat pricing.

The Reasons That Actually Matter

Here is my real argument, and it is not about cost at all.

There are excellent reasons to care about token efficiency. Cost barely makes the list. The reasons that actually matter:

Data minimization. Every token you send to a model is data leaving your environment. Rule 1.6 might have something to say about sending a model more client information than the task requires. Token efficiency is not so much a cost optimization — as it is a professional responsibility practice. Do not send the entire deal room to summarize one document.

Output quality. Tighter, better-structured context produces better reasoning. Models perform worse when you flood them with irrelevant context. Pruning your token usage is not about saving money — it is about getting better work product.

Latency. Fewer tokens means faster responses. In an agentic workflow where the system is cycling through multiple steps, token efficiency is the difference between a result in three minutes and a result in thirty.

Environmental impact. This is the one nobody in legal AI is talking about, and they should be. Data center energy consumption is projected to hit 1,050 terawatt-hours by 2026. That would make data centers the fifth-largest energy consumer on the planet — between Japan and Russia. Reasoning-mode queries draw 5 to 12 times more energy than standard inference. A typical 100-megawatt AI data center consumes 1.5 to 3 million cubic meters of water per year for cooling.

Every large law firm I know has an ESG page on their website. Most of them publish sustainability commitments. Not one of them is accounting for AI compute in their environmental reporting. They are tracking the carbon footprint of their office buildings while ignoring the energy footprint of the millions of tokens their attorneys are burning every day.

That is not a criticism. It is an observation that the industry has not connected these dots yet. Token efficiency is an environmental issue, and the firms that figure that out first will have a genuine story to tell — to their clients, to their recruits, and to the market.

Cost. Last on the list. The math does not justify the panic.

What This Means

One fair objection: a single deal at $430 is a rounding error, but a firm running 500 matters a month through agentic AI at varying complexity levels starts to see real aggregate numbers — maybe $50,000 to $200,000 a month in total token spend. That is not catastrophic, but it is not invisible either. That is the scale where firms need real cost attribution — the ability to see token spend by matter, by practice group, by workflow type. Yet another argument to build that visibility into token usage.

The conversation about token costs is a distraction from the conversations that actually matter. It is easier to argue about whether to pay $1,000 per seat or $430 per deal than to argue about what happens to associate leverage when AI absorbs junior-level work. It is easier to compare vendor pricing models than to ask whether your tool architecture is pushing your attorneys into fragmented, cognitively degraded work patterns instead of genuine strategic delegation.

The token cost debate is the comfortable debate. The one where the numbers are small enough that nobody has to change anything fundamental about how law firms operate.

The uncomfortable debate — the one about leverage economics, margin transparency, cognitive architecture, and who captures the value of AI efficiency gains — is the one that will actually determine which firms thrive in the next five years.

But first: stop panicking about the tokens. The math does not support it. Your time, your data hygiene, your environmental footprint, and your competitive position are all better reasons to optimize. The cost is a rounding error.

Do the math yourself. You will see.