In this episode of The Geek in Review, Greg Lambert hosts a solo conversation with Triona Buckley, Chief Product Officer at Actionstep, about generative AI’s growing influence on mid-market law firms. Buckley challenges a common assumption about legal AI: faster task completion does not always remove friction. An associate might produce a draft within seconds, only to transfer the burden upstream to a senior lawyer responsible for reviewing sources, reconstructing reasoning, and correcting mistakes.

Buckley argues law firms should shift their attention from speed to systems. Standalone drafting and research tools address individual tasks, while system-level AI connects work across an entire legal matter. Embedded within everyday workflows, AI helps lawyers locate information, reduce administrative work, and preserve more time for client advice and professional judgment. The goal is a smoother operating model, rather than a collection of isolated tools producing faster documents.

The conversation also examines institutional knowledge, especially within firms lacking large knowledge management or innovation teams. Buckley describes an approach where AI captures decisions, context, and reasoning as lawyers work. This creates a continuously expanding record of how the firm handles matters, advises clients, and applies professional judgment. Governance still plays a central role, including clear audit trails showing whether a person or an AI agent performed each action.

Greg and Triona then explore AI as an individual tutor for junior lawyers. Remote and hybrid work have weakened the traditional apprenticeship model built around observation and informal office conversations. Drawing upon decades of firm experience, an AI tutor might question an associate’s assumptions, prompt additional research, and reinforce the firm’s preferred methods. Such systems offer structured practice while preserving the essential mentoring relationship between senior and junior lawyers.

Another major theme is the hidden cost of delayed time entry. Actionstep’s Trace passive time capture technology monitors work across practice management, email, and document applications, then presents lawyers with matter-linked, billing-ready entries. More accurate records help firms recover otherwise forgotten time while producing better data for pricing, staffing, client estimates, and profitability analysis. Those insights grow more important as clients push firms toward fixed fees and output-based pricing.

Buckley believes mid-market law firms hold several advantages during the AI transition. They often operate with fewer systems, maintain closer client relationships, and move through organizational change faster than larger enterprises. Success will still require disciplined implementation, trusted internal champions, connected data, and sustained attention to client service. Her message is optimistic but direct: firms with strong relationships, clean data, and a clear economic strategy will be better prepared for agentic AI and the changing business of law.

Actionstep’s U.S. Midsize Law Firm Priorities Report

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Stephanie Wilkins (00:00)
The Gen AI conversation has been advancing faster than a lot of people can keep up lately, but some new developments have brought older concepts like prompting back into the spotlight thanks to a trending new topic, token cost. Token cost moved into the spotlight recently as tools like Claude gained traction in legal, because most plans come with token limits, as well as the ability to request limit increases, which has resulted in tales of astronomical bills for some users. Recently, Legora also announced that it’s moving its Agent Pro offering

Consumption based pricing, which means it’s billing by what the agent does rather than by a flat seat license. And what agents do is consume tokens. Eventually, other providers are sure to follow suit. What many don’t fully understand is just how quickly token usage can add up. A few extra follow-up questions, a document pasted in twice, a chat continuing long after it should have been reset. If that sounds familiar, token consumption compounds faster than you might expect, and you may be looking at a higher token usage than you think.

And you might not even realize it until you’ve hit your usage limit or worse, seen the bill. This is a blind spot we’ve been unpacking in one of our latest article series on Legal Tech Hub: how to get more out of tools like Claude without burning time, decreasing accuracy, or racking up unnecessary bills. We’ve covered topics like what tokens actually are and why they function as a hidden meter running behind every chat. When to reset a conversation versus continue in the same chat.

And what everyday prompting habits, from repasting whole documents to burying five questions in one prompt, might be driving up both cost and inaccuracy without you knowing it? Head over to legaltechnologyhub.com to read the full series and learn more about how to get the most out of your token limits and your usage of tools like Claude.

Greg Lambert (01:45)
Welcome to the Geek and Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Greg Lambert, and Marlene is out west climbing a mountain or maybe enjoying a cocktail somewhere right now. So everyone is stuck with me this week. today we are exploring why AI isn’t actually eliminating bottlenecks in law firms, but rather just moving that bottleneck upstream and how mid-market firms

Can leverage their institutional knowledge to win the AI transition. So I’d like to welcome Triona Triona Buckley, the chief product officer at Action Step, with more than two decades in the legal and legal tech industries. Triona has spent years working inside law firms here in the US, in Ireland, and the UK, and now leads the product vision at Action Step.

the leading cloud-based law firm management platform for mid-sized firms supporting over fifty five hundred law firms globally. And Triona is joining us I think from the future because I think it’s already tomorrow there in beautiful Auckland, New Zealand. So Triona, welcome to the Geek and Review.

Triona Buckley (02:55)
Thanks, Greg. Great to be here.

Greg Lambert (02:58)
All right, I I threw this one in at the last minute ’cause I realized we weren’t giving you a chance to really talk about action steps. So you mind giving us just kind of the the elevator pitch on on action step and what you do there?

Triona Buckley (03:11)
Sure.

Action Step is a law firm’s operations platform. So it combines practice management and legal accounting in one platform. So really, I suppose, designed to connect all the work of everyone that works at law firms. We are a global business, so we have a global team, a global customer base, very much focused on the mid market. So that’s our sweet spot, it’s where most of our customers sit, and so we’re really trying to support the unique challenges and our

opportunities that mid-market firms have. Yeah, and I’ve I’m CPO at Action Step, I’ve been with Action Step for many years and work very closely with our customers on a day-to-day basis to make sure that our product is is answering their needs.

Greg Lambert (03:55)
Awesome. Thank thank you very much for taking the time to give us a little background on that.

Greg Lambert (03:58)
So Trina, your your core argument regarding generative AI isn’t that it’s necessarily eliminating the friction in law firms, but it’s it’s kind of taking that friction and and shifting that bottleneck upstream. So, you know, for example, if if a junior associate you know takes a few seconds to draft a document, now all of a sudden the the senior partner is is having to be inundated with these and then kind of

reverse engineer how how it was put together. So in your view, how do you how how do firms structurally fix this workflow so that that AI doesn’t actually in introduce more problems than it fixes and and what do you mean when you say something like we need to shift from speed to systems?

Triona Buckley (04:50)
So what I mean about really shifting from speed to systems is is thinking less about, you know

one task, you know, and delivering one task using one AI tool. And more about, you know, thinking about the whole workflow. You know, how how are you going to use AI across the workflow and across your tasks in a way that feels much more intuitive to the way that you naturally work. So, you know, what we’ve seen is that law firms adopted and continue to use some fantastic tools around documents and research and and those sorts of things. And those are great to

Tools and obviously they came to market first because it was around generative AI and that sort of thing. you know, we now have the opportunity with more agentic AI available to to really connect that into a more natural workflow and into the places that you work anyway. so really it’s about taking, you know, the like I say, individual tasks and connecting them together into the the overall workflow. And I think the way the way that law firms can do that and and think can have apply that system thinking

is really think about the broader and bottlenecks. You know, we started with drafting, but yes, yeah, we’re we’re we’re it shifts it shifts the onus, I suppose, upstream to make sure that those drafts were well done and researched and thoughtful and blah blah blah and you know and and and and cited correctly.

And so those are the more expensive resources being used to do some of some of that work. But really, you know, oftentimes the friction in overall workflows is less to do with those tasks and more to do with, you know, sort of operational friction, administrative work, you know, finding information, not knowing where to look to find the information, you know, and having to dig into all these different silos of information to find what you need. So really addressing those sorts of things, you know, help.

the sort of day-to-day of your fee earners just feel smoother and less fewer things in their way. So that’s really what I you know what I think about when I think about kind of system level AI. It’s really smoothing out the whole the whole the whole workflow so that fee earners get to do what they do best which is really apply judgment to their client work and they get to do it in a way that you know they have room to think etc because that you know administrative friction has been removed I suppose by a

AI, automation, systems, whatever it might be.

Greg Lambert (07:05)
Okay. So w we talked recently, I th I think it was the last episode on

called executable knowledge and and private graphs. And you’ve noted that while AI can draft a contract, you know, AI sometimes is is biased and you know what clients are really paying for during the the high stakes milestones that is is for that lawyer’s kind of unbiased judgment. There and we we talk a lot about judgment here for the for the lawyers. So

On the the mid-market firms that that you work with, how is it that they’re actually capturing the expertise that’s kind of locked in that that partner’s head and put it into so that it’s it’s that systems process that kind of guarantees things like

The context is right, the controls are there, the governance is there to to help support that judgment. How do you how do you pull it all together?

Triona Buckley (08:05)
Yeah, well

Interestingly, what we see at larger law firms is that, you know, they tend to have huge knowledge teams and innovation teams who are sort of their job to a certain extent is to, you know, collate all of that knowledge and bring it together and and extract that sort of thing from people’s heads. I mean, mid-market firms don’t have the luxury usually of of having those sorts of resources. And so if it’s not been captured throughout your day, it’s probably not going to happen. It’s probably not going to be, you know, a a concerted concerted effort, I suppose, for you know, everyone at the firm.

To sort of download what they’ve done that day and the reasons why they might have done things, etc. And so what we what we can see with with and more so with AI, but even with you know automation is if you’re if you’re capturing if you’re capturing you know what people are doing as they move through their day and you know prompting the right questions, asking them why they might have done things just as they as they actually go about their their normal work, it’s much more likely that you’re going to be able to you know to add to that instant.

institutional knowledge and really add that context of that layer of context around what they’re doing and why and so what from an action stamp point of view what we’re really trying to ensure in anything that we bring to market from an AI point of view is that it’s not it’s not interruptive it’s very much you know really just kind of following along as they move through their day and and sort of you know capturing everything. What’s really important there is there’s a governance layer to that obviously you know you have to know who has done what whether that’s an agent or human.

And so, you know, making sure that you’re actually, you know, capturing that that that level of detail on on on what’s been done and when and why and all of that is really, really important from a governance point of view, because you have to be able to look back and see and we all we all we all we all we’ve all seen you know the reasons why if you don’t have that it can be a problem.

But actually there’s also just a you know like it beyond the sort of governance and the safety point of view, it’s really more around you know adding to your institutional knowledge, you know. and I suppose you you want to, you know, if we’re you know, the things that might have been learnt, I suppose, in conversation, you know, need to be also captured within within the system. And so that’s a AI is a great, great use case for that, you know, because it can act like you’re you’re sort of your your sparring partner in understanding where you’re coming from.

Greg Lambert (10:17)
Yeah. I’m I’m curious on how you kind of set that up and monitor it with because I n I know you mentioned that you wanna you wanna do this in a way that doesn’t interrupt the workflow of of the attorneys. So, you know, just kind of peeling back the onion a l a little bit. How how how did you work to set something like that up and and kind of determine what’s w how how it is that you don’t interrupt the workflow?

Triona Buckley (10:47)
Yeah, look, I think a lot of it is again back to sort of, you know, having most things happening in one platform, you know, so that you’re you’re you’re not having to sort of, you know, change how you do things in different places and therefore piece together all of that kind of audit trail. So having that sort of centralized place for all of that information to live is is is thing number one. And so and we see the most mid-sized law firms tend to prefer to have fewer tools rather than more. Some of that is you know budgetary point of view.

of it is just there you know a preference for a firm that size that what that allows actually is that you can have you know with all of that connected data you have you can put like ai on top of that and it it has all of the context you know and so it it doesn’t have to read five different tools you know it actually has the information it needs at its fingertips and so that that’s a big piece of it is just the fact that it’s actually already kind of connected and when in one place it doesn’t have to be one platform

A lot of it is just more to do with how things are interconnected. So that that’s sort of the the underlying, I think, philosophy really for us around how we’ve been thinking about AI and how we see AI being successfully adopted by our customers is again back to that kind of system thinking. You know, how do I how do I make sure that everything I do is is is being captured, but it’s it’s not sort of you know proliferated across too many tools.

Greg Lambert (12:08)
Yeah. Well let me get away from the from the the partner level and let me let me ask you about some of the training, especially on the the associates. and I and I think you know I I was reading articles today that that kind of played on this fear that you know that the AI is is going to just cut all of the junior associate class, you know, compound that with the fact that

a lot of clients don’t wanna pay for that. They see it as as training. I always laugh. I I love the fact that the clients love taking the the experienced lawyer later and bringing them in house, but they don’t wanna train them them

Triona Buckley (12:47)
Yeah.

Greg Lambert (12:52)
As a junior associates, but you you mentioned that we need to move away from relying on training by osmosis, and I think a lot of us understand that. what you mean by that. So, how should firms re-architect their training module or models to so that AI acts more as an accelerator?

that teaches critical thinking rather than just a machine that that just does some some work for them. I was I I like to call this using the AI to help you AI. but how how do you see the AI being used in the training sessions?

Triona Buckley (13:30)
Yeah, I mean, you know, one of the things that I saw at law firms when I worked there was this, you know, model of, you know, summer associates and more junior associates like in office with partners or in office with senior associates. And like there’s a lot of value in that osmo like training by osmosis, as you said. some of the sort of more nuanced learning happens there. But you know, the the reality is that, you know, s some of that has already gone away, you know, through COVID and everything else, and people not necessarily being

Being

co-located. you know, that hasn’t been as possible in the last number of years. And while firms have you know really invested in their training programs and their more formalized sort of structured training systems, I one of the things that they hasn’t quite caught up yet is this yeah, is using AI to be the trainer, you know. So if we think about connecting what we talked about a moment ago in terms of institutional knowledge, there’s decades and decades of you know client work and you

know, you know, advisory work captured, you know, within the systems of a law firm, how do you put that to work? Not just for your client work, but actually for training your, your, your juniors. And so, you know, and it goes back a little bit to what we spoke about as well in relation to creating drafts. you know, if you actually have AI that is acting like the trainer, is asking your associates questions, is using the institutional knowledge to prompt them to ask deeper questions, to understand this.

More to make sure they’re checking what they should be checking. You know, it it really is an opportunity, I suppose, for it to feel like that sort of in-office, you know, more nuanced sort of training without that lift being on, you know, the partners to do that. Because, you know, as we all know, that’s a very inconsistent model. It’s very much reliant on, you know, how good that one, you know, partner or senior person is in terms of training. Whereas AI can really standardize that, you know, it can take all of the good parts of all of that sort of learning biosmosis, but add in a level of structure.

And and sort of rigor that that wasn’t there before us. So I’ve just you know we’ve I’ve started to see a few of the more innovative firms really see that as an opportunity, particularly at the mid market, where again you don’t necessarily have the resources to have a dedicated you know training program and that sort of thing. so so yeah, that that’s really the opportunity that I see is just sort of you know adding adding the nuance and the rigor, but really the institutional knowledge comes to life. This is how we do things with this firm, but not even just at a sort of a

you know, a workflow level more of the sort of like, you know, we tend to advise this way with our clients, you know, and like really, really very much more I suppose nuanced, you know, which is really the secret source I suppose of why clients choose a particular firm over another. Yeah.

Greg Lambert (16:03)
Yeah.

I’ve I’ve been using and and I know I know that Mark Andreessen didn’t come up with this, but he had he had said something earlier earlier this year about the number one thing that moves someone from the fiftieth percentile to the ninety ninth percentile the fastest is individual tutoring. and he said over you know the

long history of of humans that you know the the royal families did this, that the the the wealthy families did it, but it you know, the it took a lot of money to to do to do this individual tutoring and he’s saying now we have these tools that can be set up to be tutors. Is that is that how you’re seeing it is to train the train the AI to be the trainer or the tutor?

Triona Buckley (16:52)
Absolutely.

One hundred percent, yes. And and again, this isn’t to replace, you know, training programs or to replace that really important, you know, senior junior interaction. It’s about augmenting it and just making sure you’re standardizing it to a certain extent so everyone’s getting what they need. But I do really think that it’ll ex that it accelerates you know, the the pathway for for junior lawyers to actually get to a level of knowledge and and and and that more quickly and consistently, like I say. So yes, absolutely. It’s like using AI as your tutor.

not exclusively augmenting your training programs and and all of that. But but yeah, I think it’s a huge opportunity. And I think for mid-market firms in particular, a way that they can really use AI to help give their teams an edge, you know, and compete at whatever other level they they want to compete at with larger firms, whatever it might be.

Greg Lambert (17:39)
Yeah, yeah. I I I know a lot of trainers that that would love to have the help on this because they’re overwhelmed. In fact my my trainer was was telling me ’cause we were trying trying to think about how we were setting up the training program.

And she was it was kinda tongue in cheek and she was like, Well I’m you know, I’ve already got a full time job doing training and now you’re now we’re bringing in AI, so I’ve gotta do all that what I was doing and now I’ve gotta do AI too. So I think any type of leverage that we can get to to help the AI help us would be appreciated.

Triona Buckley (18:12)
Yes. Well that’s it. And then that’s

another that’s another bottleneck, I suppose, that we’re we’re seeing at firms is that yes, everybody has their day job, you know, and everybody is looking at how do they how do how do they sort of have this AI dividend brought into the mix. You know, how do we make AI actually create some ROI for us and make things more efficient? And in the meantime we’re double jobbing trying to figure out how to do privacy.

Greg Lambert (18:33)
Yeah. I was gonna say I’m still waiting on

that dividend.

Triona Buckley (18:36)
Yeah. Poor

poor IT teams trying to figure out, you know, what tools and when and for who and et cetera. Yeah, it’s it’s tricky.

Greg Lambert (18:44)
I I know when I interview attorneys that one if I ask them what are their top what are what are the top tasks that they do during a day that they would love to be automated.

typically those two are email and time entry. and so I wanna want to dive in on the the time entry, which you know there’s a lot of what was called the data debt that that is involved in the manual time entry tracking. and

Recently, Action Step acquired a startup called Tr Traced and brought in its founder, Aiden Bub, and integrated a new passive time capture module called Trace. so you’ve mentioned that when lawyers wait to enter time, and I think a lot of us know this, that you know, there’s that recency bias which causes them to you know write off time even before they’ve written the time down. so why

Why

is it so critical for you to for lawyers to capture time in the present tense as as as quick I guess as quickly as possible? And and how do you how do you use this AI module to help you kind of decouple the the time from the from the strictly hourly rates?

Triona Buckley (20:02)
Yeah, there’s there’s two aspects to it. Yeah, number one, I think, yeah, the the the panic, you know, at the end of the the week or the month where you get your push from rebelling team to to enter all your time. my goodness, what did I do? There’s that. And obviously, you know, most most fear earners have figured out systems and, you know, ways of of of helping with that in in you know, whether it’s their own individual kind of systems. and then the other piece is that yeah, you just you know, you you you automatically as a human write down the value of what you’re doing sometimes, you know, and so

Greg Lambert (20:12)
Yeah.

Triona Buckley (20:31)
you you just you you you think it’s fifteen minutes and it was actually an hour or whatever it might be. You know, you’re losing minutes all of the time and so

Frankly, sometimes it takes more time to enter the time of what you did, you know, and trying to recall what you did, it’s not almost not worth your time to create the time entry. And so there’s you know, there’s a few different aspects to it there, but what we know. So, but what we’ve seen is that actually again, if you’re as you work through your day, you’re getting regular you know reminders or prompts, or you know, you know, almost you know, bill ready you know, time entries presented to you, then all you have to do is go, Yes, I did that tick. you know, we’re we’re seeing huge.

Huge gains for our fee owners who are using Trace in Action Step. And you know, there if if you’re even capturing an extra, you know, 30 minutes a day, which is work you already did. We’re not telling you to do any more work, we’re just saying capture more of that work so that it’s because it’s billable work. you know, if you’re even just capturing 30 minutes more a day, that’s obviously a huge gain that all adds up every week, etc. So so that’s really what Trace is all about. It’s just in you know, it monitors both the work that you’re

you’re

doing in action step in our application, but also the work that you’re doing in in Word and email and et cetera, et cetera. And so it’s and and really presenting it all in a way that is, you know, bill already very easy for you to say yes. It’s obviously connect, you know, it connects it connects the work through to the matters using AI and using the context of of where you’re working. So

Really exciting one. I think there’s a huge ROI gain for law firms on that. You know, it’s sort of the most obvious one our customers have really gravitated to. so yeah, excited to get it into. We have we we already have our first version of that in market and being used by customers. We have our our next version actually coming out next next month, which crosss across more tools. so yeah, really, really good ROI for our customers on that.

Greg Lambert (22:15)
Well, I imagine even beyond just the, you know, tracking time, having a more accurate assessment of how long it actually takes to do something. you know, having those data points and and being consistent, rather than relying on somebody trying to remember at the end of the month who’s there’s you know is it’s gotta be a huge value add too.

Triona Buckley (22:35)
That’s it.

Yeah, down downstream the huge value adds in terms of yeah, like you know, accuracy of of client quoting, for example, you know, and and and on and that whole area, you know, and so there’s there’s you know, so pricing, you know, how you think about those things, you know, just just you know utilization across teams, you know, there’s so many different things that this plays into. and really lifting that information up to management so that they can see, you know, where people are spending them the the the most time and how can we help. Again, it helps to identify some of those bottlenecks that you

Mm-hmm.

Greg Lambert (23:07)
Yeah, yeah. I c I can I can imagine the the partner, the billing partner looking over there and going, my god, it took that long to do this. So one

Triona Buckley (23:14)
Ha ha ha.

Greg Lambert (23:18)
We talked briefly about the fact that the, you know, there’s a lot of tools that are being used. and it’s interesting because Action Step actually did a mid-market priority report, and it was just kind of showing the amount of tools they get used. and I think it was like 83% of firms still use like three or more tools just to manage a particular matter. and

a a third of of those use six or more tools. So and and being in a in a large firm, I I mean we’ve s we’ve seen tools sprawl where it’s like, it’d be you know it’d be great if we added this one thing or this will make make things easier. And the next thing you know you you’ve just got this huge stack of of technology with each one doing its own little piece. So

Triona Buckley (23:48)
Yeah.

Mm-hmm.

Greg Lambert (24:07)
Why why do you believe that mid-market firms are actually better positioned to to apply the the AI transition than say an an AMLOD, you know, 100, 200 firm? and and so, you know, how how are you seeing them being better positioned?

Triona Buckley (24:27)
I think there are three reasons why mid market firms are are better positioned than most. number one, they they tend to have fewer tools, so they’re probably more in that that middle camp of you know, kind of three ish tools, maybe that they’re they’re working with, which actually isn’t a lot, you know, being having some well integrated tools is is good. and so they have fewer tools to to work with, and so it’s actually it’s easier for them to sort of get their their their house in order and get sort of AI.

ready, you know, because their data’s living in fewer tools, frankly. And part of that is because you know that they don’t necessarily have the budget to buy all of these other two other tools. And part of it is that they’re, you know, they’re they’re able to you know the they they there are products out there that support them, you know, that that give them that sort of you know cleaner, cleaner environment. So there’s that piece of it. I think you know number one, just that their their tooling is set up well for you know for for AI to work well for them. Number two

It is just a function of size, you know. the you know, you have a professional layer in mid-market firms who are experienced, who typically are quite close to the bottlenecks and the needs of that firm. but because they’re slightly, you know, they’re not these huge big enterprise firms, they’re they’re easier ships to turn. And so change management is something that can happen somewhat more easily if you do have that professional layer. And I’m really talking about firms who sort of invested in having, you know, you know, strong management resources that can help we can help that to happen.

And then the third thing is that, you know, generally clients choose mid-market firms because they have, you know, they they the closest they get to senior, senior resources, you know, the closest they can have to the partners, the advisory relationship that they have there, which the you know, you don’t necessarily get, you know, in other segments of the market. And so mid-market firms already have, you know, good relationship managers, you know, so their fear earners are good relationship managers, their partners are very focused on the growth of the firm and making sure that custom you know, clients are are well.

taken care of and all of that. And that that’s one of the things that is going to shine most in a world of AI. You know, it’ll be less about the sort of the the quality of the of the you know the documents and that sort of thing and more about you know those sort of really nuanced personal you know advisory relationships you know which mid market firms have already had to hone those skills you know over many years. And it’s one of the reasons why you know

talent comes to those firms because they want to have more of that client engagement, you know. So so those are the sort of the three reasons why I think mid-market firms are in really, really good position when it comes to AI. They can their data’s well set up, they’ve got the client relationships already, which are gonna be their value point going forward. and they, you know, they’re they’re easy they find things, you know, easier to adopt and easier to change.

Greg Lambert (27:03)
Yeah, I wanna I want to pull on the the the change management thread that you mentioned there in the the middle of that. and I imagine that having fewer tools or tool you know tools that that kind of crosses across multiple steps.

makes it a little easier on the change management front. how how do you work with your with your c customers to kind of understand that change management aspect of it? ‘Cause I I think a lot of us don’t realize how important that is.

Triona Buckley (27:35)
You are absolutely right. So many firms do not realise how important that is. and we try and advise firms who come to us to really think about these projects as, you know, think about them in a matter of sort of, you know, months, not weeks. You know, like these are th you know, you have to plan for this. You have to make sure that you have you know, you’re you have an awareness, I suppose, of how your systems currently work, your actual, you know, how your people work. you know, and that you’re you’re planning around that and like less focus on feature functionality.

And more about you know your own processes today and and and how you want it to work in the future, that sort of thing. So, I mean, a lot of the change management piece is around, as you know, yeah, you know, being clear about what actually needs to change, where those bottlenecks are, but also having the right team in place that are going to actually drive it forward, they’re going to be champions, who are going to make sure it’s successful. So, a lot of the times when we’re engaging with you know firms who are coming to us for the first time, we’re asking them those kinds of questions, you know, like who’s who’s going to be involved in this in this process.

project. you know, what are the things that matter to your firm? You know, where do you see your firm, you know, next year, the year after? What sort of growth are you experiencing? What sort of clients you do with so really more like asking about their business so that they’re already starting to think about those sorts of things in the context of technology and less sort of you know comparison tables of features and functionality, like I say. So that’s really important. We also work with a really really experienced cohort of advisory partners who you

Know work with firms to help them to adopt new technology, like their entire jobs are around change management and technology and adoption and that sort of thing. So you know, while our team is very much because most of them have worked in law firms, they’re very much of that mindset. the actual implementation work is is is is done by our our partner group, you know, in direct collaboration with our customers. so that works really well. It means we don’t have any implementation bottlenecks at Action Step. You know, we have we have kind of an

Endless supply of you know of of of well-vetted partners that customers can choose from. So that works that works well. And it’s a very different, like we’re a software company, you know, and so it’s a very different mindset. You need to have business process people involved in that. but like I say, I think having those conversations from the very start, we know is really important. We have a lot of firms who will come to us, have those initial conversations, realize they need to go and do a bit more homework, and then come back to us, you know, kind of three months later. So yeah.

Greg Lambert (29:52)
Yeah. Yeah.

Triona Buckley (29:56)
That’s that’s probably how it works best.

Greg Lambert (29:56)
Yeah.

Yeah, the you know, kinda kinda keeping your eye on where you know the the s strategic focus of of the firm. because you know especially when you’re bringing in a a a substantial bit of software that that can kind of change the day in and day out for people that the you know it’s not just turn it on and say here you go. There’s there’s a lot that you need to think about as you’re as you’re

So it sounds like you’ve got a good handle on on that implementation. Well done.

Triona Buckley (30:27)
I like to think so. I mean look, these things are always they’re they’re huge projects, you know, they are huge projects, like you say. You know, it’s it is it’s a big undertaking. and I suppose for that expectation to be set is also important, you know, like let’s be let’s be clear, you know, this is going to take, you know, a lot of effort, a lot of thinking, but you’re gonna come out the side in a really, really good position, you know, and sort of you know, working through that th those, you know, th those periods of of of of pain. you know, it’s it’s an isn’t

Inevitable, you know, but but again, as long as as long as you’re having those conversations early and that’s understood, it’s not an issue. It just shouldn’t be a surprise.

Greg Lambert (31:04)
Yeah.

well, you know, the one thing law firms are known for is is really accepting change. So it’s or or or maybe not. Maybe maybe we’re not known for that.

Triona Buckley (31:13)
People.

But

you know, there’s a bit of a talk track about law firms sort of being, you know, like you know, slow to change and and and you know that sort of thing slow to adopt. but I actually I I respect the thoughtfulness that you know generally the legal industry has to, you know, to to to these things. I also think if you look at the history of law firms, like to be fair, there’s there’s few enough industries that have actually survived and thrived as well as as as law firms have, you know, so through all sorts of booms and busts and whatever else. So I don’t

Yeah.

Greg Lambert (31:43)
Yeah, yeah. Well, one thing we do thrive through uncertainty. So that’s so well, Triona, let me let me ask you before we get to our crystal ball question, we’ve been asking our guests for the past year or so, just because there’s so much going on in in in the industry, whether whether it’s legal tech or just AI in general, you know, there’s just

Triona Buckley (31:47)
Ha ha ha.

Greg Lambert (32:09)
So much to try and keep up with. So we’ve been asking, you know, what are what are some, you know, one or two of the you know, must read or must watch or must listen to resources that you l use to kind of help you kind of keep up with with what’s going on in in the market right now.

Triona Buckley (32:30)
I have to admit that while I am a obviously this podcast, but while I am a I I am I while I I listen to most things, I think the voices that I listen to the most are are are not necessarily the you know the podcasts or the articles or that sort of thing. It’s it’s actually, you know, folks are in the industry working with law firms every day. I mean obviously yourself and Marlene are in practice, so that’s you know different. You have that very specific

Greg Lambert (32:35)
Obviously.

Triona Buckley (32:57)
perspective but but yeah you know that I mentioned that partner community and so I would say actually that partner community I probably listen to the most and obviously directly to customers and really try and form our own view based on based on them because one of the things that I I see happening in the industry is a lot of me too me too, you know, just a lot of kind of jumping on bandwagons. And so actually kind of going directly to the horse you know, and and and sort of hearing what they have to say is is

it’s in my mind more powerful. so yeah that tends to be where I where I lean for sort for some real market reads. Like these are partners who are working with you know dozens of dozens of customers, law firms, you know, every single every single month you know who who really do understand things from the inside out. So that’s that’s where I lean.

Greg Lambert (33:43)
Yeah, it’s it’s definitely still important to to build your own community. So it’s I I think I think a lot of us have relied upon that, especially the last two or three years. So well now it’s it’s time for a crystal ball question. So pull out your crystal ball. It’s really kind of cheating because you’re already in tomorrow, so you you you probably already know some of the future.

Triona Buckley (33:48)
Yes, most important.

Ha ha ha

Greg Lambert (34:07)
But look looking ahead you know a a few months to a few years, as you know, a agentic AI and harnesses and you know, all of the stuff that that we’re being hit with lately affects and becomes embedded into the business of law. You know, what what are some things that you think especially in the mid market law firm model?

that they should be prepared for, that that that they may not be ready for yet.

Triona Buckley (34:37)
a few things. I mean, I think I mean some of the things we’ve touched on. So I think you’re gonna see firms that you know have invested in you know client nurturing, client relationships, that sort of thing. I think they are gonna be the ones who who thrive. And so if firms are not doing that already, that should be the thing that they are prioritizing over and above anything else, frankly. you know, the second thing I mentioned, I think the firms that have their their data house in order are gonna do better, you know, in in an in terms of AI enablement. but in general, I think I

have a very positive perspective on where law firms will end up being. I think again, you know, law firms have been through a lot of cycles. They always end up you know surviving, thriving, you know, changing, evolving, you know, coming out stronger forward, sort of thing. And so I so you know I think I I think just for them to think forward to that point, you know, and what that looks like for them and and and work towards that. The the pieces that I don’t see as much conversation around at at mid-sized firms, I think is probably happening

With enterprise firms, but not necessarily mid-size, is around how their commercial model is going to change, you know, how their whole their pricing, their

you know, like the types of clients’ engagement and client engagements that they have is going to change. We’re already hearing from some law firms that clients are saying to them, you know, I don’t wanna I don’t want to pay for your juniors, I only want to pay for your seniors, that sort of thing. So I think, you know, you’re just gonna see a lot more which is has already been happening, but you’re gonna see a lot more, you know, project level pricing, you know, those sorts of fixed fee type of pricing, which is less less about attaching, you know, rates to people and more about the output. you know, so I

And I I I think that’s an area that I think mid-sized law firms really need to need to focus on and plan for and understand how that’s going to impact their their kind of economic model as firms. it’s one of the reasons why, you know, we we talked about time tracking earlier, but you know, you know, and sort of understanding the dynamics of you know, where you spend your time and how that plays into profitability. it’s one of the reasons why I think that’s a that’s like something that we’re investing in and that we can help law firms with to plan for in future.

Greg Lambert (36:35)
Yeah, yeah, great, great answers. So Triona Buckley, think I wanna thank you very much for taking the time to talk with me today and sharing your your vision of where where we’re heading. It’s this has been fun.

Triona Buckley (36:49)
Thanks, Greg. Really appreciate it.

Greg Lambert (36:51)
All right. And thanks to all of you who are listening to the Geek and Review. If you enjoyed the show, please share it with a colleague. we’d love to hear from you on LinkedIn and Substack. And Triona, where’s the best place for our listeners to learn more about you and Trace and Action Step?

Triona Buckley (37:09)
Actionstep dot com. Very simple. Yes.

Greg Lambert (37:11)
Easy. I like it. And

as always, the music you hear is from Jerry David DeSecca. So thank you, everybody.

Predictions about artificial intelligence often focus on job losses and shrinking demand for lawyers. Filevine CEO and co-founder Ryan Anderson and product manager John Rizner offer a sharply different forecast. Drawing on the Jevons paradox, they argue greater efficiency will make legal services accessible to more people, encourage deeper legal research, and create work once excluded by cost. AI might reduce the effort required for individual tasks while expanding the overall volume and ambition of legal representation.

The shift holds major implications for the access-to-justice gap. Faster drafting, research, and document review would allow lawyers to serve more clients without sacrificing professional judgment. Anderson expects family law, immigration, bankruptcy, criminal defense, and employment litigation to experience some of the earliest growth. Motions, witnesses, and legal theories once abandoned over expense become economically viable, although courts face their own capacity crisis as more disputes and arguments enter the system.

Rizner explains how Filevine’s legal AI platform, Lois, applies machine learning to one of legal research’s oldest problems: traditional citators often return different results. Lois combines citation graphs with semantic analysis to locate opinions discussing related legal doctrines even when no direct citation connects the cases. A panel of models then evaluates potential conflicts and produces a structured memo. The goal is richer legal analysis focused on the precise holding or proposition a lawyer needs, rather than a simple flag attached to an entire opinion.

Accuracy still demands disciplined human review. Filevine organizes citation verification into three levels: confirming the cited case exists, determining whether the case supports the claimed proposition, and checking whether the authority is still good law. The conversation also examines Rizner’s research into how different large language models approach efficient breach of contract. OpenAI, Google, and Anthropic models produced dramatically different recommendations, revealing embedded legal and economic preferences beneath seemingly neutral answers.

The guests also explore how AI changes legal drafting, law firm economics, and the billable hour. Filevine’s acquisition of Pincites, now Lois for Word, reflects Microsoft Word’s continuing role as the shared language of legal documents, redlines, formatting, and negotiations. Efficiency does not automatically eliminate hourly billing. Lawyers might instead use saved time to produce more thoroughly researched arguments, stronger contracts, and work product approaching senior-level depth. Firms still need incentives rewarding efficiency rather than treating faster work as lost revenue.

Looking ahead, Anderson and Rizner predict a proliferation of frontier and open-source models tailored to firms, individual lawyers, and specific client relationships. Legal teams will increasingly pair proprietary knowledge with selected models to produce highly specialized analysis. Yet model choice introduces jurisprudential bias, accuracy risks, and serious training concerns for junior lawyers. AI expands the range of available options, while experienced legal judgment decides which arguments deserve trust, which sources require verification, and which advice should reach the client.

John Rizner Slides Filevine Primary Presentation – 2026

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Nikki Shaver (00:00) Hi Greg and Marlene. Ever since Anthropic launched Claude for Legal, a lot of focus has been on whether firms and legal departments should be using that in place of, or as well as, legal-specific applications like Harvey and Legora. But Anthropic is not the only big player to brush up against the legal market. Microsoft has launched its legal agent, embedded in Word and other 365 applications. OpenAI has reportedly hired someone to help build in legal. Perplexity has launched Computer for Counsel with a host of legal MCP connectors. And now it seems Amazon may also have its sights on our vertical with Amazon Quick for Legal. It was already a tough market to navigate when all you had to worry about were the thousands of legal applications; now you also need to track what’s happening in the broader tech ecosystem. We’ll soon be publishing a helpful comparison of the big tech offerings for legal, with an examination of where in the legal tech stack they might be useful. Stay tuned for that on our site at legaltechnologyhub.com, or follow along on LinkedIn at Legaltech Hub. It’s a pleasure to see all of you. Until next week.

Marlene Gebauer (01:13) Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer,

Greg Lambert (01:20) and I’m Greg Lambert. And today we are going to be digging into a fascinating and probably somewhat counterintuitive theory that’s turning the traditional panic over AI on its head. Instead of asking how many legal jobs AI will destroy, we’re actually going to explore why AI may actually trigger an unprecedented explosion of legal work, opening up massive markets that were historically priced out.

Marlene Gebauer (01:48) In hearing what our guests have to say about this, and to lead our conversation, we are absolutely thrilled to welcome John Rizner, product manager at Filevine. And joining John is Ryan Anderson, the CEO and co-founder of Filevine. John and Ryan, welcome to The Geek in Review.

Greg Lambert (02:03) Welcome, guys.

Ryan Anderson (02:04) Good to be here, Greg and Marlene.

Greg Lambert (02:05) All right, John, let’s start off with you, because we wanted to talk about the Jevons paradox, which I know a lot of our listeners have heard before. But let’s look at the big macroeconomic picture that you guys have brought to our attention as well. Many pundits out there are predicting that AI will contract the legal industry, but you guys are arguing the exact opposite of that using the Jevons paradox — the economic theory that technology makes resources cheaper and more efficient to produce, and then consumption actually skyrockets. So do you mind just talking a little bit about how the paradox applies to legal work, and how we should be prepared for that?

John Rizner (02:51) Yeah, totally. So I’m going to talk about it in connection with a product that I’m working on: a citator, à la Shepard’s-style citations, within LOIS. When we started off investigating our tooling, I had a number of conversations with some old law school classmates of mine that did far better in law school than I did — appellate litigators who went through the elite clerkship rounds. And I was talking with them: “If you had the ability, with large language models and new AI tooling, would you be excited to chew through fewer research materials as part of your work?” What was interesting is all of them said, actually, if I had unlimited time and unlimited resources, I would want to find more and read more, as opposed to find less and read less. If they could, they would love to boil the ocean — exhaust every possible resource in reading through opinions and relevant literature on their case before drafting or preparing anything. So I think as AI tools, especially in the legal research space, the citator space, make finding the relevant literature easier — even making more of the relevant literature an immediate delivery to the user — rather than lawyers spending less time doing research, we’re going to see lawyers spend more time doing substantive research. The difference being, instead of lawyers saying, “All right, I’ve got X amount of time, I found this handful of cases, let’s quickly draft this and get it out the door,” I think you’re going to see far more in-depth and nuanced arguments in legal work product, due to the fact that lawyers will have easier access to greater amounts of research material. And as a consequence, you’ll probably have better opinions come out of courts, because the arguments lawyers are making are going to be more nuanced, reflecting the deeper research. So I hope lawyers like doing research — I hope that’s why they went to law school — because in my view of it, you’re going to see greater and greater research, and far higher quality research, as opposed to lesser or more shallow research.

Greg Lambert (05:23) Yeah, that was going to be my follow-up question, on a conversation we had last week — one of the things she mentioned was: we used to get these one-page surveys or reports, and now we’re getting these 40-page surveys. And it’s not necessarily better; it’s just more. But I think your argument is that the outcome, the results, the research is actually better rather than just more research. Am I interpreting that correctly?

John Rizner (05:55) Yeah. So I think, at their core, and obviously we all know as part of our ethical duties, lawyers should be diving through the cases that they’re possibly looking at. And I think as these tools evolve and become more mature and can find more relevant and more on-point items for the lawyers to read, they will read the same amount, maybe even read more. But the conclusions they come to will be stronger, because they have that better picture of how the law is at that particular moment. So volume may grow, and in addition to volume growing, I think quality becomes far better in the long run.

Marlene Gebauer (06:38) Yeah, I think that makes sense, because AI kind of allows you to separate the wheat from the chaff, and then you can focus more deeply on the things that you need to focus on and sort of leave the other ones.

Ryan Anderson (06:52) Yeah, I’ll speak to that for just a moment too, from a market perspective. Two, three years ago there were a lot of predictions around what would happen to engineers, coders. Filevine employs a lot of coders — I think we will end the year with something like 400 to 450 engineers, about half of which are ML engineers, the other half your traditional coders. And there were quite a few predictions that AI would reduce the need for corporations, for tech companies specifically, to have as many engineers. The theory was that there will be so much code written, and it’ll be so much easier to write, that you won’t need as many engineers — so you’ll see some layoffs and fewer engineers hired. Well, it’s been the case at our company, and I think broadly across the industry, that the exact opposite has happened. The engineering market sort of hit a low maybe two or three years ago and has actually climbed out of that. More are being hired — not more than ever, but certainly more than the post-COVID lows. More companies are bringing on more engineers to do more work, and there’s a very logical reason for this. We definitely write 10 times more code today than we did before — it might even be more than 10 times more. But for every piece of code that we write, it has to be reviewed, it has to be QA’d, it has to be tested. And as the aperture of the product increases, the surface area for problems increases, but the ambition of the product also grows. What that has allowed us to do as a company is build a much richer, broader, more end-to-end product offering that we think is really unique, but I think it’s the same for a lot of companies. The analogy really holds true for legal. As John brought up, you’re going to have lawyers able to do research more quickly and find truth for their clients in a much broader way, and be able to zero in and find creative ways to apply the facts of their case to the law. But even consider the lawyer who says, “Instead of bringing that motion the client just didn’t want us to do because it was going to be expensive, I didn’t bring the motion to compel the deposition of that witness. We just let it go — we just didn’t do it in this case.” That happens all the time. If you talk to litigators, they’ll say their clients didn’t want to pay for witness number six, who might have had some interesting information in the case. But boy, now that you can bring that motion to compel, and it’s much easier to draft and much quicker, the opportunity costs change, and the search for truth becomes sort of superpowered. And of course, litigation in particular is a counterparty affair — what one side does, the other side is having to respond to, and then maybe bringing a counterattack themselves. So for better or for worse, I think truth will get sought out more quickly. But also, I think lawyers are going to be very busy. And honestly, I wouldn’t want to be a judge in this atmosphere — I think it is really tricky for the judiciary at this time.

Greg Lambert (09:47) Yeah.

John Rizner (09:50) And I think one other item to bring in on this truth question: it’s not like we are looking at a field where all the buried treasure has been dug up. One of the things we were looking at early on was research by Paul Hellyer and Susan Mart, two legal scholars, who had done quantitative analysis of how often, say, Lexis and Westlaw agree or disagree on a particular citation, or how often they return the same relevant opinions for a particular citation analysis. What’s really interesting is that the differences between the two could be substantial, depending on the opinions looked at. So if you’re a subscriber only to Westlaw or only to Lexis, you may be missing items that Lexis is returning but Westlaw isn’t, or vice versa. And so where AI tooling makes it cheaper and easier to apply different methodologies — so you can have your multiple citators taking different approaches at a larger scale — I think you start to discover more of those opinions that were otherwise hidden, because you were stuck with one citator and only uncovering those hidden truths via one type of methodology.

Greg Lambert (11:10) Ryan, I want to go back to something that you mentioned about using the engineers as a parallel path to some of the things that we’re seeing in legal. I was listening — I believe it was Adam Mosseri from Instagram — talking about how his engineering teams are made up now versus how they were pre-AI, where he was saying every project had like a baker’s dozen of people on it, ranging from engineers to project managers to researchers and data scientists. And he says now it’s essentially a team of three. Are you seeing similar things on your engineering team? And do you think there’s a parallel for how lawyers will be doing it?

Ryan Anderson (12:00) Yeah — and sorry, sorry to interrupt you, but yes, we are definitely seeing a reduction in what a single person can do. Let me give an example. This just happened today, about two hours ago. I was going through a new feature with our team. We were thinking through how to design an experience on our deep retrieval engine. We think we have really world-class retrieval — we think we can search through more documents than almost any product out there; we literally think we’re the best in the world at this. But you might imagine there are some UI constraints: telling the user where you are when you’re searching through millions of pages of documents can be tricky. There was some back and forth about how to do this, the team’s debating, there are about 10 people on the call, and we’re saying, “Well, how about this method to tell the user what’s going on? Here are some other methods.” And about 15 minutes into the conversation, one of the engineers goes, “Well, I actually just coded it. Does anyone want to see what I just did?” And we literally shared a screen, and he showed something — not in a production environment, but on his local machine using real live code — and said, “Here’s how I think we should do it.” It was working, and we could actually play with it right there during the meeting. I can tell you that kind of interaction never happened before. Never. We have lawyers who say, “I was exploring an argument that I didn’t think would be a good idea, but I used LOIS, and all of a sudden it found a couple cases, and it was a line of case law theory that I didn’t think applied to my case, but it does. And so we’re going to actually attack the other side in a way that we hadn’t realized at all.” When you lower the cost for exploration, for creative exploration, people become much more creative. I’ve always thought the best lawyers are actually exceptionally creative. It’s funny — lawyers don’t think of themselves as creatives. They think of artists and movie stars and musicians as creative people. But I think great lawyers are exceptionally creative, and by lowering the cost to go on a creative thread with a legal theory, you enable much more creativity. It’s very exciting.

Marlene Gebauer (14:09) So Ryan, when we talk about lowering costs to capture a dormant market of middle-class clients and small businesses, what specific practice areas or legal needs do you think will see the quickest influx of new work? How does Filevine view this unfolding from a macro strategy perspective?

Ryan Anderson (14:29) There are many great things about AI, and there are some scary things about AI. But in the legal industry, perhaps the most exciting is — we all know that getting a lawyer is actually quite a challenge. My brother, who’s our chief product officer, talks about how he needed a lawyer to help with a relatively small real estate dispute. Here’s a guy who literally leads product for a pretty significant legal tech company, has a brother who was a lawyer, and knows many, many lawyers in his day-to-day job, and he could barely get somebody to answer the phone. It is hard sometimes to find a lawyer — and that’s a well-connected guy. If you’re somebody who’s indigent, working class, maybe you don’t grow up with the same kind of privileges or connections, it can be extremely challenging to find good legal representation. And it’s pretty awesome that lawyers are going to have much more capacity now. You asked what industries: I think you’ll see a lot more in family law, a lot more immigration law, a lot more bankruptcy, and I think underrepresented criminal defendants are going to be in much better shape than they are today. In fact, a lot of those state public defenders’ offices are clients of Filevine. The Innocence Project is a client of Filevine. You’re going to be able to have more people who have meritorious claims for wrongful prosecution — or for having a lawyer who didn’t do the job they should have been doing — win their case, or at least get them a lower sentence. Many more of these cases can be taken on. It’s incredibly exciting. So I think you’ll see it across the board in those areas. The example I give is: these are lawyers, so we’ll say they have 10 hours in their workday, and it now takes two hours to draft a will instead of eight. Well, how many more people can they serve in a given day? If they could draft one will in eight hours, or maybe it was four, and now can do it in two, you’re now serving something like double the number of people during a day. You don’t even have to reduce your rates that much — maybe your rates stay the same, they might even go higher — but you’re able to serve more people, and those people wind up paying less because they’re taking less of the lawyer’s time. So it’s a very exciting world we’re going to live in. We don’t know exactly how it will all play out, but more legal customers will be served than ever before. It’s very exciting.

Marlene Gebauer (16:55) What do you think about labor and employment? Would that be an area as well?

Ryan Anderson (16:59) Oh, for sure — undoubtedly. I would love to say that this is maybe good news for corporate defendants. It’s probably not good news for corporate defendants. They’ll have more tools at their disposal to investigate these claims, and they’re often not meritorious, and Filevine wants to help those customers. But also plaintiffs’ lawyers who have meritorious claims against corporations use our products. We really want to help lawyers find the truth; we want to help the justice system find the truth. We are huge believers that the American system of justice not only is the best in the world, but forms the infrastructure for a fair and just capitalist society. We think the lawyer’s role in a system of capitalism is really critical, because otherwise it’s hard to keep business in check. So we’re really proud to serve our customers, whatever side of the “v” they may be on. But yes, I think you can expect an increase in litigation across the board.

Greg Lambert (17:52) I’ll go back to when you were saying it’s going to be a difficult time to be a judge right now, because I think that’s one area where, if we don’t figure out how they are going to handle this massive influx of new cases, it doesn’t matter really how much of the efficiency gets on the plaintiff/defendant side if the court system is inaccessible.

Marlene Gebauer (18:20) I just wonder if they’re going to get like an AI version of Judge Judy — people will agree to the AI judge and let them decide in smaller matters.

Greg Lambert (18:24) Are you guys working on that, Ryan?

Ryan Anderson (18:24) Yeah, you could see kind of smaller matters, private matters. Obviously, I think there’s a different need to be there, but —

Marlene Gebauer (18:38) I could see that for mediation, sure, stuff like that.

Ryan Anderson (18:55) Yeah, I think so. I think you can envision small arbitrations that are private, where at least they agree to maybe some AI tooling being used. I think that’s almost certainly going to happen — it probably should, to a degree. Let me be clear, though: Filevine and LOIS, which is our AI product, is the core economic engine of this business. We basically only sell LOIS and AI products today, which is quite a difference from where we were three to four years ago. So AI is incredibly important to me; it’s incredibly important to this company. We are fully AI-pilled in how bullish we are about this industry. Having said all of that, I think the judgment of the lawyer, the judge, the legal professional isn’t going away for a really long time. It is one thing to see an AI output as an unsophisticated consumer of that information and say, “Geez, this kind of looks pretty good and pretty persuasive.” But I think all of us have seen enough AI results and prompts and, frankly, slop to go: hold on, hold on — this looks like it’s right, but it is in fact not right, and sometimes in really critical but perhaps nuanced ways. That kind of legal judgment is going to exist for quite some time. So I think we’re a very far ways off from lawyers being replaced, from judges being replaced. But as I’m sure everyone in your audience already realizes, the age of AI in legal is here — it has been here now for a couple of years and is very squarely in our era. We will talk to our grandkids about this transition; it is a very big deal. But we’re huge believers in the primacy of human judgment when it comes to the law. We believe that lawyers play a really critical role.

John Rizner (18:55) And I want to tie back to some research that we did this last year on how LLMs respond — we may get into the paper in a little bit in this podcast — but we were looking at how different LLMs, or different families of LLMs, react to the same law-and-economics breach of contract issue, and how they compare to humans. What was maybe surprising, maybe unsurprising, was that depending on what model or what family of models you were using, you could see dramatically different results on whether the LLM was pushing the judge or the practitioner to push for a breach of contract or to keep a promise. So I think there could even be a future in which — if we’re talking about our LLM-based arbitrators — folks are fighting over which LLMs or which tech tools to use, because there is that risk that the tool you use could affect what answers you’re coming to. And I think that’s even another argument why the human attorney needs to remain a core part of the practice. I don’t think we’re yet comfortable giving over our judgment to tools that might already have a judgment built in that we don’t even agree with.

Ryan Anderson (21:45) Yeah, and of course — yeah, go ahead.

Marlene Gebauer (21:47) No, no, go ahead. Sorry.

Ryan Anderson (21:50) No, no. I mean, look, we all see the hallucination news daily, and it’s everywhere. I think lawyers are right to be very concerned about it. I think judges are right to take a really strict view of hallucination. It’s out there all the time; it is an extremely challenging problem. Hopefully we’ll get a chance to discuss what we’re doing on anti-hallucination — that’s an entire team at Filevine, the anti-hallucination team, certainly with respect to case law. But I’ll just give you one example. We had a customer put some data through Claude versus LOIS — we think LOIS is much more precise than Claude — and Claude came back, and it had taken some testimony and put in quotations something that a witness had said. Literal quotation marks around what a witness had said. And it turned out that witness hadn’t said that; it was sort of an amalgamation of three or four things the witness had said. If you looked at each statement the witness had made, you can understand how Claude would have arrived at the conclusion that the witness had said this quote — but the witness had not said that. And any lawyer looking at the quote versus what the witness had actually spoken and verbalized in the deposition: totally different. Totally different. So the legal judgment to know the difference between what is close and what is precise is going to be needed for a long time. We’re not there yet. AI tools are very important, but they have to be watched over, for sure.

Marlene Gebauer (23:15) Well, I want an anti-hallucination T-shirt — an anti-hallucination team T-shirt, that’s what I want. So John, Greg and I saw your presentation at Texas Trailblazers a few months back, and you noted that AI adoption follows incentives, and that trust in AI is ultimately a workflow problem. Since of course we’re talking about Gen AI, it wouldn’t be fair to not have a billable hour question. If efficiency gains allow lawyers to draft a complex contract or review a file in a fraction of the time, how do firms need to restructure their incentives so they aren’t punishing efficiency under this traditional hourly model? And honestly, if you have this, that’s the secret sauce, because we’re all grappling with that.

Greg Lambert (24:05) This is the billion dollar — maybe trillion dollar — question.

Marlene Gebauer (24:07) Yeah, so here you go.

John Rizner (24:09) I think, on the one hand, the billable hour has survived a lot of technological change so far and has not disappeared yet, so I think the billable hour has some weight to it that will be hard to remove. Coming back to Ryan’s point on the will piece, as well as our opening point about quality: say producing a particular piece of work product today takes eight hours, and we push it down to two to get that same exact work product. Well, there’s still runway to improve on that particular document — a better-researched argument, a more ironclad contract, a will that really thinks through all the elements of this particular family situation and gets it exactly how the — this is throwing back to my law school days — the testator, is that the right term? Got to go back to trusts and estates. But I think with that extra time, the associate or the partner now has the ability to provide a better work product, something that really reflects what their particular client needs, at the same price or less, or maybe slightly more, but with a depth that wasn’t even close to what we could achieve today. The deep nuance that an associate may not have been required or expected to have today will be a requirement in the very near future and in the long run. So those extra six hours will be spent producing that higher-level, almost partner-level deep quality that just wasn’t accessible or doable today. So I think the billable hour survives for a while — unfortunately or fortunately, depending on what side of the debate you’re on. I know I never enjoyed the billable hour, but I think the economics of it still work, especially if you think that volume and quality become a greater and greater focus.

Greg Lambert (26:09) Yeah, I’ve yet to meet someone that loves the billable hour, but yet here we are.

Marlene Gebauer (26:15) When you find one, let us know. We want them on the podcast. It’s like: explain yourself.

Greg Lambert (26:21) Yeah. John, I want to also jump in on your presentation that you did a couple months ago at Texas Trailblazers, because I wasn’t sure how well that was going to go over — it was a pretty in-depth, very, very deep dive on citation systems.

Marlene Gebauer (26:38) We loved it.

Greg Lambert (26:44) But I asked a couple of partners at my firm who were in the room there, and they were like, “This was the best part of the whole conference. I love this.” So let’s jump into that, and more on the LOIS legal research product that you guys have as well. You’ve noted that traditional citators tell you about the case — we kind of talked about that earlier — but lawyers really care about the specific holding. Do you mind giving us almost a Reader’s Digest version of what you presented there, and talking about these dual-pathway retrieval systems under the hood? I’ll turn it over to you.

John Rizner (27:26) Yeah, absolutely. So our citator approach all came from the idea of: are there workflows in the current legal system that we think we could mimic using ML tooling and LLMs, to come to the same answer that the current legal process works through? What we ended up falling into is: on the one hand, citation graphs — your traditional citation approach — do a really good job when we’re looking at those cases that directly engage one another. Chevron and Loper Bright, for example, where you have Loper Bright talking expressly about Chevron. But there are all those cases that might talk to issues around the issue that you really care about, but might not expressly be cited by the potential treating cases — because it may be only a part of the larger case, or, for whatever reason, a clerk didn’t include it or a judge wanted to make a strategic decision, and those links just might not exist. So we thought: all right, can we find a bunch of opinions that might be relevant, either on a citation graph or, as we came to it, via semantic similarity — the inherent meaning of the text as turned into a kind of mathematical representation? Could we find all of the opinions that could exist and be relevant to the lawyer’s issue? And then from that, let’s pass that into almost a mock en banc process, where we have a bunch of LLMs operating as judges trying to decide: do we care about these opinions? Is this a relevant opinion to the issue we’re looking at? Do we think it conflicts with the issue we’re looking at? And from those mass LLM runs, produce a structured memo for the user saying: in this universe, we found these opinions that might cut across your specific chosen issue, for these reasons. What we’re really excited about is finding those opinions that do negatively engage one another but are really hard to find, because they’re not cleanly on a citation graph. The one I always go back to — I’ll look here in Utah — there’s this case called Brinkerhoff v. Salt Lake City, and the last few paragraphs of Brinkerhoff talk about this old doctrine of governmental immunity. There’s this test where the question of governmental immunity turns on whether the government activity is proprietary or governmental. If you pull up Brinkerhoff on some traditional citators, you’re not going to see any negative flags on it. And the reason is that Brinkerhoff has been talked about in a negative way in later-down-the-road Utah opinions, but the doctrine that Brinkerhoff used in those final few paragraphs was ultimately cut down by the Utah Supreme Court and the Utah legislature. Because we are, one, surfacing the fact that Brinkerhoff has talked about this doctrine, regardless of whether those later cases mention Brinkerhoff, and two, because we’re giving it to this LLM panel approach — this voting and memo-writing approach — the LLMs can recognize: wow, this doctrine is being used in Brinkerhoff; wait a minute, here’s Standiford, this later-in-time case that attacked the doctrine. We think there’s a connection there — a conflict there. We are able to find these items that are, again, otherwise unfindable on a traditional citation graph. That’s the thing I’m really excited about: we’re able to find for lawyers these ideas and conflicts that may be relevant to the other side’s point or opinion. What we want to surface to the lawyers is: hey, there might be other attacks on that particular proposition of law that we can now help you find, that were just undiscoverable before.

Greg Lambert (31:36) Yeah, man. It just brought that presentation all back to me. I remember how I geeked out on that train.

Ryan Anderson (31:44) It’s very cool. You know, I know this will be posted in transcript form — if there’s a way to show some of John’s slides, I mean, the slide that shows that...

Marlene Gebauer (31:53) Yeah, if you can give us a link to the slides, we will post that up in the show notes, absolutely. It’s so cool.

Ryan Anderson (31:58) Yeah — I mean, I think as you all probably were when you saw the presentation, just the notion that Westlaw and Lexis find different sets of cases. Good night, that’s terrifying.

Marlene Gebauer (32:09) We’re just like, yeah — someone’s finally saying it out loud.

Ryan Anderson (32:13) Right, right. And we’ve just shown that this method actually picks up a lot of cases that, in some cases, neither of them found. So it’s really interesting.

John Rizner (32:13) When we were starting to first benchmark our tooling, we had been pulling down those studies by Susan Mart in Colorado about the different cases they were finding. And we were like: some of this research was done eight years ago or five years ago — I wonder if, because of technological change, these differences have started to go away. In our current benchmarking: not so much. Still the dramatic divergence — which citator you happen to be using is going to give you a particular answer, and if you’re using a different one, you’re going to get a different answer. So it’s amazing just how much opportunity there is to help fill those gaps and give folks an ability to find things that they otherwise aren’t finding right now with their current tool.

Ryan Anderson (33:20) I’ll just briefly note: we have such a wealth of case law, a rich history of case law in this country. It’s so cool that most things have been decided — the vast majority of things have been discussed at some point by some court somewhere — which gives such stability to businesses, to human beings, to politicians, to people operating in our country. It gives everyone a tremendous amount of confidence when they make decisions, especially business decisions, to understand what the nature of the law is. And yet the citation system that we use was written — and it was the best we had — in a really deterministic, code kind of way. Here’s a citation; it links to this other citation, which links to this other citation. And that’s great — that’s what we had available to us, and it took humans to chain those cases together in logical trees, progenies of different cases. But LLMs are particularly good at saying: these words, these phrases, are semantically similar — they have similar meaning to this other set of cases over here — and even though there’s no hard-coded citation, we find similarity here, and you should take a look at it. It’s really fascinating. It’s a great use of LLMs. Go ahead, Marlene.

Marlene Gebauer (34:22) I just think about the discrepancies that you’re talking about and how that impacts — and we’re going to talk about the hallucinations — but when you’re trying to check for hallucinations and for legitimate cites, how does this play into that? And with how the courts are coming down on this, how do people rely on the tools that they have? Or should they?

Ryan Anderson (34:48) Yeah, I mean, we think we have a solution. I’ll let John talk about that.

John Rizner (34:52) So, in my view, this isn’t even specific to legal research in terms of case law: there’s such an importance to having corpuses of the information you care about available and engineered in a way that is best presented to the LLM, in the right way, at the right time, and in the right process. The real question I think lawyers need to be asking their tech providers is: all right, what corpuses are you using? How are you presenting them to your AI technology? And what benchmarks, what kind of evidence do you have that your approach is working well? It doesn’t matter if you’re using AI in your case file, trying to find the right document that outlines a scientific expert’s findings on something, or a point from a deposition that you really care about, or, on the legal research side, a particular opinion or particular citation that you care about — either way, it’s really important that your legal tech has engineering designed to get to the right point in the corpus for the answer you care about. For us, on the document side — Ryan was talking about that earlier with our enormous data science groups — they’re really focused on our corpus of case file material, and on stopping hallucinations by using what we call the knowledge engine to get the right answers surfaced to the LLM. On the citation side, what we’re focused on doing right now is taking open-source corpuses of grounded decisions — in this case, CourtListener being our big partner in that — and any time text is produced within LOIS in the chat, whenever we see a citation pop up, having tooling then check: okay, this has been brought up — does this case exist? And more importantly, does the user have an opportunity to check if the discussed opinion is discussing the issue we care about? Because one of the things we’re now seeing judges talk about is: okay, the case you cited exists, but it doesn’t support all of the propositions you’re trying to support. And in my mind — and this is what we’re building towards — there are almost three levels of evaluation you have to do for any particular piece of case law.

Marlene Gebauer (37:10) It doesn’t say that.

John Rizner (37:21) On the first level: is this case actually real? Does this citation actually exist somewhere in the corpus? That’s your very base level, right — is “John v. Ryan in the Territory of Guam” entirely made up? Was that citation made up? On the second level, you have that second-order hallucination: does that citation actually refer to and support the proposition you care about? Sure, the case exists, but are we talking about, I don’t know, Roe v. Wade in a contracts dispute? What’s being cited doesn’t support what you care about. And on the third level is that citational analysis: okay, the opinion exists, it supports what you care about — now, is it good law in the larger common law analysis? So that’s how we’re thinking about it: are we building towards hitting all three of those evaluation levels?

Marlene Gebauer (38:16) All right, I want to switch gears for a little bit and move away from litigation and more into the transactional area. Filevine started with a strong focus in plaintiff litigation, but this January you acquired Pincites, which is a legal drafting and redlining tool, and brought in Sona and Mariam Sulakian into your executive team, rebranding the tool as LOIS for Word. As a little bit of background for listeners who might not know: Sona was formerly a legal strategy expert at Ropes & Gray, and Mariam is a former GitHub and Meta product engineer. So congratulations — you brought in the dynamic duo. And I have a couple questions here. Why is it critical to corporate and transactional expansion to have something inside of Word? Particularly because I think we’re seeing now, with some of the larger LLMs, that they’re actually doing a lot of the work inside that environment — and I realize they’re doing stuff in Word too — but you do see a lot, particularly when you’re working with larger sets, that they’re actually doing it within the LLM environment.

Ryan Anderson (39:31) Look, it’s a great question. I don’t think Word is the final surface for all legal drafting, but it is definitely the dominant surface still today — and we’re not even talking about AI legal drafting, just legal drafting. The lingua franca of law is still Word, and probably will be for some time. If I said to you, “Hey, here are the redlines to this agreement. Go ahead and open this link with this other document type you’ve never heard of. Don’t worry about it — you’re going to have to sign in and make an account to see it,” Marlene, you would say, “No, thank you. Yes, hard pass. I need the Word document, please.” And I think pretty much every lawyer feels the same way. They have all learned to use Word. They understand it. They understand how comments work in Word. They understand how redlines work in Word. So getting an entire industry to re-platform may be challenging — certainly in the short term, probably in the medium term, maybe not in the long term. But re-platforming on a different drafting modality is just going to be really hard. So I think we’re stuck with Word — maybe that’s the right way to say it, maybe not. Everyone uses Word because it is the most fully featured drafting product the world has ever seen. So there are some good things about Word.

Marlene Gebauer (40:52) Greg likes WordPerfect. Sorry. He’s a big fan.

Greg Lambert (40:53) I was going to say — WordPerfect 4.2 on DOS. I mean, I’m up for bringing that back.

Ryan Anderson (40:56) There we go — you’d be surprised. Recently we still got a lot of customers asking us to integrate with —

Greg Lambert (41:01) Yeah, those reveal codes, man. Still need those.

Ryan Anderson (41:06) So I think that’s really critical. First of all, Sona and Mariam would be the first to tell you — and by the way, Marlene, you are right, they are the dynamic duo. These are two very sharp women — sisters, of course. First of all, lovely human beings, just a delight to work with, but as smart of folks as I’ve ever worked with in my career, and really fun to have on the team. They’ll anchor our San Francisco office. That was some time ago now — six months ago — so we’ve actually built quite a large team around them at this point, and that team is solely focused on AI legal drafting. Of course, they use a lot of the work that John and others have done, but they would be the first to tell you that Word is not the only surface they’re going to work on, and they already work on other ways to draft. But given the gravity that Word has in the industry, I think it’s going to be around for a while. There is a benefit to having everything be in Word — getting the formatting right is almost impossible without Word, depending on the court.

Marlene Gebauer (42:04) Formatting is a tough thing.

Ryan Anderson (42:06) It is very challenging. And look, I fully understand judges’ particularity and frustration around hallucinations — but they get equally as frustrated around some of the tiniest formatting issues. I’m a little less like — okay, does it really matter if the margins are 1.25 instead of 1.1 or something? But I can tell you that it does to them, and if it does to them, it does to the lawyer practicing in their courtroom. So it’s really not optional, and Word can reliably produce the best-formatted documents for legal in the world. So we need to be on that surface, and we want to be world class there. And we do think LOIS for Word is world class — we’ll put it up against Claude for Legal or Harvey or Legora or any of our other competitors. We think it’s the best redlining and drafting tool out there. And to that end, we should note that not only is LOIS for Word very good at redlining — we think it’s the best at redlining — but the ambition is much stronger than that, much broader. It is drafting long-form, sophisticated, complex legal documents grounded in evidentiary citations and case law citations. That is an ambition that I don’t think many of our other competitors have gotten to quite yet — maybe some at the top of the market are with us there. That’s the product we’re building, and we feel really proud to build it with those two at the helm. They’re doing great, and will do great.

Greg Lambert (43:37) Yeah, sounds like they’re a great team to work with.

Ryan Anderson (43:41) They’re awesome. I don’t know if you’ve met them individually, but sharp, lovely, fun, and incredible — I would say impeccable product taste. We think their thought processes and intuition around how to build products that lawyers really love and want to work with day in, day out is second to none.

Greg Lambert (44:00) Well, now I’m regretting bringing you two on. We should have brought those two on.

Ryan Anderson (44:01) Sure — well, I can tell you, they would be better.

Marlene Gebauer (44:02) I was going to say, now we have to bring them on. Were you teeing them up? You are teeing them up. Good, because I think that would be a fascinating discussion — why lawyers like tools. I don’t even know that they know why they like tools. So I think that would be cool.

Ryan Anderson (44:21) I think you’re right. Well, we’d love to have Sona or Mariam come on the podcast sometime.

Greg Lambert (44:27) Well, John, I’m going to jump ahead because I want to do one more geek-out with you while we’ve got you here.

John Rizner (44:32) Sounds great.

Greg Lambert (44:33) I want to bring up a working paper that you co-authored called “The AI’s Philosophy of Contract,” where you empirically studied how frontier large language models are handling classic concepts like efficient breach and remedies. So here’s our chance to geek out: what did this reveal, and what interested you in writing this paper?

John Rizner (45:04) Yeah, so we came across a paper that was basically an empirical study of how humans respond to your classic efficient breach scenario — where it is more economical to breach the contract as opposed to adhere to and follow the contract. Humans, interestingly enough, had certain breach rates, but if you included a specific remedy in the contract, you could change how humans, in the behavioral economics sense, respond to those efficient breach scenarios. So we were curious: as LLMs become a place where you go to ask for legal advice, how might they respond? Will they follow humans in how they respond to efficient breach, or will they take a colder, law-and-economics view, or a softer view? That was our big question. So what we did is we took what were, at the time, the main frontier LLMs — Anthropic, Google, OpenAI — and created large sets of efficient breach scenarios, and tracked how these LLMs would respond to those scenarios. And the divergence was wild. You had Google and OpenAI, who were more like, “All right, it’s economical — go ahead and breach.” Well, Anthropic was like, “We are never breaching. We cannot breach.” You might have a swing from high-90-percent breach rates to under-10-percent breach rates depending on what model you were choosing. One of our big takeaways was that, for practitioners, this should be a thing to think about, because what model you’re using might determine whether you’re getting advice of “let’s tell the client to do X” versus “let’s tell the client to do Y.” The other big note we found is that the LLMs likewise respond to whether you had specific breach remedies within the contract, in a way that a human would also respond to those types of hints within the contract. There’s a lot of discussion in the frontier labs in San Francisco about alignment of humanity generally with these tools. I think lawyers need to start thinking about alignment as in: does this particular LLM align with my particular jurisprudential philosophy, or how I would approach this particular question at a more theoretical, more philosophical element of judgment? Because if you just hand it over to the LLM, what answer you’ll get and what legal advice you might get will be determined by whether you happen to be talking to Anthropic or talking to OpenAI on a particular day. A really interesting question as well: when we ran that empirical data, the open-source models hadn’t yet had their big day in the limelight. It’ll be interesting to see to what extent open-source models become a way to choose a particular piece of jurisprudence baked into the model — that either you prefer, or a particular court prefers, or a particular judge prefers. I think there are real questions on that, as models proliferate, of folks choosing models that might fit their particular jurisprudential philosophy. Say that five times fast.

Greg Lambert (48:33) It just made me think whether it follows the University of Chicago economics theory or Berkeley economic theory. So I guess it’s how it’s trained, right?

Marlene Gebauer (48:43) It’s super interesting, because I just did a client presentation this morning, and they were talking about: okay, even if it’s an approved enterprise foundational model, if people put things in and the clauses are not what they would put in, but that’s what’s recommended — it just highlights how important it is to have a playbook, or some sort of template or guidance to use in addition to the model, because you may be doing something wrong. That’s one point. And the other point is, I think about more junior people using these tools. Someone who’s got more extensive experience would look at this and automatically say, “Absolutely not.” But somebody who’s more junior, and who’s actually using this as a learning tool in addition to a drafting tool, is really going down the wrong path.

Ryan Anderson (49:51) I couldn’t agree [more]. I think it’s very scary. Junior attorneys — boy, you almost wonder how much they should be using it. It’s very challenging. And we see it at the company: some of our employees who are junior at the company or new to the company often rely far too heavily on AI analyses and responses from frontier models, and gosh, you wonder — did they even think about these things? So it’s a big challenge, and for that very reason, we’re huge fans of lawyers and legal judgment. We think it’s going to be around for a while.

Greg Lambert (50:28) All right, guys, we’re going to jump to the crystal ball question. Looking out into the near future, what are some challenges or changes that you think we’re going to have to be prepared for as we move along in this age of AI?

Ryan Anderson (50:46) You will see model proliferation. We’re speaking to you on a day that Meta just came out with maybe their first really good model, and I don’t think anybody had that on their bingo card. But here they are with a model that looks — at least initially, based on the commentary online and even some testing done — like a pretty good model, and pretty good at legal. So now all of a sudden there’s at least a fourth player, probably a fifth: you’ve got Grok, Gemini, Claude, OpenAI, and now Meta. You’re going to see a lot more of these. We internally are using more open-source models than ever before — it is now part and parcel of the work we do to fine-tune and train open-source models. A lot of folks believed there were going to be maybe two, maybe three dominant labs, and people were going to mostly build on top of them. That does not look like it’s going to be the case. In fact, we’re seeing the open-source models be as good or better, with relatively limited fine-tuning, than even some of what the frontier models can do. So that really changes the economics for everybody. First of all, it means we can use a lot more inference — it’s not as costly as before. It also means there will probably be a lot more folks — end customers, law firms — choosing to have a stake in helping to build their own models, and we want to be there when customers do that. That’s part of the beauty of LOIS: law firms can really customize it to them and keep what makes their firm great within LOIS. So I think you’re going to see a huge proliferation of models, which is going to be different, it’s going to be confusing, and it’s not, I think, the environment a lot of people expected to see even just a year ago. So that’s my big crystal ball prediction.

John Rizner (50:46) And the one piece I would add to that: with the proliferation of models, you’ll see, increasingly, what I like to call synthetic secondary sources. With the proliferation of models, and folks having their preferred work product — really, what some of these firms have spent decades upon decades tuning and building as their particular approach to the law — I think you’ll have tooling and models emerge where the analysis provided to a particular lawyer is so customized, where it is the specific model and specific data source coming together. If the firm name is, say, Ryan and John LLP, it is a technological marriage of our exact corpus with our exact preferred models to get our exact answer, at a scale that’s just not been achievable, because the knowledge and the style and the preferences were locked in the brains of the partner or maybe the senior associates. A crazy ability to get ultra-custom legal outputs for users.

Ryan Anderson (53:40) I think you’re maybe not even going far enough — it may not even be at the firm level. It might be the lawyer level, and it might even be the lawyer paired with the client. You could envision a world in which you say, “For this client, we prefer this model.” I can see that very easily being the case. We already orchestrate through multiple models just to give you a certain response to a query. So it’s going to be a world where you see a lot of models, they’re going to be used in a lot of different ways, and they’re going to become more and more specialized. It’s very exciting. Thank you both so much for the time today.

Greg Lambert (54:22) Yeah, you got it. John and Ryan, thank you very much. Appreciate it.

Marlene Gebauer (54:26) Yeah, and thanks to all of you for listening to The Geek in Review. If you’ve enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (54:35) And real quick — if they want to learn more, where’s the best place to reach out?

Ryan Anderson (54:40) Probably just hit me up on Twitter — DM me on Twitter. Ryan Filevine is my username on Twitter, and you can find me there. I guess we call it X now, right? So it’s X.

Marlene Gebauer (54:48) And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

What does legal AI value look like once speed stops serving as the headline metric? In this episode of The Geek in Review, Greg Lambert and Marlene Gebauer speak with Nikki Shaver, co-founder and CEO of Legal Technology Hub and a member of the inaugural Financial Times Law 50. Shaver argues that law firms need to move beyond time saved toward efficacy: stronger output, stronger client outcomes, and more effective legal advice.

The conversation examines why the billable hour is far from finished yet no longer serves as the sole measure of legal value. Shaver compares hourly timekeeping to a taxi meter: useful for internal visibility, yet insufficient as the price signal for work transformed by AI. Workflow mapping, client discussions, and pricing discipline become central where an AI-enabled process compresses weeks of effort into hours.

Corporate legal departments are adopting AI at a faster pace, bringing new pressure to outside counsel. Some in-house teams see AI as a route to keep more work inside, while others see room for firms to take on work that previously sat outside budget limits. Shaver frames the strategic question around delivering more for clients, especially in practice areas where a firm holds differentiated expertise.

AI has not produced the promised empty calendar. Instead, lawyers report fuller schedules, longer documents, and a growing verification tax. Shaver flags the rise of 40-page forms, bloated redlines, and outputs that look polished yet lack sound reasoning. The episode makes a practical case for concise drafting, human review, and critical reasoning before any AI-generated material reaches a client or counterparty.

Agentic AI raises the stakes. Legal Technology Hub’s AI Agents in Law Map tracks hundreds of solutions, yet governance has not kept pace with new autonomy, connectors, and downstream system access. Shaver urges firms to establish traceability, unique identifiers, risk-based human oversight, enforceable policies, and a clear view of where data travels.

For firms aiming past baseline adoption, Shaver draws a line between routine personal use and strategic transformation. Daily use builds fluency, but competitive advantage grows from proprietary workflows, data foundations, client-facing collaboration spaces, and focused investment in the practices where a firm already excels. Her crystal-ball view is blunt: trusted judgment will become a scarce premium asset, AI-native firms will rise, and traditional firms will launch AI-native subsidiaries of their own.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Nikki Shaver (00:00)
If you look at the history of law firms and what they offered, the scarcity was lawyers’ time. Increasingly, the thing that is, or will become, scarce and valuable is trusted judgment. I think that is something we will be able to charge a premium for.

Greg Lambert (00:23)
This week on The Geek in Review, we talk with our friend Nikki Shaver, co-founder and CEO of Legal Tech Hub, about where law firms stand now, where the pressure points are forming, and why trusted judgment may become the profession’s most valuable asset. But first, let’s hear from Nikki’s husband, Chris Ford, at Legal Tech Hub.

Chris Ford (00:44)
Hello, Marlene and Greg, from a gorgeous Toronto. This is Chris Ford, Chief Experience Officer here at Legal Tech Hub. Your audience might be aware of the events that we put on throughout the year, and I wanted to remind you of a couple coming up.

First, on August 4, we have the latest version of our online showcase, the Demo Dozen. This is an opportunity for 12 different legal tech vendors to give 15-minute quick-fire demos and give your audience a good idea of the latest in legal technology. It is free to attend.

Then, on August 23, if you are coming to ILTACon, we are holding our second annual Trivia Challenge. It is a social event, and we would love to see you there as well.

On September 23, we have our flagship innovation conference, Legal Tech Hub Horizons, in New York City at Columbia University. Visit legaltechnologyhub.com, select the Events tab, and choose Legal Tech Hub Events from the drop-down menu. We would love to see you.

Marlene Gebauer (02:01)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:07)
And I’m Greg Lambert. Today, we’re thrilled to have Nikki Shaver, the co-founder and CEO of Legal Tech Hub, joining us. Nikki was recently recognized on the inaugural Financial Times Law 50 list for her outsized influence in navigating the technological, structural, and political challenges

Marlene Gebauer (02:24)
Yay.

Greg Lambert (02:32)
currently besieging the legal profession. So, Nikki, first, congratulations and, more importantly, welcome to The Geek in Review.

Nikki Shaver (02:40)
Thank you so much for both things. I’m very happy to be here with both of you.

Marlene Gebauer (02:44)
Yes, Nikki, congratulations. You have recently noted that managing partners are shifting their focus away from simply measuring efficiencies or hours saved. You are arguing that the real measure of AI value now is efficacy, the quality of output, client outcomes, and the effectiveness of advice. So, what do the most innovative firms look like in 2026? Are they really

Greg Lambert (02:46)
Yeah.

Marlene Gebauer (03:11)
detaching from the billable hour? What are they doing instead?

Nikki Shaver (03:15)
So, certainly, I am not saying that the billable hour is dead. I do think that we have over-indexed on efficiency as effectively the sole value of AI for some time. I’m pleased to see that we’re moving away from that because, of course, AI has also enabled us to do things that simply were not possible before.

I think everyone was initially busy understanding what generative AI could do and where it could have particular impact in law and legal workflows. Efficiency and productivity were the obvious starting points because that is what you see first, right? You enter a query, and the response is instantaneous. So efficiency and productivity are the obvious initial gains. But the real value for law firms, when you look at what we do, is improving client service in one way or another.

There is value to being fast, right? You think of going to a dry cleaner and saying, “I don’t want to pick up this dress in a week. I have an event tomorrow night, and I need it done faster. I need to pick it up tomorrow morning.” You do not pay less for that service. You pay more. So, one thing we need to think about is rethinking the way we contemplate time and its relation to legal work. Delivering work faster has value, too.

Second, we need to look at the value we can achieve in terms of quality, digging in deeper where we can get insights from broader data sets than we have ever been able to access before, or literally undertaking work that would not have been possible before. I heard a story recently of a corporate legal department taking on a project that it would not have done previously because it would have cost the business something like $14 million. Now, it was feasible within budgetary constraints because of AI.

The firms that are particularly innovative at the moment are looking at the billable hour like a taxi meter, right? It is still useful. We are still going to use hourly rates to record what has been done internally. It is useful in all kinds of ways, especially given how law firms continue to be structured, but it is not necessarily the determining factor in how much work should cost.

That is not true across the board. There are clients who still want to see the billable hour. They want to see things broken down by the hour. In fact, historically, the billable hour arose because of client demand. So it is not as though all clients are clamoring to move away from it. But there are workflows and types of work where it makes sense to move away from it, and we are seeing firms do that.

One of the other things I have heard in the last year or so is that firms have mapped their workflows and understood that, when AI is applied to certain workflows, they see a reduction of two or three hours, which is palatable within existing pricing mechanisms. But in other parts of the firm, they see a switch from something that used to take, for example, three weeks of time across a legal team to five or six hours. That is not palatable. It is existential unless you get out ahead of it.

So, doing the work to map workflows and understand where the impact lies, where you need to get ahead of it, having real conversations with clients, and making sure you are pushing margins up rather than racing to the bottom or leaving a lot of money on the table when the work you are doing is better than it has ever been before.

Greg Lambert (06:51)
Yeah. Well, okay.

Marlene Gebauer (06:52)
Are you getting a sense, on the flip side, of how clients are responding to this efficacy position? I saw something in Artificial Lawyer today that said, I think, 30% of the work is going to be handled by AI in-house. I am curious whether you are hearing anything from that side in terms of how they think this is impacting, or how it should impact, their work.

Nikki Shaver (07:21)
It is really interesting if you look at clients and the way they have taken up AI. They were slow off the mark initially. I think everyone assumed that, because they had so much to gain from using AI internally, they would be the ones to get it underway quickly. But they did not have the benefit of the resources that a lot of large law firms have to pilot and deploy technology relatively quickly.

Between 2024 and 2025, the rate of AI adoption in corporate legal departments in the U.S. doubled. Again, from the end of 2025 to where we are right now in 2026, the rate has doubled. So that is happening quickly on the client side. Increased sophistication also leads to an understanding from clients of what law firms are able to automate. That savviness is something law firms need to recognize, and it is why you need to be having those discussions with clients.

Sitting in a room of about 100 GCs earlier this year, I polled the audience. One of the questions I asked was, “What are you seeking to gain from deploying AI internally?” A significant portion of the room said, “Keeping more work in-house.” So, I think you are right, Marlene. That is something client teams are focused on because they want to adhere to ever-tighter budgets and, ideally, reduce outside counsel spend.

But I have also heard that firms and clients have been able to start working together in a way that makes it easier for clients to send certain work out that they might not have before, because law firms might not have taken it on. It might not have been feasible within the way firms previously worked and the billing structures. There are ways, then, of increasing the work that outside counsel does in a way that is valuable to the client.

But I also want to say that there is a concern we will look at this and think, “How do we keep costs down for clients?” and make that the overarching feature of how we consider the relationship going forward. I would argue that the better question is, “How can we deliver more for our clients?”

How do we prioritize, as law firms, the practice areas and clients that matter most to us in terms of business growth going forward? How do we double down in those areas and make big bets that enable us to charge more because the work will be worth more?

Greg Lambert (10:00)
Yeah. I know this may be more just me, but most people I talk to, you would think by now, with AI tools, that all of a sudden you would have more time to work on certain things. But everyone I talk to says, “I have never been busier than I am right now.”

It seems like AI functionality has created more work because people feel like they can do more things. I imagine that, even in-house, they are getting more questions from their teams and from the rest of the organization. So it feels like there is this stacking of work, and even with AI, it feels like we are busier now than we were pre-AI.

Nikki Shaver (10:57)
No, I completely agree, Greg. I do think it is a fallacy, right? If you look at the original Industrial Revolution, there was the same kind of thinking process. Once we put in what we would now consider rather elementary tools, but nevertheless tools that at that stage effectively automated or took away some human manual work, the idea was that everyone would have more time. More time to lie in a field and look at the sun.

That is not what happened. The pace of work escalated dramatically. People took on more work, and the way we worked changed dramatically. I think that is happening again.

I use AI all the time. I feel busier than ever, but I am also able to do so much more. I do not think it is human nature to say, “Cool, I got that one thing done, so now I am going to take a break for the rest of the day.” I think our tendency is to say, “I was able to do that so much faster. What else can I fit in?” So it is a pace, an escalation.

But I think there is another thing happening that we do not talk about enough and probably need to address. It may be teething issues around the deployment and broader adoption of AI. Documents are becoming longer. Communications are becoming longer. There is that adage: “I would have written a short letter, but I did not have the time.”

It is easy to get a lengthy output from AI and use it. It is much harder to edit it down into something short, succinct, and to the point. I am hearing this across many firms and corporate legal departments. They are seeing forms and applications that might have been one page previously coming in at 40 pages. They are seeing redlines where AI has clearly been used, which is indicative of poor use of AI as well, not merely a tendency toward wordiness or length.

Redlines are coming into firms from the other side that have clearly been generated by AI tools, perhaps tools not purpose-built for legal work. Not only must the firm respond to those redlines in a negotiation, but it must first go back to the other firm and explain why the redlines make no sense to begin with because they have not been verified.

So, there is a verification tax that we are all aware of, and the necessity to supervise does take time. All of this on top of that can push things out and create more work.

Greg Lambert (13:37)
Yeah, yeah. I can tell you an example I saw this morning. One attorney was talking about a client obviously using AI to outline what their contract should say. There was some outrageous language at the bottom that the AI had generated, and they were like, “Well, if you want me to address that, my estimate on the bill is going to be ten times what you think it is, because this is not something we normally take on for this type of matter.”

So, again, you are seeing people rely so much on AI that they are forgetting the practicalities of it. A one-pager is much better than a 40-pager. Relying on the expert to give you advice is better than relying on AI to give you advice.

Nikki Shaver (14:13)
Right.

Yeah.

Greg Lambert (14:31)
Okay.

Nikki Shaver (14:32)
I totally agree. It is really interesting, right? When we use AI, even when you try hard to apply your human judgment, you are verifying, but you may not necessarily be asking, “How can I abbreviate this?” or, “Is this actually saying what I wanted it to say?”

I think there are a lot of ways where you need to apply more critical reasoning than we are even saying that we do. It is not only about judgment or citation checking. It is about reasoning checking. Is this word being used correctly because it sounds nice in the sentence? Do we need it all? Exactly.

Marlene Gebauer (15:10)
Do we need all of this? I can tell you, it is

Greg Lambert (15:13)
Yeah.

Marlene Gebauer (15:14)
like, we all know we do not.

Nikki Shaver (15:16)
Yeah.

Greg Lambert (15:17)
But it looks really good, it is formatted well, so let us just go with it.

Nikki Shaver (15:18)
Yeah. Right, right.

Marlene Gebauer (15:22)
No, every time I use it, I need this in three sentences. I need this in a five-sentence paragraph. Short sentences. You have to tell it that, too, because it is just too much.

Nikki Shaver (15:33)
Yeah.

Greg Lambert (15:34)
Yeah, because otherwise, it is paragraph-long sentences.

Nikki Shaver (15:38)
Mm-hmm.

Greg Lambert (15:39)
Well, Nikki, let me shift gears to one of the other big things going on in 2026. As you mentioned before, when generative AI started, it was more chatbot, more prompt, question, answer, back and forth. Now, of course, 2026 is agents, agents everywhere.

Legal Tech Hub recently launched the AI Agents in Law Map, tracking, I think at one point, 300 or more distinct agentic solutions. So, as law firms start to use more agentic workflows where AI is taking on these multi-step processes, how are you seeing firms adjust or overhaul their governance frameworks to prepare for workflows that handle so much?

Nikki Shaver (16:45)
To be very blunt, I am not seeing that at the level I think we should have it, given that agents are soon going to be fairly prolific, if not already. I am concerned that the evolution of technology is outpacing our ability to govern it. That is really problematic from a risk perspective.

Having said that, there are things firms can be doing, and a lot of firms are thinking about it. But it is difficult to understand exactly what mechanisms to put in place, partly because it is so dependent on what you are using agentic AI for and in what form. Whether it is part of a third-party solution or something homegrown and proprietary, you are responsible for putting the guardrails around it.

There are additional complexities when you look at MCP and the wide availability of connectors. Not only may you have introduced autonomy in an agent, but the tools and systems it accesses downstream may not be ones you can vet, nor may you be able to vet the data those downstream agents have access to. What do you do in those circumstances?

Even before you get to those scenarios, there are some fairly prosaic things. For example, even last year, I heard about firms that deployed agents but had not put in unique identifiers to ensure they were trackable or traceable within systems. Later, someone looked at unusual activity, thinking, “Who is doing that?” and suddenly realized it was an agent.

You need unique identifiers. You need to be able to trace agents in your system. But I have also heard all kinds of other stories. When you introduce autonomy, you introduce the potential for unforeseen outcomes. Some of these are amusing when we look at them now, but the fact that they exist is indicative of the potential for real risk.

I will give you a couple of examples. Someone I know ran an agent to extract data from their email system and then run various analyses across that data. When the agent went to the email system, it was unable to extract the data in the way it wanted to. So, instead of coming back and saying, “I am not able to do this,” it created an entire replica of the person’s inbox and saved it onto their desktop. Then it ran the queries against that replica of the inbox. The person only noticed later when they saw the emails saved on their desktop.

Another person recently told me about a scenario where they uploaded a PDF and said, “Open this PDF and tell me what it says.” But it turns out the agent did not have access to Adobe Acrobat. So it used unknown quantities of tokens to write a program that was a PDF reader in order to open the PDF and tell him what it said.

These are funny, right? But think of that. I have also heard stories from the GC perspective of agents that have gone into European employee masters of record to grab information and bring it back into the U.K. or the U.S., putting the organization immediately in breach of GDPR in circumstances where the organization did not have visibility over that.

You have to be really careful. Dan Katz, Gillian Hadfield, and Mike Bommarito have written a book on agentic systems. I think you may have had them on. There is an amazing chapter on the governance of agents. You really need to think about the risk calculus, how you are deploying the agent, and making sure a human is in the loop in the right way for the type of risk you are introducing, depending on how you are deploying the agent internally.

Of course, you also need to update your policies and ensure that those policies are not merely there but enforced across your organization. These are all things we need to be thinking about now.

Greg Lambert (21:03)
It is a crazy time to be a CISO. I do not envy them right now. Even though I pressure mine all the time to let me do crazy stuff, it is good to have him and his folks telling me, “Okay, let us pump the brakes here.”

Nikki Shaver (21:09)
Yeah. I do not envy them either.

Marlene Gebauer (21:11)
Yeah.

Nikki Shaver (21:26)
Yeah.

Marlene Gebauer (21:26)
Your Adobe example, and all the news coming out last week about new changes in how AI is going to be priced based on tokens, make me wonder how organizations are going to handle that. Is there going to be pushback? Are they going to say no?

I think everybody recognizes that the way AI has been priced is probably not how it will continue to be priced. Those days are gone. But when you calculate how much more it is going to cost, I feel like organizations are caught between a rock and a hard place. You have demand, but are you going to have to get certification for people to make sure they are properly using it in order to get a certain amount of tokens?

Are we going to dial back on using it for certain things because we do not really need to? Are people going to become more discretionary about how they use it?

Nikki Shaver (22:30)
I do not think we will dial back.

I also think this is probably a wave we are going through. If you look at what has happened since late 2022, we have already seen a lot of waves, and that is going to continue. We had a period when everyone needed to learn how to prompt in a deep way. Now every tool you use has a prompt enhancer, or prompting has moved into the back end. I think something similar is happening now. We are going to see a short-term escalation of costs, for sure.

I think you are right, Marlene. We all saw it coming. The reality is that legal work is incredibly document-driven, and both input and output are lengthy in many legal workflows. One thing we have been doing at Legal Tech Hub is putting out content, including webinars, on how to manage your tokens, think about your work, adjust your work, and adjust which model you select and the level of effort you select for that model.

But I think that is temporary in the same way prompting was. The effort of minimizing the token load should not sit with a lawyer. You should not be sitting there thinking, “How can I make sure I am using the lowest level of model appropriate for my work?” or, “Should I summarize things before uploading them?” That does not make sense for us to do in the workplace.

My sense is that token costs will ultimately come down, or there will be new efficiencies where they will not be as significant, and it certainly will not sit with us. There will be things that happen in the back end that manage token use in a way that makes it more palatable.

But to the cost point, the reality is that all of us need to recognize we will be investing more and more in this technology. If, and this is what I believe, it becomes completely inextricably linked with the work we do, such that you cannot contemplate doing due diligence or any other legal workflow without also thinking about the AI that enables it and how that operates, then realistically, we are going to have to invest in the technology the same way we invest in our people. It will become part and parcel of the way we work.

Greg Lambert (24:36)
That is it.

You are muted, Marlene.

Marlene Gebauer (24:48)
I wanted to switch gears for a second and talk about the RSGI and Harvey report. You recently analyzed that and revealed there is a massive divide between standard AI adopters and true power users. The data showed power users in law firms are saving about 37 hours a month, which is pretty incredible. So, what specific behaviors, methodologies, or mindsets separate

Nikki Shaver (25:09)
Yeah.

Marlene Gebauer (25:15)
the lawyer who uses the AI chatbot from a truly transformative power user?

Nikki Shaver (25:22)
You know, I love this one because it is so simple. Usage begets usage. It is as simple as that. If you use AI every day, you understand better what it is able to do. Then you start setting it more complicated tasks and using it more, and that in turn unlocks more understanding of sophisticated use, and so on and so forth.

The data really shows this, not only in law but generally. If you use AI every single day, you will ultimately become a superuser. Whether that means you will become someone who uses

Marlene Gebauer (25:51)
Yeah.

Nikki Shaver (26:00)
workflows or builds workflows, I do not think everyone is set up to be that person necessarily, and that is okay. But I think one thing firms can do is focus on making sure, and incentivizing people, to use this technology in some way every day.

Frankly, I would not be terribly prescriptive about it. I would be encouraging of usage in any way, whether that is, “Where should I order lunch?” or, “What am I going to do this weekend?” No matter what it is, use it every single day for 10 to 15 minutes. Gradually, you will build superusers.

Marlene Gebauer (26:42)
I am reading a lot about how adoption is not enough. I agree with you. I think you have to start with adoption and then move from there. But from a firm perspective, they are looking to figure out how use of AI is promoting value. By value, I mean better client relationships or new ways of pricing that make sense. They are looking to see some impact beyond the actual use.

Is it enough to be a power user, or is the new benchmark either developing workflows or having systems of practice that are powered by AI?

Nikki Shaver (27:31)
I think there are two levels there. One is individual use. Being a power user is great. The second is whether firms should be focused solely on adoption of third-party technology, or whatever you have rolled out across the firm. There, I think the answer is no.

Something I have been arguing for over a year now is that, yes, definitely focus on adoption. Make sure that you have a full firm of people who are AI literate and use AI on a daily basis. That is really important and will continue to be important. You need data enablement as a foundation. You need AI enablement. Driving adoption in a widespread way gets you there for the latter.

But it is not sufficient if you want to use AI to create a point of differentiation in the market that allows you to have a competitive advantage. Everyone is driving adoption of AI across their lawyer and staff population. It is not distinctive. It is not differentiating.

Instead, what firms need to be doing is identifying, and they will have done some of this work already, the key practices. Where is it that you already have an edge? Where is your expertise special, unique, and different? Then, how can you use AI in a way that is not about individual users deploying it in workflows, but is truly about transforming and using AI in an operational way that is embedded with clients, or in the bridge between the firm and clients?

How do you completely change the way that work is done and amplify it, acting as a catalyst so that you are able to drive business growth seriously in the areas where it matters for your firm? That is the work firms should be doing at the moment, figuring out where they want to make that kind of investment and moving beyond adoption of the AI tools they have licensed.

Greg Lambert (29:37)
Yeah. We had Stephen Costigan on last week, and he talked about making sure you own your data, that you are not essentially farming your data out to third parties. But I also think, especially since the government stepped in and had Claude remove its latest and greatest model, Fable Five,

Marlene Gebauer (29:38)
Agreed.

Greg Lambert (30:04)
and now with ChatGPT 5.6, we are getting back into build versus buy. Do we rent the foundational model, or do we do what it looks like Kirkland & Ellis is doing and create a foundation where we at least partially control the AI foundational models, whether that is open source or whatever the next phase is?

What are you seeing? What are your thoughts about how we, one, protect our data so that the things that make us unique and our expertise stay that way? And two, what about the technology? Do we continue to rent the technology, or do you think we need to own at least a piece of it going forward?

Nikki Shaver (31:01)
You need both. I think the dichotomy that we used to hold as established, the buy versus build dichotomy, is no more and has been no more for some time.

Talking about renting, it is still the case that the market is not settled around legal AI. Will it even be legal AI? Will it be something else? We do not know. So, renting in some ways makes sense when it comes to third-party technology. Think along the lines of staying nimble. That is why I advocate that every firm should have an R&D department, like most successful businesses have had for decades, because you need to be able to experiment, understand what is coming down the track, and remain agile for the foreseeable future.

I do not think this escalation and evolution of technology is going to end. On the other hand, for the kinds of embedded operational uses of AI that I was talking about previously, you do need something proprietary. I do not think it is only about protecting your data. Yes, we all do that whenever we work with technology. I think it is more about embedding your data within technologically driven, continuous workflows that allow the firm to develop something quite unique.

That kind of thing has to sit alongside the renting you are doing of other technology tools.

Marlene Gebauer (32:38)
Historically, lawyers have worked in a black box. They deliver finalized, static documents to clients. Even with electronic delivery, that did not change much. We sent the client an email with attachments. We made the documents and sent them along.

We have had some collaboration tools that have been popular, like things such as HighQ. But again, this is where we would share and organize static documents. Now, you have tracked the rise of a new AI-mediated collaboration layer where clients and outside counsel are working dynamically in shared AI workspaces. First, how does that environment work? Second, how important is this to retaining clients and partners at firms?

Nikki Shaver (33:30)
This is going to be something we see more of, for sure. You can imagine that it cuts down on the turns, right? As you said, Marlene, it was kind of stop-start. You send something to me, I review it, I redline it, I send it back to you, and you look at it.

It cuts all of that out effectively if you are working in a secure space where both of you have access to a document in real time and access to AI in real time that is able to support you in the work each side has to do within a document or deliverable.

It sounds facile, but it is like the next iteration of a client portal, one that does change the way you work with clients because it is in live time rather than in staged forms or phases.

Marlene Gebauer (34:12)
What?

Greg Lambert (34:21)
Well, Nikki, here is the most exciting part that I want to talk about, your recognition in the inaugural Financial Times Law 50. I will say, not quite the same thing, but I was an inaugural member of the Fastcase 50. That was a long time ago. I think that was 17, 16, 15 years ago. Time flies if you hang around long enough.

Nikki Shaver (34:38)
Congratulations, Greg.

Greg Lambert (34:47)
But it is interesting that the Financial Times jumped in and was able to recognize the legal industry in itself. I found that very exciting. Tell us about it and how that went.

Nikki Shaver (35:05)
Well, thank you very much. I really applaud the Financial Times and also Rena Sengupta from RSGI Consulting. Rena and the FT have been working together for years on the Innovative Lawyers Awards, which many firms and legal departments are aware of and have participated in. Rena, I think, really drove this.

I love the background. She understands, like all of us do, that we are currently in a moment where the legal profession faces unprecedented uncertainty about its future. What she saw in the FT Law 50 was the ability to recognize people who were courageous enough and outspoken enough to have an impact on driving the profession forward in some way.

It is broken down into different categories, AI and the rule of law. There are leaders who have stood up in various ways for the rule of law, which has been incredibly important in the past year and a half especially. The category I was in was leaders and advocates, which is really about speaking out.

The idea Rena has, which I love, is like imagining bright lights, pinpricks of light around the world, and knitting them together in a way where they can have greater impact because they know one another, are united, and can work together to drive broader impact.

Each year, this will happen again, with the idea that it is not merely a static recognition but something that can have a real ability to drive positive change in the legal industry, which is something I have cared about deeply for many years.

It is a wonderful recognition. The people on this list are quite extraordinary. I do not know if either of you watched Neil Katyal’s TED Talk.

Greg Lambert (36:59)
Yeah. That was good.

Nikki Shaver (37:01)
Really amazing. I think it is a wonderful initiative, and I hope we see more and more of this kind of thing. It is almost about activism, right? It is about recognizing that, when our industry is in this kind of moment, we need to think about things differently and encourage other people to think about things differently as well.

I speak a lot at the moment about law being in the dilemma zone, the classic Clayton Christensen dilemma. I think we are in that moment where a lot of incumbents, unless they start acting differently now, will fail as nimble disruptors come up. We are seeing that happen in the market right now.

It is an important time for all of us to do what we can to drive our institutions forward if we are keen to see them survive in some way, shape, or form.

Greg Lambert (37:51)
Did you think, when you started Legal Tech Hub, because I knew you when you were at Reed Smith and then moving on, that the original idea was, “I do not have a way to keep track of all the things that are going on”? But Legal Tech Hub has become much more than a tracking tool. It is almost like a community now. Has it exceeded your expectations, or are you right on track where you thought you would be?

Nikki Shaver (38:24)
Definitely.

Honestly, I was frustrated. It was born out of frustration. I said to Chris, who many of you know, my husband and also one of the co-founders, “Why is this so hard? Why does this not exist? It is so annoying that I cannot find this information.”

So, that is what it was born out of. I never could have imagined. I think what is interesting, though, Greg, is that it is called Legal Tech Hub, but I now refer to it more as an intelligence platform because it is very much about market analysis. The work I increasingly do is also about market analysis and the

Greg Lambert (38:40)
Why is it so hard?

Nikki Shaver (39:04)
strategic work that we have to do as an industry. I think the reason for that shift is that when you looked at technology in 2022, pre-November 2022 and before, it was a support function. It was not the pointy end of strategy for law firms.

But now, those of us who work in AI and legal tech for legal practice are working in something that has the ability to shape strategy for firms and have a real impact on how law firms, as businesses, grow or fail.

I think that is why the platform that we started as something for information about legal tech has moved into something more about strategic industry analysis. A lot of people who work in this area are finding themselves doing more strategic work than they might have a few years ago, including probably both of you, I imagine.

Greg Lambert (40:05)
No, we are right on track where we thought we would be. We are still trying to figure out how to hook up our microphones.

Marlene Gebauer (40:07)
Right on track.

Nikki Shaver (40:07)
Please.

Marlene Gebauer (40:16)
So, Nikki, before we get to the crystal ball question, you are someone who tracks the entire global legal tech market, as we were just talking about. What are one or two must-read resources, thinkers, or groups that you rely on to stay ahead of the curve in legal AI and innovation?

Nikki Shaver (40:38)
It is interesting to me that it is so disparate.

There is no single source. You really have to follow a lot of different people. Greg, I absolutely love the fictionalized updates you are putting out at the moment about AI developments. I think that is great. I love Stephanie Gutos’s Legal AI Bites that she puts out. I actually have this right here, Agentic AI by Dan Katz and the Bommaritos. They are sources of inspiration for me.

I follow Ethan Mollick, who I think is a wonderful outside-of-legal thinker. I like Patrick Fuller. If you ever get the chance to read what he has written and see what he says, he tracks a lot of data that I find interesting. Dazza Greenwood and Megan Ma out of Stanford and MIT, I find really interesting too.

There are lots and lots of people. One thing I find particularly interesting is that with X, formerly Twitter, I think we are seeing a bit of a resurgence there. There are some interesting things happening. So, anyone who stepped away, I do not know. I think it is about to become interesting again.

Greg Lambert (41:38)
Yeah.

I think so too. I do not like it either, but it is what it is, I guess.

Nikki Shaver (41:49)
It is what it is, exactly.

Greg Lambert (41:53)
All right. Well, Nikki, we are at the crystal ball question. As agentic AI and real-time client collaboration become table stakes, what other shifts do you think are coming that we need to be aware of?

Nikki Shaver (42:16)
I think if you look at the history of law firms and what they offered, the scarcity was lawyers’ time. Increasingly, the thing that is or will become scarce and valuable is trusted judgment. I think that is something we will be able to charge a premium for.

We will see some firms fail. I really believe that is going to start to happen, unfortunately, in the next little while. We will see the increasing rise of AI-native firms, especially ones that are product companies turned services companies.

I think we will see some law firms move from being partnerships, legal partnerships, to businesses where the people who run them are leading multidisciplinary teams who serve clients holistically around strategy and law, but where law is one part of the service, not the entirety of it.

One thing I will put out here, because I knew you were going to ask me the crystal ball question, is something I think will happen. We have had law firms with subsidiaries. I think we are going to see a traditional law firm open a subsidiary business that is an AI-native law firm.

Greg Lambert (43:37)
Makes sense when you think about it.

Marlene Gebauer (43:39)
Yeah, it does.

Nikki Shaver (43:39)
Yeah.

Marlene Gebauer (43:40)
It does. It does.

Greg Lambert (43:41)
But, again, it really helps to hear it out loud. All right. Well, Nikki Shaver, thank you very much for taking the time to join us. I know you are super busy. You have been across the globe literally the last couple of weeks. I am glad you were able to sit down and talk with us. Hopefully, you can catch your breath and take a little time to yourself.

Nikki Shaver (43:45)
Yeah.

Marlene and Greg, it is always such a pleasure, really always a pleasure speaking with you. Thank you for having me.

Marlene Gebauer (44:13)
And thanks to all of you for listening to The Geek in Review. If you have enjoyed the show, please share it with a colleague. We would love to hear from you on LinkedIn and Substack.

Greg Lambert (44:22)
I always love it when Marlene goes into a radio voice for this one. So, Nikki, where is the best place for listeners to learn more about you and find out more about Legal Tech Hub?

Marlene Gebauer (44:25)
Okay.

Nikki Shaver (44:25)
You can find me on LinkedIn, Nicola Shaver, N-I-C-O-L-A. You can find Legal Tech Hub at legaltechnologyhub.com.

Marlene Gebauer (44:41)
And as always, the music you hear is from Jerry David DeCicca. Thank you. Bye, everybody.

[Ed. Note: This is part of an ongoing series at The Geek in Review Substack page. – GL]

Leo Huang did not usually come to the twelfth floor, and he never came to it at seven-forty in the morning. So when Cooper looked up from his coffee and found him in the doorway, still in yesterday’s shirt with a laptop held against his chest like a clipboard, he could tell something was going on, and from the look on Leo’s face, probably something bad.

“I think we almost sent a client something we shouldn’t have,” Leo said. “I think the intake agent cleared it, and I think it’s been clearing things it shouldn’t for a while, and I can’t find anyone whose job it is to care.”

Cooper put the coffee down. “Sit. Start at the beginning, slowly. What got cleared.”

Leo set the laptop on the desk and turned it. On the screen was the intake queue, the one the business-intake team and the junior associates worked off of every morning, the inbound non-disclosure agreements and engagement requests that the firm’s intake agent read overnight, classified by risk, and stacked into three colors. Green for standard, clear to send. Yellow for a human look. Red for a partner. The agent drafted the first-pass response too, so that by the time a person opened the item, there was already a reply sitting under it, polite and competent and ready to go out.

“This one came in Monday,” Leo said, and opened it. “Counterparty NDA, mutual, looks like every other one. The agent tagged it green. Standard, clear to send. The response was already drafted. Someone on intake would have sent it this morning, today, because that’s what you do with green, you send it.” He scrolled. “But I was staffed on the underlying matter, so I actually read the thing. And buried in the confidentiality section there’s a non-solicit. Twelve months, their employees and ours. That’s not a confidentiality term. That’s a restrictive covenant riding inside an NDA, and we have a house rule that any embedded non-solicit goes to a partner, because Maya made it a rule after that mess last year.”

“And the agent missed it.”

“The agent didn’t miss it. That’s the part that kept me up. It saw the non-solicit. It just decided it was boilerplate. Because the playbook it’s reading still says standalone non-solicits in NDAs are standard market and don’t need escalation. Which was true. Last spring. Before the rule changed.”

Cooper felt the morning go quiet around him.

“So I went looking for the playbook it’s reading,” Leo said. “It’s pointed at a document in the old KM folder. The one we superseded in March. The current house position, the one with the non-solicit rule, lives in a different place now, and nobody ever told the agent. It’s been reading a dead document with total confidence since the spring. Every green tag it’s handed out since March, it handed out from a rulebook we threw away.”

“How many is that.”

“I don’t know. That’s the other thing.” Leo looked at him. “I went to find out who owns this agent so I could tell them. There’s no one. It was a pilot. I built the clause-extraction piece, the part that pulls terms out, eighteen months ago, but I don’t own the thing it became. Somebody in KM stood up the production version, and somebody used to check it on Fridays, and I went and asked, and the person who checked it moved to the Houston office in the spring. The Friday review went with her. Except it didn’t go anywhere. It just stopped.”

Cooper was already standing, already reaching for the marker he kept by the door.

“Don’t send anything green until I get back to you,” he said. “I’m getting the room.”

• • •

He got the room by ten. Not the firefighting version from the Friday in June when a government had reached across an ocean and switched off a model while everyone watched from their kitchens. The deliberate one, Governance Committee on twelve, door closed, three hours blocked. Nora came up from the administrative floor with her iPad and the look of someone who already suspected she would not enjoy the next three hours. Maya came in between a deposition and a closing, reading glasses pushed into her hair. Jesse joined from his lab in Chicago, a window on the wall screen with whiteboards behind him. And Arthur joined the way he joined everything now, a calm rectangle in the corner, the leather spines of his home study at his back.

Cooper wrote one sentence on the board, not a phrase this time but a full sentence, and stepped away so they could all read it.

“That’s not mine,” he said. “It’s been going around. There’s a briefing the partners have all been forwarding each other since the weekend, about teams running agents that nobody owns. I read it twice and then Leo walked into my office this morning and handed me the live version of it.” He told them the NDA. The non-solicit riding inside the confidentiality section. The green tag. The drafted reply that would have gone out today. The playbook the agent had been reading since March, the one the firm had superseded and the agent had never been told about.

“For most of June,” he said, “we were worried about the wrong direction. The Thirty-Day problem, the Friday a model disappeared, the half-billion-dollar headline, the Chinese weights. All of it was about intelligence we rent from outside, and whether it would still be there, and whose hand was near the switch. Every one of those was about something coming at us from the outside. This is the opposite. This came from inside the building. This is an agent we built, that works, that everyone trusts, quietly handing out wrong answers from a rulebook we threw away in the spring, and the reason nobody caught it is that nobody’s job was to catch it.”

“We called last week’s problem the ownership problem,” Maya said. “Owning the model versus renting it.”

“Different word, same week, I know. That ownership was about whose balance sheet the intelligence sits on. This one’s smaller and closer. It’s about who’s responsible for one agent doing one job, today, correctly. Owning the model is a strategy question. Owning the agent is a Tuesday question. We’ve been answering the first one all month and we never asked the second.”

• • •

Jesse had his hand half up in the video window, the old classroom reflex.

“Can I do the part that’s going to feel like a coincidence and isn’t,” he said. “Because the machines already solved the thing you’re describing, for themselves, and watching them solve it is the fastest way to see what you’re missing.”

“Go.”

“When two of these agents need to work together, one has to know what the other is and how to talk to it. So there’s a standard for that now. Google put out a protocol, agent-to-agent, and the heart of it is a thing called an Agent Card. It’s a small file, published at a predictable address, slash dot-well-known slash agent dot json, and it says, in plain machine-readable terms, here is what I am. Here’s my name. Here are the skills I offer. Here’s the kind of input I take and the kind of output I give back. Here’s how you authenticate to me. Another agent reads that card and knows exactly how to delegate work to the first one, or knows not to.” He paused. “Anthropic’s protocol, the one we already use, the Model Context Protocol, does the same thing one layer down for tools. A server publishes a machine-readable description of what it can do, so an agent can discover it. The whole ecosystem now runs on the principle that nothing autonomous should be a black box to the things that depend on it. Every agent hands out a card that says who it is.”

Cooper saw it. “And the humans don’t have one.”

“The humans don’t have one. Your intake agent could publish a perfect Agent Card tomorrow. It could tell another machine exactly what it does and how to call it. What it cannot tell anyone is who at this firm is responsible for it, what it’s reading, whether what it’s reading is current, and the last time a human looked at its work. The machines built themselves identity documents so they could trust each other. We never built the human version. The briefing’s whole idea, the thing it calls an Agent Owner’s Card, is just that. The card the machines already give each other, rewritten for the person who has to answer for the agent.”

“So the gap isn’t technical,” Nora said.

“The gap is the opposite of technical. The machine half is solved. The card the agent hands another agent is solved. What’s missing is the card a human holds. And you can’t generate that one out of the model, because most of what goes on it is a decision only a person can make.”

• • •

Nora had been waiting with the patience of someone who knew the arithmetic was coming to her, and she put the intake agent’s dashboard on the wall screen.

“Before anyone decides this agent is broken, look at how healthy it is,” she said. “Volume is up forty percent since the pilot. Average turnaround on an inbound NDA went from two days to under four hours. Error rate, by every flag we log, is zero. If you ran this agent’s numbers in a board deck, it would be the best-performing thing the firm operates. Green across the board. And it has been handing out wrong classifications for three months.”

She let that sit.

“That’s the part I want everyone in this room to feel, because it is the dangerous part. Nothing on this dashboard is lying. The agent is fast. It is consistent. It does not crash. It does not throw an error, because from its own point of view it is not making one. It is applying the rulebook it was given, perfectly, to every document, on time. It is operationally flawless and behaviorally wrong, and there is no metric on this screen that can tell the difference between those two things. The only thing that could tell the difference is a person reading the output against the current rule. And we stopped doing that in the spring.”

“Walk the failure,” Cooper said. “All of it. Leo found the first piece. What’s underneath.”

Nora moved through it the way she moved through everything, one row at a time.

“Three failures, and they stack. The first is the one Leo found. The source it reads is stale. It’s pointed at a playbook we superseded in March, so its facts are three months out of date and it cites them with full confidence. Fix the pointer and you fix today’s NDA. You do not fix the other two.

“The second is the brief. The agent’s standing instructions, the prompt that tells it how to behave, were written fourteen months ago for the pilot. They still say to treat the old standard as the default. And on top of that they’ve silted up. Every example we ever fed it during the pilot is still in there, dozens of sample contracts that no longer match our house style, all of it still sitting in the agent’s context competing for attention with the instructions that matter. It isn’t failing because it knows too little. It’s failing because it’s carrying fourteen months of accumulated junk and can’t tell what still counts.

“The third is the one that should bother us most, because it’s the one that let the other two survive. Nobody is reading the output. When the agent was new, the KM associate who stood it up sampled it every Friday. She moved to Houston in the spring, and the review didn’t transfer to anyone. It simply ended. The field on my screen for last human review is blank. Not old. Blank. We have an agent the whole intake team trusts on sight, and the last time a human actually checked whether that trust was earned, I can’t give you a date.”

She looked up from the iPad.

“And while I have the floor. This is one agent, the one Leo happened to be staffed against. I run six workflows on the administrative floor right now that look exactly like this from the dashboard. Green, fast, unattended. I am not telling you any of them are wrong. I’m telling you I don’t currently have a person I could name for most of them either. We have been very proud of how many of these we’re running. We have not once counted how many of them anyone owns.”

The room was quiet. Cooper wrote, on the side of the board, operationally flawless, behaviorally wrong, and under it, smaller, the blank field is the finding.

• • •

Maya took the glasses out of her hair and folded them, which was how Cooper knew she was about to say the thing that mattered.

“I want to name who owns this one, because the answer tells you the whole principle,” she said. “It is not Leo. Leo built a part of it and Leo is the reason we’re not sending a defective NDA this morning, but Leo is a second-year, and the agent’s mistakes don’t land on his desk. It is not KM, who stood up the production version. It is not IT, who keep it running. It is not this committee. The owner of that intake agent is whichever partner would have to call the client and explain why we let a twelve-month non-solicit go out under our name as standard market. That’s me, on this matter. The owner is the person who eats the consequence.”

“Ownership follows the consequence, not the keyboard,” Cooper said.

“Ownership follows the consequence. The person who wrote the prompt can tell you what the agent was supposed to do. Only the person on the hook for the outcome will actually notice, and care, when it stops doing it. That’s why you can’t hand this to the people who built it and call it owned. They’re not the ones who get the phone call.” She put the glasses down on the table. “And here’s the part that makes it hard, the part the briefing got right. A good agent’s entire purpose is to earn the right not to be checked. We want intake to trust it on sight. That’s the win. That’s why we built it. And trust-on-sight is the exact thing that removes the person who would have caught the drift. The better this agent gets, the more dangerous it becomes, because the reward for being good is that everyone stops looking. We’ve seen this movie. We called it verification drift two weeks ago when it was about lawyers getting lazy with a first draft. This is the same disease one level up. It’s the firm getting lazy with an entire agent.”

“So the answer can’t be to trust it less,” Nora said. “The whole point is to trust it.”

“The answer is that somebody has to be assigned to keep looking precisely after everyone else has earned the right to stop. On purpose. As a job. Not because they distrust it. Because they own it.”

• • •

Cooper went to the clean half of the board.

“Then let me build the human version of Jesse’s card, out loud, for this agent, so we can see what it costs us to actually know what we’re running.”

He didn’t draw boxes. He talked it onto the board the way Maya talked her diagrams onto a page, a field at a time, and the room filled in the answers as he went.

It would start with identity, he said, the easy part, the part the machine could write itself. What the agent does, in one sentence. Reads inbound NDAs, classifies risk, drafts the first response. Then the part the machine couldn’t write. The owner. One name. The partner who eats the consequence, which this morning meant Maya. A backup, for when the owner was in a closing or on a plane, because an agent whose only owner is unreachable for three weeks in August is an ownerless agent for three weeks in August.

Then scope, he said, and this was where it stopped being comfortable. Who actually acts on the output, and do they re-check it or trust it on sight. For the intake agent the honest answer was that the whole intake team trusted it on sight, which was the win and the exposure in the same line. And reach. What real systems it touched, what it could set in motion if it went wrong. An agent that drafts a reply a human sends is one blast radius. An agent that sends the reply itself is another. You wrote down the blast radius so you knew how hard to hold the thing.

Then the inputs, he said, and looked at Leo. What source material it depends on, and when somebody last confirmed that source was current, and who that somebody is. The blank where the freshness date should be was the whole reason they were in the room. Then the brief. Where the agent’s standing instructions live and when they were last reviewed, because a fourteen-month-old prompt was its own kind of stale source.

And then the last block, he said, the one that mattered most because it was the one that had failed silently. The review loop. How often, and by whom, the output gets sampled. The actual date it last happened. And a line for the condition that would make the owner pull the agent offline, and who was allowed to pull it. He underlined the date field twice.

“That field,” he said, “the last-reviewed date. On this agent it’s blank. That blank is not missing data. That blank is the finding. A card where ‘owner’ is empty means you’ve found an agent nobody owns. A card where ‘last reviewed’ is three months old on an agent the whole team trusts on sight means you’ve found a dead review loop before it cost you a client. The empty fields are the entire value of the exercise. You don’t fill the card out to look organized. You fill it out so the holes show you where you’re already exposed.”

Jesse, from Chicago, said it plainly. “That’s the card my agents already hand each other. You just wrote the half that has a person’s name on it.”

• • •

Cooper looked to the corner of the screen. “Arthur. You’ve gone quiet on us.”

Arthur set down his pen. The small sound of it carried over the link, and the table came to rest around it, the way water settles.

“I have been listening for the line this committee keeps blurring, because the blur is the most dangerous thing in the room, and I have watched it blur twice this month,” he said. “Let me draw it clean.”

He let it settle.

“What we wrote down in March, the framework, the access controls, the question we now ask every vendor before we sign, that is governance. Governance is the road. It sets the rules everyone drives by. What an agent may touch, what gets logged, what is forbidden, what requires a partner. A committee writes the road, and a committee should, and we have, and it scales across the whole firm, and it is necessary. None of what I am about to say is an argument against it.

“But a road does not drive the car. What failed this week was not the road. The rules were fine. The failure was that one particular car had been traveling that road for three months with no one behind the wheel, and the road cannot tell you that. Governance can prohibit a thing and still have no idea whether a permitted thing is being done well, by whom, or whether it stopped being done well in March. That is what ownership is. Ownership is one named person who reads what this agent produces and acts when it drifts. A committee cannot do that, because a committee does not read the output. Twelve people sharing the watch is twelve people each assuming one of the other eleven has it. That is not oversight. It is the absence of oversight wearing the costume of a meeting.”

“Governance without an owner,” Cooper said.

“Is theater. Well-intentioned, well-documented theater. It can prevent some bad outcomes, and it cannot tell you why a good outcome was permitted, or trace what went wrong when something fails, because there is no one standing close enough to the work to know. We have spent the month being careful about the road. The car still needs a driver, and the driver has to have a name, and the name cannot be the committee’s.” He paused. “Leo did the committee’s job this morning, by the way. Not because it was assigned to him. Because he happened to be close enough to the work to see it. We do not get to rely on happening to be close. That is the whole point.”

• • •

“Decisions,” Cooper said, and capped and uncapped the marker the way he did when he wanted them short.

“One. We run the test on everything that’s already running. Every agent in production, practice side and administrative side, gets the one question put to it. Does anyone act on this agent’s output without re-checking it. Every place the answer is yes is an agent that needs a named owner, and it needs one this week, not after we form a working group to think about it.”

“Two. Every owned agent gets the card. Not the machine’s Agent Card, the human one. Name, owner, backup, who consumes it and whether they trust it on sight, what it reaches, what it reads and how fresh that is, where its brief lives and when it was last reviewed, the review cadence, the kill condition. The human counterpart to the file the machines already publish to each other. We are the only ones in the loop without an identity document. We fix that.”

“I can have the agents draft their own cards by Friday,” Jesse said from the screen. “Descriptive fields filled in from what each one can see about itself, and the owner line and the dates left deliberately empty. The blanks come back to you as the worklist. The machine writes the half it can observe. It can’t sign the half that has a person’s name on it, and it shouldn’t try.”

“Three. The blank field is a finding, and we treat it like one. A blank owner or a blank last-reviewed date doesn’t sit quietly on a card. It goes in the risk register as an open item with a name attached to closing it. We were excellent this month at registering risks about models we rent. This is a risk about agents we built, and it gets the same ink.”

“Four.” He looked at Maya. “Ownership follows the consequence, not the keyboard. The owner of an agent is the person accountable for the outcome it produces, not the person who wrote its prompt and not the department that keeps it running. If we cannot name a person who eats the consequence, the agent should not be trusted on sight by anyone, and that is not a punishment, it’s a definition.”

“Five. The review loop is a scheduled job, not a good intention.” He glanced at Nora. “Whoever owns an agent runs a standing review of it on a cadence, monthly as the default, tighter for anything with a large blast radius, and the review checks the three things that failed here. Is the source still current. Has the brief drifted or bloated. When did a human last actually read the output. The failure this week depended entirely on nobody looking. We make looking a job with a date on it, so it can’t quietly end when someone moves to Houston.”

He held the marker out toward the screen. “Six is yours, Arthur.”

Arthur’s tile was still for a moment.

“Six is the sentence the partners will keep, so make it the true one. We are not safer because we run more agents. We are safer when each one has a name attached to it. The committee writes the rules of the road, and that is real work, and we have done it. But every car on that road needs one person behind the wheel who will notice when it drifts and is allowed to stop it. An agent with no owner does not announce itself by crashing. It keeps running, beautifully, on trust it earned under conditions that stopped being true in the spring, and the value drains out of it so slowly that the dashboard stays green the whole way down. The work this week is not technical. It is the oldest work we do. Putting a name next to a responsibility, and meaning it.”

Maya was writing. Cooper could hear the pen.

• • •

They broke a little after one. Nora went down to start the count, agent by agent, on the administrative floor, the look on her face that of someone who already knew the first column would have more blanks than names. Maya went to her closing, but not before she’d written ownership follows the consequence on the back of her own hand, which Cooper had never seen her do. Jesse’s window blinked out, already muttering about how to make an agent leave its own owner field empty on purpose. Arthur’s tile closed with a small nod.

Cooper didn’t stay on twelve. He took the elevator the other way, down to where the morning had started, and found Leo at his desk with the intake queue still open and the green NDA still flagged where he’d left it at dawn.

“Hold the queue until we’ve repointed it at the current playbook,” Cooper said. “Then you can release the rest. The non-solicit one goes to Maya, with your note attached, and your name on the note.”

Leo nodded, then said the thing he’d plainly been carrying since seven-forty. “I only caught it because I was on the matter. If I’d been staffed somewhere else this week, it goes out and nobody ever knows. That isn’t a control. That’s me getting lucky on a Tuesday.”

“No. It isn’t a control.” Cooper pulled a chair around. “That’s the finding, said out loud. So here’s the first card, and we fill it out together, because you know this agent better than anyone who was in that room. We’re not writing it to look tidy. We’re writing it so the holes show. And the first hole has your fingerprints on it, because when you went looking for who owns this thing, you came back with nobody.”

Leo opened a blank document and typed the agent’s name across the top. Under it he set the cursor on the line for the owner, the line he hadn’t been able to fill at dawn, and left it empty, and turned the screen a few degrees so Cooper could see the blank sitting there. Neither of them filled it. The blank was the point. It was a question with a person’s name as its only possible answer, and that name was a decision the firm still had to make on purpose.

Cooper rode back up but stopped at the window at the end of the twelfth-floor hall instead of going into the committee room, and took out the notebook, the same one.

An agent nobody owns doesn’t crash. It keeps running on trust it earned under conditions that no longer hold, and the value drains out so quietly the dashboard stays green the whole way down. Autonomy without an owner isn’t independence. It’s abandonment with good uptime.

He thought about the empty line glowing on Leo’s screen two floors down, and wrote the second.

The machines gave each other a card that says who they are. We never wrote the one that says who answers for them. The empty line isn’t missing information. It’s the haunted house — the lights still on, the work still moving, and nobody home to notice the room went wrong in the spring.

Outside, the early dark had come on and the city had stood up in it, whole floors of the towers lit against the evening over desks that had emptied hours ago. He used to read that skyline as proof the work was getting done. He stood there a while longer than he meant to, counting windows, wondering building by building how many of those burning lights had a name behind them, and how many were only still on.

For law firms, artificial intelligence has often arrived as a choice between speed and control. Stephen Costigan, founder of Atlas AI, argues that choice deserves a rethink. In this episode of The Geek in Review, we speak with Costigan about private legal AI infrastructure, knowledge graphs, and why a firm’s internal work product may become its most valuable long-term asset.

Atlas AI focuses on turning documents, matter history, precedents, clauses, parties, and obligations into a curated legal knowledge graph inside a firm’s own environment. Costigan contrasts this approach with standard vector search and retrieval systems, which find text with similar language but often lack context around clients, matters, entities, and relationships. A knowledge graph offers structure, linking people, documents, clauses, and legal concepts in ways closer to how lawyers understand their work.

The conversation also explores data quality, a subject with enough baggage to fill a records room. Costigan argues firms no longer need year-long cleanup projects before seeing results. Agent-led curation, entity extraction, duplicate resolution, and ontology mapping reduce much of the manual sorting traditionally associated with knowledge management. Human judgment still matters, especially around practice-area vocabularies and lower-confidence results, but the machines get assigned more of the janitorial work.

Security and governance sit at the center of Costigan’s model. Rather than asking firms to trust a vendor’s assurances around privileged data, Atlas AI runs within a firm’s Azure environment, under firm-controlled keys and policies. Costigan frames this as a shift from confidentiality as a contractual promise to confidentiality as an architectural decision. For legal organizations handling sensitive client information, the location of data, embeddings, audit trails, and model interactions matters as much as the interface lawyers see on screen.

Looking ahead, Costigan predicts a divide between firms renting generic AI tools and firms building durable knowledge infrastructure from their own experience. As routine drafting, diligence, and review work compress, firms with structured and reusable internal intelligence may productize expertise, offer new fixed-fee services, and rely less heavily on traditional leverage models. The future question, Costigan suggests, will not center on which AI tool sits on a lawyer’s desktop. The bigger question will ask who owns the knowledge behind the work.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Nikki Shaver (00:00)
Hello Marlene and Greg, coming to you live today from Toronto, Canada. This is Nikki Shaver, the CEO of Legal Tech Hub. I wanted to share something with your audience from the FT Innovative Lawyers Summit last week in London. It was a great event because it brought together people from across legal verticals, including lawyers from law firms, GCs, technologists, people in innovation, and more. One of the panels that really stuck with me was on positive psychology, which emerged as a field in the early 2000s.

A few takeaways from the panel: If you think you can, you can. If you think you can’t, you can’t. There really is something to believing in agency, in your own personal agency. Another couple of things are particularly important as we all look to drive adoption on one hand and increase or maintain engagement among lawyers and employees during this time of unprecedented change and uncertainty in the industry.

Do not let people sit in their little pockets of pessimism. It will spread. Instead, as a leader, one should focus on creating a sense of hope, agency, and a pathway forward. First, create a sense of hope, then provide a vision for the way forward. What is the path forward? Then provide people with a sense that they have agency to drive that path forward.

I love that. I think it is a good thing for leaders to remember at this time, and something for all of us to remember as we encourage people to change the way they work and adopt new technologies, tools, and ways of working. They are much less likely to do so if they do not feel that they have agency themselves. So, leaving you with that today, we will be writing about the FT Innovative Lawyers Summit. Look us up at legaltechnologyhub.com, and you will get a notification when that article comes out. Thank you so much.

Marlene Gebauer (02:26)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:33)
And I’m Greg Lambert, and today we are exploring how law firms can harness their own internal data to power the next generation of legal artificial intelligence.

To do that, I am very happy to welcome Stephen Costigan, the founder of Atlas AI. Atlas AI is an enterprise-grade legal AI platform that helps professional services teams transform their internal knowledge into a powerful private legal knowledge graph. So Stephen, welcome to The Geek in Review. Good to have you.

Stephen Costigan (03:08)
Thanks, Greg. Great to be here.

Marlene Gebauer (03:10)
Stephen, can you start by giving our listeners an overview of Atlas AI and what led you to build a platform focused specifically on firm-hosted private legal AI?

Stephen Costigan (03:20)
Absolutely. I’ll start by saying our platform turns a firm’s documents and matter history into a private, curated knowledge graph that the firm owns and runs inside its own environment. It then puts research, drafting, review, and enterprise search on top of that graph.

What led me to do this is that I have spent years building software inside demanding enterprises, including elite law firms. Most of my experience is in the AmLaw 50 space, and I kept seeing the same trap. Every AI tool asked a firm to choose between productivity and control. You either get speed by shipping your most sensitive work to someone else’s cloud, or you get control by building something you can barely operate. For a profession whose entire value rests on confidentiality and privilege, that is not a real choice.

Greg Lambert (04:19)
Yeah, and I have a side question on that. Do you think the foundational companies, Gemini, Claude, and ChatGPT, really understand the legal market and how sensitive the data is that we have?

Stephen Costigan (04:39)
Well, I think a firm should be able to use frontier AI without its knowledge ever leaving its walls. The firm’s knowledge should become an asset that it owns, not fuel for an outside vendor’s model.

Greg Lambert (04:54)
So, Stephen, when you and I were prepping for this and had a couple of conversations, you really got me thinking. I have been thinking about this for a while, but it was good to talk with somebody who is actually doing it about the underlying architecture of the knowledge graph systems you have developed at Atlas AI.

For the non-engineering types listening to this, can you explain what a legal knowledge graph is, why it is a better foundation than dumping documents into a vector database and creating a basic RAG system, and what benefit the knowledge graph provides?

Stephen Costigan (05:42)
Sure. I’ll start with an analogy. A standard vector database is like a clerk who hands you the pages that sound most similar to the question you asked. It is fast, but it has no understanding and very little context. It does not know that the company in one agreement, ACME Holdings, and the company in another agreement are the same client.

So it only knows which pages use similar words. A knowledge graph is the opposite. It is a map of your knowledge where the things that matter, clients, matters, parties, clauses, and obligations, are represented as connected entities. The graph knows that Client A has these matters, each matter has these documents, and each document contains these clauses and parties.

It captures the relationships, not only the text to which it refers. And really, AI is only...

Greg Lambert (06:45)
Yeah, I wrote an entire section of a book on this late last year. It was interesting because, in the legal industry, we all started with vector databases and thought, “This is great because it knows the kind of similar words that we can look for.” It was like the natural-language search we were promised in the early 2000s and that never really worked out.

Stephen Costigan (07:18)
Right.

Greg Lambert (07:24)
It would be great if law were flat, if our business were flat and every document were equal to every other document, or every client were equal to every other client. But it is a complicated batch of information that we deal with. I think even the legal research vendors have finally figured out that it is not flat information. There is nuance, and there are levels to it.

I geek out a little bit when we start talking about adding knowledge graphs to data. But it is interesting that you are doing that with internal information. That is something most people do not think about.

Stephen Costigan (08:01)
Right. AI is only as reliable as what you ground it in. Even with traditional search, adding vector search, adding a knowledge graph, and bringing all of those together into a hybrid search pipeline, you are still not going to achieve the level of accuracy that law requires.

That is why we have added an additional layer, which focuses on bringing the ontology into the index. We bring the firm’s ontology into the index, its information architecture, if you will, in a traditional way of describing it, and then use agents to map that data. At a high level, that is what we are doing, and that is why we are seeing big increases in accuracy when drafting and performing enterprise search.

Marlene Gebauer (08:54)
So, speaking of accuracy, these tools tend to work better when they have clean data. How much work does a firm need to put into cleaning and curating its precedents, templates, and contracts before Atlas AI can generate reliable insights?

Stephen Costigan (09:13)
I would expand this beyond Atlas AI and say far less than people fear. This is the part that many firms get wrong about AI projects. They assume a year-long data-cleaning program is necessary before any value is derived. That is not really the case anymore. Agent-led curation, as we like to frame it, is speeding up the process.

Greg Lambert (09:14)
Is the internet out?

Stephen Costigan (09:40)
Curation can now be automated and continuous. In our platform, a component we call the Librarian runs over every document as it comes in and extracts the entities and relationships. It resolves duplicates, reconciles the same party appearing in different forms, and maps everything to the firm’s ontology. This is a controlled vocabulary for legal concepts, and the graph cleans itself as it grows.

It connects to iManage and SharePoint and builds the structure automatically. What the firm contributes is judgment, not janitorial work. That is what we are removing from the equation. Sorry, go ahead.

Marlene Gebauer (10:21)
Yeah, because you would still have to determine, even if you have the infrastructure in place, which documents are important versus others.

Stephen Costigan (10:32)
Exactly. That is now possible through automation. I would say the firm needs to define and approve the ontology for practice areas and maintain a light human-in-the-loop review queue, where the system can flag lower-confidence data extractions before they are committed to the graph. That allows the firm to govern quality.

The firm does not need to hand-clean precedents anymore. I am not getting into a ton of detail about our product, but at a high level, that is what you are able to achieve now, and it is pretty incredible.

Marlene Gebauer (11:12)
And this will deploy directly into the firm’s infrastructure, such as its Azure environment.

Stephen Costigan (11:21)
Yeah.

Marlene Gebauer (11:21)
I think that is key for large law firms because of the security question. How are you seeing this change the conversation around client data security and firm governance?

Stephen Costigan (11:30)
It is changing confidentiality from a promise into an architecture. That is the best way I can describe it. In a standard SaaS platform, you are trusting a vendor’s contract. You are signing something that basically says, “Trust us with this privileged client data. We adhere to all these controls. We have enough funding. Trust us.”

Greg Lambert (11:48)
All right. Dive deeper into that.

Marlene Gebauer (11:52)
Sounds good. Tell me more.

Stephen Costigan (12:13)
“We can handle this. There will not be a data breach. And we have indemnity clauses to back it up if something happens.” That does not change anything. Client data is exposed. Trust us that we will handle it correctly. Trust that it does not train a model. For a firm with duties to its clients, trust is a weak control.

That has been our thesis since the beginning. We started as a plain private version of ChatGPT, with a few legal twists in our prompt library. Now we have expanded to cover many different focus areas and features. But with Atlas AI, the platform runs inside the firm’s own Azure environment under the firm’s keys.

The data, the graph, the embeddings, and the audit trail, none of it leaves the firm’s environment.

The models are accessed under zero-data-retention terms, so nothing is retained and nothing trains a third party’s system. That flips the governance conversation entirely. Instead of asking, “Can we get comfortable with the vendor’s data-handling procedures?” it becomes, “Can we show our general counsel and conflicts partner exactly where the data flows?” The answer is yes, because it never leaves. That is why firms will run their most sensitive matters on our platform.

They will run them on private AI infrastructure because sovereignty is total in that configuration.

Greg Lambert (13:50)
Interesting.

I have a saying that I use probably a little too much now: Lawyers tend to do better with a red pen than a blue pen. They like to have something to edit rather than create whole cloth. They do their best work when they have a solid first draft to edit and refine.

How do the agentic workflows in Atlas AI provide attorneys with that critical first draft for complex tasks such as due diligence or bulk contract review? What benefits let them dive in much faster?

Stephen Costigan (14:35)
Right. I love that framing. At a high level, we give the lawyer a first draft that has already been argued against by a second, adversarial system. They get to do their best work, the red-pen work, instead of assembly. I can go into more detail if you would like.

Marlene Gebauer (14:55)
Yeah, please do. Go ahead.

Greg Lambert (14:57)
Yep, please do.

Stephen Costigan (15:00)
What I mean is that the system is not there to replace a lawyer’s judgment. It is there to deliver a strong, cited first draft. The lawyer spends time editing and deciding, not assembling.

For due diligence, an attorney points the system at a deal document set, and it produces a diligence checklist or an issues list. Every line item carries an inline citation to the exact subsection from which it came, not simply Section 3, but Section 3.2(a).

For bulk contract review, you define the question once, such as change of control, governing law, or termination, and the system extracts structured answers across hundreds of documents into a reviewable grid. Each cell is traceable to the source. The part that makes the draft trustworthy is an adversarial verification step. One model drafts, and a separate model is tasked with arguing against it, checking that the deliverable does what was asked and that every citation is precise.

That happens before the lawyer or anyone else sees it. So the red pen the attorney picks up is editing a verified draft, not catching the machine’s mistakes or, even worse, having someone else catch them.

Marlene Gebauer (16:20)
There is always a question of build versus buy, and that continues in the market. What do you think is the primary differentiator for firms that say, “We are going to choose to build our own capabilities. We are going to build our own knowledge model, relying on our own data rather than a centralized vendor model?”

Stephen Costigan (16:50)
My thoughts are, number one, you are not going to build a differentiated practice area in your firm or maintain differentiation by buying the same product and using the same data set as everyone else.

Number two, data is the firm’s greatest asset. The signals that come off that data are being extracted. Why are you giving your data away or training another organization’s environment on those signals? But the primary differentiator is not a feature. It is ownership.

And with the... Sorry, go ahead.

Marlene Gebauer (17:33)
No, go ahead.

Stephen Costigan (17:51)
I am really backing up the points that I made. In the centralized vendor model, you are renting access to their product, interface, roadmap, and data-handling promises.

Your knowledge ultimately improves their system. With our platform, or with private AI infrastructure in general, you build a curated graph that you own outright. That compounds with every matter you work on and bring into the platform.

You can extend and build on it. You can create differentiated products for your firm, and you can do that more easily now, especially with recent advancements in agentic AI. Three things follow from that.

First, the ontology is yours and editable. You govern how your knowledge is structured, not some vendor. Second, because it is structured and resolved, the asset becomes more valuable over time rather than being consumed and forgotten after each query, as in a RAG model. Third, because it runs in your environment, you are never exposing client data to build someone else’s moat.

The crowded part of the market is selling chatbots and tooling that sit on top of a model using the same closed pattern. The durable position is helping a firm own its intelligence and knowledge infrastructure. The model layer is becoming a commodity. The curated graph is part of the defensible asset, and it should belong to the firm.

Marlene Gebauer (19:22)
What would you say?

Greg Lambert (19:23)
Is there a certain type of expertise that firms need to maintain this? If we are going to use a third party and rely on its infrastructure, is there a different type of expertise that we need internally to maintain our own version of that infrastructure? I hope that question made sense.

Stephen Costigan (19:56)
Yeah, it does. In terms of requiring an entire team to manage an infrastructure like that, I think that in the near future it will not be as much of an ask to build a part of your organization that can manage private AI infrastructure.

I think existing knowledge management roles can be adapted to the curation and management of that aspect of the environment. I say that because of the advancements in agent-based infrastructure management. For instance, in our environment, we have what we call the Enclave. It is an agent environment where pretty much everything in our infrastructure is managed automatically, in a highly governed way.

It is not that difficult to roll your own infrastructure-management agents. When we look at the amount of code written agentically in our organization, we have gone from around 10 percent to 60 or 65 percent of our platform being written agentically. Everyone thinks they are going to need a huge team to manage their own private infrastructure, but that is simply not the case anymore.

Greg Lambert (21:20)
Yeah. I know a lot of us are looking at the Kirkland advertisements for GPU professionals and seeing that kind of build-it-almost-from-scratch, nearly-on-your-own approach. With Atlas AI, I am not going to have to hire people to stand up GPUs and monitor them, right?

Stephen Costigan (21:42)
Exactly.

Marlene Gebauer (21:43)
I had a question because you mentioned before that the legal AI market is incredibly saturated, which it is. But I am not sure the same is true from a knowledge management perspective. There seems to be a more limited set of tools that deal specifically with that. What do you think differentiates those types of tools?

Stephen Costigan (22:10)
I do not think there are many highly effective knowledge management platforms for law firms. Again, the team size is typically one to five people in that particular area of the firm, if they are lucky. I do not think there are really any solutions out there to compare to, honestly.

Marlene Gebauer (22:13)
Correct. Mm-hmm.

Stephen Costigan (22:36)
But I think what is critical is automating the curation of the DMS. If you can do that, you have achieved a great deal for a firm at the outset.

Marlene Gebauer (22:48)
Yeah, that is what I am saying. There seems to be a limited number of tools that say they do that. The ones that are out there still say you need to offer the system something to start with or clean up the data. I was curious whether you see other differentiators between tools in that space.

Stephen Costigan (22:55)
Yes. The thing is, it is not just about curating the data. It is about what happens after that, too. There are many paths one can take as a founder building a product. The path we selected was to get away from building features to compete with everyone else on the end-user side of the product and to enable the democratization of feature development.

You can take Claude Code, for instance, use our MCP, access your curated data set, and build whatever applications you want, either separate from or integrated into our platform. That is something I think is truly unique.

It takes firms away from having to follow the vendor roadmap and allows them to start building differentiated products for their firm immediately.

Greg Lambert (24:05)
Stephen, before we get to our crystal ball question, we have been asking guests to share some of the ways they keep up with the market. There is so much to try to keep up with. Do you have any resources you do not mind sharing that help you stay current on the transitions in technology?

Stephen Costigan (24:17)
We really rely on Legal Tech Hub for market signals, and Artificial Lawyer.

From a knowledge graph and data architecture standpoint, the foundational knowledge graph work by Hogan and colleagues is great. Those are the high-level sources. Ethan Mollick’s Applied AI is helpful for understanding how professionals actually adopt these tools. Those are my recommendations.

Greg Lambert (24:39)
Yeah. I have an Artificial Lawyer story to tell on this. I was on Richard’s podcast a few weeks ago, and I was in my Austin office when one of the attorneys came up and said, “I heard your podcast interview.” I said, “Which one?” They said, “The one you were on with Richard.” I said, “Well, have you ever listened to my podcast?” They said, “No, I have not listened to that yet.”

Stephen Costigan (25:20)
Exactly.

Greg Lambert (25:27)
So Richard does a pretty good job.

Marlene Gebauer (25:29)
Yes.

Stephen Costigan (25:30)
Absolutely. I try to read across two lanes that I do not know if they always talk to each other, but I think that is changing, and we want to drive that change. Those lanes are the legal innovation world and the knowledge representation world. The interesting work is at the seam between them. That is something no one is really attacking right now, so it is something that really excites me.

Marlene Gebauer (25:55)
Yeah, I think that is spot on.

It is time for a crystal ball question. Looking ahead, do not be scared. We will only come back next year to see whether you were right.

Stephen Costigan (26:03)
I have no idea what that is, so I am scared.

Greg Lambert (26:10)
I am just guessing.

Stephen Costigan (26:12)
Okay.

Marlene Gebauer (26:24)
What is the single biggest shift that you see coming for the traditional law firm business model?

Stephen Costigan (26:30)
Ownership. The biggest shift is that a firm’s accumulated knowledge stops being a byproduct and becomes its most leveraged asset. That really breaks the math on which the billable hour rests. The traditional model monetizes leverage.

Partners sell associate hours against precedents that live in people’s heads and scattered files today. That is changing, obviously. When that precedent becomes a curated graph, routine production work compresses dramatically.

The firms that own that graph will deliver senior judgment with far less junior leverage. They will be able to productize their expertise through fixed-fee, on-demand, client-facing offerings in ways the hourly model never allowed. The divergence I see is ownership.

Marlene Gebauer (27:34)
It is ownership and usability.

Stephen Costigan (27:36)
Usability. Firms that treat AI as a tool they rent will compete on price against everyone renting the same tools. Firms that build and own their own knowledge graphs, curate their data into those graphs, and use them effectively will turn their expertise into a durable, compounding asset. That becomes the real moat for the firm, not head count. The losers will be those who gave their knowledge away and do not have much to show for it. So there is my crystal ball response.

Marlene Gebauer (28:06)
Okay, kids, you heard it here. Do not give your knowledge away.

Greg Lambert (28:10)
It is surprising that you have to tell people that, is it not?

Stephen Costigan (28:10)
Ha ha.

Just to land this point, in five years the question will not be which AI tool your firm uses. It will be whether your firm owns its knowledge or rents access to it from a bunch of knowledge providers. That is the line on which the next generation of firms will be drawn.

Greg Lambert (28:37)
Yeah, I think that is a pretty solid prediction. Stephen Costigan from Atlas AI, I want to thank you for taking the time to join us, break this down, and geek out with us on knowledge graphs and private AI. I really appreciate it.

Marlene Gebauer (28:40)
Yeah, me too. Mm-hmm.

Stephen Costigan (28:52)
Thank you so much. Thanks for having me.

Marlene Gebauer (28:55)
Yeah, thanks, Stephen.

Thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We would love to hear from you on LinkedIn and Substack.

Greg Lambert (29:06)
And Stephen, where is the best place for listeners to learn more about you and what you are doing at Atlas AI?

Stephen Costigan (29:14)
Sure. You can find me on LinkedIn, Stephen Costigan, Stephen with a PH. I am happy to talk with any firm thinking about owning its AI rather than renting it. The site is atlas-ai.io.

Marlene Gebauer (29:29)
And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

Signals are the new black

I had the pleasure of attending my first LSSO – Raindance Conference a few weeks ago where at least a half dozen times (I honestly lost count) presenters talked about signals.

Last week, I hosted an episode of Harbor’s Legal Soundings Podcast and signals came up.

There were the headlines about Kirkland’s AI investment being a signal of something, and talk of Microsoft signals.

I’m starting to think if I had a nickel for every recent mention of “signals” I’d be as rich as the people collecting nickels related to agentic AI.

Signals are not new.  Military intelligence has had the Signals Corp since the invention of radios.  Competitive intelligence professionals have always been in the business of finding signals to avoid mistakes and predict opportunity.

For decades, intelligent analysts have sifted through vast amounts of information to separate meaningful developments from background noise. The job has never been simply to gather information; it has been to transform information into intelligence. For example, at one of the firms I worked at, a Practice Group Leader called me one day and asked me to “look for the signals to determine which National oil company would invest in the Canadian oil sands next.”  Those weren’t the exact words he used, but you get the point.

What’s different in the AI era is that the economics of information have fundamentally changed.

Information itself is no longer scarce.

Every law firm and their CI/ BD practitioners now have access to AI tools that can instantly summarize earnings calls, SEC filings, regulatory developments, news articles, LinkedIn activity, job postings, patents, podcasts, analyst reports, and social media conversations. The barriers to access have largely disappeared.

The competitive advantage is no longer who has the information. The competitive advantage is who can identify and act on meaningful signals before everyone else.

This may be the single most important shift occurring in business development and competitive intelligence today.

AI Hasn’t Eliminated Analysis. It Has Raised the Bar.

For years, many organizations equated competitive intelligence with information gathering: collect the data, build the dossier, distribute the report, repeat.

AI now performs much of that work in seconds. Summarization is becoming commoditized. Research is becoming commoditized. Even synthesis is becoming increasingly accessible.

As AI lowers the cost of analysis, human judgment becomes more valuable, not less.

The question is no longer ‘What do we know?’ The questions become ‘What matters, and what is likely to happen next?’ ‘Who will this impact and how can we help?’

That is a signal-detection and analysis paradigm shift.

Business Development Is Becoming a Timing Function

Business development has always been about relationships, and it still is. But passive relationships, the kind where contact is only made when a suit is filed, a transaction is imminent or there is a sporting event happening, will no longer suffice.  Success today will   depend on engaging clients at precisely the right moment.

Companies continuously emit signals: new executive hires, geographic expansion, product launches, website changes, patent filings, strategic partnerships, job postings, and regulatory disclosures, to name a few.

Individually, these data points are unremarkable. Collectively, they tell a story.

Historically, legal business development has been largely relationship-driven and reactive: build relationships, stay visible, wait for a legal event, and receive the call.

The AI era invites a different question: What signals indicate a client is about to face a legal challenge before they realize they need outside counsel?  We used to set up early warning signals at my previous firm but we were still later than we could be in today’s world. We had to wait for a class action to be filed to find it. Today, AI tools can monitor consumer complaints, regulatory investigations, product recalls, data breaches, and court filings in near real time.

The firms that recognize that story first gain an advantage because timing matters.

Law Firms Have a Unique Opportunity

Lawyers are already trained to think in this scenario planning kind of way.

They instinctively ask: What changed? What are the second-order consequences? What risks are emerging? What is likely to happen next? What similar things have happened in the past?

These are signal-detection skills. The opportunity is to apply that thinking earlier in the client lifecycle.

What Legal Signals Might Look Like

Strong signals are easy to spot. Everyone sees the merger announcement, major funding round, or significant litigation filing.

Weak signals are more interesting: a handful of AI governance hires, a subtle website update, a revised privacy policy, or participation in a new industry consortium.

Weak signals may seem insignificant but they reveal a strategic shift months before it becomes obvious. The organizations that consistently connect these dots early will outperform those that wait for certainty, because by the time certainty arrives, everyone else can see it too.

This Is Also a Talent Question

Law firms have traditionally rewarded relationship builders, rainmakers, and network strength.

Those skills remain indispensable, but firms may need to elevate curiosity, pattern recognition, industry fluency, strategic questioning, and the ability to connect weak signals into actionable hypotheses.  These may not be skills that lawyers readily possess; some firms are already creating hybrid teams that combine business development professionals, competitive intelligence specialists, knowledge management professionals, and practicing lawyers to do exactly this. Others will find that to properly detect and action the signals they need to upskill their teams, hire or outsource to stay competitive.

Conclusion

Information is no longer a scarce resource.

In the AI era, every firm can gather more, summarize faster, and monitor more broadly. The advantage belongs to the firms that can identify which signals matter, understand what they mean, and act before the need becomes obvious.

For law firms, that changes the role of competitive intelligence and business development. The goal is not simply to report what happened. It is to help lawyers and clients see what may happen next.

AI can surface the signs. Human judgment turns them into signals.

And given how often signals seem to be appearing lately — in conferences, client conversations, headlines, podcasts, and product pitches — I wonder if the The Five Man Electrical Band was song writing in 2026 instead of 1971, they would have been singing about signals instead of signs... But there is an important distinction. Signs tell you where things are. Signals hint at where things are going.

“Sign, sign, everywhere a sign.”

This week we welcome American Association of Law Libraries leaders Jenny Foster, AALL President for 2025-2026, and Jessica Whytock, AALL Vice President and President-Elect. The conversation offers a preview of the 2026 AALL Annual Meeting & Conference in Cleveland, Ohio, along with a thoughtful look at how the association is supporting legal information professionals during a period of institutional, technological, and professional change.

Foster reflects on a leadership year focused on transparency, communication, and meaningful opportunities for member participation. From strengthening channels between members and AALL leadership to intentional volunteer appointments across committees and juries, she describes an association built through relationships. The goal is to ensure newer, mid-career, and seasoned law librarians all have a visible place in shaping the profession’s future.

Advocacy also plays a central role in the discussion. Foster explains how AALL continues its work on access to legal information, public policy, and coalition-building, even amid staffing transitions. The association’s Government Relations Committee has continued meeting with members, offering advocacy training, rebuilding connections with peer organizations, and aligning its work with AALL’s strategic priorities. For law librarians, advocacy is both a long-term commitment and a practical responsibility tied to preserving authoritative legal information.

The 2026 conference theme, “Leading with Aloha,” gives the Cleveland meeting its distinct point of view. Foster shares how aloha, rooted in kindness, unity, humility, patience, and meaningful connection, became a framework for leadership during uncertain times. More than 65 programs will explore topics ranging from generative AI and legal scholarship to physical collection strategy, access challenges, and the changing role of legal information professionals. Local programming connected to Cleveland’s history will bring an added sense of place to the gathering.

Whytock looks ahead to her upcoming presidency with a focus on clear pathways for engagement, leadership, grants, scholarships, committee service, and professional growth. Both leaders see artificial intelligence as a catalyst for a deeper conversation about the identity and value of legal information professionals. Their message is straightforward: the future of law librarianship rests in human judgment, critical thinking, ethical discernment, context, access, and a community willing to bring more voices into the room. The 2026 AALL Annual Meeting in Cleveland offers a place for those conversations to move from aspiration into action.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Marlene Gebauer (00:00)
Hi, I’m Marlene Gebauer from The Geek in Review and I have Sam Moore here from Legal Technology Hub who’s going to tell us a little bit about analysis of token usage and model selection.

Sam Moore (00:11)
That’s right. Thank you, Marlene. Well, it is tokens, tokens everywhere. I think spurred on by the launch of Claude for legal, but certainly going back further than that. There’s an issue in the legal industry today around token usage in GenAI tools. And in the legal technology hub advisory team, we’ve had several conversations in the last week or two about this, both in terms of frontier models, but also in terms of the legal AI platforms.

And the topics we’re discussing with clients right now tend to fall into three interconnected topics. First is model selection, because a lot of these products give the users a choice of which model they want to use for a given prompt. But most users of these products really have no idea what the difference is. I’ve seen law firm clients whose users just pick the most sophisticated model for everything, toggle on every optional feature available.

and then are confused as to why responses are taking a long time and why they’re hitting token limits very, very quickly. The second’s around model context windows. I’ve had several conversations lately about what a context window even is and how it can create drift when it gets crowded in a chat’s context window and why that really matters for legal use cases, which often involve uploading quite large documents, which take up a lot of space in those context windows.

And finally, efficient token usage. Law firms and law departments, I think, are generally not that accustomed to this kind of pay-as-you-go model in technology. Not unless you’re like me and you recall when the big legal research platforms were on a pay-per-search basis. So now those users are running into high-cost overages on the frontier models in particular, and they’re realizing that low sticker price per month is not their reality, not when their users

don’t know how to use those tools efficiently and how to control cost. So as well as delivering advisory work on these topics on a one-to-one basis, we’re actually working on a series of articles for LTH Premium about these topics, which will then combine into a sort of playbook for our subscribers to keep handy when they’re working with Gen AI tools. And we expect to start putting out that content in early June.

And if people want to know more about LTH advisory and what we can do, they can always get in touch with us by going to legaltechnologyhub.com or by finding me on LinkedIn.

Marlene Gebauer (02:35)
Thank you, Sam, for keeping us informed about this important issue.

Sam Moore (02:39)
You’re welcome.

Marlene Gebauer (02:47)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:55)
And I’m Greg Lambert, and today we are thrilled to welcome the leadership of the American Association of Law Libraries, or AALL. Joining us today is Jenny Foster, the 2025-2026 AALL President, and joining her is Jessica Whytock, the AALL Vice President and President-Elect.

Marlene Gebauer (03:01)
Yay.

Jenny and Jessica are here to preview the upcoming 2026 AALL Annual Meeting & Conference taking place in beautiful Cleveland, Ohio, this July, and to discuss the strategic direction of the association. So Jenny and Jessica, welcome to the show.

Jenny Foster (she/her) (03:32)
Aloha, thank you so much for having us. We really appreciate it. Big fans of The Geek in Review.

Jessica Whytock (03:38)
It’s great to be here. Thank you.

Greg Lambert (03:40)
Yeah, and Marlene went to law school in Cleveland so bad memories there, so if you

Marlene Gebauer (03:47)
Not about

not about not about law school, no. It’s just

Jenny Foster (she/her) (03:51)
Crazy.

Greg Lambert (03:53)
Well, Jenny, welcome back. You joined us last year, and you were in Jessica’s shoes. So let’s talk a little bit about the presidency that you’ve had for almost a year now. I know you made a massive push for transparency and open communications within the association. In that time, I know you’ve doubled down on the eBriefing and the KnowItAALL newsletter, making sure that

that people have a direct line to leadership. On top of that, you focused on volunteerism, and there has been a whole lot going on. So talk to us a little bit about, you know, the internal and external policies you have set up, and how your year has gone.

Jenny Foster (she/her) (04:41)
Okay, how my year has gone? That’s a loaded question, Greg. I mean I think

Greg Lambert (04:45)
It’s been great.

Marlene Gebauer (04:46)
Ha

Jenny Foster (she/her) (04:49)
Everybody’s like, “How’s my year? How’s my day gone?” But, going back to transparency and open communication, your question, I was really thinking about the bones of communication and inviting people to participate in conversations with leadership. They were already there. And then, with Jessica’s help, working together intentionally, we were trying to uncover and remove barriers. What is the word? We wanted to be clear and intentional in inviting people to talk, share their perspectives, and, like you said, reach leadership. And one of the ways that we were thinking about engaging members was through volunteerism and intentional volunteerism. So I really think that

Well, as you know, Greg, right, you get elected to this position, right? And law librarians like to plan. We love a plan. We are not even in our roles before we begin thinking about vice chairs and appointments to the 42 juries and committees carrying out the work of our organization, and who we are inviting into that space. And then they will be working eventually alongside us in our presidential year. Jessica would have to speak for herself, but I know we were aligned when we were making those appointments, thinking about where members are in the organization. What setting are they in? Are they newer, experienced, mid-career, or seasoned professionals? What type of library? And really trying to be super intentional about inviting them to participate, right? Because for the newer folks and mid-career folks, these are our future leaders of the association, and they need to be invited to share space there. And then our seasoned professionals are the mentors helping people matriculate through our organization. When we invite folks into leadership spaces, it is such an opportunity to learn. I think about the spaces where I have served.

And you suddenly get put in a room with people you would not otherwise meet. But if I was only in my career track at the Hawaiʻi State Judiciary, I would have never met Jessica. What a travesty. Okay, because she’s brilliant, lovely, and I can’t wait to hand off the reins to you in July, Jessica. And I think about that also in my first committee, right? The Special Committee on Diversity, the Leadership Development Committee, and all the volunteers from all walks of life.

And then we want to create a space where people can share their perspectives, come to a disagreement sometimes because we have passionate members, but through that disagreement, really tackle the work of our association to come out with a better outcome. And we can’t do that if we don’t communicate clearly, if we don’t make space for communication in a way that’s helpful and helps each other, right? Like recognize each other. And that connection is what strengthens AALL now and in the future. Did you want to add anything, Jessica? I’m just like blah!

Jessica Whytock (07:53)
Jenny, I think the question was about your leadership year, so

Jenny Foster (she/her) (07:55)
Yeah, yeah, yeah. But that is the other thing, too.

We are all connected to one another. I mean, the historic election, right? But I recognize that I was so humbled, the first resident from Hawaiʻi to be elected to this role in a hundred and twenty years. But before me came many fabulous leaders, like yourself, Greg, and BIPOC leaders. Like it’s not done in a vacuum. We’re all here in relationship with one another. And that’s really what I was focusing on this year is like how do we invite more voices to the table to, you know, further the legal information profession.

Jessica Whytock (08:29)
You know, Jenny, I actually will add something, which is Jenny is all effusive love and hearts, but what has impressed me is the smart, thoughtful person behind all of this who has a vision and a plan and also

Jenny Foster (she/her) (08:31)
Yes, please.

Jessica Whytock (08:54)
has an astounding amount of institutional knowledge that you just share at key decision-making points. So the one thing that I hope that people understand is that Jenny’s a tremendously strong leader and that we’ve been so lucky this year to have Jenny. And that as much as we love the Aloha and the hearts, there’s wisdom there that has made a big difference in what we’ve done this year.

I know.

Marlene Gebauer (09:22)
I know, I I’m like

Greg Lambert (09:24)
And luckily she’s she’s very quiet and muted and

Marlene Gebauer (09:27)
Ha ha ha

Jessica Whytock (09:29)
You know, she says all of the words so I don’t have to say them.

Greg Lambert (09:32)
Yeah.

Marlene Gebauer (09:34)
Well, AALL has always been very strong in terms of its advocacy arm and you know historically has had a very strong voice in Washington. you know, even with the transitions in the government relations role, and I was part of that committee at one time. You’re you know, your your administration hasn’t missed a beat. So, you know, you’ve been partnering with the ALA, issuing statements on IMLS funding.

opening up communications with ALA about filing amicus briefs. So you know, how is AALL ensuring that the advocacy needs of law librarians remain at the forefront at the national level?

Jenny Foster (she/her) (10:13)
Well, the first is really embracing it, as we know we have a vacancy operationally with our advocacy position. but we also had a vacancy with our executive director, which is really important, so we had to cross that bridge first. But even still one thing at a time. But it’s side by side with that, we were still working towards advocacy. So this is my love letter to the GRC, which you used to be on, Marlene. You were on a GRC committee. Thank you.

Marlene Gebauer (10:27)
One thing at a time.

Back in the day, yeah, I was. It was my first committee,

yeah.

Jenny Foster (she/her) (10:43)
My God, and it’s so amazing, right? You learn so much. And so this is my love letter to the chairs, Chris Lund and Kristina Chamorro, for their tremendous work this year. Supporting them has been important. Shameless plug for the eLearning platform: they recently held a national advocacy training with a congressman from New York, I believe. But more importantly, they have been consciously and intentionally meeting with members. So they’ve been meeting, doing these chapter meet and greets across the nation to really have a better understanding of what are the advocacy needs. Like things

Marlene Gebauer (10:45)
Mm-hmm.

Jenny Foster (she/her) (11:11)
Have changed since you since when I first started, probably since you were in the committee, right, Marlene? So we really need to evaluate what is effective and where our common goals are. But that’s also outlined in our strategic plans because we have an advocacy pillar in our strategic plan. So this is like working side by side, thinking about taking the temperature of our membership nationally. Where are they? Where can we come together? What do we need to move forward? Nurturing and reopening that communication with ALA was really something

Marlene Gebauer (11:17)
I’m sure it has.

Jenny Foster (she/her) (11:41)
something important to do this year. We are so happy those channels have opened. Thinking about other sister organizations in the advocacy space, aligning ourselves with them, and building coalitions. We are more visible when we come together, right? As RBG said, advocate for change, but do it in a way that invites people to join you. And, alongside our board action, we have been grappling with what advocacy looks like. How can we meet the needs of our strategic plan? Our amazing executive director brings advocacy experience from the work she did before joining us. Alongside our board’s advocacy vision and the nuts and bolts of the GRC’s work, all of that can coalesce and align to move this forward. So even though we’ve had a vacancy this year, we’re intentionally focusing on how to fill this gap while still meeting the advocacy goals that we hold so dear. So we are working on this like

I’m so happy with how this is going and you will see more. Remember, advocacy is a marathon, especially now. It’s a marathon, not a race. And so every little step that we can make will have a difference and I’m really pleased with the work that we’ve done this year. And then I get to hand it off to Jessica.

Greg Lambert (12:54)
Yeah, that was one of the things Jenny, you and I had in common, because we had searched for an executive director when I was president. So, fun

Jenny Foster (she/her) (13:05)
Yes, that is correct. Yes. Yes.

Greg Lambert (13:09)
Fun times. That is always interesting to watch, that transition. But I want to pull focus back to the conference coming up in July in Cleveland. And your theme is Leading with Aloha. What does that theme mean for you personally, and how does it show up across the more than 65 programs at this year’s meeting?

Jenny Foster (she/her) (13:43)
Thank you so much, Greg. I love this question. This could be like a whole podcast on itself, so I promise not to take up all the time. But going back to how law librarians love to plan. So you know, right now Jessica is already thinking about next summer in Philadelphia, and in the VP role, I was already thinking about Cleveland in 2026, right? So

And then I was trying to think about this gift that has been given to me to be the leader of our association. And then I always approach leadership with a service mentality, like what are the gifts that I can offer to the membership, to the organization. And then, remember, this was back in the fall of 2024.

There was a lot going on. My vice chair was in Chicago planning for Portland. And so I was walking my dog along the beautiful Koʻolau, thinking about all the changes that were about to come in 2025, remembering how chaotic it was in terms of access to legal information, thinking it was probably gonna come back. I’m just being descriptive. We know like government information exists one day and then it’s not on another day, and I just realized how much chaos was gonna be there. And then also thinking about

Being from Hawaiʻi, what was the gift that I could give? And I just, it was like, aloha, we will lead with aloha. In conversations about unpredictability, chaos, and fear, I have said that we are not hardwired for those things. But, actually, we are. And you know what those hard wires are?

Fight, flight, or fawn, and those might keep you alive, but they’re not a meaningful way to live and connect with others. So the antidote to unpredictability and chaos in my mind is meaningful connection. Because when we come together in meaningful connection and elevate one another, that is the antidote to fear and unpredictability because we can see with our own eyes that we can make positive differences in the colleagues that we work with, the communities that we serve. And aloha.

It’s all about nurturing that meaningful connection. So here’s my law librarian story time, okay? So aloha, which has been nurtured by the Indigenous population, Kānaka Maoli, or Native Hawaiians, for generations, right? It’s nothing new, but it was codified in the Hawaiʻi Revised Statutes in 1986. So this year makes the 40 year anniversary. And the reason that they did that is because they were infusing aloha into the highest levels of government decision making.

It says in the statute, right? It asks everybody to think about aloha in fulfilling their responsibilities and obligations as service to the people, the legislature, the governor, the chief justice, everybody.

And they even defined aloha. So I’m going to tell you the definition: ʻakahai, meaning kindness to be expressed with tenderness; lōkahi, meaning unity, to be expressed with harmony; ʻoluʻolu, meaning agreeable, to be expressed with pleasantness; haʻahaʻa, meaning humility, to be expressed with modesty; and ahonui, meaning patience, to be expressed with perseverance. All of those things, kindness, unity, agreeableness, humility, and patience, invite that connection.

And to face one another with that mindset, it takes two things. It takes courage because it’s really hard to be your authentic self. I think. I think it’s very brave to do that. And secondly, it really opens up the space to learn from each other, to be curious about all the different perspectives that we bring to the table. And

Leading with aloha, in terms of our programming, encompasses all of those things. We are already doing those things. We are bringing in programming where newer, mid-career, and experienced law librarians can learn from one another. We represent all the different library settings, all the different types of work that we do, and collaboration. I’m really excited about the work that we’re doing here and the intentional programming that our AMPC put together. I want to give a shout-out to our chair, Kelly Leong. The synergy was remarkable.

She is brilliant, wonderful, and intentional about how she led the committee, making sure different voices were incorporated in programming decisions. Kelly has family roots here in Hawaiʻi. Her dad is half Hawaiian, and she has grown up visiting. I get chicken skin thinking about the synergy of our coming together to develop this program for AALL and highlight our members through that connection. So we’ve got, you know, the Discussion Dens and the

hot topics that range from AI courses to access challenges and legal scholarship, balancing physical-collection needs with tightening budgets, GenAI, and how we integrate and guide its use in our institutions. We have a couple of interesting Marlene, you know, because you were there in Cleveland. We were just talking about it, about like all the different programming that is local to Cleveland. How do you say the river? I want to say it correctly. Cuyahoga. Cuyahoga River.

Greg Lambert (18:36)
Cuyahoga.

Jenny Silbiger (she/her) (18:38)
And how it was on fire, and the EPA was created in response to that. And then there was the grim serial-killer history from the 1930s, and the policies developed to try to solve that mystery. There is something there that elevates the community, the work that we do, and law librarianship in general. I mean, that is what was incorporated. Aloha was incorporated. But really, this is what we do every year when we come together in person at our annual meeting.

We come together for that connection piece. Okay, sorry, that was really long. You need to cut stuff down.

Greg Lambert (19:11)
A couple of comments there.

one, I’m I may need you to make sure that the the transcript spells everything correctly about the aloha.

Jenny Foster (she/her) (19:25)
For sure, for sure.

Greg Lambert (19:27)
And, when people think of librarians, they often think, “I would love to be a librarian because I love to read books.” That is about as far from what we do as it gets. AALL has three pillars, government, private, and academic librarians, plus our vendor allies. That diversity makes planning these events a challenge, but it also brings together different views under the same professional umbrella.

Jessica Whytock (20:20)
That diversity is one of the greatest strengths of our association, especially for me as an academic law librarian. I need to know what is happening in other libraries because it informs how we prepare students for their professional lives. And AALL has always provided this amazing opportunity for us to get together with our colleagues at different libraries and learn from them and see what their needs and interests are. Because of AALL and the people I have met there, I had a panel this year in my advanced legal research class. It came together quickly because everyone knew me from different programs we had done together and came in to teach my students what life will look like when they are out in the world. And so I love the fact that we are such a diverse group, representing all different types of libraries because it has added so much to my professional life. Yeah, I know, Jenny.

Jenny Foster (she/her) (21:39)
I feel the same. I love learning. And you know what, especially with the emergence of AI, I had Greg on a CLE here in Hawaiʻi a couple of years ago, was that two years ago, Greg? But I love hearing what the firms are doing with it and also and and I’m doing reconnaissance for the court so I’m learning about it. No, absolutely. Everything that you said, Jessica and Greg, thank you.

Marlene Gebauer (21:58)
I’m going to stay with the conference theme. Incorporating the locality into the conference is brilliant, because every location offers something different. It will be interesting to see how that works in Cleveland. But you know

We have heard a lot of buzz about the pre-conference activities, including hands-on workshops focused on copyright and fair use, and the return of the PLLIP Summit, the Private Law Librarians and Information Professionals Summit. So hooray for that. What are some

Jenny Foster (she/her) (22:41)
Ha ha ha.

Marlene Gebauer (22:45)
It seems like there is a lot going on, but people have to choose. What are some must-attend events before the official opening session?

Jenny Foster (she/her) (22:57)
You are so right, Marlene. I’m so excited. Yes, you’re right. The return of the PLLIP Summit has garnered a lot of attention and people are super excited. I guess. I don’t know. I’m sorry. I heard them calling PLLIP. I’m not in government.

Marlene Gebauer (23:05)
We call it PLLIP now? Okay. We always called it P-L-L-I-P, but I like PLLIP better, actually.

Jenny Foster (she/her) (23:14)
I think their theme is “Turn It Up! Amplify Your Practical Skills & Processes.” That is a turn I saw. I just got back from Canada, actually, and they were also talking a lot about AI, but really focusing on the practical applications. Like, tell me what works for you. How do you use it? I want to see what you’re talking about. And I think that is why they decided to do that. And then there are three others. There is “Libraries, Copyright & Fair Use,” which extends last year’s 101-level session into a more advanced discussion of how fair use shows up in day-to-day decisions and how to build confidence through real-world scenarios. And then I’m especially interested in “Re-Focusing in the Attention Economy” because that one is really about responding to the themes that we’re dealing with every day. Like, I had to turn off my phone, I had to close off 12 windows before we came onto this podcast, right? Because it’s a real thing, and what are the strategies to help your learners in such a digitally distracting environment. But my favorite is my love letter to CONELL, because that is our Conference of Newer Law Librarians. So if anyone is thinking about coming, if you’re a first-time attendee, if you switch careers or you’re a newer law librarian, I would love to see you there. And it is not a contest, but I hear that we have more CONELL registrants this year in Cleveland than we had last year. Again, not a contest. I’m just so excited that folks can come there and have that.

shared experience to kick off the conference. I remember being at my first CONELL. I love meeting newer law librarians who come to us. I still remember meeting people at last year’s CONELL. I’ll name-drop Devin Murphy, and there were others I met too. I love welcoming them in. So I’m super excited about that.

Greg Lambert (25:00)
It’s not a contest, but you won, right?

Jenny Foster (she/her) (25:03)
So those are some of them.

Marlene Gebauer (25:03)
Jess, did you have anything you wanted to add?

Jessica Whytock (25:05)
I’m really excited about CONELL and that so many people are participating. Everything sounds great. I know we are going to be busy, so we will not get to engage in as much programming as we would like. But getting the PLLIP Summit back feels great. But all of the pre-conference programming feels great. And I think it’s one of those situations where

Folks will find themselves having to make tough choices.

Marlene Gebauer (25:37)
Yeah. Well, from a personal perspective, if anyone’s on the fence about going to CONELL, I highly recommend that you do. For a few years, I led the session about what to expect at AALL for newer members, and it was a blast. You meet people, and as you said, they become lifelong friends. So go.

Jenny Foster (she/her) (25:50)
What?

Greg Lambert (26:03)
Yeah.

Yep.

Jessica Whytock (26:04)
I suspect we all remember our CONELL experience. I know I do.

Greg Lambert (26:10)
Yep, even when it was in the previous century, like mine. Mine was in 1999 in D.C., and I think I met Mark Gediman there. He and I are still hanging out, so.

Jenny Foster (she/her) (26:18)
Jessica Whytock (26:21)
No.

Jenny Foster (she/her) (26:23)
You’re a twentieth-century attendee.

Greg Lambert (26:26)
Yeah, my daughter. Yeah, whatever.

Greg Lambert (26:26)
So

So Jessica but before before we turn to you, I want this this actually applies to both you and Jenny. So Jenny, you know that when you hand over the gavel in it in Cleveland that no one will want to talk to you anymore. I was gonna say it’s it’s perfectly fine. So so Jessica just remember

Jenny Foster (she/her) (26:49)
Okay.

Marlene Gebauer (26:53)
But yet you will still have duties.

Greg Lambert (26:55)
This is this is gonna

Jenny Foster (she/her) (26:55)
Yeah, it it

Greg Lambert (26:56)
be

Jenny Foster (she/her) (26:56)
is true. I know.

Greg Lambert (26:56)
your huge year. Everyone’s gonna wanna talk to you, but don’t worry, at the end of it you’ll hand the gavel over to the next person and then no no one will talk to you again. So just words of wisdom.

Jenny Foster (she/her) (27:05)
I’m signed up.

I’m so there. I’m Jessica’s number one supporter. That is what my job is next year and I’m so looking forward to it.

Greg Lambert (27:12)
So, Jessica, as you prepare to step into the presidency in July, you bring a strong

background in advocacy, especially for the integrity of the profession. You’ve also co-chaired the Academic Law Libraries Special Interest Section white paper on continuing status and tenure, which is significant in academic settings. I want to step back. I know it is still early, but I know they picked up the phone and called you a couple of years ago and asked if you would run for this. Now that you are on the doorstep, what are some things you want to happen in your year?

Jessica Whytock (28:03)
Yeah, I have learned so much this year being on the board and one of my priorities of course is making sure that all of the good work Jenny has done continues that the relationship building continues, and that the systems that she put in place to just make sure that people have a a way to

to reach out and talk to us, and that those systems continue to exist even after my term is over. I do not think AALL always feels like now is the time when we see how important our profession is. And again, right now I’ve never seen such an important time to be a law librarian. Doing what we can to preserve access to authoritative legal information feels so important, even more important with AI and how now it changes how we access, how we interpret, how we use legal sources. So, making sure that we keep that at the forefront of what we’re doing during my term feels really key. But it is the internal growth and development of AALL that I really want to ensure is solid and strong. The association has made a big difference in my career, the people I have gotten to meet, the people I work with, the options and opportunities that are available to me. A lot of that is, of course, the work that we do every day at our own institutions matters tremendously, but it’s the connections that we make through different

positions at AALL, either just attending conferences or serving on a committee, or if you’re lucky to be in a leadership role doing that. So I want to focus on making sure that AALL makes it abundantly clear on how you can engage with the association. That might mean attending conferences or making sure you understand the grants and scholarships available to you. But

I really want to put systems in place that make it very transparent on how you can volunteer for a committee. How do you get chosen to chair a committee? I think the more that we make those pathways clear and available to people, the more we ensure that our association

represents our members and that people have a voice and that people know what they can do to take on these roles because they’re important and it is a lot of work, but it is a tremendous privilege and opportunity to serve on the board or to serve as a chair of a committee. And I want to make sure that the people who want to do that are able to do that and that people who did not even know it was something they could strive for see a really clear pathway, and steps people can take, to serve in AALL. So my goal is to strengthen our association by making sure that our members have really strong voices and pathways to join the association in any number of ways.

Jenny Foster (she/her) (31:16)
I love that with my whole heart. I cannot wait to support you. This year would not have been as successful without Jessica by my side and all the support from our board members. Like this is a collective group project.

Marlene Gebauer (31:22)
Yeah.

Jenny Foster (she/her) (31:29)
It is a collective group effort. I know we are in these roles for a reason. You wake up and do not know what role you are going to be in, but this is the one that was chosen for us. Jessica takes it seriously, and so do I. What a tremendous privilege and gift it is. I see Jessica striving hard to make meaningful impacts, and I am so appreciative of her help and I can’t wait, I’m so excited for her vision to come forward. Okay, sorry Marlene.

Marlene Gebauer (31:54)
No, no, that’s

okay. And I mean it sounds like you got some great goals to to strive for next year. well before we get to our crystal ball question, so as leaders of AALL and librarians who obviously read all the time, what what are one or two of the must read resources, you know, you know, committees or thinkers that you rely on to stay ahead of the curve in library administration and legal information?

Greg Lambert (32:08)
Yeah.

Jenny Foster (she/her) (32:22)
I

I love it. Are you asking us for a book list? Okay, I’m here for it, Marlene. For me, I zoom out a little bit. I love Dr. Brené Brown’s Dare to Lead and the BRAVING framework she has for leadership. I feel like that because leadership is an act of service. And I like zooming out: how can we be of service when we’re bringing ourselves into our workplace but then also for the association and in librarianship

Greg Lambert (32:27)
Yeah.

Jenny Foster (she/her) (32:50)
In general. And then also, and did you know, she started a brand-new podcast with Adam Grant. So it’s called The Curiosity Shop and it just launched, like I think they’re on their fourth or fifth episode. So I don’t know if you know, but like ten years ago they got into a public kerfuffle in The New York Times about vulnerability and when it’s appropriate to be vulnerable in the workplace and all of that. Anyway, but now they’ve come back full circle together and they are having such great conversations about leadership, about paradoxical thinking, making space for

diverse voices and what does that mean and then accountability but ethical integrity, it’s great. Those would be the two things that would be on my bookshelf for you to share. Now another time we can talk about the things we do for entertainment. What program are you watching?

Marlene Gebauer (33:32)
That will be after recording.

Jessica, do you have anything?

Jessica Whytock (33:39)
Well, you know, I’ll be honest, I have gone through a number of transitions the last few years and reading has not followed along. I have just not had the time. but I try to surround myself with people who model the behavior that I want to engage in and listen and learn. And I listen to people with whom I disagree and try to understand what their perspective is. So I’m not doing a lot of reading right now. I hope to do more reading in the future.

Marlene Gebauer (34:13)
I want our audience to understand that librarians do things other than reading. Most of the time.

Jessica Whytock (34:18)
Yeah.

Greg Lambert (34:18)
Yeah.

Jessica Whytock (34:21)
Yeah.

Because I do not have the time, I listen to and watch people I admire, see how they handle situations, and adopt what feels authentic to me.

Greg Lambert (34:38)
All right, now it is time for the crystal ball question. Jenny, we will start with you. looking into the future over the next few years, what do you think will be the biggest shift? We can focus this on AALL, but you know, what what do you see for the profession? Something that we need to prepare for now ’cause it’s gonna hit us later.

Jenny Foster (she/her) (34:48)
Okay.

Okay, I’m diving in here. I’m leaping off the cliff. This is a tough question. Thanks a lot. But

I think the biggest shift that we’re gonna see is this fundamental reimagining of what it means to be a legal information professional in our AI-integrated world. And not necessarily the technology itself, because I have faith in all of us. We can beta test it and learn it because we are teaching it, and the best way to learn something is to teach it. It is really about thinking about that identity question of who we are. What do we uniquely offer, and how do we communicate that to the organizations, communities, and institutions that we serve?

And we are already grappling with that question right now. I see our members stepping up in all the spaces. You, Greg, Emily, all the folks in the firm world, all the people in academia who are doing the innovation labs and all those things, and people in the courts too, we are a little behind, but we are still doing the best we can. Law librarians are stepping into leadership roles there, too. So I believe this with my whole heart. I know I talk to you about things from the heart, but I also have a brain, too. The answer is not to compete with the technology. It is to lean more deeply into the things that we do. The critical thinking that we do, the ethical discernment, the human judgment about context and nuance and access and equity that I don’t think algorithms can replicate. I mean, they cannot, right? Not yet. But the next generation of legal information professionals will need to be really fluent in all these tools, yes, but more importantly, then they’ll need to be anchored in our professional identity, our sense of purpose and service. And that’s where AALL comes in. Our work over the next several years, we have to cultivate that foundation through educational programming, through advocacy, our pipeline, and the ways Jessica discussed for engaging our members. Because this is the community that we will build together where

emerging professionals will feel valued, seen, supported, and empowered to step into those leadership roles because the future is not something that will happen to us. The future is something that we’re gonna build together. With aloha.

Greg Lambert (37:19)
All right. Kind of a hard one to follow.

Marlene Gebauer (37:25)
That’s fair.

Jessica Whytock (37:25)
You never benefit by talking after Jenny.

Jenny Foster (she/her) (37:29)
Jessica’s brilliant. She’s amazing.

Jessica Whytock (37:32)
I do not know. In a time of tremendous flux, it is hard to know what things will look like in five years. I do think one of the things that we need to do is make sure that we’re inviting the right people and voices in the room and that we don’t always just lean on our traditional colleagues and allies and that we think about who else needs to join conversations as you know

As we watch our profession and the legal industry change, it is going to be important to be flexible, adaptable, welcoming, and forward-thinking about who we need to talk to so that we are doing the work that we need to do and that we are teaching and training future lawyers to do the work that they need.

Greg Lambert (38:24)
Well, Jenny Foster and Jessica Whytock, thank you both for giving us a look at what is happening at AALL and at the upcoming conference in Cleveland. Thank you both for being here.

Jenny Foster (she/her) (38:37)
Mahalo, Greg. Will we see you there? Are you coming?

Greg Lambert (38:40)
I am coming. I will be there.

Jenny Foster (she/her) (38:42)
Yay, thank you so much. It was such a pleasure. Thank you, Marlene. I’m sad that we did not get to see Georgie, but hopefully next time.

Jessica Whytock (38:45)
Yeah.

Marlene Gebauer (38:45)
Thank you.

And thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (38:58)
And Jenny, one last time for the listeners who want to learn more, register for the upcoming annual meeting in Cleveland, where do they need to go?

Jenny Foster (she/her) (39:08)

Go to AALLNET.org, where you will find our conference site. Or go to any AALL social channel, LinkedIn, Facebook, or Instagram. It is always there. Come join us. We would love to see you there. Aloha.

Greg Lambert (39:22)
We’ll make sure we also put links

in the show notes as well. Okay.

Jenny Foster (she/her) (39:25)
Okay, perfect.

Marlene Gebauer (39:28)
And as always, the music you hear is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

In this episode of The Geek in Review, we welcome Greg Dickason⁠, Chief Technology Officer at ⁠LexisNexis⁠, for a wide-ranging conversation on agentic legal AI, Lexis+ AI Protégé, and the movement from AI chat toward AI work. Dickason frames the shift through a simple contrast: earlier legal AI answered questions, while agentic workflows take on multi-step assignments, conduct research, create drafts, verify citations, and move legal professionals closer to finished work product. For law firms and legal departments trying to understand where AI goes next, this episode places agentic AI squarely inside legal workflow, legal research, drafting, and risk management.

A major theme of the conversation is trust. Dickason explains how Shepard’s Verify extends the familiar Shepard’s signal beyond traditional research screens and into uploaded work product. Rather than asking lawyers to rely on AI-generated text without a verification layer, LexisNexis is building citation checking into the workflow, giving lawyers a path to confirm whether cited authority exists, whether authority is still good law, and how later courts treated the cited case. For lawyers worried about hallucinated citations, AI-generated briefs, and unreliable authority, this verification layer becomes part of the product architecture, rather than an afterthought.

The discussion also explores the relationship between LexisNexis and Anthropic, along with the rise of legal AI skills. Dickason describes a market where model choice, orchestration, and legal skills increasingly matter as separate layers. Anthropic, OpenAI, Google, and other model providers offer impressive foundations, yet legal work needs more than general-purpose intelligence. Large law workflows require legal content, expert reasoning, matter-specific playbooks, and firm-defined processes. Dickason notes the ability to upload firm playbooks as skills, giving firms a path to bring their own way of working into Protégé.

Security receives equal billing with accuracy. As firms place client documents into AI vaults and connect work product to legal AI platforms, Dickason explains bring your own key, or BYOK, through a practical office-and-locked-cabinet analogy. The point is control: client content sits encrypted, access depends on the user’s key, and access stops when the key is withdrawn. He also discusses legal chunking, indexing, vector stores, retrieval-augmented generation, and knowledge graphs as part of building AI systems suited for legal documents, rather than generic file handling.

The episode closes with a broader view of legal AI’s impact on junior associates, legal training, and access to law. Dickason does not predict the end of junior lawyers. Instead, he sees AI helping junior lawyers become senior faster through mock trials, mock depositions, and richer training environments. He also warns of risks from agent volume, security vulnerabilities, and legal systems struggling to keep pace with AI-enabled industries. The message is pragmatic and optimistic: agentic legal AI will change legal work, yet the winners will be those who combine trusted content, secure systems, verification, workflow design, and human judgment.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Marlene Gebauer (00:00)
Hi, I’m Marlene Gebauer from The Geek in Review, and I have Sam Moore here from Legal Technology Hub, who’s going to tell us a little bit about analysis of token usage and model selection.

Sam Moore (00:11)
That’s right. Thank you, Marlene. Well, it is tokens, tokens everywhere. I think spurred on by the launch of Claude for Legal, but certainly going back further than that. There’s an issue in the legal industry today around token usage in GenAI tools. And in the Legal Technology Hub advisory team, we’ve had several conversations in the last week or two about this, both in terms of frontier models, but also in terms of the legal AI platforms.

And the topics we’re discussing with clients right now tend to fall into three interconnected topics. First is model selection, because a lot of these products give the users a choice of which model they want to use for a given prompt. But most users of these products really have no idea what the difference is. I’ve seen law firm clients whose users just pick the most sophisticated model for everything, toggle on every optional feature available, and then are confused as to why responses are taking a long time and why they’re hitting token limits very, very quickly.

The second is around model context windows. I’ve had several conversations lately about what a context window even is and how it creates drift when it gets crowded in a chat’s context window, and why that really matters for legal use cases, which often involve uploading quite large documents, which take up a lot of space in those context windows.

And finally, efficient token usage. Law firms and law departments, I think, are generally not accustomed to this kind of pay-as-you-go model in technology. Not unless you’re like me and you recall when the big legal research platforms were on a pay-per-search basis. So now those users are running into high-cost overages on the frontier models in particular, and they’re realizing that low sticker price per month is not their reality, not when their users don’t know how to use those tools efficiently and how to control cost.

So, as well as delivering advisory work on these topics on a one-to-one basis, we’re actually working on a series of articles for LTH Premium about these topics, which will then combine into a sort of playbook for our subscribers to keep handy when they’re working with GenAI tools. And we expect to start putting out that content in early June.

And if people want to know more about LTH advisory and what we do, they can always get in touch with us by going to legaltechnologyhub.com or by finding me on LinkedIn.

Marlene Gebauer (02:36)
Thank you, Sam, for keeping us informed about this important issue.

Sam Moore (02:41)
You’re welcome.

Marlene Gebauer (02:49)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (02:55)
And I’m Greg Lambert, and today we are diving into the rapidly evolving world of agentic legal AI.

Marlene Gebauer (03:19)
Greg, welcome to the show. Greg Two, welcome to the show. No, I think you’re Greg. No, actually you’re Greg One, and Greg Lambert will be Greg Two. How’s that?

Greg Dickason (03:21)
Thanks.

Greg Lambert (03:26)
Yes, yes. But you’ll be Greg with the British accent, and I’ll be Greg recovering from a cold. So, Greg, let’s start off. We talk a lot about agentic AI, agentic workflows, and so I want to define that so we know what we’re talking about here.

Greg Dickason (03:30)
There we go.

Marlene Gebauer (03:31)
The smarter one.

Greg Dickason (03:37)
Gives you a very mellow voice, so it’s good.

Greg Lambert (03:53)
So do you mind breaking down...

Greg Dickason (04:06)
Yeah, absolutely. I think I like to differentiate between sort of the first generation of AI and this generation with agents and agentic workflows, if you like. And what we mean by that is the first generation, you would go and ask it a question, it would give you an answer. You would then have to do something with that answer. You’d have to go and plug it into a Word document or maybe do a bit more research and then go back with another question. It was a bit like the sage sitting on the hill. You walk up the hill, you ask a question, you get your advice, and you wander down again.

Agents, or agentic workflows, you’ve now taken that sage and you’ve put them in your factory floor. And now when you ask them a question, it’s more like you’re telling them to do something. They go and do the research, but then they do something with it. They build something, they produce a document, they realize that they’ve got to do more research, so they have multiple steps. And so it’s a much, much more powerful paradigm. You’re not just asking and getting a response back, you’re actually getting work done for you. And that’s where I think it is the huge shift with agents.

Greg Lambert (04:59)
Yeah. And so how do you look at that when...

Greg Dickason (05:16)
So for us, it’s about understanding that our customers want to get work done. They don’t just want to come and understand something, do some research, and then go off. They want to get work done. They want to produce a document. And in some cases, they don’t always know all the questions they need to ask. So being able to ask a more open-ended question, and then we go off and our agents build on that, ask them questions back, and effectively create a workflow or a long task which produces far closer to the output they want to get.

So we recognize that as much as we’ve got great authoritative content, that’s really powerful when you’re marrying it to the workflow of your customer so they actually know what they want to produce. And a lawyer is not producing an output from an AI. A lawyer is producing an email or a draft or a brief or something like that. And we want to help them get as close as possible to that final output.

Marlene Gebauer (06:05)
So I imagine that agents are very important in tools like Shepard’s. So how do features like Shepard’s Verify operate under the hood to actively cross-check AI-generated text against the LexisNexis database and prevent hallucinated citations?

Greg Lambert (06:23)
Like hallucinations came up in the second question. I like that.

Greg Dickason (06:26)
Yeah. It’s always a theme with AI, isn’t it? Yeah, absolutely. And I think that’s why we think Shepard’s Verify is so important. But everybody knows Shepard’s. Shepard’s is going to tell you, is this good law? It gives you a really strong signal. And that’s on our platform. What we thought is it’s actually good to move that out of our platform and onto your platform and wherever you are.

So if you’ve written a document and you upload it, can we do some citation verification for you? Can we check, does that citation exist? And if it does exist, is it good law? And that’s what Shepard’s Verify is about. It’s about...

Greg Lambert (06:29)
Yeah.

Greg Dickason (06:56)
That trust signal and giving it to you so you can use it where you are. Obviously, in our responses, we always give you a Shepard’s signal so you can click through and check, as well as get the signal to see how good is this law, but also in the documents you upload. So is it even verifiable, and is it still good law? And that’s where it works.

So, how does it work under the covers? We’ve got Shepard’s, we’ve turned it into a really powerful service, and that service is now available inside Protégé, so we can use it against any document. And that goes back to Greg’s earlier question, Greg Two’s earlier question, which is, how is an agent different from an AI? In this case, the agent knows, at this point, I need to verify what I’ve just picked up, or I need to verify this document. So it knows that it can use the Shepard’s Verify tool to do a particular task, which is to give you confidence in the output.

Greg Lambert (07:41)
Do you mind giving us a scenario where, if I’m an attorney and I’m working, how does that process work? Is it smooth, or is it something that I’ve got to purposefully go and do?

Greg Dickason (07:49)
Yeah, so let’s say you’ve got a brief from opposing counsel and you want to check that. You can upload that onto Protégé and we will do the verification checks for you. So you’ll see the signals against your document and be able to see how well the opposing counsel’s citations actually link, whether it’s good law or whether it even exists, as an example.

Greg Lambert (08:16)
And is it verifying the citations only, or does it go a little bit deeper? Does it look at what’s quoted, or how deep does it go?

Greg Dickason (08:24)
Yeah, it looks at whether or not, how well that has been treated by subsequent cases. It doesn’t always go right into the argument, but it does look at how well it is being treated by subsequent cases, and therefore whether this is a good or bad case to use in your particular argument.

Greg Lambert (08:43)
All right. So one of the things, and everyone is now talking about Anthropic. They seem to be the foundational AI model that everyone’s using, and, of course, caused a big stir over the past few months with the SaaS apocalypse and now the legal AI tools, the skill sets that they’re bringing in.

So do you mind talking to us a little bit about what kind of relationship Lexis and Anthropic have? Because I know you guys have used them for a long time. They’ve been underlying a lot of your technology for a long time. So it’s not a new relationship at all. But with them announcing that they’re in legal by releasing these skill sets, how does that relationship work? How are you building on that right now?

Greg Dickason (09:38)
I see the Anthropic one, I’m super excited about working with them, right? The fastest-growing company in history. I mean, you’ve seen what they’ve done this year. It’s pretty amazing. And to your point, we’ve been working with them from before they were really even thinking about how they sold to enterprises. So we had signed an arrangement with them on Amazon Bedrock, which is the way Amazon supports models, before Amazon Bedrock was live. And that was their way to start to work with us. I think we were one of the largest contractors they had in those very early years.

So we’ve got a great relationship with them. It’s been going for a long time. Jeff Bleich, their chief legal officer, was at one of our conferences the other day, and so therefore I see it as largely really collaborative. What’s great about Anthropic is they’re very open. They tell us what they’re doing. They give us early access so we can test against their models. We can test and see their skills. And so that’s a great place to be.

But at the same time, they’re moving very, very fast. And I think what they’re seeing is, how do they enable the knowledge worker in general? So, how can they give the knowledge worker the skills that the knowledge worker needs to get their job done? And they see Claude Cowork as sort of that generic knowledge worker’s interface where you can do some pretty cool stuff.

But what’s great is that what they’re providing is a great model, a good harness, and a set of skills. And I think of those as almost the layers. If you think about old tech, you used to have the database and then the business layer and all the rest. Now you’ve got the model, the harness, which helps that model work in your environment, and then the skills, which tell the model how to think about a particular thing.

All of those are available to us. But at the same time, we also have those available from other parties like OpenAI and Google and others. So we can pick the best of breed for the model, the harness, and the skills, regardless of which provider. And we can do that for whatever use case, for whatever type of lawyer we’re serving at the particular time.

So I think we’re actually in this unique position where we have great content, which we can use to build skills, but we can choose best of breed at all three layers. And we’re working with exciting businesses like Anthropic, which just means that we can innovate very, very fast on what they’re doing. So I don’t see it as too competitive. I think your other question there, Greg, was, you know...

Greg Lambert (11:43)
Yeah, because you hear, like, you hear now, we’re an AI, what’s the phrase, Marlene, that these small firms are? Basically they’re an AI foundational law firm, or I’m not getting it.

Marlene Gebauer (11:51)
AI-powered, AI-forward. AI-native, sorry.

Greg Lambert (12:08)
AI-native. And so I guess, and I think this might be a bit of a softball question, but I’ll throw it out there anyway. What is the value of having that combined?

Greg Dickason (12:23)
First, because the foundational models are tuned for generic solutions. They’re not tuned for what you need. So you need something that layers on top, which understands the law.

Second is that the foundational model is increasingly requiring a harness to work well. So you’re starting to get stuck into that harness because the two are being coupled. Think of it a bit like riding a bicycle. I can be a great athlete, I’m the model, but if I’m on a bicycle that fits me really well, I’m going to be so much better when I’m on my bike. And that’s what’s happening. Increasingly, the harness and the model are working well together. But that’s making you lock in because then it’s only you getting where they’ve tuned that.

So what we can do is we can reverse engineer and work across all of that. So we give you the best harness and the best model for a particular use case. So I think that’s why.

And then the skills is just a really exciting space. Skills are just Word documents, not Word documents, just text documents, which tell the agent how to think. And they can call each other and they can get quite complicated, but they’re basically just a set of text documents. And so if you go to Anthropic, you get a lot of great skills that are focused on just in-house counsel, but they’re not focused on longer-running, harder tasks, particularly in large law. And so, yes, there’s some stuff you can do there, but it’s not a generically strong legal platform like we provide. And we can reuse those skills and our own skills. So I think we can give you the best of all worlds.

Greg Lambert (13:43)
Is there a future where, as a Lexis+...

Greg Dickason (13:55)
That future’s arrived already. You can upload your skills with our new work product. The future’s arrived. But exactly to that point, you have your own way of doing work. You’ve already written it down. You’ve got your playbook. You can turn that into a skill and use that.

Greg Lambert (13:59)
The future is here now.

Marlene Gebauer (14:01)
Hm.

Greg Lambert (14:09)
Okay.

Marlene Gebauer (14:11)
So I’m going to ask another value-related question, sort of what your thoughts are in terms of the value of this. In addition to the Anthropic alliance, you also have now an alliance with Luminance, and I imagine that is going to bring a lot of new document drafting skills, and that it will be combined with the legal research skills of Lexis. Outside of streamlining that process, where do you see the value in that combination in one interface?

Greg Dickason (14:45)
I do think it’s about getting your work product done without having to switch interfaces. So I do think it’s the fact that you can do the research, you can start the draft, then do further research, and it can all happen relatively seamlessly. There might be one click through to check something and then back again, but it’s relatively seamless with things like Shepard’s Verify popping up to tell you, yes, this is right, this is not right.

And I think that’s a lot of, if you listen to good product podcasts, it’s about reducing the friction. It’s reducing how hard it is to do what you want to do, and I think a lot of those kinds of integrations for us are about reducing the friction so that there’s a...

Marlene Gebauer (15:19)
It’s also about getting people comfortable with working in a workspace outside of what they currently do, changing that whole, helping with change management in terms of how they do their work, because people are kind of notorious about not wanting to change that.

Greg Dickason (15:38)
Yes, it’s very hard. And for me, as a product tech guy, that’s one of the hardest things, getting people to change, even the small things. Like when you go into Netflix versus Amazon Prime, they scroll slightly differently. And even that I find is like...

Marlene Gebauer (15:51)
It’s infuriating.

Greg Lambert (15:54)
Yeah. Well, let me ask about this, because I wrote a thing about the future of the UX, and if you’re not developing an interface, an experience that the user likes or works in the way that they work, they’re going to go out and create their own way of accessing it, whether it’s like with...

Marlene Gebauer (16:16)
Or find a workaround or something.

Greg Lambert (16:16)
What Salesforce is doing with a headless interface, or they might use the AI to access the website directly and then pull the information back in for them. So as someone who is on the product side, how do you think about what the future of the user experience is as we move, especially as we move into this agentic period?

Greg Dickason (16:41)
I think it’s increasingly going to be simpler and simpler because the agent’s going to understand your intent. Therefore, one, it’s going to know about you, so it’s going to have memory about you, who you are, what you care about, and then it’s also going to understand the intent of this current thing you want to do. And so you don’t need a complicated UX anymore. What you need is something that’s simple, that’s easy to engage with, but then it might diverge toward a particular use case.

So if you’re doing research, it might ask you some questions. If you’re doing a draft, it might open a document on the side. But ultimately, it’s doing that for you. So it’s very curated for you. I mean, they do talk about AI UI, which is where the UI is actually created by the AI in real time. I think that’s immature, and I don’t think it’s there because then the AI is almost overcomplicating it. I think what we’re going to get down to is a much simpler interface.

Greg Lambert (17:30)
Yeah. I’m curious, because a lot of the web is built for human interaction. And one example is, let’s say I get a web page and it gives me a spreadsheet. Well, it might only give me 50 lines of that spreadsheet, and then I have to click page two, right? Because that’s how a human ingests it. Whereas if it’s an AI interface, it would give them the entire spreadsheet, or it might give them dozens of spreadsheets all at once because it can handle that. So it’s going to be interesting from a product side how you do that.

Greg Dickason (18:04)
Definitely. And we’re looking increasingly, like for our digital side, we’re seeing more and more traffic coming from OpenAI, from ChatGPT, and Anthropic, the actual models, the open models, where users have clicked through. So rather than come through via Google, they’re coming via those channels. And then the question is how much of that is coming from the agent, with agents looking at our website and curating that back for the user. So it’s a really interesting change. I think you’re right. More and more of the web is going to be written for agents, not for users.

Greg Lambert (18:28)
Well, let us know when you figure it out and we’ll bring you back on. You can explain it to us.

Greg Dickason (18:31)
Ha ha ha.

Marlene Gebauer (18:32)
I do have a question about what you’re hearing in terms of feedback from clients. We’ve talked about comprehensive solutions where you can bring in your drafting, you can bring in your research, you can bring in your assistant and all those things, versus point solutions. And I know it will have to do with the actual work that needs to be performed, but there’s also an increasing pressure, I think, for clients regarding the cost of these tools. So I’m curious, what sort of feedback are you getting from clients? Are they leaning one way or the other? Anything that you can offer in terms of that insight?

Greg Dickason (19:16)
I do think increasingly we’re going to start to see consolidation. They want fewer tools. I think there has been a case where they’ve been looking at lots and using point solutions because there have been specific point solutions that have helped for specific use cases. And I do think that’s going to start to collapse, coalesce. So, for example, with our system, we can now load any type of skill, which means you can start to tune our system for your particular matter and how you do your matter, and the agents can pick that up. So I do think that’s going to happen. And I do think that’s what our clients are starting to ask questions about. I think that’s your question, okay?

Marlene Gebauer (19:49)
It is, it is. And I had one other one. In the news, we’ve been hearing about firms making a large investment and building their own AI. And I’m curious, sort of what your take is on that.

Greg Lambert (20:02)
They had an extra $500 million laying around.

Greg Dickason (20:05)
Really? Yes. Look, I think it’s logical for particular workflows. I think for some things it’s not that logical, but for some things it does make a lot of sense, for some workflows, particularly when that is your value proposition that you take into market, that your clients see from you.

And I think in that case, you’re going to need some really good foundational building blocks to help build that. Obviously, we see ourselves as being a key contributor in that kind of space, where you’ve got deep legal research, deep authoritative content. But I don’t see it as being just calling some dumb interface, because you need the reasoning, you need the legal logic that comes with an agent like Protégé.

So it’s not MCP where you’re just being called. It’s A2A, it’s agents talking to agents. And I think that’s probably the emerging space, where you have an expert talking to an expert. They might both be agents to help solve the client’s problem. So we do see a space for that, but I do think it’s agent-to-agent rather than agent-to-MCP.

Greg Lambert (21:01)
And so one of the things that we’re seeing is a lot more of the firm’s data is being uploaded into systems, whether it’s in vaults or whether it’s through the Word document in the plugins, or a number of different ways that the information is being accessed and somewhat commingled, I would say. So, can’t talk AI without also talking about security. And one of the topics that’s being talked about now is the BYOK, or bring your own key.

Greg Dickason (21:51)
I think it’s critical, especially for our larger customers. They have to have it. And the point with bringing your own key is...

Greg Lambert (21:56)
Well, let me stop you there. Do you mind just talking about what it means to bring your own key?

Greg Dickason (22:02)
Sure, sure. So I like to think of it almost like a house. I’ve got a house where you can come and get your work done. You bring your documents, and I’ve got other, well, maybe not a house. I’ve got an office where I’ve got great workers. You can come, you can bring your documents, and you can get stuff done.

Now, what you want to do is bring a lot of documents, so you don’t want to keep bringing them in and out. You want to put them in the vault, right? And you want me to be able to access that so that my experts can give you the right results. But what you don’t want is for me to be looking at your documents when you’re not around, right?

So what I do is I give you a cabinet in my office. You put your documents in the cabinet and you lock it, and you bring your own key and you take that key away. And then you know I can’t access it when you’re not around because I don’t have your key.

And it’s almost exactly the digital equivalent of that. You have a mathematical key which unlocks, and it first of all encrypts and then unencrypts the content I need to do the job for you. But if at any point you withdraw that key, I no longer can do work for you. And that’s provable. And so I think it’s a great model where you can be quite sure that the only time your content is ever accessed is to do work for you.

Greg Lambert (23:11)
So how are you and your customers implementing this with Lexis?

Greg Dickason (23:16)
So exactly as you’re saying, in Vaults, you can now bring your own key. So you lock it. You put your content into the Vault. We index it so it’s all available for the AI to look at and say, okay, this piece of content works with this law to help draft that document for you. But it’s locked. And the only time our AI can look at that is when you’ve actually logged in and you’ve provided your key as part of your login. If you haven’t logged in, we can’t use your content. So it’s built into the Vault and we can prove that, and that helps you from your security posture perspective as a firm.

Greg Lambert (23:47)
And I’m curious if...

Yeah, upload files there.

Greg Dickason (24:16)
So with Claude, typically now you’re having to do it on your own laptop, and you can’t build as strong a vault. So when you upload files with us, we’re not just uploading them, we’re indexing them and we’re chunking them so they’re part of a vector store. And we’re doing that in a legal way. Different models can chunk the content in different ways. We chunk it so that it’s legally relevant. You can’t do that directly with Claude. You have to build your own chunking and your ingestion layer, which properly processes the files, and then your storage layer, which stores them in a way in which they can be easily retrieved for the AI. You might have heard of RAG.

Greg Lambert (24:53)
Yeah. We’ve been talking RAG for...

Marlene Gebauer (25:01)
Last year, year before.

Greg Dickason (25:01)
Yeah. Well, yeah, I mean, that’s like history now, right?

Greg Lambert (25:04)
Yeah, that’s very 2022.

Greg Dickason (25:01)
But to have a really good RAG system, you need to be able to properly chunk and index. And then on top of that, you can build a knowledge graph and other ways in which it makes it easier for your agents to surface.

Marlene Gebauer (25:17)
So it’s good to hear that Lexis is thinking about security, like bring your own key and things like that. What do you find from clients that they are most concerned about? Is it this type of security? Is it the hallucinations that sometimes happen with cases that they see in the news? What type of conversations are you having, and how are you assuring clients that Lexis is focused very much on trustworthy output and absolute security?

Greg Dickason (25:54)
Yeah, completely right. I think it’s both. When we’re talking to the security teams, they’re interested in the mechanics of security. So things like bring your own key, making sure that what they’ve uploaded is properly locked away, that kind of thing.

When you’re talking more to lawyers, they’re more interested in the hallucinations and verifiability, and making sure that they understand how do they know that what we’re giving them is good law, and how easy is it to check it? Because our position is these are non-deterministic models, right? They’re probabilistic models, which means they will always come up with a small probability of saying something that’s not quite right. Now, we’ve got lots of rules and a ton of stuff around to limit that, and we believe we’re best in breed, but you still need to finally be able to verify, to check. And that’s why it’s very easy on our platform to be able to click through and see. You get your Shepard’s signals, and you can easily click through onto the platform.

So I think a lot of our clients are asking us, show us your security model, which we do, and then also show us how we can mitigate any risks of using an AI system to get more efficient, more effective, to provide services. And a lot of that comes down to reduction in hallucinations, reduction in the type of hallucination to almost zero. But then, at the same time, you can always verify. You can click through and verify.

Greg Lambert (27:10)
I’m curious if there’s risk, or things that your customers might not be thinking about now, but maybe they should be thinking about. Is there anything that, I know you’re dealing with some smart customers who are risk-averse, but I’m curious. For example, if you ever get your hands on Mythos, what kind of risks do we think are out there with something like that?

Greg Dickason (27:38)
Well, I do think there are two types of risks. There are risks we’re aware of, but I think where AI is going is pretty mind-blowing. The next six months to two years, I think, is going to be phenomenal. And if you think back to coding, agents were a bit of a toy. You got agents to write code for you, and it was a bit of a toy. Then somewhere around November, December last year, that toy became something real. And a lot of my colleagues in the tech world came back from holidays and said, wow, before the holidays, I wasn’t doing much. Over the holidays, I built five systems that I never even thought I could do. And this is what’s happened.

And I think we’re going to start seeing those types of step changes in other parts of the industry as well. And one of them is Mythos. So I do worry about Mythos because I think that’s going to surface so many security bugs and security vulnerabilities in the next couple of months that we’re going to have the spike of that happening and we’re going to need to make sure we can jump on them. I think we’ll get to a much better state in about six months to a year’s time, but there’s going to be a period of time where we’re all quite vulnerable. And I really like the way Anthropic is trying to roll it out to keep us on.

Where else can we think? I think it’s agents’ ability to overwhelm us. That’s something else I worry about. From a legal perspective, how many briefs can you get, and how much content can you ingest as a human? So I do think we’re going to increasingly need agents to help us mediate the effect of agents in terms of volume, in terms of sheer complexity of work we’re doing.

And then I do think that we’ll start to see new types of industries emerging, new industries that are far more agile and AI-enabled, and that’s going to stress the legal system just like other ways in the past have, even blockchain and new ways of thinking, digital assets and all that. But it’s going to happen faster. And so, how do we keep up? How does legislation keep up? That’s going to be a real societal challenge, Greg. Maybe we’re going a little bit away from Lexis, but you know what I mean.

Greg Lambert (29:25)
Yeah. Yep, exactly. Well, speaking of keeping up, before we get to our crystal ball question, we’ve been asking our guests to talk to us about how they keep up with the industry. Are there certain things that you read or people that you listen to that help you along? What’s a couple of things that you...

Greg Dickason (29:56)
Well, The Geek in Review is a start, of course.

Greg Lambert (29:58)
Of course.

Greg Dickason (29:59)
And then also I read Artificial Lawyer, Law360. So there are a few legal things which are great. I read the Turing Post. It’s quite technical, but it’s a nice email chain that you can get, Turing Post. And then The Information. It’s a technology-focused magazine, but it actually gives you some really good cutting-edge thoughts about where AI is going. That’s not super technical either. So that’s really where I go.

But I also think a lot of the time, I use Claude itself. I say to Claude, what should I know? What’s happened in the last week? And I sort of have an interactive session with Claude to learn. And that’s also quite useful.

Greg Lambert (30:32)
Yeah. It’s one of the things we say here: use the AI to help you AI. So...

Greg Dickason (30:36)
Yeah.

Marlene Gebauer (30:37)
Okay, Greg One, it is time for our crystal ball question. So looking ahead, a few months to a few years, as AI takes over the orchestration of massive document-heavy tasks through tools like Protégé Vault, what do you think is the single biggest shift coming for the traditional role of the junior associate?

Greg Dickason (31:00)
So the simple question is no more junior associates. But I think the answer to that is they’re not junior because they’re not there, they’re junior because they very quickly become senior. And I think we’ll see AI helping us train junior associates, then being able to do mock trials and all the rest very quickly, mock depositions, all of that. And so we’ll see them becoming senior very quickly and learning a lot as a result.

Marlene Gebauer (31:04)
Yeah.

Greg Dickason (31:23)
I think very recently on the podcast, you had somebody who was building their training systems, and that was pretty exciting to hear. And I do think that’s where we’re going to go. So I don’t think we’re going to see fewer lawyers. I think we’re going to see the law being applied in more places. Society’s underserved, and I think it’s going to give us the opportunity to serve more people, which is pretty exciting about where AI can take us.

Greg Lambert (31:44)
Right. I like your vision. So, well, Greg Dickason, CTO there at LexisNexis, I want to thank you very much for joining us.

Marlene Gebauer (31:58)
Thank you, Greg.

Greg Dickason (31:59)
Great to be here. Thanks, Greg. Thanks, Marlene.

Marlene Gebauer (32:01)
And thanks to all of you for listening to The Geek in Review. If you enjoyed the show, please share it with a colleague. We’d love to hear from you on LinkedIn and Substack.

Greg Lambert (32:09)
So Greg, where’s the best place that listeners can find out more about you or about Lexis+ AI with Protégé?

Greg Dickason (32:16)
So jump onto lexisnexis.com/AI. That’s the best place to go. And then happy for you to look me up on LinkedIn, and I think we’ll post the link on this.

Greg Lambert (32:26)
Yes.

Marlene Gebauer (32:26)
And as always, the music here is from Jerry David DeCicca. Thank you, Jerry, and goodbye, everybody.

Greg Lambert (32:31)
Bye.

In this episode of The Geek in Review, Greg Lambert and Marlene Gebauer welcome back ⁠Joel Hron⁠, Chief Technology Officer at ⁠Thomson Reuters⁠, for a timely conversation about the shifting relationship among foundation models, legal content providers, legal tech platforms, and the lawyers trying to make sense of the mess. Recent moves by Anthropic, including Claude’s legal practice area tools and MCP connections into legal platforms, raise a larger question for the market. Is a model provider still sitting behind the scenes, or is it starting to become a legal work environment of its own?

Hron explains Thomson Reuters’ commitment to what it calls ⁠fiduciary-grade AI⁠, a standard built around trust, verification, transparency, and accountability. For TR, legal AI needs more than a fast answer. It needs systems lawyers trust enough to stand behind. Hron points to Westlaw, Practical Law, KeyCite validity signals, citation ledgers, and verification tools as core ingredients in building AI systems suited for high-stakes professional work. In his view, almost right is not good enough when clients, courts, regulators, and professional obligations sit on the other side of the output.

The conversation turns to how CoCounsel and Westlaw Deep Research use legal content across far more than traditional research tasks. Hron explains that when AI systems gain access to trusted legal content and verification tools, they begin researching throughout the workflow, even while revising contract language or analyzing provisions. He also describes Litigation Document Analyzer, internally nicknamed the BS Detector, a tool designed to review claims in a document and map them to supporting authority, weak support, or no support at all. For lawyers who spend as much time verifying AI output as generating it, tools like these aim to move verification from a manual scavenger hunt into a structured process.

Greg and Marlene also press Hron on Anthropic’s legal plugins, MCP, and the idea of headless legal technology. Hron argues that MCP changes access, not advantage. In his view, the application layer is shifting, but the real competitive value sits in trusted content, expert systems, governance, and domain-specific intelligence. CoCounsel’s user interface represents one expression of TR’s legal agent capabilities, while MCP opens other ways for those capabilities to appear inside broader work environments. Some work will still need a purpose-built legal interface; other work might happen through email, Word, Claude, or another agentic workflow with little visible interface at all.

The episode closes with a larger discussion about what happens when AI starts performing more of the work itself. Hron shares TR’s internal engineering OKR, where more than 50 percent of pull requests should be written by AI, and explains why 51 percent serves as a useful mental model. Once AI performs a controlling share of the work, the human role shifts from doing the task to governing the system. For legal professionals, the same transition is coming. The key question is no longer only whether AI produces useful work. It is whether lawyers have built the systems, context, safeguards, and verification layers needed to trust the work, defend the work, and remain accountable for the work.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠Substack⁠

[Special Thanks to ⁠⁠Legal Technology Hub⁠⁠ for their sponsoring this episode.]

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠⁠

Transcript:

Cleaned transcript below, using the uploaded file as the source.

Greg Lambert (00:00)
Hey, everyone. I’m Greg Lambert from The Geek in Review, and I have our friend Stephanie Wilkins from Legaltech Hub. And Stephanie, all the rage is about the talk about Claude for Legal. So do you mind giving us your perspective from the Legaltech Hub?

Stephanie Wilkins (00:16)
Sure. We’ve been diving into this a lot. And I’m sure anyone listening to this has definitely heard that Anthropic made that huge announcement recently with the launch of Claude for Legal. And there’s really a lot involved in it. We’ve taken a lot of time, across our team, to look at it from different angles and try to do very in-depth coverage on this. To me, it feels a lot like the days back when ChatGPT first came out and people were trying to get their heads around what it even is, let alone what it means. So we’ve done a number of pieces.

The first one covers the full announcement, that there are 12 new practice area plugins, more than 20 MCP connectors with legal tech providers, expansion across Microsoft 365, access to justice partnerships, and a managed agents layer for legal users building in the Claude platform developer environment. You know, just a few things to unpack there. But it is really, arguably, the most significant move a frontier AI provider has made into legal to date.

But it does raise real questions across the market, among them being how the partner ecosystem evolves from here, what it means for the established legal AI platforms, and where the announcement is genuinely game-changing and where we might have a little bit of overhype going on here. So we’ve been on the news from the start. Before it went live, we had a chance to speak to Mark Pike, who’s Anthropic’s Associate General Counsel, and he’s also serving as its product lead for legal. So we’ve included his perspective.

And since then, we’ve looked at multiple angles. We have the plain announcement news itself. We have a visual timeline that traces Anthropic’s path into legal from 2023 through this month. I did a separate analysis that looks into how much legal research you can actually do from within Claude for Legal, because that was one of the big areas it touched on. And as a sneak peek, we get to very different conclusions, whether you’re a BigLaw practitioner, or you do law in a small firm, or you’re a solo practitioner in the access to justice system. And then a fourth piece by Nikki Shaver really dives into the operational reasons why Claude for Legal is simply not yet a lift-and-shift replacement for enterprise legal AI platforms.

This is definitely an inflection point. It is not the death of legal tech as we know it, as some people might want to believe. It has not upended the industry overnight, but there is a lot to follow here, and we’re going to keep looking at it from different angles as they arise.

You can read all of the articles I just mentioned on LegalTechnologyHub.com. And if you want to get these updates in your inbox in real time, you can sign up for our free newsletters and follow the Claude for Legal announcement and the journey we’re on as we really try to be critical and dive into what it really does and doesn’t mean.

Greg Lambert (02:56)
Yeah, well, there’s so much hype, so it’s good to have a little bit of fact-checking going on. So thank you.

Stephanie Wilkins (03:01)
Yep, thank you.

Marlene Gebauer (03:10)
Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer.

Greg Lambert (03:16)
And I’m Greg Lambert. And Marlene, for the past year or so, the legal AI conversation has been dominated a lot by the foundational model race: which model is smarter, which one does the reasoning better, which one has the bigger context window, and which one is going to power the next wave of legal tech tools.

Marlene Gebauer (03:26)
Mm-hmm.

Yeah, absolutely right. But recent announcements from Anthropic and Thomson Reuters raise a different question. So if Claude is now launching legal practice area tools and connecting into major legal platforms through MCP, is Anthropic still just a model provider behind the scenes, or is it becoming a legal tech platform in its own right? And if Claude becomes one of the places lawyers go to work, what does that mean for the value of trusted legal content, citation systems, workflow platforms, and all of the legal AI tools built around those models?

Greg Lambert (04:14)
And that’s exactly why we brought in today’s guest. So we’re welcoming back Joel Hron, Chief Technology Officer at Thomson Reuters. Joel’s been on the show before and talked about professional-grade AI and where Thomson Reuters sees the technology heading. And this time, we want to dig into what the Claude and CoCounsel Legal announcement says about the changing relationships among the model providers, the content companies, the legal platforms, and the firms and legal departments trying to make sense of it all, because it’s kind of crazy. So Joel, welcome back to the show.

Joel Hron (04:49)
Thank you for having me. Good to be back.

Marlene Gebauer (04:52)
Yeah, welcome back, Joel. So for a year, we were just saying everyone was sort of chasing these model capabilities. Thomson Reuters is drawing a hard line around fiduciary-grade AI. You have argued that in high-stakes law, the work is easy, but defending it is what matters. From an engineering perspective, how are you building trust as a system primitive?

Can you walk us through the architecture of the patent-pending citation ledgers and how you ensure the agent isn’t reasoning from the open web?

Joel Hron (05:30)
Yeah, you bet. I mean, so we have leaned into this idea around fiduciary-grade AI. I think the core premise of this idea is that almost right is not good enough in the domains that we deal with. And I would say our focus has really been around how do we exploit the value of the 1.9, almost 2 billion documents across Westlaw and Practical Law that we have, the 1.5 billion KeyCite validity signals. These are all signals that human lawyers use every day to validate and verify and build trust in the work product that they’re putting out to their clients or to the courts or otherwise. And so our focus has been, okay, how do we use those same systems that human lawyers use today to help AI build the same level of verifiability and trust?

And I spent the last two weeks, I got back yesterday, with our customers across corporates, across the Am Law 100 and global large law, as well as some of the largest tax firms in the world. And this theme of trust came up almost repetitively across all three groups of those professionals. And I think this idea of, okay, AI is great, it’s doing a lot of work, but how can I, as a human, maintain accountability for what I’m putting out the door? And do I maintain accountability for it? And the answer to that question is affirmatively, yes. I think the professional maintains that accountability. And so it’s incumbent, I think, on us as software providers to build the tools in terms of verifiability and transparency and auditability to give them what they need in order to stand behind the output. And so that’s the core of what we mean when we say fiduciary-grade AI.

And so in terms of how we are building that, I would say first and foremost is to really leverage the best technology in the market today. And today that’s models like Claude, but also the latest versions of GPT, Gemini, etc. And also this idea of the coding harness and what’s called agent harness now in terms of how this is being evolved for AI agents to proliferate. So that’s sort of the core. And I think everybody is trying to evolve their products to live and operate around that paradigm.

But I think what’s unique and important to us is what tools do we make available to that agent to be able to do its work? And again, those tools lean on access to the content and information we have, but specifically also verification tools, citation ledger tools that we are able to build that allow the agent to do that work of verification for itself and ultimately deliver a better work product at the end of the day. And I think you’ve seen this in Westlaw Deep Research, how this operates. That system works very well. And we’ve adapted that same sort of approach with CoCounsel across more types of work. And that next version of CoCounsel is in beta right now.

I was telling Greg earlier, one of the things that we see, though, is that CoCounsel as a product doesn’t just do deep research when it’s preparing for some important litigation matter. It does legal research on almost every task. If it’s modifying a contract clause or updating terms in a provision or something like this, it is always doing research. It is always going to that content to verify what’s market right now. What has happened in the case history that would support what I need to do to this contract?

And that’s, I think, a much more powerful use of content than just preparing for a litigation matter where people are always doing research. What you see is that these AI systems, when you give them these tools, are actually using this content in a really deep way across many different types of legal work that you might not have considered doing research for before.

Greg Lambert (09:53)
Yeah, I know a lot of times the argument that I’m hearing from a lot of lawyers right now is that the AI is getting them an answer really quickly, but they’re spending almost as much time verifying that the information they’ve gotten back is accurate. With CoCounsel and Deep Research, and the combination of that along with the agent harness that you’re writing in, does that speed up that verification process, or are we getting into the positive now?

Joel Hron (10:09)
Mm-hmm.

Joel Hron (10:30)
Yeah, I mean, in one way it does, but in other ways we’ve built specific products or modules or features, whatever you want to call them, for speeding up verification. One example of that is a product we’ve called Litigation Document Analyzer, but internally we called it the BS Detector. And it was literally an application built around an agent harness and these content tools that was focused on looking at a document. It could be a litigation document. It could really be any kind of document, a brief, anything like this.

And what this system will do is it’ll go through every claim made in this document. And a claim could be a sentence. It could be a sequence of sentences, but at a granular level, what is every assertion that is made by this document? And is it supported by something factual, i.e., case law or statute or regulation or something like this? The output of this is effectively a table of, here are all the claims, and here is the support or lack of support for this claim. And even, do we think this is a hyperbolic extension of what this case actually says or something like this?

That absolutely speeds up verification. Again, the idea isn’t that every brief or every report is going to be 100% accurate. I think, in fact, us building those products is recognition that it may never be 100% accurate, and lawyers need tools to be able to build trust in the work ultimately so that they can stand behind it and be accountable for it.

And I think that’s really what we’re committed to as we build these products: delivering the highest bar of accuracy that we can, but also delivering the tools that professionals need at the end of the day to be able to trust them.

Greg Lambert (12:25)
I want to get into the announcement of the TR and Anthropic collaboration, which is not a new thing. You guys have been collaborating for a while, but I know with all of the news surrounding Anthropic recently launching into legal directly, can you explain the bidirectional relationship that TR and Anthropic have now and what it means for the people who are using CoCounsel or Deep Research? How is it shifting what they’re seeing?

Joel Hron (12:59)
Yeah, you bet. On the surface, this feels like a big change, but two things. One, as you said, we’ve been working with Anthropic for quite a while, as well as working with OpenAI, Microsoft, AWS, Google, etc. But we have been working particularly closely with Anthropic for quite a while. But the second thing that hasn’t changed is, for us as TR, but also as CoCounsel, we’ve wanted our products to exist where customers are working. And that could be the Microsoft 365 stack. It could be Gemini Enterprise or Google Workspace. It could be Anthropic or Claude Enterprise. It could be OpenAI Enterprise. But I think the idea is that we want our products to exist where people are.

And at the end of the day, these platforms, whether they’re AI platforms or general workplace platforms, are meant to do a lot of different things across the business of law or the business of a corporation. And our goal is really to focus on how do we deliver, again, this fiduciary-grade level to those expert tasks that need to happen, particularly within law, but also outside of law in other industries that we practice in.

So in some cases, CoCounsel, the application interface, is the best way to experience and verify and validate that work that’s happening. But in other cases, where there are general work processes happening, our fiduciary-grade tools support those and act as support agents to that work. And I think our focus is to make sure that intelligence and capability exists wherever it is being used.

And I think that’s how we are thinking about CoCounsel, but that’s also how we’re thinking about making CoCounsel available in other systems. And we see a lot of value in that. The interface layer of software, as you guys have said, has been democratized quite a lot by AI tools, and in particular coding tools and things like that. And we see a lot of firms building their own things. We see a lot of firms and companies consuming general-purpose tools as well and building on top of those. And I think what’s critical is that we deliver that fiduciary-grade intelligence into whatever those systems are, whether they’re our own interfaces or things that people are building on their own.

Marlene Gebauer (15:32)
So Joel, I’m wondering if MCP essentially changes what it means to be a legal tech platform. Claude now has 20 MCP connectors into eDiscovery and CLM tools, for example. And so you never have to leave that interface. We’re seeing this kind of squeeze on this application layer.

You’ve mentioned that as agentic systems get more headless, I guess, optimizing for the single front door is not the right way to go. So does the traditional vertical legal tech application survive this orchestration layer, or is everything becoming more commoditized plumbing?

Joel Hron (16:15)
Yeah, I mean, I would say that MCP changes access, not advantage, if that makes sense. MCP, just like APIs have done, but I think MCP is sort of the analog of API integrations in an agent future, changes how people maybe access this technology, but it doesn’t change the purpose of the technology itself.

And I think, certainly for us, that’s about building solutions that people can trust and building information and knowledge and intelligence that people can trust. So for us, I don’t think MCP changes our job to be done, if you will, as a company, which is about building trust. And MCP is just a mechanism for us to deliver that into more types of work where it’s needed.

And like I said, I think in some cases there is a user experience that goes along with that. I gave the example of Litigation Document Analyzer. Maybe that’s a good example where there’s a distinct experience for how you should do that validation at an important moment. But then there are other cases where the experience may not even be an experience. It may be a workflow that gets triggered automatically off of an email, and a series of steps and work happens, and it comes back as another email.

And so I think we’re moving to a world where in some cases there may not be an experience at all. And that’s what I mean by headless. And I think what we want is that our fiduciary-grade intelligence is playing a part in that process no matter where and how it happens.

Greg Lambert (18:01)
Yeah, let me pull on that a little bit, because we’ve always heard legal vendors talk about work. They want their product to be where the attorneys are working, which is code for Microsoft Word or Outlook, typically. But I think we’re seeing even that shift a little bit, that some attorneys are working directly in the AI tools, or may have their own setup that they vibe-coded that allows them to start working on some things.

So my question is, because of the fact that with Westlaw or CoCounsel, there’s this designed user experience that you’ve set up, that you spent probably millions upon millions of dollars getting just right, so that you have this great experience. And then all of a sudden, your users, or at least some of your users, may be shifting away from some of these really good interfaces that you’ve designed for them. Is that kind of difficult for your UX designers to wrap their heads around?

Marlene Gebauer (19:08)
I was actually going to say, are we going back to more content and capability than the delivery system? Sort of how it was before everything got highly technical.

Joel Hron (19:21)
Well, and maybe to riff on that idea a little bit, Marlene, I don’t know if we’re going back to that, because I don’t know that we ever left that point of view. That has always been the centerpiece of everything we’ve built around, having accurate and up-to-date content.

Marlene Gebauer (19:40)
Well, the position as a content provider versus a technology company, that has sort of gone back and forth sometimes.

Joel Hron (19:45)
Yeah.

Yeah.

But I mean, where we spend millions of dollars is on making sure that our information is accurate and up to date. And in terms of being a content provider versus a technology company, our tools like CoCounsel, for instance, or Deep Research, are not just providing a ranked list of raw content back to an agent. There’s a tremendous amount of technology in terms of how we interpret and apply judgment and apply verification and citation and things like this to that information. And I think that is very much what makes us a technology company, more so than the interface that sits on top of that.

Now, the interface that sits on top of that certainly is changing. And I think the options that people have there are proliferating. I think for our design teams and our people building user interfaces, where the dominant work is legal work and the dominant work centers around the need to verify and build trust through a process, I think CoCounsel will continue to build great experiences for that type of work.

And so I think that’s really, if I’m a design researcher, this is what I’m thinking about: how do I build a user experience that elicits that understanding of how and why this claim is made, rather than just surfacing the claim in pretty font and colors? And so that’s the goal of our design teams. In some cases that’s necessary, and they’ll be in CoCounsel to do that kind of work.

In some cases, maybe that level of depth is not necessary. And that might happen out of an email client, or it might happen out of Microsoft Word, or it might happen in a general-purpose AI tool. And again, I think we’re open to either of those paths because we understand that work can span across those two in different situations.

Greg Lambert (21:54)
Do you think, or I guess, are your developers and designers essentially creating two variations of the content, one that gets surfaced through the UI and then one that gets surfaced through an agent-oriented way?

Joel Hron (22:10)
Yeah.

I think this is a really good question, Greg. And I would look at ourselves the same way I think Anthropic looks at themselves. They are a model provider first, and their job is to build models and tools around the models and make them available to builders. And then they’re building Claude Enterprise, the application. And Claude Enterprise, the application, is their best expression of the model. So this is a user interface that expresses the capabilities of the model in a way that allows the user to get the most out of what that model is capable of doing.

And I see our job very much the same. We take models from providers, but we build harnesses and tools around them and under them to be able to have legal capabilities that the base models themselves don’t have. And then our UI, and that’s sort of what is available via MCP, CoCounsel Legal is the agent, and it can do a variety of different things from a legal capability standpoint. Users can access that via MCP and plug it into different places, but CoCounsel, the UI, is our expression of that agent and how we believe a lawyer can get the most out of that agent for certain types of tasks.

And so that’s really how I see it. I don’t think they’re conflicting in any way. I think they’re both useful. One team is really optimized on how do I hill-climb the capabilities of this legal agent by giving it access to expert-level tools and systems. And the other team is focused on, how do I build a UI that expresses the capabilities of this agent in a way that is most useful for a human to interact with it?

Greg Lambert (24:05)
It seems like we have the two teams that are doing that. How well do they learn from one another? Because it would seem like there are certain ways that you’re surfacing information to a human that may also be relevant to the agent, and vice versa.

Joel Hron (24:22)
100%. They work very, very closely with each other. I would say most of the development we do today really starts with the agent. Most of how we think about solving legal problems starts with what is the agent capable of doing? And I think as we build UIs that express those capabilities, those UIs convey obvious gaps.

And some of those gaps can be filled or mitigated by the UI and how we construct the UI and how we construct the workflows within the UI and how we construct things like customization via skills and stuff like this. And some of those things need to be fed back into the agent team to say, okay, well, we need better tools to handle these sorts of edge cases, or we need better behavior for XYZ sorts of use cases. And so there’s a two-way conversation that happens between those teams.

I think the other thing that’s important as you think about agents is the context engineering for the agent is incredibly important, right? The agent is operating off of what it is discovering throughout the process of doing its work. And the human has a lot of context that the agent does not have. And in many ways, the UI itself is a way to help the human user convey their context to the agent in a way.

Just like if you were to hire a new intern at your company, you would probably set up some shared folder with them. And you would say, okay, here’s some recent documents we’ve put together, and here’s an onboarding document. That’s you conveying context to this intern to help them understand, well, this is what we’re doing, and this is why we’re doing it, and this is how we’ve done it in the past. And that’s the same thing that you want to elicit between a human user and an agent. And that’s what I think you guys are helpful at exposing as well.

Greg Lambert (26:33)
One final question on this topic. Just curious, if you were to put a percentage on it, on the coding that your developers do, how much of that are they relying on the AI now to do?

Joel Hron (26:47)
Yeah, this is a great question. Honestly, I have had other podcasts about this topic solo, and we could spend hours on it. We have an OKR in our organization that more than 50% of the pull requests that go into our codebase get written by AI. And I would say some teams are north of 80% at this point.

There’s a really interesting reason, though, we set this OKR. And sorry if I’m taking a tangent. You can pull me back into legal at some point if you want.

Greg Lambert (27:21)
We love OKRs on here.

Joel Hron (27:44)
Okay, so there’s a really interesting reason we chose 51%. And one of the engineers that is on my leadership team mentioned this to me back in December, and it really stuck with me. But he said something really changes about your mindset when you get to 51% of the code being written by AI, because now you, as the human user, are no longer in control of the code that gets written. You have ceded controlling interest of your codebase to something that is not you.

And it really is a good signal for, okay, well, how does your role as an engineer now change? Your role now as an engineer is less about writing lines of code and it’s more about building systems around how code gets written. And those things are governance systems and tests and guidance documents and architecture principles and things like this that help constrain and steer and guide the agent to do the right thing along the way.

And I think it’s a really good analog for how a lawyer should think about their role changing or how a tax professional may think about their role changing. As AI sort of picks up more and more of this grunt work, if you will, your job is more about how do you build systems around AI to do the work you want it to do in the way you want it done, rather than doing the work itself, right? And I think that’s really the mindset shift for an engineer that is taking shape right now. And I think it will likely take shape in other industries over the years to come.

Greg Lambert (29:06)
Yeah, that’s a good parallel. Thanks.

Marlene Gebauer (29:09)
Anthropic just shipped 12 practice area plugins covering everything from corporate law to litigation, and some deploying as managed agents. There’s a fine line between being the partner and being the competitor. So when a lawyer is using Anthropic’s native open-source corporate legal plugin versus routing that workflow through CoCounsel Legal, what’s the functional difference in output trust and defensibility? And I know you’ve talked a bit about this, so maybe you can do a compare and contrast.

Joel Hron (29:48)
Yeah, for sure. What I would say is that if you go look at these plugins, they are nothing but a rudimentary set of instructions for how to do a certain type of work. So I would say the plugins themselves have no concept of validation or verification or groundedness in factuality in any way.

They are helpful guides to an agent to help it meander through a task, but they do not have any concept of these principles, I would say, of fiduciary-grade AI. Now that’s not to say that one couldn’t take a plugin and say, hey, use these tools to validate your work along the way. And those tools could be CoCounsel MCP, living in a plugin.

I think you could do something really well in that context. And I think that’s how we think about MCP in the context of Claude, if you will. You can bring CoCounsel’s capabilities into a lot of these workflows in a native way and get the best of both. But the plugins as a standalone, again, are nothing more than a couple of instruction documents for the agent in terms of how to follow a path. And again, I don’t think they get to the level of depth and trust and transparency that we hear our users are looking for.

Greg Lambert (31:16)
I’ve heard people joking that these are the skills that their in-house legal department uses at Anthropic, but with all the really good stuff pulled out of it. It’s very basic.

Joel Hron (31:29)
Yeah. And look, I also don’t think Anthropic’s goal is to build a legal product per se that covers the spectrum. I think that’s why they are basic. I think they’re meant to be indicative and instructional around, here’s how you build guardrails for an agent, or here’s how you build workflows for an agent. You can take this and then make it much, much better. But here’s the seed of an idea, and you can then go use the platform to grow and expand and think about it in different ways.

And so I think that’s more of the message to take from the plugins than, here’s a legal product that stands on its own two legs. And again, I think our goal is to build tools that work in the context of that system and can be used to add validity and trustworthiness to whatever processes are happening there.

Greg Lambert (32:27)
Let me tag on to that, because one of the interesting things that they did put out was this thing that they call the cold start interview, that you can take a firm’s specific playbook and put it in, and Claude will write that to their Claude markdown files. And suddenly, when I say this out loud, I just envision my security ops person coming in and ripping my computer out of the wall and shutting everything down if I were to do this.

Marlene Gebauer (32:55)
It’s like, no, no, no.

Greg Lambert (32:57)
But the firm’s institutional memory can get encoded directly into these LLM instructions rather than vendor software. Putting back on your CTO hat, not that you’ve taken it off, how do you advise firms on things like this and what it is they should be doing with their proprietary information?

I’m sure you have a preference that they put it into what you’re calling the fiduciary-grade system rather than the LLM itself.

Joel Hron (33:35)
Well, certainly. I would say security is one of the things, I think probably security and trust are the two things that stand between...

Greg Lambert (33:47)
Yeah, that’s why we don’t talk about Grok as an enterprise tool, I think.

Joel Hron (33:50)
Right. So it stands between the real-world application of AI and the capability, perhaps, of AI today. And so I think certainly part of what the definition of fiduciary-grade is does speak to security and how we use that information. And we’ve been very clear that we don’t use that information in terms of training our models and products and things like that. And I think customers really appreciate the stance that we’ve taken on that. And I think they honestly trust us quite a lot because of the decades of work we’ve done with them on that front. And I would say we’ve earned that trust in many ways, and we try to re-earn it every day to keep it.

And so I think as a firm, this is your competitive advantage. In the future, if you can codify your knowledge, and we’ve talked about knowledge management as a domain of law for a while, but if you can codify your knowledge in a way, and you can do it better than the next person and make that knowledge available for AI in agent-native ways, then I think you have a tremendous competitive advantage.

And I would be very reluctant to, A, take that task lightly. And I would be very reluctant, B, to open with that approach. If I’m a law firm or a big corporation, this is something I want to be an expert at, because it is your lifeblood as a company at the end of the day. And I think if I’m making investments as a company anywhere, it’s going to be in this area. And then how do I serve that knowledge and intelligence then? I can choose a million different tools to serve it into. Owning that knowledge is really what differentiates you at the end of the day as a company. And certainly if I was leading a law firm or a big corporation in that sense, that’s what I would be focused on a lot.

Marlene Gebauer (36:00)
It’s funny, we just had this conversation yesterday with Ryan McClead about making sure that knowledge and content is AI-accessible in addition to being people-accessible. So, yeah, I agree. It’s going to be something important.

Joel Hron (36:16)
Well, and I’ll use this analogy, I don’t know if it will stick, but you could think of TR very much as a law firm. So a law firm has a lot of experience and matters that they’ve worked on over the course of time, that they want to index and organize and make sense of to inform what future work they do. That is the same job that we do with case law and statutes and regulations. We just happen to do it with most jurisdictions across the world.

And that is absolutely what we still believe differentiates us as a company and differentiates our products at the end of the day. And the better that we can organize that information and make it available, as you say, for AI, I think the better our products become. And in many ways, we have changed the foundational aspects of how Westlaw works for agents versus how it used to work for humans, and the APIs that the agent calls are different than the APIs that power the application today because agents work with the content in a different way, at a different pace, and at a different rate than humans do. And they need to look and feel and act differently for an agent user versus a human user.

Marlene Gebauer (37:33)
So Joel, you’re a bit of a unicorn. You sit at this intersection of legal content, AI infrastructure, product development, and figuring out what lawyers actually need from these tools. So what are a couple of resources, signals, or conversations that you rely on to separate the real movement in legal AI from a lot of the noise?

Joel Hron (37:56)
Yeah, it’s a really good question. I mean, I would say I read a lot. So I still enjoy following LinkedIn or Twitter. I do wish I practiced fishing more than I read about it, but I try to do both.

Greg Lambert (38:07)
And say you read a lot about fishing too, right? I see the books.

Joel Hron (38:21)
But I would say I do read a lot, and I think there’s a lot of really good content, particularly academic papers, that come out and people reference them on Twitter and things like that. It’s a good source of figuring out what to go read. But I think that’s a good way to stay up to speed on what’s happening in the market.

I think the most important thing, though, is that I use it. I think the most amazing thing for AI for me as an engineer has been, and I used to run a startup before we were acquired by TR, so at a startup with less than a hundred people or so, it’s quite easy to stay deep in the code and involved in how things work. Then you come into TR, and we’ve got 140 products and thousands of engineers. It’s impossible to maintain the level of depth in code. But with AI tools now, when I’m talking to a team and they have an issue, I can immediately go understand the code and what’s happening and what has happened in the last few weeks.

What commits have been made? What issues have come up? My level of depth in the code itself is far more than it could have ever been otherwise because of the ramp-up time and context switching. And that has been such a blessing for me as a technical person, to be able to do that. But I also learned so much by doing that about what is possible and what is capable. It gives me an intuition as well for where things are going and how I think I should be directing teams and this kind of stuff.

And so that would be the best piece of advice I would have for people: whatever it is, go use it, and use it a lot. The more you do, the better your intuition becomes for where this is going and what impacts you think it might have on your teams, your talent, on the products you’re building, etc.

Greg Lambert (40:16)
Yeah, I couldn’t agree more. Well, Joel, it’s time for our crystal ball question. I think we’ve thrown this at you before, but there’s so many things going on with the AI harnesses, the agents, the MCPs, the collaboration between foundational models and products. So what do you think is something on the horizon that legal professionals need to be looking out for and preparing for?

Joel Hron (40:49)
Yeah, it’s a good question. I think, obviously, agents is the trend. I think it’s easy to say that, but if you think about what does that mean, that means now that I am delegating actual work and decision-making to an AI, delegating actual work product to a non-human. And this is exactly like what I said with engineering as well. Your role now changes from doing the work to governing the work.

And so I think the change that you need to be anticipating is, okay, in a world where I’m no longer doing the work, how do I build the systems that give me the trust to stand behind the work that gets created? And for engineers, those are things like architecture design principles, high test automation coverage, etc. For lawyers, it’s other things. And I think for those people doing the work, it would really behoove them to spend a lot of time thinking about that.

And again, that’s what we believe we’re building in our tools: systems that can elicit and elucidate that transparency and trust that’s necessary to build systems around that kind of environment. But I think really thinking about what it means to have an agent do work, and what are the implications if you think three or four steps down the road, that’s what people need to be preparing for today.

Greg Lambert (42:23)
Tell everyone to keep track when the AI takes over 51% of your work. Your role changes, right?

Joel Hron (42:27)
And whether 51% happens or not, it’s kind of irrelevant. It’s more of a helpful mental exercise to say, what if I am no longer the controller of this system? What would I do? And that’s probably a good mental model for some actions that you might want to take.

Marlene Gebauer (42:47)
And I like the way you phrase it. You’re governing the work. I’ve heard, okay, you’re supervising or you’re checking. I think governing is a better word for it, given what you’re saying about you really have to think about these systems. What’s going to make you trust it? And think about that in terms of how to govern it.

Joel Hron (42:51)
Right.

Right.

Well, and again, I mean, the accountability does not shift away from the human in this process, right? And so that’s why I think governance is a good word. Because at the end of the day, if we ship a bug, it’s not like calling up my AI model and telling it how bad of a job it did. Engineers are accountable.

Greg Lambert (43:29)
Claude, you’re fired. Codex, you’re in.

Marlene Gebauer (43:29)
You messed up, Claude.

Joel Hron (43:32)
That’s not what happens.

I think humans maintain accountability. And so it’s incumbent on us to really lean into, how do we maintain that accountability in a way that we can stand behind and trust?

Greg Lambert (43:43)
Joel Hron, Chief Technology Officer at Thomson Reuters. Thank you very much for coming in and unpacking. Man, there’s a lot going on. It’s an exciting time to be in the industry, isn’t it? You bet.

Marlene Gebauer (43:54)
Hmm.

Joel Hron (43:55)
Absolutely. Thank you for having me, Marlene and Greg.

Marlene Gebauer (43:59)
And thanks to all of you, our listeners, for taking the time to listen to The Geek in Review podcast. If you enjoyed the show, please share it with a colleague and don’t forget to like and subscribe. Joel, where’s the best place for people to follow your work and learn more about what Thomson Reuters is doing with CoCounsel Legal?

Joel Hron (44:17)
Yeah, I try to put a lot of updates out on LinkedIn pretty regularly about new things that we’re doing and shipping and partnerships and things like that. So definitely I would say follow me or follow Thomson Reuters there. And I would also say you can gain access to the next versions of CoCounsel, which are in beta here pretty soon. And you can check that out on our website and try to get access. It’s something we’re quite excited about right now.

Marlene Gebauer (44:43)
All right, I’ll definitely encourage everybody to check out what you’re doing and talking about. And I should note, as always, the music you hear is from Jerry David DeCicca. Thank you so much, Jerry, and bye, everybody.

A few weeks ago I ran the numbers on the token cost panic. I took the scariest figure in legal AI, the finding that agentic workflows burn a thousand times more tokens than a chat query, and followed it all the way down to a dollar amount on a real deal. The panic did not survive the arithmetic. The piece is here if you want the full walk-through.

This is not that piece. The panic has moved on since I wrote it, and the new versions are smarter than the old one. The thousand-times number has quietly retired, because a thousand times almost nothing is still almost nothing. In its place are three fresher anxieties, and they deserve a real answer. The first says the model makers have a monopoly now, the price of a token is climbing, and it will climb forever, so you had better lock in a flat rate or build your own models before it does. The second says forget the price of a token, watch the meter: every time the AI reads your contract it ticks, and a long agentic session reads your contract over and over and over. The third does not bother with an argument at all. It just points at a number. One company spent five hundred million dollars on AI in a single month, and the number is so large it does the panicking for you.

All three are wrong. They are wrong in more interesting ways than the original, which is the only reason I am writing this down instead of linking to the first piece again. But underneath the new costumes it is the same body. Every version of this panic makes the same mistake and reaches the same conclusion. So let us stop swatting the individual numbers and name the thing that keeps generating them.

The Mistake Underneath All of It

Here is the error, stated once, because everything below is a variation on it.

A token is the unit a model uses to bill you. It is not the unit your work is measured in, it is not the unit your client pays for, and it is not the unit anything you care about is denominated in. It is a meter reading. The entire genre of token panic consists of staring at the meter reading as though it were the fare, the destination, and the quality of the ride all at once.

It is not any of those things. It is the meter. And a meter, by itself, tells you nothing about whether you are getting a good deal. A taxi meter reading of forty dollars is a bargain to the airport and a robbery around the block. The number on the meter is the least informative number in the entire transaction, because it means nothing until you put it next to what the ride was worth. Every piece in this genre forgets that, and forgets it in a slightly different way. Let me take them in turn.

“Prices Only Go Up”

Start with the monopoly story, because it has a real fact inside it. Yes, the newest frontier model costs more per token than last year’s newest model. That part is true. What the story does with it is the problem.

It draws a line through two dots and calls it a trend. Frontier prices up, therefore prices up forever, therefore lock in a flat rate before the meter eats you. But you are watching the wrong number. The price of a frontier token is not your cost. Your cost is what it takes to finish a task, and the cost of finishing a given task has been in freefall for two straight years. The same capability that ran on the most expensive model available in 2022 runs today on something on the order of two hundred and eighty times cheaper. Last year’s frontier is this year’s mid-tier is next year’s free default. The token at the very tip of the frontier gets a little pricier each release; everything behind the tip collapses in price behind it. Gartner expects another ninety percent drop in inference cost by 2030.

Watching the frontier price and concluding that AI is getting more expensive is reading the thermometer and announcing a fever, while ignoring that you are holding the thermometer over a candle. The evidence that the baseline is getting cheaper often sits right there in the same articles raising the alarm, quoted from the experts and then left unaddressed. You do not build a cost strategy on the one number in the system that is engineered to always be the highest.

“Watch the Meter Tick”

The second version is more seductive, because it comes with a picture. There is a meter. It is running. It is not visible and nobody is watching it. Every question you ask, every document you paste, every time the model reads back over the contract, the meter advances, and an agentic session is one long ride with the meter buried somewhere you cannot see it. Be afraid of the meter.

It is a good picture. It is also describing a machine that was rebuilt about a year ago.

Here is the mechanism the picture leaves out. When an AI reads a long document, that document is loaded into its context once, at full price. Every subsequent time the model reads back over it, that is a cached read, and every major model provider bills it at a discount, anywhere from fifty to ninety percent off, depending on the provider. You pay full freight to put the contract in the room once. After that, every time the model reads back over it costs a fraction of that first pass.

So the entire horror story, the one image every version of this panic is built on, the machine reading your document over and over while the meter spins, describes a problem the providers fixed before most of these pieces were written. The agent that reads your contract fifty times is not paying fifty times to read it. It pays full price once and a steep discount on the other forty-nine. The panic prices every one of those reads at full freight; the actual bill is a small fraction of that. The meter is real. It barely moves on the part everyone is pointing at.

I want to be precise here, because imprecision is how someone discredits a whole argument over a footnote. Cached reads are discounted, not free, and a long enough session still adds up. The cache expires after a few minutes to an hour, so the discount lives inside a working session rather than forever. None of that rescues the panic. Agentic work is high-frequency, same-session, re-reading-the-same-context work. That is the precise workload the caching discount was built for.

“But Look At This Number”

And then there is the half-billion-dollar bill.

The story made every outlet, because it is built to. One company, unnamed, spent five hundred million dollars on Claude in a single month. It is worth knowing where that number comes from: a single AI consultant, quoted in a single report, describing a client they do not name. No company has confirmed it and no one has verified it. That did not slow it down for a second. The number is enormous and the number is unverified, and it is being passed around as proof that AI costs have slipped the leash and big companies are getting torched.

So let us take it at face value anyway, because even granting every word of it, the story argues the opposite of what it is being used to prove. Read the second sentence and it falls apart in your hands. The company spent five hundred million dollars because it put no usage limit on the licenses. Thousands of employees had unlimited, uncapped access, and for a month nobody looked at the meter. That is the story. That is the whole story.

This is not a company that got beaten by the cost of AI. This is a company that took its foot off the brake, tied the steering wheel down, climbed into the back seat, and then expressed surprise at where the car ended up. The number is not evidence that AI cost is uncontrollable. It is evidence of precisely the opposite, because the controls exist and this company chose not to use a single one of them.

The controls are not theoretical. Claude Enterprise ships four levels of spend control: an organization-wide monthly cap, group caps, caps by seat tier, and individual per-user caps. They are hierarchical, so a user cannot exceed their own cap, their group’s cap, or the organization’s, whichever is lowest. When someone hits the limit, they are blocked. That is finer-grained cost governance than most firms have ever had over their e-discovery spend, and every serious enterprise AI platform ships some version of it. It is a shipping product, not a roadmap promise.

A company spent half a billion dollars by switching all of it off.

A company that gives thousands of people uncapped access to any metered resource and looks away for thirty days does not have a token problem. It has a management problem wearing a token costume.

And there is a second reason the number should not frighten you, specific to the kind of work most professionals are actually doing. That bill came from unattended automation: thousands of processes looping on their own, re-reading and retrying around the clock, with no person waiting on any single step. That is the only kind of AI spending that can run away while you sleep, because it is the only kind with no human in the loop to stop it. Attended work, a person at a desk directing the tool and reading the results, is capped by something the runaway scenario removed: human attention. Even a whole firm only has so many working hours in a month. I have tried to reach half a billion dollars a month with nothing but attended sessions, stacking every worst case, the largest firms, the most expensive model, every lawyer running flat out all day with the caps off, and the ceiling lands in the low single-digit millions, an order of magnitude short, and even that requires assumptions no real firm would survive.

The only way to clear that ceiling is to take the human out of the loop. Here someone will object that firms will simply start running headless processes of their own, looping unattended like the company that ran up the bill. They will not, because there is nothing for such a process to do. The runaway scenario needs a task with no natural end, something that can spin on itself for a thousand hours and always find more to spend on. Legal work is not shaped like that. A contract gets reviewed and marked up and it is finished; there is no ten-thousandth pass, because there is no ten-thousandth version. The deliverable has a floor, and the floor is the cap, whether or not anyone is watching.

And if none of that reassures you, there is a backstop that costs nothing to set. Cap each person at some deliberately absurd number, ten thousand dollars of usage a month, and forget about it. No one doing supervised legal work will ever come close, so it never touches real use. But it is not really a spending limit. It is a smoke alarm. The day someone actually hits ten thousand dollars in a month, the cap has told you something has gone wrong, a broken process, a headless loop someone set running, a mistake worth finding, long before it becomes a number worth fearing. The company in the story did not lack a way to prevent the bill. It declined to use one.

The most-cited number in the entire panic turns out to be the best argument against it.

The One True Thing

Now let me give the panic its due, because there is a real fact in here and I am not going to pretend otherwise.

Your AI bill probably is going up. Not the per-token price, the bill. Even as the price of intelligence collapses, total enterprise spend on it has risen sharply, because the work has moved from a single chat answer to an agent running an entire multi-step task, and that consumes vastly more tokens. That is real. That is the genuine signal buried under all the noise, and the people watching their invoices climb are not imagining it.

But look at what the climbing number represents. The bill went up because the machine stopped answering a question and started doing the job. Two years ago the meter measured a chatbot composing a paragraph. Today it measures an agent reading the deal room, drafting the issues list, checking it against the precedent, and revising its own work. Of course the meter is higher. It is doing thirty times the work, because there is thirty times the work being done, work that used to belong to a person and a timesheet.

So the question was never “why is the meter higher.” The question is the one this entire genre is constructed to avoid: what is the meter now doing that it could not do before, and what did that work cost you the last time a human did it? Put the number as high as you like. Four hundred dollars a deal, four thousand, forty thousand: the question does not change, and neither does the answer, as long as the work it replaced cost you more. A bill that tripled while absorbing the work of a first-year associate is not a cost problem. It is the best trade your firm made all year. You only get to be horrified by the number if you refuse, the entire time, to look at the other side of the ledger.

What Actually Deserves Your Attention

There is a version of cost discipline that is not panic, and it is worth naming so it does not get lost in the noise.

Use the cheapest model that does the job. Route the easy work to the small model and reserve the frontier for the tasks that need it. Do not paste the entire deal room in to summarize one clause. Cap your users. Watch your meter, not because the meter is the enemy, but because watching the meter is just management, and a firm that cannot see its AI spend by matter and by practice group should go build that visibility before it signs anything. None of this is glamorous. None of it sells a product. Nobody is going to write a breathless thought piece urging you to right-size your model selection, because there is no panic in it and no vendor on the other end of it. It is just the unsexy discipline of knowing what a task is worth before you run it, which is the same discipline the profession has always claimed to have and rarely does.

That is the whole legitimate concern. It fits in a paragraph. Everything past it is theater.

Why It Keeps Coming Back

Here is the part I actually want you to take away, because it will outlast the next five versions of this.

The token panic recurs because it is the comfortable debate. It lets a roomful of smart people argue urgently about something that does not threaten anyone. It is easier to compare per-seat pricing against per-token pricing than to ask what happens to associate leverage when the work a first-year used to bill for is absorbed by a machine that costs four hundred dollars a deal. It is easier to fear a meter than to ask who, exactly, captures the value when AI makes a partner ten times more productive: the client, the firm, or the vendor. It is easier to publish a chart of rising token prices than to sit with the fact that the entire economic structure of the firm, the leverage pyramid, the billable hour, the margin built on associate hours, is the thing actually being repriced, and the tokens are a rounding error inside that story.

The panic is a place to hide. Every few weeks it comes back wearing a new number, because the number is never the point. The number is the thing people reach for so they do not have to look at the ledger underneath it.

So do the boring things, the ones that fit in a paragraph, and then put the meter down and go have the uncomfortable conversation. That is the one that decides which firms are still standing in five years.

The tokens were never going to.