Joe Moura, CEO at Crew AI

If it’s a problem that the system gives a different answer each time, then maybe it shouldn’t be an agent.

This one cuts through a lot of the hype. Agents are amazing at many things, but it doesn’t mean they are the right tool for everything. Business systems need determinism (audit trails, controls, predictability, explainability etc). Agents are fundamentally non-deterministic.

Does this mean that agents suck? Absolutely not. If you need to do an API call then there are simpler ways of doing it, but if you need some open-ended research then an LLM call is going to be a better choice.

It squares with other stories I heard of people starting out with the naive view that agents would solve everything, only to then realise that they had to add in lots of old-school deterministic code to make things work robustly.

Malte Ubl, CTO at Vercel 

I didn’t write down his quote verbatim, but the thrust was this: He is seeing interns outperforming engineers with 10 years of experience because the interns are comfortable working with AI agents in a way that some established software engineers aren’t.

This is a good antidote to the general story out there that junior software engineers are disappearing.

And it wasn’t just Malte making this point – I heard it from a lot of other places too.

Feels exactly like starting a new Claude Code session. Claude keeps a patchwork of markdown files with its “memory” in. As an example, one of my projects has a MEMORY.md and 11 other markdown files in its memory folder. Every time a new session starts, Claude has to find and re-read the relevant bits of memory. Every time it gets something wrong it has to update its memory. Handling memory in your AI agent is a big deal these days: here are some examples from a quick search of people working on this problem.

Until this is a solved problem, working with an AI agent feels like working with the protagonist in Memento, except with markdown files instead of tattoos.

Contrast that with a human being. A human being isn’t as fast at writing code, but a human being learns over time while they are doing the job. A human being might break production once but will learn from that and won’t do it again. An AI agent? Not so sure.

Two conclusions:

1. We are miles away from AGI. For AGI to happen, AI will need to learn from experience in a similar way to humans. Markdown files and vector data stores ain’t it.
2. Companies who are replacing humans with AI today need to think very carefully. They are risking their longevity. Sure, today is the worst the models will ever be, and models might be mind-blowing at specific tasks, but fundamentally AI models are not going to learn your business through experience the way human beings will.

So the design is built around three defences.

Anonymise before judging. The five providers (Exa, Tavily, Linkup, Perplexity and my own, Syracuse) have their names shuffled and replaced with the letters A–E before any data reaches the model. The decode key is written to a local file only after scoring, and it’s re-randomised every run, so A in one run isn’t A in the next. The judge literally cannot defer to “mine” because it doesn’t know which letter is mine.

Force a verdict, ban the hedge. The judge is told explicitly that “different providers suit different needs” is not an acceptable conclusion. It has to produce a strict 1-to-5 rank order, back every negative claim with a specific example article, and describe exactly what each lower-ranked provider would need to fix to reach first place. That last instruction makes it articulate concrete gaps instead of waving vaguely at quality.

Hold every provider to the same bar. The same criteria apply to all five: precision (wrong-entity false positives), coverage of obscure companies, date accuracy, whether the summary is usable without clicking through, source quality, paywall accessibility, and hallucination risk. A provider with lots of undated or stale results can’t rank first no matter what else it does, because undated news is non-actionable.

Did it work? Well enough that the benchmark cheerfully tells me where I lose. My product wins on company news and comes mid-table on industry/region news, and the write-ups for the runs where I rank last are as unsparing as the rest. That seems a decent signal that the anti-bias machinery is doing its job.

It’s all open source if you want to poke holes in it or point it at your own provider. https://github.com/alanbuxton/news-comparison.

The product itself is Syracuse Company News.

Yet they both used the same word to describe a great data person: “Curiosity”.

A curious data person is someone who spends time in the data, finds interesting patterns and then uses that to create insights. Whether you’re a data analyst working for a large company or a data journalist writing investigative reports, curiosity seems to be the common thread.

You could also say that “curiosity” is an important trait for any product person or software engineer in today’s world. Thanks to AI, the cost of exploring a hunch is basically zero, so coming up with hunches becomes more and more important.

Interestingly…. of all the things that people praise AI for, curiosity is not one of those things. Curiosity remains a very human attribute.

The hype is overbearing. There is no end of people writing about the business they launched last weekend off a vibe-coded app.

With all this noise, it’s rational to ask yourself “do I need a vendor to provide this product for me, or can I code it myself?” Especially when, after your first few hours of vibing, you can get something that will inevitably blow you away.

But Chamath’s 80/90 rule holds. With an AI tool, you can do 80% of your vendor’s functionality at 90% less cost. For many people that will be just fine. But it will turn out that a lot of people need 81%. The question is how long they spend experimenting with vibe-coded apps before they come to that conclusion.

That’s why I think SaaS is seeing more of a pause than a collapse. Probably for the next 6-12 months many organisations will sink a lot of money into experimenting with AI tools. Most will realise that running your own back-office systems is a world of pain. Anthropic already know this. They didn’t create their own sales support tooling. They use Gong.

The patterns are similar to what we’ve seen before. A new technology meant “anyone could do X”. It didn’t mean that it made sense for everyone to do X. Instead, some organisations figured out ways of doing brand new things, enabled by this new tech.

In the 1990’s anyone could build a website². All you needed was to know a bit of HTML. This led to a flurry of websites, at least in some sectors³. But it didn’t mean that everyone ended up with their own website. It meant a few companies figured out how to create new types of product that didn’t exist before.

In 2005, Web 2.0 was on its way. David Heinemeier Hansson released a video about how anyone could build a blog site in 15 minutes. All you needed to know was this really simple tech stack he had created called Ruby on Rails. This didn’t mean that everyone created their own blog site. But it enabled new types of product that didn’t exist before. As an example: Twitter only exists now thanks to how quickly programmers could ship code with Ruby on Rails back then.

Fast forward to today and we’re in a similar place. Anyone can build a web-based business. All you need is a prompt, or perhaps a few Markdown files. Does this mean that everyone will? If history is any guide, clearly not. A few organisations will figure out new things that we couldn’t do before and they will win big.

But, for some reason, the great and good think this time it’s different.

Truth is, it is different in one important way. But not the way that they probably think.

Back in the Web 1.0 and Web 2.0 days you could take a prototype and be pretty confident that you could build it out into a real product by investing in more programmers⁴. That logic doesn’t hold in a world of prompt engineering⁵. There is little predictable relationship between a prompt and the result. There is no shortage of people online advertising their prompt packs. At the same time, this guy talks to his Claude in “caveman” to save on tokens: “Same fix. 75% less word. Brain still big.“

You can’t take an LLM and a prompt and predict how much you need to spend on tokens and how much you need to spend on engineering to build it out into a real product. From what I’ve seen you need a fair amount of good old fashioned engineering. Anthropic already know this. There is a lot of traditional if/else in Claude Code.

The question is how long everyone else takes to also figure it out. The trouble is that there is plenty of money to be made selling more and more shovels to people out there vibing for gold, so I expect it will be a while yet.

Footnotes

1. On one hand I work for a SaaS company so perhaps I’m biased. On the other hand, I’ve been building products through Web 1.0, Web 2.0 and with AI and text processing since 2015 which also colours my perspective. ︎
2. At the Science Museum in London they have a video of Sir Tim Berners Lee explaining just this – how easy it was for anyone to make a website. I can’t find it online so you’ll have to believe me. Or visit the museum. ︎
3. My 1995 highlight was writing the first website for Hydrogen Dukebox, a record label. ︎
4. In fact, back in the mid-2000’s someone once told me that the rule of thumb for valuing a company was $1m per engineer. ︎
5. Or “prompt shamanism” as an engineer I work with calls it. A far more accurate term than engineering. ︎

Background: I wanted to compare Syracuse against some other similar providers. In my last test run, Syracuse didn’t do great. So I got AI to help:

1. I got Claude Code to write me some code to run comparisons. Syracuse was the winner for companies but didn’t do great for industries/locations.
2. Then I got Claude Cowork to analyse the results and identify gaps.
3. One of the gaps it pointed out was that Syracuse scrapes fewer sources than the established players. It gave me a list of sources to prioritize.
4. I got Claude Code to write spiders for those sources.

So far so good, but now I’ve got a new problem. Claude Code is writing so many spiders for me that I can’t usefully keep on top of them any more. I asked Claude Code how to organise them better. It created subdirectories for them and explained how those subdirectories need an empty __init__.py file in so that scrapy can find them.

At this point I could have stopped but something felt not quite right.

Let’s say I wanted to reorganise the spiders, created a new folder and I forgot to put this __init__.py file in. Any spiders in this directory simply wouldn’t run. There would not be any errors or notifications that they didn’t run. The spiders would be completely invisible to scrapy.

I told Claude Code to write some tests to make sure this didn’t happen. It obliged and made some tests that I could run which would give an error if any expected subdirectory didn’t have this file in.

Great, but what if something changed and I didn’t end up running this test file, or if the test was inadvertently deleted? Then I’m back at square one, hoping that all the subdirectories have the magic __init__.py file in.

So I told Claude Code to update the application to check the spider subdirectories every time it starts up and to abort if an expected subdirectory didn’t have the special file. And then to write some test code to validate that this protection would always work. It did this task very quickly and very well.

The point is that Claude is pretty good at following instructions. It can write a lot of code very fast. The cost of writing code is now roughly zero.

The bad news for software engineers is that “producing code in response to a specification” is now something that AI can do. If you see your software engineering job as taking a spec and implementing that into code then it’s only a matter of time before you get replaced by AI.

The good news is that producing code in response to a specification was never really the most important part of being a software engineer. The most important part, certainly over the past 20 years or so, if not longer, is figuring out what problems to solve and how to best fix those problems. Not all problems are worth fixing, and not all problems are worth fixing with fresh code.

This isn’t necessarily a “senior engineer” skill. It’s about not being scared to ask dumb questions about how the code might behave or misbehave. It’s about thinking through two or three levels of “am I really solving the whole problem” and “what could go wrong”.

So I am feeling really positive about what this means for us. It means we can focus on the more interesting problems.

Hat tip at this point to Seb Bacon who discusses this topic from the point of view of academic research and public sector services https://bacon.boutique/sloponomics-and-the-coming-storm/. His post is well worth a read.

And from Syracuse’s point of view? The even better news is that, since Claude’s latest intervention, Syracuse is now top of the pack for cleanly-structured, non-hallucinated, relevant, auditable data for agents that need to operate on company or industry news. If you’re building an agents that need this sort of data to work on then you should check it out.

My favourite definition was Tesler’s Theorem:

AI is whatever hasn’t been done yet.

Once it’s been done, it’s “just software”.

This has been a good rule of thumb over the past decades. Something is considered magical, researcher figures it out, everyone takes it for granted and agrees that it didn’t need human-level intelligence after all. For example think of number plate recognition or self driving cars. Things that 20 years seemed to need a high level of human-level understanding turned out to be solved with “just (clever) software”.

This all changed in 2022. That was the year that that language models became rapidly more capable. In June 2022, a Google engineer thought their chatbot was becoming conscious. Meta launched Galactica which was amazing for about 24 hours before it turned into a shambles. Then ChatGPT came on the scene in November.

And everything changed.

On the one hand we saw a machine that writes text as if it’s talking to you and collectively we said “Ah, this is AI. It’s here”. Today, when anyone says “I’ll use AI for this” they mean a ChatGPT type of experience: something that can communicate with you using normal language like a friend or co-worker would.

On the other hand, it wasn’t what we imagined AI would be. It’s far from the sort of science fiction level of consciousness that you would have imagined an AI to be.

Kent Beck, the software engineering guru, likes to call his GenAI coding assistant his “Genie“. It’s a neat term that captures what it’s like working with what we call “AI” these days: powerful, mysterious, frustrating, capricious. It’s more like alchemy than science fiction.

This is absolutely not what AI was supposed to be. Far from being super smart, AI today comes with disclaimers. If someone asked you 10 years ago how you imagined “AI”, I bet you wouldn’t have said “it will come with disclaimers about not getting stuff right”. But that’s where we are:

Claude:

Copilot:

ChatGPT

VSCode

Now it’s here, it’s just software. And fallible software, at that.

Does this disprove Tesler’s Theorem? Not at all, but things have got messier.

1. Now that “AI” specifically means the ChatGPT-type of experience, we needed a new term to describe the sorts of AI we saw in science fiction films. Enter “AGI” or “Artificial General Intelligence”. AGI is still what hasn’t been done yet. Tesler’s theorem holds, just change “AI” to “AGI”.
2. Most people aren’t terminally online and aren’t up to speed with the latest moving of definitional goalposts. So most people are having to hold both “AI is the stuff of science fiction” and “AI is prone to errors” in their heads at the same time.

It’s a recipe for confusion and mis-matched expectations.

Bad news: The questions highlight a blind spot at the heart of the UK establishment’s view of entrepeneurship. They got really close but missed out on asking a critical question.

To see what I mean, here’s the PDF that goes along with the call for evidence. Here is the background to one of the sections:

4.27 The government would also like to gather views on whether the tax system could better support a strong reinvestment cycle among entrepreneurs and, if so, the best mechanism for doing this.
4.28 In the US, there are strong cases of successful entrepreneurs reinvesting the gains from their businesses into the earliest stage companies, injecting capital from successful growth back into the ecosystem and providing mentorship for new founders.
4.29 An example of this is the so-called ‘PayPal Mafia’ where founders and key employees of PayPal used gains from its success to create a series of new start-ups, many of which became successful companies.
4.30 The UK has a much weaker reinvestment cycle, with fewer examples of this kind of successful recycling of talent and capital, despite this being a key part of a healthy entrepreneurial and investment ecosystem.

This line jumped out at me: “injecting capital from successful¹growth back into the ecosystem and providing mentorship for new founders”. These are good things to want, but they are missing something big that they gloss over in the next sentences. The PayPal mafia that they describe did far more than injecting capital and providing mentorship. They also set up brand new companies themselves.

Now note this line “this kind of successful recycling of talent and capital”. Now we’re talking. They are so close to the answer, but here is the blind spot: recycling talent means people getting back into the arena to start a new company, like so many of the PayPal mafia did. It’s not about just providing cash and mentorship from arms length.

Let’s look at the actual questions now²:

Box 4.D In order to better understand how the tax system can support reinvestment, the government is seeking views on the following questions:

17. What are the main factors that influence whether entrepreneurs reinvest in other start-ups or scale-ups after a successful business exit, and to what extent is tax an appropriate lever for encouraging this?
18. Is tax an appropriate lever to incentivise reinvestment? If so, how can the UK tax system encourage stronger reinvestment activity, including through removing any existing barriers that might disincentivise this?
19. To what extent does Business Asset Disposal Relief (BADR) influence decision-making when considering the sale of a business, compared to other factors e.g. market conditions or personal circumstances?
20. Do you consider BADR to be well-targeted at supporting entrepreneurial activity, or are there ways that it could be changed, or a better alternative?

Spot the problem?

Check question 17: “What are the main factors that influence whether entrepreneurs REINVEST in other start-ups”.

It’s not asking “What are the main factors that influence whether entrepreneurs START other start-ups”.

Big difference. I’ve been shouting into the void for a long time about how the UK tax regime discourages entrepreneurs starting new startups in the UK. It’s because the UK tax regime actively encourages investors but takes entrepreneurs for granted.

Remember the line about the PayPal mafia? The PayPal mafia are the early employees at PayPal who took the money they made and the network they were a part of to build a whole ecosystem of other companies. Peter Thiel is one example. Famously, he held his PayPal shares in a self-directed Roth IRA. After PayPal he went on to co-found Palantir. Co-found, as opposed to just invest in. Whatever your personal views on Thiel or Palantir, he is an example of the PayPal mafia, and one that wouldn’t happen in the UK because we don’t have anything like the same incentives. Incentives matter, certainly for sophisticated startup builders. The UK would do well to stop discouraging sophisticated startup builders³.

There is a simple solution (perhaps not easy, but certainly simple). It is to level the playing field between investors and entrepreneurs. But, until the government starts asking the right question, it won’t get to hear the right answer.

Notes

1. “Successful” is a bit of a tell here too. It ignores the vast majority of founders who are not spectacularly successful. Failure is an inevitable and healthy part of entrepreneurship. A healthy entrepreneurial ecosystem is about recycling talent at all levels of success or failure. Not just about how to extract investment ££ out of the most financially successful. ︎
2. Curious that 2 of the 4 questions are specifically about BADR. I wonder why. For what it’s worth, my simple solution would involve getting rid of BADR and replacing it with something simpler and fairer. ︎
3. I’m trying to think of a UK analogy to Peter Thiel. The best I can come up with is Nik Storonsky of Revolut. Peter Thiel had plenty of incentives to build more businesses in the USA. The UK essentially pushed Storonsky out. ︎

It’s because of the sorts of jobs that it is disrupting.

It’s affecting programmers. When social media started killing journalism there was a dark joke going round: “learn to code”. Well, now the coders are among the ones being disrupted.

It’s affecting thought leaders, senior managers, consultants. They (we) can feel the AI disruption in our bones, in a way that we couldn’t when the disruption was about automatic a manufacturing production line.

It’s not surprising that people who are experiencing AI tools in their daily lives – whether using ChatGPT to plan a holiday or Claude to write some code or Copilot to do a business task – are extrapolating from these personal experiences.

We are only human. We understand things better when we experience them first hand than when we just read about them.

The people whose job it is to write about the future of AI are also those who are most directly affected by the evolution of AI.

An unfortunate side-effect of this is that there is still more hype than real analysis of what is going on.

I’ve no idea why YouTube says “foreign”. The vocals are in English, even if no-one can hear the words. But I agree with YouTube that there is definitely lots of [Music] going on.

Given the nerd I am, I wondered how an AI agent might interpret this. I’ll blame Moltbook’s viral rise for this train of thought. But bear with it a moment.

I’m a human being seeing YouTube’s machine-generated subtitles. I can interpret those however I like. I’m unlikely to act any differently as a result of whatever YouTube may or may not show there.

What about a future world where we all have AI agents acting on our behalf, perhaps making purchasing decisions. If you think “the algorithm” that the likes of YouTube and Netflix use to make recommendations is dubious, wait until Large Language Models start sitting in the middle of all our communications.

We are seeing the beginngs of this. People have agents that look up YouTube videos, get the transcripts and generate marketing content. But this is still content that is intended for a human being to interpret. What about when the content is being created for an AI to consume? Who wins in this scenario? If you are creating foreign [Music] then will you struggle to get your message across? And does it matter?

I’ll let Klangphonics have the last word.