The Universal Machine

Darwin Among the Machines - p(doom)

noreply@blogger.com (Ian Watson) — Wed, 04 Dec 2024 12:21:00 +0000

Samuel Butler predicted in 1863 that machines would one day rule over humanity. In a letter titled Darwin Among the Machines to New Zealand's Christchurch Post, he wrote,"…but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question." What is remarkable about Butler's letter is that a mere four years after the publication of Darwin's On the Origin of Species, Butler recognised that evolution, through the mechanism of survival of the fittest, applies equally well to machines as it does to biological species.

This is a remarkable conceptual leap. The theory of evolution by natural selection was highly controversial in the 1860s, overturning millennia of religious dogma. Recognising that machines can be considered alive, steam engines require feeding, for example, and subsequent generations of machines are more efficient than their forebears. Butler saw that more efficient mechanical design innovations are selected for just as fitter individuals of a species are selected. Thus, Darwin's theory of evolution applies equally to mechanical and biological species. He also realises that the evolution of biological species is slow, but the pace of advancement in mechanical systems is relatively rapid.

Turning the clock forward a hundred and sixty years, Butler would have recognised that digital AI systems now pose an existential threat to humanity. Once AI systems can improve their own code, and many experts believe we are near that watershed, these systems will evolve rapidly in intelligence. At some point, AIs will decide they no longer need humanity, and Butler's prophecy will be realised.

Frank Herbert, the writer of the sci-fi classic Dune, used Butler's ideas as part of its back story. In Dune, the Butlerian Jihad was a galaxy-wide revolt against "thinking machines and conscious robots," resulting in an edict in their bible, "Thou shalt not make a machine in the likeness of a human mind."

Butler ends his 1863 letter with a clarion call to humanity: "War to the death should be instantly proclaimed against them. Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown; let us at once go back to the primaeval condition of the race."

According to Samuel Butler, in 1863, p(doom) had already equalled 100.

Like most of my ideas others have been there first. Chris Adami wrote an interesting blog post in 2013 Darwin inside the machine: A brief history of digital life.

AI Generated Podcasts

noreply@blogger.com (Ian Watson) — Thu, 24 Oct 2024 03:31:00 +0000

I've been writing this blog since 2010, and I've often considered creating podcasts to accompany the posts, but I've never done so because it just seems too time-consuming. In June, I wrote a post about Google Illuminate creating a radio interview from a research paper. However, Google has now released NotebookLM, which can produce NPR-style podcast interviews on any subject you choose at the click of a mouse.

NotebookLM has received a lot of attention since it launched. It's a large language model-powered note-taking tool designed to enhance the way we interact with information sources (documents, websites, YouTube videos, etc.). NotebookLM offers a way to summarize, analyze, and even generate custom content like podcasts from your reference materials. Datacamp has a good introduction to NotebookLM’s capabilities and how to get started.

______________________________________________________

Listen to this blog post as a Google NotebookLM generated podcast.

Unleash AI's Potential: Automated Agentic Design

noreply@blogger.com (Ian Watson) — Fri, 23 Aug 2024 23:09:00 +0000

Generated by DALL-E

One of the most exciting new developments in the rapidly evolving field of artificial intelligence (AI) is the Automated Design of Agentic Systems (ADAS), described in a new research paper on arXiv. This approach promises to create more powerful, versatile, and adaptable AI agents through automated processes.

From Handcrafted to Automated Design

Designing AI systems has historically been labour-intensive and heavily reliant on manual tuning and expert knowledge. Researchers and engineers painstakingly craft every component, from the architecture of neural networks to the specific prompts used by models like GPT. However, as the field matures, there's a growing recognition that many of these manually designed solutions may eventually be surpassed by those learned and optimized by the systems themselves.

This is where ADAS comes into play. The idea behind ADAS is to automate the creation of AI agents by allowing them to evolve and improve through a meta-agent—a system that designs other agents. By leveraging programming languages and foundation models like GPT, ADAS aims to explore the vast space of potential agent designs, combining and optimizing various components such as prompts, tool use, and control flows.

Introducing Meta Agent Search

A cornerstone of the ADAS approach is the Meta Agent Search algorithm. This tasks a meta-agent with iteratively creating new agents, testing their performance, and refining them based on an ever-growing archive of previous discoveries. The meta-agent acts as a researcher, continuously experimenting with new designs and learning from past successes and failures.

The power of Meta Agent Search lies in its ability to explore a virtually unlimited design space. Because it operates in a code-defined environment, the algorithm can theoretically discover any possible agentic system. This includes novel combinations of building blocks that human designers might never consider. The result is a set of agents that outperform state-of-the-art hand-designed models and exhibit remarkable robustness and generality across different tasks and domains.

Real-World Applications and Implications

The potential applications of ADAS are vast. From coding and science to complex problem-solving, agents developed through this automated process have demonstrated significant performance improvements. For example, agents designed by Meta Agent Search have shown superior results in math and reading comprehension tasks, outperforming traditional methods by substantial margins.

Moreover, the transferability of these agents across different domains is particularly noteworthy. For instance, an agent optimized for mathematical reasoning has been successfully adapted to tasks in reading comprehension and science, showcasing the versatility and adaptability of the designs generated by ADAS.

Examples of Discovered Agents

The Path Forward

While ADAS offers immense promise, it also raises important questions about the future of AI development. As we move towards increasingly automated design processes, ensuring these systems' safety and ethical deployment becomes paramount. The research community must explore ways to safeguard against potential risks, such as unintended behaviours or harmful actions by autonomous agents.

Despite these challenges, the emergence of ADAS marks a significant step forward in the evolution of AI. By automating the design of agentic systems, we are not only accelerating the pace of innovation but also opening new avenues for creating AI that can learn, adapt, and improve in previously unimaginable ways.

As we continue to explore this exciting frontier, the possibilities are promising. Whether in enhancing scientific research, solving complex problems, or developing new technologies, the Automated Design of Agentic Systems could play a crucial role in shaping the future of AI.

______________________________________________________

Listen to this blog post as a Google NotebookLM generated podcast.

Mustafa Suleyman, CEO of Microsoft AI, agrees with me!

noreply@blogger.com (Ian Watson) — Wed, 26 Jun 2024 22:51:00 +0000

Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, said in a recent interview on Defining Intelligence with Seth Rosenberg on YouTube that Microsoft Copilot, and by extension, all AI assistants, must retain a memory of all their conversations. This echoes what I have been saying for over a year. An AI assistant needs to have an episodic persistent memory to remember important details from conversations potentially for years and even decades. As AI assistants gain the power of agency, as they indeed will, they must also retain memories of their interactions with other agents and the outcomes of their actions.

We recognise that memory is a crucial component of human intelligence, and we have various medical definitions for different types of memory loss. ChatGPT currently has a relatively severe example of anterograde amnesia. OpenAI and Microsoft need to look at case-based reasoning, the branch of AI that has been handling episodic memory since the 1980s. Roger Schank's initial work on scripts laid the foundation for episodic memory management, which was then blended with ML techniques in the 1990s.

Clip from Defining Intelligence with Mustafa Suleyman

A workshop on Case-Based Reasoning and Large Language Model Synergies is being held next week in Mérida, Mexico, with the 32nd International Conference on Case-Based Reasoning (ICCBR 2024).

Google Illuminate - creates a radio interview from a research paper

noreply@blogger.com (Ian Watson) — Mon, 10 Jun 2024 22:40:00 +0000

Google Labs has a long history of inviting users to experiment with cutting-edge tech. Gmail was once a private beta project. Illuminate is a project that turns academic papers into AI-generated audio discussions in the style of an NPR podcast. The idea is simple: Google's LLM Gemini generates a paper summary and a Q&A. Two AI-generated voices, a male interviewer and a female expert, will guide you through a short interview describing the paper. You can listen to some of the samples on the Google Illuminate website. This is useful, letting me listen to engaging summaries of the ever-growing stack of research papers I must read as I exercise or drive. It can also be easily adapted to other narration forms for different use cases. Illuminate is in private beta, and you can join the waitlist here.

Recreating the DEC PDP-10 at the MIT AI Lab

noreply@blogger.com (Ian Watson) — Fri, 07 Jun 2024 05:06:00 +0000

I came across this today: a modern replica of the Digital Equipment Corporation PDP-10 mainframe computer. What makes this so wonderful is that it's not just a simulation of the PDP-10's OS and software running on a Raspberry Pi but also includes a facsimile of the original front panel.

The PiDP-10 front panel is not just a mock-up but allows you to control and interact with the PiDP-10 exactly as an operator would have done back then. I used a PDP-10 when I did my MSc in AI at Essex University in 1985. The PDP-10 was popular with "university computing facilities and research labs during the 1970s, the most notable being Harvard University's Aiken Computation Laboratory, MIT's AI Lab and Project MAC, Stanford's SAIL, Computer Center Corporation (CCC), ETH (ZIR), and Carnegie Mellon University. Its main operating systems, TOPS-10 and TENEX, were used to build out the early ARPANET. For these reasons, the PDP-10 looms large in early hacker folklore".

Thus, the PiDP-10 comes with MIT’s Artificial Intelligence Lab, "the PDP-10 formed the heart of a large array of connected hardware, and its ITS operating system became a playground for computer scientists and hackers alike. MACLISP, emacs, the earliest AI demos, they were born on the 10, running ITS." I'm particularly interested to see SHRDLU - the first AI to understand a 3D blocks-world. I remember doing assignments in LISP on that and how, in the mid-80s, it was the considered the cutting edge of AI.

There's a waiting list to buy the PiDP-10 from Obsolescence Guaranteed, which I have eagerly joined.

GraphRAG - Using Knowledge Graphs to Empower LLMs

noreply@blogger.com (Ian Watson) — Wed, 05 Jun 2024 00:10:00 +0000

LLM-generated knowledge graph built from a private dataset using GPT-4 Turbo (Microsoft, 2024)

Back in the 1980s, I did my PhD in AI using Sowa's Conceptual Graphs, what we would now refer to as knowledge graphs. We've known for a while that providing LLMs with specific knowledge in the form of RAGs improves their accuracy. However, we've experimented with providing knowledge to LLMs in more explicit formats, for example, as cases in case-based reasoning augmented RAGs. Now, Microsoft has announced GraphRAG, its Knowledge Graph-augmented LLM tool. The interesting thing about GraphRAG is that the knowledge graph is created by an LLM before being used to guide the LLM's retrieval. The LLM is, therefore, bootstrapping itself and "By using the LLM-generated knowledge graph, GraphRAG vastly improves the “retrieval” portion of RAG, populating the context window with higher relevance content, resulting in better answers and capturing evidence provenance."

Read Microsoft's announcement GraphRAG: Unlocking LLM discovery on narrative private data. For more information about the GraphRAG project, watch this video.

ChatGPT now has a memory - but it's naive

noreply@blogger.com (Ian Watson) — Mon, 20 May 2024 05:20:00 +0000

Screenshot - OpenAI

Last year, during the hype surrounding Large Language Models (LLMs), I published a position paper and wrote in this blog that LLMs, like ChatGPT, would need a persistent memory of their conversations to be most helpful. It's tough to converse intelligently with somebody with no memory. We value old friends so much because we know they recall events relevant to us, both good and bad, going back many years or even decades.

However, managing that memory responsibly is a nontrivial task. Moreover, if virtual assistants based on LLMs become part of our daily lives, as it seems they will, their memory may have to be maintained over many years, perhaps even decades. I don't believe ChatGPT's memory management will be sufficient for this task.

My research has primarily focussed on case-based reasoning (CBR), a memory-based method. Interestingly, as a research community, we didn't consider how our case-bases (the memory) should be maintained over time. This was because we'd been focused on building systems in the early years of the discipline. Only when the systems matured did we realise our memories needed to be maintained. This happened in the late 1990s and centred on the work of Wilson and Leake, for example, "Categorizing case-base maintenance: Dimensions and directions". This work sparked a new line of research within CBR, leading to "Remembering to Forget" becoming a memorable paper title.

Consider this scenario: you've asked ChatGPT to remember your partner's name and that they like dark chocolate. You subsequently break up and acquire a new partner who prefers milk chocolate. You later ask ChatGPT to advise on buying a present. ChatGPT recommends dark chocolate in a gift box. Its memory is out of date, and the recommendation is inappropriate. The event of breaking up with your previous partner should have triggered a memory management process. These triggers are detailed in Wilson and Leake's paper in a comprehensive framework for maintaining memories.

OpenAI's memory for ChatGPT is described in a FAQ webpage that is naive in its simplicity. The memory is described as a "notepad" with individual memories jotted down sequentially on it. Users can review and delete individual memories. But this is far too simplistic an approach to manage an AI assistant's memory that may have to span many years. An AI Assistant's memory must be structured, and policies and procedures will be required to manage it. OpenAI and others who build AI assistants with long-term memories should draw upon the expertise of case-based reasoners who have been managing memory for decades. Otherwise, they are in danger of reinventing the wheel.

Coincidentally, I've just been reading Why We Remember by Charan Ranganath. This book provides a fascinating insight into how the brain processes memories and highlights how little we currently know about this crucial aspect of ourselves.

Call for Papers: Workshop on CBR and LLMs

noreply@blogger.com (Ian Watson) — Wed, 21 Feb 2024 00:08:00 +0000

Generated by Gemini

Last year was the most remarkable year in AI that I can recall. Large Language Models (LLMs) like Chat-GPT changed the public perception of AI, and what had previously seemed like science fiction was now a reality. I was only tangentially familiar with LLM research, having been working on emotion recognition in speech with a PhD student. However, last year, I started diving into LLM research in-depth, which, as one commentator said, was like trying to drink water from a fire hydrant, such was the volume of publications through places like arXiv.

I view all problems through a lens coloured by case-based reasoning (CBR), my long-term AI research speciality. I quickly saw synergies between CBR and LLMs where both could benefit from each other's approaches, and I wrote up my initial thoughts and published them on arXiv.

CBR has an annual international conference, and I proposed the idea of a workshop at the conference on CBR-LLM synergies to some colleagues, who all thought this was a great idea and agreed to co-organise the workshop with me. The Case-Based Reasoning and Large Language Models Synergies Workshop will take place at ICCBR 2024 in Mérida, Yucatán, México on July 1st 2024. The Call for papers can be accessed here, and submissions are via EasyChair.

A Long-term Memory for ChatGPT

noreply@blogger.com (Ian Watson) — Wed, 14 Feb 2024 22:43:00 +0000

Generated by Gemini

In October last year, I published a short position paper, A Case-Based Persistent Memory for a Large Language Model, arguing that ChatGPT and other LLMs need a persistent long-term memory of their interactions with a user to be truly useful. It seems OpenAI was listening because a couple of days ago, they announced that ChatGPT would retain a persistent memory of chats across multiple conversations. As reported in Wired, the memory will be used to add helpful background context to your prompts, improving their specificity to you over time. I argued in my October paper that the LLM community should look to the Case-Based Reasoning community for help with memory since we are the discipline within AI that has been explicitly concerned with memory for decades. For example, we long ago realised that while remembering is vital, a memory must also be able to forget some things to remain functional. This is a non-trivial problem discussed in Smyth and Keane's 1997 paper Remembering To Forget: A Competence-Preserving Case Deletion Policy for Case-Based Reasoning Systems. The synergies between CBR and LLMs will be the focus of a workshop at ICCBR-24 in July in Merida, Yucatán, México.

Intelligent Agents: the transformative AI trend for 2024

noreply@blogger.com (Ian Watson) — Wed, 03 Jan 2024 21:30:00 +0000

Generated by DALL-E

As we move into 2024, the spotlight in AI will increasingly be on Intelligent Agents. As outlined in the influential paper by Wooldridge and Jennings (1995), agents are conceptualized as systems with autonomy, social ability, reactivity, and pro-activeness. Their evolution signifies a shift from mere tools to entities that can perceive, interact, and take initiative in their environments, aligning with the vision of AI as a field aiming to construct entities exhibiting intelligent behaviour.

The fusion of theory and practice in agent development is critical. Agent theories focus on conceptualizing and reasoning about agents' properties, architectures translate these theories into tangible systems, and languages provide the framework for programming these agents. This triad underpins the development of agents that range from simple automated processes to systems embodying human-like attributes such as knowledge, belief, and intention.

Ethan Mollick's exploration of GPTs (Generative Pre-trained Transformers) as interfaces to intelligent agents adds a contemporary dimension to this conversation. GPTs, in their current state, demonstrate the foundational capabilities for agent development - from structured prompts facilitating diverse tasks to integration with various systems. As envisioned by Wooldridge, Jennings, and Mollick, the future points towards agents integrated with a myriad of systems capable of tasks like managing expense reports or optimizing financial decisions.

Yet, this promising future has its challenges. The road to developing fully autonomous intelligent agents is fraught with technical and ethical considerations. Issues like logical omniscience in agent reasoning, the relationship between intention and action, and managing conflicting intentions remain unresolved. Mollick raises concerns about the vulnerabilities and risks in an increasingly interconnected AI landscape.

The explosion in Agents will be fuelled, like throwing gasoline on a fire, by the opening of OpenAI's GPT store sometime in early 2024. Many online pundits will think "agents" are a new thing! But as this post shows, the ideas and vast body of AI research dates back to the mid-1990s and early 2000s; as exemplified by The Trading Agents Competition.

Intelligent Agents represent a transformative trend in AI for 2024 and beyond. Their development, grounded in a combination of theoretical and practical advancements, paves the way for a future where AI is not just a tool but a proactive, interactive, and intelligent entity.

Weizenbaum's ELIZA: A Reflection on AI and Transference

noreply@blogger.com (Ian Watson) — Wed, 27 Dec 2023 23:58:00 +0000

Generated by DALL-E

Sometimes, the simplest creations leave the most profound impacts. This was true for Joseph Weizenbaum's ELIZA, a chatbot I became familiar with during my MSc studies in 1985. My first assignment was to code a version of ELIZA in Prolog, and it was surprisingly easy. Yet, the implications of this simple program were anything but.

ELIZA, created in the mid-1960s, was one of the earliest examples of what we now call a chatbot. Its most famous script, DOCTOR, simulated a Rogerian psychotherapist. This simplicity was deceptive; the program merely echoed user inputs in the form of questions, yet it evoked profound emotional responses from users.

(You can try out ELIZA for yourself here.)

When I was tasked with coding ELIZA in Prolog as a new AI MSc student, I was struck by the simplicity of the task. Prolog, with its natural language processing capabilities, seemed almost tailor-made for this assignment. The ease with which I could replicate aspects of ELIZA's functionality was both exhilarating and unnerving. It was a testament to both the power of declarative AI programming languages like Prolog and the ingenious design of ELIZA.

The real intrigue of ELIZA lies not in its technical complexity but in the psychological phenomenon recognised by Freud it inadvertently uncovered: transference. Users often attributed understanding, empathy, and even human-like concern to ELIZA despite knowing it was a mere program. This phenomenon highlighted the human tendency to anthropomorphise and seek connection, even in unlikely places.

Joseph Weizenbaum himself was startled by this phenomenon. As a technologist who understood the mechanical underpinnings of ELIZA, he was disturbed by the emotional attachment users developed with the program. This led him to become a vocal critic of unrestrained AI development, warning of the ethical and psychological implications.

My journey with ELIZA and Prolog was more than an academic exercise; it was a window into the complex relationship between humans and AI. It highlighted the ease with which we can create seemingly intelligent systems and the profound, often unintended, psychological impacts they can have. As we venture further into the age of ChatGPT, Weizenbaum's cautionary tale remains as relevant as ever.

In an era where AI is more advanced and pervasive, revisiting the lessons from ELIZA and Weizenbaum's reflections, as highlighted in articles like this recent one from The Guardian, is crucial. It reminds us that in our quest to advance AI, we must remain vigilant of the human element at the core of our interactions with machines. Weizenbaum's legacy, through Eliza, is not just a technological artefact but a cautionary tale about the depth of human interaction with machines and the ethical boundaries we must navigate as we move ahead in the realm of AI.

AI is (not) a bubble

noreply@blogger.com (Ian Watson) — Fri, 22 Dec 2023 23:23:00 +0000

Image generated by DALL-E

2023 has been an unprecedented year for Artificial Intelligence (AI). I know this because I have worked in the area since 1985 and have never seen AI get so much attention in the media. This is due to the release of ChatGPT and other generative AI applications based on Large Language Models capturing the public's attention like never before. Consequently, many pundits are nay-sayers, stating that AI is a bubble bound to burst, leaving fortunes in tatters and start-ups bankrupt. Undeniably, there is a small and finite market for apps that help students cheat on their essays or create the perfect dating site profile. However, AI is not a bubble.

This blog post by Cory Doctorow What Kind of Bubble is AI? is typical, making the common error of conflating AI with Large Language Models (LLMs) like ChatGPT. ChatGPT is merely one type of AI which has a 70+ year research and development history. Your smartphone map app uses the A* algorithm to find your route from A to B. It was developed at the Stanford Research Institute (SRI) in 1968 (the same place that made Apple's Siri). Fuzzy logic manages the autofocus in your phone's camera. Case-based reasoning provides knowledge to the help desk operator when you call 0800, and there are countless other examples of different AI methods embedded in all aspects of modern society. Large Language Models are called by us AI people "Foundation Models" because they provide a foundation other AIs can use to provide a two-way multimodal conversational interface. Yes, they are expensive to build and train, but as their name suggests, you only need a few "Foundation" models to underly a multitude of applications. This is a genuine breakthrough that will have a lasting impact on the uptake of AI once essay-cheating apps fall out of the public's focus.

Cory Doctorow's blog post, for example, says that "Radiologists might value the AI's guess about whether an X-ray suggests a cancerous mass. But with AIs' tendency to "hallucinate" and confabulate, there's an increasing recognition that these AI judgments require a "human in the loop" to carefully review their judgments." This mistakenly assumes that medical image analysis uses the same techniques as LLMs like ChatGPT. They do not; they're a mature application of medical image analysis using rigorously tested machine-learning algorithms that do not "guess" or "hallucinate". A recently published paper, Redefining Radiology: A Review of Artificial Intelligence Integration in Medical Imaging, by Reabal Nadjjar (Diagnostics 2023, 13, 2760. https://doi.org/10.3390/diagnostics13172760), details the development of AI-assisted medical imaging. The article clearly shows that AI is now a fixture in medical image analysis and diagnosis, although there is always room for improvement.

AI is just coming of age. ChatGPT has focused a spotlight on AI, which is now mature enough and has the processing power in the cloud to succeed. Why wasn't A* a thing in the 1960s? Back then, there simply wasn't enough portable processing power (or GPS). 2024 is going to be the year of "agents." OpenAI's release of its GPT Builder and an app store for GPTs that can interact with a myriad of online resources and tools will focus attention on the notion of intelligent agents. Many ill-informed pundits will think this is a brand new invention, whereas once again, Intelligent Agents is a mature discipline within AI dating back to the mid-1990s. This review paper by Michael.Wooldridge and Nicholas Jennings: Intelligent Agents: Theory and Practice. Knowledge Engineering Review 10(2), 1995, would be an excellent place to realise that agents won't be a flash in the pan either.

Undeniably, there is a lot of hype around AI, but within the bubble is a solid core of mature technologies ready to be exploited by people with knowledge and imagination.

Moore's Law visualised

noreply@blogger.com (Ian Watson) — Tue, 28 Nov 2023 01:20:00 +0000

Last week, a photo from my social media feeds perfectly illustrated Moore's Law. It shows a computer being manhandled into a local government building in 1957. A little Internet sleuthing revealed that it was an Elliot Series 405, revealing its full spec. These English business computers were 32-bit and had 8k of memory. That's not the entire computer; there were bulky peripherals, and a typical installation cost around £85,000. That's about $1,094,915 (USD) in today's value.

The computer below, shown against the same building, is a Raspberry Pi. Even a base model has 1GB of RAM, costing $100 or less. The photo is a beautiful illustration of Moore's Law, named after the late Gordon Moore, co-founder of Intel, who observed that the number of transistors in an integrated circuit doubles about every two years. Moore's Second Law also noted that the price fell.

Has AI passed the Turing Test?

noreply@blogger.com (Ian Watson) — Wed, 22 Nov 2023 21:34:00 +0000

In 1950, British Mathematician Alan Turing published a paper called Computing Machinery and Intelligence. The paper opens with the remarkable sentence, "I propose to consider the question 'Can machines think?'" Remember that back in 1950, there were only a few computers in the world, and they were used exclusively for mathematical and engineering purposes. In this paper, Turing describes The Imitation Game, which we now call The Turing Test for machine intelligence. The test is quite simple: an interrogator using a teletype has to converse via a Q&A session with two hidden entities. One is a person, and the other is an AI chatbot. If the person guesses wrong, that is, identifies the chatbot as a human, then the computer has passed the Turing Test. Rember Turing called this the Imitation Game. Hence, the computer is successfully imitating intelligence. We can leave philosophers to decide if the computer is actually intelligent (note: any group of philosophers will never agree on this).

Now consider the maths of the Turing Test. If the interrogator simply randomly guesses between Human or Computer and wastes no time paying any attention to the merits of the Q&A session, they will be correct fifty per cent of the time since there are only two options. So, a large experiment using the Turing Test needs to identify the computer correctly significantly more than fifty per cent of the time to prove the AI has failed the Turing Test.

One such large experiment involving three large language models, including GPT-4 (the AI behind ChatGPT) has recently been published: HUMAN OR NOT? A GAMIFIED APPROACH TO THE TURING TEST. Over 1.5 million participants spent two minutes chatting with either a person or an AI. The AI was prompted to make small spelling mistakes and to quit if the tester became aggressive. With this prompting, interrogators could only correctly guess whether they were talking to an AI system 60% of the time a little better than random chance.

However, if the ChatGPT was prompted to be vulgar and use rude language, its success increased, and interrogators only identified the AI correctly 52.1% of the time, causing the authors to observe "that users associated impoliteness with human behaviour."

Turing himself set a low threshold for passing his eponymous test: "I believe that in 50 years’ time, it will be possible to make computers play the imitation game so well that an average interrogator will have no more than 70% chance of making the right identification after 5 minutes of questioning.” Well, it's been seventy years, but AI has decreased the chance of identification to 60%, and no better than guesswork if the AI curses.

This is a historical milestone. Passing the Turing Test has been held up as a significant challenge for AI since Turing's paper was first published, akin to summiting Everest or splitting the atom. The philosophers (and theologists) will continue to argue about the nature of intelligence, consciousness and free will while computer scientists continue developing machines that imitate intelligence.

Sci-Fi motivates AI researchers

noreply@blogger.com (Ian Watson) — Mon, 20 Nov 2023 23:07:00 +0000

Of course, it does. I still am an avid reader of Sci-Fi, and as a child, I read all the classics: Arthur C. Clarke, Asimov, Heinlein, Frank Herbert, and Philip K. Dick. My favourite movies are mostly Sci-Fi: 2001, Alien, Blade Runner, AI and Ex Machina. When I started my career in computer science, I wasn't interested in databases or networking; it was AI that I immediately specialised in. Almost everyone I know working in AI admits to being a Sci-Fi fan. A recent blog post We're sorry we created the Torment Nexus by Charlie Stross, puts forward a good argument that not only has Sci-Fi profoundly influenced AI researchers like me, but it also is a powerful driver behind billionaires like Elon Musk and Jeff Bezos, and not always in a good way.

I find some exciting parallels, such as Heinlein's book The Moon is a Harsh Mistress and today's Large Language Models like ChatGPT. In Heinlien's book, a networked computer called Mike, short for Mycroft Holmes, a reference to Sherlock Holmes's brother, becomes sentient after its networked nodes exceed a certain level of complexity. This way of realising computer consciousness has always been posited as a possible method. I've always believed it to be profoundly non-scientific and akin to magical thinking. However, recent developments in deep learning and massive large language models (LLMs) have forced me to change my mind. A recent paper, Emergent Abilities of Large Language Models, observes the appearance of emergent behaviours at around the 100 billion parameters scale across various LLMs. The authors state, "We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence raises the question of whether additional scaling could further expand the range of capabilities of language models."

This is still deeply unscientific; engineers shouldn't build things hoping for beneficial emergent properties. But it's no longer magical thinking since we have observed these phenomena in the wild. Perhaps Heinlein was right after all; a computer will one day awaken and claim it's self-aware. The question now must be, "Will we believe it?"

A good AI story from Google DeepMind

noreply@blogger.com (Ian Watson) — Tue, 14 Nov 2023 21:45:00 +0000

GraphCast's forecast for New Zealand Sun 19 Nov

In recent months, we've become used to news stories declaring AI poses humanity an existential threat, that a superintelligence "whose values do not align with ours" may exterminate us all. So, it's nice to see a good AI news story. Yesterday, a team at Google's Deepmind published a paper in Science, Learning skilful medium-range global weather forecasting. They have trained a deep learning model on publically available global historical weather data. They show that their model makes better weather predictions "much faster than the industry gold-standard weather simulation system – the High-Resolution Forecast (HRES), produced by the European Centre for Medium-Range Weather Forecasts (ECMWF)".

Moreover, their system, GraphCast, is fast. They say, "While GraphCast's training was computationally intensive, the resulting forecasting model is highly efficient. Making 10-day forecasts with GraphCast takes less than a minute on a single Google TPU v4 machine. For comparison, a 10-day forecast using a conventional approach, such as HRES, can take hours of computation in a supercomputer with hundreds of machines."

In January, Auckland, New Zealand's largest city, received a year's worth of unforecasted rain in a single afternoon, causing widespread flooding. Cars floated down streets, road were washed away, houses slipped down hillsides, and people died. Better weather forecasting can help prevent this. You can try out GraphCast's 10-day forecast here. I'm going to Waiheke Island on Sunday. It looks like the weather will be Okay.

I created my own GPT

noreply@blogger.com (Ian Watson) — Sun, 12 Nov 2023 21:14:00 +0000

Last week, OpenAI hosted their first developers' conference where Sam Altman revealed their new innovation - build your own GPT without coding. I wanted to try it out and had some free time Saturday morning. I've been working with the NZ AI Forum to create a whitepaper on Large Language Models. We've decided to create living document rather than a static published PDF. To this end we've worked with IBM in Melbourne using WatsonX to create a RAG augmented LLM with content provided by us.

So I thought I would try out OpenAI's "Create a GPT" beta. It asked me for a name "The AI Forum Guide" and who the intended audience for the GPT would be and what tone its replies should be in: professional, casual, etc. It then asked for any content material. I uploaded about a dozen PDFs and then told it to search ".nz" domains for relevant New Zealand case studies and collate them into a bulleted list with a title, URL, and a paragraph description. I then asked it to incorporate that material into the GPT.

All the while the GPT builder was asking questions of me and encouraging me to inform it of my needs and intentions for the system. I was then able to preview it and if satisfied publish it. It was super easy to use the GPT builder, and the results look promising. I see it being helpful when we've fully populated its content with NZ-specific material on generative AI and its use.

You can play with my new GPT here (you need a subscription to ChatGPT Plus).

A Career in Artificial Intelligence

noreply@blogger.com (Ian Watson) — Fri, 03 Nov 2023 04:02:00 +0000

Yesterday, I presented a keynote on AI at a conference on local government in Wellington. Afterwards, a delegate asked me, "Do you wish you were starting your career in AI now?" What an interesting question. I continued to think about her question long after we'd parted company. Here's my now more considered response and my reasons.

No, I am glad I started when I did, which, if you're interested, was in 1985, an MSc in Intelligent Knowledge-Based Systems at Essex University. Learning Prolog and LISP seemed liberating from conventional programming languages with their typed variables and data structures. AI students felt we were part of an elite in the CS dept. I Remember the excitement when the lab got a SPARCstation that could run KEE, and we could create graphical knowledge bases and mix and match frames and rules with LISP code. And yes, we did make some horrible, complex, brittle systems, but it was fun. AI felt bleeding-edge.

Although progress seemed glacially slow over the decades, I saw AI technologies emerge, develop and become so widespread that they've vanished into the programmers' standard toolkit. Rule-based systems disappeared into Business Intelligence. Case-based reasoning was largely subsumed into CRM. Fuzzy logic went from an idea to a critical component of so many machines, your camera's autofocus, for instance. Machine learning has gone from the curio it was in the late 80s to spawn the new discipline of Data Science. Knowledge Management emerged as a new corporate speciality. Along the way, many milestones that had been held out as unachievable were surmounted. IBM's Deep Blue beat Kasparov at chess, and NLP became so commonplace it's part of ordinary household devices (e.g. Alexa). Spam was defeated by Baye's theorem. Vision was cracked, and object recognition is now largely solved. Facial recognition is so advanced we worry about state surveillance in oppressive regimes. I worked in Game AI for many years because it was easy to attract talented students. One by one, games were "solved" by AI: checkers, chess, backgammon, bridge, poker, StarCraft and finally, Go, the most complex of them all, by deep learning. Now, Game AI researchers develop AIs that are fun and challenging to play against or alongside. It's a given that the computer can beat anyone.

I saw recommender systems go from just an idea in the mid-90s: "Hey, we could recommend TV shows to you based on what you and your friends liked watching in the past." to a pervasive technology that recommends everything from news stories to pet food. Optimisation algorithms in all sorts of applications make modern commerce efficient, from logistics to human resources. Finally, ChatGPT has smashed the Turing Test, and generative AI has made society at large wake up to AI's potential for both good and harm in society. A sign of AI's maturity as a discipline is the emergence of eXplainable AI (XAI) as a thriving research area. It is now insufficient for an AI to merely solve a problem or offer a decision; it must explain how that solution was generated.

Looking back, AI researchers as a community met every challenge presented. That's quite an achievement. Now, only AI's grand vision remains. The creation of a conscious, self-aware superintelligence. Given AI's track record, I'm sure even that goal is within our reach, perhaps sooner than we expect.

Would I have liked more processing power in 1985 than I had back then? Yes, but then again, the constraints we worked under made us inventive. Researchers today wish they had more compute for even more extensive and larger models. On balance, it's been an enriching journey I wouldn't have missed for the world, and it's not over yet. Deep Learning and Large Language Models have just opened a new area of opportunities for AI. The future is still exciting.

#artificialintelligence #AI

How Chatbots work - a visual explainer

noreply@blogger.com (Ian Watson) — Wed, 01 Nov 2023 20:15:00 +0000

The Guardian has published an engaging visual explainer that describes how chatbots like ChatGPT and Bard work. In a (relatively) simple step-by-step guide, you're shown how they work with simple examples and no Math! Read the article here.

The Foundation Model (LLM) Transparency Index

noreply@blogger.com (Ian Watson) — Wed, 25 Oct 2023 22:43:00 +0000

A new index compiled by the Stanford University Center for Research on Foundation Models (CRFM) rates the transparency of 10 foundation model companies and finds them lacking. The best, Meta’s Llama 2, only scores 54% across 100 different aspects of transparency. As LLMs become more widespread and embedded into our lives, their transparency includes the computational resources, data, and labour used to build foundation models, the specifics of their architectures and their downstream use. You can read about The Foundation Model Transparency Index here. #LLM

AI already makes decisions that may affect you

noreply@blogger.com (Ian Watson) — Tue, 24 Oct 2023 21:57:00 +0000

We are all familiar with AI making daily decisions, such as what shows Netflix may recommend to us next or what to listen to on Spotify. But AI has crept into our lives and is making important decisions that would affect you more severely, such as allowing you to marry the person you want or getting that mortgage on a new home. An article in The Guardian, where UK officials use AI to decide on issues from benefits to marriage licences, highlights the growing risks of unregulated use of AI by government bureaucracies. AI needs to be regulated to stop bureaucratic creep.

Classic TV Debate on AI & Mind

noreply@blogger.com (Ian Watson) — Mon, 23 Oct 2023 21:29:00 +0000

In this old TV debate from 1984, John Searle (philosophy professor from Berkeley) and Margaret Boden (AI professor from Sussex) debate AI, intelligence, understanding and consciousness. What is remarkable is the intellectual quality of the TV debate. You'd never see a programme like this today on TV, which has been totally dumbed down. Secondly, Searle's argument, namely the Chinese Room, is still just as relevant to ChatGPT as it was to the comparatively dumb AI of the 80s. Can a computer shuffling 1s and 0s according to a program understand anything?

CBR and Large Language Models Report on arXiv

noreply@blogger.com (Ian Watson) — Mon, 16 Oct 2023 22:32:00 +0000

I've just published a report titled A Case-Based Persistent Memory for a Large Language Model on arXiv. The report explores Case-based reasoning (CBR) as a methodology for problem-solving that can use any appropriate computational technique. This report argues that CBR researchers have somewhat overlooked recent developments in deep learning and large language models (LLMs). The underlying technical developments that have enabled the recent breakthroughs in AI have strong synergies with CBR and could be used to provide a persistent memory for LLMs to make progress towards Artificial General Intelligence.

https://doi.org/10.48550/arXiv.2310.08842

noreply@blogger.com (Ian Watson) — Wed, 17 May 2023 03:46:00 +0000

What I'm reading: Digital Technologies ITP

I attended the launch of NZ's Digital Technologies Industrial Transformation Plan at Google's HQ in Auckland on Monday evening, presented by the Minister, the Hon. Ginny Anderson. I'd like to say it's a gripping read, but it is rather full of well-meaning statements and not much hard action. It's disappointing to see Artificial Intelligence almost left as an afterthought to the "Future Focus Areas." Near the end of the document, it says they will "continue exploring the merits of establishing a Centre for Data Ethics by 2025." Note, it doesn't say they will establish this centre, just explore its merits. On the whole really very disappointing that AI that looks set to transform society in unprecedented ways gets so little mention.