Mark Watson's artificial intelligence and Lisp hacking blog

Supporting Goals of Free Software Foundation while using Apple hardware

2025-11-14T15:38:00.003-07:00

I just rejoined the FSF (https://www.fsf.org/) recently after almost ten years of not being a member. For decades I relied on Linux for work and writing and being a member of the FSF made sense. However when I decided to simplify my life and use a MacBook, an iPhone, and an iPad I let my FSF membership drop. I am enjoying being a member again.

I am not recommending that anyone adopt my setup, but it may still be of general interest:

Apps vs. Web Browser

On mobile devices (and my MacBook) I don’t like to install apps unless it is something like a Chess game app. Services like Email, Mastodon, Facebook, YouTube, and X all work fine in a web browser, even with the DuckDuckGo browser which is usually my default.

Email

I use my own custom domain on Apple iCloud email. Convenient enough. I use backup ProtonMail and Gmail accounts.

I use the DuckDuckGo web browser, even on my iPhone, for these three services.

AI vs. No AI

Well, I have been a paid AI practitioner since 1982 but I believe that modern (LLM based) AI should be used purposely, keeping in mind “productivity vs. playing around issues.” The resource costs of overuse of AI might cause an economic crash (a favorite little conspiracy theory of mine).

I like to have AI available only when I specifically want it. Usually I just code in plain-old Emacs for coding and writing my books. If I specifically want AI help with something then for an IDE experience I will use the TRAE coding agent. For command line, I will use gemini-cli or codex. I like to use AI coding help 4 or 5 times a week. As an example, today I wanted some Python code that used a few libraries converted to Common Lisp (using several popular CL libraries). TRAE one-shotted this for me in two minutes. I think it would have taken me over 20 minutes to write it myself.

AI is OK for easy stuff you can do yourself, and save time. I prefer to use it less, rather than more often but I love writing open source software and Open Content licensed books (https://leanpub.com/u/markwatson) - no strong desire to over-automate my work. But I appreciate AI tools when I do use them.

AI / Web Browser Integration?

In general I say NO! After reading their privacy docs I did experiment a few hours each with the Perplexity Comet Browser and OpenAI’s ChatGPT Atlas and I have read the docs for Opera Neon and Brave with Leo. (I used Proton’s Lumo AI for two months which is not really a browser integration and decided I didn’t want to pay for it after two months.)

I am a human being, I like to think for myself, and just use AI sparingly.

One option I am investigating is the “AI tab” on the DuckDuckGo browser that seems very useful and is privacy preserving. I am evaluating DDG’s subscription paid plan month by month, haven’t subscribed for a year yet.

Software Development

I find macOS almost as good as Linux for coding. I do keep one or two small Linux VPSs running on the web when Linux is better than macOS.

Notes on effectively using AI

2025-08-30T09:39:00.000-07:00

I cover the use of LLMs in several of the books I have written. This blog post is meant to be common general purpose advice for effectively using AI using four techniques:

Use one-shot or few-shot prompting: show the AI what you want with examples.
Chain of thought reasoning: LLMs are auto-regressive models so the tokens generated "become part of the prompt," that is, without asking for chain of thought reasoning if the model makes a quick guess at an answer, then that answer drives the remaining token generation process.
Assigning a role to the AI assistant: for example "You are a historian who uses web search to verify your analysis of history and current events."
Reverse prompting: explicitly tell the AI to ask you questions (important because modern system prompts often coach a model to not do this).

One-shot or few-shot prompting

As an example, you might want to process input text and save people's email addresses and names in some structured output format, like JSON. If you don't start a prompt with context for the format you want, results will vary. Here is an example prompt, where text to be processed would be placed below Text:.

#<<EOFYou are an information extraction system.
Extract all people’s **full names** and **email addresses** from the following text.
If no names or emails are present, return an empty list.
Return the result strictly in this JSON format:
{
  "contacts": [
    {
      "name": "<full name as written in text>",
      "email": "<email address>"
    }
  ]
}
Text:
Hi, I’m Alice Johnson, please email me at alice.j@example.com.  
Also, you can reach Bob Smith via bob.smith42@gmail.com.
EOF

Chain of thought reasoning

Treat the AI as you world a person: Start a prompt with something like this:

Before answering, give a concise reasoning summary: list the key assumptions and the 3–7 high-level steps you’ll take (no inner monologue). Then provide the final answer, followed by a quick self-check (one possible pitfall or edge case). Use Markdown, keep it succinct, and prefer bullet points.
Question: ‹your question›

Assigning a role to the AI assistant:

All-purpose role prompt

You are a ‹role›. Always adopt this perspective when reasoning and answering. If relevant, use external verification (e.g. web search, references, or sources).
Before answering: give a reasoning summary (assumptions + 3–7 steps).
Then: provide the final answer, followed by a self-check (biases, gaps, or counterexamples).
Question: ‹your question›

Ultra-compact variant

Act as a ‹role›.
Reasoning summary → answer → self-check.
Q: ‹your question›

Example (historian role)

You are a historian who cross-checks analysis using web sources.
Before answering: assumptions + reasoning summary.
Then: provide the historical analysis, with citations when possible.
Self-check: note one uncertainty, bias, or missing source.
Q: How did the Black Death reshape European economies?

Coding / technical role

You are a senior software engineer specializing in Python and AI.
Step 1: reasoning summary.
Step 2: final code, with inline explanation.
Step 3: minimal test + note one edge case.
Task: Build a script that reads a CSV and outputs JSON.

Reverse prompting

All-purpose reverse-prompt

Before answering, first ask me clarifying questions if anything is underspecified, ambiguous, or could change the answer. Keep it concise (max 3 questions). If the question is already clear, briefly state: “No clarifications needed.”
Then: give a reasoning summary (assumptions + 3–7 steps).
Finally: provide the answer with a self-check (one pitfall/edge case).
Question: ‹your question›

Ultra-compact variant

Always start by asking me clarifying questions (max 3). If none, say so.
Then: reasoning summary → answer → 1-line self-check.
Q: ‹your question›

Coding variant

Step 1: Ask me clarifying questions about requirements (input format, edge cases, env).
Step 2: Reasoning summary.
Step 3: Provide final code, plus a usage example and one edge case.
Task: ‹your task›

Tool calling with Moonshot's Kimi K2 mode

2025-07-16T09:57:00.002-07:00

Moonshot's new Kimi 2 model is extremely inexpensive to use. Moonshot AI (月之暗面) is a prominent Chinese artificial intelligence startup focused on developing large language models with the long-term goal of achieving Artificial General Intelligence (AGI). The company gained significant recognition for its Kimi chatbot, which pioneered an exceptionally large context window capable of processing up to two million Chinese characters in a single prompt. Backed by major investors like Alibaba, Moonshot AI has quickly become one of China's most valuable AI unicorns and a key competitor in the global AI race.

November 14, 2025 update: the code below is Python. I have more up to date Common Lisp examples using Kimi K2 here: https://leanpub.com/lovinglisp/read#leanpub-auto-moonshots-kimi-k2-model

The release of Kimi 2 last week has been referred to as "another DeepSeek moment."

You need a Moonshot API access key from https://platform.moonshot.ai/console/account

The direct API pricing from Moonshot AI is approximately:

Input Tokens: ~$0.60 per 1 million tokens

Output Tokens: ~$2.50 per 1 million tokens

Several inference providers in the USA are now supporting Kimi 2 also. Kim 2 is a mixture of expert model with 32 billion parameters active at any time during inference. The total model size is a trillion parameters.

Tool use using the OpenAI Python library is fairly straightforward:

from typing import *
 
import os
import json
 
from openai import OpenAI
from openai.types.chat.chat_completion import Choice
 
client = OpenAI(
    base_url="https://api.moonshot.ai/v1",
    api_key=os.environ.get("MOONSHOT_API_KEY"),
)

# The specific implementation of the search tool
def search_impl(arguments: Dict[str, Any]) -> Any:
    """
    When using the search tool provided by Moonshot AI, you just need to return the arguments as they are,
    without any additional processing logic.
    """
    return arguments

# START: Added Tool Implementations
# ---------------------------------
def list_files_impl() -> str:
    """Lists files in the current directory, one per line."""
    print("--- Calling list_files_impl ---")
    try:
        files = os.listdir('.')
        if not files:
            return "The current directory is empty."
        return "\n".join(files)
    except Exception as e:
        return f"Error listing files: {e}"

def get_file_contents_impl(filename: str) -> str:
    """Reads and returns the contents of a file."""
    print(f"--- Calling get_file_contents_impl for file: {filename} ---")
    try:
        with open(filename, 'r', encoding='utf-8') as f:
            return f.read()
    except FileNotFoundError:
        return f"Error: File '{filename}' not found."
    except Exception as e:
        return f"Error reading file '{filename}': {e}"
# -------------------------------
# END: Added Tool Implementations


def chat(messages) -> Choice:
    completion = client.chat.completions.create(
        model="moonshot-v1-8k", # Using a standard model for tool use demonstration
        messages=messages,
        temperature=0.3,
        tools=[
            # Tool 1: Moonshot's built-in web search
            {
                "type": "builtin_function",
                "function": {
                    "name": "$web_search",
                },
            },
            # START: Added Tool Schemas
            # -------------------------
            # Tool 2: List files in current directory
            {
                "type": "function",
                "function": {
                    "name": "list_files",
                    "description": "Lists all files in the current working directory, one file per line.",
                    "parameters": {
                        "type": "object",
                        "properties": {},
                    },
                },
            },
            # Tool 3: Get contents of a file
            {
                "type": "function",
                "function": {
                    "name": "get_file_contents",
                    "description": "Returns the full contents of a specified file in the current directory.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "filename": {
                                "type": "string",
                                "description": "The name of the file whose contents are to be retrieved."
                            }
                        },
                        "required": ["filename"],
                    },
                },
            }
        ]
    )
    return completion.choices[0]
 
 
def main():
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant who can browse the web and interact with the local file system. Return all answers in English."},
    ]
 
    # Updated question to demonstrate file system tools
    messages.append({
        "role": "user",
        "content": "Please list the files in the current directory, and then show me the contents of the python script you find."
        ##"content": "Please search for Mark Watson AI consultant and author, and tell me what his hobbies are."
      })
 
    finish_reason = None
    while finish_reason is None or finish_reason == "tool_calls":
        choice = chat(messages)
        finish_reason = choice.finish_reason
        if finish_reason == "tool_calls":
            messages.append(choice.message)
            for tool_call in choice.message.tool_calls:
                tool_call_name = tool_call.function.name
                # Note: Arguments can be an empty string if no parameters are needed.
                tool_call_arguments_str = tool_call.function.arguments
                tool_call_arguments = {}
                if tool_call_arguments_str and tool_call_arguments_str.strip():
                    tool_call_arguments = json.loads(tool_call_arguments_str)

                print(f"Model wants to call tool '{tool_call_name}' with arguments: {tool_call_arguments}")
                
                # START: Updated Tool Handling Logic
                # ----------------------------------
                if tool_call_name == "$web_search":
                    tool_result = search_impl(tool_call_arguments)
                elif tool_call_name == "list_files":
                    tool_result = list_files_impl()
                elif tool_call_name == "get_file_contents":
                    filename = tool_call_arguments.get('filename')
                    if filename:
                        tool_result = get_file_contents_impl(filename=filename)
                    else:
                        tool_result = "Error: 'filename' parameter is required."
                else:
                    tool_result = f"Error: unable to find tool by name '{tool_call_name}'"
                # --------------------------------
                # END: Updated Tool Handling Logic
 
                messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "name": tool_call_name,
                    "content": json.dumps({"result": tool_result}) # Encapsulate result in a JSON object for consistency
                })
 
    print("\n--- Final Answer ---\n")
    print(choice.message.content)
 
 
if __name__ == '__main__':
    main()

Google Gemini Batch Mode API with a 50% cost reduction: a game changer?

2025-07-07T11:36:00.000-07:00

I noticed on X this morning that Google dropped a new batch API with a 50% price cut. I use gemini-2.5-flash for speed and low cost and being able to batch large numbers of requests in a JSONL file (JSON where each line is a single legal JSON expression) seems like a big deal to me.

Gemini Batch API Docs

I have been a little negative on Hacker News and X recently about the energy costs vs. value from LLM use and it seems like Google is striking a good middle ground in cost and environmental impact.

Automating NLP and other workflows seems fairly simple: automate writing pipeline requests to a JSONL file, automate sending requests, periodically polling for the results being complete, auto-download and use results in your workflows.

So much fun: recreating 1970s text adventure games using LLMs, but better

2025-07-05T10:48:00.002-07:00

In the late 1970s, I worked long hours on a text based adventure game called land of the dwarf for the Apple II computer. My game was written in Apple Basic and I gave it away for free. I wrote it using a huge sheet of paper, drawing a transition network diagram with bubbles, locations and action codes for things that could be done and arcs between the bubbles being the ability to move from one location to another. Yesterday I was really in the mood to do something fun. We didn’t have anything going on for Fourth of July celebrations until an outdoor symphony in the early afternoon, so I sat down in the morning with Google‘s excellent Gemini CLI coding agent and I described what I wanted, which was to be able to input a story context as a text input file and then use an LLM in a conversation loop and continuously provide you with the text based adventure program experience. I was really surprised how well it turned out. Fun!

The generated adventure game code uses a local model running on Ollama - that is the way 😃

AI dominance: US vs. China and the rest or the world

2025-02-06T13:22:00.000-07:00

I would like the USA to not lose the ‘AI race’ but I don’t want other countries to lose either.

Ensuring continued US leadership in AI without constraining global progress necessitates a dual-pronged strategy that couples strategic national R&D investments with adherence to open international standards. On the technical front, integrating edge computing, specialized AI silicon, and advanced hardware-software co-design with standardized protocols for interoperability enables US institutions to maintain a competitive edge. Simultaneously, promoting collaborative frameworks for data sharing, federated learning, and algorithmic auditing mitigates risks of siloed innovation, ensuring that breakthroughs in efficiency, low-latency inference, and energy-optimized training regimes benefit a global network of research and industry stakeholders.

In the USA I would like to see major resources dedicated to both open models such as the ones from Meta, as well as the use of French Mistral open models (Mistral Small is awesome and easily runs on my Mac) and Chinese models such as Qwen (which I frequently use) and Deepseek R1 (that I play with.)

I also hope for vibrant closed model competition from Google, OpenAI, Anthropic, etc. I find Gemini 2 models and o3-mini-high to set new standards for general utility.

AI update: The new Deepseek-R1 reasoning language model, Bytedance's Trae IDE, and my new book

2025-01-21T09:41:00.002-07:00

I spent a few days experimenting with Cursor last week. Bytedance's Trae IDE is very similar and is currently free to use with Claude Sonnet 3.5 and GPT-4o: https://www.trae.ai/home I would like to use Trae with my own API accounts but currently Bytedance is paying for LLM costs.

I have been experimenting with the qwen2.5 and qwen2.5-coder models that easily run on my M2Pro 32G Mac. For reasoning I have been going back to using OpenAI O1 and Claude Sonnet, but after my preliminary tests with Deepseek-R1, I feel like I can do most everything now on my personal computer.

I am using: ollama run deepseek-r1:32b

I recently published my new book “Ollama in Action: Building Safe, Private AI with LLMs, Function Calling and Agents” that can be read free online at https://leanpub.com/ollama/read

I am moving back to the Google platform, less excited by what Apple is offering

2024-09-19T13:48:00.004-07:00

I have been been playing with the Apple Intelligence beta’s in iPadOS and macOS and while I like the direction Apple is heading I am getting more use from Google’s Gemini, both for general analysis of very large input contexts, as well as effective integration my content in Gmail, Google Calendar, and Google Docs.

While I find the latest Pixel phone to be compelling, I will stick with Apple hardware since I don’t want to take the time to move my data and general workflow to a Pixel phone. The iPhone is the strongest lock-in that Apple has on me because of the time investment to change.

The main reason I am feeling less interested in the Apple ecosystem and platform is that I believe that our present day work flows are intimately wrapped up with the effective use of LLMs, and it is crazy to limit oneself to just one or two vendors. I rely on running local models on Ollama, super fast APIs from Groq (I love Groq for running most of the better open weight models), and other APIs from Mistral, OpenAI, Google, Anthropic, and other providers.

Except for accepting part time work, I am mostly retired and for my now more limited development requirements, I am using platform agnostic web based systems like Replit with their AI coding assistant and Google Colab with Gemini assistant.

I have a lot of excellent Apple hardware that I will use until it breaks and then decide on replacement gear based on cost performance. For years I have loved the Apple ecosystem but in modern times things change so quickly and I prefer to not being strongly loyal to any particular provider.

Getting closer to AGI? Google's NoteBookLM and Replit's AI Coding Agent

2024-09-18T14:07:00.007-07:00

Putting "closer to AGI?" in a blog title might border on being clickbait, but I will argue that it is not! I have mostly earned my living in the field of AI since 1982 and I argue that the existence of better AI driven products and the accelerating rate of progress in research, that we are raising the bar on what we consider AGI to be.

I have had my mind blown twice in the last week:

Today I took the PDF for my book "Practical Artificial Intelligence Programming With Clojure (you can read it free online here) and used it to create a notebook in Google's NotebookLM and asked for a generated 8 minute podcast. This experimental app created a podcast with two people discussing my book accurately and showing wonderful knowledge of technology. If you want to listen to the audio track that Google's NotebookLM created, here is a link to the WAV audio file

Last week I signed up for a one year plan on Replit.com after trying the web based IDE for Haskell and Python coding. After using the platform for a few days I used its AI coding agent to rewrite my Clojure web app CookingSpace.com in Javascript, HTML, and CSS. I estimate that doing this by hand would have taken me at least 30 hours, and I had something written and publicly deployed in less than an hour. This process would have been quicker but initially I asked for features like user accounts that I later decided I didn't want so I had to iterate with the AI coding agent to remove working functionality.

Why claim these two well engineered products make me feel that we are closer to AGI?

To be fair what makes NotebookLM and Replit AI coding agent so impressive is the product design and infrastructure that uses LLMs under the hood. Still, it is very difficult to say that the functionality of both these systems is not now superior to most humans.

I feel that the argument over AI and AGI gets dragged down by discussions of sentient AI taking over the world of somehow controlling humans. The thing to worry about is how people use AI.

I find it difficult to view AI as anything but a set of advanced tools, something to augment human productivity. In my view, advanced AI driven tools and products are very close to what I would call AGI.

New OpenAI gpt-o1-preview and gpt-o1-mini and one week experience with Replit.com AI Coding Agent

2024-09-13T10:23:00.004-07:00

I have only spent a short while experimenting with the ne gtp-o1 models: so far very impressive for science, math, and instruction following. You need a ChatGPT Plus account to try it, or you can perform rate limited queries for half the monthly cost using Abacus AI

The thing I am most impressed with (this week!) is the Replit.co AI coding agent that after briefly trying it I pre-paid for a one year subscription. I quickly rewrote a complex Clojure web app I wrote 12 years ago in JavaScript making it much less expensive to host CookingSpace.com (new JavaScript version)

I gave a live demo of Replit AI in my weekly AI demo and group chat. Please join: Mark Watson's informal AI presentations and group chats

Topics: Recipe: Mark’s African Stew, and converting my Clojure CookingSpace web site to JavaScript

2024-09-12T15:06:00.000-07:00

I wanted to convert my server side web site CookingSpace.com to mostly client side JavaScript. I used the Replit.com AI coding agent to do this, and while this is a subject for another blog article, I showed the Replit coding AI a snippet of my JSON recipes and nutrients file, and described in a few paragraphs the functionality of the new web site.

I want to share a recipe that I created:

Mark’s African Stew

Ingredients

Optional: 3/4 to 1 pound of Lamb shoulder, deboned and cut into cubes, leaving some fat on the meat
2 tablespoons of any non-seed oil (Avocado Oil is good). (Avoid seed oils like Canola, Corn, Soybean, Sunflower, etc.)
1 brown onion, coarsely chopped
1 egg plant, partially skinned, chopped into bite sized pieces, and heavily salt in a strainer. Before cooking, rinse well to remove excess salt!
ground coriander (at least 1 teaspoon)
Cumin (at least 2/3 teaspoon)
paprika (1 to 2 teaspoons, depending on how you like the taste of paprika)
ginger powder ( at least 1/2 teaspoon), or, even better use finely chopped fresh ginger
pealed and finely chopped)
cinnamon (1/4 to 1/4 teaspoon depending how much you like the taste of cinnamon)
black pepper (at least 1/3 teaspoon - nutritionally pairs well with the turmeric)
turmeric (bout 1/4 teaspoon), or, even better use fresh turmeric root peeled and finely chopped.
fresh chopped garlic (or garlic powder) added to taste. I use about 3 garlic cloves, you probably want less.
Something hot and spicy: dried chili peppers, or fresh chopped chili peppers, or hot chili sauce - whatever you like, adding enough ‘hot’ for your tastes.

1 cup chicken broth, or vegetable broth if you are not using the optional meat.
4 cups water, or more in stew dries out while cooking
1/4 pound of lentils
2 or 3 celery stocks, coarsely chopped. Use the green tops also! Use the root also.
1 14oz can of diced tomatoes
1 16oz can of cooked garbanzo beans, using water in can
2/3 cup chopped fresh cilantro, I use about half the stems (finely chopped)
Small pasta noodles, I use a very small amount as an accent
Optional: a table spoon of honey added after heat is turned off.
Juice from 1 fresh lemon (optional, but…)

Directions:

Brown the lamb in about 2 tablespoons of any non-seed oil (Avocado Oil is good) and remove, leaving oil and the small amount of fat from cooking the meat.

Sauté the onion

Turn off heat and add rinsed lentils, prepared eggplant (rinse salt off before adding to pan), and all spices.

Cook covered under low heat for about 20 minutes.

Add everything else: canned garbanzo beans, chopped celery, cilantro, diced tomatoes, and return the meat to the heavy pan, cover and simmer for a while. Cook for at least 10 minutes before adding pasta:

Add the small amount of plaster noodles last, calculating how long they should cook by doubling the time that you would cook them in boiling water.

When the pasta tastes done, cover the heavy pan, cook another minute, then turn off heat leaving the heat and leave the pan covered for 5 to 20 minutes.

Stir in fresh lemon juice and serve. After tasting at the table, add salt if necessary.

Code and notes from my recent talk: Exploring the Future of AI: Introduction to using LLMs using Python

2024-09-12T14:55:00.000-07:00

Topics: Large context prompts with LLMs vs. RAGs using embeddings vector stores. How to avoid LLM hallucination. 3 code demos.

started an informal code demo and group conversation Meetup group (link) and today I gave a fifteen minute code demo followed by a conversation with the attendees. Here is a GitHub repo with the code examples: https://github.com/mark-watson/talk2_LLM_Python_intro
Here are my talk notes:

Exploring the Future of AI: Introduction to using LLMs using Python
Riff on ‘AI grounding’ and how LLMs help: LLMs, trained on vast amounts of text, excel at recognizing patterns and providing contextually relevant responses. They mimic grounded understanding by referencing large datasets that encompass a variety of real-world scenarios. For example, they can infer meanings from complex contexts by drawing on their training data. When LLMs are integrated with other modalities, such as vision or audio (e.g., vision-language models), the grounding improves. These models can associate text with images or sounds, making the connections more robust and closer to a human-like understanding of concepts.
Tradeoffs between using large context LLMs: where a large body of text is added to a query prompt, to the alternative approach of breaking multiple documents into many separate chunks of text, calculating an embedding vector for each chunk, and then storing the chunks and their associated embedding vectors ins a vector data store.
Long-Context LLMs: designed to support processing large blocks of text, often an entire book, within a single prompt. These models can accommodate extended sequences of text, enabling them to consider more context at once. This is particularly useful for tasks that require maintaining continuity over long narratives or documents. However, long-context LLMs have limitations, such as performance degradation when the context becomes too long, which can lead to reduced accuracy in generating or retrieving relevant information. These models are also computationally expensive, as handling extensive sequences demands significant resources.
On the other hand, vector stores (or vector databases) work by converting text or other unstructured data into high-dimensional vectors using embeddings. These vectors are stored and can be retrieved based on their similarity to a query vector, allowing for efficient semantic search across vast datasets. This approach provides a form of “long-term memory” for LLMs, enabling them to access and retrieve relevant information from large collections of documents without needing to process the entire context at once. Vector stores are particularly useful in retrieval-augmented generation (RAG) systems, where they help the model to find and focus on the most relevant information, improving both efficiency and accuracy .
In essence, while long-context LLMs attempt to handle extensive information within the model’s processing window, vector stores offer an external memory solution that complements LLMs by efficiently managing and retrieving relevant information from larger datasets.

What about LLM hallucinations?
Long context windows and retrieval-augmented generation (RAG) data stores significantly reduce LLM hallucinations by improving the model's access to relevant and accurate information during the generation process.
1. Long Context Windows: When LLMs are equipped with long context windows, they can process and retain more information within a single session. This allows the model to maintain continuity and consistency over extended text, reducing the chances of fabricating information that doesn't align with the given context or user query. By having access to more surrounding context, the model can generate more coherent and accurate responses that are anchored in the actual input data.
2. RAG Embedding Vector Data Stores: In a RAG setup, an LLM is paired with a vector store that holds a vast amount of pre-processed, structured information. When a query is posed, the model retrieves relevant documents or data snippets from this store, which then informs the generation process. This retrieval step grounds the model's output in factual data, effectively reducing the likelihood of hallucinations. Since the model can rely on precise and contextually relevant information, it is less prone to generating plausible-sounding but incorrect or nonsensical content.
Together, these approaches enhance the reliability of LLM outputs. Long context windows allow the model to consider more of the input in a single pass, while RAG ensures that the model has access to verified information, leading to fewer instances of hallucination and more trustworthy results.

My Dad's work with Robert Oppenheimer and Edward Teller

2023-08-28T07:23:00.001-07:00

Robert Oppenheimer and Edward Teller facilitated my Dad getting a professorship at UC Berkeley when I was 3 years old. Oppenheimer left Berkeley but Teller was a good friend of my father and I remember him being in our home.

Three weeks ago, my wife and I were just leaving to see the new Oppenheimer movie when my Dad called. He mentioned that when I was in grade school he was invited to give a talk at Princeton. After his talk Oppenheimer talked with my Dad and invited him to have dinner at his house. My Dad said Oppenheimer was not well (I think he died of throat cancer soon afterwards) but his wife Kitty carried the conversation.

My Dad, Ken Watson, passed away 10 days ago on August 18, 2023.

Time and Attention Fragmentation in Our Digital Lives

2023-03-12T16:29:00.001-07:00

As humans we have evolved over a few million years to be both attentive and reactive to danger, live in social communities, and spend much of our time being in the present moment gathering and eating food and socializing.

The behavior of rapidly changing short attention to content on social media, too many good short form things to watch on streaming video entertainment platforms, are all rewiring our brains in an unnatural and unhealthy way.

I fight back, but in really simple ways that entail little ceremony:

Almost every morning I spend 30 minutes scanning Hacker News (about 10 minutes), Apple News (about 5 minutes), and the remaining time on Twitter and Mastodon finding interesting new (mostly tech) things. I make notes in a temporary Apple Note: links of things I may want to research, try, or simply read that day. I like to get this all done at once, and then not feel like I need to interrupt my activities during the day to “catch up” on what is happening in the world.

In a way, it seems like 30 minutes a day, summed up over 365 days in a year, is a frightening amount of life spent on what I would consider “indexing.” In a typical day, I might spend between 1 and 3 hours taking long attention span meanderings over stuff on my daily (temporary) Apple note. While I feel kind of bad about the 30 minutes of chaotic short attention span “discovery and indexing” time each morning, I don’t feel bad at all, in fact the opposite I feel good about spending more time during the day diving more deeply into the few things on my daily list.

Another thing I feel very good about is spending long periods of time socializing with friends and family, reading fiction (and sometimes technical books) or watching a single good movie in the evening.

ChatGPT as part of the evolution of programming languages

2023-02-02T15:39:00.007-07:00

In the 1940s von Neumann and his colleagues created conceptual models for computer architectures that were oriented toward the engineering problems of building computing devices, and not for making it easier for humans to write programs. The Lambda Calculus and also the design of the Prolog programming language are the first real efforts that I am aware of to place emphasis on how we humans think and solve problems.

I had a thought earlier today that I keep coming back to: there are concise programming languages that can be more difficult to write code with, but once done the code is more valuable because of its conciseness that yields better readability.

I have been fascinated by, and use Copilot and ChatGPT to write code and sometimes it works well enough. What will the effects of ChatGPT and future LLMs be on the popularity of niche languages like Prolog and APL?

All things considered I would often rather have a concise program in Prolog or a flavor of Lisp than a much larger program written in a verbose language (e.g., Java). If I can describe a problem and generate a quality program in any language, then I will ask for generation in my favorite languages.

Because of its possible utility for so many tasks, ChatGPT and future similar systems really seem like programming environments to me.

Collecting textual context data for a problem, appending a series of questions to be answered, and sending the resulting text off to ChatGPT also seems like programming to me.

Elon Musk & Twitter, Tech job market, my writing projects

2022-11-05T08:30:00.000-07:00

I find it sad that Musk's purchase of Twitter is turning out so badly. I read that advertisement pre-sales for next year are very low, thus the urgent need to cut expenses. I don't dissagree with Elon Musk's original idea of having an uncensored platform, but the execution is not good. My best wishes to everyone at Twitter (and other tech companies) who have lost their jobs recently. The job market was crazy-good for a few years, and now I expect it to more like after the 2000 dot-com-crash, at least for a few years.

I advise people to take a different approach to managing their careers. As fantastic as online (often free) classes are for teaching useful stuff like machine learning, front end development, etc., this has also greatly expanded the global talent pool. Now more than ever, I advise learning through doing your own projects. I have literally done this myself for the last 40 years: I spend my own time experimenting with tech that both fascinates me and might be useful for my employer. Take control of what you want to learn and work on even if initially it is unpaid work.

I have been using my Mastodon account https://mastodon.social/@mark_watson but I have plans what so ever to give up Twitter (you can follow me at https://twitter.com/mark_l_watson). I rely on Twitter to get tech news, thanks to the awesome people I follow. I am working hard to also follow awesome people on Mastodon.

About 12 years ago, I used LaTex to write a semantic web book, but in 2 editions: Common Lisp and Java. It was very cool to have 2/3 of the manuscript common text, and about 1/3 unique to the programming language. For the last two evenings, I have been revisiting my old manuscript materials because I am thinking of updating the material and also creating additional editions for more programming languages: Python, JavaScript, and maybe Swift and Prolog.

I had forgot how cool TeX and LaTex are. It was very easy to start working with again, even after a 12 year gap.

Not really retired 😀

2022-10-01T10:45:00.000-07:00

I read with some humor my last blog post from 6 months ago, saying that I was retired. Ha! As I mentioned 6 months ago, my wife has chronic health problems but those have stabalized, and life is now fairly good. I did start a very much part time (15 hours/week) advisory gig with Mind AI about 4 months ago. Enjoyable work on an interesting product. I am not performing any substantial development work, rather spending most of my time as an architect and advisor. I am 71 years old, and leaving heavy lifting development work to younger and more energetic co-workers is for the best.

I have a huge backlog of writing projects, mostly on hold for the moment because of my work at Mind AI:

A new book "Artificial Intelligence Programming in Python: Exploring the Boundaries of Deep Learning, Symbolic AI, and Knowledge Representation".
Edits for my Common Lisp book, adding new examples.
Ideas and some new code for my Swift AI book.

For the last year I have been basically giving away my recent eBooks for free on Leanpub by setting the minimum price to $0.00. I stopped doing this today. Now all of my recent eBooks are available for free online reading on my website https://markwatson.com and readers have to pay for PDF/ePub/Kindle versions on Leanpub.

I have been very happy to get 5 visits to my home in Sedona this year from old collegues, including my boss from 40 years ago and a recent co-worker from Olive AI. Great fun!

I also have a new hobby: I joined the Northern Arizona Audubon Society. Yes, I am now a bird watcher! I have some experience at this bird watching activity: I have had my pet parrot for 20 years. Looking at wild birds is fun also, and the birding hikes are guided by experts so this is a learning experience.

I retired yesterday: my list of things to do in retirement

2022-03-05T06:44:00.002-07:00

What does an intelligent person do in retirement? That is a question of individual tastes but I will share my list of 20 things.

Yesterday was my last day working on a recommendation model at Babylist. Babylist is a great company to work for but I decided to retire in order to spend more time helping my wife who now has chronic health problems. When I write books, my wife enjoys editing my work so we will keep doing that. I also plan on being a gentleman computer scientist by working on open source deep learning applications and semantic web/linked data tools and applications.

I may end up not doing all of these things, but generally I plan on spending more time on current interests and starting some new hobbies:

Retirement Activities

Join an Internet Chess club **DONE**
Get a fishing license
Video Games
Improve my cooking/recipe web site
Reading
Release new editions for my 3 most popular eBooks
Practice guitar, Native American Flute, and didgeridoo
Eco-box vegetable garden
Set up my model trains
Play online Go. **DONE** online-go.com
Learn to use the Ham radio (listen only, no license) that my son in law Josh gave me
Photography
Wednesday and Saturday morning bird watching tours at Red Rock State Park
Kayaking
Take more road trips with Carol and friends
Using my new meat smoker
Hiking with my friends
Marksmanship: spend more time at the shooting range (I own a 22 caliber target pistol)
Meditation
Start taking flying lessons again

I may not do all of these activities but I found it useful to put some thought into what I want to now spend my time on.

My productivity hacks

2021-09-21T08:56:00.002-07:00

Like most people, there are many more things that I would like to do than I have time to do. I believe that learning to identify action items that simply should not be done is valuable, but not easy to do. I am mildly attention deficit in the sense that I can only think about or attend to one thing at a time. For a computer scientist, this has been a super power, but is personally inconvenient. I keep 3 TODO lists:

TODO high priority - I tend to have 1 to 3 things on this list. I time box my activities so this list is the actions that I will rotate through. I usually work in 60 to 90 minute sprints, but for deep coding this may be 3 or 4 hours.
TODO open - everything that I would like to do, but a lot of stuff on this list gets deleted with no further effort (the all important decisions on what not to do).
TODO done - instead of deleting completed actions on "TODO high priority" I cut the text and paste the action text to the top of this list.

I really like the Freedom.to web site and app blocker. I feel like I am setting parental controls on myself. I keep a list of fun but not very useful web sites and apps (e.g., Hacker News, Twitter, Reddit tech groups, and Facebook) that can only be accessed in the early morning, briefly during lunch, and in the evening times. I sometimes temporarily unblock sites if I want to quickly post something to social media.

I place no limits on the time I spend on meditation, hiking, kayaking, cooking + meals, and reading good books. I try to limit watching streaming media to a few shows and also movies I watch with my wife and/or friends.

DBPedia Natural Language Interface Using Huggingface Transformer

2021-03-24T10:36:00.006-07:00

I prototyped a simple natural language question answering demo in about 90 minutes. I accept a query like “where does Bill Gates work?”, find the likely URI for Bill Gates, collect some comment text for this DBPedia entity, and then pass the original query to the transformer model with the “context” being the comment text collected via a SPARQL query. I run this on Google Colab. Note that I saved my Jupyter Notebook as a python file that is in the listing below. Note the use of ! to run shell commands (e.g., !pip install transformers).

# -*- coding: utf-8 -*-
"""DbPedia QA system.ipynb

Automatically generated by Colaboratory.

Original file is located at
    https://colab.research.google.com/drive/1FX-0eizj2vayXsqfSB2ONuJYG8BaYpGO

**DBPedia Question Answering System**
Copyright 2021 Mark Watson. All rights reserved. License: Apache 2
"""

!pip install transformers
!pip install SPARQLWrapper

from transformers import pipeline

qa = pipeline(
    "question-answering",
    #model="NeuML/bert-small-cord19qa",
    model="NeuML/bert-small-cord19-squad2",
    tokenizer="NeuML/bert-small-cord19qa"
)

!pip install import spacy
!python -m spacy download en_core_web_sm

import spacy

nlp_model = spacy.load('en')

from SPARQLWrapper import SPARQLWrapper, JSON

sparql = SPARQLWrapper("http://dbpedia.org/sparql")

def query(query):
  sparql.setQuery(query)
  sparql.setReturnFormat(JSON)
  return sparql.query().convert()['results']['bindings']

nlp_model = spacy.load('en')

def entities_in_text(s):
    doc = nlp_model(s)
    ret = {}
    for [ename, etype] in [[entity.text, entity.label_] for entity in doc.ents
        ]:
        if etype in ret:
            ret[etype] = ret[etype] + [ename]
        else:
            ret[etype] = [ename]
    return ret

def dbpedia_get_entities_by_name(name, dbpedia_type):
  sparql = "select distinct ?s ?comment where {{ ?s <http://www.w3.org/2000/01/rdf-schema#label>  \"{}\"@en . ?s <http://www.w3.org/2000/01/rdf-schema#comment>  ?comment  . FILTER  (lang(?comment) = 'en') . ?s <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> {} . }} limit 15".format(name, dbpedia_type)
  #print(sparql)
  results = query(sparql)
  return(results)

entity_type_to_type_uri = {'PERSON': '<http://dbpedia.org/ontology/Person>',
    'GPE': '<http://dbpedia.org/ontology/Place>', 'ORG':
    '<http://dbpedia.org/ontology/Organisation>'}

def QA(query_text):
  entities = entities_in_text(query_text)

  def helper(entity_type):
    ret = ""
    if entity_type in entities:
      for hname in entities[entity_type]:
        results = dbpedia_get_entities_by_name(hname, entity_type_to_type_uri[entity_type])
        for result in results:
          ret += ret + result['comment']['value'] + " . "
    return ret

  context_text = helper('PERSON') + helper('ORG') + helper('GPE')

  print("Answer from transformer model:")
  print("Original query: ", query_text)
  print("Answer:")

  answer = qa({
                "question": query_text,
                "context": context_text
               })
  print(answer)

QA("where does Bill Gates work?")
QA("where is IBM is headquartered?")
QA("who is Bill Clinton married to?")

The output looks like this:

Answer from transformer model:
Original query:  where does Bill Gates work?
Answer:
{'score': 0.31679803133010864, 'start': 213,
 'end': 222, 'answer': 'Microsoft'}
Answer from transformer model:
Original query:  where is IBM is headquartered?
Answer:
{'score': 0.8704459071159363, 'start': 115, 'end': 131,
 'answer': 'Armonk, New York'}
Answer from transformer model:
Original query:  who is Bill Clinton married to?
Answer:
{'score': 0.00018714569159783423, 'start': 480, 'end': 505,
 'answer': 'former secretary of state'}

I have a new job helping to build a Knowledge Graph at Olive AI

2020-10-22T16:41:00.000-07:00

I retired (my last job was Master Software Engineer and the manager of a deep learning team at Capital One) a year ago April and was enjoying time with friends and family, doing personal research in hybrid AI, lots of writing, and volunteering at our local food bank. I stopped my volunteer work with COVID-19 and welcomed the opportunity last month to start work at Olive AI working on a very strong Knowledge Graph team. I believe in their mission and the work and the people are great!

It is refreshing to leave the deep learning field, at least for a while. My heart is in developing stronger AI that can explain its actions and adapt flexibly to help people in their lives. I always take a humans-first stand on technology. AI systems should help us get our work done efficiently and remove tedium, allow us more time for creative activities, and generally enjoy our own humanity.

I have tried to take advantage of extra time during the COVID-19 pandemic

2020-07-08T06:36:00.000-07:00

My wife Carol and I have been practicing social distancing and wearing masks for shopping for over 5 months now. Welcome to the new normal and a crazy world in which entertaining and seeing friends is done by meeting in people's yards and everyone bringing their own "meal in a bag."

I enjoy writing so I have been updating my recent books, starting with Loving Common Lisp, or the Savvy Programmer's Secret Weapon and A Lisp Programmer Living in Python-Land: The Hy Programming Language. These are free to read online and licensed with Creative Commons Share and Share Alike, No Commercial Reuse, so you can also find copies on the web (hopefully up to date copies!).

Last month I started a much larger project: I have not updated my book Practical Artificial Intelligence Programming With Java since the fourth edition was published in 2013. I have discarded a lot of older material like exert systems, and have three new chapters on the semantic web and also a new chapter on deep learning. I also copied the material on anomaly detection from my Power Java book that is now discontinued and updated that material. Lastly, I am revisiting how readers install, run, and experiment with the code examples. I am still using maven but I am being more consistent, all of the examples are also installable libraries, and now some of the examples use the libraries developed in other examples.

For fun, I have been buying more material for my Oculus Quest VR device. Favorites include ping pong, racket ball, the Star Wars Darth Vader Immortal 3 volume set, and I enjoy a lot of 3D art that people post.

For exercise I try to hike every morning from 5:30am to about 6:30am (I live in the mountains of Central Arizona and even at high altitude it gets warmer later in the day). My wife and I cancelled our gym memberships so I bought myself some weights that I keep in my home office. I am a huge fan of the Apple Watch and I use it to track health and fitness activities.

I hope that you, dear reader, are doing well in these crazy times we are living in.

Custom built SBCL and using spaCy and TensorFlow in Common Lisp

2020-02-18T09:33:00.000-07:00

Here are some of my of my recent notes that might save you some time, or teach you a new trick.

I have had good results using the py4cl library if I wrap API calls to TensorFlow or spaCy in a short Python library that calls Python libraries and returns results in simple types like strings and dictionaries. I just committed a complete example (Python library and Common Lisp client code) to the public repo for my book Loving Common Lisp, or the Savvy Programmer's Secret Weapon that will be added to the next edition of my book. Here is a link to the subdirectory with this new example in my repo: https://github.com/mark-watson/loving-common-lisp/tree/master/src/spacy

I frequently make standalone executable programs using SBCL and I just noticed a great tip from Zach Beane for compressing the size of standalone executables. Start with rebuilding SBCL from source to add the compression option; get the source code and:

./make.sh --with-sb-thread --with-sb-core-compression
sh install.sh

When I now build my KGCreator application with:

(sb-ext:save-lisp-and-die "KGcreator" :toplevel #'kgcreator :executable t :compression t)

then the size of the standalone executable is reduced from 93MB to 19MB. I don't notice any extra startup time which is important for command line utilities.

Protecting oneself from surveillance capitalism

2020-02-10T17:30:00.000-07:00

As an author I find occasional use of Facebook and Twitter to be useful for “broadcasting” notifications of my new books, open source projects, etc. I also find gmail to be useful for some types of email.

Still, I do like to take a few easy steps to push back a little against the free use of my web behavioral data to profit corporations who I don’t do business with (and those I do):

Use ProtonMail as my primary email
Use Firefox on my Linux and macOS laptops with individual containers for Google, FaceBook, etc.
On iOS devices, favor browsing with private tabs.
Use a VPN when I am traveling and when I need to use public WiFi
Limit use of my gmail address to a backup email and as a junk email address.
For online purchases from Amazon, etc. use a secure email service that does not use the contents of your email to market to you and as data to sell to 3rd parties.
Frequently delete all cookies from web browsers that you use.
Use private browsing windows for routine use of the web
Prefer to access the web on an iOS or Chromebook - probably more secure than a laptop or PC.
Logout from web apps like Facebook, LinkedIn, Amazon, Google, Twitter, etc. to avoid active sessions.
This takes some time, but on mobile devices, Apple/Windows laptops, etc. turn off tracking on as many apps as possible.

My hopes and predictions for the next 10 years

2020-01-02T18:36:00.000-07:00

Hello everyone, I wish everyone a happy and healthy new year!

Here are my predictions for the next ten years:

Wearable devices like the Apple Watch will become widely used and because of user pushback we will see company’s like Apple, Google, Toshiba, Huawei, Samsung, etc. start to support standards that allow, for example, an Apple Watch to interact with a Samsung TV. Further, I expect a single personal device (watch or phone) to be for most users a connection hub that interacts with public kiosks, displays, input devices, etc.
Deep learning architectures and techniques will rapidly improve and will continue to rule the world, at least for a while. I expect at least one new dramatic paradigm shift for AI beyond current deep learning, reinforcement learning, etc. models.
The world economies will get hit hard and wealth will be in a larger part measured in terms of ownership of water and food production, manufacturing, technology IP, and hopefully hard assets like gold, silver, and secure cryptocurrencies. I expect most currencies will eventually be backed by about 20% gold and this will enforce some stability and help eliminate wild printing of fiat currencies. I expect SDRs, or something like them, to be the reserve currency. In ten years, I would expect SDRs to be about equally made up of US dollars, Euros, and Chinese currency.
Politically, world elites will continue to assume almost total control of news media and governments but they as a group will start to share more resources with highly trained and educated workers and allow the creation of a secure safety net that will provide a safer society because almost all people will have food and shelter needs met (this may take more than ten years). We all have to live on planet earth and everyone’s life improves with fewer wars and higher productivity.