Random thoughts on coding and technology

Building a Private Local LLM Server on Windows with llama.cpp, Vulkan and Qwen

noreply@blogger.com (Unknown) — Mon, 24 Aug 2026 19:36:58 +0000

I wanted to turn an existing Windows machine into a fully self-hosted LLM server: no cloud inference, no OpenAI dependency, and an API that my own applications could call in essentially the same way they call OpenAI or Ollama.

The hardware was not designed as an AI workstation:

CPU: AMD Ryzen 5 7430U
Cores: 6 cores / 12 threads
RAM: 32 GB
GPU: AMD Radeon integrated graphics
GPU backend: Vulkan
OS: Windows
Dedicated VRAM: only around 512 MB
Shared GPU memory: approximately 16 GB

Despite those limitations, the final result was:

Windows Server
     │
     ▼
llama.cpp + Vulkan
     │
     ▼
Multi-model llama-server router
     │
     ├── Qwen3-14B Q5_K_M
     │      "fast"
     │
     └── Qwen3-Coder-30B-A3B Q4_K_M
            "coder"
     │
     ▼
OpenAI-compatible API
     │
     ├── Browser UI
     ├── Python
     ├── C#
     ├── agents
     └── other machines on the LAN

The server now starts automatically after a reboot and exposes both models through one endpoint.

This post documents the complete process, including the failures along the way.

1. Starting with `llmfit`

I first used llmfit to work out what models were theoretically capable of running on the machine.

It detected roughly:

CPU: AMD Ryzen 5 7430U
RAM: ~31 GB
GPU: AMD Radeon Graphics
Backend: Vulkan

The important discovery was that although Windows initially reported only around:

0.5 GB VRAM

this was misleading.

The Radeon is an integrated GPU using Unified Memory Architecture (UMA), meaning it can borrow system memory.

After installing llama.cpp, this became much clearer:

llama-cli.exe --list-devices

returned:

Available devices:
  Vulkan0: AMD Radeon (TM) Graphics (16345 MiB, 15527 MiB free)

So Vulkan could address approximately 16 GB of shared GPU memory.

That dramatically changed what models were practical.

2. Why llama.cpp instead of Ollama

Ollama is excellent for convenience, but I wanted lower-level control over:

GPU layer offloading
Vulkan
context size
batch size
micro-batch size
KV-cache placement
memory allocation
model quantization
multi-model routing

llama.cpp also provides an OpenAI-compatible API, so applications do not need to be coupled to a llama.cpp-specific protocol.

The resulting architecture is:

GGUF
 ↓
llama.cpp
 ↓
llama-server
 ↓
/v1/chat/completions

This means the same client abstractions can later work with:

OpenAI
llama.cpp
Ollama's OpenAI compatibility endpoint
vLLM
LM Studio
LocalAI
other OpenAI-compatible servers

3. Installing llama.cpp on Windows

There was no need to compile llama.cpp.

I downloaded the official Windows x64 Vulkan build from the llama.cpp releases.

The relevant package is named approximately:

llama-bXXXXX-bin-win-vulkan-x64.zip

I extracted it to:

C:\llama.cpp

The directory contained binaries such as:

C:\llama.cpp\
    llama-cli.exe
    llama-server.exe
    llama-bench.exe
    ...

Testing the executable:

C:\llama.cpp\llama-cli.exe --version

gave:

version: 0.1.2-dev
build: 10517
commit: dc72703fc
built with Clang 20.1.8 for Windows x86_64

4. The first problem: Windows "Bad Image"

Initially Windows refused to start one of the llama.cpp DLLs with a message similar to:

llama-cli.exe - Bad Image

Error status 0xC0E90002

The failing DLL was:

llama-cli-mtmd.dll

This turned out not to be a Vulkan problem.

Windows security controls were blocking one of the unsigned/open-source binaries.

After allowing the application appropriately, this worked:

C:\llama.cpp\llama-cli.exe --version

This was an important distinction.

A true Vulkan failure would more likely produce something such as:

VK_ERROR_...
failed to load Vulkan backend
no Vulkan devices found

Instead, Windows had been stopping the process before Vulkan initialization even happened.

5. Confirming Vulkan actually worked

The definitive test was:

C:\llama.cpp\llama-cli.exe --list-devices

Output:

Available devices:
  Vulkan0: AMD Radeon (TM) Graphics (16345 MiB, 15527 MiB free)

At that point I knew:

the Vulkan llama.cpp build worked;
the AMD driver worked;
llama.cpp could see the Radeon;
approximately 15.5 GB could potentially be allocated by Vulkan.

6. Choosing the first model

The first model I tested was:

Qwen3-Coder-30B-A3B-Instruct

using:

Q4_K_M

quantization.

The file was approximately:

18.6 GB

and was stored as:

C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

The model uses a Mixture-of-Experts architecture, which made it particularly interesting for relatively modest hardware.

7. Hugging Face SSL problems

Initially I tried letting llama.cpp download the model directly:

llama-cli.exe -hf lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M

Instead I received:

HTTPLIB failed:
SSL server verification failed

error:
--model is required

The second error was just a consequence of the first: llama.cpp failed to download anything and therefore had no model to load.

Rather than waste time debugging Windows certificate handling, I downloaded the GGUF manually.

The resulting structure was:

C:\
├── llama.cpp\
│   ├── llama-cli.exe
│   ├── llama-server.exe
│   └── ...
│
└── models\
    └── Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

I actually prefer this arrangement for a server.

Once downloaded, inference no longer depends on:

Internet connectivity
Hugging Face
SSL certificates
authentication
external availability

8. The first GPU memory failure

I initially tried maximum GPU offloading:

C:\llama.cpp\llama-cli.exe ^
  -m C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf ^
  -c 8192 ^
  -ngl 99 ^
  -p "Write a high performance C# lock-free ring buffer"

This failed with:

ggml_vulkan:
Device memory allocation failed

vk::Device::allocateMemory:
ErrorOutOfDeviceMemory

This made sense.

The model itself was approximately:

18.6 GB

while Vulkan could address around:

15.5 GB

and Vulkan also needs memory for:

compute buffers
KV cache
staging buffers
runtime allocations
graph execution
Windows/WDDM allocations

So -ngl 99 was too aggressive.

9. Partial GPU offloading

I next tried:

-ngl 20

with a smaller 4K context:

C:\llama.cpp\llama-cli.exe ^
  -m C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf ^
  -c 4096 ^
  -ngl 20 ^
  -p "Write a high performance C# lock-free ring buffer"

This got further, but failed while creating the compute buffers:

failed to allocate Vulkan0 buffer
failed to allocate compute pp buffers

The allocation that failed was only around:

228 MB

which showed how close the machine was to its Vulkan memory ceiling.

10. Reducing batch and micro-batch sizes

The eventual working configuration was:

C:\llama.cpp\llama-cli.exe ^
  -m C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf ^
  -c 4096 ^
  -ngl 10 ^
  -b 128 ^
  -ub 128 ^
  --no-kv-offload ^
  -p "Write a high performance C# lock-free ring buffer"

Or on one line:

C:\llama.cpp\llama-cli.exe -m C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf -c 4096 -ngl 10 -b 128 -ub 128 --no-kv-offload -p "Write a high performance C# lock-free ring buffer"

This worked.

The important settings were:

-c 4096

4,096-token context.

-ngl 10

Only ten model layers were explicitly offloaded to Vulkan.

-b 128

Small prompt-processing batch.

-ub 128

Small micro-batch.

--no-kv-offload

Keep the KV cache out of the already-constrained Vulkan allocation.

11. First benchmark

The model produced approximately:

Prompt:     14.6 tokens/s
Generation: 4.6 tokens/s

The UI later showed approximately:

4.8 tokens/s

for generation.

That is not particularly fast, but it is entirely usable for a 30B-class coding model running on a low-power Ryzen APU.

The main limitation is memory bandwidth.

The architecture looks roughly like this:

             32 GB DDR RAM
                   │
          ┌────────┴────────┐
          │                 │
     Ryzen CPU         Radeon iGPU
          │                 │
          └────────┬────────┘
                   │
             Same memory bus

Unlike a discrete GPU, the Radeon does not have hundreds of GB/s or ~1 TB/s of dedicated VRAM bandwidth.

Both CPU and GPU are competing for system-memory bandwidth.

12. Starting the browser UI

Once command-line inference worked, the next step was llama-server.exe.

I used:

C:\llama.cpp\llama-server.exe ^
  -m C:\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf ^
  -c 4096 ^
  -ngl 10 ^
  -b 128 ^
  -ub 128 ^
  --no-kv-offload ^
  --host 127.0.0.1 ^
  --port 8080

Then I opened:

http://127.0.0.1:8080

and the built-in llama.cpp chat UI appeared.

At the same time, the OpenAI-compatible API became available under:

http://127.0.0.1:8080/v1

13. Calling the local model from Python

Because llama.cpp exposes OpenAI-compatible endpoints, the normal OpenAI Python client can be used.

For example:

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key="local"
)

response = client.chat.completions.create(
    model="local",
    messages=[
        {
            "role": "user",
            "content": "Write a C# bounded concurrent queue."
        }
    ]
)

print(response.choices[0].message.content)

The application does not need to know much about the underlying inference engine.

Conceptually:

Application
    │
    │ OpenAI API
    ▼
localhost:8080/v1
    │
    ▼
llama-server
    │
    ▼
Qwen GGUF

14. The context-size problem

Eventually the UI produced:

request (7166 tokens) exceeds the available context size (4096 tokens)

This happened because the server was started with:

-c 4096

and the conversation had grown beyond that.

A context contains more than just the latest user message:

system prompt
+
previous user messages
+
previous assistant responses
+
new request
+
generated response

A larger context can be configured, for example:

-c 8192

or:

-c 12288

or:

-c 16384

but larger context windows have costs:

larger KV cache
more RAM
slower prompt processing
longer cold-context ingestion

On this machine, a roughly 8K–16K context is far more sensible than chasing enormous theoretical context windows.

15. Adding a faster model

The 30B coding model worked, but approximately:

4.8 tokens/s

is slow for everyday interactive use.

So I added:

Qwen3-14B

using:

Q5_K_M

quantization.

The second model became:

C:\models\Qwen3-14B-Q5_K_M.gguf

The final model directory was therefore:

C:\models\
├── Qwen3-14B-Q5_K_M.gguf
└── Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf

The idea was:

Qwen3-14B
    ↓
fast/default work

Qwen3-Coder-30B
    ↓
harder coding work

16. Multi-model routing

I did not want two permanent llama-server.exe processes consuming RAM simultaneously.

Instead, llama.cpp supports model presets and routing.

The goal was:

                  llama-server
                       │
                --models-max 1
                       │
          ┌────────────┴────────────┐
          │                         │
       "fast"                    "coder"
     Qwen3-14B              Qwen3-Coder-30B
          │                         │
          └────────────┬────────────┘
                       │
              only one loaded
                 at a time

That is exactly what a 32 GB machine needs.

17. Creating `models.ini`

I created:

C:\llama.cpp\models.ini

with:

version = 1

[*]
parallel = 1
jinja = true
cache-prompt = true

[fast]
model = C:/models/Qwen3-14B-Q5_K_M.gguf
ctx-size = 8192
n-gpu-layers = 99
batch-size = 128
ubatch-size = 128
no-kv-offload = true
load-on-startup = true

[coder]
model = C:/models/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
ctx-size = 8192
n-gpu-layers = 10
batch-size = 128
ubatch-size = 128
no-kv-offload = true
load-on-startup = false

The names:

[fast]
[coder]

become model aliases.

18. Another mistake: unsupported `default-model`

Initially the configuration contained:

default-model = true

under the fast model.

The server failed with:

failed to initialize router models:
option 'default-model' not recognized in preset 'fast'

That option was not supported by the llama.cpp build I was using.

Removing it solved the issue.

The important startup option was instead:

load-on-startup = true

for the fast model.

19. Starting the multi-model router

The router was started using:

C:\llama.cpp\llama-server.exe ^
  --models-preset C:\llama.cpp\models.ini ^
  --models-max 1 ^
  --host 127.0.0.1 ^
  --port 8080

The critical option is:

--models-max 1

Without it, multiple models might remain loaded simultaneously.

That would be a bad idea with:

~10 GB 14B model
+
~19 GB 30B model
+
Windows
+
KV caches
+
compute buffers

With a one-model limit, llama.cpp can effectively behave like:

request model="fast"
        ↓
load fast

request model="coder"
        ↓
unload fast
        ↓
load coder

The load transition takes time, but RAM remains manageable.

20. Calling different models through the same API

Now one server can serve both models.

The base endpoint remains:

http://127.0.0.1:8080/v1

For normal work:

response = client.chat.completions.create(
    model="fast",
    messages=[
        {
            "role": "user",
            "content": "Explain async iterators in C#."
        }
    ]
)

For harder coding tasks:

response = client.chat.completions.create(
    model="coder",
    messages=[
        {
            "role": "user",
            "content": """
            Review this MPMC queue implementation for
            memory-ordering and publication bugs.
            """
        }
    ]
)

Same API.

Different backend model.

21. Making the server accessible over the LAN

For access from other machines, I changed:

--host 127.0.0.1

to:

--host 0.0.0.0

The resulting server command became approximately:

C:\llama.cpp\llama-server.exe ^
  --models-preset C:\llama.cpp\models.ini ^
  --models-max 1 ^
  --sleep-idle-seconds 600 ^
  --host 0.0.0.0 ^
  --port 8080

Now another machine could use:

http://SERVER-NAME:8080

For example, the final browser UI was reachable using the Windows hostname directly:

http://win-2nsdp31hmq3:8080

22. Windows Firewall got in the way

During unattended startup, I eventually discovered another issue.

Windows displayed:

Do you want to allow public and private networks
to access this app?

llama-server.exe

This explained why startup appeared to fail.

The process was waiting for an interactive firewall decision.

When no user was logged in, nobody could click the dialog.

For a LAN-accessible server, the cleaner solution is a narrow Windows Firewall rule.

From elevated PowerShell:

New-NetFirewallRule `
  -DisplayName "llama.cpp API - Private LAN" `
  -Direction Inbound `
  -Program "C:\llama.cpp\llama-server.exe" `
  -Protocol TCP `
  -LocalPort 8080 `
  -RemoteAddress LocalSubnet `
  -Profile Private `
  -Action Allow

This permits:

llama-server.exe
TCP 8080
Private profile
Local subnet

rather than allowing arbitrary public-network access.

23. Making llama.cpp start after reboot

Initially I tried a Task Scheduler trigger:

At startup

but this was unreliable.

The machine also runs Docker Desktop, which normally expects an actual Windows user session.

There is also a practical issue with running GPU/Vulkan applications in a non-interactive Windows session.

The eventual solution was:

reboot
   ↓
automatic Windows login
   ↓
interactive desktop session created
   ↓
Docker Desktop starts
   ↓
llama.cpp starts
   ↓
workstation locks

This has proved much more reliable.

24. Windows automatic logon

For automatic logon I used Microsoft's Sysinternals Autologon utility rather than manually storing a plaintext password in Winlogon registry keys.

After enabling automatic logon for the server account:

Windows boots
      ↓
user automatically logs in

This creates the normal user session required by the software stack.

25. Starting llama.cpp at logon

Instead of:

At startup

I changed the llama.cpp Task Scheduler trigger to:

At log on

for the server user.

I also added a short delay so Windows, networking and the graphics stack could initialize first.

Approximately:

At log on
Delay: 30 seconds

The task runs:

C:\llama.cpp\start-router.cmd

26. The final startup script

The launcher is approximately:

@echo off

cd /d C:\llama.cpp

echo ============================================== >> C:\llama.cpp\llama-server.log
echo Starting llama.cpp at %date% %time% >> C:\llama.cpp\llama-server.log
echo ============================================== >> C:\llama.cpp\llama-server.log

C:\llama.cpp\llama-server.exe ^
  --models-preset C:\llama.cpp\models.ini ^
  --models-max 1 ^
  --sleep-idle-seconds 600 ^
  --host 0.0.0.0 ^
  --port 8080 ^
  >> C:\llama.cpp\llama-server.log 2>&1

This gives me a persistent log:

C:\llama.cpp\llama-server.log

which is very useful for diagnosing unattended-startup problems.

27. Automatically locking the workstation

Automatic login would otherwise leave the machine unlocked.

So after login and startup I schedule:

rundll32.exe

with:

user32.dll,LockWorkStation

after a short delay.

The resulting sequence is:

Machine reboots
       ↓
Windows boots
       ↓
Automatic login
       ↓
AMD/Vulkan session initialized
       ↓
Docker Desktop starts
       ↓
llama.cpp router starts
       ↓
Firewall already configured
       ↓
Windows locks
       ↓
Services remain running

Locking is important.

It preserves the logged-in session; logging out would terminate processes belonging to that user.

28. The final architecture

The finished environment looks like this:

                    WINDOWS MACHINE
                           │
                           ▼
                    automatic login
                           │
              ┌────────────┴────────────┐
              │                         │
       Docker Desktop             llama.cpp
                                        │
                                        ▼
                                llama-server router
                                   port 8080
                                        │
                               --models-max 1
                                        │
                   ┌────────────────────┴────────────────────┐
                   │                                         │
                   ▼                                         ▼
             fast                                       coder
        Qwen3-14B Q5_K_M                    Qwen3-Coder-30B-A3B
             ~10 GB                                 ~18.6 GB
          full/large GPU                         partial Vulkan
            offload                                 offload
                   │                                         │
                   └────────────────────┬────────────────────┘
                                        │
                                ONE MODEL AT A TIME
                                        │
                     ┌──────────────────┼──────────────────┐
                     │                  │                  │
                     ▼                  ▼                  ▼
                  Web UI             Python               C#
                     │                  │                  │
                     └──────────────────┼──────────────────┘
                                        │
                              OpenAI-compatible API
                                        │
                         http://SERVER:8080/v1

29. What I learned

Integrated GPUs are more useful than the VRAM number suggests

Windows initially showed roughly:

512 MB dedicated VRAM

which sounded hopeless.

But Vulkan reported:

~16 GB accessible GPU memory

because the Radeon uses shared system RAM.

That made reasonably large models practical.

Shared memory is not extra memory

The machine does not have:

32 GB system RAM
+
16 GB VRAM
=
48 GB

It has:

32 GB physical RAM

shared between:

Windows
CPU inference
GPU inference
KV cache
compute buffers
other applications

The 16 GB figure is effectively a GPU allocation budget over the same physical memory.

Maximum GPU offload is not necessarily optimal

For the 30B model:

-ngl 99

failed.

Even:

-ngl 20

eventually exhausted Vulkan compute-buffer memory.

The stable configuration became:

-ngl 10

There is little value in maximizing GPU layers if doing so makes the system unstable.

Batch size matters enormously

Reducing:

-b

and:

-ub

made the difference between failure and successful inference.

For interactive use, giant prompt-processing batches are unnecessary.

Keep enough memory for runtime buffers

A model fitting numerically into GPU memory does not imply it will run.

You also need room for:

model tensors
+
KV cache
+
compute buffers
+
graph buffers
+
Vulkan overhead
+
Windows/WDDM

Always leave headroom.

Large context windows are expensive

A model may advertise huge context sizes, but hardware reality matters more.

On this machine:

8K–16K

is far more useful than trying to force:

32K
64K
128K

contexts into memory.

Prompt processing also becomes increasingly expensive as conversations grow.

A smaller model can be a better default

The 30B model gives better quality on harder tasks, but around:

4.8 tokens/s

is not ideal for every interaction.

Using a smaller model for everyday work and escalating difficult requests to a larger model is a better architecture.

Hence:

fast
  ↓
Qwen3-14B

coder
  ↓
Qwen3-Coder-30B

30. Current model configuration

The final models.ini is:

version = 1

[*]
parallel = 1
jinja = true
cache-prompt = true

[fast]
model = C:/models/Qwen3-14B-Q5_K_M.gguf
ctx-size = 8192
n-gpu-layers = 99
batch-size = 128
ubatch-size = 128
no-kv-offload = true
load-on-startup = true

[coder]
model = C:/models/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
ctx-size = 8192
n-gpu-layers = 10
batch-size = 128
ubatch-size = 128
no-kv-offload = true
load-on-startup = false

And the router starts with:

C:\llama.cpp\llama-server.exe ^
  --models-preset C:\llama.cpp\models.ini ^
  --models-max 1 ^
  --sleep-idle-seconds 600 ^
  --host 0.0.0.0 ^
  --port 8080

31. Testing the server

Local health check:

curl http://127.0.0.1:8080/health

Model list:

curl http://127.0.0.1:8080/v1/models

From another machine:

curl http://SERVER-NAME:8080/health

Browser:

http://SERVER-NAME:8080

API:

http://SERVER-NAME:8080/v1

32. OpenAI-compatible Python example

from openai import OpenAI

client = OpenAI(
    base_url="http://SERVER-NAME:8080/v1",
    api_key="local"
)

response = client.chat.completions.create(
    model="fast",
    messages=[
        {
            "role": "system",
            "content": "You are an experienced software engineer."
        },
        {
            "role": "user",
            "content": "Explain the LMAX Disruptor architecture."
        }
    ]
)

print(response.choices[0].message.content)

For the larger model:

response = client.chat.completions.create(
    model="coder",
    messages=[
        {
            "role": "user",
            "content": """
            Design a bounded lock-free MPMC queue in C#.
            Discuss memory ordering, ABA issues and false sharing.
            """
        }
    ]
)

33. What I would improve next

There are several obvious next steps.

API authentication

For anything beyond a trusted local machine, add:

--api-key

and never expose port 8080 directly to the public Internet.

Tailscale

For remote access away from home, I would use:

Laptop
   │
Tailscale
   │
Home LLM server

rather than router port forwarding.

Open WebUI

llama.cpp's native UI is already surprisingly usable.

A future enhancement could be:

Open WebUI
    ↓
llama.cpp /v1

for richer:

conversation management
file handling
RAG
authentication
multiple users
persistent histories

A genuinely fast small model

I may also add a third model in the 7B–8B range:

fastest
    ↓
7B model
    ↓
quick tasks / summarization / simple coding

leaving:

14B

for normal work and:

30B

for difficult coding/reasoning.

That would effectively create three local inference tiers:

FASTEST
7B
 ↓
cheap/simple work

FAST
14B
 ↓
default

QUALITY
30B
 ↓
hard problems

34. Final result

What started as a modest Windows machine with an integrated Radeon GPU has become a completely private LLM inference server.

It now provides:

local inference;
no per-token fees;
no external inference API;
no model prompts leaving the machine;
AMD GPU acceleration through Vulkan;
multiple GGUF models;
automatic model routing;
an OpenAI-compatible REST API;
a browser-based chat interface;
LAN access;
automatic startup after reboot;
Docker running alongside it;
and unattended operation.

The most surprising part was that the hardware was substantially more capable than its specification initially suggested.

The key was not trying to force desktop-GPU assumptions onto an integrated Radeon.

Instead, the working solution used:

Vulkan
+
shared GPU memory
+
partial model offload
+
small batches
+
CPU-side KV cache
+
quantized GGUF models
+
dynamic model routing

The end result is not going to compete with a high-end NVIDIA inference server.

It does not need to.

For private coding, experimentation, agents, document processing and local application integration, it is already a remarkably capable little LLM server sitting on hardware that was available anyway.

And, perhaps most importantly:

The API is mine.
The models are mine.
The data stays local.

From 1,880th to 8th: A Six-Day, Validation-First Kaggle Sprint

noreply@blogger.com (Unknown) — Sat, 01 Aug 2026 18:49:01 +0000

From 1,880th to 8th: A Six-Day, Validation-First Kaggle Sprint

How a disciplined experiment framework, leakage-safe target statistics, class-aware decision rules, and a GPU-trained FT-Transformer produced an 8th-place private leader board finish in Playground Series S6E7.

The result

I finished Kaggle's Playground Series S6E7: Predicting Student Health Risk at 8th on the private leaderboard and 63rd on the public leaderboard, out of 3,356 teams.

Result	Rank	Score	Approximate percentile
Final private leaderboard	8 / 3,356	0.95084	Top 0.24%
Final public leaderboard	63 / 3,356	0.95306	Top 1.88%

The first repository commit was made on 26 July 2026. The final public submission landed on 31 July, less than five hours before the deadline. In roughly five and a half days, I recorded 161 completed experiments across 28 model and ensemble families and made 10 scoring Kaggle submissions.

The headline is exciting, but the most useful part of the story is the distinction between the two final scores:

The private score of 0.95084, which produced the 8th-place finish, came from the independently trained FT-Transformer pipeline. This is the result that best represents model generalization.
The public score of 0.95306 came from a late, leaderboard-guided 29-row cleanup of a public 0.95303 anchor. That artifact scored only 0.95018 privately. It was a legitimate, reproducible competition experiment, but it was not a universal model improvement and it clearly overfit the 20% public split.

That separation matters. The public leader board rewarded the final clean-up; the private leader board rewarded the model.

The competition in one page

The task was a three-class classification problem. For every test row, the model had to predict one of:

at-risk;
fit;
unhealthy.

The training set contained 690,088 rows, the test set 295,753 rows, and the model received 13 health and behaviour features: seven numeric and six categorical. Missingness was part of the problem—there were 449,496 missing training cells, with per-feature missing rates ranging from roughly 1% to 12%.

The class distribution was extremely uneven:

Class	Training rows	Share
at-risk	592,561	85.87%
unhealthy	57,724	8.36%
fit	39,803	5.77%

Kaggle evaluated submissions with balanced accuracy, the unweighted mean of the three class recalls:

BA = \frac{1}{3} ({Recall}_{at-risk} + {Recall}_{fit} + {Recall}_{unhealthy}) .

That metric changes the strategy completely. A model that predicts the 85.87% majority class for every row has high ordinary accuracy but a balanced accuracy of only one third. The model must recover the two small classes without sacrificing too much majority-class recall.

The data audit found no duplicate IDs, no exact train/test overlap, and no automatic leakage flags. Adversarial validation did, however, produce an AUC of 0.5850 in the initial audit and 0.6199 with the expanded diagnostic feature set. In other words, train and test were similar, but not interchangeable. That moderate drift was one reason not to trust tiny cross-validation gains or public-leaderboard movement on its own.

Build the laboratory before chasing the score

The project started as more than a single notebook. I built a reusable, configuration-driven framework around the competition:

append-only experiment and submission registries;
fold-aligned out-of-fold and test probabilities;
fold-local feature fitting and target encoding;
stratified cross-validation with fixed, reproducible seeds;
candidate comparison with fold wins, bootstrap intervals, per-class recall, and disagreement analysis;
submission schema, ID-order, label-domain, duplicate, and missing-value checks;
SHA-256 hashes for important notebook inputs, probability files, and submissions;
guarded upload commands, so training and submission remained separate actions;
unit, integration, lint, formatting, and type-checking gates.

This investment paid off when the experimentation became frantic. By the last two days, several long-running GPU notebooks were in flight and many candidate scores differed by less than 0.0002. Without stable folds, stored OOF probabilities, and lineage, it would have been impossible to tell a real improvement from noise.

The promotion rule for a full candidate was deliberately strict. Relative to the verified incumbent, it had to:

improve balanced accuracy by at least 0.00020;
win at least five of seven folds;
avoid reducing any class recall by more than 0.00050.

That rule rejected some attractive-looking results—including the best XGBoost/FT blend—and protected the campaign from spending submissions on fragile OOF wins.

The path from 0.92933 to 0.95054

The first scoring model was a five-fold LightGBM baseline. It established that the pipeline worked, but also exposed the central issue: the default probability argmax was optimized for ordinary accuracy, not balanced accuracy.

Milestone	Honest local OOF BA	Public score	Private score
LightGBM baseline	0.933069	0.92933	0.92979
LightGBM + cross-fitted class decisions	0.939377	0.93520	0.93597
CatStat XGBoost	0.949837	0.95006	0.95031
Two-seed XGBoost blend	0.949802	0.94972	0.95012
Exact-statistic XGBoost	0.950398	0.95054	0.95027
FT-Transformer v2	0.950549	0.95094	0.95084
Public-anchor 29-row cleanup	not a trained model	0.95306	0.95018

The biggest early gain was the decision rule

For each class, I added a learned offset to the log probability:

\hat{y} = \arg \max_{k} (\log p_{k} + b_{k}) .

The offsets were cross-fitted. When scoring fold $f$ , its offset vector was learned from all other folds, never from the labels being evaluated. A final vector was then fitted on all OOF rows and applied to test predictions.

On the LightGBM model, this increased OOF balanced accuracy from 0.933069 to 0.939377 and improved all five folds. On the stronger tree and neural models, the same idea remained the dominant post-processing lever.

The FT notebook expressed the equivalent intuition as prior correction:

\hat{y} = \arg \max_{k} \frac{p_{k}}{π_{k}^{β}},

where $π_{k}$ is the training prior for class $k$ . Setting $β$ near one asks the classifier to behave as if the classes were closer to equally likely, which aligns the decision rule with balanced accuracy.

Exact values were more informative than they looked

The next breakthrough was fold-local multiclass target statistics. The data was synthetic, but many numeric values repeated exactly. A value such as a step count or heart rate could therefore behave like a high-cardinality category.

For each selected feature value, I estimated the conditional class distribution on the training portion of a fold and transformed only the held-out portion. The tree pipeline expanded exact-value statistics across all seven numeric features and three ordinal feature interactions. The enhanced XGBoost model improved every corrected fold over its predecessor and reached 0.950398 honest OOF and 0.95054 public.

This feature family also taught an expensive lesson: do not screen a sample-size-dependent encoder only on a small sample. Exact numeric target encoding looked weak on tens of thousands of rows and became valuable at 690,088 rows, where repeated values finally had enough support.

The model that delivered 8th place: FT-Transformer v2

Most strong solutions in this competition were tree-based. The final private result came from adding a genuinely different family: an FT-Transformer trained with masamlp on Kaggle's T4 GPU.

The final v2 specification was:

Component	Setting
Outer validation	7-fold shuffled StratifiedKFold, seed 42
Raw inputs	13 competition features
Additional inputs	39 fold-local exact-value target-statistic features
Total model inputs	52
Categorical handling	learned embeddings for six categorical columns
Numeric handling	quantile scaling and `plr-lite` embeddings
Transformer	128-dimensional blocks, 2 blocks, 8 attention heads
Training	AdamW, learning rate 0.001, batch size 4,096
Schedule	cosine, 16 fixed epochs, no early stopping
In-model ensemble	4 independently initialized members per fold
Test prediction	mean probabilities across seven fold models

The target encoder was fitted inside every outer training fold. Its training-side values were themselves generated out of fold, while validation and test were only transformed. Quantile scaling, categorical vocabularies, imputation, embeddings, and model weights were also fold-local.

Fixed epochs were deliberate. Restoring the checkpoint that performed best on the same validation fold would have added roughly 0.0003 of optimistic validation score, not necessarily 0.0003 of generalizable skill.

The neural model's raw argmax OOF score was 0.891869. After honest cross-fitted decision correction, it reached 0.950549 ± 0.001740. It beat the corrected XGBoost incumbent by 0.000151 and won four of seven folds. Its raw probabilities had classwise correlations of 0.9984–0.9988 with XGBoost: similar, but different enough to create useful disagreement.

Under the predeclared local gate, that was not enough to claim a confirmed standalone improvement: the gain was below 0.00020, only four folds improved, and unhealthy recall fell by 0.000624. Near the deadline, however, the independently trained neural submission was still a rational model-family hedge. The distinction is important: the repository recorded it as a valid full model without overstating the OOF gain.

The Kaggle notebook submission scored 0.95094 public and 0.95084 private. The private score was the best of the campaign and placed the team 8th on the final leaderboard.

Why the notebook infrastructure mattered

The local full FT run had to be stopped after more than six hours. The RTX 2060 was pinned at full utilization and reached 91°C, while other experiments still needed the machine. Smaller local FT screens also underestimated the exact-value feature family because they lacked enough rows.

The practical solution was to move the full neural confirmation to a private Kaggle Notebook, save raw OOF and test probabilities, download them, verify their hashes and row alignment, and import them into the local experiment registry. This let the T4 do the expensive training while the repository remained the source of truth for honest cross-fitting, blending, comparison, and lineage.

The completed v2 run trained 28 neural networks in total: four ensemble members for each of seven folds. Under the deadline, the notebook was not just a convenient UI; it was the only realistic way to complete the full-scale neural experiment.

The blend that almost passed

The FT model was introduced partly for diversity. I evaluated a predeclared grid of XGBoost/FT probability blends, correcting every blend with fold-external class decisions.

The best blend used 75% XGBoost and 25% FT. It reached 0.950595, improved six of seven folds, and had a paired bootstrap interval of +0.000035 to +0.000357 versus the incumbent.

It still failed the promotion rule:

the exact gain was +0.000197, missing the +0.00020 threshold by about three millionths;
unhealthy recall fell by 0.001351, well beyond the 0.00050 guardrail.

That is the kind of candidate the public leaderboard tempts you to submit. The framework rejected it. The final private result vindicated the value of having a standalone neural submission without claiming that every small OOF blend gain was real.

What did not work

The negative results were as useful as the winners because they narrowed the search under severe time and compute constraints.

Experiment family	Best relevant result	Decision
Native CatBoost screen	0.948367	Below matched XGBoost
Hand-built health-range features	−0.000565 vs screen control	Rejected
Independent exact-stat XGBoost seed	0.950362	Confirmed family, not a gain
Missing-count features, full run	0.950293	Screen gain did not transfer
Structural XGBoost trial	0.950452	Only +0.000054; recall guards failed
RealMLP-TD screen	0.948076	Weaker and insufficiently diverse
Class weighting, best corrected screen	+0.000082 vs control	Below promotion threshold
Richer temperature scaling	temperatures converged to 1.0	No value beyond offsets
Two newly probed interactions, full run	0.950303	Two fold wins; unhealthy recall failed

The common pattern was important: many ideas improved raw argmax accuracy or a three-fold screen, then disappeared after class-aware correction or full seven-fold confirmation. Balanced accuracy was won at the interaction between probabilities and decisions, not by maximizing a convenient proxy.

How the 0.95306 public score was produced

The final public submission was a separate, late competition experiment. It began from the public V98 anchor, which scored 0.95303, and applied an explicit ledger of 29 row-level label corrections recovered through controlled small-block ablations.

Transition	Rows
`unhealthy` → `at-risk`	20
`fit` → `at-risk`	5
`at-risk` → `unhealthy`	4
Total	29

The notebook checked the anchor's exact SHA-256, schema, row count, row order, label domain, and every expected source label before changing a row. It then round-trip validated the CSV and asserted the final output hash:

4171D46147B36CEA16AF087FED7BE36BFD9B4BD48098B3D5D20A129123AFBB81

That tiny patch raised the public score from 0.95303 to 0.95306, producing public rank 63. Its private score was 0.95018, substantially below the FT model's 0.95084. The result is therefore best understood as a reproducible public-leader board clean-up, not evidence of a better classifier.

I am including this detail because a useful competition retrospective should explain both what worked and what merely scored. The 29-row artifact explains the latest public score; the FT-Transformer explains the private finish.

What I would carry into the next competition

Optimize the competition metric at the decision layer. On an imbalanced multiclass task, probability argmax can leave a huge amount of balanced accuracy unused.
Cross-fit every learned post-processing step. Offsets, thresholds, target statistics, calibration, and ensemble weights can all leak if fitted and scored on the same OOF labels.
Test scale-dependent features at the scale where they should work. Exact-value statistics were a false negative on small samples and a real gain on the full data.
Use a different model family for real diversity. A second tree seed barely helped; the FT-Transformer created different errors and delivered the best private score.
Keep raw probabilities, not just labels. Stored OOF and test probabilities made later calibration, disagreement analysis, and blending possible without retraining.
Predeclare promotion gates. A +0.000197 blend can look irresistible until its class-recall regression is measured.
Treat public leaderboard engineering as a separate layer. It may be a valid competition tactic, but it should never be confused with private generalization.
Use hosted notebooks strategically. Under a six-day deadline, Kaggle's T4 was essential for experiments that could not be completed safely on local hardware.

Final reflection

Moving from a 0.92933 first submission and public rank around 1,880 to 8th private and 63rd public was not the result of one magic hyperparameter. It came from a sequence of compounding decisions:

audit the data;
align validation with balanced accuracy;
fit target statistics inside folds;
preserve OOF probabilities;
add class-aware decisions;
demand repeated evidence;
use Kaggle Notebooks when local compute became the bottleneck;
record negative results instead of repeating them;
separate a generalized model gain from public leaderboard post-processing.

The most satisfying number is not 0.95306. It is 8th on the private leaderboard. That is the result produced by the modelling pipeline when the hidden 80% of the test labels finally decided the competition.

Reproducibility and links

All local validation figures in this post come from aligned stored OOF predictions. Final public/private submission scores and leader board positions were reconciled against the authenticated Kaggle API after the competition closed on 31 July 2026.

Building a GPT-Style Language Model From Scratch, One Understandable Step at a Time

noreply@blogger.com (Unknown) — Thu, 23 Jul 2026 21:31:06 +0000

Large language models can feel mysterious because we usually meet them only after they have been trained on enormous datasets using equally enormous computers. FrankenGPT takes the opposite approach: make the model small enough to run locally, keep the implementation readable, and expose the complete path from a text file to generated text.

The result is not a replacement for ChatGPT, and that is not the goal. It is a compact, working GPT-style language model that lets us inspect the same essential ideas: tokenization, causal self-attention, next-token prediction, gradient-based optimization, validation, checkpoints, and autoregressive generation.

This article walks through that path without assuming that you already speak fluent Transformer.

Repository: JordiCorbilla/llm-from-scratch

What does "from scratch" mean here?

The phrase is important. In FrankenGPT's scratch workflow:

no pretrained model weights are downloaded;
no pretrained tokenizer is reused;
the vocabulary is built from the text we provide;
every model parameter starts with a random value; and
every saved checkpoint is the result of our own training run.

The repository also contains an optional Hugging Face fine-tuning example, but that is a separate experiment and is clearly labelled as pretrained. Everything in this article uses the scratch model.

It is also worth being precise about the word large. FrankenGPT follows the architecture and training objective used by GPT-like language models, but our experiments range from roughly 90,000 to 2.9 million parameters. That is deliberately small by modern LLM standards. Small models are much easier to understand, test, and train on an ordinary computer.

The whole journey in one view

The model's job can be summarized as a pipeline:

corpus text
  -> tokenizer and vocabulary
  -> shifted input/target sequences
  -> token and position embeddings
  -> causal Transformer blocks
  -> next-token scores
  -> cross-entropy loss
  -> AdamW parameter updates
  -> checkpoint
  -> one-token-at-a-time generation

Training repeatedly moves from left to right until the model becomes better at predicting the next token. Generation then reuses the trained model one prediction at a time.

Step 1: Create a clean environment

FrankenGPT requires Python 3.10 or newer. A virtual environment keeps this experiment separate from other Python projects:

git clone https://github.com/JordiCorbilla/llm-from-scratch.git
cd llm-from-scratch

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

On macOS or Linux, activate the environment with:

source .venv/bin/activate

The editable installation gives us the frankengpt command while keeping the source code available to inspect and change.

Confirming GPU support

CPU training works everywhere and is useful for the first smoke test. An NVIDIA GPU makes the larger experiments much faster, but PyTorch must have been installed with CUDA support:

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

A CUDA-enabled installation prints a version containing +cu... and then True. If it prints False, use the official PyTorch installation selector to choose the wheel that matches the machine.

FrankenGPT accepts --device auto, --device cpu, or --device cuda. On CUDA it automatically uses mixed-precision training: some operations use lower-precision numbers to improve GPU throughput and reduce memory use, while gradient scaling protects the optimization process from numerical underflow.

Step 2: Choose the training text

A language model learns statistical patterns from its corpus. It does not receive a grammar book or a database of facts; it only sees sequences of tokens and tries to predict what follows.

For the first experiment, FrankenGPT can download Mary Shelley's Frankenstein from Project Gutenberg:

frankengpt train --download --data data/pg84.txt ...

The loader removes the Project Gutenberg catalogue header and licence footer before training. It then makes a contiguous split:

approximately 90% of the tokens are used for training;
approximately 10% are held out for validation.

Those partitions do not overlap. The training portion updates the weights; the validation portion measures whether the learned patterns also work on unseen text.

For a broader experiment, the project can download several public-domain classics:

frankengpt fetch-data --output-dir data/classics

More varied data matters. A model trained on one novel mostly learns the vocabulary, punctuation, characters, and style of that novel. A multi-book corpus exposes it to more sentence structures and word combinations.

Step 3: Turn text into tokens

Neural networks operate on numbers, not strings. A tokenizer divides text into units and assigns an integer ID to each unit.

FrankenGPT includes two intentionally simple tokenizers:

Tokenizer	Example units	Trade-off
Character	`["T", "h", "e"]`	Small vocabulary and no unknown words, but long sequences
Word	`["The", "creature", ","]`	More readable output and shorter sequences, but a larger vocabulary

Both vocabularies are built only from the selected corpus. For example:

text:       The creature
tokens:     ["The", "creature"]
token IDs:  [418, 937]

The IDs themselves have no linguistic meaning. They are indexes into a learned embedding table. During training, the vector at each index gradually becomes useful for predicting tokens that tend to appear nearby.

Step 4: Create the question and the answer

The training objective is next-token prediction. Given a sequence of token IDs, the target is the same sequence shifted by one position:

token stream:  [12, 31, 7, 44, 9]
input:         [12, 31, 7, 44]
target:        [31, 7, 44, 9]

This produces several predictions in one forward pass:

after 12, predict 31;
after 12, 31, predict 7;
after 12, 31, 7, predict 44; and
after 12, 31, 7, 44, predict 9.

With batch size B and context length T, the input and target tensors both have shape [B, T].

Step 5: Pass the tokens through a GPT-style decoder

The model first combines two learned representations:

a token embedding, which represents the current token;
a position embedding, which represents where that token appears in the sequence.

The combined vectors pass through a stack of Transformer decoder blocks. Each block contains pre-layer normalization, multi-head causal self-attention, a GELU feed-forward network, dropout, and residual connections.

Causal self-attention, gently explained

Self-attention lets each position decide which earlier positions are useful. Internally, the model projects each token representation into queries, keys, and values:

queries, keys, values  [B, heads, T, head dimension]
attention scores       [B, heads, T, T]

The core operation is:

attention(Q, K, V) = softmax((Q K^T / sqrt(d_k)) + causal_mask) V

The notation is more compact than the idea. Q K^T measures which tokens are relevant to one another, the scale factor keeps the numbers stable, the mask removes future tokens, softmax turns the remaining scores into proportions, and those proportions select a mixture of the value vectors.

The causal mask blocks every future position. When predicting token 5, the model may use tokens 0 through 4, but it cannot peek at token 5's answer. Without this restriction, training loss would look excellent while generation would fail: the model would have learned to copy information that is unavailable when writing new text.

Residual connections preserve a direct path through the network, while the feed-forward layers transform each position independently. Repeating the block lets the model build progressively richer contextual representations.

Finally, a linear language-model head produces one score, or logit, for every item in the vocabulary:

hidden states  [B, T, model width]
logits         [B, T, vocabulary size]

FrankenGPT ties the output head to the token embedding table. The same learned token representations therefore support both reading input and scoring output, while reducing the parameter count.

Step 6: Measure the error and update the model

Cross-entropy loss compares the logits with the correct shifted targets. Lower loss means that the model assigned more probability to the actual next tokens.

One training update performs:

forward pass
  -> cross-entropy loss
  -> backpropagation
  -> gradient clipping
  -> AdamW update
  -> learning-rate schedule

The optimizer is AdamW. The learning rate warms up gradually and then follows a cosine decay. Gradient clipping limits unusually large updates, and weight decay discourages parameters from growing without bound.

There are two losses to watch:

training loss is measured on text used to update the model;
validation loss is measured on held-out text that never updates the model.

We want both to decrease. If training loss continues downward while validation loss begins to rise, the model is memorizing the training corpus more than it is learning patterns that transfer. That is overfitting.

Step 7: Prove the complete pipeline with a smoke run

Before spending time on a long training run, it is sensible to test every stage with a small CPU model:

frankengpt train --download --data data/pg84.txt --tokenizer word --max-vocab 2048 `
  --device cpu --output runs/readme-smoke --max-steps 10 --batch-size 8 `
  --context-length 16 --d-model 32 --n-heads 4 --n-layers 2 --eval-interval 5

This model has 91,520 parameters. In a verified run, training loss moved from 7.6218 to 7.5856, while validation loss moved from 7.6328 to 7.6039.

That is a small improvement, as expected after only ten updates. At this stage, success means that:

data was loaded and tokenized;
forward and backward passes completed;
both losses were recorded;
checkpoints were saved; and
the checkpoint could be loaded for generation.

The generated text is not supposed to be fluent yet:

The earth interested does desires enemy affectionate desires disappeared figure figure
figure desires distinct noble.. the familiar valley valley safety disappeared disappeared
the

This output is useful precisely because it sets an honest baseline. Ten steps validate the machinery; they do not teach a language model to write.

Step 8: Train the larger model on a GPU

The more interesting run used the word tokenizer and a collection of public-domain classics:

frankengpt fetch-data --output-dir data/classics

frankengpt train --data data/classics/*.txt --tokenizer word --max-vocab 16384 `
  --device cuda --output runs/classics-word-v2 --max-steps 2000 `
  --batch-size 32 --context-length 64 --d-model 128 --n-heads 4 `
  --n-layers 4 --eval-interval 200

Its configuration was:

Setting	Value
Parameters	2,898,688
Vocabulary	16,384 word and punctuation tokens
Context length	64 tokens
Transformer blocks	4
Attention heads	4
Model width	128
Batch size	32
Training steps	2,000
Device	CUDA

On the test machine, an NVIDIA GeForce RTX 2060 ran the model through CUDA mixed precision. The recorded training segment processed approximately 44,868 tokens per second. Throughput is hardware- and software-dependent, so the loss trend is the more portable result.

Step 9: Save checkpoints and keep the best model

At each evaluation interval, FrankenGPT writes:

checkpoint_last.pt, containing the newest state;
checkpoint_best.pt, containing the lowest validation loss seen so far; and
metrics.json, containing throughput and loss history.

The checkpoint includes the model configuration, tokenizer vocabulary, model parameters, optimizer, learning-rate scheduler, training step, and history. That makes it possible to resume training rather than restart:

frankengpt train --data data/classics/*.txt --tokenizer word --max-vocab 16384 `
  --device cuda --output runs/classics-word-v2 `
  --resume runs/classics-word-v2/checkpoint_last.pt `
  --max-steps 3000 --batch-size 32 --context-length 64 `
  --d-model 128 --n-heads 4 --n-layers 4 --eval-interval 200

Model shape and tokenizer settings must remain unchanged when resuming. Only the training budget and other compatible operational settings should change.

Load PyTorch .pt checkpoints only from trusted sources because they can contain pickled Python data.

What did the model learn?

The multi-book model's measured losses were:

Step	Training loss	Validation loss
100	6.6836	6.7742
1,000	5.1567	5.4137
2,000	5.0342	5.3361

Both values fell substantially, and validation loss was still improving at step 2,000. That is evidence that the model learned patterns that also applied to the held-out text.

Loss values from the word model should not be compared directly with the character model's losses. Their vocabularies and prediction problems are different: choosing among 16,384 words is not the same task as choosing among 93 characters. Loss trends within one run are what matter.

The eventual output

Generation begins with a prompt and repeatedly samples one next token:

frankengpt generate --checkpoint runs/classics-word-v2/checkpoint_best.pt `
  --prompt "I had worked hard" --max-new-tokens 80 `
  --temperature 0.7 --top-k 20 --device cuda

A fresh sample from the trained checkpoint produced:

I had worked hard, and all the house of a moment of my own friend, and that I am, with a
great. But I shall not not be no time, you shall have been no, it is all the door, my own,
to be so. What may not know. I see that I can do you will be a man, I cannot know you have
been as I can be so that my

The difference from the ten-step smoke output is visible. The larger model learned:

plausible English punctuation;
common phrase and clause structures;
agreement between many neighbouring words; and
a rough literary rhythm from the source corpus.

It also repeats itself, contradicts itself, and loses the thread. That is the expected boundary of this experiment, not a hidden failure. A 2.9-million-parameter model trained for 2,000 updates on a handful of books has learned local language patterns, not robust reasoning, factual knowledge, or long-range narrative planning.

Sampling settings change the style but cannot create knowledge the model did not learn. A lower temperature and smaller top-k make output safer and more repetitive. Higher values add variety along with more mistakes.

Step 10: Verify and benchmark the result

The repository includes automated checks for tokenization, shifted targets, causal masking, model shapes, loss handling, checkpoint loading, resumption, compilation fallbacks, and generation:

python -m ruff check .
python -m pytest

Benchmark a saved model separately:

frankengpt benchmark --checkpoint runs/classics-word-v2/checkpoint_best.pt --device cuda

Forward-pass throughput and autoregressive generation throughput are different measurements. A forward pass predicts many positions in parallel; generation must append one token and run the model again before it can choose the next.

What would improve it next?

The project is intentionally a foundation rather than an endpoint. The highest-impact next experiments are:

Use more varied text. Data diversity improves vocabulary and generalization.
Add a subword tokenizer. Subwords balance the long sequences of characters against the very large vocabulary and unknown words of a simple word tokenizer.
Increase context length. The model can then condition on more history, at a higher memory and compute cost.
Scale width and depth carefully. More parameters increase capacity only when the dataset and training budget grow with them.
Track validation first. More training is useful until held-out performance stops improving.
Evaluate multiple prompts and seeds. One attractive sample is not a complete model evaluation.

Closing thought

The most valuable output of FrankenGPT is not a perfect paragraph. It is the removal of the black box.

We can point to the corpus, vocabulary, shifted targets, embeddings, masked attention, loss, gradients, optimizer, checkpoint, and sampling loop. We can see the validation loss fall, resume the exact training state, move the computation to a GPU, and inspect what the model learned—and what it did not.

That is the real benefit of building a language model from scratch: every generated token has a path we can follow.

RiskOptima in 2026: Turning a Quant Toolkit into a Five-Project Portfolio Platform

noreply@blogger.com (Unknown) — Sat, 27 Jun 2026 17:43:19 +0000

In April 2025, I wrote about the RiskOptima Options Analysis Suite: a practical options toolkit for implied volatility screening, Greeks simulation, and straddle backtesting.

That version had a clear purpose: give traders and quant developers a compact way to look at options behaviour, volatility, and earnings-event risk using Python. It was deliberately hands-on: fetch market data, generate charts, inspect Greeks, and test a simple volatility trade.

One year later, RiskOptima has grown into something broader.

It is no longer just an options analysis script. It is becoming a structured quantitative finance toolkit that can demonstrate five canonical quant portfolio projects in one repository:

Caption: RiskOptima now includes dashboard-ready analytics, pricing engines, backtesting utilities, optimization workflows, and credit risk modelling.

Algorithmic trading and backtesting
Portfolio optimization
Market risk dashboarding
Option pricing and Greeks
Credit risk modelling

That matters because a quant portfolio should not just show isolated notebooks. It should show that you can build reusable models, package them cleanly, test them, and explain how they fit into real investment and risk workflows.

From Options Suite to Quant Risk Platform

The original options post focused on three practical workflows:

Implied volatility screening
Greeks simulation
Earnings straddle backtesting

Those pieces are still valuable, but they now sit inside a wider architecture.

RiskOptima now includes modules for:

Portfolio analytics
Mean-variance optimization
Factor risk modelling
Backtesting with transaction costs
VaR and CVaR
Monte Carlo simulation
Fixed-income analytics
Options pricing
Credit risk modelling
Dashboard-ready market risk reporting

The main shift is architectural. Instead of treating each notebook as a standalone experiment, RiskOptima now exposes cleaner package APIs that can be imported, tested, reused, and extended.

That is the difference between a tutorial project and a library.

Project 1: Algorithmic Trading Backtester

The backtesting layer provides strategy interfaces, portfolio state tracking, transaction cost modelling, and daily rebalancing workflows.

Caption: A clean backtesting layer should produce an inspectable equity curve, weights through time, and trade/risk diagnostics.

The goal is not to compete with institutional backtesting engines. The goal is to show the mechanics clearly:

Generate target weights
Rebalance a portfolio
Apply spread and impact costs
Track equity curves
Inspect weights through time

For example, a simple SMA crossover strategy can be expressed as a reusable strategy object and passed into the backtest engine.

from riskoptima import BacktestConfig, SMACrossStrategy, SimpleCostModel, run_backtest

strategy = SMACrossStrategy(short_window=20, long_window=50)
config = BacktestConfig(initial_cash=1_000_000, rebalance_rule="D")
cost_model = SimpleCostModel(spread_bps=2.0, impact_coeff=0.0)

equity_curve, weights = run_backtest(
    prices=prices,
    strategy=strategy,
    config=config,
    cost_model=cost_model,
)

The important portfolio signal here is not that an SMA strategy is sophisticated. It is that the repo demonstrates the plumbing required to evaluate systematic strategies properly.

The SMA code has also been moved into the library itself instead of living only inside the notebook. That makes the notebook easier to read and gives users reusable functions for building signals, generating trade logs, and plotting entries and exits.

Project 2: Portfolio Optimization

RiskOptima already had strong portfolio optimization foundations: efficient frontiers, Monte Carlo portfolio generation, Sharpe maximization, minimum variance portfolios, and allocation visualizations.

Caption: Portfolio optimization is where expected return, volatility, constraints, and risk-adjusted objectives come together.

The newer modular optimizer layer makes that cleaner:

from riskoptima import Constraints, optimize_max_sharpe

constraints = Constraints(
    weight_bounds=(0.0, 0.30),
    factor_bounds={"MKT": (-0.2, 0.8)},
)

weights = optimize_max_sharpe(
    expected_returns=expected_returns,
    cov=covariance_matrix,
    constraints=constraints,
    factor_exposures=factor_exposures,
    risk_free_rate=0.02,
)

This gives the project a more realistic optimization story:

Long-only or bounded portfolios
Factor exposure controls
Risk-adjusted objective functions
Reusable optimizer APIs

For a quant portfolio, this is one of the most important sections. It shows that you understand portfolio construction, not just individual signal generation.

Project 3: Market Risk Dashboard

One of the biggest additions is the dashboard-ready market risk report layer.

Caption: The market risk layer turns raw returns into dashboard metrics such as VaR, CVaR, drawdown, rolling volatility, and benchmark-relative risk.

The new reporting module produces a structured risk report with:

Annualized return
Annualized volatility
Sharpe ratio
Sortino ratio
Maximum drawdown
Historical VaR
Parametric Gaussian VaR
CVaR / expected shortfall
Beta versus benchmark
Tracking error
Information ratio
Rolling volatility
Rolling drawdown

Example:

from riskoptima.reporting import build_market_risk_report

report = build_market_risk_report(
    returns=asset_returns,
    weights=portfolio_weights,
    benchmark_returns=benchmark_returns,
    confidence_levels=(0.95, 0.99),
)

print(report.metrics["annualized_volatility"])
print(report.metrics["historical_var"][0.99])
print(report.metrics["cvar"][0.99])

There are also plotting helpers for:

Drawdown curves
Rolling volatility
VaR / CVaR loss distributions
Correlation heatmaps

This turns RiskOptima into something that can support an actual risk dashboard rather than just isolated calculations.

Project 4: Option Pricing Engine

The original options suite has now been refactored into a cleaner option pricing engine.

Caption: The options engine separates pricing logic from notebook exploration, making Black-Scholes, Greeks, implied volatility, binomial trees, and Monte Carlo pricing reusable.

The new riskoptima.options package exposes:

black_scholes_price
black_scholes_greeks
implied_volatility
binomial_tree_price
monte_carlo_european_option

Example:

from riskoptima.options import (
    black_scholes_price,
    black_scholes_greeks,
    implied_volatility,
    monte_carlo_european_option,
)

S, K, T, r, sigma = 100, 100, 1.0, 0.05, 0.20

call = black_scholes_price(S, K, T, r, sigma, option_type="call")
put = black_scholes_price(S, K, T, r, sigma, option_type="put")
greeks = black_scholes_greeks(S, K, T, r, sigma, option_type="call")
iv = implied_volatility(call, S, K, T, r, option_type="call")
mc = monte_carlo_european_option(S, K, T, r, sigma, option_type="call", random_state=42)

This is a better portfolio story than the original single-file approach because it separates pricing models from analysis notebooks.

The legacy RiskOptima class methods are still preserved for backward compatibility, but the cleaner package-level API is now the preferred interface.

Project 5: Credit Risk Model

The newest addition is the credit risk module.

Caption: The credit module adds a different risk lens: PD, LGD, EAD, expected loss, rating migration, Merton default probability, and simulated loss tails.

This matters because credit risk is a different modelling domain from market risk and options. It shows that RiskOptima can cover more than asset returns and volatility.

The new riskoptima.credit package includes:

Logistic PD model helpers
PD prediction
Expected loss
Portfolio expected loss
Unexpected loss approximation
Transition matrix validation
Rating migration simulation
Merton distance to default
Merton default probability
Credit loss Monte Carlo
Credit VaR
Credit CVaR

Example:

import pandas as pd
from riskoptima.credit import (
    portfolio_expected_loss,
    simulate_credit_losses,
    credit_var,
    credit_cvar,
    merton_pd,
)

portfolio = pd.DataFrame({
    "obligor": ["A", "B", "C"],
    "PD": [0.01, 0.025, 0.04],
    "LGD": [0.40, 0.45, 0.55],
    "EAD": [1_000_000, 750_000, 500_000],
})

expected_loss = portfolio_expected_loss(portfolio)
losses = simulate_credit_losses(portfolio, n_sims=20_000, random_state=42)

var_99 = credit_var(losses, confidence=0.99)
cvar_99 = credit_cvar(losses, confidence=0.99)
merton_default_probability = merton_pd(
    asset_value=150,
    debt_face_value=100,
    asset_vol=0.25,
    risk_free_rate=0.03,
    maturity=1.0,
)

This is the sort of feature that helps the project read like a complete risk platform rather than a collection of market-data notebooks.

The Notebook Layer Still Matters

RiskOptima still includes notebooks because notebooks are useful for storytelling.

The current notebook set covers:

Bond analytics
Portfolio optimization
Index / volatility divergence signals
Stochastic volatility models
SMA strategy backtesting
Options analytics
Core modular features
Credit risk modelling

The important change is that notebooks are now demonstrations of package functionality, not the only place where the functionality exists.

That makes the project easier to test and easier to maintain.

Caption: The notebooks are now the storytelling layer. The reusable code lives in the package modules.

Testing and Reliability

Another important update is test coverage.

RiskOptima now includes tests around:

Backtesting engine behaviour
Optimizer constraints
Factor risk model covariance
Credit risk calculations
Market risk reporting
Option pricing accuracy
Public API imports
Core dataclasses
Legacy analytics
Empty-trade SMA edge cases

The project also validates all notebooks with execution checks. That is useful because quant notebooks often break quietly when market data changes, dependencies move, or a strategy produces no trades.

In the latest pass, all eight notebooks executed successfully.

What This Project Demonstrates

RiskOptima is designed to show practical quant engineering skill across several layers:

Financial modelling
Python package design
Numerical methods
Data analysis
Risk reporting
Backtesting
Testing
Notebook-based communication

That combination matters.

A recruiter or interviewer does not just want to see that you can calculate a Sharpe ratio. They want to see that you can build something coherent, test it, explain it, and extend it.

RiskOptima now tells that story much better than it did a year ago.

What I Would Add Next

The next highest-ROI improvements are not just more models. They are product and engineering polish:

Documentation site A small MkDocs or Sphinx site with API examples would make the project easier to navigate.
Streamlit dashboard The market risk report layer is already dashboard-ready. A Streamlit app would make it visual and recruiter-friendly.
CI notebook smoke tests Run unit tests on every push and execute a lightweight subset of notebooks on scheduled builds.
Example datasets Add small synthetic datasets so every demo can run without live market data.
Model validation pages For options, credit risk, and VaR, include known-value tests and methodology notes.
Portfolio project landing page A single README section that maps each canonical quant project to notebooks, APIs, and tests.

Final Thoughts

The original RiskOptima Options Analysis Suite was useful because it solved a focused problem: options analysis with practical charts and workflows.

The updated RiskOptima is more ambitious.

It is a quant finance toolkit that demonstrates how portfolio optimization, market risk, options analytics, backtesting, and credit risk can live together in one coherent Python project.

That is the direction I want the project to keep moving in: not just more notebooks, but a stronger bridge between research, engineering, and real-world risk analysis.

Building a Local MiFID Regulatory Copilot

noreply@blogger.com (Unknown) — Sat, 13 Jun 2026 12:32:31 +0000

I built this project for fun as a local-first regulatory copilot for MiFID II, MiFIR, UK MiFIR, transaction reporting, transparency, ARM/APA workflows, and source-backed analysis. The goal was not to build a generic chatbot. The goal was to build a reviewable regulatory workbench: a tool that starts from official documents, shows the evidence it used, and fails conservatively when the corpus is incomplete.

The app now runs as a FastAPI service with a browser UI, local Ollama generation, a manifest-driven regulatory corpus, hybrid retrieval, evidence scoring, citations, and a corpus explorer. It is designed to be deployed on a private server and exposed through Cloudflare Tunnel, while keeping the source documents and indexes under local control.

Why Local First

Regulatory analysis has two constraints that are easy to underestimate.

First, the answer must be traceable. If an assistant says a transaction is reportable, or that a transparency deferral may apply, a reviewer needs to see the exact document, page, article, and passage behind that claim.

Second, the source set must be controlled. MiFID and MiFIR materials exist across Level 1 legislation, RTS, ESMA Q&A, reporting instructions, XML schemas, FCA pages, FCA Handbook material, discussion papers, consultation papers, and policy statements. A useful assistant needs to know which sources are current, which are historical, which are proposals, and which jurisdiction they belong to.

Those constraints pushed the project toward retrieval-augmented generation instead of fine-tuning. Fine-tuning can help with style or workflow patterns, but it should not be the source of truth for current law. The law and guidance should live in an auditable corpus.

Architecture

The architecture has five layers:

Source manifest
Ingestion and chunking
Keyword and vector indexing
Route-aware retrieval and answer generation
UI, evidence quality, and corpus inspection

The source manifest is the control plane. Each entry records the source ID, title, publisher, jurisdiction, regime, document type, domain tags, priority, status, official URL, and expected local path. The app does not silently scrape the internet at query time. Documents are downloaded into data/raw, then parsed, chunked, and indexed.

FastAPI serves the query endpoints and the static UI. Ollama provides local generation. The deployment model binds the API to 127.0.0.1; Cloudflare Tunnel can expose it through HTTPS without opening an inbound firewall port. HTTP Basic authentication is loaded from an ignored local config/auth.yaml, and a public showcase should additionally use Cloudflare Access.

Corpus Chunk Methodology

The most important implementation detail is chunking. Early versions of the corpus produced low-value chunks such as cover pages, titles, and fragments that looked meaningful in the UI but did not carry legal substance. That created bad retrieval: if the model receives weak context, it will either answer weakly or over-interpret the wrong passage.

The current chunker is legal-boundary aware. It looks for headings such as:

Article 26
Recital (12)
CHAPTER IV
SECTION 2
ANNEX I
Question 3
Table 2

When a boundary is found, the chunker keeps the surrounding legal unit together where possible. It then enforces a size window from config/retrieval.yaml: chunks should be large enough to carry meaning, but small enough to fit into retrieval and prompt context. Oversized blocks are split by paragraph. Undersized blocks are merged with nearby legal blocks until they become useful.

Each chunk carries metadata:

source_id
document title and publisher
jurisdiction, regime, and domain tags
document type, priority, and status
page number
detected article, section, annex, question, or table
original extracted text

The chunker also filters low-value front matter. Short policy-statement covers, tables of contents, and similar non-substantive fragments are excluded unless they contain strong legal signals. This matters because the corpus explorer is not just a debug tool; it shows exactly what the answer engine can see.

Retrieval And Accuracy Controls

Retrieval is hybrid. The app builds:

a SQLite FTS5 keyword index for exact legal terms such as Article 26, RTS 22, FIRDS, deferral, or APA
a local vector index for semantic matching

The hybrid retriever merges both result sets, applies route filters, and adds ranking boosts for source priority, active status, consolidated source material, and domain-specific intent.

The router classifies each question by:

jurisdiction: EU or UK
regime: MiFIR, MiFID II, or UK MiFIR
domain: transaction reporting, transparency, reference data, algorithmic trading, order records, or overview

That route is important. A UK transparency question should not accidentally retrieve EU RTS material unless the user asks for a comparison. A MiFID overview should prefer Level 1 Directive scope material instead of a narrow RTS. A transaction reportability question should route to MiFIR Article 26 and RTS 22, even if a user casually says "under MiFID II".

That last example became a useful design test. A question like "What information is needed to determine transaction reportability under MiFID II?" was initially pulling in RTS 25 clock synchronisation because RTS 25 is MiFID II material and includes transaction-adjacent wording. The fix was not to hard-code one answer. The fix was to improve routing and retrieval:

transaction reportability now routes to MiFIR Article 26 / RTS 22
reportability retrieval seeds canonical chunks from MiFIR Article 26, RTS 22 Article 1, RTS 22 Article 2, and RTS 22 Table 2
RTS 25 clock synchronisation is penalised for reportability questions unless the user asks about clocks, timestamps, or RTS 25
a regression test locks this behaviour

This is the general pattern for accuracy: make the route explicit, prefer canonical sources, show citations, and add tests for retrieval failures.

How An Answer Is Derived

When a user asks a question, the app does the following:

Classifies the question into jurisdiction, regime, and domain.
Expands the retrieval query with domain-specific terms.
Retrieves source chunks using keyword search, vector search, metadata filters, and seeded canonical evidence.
Builds a prompt containing only the retrieved context and route assumptions.
Calls the local Ollama model (using gemma3n).
Converts the generated answer into a deterministic report format.
Builds citations directly from the retrieved chunks.
Scores the evidence quality and shows warnings.

The deterministic formatter is deliberate. LLMs are useful for summarising and connecting evidence, but the outer structure should be stable. The app controls the headings, assessment status, citation list, required data fields, missing facts, systems impact, validation rules, and confidence statement. This avoids UI-breaking formatting drift and keeps reviewer-facing output predictable.

Evidence Quality

Every answer gets an evidence-quality report. The score considers:

number of retrieved sources
number of retrieved chunks
source priority
document type
source status
jurisdiction match
regime match
domain match

The UI treats this as a gate, not decoration. Low means the answer should not be relied on. Medium means draft use only. High means the evidence base is strong enough for analyst review, still subject to compliance sign-off.

Citations show the source ID, title, document type, page, article or section when available, and a short source preview. Retrieved chunks are also listed so a reviewer can inspect what the model saw.

The Corpus Explorer

The corpus explorer makes the knowledge base inspectable. It answers questions like:

Which official files are loaded?
Which sources are missing?
How many chunks were produced per source?
What exact text is available to retrieval?
Which article, page, or table did a chunk come from?

This is essential for trust. When an answer looks wrong, the first question should not be "why did the model say that?" It should be "what evidence did retrieval give the model?" The corpus explorer gives that answer directly.

Deployment Model

The deployment shape is intentionally small:

Clone the repository on the server. (Source code is still private, if you want to see it, reach out)
Create config/auth.yaml locally and keep it out of git.
Add the official source files under the paths listed in config/source_manifest.yaml.
Run manifest validation.
Build the corpus.
Build the indexes.
Start FastAPI bound to 127.0.0.1.
Expose it through Cloudflare Tunnel.

The current helper script can start the API and quick Cloudflare tunnel for demos. For a stable external setup, use a named Cloudflare Tunnel, a stable hostname, Cloudflare Access, log rotation, dependency scanning, rate limits, and a regular corpus refresh process.

Testing Strategy

The test suite covers the mechanics that matter for this kind of app:

source manifest validation
chunk metadata extraction
retrieval filtering
route classification
overview retrieval for MiFID and MiFIR
transaction reportability routing to MiFIR / RTS 22
evidence scoring
acceptance questions with expected source IDs and expected terms

The acceptance file is intentionally simple: each question defines the expected jurisdiction, regime, domain, minimum evidence level, expected sources, and expected terms. This provides a repeatable sanity check without requiring the LLM to produce byte-identical prose.

For a regulatory copilot, this is the right kind of test. The generated wording can vary, but the evidence path should be stable.

What This Enables

The result is a local regulatory copilot that can:

answer MiFID / MiFIR reporting questions from local official sources
separate EU and UK material
cite source chunks
flag weak evidence
expose the corpus for review
support controlled external demos through Cloudflare
grow by adding new manifest sources and domain-specific retrieval seeds

The app is usable today as a showcase and analyst draft tool. It is not a replacement for legal or compliance sign-off. The value is that it makes the evidence path visible and testable, which is exactly what a regulatory assistant needs before it can become operationally useful.

Next Extensions

The strongest next improvements would be:

replace fallback hash embeddings with a production embedding model
add a reranker for better long-document retrieval
improve table extraction for RTS field tables and FCA policy statements
add source freshness checks and supersession metadata
add Cloudflare Access and request rate limiting for public demos
add structured audit logs for questions, routes, retrieved chunks, and evidence scores
build workflow-specific dashboards for transaction reporting, transparency, ARMs, APAs, and exception management

The architecture is intentionally extensible: new source types enter through the manifest, new documents become chunks, chunks become searchable evidence, and route-specific retrieval rules can be added without changing the UI contract.

Exposing a Local Web App With a Cloudflare Tunnel

noreply@blogger.com (Unknown) — Sun, 07 Jun 2026 14:44:48 +0000

When you want to open a local web app from a phone, tablet, teammate's machine, or temporary external device, a Cloudflare Tunnel is a simple way to expose it without changing router settings. It gives you a public HTTPS URL that forwards traffic back to a local development server.

No port forwarding, no public IP, no DNS setup, and no inbound firewall exposure are required for the quick version.

What We Are Building

A common local full-stack app has two processes:

Frontend: Vite on http://127.0.0.1:5173
Backend API: for example ASP.NET Core, Node, Python, or another API on http://127.0.0.1:5069

The external browser only talks to the public Cloudflare URL. Cloudflare forwards that traffic to Vite, and Vite proxies backend calls to the local API:

External browser
  -> https://random-name.trycloudflare.com
  -> Cloudflare edge
  -> cloudflared on this machine
  -> Vite dev server on 127.0.0.1:5173
  -> /api and optional WebSocket routes proxied to the backend on 127.0.0.1:5069

For short-lived demos this is ideal. The URL is random, disposable, HTTPS-enabled, and does not require a Cloudflare account.

Why Cloudflare Tunnel

The normal LAN setup works if your phone and laptop are on the same WiFi, but it can fail because of hotel networks, mobile hotspots, VPNs, firewall rules, or router isolation.

A tunnel avoids that by making an outbound connection from your machine to Cloudflare. Your laptop does not need to accept inbound internet traffic directly.

Cloudflare calls this short-lived mode a Quick Tunnel or TryCloudflare tunnel. The command looks like this:

cloudflared tunnel --url http://localhost:5173

Cloudflare then prints a URL like:

https://some-random-words.trycloudflare.com

That is the URL you open from the external device.

Step 1: Install `cloudflared`

Download cloudflared from Cloudflare:

https://developers.cloudflare.com/tunnel/downloads/

On Windows, you can put cloudflared.exe somewhere simple, for example:

C:\Tools\cloudflared\cloudflared.exe

Then either add that folder to PATH, or run it with the full path:

C:\Tools\cloudflared\cloudflared.exe --version

If it prints a version, the binary is ready.

Step 2: Start the Backend API

Start whatever backend your app uses. For example, a .NET API might look like this:

cd C:\repo\my-app
dotnet run --project src\MyApp.Api\MyApp.Api.csproj --launch-profile http

Or a Node API might look like this:

cd C:\repo\my-app\backend
npm run dev

Check the backend health endpoint if you have one:

Invoke-WebRequest -UseBasicParsing -Uri http://127.0.0.1:5069/api/health

You want HTTP 200.

Step 3: Start the Frontend

In another terminal:

cd C:\repo\my-app\frontend
npm run dev:lan

For a Vite app, dev:lan typically runs Vite with:

vite --host 0.0.0.0

That makes the dev server reachable beyond only localhost. The Cloudflare tunnel can still point at 127.0.0.1, but LAN binding also keeps the same command useful for direct local-network testing.

Check the frontend locally:

Invoke-WebRequest -UseBasicParsing -Uri http://127.0.0.1:5173

Again, you want HTTP 200.

Step 4: Start the Cloudflare Tunnel

In a third terminal:

cloudflared tunnel --url http://127.0.0.1:5173

If cloudflared is not on PATH, use the full path:

C:\Tools\cloudflared\cloudflared.exe tunnel --url http://127.0.0.1:5173

After a few seconds, cloudflared prints a public URL ending in:

.trycloudflare.com

Open that URL from the external device.

Why We Tunnel the Frontend, Not the Backend

For a React/Vite app backed by an API, expose Vite on port 5173, not the backend API directly.

That matters because the browser app loads frontend assets from Vite, and Vite can proxy backend traffic:

/api goes to the backend API
/hubs, /ws, or another WebSocket route goes to the backend when needed

So from the external browser's point of view, everything appears to come from the same public origin:

https://random-name.trycloudflare.com

That avoids mixed-origin problems for normal API calls and WebSockets.

If you only need to expose an API without a frontend, point the tunnel directly at the API instead:

cloudflared tunnel --url http://127.0.0.1:5069

Vite Proxy Requirement

For a frontend + backend setup, the frontend dev server should proxy API and WebSocket requests to the backend. In a Vite app, this lives in vite.config.ts.

Conceptually it needs this:

server: {
  proxy: {
    '/api': 'http://127.0.0.1:5069',
    '/hubs': {
      target: 'http://127.0.0.1:5069',
      ws: true,
    },
  },
}

The important bit is ws: true for any WebSocket route. For example, SignalR often uses /hubs, while other apps might use /ws, /socket, or /socket.io.

Backend CORS

In this setup, CORS is less painful because the external browser calls the Cloudflare URL, and Vite proxies backend calls server-side during development.

Still, your backend CORS policy should support the local dev origins used by Vite:

http://localhost:5173
http://127.0.0.1:5173

For direct LAN testing, you may also want to allow private network origins on port 5173, such as:

http://192.168.x.x:5173
http://10.x.x.x:5173
http://172.16.x.x:5173

That is separate from the Cloudflare tunnel, but useful when testing directly on WiFi.

Quick Tunnel vs Named Tunnel

The command above creates a Quick Tunnel:

cloudflared tunnel --url http://127.0.0.1:5173

That is best for:

A temporary demo
Sharing a local app briefly
Testing from a phone or tablet
Avoiding Cloudflare account setup

The tradeoffs:

The URL is random.
The URL changes when you restart the tunnel.
It is not meant to be permanent infrastructure.
Anyone with the URL can try to access it.

For a longer-running setup, use a named tunnel with a Cloudflare account and your own domain. That gives you stable DNS, access policies, service installation, and better operational control.

Security Notes

Treat the random URL as public.

It is obscure, but not authentication. For a quick demo that may be enough. For anything long-lived, add one of:

Cloudflare Access
App-level authentication
A private token in the app
A named tunnel behind your own domain

Also remember that the local app is still your development instance. If it has admin endpoints, debug tools, seed data, or destructive buttons, the tunnel exposes those too.

As a rule of thumb, a Quick Tunnel is fine for short private checks, but it should not be left running indefinitely without authentication.

Troubleshooting

If the Cloudflare URL opens but the page is blank:

Confirm Vite is running on 5173.
Open http://127.0.0.1:5173 locally.
Check the Vite terminal for errors.

If API data does not load:

Confirm the backend is running on 5069.
Open http://127.0.0.1:5069/api/health.
Check that Vite is proxying /api to the backend.

If WebSockets do not connect:

Confirm your WebSocket route is proxied with ws: true.
Check browser dev tools for WebSocket errors.
Restart Vite and cloudflared.

If the tunnel command starts but no URL appears:

Check your internet connection.
Try a newer cloudflared binary.
Make sure no local Cloudflare config is interfering with Quick Tunnels.

If the URL stops working:

Check that the cloudflared process is still running.
Quick Tunnel URLs are tied to that running process.
Restarting the process normally creates a new URL.

Useful Commands

Backend:

cd C:\repo\my-app
dotnet run --project src\MyApp.Api\MyApp.Api.csproj --launch-profile http

Frontend:

cd C:\repo\my-app\frontend
npm run dev:lan

Tunnel:

cloudflared tunnel --url http://127.0.0.1:5173

Health checks:

Invoke-WebRequest -UseBasicParsing -Uri http://127.0.0.1:5069/api/health
Invoke-WebRequest -UseBasicParsing -Uri http://127.0.0.1:5173

References

Cloudflare Quick Tunnels: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/
Cloudflare cloudflared downloads: https://developers.cloudflare.com/tunnel/downloads/
Cloudflare Tunnel configuration: https://developers.cloudflare.com/tunnel/configuration/

Building a Local Voice Cloning Studio That Actually Runs on My Machine

noreply@blogger.com (Unknown) — Sun, 19 Apr 2026 06:33:00 +0000

I wanted a local voice cloning workflow that felt like a real product, not a notebook demo.

The goal was simple: record or upload a short reference clip, create a reusable voice profile, type arbitrary text, synthesize speech locally, and keep every file on my own machine. No accounts, no cloud voice API, no remote inference.

The result is local-voice-studio: a React and FastAPI application with local profile management, reference audio handling, generation history, diagnostics, and pluggable local TTS engines.

The Product Shape

The app is built around a few core workflows.

You can create multiple voice profiles, each with notes, language preference, tags, saved reference clips, a primary clip, and synthesis defaults. Reference clips can come from browser microphone recording or from uploaded audio files. The backend normalizes every clip to a model-friendly WAV file, stores metadata in SQLite, and keeps the original upload on disk.

The generation page is a focused workspace: choose a profile, enter text, optionally provide delivery instructions, tweak advanced settings, generate audio, and then preview or download the WAV result. Every generation is stored in history with status, parameters, input text, output path, and errors if something fails.

The settings page shows the local runtime state: active engine, model name, GPU detection, FFmpeg availability, directories in use, and whether local transcription is available.

Everything lives under the local data/ directory.

Architecture

The backend is FastAPI with a deliberately boring service/repository split.

Routes handle HTTP only. Services coordinate profile management, audio processing, generation jobs, runtime diagnostics, and TTS orchestration. Repositories isolate SQLite persistence. Audio utilities handle normalization and metadata probing through FFmpeg and FFprobe.

The TTS runtime sits behind a TtsEngine interface. The generation service does not care whether the active engine is XTTS or Qwen3. It prepares a profile, builds a synthesis payload, writes the output to data/generated, and records status updates in SQLite.

Synthesis runs as an in-process background job. That is enough for a local desktop-style app and avoids adding Redis, Celery, or other distributed infrastructure too early.

The frontend is React, TypeScript, Vite, React Router, and TanStack Query. The UI is desktop-first, dark, and intentionally simple: left navigation, profile cards, reference clip panels, a generation workspace, history, and diagnostics.

The XTTS Lesson

The first implementation used Coqui XTTS v2.

XTTS was a useful baseline because it can do zero-shot voice cloning from reference audio and it is relatively well-known in local TTS experiments. The app normalized clips, cached conditioning artifacts when possible, and used the selected primary clip as the conditioning source.

But in practice, the output did not sound close enough. The voice often drifted into a generic speaker. In one test, the result sounded like a random English man rather than the target voice.

That was an important product lesson: a working local model integration is not the same thing as a good cloning experience.

The app was technically generating speech, but the speaker identity was not good enough.

Why Transcript-Aware Cloning Helped

The breakthrough was moving from audio-only cloning to transcript-aware cloning.

Qwen3-TTS expects the reference audio and the exact text spoken in that reference audio. That small workflow change matters. Instead of asking the model to infer everything from the waveform alone, the app gives it aligned acoustic and linguistic context.

So the profile workflow changed:

Record or upload a clean reference clip.
Mark the best clip as primary.
Save the exact transcript of that clip.
Generate using Qwen3-TTS.

The app now stores reference_text on each clip. The primary clip transcript becomes part of the conditioning fingerprint, so changing the transcript invalidates old cached prompts. There is also an optional local transcription path using a Whisper-style local ASR dependency, but manual transcript entry is always available and often better if the user knows exactly what they said.

This made the system feel much closer to the Voicebox-style workflow that worked well in practice.

UX Details That Matter

Local AI workflows often have long, silent waits. That is bad UX.

The app now shows loading feedback where it matters:

The Transcribe locally button shows a spinner while transcription is running.
The transcription button is disabled while the action is in progress, preventing accidental duplicate requests.
The Generate audio button shows a spinner while the job is starting or running.
The generate page warns if the selected primary clip is missing its transcript.

These are small details, but they make the app feel more trustworthy. When local models take time to load, users need visible progress.

Running It Locally

The recommended path is Qwen3-TTS.

From the repository root:

powershell -ExecutionPolicy Bypass -File .\scripts\setup.ps1
& .\.venv\Scripts\python.exe -m pip install -e ".\backend[qwen,transcription]"

Start the backend:

$env:LVS_TTS_ENGINE = "qwen3"
powershell -ExecutionPolicy Bypass -File .\scripts\dev-backend.ps1

Start the frontend:

powershell -ExecutionPolicy Bypass -File .\scripts\dev-frontend.ps1

Then open:

http://localhost:5173

If another local app is already using port 8000, run the backend on 8010 and point the frontend proxy at it:

$env:LVS_TTS_ENGINE = "qwen3"
$env:LVS_BACKEND_PORT = "8010"
powershell -ExecutionPolicy Bypass -File .\scripts\dev-backend.ps1

$env:VITE_API_TARGET = "http://127.0.0.1:8010"
powershell -ExecutionPolicy Bypass -File .\scripts\dev-frontend.ps1

What I Would Improve Next

The next improvements are mostly product polish and runtime hardening.

The app should make the active engine more visible in the main UI. Profiles should get a clear "Qwen-ready" badge when their primary clip has a transcript. The transcription workflow could show progress beyond a spinner, especially on first model load. The settings page could provide stronger guidance when the environment is missing Qwen, Whisper, CUDA, FFmpeg, or model weights.

Long-form generation also needs more work. The app currently supports normal generation jobs, but paragraph-aware chunking, concatenation, and better history grouping would make it more useful for narration.

Finally, Qwen and XTTS have different Python/runtime expectations. A future version could support separate engine environments or a small process boundary so users do not have to fit every model into one Python virtual environment.

Takeaway

The biggest lesson was that local-first voice cloning is not just about wiring up a model.

The workflow matters.

For this project, XTTS proved the architecture and local storage model. Qwen3-TTS made the voice identity closer by adding the missing reference transcript. The resulting app is still early, but it now has the shape of a real local voice studio: profiles, clips, transcripts, generation history, runtime diagnostics, and no cloud dependency.

Beyond Prompts: Building a Codex Playbook for Real Application Developmentnoreply@blogger.com (Unknown) — Mon, 13 Apr 2026 20:13:00 +0000
One-off prompting is weak engineering infrastructure.
It can be useful for getting unstuck, drafting code, or exploring ideas, but it does not produce repeatable engineering behavior on its own. The same prompt that works acceptably on one backend feature often degrades into generic advice or inconsistent implementation on the next one. That happens because a prompt is usually trying to do too many jobs at once: repository policy, architectural philosophy, feature workflow, review heuristics, and implementation detail.
This repository was built to separate those concerns and make them durable. 
JordiCorbilla/codex-engineering-playbook
The Core ProblemMost teams experimenting with coding agents start with giant instruction blobs. They pour style preferences, architecture opinions, code review heuristics, framework conventions, and edge-case warnings into a single file or prompt. That usually fails in predictable ways:
durable guidance gets buried under temporary workflow detail
the instructions become too large to scan and too vague to route reliably
project structure becomes secondary to the agent's last prompt
implementation becomes inconsistent across languages and application layers
The problem is not that the model needs more words. The problem is that the repository needs a better operating model.
What This Repository Builds InsteadThis playbook uses four layers:
AGENTS.md for durable repository-wide rules
.agents/skills for reusable workflows
templates and scripts for deterministic implementation support
examples for concrete, inspectable reference projects
That split is the point of the repository.
The top-level AGENTS.md stays small. It says what the repository is for, when to prefer skills, what "done" means, and what kinds of validation are expected. It does not try to teach every implementation pattern directly. That restraint is important. Durable instructions should be hard to invalidate.
The detailed workflow logic moves into focused skills such as:
csharp-backend-feature
python-backend-review
react-typescript-feature
api-contract-review
layered-architecture-recon
This is better than a giant style blob because routing matters. Reviewing a Python backend diff is not the same task as implementing a React feature or mapping an unfamiliar layered architecture. Putting those workflows into separate skills makes the instructions sharper and more reusable.
Why Durable Repo Instructions MatterAgent behavior improves when the repository teaches stable defaults instead of relying on the operator to restate them every time.
In this repository, the durable defaults are simple:
inspect existing structure first
keep boundaries clear
put logic in the right layer
validate the touched area before calling the work done
explain trade-offs when architecture changes
Those rules are broad enough to matter across C#, Python, and React, but narrow enough to remain useful. They establish an operating model without pretending the languages are interchangeable.
Why Skills Beat Giant Prompt FilesA single giant prompt tends to flatten all engineering work into "follow best practices." That is almost always too soft. Real engineering work has modes, and the quality bar changes with the mode.
Feature work needs placement discipline. Review work needs risk detection. Contract review needs attention to caller-visible behavior. Architecture reconnaissance needs an accurate map, not rewrite advice.
That is why the skills in this repository are intentionally separate:
csharp-backend-feature cares about thin controllers, service-led behavior, async correctness, DTO boundaries, and targeted tests.
csharp-backend-review cares about fat controllers, sync-over-async mistakes, leaky entity boundaries, and weak exception flows.
python-backend-feature emphasizes typed boundaries, thin request handling, explicit services, and restrained abstraction.
frontend-ux-polish is about hierarchy, spacing, labels, state clarity, and affordances, not broad frontend architecture.
Separating those workflows does two things. It makes routing more reliable, and it avoids making every task carry the weight of every other task's instructions.
The C# GuidanceThe C# portion of the playbook favors a layered ASP.NET Core style, but it is not blindly enterprise. The guidance in conventions/csharp.md pushes toward controllers, services, and repositories when those boundaries provide real value. It also says something many style guides avoid saying clearly: repository abstractions can become ceremony.
That trade-off matters. In the C# example app, the repository boundary exists because it helps keep persistence concerns separate and makes the service easy to test. But it stays intentionally small and in-memory. The goal is to demonstrate responsibility boundaries and async flow, not to perform ORM theater.
The example includes:
OrdersController.cs for HTTP concerns
OrderService.cs for validation, orchestration, and business flow
InMemoryOrderRepository.cs for isolated storage behavior
ExceptionMappingMiddleware.cs for centralized error translation
That is a good example of the repository's overall stance: use structure to clarify responsibilities, not to multiply files for their own sake.
The Python GuidancePython code often swings between two bad extremes when teams talk about architecture. One extreme is route handlers full of business logic and IO. The other is an imported enterprise pattern that creates classes, abstract base types, and indirection that the codebase has not earned.
The Python guidance in conventions/python.md tries to avoid both.
It emphasizes:
thin request handling
typed boundaries
explicit validation
clear service or module ownership
simplicity over cleverness
In the Python example app, the route module translates HTTP concerns, the service owns the business path, and the data-access layer stores records in memory. The example is intentionally small, but the boundaries are inspectable:
orders.py keeps request handling thin
orders.py owns the business behavior
orders.py isolates storage concerns
orders.py defines typed request and response models
That structure is useful because it makes future changes cheaper. A new validation rule or storage implementation has an obvious home.
The React + TypeScript GuidanceFrontend guidance is often too aesthetic or too abstract. This playbook is trying to be engineering guidance, so the React conventions focus on code structure, state handling, and contract discipline.
The priorities in conventions/react-typescript.md are:
keep components focused
separate API access from rendering
make loading, error, and empty states explicit
type props and async data honestly
keep hooks disciplined
avoid business logic hidden in JSX
The React example app is not a design system demo. It is a small application slice showing the separation between transport, state, and presentation:
orders.ts owns data access behavior
useOrders.ts models async state transitions
OrderSummaryList.tsx renders typed data
App.tsx composes the UI and makes loading, empty, and error states visible
That is deliberate. Many generated React examples hide the interesting part of the problem inside a single file. This one does not.
Why The Examples MatterExample apps are not filler in a repository like this. They are operational anchors.
Without examples, convention documents drift toward abstract advice. Skills become more likely to overfit to generic framework knowledge. Templates become harder to evaluate because there is no concrete reference point for what "good" looks like in the repository.
The examples in this playbook make the guidance testable. They show the relationship between the documents and actual code. They also give Codex stable insertion points when exploring an unfamiliar but similarly structured codebase.
The practical run instructions live in the repository README. That split is intentional: the blog explains the design, while the README acts as the operational starting point.
Scripts And Templates As Engineering ToolsAnother failure mode in agent-oriented repositories is over-explaining deterministic work in prose. If a C# controller shape is predictable, a template is better than another paragraph. If you need to discover likely insertion points or summarize changed files, a small script is usually better than asking the model to infer everything from scratch every time.
That is why this repository includes:
templates/csharp
templates/python
templates/react
scripts/scan_repo.py
scripts/find_feature_insertion_points.py
scripts/discover_tests.py
scripts/summarize_changed_files.py
scripts/validate_playbook.py
These are small on purpose. They exist to support repeatability, not to introduce another platform inside the repository.
StrengthsThe repository is structured for reuse, not just demonstration.
The top-level instructions are durable enough to survive ordinary growth.
The skills are narrow enough to route reliably.
The examples make the guidance concrete instead of rhetorical.
The validation bar is lightweight but real.
LimitationsThe playbook is intentionally opinionated, which means it will not fit every team culture.
The examples are small; they demonstrate boundaries, not the full complexity of production systems.
The repository does not solve framework-specific edge cases beyond the patterns it chooses to illustrate.
Skills can become stale if a team's real architecture evolves and the playbook does not.
There is also a rigidity trade-off. If a team treats the playbook as law rather than guidance, it can become a local maximum. Not every codebase needs controller/service/repository layering. Not every React feature needs another hook. Not every Python service needs another module boundary. Good playbooks create defaults, not dogma.
Where This Helps MostThis approach helps most when:
a team wants more repeatable coding-agent behavior
the codebase already values clear boundaries
multiple languages need a shared architectural vocabulary
implementation and review workflows should be routable and explicit
It helps less when:
the codebase is intentionally experimental
architecture changes weekly
framework-specific constraints dominate general engineering structure
the team wants maximal freedom and minimal convention
Pros And ConsPros:
better repeatability than one-off prompts
clearer separation between policy, workflow, and implementation detail
easier extension through focused skills
stronger grounding through examples and validation
Cons:
more repository structure to maintain
more upfront design work than dropping a prompt file into a repo
some duplication between conventions, skills, and examples is intentional and needs upkeep
can become too rigid if teams stop exercising judgment
Final ViewThe point of this repository is not to prove that Codex can follow instructions. The point is to make those instructions worth following.
That requires durable repository guidance, focused workflow skills, deterministic helpers, and concrete examples. It also requires saying no to the usual temptation to dump everything into a single mega-prompt and hope the agent will sort it out.
That temptation is understandable. It is also a weak substitute for engineering structure.

Skills for Codex: The missing layer between prompts and reliable workflows

noreply@blogger.com (Unknown) — Sun, 12 Apr 2026 18:27:00 +0000

The real unlock for coding agents is packaging workflows into AGENTS.md, skills, and scripts

Most teams experimenting with coding agents are still operating at the wrong layer.

They keep trying to get reliability by writing better prompts. That helps a little. It does not solve the real problem.

The hard part is not telling the model to "be careful" or "think step by step." The hard part is turning repeated engineering workflows into artifacts the agent can reuse: repo instructions, narrowly scoped skills, and deterministic helper scripts.

That is what I built in this demo repository.

The repo is small on purpose. It contains a tiny legacy-style Python service, four concrete Codex skills, a short AGENTS.md, several Python scripts, and lightweight tests. The point is not to show off a toy prompt library. The point is to show what a serious workflow layer looks like when you want an agent to behave more like an engineer and less like an improv partner.

One-line takeaway: prompting is too ephemeral for repeated engineering work.

Another one: if a workflow matters more than once, it should probably exist as a repo artifact.

Why prompting alone breaks down

One-off prompting degrades for boring reasons:

The same task gets described slightly differently every time.
The agent re-derives the workflow from scratch.
Deterministic steps get handled as prose instead of code.
The model mixes operational guidance with local reasoning and drops part of it under pressure.

Senior engineers already know this pattern from human systems. If a workflow matters, we do not rely on memory alone. We add runbooks, scripts, templates, checks, and conventions.

Coding agents need the same thing.

"Please inspect the code carefully before changing anything" is not a reliable mechanism. It is a wish. A skill with explicit triggers, steps, outputs, and boundaries is a mechanism.

The missing layer: `AGENTS.md` + skills + scripts

The repo uses three layers, each doing a different job.

AGENTS.md is the repo-level contract. It tells Codex how work should be done here:

prefer skills for recurring workflows
use scripts for deterministic tasks
validate changes before calling them done
plan first when risk or ambiguity is non-trivial

That file is intentionally short. Repo instructions are more useful when they set constraints and decision rules, not when they try to micromanage every possible action.

The skills are the workflow layer. Each one has a sharp name, obvious routing description, trigger conditions, workflow steps, expected outputs, and boundaries. That matters because routing quality is part of the problem. If skill names are vague, the agent will miss them or apply them at the wrong time.

The scripts are there for the parts that should not depend on language generation at all. Parsing test failures, scanning a repo for hotspots, and extracting risk signals from a diff are all better handled with code than with a paragraph of instructions.

One-line takeaway: natural language is a poor substitute for a parser.

The example repo

The demo repo has a compact but realistic structure:

skills/ contains four engineering workflows
scripts/ contains Python helpers for deterministic scanning and summarization
demo/legacy_service/ contains a deliberately awkward Python service
demo/artifacts/ contains failing test output and a risky sample diff
tests/ validates the scripts

The service is small, but it has the right kind of mess:

module-level in-memory storage
a module-level cache
configuration read from environment at import time
time-sensitive pricing logic
hidden coupling between service, storage, and cache code

That is enough to make the skills meaningful. The repo is not pretending that agents only work on greenfield, well-factored code.

How to get the repo and run it

The repository is here:

https://github.com/JordiCorbilla/engineering-skills-for-codex

If you want the full example locally, clone it and run the lightweight validation:

git clone https://github.com/JordiCorbilla/engineering-skills-for-codex.git
cd engineering-skills-for-codex
python -m unittest discover -s tests -p "test_*.py" -v
python -m unittest discover -s demo/tests -p "test_*.py" -v

If make is available in your environment, the equivalent entry points are simpler:

make test
make validate

The skills themselves live under skills/. In practice, you use them by opening the repo in Codex and invoking the workflow by name when the task matches. For example:

ask Codex to use investigate-before-patching before editing demo/legacy_service/services/order_service.py
ask Codex to use summarize-test-failures against demo/artifacts/pytest_failure_output.txt
ask Codex to use legacy-codebase-recon on demo/legacy_service
ask Codex to use review-change-safely on demo/artifacts/sample_change.diff

The deterministic parts can also be run directly without the skill wrapper:

python scripts/inspect_patch_context.py --root demo --target demo/legacy_service/services/order_service.py
python scripts/summarize_test_failures.py --input demo/artifacts/pytest_failure_output.txt
python scripts/legacy_recon.py --root demo/legacy_service
python scripts/diff_review_summary.py --input demo/artifacts/sample_change.diff

If you want to reuse the skills in another repository, the practical starting point is to copy the relevant folder from skills/, bring over any supporting script from scripts/, and adapt AGENTS.md so the repo-level instructions match the codebase you actually have. Treat these skills as templates for repeatable workflows, not drop-in magic.

The skills

`investigate-before-patching`

This skill exists to stop speculative editing.

Before changing code, the workflow inspects the target module, local dependencies, related tests, and obvious coverage gaps. The helper script, scripts/inspect_patch_context.py, gives a deterministic first pass over imports, sibling modules, and nearby tests.

That sounds basic. It is also exactly the sort of discipline agents tend to skip when left alone. They jump to the patch because producing a patch is easier than earning one.

The important design choice here is scope. The skill does not promise a full call graph. It explicitly says the scan is a starting point and that critical modules still need to be read.

`summarize-test-failures`

This skill handles a very common failure mode in agent-driven work: drowning in raw test output.

The script ingests a test log, extracts parseable failures, clusters them by likely cause, and produces a debugging brief. In the demo, the sample pytest log gets reduced to three concrete buckets:

stale state or cache invalidation
pricing rule regression
configuration drift

That is already more useful than dumping forty lines of traceback back at the user.

The limit is important though: this is a triage tool, not a diagnosis engine. If the log is messy or incomplete, the summary should stay humble.

`legacy-codebase-recon`

This is the first-pass orientation skill for inherited systems.

The script looks for entry points and scores modules for hotspots using simple heuristics: mutable globals, environment coupling, cache usage, time-sensitive behavior, and general structural weight. On the demo service, it correctly flags the order service and repository layer as the riskiest modules.

This is exactly where skills can help: not by pretending to understand a legacy system instantly, but by providing a disciplined first pass that makes the next manual inspection smarter.

The failure mode is also obvious. Heuristics are directional. A hotspot score is not architecture truth.

`review-change-safely`

This skill is designed for reviewing diffs with an emphasis on unintended side effects.

The demo diff changes a function signature, alters the returned payload shape, and replaces a guarded key lookup with a required-key lookup. The script surfaces all three and also notes that no tests moved with the behavior change.

That is a useful baseline because many agent reviews default to a glorified summary. Serious review work needs findings, not narration.

The skill therefore forces a review shape:

findings first
severity and confidence
file references
residual risk

That structure matters. It is much easier to trust and act on than a free-form paragraph about what changed.

Why scripts matter

The scripts are not there because Python is exciting. They are there because some tasks are simply better when the machine is forced to be literal.

Three examples:

First, parsing failure lines. Asking a model to "read this pytest output and summarize it" works until the output is long, repetitive, or messy. A parser is cheap and stable.

Second, scanning code for risk signals. Telling the model to "look for tight coupling and hidden state" is too vague on its own. A script can make that scan explicit and repeatable, even if the results are only heuristic.

Third, diff review. Function signature changes, payload key changes, and test coverage gaps are not subtle concepts. They should be machine-detectable.

The right split is simple:

use scripts for deterministic extraction
use skills for workflow logic
use the model for judgment, prioritization, and synthesis

What worked well

The most useful part of the repo is the layering.

AGENTS.md gives the repo a stable operating model. The skills give recurring tasks structure and routing clarity. The scripts prevent a class of drift where every run invents a slightly different method. The tests keep the helpers from silently rotting.

The second useful part is restraint. The skills are short. They are not trying to encode every engineering principle. They are only trying to make a few repeated workflows reliable enough to be worth using.

One-line takeaway: a narrow skill you can trust is worth more than a broad skill you cannot.

What still does not work well

Skills do not remove the need for real engineering judgment.

They do not understand runtime behavior. They do not replace reading the code. They do not magically fix weak tests, misleading names, or missing context from the user. They also do not help much when the task is genuinely novel and there is no repeated workflow to package.

There is also a maintenance cost. Once you create a skill, you own it. If the repo changes and the skill instructions or script behavior drift, you have created a false sense of reliability.

This is why I would be cautious about building fifty skills too early. Most teams should start with a handful of painful, recurring workflows and make those solid.

When I would use this in production

I would use this pattern in production when:

the team is repeatedly using agents on the same codebase
the work includes bug triage, reviews, legacy recon, or change planning
there is enough repetition to justify a workflow artifact
the team cares more about consistency than about model theatrics

I would not bother in a repo where agent use is rare, the workflows are highly bespoke, or the codebase is changing so quickly that skill maintenance would dominate the value.

The threshold is not "do we use AI?" The threshold is "do we keep repeating the same engineering moves?"

Closing thoughts

The claim here is modest.

Coding agents do not become reliable because we found the perfect prompt. They become more useful when we stop forcing the model to reinvent stable workflows on every task.

That means packaging the workflow.

Write the repo contract in AGENTS.md. Turn repeated tasks into skills with real boundaries. Move deterministic steps into scripts. Test those scripts. Then let the model operate inside that frame.

That is less glamorous than prompt alchemy. It is also much closer to how engineering systems usually become dependable.

Building Fund Overlap Lab: From Surface Labels to Real Exposure

noreply@blogger.com (Unknown) — Wed, 08 Apr 2026 15:34:00 +0000

If you have ever compared two multi-asset funds and thought, "These look different, but are they actually different?" this project is for that exact problem.

Fund Overlap Lab is a Python tool for Vanguard UK funds that retrieves holdings, normalizes names, and calculates overlap. It includes a CLI for fast checks and a Streamlit app for interactive analysis.

JordiCorbilla/fund-overlap-lab: fund-overlap-lab

Why It Exists

Wrapper funds such as LifeStrategy and Target Retirement can look distinct at the top level while sharing large portions of the same underlying exposures.

The project aims to make that visible by combining:

look-through holdings
transparent overlap math
risk and cost context
practical UI workflows for real portfolio conversations

Core Overlap Model

For two funds, overlap is calculated as:

sum(min(weight_a_i, weight_b_i))

where each i is a normalized underlying holding.

The app then reports:

overlap percentage
shared and distinct holdings
only-in-A and only-in-B tables
bucket-level aggregation

Data Pipeline: What Changed and Why

Early versions parsed portfolio HTML tables. That broke when pages shifted to JS-driven rendering. The current pipeline is built to survive those changes.

Current strategy:

primary source: GraphQL endpoint for detailed holdings
fallback source: HTML table/text extraction where possible
resilience fallback: API asset-allocation level extraction
dynamic product lookup from Vanguard product catalog

This layered approach keeps the tool operational when upstream page structures move.

How Data Access Works

Under the hood, the provider follows a staged resolution and retrieval flow:

Product resolution

the app accepts multiple identifiers (fund code, SEDOL, slug)
it queries Vanguard UK product metadata and resolves those inputs to canonical product fields (including portId)

Primary holdings retrieval (GraphQL)

with the resolved product, it calls Vanguard UK GraphQL endpoints to fetch holdings details
this path returns richer data than static page scraping, including constituent-level rows where available

Recursive expansion (optional)

in Ultimate Look-Through mode, holdings that are themselves funds or ETFs can be expanded recursively
each child weight is multiplied through the parent path weight
recursion is bounded by max depth and protected against cycles

Fallback retrieval

if detailed holdings cannot be retrieved, the provider falls back to HTML extraction and then allocation-level API data
this keeps the app functional even when one upstream path changes

Normalization and comparison preparation

holding names are normalized for robust joins
duplicate rows are consolidated
percentage fields are standardized before overlap calculations

This design keeps the user workflow simple while handling real-world upstream variability in a resilient way.

Ultimate Look-Through Mode (Recursive)

A major upgrade is recursive decomposition of fund-of-funds structures.

You can now choose between:

Direct Holdings mode: one layer of holdings
Ultimate Look-Through mode: recursively expands eligible underlying funds and ETFs

Both Two-Fund Compare and Portfolio Analysis now include:

an Ultimate Look-Through toggle
a max depth control to limit recursion
cycle protection so recursive expansion does not loop

This matters for mixed wrappers where one level of holdings is still another basket.

Example Questions It Answers Quickly

How much overlap exists between a LifeStrategy fund and a Target Retirement fund right now?
Is a portfolio diversified across managers and wrappers, or concentrated in repeated underlyings?
Do risk and OCF differences align with actual exposure differences?
Does recursive look-through materially change the diversification story?

Project Structure

fund_overlap_lab/providers.py: data access, product resolution, holdings retrieval
fund_overlap_lab/compare.py: overlap and portfolio analytics
fund_overlap_lab/models.py: data models
fund_overlap_lab/buckets.py: coarse asset bucketing
fund_overlap_lab/cli.py: command-line workflows
app.py: Streamlit experience

Run It

Install:

python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt

CLI compare:

python -m fund_overlap_lab.cli compare VGL100A VAR45GA

Streamlit app:

streamlit run app.py

Final Thought

Fund Overlap Lab is intentionally practical. It is built to help answer a real question before making allocation changes:

"Am I truly diversifying, or buying the same exposure through different wrappers?"

Building a Quant Developer Reference Librarynoreply@blogger.com (Unknown) — Tue, 07 Apr 2026 09:43:00 +0000
For a long time, I wanted a single place that explained quantitative finance the way quant developers actually need it explained.
Not as a pure math textbook. Not as a collection of interview flashcards. Not as a random pile of disconnected notes.
I wanted something practical. Something that connects pricing theory to implementation. Something that helps you move from "I know the formula" to "I can build this correctly, test it, and trust it in production."
So I started building the Quantitative Developer Reference Library. JordiCorbilla/Quantitative-Developer-Reference-Library: Quantitative-Developer-Reference-Library
Why This ExistsA lot of great quantitative finance material already exists. The problem is not the lack of information. The problem is fragmentation.
If you are a quant developer, the knowledge you need is usually scattered across:
textbooks that go deep on theory but say little about implementation,
code samples that skip market conventions and edge cases,
desk knowledge that lives in someone's head,
internal wikis that are useful but never portable,
and years of trial and error that are expensive to repeat.
That leaves a gap between understanding the model and building the system.
In practice, that gap is where most of the pain lives:
curves built with the wrong conventions,
volatility surfaces that look smooth but produce nonsense risk,
pricing engines that work for the happy path but fail on real trades,
PnL explain that does not reconcile,
market data that is technically available but not analytically usable,
and analytics platforms that become hard to trust as they grow.
This repository is meant to close that gap.
What The Library Tries To DoThe goal is simple:
Build a practitioner-focused reference for quant developers that connects products, pricing, risk, market data, numerical methods, and production architecture.
This is not meant to be read cover to cover like a course.
It is designed to work as:
a reference you can return to when building something real,
a bridge between theory and implementation,
a map of how the major pieces fit together,
and a foundation that can keep growing over time.
What Is In The RepoThe library is organized to cover both the instrument side and the engineering side.
Core product chaptersoptions,
futures and forwards,
equities,
FX,
fixed income,
interest rates,
credit,
commodities,
and cross-asset topics.
Quant engineering chaptersmarket data,
pricing architecture,
risk and PnL,
testing and validation,
performance and production.
There is also a numerical methods chapter because, in practice, almost every serious pricing problem ends up there eventually.
The idea is that a quant developer should not need to mentally separate:
"the finance part,"
"the code part,"
and "the production part."
Those are usually the same problem, just seen from different angles.
The Point Of ViewThis repo is opinionated.
It assumes that quantitative correctness is not just about deriving the right formula. It is also about:
using the right conventions,
consuming the right market data,
choosing the right numerical method,
exposing the right risk,
and building the system so the result is reproducible and explainable.
That is why the library focuses on questions like:
What does the desk actually quote?
What data do you really need?
What assumptions are hidden in this model?
What tends to break in production?
How should this be tested?
How do you know the output is sane?
Those questions matter just as much as the equation itself.
Who It Is ForI built this primarily for:
quant developers,
engineering-minded quants,
developers moving into front-office or analytics roles,
and practitioners who want a more implementation-grounded way to study the field.
If you are preparing for interviews, building pricing libraries, working on market data or risk systems, or trying to connect multiple years of learning into one coherent framework, this should be useful.
It should also be useful to people already in the field who want a clean reference instead of searching across old notes and half-remembered documents.
What Makes It DifferentThere are plenty of resources that explain option pricing. There are plenty of resources that explain fixed income. There are plenty of resources that explain numerical methods.
What is rarer is a resource that keeps all of the following in the same frame:
product intuition,
pricing formulas,
market conventions,
implementation shape,
validation logic,
and production pitfalls.
That is the standard I want this repository to meet.
For example, an options chapter should not stop at Black-Scholes. It should also talk about:
implied volatility surfaces,
no-arbitrage checks,
surface risk conventions,
American exercise intuition,
implementation pitfalls,
and the difference between a formula that works on paper and an analytics workflow that works on a desk.
The same standard applies across rates, fixed income, credit, market data, risk, and architecture.
Why I Am Sharing It EarlyI do not want this to become a private note system that only becomes visible once it is "finished."
A library like this gets better in the open.
It improves when people use it, challenge it, extend it, and point out where the real complexity lives. Quant development is too broad, too practical, and too experience-driven for one person to pretend they can write the final word on it alone.
So I am sharing it as a growing public reference.
The current version already has the foundation:
a consistent structure,
broad chapter coverage,
deeper treatment in options, fixed income, rates, and numerical methods,
and dedicated chapters for the engineering side of quant work.
From here, the plan is to keep making it more useful.
Where I Want To Take ItOver time, I want this repository to become a serious long-term resource for the quant developer community.
That means:
deeper product coverage,
better worked examples,
more implementation patterns,
more sanity-check frameworks,
stronger cross-links between topics,
and eventually a body of material that is genuinely useful both for study and for real-world system design.
Some of the areas I especially want to deepen:
volatility modelling,
multi-curve rates,
fixed-income relative value and spread analytics,
calibration workflows,
pricing architecture patterns,
risk explain,
and the operational side of production quant systems.
If You Work In This SpaceIf you are a quant developer, quant, trader, or engineer working near pricing and risk systems, I would love for you to take a look.
Use it. Challenge it. Steal from it. Improve it.
If a section is missing something important, I want to know. If a convention should be clarified, I want to know. If there is a subtle implementation trap that every real system hits, that is exactly the kind of knowledge worth capturing here.
The best technical references are not just correct. They are useful.
That is the bar.
RepositoryIf this sounds useful, you can explore the repo here:
Quantitative Developer Reference Library
Start with:
README.md
00-overview.md
01-options.md
05-fixed-income.md
06-interest-rates.md
10-numerical-methods.md
Closing ThoughtQuant finance has no shortage of complexity. What it often lacks is structure.
The point of this library is to create more structure:
structure around the products,
structure around the implementation,
structure around the failure modes,
and structure around the knowledge itself.
If it saves someone time, helps someone build something better, or gives the community a shared reference that is more practical than what already exists, then it is worth doing.
That is the project.
Short Version For Social PostsI built a Quantitative Developer Reference Library: a practical, growing repo for quant developers that connects products, pricing, risk, market data, numerical methods, and production architecture.
It is designed to bridge the gap between "I know the formula" and "I can build this properly."
Current coverage includes options, futures, equities, FX, fixed income, rates, credit, commodities, cross-asset topics, plus market data, pricing architecture, risk/PnL, testing, and production engineering.
If you work in quant dev, pricing, risk, or analytics engineering, I would love feedback.

Building a CLI Firework Simulation in Python, Just for Fun

noreply@blogger.com (Unknown) — Tue, 07 Apr 2026 07:36:00 +0000

Every now and then it is worth building something with no real business value at all.

JordiCorbilla/firework-simulation: firework-simulation

This terminal firework simulation was one of those projects.

It was not meant to be a serious graphics engine, a physics experiment, or a polished game. It was just a fun idea: could a terminal window, a handful of ASCII characters, and some ANSI colors feel enough like fireworks to make me smile? The answer turned out to be yes.

The basic idea is simple. A rocket launches from the bottom of the terminal, rises with velocity and gravity, bursts near its apex, and then scatters particles across the screen. Those particles fade out over time, slow down slightly, and disappear. Render that fast enough in a loop and you get something that feels lively, colourful, and surprisingly expressive for such a low-tech medium.

What makes a project like this enjoyable is that the constraints are part of the charm. A terminal is a terrible place for realistic animation. Resolution is coarse, circles are approximate, motion is chunky, and different terminals behave differently. But that is exactly why the project works best as a stylized effect rather than a realistic one. Bright colours, simple trails, exaggerated bursts, and particles fading from @ to o to * to . do more for the effect than realism ever could.

The implementation uses plain Python and ANSI escape codes, which keeps it lightweight and easy to run. There is a render loop, a small particle system, a few burst patterns, and some keyboard controls for launching rockets, triggering bursts, toggling finale mode, pausing, and quitting. Nothing too deep. Just enough structure to make the show feel varied.

I also leaned into the playful side of it. There are different shell patterns like chrysanthemum, ring, willow, palm, and crackle. Rockets leave trails. The background has stars. There is an automatic finale mode that ramps up the pace. None of this was necessary. That was the point.

Projects like this are a good reminder that programming does not always need to be optimized around utility. Sometimes it is enough to build something because it is amusing, visually satisfying, or slightly ridiculous. A CLI firework show definitely qualifies.

In the end, this was just for fun, and that is exactly why it was worth making.

If you want to try it yourself, run:

python -m fireworks

And if you want to interact with it:

space launches a rocket
r triggers a random burst
f toggles finale mode
p pauses
q quits

It is a small project, but it captures something I like about programming: sometimes a silly idea is reason enough.

Running Gemma 4 Locally on Android: Surprisingly Good

noreply@blogger.com (Unknown) — Tue, 07 Apr 2026 07:25:00 +0000

I’ve been experimenting with on-device LLMs for a while—mostly via Ollama, OpenClaw, and various local inference stacks—but this is the first time I’ve seen something that feels actually usable on a phone without compromise.

I just tested Gemma 4 (E2B-it variant) directly on Android, and the experience is… unexpectedly solid.

TL;DR

Area	Verdict
Setup	Extremely simple (native integration)
Performance	Fast enough to feel interactive
Quality	Strong for a ~2.5GB model
UX	Clean, minimal, no friction
Practicality	Finally viable for real usage

What This Is

This is Google’s Gemma 4 model running fully on-device, exposed through an Android-native interface using LiteRT-LM.

Key characteristics:

~2.5GB footprint (E2B-it variant)
Runs entirely locally
Supports multimodal input
~32K context window
No cloud dependency
No latency spikes from network

This matters more than it sounds—because most “local LLM” setups are either:

Too heavy (desktop GPU required), or
Too slow (toy-level mobile inference)

This sits right in the middle: practical local intelligence.

First Impressions

1. Setup Experience

This is where it stands out immediately.

No:

Docker
Python envs
CUDA nonsense
CLI gymnastics

Just:

Open app
Tap download
Done

Compared to typical local setups (Ollama, llama.cpp, etc.), this is orders of magnitude simpler.

2. Performance

This is the surprising part.

Responses are fast enough to feel conversational
No obvious stalling or token starvation
Latency feels closer to edge inference than “local hack”

This suggests:

Aggressive quantization
Optimized runtime (LiteRT-LM is doing heavy lifting here)
Likely hardware acceleration (NNAPI / GPU paths)

It’s not desktop-level—but it’s good enough to actually use.

3. Model Quality

For a 2.5GB model, the quality is impressive:

Coherent reasoning
Good instruction following
Decent structure in responses
No obvious collapse under moderate prompts

Where it likely struggles (as expected):

Deep multi-step reasoning
Heavy coding tasks
Long-chain logical consistency

But for:

Notes
Quick analysis
Idea generation
Lightweight coding help

…it’s absolutely viable.

Why This Matters (Strategically)

This is bigger than just “cool mobile AI”.

1. True Edge AI Is Finally Here

We’re crossing a threshold:

Before	Now
Cloud-only intelligence	Local-first viable
Privacy tradeoffs	Fully private inference
Latency issues	Instant response
API costs	Zero marginal cost

This changes:

Enterprise workflows
Personal productivity
Privacy models

2. Cost Model Disruption

If you can run:

A good-enough model locally
With zero infra cost

Then:

Not every task needs GPT-5 / Claude Opus
You offload 70–80% of interactions locally
Cloud becomes premium tier, not default

This is exactly the direction your current stack (OpenClaw + Ollama) is already heading—this just compresses it into mobile.

3. UX Is the Real Breakthrough

The real innovation here is not the model—it’s the delivery.

Compare:

Stack	Friction
Ollama + CLI	Medium
OpenClaw + agents	High (powerful but complex)
Android Gemma app	Near zero

Who wins?
The one users can install in 30 seconds.

Where It Fits in a Serious Stack

Given your current setup (agentic + local infra), this opens interesting architecture options:

Hybrid Model Strategy

Mobile (Gemma 4)
→ Quick queries, notes, offline usage
Local Desktop (Ollama / OpenClaw)
→ Agent workflows, automation, coding
Cloud (Claude / GPT)
→ Heavy reasoning, critical tasks

This becomes a tiered inference system:

Cheap → Fast → Local
Expensive → Smart → Cloud

Limitations (Be Realistic)

This is not magic.

Constraints:

Memory-bound → limited reasoning depth
Smaller parameter count → weaker abstraction
Likely struggles with:
- complex code generation
- financial modelling
- deep system design

Hidden Tradeoffs:

Quantization artifacts
Potential hallucination under pressure
Performance tied to device hardware

Still—none of these are deal-breakers for its target use.

Opinionated Take

This is the first time I’d say:

Local LLMs on mobile are no longer a gimmick.

We’re not at parity with cloud models—but we don’t need to be.

We just need:

70% of capability
0% latency
0% cost
100% privacy

And this hits that balance.

What I’d Do Next (If You Want to Push This Further)

Given your background, this is where it gets interesting:

1. Build a Mobile → Agent Bridge

Phone handles prompts locally
Escalates complex tasks to your OpenClaw backend

2. Local RAG on Mobile

Index notes / PDFs on-device
Use Gemma as query layer

3. Trading / Quant Use Case (Lightweight)

Quick portfolio queries
Market summaries (cached locally)
Decision journaling

4. Telegram Bot + Mobile LLM Hybrid

Local inference first
Cloud fallback only when needed

Final Verdict

Dimension	Score
Innovation	8/10
Practicality	9/10
Performance	7.5/10
UX	9.5/10
Strategic impact	9/10

This is the direction everything is going.

Not bigger models.
Not more GPUs.

Smarter distribution of intelligence across edge + cloud.

Closing Thought

If this trajectory continues:

Your phone becomes your primary AI interface
Your laptop becomes your agent orchestration layer
The cloud becomes optional, not required

That’s a very different world than where we were even 12 months ago.

Build a Simple MCP Agent with OpenAI: Describe and OCR Local Images

noreply@blogger.com (Unknown) — Sun, 05 Apr 2026 13:31:00 +0000

If you want a practical MCP example that is easy to run and easy to explain, image analysis is a great fit.

Source code: JordiCorbilla/mcp-image-analysis-agent: mcp-image-analysis-agent

In this walkthrough, we will build a minimal Model Context Protocol (MCP) setup:

An MCP server that exposes tools to list images, download files, and analyse local images
An MCP client that uses OpenAI tool-calling to decide when to call that tool
A CLI flow that returns both:
- image description
- extracted text (OCR)

This example is intentionally small so you can publish it, teach it, and extend it quickly.

What You Will Build

You will run two Python programs:

server.py: MCP server with practical tools for image workflows
client.py: OpenAI-powered agentic client that connects to the server and can invoke MCP tools

The interesting part is the split of responsibilities:

The client handles conversation, model calls, and tool orchestration.
The server handles local file access safely and image analysis logic.

Why This Is a Good MCP Example

MCP is most useful when a model needs controlled access to local capabilities. In this project:

The model cannot directly read files.
The model must call a tool exposed by your MCP server.
The server validates paths and performs the action.

That makes this architecture both practical and safer than giving direct unrestricted local access.

Prerequisites

Python 3.10+
OpenAI API key
A local image file in the lab folder (for example sample.png)

Project Files

client.py
server.py
requirements.txt

Install Dependencies

From the lab folder:

python -m pip install -r requirements.txt

Set Environment Variables

Use a .env file in the same folder:

OPENAI_API_KEY=sk-your-key-here
OPENAI_MODEL=gpt-4.1-mini
# Optional override for server-side vision model
OPENAI_VISION_MODEL=gpt-4.1-mini

Run the Demo

Start the client and point it to the server:

python .\client.py .\server.py

From the menu:

Choose Analyse Local Image (Description + OCR)
Enter an image path relative to the project folder
Optionally add a focus instruction (for example: extract receipt totals)

How the Agentic Loop Works

The client does this repeatedly:

Sends user query + available MCP tools to OpenAI.
If model requests tool calls, client executes them through MCP.
Tool results are appended back into the conversation.
Model returns final answer.

That means the model decides when to use tools, while your server decides how tools operate.

Core MCP Tools

The server exposes these core tools:

list_images_in_current_folder()
download_file_from_url(url, output_path="")
analyze_local_image(file_path, focus="")

1) List Images in the Current Folder

Use this when you want the agent to discover available images before analysis.

Example request:

List images available in this folder.

Returned data includes:

count
images[] with name, relative path, size, and modified timestamp

2) Download a File from a URL

Use this to bring an image into your project folder before analysis.

Example request:

Download https://example.com/photo.jpg to downloads/photo.jpg.

The tool restricts downloads to http/https and saves only inside your project directory.

3) Analyse a Local Image

This performs description + OCR in one pass.

Behaviour:

Validates the path stays inside the project directory.
Checks that the file exists and is a supported image format.
Converts image bytes to a data URL.
Sends image + instruction to OpenAI Vision.
Returns structured output:
- description
- extracted_text
- notable_details

Example Prompt

User asks:

Analyze image sample.png. Use MCP tools. Return a short description, extracted text, and two useful follow-up actions.

You can also run a
two-step flow:

Ask the agent to list images with list_images_in_current_folder.
Pick one and ask the agent to analyse it with analyze_local_image.

Possible response:

Description: A photographed whiteboard with sprint tasks and dates.
Extracted text: "Sprint 12, Demo Friday, Fix auth timeout"
Follow-up actions:
- Convert items into a task checklist.
- Create calendar reminders for deadline lines.

Error Handling Included

This sample handles common failures cleanly:

Missing OPENAI_API_KEY
Invalid image path
Path traversal attempts outside project folder
Unsupported image extension
Non-JSON model output fallback

Security Notes

Even in a demo, do not skip path validation.

The server uses a safe resolver that enforces all tool file access under the project root. This prevents prompts from tricking your tool into reading arbitrary files outside the workspace.

Cost and Model Tips

Start with gpt-4.1-mini for lower cost.
For higher OCR fidelity, test a stronger model in OPENAI_VISION_MODEL.
Keep outputs concise to reduce tokens.

Troubleshooting

`OPENAI_API_KEY is not set`

Add the key to .env or set it in your shell, then restart.

`Image file not found`

Use a path relative to the project folder where you run the client.

`Unsupported image extension`

Use one of: .png, .jpg, .jpeg, .webp, .gif, .bmp, .tif, .tiff.

No OCR text returned

Some images contain no readable text or low-quality text. Try a clearer image or add focus guidance.

Extension Ideas

Once this baseline works, you can add:

Batch image processing folder mode.
Structured OCR outputs (fields for receipts/invoices/forms).
A second tool to summarize OCR into action items.
Markdown report generation from analysis results.

Final Thoughts

This project is a compact, publishable MCP example that demonstrates real agent behaviour:

OpenAI handles reasoning and tool decisions.
MCP exposes local capabilities safely.
You get immediately useful output from local images.

If you are teaching MCP, this is a strong first demo before moving on to multi-tool, multi-agent workflows.

Rowlens: finding one exact CSV row in a 17 GB file without blowing up memory

noreply@blogger.com (Unknown) — Sun, 29 Mar 2026 15:48:00 +0000

Most CSV tooling assumes the file is still small enough to load, sort, or inspect interactively. That breaks down fast once the file is measured in gigabytes. rowlens was built for the opposite case: a CLI you can point at a very large CSV when you already know the clues you are looking for and you want the exact matching rows back in a readable format.

The design constraint was simple: never read the full file into memory. rowlens opens the CSV as a stream, reads the header once, and then processes each record one at a time with Python's standard csv reader. That keeps memory usage effectively flat even when the input file is 17 GB or more. The tool does not build an index, cache rows, or attempt an in-memory dataframe workflow. It just walks the file once and stops early if you set --max-results.

The matching model is intentionally narrow for version 1.0. Repeated --keyword arguments are treated as exact cell-value matches. Repeated --filter arguments are treated as substring checks. A row is returned only when it satisfies every supplied condition. That gives a useful two-stage search pattern in practice: use --keyword for the hard identifier, then --filter to narrow the surrounding context.

Example:

rowlens --file "huge.csv" --keyword "1213131" --filter "AAA" --output "results.txt"

If a row matches, the output is not dumped as raw CSV. Instead, rowlens renders a bordered CLI report with summary metadata at the top and then a per-match table that lists each column name beside its value. That matters more than it sounds. When you are debugging a production extract or validating a record in an enormous export, the important part is not just finding the row. The important part is understanding the row immediately.

That output style was inspired by terminal-first tooling such as csv-stream-diff: strong borders, obvious sections, and tabular structure that still works in plain text when redirected to a file. The same rendered report can be written with --output, which makes it easy to attach to a ticket, share in chat, or keep as a trace artifact from an investigation.

Under the hood, Python was a good fit here because the problem is mostly streaming IO plus deterministic row checks. The package is structured with Poetry from the start so it can be shipped cleanly to PyPI. The CLI entry point lives behind the rowlens console script, dependencies are minimal, and the test suite covers the main behaviors: combined keyword and filter matching, case-insensitive search, support for extra cells beyond the header, and output-file generation.

Version 1.0 is deliberately focused. It solves one job well: find the exact row you care about in a file too large for the usual tools, then present it cleanly. Future iterations could add column-scoped matching, JSON output, compressed input support, or richer summary stats. But the first release already hits the core operational need: streaming search for massive CSVs with output that humans can read immediately.

Building sqlcsv-exporter: a Python CLI to Stream SQL Server Data into CSV

noreply@blogger.com (Unknown) — Sat, 28 Mar 2026 22:00:00 +0000

I recently built a small Python package called sqlcsv-exporter to solve a very practical problem: export the results of a SQL Server query into a CSV file, from the command line, without dragging an entire dataset into memory.

The tool is packaged as a Poetry project, designed to be publishable to PyPI, and focused on a workflow that is common in reporting and data operations teams:

store the SQL in a .sql file
pass connection details at runtime
stream rows in chunks
write the CSV incrementally
show visible progress while the job runs

This post walks through why I built it, how it works, and what I learned while testing it against a local SQL Server instance.

The Problem

The starting point was familiar: a one-off Python script that connected to SQL Server, ran a query, and wrote a CSV. That works for quick experiments, but it gets messy fast.

Typical issues show up immediately:

query text is embedded inside Python instead of living in a .sql file
authentication logic is mixed into export logic
the script is hard to reuse across environments
large result sets become memory-heavy
the UX is poor when a query runs for several seconds or several minutes

The goal was to replace that with a proper CLI package that feels like a real tool rather than a disposable script.

The Goal

I wanted a command like this:

sqlcsv-exporter `
  --sql .\queries\report.sql `
  --output .\exports\report.csv `
  --server "DESKTOP-TTUSQLJ\SQLEXPRESS" `
  --database QuantDevTest

The requirements were straightforward:

read SQL from file
connect to SQL Server with trusted auth or SQL auth
stream the result set with fetchmany()
write CSV rows incrementally
avoid pandas for the write path
provide useful progress output
keep the package testable and publishable

The Package Structure

I split the project into small modules so each concern is isolated:

cli.py: argument parsing and process exit behavior
config.py: config model and validation
connection.py: ODBC connection string and connection creation
sql_rewriter.py: SQL file loading and optional variable rewriting
exporter.py: query execution, chunked fetch, CSV writing, and progress rendering

That separation matters because it makes the code easier to test. The CSV writer can be tested without a database. The SQL rewriting logic can be tested with simple strings. The export workflow can be tested with mocked connections and cursors.

Why Streaming Matters

The core design choice was to stream results instead of loading everything into memory.

The exporter:

opens the SQL file
optionally rewrites a declared date variable such as @InAsOfDate
executes the query with pyodbc
fetches rows in chunks
writes each chunk directly to a CSV file

That keeps the memory profile stable even when the result set gets large.

Instead of doing something like this:

rows = cursor.fetchall()
writer.writerows(rows)

the tool does this conceptually:

while True:
    rows = cursor.fetchmany(chunk_size)
    if not rows:
        break
    writer.writerows(rows)

That small change is the difference between a toy exporter and something you can trust on real datasets.

A Better CLI Experience

Command-line tools that do database work have one recurring UX problem: silence. If a query takes ten seconds, users start wondering whether the command is stuck, blocked, or dead.

So the exporter prints:

a short run summary before execution
detected column count after the query starts returning metadata
a live progress display while rows are written
a final completion summary with rows, columns, file size, and elapsed time

I used rich for the terminal output instead of hand-rolled print() calls. It gives the tool a more deliberate interface without turning it into a full TUI.

Handling SQL as a File

One design decision I liked immediately was keeping SQL in a separate file. That brings a few benefits:

the query is easier to inspect and review
analysts or DBAs can edit SQL without touching Python code
the same CLI can be reused for many exports
long or multi-step SQL scripts remain readable

For example, this simple test query lives in its own file:

SELECT TOP (1000)
    [Id],
    [Stock],
    [LowPrice],
    [MaxPrice],
    [AvgPrice]
FROM [QuantDevTest].[dbo].[StockMetrics];

That file can be passed directly to the CLI without changing the Python package at all.

Optional SQL Rewriting

Some teams keep parameter values in SQL scripts using a declared variable like:

DECLARE @InAsOfDate DATE = '2026-03-01';

To support that style, the tool includes a lightweight SQL rewriting step. If the variable exists, the CLI can replace its value based on --date. If the variable does not exist, the SQL runs unchanged.

This keeps the interface flexible:

SQL files can remain mostly static
operators can adjust date-driven exports without manually editing the file

It is intentionally narrow in scope. It is not trying to become a generic SQL templating engine.

A Real-World Bug Caught by Live Testing

The first end-to-end run against my local SQL Server instance did not succeed.

The CLI connected successfully, but then failed with:

AttributeError: 'pyodbc.Cursor' object has no attribute 'timeout'

That was useful. The mocked tests passed, but the live run exposed a compatibility assumption in the code. Some pyodbc environments expose timeout behavior differently.

The fix was simple:

try setting the timeout on the cursor if that attribute exists
otherwise fall back to the connection object

That is exactly why live testing matters. A tool can look clean in unit tests and still fail in a real environment for reasons that only surface with the actual driver and actual database connection.

Stress Testing with a Slower Query

A fast export is not a great demo for progress reporting, so I added a dedicated slow test query.

It:

copies base rows into a temp table
creates two multiplier temp tables
cross joins them to expand the result set
adds a short WAITFOR DELAY

That makes it easy to simulate a longer-running export without needing a large production-sized table.

Conceptually, the query looks like this:

SELECT TOP (1000) ... INTO #BaseStockMetrics ...
SELECT TOP (200) ... INTO #MultiplierA ...
SELECT TOP (200) ... INTO #MultiplierB ...

WAITFOR DELAY '00:00:03';

SELECT ...
FROM #BaseStockMetrics AS b
CROSS JOIN #MultiplierA AS a
CROSS JOIN #MultiplierB AS c;

With a small base table, this is enough to create a large CSV and make the progress output visible. With a large base table, it can scale very aggressively, so it is the kind of test query you use carefully.

Testing Strategy

The package includes pytest coverage around the parts that are most likely to break:

config validation
date resolution
SQL file loading
SQL variable rewriting
connection string generation
CSV writing
mocked end-to-end export behavior

That balance is important. Not every code path needs a deep integration test, but the components that define correctness should have coverage.

The live SQL Server run complements those tests by exercising:

the actual ODBC driver
the actual SQL Server instance
actual filesystem writes
real CLI behavior

Packaging for PyPI

The project is set up as a Poetry package and versioned as 1.0.1.

Build artifacts are created with:

poetry build

That produces:

a source distribution
a wheel

Those are the exact artifacts needed for publishing to PyPI.

Example Usage

Trusted connection:

poetry run sqlcsv-exporter `
  --sql .\queries\stock_metrics_top_1000.sql `
  --output .\exports\stock_metrics_top_1000.csv `
  --server "DESKTOP-TTUSQLJ\SQLEXPRESS" `
  --database QuantDevTest

SQL authentication:

poetry run sqlcsv-exporter `
  --sql .\queries\stock_metrics_top_1000.sql `
  --output .\exports\stock_metrics_top_1000.csv `
  --server "my-server" `
  --database "Reporting" `
  --sql-auth `
  --username "report_user" `
  --password "secret"

Final Thoughts

This is not a huge project, but it is a useful example of turning a script into a tool.

The interesting part was not the CSV writing itself. It was the engineering around it:

separating concerns into modules
choosing streaming over eager loading
designing a usable CLI
writing tests around the core behavior
validating the tool against a real SQL Server environment

That is usually the difference between “something that works on my machine” and “something I would actually hand to another team.”

If I extend it further, the next areas I would look at are:

environment-variable support for connection defaults
optional gzip output
row count estimates before execution when possible
better handling for very wide result sets
structured logging for scheduled runs

For now, sqlcsv-exporter does the job it was meant to do: run a SQL Server query from a file and stream the results into CSV in a way that is practical, testable, and ready to package.

Building csv-stream-diff: A Fast, Streaming CSV Comparison Tool for Very Large Files

noreply@blogger.com (Unknown) — Sun, 22 Mar 2026 20:30:25 +0000

Comparing two CSV files sounds simple until the files are no longer small.

Once the datasets move into the millions of rows, the usual approaches start to fall apart. Loading both files into memory is expensive. Spreadsheet tools stop being useful. Even many ad hoc scripts become slow, fragile, or impossible to run reliably in production-like environments.

That is the problem I wanted to solve with csv-stream-diff.

The Problem

In real systems, CSV comparison is rarely just "diff these two files."

Usually the job looks more like this:

The files are large enough that a full in-memory load is risky or impossible
The key columns on the left and right files do not use the same names
The comparison should only consider a selected subset of columns
Duplicate keys may exist and need to be reported clearly
Sometimes a full comparison is required, but sometimes a statistically useful sample is enough
The output needs to be machine-readable so it can feed downstream validation or remediation workflows

I wanted a tool that could handle that cleanly, with minimal dependencies, and still be easy to package and run anywhere.

What `csv-stream-diff` Does

csv-stream-diff is a Python CLI tool for comparing very large CSV files using:

streaming reads
hash-based partitioning
multiprocessing
YAML-driven configuration

It produces structured output files for:

rows only on the left
rows only on the right
rows with cell-level differences
duplicate keys
summary metadata

It is designed to be practical rather than clever.

The Core Design

The main design constraint was memory.

If a tool tries to build a single giant in-memory index for both files, it will eventually hit a limit. So instead of comparing the full files at once, csv-stream-diff uses a two-phase approach.

1. Partition both files into hashed buckets

Each row key is normalized and hashed into a bucket. The left and right files are streamed row by row and written into matching bucket files on disk.

That matters because rows with the same normalized key always land in the same bucket. Once that is true, each bucket can be compared independently.

2. Compare buckets in parallel

After partitioning, the tool compares bucket pairs using multiple worker processes. Each worker only needs to index one bucket of the left file at a time, not the entire dataset.

This keeps memory bounded while still taking advantage of all available CPU cores.

The result is a design that scales much better for heavy loads than a naïve single-process implementation.

Why YAML Configuration

I did not want the CLI to become a wall of flags.

The comparison usually needs several pieces of information:

the left and right file paths
the left and right key columns
the left and right comparison columns
CSV dialect options
output paths
sampling settings
performance settings

That is much easier to manage in a YAML file than on the command line.

The CLI still supports overrides, but the YAML file is the primary contract. That makes runs reproducible and easier to version alongside data validation jobs.

Exact Sampling for Large Validation Runs

Sometimes you do not want to compare every row.

For example, if the source files contain tens of millions of records, you may want to run a fast validation pass against an exact random sample of keys before committing to a full comparison.

csv-stream-diff supports that:

sampling.size: 0 means compare everything
sampling.size > 0 means compare an exact random sample of left-side unique keys
sampling.seed makes the sample reproducible

This gives you a useful middle ground between tiny spot checks and full heavy-load comparisons.

Handling Duplicate Keys

Duplicate keys are one of the most annoying edge cases in file comparison work.

If a key appears multiple times, the comparison becomes ambiguous. Instead of failing silently or hiding the problem, the tool reports duplicate keys explicitly and continues using the first occurrence for the main comparison.

That behaviour is deliberate:

you get a warning
you get a separate duplicate-key artifact
you still get a usable comparison result

This makes the tool better suited for messy real-world data.

Keeping Dependencies Small

I wanted the runtime dependency footprint to stay minimal.

The tool is built mostly with the Python standard library. The only runtime dependency is PyYAML, which is used for configuration loading.

That keeps installation simple and reduces operational friction when the tool needs to run in different environments.

Outputs That Are Actually Useful

One important goal was to avoid producing a human-only report.

The tool writes separate output files for each class of result, which makes it easier to automate downstream processing:

only_in_left.csv
only_in_right.csv
differences.csv
duplicate_keys.csv
summary.json

The differences.csv file is especially useful because it reports cell-level differences with both the left and right column names and values.

That means you can do more than say "this row changed." You can say exactly how it changed.

Testing the Tool Properly

I also wanted the project to be easy to validate.

So the repository includes:

unit tests with pytest
BDD-style acceptance tests with behave
a fixture generator that creates two baseline-identical CSV files and then introduces controlled differences

The generator makes it easy to create realistic comparison scenarios involving:

changed values
left-only rows
right-only rows
duplicate keys

That is useful both for development and for demonstrating the tool to others.

A Few Practical Lessons

Building this reinforced a few engineering lessons:

For large-file tooling, streaming and partitioning beat clever in-memory shortcuts
Exact sampling is worth implementing properly because it gives a fast validation mode without becoming a toy feature
Duplicate handling should be explicit, not implicit
Machine-readable outputs matter as much as console output
Minimal dependencies make utility tools easier to adopt

Example Usage

With a config file in place, the tool is intentionally simple to run:

python -m csvstreamdiff.cli --config config.yaml

You can also override selected settings from the CLI:

python -m csvstreamdiff.cli --config config.yaml --sample-size 100000 --workers 8

Why I Built It

This project came from a practical need: compare large CSV datasets reliably, with clear outputs, and without depending on heavy frameworks or fragile one-off scripts.

The result is a tool that is meant to be packaged, published, and reused anywhere.

That was the bar from the start.

Closing

If you work with large exports, migration validation, reconciliation jobs, or data quality checks, CSV comparison becomes infrastructure very quickly.

csv-stream-diff is my attempt to make that infrastructure solid:

reproducible
scalable
explicit
easy to automate

If you want to explore the project, the repository includes the CLI, example configuration, test generator, and packaging setup needed to build and publish it.

Coding from Telegram: my OpenClaw + Ollama setup

noreply@blogger.com (Unknown) — Sun, 15 Mar 2026 12:44:00 +0000

I finally got a setup working that feels a bit like the future.

I launched OpenClaw with:


ollama launch openclaw --model kimi-k2.5:cloud

and connected it to Telegram, which means I can now talk to my coding assistant directly from my phone.

What makes this especially satisfying is that the whole thing is running on a very inexpensive mini PC at home:

Mini PC, 12GB RAM + 128GB ROM, Intel Celeron J4125 (up to 2.7 GHz), Windows 10 Pro, dual Wi-Fi 2.4/5G, Bluetooth 4.2, 4K HD, 2 HDMI + 1 VGA.

Nothing exotic. Nothing expensive. Just a small, affordable box quietly sitting there, acting as the bridge between my workspace and my phone.

That is part of the appeal. This setup feels surprisingly capable for something that costs very little and, beyond the hardware I already own, is effectively free to run.

The interesting part is not just that it chats back. It has access to my workspace, can help create projects, work with files, and act more like a real development companion than a simple question-answer bot. That changes the experience completely. It stops being something I occasionally open in a browser and starts feeling more like an always-available engineering copilot.

What I like most is the shift in how and where coding can happen.

I can be away from my desk, out doing groceries, commuting, or just walking around, and still make progress. I can send a quick message to scaffold an idea, create a project structure, inspect something in the workspace, or prepare the next task before I am back in front of the machine. It turns dead time into useful time.

That is the part that feels powerful: development becomes more ambient.

Instead of waiting for the perfect focused block at a desk, I can stay connected to the flow of a project throughout the day. A thought appears, I message it. A small task comes to mind, I delegate it. A project idea starts forming, I capture it immediately. By the time I sit down again, some of the groundwork is already done.

There is also something elegant about the stack itself. Ollama provides a clean way to launch the model workflow, OpenClaw gives the agent a useful operating surface, and Telegram becomes the lightweight interface that makes it all feel natural. No heavy setup on the phone, no friction, no need to open a laptop just to move a task forward.

The other thing I genuinely like is how accessible it is. You do not need a powerful workstation, a costly hosted service, or a complex mobile setup. A cheap mini PC, a Telegram account, and the right tooling are enough to build something that feels surprisingly close to a personal remote engineering assistant.

It is still early, of course, and this kind of workflow needs trust, guardrails, and careful handling when it comes to file access and automation. But even in its current form, it already feels genuinely useful.

A coding assistant in Telegram sounds like a gimmick until it starts helping you build real things while you are standing in a supermarket queue.

That is when it stops sounding like a demo and starts feeling like a new way of working.

Tina the Destroyer: Turning Family Photos Into a Playable Arcade Game

noreply@blogger.com (Unknown) — Sun, 08 Mar 2026 15:39:00 +0000

I built a small local arcade game called Tina the Destroyer with Python and Pygame.

The idea was simple and personal: I took pictures of my daughter, converted them into game-ready pixel sprites, and built a one-screen game around that character. The result is a fast arcade loop where Tina smashes falling rats before they hit the ground.

The Core Idea

I wanted a game that was:

Fast to play
Easy to understand
Personal and fun

So I designed a tight gameplay loop:

Move left and right
Press Space to perform a ground smash
Squash rats for points
Lose a life when a rat reaches the bottom
Game over after 3 missed rats

From Photos to Sprites

The visual pipeline started with real photos. I cleaned and adapted the character images to work as sprite sheets and exported key poses:

player_idle.png
player_smash.png

I also prepared enemy and effect assets:

rat.png
rat_squashed.png
impact.png
title.png

For audio, I added smash, miss, and looped music effects, with fallback loading so the game still runs if a file is missing.

Building the Game

The game is organized into small modules:

main.py entrypoint
game.py state machine and game loop
entities.py player, rats, and impact effects
settings.py tuning values
utils.py asset loading and fallbacks

Key features I focused on:

Clean state flow (TITLE, PLAYING, GAME_OVER, QUITTING)
Responsive movement and smash timing
Difficulty scaling over time
Score/lives HUD
Best-score persistence to JSON
Graceful fallback behavior for missing assets

Vibe Coding the Project

I vibe coded this with gpt-5-3.codex, iterating quickly on gameplay feel, tuning values, and production details (asset fallbacks, restart flow, audio handling, and UI polish).

Source code: JordiCorbilla/tina-the-destroyer: tina-the-destroyer

That pairing made it easy to keep momentum: prototype fast, test, adjust, repeat.

Why I Loved This Build

This project sits at the intersection of family creativity and engineering. It started as a fun idea with photos and ended up as a fully playable local arcade game I can run anytime with:

pip install pygame
python main.py

Small scope, real personality, and a complete game loop. Exactly the kind of build I enjoy.

Building a Portfolio CLI in 2026 (and Why CLIs Feel Cool Again)

noreply@blogger.com (Unknown) — Sun, 15 Feb 2026 21:09:00 +0000

Over a single weekend, I decided to build a portfolio tracker that lives entirely in the terminal. Not a web app. Not a spreadsheet. A real, interactive CLI with panels, charts, and daily tracking.

I also wanted it to fit how I actually work: Excel as the source of truth, but with a console experience that makes daily updates fast and visual.

This was vibe-coded with Chat GPT-5-2-Codex over a weekend, which made it surprisingly fun to iterate on UI tweaks, edge cases, and the overall feel.

Why a CLI?

CLIs are back. Not because they're nostalgic, but because they're fast, distraction-free, and feel like tools instead of products. A good CLI lets you focus on the data and decisions rather than the UI chrome.

What I Built

The result is my-portfolio-cli, an interactive portfolio dashboard that:

Reads data from an Excel workbook.
Renders a rich TUI with Spectre.Console.
Shows daily PnL, MTD performance, and FY summaries.
Displays a simple PnL bar chart right in the terminal.
Lets you add a new daily snapshot instantly.
Creates new monthly sheets when you need them.

The Flow

Start it and it opens directly in interactive mode. Use arrow keys to move across days and months. Press A to add a new entry. If the month doesn't exist yet, it creates it automatically.

If you're starting from scratch (no workbook or no month sheets), the CLI prompts you to create the first month, add accounts, and enter initial values.

Design Notes

The key was making it feel like a real console app:

Clean layout with strong visual hierarchy.
Color-coded gains and losses.
A compact FY summary that's readable at a glance.
A minimal bar chart for quick trend cues.

It isn't about being fancy. It's about being useful and enjoyable to use every day.

What's Next

This project is going online soon, which means:

A demo workbook and screenshots.
A short walkthrough.
Possibly a minimal installer or single-file publish.

For now, it's a tight little tool that does exactly what I need.

If you want to build your own CLI tool, I highly recommend it. The feedback loop is short, and it's easy to keep refining the experience until it feels right.

Running Claude Code Fully Locally with Ollama (Yes, It’s Possible — With Caveats)

noreply@blogger.com (Unknown) — Sun, 25 Jan 2026 17:37:00 +0000

Claude Code is designed as a cloud-first, tool-centric coding agent tightly coupled to Anthropic’s infrastructure. Out of the box, it does not support local models.

However, with a combination of:

endpoint redirection,
model aliasing inside Ollama,
and a tool-capable local model,

you can run Claude Code entirely locally against Ollama — including file edits, refactors, and shell tools.

This article documents the exact path that works, the failure modes you will hit, and the engineering trade-offs.

Why Claude Code Is Hard to Run Locally

Claude Code makes three strong assumptions:

Anthropic authentication is always available
Model names are fixed (claude-sonnet-*)
Tool calling is mandatory

Most “local Claude” guides fail because they only solve one of these.

To succeed, you must solve all three.

Architecture That Actually Works


VS Code
 └─ Claude Code extension
     ├─ hardcoded model name (claude-sonnet-*)
     ├─ mandatory tools
     └─ Anthropic-style requests
           ↓
      Ollama (localhost)
           ↓
   Tool-capable local model (Qwen)

Key insight:

You don’t change Claude Code — you adapt Ollama to look like Anthropic.

Step 1: Redirect Claude Code to Ollama

In VS Code Command palette -> Preferences: Open User Settings (JSON):


{
  "claudeCode.environmentVariables": [
    {
      "name": "ANTHROPIC_BASE_URL",
      "value": "http://127.0.0.1:11434"
    },
    {
      "name": "ANTHROPIC_AUTH_TOKEN",
      "value": "ollama"
    }
  ],
  "claudeCode.disableLoginPrompt": true
}

This bypasses cloud calls and sends all traffic to Ollama.

At this point:

UI works
Requests flow
But you will get 404 model not found

That is expected.

Step 2: Understand the Model Name Problem

Claude Code always sends model names like:


claude-sonnet-4-5-20250929

Ollama does not have such models.

You cannot configure Claude Code to use gemma, llama, or qwen directly.

Solution: Model aliasing

Ollama lets you create lightweight aliases that map one model name to another.

Step 3: Why Gemma Fails (Important)

Gemma models (including gemma3n) do not support tool calling.

Claude Code always sends a tools schema.

Resulting error:


does not support tools

This is a hard failure, not a timeout or config issue.

Conclusion

Gemma works for chat, not for Claude Code.

Step 4: Use a Tool-Capable Model

You need a model that supports:

structured tool calls
JSON responses
streaming + edits

Step 5: Create Claude-Compatible Aliases

Create a Modelfile:


FROM qwen2.5-coder:7b

Then create aliases Claude Code expects:


ollama create claude-sonnet -f Modelfile
ollama create claude-opus   -f Modelfile
ollama create claude-haiku  -f Modelfile

No weights are duplicated.
Only metadata is created.

Now Ollama can answer requests for claude-sonnet-*.

Result: Fully Local Claude Code

At this point, Claude Code can:

create files
edit files
run shell commands
refactor code
reason about repos

All without cloud access.

Building a Multi‑Agent Finance Chatbot with AG2

noreply@blogger.com (Unknown) — Sun, 12 Oct 2025 19:23:00 +0000

Financial conversations often touch on complex markets, risk preferences and compliance constraints. A single large‑language model may struggle to handle every nuance, so AG2 (formerly AutoGen) offers a multi‑agent architecture where each agent specializes in part of the workflow. This article demonstrates how to create a finance chatbot using AG2 and Python. We will define specialised agents, orchestrate them via a group chat and show a sample interaction. The approach is inspired by multi‑agent healthcare chatbots but adapted to investment discussions.

Why multi‑agent chatbots?

Recent tutorials explain that multi‑agent chatbots divide complex interactions into specialised roles. Each agent focuses on one task—answering questions, providing recommendations or analysing data—so the overall system can handle nuanced conversations more accurately (analyticsvidhya.com). The AG2 documentation notes that multi‑agent architectures are easier to maintain and extend; they allow developers to add or remove functionality without redesigning the entire system (docs.ag2.ai). Multi‑agent designs also support different conversation patterns (sequential, group or nested chat) and enable tool use and code execution (docs.ag2.ai). In regulated industries such as healthcare or finance, this modularity improves accuracy and reduces risk (docs.ag2.ai).

Another advantage is practicality. Building a chatbot for stock‑market questions is complex, but packages like Chainlit and AG2 simplify the process by handling chat orchestration and tool calls (tinztwinshub.com). These frameworks allow you to run large‑language models locally or through an API and to register Python functions as tools (tinztwinshub.com). As open‑source models like Meta’s Llama 3.1 gain popularity, developers can deploy multi‑agent systems on local machines (tinztwinshub.com).

Pros and cons

Aspect	Pros	Cons & Considerations
Modularity	Each agent has a focused role, making the system easier to extend or modify (docs.ag2.ai).	More agents increase the number of interactions to manage and may introduce coordination overhead.
Specialisation	Domain‑specific agents can outperform a monolithic model on their task (analyticsvidhya.com).	Requires careful prompt design to keep agents within their remit and avoid conflicting advice.
Resource usage	Multi‑agent architectures can run cheaper models for some roles while leveraging larger models only where needed (docs.ag2.ai).	Running multiple models in parallel increases latency and can be costly for high‑volume workloads.
Transparency	By exposing distinct agent responses, users can understand how conclusions are derived (e.g., one agent summarises market data, another suggests allocations).	In some contexts users may prefer a single, unified answer; too much detail may confuse non‑experts.

Defining the finance chatbot agents

Our finance chatbot will engage the user to understand their goals, analyse market conditions and provide general asset‑allocation suggestions. It is not a licensed financial adviser; recommendations are for educational purposes. We adapt the roles from a mental‑health chatbot example into finance:

Client Agent – Captures the user’s financial situation, risk tolerance and investment goals.
Market Analysis Agent – Analyses the market based on user input. It summarises macro‑economic trends, asset‑class performance and volatility without giving personalised advice.
Portfolio Recommendation Agent – Suggests a high‑level asset allocation based on the analysis and user profile. It does not trade or recommend specific securities.

The roles can be summarized as follows:

Agent	Purpose	Key instructions
Client Agent	Collects information about the user’s financial goals, investment horizon and risk tolerance.	Ask the user about their objectives, constraints and any particular interests or concerns.
Market Analysis Agent	Summarises market conditions, including recent trends in equities, bonds, commodities and macro‑economic indicators.	Do not provide personal investment advice; instead, report high‑level insights such as whether markets are volatile or stable.
Portfolio Recommendation Agent	Provides general asset‑allocation suggestions based on the user’s profile and the market summary.	Recommend broad categories (e.g., 60 % equities, 30 % bonds, 10 % alternatives); include a disclaimer that this is not financial advice.

Setting up the environment

To follow along you need Python 3.11+ and the ag2 library. You can install AG2 via pip (pip install ag2[openai]). If you plan to run local models, install an API wrapper like ollama and download a model such as Llama 3.1 (tinztwinshub.com). You will also need API keys if using cloud models. In this example we configure the LLM via llm_config without specifying an API key; supply your own key as needed.

Implementing the agents

The following Python code illustrates how to create the finance chatbot. It borrows the group‑chat structure from the mental‑health example and uses AG2’s ConversableAgent, GroupChat and GroupChatManager classes:


from autogen import ConversableAgent, GroupChat, GroupChatManager

# Replace None with your OpenAI or local model API key if required
llm_config = {"config_list": [{"model": "gpt-4o", "api_key": None}]}

# 1. Client Agent: collects user goals and risk tolerance
client_agent = ConversableAgent(
    name="client",
    system_message=(
        "You are the client interface.\n"
        "Your job is to describe your financial goals, risk tolerance, investment horizon, "
        "and any budget constraints or ethical considerations."
    ),
    llm_config=llm_config
)

# 2. Market Analysis Agent: summarises current market conditions
market_analysis_agent = ConversableAgent(
    name="market_analysis",
    system_message=(
        "You analyse the current market based on the client's input."
        "Summarise macroeconomic factors (interest rates, inflation), sector trends and recent market volatility. "
        "Do not provide personalised investment advice; simply report high-level observations."
    ),
    llm_config=llm_config
)

# 3. Portfolio Recommendation Agent: proposes a high-level allocation
portfolio_recommendation_agent = ConversableAgent(
    name="portfolio_recommendation",
    system_message=(
        "You suggest a general asset allocation based on the analysis from the Market Analysis Agent."
        "Consider the client's risk tolerance and goals."
        "Recommend proportions across asset classes such as equities, bonds, and alternatives."
        "Include a disclaimer that the suggestions are educational and not financial advice."
    ),
    llm_config=llm_config
)

# Configure a group chat where the analysis and recommendation agents converse
# The manager orchestrates turn-taking in round‑robin fashion
finance_groupchat = GroupChat(
    agents=[market_analysis_agent, portfolio_recommendation_agent],
    messages=[],
    max_round=3,
    speaker_selection_method="round_robin"
)

manager = GroupChatManager(name="manager", groupchat=finance_groupchat)

# Function to start the finance chatbot

def start_finance_chat():
    print("\nWelcome to the AI Finance Chatbot!")
    user_input = input("Please describe your financial goals and risk tolerance: ")

    # The client agent initiates the conversation with the manager
    print("\nAnalysing your profile and market conditions...")
    response = client_agent.initiate_chat(
        manager,
        message=(
            f"My financial goals and risk tolerance are: {user_input}. "
            "Based on this, what insights and recommendations can you provide?"
        )
    )

    # Fallback in case the first response is empty
    if not response:
        response = portfolio_recommendation_agent.initiate_chat(
            manager,
            message="Based on the client's goals, please provide portfolio recommendations."
        )

# Run the chatbot
start_finance_chat()

How it works

User input – The client_agent collects the investor’s goals and risk tolerance. In a real deployment you might ask follow‑up questions to ensure clarity.
Market analysis – The market_analysis_agent reviews the input and provides a concise summary of current market conditions. It does not provide personalised advice, reflecting a common compliance requirement for general market commentary.
Portfolio suggestion – The portfolio_recommendation_agent uses the market summary and client profile to recommend an asset allocation. It emphasises diversification and includes a disclaimer.
Group chat management – The GroupChatManager coordinates the exchange. We use a round‑robin method to ensure both agents contribute (analyticsvidhya.com). The max_round parameter prevents infinite loops.

Sample interaction

Suppose a user says: “I’m saving for retirement in 20 years, have moderate risk tolerance and want to invest ethically.” The agents might respond as follows:

Market Analysis Agent: “Based on the current environment, central banks are signalling that interest rates may remain high to curb inflation, equities have been volatile with technology and renewable‑energy sectors outperforming, and bond yields offer moderate returns. Commodities like gold have appreciated due to geopolitical tension.”

Portfolio Recommendation Agent: “Given your 20‑year horizon and moderate risk tolerance, a diversified allocation could be 60 % equities (with a tilt towards ESG‑focused funds), 30 % bonds (combining government and investment‑grade corporate bonds), and 10 % alternatives such as real‑estate investment trusts or commodities. This is general information and not personalised financial advice.”

In a real‑world application you could register functions that fetch real‑time market data or compute portfolio statistics using AG2’s register_function API (tinztwinshub.com), but our simple example relies on the language model.

Pros and cons of this approach

Pros:
- Modularity and specialisation – Each agent has a clear purpose, which can improve the relevance and quality of responses (docs.ag2.ai). For example, one agent summarises macro data while another focuses on portfolio construction.
- Ease of extension – You can add agents to handle regulatory checks, tax considerations or portfolio rebalancing without rewriting existing components (docs.ag2.ai).
- Transparency – Users can see how the system arrives at recommendations because each agent’s contribution is visible (docs.ag2.ai).
Cons:
- Coordination overhead – More agents mean more messages and potential latency. Tuning max_round and speaker selection helps avoid long conversations.
- Prompt management – Agents must be carefully prompted to avoid stepping outside their roles. If the market‑analysis agent gives advice, it could cause compliance issues.
- Resource usage – Running multiple models or using powerful models for each agent can be costly. Techniques like mixing cheap and expensive models or limiting context length can help (docs.ag2.ai).

Future enhancements

Real‑time data integration: Register functions to call APIs (e.g., yfinance, Bloomberg, or custom data feeds) so the analysis agent can deliver up‑to‑the‑minute market summaries. The Tinz Twins example shows how to plot year‑to‑date gains using a registered function (tinztwinshub.com).
Regulatory compliance: Add an agent that checks whether recommendations adhere to regional regulations (e.g., MIFID II or SEC guidelines). It could flag issues before suggestions reach the user.
Advanced conversation patterns: Use nested or constrained group chats to handle multi‑step workflows, such as gathering risk questionnaires, performing Monte‑Carlo simulations and then generating a report. AG2 supports these patterns (docs.ag2.ai).
Multi‑modal output: Integrate charting tools to visualise asset allocations or risk/return trade‑offs. For instance, register a function that plots allocation pie charts and returns an image.

Conclusion

AG2’s multi‑agent architecture makes it straightforward to build tailored chatbots for complex domains such as finance. By dividing the conversation into specialised roles, developers can achieve greater accuracy, transparency and extensibility compared with monolithic chatbots (analyticsvidhya.com). The simple example above illustrates how you can adapt a mental‑health chatbot pattern to a personal‑finance advisor. Although multi‑agent systems introduce coordination challenges and resource overhead, careful design and modern frameworks like AG2 or Chainlit can mitigate these issues (docs.ag2.ai). As open‑source models and tooling evolve, we can expect multi‑agent finance chatbots to become more capable—potentially integrating real‑time data, compliance checks and sophisticated analytics.

Building a MiFID II Compliance Chatbot with CrewAI

noreply@blogger.com (Unknown) — Sat, 11 Oct 2025 18:41:00 +0000

Financial regulation is notoriously complex. Investment firms must interpret and comply with lengthy documents such as the Markets in Financial Instruments Directive (MiFID II) and its companion regulation (MiFIR), while preparing for MiFID III. Compliance officers routinely pore over hundreds of pages of policy statements, ESMA Q&As and Regulatory Technical Standards (RTS). At the same time, regulators are tightening expectations: by September 2025 firms will need to implement the MiFID III amendments and be ready to demonstrate that surveillance and inducement controls work across every channel (luware.comluware.com). Manual search is inefficient and error‑prone.

This article shows how to build a MiFID II compliance chatbot using CrewAI, a multi‑agent framework. You will learn how to organise the workflow, assign tools at the agent or task level, avoid common validation errors, and anticipate upcoming regulatory changes. The techniques apply beyond finance; they illustrate a pattern for building reliable agentic systems that combine retrieval with language models.

Framework
CrewAI orchestrates multiple AI agents to search MiFID-related PDFs and draft compliance answers

Approach
Compare agent‑centric vs task‑centric tool assignment for reliability and predictability

Key Fix
Teach the language model to include the mandatory {"query": "<text>"} argument when calling the PDF search tool

Regulatory Horizon

MiFID III amendments enter force March 28 2024 with a main implementation deadline of September 29 2025 (luware.comluware.com)

New Features

CrewAI’s MCP standardises tool access and data sources (blog.crewai.com) while guardrails and event buses increase reliability (blog.crewai.comblog.crewai.com)

Agent‑centric vs Task‑centric Tool Assignment

CrewAI allows tools—such as a PDF searcher or a web search API—to be provided either to the agent as a global toolbox or to individual tasks. These two patterns affect reliability and maintainability.

Agent‑centric (Generalist)

In this common setup the agent receives a list of tools that it may call at any time. The agent decides which tool to use based on the prompt and its internal reasoning. This is flexible and quick to prototype, but it can lead to mis‑calls. For example, the PDF search tool expects a JSON argument with a query string. Without explicit instructions, the LLM may omit the argument, leading to the familiar Pydantic error:


Error: Arguments validation failed: `query` field required

When the agent owns the entire toolbox, every task must encode tool‑calling rules in its prompt to avoid such errors. In our experience this reduces predictability as the system grows; even seasoned engineers forget to repeat the tool signature in each description.

Task‑centric (Specialist)

The task‑centric approach grants tools only to the tasks that need them. When the agent begins a task it is temporarily given the required tools and cannot access others. This design leads to more predictable tool calls because the agent knows which tools are available for that job and the prompt can focus on how to use them.

In regulated domains this scoping is critical. It reduces the chance that a general‑purpose LLM will call a web search when only document evidence is allowed, and it makes unit testing straightforward: each task can be executed in isolation with its own tools and guardrails (blog.crewai.com).

Pros and Cons

Agent‑centric:
- Pros: Simple configuration; fewer moving parts; agent decides which tool to use.
- Cons: LLM may mis‑call tools without guidance; prompts must repeat tool signatures; difficult to test tasks in isolation.
Task‑centric:
- Pros: Predictable and secure; tools scoped to tasks; easier to unit‑test and update; better adherence to compliance restrictions.
- Cons: Slightly more verbose setup; requires careful mapping of tasks to tools.

Code Walkthrough

Let’s examine the core elements of the chatbot. The code below instantiates an LLM, sets up PDF search tools for each regulatory document, defines an agent with a system prompt and tool‑calling guide, and creates two tasks orchestrated sequentially by a crew. Code can be found here: (langgraph-cookbook/09 - Mifid CrewAI Chatbot at main · JordiCorbilla/langgraph-cookbook)


from crewai import Agent, Task, Crew, Process
from crewai import LLM
from crewai_tools import PDFSearchTool, SerperDevTool

llm = LLM(model="openai/gpt-4o-mini")

web_search_tool = SerperDevTool()

pdf_search_tool = PDFSearchTool(
    pdf="https://www.fca.org.uk/publication/policy/ps17-14.pdf",
    config={ 'embedder': { 'provider': 'huggingface', 'config': { 'model': 'sentence-transformers/all-MiniLM-L6-v2' } } }
)
pdf_search_tool2 = PDFSearchTool(
    pdf="https://www.fca.org.uk/publication/consultation/cp16-19.pdf",
    config={ ... }
)
pdf_search_tool3 = PDFSearchTool(
    pdf="https://www.esma.europa.eu/sites/default/files/library/2016-1452_guidelines_mifid_ii_transaction_reporting.pdf",
    config={ ... }
)

COMPLIANCE_SYSTEM_PROMPT = """
You are a MiFID II / MiFIR compliance analyst. Answer ONLY with information grounded in the provided PDFs.  Always include article‑level citations.  Prefer the latest ESMA RTS for conflicts.  No speculation.
"""

CITATION_FORMAT_PROMPT = """
Format citations as markdown footnotes: [MiFID II Art X(Y)](SOURCE_URL#page=?), [MiFIR Art X(Y)](SOURCE_URL#page=?).
"""

TOOL_CALLING_GUIDE = """
When using "Search a PDF's content", you MUST pass a JSON argument like {"query": "<short search string>"}.  Never omit 'query'.
"""

agent_centric_agent = Agent(
    role="MiFID Compliance Analyst",
    goal="Answer MiFID II / MiFIR questions with exact citations.",
    backstory="Seasoned regulatory analyst trained on MiFID II, MiFIR and FCA/ESMA RTS/Q&A.",
    tools=[pdf_search_tool, pdf_search_tool2, pdf_search_tool3, web_search_tool],
    llm=llm,
    system_prompt=COMPLIANCE_SYSTEM_PROMPT + CITATION_FORMAT_PROMPT + TOOL_CALLING_GUIDE
)

# Define tasks and crew
agent_centric_task = Task(
    description=(
        "Answer '{customer_query}'. First, search the PDFs using 'Search a PDF's content' with {\"query\": \"<your search string>\"}. Extract relevant passages and synthesise the answer."),
    tools=[pdf_search_tool, pdf_search_tool2, pdf_search_tool3],
    agent=agent_centric_agent
)

response_drafting_task = Task(
    description="Use the gathered information to draft a comprehensive response to '{customer_query}'.",
    agent=agent_centric_agent,
    context=[agent_centric_task]
)

task_centric_crew = Crew(
    agents=[agent_centric_agent],
    tasks=[agent_centric_task, response_drafting_task],
    process=Process.sequential
)

The system prompt instructs the agent to answer using only information from the PDFs, include citations, and avoid speculation. We append a tool‑calling guide that explicitly defines the JSON argument required by the PDFSearchTool. Without this guide the agent might call the tool incorrectly, causing the validation error described earlier.

The first task instructs the agent to search the PDFs for relevant text using a JSON query. The second task uses the extracted passages to draft the final answer. By assigning the PDF search tools only to the search task, we prevent accidental web queries during drafting.

Preparing for MiFID III

Our chatbot currently answers questions under MiFID II and MiFIR. However, regulatory evolution continues. MiFID III amendments were published in February 2024, entered into force in March 2024 and must be implemented by September 29 2025 (luware.com). The third iteration does not overhaul the framework but raises the bar on surveillance and inducements: compliance teams must prepare for heightened expectations around off‑channel communications and record‑keeping (luware.com). MiFID III builds on MiFID II by widening the focus to behavioural risks such as conflicts of interest and pricing opacity (luware.com), and regulators expect active monitoring rather than passive archiving (luware.com).

To stay ahead:

Integrate multi‑channel monitoring: capture voice, chat, video and messaging across platforms; apply AI to detect misconduct patterns (luware.com).
Extend the knowledge base: ingest ESMA Q&As, RTS documents and MiFID III draft texts; update embeddings regularly.
Use CrewAI’s guardrails: implement output‑length checks or hallucination detection to ensure answers remain grounded (blog.crewai.com).
Leverage event buses for audit logs and compliance evidence: CrewAI’s event bus emits events such as task started, tool use error, and LLM call completed (blog.crewai.com).
Adopt query rewriting and agentic RAG to improve retrieval. CrewAI now supports query rewriting to focus on relevant keywords and integrates with vector databases like Qdrant and Weaviate (blog.crewai.com).
Plan for MiFID III deadlines: ensure your system can adapt to new PFOF bans, stricter inducement rules, and expanded communication capture (luware.com).

Beyond Compliance – General Lessons

While this capstone project addresses a specific regulatory domain, the design patterns are broadly applicable:

Be explicit about tool signatures: LLMs cannot infer parameter names; include examples and constraints in prompts. For production systems, consider using structured tool calling (JSON Schema) or wrappers that handle input validation.
Scope tools to tasks: Minimises misuse and aids debugging. For high‑impact domains (finance, healthcare) this can be critical for legal and safety reasons.
Instrument your agents: Use CrewAI’s event bus to collect telemetry on tool use and LLM calls. This data helps debug issues and demonstrates compliance (blog.crewai.com).
Stay up to date: Regulations change. Regularly ingest new documents and search the web to supplement your knowledge base. MiFID III emphasises behaviour monitoring, so your system must adapt to detect tone and sentiment across channels (luware.com).
Embrace guardrails and RAG: Guardrails catch excessive or irrelevant outputs; agentic RAG (retrieval augmented generation) tailors the knowledge retrieval to each query, increasing precision and reducing hallucinations (blog.crewai.com).

Conclusion

By combining CrewAI’s multi‑agent orchestration with dedicated tools for PDF and web search, we built a robust MiFID II compliance chatbot. Assigning tools at the task level rather than the agent level yields more predictable and testable workflows. The critical missing‑query error was resolved by teaching the LLM to call tools correctly. Looking ahead, MiFID III will demand even greater surveillance and behavioural oversight; our modular architecture can adapt by adding new documents, query‑rewriting agents, guardrails and event logging.

Agentic AI is evolving rapidly. CrewAI’s adoption of the MCP, guardrails and event bus underscores this progress (blog.crewai.comblog.crewai.comblog.crewai.com). With careful design, developers can harness these advances to build trustworthy, compliance‑ready chatbots that empower experts rather than replace them. Start small, iterate quickly, and always validate your tools.

Automating MiFID Compliance with CrewAI

noreply@blogger.com (Unknown) — Sat, 04 Oct 2025 19:04:00 +0000

	Regulatory context The MiFID II/III reviews introduce a single 7 % dark‑trading volume cap, ban payment for order flow (PFOF) and require Member States to ensure data‑quality standards for consolidated tape providers (legal.pwc.de). MiFID III mandates extended data fields and near‑real‑time reporting by September 2025 (asctechnologies.com). PFOF will be fully banned and a consolidated tape for equities, ETFs and derivatives will launch in June 2026 (asctechnologies.com). Key deadlines
	Member States must transpose the MiFID II/MiFIR review into national law by 28 September 2025 (legal.pwc.de). September 2025 is also the deadline for MiFID III reporting obligations (asctechnologies.com); the PFOF ban and consolidated tape come into force in June 2026 (asctechnologies.com). Challenges
	Firms face increased operational complexity: they must implement a single‑volume cap, monitor dark trading, ensure data quality for consolidated tapes and update policies and procedures (legal.pwc.de). Near‑real‑time reporting and expanded data fields demand modern IT infrastructure (asctechnologies.com). All customer conversations across channels must be recorded and analysed for misconduct, requiring AI‑powered risk detection (asctechnologies.com). Opportunities
	Greater transparency and harmonised data help investors and regulators. The EU‑wide consolidated tape offers real‑time market data (asctechnologies.com). AI and automation can transform compliance from a cost centre into a strategic differentiator: multi‑agent systems can handle large volumes of data, detect compliance risks and produce accessible reports. Automation approach
	CrewAI is a framework for building teams of autonomous agents that work together. A research agent gathers the latest MiFID updates using web‑search tools and summarises them; a writer agent turns this research into an engaging blog post. Tasks are executed sequentially, ensuring that the output of research feeds directly into writing.

Why MiFID Updates Matter

MiFID II was the EU’s flagship reform for securities markets. Seven years after its introduction, financial markets look very different: algorithmic trading, retail investing apps and AI‑assisted research have become mainstream. The European Commission therefore adopted a MiFID II/MiFIR review in February 2024. The review aims to boost transparency and investor protection by removing redundant provisions, creating a consolidated tape and banning PFOF (payment for order flow) (legal.pwc.de). Member States have until 28 September 2025 to transpose the amendments into national law (legal.pwc.de).

In parallel, the so‑called MiFID III package—an informal term for amendments to MiFID II and MiFIR—modernises the rulebook to reflect digitalisation. It requires firms to document transactions in greater detail and report them to supervisory authorities in near real time (asctechnologies.com), introduces a single 7 % dark‑trading cap (asctechnologies.com), bans PFOF (asctechnologies.com) and launches a consolidated tape for equities, ETFs and derivatives (asctechnologies.com).

These changes present both challenges and opportunities for financial institutions. On the one hand, compliance will demand investments in data governance, monitoring tools and process redesign (legal.pwc.de). On the other hand, harmonised data and advanced analytics can enhance risk management and investor confidence. Adopting multi‑agent AI systems such as CrewAI can help firms navigate this landscape efficiently.

Building a Multi‑Agent MiFID Compliance Workflow

The traditional approach to regulatory change management involves human analysts reading lengthy legislative texts, summarising them and drafting internal or external communications. This process is time‑consuming and error‑prone. CrewAI offers an alternative: you can orchestrate multiple AI agents to collaborate on complex workflows.

Below is a high‑level example of how to automate MiFID research and content creation:

Research agent – A specialist agent queries real‑time web search tools (such as Serper) for the latest MiFID II/III developments. It extracts key facts—deadlines, major amendments, compliance obligations—and generates a structured report. The report highlights the 28 September 2025 deadline for transposition (legal.pwc.de), the September 2025 start of near‑real‑time reporting (asctechnologies.com), and the June 2026 PFOF ban and consolidated tape (asctechnologies.com).
Content strategist – This agent receives the research report and crafts a reader‑friendly article. It explains why the single‑volume cap and consolidated tape matter, discusses the pros and cons of the new regime, and includes practical advice for compliance teams. It also emphasises how AI and automation can help firms handle increased data volumes and monitoring requirements (asctechnologies.com).
Workflow orchestration – A Crew object ties these agents and tasks together in a sequential process. The research task must finish before the writing task begins, ensuring that the output is coherent.

Sample Implementation

Below is a simplified Python script (see crewai_mifid_example.py) that demonstrates this workflow. It uses the CrewAI framework and assumes you have set a valid Serper API key and configured a supported large language model.


import os
from crewai import Agent, Task, Crew, Process, LLM
from crewai_tools import SerperDevTool

# Set your Serper API key
os.environ['SERPER_API_KEY'] = 'your_serper_api_key'

# Configure your LLM
llm = LLM(
    model="gtp-4",
    max_tokens=2000,
)

# Define the research agent
research_agent = Agent(
    role="MiFID Research Analyst",
    goal="Investigate and summarize recent MiFID II and III updates",
    backstory="Expert in EU financial regulation",
    verbose=True,
    allow_delegation=False,
    llm=llm,
    tools=[SerperDevTool()],
)

# Define the writer agent
writer_agent = Agent(
    role="RegTech Content Strategist",
    goal="Write an engaging article about MiFID updates and AI compliance",
    backstory="Skilled communicator for tech audiences",
    verbose=True,
    allow_delegation=True,
    llm=llm,
)

# Define tasks
research_task = Task(
    description="Gather and analyse MiFID II/III deadlines, reporting requirements and PFOF ban.",
    agent=research_agent,
    expected_output="A structured report on MiFID updates",
)
writer_task = Task(
    description="Draft a 4‑paragraph blog post based on the research.",
    agent=writer_agent,
    expected_output="A blog post summarising MiFID updates and the role of AI in compliance.",
)

crew = Crew(
    agents=[research_agent, writer_agent],
    tasks=[research_task, writer_task],
    process=Process.sequential,
    verbose=True,
)

if __name__ == '__main__':
    result = crew.kickoff(inputs={'topic': 'MiFID II and MiFID III updates 2025'})
    print(result)

Pros and Cons of Using CrewAI for Regulatory Compliance

Pros
- Efficiency and scale: AI agents can monitor regulatory updates 24/7, quickly summarising amendments like the MiFID reviews. This reduces manual labour and speeds up response times.
- Consistency: Automated pipelines deliver uniform analysis and writing quality, avoiding variability in human reporting.
- Adaptability: Changing the topic (e.g., from MiFID to ESG or AI regulation) simply requires passing a different input string; no need to redesign the workflow.
- Integrative power: Agents can be equipped with multiple tools—web search, document parsing, API connectors—allowing them to pull data from diverse sources.
Cons
- Context limitations: AI models may misinterpret nuanced legal language or miss implicit regulatory interactions. Human oversight remains essential.
- Data‑source constraints: Tools like Serper rely on publicly available information. Some proprietary regulatory databases may be inaccessible.
- Initial setup: Configuring secure LLM access and API keys requires careful attention to privacy and compliance policies.
- Regulatory liability: Over‑reliance on automated interpretations could pose risks if the system outputs incorrect advice. Always validate AI‑generated content with legal experts.

Conda Environment

To reproduce the workflow locally, create a conda environment named crew with the following specification (see environment.yml):


name: crew
channels:
  - defaults
  - conda-forge
dependencies:
  - python=3.10
  - pip
  - pip:
      - crewai==0.193.2
      - crewai-tools==0.38.0
      - langchain==0.3.20
      - langchain-community==0.3.19
      - databricks-sdk==0.57.0
      - jupyter
      - ipykernel
      - openai

Create the environment with:


conda env create -f environment.yml
conda activate crew

Install any additional models or API keys (e.g., for Serper or your chosen LLM) as environment variables.

Final Thoughts

The MiFID II/MiFIR review and the forthcoming MiFID III rules represent the biggest shake‑up of EU securities regulation since 2018. Firms must prepare for tighter reporting timelines, a 7 % dark‑trading cap and the end of PFOF. At the same time, greater transparency and harmonised data open new opportunities for data‑driven trading strategies and improved investor protection.

Leveraging frameworks like CrewAI allows compliance teams to automate the tedious parts of regulatory change management. By delegating research and writing tasks to specialised agents, practitioners can focus on high‑level analysis and strategic decision‑making. As always, technology is a tool—not a substitute for legal judgement—but used wisely it can turn regulation into a competitive advantage.

Reflection Pattern for Investment Advisory: A Fictional Take

noreply@blogger.com (Unknown) — Sun, 28 Sep 2025 18:14:00 +0000

Reflection Pattern for Investment Advisory: A Fictional Take

This article explores how the reflection pattern, an agentic AI design pattern in which a model iteratively generates, critiques and refines its own work, can be applied to investment advisory. We’ll introduce fictional superstar advisors inspired by pop culture, explain how the reflection loop fits within an orchestrator–worker architecture, and provide a full Python implementation. Along the way, we highlight benefits, drawbacks and speculative future applications.

Summary at a Glance

Persona	Style	Typical Allocation (Equities / Alternatives / Fixed‑Income / Cash)
Bobby Axelrod	Aggressive hedge‑fund maverick willing to ride volatility	70% / 20% / 5% / 5%
Gordon Gekko	Opportunistic raider who loves leverage, options and arbitrage	75% / 15% / 5% / 5%
Richie Rich	Cautious wealth‑preserver who favours blue chips and bonds	30–60% / 10–20% / 15–45% / 5–15%

The Reflection Pattern Explained

The reflection pattern originates in cognitive science and has recently been adopted for large language models (LLMs). The idea is simple yet powerful: rather than trusting a single draft, the model runs a generate–critique–refine loop. According to Analytics Vidhya, the pattern “is a method where the model generates, critiques and refines its outputs through an iterative self‑assessment process” (analyticsvidhya.com). This loop mimics human editing, catching mistakes, clarifying ambiguities and improving quality across multiple passes (analyticsvidhya.com). A second LLM (or the same model with a different prompt) acts as a critic, evaluating the initial output against well‑defined criteria and feeding feedback back to the generator (philschmid.de).

Key steps in the reflection pattern include:

Generation: produce a first draft based on the user’s prompt. This can be done in a zero‑shot manner, generating a candidate without examples (analyticsvidhya.com).
Self‑reflection: the model reviews its own work, identifies errors, missing details or stylistic issues, and generates feedback (analyticsvidhya.com).
Iteration & refinement: the feedback is incorporated into a new draft, and the cycle repeats until the output meets quality thresholds or a maximum number of iterations is reached (tpiros.dev). Stopping conditions prevent infinite loops and include quality targets or iteration limits (analyticsvidhya.com).

Practical applications of the reflection pattern include code generation, content drafting, data analysis and decision‑making. Because the model critiques itself, it can catch inconsistencies and gradually improve the output. However, iterative generation consumes time and compute, and repeated prompts can induce drift or hallucination. A disciplined stopping criterion and robust evaluation criteria are essential.

The Orchestrator–Worker Pattern

The reflection loop fits naturally into an orchestrator–worker architecture, another agentic design pattern. In this setup, a central orchestrator decomposes a complex task into subtasks and dispatches them to specialized worker agents. The orchestrator maintains global oversight, monitors worker performance and synthesizes partial results into a final answer (dzone.com). Workers operate as domain‑specific experts, focusing solely on their assigned tasks (dzone.com).

This separation of concerns brings several benefits:

Modularity & extensibility: new worker agents can be added without rewriting the orchestrator, making the system scalable.
Parallelism: tasks can run concurrently, reducing latency and improving throughput.
Resilience: if one worker fails, the orchestrator can still produce a meaningful result from the remaining outputs (pytrick.medium.com).
Central control: centralized monitoring ensures that quality standards are enforced across workers and that fallback strategies are applied when errors occur (dzone.com).

The orchestrator–worker pattern is particularly useful for workflows that combine planning, data retrieval, analysis and synthesis. For example, in a technical stock analysis system, an orchestrator fetches data once, sends it to workers computing RSI, MACD and other indicators, then aggregates the signals into a report (pytrick.medium.com). Our fictional investment advisor will reuse this structure: the orchestrator iterates through different personas (workers), each generating an investment plan; an evaluator agent critiques the plan; and the orchestrator decides whether to accept or request another draft.

Fictional Investment Superstars

Real investment icons like Cathie Wood or Warren Buffett offer recognisable strategies, but they also introduce brand‑specific bias. To avoid conflating advice with real identities, and to show that agentic workflows can model a variety of styles, we propose three fictional personas:

Bobby Axelrod: loosely modelled on a hedge‑fund manager character. Bobby favours high‑growth sectors and alternative assets. In our template he allocates ~70 % to equities and 20 % to alternatives, leaving minimal fixed income and cash. This results in an aggressive risk grade.
Gordon Gekko: inspired by the 1980s corporate raider, Gordon prioritises concentrated bets, options and arbitrage. He rarely recommends conservative portfolios, pushing equities to 75 % and leaving a token 5 % to fixed income. Another aggressive profile.
Richie Rich: a wealthy heir who values security. Richie’s allocation varies by target grade: 60–30 % equities, 10–20 % alternatives, 15–45 % fixed income and the rest cash. He can satisfy conservative or moderate risk tolerances.

These personas serve as “workers” in our orchestrator. They each provide a distinct draft plan based on the investor’s profile and desired risk grade. A simple evaluator reads the plan, calculates the equity allocation and assigns a risk grade (conservative, moderate or aggressive). If the assigned grade doesn’t match the target, the orchestrator calls another persona to propose a new plan. After a preset number of iterations, the best plan is returned.

LLM Integration with LangChain

While deterministic templates offer repeatable outputs, they lack the spontaneity and nuance of natural language generation. To make the advisor more generative and human‑like, the latest version integrates an optional language model using LangChain’s ChatOpenAI interface. At the top of the code we attempt to import ChatOpenAI and ChatPromptTemplate. A helper function get_llm() tries to instantiate a gpt‑4o‑mini model with a moderate temperature (0.7). When this succeeds, each persona constructs a structured prompt containing the investor’s profile and target grade; the model then generates a bespoke Markdown plan with asset allocations, rationale and rebalancing advice.

The beauty of this design is its graceful fallback: if LangChain isn’t installed or the model fails to initialize (for instance, due to missing API keys), the code reverts to deterministic templates. The reflection loop and evaluator remain unchanged, so the orchestrator–worker architecture stays intact. Only the plan generation mechanism becomes dynamic when an LLM is available. This modularity demonstrates how real‑world systems can incrementally adopt generative AI without sacrificing robustness.

Code Implementation

To put these ideas into practice, we provide a fully working Python module that mirrors the reflection loop described in the Skills Network lab while swapping the real investors for our fictional superstars. The module defines a shared state dictionary, prompts for each persona, an evaluator inspired by Richie Rich’s prudence, and a simple Python loop that iteratively generates and critiques investment plans until the risk grade matches the target.

Key components include:

determine_target_grade – uses an LLM to pick a suitable risk grade for the investor based on their profile.
investment_plan_generator – produces an initial Bobby Axelrod–style plan and refines it using Gordon Gekko’s opportunistic rules when feedback exists.
evaluate_plan – calls a Richie Rich–style evaluator to classify the risk level and provide feedback on diversification, volatility and capital preservation.
route_investment – decides whether to accept the current plan or request another iteration, enforcing a maximum number of loops.
run_reflection_workflow – orchestrates the entire generate–evaluate–route cycle in plain Python without relying on LangGraph, returning the final plan along with feedback and iteration count.

Here is a high‑level sketch of the module’s structure (see langgraph-cookbook/06 - Investment Agent at main · JordiCorbilla/langgraph-cookbook for complete code):


from typing import TypedDict, Dict, Literal, Annotated
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

grades = Literal["ultra-conservative", "conservative", "moderate", "aggressive", "high risk"]

class State(TypedDict):
    investment_plan: str
    investor_profile: str
    target_grade: grades
    feedback: str
    grade: grades
    n: int

# Initialise the LLM (gpt-4o-mini) and build prompt templates for each persona

# … Bobby Axelrod prompt and pipeline
# … Gordon Gekko prompt and pipeline with adaptation rules
# … Richie Rich evaluator prompt and structured output

def determine_target_grade(state: State) -> Dict[str, grades]:
    # Ask the LLM to choose the target risk grade based on the profile
    pass

def investment_plan_generator(state: State) -> Dict[str, str]:
    # Generate or refine the plan using Bobby or Gordon prompts
    pass

def evaluate_plan(state: State) -> Dict[str, object]:
    # Evaluate the plan with the Richie Rich evaluator
    pass

def route_investment(state: State, iteration_limit: int = 5) -> str:
    # Decide whether to accept the plan or loop again
    pass

def run_reflection_workflow(profile: str) -> State:
    # Orchestrate the reflection loop until the target grade is met or the iteration limit is reached
    pass

The complete implementation can be downloaded as custom_investment_reflection.py (file ID :agentCitation{citationIndex='0'}). This script uses LangChain’s ChatOpenAI to generate and evaluate plans, falls back gracefully if the LLM isn’t available, and demonstrates how to adapt the reflection pattern to custom personas.

Sample Execution

As an example, suppose you run run_reflection_workflow() from the custom_investment_reflection.py module with a 35‑year‑old investor earning $100 k, holding $100 k in assets, aiming to retire by 55 with a generous travel budget, and tolerating high risk. The workflow determines an aggressive target grade and iteratively refines the plan. One possible output (actual results vary slightly with the generative model) is summarised below:


🎯 Final Investment Plan Summary
========================================

📌 Investor Profile:
Age: 35
Salary: $100,000
Assets: $100,000
Goal: Retire comfortably by age 55 with ample travel budget
Risk tolerance: high

📈 Target Risk Grade: aggressive
📊 Final Assigned Grade: aggressive
🔁 Iterations Taken: 2

📝 Evaluator Feedback:
----------------------------------
This portfolio is classified as aggressive due to its heavy allocation to equities (75%) and lower commitment to fixed income.  Ensure the investor is comfortable with higher volatility and periodic drawdowns.

📃 Final Investment Plan:
----------------------------------
Gordon Gekko advises the investor to pursue an assertive course:

### Asset Allocation
- **Equities (75%)**: Concentrated bets in blue chips and cyclical industries, supplemented by option strategies.
- **Alternatives (15%)**: Distressed debt, merger arbitrage and real‑estate investment trusts.
- **Fixed Income (5%)**: Opportunistic short‑dated corporate notes.
- **Cash (5%)**: Dry powder for takeover opportunities.

### Rationale
Prioritizing blue chip equities and sector ETFs with active options overlays, complemented by distressed debt and merger arbitrage funds.  Minimal fixed income ensures capital is working hard.

### Risk Management
Use stop‑loss orders and hedges; rebalance monthly to maintain leverage discipline.

The first iteration used the Bobby Axelrod persona, which produced a 70 % equity allocation (moderate grade). The evaluator recommended increasing risk exposure, and the orchestrator rotated to Gordon Gekko, whose 75 % equity allocation met the target. The system stopped after two iterations.

Pros and Cons of the Reflection Pattern in Finance

Advantages

Improved quality through self‑critique: By iteratively evaluating its own recommendations, the advisor reduces the likelihood of omissions or contradictory allocations. It mirrors the human practice of drafting and revising (philschmid.de).
Personalisation: Different personas allow the system to explore varied strategies. Investors can compare aggressive, moderate and conservative approaches without requiring multiple human advisors.
Generative creativity: When an LLM is available, each persona can craft bespoke investment narratives instead of fixed templates. This results in plans that read more naturally and incorporate subtle nuances while still adhering to the requested risk grade.
Reduced human bias: Using fictional characters avoids overreliance on real gurus. The system still captures different risk appetites but does not misappropriate any celebrity’s actual statements.
Extensibility: New personas or evaluators can be added without changing the core loop, demonstrating the modularity of the orchestrator–worker pattern (dzone.com).
Autonomy and scalability: The loop can be run asynchronously, and evaluation metrics can be refined over time to align with regulatory guidelines or individual preferences.

Limitations

Heuristic evaluator: Our evaluation function parses simple percentages and assigns grades. Real‑world risk analysis is far more complex, involving volatility, correlation and drawdown metrics. More sophisticated evaluators (e.g., risk‑parity or Monte Carlo simulators) could improve accuracy but also increase complexity.
Computational cost: Iterative generation and evaluation consume more compute cycles and may require multiple LLM calls. For high‑frequency use cases, latency and API cost could become prohibitive.
Unpredictability: Generative models may occasionally omit required allocation details or introduce inconsistent formatting. Although the evaluator catches gross mismatches, manual review is advisable when using LLM‑generated plans.
Potential oscillations: Without careful stopping rules, the system might ping‑pong between personas or produce contradictory adjustments. Clear termination criteria and penalizing repeated mistakes are essential (analyticsvidhya.com).
Lack of external context: Our fictional personas rely on templates rather than real market data. Integrating live data feeds and dynamic modelling would make the advice more actionable but also raises regulatory and liability considerations.

Conclusion

The reflection pattern offers a simple yet powerful mechanism for improving AI outputs by letting the model serve as its own critic. When combined with an orchestrator–worker architecture, it provides a modular and extensible framework for tasks like investment advice, where balancing risk and return requires careful iteration. By experimenting with fictional personas, we avoid real‑world bias while illustrating how different risk appetites can be encoded in code. Although our current implementation uses heuristic evaluators and templated advice, it lays the foundation for more sophisticated systems that combine generative AI, financial models and ethical constraints.

Random thoughts on coding and technology

Building a Private Local LLM Server on Windows with llama.cpp, Vulkan and Qwen

1. Starting with llmfit

2. Why llama.cpp instead of Ollama

3. Installing llama.cpp on Windows

4. The first problem: Windows "Bad Image"

5. Confirming Vulkan actually worked

6. Choosing the first model

7. Hugging Face SSL problems

8. The first GPU memory failure

9. Partial GPU offloading

10. Reducing batch and micro-batch sizes

11. First benchmark

12. Starting the browser UI

13. Calling the local model from Python

14. The context-size problem

15. Adding a faster model

16. Multi-model routing

17. Creating models.ini

18. Another mistake: unsupported default-model

19. Starting the multi-model router

20. Calling different models through the same API

21. Making the server accessible over the LAN

22. Windows Firewall got in the way

23. Making llama.cpp start after reboot

24. Windows automatic logon

25. Starting llama.cpp at logon

26. The final startup script

27. Automatically locking the workstation

28. The final architecture

29. What I learned

Integrated GPUs are more useful than the VRAM number suggests

Shared memory is not extra memory

Maximum GPU offload is not necessarily optimal

Batch size matters enormously

Keep enough memory for runtime buffers

Large context windows are expensive

A smaller model can be a better default

30. Current model configuration

31. Testing the server

32. OpenAI-compatible Python example

33. What I would improve next

API authentication

Tailscale

Open WebUI

A genuinely fast small model

34. Final result

From 1,880th to 8th: A Six-Day, Validation-First Kaggle Sprint

From 1,880th to 8th: A Six-Day, Validation-First Kaggle Sprint

The result

The competition in one page

Build the laboratory before chasing the score

The path from 0.92933 to 0.95054

The biggest early gain was the decision rule

Exact values were more informative than they looked

The model that delivered 8th place: FT-Transformer v2

Why the notebook infrastructure mattered

The blend that almost passed

What did not work

How the 0.95306 public score was produced

What I would carry into the next competition

Final reflection

Reproducibility and links

Building a GPT-Style Language Model From Scratch, One Understandable Step at a Time

What does "from scratch" mean here?

The whole journey in one view

Step 1: Create a clean environment

Confirming GPU support

Step 2: Choose the training text

Step 3: Turn text into tokens

Step 4: Create the question and the answer

Step 5: Pass the tokens through a GPT-style decoder

Causal self-attention, gently explained

Step 6: Measure the error and update the model

Step 7: Prove the complete pipeline with a smoke run

Step 8: Train the larger model on a GPU

Step 9: Save checkpoints and keep the best model

What did the model learn?

The eventual output

Step 10: Verify and benchmark the result

1. Starting with `llmfit`

17. Creating `models.ini`

18. Another mistake: unsupported `default-model`

Step 1: Install `cloudflared`