Random thoughts on coding and technology

Rowlens: finding one exact CSV row in a 17 GB file without blowing up memory

noreply@blogger.com (Jordi Corbilla) — Sun, 29 Mar 2026 15:48:00 +0000

Most CSV tooling assumes the file is still small enough to load, sort, or inspect interactively. That breaks down fast once the file is measured in gigabytes. rowlens was built for the opposite case: a CLI you can point at a very large CSV when you already know the clues you are looking for and you want the exact matching rows back in a readable format.

The design constraint was simple: never read the full file into memory. rowlens opens the CSV as a stream, reads the header once, and then processes each record one at a time with Python's standard csv reader. That keeps memory usage effectively flat even when the input file is 17 GB or more. The tool does not build an index, cache rows, or attempt an in-memory dataframe workflow. It just walks the file once and stops early if you set --max-results.

The matching model is intentionally narrow for version 1.0. Repeated --keyword arguments are treated as exact cell-value matches. Repeated --filter arguments are treated as substring checks. A row is returned only when it satisfies every supplied condition. That gives a useful two-stage search pattern in practice: use --keyword for the hard identifier, then --filter to narrow the surrounding context.

Example:

rowlens --file "huge.csv" --keyword "1213131" --filter "AAA" --output "results.txt"

If a row matches, the output is not dumped as raw CSV. Instead, rowlens renders a bordered CLI report with summary metadata at the top and then a per-match table that lists each column name beside its value. That matters more than it sounds. When you are debugging a production extract or validating a record in an enormous export, the important part is not just finding the row. The important part is understanding the row immediately.

That output style was inspired by terminal-first tooling such as csv-stream-diff: strong borders, obvious sections, and tabular structure that still works in plain text when redirected to a file. The same rendered report can be written with --output, which makes it easy to attach to a ticket, share in chat, or keep as a trace artifact from an investigation.

Under the hood, Python was a good fit here because the problem is mostly streaming IO plus deterministic row checks. The package is structured with Poetry from the start so it can be shipped cleanly to PyPI. The CLI entry point lives behind the rowlens console script, dependencies are minimal, and the test suite covers the main behaviors: combined keyword and filter matching, case-insensitive search, support for extra cells beyond the header, and output-file generation.

Version 1.0 is deliberately focused. It solves one job well: find the exact row you care about in a file too large for the usual tools, then present it cleanly. Future iterations could add column-scoped matching, JSON output, compressed input support, or richer summary stats. But the first release already hits the core operational need: streaming search for massive CSVs with output that humans can read immediately.

Building sqlcsv-exporter: a Python CLI to Stream SQL Server Data into CSV

noreply@blogger.com (Jordi Corbilla) — Sat, 28 Mar 2026 22:00:00 +0000

I recently built a small Python package called sqlcsv-exporter to solve a very practical problem: export the results of a SQL Server query into a CSV file, from the command line, without dragging an entire dataset into memory.

The tool is packaged as a Poetry project, designed to be publishable to PyPI, and focused on a workflow that is common in reporting and data operations teams:

store the SQL in a .sql file
pass connection details at runtime
stream rows in chunks
write the CSV incrementally
show visible progress while the job runs

This post walks through why I built it, how it works, and what I learned while testing it against a local SQL Server instance.

The Problem

The starting point was familiar: a one-off Python script that connected to SQL Server, ran a query, and wrote a CSV. That works for quick experiments, but it gets messy fast.

Typical issues show up immediately:

query text is embedded inside Python instead of living in a .sql file
authentication logic is mixed into export logic
the script is hard to reuse across environments
large result sets become memory-heavy
the UX is poor when a query runs for several seconds or several minutes

The goal was to replace that with a proper CLI package that feels like a real tool rather than a disposable script.

The Goal

I wanted a command like this:

sqlcsv-exporter `
  --sql .\queries\report.sql `
  --output .\exports\report.csv `
  --server "DESKTOP-TTUSQLJ\SQLEXPRESS" `
  --database QuantDevTest

The requirements were straightforward:

read SQL from file
connect to SQL Server with trusted auth or SQL auth
stream the result set with fetchmany()
write CSV rows incrementally
avoid pandas for the write path
provide useful progress output
keep the package testable and publishable

The Package Structure

I split the project into small modules so each concern is isolated:

cli.py: argument parsing and process exit behavior
config.py: config model and validation
connection.py: ODBC connection string and connection creation
sql_rewriter.py: SQL file loading and optional variable rewriting
exporter.py: query execution, chunked fetch, CSV writing, and progress rendering

That separation matters because it makes the code easier to test. The CSV writer can be tested without a database. The SQL rewriting logic can be tested with simple strings. The export workflow can be tested with mocked connections and cursors.

Why Streaming Matters

The core design choice was to stream results instead of loading everything into memory.

The exporter:

opens the SQL file
optionally rewrites a declared date variable such as @InAsOfDate
executes the query with pyodbc
fetches rows in chunks
writes each chunk directly to a CSV file

That keeps the memory profile stable even when the result set gets large.

Instead of doing something like this:

rows = cursor.fetchall()
writer.writerows(rows)

the tool does this conceptually:

while True:
    rows = cursor.fetchmany(chunk_size)
    if not rows:
        break
    writer.writerows(rows)

That small change is the difference between a toy exporter and something you can trust on real datasets.

A Better CLI Experience

Command-line tools that do database work have one recurring UX problem: silence. If a query takes ten seconds, users start wondering whether the command is stuck, blocked, or dead.

So the exporter prints:

a short run summary before execution
detected column count after the query starts returning metadata
a live progress display while rows are written
a final completion summary with rows, columns, file size, and elapsed time

I used rich for the terminal output instead of hand-rolled print() calls. It gives the tool a more deliberate interface without turning it into a full TUI.

Handling SQL as a File

One design decision I liked immediately was keeping SQL in a separate file. That brings a few benefits:

the query is easier to inspect and review
analysts or DBAs can edit SQL without touching Python code
the same CLI can be reused for many exports
long or multi-step SQL scripts remain readable

For example, this simple test query lives in its own file:

SELECT TOP (1000)
    [Id],
    [Stock],
    [LowPrice],
    [MaxPrice],
    [AvgPrice]
FROM [QuantDevTest].[dbo].[StockMetrics];

That file can be passed directly to the CLI without changing the Python package at all.

Optional SQL Rewriting

Some teams keep parameter values in SQL scripts using a declared variable like:

DECLARE @InAsOfDate DATE = '2026-03-01';

To support that style, the tool includes a lightweight SQL rewriting step. If the variable exists, the CLI can replace its value based on --date. If the variable does not exist, the SQL runs unchanged.

This keeps the interface flexible:

SQL files can remain mostly static
operators can adjust date-driven exports without manually editing the file

It is intentionally narrow in scope. It is not trying to become a generic SQL templating engine.

A Real-World Bug Caught by Live Testing

The first end-to-end run against my local SQL Server instance did not succeed.

The CLI connected successfully, but then failed with:

AttributeError: 'pyodbc.Cursor' object has no attribute 'timeout'

That was useful. The mocked tests passed, but the live run exposed a compatibility assumption in the code. Some pyodbc environments expose timeout behavior differently.

The fix was simple:

try setting the timeout on the cursor if that attribute exists
otherwise fall back to the connection object

That is exactly why live testing matters. A tool can look clean in unit tests and still fail in a real environment for reasons that only surface with the actual driver and actual database connection.

Stress Testing with a Slower Query

A fast export is not a great demo for progress reporting, so I added a dedicated slow test query.

It:

copies base rows into a temp table
creates two multiplier temp tables
cross joins them to expand the result set
adds a short WAITFOR DELAY

That makes it easy to simulate a longer-running export without needing a large production-sized table.

Conceptually, the query looks like this:

SELECT TOP (1000) ... INTO #BaseStockMetrics ...
SELECT TOP (200) ... INTO #MultiplierA ...
SELECT TOP (200) ... INTO #MultiplierB ...

WAITFOR DELAY '00:00:03';

SELECT ...
FROM #BaseStockMetrics AS b
CROSS JOIN #MultiplierA AS a
CROSS JOIN #MultiplierB AS c;

With a small base table, this is enough to create a large CSV and make the progress output visible. With a large base table, it can scale very aggressively, so it is the kind of test query you use carefully.

Testing Strategy

The package includes pytest coverage around the parts that are most likely to break:

config validation
date resolution
SQL file loading
SQL variable rewriting
connection string generation
CSV writing
mocked end-to-end export behavior

That balance is important. Not every code path needs a deep integration test, but the components that define correctness should have coverage.

The live SQL Server run complements those tests by exercising:

the actual ODBC driver
the actual SQL Server instance
actual filesystem writes
real CLI behavior

Packaging for PyPI

The project is set up as a Poetry package and versioned as 1.0.1.

Build artifacts are created with:

poetry build

That produces:

a source distribution
a wheel

Those are the exact artifacts needed for publishing to PyPI.

Example Usage

Trusted connection:

poetry run sqlcsv-exporter `
  --sql .\queries\stock_metrics_top_1000.sql `
  --output .\exports\stock_metrics_top_1000.csv `
  --server "DESKTOP-TTUSQLJ\SQLEXPRESS" `
  --database QuantDevTest

SQL authentication:

poetry run sqlcsv-exporter `
  --sql .\queries\stock_metrics_top_1000.sql `
  --output .\exports\stock_metrics_top_1000.csv `
  --server "my-server" `
  --database "Reporting" `
  --sql-auth `
  --username "report_user" `
  --password "secret"

Final Thoughts

This is not a huge project, but it is a useful example of turning a script into a tool.

The interesting part was not the CSV writing itself. It was the engineering around it:

separating concerns into modules
choosing streaming over eager loading
designing a usable CLI
writing tests around the core behavior
validating the tool against a real SQL Server environment

That is usually the difference between “something that works on my machine” and “something I would actually hand to another team.”

If I extend it further, the next areas I would look at are:

environment-variable support for connection defaults
optional gzip output
row count estimates before execution when possible
better handling for very wide result sets
structured logging for scheduled runs

For now, sqlcsv-exporter does the job it was meant to do: run a SQL Server query from a file and stream the results into CSV in a way that is practical, testable, and ready to package.

Building csv-stream-diff: A Fast, Streaming CSV Comparison Tool for Very Large Files

noreply@blogger.com (Jordi Corbilla) — Sun, 22 Mar 2026 20:30:25 +0000

Comparing two CSV files sounds simple until the files are no longer small.

Once the datasets move into the millions of rows, the usual approaches start to fall apart. Loading both files into memory is expensive. Spreadsheet tools stop being useful. Even many ad hoc scripts become slow, fragile, or impossible to run reliably in production-like environments.

That is the problem I wanted to solve with csv-stream-diff.

The Problem

In real systems, CSV comparison is rarely just "diff these two files."

Usually the job looks more like this:

The files are large enough that a full in-memory load is risky or impossible
The key columns on the left and right files do not use the same names
The comparison should only consider a selected subset of columns
Duplicate keys may exist and need to be reported clearly
Sometimes a full comparison is required, but sometimes a statistically useful sample is enough
The output needs to be machine-readable so it can feed downstream validation or remediation workflows

I wanted a tool that could handle that cleanly, with minimal dependencies, and still be easy to package and run anywhere.

What `csv-stream-diff` Does

csv-stream-diff is a Python CLI tool for comparing very large CSV files using:

streaming reads
hash-based partitioning
multiprocessing
YAML-driven configuration

It produces structured output files for:

rows only on the left
rows only on the right
rows with cell-level differences
duplicate keys
summary metadata

It is designed to be practical rather than clever.

The Core Design

The main design constraint was memory.

If a tool tries to build a single giant in-memory index for both files, it will eventually hit a limit. So instead of comparing the full files at once, csv-stream-diff uses a two-phase approach.

1. Partition both files into hashed buckets

Each row key is normalized and hashed into a bucket. The left and right files are streamed row by row and written into matching bucket files on disk.

That matters because rows with the same normalized key always land in the same bucket. Once that is true, each bucket can be compared independently.

2. Compare buckets in parallel

After partitioning, the tool compares bucket pairs using multiple worker processes. Each worker only needs to index one bucket of the left file at a time, not the entire dataset.

This keeps memory bounded while still taking advantage of all available CPU cores.

The result is a design that scales much better for heavy loads than a naïve single-process implementation.

Why YAML Configuration

I did not want the CLI to become a wall of flags.

The comparison usually needs several pieces of information:

the left and right file paths
the left and right key columns
the left and right comparison columns
CSV dialect options
output paths
sampling settings
performance settings

That is much easier to manage in a YAML file than on the command line.

The CLI still supports overrides, but the YAML file is the primary contract. That makes runs reproducible and easier to version alongside data validation jobs.

Exact Sampling for Large Validation Runs

Sometimes you do not want to compare every row.

For example, if the source files contain tens of millions of records, you may want to run a fast validation pass against an exact random sample of keys before committing to a full comparison.

csv-stream-diff supports that:

sampling.size: 0 means compare everything
sampling.size > 0 means compare an exact random sample of left-side unique keys
sampling.seed makes the sample reproducible

This gives you a useful middle ground between tiny spot checks and full heavy-load comparisons.

Handling Duplicate Keys

Duplicate keys are one of the most annoying edge cases in file comparison work.

If a key appears multiple times, the comparison becomes ambiguous. Instead of failing silently or hiding the problem, the tool reports duplicate keys explicitly and continues using the first occurrence for the main comparison.

That behaviour is deliberate:

you get a warning
you get a separate duplicate-key artifact
you still get a usable comparison result

This makes the tool better suited for messy real-world data.

Keeping Dependencies Small

I wanted the runtime dependency footprint to stay minimal.

The tool is built mostly with the Python standard library. The only runtime dependency is PyYAML, which is used for configuration loading.

That keeps installation simple and reduces operational friction when the tool needs to run in different environments.

Outputs That Are Actually Useful

One important goal was to avoid producing a human-only report.

The tool writes separate output files for each class of result, which makes it easier to automate downstream processing:

only_in_left.csv
only_in_right.csv
differences.csv
duplicate_keys.csv
summary.json

The differences.csv file is especially useful because it reports cell-level differences with both the left and right column names and values.

That means you can do more than say "this row changed." You can say exactly how it changed.

Testing the Tool Properly

I also wanted the project to be easy to validate.

So the repository includes:

unit tests with pytest
BDD-style acceptance tests with behave
a fixture generator that creates two baseline-identical CSV files and then introduces controlled differences

The generator makes it easy to create realistic comparison scenarios involving:

changed values
left-only rows
right-only rows
duplicate keys

That is useful both for development and for demonstrating the tool to others.

A Few Practical Lessons

Building this reinforced a few engineering lessons:

For large-file tooling, streaming and partitioning beat clever in-memory shortcuts
Exact sampling is worth implementing properly because it gives a fast validation mode without becoming a toy feature
Duplicate handling should be explicit, not implicit
Machine-readable outputs matter as much as console output
Minimal dependencies make utility tools easier to adopt

Example Usage

With a config file in place, the tool is intentionally simple to run:

python -m csvstreamdiff.cli --config config.yaml

You can also override selected settings from the CLI:

python -m csvstreamdiff.cli --config config.yaml --sample-size 100000 --workers 8

Why I Built It

This project came from a practical need: compare large CSV datasets reliably, with clear outputs, and without depending on heavy frameworks or fragile one-off scripts.

The result is a tool that is meant to be packaged, published, and reused anywhere.

That was the bar from the start.

Closing

If you work with large exports, migration validation, reconciliation jobs, or data quality checks, CSV comparison becomes infrastructure very quickly.

csv-stream-diff is my attempt to make that infrastructure solid:

reproducible
scalable
explicit
easy to automate

If you want to explore the project, the repository includes the CLI, example configuration, test generator, and packaging setup needed to build and publish it.

Coding from Telegram: my OpenClaw + Ollama setup

noreply@blogger.com (Jordi Corbilla) — Sun, 15 Mar 2026 12:44:00 +0000

I finally got a setup working that feels a bit like the future.

I launched OpenClaw with:


ollama launch openclaw --model kimi-k2.5:cloud

and connected it to Telegram, which means I can now talk to my coding assistant directly from my phone.

What makes this especially satisfying is that the whole thing is running on a very inexpensive mini PC at home:

Mini PC, 12GB RAM + 128GB ROM, Intel Celeron J4125 (up to 2.7 GHz), Windows 10 Pro, dual Wi-Fi 2.4/5G, Bluetooth 4.2, 4K HD, 2 HDMI + 1 VGA.

Nothing exotic. Nothing expensive. Just a small, affordable box quietly sitting there, acting as the bridge between my workspace and my phone.

That is part of the appeal. This setup feels surprisingly capable for something that costs very little and, beyond the hardware I already own, is effectively free to run.

The interesting part is not just that it chats back. It has access to my workspace, can help create projects, work with files, and act more like a real development companion than a simple question-answer bot. That changes the experience completely. It stops being something I occasionally open in a browser and starts feeling more like an always-available engineering copilot.

What I like most is the shift in how and where coding can happen.

I can be away from my desk, out doing groceries, commuting, or just walking around, and still make progress. I can send a quick message to scaffold an idea, create a project structure, inspect something in the workspace, or prepare the next task before I am back in front of the machine. It turns dead time into useful time.

That is the part that feels powerful: development becomes more ambient.

Instead of waiting for the perfect focused block at a desk, I can stay connected to the flow of a project throughout the day. A thought appears, I message it. A small task comes to mind, I delegate it. A project idea starts forming, I capture it immediately. By the time I sit down again, some of the groundwork is already done.

There is also something elegant about the stack itself. Ollama provides a clean way to launch the model workflow, OpenClaw gives the agent a useful operating surface, and Telegram becomes the lightweight interface that makes it all feel natural. No heavy setup on the phone, no friction, no need to open a laptop just to move a task forward.

The other thing I genuinely like is how accessible it is. You do not need a powerful workstation, a costly hosted service, or a complex mobile setup. A cheap mini PC, a Telegram account, and the right tooling are enough to build something that feels surprisingly close to a personal remote engineering assistant.

It is still early, of course, and this kind of workflow needs trust, guardrails, and careful handling when it comes to file access and automation. But even in its current form, it already feels genuinely useful.

A coding assistant in Telegram sounds like a gimmick until it starts helping you build real things while you are standing in a supermarket queue.

That is when it stops sounding like a demo and starts feeling like a new way of working.

Tina the Destroyer: Turning Family Photos Into a Playable Arcade Game

noreply@blogger.com (Jordi Corbilla) — Sun, 08 Mar 2026 15:39:00 +0000

I built a small local arcade game called Tina the Destroyer with Python and Pygame.

The idea was simple and personal: I took pictures of my daughter, converted them into game-ready pixel sprites, and built a one-screen game around that character. The result is a fast arcade loop where Tina smashes falling rats before they hit the ground.

The Core Idea

I wanted a game that was:

Fast to play
Easy to understand
Personal and fun

So I designed a tight gameplay loop:

Move left and right
Press Space to perform a ground smash
Squash rats for points
Lose a life when a rat reaches the bottom
Game over after 3 missed rats

From Photos to Sprites

The visual pipeline started with real photos. I cleaned and adapted the character images to work as sprite sheets and exported key poses:

player_idle.png
player_smash.png

I also prepared enemy and effect assets:

rat.png
rat_squashed.png
impact.png
title.png

For audio, I added smash, miss, and looped music effects, with fallback loading so the game still runs if a file is missing.

Building the Game

The game is organized into small modules:

main.py entrypoint
game.py state machine and game loop
entities.py player, rats, and impact effects
settings.py tuning values
utils.py asset loading and fallbacks

Key features I focused on:

Clean state flow (TITLE, PLAYING, GAME_OVER, QUITTING)
Responsive movement and smash timing
Difficulty scaling over time
Score/lives HUD
Best-score persistence to JSON
Graceful fallback behavior for missing assets

Vibe Coding the Project

I vibe coded this with gpt-5-3.codex, iterating quickly on gameplay feel, tuning values, and production details (asset fallbacks, restart flow, audio handling, and UI polish).

Source code: JordiCorbilla/tina-the-destroyer: tina-the-destroyer

That pairing made it easy to keep momentum: prototype fast, test, adjust, repeat.

Why I Loved This Build

This project sits at the intersection of family creativity and engineering. It started as a fun idea with photos and ended up as a fully playable local arcade game I can run anytime with:

pip install pygame
python main.py

Small scope, real personality, and a complete game loop. Exactly the kind of build I enjoy.

Building a Portfolio CLI in 2026 (and Why CLIs Feel Cool Again)

noreply@blogger.com (Jordi Corbilla) — Sun, 15 Feb 2026 21:09:00 +0000

Over a single weekend, I decided to build a portfolio tracker that lives entirely in the terminal. Not a web app. Not a spreadsheet. A real, interactive CLI with panels, charts, and daily tracking.

I also wanted it to fit how I actually work: Excel as the source of truth, but with a console experience that makes daily updates fast and visual.

This was vibe-coded with Chat GPT-5-2-Codex over a weekend, which made it surprisingly fun to iterate on UI tweaks, edge cases, and the overall feel.

Why a CLI?

CLIs are back. Not because they're nostalgic, but because they're fast, distraction-free, and feel like tools instead of products. A good CLI lets you focus on the data and decisions rather than the UI chrome.

What I Built

The result is my-portfolio-cli, an interactive portfolio dashboard that:

Reads data from an Excel workbook.
Renders a rich TUI with Spectre.Console.
Shows daily PnL, MTD performance, and FY summaries.
Displays a simple PnL bar chart right in the terminal.
Lets you add a new daily snapshot instantly.
Creates new monthly sheets when you need them.

The Flow

Start it and it opens directly in interactive mode. Use arrow keys to move across days and months. Press A to add a new entry. If the month doesn't exist yet, it creates it automatically.

If you're starting from scratch (no workbook or no month sheets), the CLI prompts you to create the first month, add accounts, and enter initial values.

Design Notes

The key was making it feel like a real console app:

Clean layout with strong visual hierarchy.
Color-coded gains and losses.
A compact FY summary that's readable at a glance.
A minimal bar chart for quick trend cues.

It isn't about being fancy. It's about being useful and enjoyable to use every day.

What's Next

This project is going online soon, which means:

A demo workbook and screenshots.
A short walkthrough.
Possibly a minimal installer or single-file publish.

For now, it's a tight little tool that does exactly what I need.

If you want to build your own CLI tool, I highly recommend it. The feedback loop is short, and it's easy to keep refining the experience until it feels right.

Running Claude Code Fully Locally with Ollama (Yes, It’s Possible — With Caveats)

noreply@blogger.com (Jordi Corbilla) — Sun, 25 Jan 2026 17:37:00 +0000

Claude Code is designed as a cloud-first, tool-centric coding agent tightly coupled to Anthropic’s infrastructure. Out of the box, it does not support local models.

However, with a combination of:

endpoint redirection,
model aliasing inside Ollama,
and a tool-capable local model,

you can run Claude Code entirely locally against Ollama — including file edits, refactors, and shell tools.

This article documents the exact path that works, the failure modes you will hit, and the engineering trade-offs.

Why Claude Code Is Hard to Run Locally

Claude Code makes three strong assumptions:

Anthropic authentication is always available
Model names are fixed (claude-sonnet-*)
Tool calling is mandatory

Most “local Claude” guides fail because they only solve one of these.

To succeed, you must solve all three.

Architecture That Actually Works


VS Code
 └─ Claude Code extension
     ├─ hardcoded model name (claude-sonnet-*)
     ├─ mandatory tools
     └─ Anthropic-style requests
           ↓
      Ollama (localhost)
           ↓
   Tool-capable local model (Qwen)

Key insight:

You don’t change Claude Code — you adapt Ollama to look like Anthropic.

Step 1: Redirect Claude Code to Ollama

In VS Code Command palette -> Preferences: Open User Settings (JSON):


{
  "claudeCode.environmentVariables": [
    {
      "name": "ANTHROPIC_BASE_URL",
      "value": "http://127.0.0.1:11434"
    },
    {
      "name": "ANTHROPIC_AUTH_TOKEN",
      "value": "ollama"
    }
  ],
  "claudeCode.disableLoginPrompt": true
}

This bypasses cloud calls and sends all traffic to Ollama.

At this point:

UI works
Requests flow
But you will get 404 model not found

That is expected.

Step 2: Understand the Model Name Problem

Claude Code always sends model names like:


claude-sonnet-4-5-20250929

Ollama does not have such models.

You cannot configure Claude Code to use gemma, llama, or qwen directly.

Solution: Model aliasing

Ollama lets you create lightweight aliases that map one model name to another.

Step 3: Why Gemma Fails (Important)

Gemma models (including gemma3n) do not support tool calling.

Claude Code always sends a tools schema.

Resulting error:


does not support tools

This is a hard failure, not a timeout or config issue.

Conclusion

Gemma works for chat, not for Claude Code.

Step 4: Use a Tool-Capable Model

You need a model that supports:

structured tool calls
JSON responses
streaming + edits

Step 5: Create Claude-Compatible Aliases

Create a Modelfile:


FROM qwen2.5-coder:7b

Then create aliases Claude Code expects:


ollama create claude-sonnet -f Modelfile
ollama create claude-opus   -f Modelfile
ollama create claude-haiku  -f Modelfile

No weights are duplicated.
Only metadata is created.

Now Ollama can answer requests for claude-sonnet-*.

Result: Fully Local Claude Code

At this point, Claude Code can:

create files
edit files
run shell commands
refactor code
reason about repos

All without cloud access.

Building a Multi‑Agent Finance Chatbot with AG2

noreply@blogger.com (Jordi Corbilla) — Sun, 12 Oct 2025 19:23:00 +0000

Financial conversations often touch on complex markets, risk preferences and compliance constraints. A single large‑language model may struggle to handle every nuance, so AG2 (formerly AutoGen) offers a multi‑agent architecture where each agent specializes in part of the workflow. This article demonstrates how to create a finance chatbot using AG2 and Python. We will define specialised agents, orchestrate them via a group chat and show a sample interaction. The approach is inspired by multi‑agent healthcare chatbots but adapted to investment discussions.

Why multi‑agent chatbots?

Recent tutorials explain that multi‑agent chatbots divide complex interactions into specialised roles. Each agent focuses on one task—answering questions, providing recommendations or analysing data—so the overall system can handle nuanced conversations more accurately (analyticsvidhya.com). The AG2 documentation notes that multi‑agent architectures are easier to maintain and extend; they allow developers to add or remove functionality without redesigning the entire system (docs.ag2.ai). Multi‑agent designs also support different conversation patterns (sequential, group or nested chat) and enable tool use and code execution (docs.ag2.ai). In regulated industries such as healthcare or finance, this modularity improves accuracy and reduces risk (docs.ag2.ai).

Another advantage is practicality. Building a chatbot for stock‑market questions is complex, but packages like Chainlit and AG2 simplify the process by handling chat orchestration and tool calls (tinztwinshub.com). These frameworks allow you to run large‑language models locally or through an API and to register Python functions as tools (tinztwinshub.com). As open‑source models like Meta’s Llama 3.1 gain popularity, developers can deploy multi‑agent systems on local machines (tinztwinshub.com).

Pros and cons

Aspect	Pros	Cons & Considerations
Modularity	Each agent has a focused role, making the system easier to extend or modify (docs.ag2.ai).	More agents increase the number of interactions to manage and may introduce coordination overhead.
Specialisation	Domain‑specific agents can outperform a monolithic model on their task (analyticsvidhya.com).	Requires careful prompt design to keep agents within their remit and avoid conflicting advice.
Resource usage	Multi‑agent architectures can run cheaper models for some roles while leveraging larger models only where needed (docs.ag2.ai).	Running multiple models in parallel increases latency and can be costly for high‑volume workloads.
Transparency	By exposing distinct agent responses, users can understand how conclusions are derived (e.g., one agent summarises market data, another suggests allocations).	In some contexts users may prefer a single, unified answer; too much detail may confuse non‑experts.

Defining the finance chatbot agents

Our finance chatbot will engage the user to understand their goals, analyse market conditions and provide general asset‑allocation suggestions. It is not a licensed financial adviser; recommendations are for educational purposes. We adapt the roles from a mental‑health chatbot example into finance:

Client Agent – Captures the user’s financial situation, risk tolerance and investment goals.
Market Analysis Agent – Analyses the market based on user input. It summarises macro‑economic trends, asset‑class performance and volatility without giving personalised advice.
Portfolio Recommendation Agent – Suggests a high‑level asset allocation based on the analysis and user profile. It does not trade or recommend specific securities.

The roles can be summarized as follows:

Agent	Purpose	Key instructions
Client Agent	Collects information about the user’s financial goals, investment horizon and risk tolerance.	Ask the user about their objectives, constraints and any particular interests or concerns.
Market Analysis Agent	Summarises market conditions, including recent trends in equities, bonds, commodities and macro‑economic indicators.	Do not provide personal investment advice; instead, report high‑level insights such as whether markets are volatile or stable.
Portfolio Recommendation Agent	Provides general asset‑allocation suggestions based on the user’s profile and the market summary.	Recommend broad categories (e.g., 60 % equities, 30 % bonds, 10 % alternatives); include a disclaimer that this is not financial advice.

Setting up the environment

To follow along you need Python 3.11+ and the ag2 library. You can install AG2 via pip (pip install ag2[openai]). If you plan to run local models, install an API wrapper like ollama and download a model such as Llama 3.1 (tinztwinshub.com). You will also need API keys if using cloud models. In this example we configure the LLM via llm_config without specifying an API key; supply your own key as needed.

Implementing the agents

The following Python code illustrates how to create the finance chatbot. It borrows the group‑chat structure from the mental‑health example and uses AG2’s ConversableAgent, GroupChat and GroupChatManager classes:


from autogen import ConversableAgent, GroupChat, GroupChatManager

# Replace None with your OpenAI or local model API key if required
llm_config = {"config_list": [{"model": "gpt-4o", "api_key": None}]}

# 1. Client Agent: collects user goals and risk tolerance
client_agent = ConversableAgent(
    name="client",
    system_message=(
        "You are the client interface.\n"
        "Your job is to describe your financial goals, risk tolerance, investment horizon, "
        "and any budget constraints or ethical considerations."
    ),
    llm_config=llm_config
)

# 2. Market Analysis Agent: summarises current market conditions
market_analysis_agent = ConversableAgent(
    name="market_analysis",
    system_message=(
        "You analyse the current market based on the client's input."
        "Summarise macroeconomic factors (interest rates, inflation), sector trends and recent market volatility. "
        "Do not provide personalised investment advice; simply report high-level observations."
    ),
    llm_config=llm_config
)

# 3. Portfolio Recommendation Agent: proposes a high-level allocation
portfolio_recommendation_agent = ConversableAgent(
    name="portfolio_recommendation",
    system_message=(
        "You suggest a general asset allocation based on the analysis from the Market Analysis Agent."
        "Consider the client's risk tolerance and goals."
        "Recommend proportions across asset classes such as equities, bonds, and alternatives."
        "Include a disclaimer that the suggestions are educational and not financial advice."
    ),
    llm_config=llm_config
)

# Configure a group chat where the analysis and recommendation agents converse
# The manager orchestrates turn-taking in round‑robin fashion
finance_groupchat = GroupChat(
    agents=[market_analysis_agent, portfolio_recommendation_agent],
    messages=[],
    max_round=3,
    speaker_selection_method="round_robin"
)

manager = GroupChatManager(name="manager", groupchat=finance_groupchat)

# Function to start the finance chatbot

def start_finance_chat():
    print("\nWelcome to the AI Finance Chatbot!")
    user_input = input("Please describe your financial goals and risk tolerance: ")

    # The client agent initiates the conversation with the manager
    print("\nAnalysing your profile and market conditions...")
    response = client_agent.initiate_chat(
        manager,
        message=(
            f"My financial goals and risk tolerance are: {user_input}. "
            "Based on this, what insights and recommendations can you provide?"
        )
    )

    # Fallback in case the first response is empty
    if not response:
        response = portfolio_recommendation_agent.initiate_chat(
            manager,
            message="Based on the client's goals, please provide portfolio recommendations."
        )

# Run the chatbot
start_finance_chat()

How it works

User input – The client_agent collects the investor’s goals and risk tolerance. In a real deployment you might ask follow‑up questions to ensure clarity.
Market analysis – The market_analysis_agent reviews the input and provides a concise summary of current market conditions. It does not provide personalised advice, reflecting a common compliance requirement for general market commentary.
Portfolio suggestion – The portfolio_recommendation_agent uses the market summary and client profile to recommend an asset allocation. It emphasises diversification and includes a disclaimer.
Group chat management – The GroupChatManager coordinates the exchange. We use a round‑robin method to ensure both agents contribute (analyticsvidhya.com). The max_round parameter prevents infinite loops.

Sample interaction

Suppose a user says: “I’m saving for retirement in 20 years, have moderate risk tolerance and want to invest ethically.” The agents might respond as follows:

Market Analysis Agent: “Based on the current environment, central banks are signalling that interest rates may remain high to curb inflation, equities have been volatile with technology and renewable‑energy sectors outperforming, and bond yields offer moderate returns. Commodities like gold have appreciated due to geopolitical tension.”

Portfolio Recommendation Agent: “Given your 20‑year horizon and moderate risk tolerance, a diversified allocation could be 60 % equities (with a tilt towards ESG‑focused funds), 30 % bonds (combining government and investment‑grade corporate bonds), and 10 % alternatives such as real‑estate investment trusts or commodities. This is general information and not personalised financial advice.”

In a real‑world application you could register functions that fetch real‑time market data or compute portfolio statistics using AG2’s register_function API (tinztwinshub.com), but our simple example relies on the language model.

Pros and cons of this approach

Pros:
- Modularity and specialisation – Each agent has a clear purpose, which can improve the relevance and quality of responses (docs.ag2.ai). For example, one agent summarises macro data while another focuses on portfolio construction.
- Ease of extension – You can add agents to handle regulatory checks, tax considerations or portfolio rebalancing without rewriting existing components (docs.ag2.ai).
- Transparency – Users can see how the system arrives at recommendations because each agent’s contribution is visible (docs.ag2.ai).
Cons:
- Coordination overhead – More agents mean more messages and potential latency. Tuning max_round and speaker selection helps avoid long conversations.
- Prompt management – Agents must be carefully prompted to avoid stepping outside their roles. If the market‑analysis agent gives advice, it could cause compliance issues.
- Resource usage – Running multiple models or using powerful models for each agent can be costly. Techniques like mixing cheap and expensive models or limiting context length can help (docs.ag2.ai).

Future enhancements

Real‑time data integration: Register functions to call APIs (e.g., yfinance, Bloomberg, or custom data feeds) so the analysis agent can deliver up‑to‑the‑minute market summaries. The Tinz Twins example shows how to plot year‑to‑date gains using a registered function (tinztwinshub.com).
Regulatory compliance: Add an agent that checks whether recommendations adhere to regional regulations (e.g., MIFID II or SEC guidelines). It could flag issues before suggestions reach the user.
Advanced conversation patterns: Use nested or constrained group chats to handle multi‑step workflows, such as gathering risk questionnaires, performing Monte‑Carlo simulations and then generating a report. AG2 supports these patterns (docs.ag2.ai).
Multi‑modal output: Integrate charting tools to visualise asset allocations or risk/return trade‑offs. For instance, register a function that plots allocation pie charts and returns an image.

Conclusion

AG2’s multi‑agent architecture makes it straightforward to build tailored chatbots for complex domains such as finance. By dividing the conversation into specialised roles, developers can achieve greater accuracy, transparency and extensibility compared with monolithic chatbots (analyticsvidhya.com). The simple example above illustrates how you can adapt a mental‑health chatbot pattern to a personal‑finance advisor. Although multi‑agent systems introduce coordination challenges and resource overhead, careful design and modern frameworks like AG2 or Chainlit can mitigate these issues (docs.ag2.ai). As open‑source models and tooling evolve, we can expect multi‑agent finance chatbots to become more capable—potentially integrating real‑time data, compliance checks and sophisticated analytics.

Building a MiFID II Compliance Chatbot with CrewAI

noreply@blogger.com (Jordi Corbilla) — Sat, 11 Oct 2025 18:41:00 +0000

Financial regulation is notoriously complex. Investment firms must interpret and comply with lengthy documents such as the Markets in Financial Instruments Directive (MiFID II) and its companion regulation (MiFIR), while preparing for MiFID III. Compliance officers routinely pore over hundreds of pages of policy statements, ESMA Q&As and Regulatory Technical Standards (RTS). At the same time, regulators are tightening expectations: by September 2025 firms will need to implement the MiFID III amendments and be ready to demonstrate that surveillance and inducement controls work across every channel (luware.comluware.com). Manual search is inefficient and error‑prone.

This article shows how to build a MiFID II compliance chatbot using CrewAI, a multi‑agent framework. You will learn how to organise the workflow, assign tools at the agent or task level, avoid common validation errors, and anticipate upcoming regulatory changes. The techniques apply beyond finance; they illustrate a pattern for building reliable agentic systems that combine retrieval with language models.

Framework
CrewAI orchestrates multiple AI agents to search MiFID-related PDFs and draft compliance answers

Approach
Compare agent‑centric vs task‑centric tool assignment for reliability and predictability

Key Fix
Teach the language model to include the mandatory {"query": "<text>"} argument when calling the PDF search tool

Regulatory Horizon

MiFID III amendments enter force March 28 2024 with a main implementation deadline of September 29 2025 (luware.comluware.com)

New Features

CrewAI’s MCP standardises tool access and data sources (blog.crewai.com) while guardrails and event buses increase reliability (blog.crewai.comblog.crewai.com)

Agent‑centric vs Task‑centric Tool Assignment

CrewAI allows tools—such as a PDF searcher or a web search API—to be provided either to the agent as a global toolbox or to individual tasks. These two patterns affect reliability and maintainability.

Agent‑centric (Generalist)

In this common setup the agent receives a list of tools that it may call at any time. The agent decides which tool to use based on the prompt and its internal reasoning. This is flexible and quick to prototype, but it can lead to mis‑calls. For example, the PDF search tool expects a JSON argument with a query string. Without explicit instructions, the LLM may omit the argument, leading to the familiar Pydantic error:


Error: Arguments validation failed: `query` field required

When the agent owns the entire toolbox, every task must encode tool‑calling rules in its prompt to avoid such errors. In our experience this reduces predictability as the system grows; even seasoned engineers forget to repeat the tool signature in each description.

Task‑centric (Specialist)

The task‑centric approach grants tools only to the tasks that need them. When the agent begins a task it is temporarily given the required tools and cannot access others. This design leads to more predictable tool calls because the agent knows which tools are available for that job and the prompt can focus on how to use them.

In regulated domains this scoping is critical. It reduces the chance that a general‑purpose LLM will call a web search when only document evidence is allowed, and it makes unit testing straightforward: each task can be executed in isolation with its own tools and guardrails (blog.crewai.com).

Pros and Cons

Agent‑centric:
- Pros: Simple configuration; fewer moving parts; agent decides which tool to use.
- Cons: LLM may mis‑call tools without guidance; prompts must repeat tool signatures; difficult to test tasks in isolation.
Task‑centric:
- Pros: Predictable and secure; tools scoped to tasks; easier to unit‑test and update; better adherence to compliance restrictions.
- Cons: Slightly more verbose setup; requires careful mapping of tasks to tools.

Code Walkthrough

Let’s examine the core elements of the chatbot. The code below instantiates an LLM, sets up PDF search tools for each regulatory document, defines an agent with a system prompt and tool‑calling guide, and creates two tasks orchestrated sequentially by a crew. Code can be found here: (langgraph-cookbook/09 - Mifid CrewAI Chatbot at main · JordiCorbilla/langgraph-cookbook)


from crewai import Agent, Task, Crew, Process
from crewai import LLM
from crewai_tools import PDFSearchTool, SerperDevTool

llm = LLM(model="openai/gpt-4o-mini")

web_search_tool = SerperDevTool()

pdf_search_tool = PDFSearchTool(
    pdf="https://www.fca.org.uk/publication/policy/ps17-14.pdf",
    config={ 'embedder': { 'provider': 'huggingface', 'config': { 'model': 'sentence-transformers/all-MiniLM-L6-v2' } } }
)
pdf_search_tool2 = PDFSearchTool(
    pdf="https://www.fca.org.uk/publication/consultation/cp16-19.pdf",
    config={ ... }
)
pdf_search_tool3 = PDFSearchTool(
    pdf="https://www.esma.europa.eu/sites/default/files/library/2016-1452_guidelines_mifid_ii_transaction_reporting.pdf",
    config={ ... }
)

COMPLIANCE_SYSTEM_PROMPT = """
You are a MiFID II / MiFIR compliance analyst. Answer ONLY with information grounded in the provided PDFs.  Always include article‑level citations.  Prefer the latest ESMA RTS for conflicts.  No speculation.
"""

CITATION_FORMAT_PROMPT = """
Format citations as markdown footnotes: [MiFID II Art X(Y)](SOURCE_URL#page=?), [MiFIR Art X(Y)](SOURCE_URL#page=?).
"""

TOOL_CALLING_GUIDE = """
When using "Search a PDF's content", you MUST pass a JSON argument like {"query": "<short search string>"}.  Never omit 'query'.
"""

agent_centric_agent = Agent(
    role="MiFID Compliance Analyst",
    goal="Answer MiFID II / MiFIR questions with exact citations.",
    backstory="Seasoned regulatory analyst trained on MiFID II, MiFIR and FCA/ESMA RTS/Q&A.",
    tools=[pdf_search_tool, pdf_search_tool2, pdf_search_tool3, web_search_tool],
    llm=llm,
    system_prompt=COMPLIANCE_SYSTEM_PROMPT + CITATION_FORMAT_PROMPT + TOOL_CALLING_GUIDE
)

# Define tasks and crew
agent_centric_task = Task(
    description=(
        "Answer '{customer_query}'. First, search the PDFs using 'Search a PDF's content' with {\"query\": \"<your search string>\"}. Extract relevant passages and synthesise the answer."),
    tools=[pdf_search_tool, pdf_search_tool2, pdf_search_tool3],
    agent=agent_centric_agent
)

response_drafting_task = Task(
    description="Use the gathered information to draft a comprehensive response to '{customer_query}'.",
    agent=agent_centric_agent,
    context=[agent_centric_task]
)

task_centric_crew = Crew(
    agents=[agent_centric_agent],
    tasks=[agent_centric_task, response_drafting_task],
    process=Process.sequential
)

The system prompt instructs the agent to answer using only information from the PDFs, include citations, and avoid speculation. We append a tool‑calling guide that explicitly defines the JSON argument required by the PDFSearchTool. Without this guide the agent might call the tool incorrectly, causing the validation error described earlier.

The first task instructs the agent to search the PDFs for relevant text using a JSON query. The second task uses the extracted passages to draft the final answer. By assigning the PDF search tools only to the search task, we prevent accidental web queries during drafting.

Preparing for MiFID III

Our chatbot currently answers questions under MiFID II and MiFIR. However, regulatory evolution continues. MiFID III amendments were published in February 2024, entered into force in March 2024 and must be implemented by September 29 2025 (luware.com). The third iteration does not overhaul the framework but raises the bar on surveillance and inducements: compliance teams must prepare for heightened expectations around off‑channel communications and record‑keeping (luware.com). MiFID III builds on MiFID II by widening the focus to behavioural risks such as conflicts of interest and pricing opacity (luware.com), and regulators expect active monitoring rather than passive archiving (luware.com).

To stay ahead:

Integrate multi‑channel monitoring: capture voice, chat, video and messaging across platforms; apply AI to detect misconduct patterns (luware.com).
Extend the knowledge base: ingest ESMA Q&As, RTS documents and MiFID III draft texts; update embeddings regularly.
Use CrewAI’s guardrails: implement output‑length checks or hallucination detection to ensure answers remain grounded (blog.crewai.com).
Leverage event buses for audit logs and compliance evidence: CrewAI’s event bus emits events such as task started, tool use error, and LLM call completed (blog.crewai.com).
Adopt query rewriting and agentic RAG to improve retrieval. CrewAI now supports query rewriting to focus on relevant keywords and integrates with vector databases like Qdrant and Weaviate (blog.crewai.com).
Plan for MiFID III deadlines: ensure your system can adapt to new PFOF bans, stricter inducement rules, and expanded communication capture (luware.com).

Beyond Compliance – General Lessons

While this capstone project addresses a specific regulatory domain, the design patterns are broadly applicable:

Be explicit about tool signatures: LLMs cannot infer parameter names; include examples and constraints in prompts. For production systems, consider using structured tool calling (JSON Schema) or wrappers that handle input validation.
Scope tools to tasks: Minimises misuse and aids debugging. For high‑impact domains (finance, healthcare) this can be critical for legal and safety reasons.
Instrument your agents: Use CrewAI’s event bus to collect telemetry on tool use and LLM calls. This data helps debug issues and demonstrates compliance (blog.crewai.com).
Stay up to date: Regulations change. Regularly ingest new documents and search the web to supplement your knowledge base. MiFID III emphasises behaviour monitoring, so your system must adapt to detect tone and sentiment across channels (luware.com).
Embrace guardrails and RAG: Guardrails catch excessive or irrelevant outputs; agentic RAG (retrieval augmented generation) tailors the knowledge retrieval to each query, increasing precision and reducing hallucinations (blog.crewai.com).

Conclusion

By combining CrewAI’s multi‑agent orchestration with dedicated tools for PDF and web search, we built a robust MiFID II compliance chatbot. Assigning tools at the task level rather than the agent level yields more predictable and testable workflows. The critical missing‑query error was resolved by teaching the LLM to call tools correctly. Looking ahead, MiFID III will demand even greater surveillance and behavioural oversight; our modular architecture can adapt by adding new documents, query‑rewriting agents, guardrails and event logging.

Agentic AI is evolving rapidly. CrewAI’s adoption of the MCP, guardrails and event bus underscores this progress (blog.crewai.comblog.crewai.comblog.crewai.com). With careful design, developers can harness these advances to build trustworthy, compliance‑ready chatbots that empower experts rather than replace them. Start small, iterate quickly, and always validate your tools.

Automating MiFID Compliance with CrewAI

noreply@blogger.com (Jordi Corbilla) — Sat, 04 Oct 2025 19:04:00 +0000

	Regulatory context The MiFID II/III reviews introduce a single 7 % dark‑trading volume cap, ban payment for order flow (PFOF) and require Member States to ensure data‑quality standards for consolidated tape providers (legal.pwc.de). MiFID III mandates extended data fields and near‑real‑time reporting by September 2025 (asctechnologies.com). PFOF will be fully banned and a consolidated tape for equities, ETFs and derivatives will launch in June 2026 (asctechnologies.com). Key deadlines
	Member States must transpose the MiFID II/MiFIR review into national law by 28 September 2025 (legal.pwc.de). September 2025 is also the deadline for MiFID III reporting obligations (asctechnologies.com); the PFOF ban and consolidated tape come into force in June 2026 (asctechnologies.com). Challenges
	Firms face increased operational complexity: they must implement a single‑volume cap, monitor dark trading, ensure data quality for consolidated tapes and update policies and procedures (legal.pwc.de). Near‑real‑time reporting and expanded data fields demand modern IT infrastructure (asctechnologies.com). All customer conversations across channels must be recorded and analysed for misconduct, requiring AI‑powered risk detection (asctechnologies.com). Opportunities
	Greater transparency and harmonised data help investors and regulators. The EU‑wide consolidated tape offers real‑time market data (asctechnologies.com). AI and automation can transform compliance from a cost centre into a strategic differentiator: multi‑agent systems can handle large volumes of data, detect compliance risks and produce accessible reports. Automation approach
	CrewAI is a framework for building teams of autonomous agents that work together. A research agent gathers the latest MiFID updates using web‑search tools and summarises them; a writer agent turns this research into an engaging blog post. Tasks are executed sequentially, ensuring that the output of research feeds directly into writing.

Why MiFID Updates Matter

MiFID II was the EU’s flagship reform for securities markets. Seven years after its introduction, financial markets look very different: algorithmic trading, retail investing apps and AI‑assisted research have become mainstream. The European Commission therefore adopted a MiFID II/MiFIR review in February 2024. The review aims to boost transparency and investor protection by removing redundant provisions, creating a consolidated tape and banning PFOF (payment for order flow) (legal.pwc.de). Member States have until 28 September 2025 to transpose the amendments into national law (legal.pwc.de).

In parallel, the so‑called MiFID III package—an informal term for amendments to MiFID II and MiFIR—modernises the rulebook to reflect digitalisation. It requires firms to document transactions in greater detail and report them to supervisory authorities in near real time (asctechnologies.com), introduces a single 7 % dark‑trading cap (asctechnologies.com), bans PFOF (asctechnologies.com) and launches a consolidated tape for equities, ETFs and derivatives (asctechnologies.com).

These changes present both challenges and opportunities for financial institutions. On the one hand, compliance will demand investments in data governance, monitoring tools and process redesign (legal.pwc.de). On the other hand, harmonised data and advanced analytics can enhance risk management and investor confidence. Adopting multi‑agent AI systems such as CrewAI can help firms navigate this landscape efficiently.

Building a Multi‑Agent MiFID Compliance Workflow

The traditional approach to regulatory change management involves human analysts reading lengthy legislative texts, summarising them and drafting internal or external communications. This process is time‑consuming and error‑prone. CrewAI offers an alternative: you can orchestrate multiple AI agents to collaborate on complex workflows.

Below is a high‑level example of how to automate MiFID research and content creation:

Research agent – A specialist agent queries real‑time web search tools (such as Serper) for the latest MiFID II/III developments. It extracts key facts—deadlines, major amendments, compliance obligations—and generates a structured report. The report highlights the 28 September 2025 deadline for transposition (legal.pwc.de), the September 2025 start of near‑real‑time reporting (asctechnologies.com), and the June 2026 PFOF ban and consolidated tape (asctechnologies.com).
Content strategist – This agent receives the research report and crafts a reader‑friendly article. It explains why the single‑volume cap and consolidated tape matter, discusses the pros and cons of the new regime, and includes practical advice for compliance teams. It also emphasises how AI and automation can help firms handle increased data volumes and monitoring requirements (asctechnologies.com).
Workflow orchestration – A Crew object ties these agents and tasks together in a sequential process. The research task must finish before the writing task begins, ensuring that the output is coherent.

Sample Implementation

Below is a simplified Python script (see crewai_mifid_example.py) that demonstrates this workflow. It uses the CrewAI framework and assumes you have set a valid Serper API key and configured a supported large language model.


import os
from crewai import Agent, Task, Crew, Process, LLM
from crewai_tools import SerperDevTool

# Set your Serper API key
os.environ['SERPER_API_KEY'] = 'your_serper_api_key'

# Configure your LLM
llm = LLM(
    model="gtp-4",
    max_tokens=2000,
)

# Define the research agent
research_agent = Agent(
    role="MiFID Research Analyst",
    goal="Investigate and summarize recent MiFID II and III updates",
    backstory="Expert in EU financial regulation",
    verbose=True,
    allow_delegation=False,
    llm=llm,
    tools=[SerperDevTool()],
)

# Define the writer agent
writer_agent = Agent(
    role="RegTech Content Strategist",
    goal="Write an engaging article about MiFID updates and AI compliance",
    backstory="Skilled communicator for tech audiences",
    verbose=True,
    allow_delegation=True,
    llm=llm,
)

# Define tasks
research_task = Task(
    description="Gather and analyse MiFID II/III deadlines, reporting requirements and PFOF ban.",
    agent=research_agent,
    expected_output="A structured report on MiFID updates",
)
writer_task = Task(
    description="Draft a 4‑paragraph blog post based on the research.",
    agent=writer_agent,
    expected_output="A blog post summarising MiFID updates and the role of AI in compliance.",
)

crew = Crew(
    agents=[research_agent, writer_agent],
    tasks=[research_task, writer_task],
    process=Process.sequential,
    verbose=True,
)

if __name__ == '__main__':
    result = crew.kickoff(inputs={'topic': 'MiFID II and MiFID III updates 2025'})
    print(result)

Pros and Cons of Using CrewAI for Regulatory Compliance

Pros
- Efficiency and scale: AI agents can monitor regulatory updates 24/7, quickly summarising amendments like the MiFID reviews. This reduces manual labour and speeds up response times.
- Consistency: Automated pipelines deliver uniform analysis and writing quality, avoiding variability in human reporting.
- Adaptability: Changing the topic (e.g., from MiFID to ESG or AI regulation) simply requires passing a different input string; no need to redesign the workflow.
- Integrative power: Agents can be equipped with multiple tools—web search, document parsing, API connectors—allowing them to pull data from diverse sources.
Cons
- Context limitations: AI models may misinterpret nuanced legal language or miss implicit regulatory interactions. Human oversight remains essential.
- Data‑source constraints: Tools like Serper rely on publicly available information. Some proprietary regulatory databases may be inaccessible.
- Initial setup: Configuring secure LLM access and API keys requires careful attention to privacy and compliance policies.
- Regulatory liability: Over‑reliance on automated interpretations could pose risks if the system outputs incorrect advice. Always validate AI‑generated content with legal experts.

Conda Environment

To reproduce the workflow locally, create a conda environment named crew with the following specification (see environment.yml):


name: crew
channels:
  - defaults
  - conda-forge
dependencies:
  - python=3.10
  - pip
  - pip:
      - crewai==0.193.2
      - crewai-tools==0.38.0
      - langchain==0.3.20
      - langchain-community==0.3.19
      - databricks-sdk==0.57.0
      - jupyter
      - ipykernel
      - openai

Create the environment with:


conda env create -f environment.yml
conda activate crew

Install any additional models or API keys (e.g., for Serper or your chosen LLM) as environment variables.

Final Thoughts

The MiFID II/MiFIR review and the forthcoming MiFID III rules represent the biggest shake‑up of EU securities regulation since 2018. Firms must prepare for tighter reporting timelines, a 7 % dark‑trading cap and the end of PFOF. At the same time, greater transparency and harmonised data open new opportunities for data‑driven trading strategies and improved investor protection.

Leveraging frameworks like CrewAI allows compliance teams to automate the tedious parts of regulatory change management. By delegating research and writing tasks to specialised agents, practitioners can focus on high‑level analysis and strategic decision‑making. As always, technology is a tool—not a substitute for legal judgement—but used wisely it can turn regulation into a competitive advantage.

Reflection Pattern for Investment Advisory: A Fictional Take

noreply@blogger.com (Jordi Corbilla) — Sun, 28 Sep 2025 18:14:00 +0000

Reflection Pattern for Investment Advisory: A Fictional Take

This article explores how the reflection pattern, an agentic AI design pattern in which a model iteratively generates, critiques and refines its own work, can be applied to investment advisory. We’ll introduce fictional superstar advisors inspired by pop culture, explain how the reflection loop fits within an orchestrator–worker architecture, and provide a full Python implementation. Along the way, we highlight benefits, drawbacks and speculative future applications.

Summary at a Glance

Persona	Style	Typical Allocation (Equities / Alternatives / Fixed‑Income / Cash)
Bobby Axelrod	Aggressive hedge‑fund maverick willing to ride volatility	70% / 20% / 5% / 5%
Gordon Gekko	Opportunistic raider who loves leverage, options and arbitrage	75% / 15% / 5% / 5%
Richie Rich	Cautious wealth‑preserver who favours blue chips and bonds	30–60% / 10–20% / 15–45% / 5–15%

The Reflection Pattern Explained

The reflection pattern originates in cognitive science and has recently been adopted for large language models (LLMs). The idea is simple yet powerful: rather than trusting a single draft, the model runs a generate–critique–refine loop. According to Analytics Vidhya, the pattern “is a method where the model generates, critiques and refines its outputs through an iterative self‑assessment process” (analyticsvidhya.com). This loop mimics human editing, catching mistakes, clarifying ambiguities and improving quality across multiple passes (analyticsvidhya.com). A second LLM (or the same model with a different prompt) acts as a critic, evaluating the initial output against well‑defined criteria and feeding feedback back to the generator (philschmid.de).

Key steps in the reflection pattern include:

Generation: produce a first draft based on the user’s prompt. This can be done in a zero‑shot manner, generating a candidate without examples (analyticsvidhya.com).
Self‑reflection: the model reviews its own work, identifies errors, missing details or stylistic issues, and generates feedback (analyticsvidhya.com).
Iteration & refinement: the feedback is incorporated into a new draft, and the cycle repeats until the output meets quality thresholds or a maximum number of iterations is reached (tpiros.dev). Stopping conditions prevent infinite loops and include quality targets or iteration limits (analyticsvidhya.com).

Practical applications of the reflection pattern include code generation, content drafting, data analysis and decision‑making. Because the model critiques itself, it can catch inconsistencies and gradually improve the output. However, iterative generation consumes time and compute, and repeated prompts can induce drift or hallucination. A disciplined stopping criterion and robust evaluation criteria are essential.

The Orchestrator–Worker Pattern

The reflection loop fits naturally into an orchestrator–worker architecture, another agentic design pattern. In this setup, a central orchestrator decomposes a complex task into subtasks and dispatches them to specialized worker agents. The orchestrator maintains global oversight, monitors worker performance and synthesizes partial results into a final answer (dzone.com). Workers operate as domain‑specific experts, focusing solely on their assigned tasks (dzone.com).

This separation of concerns brings several benefits:

Modularity & extensibility: new worker agents can be added without rewriting the orchestrator, making the system scalable.
Parallelism: tasks can run concurrently, reducing latency and improving throughput.
Resilience: if one worker fails, the orchestrator can still produce a meaningful result from the remaining outputs (pytrick.medium.com).
Central control: centralized monitoring ensures that quality standards are enforced across workers and that fallback strategies are applied when errors occur (dzone.com).

The orchestrator–worker pattern is particularly useful for workflows that combine planning, data retrieval, analysis and synthesis. For example, in a technical stock analysis system, an orchestrator fetches data once, sends it to workers computing RSI, MACD and other indicators, then aggregates the signals into a report (pytrick.medium.com). Our fictional investment advisor will reuse this structure: the orchestrator iterates through different personas (workers), each generating an investment plan; an evaluator agent critiques the plan; and the orchestrator decides whether to accept or request another draft.

Fictional Investment Superstars

Real investment icons like Cathie Wood or Warren Buffett offer recognisable strategies, but they also introduce brand‑specific bias. To avoid conflating advice with real identities, and to show that agentic workflows can model a variety of styles, we propose three fictional personas:

Bobby Axelrod: loosely modelled on a hedge‑fund manager character. Bobby favours high‑growth sectors and alternative assets. In our template he allocates ~70 % to equities and 20 % to alternatives, leaving minimal fixed income and cash. This results in an aggressive risk grade.
Gordon Gekko: inspired by the 1980s corporate raider, Gordon prioritises concentrated bets, options and arbitrage. He rarely recommends conservative portfolios, pushing equities to 75 % and leaving a token 5 % to fixed income. Another aggressive profile.
Richie Rich: a wealthy heir who values security. Richie’s allocation varies by target grade: 60–30 % equities, 10–20 % alternatives, 15–45 % fixed income and the rest cash. He can satisfy conservative or moderate risk tolerances.

These personas serve as “workers” in our orchestrator. They each provide a distinct draft plan based on the investor’s profile and desired risk grade. A simple evaluator reads the plan, calculates the equity allocation and assigns a risk grade (conservative, moderate or aggressive). If the assigned grade doesn’t match the target, the orchestrator calls another persona to propose a new plan. After a preset number of iterations, the best plan is returned.

LLM Integration with LangChain

While deterministic templates offer repeatable outputs, they lack the spontaneity and nuance of natural language generation. To make the advisor more generative and human‑like, the latest version integrates an optional language model using LangChain’s ChatOpenAI interface. At the top of the code we attempt to import ChatOpenAI and ChatPromptTemplate. A helper function get_llm() tries to instantiate a gpt‑4o‑mini model with a moderate temperature (0.7). When this succeeds, each persona constructs a structured prompt containing the investor’s profile and target grade; the model then generates a bespoke Markdown plan with asset allocations, rationale and rebalancing advice.

The beauty of this design is its graceful fallback: if LangChain isn’t installed or the model fails to initialize (for instance, due to missing API keys), the code reverts to deterministic templates. The reflection loop and evaluator remain unchanged, so the orchestrator–worker architecture stays intact. Only the plan generation mechanism becomes dynamic when an LLM is available. This modularity demonstrates how real‑world systems can incrementally adopt generative AI without sacrificing robustness.

Code Implementation

To put these ideas into practice, we provide a fully working Python module that mirrors the reflection loop described in the Skills Network lab while swapping the real investors for our fictional superstars. The module defines a shared state dictionary, prompts for each persona, an evaluator inspired by Richie Rich’s prudence, and a simple Python loop that iteratively generates and critiques investment plans until the risk grade matches the target.

Key components include:

determine_target_grade – uses an LLM to pick a suitable risk grade for the investor based on their profile.
investment_plan_generator – produces an initial Bobby Axelrod–style plan and refines it using Gordon Gekko’s opportunistic rules when feedback exists.
evaluate_plan – calls a Richie Rich–style evaluator to classify the risk level and provide feedback on diversification, volatility and capital preservation.
route_investment – decides whether to accept the current plan or request another iteration, enforcing a maximum number of loops.
run_reflection_workflow – orchestrates the entire generate–evaluate–route cycle in plain Python without relying on LangGraph, returning the final plan along with feedback and iteration count.

Here is a high‑level sketch of the module’s structure (see langgraph-cookbook/06 - Investment Agent at main · JordiCorbilla/langgraph-cookbook for complete code):


from typing import TypedDict, Dict, Literal, Annotated
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

grades = Literal["ultra-conservative", "conservative", "moderate", "aggressive", "high risk"]

class State(TypedDict):
    investment_plan: str
    investor_profile: str
    target_grade: grades
    feedback: str
    grade: grades
    n: int

# Initialise the LLM (gpt-4o-mini) and build prompt templates for each persona

# … Bobby Axelrod prompt and pipeline
# … Gordon Gekko prompt and pipeline with adaptation rules
# … Richie Rich evaluator prompt and structured output

def determine_target_grade(state: State) -> Dict[str, grades]:
    # Ask the LLM to choose the target risk grade based on the profile
    pass

def investment_plan_generator(state: State) -> Dict[str, str]:
    # Generate or refine the plan using Bobby or Gordon prompts
    pass

def evaluate_plan(state: State) -> Dict[str, object]:
    # Evaluate the plan with the Richie Rich evaluator
    pass

def route_investment(state: State, iteration_limit: int = 5) -> str:
    # Decide whether to accept the plan or loop again
    pass

def run_reflection_workflow(profile: str) -> State:
    # Orchestrate the reflection loop until the target grade is met or the iteration limit is reached
    pass

The complete implementation can be downloaded as custom_investment_reflection.py (file ID :agentCitation{citationIndex='0'}). This script uses LangChain’s ChatOpenAI to generate and evaluate plans, falls back gracefully if the LLM isn’t available, and demonstrates how to adapt the reflection pattern to custom personas.

Sample Execution

As an example, suppose you run run_reflection_workflow() from the custom_investment_reflection.py module with a 35‑year‑old investor earning $100 k, holding $100 k in assets, aiming to retire by 55 with a generous travel budget, and tolerating high risk. The workflow determines an aggressive target grade and iteratively refines the plan. One possible output (actual results vary slightly with the generative model) is summarised below:


🎯 Final Investment Plan Summary
========================================

📌 Investor Profile:
Age: 35
Salary: $100,000
Assets: $100,000
Goal: Retire comfortably by age 55 with ample travel budget
Risk tolerance: high

📈 Target Risk Grade: aggressive
📊 Final Assigned Grade: aggressive
🔁 Iterations Taken: 2

📝 Evaluator Feedback:
----------------------------------
This portfolio is classified as aggressive due to its heavy allocation to equities (75%) and lower commitment to fixed income.  Ensure the investor is comfortable with higher volatility and periodic drawdowns.

📃 Final Investment Plan:
----------------------------------
Gordon Gekko advises the investor to pursue an assertive course:

### Asset Allocation
- **Equities (75%)**: Concentrated bets in blue chips and cyclical industries, supplemented by option strategies.
- **Alternatives (15%)**: Distressed debt, merger arbitrage and real‑estate investment trusts.
- **Fixed Income (5%)**: Opportunistic short‑dated corporate notes.
- **Cash (5%)**: Dry powder for takeover opportunities.

### Rationale
Prioritizing blue chip equities and sector ETFs with active options overlays, complemented by distressed debt and merger arbitrage funds.  Minimal fixed income ensures capital is working hard.

### Risk Management
Use stop‑loss orders and hedges; rebalance monthly to maintain leverage discipline.

The first iteration used the Bobby Axelrod persona, which produced a 70 % equity allocation (moderate grade). The evaluator recommended increasing risk exposure, and the orchestrator rotated to Gordon Gekko, whose 75 % equity allocation met the target. The system stopped after two iterations.

Pros and Cons of the Reflection Pattern in Finance

Advantages

Improved quality through self‑critique: By iteratively evaluating its own recommendations, the advisor reduces the likelihood of omissions or contradictory allocations. It mirrors the human practice of drafting and revising (philschmid.de).
Personalisation: Different personas allow the system to explore varied strategies. Investors can compare aggressive, moderate and conservative approaches without requiring multiple human advisors.
Generative creativity: When an LLM is available, each persona can craft bespoke investment narratives instead of fixed templates. This results in plans that read more naturally and incorporate subtle nuances while still adhering to the requested risk grade.
Reduced human bias: Using fictional characters avoids overreliance on real gurus. The system still captures different risk appetites but does not misappropriate any celebrity’s actual statements.
Extensibility: New personas or evaluators can be added without changing the core loop, demonstrating the modularity of the orchestrator–worker pattern (dzone.com).
Autonomy and scalability: The loop can be run asynchronously, and evaluation metrics can be refined over time to align with regulatory guidelines or individual preferences.

Limitations

Heuristic evaluator: Our evaluation function parses simple percentages and assigns grades. Real‑world risk analysis is far more complex, involving volatility, correlation and drawdown metrics. More sophisticated evaluators (e.g., risk‑parity or Monte Carlo simulators) could improve accuracy but also increase complexity.
Computational cost: Iterative generation and evaluation consume more compute cycles and may require multiple LLM calls. For high‑frequency use cases, latency and API cost could become prohibitive.
Unpredictability: Generative models may occasionally omit required allocation details or introduce inconsistent formatting. Although the evaluator catches gross mismatches, manual review is advisable when using LLM‑generated plans.
Potential oscillations: Without careful stopping rules, the system might ping‑pong between personas or produce contradictory adjustments. Clear termination criteria and penalizing repeated mistakes are essential (analyticsvidhya.com).
Lack of external context: Our fictional personas rely on templates rather than real market data. Integrating live data feeds and dynamic modelling would make the advice more actionable but also raises regulatory and liability considerations.

Conclusion

The reflection pattern offers a simple yet powerful mechanism for improving AI outputs by letting the model serve as its own critic. When combined with an orchestrator–worker architecture, it provides a modular and extensible framework for tasks like investment advice, where balancing risk and return requires careful iteration. By experimenting with fictional personas, we avoid real‑world bias while illustrating how different risk appetites can be encoded in code. Although our current implementation uses heuristic evaluators and templated advice, it lays the foundation for more sophisticated systems that combine generative AI, financial models and ethical constraints.

Benchmarking Dynamic Quantization for Larger Language Models

noreply@blogger.com (Jordi Corbilla) — Sat, 27 Sep 2025 11:44:00 +0000

Benchmarking Dynamic Quantization for Larger Language Models

Modern language models can answer complex queries and generate high‑quality text, but they are heavy: hundreds of megabytes of weights have to be loaded into memory and millions of multiply and accumulate operations run per query. To reduce latency and cost, practitioners use optimisation techniques such as model distillation, quantization, pruning, dynamic/continuous batching, and KV‑cache reuse. When these methods are combined, costs can be slashed by up to 80 % while maintaining acceptable qualitylatitude-blog.ghost.io. This post focuses on dynamic INT8 quantization for a medium‑sized GPT‑2 model, shows the performance difference between full‑precision and quantised models, and discusses trade‑offs.

Why Quantization?

Quantization reduces the precision of a model’s weights from 32‑bit floating‑point values to 8‑bit (or lower) integers. PyTorch’s quantization documentation notes that INT8 quantization can shrink the model size and memory bandwidth by roughly 4× and that INT8 operations are typically 2–4 × faster than FP32 (docs.pytorch.org). For example, a 500‑million‑parameter model occupying 2 GB in FP32 can be reduced to 0.5 GB when quantized to INT8 (latitude-blog.ghost.io). This smaller footprint allows large language models to run on memory‑constrained devices and reduces inference latency.

Weight‑only quantization (e.g. 4‑bit AWQ or GPTQ) can improve speed even more. In real deployments, quantized LLMs show dramatic throughput gains: DeepSeek‑7B’s throughput on an NVIDIA RTX 4090 increases from 52 tokens/s to 130 tokens/s using AWQ, while Mistral‑7B on an AWS G5.xlarge instance jumps from 28 tokens/s to 88 tokens/s (latitude-blog.ghost.io). A 4‑bit GPTQ version of Llama‑3.2 1B maintained its F1 score and gained 30 % speed (latitude-blog.ghost.io).

Despite these benefits, quantization can degrade output quality if not applied carefully. Lower‑precision representations introduce rounding errors and remove representational capacity. Advanced techniques like AWQ/GPTQ mitigate this by weighting channel scaling per layer and calibrating on sample data. Our example uses PyTorch’s simple dynamic quantization, which is less powerful than weight‑only methods but easy to apply.

The Example: Measuring FP32 vs. INT8 GPT‑2

We built a script (inference_example_large.py) that loads GPT‑2 (124M parameters), processes a handful of prompts, quantizes the model dynamically, and then processes the same prompts. The key steps are:

Load model and tokenizer. We use Hugging Face’s AutoModelForCausalLM with the GPT‑2 checkpoint and its tokenizer. Because GPT‑2 lacks a padding token, we assign the end‑of‑sequence token (eos_token) as the pad token and set padding_side='left' to avoid runtime warnings.
Run FP32 inference. The script encodes a list of prompts, generates up to 50 new tokens for each, and measures total time.
Apply dynamic quantization. torch.quantization.quantize_dynamic quantizes only linear layers in the model (attention projections and feed‑forward layers). Embedding and layer‑norm parameters remain in FP32, so the model size reduction is limited.
Run INT8 inference with the quantized model and measure time.

Here is a condensed version of the code’s core logic (full code here: JordiCorbilla/langgraph-cookbook: langgraph-cookbook):


from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, time, psutil

model_name = "gpt2"
dev = torch.device("cpu")

# Load and patch tokenizer (GPT-2 has no pad token)
tokenizer = AutoTokenizer.from_pretrained(model_name)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token
    tokenizer.padding_side = "left"

# Load full-precision model
model = AutoModelForCausalLM.from_pretrained(model_name).to(dev)
model.eval()

prompts = ["Write a poem about quantization.",
           "Why is dynamic batching useful?",  # 4 prompts total...
          ]
max_new_tokens = 50

# FP32 inference
start = time.time()
for p in prompts:
    inp = tokenizer(p, return_tensors="pt").to(dev)
    out = model.generate(
        input_ids=inp["input_ids"],
        attention_mask=inp["attention_mask"],
        max_new_tokens=max_new_tokens,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )
end = time.time()
fp32_time = end - start

# Dynamically quantize linear layers
model_int8 = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

# INT8 inference
start = time.time()
for p in prompts:
    inp = tokenizer(p, return_tensors="pt").to(dev)
    out = model_int8.generate(
        input_ids=inp["input_ids"],
        attention_mask=inp["attention_mask"],
        max_new_tokens=max_new_tokens,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )
end = time.time()
int8_time = end - start

Results

Our run on a Windows laptop (CPU) produced the following numbers:

Model	Precision	Weight size	Total time for 4 prompts	Speedup
GPT‑2	FP32	~474 MB	~6.57 s	1×
GPT‑2	INT8 dynamic	~474 MB*	~5.78 s	1.14×

*Dynamic quantization only converts nn.Linear layers. Embedding and layer‑norm weights remain in FP32, so the overall model footprint does not shrink. Weight‑only quantization or AWQ/GPTQ can compress all weights and yield 4×–8× reductions (docs.pytorch.orglatitude-blog.ghost.io).

Interpretation

Modest gains on small models. On a 124M‑parameter GPT‑2, dynamic quantization offered only a 14 % speedup and no memory savings. This is because the model’s largest tensors are the embeddings and LM head, which were not quantized. Dynamic quantization shines on CPU‑bound workloads with many linear operations but yields diminishing returns on small models or GPU‑accelerated inference.
Larger models benefit more. Weight‑only quantization of big models (e.g., Llama‑3 70B) can reduce memory from 2 GB to 0.5 GB and double token throughput (latitude-blog.ghost.iolatitude-blog.ghost.io). AWQ and GPTQ compress all weights, producing 2×–4× speedups while preserving or even improving some evaluation metrics (latitude-blog.ghost.io).
Accuracy trade‑offs. Lower‑bit quantization introduces rounding error. While our 8‑bit dynamic model produced sensible text, 4‑bit quantization must be applied with care (e.g., per‑channel scaling, calibration) to avoid harming output quality. The Latitude article emphasises that quantization maintains accuracy “good enough for most applications” (latitude-blog.ghost.io), but domain‑specific tasks may require careful evaluation.

Continuous and Dynamic Batching

Quantization isn’t the only lever for faster LLMs. Dynamic or continuous batching groups multiple inference requests arriving close in time into a single batch. Instead of running one forward pass per request, the server merges them into micro‑batches and runs a single forward pass. Systems like vLLM achieve up to 23× throughput gains by continuously adding new prompts mid‑generation and sharing key/value caches across requests.

In our earlier example (with a tiny MLP), we saw that the first request took ~23 ms, while subsequent requests processed within 0.5–2 ms because they piggy‑backed on the existing batch. Continuous batching is especially powerful when serving interactive chat applications where many users send short messages concurrently.

Conclusion

Dynamic quantization is an easy way to reduce the computational cost of LLM inference, but its benefits scale with model size. Our GPT‑2 experiment shows only modest gains because embeddings dominate memory usage and remain in FP32. For significant improvements, practitioners should adopt weight‑only quantization and pair it with continuous batching, KV‑cache optimisation, and model distillation. These methods, when applied thoughtfully, can make deploying large language models feasible even in cost‑constrained environments.

ReAct, For Real: Building Deterministic Tool-Using Agents with LangGraph

noreply@blogger.com (Jordi Corbilla) — Sat, 20 Sep 2025 13:07:00 +0000

ReAct, For Real: Building Deterministic Tool-Using Agents with LangGraph

(feat. a Safe AST Calculator)

A practical guide to wiring a reason + act loop that’s auditable, stable, and observable. We use LangGraph for orchestration and a hardened AST calculator for math, no evals, no vibes.

Why this pattern

Determinism over vibes. The calculator is pure Python AST (whitelisted ops), so results are exact and testable.
Controlled loop. First model step is forced to call the tool; the second cannot call tools and must answer. No infinite ping-pong.
Observability. Streamed console output shows precisely when the LLM runs vs. when the tool runs, with token usage.

Architecture at a glance


┌──────────────┐     (tools_condition)      ┌────────────┐
│   agent LLM  │ ─────────────────────────► │  ToolNode  │
│ (forced 1st) │                            │ calculator │
└──────┬───────┘                            └─────┬──────┘
       │                                          │ ToolMessage
       │ no tool calls → END                      ▼
       │                                    ┌──────────────┐
       └─────────────────────────────────── │  agent LLM   │
                                            │ (free finish)│
                                            └──────┬───────┘
                                                   │ AiMessage (final)
                                                   ▼ END

Key primitives:

MessagesState: appends messages, preserving the OpenAI tool-calling protocol.
ToolNode: executes declared tools.
tools_condition: routes to tools only when the assistant produced tool calls.
One-tool-call policy: first LLM is bound to the calculator; the finisher LLM is tool-free.

The safe calculator (snippet)

Goal: evaluate math deterministically, with a tiny NL preprocessor.


# Whitelisted ops only
_ALLOWED_OPS = {ast.Add: op.add, ast.Sub: op.sub, ast.Mult: op.mul,
                ast.Div: op.truediv, ast.Pow: op.pow, ast.Mod: op.mod,
                ast.UAdd: op.pos, ast.USub: op.neg}

def _preprocess(s: str) -> str:
    s = s.strip().lower()
    s = re.sub(r"\b(please|thanks|thank you|what\s*is|what's|calculate|compute|equals?)\b", "", s)
    s = re.sub(r"\bthe\b", "", s)
    s = re.sub(r"\bplus\b", "+", s); s = re.sub(r"\bminus\b", "-", s)
    s = re.sub(r"\btimes\b", "*", s); s = re.sub(r"\bdivided by\b", "/", s)
    s = re.sub(r"\b(?:the\s+)?square root of\b", "sqrt(", s)
    s = re.sub(r"√\s*", "sqrt(", s)
    s = re.sub(r"sqrt\(\s*([0-9\.]+)\s*\)?", r"sqrt(\1)", s)
    s = re.sub(r"(\d+(?:\.\d+)?)\s*%\s*of\s*([0-9\.]+)", r"(\1/100*\2)", s)
    s = re.sub(r"(\d+(?:\.\d+)?)\s*%", r"(\1/100)", s)
    return s.replace("^", "**").strip()

def _eval_ast(node: ast.AST) -> float:
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return float(node.value)
    if isinstance(node, ast.UnaryOp) and type(node.op) in _ALLOWED_OPS:
        return _ALLOWED_OPS[type(node.op)](_eval_ast(node.operand))
    if isinstance(node, ast.BinOp) and type(node.op) in _ALLOWED_OPS:
        return _ALLOWED_OPS[type(node.op)](_eval_ast(node.left), _eval_ast(node.right))
    if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "sqrt" and len(node.args) == 1:
        return math.sqrt(_eval_ast(node.args[0]))
    if isinstance(node, ast.Expr):
        return _eval_ast(node.value)
    raise ValueError("Unsupported syntax.")

Tool wrapper:


@tool("calculator_tool")
def calculator_tool(expression: str) -> str:
    try:
        pre = _preprocess(expression)
        value = _eval_ast(ast.parse(pre, mode="eval").body)
        return str(float(value))
    except Exception as e:
        return f"Error: {e}"

Why this is safe: Only numeric literals and whitelisted operators/functions are allowed; no names, attrs, or imports can ever execute.

LangGraph wiring (snippet)

Bind two LLM phases: forced first (must call calculator) and free finish (no tools bound).


llm_forced_first = ChatOpenAI(...).bind_tools(
    [calculator_tool],
    tool_choice={"type":"function","function":{"name":"calculator_tool"}}
)
llm_free = ChatOpenAI(...)

def _has_calc_tool_call(msgs) -> bool:
    return any(isinstance(m, ToolMessage) and getattr(m,"name","")=="calculator_tool" for m in msgs)

def agent_node(state: MessagesState):
    first_phase = not _has_calc_tool_call(state["messages"])
    llm = llm_forced_first if first_phase else llm_free
    ai = llm.invoke([SystemMessage(content=SYSTEM_RULE), *state["messages"]],
                    config={"tags": ["agent_llm", "forced_first" if first_phase else "free_finish"]})
    return {"messages": [ai]}

tool_node = ToolNode([calculator_tool])

g = StateGraph(MessagesState)
g.add_node("agent", agent_node)
g.add_node("tools", tool_node)
g.add_conditional_edges("agent", tools_condition, {"tools": "tools", "__end__": END})
g.add_edge("tools", "agent")
g.set_entry_point("agent")
graph = g.compile()

Why it ends cleanly: After tools run, the graph goes back to the agent once to produce the final answer; tools_condition routes to END if no tool calls are present.

Observability: know exactly when the LLM is used

Add a tiny callback to print LLM start/end and token usage; gate with DEBUG_AGENT=1.


class ConsoleLLMHandler(BaseCallbackHandler):
    def on_llm_start(self, serialized, prompts, **kw):
        if os.getenv("DEBUG_AGENT")=="1":
            print(f"[LLM START] {serialized.get('name','LLM')} | tags={kw.get('tags',[])}", file=sys.stderr)
    def on_llm_end(self, result, **kw):
        if os.getenv("DEBUG_AGENT")=="1":
            print(f"[LLM END] tags={kw.get('tags',[])} | usage={(result.llm_output or {}).get('token_usage',{})}", file=sys.stderr)

handler = ConsoleLLMHandler()
llm_forced_first = ChatOpenAI(..., callbacks=[handler]).bind_tools(...)
llm_free = ChatOpenAI(..., callbacks=[handler])

Optional: DEBUG_CALC=1 prints NL → normalized math expression for the tool.

Running it


export OPENAI_API_KEY=sk-...
# telemetry on:
export DEBUG_AGENT=1
export DEBUG_CALC=1

python calculator_agent.py "Calculate 12% of 255 plus the square root of 244"

Sample (trimmed) output:

================================ Human Message =================================

Calculate 12% of 255 plus the square root of 244

[LLM START] ChatOpenAI | tags=['seq:step:1', 'agent_llm', 'forced_first']
[LLM END] tags=['seq:step:1', 'agent_llm', 'forced_first'] | usage={'completion_tokens': 14, 'prompt_tokens': 161, 'total_tokens': 175, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}
================================== Ai Message ==================================
Tool Calls:
  calculator_tool (call_134y7dqSU2udc2rr0OTNHKQs)
  Args:
    expression: 12% of 255 plus sqrt(244)

================================= Tool Message =================================
Name: calculator_tool

46.2204993518133

[LLM START] ChatOpenAI | tags=['seq:step:1', 'agent_llm', 'free_finish']
[LLM END] tags=['seq:step:1', 'agent_llm', 'free_finish'] | usage={'completion_tokens': 23, 'prompt_tokens': 121, 'total_tokens': 144, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}
================================== Ai Message ==================================

The result of 12% of 255 plus the square root of 244 is approximately 46.22.

Design choices: pros & cons

Pros

Deterministic math; no LLM hallucinations for arithmetic.
Single tool call guarantees predictable latency and flow.
Great DX: streaming, token accounting, and NL conveniences (% of, √, word operators).

Cons

Narrow by design (sqrt, basic ops). Extend via whitelist if you need log/exp/pi/e.
Requires an LLM key for orchestration (the math itself is local).
NL preprocessor is conservative to keep parsing predictable.

Code & assets

Source: $https://github.com/JordiCorbilla/langgraph-cookbook$

Snippet appendix

System rule (keeps the agent honest)


SYSTEM_RULE = (
  "You are a math assistant. For ANY numeric computation, "
  "call `calculator_tool` exactly once, then return the final answer. "
  "Do not call tools again after you have the numeric result."
)

Smoke test idea


# Expect: ≥2 AI messages (forced+final), exactly 1 ToolMessage, final AI has no tool_calls

A Practical, Production-Ready Tool for Summarizing YouTube Videos with LLMs and Agents

noreply@blogger.com (Jordi Corbilla) — Sun, 14 Sep 2025 11:57:00 +0000

A Practical, Production-Ready Tool for Summarizing YouTube Videos with LLMs and Agents

This post is about a tool. It’s a CLI that uses LLM tool calling and a tiny agent loop to extract IDs, fetch transcripts/metadata, and produce high-quality summaries of long YouTube videos without tripping rate limits or context windows.

TL;DR

What it is	Why it’s useful	How it works	How to run
A CLI tool that summarizes YouTube videos using an LLM with tools/agents	Handles long videos and real-world edge cases (no transcript, quotas)	LangChain tool calling+ fixed/recursive chains + chunked map-reduce summarization + bare-LLM finalization	`python youtube_tool_agent.py summarize --url "<video-url>" --language en`

What this tool does

Extracts YouTube video IDs (robust to watch?v=, youtu.be/, embed/)
Fetches transcripts (and can be extended to fall back to captions/description via yt-dlp)
Pulls metadata & thumbnails via yt-dlp (title, views, likes, chapters, image sizes)
Searches YouTube by query (PyTube)
(Best-effort) Trending by region
Generates succinct, expert-oriented summaries of long videos using a chunked map-reduce flow that avoids TPM/context explosions
Includes a CLI and verbose tracing so you can see exactly what the agent is doing

Install & first run


# Python 3.11+ recommended
pip install -r requirements.txt

# Set your model key (OpenAI shown)
export OPENAI_API_KEY="sk-..."   # Windows PowerShell:  $env:OPENAI_API_KEY="sk-..."

# Summarize a video
python youtube_tool_agent.py --verbose summarize \
  --url "https://www.youtube.com/watch?v=8TJQhQ2GZ0Y" \
  --language en

You’ll see progress like:


[LLM] OpenAI provider model=gpt-4o provider=openai
[SUM] Chunk 1/2 (30000 chars)
[SUM] Chunk 2/2 (22007 chars)
<final polished summary printed here>

Why agents for this problem?

A “just call an API and prompt” script breaks on real-world YouTube:

Some videos have no official transcript (only auto-captions or nothing).
Transcripts can be huge, exceeding model context or your org’s TPM limits.
Tool-calling models often respond with empty content + a request to call another tool, unless you orchestrate correctly.

An agent (even a tiny one) solves this:

It decides which tool to call and in what order (ID → transcript → metadata → summarize).
It loops until it has what it needs (recursive chain), then finalizes the answer.
You get traceability: every tool call is logged with payload sizes so you can reason about cost and latency.

Core ideas (architectural choices that matter)

1) Tool calling (LangChain)

We expose capabilities as annotated tools the model can call by name:

extract_video_id, fetch_transcript, get_full_metadata, get_thumbnails, search_youtube, get_trending_videos.

These functions return plain JSON/strings, which keeps the glue simple and makes debugging obvious.

Source code can be found here:
JordiCorbilla/langgraph-cookbook: langgraph-cookbook

2) Two orchestration modes

Fixed chain for deterministic summarize:
1. Extract ID → 2) Fetch transcript → 3) Summarize (chunked) → 4) Polish (bare LLM)
Universal (recursive) chain for open “ask”:
- Keep calling tools while the model requests them. Stop when it returns text.

3) Chunked map-reduce summarization

Why: long transcripts blow up context and TPM.
How: split the transcript into overlapping chunks → summarize each chunk (map) → merge the bullet lists (reduce) → optional polish for clarity.

Key tunables:

Chunk size: 3k–4k chars (or larger if you have headroom)
Overlap: 200–300 chars
Short sleep between chunk calls to smooth TPM bursts

4) Bare-LLM finalization

Tool-calling models often return empty content when they want another tool. The final step uses a bare LLM (no tools bound). That forces a textual answer and prevents “one more tool?” loops.

Architecture at a glance


┌──────────────┐      ┌───────────────────────┐
│  CLI (args)  ├────▶ |  Orchestrator Chain   │
└─────┬────────┘      └──────────┬────────────┘
      │                           │  tool_calls
      │                           ▼
      │                 ┌─────────────────────┐
      │                 │  Tools (LangChain)  │
      │                 │  - extract_video_id │
      │                 │  - fetch_transcript │
      │                 │  - search_youtube   │
      │                 │  - get_full_metadata│
      │                 │  - get_thumbnails   │
      │                 └──────────┬──────────┘
      │                            │ results (text/JSON)
      │                            ▼
      │                   ┌─────────────────┐
      │                   │  Bare LLM Call  │  ← no-tools finalization
      │                   │ (map→reduce→polish)
      ▼                   └─────────────────┘
  stdout (summary)

Why “bare” LLM at the end? Tool-calling models often return empty content when they still want to call tools. Finalizing with a no-tools model guarantees text.

CLI you’ll actually use

Point tools (fast, no LLM):


python youtube_tool_agent.py metadata   --url "https://youtu.be/8TJQhQ2GZ0Y"
python youtube_tool_agent.py thumbnails --url "https://youtu.be/8TJQhQ2GZ0Y"
python youtube_tool_agent.py search     --query "Retrieval-Augmented Generation"
python youtube_tool_agent.py trending   --region GB

Deterministic, chunked summary:


python youtube_tool_agent.py --verbose summarize \
  --url "https://www.youtube.com/watch?v=8TJQhQ2GZ0Y" \
  --language en

Agentic “do what it takes”:


python youtube_tool_agent.py --verbose ask \
  --query "Summarize this YouTube video and explain the leveraging strategy (en): https://www.youtube.com/watch?v=8TJQhQ2GZ0Y"

Under the hood (high-level)


CLI → Orchestrator
  ├─ Fixed chain (summarize): ID → transcript → map→reduce→polish (bare LLM)
  └─ Universal chain (ask): recurse while the model returns tool_calls
Tools (LangChain @tool)
  ├─ extract_video_id
  ├─ fetch_transcript  (YouTubeTranscriptAPI; optional yt-dlp captions fallback)
  ├─ get_full_metadata, get_thumbnails
  ├─ search_youtube, get_trending_videos (best-effort)
Finalization
  └─ Bare LLM (no tools) → guaranteed text

Why the fallback? In production you’ll meet videos with no official transcript. Add a yt-dlp captions fallback and, as a last resort, summarize metadata/description so you always produce something useful.

Operational tips (so it stays fast and cheap)

TPM discipline: throttle chunk calls with short sleeps; prefer smaller models for map steps, then a slightly stronger model for merge/polish.
Chapters-aware chunking: if yt-dlp returns chapters, chunk on chapter boundaries first—better topical coherence and fewer duplicates.
Cache transcripts/metadata to disk (simple JSON files). You’ll save both time and money on reruns.
Observability: log tool names and payload sizes. When costs drift, you’ll know why.

Closing thoughts

This is a tool designed for the messiness of real YouTube content. The combination of tool calling, a minimal agent loop, chunked summarization, and bare-LLM finalization makes it predictable, debuggable, and resilient. You can drop it into a pipeline today and get reliable results on long videos.

Querying and Plotting Data with LangChain’s Pandas Agent + OpenAI

noreply@blogger.com (Jordi Corbilla) — Sun, 31 Aug 2025 20:17:00 +0000

Querying and Plotting Data with LangChain’s Pandas Agent + OpenAI

Natural language interfaces for data analysis are moving from research into everyday engineering practice.
With LangChain and OpenAI, you can now query a DataFrame directly in English, let the model write and execute the Pandas/Matplotlib code, and even plot results, all inside Python.

In this post, I’ll show you how I built a Pandas DataFrame Agent with LangChain + OpenAI that can:

Answer tabular questions about a dataset.
Generate Python plotting code automatically.
Execute that code safely to produce charts.

🔧 Setup

We need a few packages:


pip install langchain langchain-openai langchain-experimental pandas matplotlib

And of course, set your OpenAI key:


export OPENAI_API_KEY=...

📊 The Dataset

For demo purposes, let’s mock up some sales data:


import pandas as pd

data = {
    "month": ["2025-01","2025-02","2025-03","2025-04","2025-05","2025-06"],
    "region": ["EMEA","EMEA","EMEA","AMER","AMER","APAC"],
    "units":  [120, 150, 130, 200, 180, 160],
    "price":  [10.0, 10.0, 10.0, 12.0, 12.0,  9.0],
}
df = pd.DataFrame(data)
df["revenue"] = df["units"] * df["price"]

🤖 Creating the Agent

The magic comes from create_pandas_dataframe_agent.
This wraps the DataFrame in a tool the LLM can call with code execution:


from langchain_openai import ChatOpenAI
from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

agent = create_pandas_dataframe_agent(
    llm=llm,
    df=df,
    agent_type="openai-tools",
    verbose=False,  # cleaner logs
)

🔎 Asking Questions in English

Now you can query the data without touching Pandas:


resp = agent.invoke({
    "input": "Compute total revenue by region as a tidy table sorted descending."
})
print(resp["output"])

Output:


| region | revenue |
|--------|---------|
| AMER   | 4560.0  |
| EMEA   | 4000.0  |
| APAC   | 1440.0  |

The agent generated and executed the Pandas code under the hood.

📈 Asking for a Plot

We can go one step further — ask the model to plot monthly revenue:


plot_code = """
import pandas as pd
import matplotlib.pyplot as plt
from pathlib import Path

df['month'] = pd.to_datetime(df['month'])
monthly_revenue = df.groupby('month')['revenue'].sum()

plt.figure(figsize=(10, 5))
plt.plot(monthly_revenue.index, monthly_revenue.values, marker='o')
plt.title('Monthly Revenue Over Time')
plt.xlabel('Month')
plt.ylabel('Total Revenue')
plt.grid()
plt.xticks(rotation=45)
plt.tight_layout()

out = Path('monthly_revenue.png')
plt.savefig(out)
print(f'Chart saved to: {out.resolve()}')
"""
agent.invoke({"input": f"Execute this exact code:\n```python\n{plot_code}\n```"})

This produces:

🚀 Why This Matters

This pattern unlocks:

Rapid ad-hoc analytics: Ask questions in English, get code + charts.
Non-technical users can explore data without writing Pandas.
Bridges RAG + analytics: Instead of only text retrieval, you can augment LLMs with structured data queries.

Of course, for production you’d want guardrails (e.g., AST linting before execution, schema checks, sandboxing). But as a prototyping tool, this workflow is incredibly powerful.

Source code can be found here: JordiCorbilla/langgraph-cookbook: langgraph-cookbook

Building a Tiny LangGraph Agent with a Tool: A Weather-Aware Trip Planner

noreply@blogger.com (Jordi Corbilla) — Sun, 31 Aug 2025 19:57:00 +0000

Building a Tiny LangGraph Agent with a Tool: A Weather-Aware Trip Planner

Most agent demos are either too simple (toy calculators) or too complex (enterprise-scale pipelines).
Here’s a middle ground: a 5-node LangGraph agent that builds a half-day itinerary for a city, adapts to weather, respects user preferences, and calls a real tool along the way.

Why this example?

Small, visualizable graph: plan → weather → search → pick → draft → finalize
Conditional routing: rainy → indoor picks, sunny → outdoor picks
Real tool usage: random_weather is a LangChain tool, called explicitly from the graph
Typed state with Pydantic v2
Optional LLM for nicer itinerary text (but not required), this example used OpenAI
Two runnable scenarios: London (rainy) and Barcelona (sunny)

Architecture

Nodes
- plan: log initial context
- weather: calls random_weather tool → Weather state
- search: load city places
- pick: filter by weather & prefs (budget, dislikes)
- draft: generate itinerary text
- fallback: relax prefs if nothing chosen
- finalize: wrap up
Edges
- Normal path: plan → weather → search → pick → draft → finalize
- Fallback path: pick → fallback → draft → finalize

The Weather Tool


from langchain_core.tools import tool
import random

@tool
def random_weather(city: str) -> dict:
    """
    Return randomized weather for the city.
    """
    conditions = ["sunny", "cloudy", "rain", "storm"]
    cond = random.choice(conditions)
    low, high = (12, 30)
    return {
        "city": city,
        "condition": cond,
        "high_c": float(random.randint(20, high)),
        "low_c": float(random.randint(low, 19))
    }

And in the weather_node:


def weather_node(s: GraphState) -> GraphState:
    w = random_weather.invoke({"city": s.city})  # tool call
    s.weather = Weather(**w)
    s.audit.append(
        f"weather: {s.weather.city} {s.weather.condition} "
        f"{s.weather.high_c}/{s.weather.low_c}°C"
    )
    return s

Full code can be found here -> JordiCorbilla/langgraph-cookbook: langgraph-cookbook

Example Run

Scenario A: London, dislikes museums, free-only budget


=== Scenario A: London + dislikes=['museum'], free_only ===
[Node] plan
[Node] weather
[Tool] random_weather(London) -> {'city': 'London', 'condition': 'sunny', 'high_c': 30.0, 'low_c': 28.0}
[Node] search
[Node] pick
[Node] draft
[Node] finalize
plan: city=London, half_day=morning, date=2025-08-31
weather: London sunny 30.0/28.0°C
search: 5 candidates
pick: Thames Riverside Walk, Hyde Park Loop, Sky Garden
draft: itinerary ready
finalize: done

Itinerary:
 - **Time:** 9:00 AM - 10:00 AM
 - **Tip:** Begin your day with a refreshing stroll along the river. Enjoy the views and the morning sun. Bring water and wear sunscreen!
 - **Transport:** Take the Tube from Embankment Station to Lancaster Gate (Circle or District Line).
 - **Tip:** This journey takes about 15 minutes. Consider grabbing a coffee from a nearby café before heading to the park.
 - **Time:** 10:15 AM - 11:30 AM
 - **Tip:** Enjoy the beautiful greenery and perhaps find a shaded spot to relax. Look out for the Serpentine Lake and the Diana Memorial Fountain.
 - **Transport:** Take the Tube from Lancaster Gate to Monument Station (Circle or District Line).
 - **Tip:** Allocate about 30 minutes for travel and check-in. Arrive by 11:45 AM to enjoy the stunning views of London.
 - **Time:** 12:00 PM - 12:45 PM
 - **Tip:** Enjoy a drink or snack at the café while taking in panoramic views of the city. Make sure to book your free entry in advance to avoid long waits!
Notes: Weather: sunny

Scenario B: Barcelona, no dislikes, mixed budget


=== Scenario B: Barcelona + no dislikes, mixed budget ===
[Node] plan
[Node] weather
[Tool] random_weather(Barcelona) -> {'city': 'Barcelona', 'condition': 'sunny', 'high_c': 29.0, 'low_c': 26.0}
[Node] search
[Node] pick
[Node] draft
[Node] finalize
plan: city=Barcelona, half_day=afternoon, date=2025-08-31
weather: Barcelona sunny 29.0/26.0°C
search: 5 candidates
pick: La Sagrada Família (Exterior), Gothic Quarter Stroll, Park Güell
draft: itinerary ready
finalize: done

Itinerary:
 - **Start at La Sagrada Família (3:00 PM - 3:45 PM)**
 - Arrive early to admire the stunning exterior of this iconic basilica. Take photos and enjoy the intricate details of Gaudí's masterpiece.
 - **Gothic Quarter Stroll (4:00 PM - 5:00 PM)**
 - Take a short taxi or tram ride (about 10-15 minutes) to the Gothic Quarter. Wander through the narrow streets, explore hidden squares, and admire the medieval architecture.
 - **Park Güell (5:15 PM - 6:45 PM)**
 - Head to Park Güell via taxi or public transport (approximately 15-20 minutes). Enjoy a leisurely walk through the park, taking in Gaudí's colorful mosaics and panoramic views of the city.
 - **Transport:** Use taxis or rideshare apps for quick transfers between locations, especially in the heat.
 - **Hydration:** Bring water to stay hydrated, as temperatures can be high.
 - **Footwear:** Wear comfortable shoes for walking on cobblestone streets and park paths.
Notes: Weather: sunny

Why this matters

Graph, not guesses: deterministic orchestration
Tools in action: explicit .invoke() calls with visible output
Policy in code: dislikes, budget filters, weather routing
Verbose logs: you can follow the agent step by step

Pros & Cons

Pros

Compact, easy-to-follow
Demonstrates both nodes and tools
Works with or without an LLM
Great for teaching graph concepts

Cons

Weather is mocked/randomized
Places are static, not from real APIs
No travel-time optimization

How to run


python trip_agent.py

Scenario A: London, free-only, dislikes museums → rainy picks
Scenario B: Barcelona, mixed budget → sunny picks

Build Your Own Local Copilot in VS Code with Continue + Ollama (CodeLlama‑7B)

noreply@blogger.com (Jordi Corbilla) — Tue, 22 Apr 2025 08:24:00 +0000

Build Your Own Local Copilot in VS Code with Continue + Ollama (CodeLlama‑7B)

TL;DR — In under 30 minutes you can have a fully‑offline, ChatGPT‑style coding assistant that reads your entire codebase and never leaks a byte to the cloud.

1 · Why bother?

Privacy first — No proprietary code leaves your laptop.
Low latency — Replies stream in milliseconds, not seconds.
Free — No API keys, no per‑token billing.
Hackable — Swap models, add custom prompts, or fine‑tune later.

2 · What we’ll build

A local stack where:

Ollama runs the codellama:7b model at http://localhost:11434.
Continue in VS Code chats with that model.
Continue indexes all your repositories so you can ask questions like:
“Where do we create the AWS SQS client?”

3 · Prerequisites

Item	Minimum
OS	Windows 11 22H2+
RAM	16 GB (32 GB recommended)
GPU (optional)	NVIDIA card with ≥ 8 GB VRAM & latest CUDA driver
WSL 2	Auto‑installed by Ollama if missing
VS Code	1.88 or newer

4 · Install Ollama

Download the Windows installer from https://ollama.ai/download/windows.
Run the MSI and accept defaults.
Verify the CLI:
```
ollama --version   # e.g. v0.1.32
```

**Heads‑up ** — The installer registers Ollama as a Windows service and listens on port 11434.

5 · Pull CodeLlama‑7B

ollama pull codellama:7b    # 4‑bit Q4_0 fits in 8 GB RAM

You’ll see a 100 % progress bar; the model lives in %USERPROFILE%\.ollama\models.

6 · Install Continue for VS Code

Open VS Code → Extensions.
Search “Continue” by Continue.dev and click Install.
Reload VS Code; a sunflower icon shows up in the Activity Bar.

7 · Point Continue at Ollama

Click the sunflower icon → gear ⚙ → Settings.
Choose Model Provider: Custom.
Fill the fields:
Field Value
Provider ollama
Base URL http://localhost:11434
Model codellama:7b
Press Save.

(This writes the same keys to your settings.json.)

8 · Index your repositories (no training required)

Add one line to User or Workspace settings:

"continue.directoryContext": [
  { "path": "D:/code", "depth": 4, "maxFiles": 25000 }
]

Continue walks the tree once, stores embeddings in %APPDATA%\continue\index, and keeps them fresh in the background.

9 · Smoke‑test the setup

Press Ctrl + Shift + L to open chat.

Type:

@Codebase Which file defines the KafkaConsumer settings?

The reply should include a Context block listing real file paths.
Optional: enable the Continue Console (Settings → Enable Console) to watch retrieved files live.

10 · Tame hallucinations (optional but recommended)

Add this to settings.json:

"continue.systemMessage": "You are a senior full‑stack engineer. Answer concisely and reference only provided context. If unsure, say 'IDK'.",
"continue.modelOptions": {
  "temperature": 0.2,
  "top_p": 0.9,
  "repeat_penalty": 1.1
}

11 · Daily cheat‑sheet

Task	How
Explain selected code	Select → right‑click → Continue: Explain
Generate tests	Same menu → Generate Tests
Inline autocomplete	Toggle Tab Autocomplete in Continue footer
Force retrieval	Prefix prompt with `@Codebase`, `@File`, or `@Folder`

12 · Troubleshooting

Symptom	Fix
`connect ECONNREFUSED 11434`	Tray → Start Ollama Service or `Start‑Service Ollama`
Model OOM on pull	Use quantised tag: `codellama:7b:Q4_0`
No Context block	Check `directoryContext` path / depth; rebuild index

13 · Next steps

Swap in StarCoder2‑15B or DeepSeek‑Coder‑33B for higher quality.
Drop temperature to 0 and let it draft unit tests you tweak.
Graduate to Tabby LoRA if you need inline completions that mimic your exact style.

From Zero to AI‑Ready: Python on Windows via WSL 2 and VS Code (GitHub + Codex Integration)

noreply@blogger.com (Jordi Corbilla) — Mon, 21 Apr 2025 10:53:00 +0000

From Zero to AI‑Ready: Python on Windows via WSL 2 and VS Code (GitHub + Codex Integration)

Why bother? Because production workloads still lean on Linux, and Windows Subsystem for Linux 2 (WSL 2) lets you run a full kernel without giving up Windows’ desktop conveniences. Follow this playbook and you’ll be writing, testing, and committing AI‑infused Python in under an hour—no VM gymnastics required.

Prerequisites at a Glance

Tool	Minimum Version	Why You Need It
Windows 10 2004 / Windows 11	with WSL 2	Provides the real Linux kernel
Ubuntu 22.04 LTS	or newer	Stable, long‑term support
VS Code 1.90 +	Editor with first‑class WSL remote support
Git 2.40 +	Version control muscle memory
GitHub account & SSH key	Passwordless pushes
OpenAI API key	Fuel for Codex AI and GitHub Copilot

1 Enable WSL 2 and Install Ubuntu

Open PowerShell as Administrator and run:

wsl --install                 # Installs WSL + default Ubuntu distro
wsl --set-default-version 2   # Make all future distros WSL 2

Reboot when prompted.

Sanity check:

wsl --status                  # Kernel ≥ 5.x, Default = 2

2 Update and Harden the Linux Side

Launch Ubuntu from the Start Menu:

sudo apt update && sudo apt dist-upgrade -y
sudo apt install -y build-essential python3 python3-pip python3-venv git

Need multiple Python versions? Install pyenv the reliable way:

curl https://pyenv.run | bash
# Follow the shell‑config instructions printed to the terminal

3 Wire VS Code into WSL

Install VS Code for Windows (normal installer).
Launch VS Code → Ctrl+Shift+X → install Remote — WSL.

Back in Ubuntu:

mkdir -p ~/code/my-python-project
cd ~/code/my-python-project
code .                     # VS Code opens inside WSL context

The status bar should now read “WSL: Ubuntu”.

4 Bootstrap a GitHub Repository (Properly)

Generate an SSH key inside WSL:

ssh-keygen -t ed25519 -C "your.email@example.com"
cat ~/.ssh/id_ed25519.pub   # Copy the public key

Add the key to GitHub → Settings → SSH and GPG keys.
Create a blank repo on GitHub (no README/LICENCE).

Clone into WSL:

git clone git@github.com:your‑org/my-python-project.git ~/code/my-python-project
cd ~/code/my-python-project

First commit:

python3 -m venv .venv && source .venv/bin/activate
echo "# My Python Project" > README.md
echo -e "venv/\n__pycache__/" > .gitignore
git add README.md .gitignore
git commit -m "chore: initial commit"
git push -u origin main

Rule of thumb: keep your SSH keys in WSL, not in the Windows ssh-agent—safer separation.

5 Add Codex AI to Your Workflow

5.1 Install Node 18 LTS & the Codex CLI

Inside WSL:

# Install Node via the Node Version Manager (nvm)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install --lts      # e.g. Node 18.x
nvm use --lts

# Install Codex CLI globally
npm install -g @openai/codex
codex --version        # should echo a semver like 0.2.x

5.2 Authenticate

export OPENAI_API_KEY="<your‑key>"   # add the line above to ~/.bashrc for permanence

5.3 First Run & Modes

cd ~/code/my-python-project
codex "Explain this repo to me."

By default Codex runs in Suggest mode (read‑only). Use --auto-edit to let it write files, or --full-auto for an autonomous agent that can run shell commands inside a sandbox. citeturn0search0

5.4 (Optional) Python SDK

If you also want to call the API directly from scripts:

pip install --upgrade openai

The same OPENAI_API_KEY environment variable is reused.

6 Commit the AI Helpers (Without Leaking Keys)

# Requirements
echo "# AI helpers\nopenai>=1.0" > requirements.txt
pip freeze --local > requirements.lock

git add requirements.txt requirements.lock codex_smoketest.py
# Double‑check no key files are staged
if git status | grep -q api_key; then echo "✗ Key leak detected"; exit 1; fi

git commit -m "feat: add OpenAI Codex integration"

git push

7 Best Practices Going Forward

One virtual environment per repo—time‑honoured and still right.
Use pre‑commit hooks for black, ruff, and isort.
Tune WSL filesystem performance: create /etc/wsl.conf inside Ubuntu:
```
[automount]
options = "metadata,umask=22,fmask=11"
```
Then wsl --shutdown from PowerShell to reload.
CI/CD parity: GitHub Actions’ ubuntu‑latest runner mirrors WSL—build once, run everywhere.

Conclusion

With WSL 2, Ubuntu, VS Code, GitHub, and Codex AI all humming together, you’ve fused a decades‑proven Linux toolchain with tomorrow’s AI‑driven coding assistants—without ever leaving Windows. Time to ship something awesome.

RiskOptima Options Analysis Suite

noreply@blogger.com (Jordi Corbilla) — Sun, 20 Apr 2025 19:15:00 +0000

Analyze IV, Simulate Greeks, and Backtest Strategies — All in One File

Built for speed. Tuned for precision. Engineered to empower.

This post lays out a fully functional options trading toolkit that combines:

🔍 Volatility screening
🧠 Greek simulation
📊 Straddle backtesting

We use live market data (via yfinance), visualize critical metrics, and ensure all functions are contained in a single script — just the way traders like us operate.

🔍 Option Moneyness: Know Where You Stand

Before trading any option, you must understand where its strike sits relative to the current market price — this is called moneyness.

✔️ Calls

In the Money (ITM): Strike < Spot — already profitable if exercised
At the Money (ATM): Strike ≈ Spot — highest gamma, most responsive
Out of the Money (OTM): Strike > Spot — cheap, speculative

✔️ Puts

In the Money: Strike > Spot — pays off below market
At the Money: Strike ≈ Spot — greatest extrinsic value
Out of the Money: Strike < Spot — directional downside exposure

Use this mental map before entering any directional or volatility trade.

📈 1. Implied Volatility Screener

❓ What It Does:

This function retrieves ATM IV across expirations and compares it to historical volatility over a 30-day period. It helps answer:

“Are options overpriced or underpriced right now?”

✅ Use It For:

Deciding whether to buy or sell premium
Structuring spreads vs naked options
Understanding term structure and skew

📊 Example Output:

🔍 Observations:

Near-term IV (May 2025) is significantly elevated — likely due to earnings or macro event.
IV sharply declines across the curve, bottoming around 42% in late 2025/early 2026.
The red dashed line = 30-day realized volatility (HV), sitting around 56.7% → options are currently rich.

This chart helps identify when to sell vol (e.g. short strangles or spreads) or buy vol (e.g. long straddles).

🧪 2. Straddle Earnings Backtester

❓ What It Does:

Backtests a simple earnings strategy:

Enter straddle 5 days before earnings
Exit 1 day after earnings
Estimate payoff using an assumed 5% straddle cost

✅ Use It For:

Gauging whether vol crush post-earnings destroys value
Validating if the move justifies option premium
Spotting overpriced earnings setups

📊 Example Output:

Summary Stats (AMZN):

🔍 Observations:

All 8 backtested trades were net losses — the moves were too small to justify the premium.
Volatility is overpriced before earnings, and the subsequent move doesn’t compensate.
Actionable Insight: Favor selling premium (like iron condors or ratio spreads) before AMZN earnings.

🧠 3. Greeks Simulator

❓ What It Does:

Calculates and visualizes option sensitivities across strikes for a specific expiry and volatility.

✅ Use It For:

Understanding where gamma is peaking
Managing theta decay
Spotting vega exposure for volatility plays

📊 Example Output:

🔍 Observations:

Gamma and Vega peak at-the-money (Strike ≈ 193) — confirming the standard shape
Theta is most negative at ATM — time decay is fastest here
Delta slope confirms call curve — higher strikes have lower delta

This simulator lets you align trades with your expectations on price, volatility, and time decay.

🎯 When to Use Calls vs Puts (Quick Guide)

Market View	Strategy	Example
Bullish breakout	Buy Call	Long 195C if AMZN ≈ 192
Bearish catalyst	Buy Put	Long 190P for downside
Big move (unknown dir)	Buy Straddle	195C + 195P
Mild bullish move	Bull Call Spread	195C / short 200C
Mild bearish move	Bear Put Spread	195P / short 190P

🧰 Toolkit Functions (Copy-Paste Ready)

RiskOptima.implied_volatility_screener(symbol='AMZN')
RiskOptima.straddle_backtester(symbol='AMZN', window_before=5, window_after=1)
RiskOptima.greeks_simulator(S=192.82, T_days=20, sigma=0.35, option_type='call')

All wrapped in a single file. Minimal dependencies. Maximum insight.

🚀 What’s Next

Add run_all() orchestrator for batch testing
Plug into riskoptima portfolio overlays
Build a Streamlit dashboard (optional)
Add edge detection: where historical P&L exceeds breakeven

Catching Market Turns with SPY & VIX Divergence

noreply@blogger.com (Jordi Corbilla) — Sun, 16 Mar 2025 20:21:00 +0000

Introduction

Ever wondered how to spot potential turning points in the stock market? One powerful signal emerges when the S&P 500 (SPY) makes a second lower low, but the Volatility Index (VIX) climbs higher than before. This suggests that while prices are pushing lower, fear is mounting—a perfect storm for an oversold bounce.

The Setup

SPY and VIX: We monitor both the S&P 500 ETF (SPY) and the CBOE Volatility Index (VIX).
Moving Average and Bollinger Bands: We use a 30-day moving average (MA) of SPY plus Bollinger bands set at ±2 standard deviations. These bands define a “reasonable” trading range for SPY.

The Signal

Local Minima: Identify points where SPY forms a short-term trough.
Second Lower Low: If SPY forms another trough lower than the first, it’s a sign of continued weakness.
Higher VIX: Confirm that the second low in SPY aligns with a higher VIX reading than at the first low, signalling growing fear.
Within ±2σ: Ensure SPY hasn’t broken too far beyond its Bollinger bands. If it’s still within the ±2σ envelope, the market might be primed for a reversion move.

Exiting the Trade
Once a position is entered, exit conditions are straightforward:

SPY Rallies Above MA: If SPY closes above its 30-day moving average, momentum might be shifting back up—take the profit (or limit losses if it didn’t bounce much).
SPY Breaks Below Lower Band: If SPY drops below the lower Bollinger band, the market could be in freefall, so exit to limit further downside.

Visualizing the Strategy
Below, you can see two charts illustrating how this Index Vol Divergence Strategy works.

Chart 1 shows only the Entry Signals. SPY (blue line) is plotted against its Bollinger bands (dotted orange lines) and 30-day moving average (solid orange line). The VIX is displayed in green at the bottom. Green triangles indicate entry points when the second lower low plus higher VIX condition is met, while SPY remains within ±2σ.
Chart 2 includes both Entry and Exit Signals. The red triangles mark exit points when SPY closes above the 30-day MA or below the lower Bollinger band.

These visuals help traders quickly see how the strategy identifies potential “buy-the-dip” opportunities and determines logical points to exit.

Why This Matters
This method combines price action (lower lows) with volatility sentiment (rising VIX). It aims to catch the moment when fear peaks and the market could bounce back. By integrating two different signals—price structure and implied volatility—it provides a unique edge for timing entries and exits.

Getting Started
We’ve encapsulated this logic in a Python function using yfinance, pandas, and matplotlib. Simply run:

from riskoptima import RiskOptima

df_signals, df_exits, returns = RiskOptima.run_index_vol_divergence_signals()

The function automatically:

Fetches SPY and VIX data from the specified start to end dates.
Identifies entry signals and exit signals.
Calculates returns.
Saves the final chart images and prints out the resulting DataFrames in the console.

Conclusion
The SPY & VIX Divergence Strategy is a practical way to detect potential buy-the-dip moments in an environment of increasing volatility. While no method is foolproof, combining price patterns with volatility insights can give you a clear, rules-based approach to timing the market. Try it out, tweak the parameters, and see if it fits your trading style!

Happy Investing!
— Jordi Corbilla

Check out the repository here:
GitHub: JordiCorbilla/index-vol-divergence-signals

Introducing RiskOptima: Your New Go-To Library for Portfolio Optimization and Risk Management

noreply@blogger.com (Jordi Corbilla) — Wed, 12 Feb 2025 10:58:00 +0000

I’m thrilled to announce the launch of RiskOptima, a Python library designed to help investors, analysts, and data scientists optimize their portfolios and manage risk with ease. Whether you’re a seasoned quant or just getting started with investment analytics, RiskOptima provides a powerful yet flexible framework for portfolio construction, performance benchmarking, and advanced visualization.

In this post, I’ll walk you through the motivation behind creating RiskOptima, highlight some of its core features, and give a quick tour of the outputs it can generate. Let’s dive in!

Why RiskOptima?

Portfolio Optimization Made Accessible
Traditional portfolio optimization methods (like Markowitz’s Modern Portfolio Theory) can be difficult to implement correctly—especially for those who are new to Python or finance. RiskOptima aims to lower the barrier by providing straightforward, well-documented functions to create, optimize, and analyze portfolios.
Combining Machine Learning and Statistical Models
As machine learning continues to transform quantitative finance, RiskOptima incorporates a variety of models to help you compare and contrast results from both statistical and ML-driven approaches. This allows you to find the best method for your specific investment strategy.
Interactive and Informative Visualizations
A picture is worth a thousand words. RiskOptima comes packed with robust visualization tools that bring your data and optimization results to life. From probability distributions to efficient frontiers, the library delivers clear, professional-looking charts to help you interpret outcomes quickly.
Open Source and Extensible
Hosted on GitHub, RiskOptima is open to contributions. We welcome bug reports, feature requests, and pull requests. The goal is to build a collaborative ecosystem around portfolio optimization and risk management.

Key Features

Multiple Portfolio Construction Methods
Compare different portfolio strategies (e.g., equal weighting, market benchmarks, or machine-learning-optimized) to see which one best suits your risk-return profile.
Monte Carlo Simulation
Generate thousands of potential outcomes for your portfolio allocations to visualize the distribution of returns and identify optimal weightings under different market scenarios.
Efficient Frontier Analysis
Plot your portfolios against the theoretical efficient frontier to see how they compare in terms of expected return and volatility.
Performance Benchmarking
Evaluate your portfolios against standard market indices like the S&P 500 (SPY) and compare key metrics such as Sharpe Ratio, Beta, and alpha generation.
Risk Decomposition and Allocation
Get detailed breakdowns of how each asset contributes to the overall portfolio risk, helping you fine-tune your allocations.
Professional Visualizations
Automatically generate publication-ready plots and charts that can be easily shared in reports or presentations.

A Quick Tour of the Outputs

Below are a few examples of the kinds of visualizations RiskOptima can produce. These charts were generated from a sample portfolio analysis:

Probability Distributions of Final Four Returns
Here, multiple distributions show the potential returns for different portfolio strategies. Each colored curve (green, blue, red, orange) represents a portfolio’s return distribution after a given period. The library also highlights key statistics (mean, VaR, etc.) for each.
Portfolio Optimization and Benchmarking
This time-series chart compares the performance of various portfolios over time, alongside benchmarks like the S&P 500. A summary table highlights the Sharpe Ratios, Betas, and other risk-adjusted performance metrics, making it easy to identify which strategy outperforms under certain market conditions.
Efficient Frontier (Monte Carlo Simulation)
A scatter plot of simulated portfolios appears in purple, with the efficient frontier in blue. Different star or dot markers represent specific portfolios (e.g., current vs. optimized). This allows you to see where your portfolio lies relative to the theoretical boundary of maximum returns for a given level of risk.
Portfolio Area Chart
This treemap visualization provides a snapshot of your holdings and their recent returns. It’s a quick way to see which assets are driving performance (or detracting from it) and how much each holding contributes to the total allocation.

Getting Started

Ready to give RiskOptima a try? Here’s how you can set it up:

Install via GitHub


git clone https://github.com/JordiCorbilla/RiskOptima.git
cd RiskOptima
pip install -r requirements.txt

Import and Initialize

from riskoptima import riskoptima

# Initialize your data
# For example, price data or returns data for each asset
# Then pass it to riskoptima for analysis
portfolio = riskoptima.YourPortfolioFunction(data=your_data)

Run Optimization

# Example function calls (subject to your actual implementation)
optimal_portfolio = portfolio.optimize(method="machine_learning")
optimal_portfolio.run_monte_carlo_simulation(n=10000)
optimal_portfolio.plot_efficient_frontier()

Visualize Results

# Generate and save plots
portfolio.plot_probability_distributions()
portfolio.plot_performance_comparison()
portfolio.plot_area_chart()

What’s Next?

RiskOptima is just getting started! In the coming weeks, I’ll be adding:

Enhanced ML Models: Additional regression/classification techniques for return forecasting.
Scenario Analysis: Stress testing portfolios under different macroeconomic scenarios.
Live Data Integrations: Hooks for popular financial data APIs to automate your data pipelines.
Documentation and Tutorials: Step-by-step guides, notebooks, and sample projects to help you get the most out of the library.

Join the Community

I’d love your feedback on RiskOptima. If you have questions, suggestions, or want to contribute:

Star the Repo: Show your support by starring RiskOptima on GitHub.
Open Issues: Encounter a bug or have a feature request? Open an issue.
Submit Pull Requests: Got a great idea? Help the library grow by contributing code.
Spread the Word: Share this project with friends and colleagues interested in portfolio optimization.

Conclusion

RiskOptima is designed to simplify the complex world of portfolio optimization and risk management by providing intuitive tools, clear visualizations, and powerful algorithms. With more features on the way, I’m excited to see how the community uses and improves this library.

Try it out, explore the charts, and let me know what you think. Together, we can make RiskOptima the go-to library for modern portfolio optimization!

Happy Investing!
— Jordi Corbilla

Check out the repository here:
GitHub: JordiCorbilla/RiskOptima

Building an AI-Powered PDF Chatbot Using OpenAI and Retrieval-Augmented Generation (RAG)

noreply@blogger.com (Jordi Corbilla) — Sat, 01 Feb 2025 21:06:00 +0000

Introduction

Have you ever struggled to find a specific piece of information buried deep inside a long PDF document? Whether it’s a financial report, a legal document, or a research paper, manually searching for answers can be frustrating and time-consuming.

What if you could simply ask a question and get an AI-powered answer instantly—without reading the entire document?

In this article, we’ll explore how to build a Retrieval-Augmented Generation (RAG) pipeline using OpenAI’s GPT models and vector databases to create a chatbot that can answer questions from any PDF document.

Why Not Just Use GPT-4 Alone?

Large Language Models (LLMs) like GPT-4 are powerful but lack context about specific documents unless that information is included in the prompt. If you were to feed an entire PDF into a GPT-4 prompt, you’d quickly hit token limits and face high API costs.

The Solution: Retrieval-Augmented Generation (RAG)

Instead of passing entire PDFs to GPT-4, we split, store, and retrieve only the most relevant sections of a document before asking the model to generate a response.

A RAG pipeline consists of three key steps:

Extracting and Splitting Text: Convert a PDF into manageable chunks.
Vectorising & Storing the Chunks: Convert text into embeddings and store them in a vector database for quick retrieval.
Querying & Response Generation: Retrieve relevant chunks and use GPT-4 to answer the user’s query.

This approach drastically reduces token usage, making queries faster and more cost-effective.

Building the AI-Powered PDF Chatbot

Tools and Technologies Used

LangChain: For orchestrating LLM calls, vector search, and retrieval.
OpenAI GPT-4: For generating responses based on retrieved content.
FAISS / ChromaDB: To store and retrieve relevant document chunks efficiently.
PyPDFLoader: To extract text from PDFs.

Step 1: Install Dependencies

First, install the necessary Python libraries:


pip install langchain openai chromadb faiss-cpu tiktoken PyPDF2

Step 2: Load and Process the PDF

We use PyPDFLoader to extract text and RecursiveCharacterTextSplitter to split it into chunks that are easier to search.


from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

def load_and_preprocess_pdf(pdf_path):
    loader = PyPDFLoader(pdf_path)
    documents = loader.load()

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000, chunk_overlap=200
    )
    chunks = text_splitter.split_documents(documents)
    return chunks

Step 3: Store and Retrieve Document Chunks

We convert text chunks into embeddings and store them in ChromaDB for retrieval.


from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings

def create_or_load_vector_store(chunks):
    embeddings = OpenAIEmbeddings()
    persist_directory = "chroma_vector_store"

    if os.path.exists(persist_directory):
        print("Loading existing vector store...")
        vectorstore = Chroma(persist_directory=persist_directory, embedding_function=embeddings)
    else:
        print("Creating new vector store...")
        vectorstore = Chroma.from_documents(chunks, embeddings, persist_directory=persist_directory)
    return vectorstore

Step 4: Query GPT-4 with Retrieved Chunks

We retrieve only the most relevant document chunks and pass them to GPT-4 for an intelligent response.


from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

def create_rag_pipeline(vectorstore):
    retriever = vectorstore.as_retriever(search_type="similarity", search_kwargs={"k": 3})
    qa_chain = RetrievalQA.from_chain_type(
        llm=ChatOpenAI(model="gpt-4", streaming=True),
        retriever=retriever,
        return_source_documents=True
    )
    return qa_chain

Step 5: Track Token Usage and Costs

To measure the cost efficiency of RAG, we log token usage:


from langchain.callbacks import get_openai_callback

def query_rag_pipeline(rag_pipeline, query):
    with get_openai_callback() as callback:
        result = rag_pipeline({"query": query})
        
        print("\nAnswer:")
        print(result["result"])

        print("\nToken Usage:")
        print(f"- Prompt tokens: {callback.prompt_tokens}")
        print(f"- Completion tokens: {callback.completion_tokens}")
        print(f"- Total tokens: {callback.total_tokens}")
        print(f"- Estimated cost: ${callback.total_cost:.5f}")
    return result

Step 6: Run the Chatbot


def main():
    pdf_path = "example.pdf"
    print("Loading and processing PDF...")
    chunks = load_and_preprocess_pdf(pdf_path)

    print("Creating/loading vector store...")
    vectorstore = create_or_load_vector_store(chunks)

    print("Setting up RAG pipeline...")
    rag_pipeline = create_rag_pipeline(vectorstore)

    while True:
        query = input("\nAsk a question (or type 'exit' to quit): ")
        if query.lower() == "exit":
            break
        query_rag_pipeline(rag_pipeline, query)

if __name__ == "__main__":
    main()

How This Improves Performance and Reduces Cost

Instead of passing entire PDFs to GPT-4, this chatbot:
✅ Retrieves only relevant parts, reducing token usage.
✅ Persists embeddings, avoiding redundant recomputation.
✅ Uses vector search, making queries faster and scalable.

Example Cost Comparison

Approach	Tokens Used	Cost per Query
Full PDF in Prompt	~20,000	$0.10
RAG-based Retrieval	~350	$0.0021

🚀 95%+ cost savings while maintaining accuracy!

Potential Applications

This RAG-based PDF chatbot can be used for:

📜 Legal Document Analysis – Quickly retrieve case laws, contracts, and compliance details.
📚 Educational Use – Answer questions from textbooks or research papers.
🏦 Finance & Regulations – Automate MiFID II, GDPR, and financial document inquiries.
🎓 Corporate Knowledge Management – Search internal company policies instantly.

Final Thoughts

The RAG-PDF chatbot is a practical solution for efficiently querying large documents using GPT-4 while significantly reducing token costs. By integrating vector search with OpenAI’s LLMs, we can build scalable and cost-effective AI applications for real-world business use cases.

👉 Try the full implementation on GitHub: RAG-PDF-Chatbot

Let me know in the comments—what would you use this for? 🚀

Optimizing Investment Portfolios with Machine Learning and Mean-Variance Analysis

noreply@blogger.com (Jordi Corbilla) — Wed, 08 Jan 2025 21:10:00 +0000

Optimizing Investment Portfolios with Machine Learning and Mean-Variance Analysis

Investing has always been about balancing risk and reward. Whether you're managing a personal portfolio or overseeing millions in institutional investments, finding the right allocation of assets to maximize returns while minimizing risk is crucial. In this post, we’ll dive into how Machine Learning (ML) and Mean-Variance Optimization (MVO) can revolutionize portfolio management, using Dividend Kings as our investment universe.

Why Dividend Kings?

By default, the repository focuses on Dividend Kings—stocks that have increased dividends for at least 50 consecutive years. These companies are renowned for their financial stability, consistent returns, and shareholder-friendly policies. Dividend Kings offer a realistic and relatively conservative set of stocks for applying portfolio optimization techniques.

This choice ensures that the portfolio is built on reliable, historically strong performers while leaving room to explore the benefits of optimization. Of course, the code is flexible—you can specify any set of tickers if you prefer growth stocks, ETFs, or other instruments.

For more on Dividend Kings, check out the full list here: Dividend Kings List.

Introduction to Portfolio Optimization

At the heart of portfolio management lies the concept of the Efficient Frontier. This theoretical boundary identifies the optimal trade-off between portfolio risk (volatility) and return. Harry Markowitz's Mean-Variance Optimization (MVO) introduced a systematic approach to achieving this balance, and it remains one of the most influential financial models.

However, traditional methods rely heavily on historical data and fixed assumptions. Markets, on the other hand, are dynamic and complex. This is where Machine Learning enters the equation—by forecasting future returns and adjusting to changing conditions, ML models can augment MVO for better decision-making.

Blending Machine Learning with MVO

Our project combines the power of predictive modeling with the rigor of statistical optimization. By using ML models like Linear Regression, we estimate asset returns based on historical data and trends. These predicted returns are then fed into the MVO algorithm to calculate the optimal allocation of assets.

Key Features:

Black-Litterman Framework: Adjusts market returns based on investor views and confidence levels.
Efficient Frontier Analysis: Visualizes the best possible portfolios for various risk levels.
Performance Metrics: Includes Sharpe Ratio, Sortino Ratio, and Information Ratio to measure portfolio efficiency.

Portfolio Details

For this project, we used the following Dividend Kings as our investment universe:

Symbol	Company Name	Weight	Sector
MO	Altria Group Inc.	4%	Consumer Staples
NWN	Northwest Natural Gas	14%	Utilities
BKH	Black Hills Corp.	1%	Utilities
ED	Con Edison	1%	Utilities
PEP	PepsiCo Inc.	9%	Consumer Staples
NFG	National Fuel Gas	16%	Utilities
KO	Coca-Cola Company	6%	Consumer Staples
FRT	Federal Realty Inv. Trust	28%	Real Estate
GPC	Genuine Parts Co.	16%	Consumer Discretionary
MSEX	Middlesex Water Co.	5%	Utilities

The total portfolio value is $100,000, with weights translating into dollar amounts. For instance, a 4% weight in MO means $4,000 invested in Altria Group.

How It Works

Input Portfolio: Define the initial portfolio with weights and market caps.
Predict Returns: Use Machine Learning models to predict future returns for each asset.
Optimize Allocation: Use MVO to find the optimal weights for the portfolio based on predicted returns, minimizing risk while maximizing returns.
Backtest: Simulate the optimized portfolio’s performance against historical data.
Visualize Results: Generate the Efficient Frontier, plot portfolio performance, and compare with the benchmark (S&P 500).

Results

Our project provides several outputs to help investors make informed decisions:

Efficient Frontier: A curve showing the best portfolios for different risk levels.
Capital Market Line (CML): A line representing portfolios that combine the risk-free asset with the market portfolio.
Performance Metrics:
- Sharpe Ratio: How much return you earn for every unit of risk.
- Sortino Ratio: Similar to Sharpe but focuses only on downside risk.
- Information Ratio: Compares portfolio performance against a benchmark.

Below is a sample visualization generated by our project:

Why This Matters

Customizable: Investors can adjust parameters like risk tolerance, minimum/maximum asset weights, and the choice of ML models.
Adaptability: The use of ML ensures that the portfolio adapts to changing market conditions.
Transparency: By visualizing the Efficient Frontier and other metrics, investors gain clarity on how decisions impact performance.

How to Get Started

Want to try it yourself? Here’s how:

Clone the GitHub repository:


git clone https://github.com/JordiCorbilla/machine-learning-optimization-portfolio.git

Install dependencies:
```
pip install -r requirements.txt
```
Define your portfolio in the fetch_portfolio_data() method.
Run the main() function:
```
python portfolio_optimization.py
```

The outputs will include:

Optimized portfolio weights.
Performance metrics and tables.
Visualization of the Efficient Frontier and portfolio performance.

Conclusion

Combining Machine Learning with Mean-Variance Optimization bridges the gap between modern predictive analytics and traditional portfolio theory. By applying this to Dividend Kings, we focus on reliable, stable companies while exploring innovative techniques to maximize returns and minimize risks.

References

Markowitz, H. (1952). Portfolio Selection. The Journal of Finance.
Dividend Kings List

Disclaimer: This project is for educational purposes only and does not constitute financial advice. Always consult a financial advisor before making investment decisions.

Optimizing a Portfolio with Monte Carlo Simulations and the Efficient Frontier

noreply@blogger.com (Jordi Corbilla) — Fri, 27 Dec 2024 19:02:00 +0000

Introduction

Welcome to my latest project—an exploration of portfolio optimization using both Monte Carlo simulations and Modern Portfolio Theory. I’ve just open-sourced the code on GitHub. In this blog post, I’ll walk through what the project does, why it’s interesting, and how you can try it yourself.

Project is hosted here:

Portfolio Optimisation · Streamlit

What is Portfolio Optimization?

Portfolio optimization is all about finding the ideal blend of assets (stocks, funds, or other instruments) that maximizes returns while minimizing risk. One classic method to visualize this is the Efficient Frontier, a concept introduced by Nobel laureate Harry Markowitz. The Efficient Frontier is the curve of “optimal” portfolios offering the highest return per unit of risk.

Risk

We often measure risk by volatility (standard deviation) of returns.

Return

Typically measured by annualized mean of returns.

Sharpe Ratio

A measure of risk-adjusted return, calculated as the (portfolio return – risk-free rate) / (portfolio volatility).

Project Overview

Monte Carlo Simulation Approach

Random Weights
We generate thousands (often 100K+) of random weight allocations across our chosen assets.
Performance Metrics
For each allocation, we calculate the annualized return, annualized volatility, and Sharpe ratio.
Best Found Portfolio
We highlight the random portfolio that yields the highest Sharpe ratio among all simulations (i.e., the “green star” in the resulting plot).

Analytical Efficient Frontier

In contrast to random allocations, we also compute the Efficient Frontier by systematic optimization. We use optimization routines (like scipy.optimize) to find the portfolio with the minimum volatility for each target return. This yields a smooth curve which is the theoretical solution to Markowitz’s Modern Portfolio Theory.

Visualizing the Results

The script generates a scatter plot of all random portfolios, color-coded by Sharpe ratio. It then overlays the Efficient Frontier curve and highlights notable points:

Market Benchmark (S&P 500) performance (red dot)
Optimal Portfolio from Monte Carlo (green star)
Potentially other reference portfolios like the Global Minimum Volatility or an Equal-Weighted portfolio.

Why Dividend Kings?

By default, the repository focuses on the Dividend Kings—stocks that have increased dividends for at least 50 consecutive years. I wanted to apply the technique to a realistic, relatively conservative set of stocks. Of course, you can specify any set of tickers if you’d prefer growth stocks, ETFs, or other instruments.

Getting Started

Clone or Download the repository:

git clone https://github.com/JordiCorbilla/monte-carlo-portfolio-optimization.git

Install Dependencies:
pip install -r requirements.txt

Run the Script: $python monte_carlo_portfolio.py --assets AAPL MSFT TSLA --start 2020-01-01 --end 2025-01-01 or use the app here :$ Portfolio Optimisation · Streamlit

By default, it saves a PNG of the Efficient Frontier (efficient_frontier_monter_carlo.png) and a CSV of the historical data (market_data.csv).

Pro Tip: Adjust parameters like risk_free_rate, portfolios, or the date range in the command line to see how different assumptions impact the results.

Key Highlights

Modular Code
- The repository separates data fetching, Monte Carlo simulations, and plotting into distinct functions for clarity.
Argument Parsing
- You can easily change inputs via --assets, --start, --end, and other flags.
Analytical Frontier + Monte Carlo
- See how a purely random approach might miss the theoretical optimum (the frontier line) unless it samples very densely.
Saving Outputs
- The script automatically saves the final chart. Perfect for including in reports or presentations.

Results: Example Chart

Below is a sample chart from the script, showing:

A cloud of random portfolio allocations color-coded by Sharpe.
A black line representing the Efficient Frontier.
Markers for the market (in red) and the optimal Monte Carlo portfolio (green star).

In a real market scenario, you might see how dividend-paying stocks cluster toward lower volatility with moderate returns, while certain growth stocks occupy a higher-risk, higher-return domain.

Results: Example via Streamlit APP

Below is a sample chart from the app, showing:

Potential Next Steps

Incorporate Additional Factors
You could add constraints like “no short selling” or “limit to 20% in any single asset” to your optimization.
Use Different Benchmarks
Compare your portfolio to an international index (like IXUS), or to bonds (like AGG).
Evaluate Performance Over Time
Extend the script to backtest the chosen “optimal” allocation through time, rebalancing periodically.

Conclusion

With this project, I’ve combined two powerful approaches—Monte Carlo and analytical optimization—to visualize and compare different portfolio allocations. By default, it focuses on Dividend Kings, but you can easily switch to any assets you like. Whether you’re an aspiring quant investor or just curious about portfolio construction, I hope this provides a helpful practical example of Modern Portfolio Theory in action.

For more details, check out the GitHub repository. Feel free to fork it, play with the parameters, and share your results. Happy investing!

References

Markowitz, H. (1952). Portfolio Selection. The Journal of Finance.
Dividend Kings List

Thank you for reading! If you have any questions or suggestions, feel free to open an issue or comment below.

Disclaimer: This project is for educational purposes only and does not constitute financial advice. Always consult a financial advisor before making investment decisions.

AI Tutor Telegram Bot Project

noreply@blogger.com (Jordi Corbilla) — Sun, 29 Sep 2024 16:18:00 +0000

It wasn't too long ago when I started experimenting with Bots using WhatsApp an I recently discovered that's way easier using telegram instead. Most of the communication layer is done for you so you don't have to worry about using web hooks and external proxies.

This time I bring you a quick project on creating a simple AI Tutor using Telegram and OpenAI. The idea is to use telegram to relay the information to a backend siting on my home server that will serve the data and communicate with OpenAI api to consume the prompts and reply the appropriate response. The idea is to use something that might help me with my children's homework. We'll also store the conversation details on a DB and because we are using telegram, we can share this bot with several people.

AI Tutor Telegram Bot Project

Overview

This project is an AI-powered Telegram bot named Merlin, designed to assist users by providing helpful and succinct answers. The bot can handle text messages, voice messages, images, and can generate images based on user prompts. The bot now supports user authorization based on a list defined in the config.ini file. Only authorized users can interact with the bot.

Features

1 User Authorization:

Access Control: Only users listed in the AUTHORIZED_USERS configuration can use the bot.

Unauthorized Access: If an unauthorized user tries to interact with the bot, they receive a message saying "You are not authorized."

2 Text Interaction:

Users can send text messages, and Merlin will respond with helpful answers generated using OpenAI's GPT-4 model.

3 Voice Interaction:

Users can send voice messages.

The bot transcribes the voice message using OpenAI's Whisper model.
Generates a response using GPT-4.
Sends back both the text response and an audio file of the response using text-to-speech.

4 Image Interaction:

Users can send images with or without captions.
The bot extracts text from the image using OCR (Tesseract).

Includes any caption text provided by the user.
Combines the extracted text and caption, and generates a response using GPT-4.

5 Image Generation:

Users can request image generation by sending a message starting with generate image:.

The bot uses OpenAI's image generation API to create an image based on the provided prompt.
Sends the generated image back to the user.

6 Text Extraction from Images:

Users can extract text from images without involving OpenAI by sending an image with the caption starting with extract text:.
The bot processes the image using Tesseract OCR and sends back the extracted text.

7 Logging and Database Interaction:

All interactions are logged into a SQL Server database for record-keeping and analysis.
Includes user ID, username, message type, user message, and bot response.

8 Configuration File:

Sensitive information like API keys, tokens, and authorized users are stored in a config.ini file.
Makes it easier to manage configurations without altering the main script.

9 Enhanced Logging and Error Handling:

The bot includes detailed logging for easier debugging and monitoring.
Exception handling ensures the bot remains operational even when unexpected errors occur.

Setup Instructions

Prerequisites

Python 3.x
Telegram account and a bot token from BotFather
OpenAI API key with access to GPT-4 and image generation
SQL Server instance for logging interactions
Tesseract OCR installed on your system
Required Python packages:
- python-telegram-bot
- openai
- gTTS
- pytesseract
- pyodbc
- pydub
- requests
- configparser
- Installation Steps

1 Clone the Repository:

git clone https://github.com/JordiCorbilla/AI-Tutor.git
cd ai-tutor

2 Install Required Packages:

pip install -r requirements.txt

3 Set Up Configuration File:

[TELEGRAM]
BOT_TOKEN = your-telegram-bot-token
AUTHORIZED_USERS = user1,user2,user3

[OPENAI]
API_KEY = your-openai-api-key

[DATABASE]
SERVER = your-database-server
DATABASE = your-database-name
DRIVER = your-database-driver

Replace placeholders with your actual configuration values.
AUTHORIZED_USERS: Provide a comma-separated list of authorized usernames, full names, or user IDs.

4 Install Tesseract OCR:

Windows: Download and install from https://github.com/UB-Mannheim/tesseract/wiki.

5 Run the bot:

PS C:\repo\AI-Tutor\venv\Scripts> & c:/repo/AI-Tutor/venv/Scripts/python.exe c:/repo/AI-Tutor/ai_tutor_telegram_bot.py

Usage

Start the Bot:
- Open Telegram and start a chat with your bot.
- Send /start to receive a greeting message.
User Authorization:
- Only users listed in the AUTHORIZED_USERS can interact with the bot.
- Unauthorized users will receive "You are not authorized."
Text Messages:
- Send any text message, and the bot will reply with an AI-generated response.
Voice Messages:
- Send a voice message.
- The bot will transcribe it, generate a response, and send back both text and audio.
Images with Optional Captions:
- Send an image.
- Optionally include a caption.
- The bot will extract text from the image and combine it with the caption to generate a response.
Image Generation:
- Send a message starting with generate image: followed by your prompt.
- Example: generate image: a futuristic cityscape at sunset
- The bot will generate an image based on your prompt and send it back.
Text Extraction from Images:
- Send an image with the caption starting with extract text:.
- The bot will extract text from the image and send it back to you.

Logging and Monitoring

Logs:
- All logs are written to bot.log in the project directory.
Database:
- Interactions are logged into the Interactions table in your specified SQL Server database.
Unauthorized Access Attempts:
- Unauthorized access attempts are logged with user details for security auditing.

Error Handling

The bot is designed to handle exceptions gracefully.
In case of errors, appropriate messages are sent to the user without exposing technical details.
Detailed error information is logged for developers to review.

Contributing

Contributions are welcome! Feel free to open issues or submit pull requests.
JordiCorbilla/AI-Tutor: Building my own AI Tutor (github.com)
Please ensure that your code adheres to the existing style and includes necessary logging and error handling.

Field	Value
Provider	`ollama`
Base URL	`http://localhost:11434`
Model	`codellama:7b`

Random thoughts on coding and technology

Rowlens: finding one exact CSV row in a 17 GB file without blowing up memory

Building sqlcsv-exporter: a Python CLI to Stream SQL Server Data into CSV

The Problem

The Goal

The Package Structure

Why Streaming Matters

A Better CLI Experience

Handling SQL as a File

Optional SQL Rewriting

A Real-World Bug Caught by Live Testing

Stress Testing with a Slower Query

Testing Strategy

Packaging for PyPI

Example Usage

Final Thoughts

Building csv-stream-diff: A Fast, Streaming CSV Comparison Tool for Very Large Files

The Problem

What csv-stream-diff Does

The Core Design

1. Partition both files into hashed buckets

2. Compare buckets in parallel

Why YAML Configuration

Exact Sampling for Large Validation Runs

Handling Duplicate Keys

Keeping Dependencies Small

Outputs That Are Actually Useful

Testing the Tool Properly

A Few Practical Lessons

Example Usage

Why I Built It

Closing

Coding from Telegram: my OpenClaw + Ollama setup

Tina the Destroyer: Turning Family Photos Into a Playable Arcade Game

The Core Idea

From Photos to Sprites

Building the Game

Vibe Coding the Project

Why I Loved This Build

Building a Portfolio CLI in 2026 (and Why CLIs Feel Cool Again)

Why a CLI?

What I Built

The Flow

Design Notes

What's Next

Running Claude Code Fully Locally with Ollama (Yes, It’s Possible — With Caveats)

Why Claude Code Is Hard to Run Locally

Architecture That Actually Works

Step 1: Redirect Claude Code to Ollama

Step 2: Understand the Model Name Problem

Solution: Model aliasing

Step 3: Why Gemma Fails (Important)

Conclusion

Step 4: Use a Tool-Capable Model

Recommended

Step 5: Create Claude-Compatible Aliases

Result: Fully Local Claude Code

Building a Multi‑Agent Finance Chatbot with AG2

Why multi‑agent chatbots?

Pros and cons

Defining the finance chatbot agents

Setting up the environment

Implementing the agents

How it works

Sample interaction

Pros and cons of this approach

Future enhancements

Conclusion

Building a MiFID II Compliance Chatbot with CrewAI

Agent‑centric vs Task‑centric Tool Assignment

Agent‑centric (Generalist)

Task‑centric (Specialist)

Pros and Cons

Code Walkthrough

Preparing for MiFID III

Beyond Compliance – General Lessons

Conclusion

Automating MiFID Compliance with CrewAI

Why MiFID Updates Matter

Building a Multi‑Agent MiFID Compliance Workflow

What `csv-stream-diff` Does