Thunderhead Explorer

The Trainee Got Better. It Still Cannot Fly Alone.

2026-09-22T14:06:52.359-04:00

I ended The Engineer and the Trainee with a plan: roll out the shielded pilot, relabel every visited state with MPC, retrain, and see whether the trainee could finally fly on its own. That is still the plan. But before getting there I did something lazier, and the result was more interesting than I expected. I also fixed something that had been bothering me every time I watched a flight.

The same recorded flight as last time, re-run with a smoother controller and a retrained model. The engine loses 60% of its thrust at ten seconds, the estimate falls to 40%, and the shield now steps in on about one command in eight instead of one in three.

The lander had the shakes

Watching a descent, the ship twitched. Not a rendering artifact — the controller really was changing its mind. I measured it: over 30 flights the beam search picked a different command on 77.7% of decisions. Every fifth of a second it re-derived the whole plan from scratch, and when two commands were nearly tied it would happily alternate between them. The tank chattered between throttle settings and the attitude jets flipped sign.

The fix is one term in the cost function: charge a candidate for differing from the command actually being flown. One knob, normalized so it covers tilt and throttle together. Here is what it bought, measured over seeds 3000–3009 on all three pads:

	No penalty	With penalty
Decisions that change command (nominal)	77.7%	20.6%
Stage-to-stage tilt jerk	5.37° RMS	1.69° RMS
Landings, all three scenarios	30/30	30/30
Fuel remaining at ×0.4 thrust	62.6	65.6

Not a trade at all, which surprised me. Smoother, and it landed the same flights using less fuel and slightly fewer decisions. The commands it stopped issuing were mostly ones it would have undone on the next stage anyway.

The same fix broke the sibling project

Naturally I tried the identical change in lunar-mpc, the Box2D controller this one descends from. On nominal flights it looked like a win: 49 of 50 strict landing checks went to 50 of 50. I nearly kept it.

Then I measured the engine-fault scenario — the one that repository exists to demonstrate — and it fell from 49/50 to 41/50. The fixed-model controller went from 42/50 to 24/50. The reason is obvious in hindsight: that lander's main engine is binary, on or off, so rapid pulsing is precisely how it holds a thrust level between the two. Charging for the pulses removes its only means of modulating thrust. Here, with three throttle settings, holding one is a real option.

So I reverted it there and wrote the numbers into that repository's research limits, because a rule that only holds in one of two codebases is worth recording where someone will trip over it. The broader lesson I have now written into my own notes: sweep a control-cost change over every published scenario before adopting it, not the one that runs quickest.

A consequence I did not plan for

Once the controller penalizes changing command, its next choice depends on what it is currently doing. Roughly four decisions in five are now “hold what you are doing.” That makes the flown command the single most predictive feature of the teacher's answer — and the trainee could not see it.

So the prompt gained one sentence: “Current command: tilt hold, engine half.” I want to be precise about why that is not a relapse into the hint-giving of the first experiment. MPC's requested command is the teacher's answer, and the earlier post showed Laya simply copies it. The flown command is actuator state, the kind any onboard controller reads from its own vehicle. The trainee still has to decide what to do next.

Retraining, on four GPUs this time

Same recipe as before, except the data came from the smoothed teacher and the run used all four RTX PRO 4500 Blackwell cards instead of three. Eight epochs in 235 seconds. Per-question validation accuracy went from 80.0% to 90.2%, and it passed 85% after the first epoch, better than anything the previous run reached in eight. A genuinely better student.

Then I flew it on the same 90 held-out flights, and this is the part worth the post:

	Previous checkpoint	Retrained
Validation accuracy	80.0%	90.2%
Landings, unshielded	0 / 90	1 / 90
Landings, with the MPC shield	90 / 90	90 / 90
Laya's own choice kept	55–62%	80–87%
Shield overrides	30–37%	12–17%

Ten points of accuracy bought exactly one landing out of ninety. The trainee is still not a pilot. One wrong command puts the ship slightly off the teacher's trajectory, the next state is less familiar, agreement collapses to about 17%, and the flight diverges. That is behaviour cloning without a corrective signal, and making the student more accurate on the teacher's own path does not fix it — which is a sharper argument for DAgger than I had before, because it says the missing signal is off-trajectory states rather than label quality.

What the better student did buy is a shield that works less. Overrides roughly halved. Four commands in five that fly the ship are now the model's own. The fused pilot leans further toward the trainee while physics still holds the veto.

About Jev, since everyone is asking

TypeSafe AI put Jev into limited early access on 15 September, and the reaction has been loud. It is not a language model: you hand it a block of state and typed questions, and it returns a probability distribution per question plus a confidence score. No text, nothing to parse.

Look at what my lander actually asks its model. A sentence of telemetry, two choice questions — tilt {left, hold, right} and engine {off, half, full} — and a probability per option. That is Jev's contract. Laya sits behind a single call, agent.predict(prompt, QUESTIONS), and nothing below it knows what produced the answer. Swapping in a decision model would be an adapter, not a redesign.

I have not measured Jev, and its speed, price and “cannot hallucinate” claims are the vendor's, not mine. But my results already answer some of what people are excited about, because those parts belong to the architecture rather than the model:

A faster, better proposer is not a pilot. I just moved a proposer from 80% to 90% and got one extra solo landing in ninety. Cheaper proposals do not change that.
Well-typed is not feasible. Every command Laya emits is already well-formed — left/hold/right is always in range — and 12–17% of them were still overridden because simulating them forward cost more than the plan. A type system rules out unparseable answers, not physically bad ones.
The usual objection has a runtime answer. People worry that a model returning bare numbers hides its reasoning. My shield never asks the model to explain itself; it rolls the proposal forward under learned physics and compares. Verification beats explanation when you can afford to simulate.
Calibration is the part I actually want, and it is not free. My shield margin is a cost delta on a scale where a plan costs 0 to about 33, with a 1000-unit crash penalty on top. Confidence is a probability. Nothing converts one into the other, and a well-calibrated probability does not supply the conversion. The real step is calibrating the event the shield decides: the chance that a proposal's best rollout exceeds the plan by more than a given gap.

Two things I would check before assuming a fit. My prompt is almost entirely numbers, and weakness on numbers is a reported limitation of this model class, so interface fit and content fit are separate questions. And a hosted API gives up the offline, on-device property that this project currently has, with roughly 14 ms per decision on a Mac and no network at all.

The usual caveats

The beam search is incomplete and the cost weights are hand-picked. The evaluation is ten held-out seeds per pad. One honest wrinkle: the original cost weights were tuned on seeds 0–9, but the new switching weight was swept on seeds 3000–3005, which sit inside the evaluation set. Every weight in that sweep landed every flight, so the choice turned on jerk and fuel rather than landings, and the landing counts are unchanged from before the penalty existed — but it is tuned on that set, and the results page says so rather than leaving it implicit. None of this is flight software.

Next up is still DAgger, and now calibration has a concrete definition to aim at. The browser lab, the measured results, the training recipe and the full write-up are at github.com/mraad/lunar-mpc-laya. Go fly it and tell me where it breaks.

The Engineer and the Trainee: Adaptive MPC Guiding and Shielding Laya

2026-09-21T16:41:20.166-04:00

Two of my recent lander experiments kept nagging at me. In One Lander, Three Controllers, an adaptive MPC learned its engine during the flight and landed through a power loss with no training data at all. In Lunar Laya, a small decision model on Apple MLX turned a text description of the flight into structured choices, but the text carried hints from a fixed feedback law that assumed a healthy engine. Cut the engine power and that model crashed every time, faithfully following bad advice. So I asked the obvious question: what happens if the engineer writes the hints for the trainee? The result is Lunar MPC + Laya.

An actual recorded flight. Laya proposes every command from telemetry alone; adaptive MPC vetoes about one in three. The engine loses 60% of its thrust at ten seconds, the estimate drops to 40%, and the lander touches down at seventy.

The engineer and the trainee

The game is the same arcade-style lander from Lunar Laya: three pads on bumpy terrain, a tilt control, an engine with off, half and full, and one decision every fifth of a second. Land upright, slowly, inside a pad, before the fuel runs out. The twist is an unannounced fault: partway through the flight the main engine loses 30% or 60% of its power, and the pilot is never told.

MPC is the engineer. It has the physics written down. Before every move it simulates thousands of command sequences three seconds ahead, picks the cheapest, executes only the first command, and replans. It also compares what it predicted with what actually happened and updates its estimate of engine strength by recursive least squares. Nobody trained it; I wrote its cost function by hand, and I ported it from the Box2D world of the earlier post to the arcade physics here.

Laya is the trainee. It reads a sentence like “Altitude 200 m, vertical velocity −8 m/s, tilt 3 degrees” and answers two multiple-choice questions: which way to tilt, how much engine. It learned from examples. It has no idea what gravity is.

Three ways they work together

1. The engineer whispers to the trainee. Laya's prompt already ended with a requested rotation and power. I swapped the source of that request from the fixed feedback law to MPC's first planned command, and changed nothing else: same weights, same prompt template, byte for byte. With the old requests, Laya landed 17 of 30 flights through a 30% power loss and 0 of 30 through a 60% loss. With MPC's requests, 30 of 30 in both. Same brain, better advice. The interesting detail is that MPC's adaptation now reaches the language model through the words in the prompt. Laya never sees the fault; it sees requests that already account for it.

2. The engineer checks the trainee's work. The original Lunar Laya overrode the model every time it disagreed with guidance. That is safe but leaves the model no room. Here the shield is predictive: MPC forces Laya's proposal as the first command of a second beam search and compares the best achievable cost with its own plan. Only a proposal that is predicted to be worse gets replaced. Laya keeps its independence; physics keeps a veto.

And here is the honest part. With the original checkpoint the shield never fired. Not once in 28,026 decisions. That model was fine-tuned to follow its prompt's request, and it follows it perfectly, so its flights are identical to MPC's own. Worse, when I appended a single sentence to the prompt with the engine estimate, request following collapsed to zero and every flight was lost. The checkpoint is welded to its training template. Useful to know before anyone puts a language model in a control loop.

3. The trainee learns from the engineer, then flies with a safety net. So I trained a second checkpoint. Same base model, same recipe as the first post, three RTX PRO 6000 GPUs, about seven minutes. This time the teacher was adaptive MPC, the rollouts included both fault levels, and the requested labels were removed from the prompt entirely. Laya has to decide from telemetry.

Pilot	Nominal	Thrust ×0.7	Thrust ×0.4	Laya's choice kept
Distilled Laya, alone	0/30	0/30	0/30	—
Distilled Laya + MPC shield	30/30	30/30	30/30	about two thirds

Alone, it cannot land a single flight. Validation accuracy was 80% per question, which sounds fine until you remember that one wrong command moves the ship slightly off the teacher's trajectory, the next state is less familiar, and agreement with MPC falls to 10–18%. That is the textbook behaviour-cloning failure, reproduced on my desk. With the shield on, the same weights land 90 of 90, and roughly two thirds of the commands that actually fly the ship are Laya's own. That is the first configuration where the fused pilot is neither controller alone.

Why bother, if the engineer alone can land?

Because the engineer can only do what its rules say. Its preferences are numbers in a cost function. It cannot be taught from examples, cannot take an instruction in plain language, and cannot tell you how confident it is. The trainee can do all three. What I want is a pilot with the trainee's flexibility and the engineer's guarantee that physics gets the last word. That pilot now exists and lands. It just needs the engineer to step in less often.

Try it in the browser

I ported the physics and the MPC to VanillaJS so you can fly it: click anywhere in the sky to place the lander, pick a pad, add a fault, and launch. A Node check verifies that the JavaScript reproduces every command of six Python-recorded flights. With the local Python server running, the same page flies both Laya checkpoints live on MLX and replays every recorded decision with the model's probabilities, MPC's request, Laya's proposal, and whether the shield fired.

The usual caveats apply. The beam search is incomplete, the cost weights were hand-tuned on ten development seeds, the evaluation is ten held-out seeds per pad, and some starts are unrecoverable for every pilot, including MPC. The shield can only be as good as MPC's own plan.

Next is the obvious one: roll out the shielded pilot, relabel every visited state with MPC, and retrain, so the trainee learns from the neighbourhood of its own mistakes. If that raises the raw landing count above zero and cuts the override rate, the loop is working. Source, docs, measured results and the browser lab are all at github.com/mraad/lunar-mpc-laya. More to come.

Lunar Laya: From State and Text to Decisions on a Mac

2026-09-21T10:54:53.632-04:00

I keep coming back to Lunar Lander. It gives me a very visible way to ask a simple question: what does a model actually decide? Turn too much, burn too late, or miss the pad, and the result is right there on the screen.

After exploring learning and control approaches to this problem, I wanted to try another angle: give a small model the current state and a text instruction, then ask it for a decision that the application can execute and measure. That experiment is now public as Lunar Laya, an Atari-inspired lander using Laya with local inference on Apple MLX.

I believe the input should be a state and a text, even a structured text. The output should always be structured and very deterministic, so it can be controlled and measured. This implementation is a representation of that idea. The fact that the model can be small enough to run locally in an MLX environment is proof that it can be done for this kind of bounded decision task.

An actual trained-model flight using FP16 MLX inference, shown at 4× simulation speed. The input includes explicit guidance requests; no post-inference overrides were applied. The smaller Q6 version was evaluated separately.

A small decision with a clear contract

The simulation exposes position, velocity, tilt, fuel and the landing target. A feedback controller calculates the requested correction. Those values and requests are serialized into text, and Laya answers two choice questions: rotate left, hold or right; and set the engine to off, half or full. Here, Laya is the decision model, not the LayaAir game engine.

The model scores the allowed choices. The runtime selects the highest-scoring option and returns a structured answer with probabilities. For example, the choices inside an answer can look like this, with probabilities omitted for readability:

{
  "rotation": {"choice": "hold"},
  "engine": {"choice": "half"}
}

The application maps those choices to a fixed command: zero rotation and 50% thrust. I can record the proposed command, what was executed, the next state and how long the decision took. That gives me something to inspect when a landing fails, and something precise to compare after changing the model.

That is what I mean by a deterministic interface: a fixed set of outputs, an explicit selection rule and a predictable mapping into application behavior. Neural scores can still change with precision or hardware. A well-defined output contract makes those changes testable; it does not make every prediction correct.

Training it, then making it smaller

I added supervised training on three CUDA GPUs, keeping the original checkpoint available. Local inference uses MLX on the Mac; Python runs the simulator and guidance. A VanillaJS application lets us replay the recorded flights, inspect probabilities and compare controllers. It displays the experiments rather than running the model in the browser.

Now the best part: the compact 6-bit version passed all 1,928 held-out validation examples and landed all 30 evaluated flights across three pads. Its 11,147 decisions agreed with the guidance requests, and its recorded state transitions matched the trained FP16 evaluation exactly.

On an Apple M4 Max, peak active MLX inference memory dropped from about 872 MiB to 370 MiB, a 57.6% reduction. The weight file went from 614 MiB to 249 MiB. Those are model-runtime measurements: total process peak memory was about 709 MiB for Q6, including the surrounding software.

The compact runtime loads packed MLX weights, removes an unused classifier and evaluates the two control questions sequentially. It also disables retained allocator caching to favor memory use. There is a tradeoff: median decision time increased from about 10.1 ms to 13.5 ms in this comparison.

I tried smaller precisions too. Four-bit missed 215 validation answers; five-bit missed two. Six-bit was the smallest configuration tested that passed every validation answer. A smaller file is only useful if the behavior survives the change! The optimization notes include the measurements and reproduction commands.

What this experiment demonstrates

The requested rotation and engine labels are already in the input. The trained model is learning to follow those requests. The feedback controller supplies the navigation knowledge, so the successful landings demonstrate guidance following, not independent discovery of a flight policy. The repository also includes a baseline that executes guidance directly, which is sufficient for this simulator.

For me, the interesting result is the working connection between state, text and a bounded action. I want that same discipline when a model selects a GIS operation or chooses the next step in a data workflow: define what it can return, validate the result, and measure what happens. The lander makes that idea easy to watch.

The next experiment I would like to try is varying the instruction while keeping the output contract fixed, then measuring where the behavior holds and where it breaks. For now, the source code, training details and animation instructions are available. Take a look, try it on your Mac, and let me know where you would use a state-plus-text decision interface. More to come. :-)

Nano FEL: GPT-2 and nanoGPT on the North Sea

2026-09-21T10:19:15.079-04:00

After LoRA on a 4B model and a 1.2B model on the Mac, I kept asking myself a simpler question. How small can the model be, and how plain can the training loop be, before this North Sea task stops working? So I went back to the basics: GPT-2 medium, 355 million parameters, fully fine-tuned with Andrej Karpathy's nanoGPT. No adapters, no chat template, no grammar, no catalog in the prompt. Just a question in, a JSON query out. That is nanofel.

The task is the same FELN translation: a question about wells, pipelines, and discoveries becomes a plan with layers, where, and relations. The twist here is the data format. nanoGPT trains on one flat stream of tokens and samples random windows from it, so each training pair is simply:

Q: Which gas wells in Norway are within 2 kilometers of an oil pipeline?
A: {"layers":["Wells","Pipelines"],
    "where":["content_type = cast(2 as SMALLINT) and (country = 'NO')","PipelinesType = cast(4 as SMALLINT)"],
    "relations":["withinDistance 2 kilometers"]}<|endoftext|>

Pairs are written one after another, and the training script from nanoGPT runs unchanged. The whole project is three vendored files from nanoGPT, a 100-line data preparation script, a normalizer that rewrites generated SQL into the gold surface style, and a 100-line inference script that decodes greedily and stops at the end-of-text token. At inference time the model has no idea a catalog exists; everything it knows about content_type = 2 meaning gas was absorbed during training.

The gold data is 3,000 records from the regenerated NorthSea project, each with a casual phrasing and a canonical phrasing of the same query. I hold out 300 records, which gives 600 evaluation questions, and keep both phrasings of a record on the same side of the split. On top of the gold, about 12,900 generated pairs from the feln-dsl grammar sampler go into training only, after the normalizer puts them in gold style. Without that normalization step the model would learn two competing ways to write the same clause and lose exact match on gold.

Training is short. Block size 256, 32 sequences per iteration, learning rate 3e-5 with cosine decay, dropout 0.1, and 3,000 iterations while keeping the checkpoint with the best validation loss. On two RTX PRO 6000 GPUs that is about five minutes including compile and checkpoint writes. The stripped checkpoint is 1.4 GB and runs on my Mac through MPS.

Here is how the runs went, all scored by exact JSON match on the held-out questions:

Run	Training pairs	Evaluation split	Exact match
v1, gold only	1,800	old gold, 200 questions	158/200 (79.0%)
v1, gpt2-large	1,800	old gold, 200 questions	158/200 (79.0%)
v2, gold + generated	17,372	old gold, 200 questions	194/200 (97.0%)
v3, new gold only	5,400	new gold, 600 questions	529/600 (88.2%)
v4, new gold + generated (shipped)	18,297	new gold, 600 questions	534/600 (89.0%)
v5, same as v4, twice the steps on three GPUs	18,297	new gold, 600 questions	532/600 (88.7%)

Two things jumped out at me. First, going from gpt2-medium to gpt2-large on the same 1,800 pairs changed nothing, while adding generated coverage for the sparse patterns took the same small model from 79% to 97%. This task is data-limited, not capacity-limited, and a 355M model has plenty of room for it. Second, the drop from 97% to 89% is not the model getting worse. The gold set was regenerated with three times more records, and the new catalog hints dropped the layer noun from the subtype descriptions. So “List gas/condensate. The returned wells must be within 15 kilometers of oil.” is now gold for wells near oil pipelines, with nothing in the sentence saying pipelines. 40 of the 66 misses in v4 are that exact situation: the secondary layer is never named, and the model picks discoveries instead of pipelines or the other way around. When every layer is named in the question, v4 scores 95%.

The v5 run was my attempt to squeeze more out of the same data, since the v4 validation loss looked like it was still falling at the last step. Three GPUs, 1.5 times the batch, twice the iterations. The validation loss bottomed out at step 1,800, drifted up while the training loss kept falling, and the best checkpoint landed two questions below v4. Same data, same result. The “still falling” signal was smaller than the evaluation noise, and I should have looked at the eval jitter before spending the compute. The remaining misses are the catalog's ambiguity, and the fix is in the data generation, not in the training loop.

A few caveats. The 89% and the 99.7% from the LoRA post are on different gold sets with different SQL conventions, so there is no head-to-head here. Exact match is strict, and a style-normalized comparison gives the same numbers, so none of the misses are formatting. But the model has no grammar and no output validation. It will happily copy an unfamiliar word into a column name, so a downstream allowlist against the catalog is still needed before anything runs.

What I like about this experiment is how little there is to it. A few hundred lines you can read in one sitting, a five-minute training run, and a model that answers on a laptop. The code, training notes, and the full miss breakdown are on GitHub. Next is regenerating the gold with the layer nouns restored in the hints, and masking the question tokens so the loss is spent on the answer only. More to come!

FELN Qwen: Train on RTX, Run on Jetson Orin

2026-09-17T14:13:00.632-04:00

After teaching a small model to query the North Sea, I wanted to take the experiment onto a Jetson. Could I train on a powerful workstation, then move the model, application, and spatial database onto an NVIDIA Jetson Orin? And could it produce the expected FELN JSON accurately enough to be useful?

That is the experiment behind feln-qwen. The job is specific: translate a question about wells, pipelines, and discoveries into FELN. The JSON identifies the layers, their attribute filters, and the spatial relationships between them. Layer order matters. Finding pipelines near discoveries is a different request from finding discoveries near pipelines!

A huge thank you to Geodata for the beautiful NorthSea data. It gave this experiment a rich geospatial vocabulary and meaningful relationships to work with. I deeply appreciate the work and care behind this dataset.

I chose Qwen3.5-9B and first verified stock-model inference on the Orin. The target machine had to support the model before committing to a full training run. Training then used two RTX PRO 6000 Blackwell GPUs, while inference and spatial execution stayed entirely on the Orin. The Mac was the browser and SSH connection; the workstation was not an inference dependency.

The data builder reads FELN.json, Layers.json, and the Open Knowledge Format files. It validates the catalog, removes duplicates, and retains provenance. The final split contains 4,790 training, 673 validation, and 534 test examples. Related query families stay together, including literal substitutions and reordered secondary layers, so a close variant cannot slip across the split boundary.

I used BF16 LoRA with rank-32 adapters, keeping the base weights frozen and applying loss to the expected answer tokens. There was enough measured GPU memory to use this recipe without quantizing the training base. A smoke run preceded the two-epoch run, which took about 58 minutes of measured wall time. Validation selected the checkpoint. The training and deployment walkthrough records the settings and commands.

For deployment, the adapter was merged into the base model and exported to a text-only GGUF Q4_K_M file. The resulting file is about 5.63 GB, served by llama.cpp with a 4,096-token context and one request at a time. Quantization is a separate deployment decision, so I evaluated the actual exported file as well as the training checkpoint.

Now the part I was most interested in: the untuned Q4 model on Orin matched 96 of 534 expected FELNs, or 17.98%. The tuned Q4 model matched 530 of 534, or 99.25%, on the same held-out set and application prompt/grammar path. These results come from the exported GGUF tested on the Orin.

Here is the matched RTX–Orin comparison using the same tuned Q4 file and all 534 questions. These are measured model-endpoint timings, with requests sent sequentially. RTX inference used one GPU; training used both. Runtime build revisions differ and are recorded with the results.

Measurement	RTX	Orin
Strict FELN matches	530/534 (99.25%)	530/534 (99.25%)
Warm median latency	0.402 seconds	3.221 seconds
Warm p95 latency	0.644 seconds	5.168 seconds
Requests per second	2.426	0.303

RTX was about eight times faster in this workload, but the Orin retained the same accuracy. The strict comparison uses FELN.same after schema validation: equivalent SQL formatting is acceptable; a wrong field, enum value, or spatial relationship is not. The comparison and underlying reports are in the repository.

The application adds schema compilation and read-only DuckDB spatial execution. A separate 18-question sample covering the nine primary-layer/layer-count strata passed 18 of 18 end-to-end checks, with a warm median of 3.97 seconds. Sampled peak system-wide shared RAM usage was about 10.1 GiB. That is an application/database measurement and a system-memory reading, distinct from the model-only table above.

I also deployed FELN Studio on the Orin. Here it is generating the expected JSON for pipelines with a dimension over 20.0 that are within two miles of a condensate discovery. The screenshot was captured from the real application, with an exact-match verdict.

FELN Studio on Orin, viewed through the Mac browser. The legacy “on this Mac” helper text is inaccurate for this deployment: model computation runs on Orin.

There are limits. These are held-out synthetic and source query families from one static catalog, not a production user study. Four test answers are still wrong, and complete Orin responses exceed one second. Studio provides generation and inspection; spatial GeoJSON comes from the separate application API. The services were restart-tested and subsequently stopped at my request, so the screenshot documents the verified deployment rather than a live public demo.

The code is now public under Apache 2.0 in feln-qwen, with data preparation, training, evaluation, serving, and measured results. Third-party models and data retain their own terms. I would like to test broader catalogs and more natural questions next. Fork the repository, explore the workflow, and let me know where you would take this in your GIS work. More to come!

FELN Liquid: Fine-Tuning a GIS Model on a Mac

2026-09-16T16:23:29.775-04:00

I have been experimenting with small language models that translate everyday questions into structured GIS queries. With FELN Liquid, I wanted to explore the training side on the Mac itself. Could I take a compact model, teach it the conventions of a North Sea catalog, and complete the experiment locally?

The first complete run used LiquidAI's LFM2.5-1.2B-Instruct with LoRA through MLX on an Apple M4 Max. Training took 48 minutes and 37 seconds. Including checkpoint evaluation and the final test, the workflow finished in just under an hour. That is a practical turnaround for trying an idea and inspecting what it learned.

The task is familiar from the other FELN projects: turn a request into a JSON plan containing layers, where, and relations. A compact Layers.json catalog accompanies each request, supplying the names, types, aliases, codes, and hints that make the data meaningful.

For example, “Show all wells with water depth > 350 meters” should produce:

{
  "layers": ["Wells"],
  "where": ["\"water_depth\" > 350"],
  "relations": []
}

The measurement belongs to the water_depth field, and its unit is already understood from the catalog. The SQL predicate needs the numeric threshold. There is no spatial join in this example, so the relations array is empty.

The completed experiment used 1,000 synthetic examples, split into 796 for training, 100 for validation, and 104 for testing. Related query families stayed together across those splits. I also froze the source files, prompt, split identifiers, and hashes for the run. As the catalog and generated questions evolve, those copies let us identify exactly what produced a result.

LoRA updated about 11.1 million parameters, roughly 0.95% of the base model, using rank-16 adapters. MLX runs this work on the Mac's Metal GPU. Peak reported MLX memory was about 30.7 GB, so the memory requirement matters even with a relatively small model. The selected adapter is about 44.5 MB and still needs the base weights to run.

The recorded results selected epoch two:

Measurement	Selected adapter
Canonical plan matches, validation	98/100
Canonical plan matches, test	99/104 (95.2%)
Strict FELN schema validity, test	104/104
Identifier and SQL compilation checks, test	104/104
Observed median generation time, test	0.784 seconds

Epoch three matched 97 of the 100 validation plans, so the final checkpoint was not the selected one. The test split was evaluated after selection. Canonical comparison normalizes the SQL representation before comparing plans; the stricter raw-output score was 96/104 on the test set. The timing is an observation from this Mac run, not a controlled, repeated inference benchmark.

Then a short human question exposed a useful problem: “Show all wells with depth > 350 meters.” The original adapter invented a depth column and left meters in the SQL expression. The available field describes water depth. A person's word “depth” could also mean drilled-well depth, which this catalog does not provide.

The guarded inference command now asks for clarification on that wording. For an explicit request, it supplies additional instructions and one example, validates the complete response against the catalog and permitted SQL expressions, and allows one retry with feedback. Its result can be clarification_required, validated, or rejected. It does not execute the query.

That distinction matters. All five errors in the original tuned test set passed the schema and SQL checks. A wrong stored code or an extra condition can still be valid SQL. Likewise, the six targeted checks that passed after adding the guard are development checks, not a new accuracy benchmark. The original scores describe synthetic query families, and execution-result accuracy has not been measured.

I am excited by how much of the training and inspection cycle can happen locally. The next step is to add more human phrasing while reserving independently written questions for evaluation. A matched RTX training comparison is still future work for this project.

The scripts, run notes, and data preparation details are available in feln-liquid. The setup currently uses workspace-specific paths, and the model weights and run artifacts are not included in a clone, so start with the README before reproducing it. Feedback on the questions your own GIS users actually ask would be especially useful. More to come!

FELN WebGPU: A Language Model, a Database, and a Map in One Browser Tab

2026-09-16T16:00:28.000-04:00

I have been exploring how small language models can turn a plain-English question into a useful GIS query. With FELN LoRA, that meant training a model on a GPU and running it locally on my Mac. For feln-webgpu, I wanted to bring the language model, the database, and the map into the web application itself.

Type “Give me all oil discoveries,” and the browser generates a query plan, compiles it to SQL, executes it, and draws the results. The example in the repository finds 562 discoveries, with 200 displayed. We can inspect the plan and SQL beside the map. I like being able to see what the model actually asked the database!

The question, query plan, SQL, and resulting features in one browser tab.

The pieces are a fine-tuned LiquidAI LFM2-350M model, Transformers.js running on WebGPU, DuckDB-Wasm with its spatial extension, and the ArcGIS Maps SDK for JavaScript. Once the assets have loaded, query generation and database execution happen on the user's machine. There is no inference server or database server behind the interaction.

There is an initial download: 443 MB for the exported model weights and 5.9 MB of Parquet data for wells, discoveries, and pipelines. The application also fetches libraries, the spatial extension, and basemap tiles from external services. Local query processing is useful, but this version still has network dependencies.

For the oil-discoveries example, the model produces:

{
  "layers": ["Discoveries"],
  "where": ["discovery_type = cast(3 as INTEGER)"],
  "relations": []
}

A JavaScript port of the FELN compiler turns that plan into DuckDB SQL. DuckDB reads the geometry from the Parquet tables, and the application sends the returned features to an ArcGIS graphics layer. The model's responsibility is translating the request into FELN. The database performs the selection.

Getting a 350-million-parameter model to do that required attention to the examples. An early version saw “water depth” during training, but a question using just “depth” could make it invent a column. Adding paraphrases addressed that gap. The current preparation also includes more natural wording and combined constraints, while keeping a query plan and all its wordings on the same side of the training/validation split.

This project uses full fine-tuning. The latest 350M run took about 31 minutes on one RTX PRO 6000. Its system prompt includes the OKF catalog documentation, roughly 10,900 tokens describing the data. That is a substantial amount of context for a short question.

The useful trick is to process that common prompt once. At startup, the application spends about seven seconds building a prefix cache, then reuses the model state for subsequent questions. In the recorded comparison, this brought generation from about seven seconds per question to 0.7–1.3 seconds. A separate 210-question browser holdout run averaged 0.46 seconds for generation. Those measurements exclude downloading the model and drawing the result.

The export experiments also helped choose the download size. On the earlier 81-question browser evaluation, FP16 and the 8-bit export both matched 74 expected plans. The 8-bit version reduced the weights from 692 MB to 443 MB. The smaller 4-bit exports lost accuracy, so the application stayed with 8-bit weights.

For the current v4 model, the recorded browser results are:

Measurement	Result
Exact FELN plans	203/210 (96.7%)
Matching result sets	208/210 (99.0%)
Mean generation time	0.46 seconds

Matching rows and matching plans answer different questions. Two different filters can happen to select the same records in this dataset. I want to keep both measurements visible. A larger 1.2B model reached 204/210 exact plans in the Python evaluation, one more than the 350M model, and was not exported into the application.

There is a spatial issue to fix before relying on distance queries: the current compiler passes metre-valued thresholds to ST_DWithin while the geometry is stored in WGS84 degrees. Those units do not match. The browser reproduces the Python implementation, so agreement between them does not establish correct distances. The correction belongs in the shared compiler and then in the browser port.

What interests me here is how much of this workflow fits in a browser tab: language, structured queries, a spatial database, and a map. The dataset is small, the model still makes mistakes, and the initial load matters. Even with those limits, this is an exciting direction for focused GIS applications.

The code and evaluation tables are available on GitHub in feln-webgpu. I would like to keep pushing on natural wording and spatial correctness as this develops. More to come!

FELN LoRA: Teaching a Small Model to Query the North Sea

2026-09-16T16:00:18.000-04:00

In the FELN RAG experiment, I used a few relevant examples to help a language model translate a question into a spatial query. That got me interested in taking the next step: teaching a small model to do this particular job, then running it locally.

I believe small language models have a big role to play in GIS. We have plenty of tasks with a well-defined vocabulary, a known catalog, and a very specific output. Turning a North Sea question into a query over wells, pipelines, and discoveries is one of them. That is the experiment behind feln-lora.

The model is NVIDIA's Nemotron-3-Nano-4B, fine-tuned using LoRA and QLoRA through NeMo AutoModel. Training happens on an NVIDIA GPU. The resulting model runs on my Mac through llama.cpp and Metal, with the question staying on the machine.

Now the best part: the latest local models produce a query in about 1.2 seconds at the warm median. That is a useful place to start for an interactive GIS workflow :-)

Consider this question from the challenge set: “Find gas wells no more than 5 km from oil pipelines.” The expected FELN is:

{
  "layers": ["Wells", "Pipelines"],
  "where": ["\"content_type\" = 2", "\"pipelinestype\" = 4"],
  "relations": ["withinDistance 5 kilometers"]
}

There is quite a bit of GIS knowledge hiding in that little object. Gas wells use content_type = 2, while oil pipelines use pipelinestype = 4. The first layer is what we want returned. The second supplies the spatial filter. Reverse the layers and we are asking for pipelines instead of wells.

layers-json supplies the catalog, and feln supplies the query model and strict comparison. The inference bundle carries the prompt, catalog, and JSON-schema grammar alongside the exported model. The grammar constrains the output structure; getting the right fields, codes, and spatial relationship remains the model's job. Query execution is a separate step, with a read-only DuckDB path in the repository.

LoRA trains small adapters while keeping the base model frozen. QLoRA adds a quantized base during training, using 4-bit NF4 here. Both recipes use rank-16 adapters. After training, I merge the adapter into the full-precision base and export a Q8_0 GGUF for local inference. The 4-bit training representation and the 8-bit deployment file are two separate choices.

For the September 16 retraining, the data preparation produced 3,463 training examples, 410 validation examples, and 419 test examples. A target query and its paraphrases stay in the same split. Otherwise, a reworded training question could make the validation score look more impressive than it deserves. Validation selects the checkpoint; the test and challenge sets are reserved for regression checks.

Here are the v2 results, with each training run using one RTX PRO 6000:

Measurement	LoRA	QLoRA
Merged model, validation	409/410 (99.76%)	409/410 (99.76%)
Q8_0 GGUF, Mac validation	408/410 (99.51%)	408/410 (99.51%)
Q8_0 GGUF, Mac challenge set	37/40	36/40
Warm median query generation, Mac	1.21 seconds	1.16 seconds
Training wall time	28 minutes 22 seconds	31 minutes 6 seconds
Peak GPU memory during training	15.6 GiB	18.4 GiB

The score uses feln.FELN.same, which normalizes SQL formatting while preserving distinctions such as operators, spatial direction, and an empty string versus NULL. A query returning some rows is not enough to make it correct. And these timings measure query generation, with database execution still to follow.

QLoRA gave me the same validation accuracy, but used more memory and took longer in this setup. That is worth measuring on the hardware and model we actually intend to use. The harder, hand-written challenge set also gives a more modest picture than the validation percentage. Prefix matching, category distinctions, and a query combining two spatial relationships still exposed mistakes. I have not measured accuracy against a real-user query log yet.

There is also the question of maintenance. When the catalog aliases, hints, and question phrasing changed, the original models scored 832/862 on the new geometry-layer questions before retraining. The preparation pipeline excludes standalone tables without geometry, including 138 questions from the regenerated gold set. Supporting those tables is still an open extension.

RAG makes it convenient to update examples and catalog context. Fine-tuning adds a training and validation cycle, but gives me a local model specialized for this workflow. The recorded RAG and fine-tuning scores use different evaluation sets, so they do not establish a head-to-head winner.

What excites me is having this translation step close to the data, on the machine in front of me. The code and training notes are available on GitHub in feln-lora. One experiment I would like to try next is adding retrieved examples to the fine-tuned model's prompt. There is room to combine the two approaches. More to come!

FELN RAG: Five Examples and a Spatial Query

2026-09-15T19:19:57.641-04:00

Ask a GIS person to find oil wells within five kilometers of gas pipelines and they can start breaking the request into pieces. Which layer contains the wells? How is “oil” encoded? Which pipelines carry gas? What spatial operation connects the two?

A language model needs those same details. “Oil” might be a numeric subtype. A country might be stored as a two-letter code. And choosing intersects when the question calls for contains can produce perfectly respectable JSON that asks the wrong question.

That is the problem behind feln-rag: give the model a description of the data and a handful of relevant worked examples, then ask it to translate a new request into a structured spatial query.

The interesting part is how little machinery the retrieval needs.

Start with the query we want to produce

The output is FELN, a JSON representation with three main pieces:

layers: the primary layer to return, followed by any spatial filter layers.
where: one SQL filter per layer, in the same order.
relations: the spatial relationship between the primary layer and each secondary layer.

Here is a small example from the bundled NorthSea corpus. A request for discoveries in the United Kingdom whose discovery wellbore name starts with 42/30- becomes:

{
  "layers": ["Discoveries"],
  "where": ["\"country\" = 'UK' AND \"discovery_wellbore_name\" LIKE '42/30-%'"],
  "relations": []
}

One layer, two attribute conditions, and no spatial join. Add another layer and a distance constraint, and the relationship becomes something like withinDistance 5 kilometers.

The repository generates this representation. Executing it against geographic data is a separate step. You can explore the translation without having the NorthSea geodatabase on your machine.

Two companion projects supply the foundation. layers-json describes the catalog, including field names, coded values, and hints. feln supplies the output models, validation, and comparison tools.

The vector store is a NumPy array

The bundled corpus contains 1,000 question-and-answer pairs. Each question has a known FELN answer attached to it.

An embedding model converts the question text into a vector. Those vectors become rows in a NumPy array, with each row still associated with its original question and FELN. The new question goes through the same encoder, and retrieval finds the five closest examples.

Because the vectors are normalized to unit length, cosine similarity reduces to this line in the index implementation:

scores = self.embeddings @ query.reshape(-1)

At this corpus size, scanning the array is a straightforward place to start. There is no separate vector database service to operate. The embeddings are cached on disk, with the encoder name and ordered question texts contributing to the cache fingerprint.

A similarity score tells us which questions resemble the new request. Whether their answers demonstrate the right spatial logic is something we still have to inspect and measure.

Put the examples where the model can use them

The retrieved examples become a short conversation. Each question is a user message, and its known FELN answer is the following assistant message. The actual question comes last:

System:    Translation instructions and layer catalog
User:      Retrieved question 1
Assistant: Known FELN answer 1
           ...four more question/answer pairs...
User:      The new question

That is the few-shot prompt. The catalog explains the data; the examples demonstrate how to translate requests involving it. The generation code sends those messages through LiteLLM and attempts to parse the response as FELN.

The model has to produce a new answer. Copying the closest example would be a very efficient way to return somebody else's query :-)

Make the context visible

FELN Studio puts the pieces on one page: the question, five retrieved examples, an editable system prompt and catalog, and the generated JSON.

The useful first click is Find examples only. Expand a card and inspect the FELN that will become an assistant message. With the default local encoder, this step does not call the generation model. Generate FELN then uses those displayed examples in their displayed order.

The repository screenshot demonstrates the workspace and retrieval; it does not show a completed generation request.

The interface is plain HTML, CSS, and JavaScript served by Python's standard-library HTTP server. It runs on loopback. Generation sends the question, catalog, and examples to the configured model provider, while credentials stay on the server. The README explains the workflow and configuration.

The VALID FELN label has a precise meaning: the output passed schema validation. You still need to read the query. Valid structure cannot tell you whether the model chose the relationship you intended.

Count the misses, too

The evaluation code separates held-out questions from the retrieval corpus and supports a zero-shot baseline using the same catalog instructions without examples. It records validity, exact match, structural and partial scores, and request failures. Studio uses the full corpus for exploration, so evaluation belongs in that separate holdout workflow. See eval.py.

The repository reports a September 15 smoke test using MPNet retrieval and azure/gpt-5.5. Retrieval was evaluated on ten held-out questions; generation used the first three:

Check	Reported result
Matching layer set among the five retrieved examples	10/10
RAG exact match	3/3
Zero-shot exact match	2/3
Generation requests	Six valid outputs, no request errors

The zero-shot miss selected intersects where the reference used contains. All six outputs were valid, which is a useful illustration of why validation and correctness need separate columns. These are reported smoke results, and three questions are far too few to establish general accuracy.

Try it

The current setup requires Python 3.13, uv, and sibling checkouts of feln, layers-json, and VectorlessGAIT. That last dependency is currently required by the package configuration even when using NumPy retrieval. Follow the setup instructions, configure LLM_MODEL_NAME and the provider credentials, then run from the repository:

uv sync
uv run --no-sync python -m feln_rag.web

Open http://127.0.0.1:8765/. The default embedding model may download its weights on first use.

Start with a question whose answer you understand. Inspect the retrieved examples. Generate the FELN. Then change the distance, reverse the requested relationship, or add an attribute condition and see what moves.

For me, the appeal is being able to follow the translation all the way through: the catalog, the examples, the prompt, and the resulting query. When something goes wrong, there are concrete pieces to examine.

The next useful experiment would be a larger held-out comparison, especially on questions that differ by one spatial relationship or one coded value. Those small differences are where a convincing answer has to become a correct query.

More to come :-)

FELN: Find Existing Location with N Layers

2026-09-15T12:27:54.402-04:00

In the layers-json post, I argued that the model needs to know what our data means before it can write a useful query. A field called STATUS with a value of 3 is not self-explanatory, and the aliases, domains, and subtypes already sitting in an ArcGIS Pro project are the cheapest explanation we will ever get. That post ended with a catalog, a Layers.json file, and a promise to do something with it.

This is the something. FELN stands for Find Existing Location with N layers, and it is a small Python library and CLI that turns a plain-language request into a structured spatial query, compiles that query to DuckDB SQL, and, the part I actually care about, measures how close one query is to another.

The shape of a question

Most of the spatial questions people ask me over the years have the same skeleton. Find the things in one layer, optionally filtered, that stand in some spatial relationship to the things in another layer, also optionally filtered. Wells near pipelines. Parcels inside a flood zone. Counties that contain at least one suspended well. The layers change, the predicates change, the relationship changes, but the skeleton does not.

So a FELN is exactly that skeleton, written down as three parallel lists:

“Find suspended wells within 5 kilometers of oil pipelines.”

{
  "layers":    ["Wells", "Pipelines"],
  "where":     ["STATUS = 2", "MEDIUM = 'OIL'"],
  "relations": ["withinDistance 5 kilometers"]
}

layers[0] is what comes back. where[i] filters layers[i], and an empty string means no filter. relations[i] relates the primary layer to layers[i+1]. One layer, no relations. Three layers, two relations, both anchored on the primary. The pipelines constrain the answer but are not returned, and the 2 in STATUS = 2 comes straight from the catalog's coded-value domain, which is the whole reason the catalog exists.

One thing to say early, because it shapes everything that follows: the WHERE clauses are simple on purpose. STATUS = 2, DEPTH > 1000, NAME LIKE 'Alpha%', a BETWEEN, a few of those joined with AND. That is the vocabulary, and it was never meant to be exhaustive. This is a starting point for the find-existing-location process, not a SQL grammar. The generator emits simple predicates so a small model has something tractable to learn, and the comparison scores them per predicate for the same reason. Nothing stops you from elaborating on it. The where entries are strings, the compiler passes them through a guard rather than a parser of its own, and a richer generator or a hand-written query can put more in them. But start simple. The interesting question is whether the loop closes at all, and simple predicates are enough to find out.

There is nothing clever in that JSON, and that is deliberate. I wanted something a model can emit, a person can read, and a program can compare. The cleverness, such as it is, lives on either side of it.

Why structure instead of SQL

My first instinct, back when I started this thread, was to have the model write the SQL directly. It works often enough to be seductive and fails in ways that are very hard to score. Two SQL strings that select the same rows can look nothing alike. Two that look almost identical can differ by one < versus <= and return different wells. If I cannot measure the output, I cannot improve the thing producing it.

The FELN structure makes the answer measurable in more than one way, and the library keeps those ways separate on purpose:

Exact match. FELN.same() normalizes the WHERE clauses with sqlglot in DuckDB dialect, so cast(2 as SMALLINT) equals 2, identifiers are case-insensitive, and conjuncts are sorted before comparison. Secondary layers match by name, not position.
Graded similarity. FELNCompare.partial gives credit per predicate. A dropped conjunct is worth about 0.67, an OR written as AND about 0.75, a swapped primary layer keeps half credit. Distances are compared in meters, so 5 miles and 8.05 kilometers can agree.
Execution. When you have result sets, FELNCompare.cost blends a 70% Jaccard on the returned OBJECTIDs with 30% of the partial score, and precision and recall fall out of the ID sets in the usual way.

That last one is where the structure pays for itself twice. The same FELN can be compiled for DuckDB, or ArcPy, or Spark, each preserving its own units, coordinate systems, and boundary semantics. The generation side improves against a measurable target; the execution side is chosen for the data. Today only the DuckDB compiler exists. ArcPy and Spark are possible backends, not integrations I have written, and I would rather say that plainly than let a README imply otherwise.

What the compiler emits

FELNToDuckDB builds a chain of CTEs: one filtered L0..Ln per layer, one join CTE per spatial relation, and a final select that keeps a primary row only if it appears in every join. withinDistance becomes ST_DWithin, which includes the boundary. notWithinDistance becomes NOT EXISTS, because “more than 500 meters from every pipeline” checks the absence of any nearby pipeline, and finding one distant pipeline is not sufficient. I got that wrong once with a plain join and a >, and the result looked perfectly reasonable until I counted the rows.

The text side has the same boundary discipline. “No more than” and “at most” generate <=. “Under” and “less than” generate strict <. Distance wording only uses “within” and “no more than”, because the relation behind it is inclusive. An earlier version mapped “no more than” to <; it is fixed, the fix does not rewrite old datasets, and seeded output changed. If you generated examples before, regenerate them.

Generating the training pairs

The part that consumes the catalog is the generator. Point it at a Layers.json and it samples layers, fields, coded values, subtypes, and spatial relations into synthetic text/FELN pairs, one JSON object per line:

uv run feln generate "$HOME/Documents/ArcGIS/Projects/NorthSea/Layers.json" \
  -n 20 --seed 0 --normalize --sql --alias-suffix -o northsea-examples.jsonl

When a layer carries subtype labels, they supply the noun in the text, so a subtype Oil wells yields “Show oil wells” with kind = 1 in the WHERE, while meta.layers keeps the catalog layer name. --alias-suffix appends the layer alias to disambiguate labels shared across layers. --layer-only 0.25 phrases a quarter of the subtyped layers by alias alone, so the subtype column has to compete as an ordinary condition. These are synthetic pairs from a template, not a natural-language parser, and I want to be clear about that: the generator produces what a model should learn to emit, it does not read what a person typed.

On the NorthSea project, five catalog layers including two tables, I ran 1,000 samples in each of five generation modes, then ran them again to check reproducibility. Every record validated against the model, referenced only layers and columns in the catalog, parsed as a single DuckDB statement, and compared to its own normalized form with a structural score of 1 and a cost of 0. Profiling put nearly all of the generation time in SQL normalization, so a bounded lru_cache from the standard library took a cold run from 0.84 seconds to 0.29 for a thousand records. No new dependency.

What it is not

FELN generates SQL; it does not execute it and it is not a sandbox. The WHERE guard rejects some unsafe syntax, but it does not stop subqueries or database functions that read files. Catalog metadata, WHERE expressions, geometry and CRS options are trusted inputs. If you are going to run model-generated queries, enforce the allowed tables and columns in your application and lock the database process down; DuckDB has good guidance on that. Structural similarity and a successful parse are not security checks.

And no accuracy claim yet. The smoke tests cover generation, compilation, and comparison. DuckDB was not installed in the environment I tested from, so result IDs, CRS correctness, and runtime types were not validated against real features. Precision and recall need reference queries and execution results, and that benchmark is the next thing on the list, not something this post can report.

What interests me here is the loop. Catalog explains the data. Generator produces measurable pairs from that explanation. Compiler runs them. Compare scores the result. Close the loop with a small model and the numbers tell you whether supplying the meaning of a field actually helps, rather than whether it feels like it should. That is the experiment I set out to make possible, and it is now possible.

The source, the CLI reference, the worked examples, and the dated review notes are in the feln repository. It depends on layers-json for the catalog model, pydantic, sqlglot, and numpy, and it does not need GDAL. Try it on a project whose data you know, read the generated pairs, and tell me where the text and the SQL disagree. Those are the bugs I want.

More to come :-)

Layers JSON: Giving LLMs the GIS Context They Need

2026-09-15T05:59:11.232-04:00

When I started writing about GenAI and geospatial analysis, the attraction was being able to ask questions in plain language and let the tools do the spatial work. There is a very practical detail behind that interaction, though: the model needs to know what our data means.

Take a field called STATUS with a value of 3. Is that an active well? An abandoned one? A record waiting for review? A model can write perfectly valid SQL around that number and still give us the wrong answer. The database will happily execute it, too :-)

That is the problem I want to address with layers-json: give applications access to the knowledge already captured in an ArcGIS Pro project and its supporting data sources.

The map already carries a lot of the explanation

If you have spent time giving fields useful aliases, defining domains, and organizing layers in Pro, you have already done some of the hard work. A field alias turns a storage name into something a person recognizes. A coded-value domain explains what those numbers mean. Subtypes tell us which kind of feature a row represents. Metadata descriptions and sample values fill in more of the picture.

I like the idea of putting that work to use beyond the map. Someone took the trouble to explain the data once; we should be able to carry that explanation into the next application.

The layers-json command reads the project's layer configuration together with its referenced File Geodatabase or PostgreSQL data and writes a Layers.json catalog. It brings together aliases, field types, domains, subtypes, sample values, and query hints. A consuming application can supply the relevant parts to an LLM when translating a question into SQL.

For our hypothetical well-status domain, that means the application can supply the actual mapping between “active” and its stored code. Now the model has something concrete to work with.

From an APRX to JSON, or something we can read

Once the package and its GDAL dependencies are installed, a catalog command looks like this. Replace the example path and map name with your own:

layers-json /data/NorthSea/NorthSea.aprx --map Map -o ./catalog

This writes catalog/Layers.json. The command runs outside ArcGIS Pro, without ArcPy. It needs Python 3.11 or newer, system GDAL, and matching GDAL Python bindings. The README has the installation steps. If you prefer working in Pro, the bundled Layers.pyt toolbox provides a Prepare Metadata tool for the active map.

There is also a format I can open and read directly:

layers-okf /data/NorthSea/NorthSea.aprx --map Map -o ./knowledge

This uses the same reader to produce an OKF knowledge bundle: an index.md and Markdown concept files for the tables. The bundle includes schema tables, decoded domains, query hints, and source references, with structured metadata in the index. It gives us a readable way to inspect the context before handing it to an application.

There are some useful boundaries here. Sampling defaults to at most 20,000 rows per layer and 20 sampled distinct values per field. Columns without sampled or domain values are pruned. I would inspect the result before treating it as the context for a query; it is a catalog built to help querying, and sampling can miss values.

And the actual rows?

The catalog describes the data. To work with the rows in a SQL database, the repository also includes export tools. From an ArcGIS Pro Python environment with the DuckDB extra installed, we can export a project's layers:

layers-duckdb "C:\Projects\NorthSea\NorthSea.aprx" --map Map

That exporter writes a sibling DuckDB database, preserves hidden attributes and GlobalIDs, honors definition queries, and converts geometry to EPSG:4326. Separate GDAL-based Bash loaders handle File Geodatabases going into DuckDB, PostGIS, or Oracle Spatial. The script reference explains their requirements and replacement behavior.

What interests me most is what this could do for smaller language models. Supplying the meaning of a field leaves less for a model to infer. That is the motivation; this project does not yet establish an accuracy improvement through model benchmarks. The application still has to select the context, generate the query, and check the answer.

Try it with a small project whose data you know well. Open the catalog, look at the aliases and coded values, and see whether it contains the explanation you would give a colleague asking about that dataset. That is a useful place to start.

The source is on GitHub. I am interested in which pieces of GIS context make the biggest difference for your queries. Feedback and questions are welcome. More to come!

Nano Face: A Mac Camera, a Jetson, and One USB Cable

2026-09-11T09:16:03.000-04:00

Let me get the obvious out of the way first. Face detection on a Jetson is not new. NVIDIA ships samples for it, half the tutorials on the internet do it, and I am fairly sure somebody has done it on a Raspberry Pi wearing a HAT. So no, this is not a post about discovering face detection.

This is a post about my two Jetsons, an Orin Nano Super and an AGX Orin, that live headless on a shelf with no monitor, no keyboard, and no camera plugged into either of them. I had just finished moving the Nano to JetPack 7.2.1 and pruning both boards down to a lean server install (no desktop, no Docker, nothing starting at boot), and I wanted a small, honest, end-to-end test that they were alive and doing real work. A face detector is a good smoke test. It either draws a box on your nose or it does not.

The twist, and the part I actually cared about, is where the camera lives. It is the one in my MacBook Pro. The Jetson never sees a camera device. Frames leave the Mac's browser, ride over the USB-C cable that was already plugged in, and land on the Jetson as plain HTTP. The Jetson runs the model and sends back coordinates. That is the whole trick, and it is the reason I bothered.

USB as a network, not as a webcam

Plug a Jetson developer kit into a Mac over USB-C and it does not show up as a camera or a disk. It shows up as a network interface. NVIDIA's device-mode service brings up a small bridge on the board with a fixed private address, and macOS gets a matching “Linux for Tegra” interface on the same tiny subnet. So I can SSH to the board with no Wi-Fi, no router, and no idea which DHCP address it picked up today.

Once SSH works, the rest is old plumbing. A launcher on the Mac copies the app to the board, starts a tiny Python HTTP server there that listens only on the Jetson's own loopback, and opens an ssh -L tunnel so that http://127.0.0.1:8765 on the Mac is that server. The browser opens the page, and here is the part I like: because the page comes from localhost, the browser treats it as a secure context and happily hands over getUserMedia. The camera stays a Mac camera. The HTML just happens to be served by a Jetson.

Mac camera
  -> browser (getUserMedia, canvas, JPEG at quality 0.8, fit within 640x480)
    -> POST /api/detect to http://127.0.0.1:8765
      -> ssh -L tunnel over the USB-C cable
        -> Jetson loopback server (OpenCV decode, longest side resized to 320)
          -> YuNet through cv2.FaceDetectorYN on two CPU threads
        <- JSON: boxes, five landmarks, confidence, timings
  <- browser draws the result on the exact frame it sent

Nothing is queued. The browser sends one frame, waits for the answer, draws the boxes on that same frame, and only then grabs the next one, capped at 20 fps. If the board gets slow, the picture gets slower; it does not fall behind. A second tab hitting a busy detector gets an HTTP 429 and a polite message. I have seen too many “live” previews with boxes floating three frames behind the face to build another one.

The model, and why the CPU

The detector is YuNet, the 2023 ONNX model from the OpenCV Zoo. It is a 232 KB file that returns a rectangle, five landmarks (eyes, nose, mouth corners), and a score, and it runs through OpenCV's DNN module on the CPU. Yes, the CPU, on a board with a GPU and TensorRT sitting right there. That was deliberate. The Nano ships with NVIDIA's OpenCV 4.8.0 already installed, YuNet is tiny, and I wanted to see what the plain path gives me before touching CUDA. The server pins the model by SHA-256 and refuses to start on a mismatch, which sounds paranoid until the day a Git LFS pointer file shows up where the model should be.

On the Orin Nano Super, with the browser capturing at 640x480 and inference at 320x240, I watched 19.7 completed frames per second, 13 to 19 ms of inference on the board, and 27 to 43 ms round trips measured from the browser. That live run was on JetPack 6.2; after the 7.2.1 upgrade the fixture tests landed in the same place, with a median of 16.1 ms inference. On the AGX Orin over the same USB tunnel, inference came in at 12.9 to 16 ms with 24 to 28 ms round trips, though that was a fixture run, not the live camera. These are readings off my desk, not a benchmark, and I would be lying if I said the AGX was breaking a sweat. For a 320-pixel face detector on the CPU, the GPU is simply not the bottleneck. JPEG encoding and the trip over the cable account for most of the rest.

One face, five dots, and the numbers I actually care about in the sidebar. The kitchen is not part of the model.

Two Jetsons, two surprises

The Nano was the easy one. The AGX Orin on JetPack 7 (L4T R39.2.1) had two surprises waiting. First, it ships with no Python OpenCV at all, and the apt package is 4.6, too old for FaceDetectorYN. A pip opencv-python-headless 4.11 wheel in ~/.local, pinned with numpy<2 so it does not fight the system NumPy, took care of that.

The second one was less obvious. macOS binds the AGX's USB gadget through NCM, which shows up on the board as usb1. NVIDIA's device-mode start script hardcodes usb0 in its ECM/NCM block, so usb1 never joins the bridge and the Mac's “Linux for Tegra” port sits there saying inactive. One line in that script, reading the interface name from the gadget's configfs instead of assuming it, and the port came alive. I keep the original next to it, because a JetPack update will happily overwrite the fix.

And one rule I learned by doing it wrong: never plug both Jetsons into the Mac at the same time. They both answer at the same fixed USB address. SSH host key aliases keep their trust records apart, but they cannot make two boards share one address.

What it is not

It is a detector, not a recognizer. There are no identities, no embeddings, no database, and nothing is written to disk on either machine; a frame lives in memory for the length of one request. The server binds only to loopback on the Jetson, the tunnel binds only to loopback on the Mac, and the server checks Host and Origin, caps bodies at 1 MB and decoded frames at 1280x720, and closes the connection after each request. It installs no service and touches nothing at boot. Close the launcher and the Jetson process exits with it, because the server watches its SSH stdin for EOF. That last one is my favorite line in the whole project.

What excites me here is not the box around my face. It is that a headless board on a shelf became a useful sidekick for my laptop over the one cable that was already plugged in, with no camera purchase, no Wi-Fi setup, and no cloud in the loop. The same shape, Mac captures, Jetson computes, browser shows, should work for anything that fits in a JPEG and comes back as JSON.

The code, the API table, the test suite, and the dated verification notes are in the nano-face repository. Next on my list is to move inference to TensorRT just to see whether it matters at this size (I suspect not), and then to try something heavier than a face detector now that the plumbing is boring and the cable is proven.

More to come :-)

ArcGIS Pro, Claude Code, and a Loopback Bridge

2026-09-08T19:04:46.027-04:00

Continuing the GenAI-with-a-GeoSpatial-twist thread I started a while back. Back then, I had a language model reason about geospatial logic. This time, I wanted it to actually do the work—on my open project, inside my running ArcGIS Pro session, while I watch.

The result is ProCowork, an experimental native ArcGIS Pro add-in that embeds the Claude Code engine in a dockable chat panel. I can ask it to list layers, add and calculate fields, select features, run a buffer, or write a more specialized ArcPy script. Claude generates the code, the add-in runs it against the live project, and the code and results come back into the same panel.

For example, I can type:

“List the layers in the current map.”
“Add a DOUBLE field POP_DEN to Parcels and set it to POP / AREASQMI.”
“Select parcels where POP_DEN > 5000 and zoom to them.”
“Buffer Roads by 100 meters and add the result to the map.”

The panel is useful, but the interesting part is not the chat UI. It is getting an external AI coding agent safely and reliably close enough to a live ArcGIS Pro project to be useful.

The constraint that shaped the design

arcpy.mp.ArcGISProject("CURRENT") only resolves inside ArcGIS Pro's own Python execution context on the foreground geoprocessing thread. Claude Code, however, runs as a headless child process. If it starts Python outside Pro, that process can work with datasets on disk, but it cannot see the open project as CURRENT.

My first design kept a long-running Python daemon inside Pro. It worked, and then it went stale in the ways long-running bridges tend to go stale. So I flipped the ownership around. A persistent C# bridge now lives inside ArcGIS Pro for the whole session. For each request it either uses the ArcGIS Pro .NET SDK directly or launches one fresh foreground ArcPy geoprocessing tool. The tool resolves CURRENT, performs the operation, returns JSON, and goes away. There is no Python daemon left behind to outlive its host.

ProCowork panel (WPF, inside ArcGIS Pro)
    -> Claude Code (headless child process)
        -> MCP over HTTP on 127.0.0.1:<ephemeral-port>
            -> C# BridgeService (inside ArcGIS Pro)
                -> .NET SDK on the CIM thread
                -> or one fresh RunScript.pyt call
                    -> live project, map, and data

MCP is the small door into the live map

The bridge exposes 14 tools through the Model Context Protocol (MCP). There are focused tools such as list_layers, get_field_list, feature_count, select_by_attribute, add_field, and run_geoprocessing. The centerpiece is run_python_current(code), which gives Claude a general ArcPy path when the curated tools are not enough.

Inspection calls and lightweight map interactions use the .NET SDK on Pro's CIM thread, without ArcPy. Data updates and arbitrary Python go through the fresh geoprocessing call. Those calls are serialized behind a semaphore because geoprocessing does not appreciate re-entrancy. Request and response data travel as geoprocessing string parameters, so there is no file-polling spool between C# and Python.

Everything stays local except the normal Claude model traffic. The MCP server binds only to loopback, chooses an ephemeral port at startup, and requires a new bearer token for the session. The add-in writes those details into a generated .mcp.json and starts Claude Code in streaming JSON mode. It can use the user's existing Claude Code login, an OAuth token, or an Anthropic API key; stored secrets are protected with Windows DPAPI.

There is also a very practical UI detail. Claude answers in Markdown, while ArcGIS Pro hosts a native WPF interface. ProCowork converts the Markdown with Markdig and renders headings, code, lists, links, and real tables using themed WPF controls. Tool calls can be shown or hidden, but I like keeping them visible: if an agent is changing my map, I want to see the code it ran.

A word of caution

By default, ProCowork runs in what I call YOLO mode. Generated code executes on the open project without an approval prompt. That immediacy is the point, and it is also exactly as dangerous as it sounds. ArcPy can edit or delete real data. The bundled instructions tell Claude to inspect fields first, use edit sessions, create backups before destructive operations, and report affected row counts. Those are useful working rules; they are not a security boundary.

So this is an MVP and an experiment, not an Esri product or a supported Esri solution. It currently requires Windows, ArcGIS Pro 3.7, the ArcGIS Pro SDK for .NET, and Claude Code. Keep backups and do not point it at the only copy of important data. The architecture can support an approval step, but the proper approval card is still future work.

What excites me here is not only that an LLM can write ArcPy. We already knew that. It is that a small, local bridge can give an agent useful context from a live desktop GIS, let it act through the correct execution path, and return the evidence to the same place where the request started.

The source code, build instructions, architecture notes, and current limitations are available in the ProCowork repository. The next step is the one I would want before using it on serious work: a clear review-and-approve experience for generated operations.

More to come :-)

One Lander, Three Controllers: Reinforcement Learning, MPC, and Quadratic Programming

2026-09-08T17:39:06.677-04:00

I was always interested in the Lunar Lander game and suddenly I heard about model predictive control and quadratic programming. I want to see if the Lunar Lander RL can be adapted to that one in here.

That question led me to build three related experiments around Gymnasium's LunarLander-v3:

Lunar RL: a causal-transformer PPO policy trained through reinforcement learning.
Lunar MPC: adaptive model predictive control using discrete beam search.
Lunar QP: model predictive control whose plan is solved as a convex quadratic program.

All three control the same lander, but they answer very different questions. RL asks, “Can experience produce a policy that maps observations directly to actions?” MPC asks, “Given an approximate physics model, which short action sequence looks best from the current state?” QP asks, “Can that short-horizon plan be expressed as a fast convex optimization problem with explicit constraints?”

The progression is not a conversion of one trained checkpoint into another controller. It is an adaptation of the problem: the observation interface, action space, randomized starts, disturbance scenarios, evaluation rules, and replay tools carry forward, while the decision-making mechanism changes completely.

One lander, three ways to decide

LunarLander exposes an eight-dimensional state: horizontal and vertical position, horizontal and vertical velocity, angle, angular velocity, and two leg-contact flags. At every simulator step, a controller chooses one of four legal actions: coast, fire the left orientation engine, fire the main engine, or fire the right orientation engine.

Question	Lunar RL	Lunar MPC	Lunar QP
Decision	One policy inference	Beam search over discrete action sequences	Convex optimization of fractional engine duties
Knowledge	Learned from simulated experience	Explicit approximate dynamics	Local affine dynamics and linear constraints
Adaptation	Requires learning or retraining	Replans and estimates engine response in flight	Re-solves and estimates engine response in flight
Action handling	Outputs one legal discrete action	Searches legal discrete actions	Allocates fractional requests into discrete pulses
Primary cost	Expensive training, cheap inference	Online combinatorial search	Online numerical optimization

1. Lunar RL: learn the behavior before flight

Lunar RL uses Proximal Policy Optimization (PPO). During training, many simulated landers collect trajectories. PPO adjusts an actor that selects actions and a critic that estimates future return. After training, control is simple: encode the latest context, evaluate the policy once, and execute the selected discrete action.

The recorded Lunar RL policy landing from an off-pad, tilted start.

The network can process either the normal eight-value state or rendered pixels. Each timestep becomes a token containing the current state, previous action, previous reward, and optionally an Impala-CNN image embedding. Three GTrXL blocks process a causal window of tokens. The actor emits four logits; the critic represents value with 41 two-hot bins in symlog space.

There is an honest architectural lesson in the repository: the standard eight-value observation is already Markov. A small multilayer perceptron can solve it, so a transformer is unnecessary in vector mode. Memory becomes useful in pixel mode because a single image shows pose but does not directly reveal velocity or angular rate. A sequence of frames lets attention recover part of that hidden state.

The advantage of RL is that the policy can absorb behavior that is difficult to write as equations. The cost is paid during training, and the result is tied to the distribution it experienced. A center-trained checkpoint landed only six of eight randomized off-pad approaches. Training with the same position and tilt randomization produced a robust checkpoint that landed all eight in that experiment.

The repository reports a historical mean return of +282.6 for its centered checkpoint and documents stronger robust-policy evaluations. It also states an important current limitation: the shipped metrics predate a correction that made the PPO update context exactly match the context used while acting. The first retrained candidate after that correction failed the strict promotion checks, so the historical checkpoint remains in place while a corrected baseline is validated. That kind of disclosure matters more than a single high score.

2. Lunar MPC: predict, act once, and predict again

Lunar MPC moves the computation from training time to control time. It starts with an approximate six-state physics model for position, velocity, angle, and angular velocity. At each observation, it searches possible future action sequences, chooses the lowest-cost plan, applies only its first action, observes what actually happened, and repeats.

observe current state
update the engine model
predict candidate action sequences
keep the best beam
execute the first action
repeat from the next observation

This repeated loop is the defining idea of model predictive control. A prediction does not have to remain accurate for the entire landing. It needs to be useful long enough to choose the next action, because the controller replans after 0.02 simulated seconds. The default horizon contains 16 blocks of four physics steps, or 1.28 seconds of foresight, with a beam width of 32.

Adaptive MPC replans after an unannounced 30% main-engine power loss.

Exhaustively expanding four actions for 16 blocks would require examining 4**16 sequences. Beam search avoids that explosion. At each depth, it expands the current candidates with all four actions and keeps only the 32 plans with the lowest accumulated cost. The cost penalizes horizontal error, velocity error, vertical descent error, tilt, angular motion, unsafe approach geometry, and excessive main-engine use.

The model also learns during the flight. Bounded recursive least squares compares the predicted velocity change with the observed change and updates estimates of main-engine acceleration, side-engine acceleration, translational bias, angular response, and angular bias. Contact impulses and extreme accelerations are excluded because they are poor evidence of airborne engine performance.

This creates two forms of adaptation. Replanning reacts to state error even when the model remains fixed. Parameter estimation tries to correct the model itself. The distinction is visible in the fault experiment: immediately before action 100, the evaluator reduces main-engine power by 30%. The controller is not told the fault time or the new multiplier; it sees only the resulting motion.

On historical evaluation seeds 200–249, adaptive MPC passed 49 of 50 strict landing checks both normally and under the fault. The fixed-model controller also passed 49 of 50 normally but fell to 42 of 50 under the fault. This is evidence that online identification helped in that specific experiment, while still leaving a known horizontal-recovery failure and no safety certificate.

3. Lunar QP: make the plan a convex optimization

Lunar QP is still MPC. The change is the planner inside the repeated loop. Instead of beam-searching discrete sequences, it linearizes the approximate dynamics around the current state and asks OSQP to choose fractional engine duties over the horizon.

minimize    1/2 z' P z + q' z
subject to  lower <= A z <= upper

The predicted dynamics have the local affine form x[k+1] = A_dx[k] + B_du[k] + c. The implementation condenses the state sequence into X = base + response*U, leaving the engine requests and slack variables as optimization decisions. With 16 prediction blocks, the QP has 96 variables: 48 engine duties and 48 nonnegative slacks.

Its quadratic objective penalizes deviation from desired position, velocity, descent rate, tilt, and angular velocity. Linear constraints bound every duty between zero and one and require the three engine fractions to sum to at most one. Soft approach limits encourage clearance, bounded tilt, and remaining inside the viewport. Explicit slack variables keep a difficult state mathematically feasible while exposing how much the preferred envelope was relaxed.

QP-based MPC uses fractional plans and executes discrete engine pulses.

The most interesting mismatch is at the actuator. The Gymnasium environment accepts one discrete action, but the convex QP may request something like 60% main engine, 10% left engine, and 30% coast. A pulse-density allocator accumulates these fractions over time, executes whichever action has the largest balance, and subtracts one from that balance. Over repeated decisions, the discrete pulse counts approximate the requested fractions.

This is a convex relaxation followed by discrete allocation, not mixed-integer optimization. The predicted fractional path and the path produced by real pulses can diverge. Replanning corrects part of that error, but a solved QP is only a numerical statement about the approximate optimization problem. It is not proof of a safe landing in the nonlinear Box2D simulator.

Lunar QP uses the same style of bounded online engine estimation as Lunar MPC. On its separate evaluation seeds 300–319, both adaptive and fixed QP passed 20 of 20 normal flights. Under the 30% engine fault, adaptive QP passed 15 of 20 and fixed QP passed 16 of 20. The result does not demonstrate an adaptation advantage for QP. It also shows why successful development seeds are not enough: all four controller/scenario combinations had passed the eight development starts.

Can the RL solution be adapted to MPC and QP?

Yes, if “adapted” means preserving the experimental system and replacing the decision rule. The reusable parts are substantial:

The same eight controller-visible observations and four legal actions.
The same randomized off-pad and tilted starting-pose wrapper.
The same separation between controller information and evaluator-only simulator truth.
The same disturbance idea: reduce engine power without telling the controller.
Strict landing checks based on termination, settling, leg contact, and foot position.
Recorded trajectories and self-contained browser replays for inspecting failures.

What does not transfer is the trained policy itself. PPO weights encode a nonlinear mapping learned from data; an MPC controller needs explicit transition equations and a cost; a QP additionally needs dynamics and constraints that remain affine or quadratic after local approximation. Trying to reinterpret neural-network weights as those matrices would solve a different and much harder identification problem.

A future hybrid could combine them more directly. RL could learn a residual correction to the MPC physics model, tune terminal costs, propose a warm-start plan, or provide a recovery policy when the optimizer fails. MPC could supervise safer exploration or generate demonstrations for an RL policy. Those designs require new matched experiments; none of the three current repositories claims to implement them.

Reading the results without fooling ourselves

The reported numbers are evidence about each repository, not a leaderboard. Lunar RL, Lunar MPC, and Lunar QP use different training histories, seed ranges, costs, horizons, discretization choices, and evaluation sample sizes. The MPC and QP controllers even optimize different cost functions. Comparing their pass counts directly would not isolate whether policy learning, beam search, or QP is better.

A fair head-to-head study would freeze the controller-visible state, starting poses, terrain seeds, engine faults, landing definition, time limit, compute budget, and reporting format. It would measure pass rate, worst-case behavior, return distribution, decision latency, deadline misses, and sensitivity to model error. For QP, it should also report slack and the error introduced by fractional-to-discrete allocation. For RL, it should report variation across training seeds and hardware backends.

Run the three experiments

Each repository is standalone and uses uv.

Reinforcement learning

git clone https://github.com/mraad/lunar-rl.git
cd lunar-rl
uv sync --locked
uv run lunar-rl-view --ckpt lunar_agent_robust.pt --seed 0 --greedy

Adaptive beam-search MPC

git clone https://github.com/mraad/lunar-mpc.git
cd lunar-mpc
uv sync --locked
uv run lunar-mpc --episodes 8 --seed 0 --thrust-scale 0.7 \
  --out dist/fault-adaptive.json --replay dist/fault-adaptive.html

QP-based MPC

git clone https://github.com/mraad/lunar-qp.git
cd lunar-qp
uv sync --locked
uv run lunar-qp --episodes 8 --seed 0 --thrust-scale 0.7 \
  --out dist/qp/fault-adaptive.json --replay dist/qp/fault-adaptive.html

The generated MPC and QP replay pages expose more than the final score. They show the actual trajectory, predicted path, chosen plan, engine estimates, and decision timing. The QP replay adds fractional requests, executed pulses, solver status, residuals, and slack. The RL viewer shows policy probabilities, critic values, and attention over the recent context. Looking inside a failed landing is often more useful than adding another decimal place to an average.

What the three controllers taught me

RL can discover a capable policy without a hand-written flight model, but its behavior depends on training coverage and careful validation. MPC makes assumptions visible and can react immediately when observed motion differs from prediction, but it pays for planning at every step. QP makes objectives and constraints especially inspectable and solves the relaxed plan efficiently, while introducing local-linearization and actuator- allocation errors.

The most useful outcome of adapting LunarLander across these methods is not choosing one universal winner. It is seeing the trade clearly: learned behavior, discrete online search, and constrained convex optimization each fail in different ways. Keeping the environment, disturbances, evidence, and replay tools consistent turns those failures into something we can study rather than hide behind a score.

References

Teaching a Lunar Lander to Stop Hovering: DQN, Prioritized Replay, and n-Step Returns

2026-09-08T17:30:39.672-04:00

LunarLander looks simple: observe eight numbers, choose one of four engine actions, and guide a small craft between two flags. In practice, it exposes several failure modes that make Deep Q-Learning interesting. My first promising agents learned to approach the landing pad, then hovered above it until the episode timed out. Other runs climbed past a score of 100 and later collapsed.

I built LunarLanderDRL to work through those problems in a compact PyTorch implementation. The final agent combines a Deep Q-Network (DQN), proportional prioritized experience replay, Double DQN, Huber loss, Polyak target updates, and three-step returns. Across three training seeds, the resulting checkpoints averaged 265.7 on 90 held-out episodes and recorded zero crashes.

Two consecutive rollouts of the shipped policy, scoring 308.1 and 272.1.

The problem

Gymnasium's LunarLander-v3 returns an eight-dimensional observation: horizontal and vertical position, horizontal and vertical velocity, angle, angular velocity, and two leg-contact flags. The discrete action space contains four choices: do nothing, fire the left orientation engine, fire the main engine, or fire the right orientation engine.

A DQN approximates the action-value function Q(s, a). Given a state, the network emits one value for each action, and a greedy policy selects the largest one. The model in this project is deliberately small:

8 inputs -> 256 -> 128 -> 64 -> 4 Q-values

The hidden layers use ReLU activations and Xavier initialization. During training, an epsilon-greedy policy sometimes takes a random action, with epsilon decaying from exploration toward exploitation. The network architecture is the easy part. Stable learning depends on what transitions we replay, how we calculate targets, and how we decide that one checkpoint is better than another.

Prioritized experience replay

Uniform replay treats every stored transition as equally informative. Prioritized experience replay (PER) focuses training on transitions where the current prediction is most wrong. This implementation assigns each transition the priority

p(i) = (|TD error(i)| + epsilon) ** alpha

alpha controls how strongly priority affects sampling. A value of zero reduces PER to uniform replay; the default is 0.6. A new transition has no TD error yet, so it enters with the highest priority seen so far. This ensures that it is replayed once before being assigned a data-driven priority.

The replay buffer stores states, actions, rewards, terminal flags, next states, and bootstrap discounts in flat NumPy arrays. Priorities live in a binary sum tree. Each parent stores the sum of its children, which lets the agent update a priority or locate a sampled leaf in O(log N) time. A batch is sampled from equal-mass segments of the total priority, reducing variance compared with drawing every item independently.

Prioritized sampling changes the training distribution, so each sampled loss receives an importance-sampling correction:

w(i) = (N * P(i)) ** -beta

The weights are normalized by the maximum weight in the batch. Beta is annealed from 0.4 to 1.0, making the correction stronger as the policy approaches convergence.

Stopping Q-value overestimation from taking over

A vanilla DQN uses a maximum over estimated values when it constructs the bootstrap target. Estimation noise makes that maximum optimistic. Because the target is itself used to train future estimates, the bias can compound until a policy that appeared to be improving suddenly collapses.

Double DQN separates action selection from action evaluation. The online network picks the next action, while the target network evaluates it:

next_action = online(next_state).argmax()
next_value  = target(next_state)[next_action]
target_q    = reward + discount * next_value

The target network is not copied in occasional hard jumps. It is updated with Polyak averaging on every training step:

target = (1 - tau) * target + tau * online

The default tau is 1e-3. This gives the target a slowly moving reference point while the online network continues to learn.

The project also uses Huber loss instead of squared error. This choice matters when it is paired with PER: the replay sampler intentionally selects transitions with large TD errors, and squaring an outlier can let one sample dominate the gradient. Huber loss is quadratic close to zero and linear for large errors. Gradient norms are clipped at 10 as a final guard against unstable updates.

Why the one-step agent learned to hover

The most revealing failure was not a crash. A one-step agent often descended near the pad, stabilized, and hovered until Gymnasium's 1,000-step limit. That behavior makes sense from the agent's temporary view of the world. LunarLander provides shaping reward for remaining close to the pad at low velocity, while the large touchdown reward is still hundreds of actions away. With one-step bootstrapping, that delayed reward moves backward through the value function slowly.

Three-step returns propagate the landing signal more quickly. Instead of storing only one immediate reward, the buffer stores a folded return:

R(t) = r(t) + gamma*r(t+1) + gamma**2*r(t+2)
target_q = R(t) + gamma**k * Q(next_state, next_action)

Two implementation details are essential. First, the bootstrap multiplier must be gamma**k, where k is the number of rewards actually folded into that transition. The buffer therefore stores a discount beside each transition. Reusing a single gamma would silently produce incorrect targets.

Second, the pending reward window must be flushed at every episode boundary. Otherwise, the last n_step - 1 transitions disappear. Those are often the exact transitions containing touchdown, so dropping them removes the examples the agent most needs. If a real terminal occurs inside the window, the fold stops there. A time-limit truncation ends the rollout but does not zero a state value that could continue beyond the artificial cutoff.

Evaluate the policy you will actually deploy

Early versions saved checkpoints using the moving average of training rewards. That metric is contaminated by epsilon-greedy exploration. A lucky random action can improve a score, and a useful exploratory action can also lower it. The checkpoint should be selected using the deterministic policy that will run after training.

The training loop now evaluates a fully greedy policy every 25 episodes on a fixed set of seeds. Evaluation uses a separate Gymnasium environment, so resetting it never changes the training environment's random-number stream. This produced a striking example: one checkpoint had a training average of 239.8 but scored only 160.6 under greedy evaluation. In another run, the training average was just 78.2 while the greedy policy scored 269.1. Selecting on training reward would have preserved the first and discarded the second.

Measured results

I trained three seeds with the following configuration:

uv run main.py --n_episodes 4000 --eps_ratio 0.35 --eps_final 0.01 \
  --learning_rate 5e-4 --warm_start 5000 --n_step 3 \
  --eval_episodes 20 --solved_score 275

The runs stopped between episodes 474 and 999 and took roughly ten minutes each on an M-series CPU. I evaluated every checkpoint on the same 30 seeds, numbered 1000 through 1029. Those held-out seeds are disjoint from the seeds used to select checkpoints.

Training seed	Mean	Median	Solved	Crashed
42	280.6	281.2	30/30	0
43	243.3	278.4	24/30	0
44 (shipped model)	273.1	286.5	28/30	0
Pooled	265.7	282.1	82/90	0/90

The comparison that changed the design was the one-step run. With otherwise matching settings, it averaged 220.3, solved 25 of 30 held-out episodes, and crashed twice. The three-step runs produced no crashes in 90 episodes. Eight episodes finished below 200, but these were soft or slow landings scoring between 21 and 46, rather than destructive impacts.

I ship the seed-44 checkpoint even though seed 42 scored higher. Seed 44 lies closer to the pooled result, so it is a more representative artifact than the luckiest run.

Run it yourself

The project uses uv, pins dependencies in uv.lock, and includes a trained model. Clone the repository and install the environment:

git clone https://github.com/mraad/LunarLanderDRL.git
cd LunarLanderDRL
uv sync

Replay the shipped policy into an MP4:

uv run render.py

# Scores only, without creating video
uv run render.py --no-save

Train a new policy and monitor it with TensorBoard:

uv run main.py --n_episodes 4000 --learning_rate 5e-4 \
  --warm_start 5000 --n_step 3
uv run tensorboard --logdir logs

The automated checks cover schedules, both replay buffers, the sum tree, n-step folding and flushing, and a short end-to-end training run against the real environment:

uv run pytest test_rl.py

What I learned

The useful lesson was not that one algorithmic trick solves LunarLander. The pieces address different sources of failure. PER spends more updates on surprising transitions. Importance weights correct its sampling bias. Double DQN reduces optimistic targets. Huber loss and gradient clipping keep high-error examples from destabilizing an update. Three-step returns move delayed touchdown information backward faster. Greedy, isolated evaluation selects the policy that will actually be used.

The project remains intentionally small enough to read end to end. The next practical improvements would be training every few environment steps instead of every step, scheduling epsilon by step rather than episode, annealing PER beta across the full run, and adding a dueling value/advantage head. The current implementation and trained weights are available under the Apache 2.0 license at github.com/mraad/LunarLanderDRL.

References

On Using AutoML for NYC Taxi Trip Duration Prediction

2024-02-03T01:10:00.000-05:00

While explaining Optuna to a client in the context of hyperparameter tuning, and performing more research on the topic, I came across AutoGluon to perform "AutoML for images, text, and tabular data". After a quick scan of the documentation, I decided to give it a try and see how it performs on a simple project.

I always loved the Kaggle competition NYC Taxi Trip Duration, as the data has spatial, temporal, and other traditional attribute information and is a great dataset to test various models and feature engineering techniques.

I used a local Apache Spark instance (as it is my go-to ETL engine) to perform some feature engineering before letting AutoGluon do its magic. As a quick proof of concept, the results are quite impressive, and here are the steps to reproduce the project and visualize the result.

Arabic SDK For Apache Spark

2024-01-28T11:16:00.000-05:00

I recently attended the Esri Saudi Arabia User Conference and was amazed by the changes in the Kingdom. The capital city of Riyadh is booming and proliferating. During the conference, I presented on integrating GenerativeAI and GIS in the plenary session and led a session on BigData and GeoAnalytics Engine. GeoAnalytics Engine, based on Apache Spark, allows spatial operations on Spark data frames. We showcased a project called "A Day in the Life," which used historical traffic data from HERE to demonstrate traffic congestion during peak hours. Traffic is notoriously bad in the city, so this was a fitting example. My colleague Mahmoud H. presented a traditional workflow process in a Jupyter Notebook off a Google Cloud DataProc cluster, efficiently processing over 300 million records (this is relatively "small"). The processed traffic information was then displayed in ArcGIS Pro in a time-aware layer to reflect the congestion visually while activating a time slider. At the end of the presentation, we surprised the audience by using ChatGPT to translate Arabic sentences to SparkSQL code, and Azure OpenAI GPT4 handled the translation very well. Look here for code snippets. This form of interaction IS the future, and I am excited to invest more in this technology and in the following areas:

Enhanced Visualization and Real-time Data Integration:

Dynamic Visualization: Integrating real-time traffic data feeds into existing models. This will not only show historical congestion but also provide live updates. Dynamic heatmaps can be particularly effective in visualizing the intensity of traffic at different times.
3D Modeling: Utilize ArcGIS's 3D scene capabilities to give a more immersive view of traffic congestion and urban planning scenarios.

Improved Data Analysis through Machine Learning:

Predictive Analytics: Integrate machine learning models to predict future traffic patterns based on historical data, weather conditions, events, and other variables.
Anomaly Detection: Implement anomaly detection algorithms to identify unusual traffic patterns, which can be crucial for incident response and urban planning.

Enhancing User Interaction and Accessibility:

Multilingual Support: While we showcased the translation of Arabic sentences to SparkSQL code, we should consider expanding this feature to include more languages, making your tool more accessible to a global audience.
Voice Commands and Chatbots: Integrate voice command functionality and develop a chatbot using Azure OpenAI GPT4 for querying and controlling the GeoAnalytics Engine, making the system more interactive and user-friendly.

Scalability and Performance Optimization:

Optimization for Large Datasets: Continue to refine the efficiency of processing large datasets. Explore the latest advancements in distributed computing and in-memory processing to handle even larger datasets more efficiently.
Cloud Integration: Ensure the solutions are cloud-agnostic and can be deployed on any public or private cloud provider, enhancing the system's scalability and reliability.

Collaboration and Sharing:

Collaborative Features: Develop features that allow multiple users to work on the same project simultaneously, including version control and change tracking for shared projects.
Export and Sharing Options: Enhance the ability to export results and visualizations in various formats and share them across different platforms, facilitating easier collaboration and reporting.

Ethical Considerations and Transparency:

Data Privacy: Address data privacy concerns by implementing robust data encryption and anonymization techniques, ensuring that individual privacy is respected while analyzing traffic patterns.
Algorithm Transparency: Provide clear documentation and explanations of the algorithms used, promoting transparency and trust in your system.

Back in Action: GenAI Meets GeoSpatial

2024-01-27T14:40:00.000-05:00

Hello, everyone. It has been a while since my last post, and I wanted to explain my absence. I have been working on demanding client projects requiring confidentiality, so I couldn't share anything.

But now I'm back and excited to dive into something new and exciting.

Generative AI (GenAI) has gained much attention lately, but I'm taking it to a different level by merging Large Language Models (LLMs) with insights from geospatial analysis. It's GenAI with a GeoSpatial twist.

I want to introduce a simple project, "ReAct geospatial logic with Ollama," which uses resources such as the Python Langchain and Ollama.

I'm thrilled to be back and can't wait to start this new journey with you. Keep an eye on this space for future updates, tips, and unique code snippets. Your feedback and questions are valuable, so please don't hesitate to reach out.

I'll see you in the next post, and as usual, you can check out the source code here.

On Machine Learning in ArcGIS and Data Preparation using Spark

2020-08-24T08:28:00.000-04:00

Artificial Intelligence / Machine Learning implementations have been part of Esri software in ArcGIS for a very long time. Geographic Weighted Regression (GWR) and Hot Spot Analysis are Machine Learning algorithms. ArcGIS users have been utilizing supervised and unsupervised learning like clustering to solve a myriad of problems and gain geospatial insight from their data. We just did not call these learning algorithms AI/ML back then. Not until the recent popularity of DeepLearning that finally blossomed from the AI Winter and made AI a household name. And guess what? Esri software has now DeepLearning implementations!

It is important to understand the difference between AI, ML, and DL. I came across this very insightful article that analogizes the relationship to Russian dolls.

Typically, collected data is very "dirty", and a lot of cleaning has to performed on it before processing it through a Machine Learning algorithm. The bigger the data, the harder is the process especially when pruning noisy outliers and anomalies. But then, that could be what you are looking for, outliers and anomalies. That is why "Data Janitor" is a new job title, as a majority of your time when dealing with data is cleaning it! That is why I love to use Apache Spark for this task. It can handle BigData very efficiently in a distributed, parallel share-nothing environment, and the usage of the DataFrame API and SQL makes the maniplation a breeze. Utilizing the latter two in an interactive environment like a Jupyter Notebook enables quick cleaning and more importantly insightful explorations.

Now the best part, we can have all 3, Jupyter, Spark and ML, in one enviroment; ArcGIS Pro!

This notebook and that notebook demonstrate the usage of Spark in Notebook in Pro to clean and explore the data, and then prepare it for Machine Learning. We are using the built-in Forest Based Regression to predict (or attempt to predict) the trip duration of a New York City taxi given a pickup and dropoff location.

The last notebook, enables the user to select pickup locations (in the below case, about JFK) and "see" the errors of the model on a map.

Some will argue that all this could have been done using Pandas, and that is true. But that is not the point of this demonstration and this assumes that all the data could be held in memory which is true in this case (1.45) but will fail when you have 100's of million of rows and you have to process all this data on one machine with limited resources. If you like Pandas, I recommend that you check out Koalas.

ArcGIS Pro, Jupyter Notebook and Databricks¶

2020-08-03T21:48:00.000-04:00

Yet another post in the continuing saga of the usage of Apache Spark from a Jupyter notebook within ArcGIS Pro.

In the previous posts, the execution was always within the ArcGIS Pro environment on a single machine, albeit taking advantage of all the cores of that machine. Here, we take a different angle, the execution is performed on a remote cluster of machines in the cloud.

So, we author the notebook locally, but we execute it remotely.

In this notebook, we demonstrate the spatial binning of AIS broadcast points on a Databricks cluster on Azure. In addition, to colocate the data storage with the execution engine for performance purposes, we converted the local feature class of the AIS broadcast points to a parquet file and placed it in the Databricks distributed file system.

More to come :-)

Virtual Gate Crossing

2020-08-02T16:58:00.001-04:00

Yet another continuation post regarding Pro, Notebook, and Spark :-). In this notebook, we will demonstrate a parallel, distributed, share-nothing spatial join between a relatively large dataset and a small dataset.

In this case, virtual gates are defined at various locations in a port, and the outcome is an account of the number of crossings of these gates by ships using their AIS target positions.

Note that the join is to a "small" spatial dataset that we can:

Broadcast to all the spark workers.
Brutly traverse it on each worker, as it is cheaper and faster to do so that spatially index it.

The following are sample gates:

And the following is a sample processed output:

More to come...

MicroPath Reconstruction of AIS Broadcast Points

2020-08-01T11:52:00.001-04:00

This is a continuation of the last post regarding ArcGIS Pro, Jupyter Notebook, and Spark. And, this is a rehash of an older post in a more "modern" way.

Micropathing is the construction of a target's path from a limited set of a consecutive sequence of target points. Typically, the sequence is time-based, and the collection is limited to 2 or 3 target points. The following is an illustration of 2 micropaths derived from 3 target points:

Micropathing is different than path reconstruction, in such that the latter produced one polyline for the path of a target. Path reconstruction losses insightful in-path behavior, as a large number of attributes cannot be associated with the path parts. Some can argue that the points along the path can be enriched with these attributes. However, with the current implementations of Point objects, we are limited to only the extra M and Z to the necessary X and Y. You can also join the PathID and M to a lookup table and gain back that insight, but that joining is typically expensive and is difficult to derive from it the "expression" of the path using traditional mapping. A micropath overcomes today's limitations with today's traditional means to express the path insight better.

So, a micropath is a line composed of typically 2 points only and is associated with a set of attributes that describe that line. These attributes are typical enrichment metrics derived from its two ends. An attribute can be, for example, the traveled distance, time, or speed.

In this notebook, we will construct "clean" micropaths using SparkSQL. What do I mean by clean? As we all know, emitted target points are notoriously affected by noise, so using SparkSQL, we will eliminate that noise during the micropath construction.

Here is a result:

More to come...

On ArcGIS Pro, Jupyter Notebook and Apache Spark

2020-07-31T09:47:00.003-04:00

Been a while since I posted something, and thank you, faithful reader, for coming back :-)

I'm intending to writing a series of posts on how to use Apache Spark and Machine Learning within a jupyter notebook within ArcGIS Pro. Yes, you can now start a jupyter notebook instance in ArcGIS Pro to create an amazing data science and data exploration experience. Check out this link to see how to get started with a Jupyter notebook in Pro. But...my favorite hidden "GeoGem", is that Pro comes with built-in Apache Spark, and y'all know how much I love Spark. People think that Spark is intended for only BigData analytics. That is so far from the truth. What I love about it, is the frictionless movement of data and analysis locally or remotely and the language fusion. In my case, I'm using Python, SQL, and Scala.

The usage of Apache Spark in Pro was demonstrated in the publically shared Covid-19 Contact Tracing Application and the Proximity Tracing Application.

In this first notebook, we will start by loading selected features into a Spark dataframe from a local feature class, process the dataframe using Spark SQL, and write the result back to an ephemeral feature class that will be displayed on the map.

Like usual, all the source code can be found here.

On Patterns Of Life: From MacroData to PicoData

2018-05-27T13:31:00.000-04:00

Insight (or GeoInsight in our case) is lost in the deluge of data that we are acquiring today from everyday sensors that are machine or humanly generated. This project is a set of heuristic Spark based implementations to reveal signals from the movement of ships in and out of the Port of Miami.

The idea is to extract small clean data (PicoData) from the overlap of a massive amount of data (MacroData). The aggregation of "clean" PicoData derived from MacroData trust into the forefront patterns of life.

For example, given the following display of AIS broadcasts:

We can mutate the data to reveal the "clean" influx of ships into the harbor at high tide:

Like usual, you can download all the source code from here.

On ML and Elastic Principle Graphs

2018-01-01T11:06:00.000-05:00

Happy 2018 all. It has been a while since my last post. Thank you for your patience dear reader. Like usual, the perpetual resolutions for every year in addition to blogging more are to eat well, often exercise and climb Ventoux.

Onward.

I genuinely believe that 2018 will be the year of the ubiquity of Geo-AI. It will be the year when Machine Learning and Spatial Awareness will blossom inside and mostly outside the GIS community.

We at Esri have had Machine Learning based tools in our "shed" for a long time. Every time an ArcGIS user performs a graphically weighted regression, trains a random trees classifier or detects an emerging hot spot, that user is using a form of Machine Learning without knowing it!

So one of my "missions" for 2018, it to make this knowledge more explicit to our users and non-traditional GIS users. Also, start to implement new forms of Machine Learning.

Machine Learning (ML), a branch of Artificial Intelligence (AI), is a disruptive force that is changing how today's industries are gaining new insight from their data. ML uses math, statistics and probability to find hidden patterns and make predictions from the data without being explicitly programmed. It is this last statement that is disruptive, "No explicit programming"! An ML algorithm iterates "intelligently" over the data, and the patterns emerge. Being iterative, the more data an ML algorithm is exposed to, the more refined the output becomes. Thus the coupling of BigData and ML is a perfect marriage fueled by cheap storage, ever more powerful computational power (think GPU) and faster networking.

This reemergence of this "No Explicit Programming" paradigm such as Deep Learning, Reinforcement Learning, and Self Organization is skyrocketing the likes of Google's AlphaGo-Zero, Facebook, and Uber.

So, I am starting this launch with something I have been fascinated by for quite some time, and that is "Elastic Principle Graphs."

It is a "deep" extension of PCA that I came across it during my research of mapping noisy 2D data to a curve and was fascinated by its self-organization.

After reading, (and rereading for the nth time) this paper, this GitHub repo is a minimalist implementation in Scala.

Happy New Year All.