Confused Life - Reloaded

From SaaS to Serviced Software -- the code was never the hard part

2026-04-03T15:00:00.000-07:00

Sometime in the last two years the bottleneck shifted. I used to spend most of a greenfield week staring at an empty editor, scaffolding routes, arguing with ORMs, and coaxing CSS into something that did not look like a government form from 2004. Today I can prompt my way to a working CRUD app with auth, a reasonable schema, and even half-decent styling before lunch. The code, it turns out, was never the hard part. The hard part was–and still is–keeping the thing alive once real users touch it.

The SaaS mental model

“Software as a Service” trained an entire generation to think the value lives in the application layer. Build a clever feature, wrap it in a subscription, ship a landing page. The implicit promise: we write the software, you pay monthly, everyone wins. That framing put the spotlight squarely on creation–new features, new integrations, new UI polish.

But anyone who has operated a SaaS product past the euphoric launch week knows where the hours actually go:

Rotating secrets and patching CVEs at 11pm on a Friday
Chasing down why the invoice PDF lambda timed out in ap-southeast-2 but not us-east-1
Fighting Terraform drift after someone clicked through the console “just this once”
Explaining to a customer why their data export is 48 hours stale because the Celery worker OOM-killed itself

The ratio of build-time to keep-it-running-time was already lopsided. AI just made it more obvious.

Enter the vibe-coded prototype

Large language models have compressed the “zero to working prototype” phase from weeks to hours. Cursor, Copilot, Aider, v0, Bolt – pick your weapon. The scaffolding phase that used to justify a two-pizza team for a quarter now fits in a solo weekend sprint. I have experienced this first-hand: prompting out a FastAPI backend with DynamoDB tables, an SPA frontend, and a deployment pipeline that mostly works. The code is not elegant. It does not need to be. It is structurally correct enough to demo and iterate.

This is genuinely magical. It is also genuinely dangerous, because it creates an illusion of completeness. The prototype works on your laptop, passes the happy path tests the LLM also generated, and looks great in the demo. Then production happens.

Production is where software goes to get serviced

Here is where the mental model needs updating. We are not really selling Software as a Service anymore. We are selling Serviced Software – and the distinction matters.

In the SaaS framing, the software is the product and the service is the delivery mechanism. In the Serviced Software framing, the service is the product and the software is just the substrate it runs on. Customers do not care that your backend is FastAPI or Express or Rails. They care that:

It is up when they need it (hosting, redundancy, failover)
Their data is safe (encryption at rest and in transit, access controls, backups that actually restore)
It stays current (dependency updates, OS patches, framework migrations)
It costs a predictable amount (no surprise egress bills, no runaway autoscaling)
Someone answers the phone when it breaks at 3am

None of those are code problems. They are operational problems. And they are the problems that AI is worst at solving, because they require sustained human judgement over months and years, not a one-shot generation pass.

The maintenance asymmetry

There is a well-known asymmetry in software engineering: building version 1.0 is perhaps 20% of the total lifetime cost. The remaining 80% is maintenance, evolution, and eventual decommissioning. AI has dramatically reduced the cost of that first 20%. But it has done almost nothing for the other 80%.

If anything, AI makes the maintenance problem worse. When code is cheap to produce, people produce more of it. More repos, more microservices, more side projects that “just need a small server.” Each one becomes a maintenance liability. Each one needs patching, monitoring, log rotation, certificate renewal, database vacuuming. The open-source maintainer burnout problem I wrote about after PyConAU 2019 is now everyone’s problem, because everyone is now a maintainer of their own vibe-coded fleet.

I keep thinking about the analogy to 3D printing. When desktop printers got cheap, everyone printed trinkets for a month. Then the printers gathered dust because the hard part was never fabrication–it was design, finishing, and material science. The bottleneck moved upstream and downstream simultaneously, leaving the newly-cheap middle step feeling oddly irrelevant.

What “Serviced Software” looks like in practice

If you accept that the value has shifted from code creation to code stewardship, a few things follow:

Platform engineering matters more than feature engineering. Internal developer platforms (Backstage, Port, Humanitec) that abstract away infrastructure and enforce guardrails are more valuable than another AI code assistant. The companies investing in paved roads for deployment, observability, and incident response will win over those investing in faster code generation.

Managed services eat custom code. Every line of custom infrastructure code is a future maintenance burden. I learned this the hard way running microservices from folders on EC2 – cron jobs, plain-text .env files, manual venv management. It worked, but every operational incident was my problem. The appeal of managed Postgres over self-hosted, or Vercel over hand-rolled CI/CD, is not laziness. It is recognizing where your scarce operational attention should go.

Security becomes the primary differentiator. When everyone can generate a working app, the ones that survive are the ones that do not get breached. Supply chain attacks, dependency confusion, credential leaks in AI-generated code that helpfully hardcoded an API key – these are the failure modes of the vibe-coding era. Security is not a feature you bolt on; it is the service layer that justifies the subscription.

Cost modeling is a core engineering skill. I wrote about EKS baseline costs being $70/month for personal projects back in 2020. That sensitivity to operational cost used to be a niche concern. Now that anyone can spin up infrastructure with a prompt, understanding what it costs to keep running is table stakes. Cloud bills are the new technical debt – invisible until they are catastrophic.

Tactical tornados vs strategic maintainers

Every engineering org has its archetypes. The tactical tornado is the superstar feature developer who can bang out a new module in a weekend, leave a trail of impressed stakeholders, and move on to the next shiny thing. They are celebrated in sprint reviews, promoted quickly, and held up as the template for “10x engineers.” Product managers love them because they turn roadmap dreams into demo-able reality at terrifying speed. AI amplifies the tornado: give them Copilot and a weekend and they will generate an entire product surface.

Then Monday arrives. The tornado has moved on to the next feature. Someone else inherits the code–no tests beyond the happy path, no runbooks, secrets in environment variables that nobody documented, an autoscaling policy copied from a blog post that assumed a different traffic shape. The tactical tornado created value in a burst. The strategic maintainer captures it over months.

Strategic maintainers are the people who go deep on the two or three features that actually drive revenue. They understand the edge cases customers hit at 2am. They know which database index is holding the query plan together and what happens when the table crosses 50 million rows. They are the ones who turn a flashy demo into a reliable product–incrementally tightening error handling, adding observability, negotiating with product managers about which “small” feature requests would actually require rewriting the payment flow.

Product managers sit in the middle of this tension. A good PM dreams up the features that will deliver the most value and sequences them so the team can ship without drowning in maintenance debt. A less experienced PM treats every sprint as a feature factory, stacking new work on top of un-serviced foundations, because the roadmap rewards visible output over invisible resilience. In the Serviced Software framing, the PM’s job is not just “what should we build next?” but “what is costing us the most to keep running, and is it worth it?”

The industry has historically rewarded tornados and feature-shipping PMs disproportionately. Promotions go to the person who launched the thing, not the person who kept it alive for three years. AI will sharpen this imbalance unless orgs explicitly revalue the strategic maintainer. When code generation is cheap, the scarce skill is not writing new software–it is understanding existing software deeply enough to keep it healthy.

The human layer

There is a deeper point here that goes beyond tooling. The shift from SaaS to Serviced Software is really a shift from building to caring. Building is exciting, creative, dopamine-rich. Caring is routine, patient, often invisible. Our industry has always undervalued the people who keep the lights on relative to the people who ship new features. AI will widen that gap unless we consciously correct for it.

The sysadmin, the SRE, the on-call engineer, the person who actually reads the CVE advisories and patches before the exploit drops – these roles are becoming more important, not less. The code is becoming commodity. The care is becoming scarce.

Where this goes

I do not think SaaS as a business model disappears. But I think the honest version of what customers are paying for will increasingly sound like: “We run and maintain this software so you do not have to.” That is not a new idea – managed hosting has existed forever. What is new is that the software layer itself is becoming thin enough that the operational layer dominates the value proposition.

We are entering an era where the question is not “can you build it?” but “can you keep it running, secure, updated, and affordable for the next five years?” The answer to that question has never been a one-shot prompt. It is a sustained commitment – and that, for now, remains stubbornly human.

Two Agents, One Codebase: An F1 Race Team Approach to Porting ACOLITE to Rust

2026-03-07T16:00:00.000-08:00

In Formula 1, every team fields two drivers. Not as a backup plan – as a strategy. One driver pushes the pace, forcing rivals to respond. The other holds position, manages tyres, and covers the alternative strategy. They share telemetry, they share a garage, but they are running different races on the same track. The team wins when both cars score points, not when one driver tries to do everything.

Porting a scientific Python codebase to Rust feels remarkably similar. You need the aggressive driver – the one who charges into unfamiliar code and lays down fast laps of Rust implementation. And you need the calculating driver – the one who reads the data, watches for degradation, and calls out when the numerical precision is drifting. Two AI coding agents, paired like Norris and Piastri, sharing a codebase but operating on different parts of the problem.

The Starting Grid: Why Rust for ACOLITE?

ACOLITE is RBINS’ atmospheric correction toolkit for aquatic remote sensing. It handles everything from Landsat and Sentinel-2 to hyperspectral sensors like PACE OCI (286 bands) and PRISMA (239 bands). The Dark Spectrum Fitting (DSF) algorithm is elegant – image-based, no external atmospheric inputs – but in Python, processing a full PACE scene involves reading 291 NetCDF variables, interpolating multi-dimensional LUTs, and correcting each pixel’s reflectance through a chain of gas transmittance, Rayleigh scattering, and aerosol models. On a decent machine, this takes around 230 seconds.

The seed was planted at FOSS4G 2025 in Auckland when Leo Hardtke ran a tutorial on Earth Observation processing with Rust. It was plagued by Nix environment issues (as I noted in my conference write-up), but when the code ran, it was fast. Zero-cost abstractions and fearless concurrency are not just slogans at that point – they are wall-clock seconds you are not spending waiting for your atmospheric correction to finish.

I had also been watching Rob Woodcock’s acolite-mp branch, which tackled the same performance problem from within Python. His approach was clever: per-band parallelism with memory budgets tuned to cloud CPU-to-RAM ratios (2, 4, or 8 GiB per core), replacing NumPy’s interpolation with the multithreaded pyinterp, and carefully managing the GIL contention that Python’s threading model inflicts on you. He got Sentinel-2 from 791s down to 197s and Landsat from 312s to 99s on a 24-core i9 – roughly a 3-4x speedup.

But the GIL is still there. The memory model is still Python’s. And as Rob himself noted, “further performance improvements are possible but require more extensive changes to the file handling” and “there is a fair amount of GIL contention which limits threading being caused by some structural choices in the implementation.” At some point, you are fighting the language rather than the problem.

Rust sidesteps all of this. No GIL. No garbage collector. Rayon gives you data-parallel iterators that map across bands or tiles with work-stealing. Memory usage is deterministic and known at compile time – you can profile it statically before deploying, which is a sentence that makes no sense in Python-land but is table stakes in systems programming.

The Pit Crew: Two Agents via ACP

Here is where the teammate analogy really kicks in. In F1, a team with only one driver is not half a team – it is no team at all. You cannot run a split strategy with a single car. You cannot use one driver to hold up a rival while the other pulls a gap. The performance of the pair exceeds the sum of the individuals because they create options that a solo driver simply cannot.

Porting 40,000+ lines of scientific Python to Rust is the same. A single AI agent writing Rust will drift – the implementation slowly diverging from Python’s numerical behaviour until your reflectance values are off by just enough to be scientifically useless. You need the second driver to keep it honest.

The solution I landed on was a multi-agent orchestration harness using the Agent Client Protocol (ACP), a JSON-RPC 2.0 protocol over NDJSON stdio that lets coding agents communicate in a structured way:

Agent	Role	F1 Equivalent
Kiro	Executor – writes Rust code, runs tests, reads files	Lead driver – pushes the pace, sets fast laps
Copilot	Proposer – reviews output, suggests next steps, cross-checks Python	Second driver – covers the strategy, watches the gaps
Human	Approver – filters proposals before dispatch	Team principal – makes the call on when to pit

The workflow per sensor port looks like this:

Human provides --task to the orchestrator (tools/agent_harness.py)
Kiro receives the task via ACP session/prompt and starts writing code
Kiro streams output via session/update chunks
Output goes to Copilot for review against the Python source
Copilot proposes ACTION: lines – “fix the gas transmittance interpolation order”, “the Rayleigh LUT needs pressure stacking”
Human approves or rejects
Approved actions go back to Kiro
Repeat until regression tests pass or maximum cycles reached

This is not vibe coding. This is a two-car team running a split strategy.

Think about how McLaren or Red Bull operate. The lead driver qualifies on pole and sets the pace in clean air. The second driver starts on a different tyre compound, runs a longer first stint, and emerges from the pits into a different part of the field. They are solving complementary problems – one optimises for raw speed, the other for strategic coverage. Neither is redundant.

Kiro is the lead driver. It attacks the Rust implementation aggressively – writing loaders, porting DSF algorithms, wiring up rayon parallelism. It sets fast laps. It also occasionally bins it into the gravel trap by hallucinating a NumPy broadcasting rule that does not exist in ndarray.

Copilot is the second driver. It reads the Python source, cross-references the Rust output, and spots where the gap to parity is growing. “The gas transmittance interpolation order is wrong” is exactly the kind of radio call a second driver makes – not flashy, but it prevents a DNF.

The human is the team principal. You do not override the drivers on every corner, but you make the strategic calls: do we pit now and fix this RMSE regression, or do we push on and address it in the next stint? Is a 0.002 RMSE difference in Sentinel-2 reflectance acceptable? (It is – that is within float32 precision.) When do we switch from tiled DSF to fixed DSF mode for this sensor?

Together, they converge faster than either alone, for the same reason that two cars gathering tyre data in free practice gives the team more information than one car doing twice as many laps.

The Telemetry: Regression Tests Against Real Data

In F1, both drivers generate telemetry. The team overlays their data – braking points, throttle application, cornering speed – to find where one is faster and why. The overlay is the truth. Not the driver’s feeling, not the engineer’s simulation, but what the car actually did on the track.

Regression tests are our telemetry overlay. The Python ACOLITE output is Driver 1’s trace. The Rust output is Driver 2’s. We overlay them pixel-by-pixel, band-by-band, and look at the delta. When the traces diverge, something real has changed and we need to understand whether it is a genuine improvement or an error we need to correct.

There are currently 141 Python regression tests that compare Rust output against Python output pixel-by-pixel across real satellite scenes:

Landsat 8/9: 13 regression + 13 Rust-vs-Python + 7 benchmark tests
Sentinel-2 A/B: 19 regression + 15 Rust-vs-Python + 9 benchmark tests
PACE OCI: 17 regression + 14 Rust-vs-Python + 12 DSF comparison + 12 ROI + 10 full-scene tests

The tolerances are tight. Sentinel-2 achieves RMSE < 0.002 (physics-equivalent). Landsat gets RMSE < 0.02. PACE full-scene (1710 x 1272 pixels x 291 bands) hits mean RMSE of 0.004 with 100% of pixels within 0.05 of Python. Correlation coefficients are R > 0.999 across all sensors.

These are not toy tests on synthetic data. They run against actual L1 scenes downloaded from USGS and NASA. When the tests break, something real is wrong.

The Performance Gap: Where the Seconds Go

Sensor	Scene Size	Rust	Python	Speedup
Landsat 8	62M px x 7 bands	66s	180s	2.7x
Landsat 9	62M px x 7 bands	56s	180s	3.2x
Sentinel-2 A	30M px x 11 bands	52s	182s	3.5x
Sentinel-2 B	30M px x 11 bands	64s	173s	2.7x
PACE OCI (full)	1710 x 1272 x 291 bands	84s	230s	2.7x

The PACE result is particularly satisfying. The key optimisation was switching from 291 per-band NetCDF reads to 3 bulk detector reads, then applying rayon-parallel atmospheric correction across tiles. Load is 12 seconds, AC is 34 seconds, write is 35 seconds. That write phase for a 291-band hyperspectral cube goes to GeoZarr V3 with gzip compression – try doing that in a Python event loop without your memory allocator throwing a tantrum.

Energy Efficiency: The Fuel Strategy Nobody Talks About

Here is the part where I get philosophical – and where the F1 analogy turns from metaphor into mirror.

Formula 1 underwent a fuel efficiency revolution in 2014. The FIA introduced hybrid power units, capped fuel flow at 100 kg/hour (monitored 2,200 times per second), and forced teams to extract maximum performance from minimum fuel. The result was not slower cars – it was faster cars that used less. The 2026 regulations go further: fossil carbon is prohibited entirely, the MGU-K will deliver three times the electrical power (350kW vs today’s 120kW), producing up to 1,000 horsepower while burning sustainable fuel. Less fuel, more power. That is not a trade-off – it is an engineering constraint that drives innovation.

The same constraint applies to scientific computing, we just pretend it does not. Cloud computing bills are denominated in dollars, but the underlying unit is energy. Every CPU cycle your atmospheric correction burns is a watt drawn from a power grid somewhere. When you are processing continental-scale Sentinel-2 archives or the full PACE ocean colour mission, those watts add up. Python is the V10 era of scientific computing – glorious, unrestricted, and profligate with resources.

Rust is the hybrid power unit. Its advantage is not just speed – it is energy per unit of work. A 3x speedup roughly translates to using a third of the compute time, which means a third of the energy, a third of the carbon footprint, and a third of your AWS bill. The Rust Foundation and others have pointed to studies showing compiled languages like Rust and C using an order of magnitude less energy than interpreted languages for equivalent workloads. Just as F1 teams discovered that fuel efficiency constraints forced them to build fundamentally better engines, switching to Rust forces you to think about memory layout, allocation patterns, and data flow in ways that Python’s garbage collector lets you ignore – until the bill arrives.

And here is the irony that would make an F1 sustainability officer wince: Earth observation processing is meant to monitor the planet’s health. Burning excess energy to do it is like running your emissions-monitoring car on leaded fuel. F1 recognised that the sport’s 20-car grid is only 1% of its total carbon footprint, but pursued fuel efficiency anyway because the technology trickles down. The same logic applies to EO processing pipelines. The individual savings per scene are modest, but at continental archive scale they compound – just like how F1’s hybrid innovations now power road cars from Ferrari’s SF90 to the electric components in every modern turbo engine.

Static memory profiling makes this tangible. In Rust, I can tell you at compile time that a Sentinel-2 full-scene atmospheric correction will peak at approximately N gigabytes of memory, because the allocations are deterministic. In Python, you find out at runtime – usually when the OOM killer visits your pod. F1 teams know their fuel load to the gram before the formation lap. Rust gives you the same certainty for compute.

Kubernetes 1.35 and Vertical Pod Autoscaling

This deterministic memory behaviour dovetails nicely with Kubernetes 1.35’s improvements to Vertical Pod Autoscaler (VPA). VPA watches your pod’s actual resource usage and adjusts CPU and memory requests/limits accordingly. When your workload has predictable resource usage – as Rust workloads tend to – VPA converges quickly to the right allocation instead of oscillating between OOM kills and wasted headroom.

For a processing pipeline that ingests satellite scenes of varying sizes (a Landsat scene is 62 million pixels across 7 bands; a PACE scene is 2.2 million pixels across 291 bands), VPA can right-size pods per sensor type. Rust’s static memory profile means the VPA recommendations stabilise fast, which means tighter bin-packing, which means more scenes processed per node, which means lower cost per scene.

Compare this to Python pods where memory usage is non-deterministic, garbage collection spikes are unpredictable, and the VPA has to overprovision to avoid OOM. The 2 GiB/core cloud ratio that Rob’s acolite-mp was carefully designed around becomes less of a constraint when your language does not waste half of it on interpreter overhead.

Out-of-Band Development: Preventing Merge Conflicts with Upstream

One design decision I am particularly happy with is keeping the Rust port on a separate feature branch (feature/rust-port) and treating it as out-of-band from the Python codebase. ACOLITE upstream is actively maintained by Quinten Vanhellemont at RBINS, with regular additions of new sensors, algorithm refinements, and bug fixes. A traditional “rewrite in Rust” approach would create an immediate fork that diverges with every upstream commit.

Instead, the Rust code lives in src/, benches/, and tests/ directories that do not exist in upstream Python ACOLITE. The Python code in acolite/ stays untouched. The regression tests are the synchronisation mechanism – they import both the Python ACOLITE modules and the compiled Rust binary, run the same scene through both, and compare outputs.

When upstream adds a new sensor or changes a gas transmittance coefficient, the regression tests fail in the Rust port. That failure is the trigger: it goes into the agent harness as a --task, Kiro investigates the numerical difference, Copilot cross-references the upstream commit, and the fix lands in Rust without touching a single Python file. No merge conflicts. No rebasing nightmares. Just tests that enforce parity.

This is how you keep an acceleration layer in sync with a moving target – you do not try to merge them. You test them against each other.

What Is Next: The Gap to Full Sensor Parity

The roadmap has the current state at 48 Rust tests, 141 Python regression tests, and three sensors fully validated (Landsat 8/9, Sentinel-2 A/B, PACE OCI). The architecture – loader, AC, writer – is clean and extensible. But three sensors out of 30+ is a qualifying lap, not a race win. Here is what closing the gap to full ACOLITE parity actually looks like.

Sensor Coverage: 3 down, 30+ to go

Python ACOLITE supports a sprawling constellation of sensors. The Rust port has ticked off the three highest-priority ones but the remaining fleet breaks into tiers:

Tier 1 – Near-term (shared loader patterns exist):

Sensor	Bands	Loader Type	Blocker
Sentinel-3 OLCI	21	NetCDF	Sensor def exists, needs full pipeline
PRISMA	239	HDF5	Shares pattern with PACE
DESIS	235	HDF5	Shares pattern with PACE
EnMAP	224	HDF5	Shares pattern with PACE
EMIT	285	NetCDF	Similar to PACE OCI

These are the low-hanging fruit. The PACE port proved out the NetCDF and hyperspectral GeoZarr writer path; PRISMA/DESIS/EnMAP share the HDF5 loader pattern. Each is a well-scoped --task for the agent harness – Kiro writes the loader and wires up the AC pipeline, Copilot validates against Python output on a reference scene.

Tier 2 – Medium-term (new loader work required):

Sensor	Bands	Notes
Landsat 5 TM / 7 ETM+	7-8	Older calibration metadata formats
PlanetScope (Dove/SuperDove)	4-8	Commercial format, GeoTIFF based
WorldView-2/3	6-29	Multi-resolution, pan-sharpening
Pleiades	5-7	DIMAP format
QuickBird-2	5	Legacy but still used
VIIRS (NPP/J1/J2)	22	Swath-based HDF5, three platforms
Aqua/Terra MODIS	36	HDF4/HDF-EOS
GOCI-2	12	Korean ocean colour mission

Each of these needs a dedicated loader – different metadata formats, different calibration approaches, different file layouts. The atmospheric correction core (DSF, gas transmittance, Rayleigh, aerosol models) is shared, but getting the radiometrically calibrated top-of-atmosphere reflectance array into the pipeline is the per-sensor work.

Tier 3 – Geostationary and niche (lowest priority for aquatic applications):

GOES ABI, Himawari AHI, MTG-I FCI, SEVIRI, Sentinel-3 SLSTR, AMAZONIA-1 WFI, CHRIS, HYPERION, HICO, HyperField, HYPSO, Tanager. Some of these (HYPERION, HICO) are decommissioned but their archives are still processed. Others (Tanager at 420 bands, HYPSO at 120) are newer hyperspectral missions that would benefit most from Rust’s performance advantage.

Beyond Loaders: The Algorithm Gap

Sensor parity is not just about reading files. Python ACOLITE has several processing features the Rust port does not yet implement:

ROI subsetting: Limit processing to a bounding box or polygon – critical for operational workflows that do not need a full scene
Ancillary data retrieval: NCEP ozone, pressure, and wind speed from NASA OBPG; currently the Rust port uses default values
DEM-derived pressure: Copernicus DEM at 30/90m for surface pressure estimation in mountainous coastal regions
Glint correction: Sun glint removal for low-latitude ocean scenes
RAdCor adjacency correction: The physics-based adjacency effect correction developed under the STEREO program
TACT thermal processing: Surface temperature from Landsat thermal bands via libRadtran – this one is architecturally interesting because it requires calling an external Fortran radiative transfer code
Interface reflectance (rsky): Sky reflection correction at the air-water interface
L2W water products: Chlorophyll-a (OC algorithms), TSS (Nechad, Dogliotti), turbidity, Secchi depth – the derived products that downstream scientists actually use

The L2W gap is the most consequential. Most ACOLITE users do not care about surface reflectance per se; they want chlorophyll maps or turbidity time series. Until the Rust port can produce L2W outputs, it remains a fast atmospheric correction engine rather than a complete aquatic remote sensing toolkit.

The Realistic Path

Closing this gap is not a sprint, it is an endurance race – appropriately enough. The agent harness makes each sensor port a repeatable, testable unit of work. The pattern is established: write loader, wire to AC pipeline, run regression tests against Python, fix deltas, validate on real data. Each sensor port takes the agents a day or two of focused work plus human review.

At the current pace, Tier 1 sensors are within reach in the near term. Tier 2 will follow as the loader library matures. The algorithm features (ancillary data, glint, TACT) are orthogonal to sensor coverage and can be developed in parallel. L2W is the final milestone – when the Rust port can ingest a Sentinel-2 scene and produce a chlorophyll-a map that matches Python to within measurement uncertainty, the port will be race-ready for production.

Each of these is a --task for the agent harness. Two drivers, one constructor’s championship. The lead driver pushes into unfamiliar sensor territory, the second driver validates against Python, and the telemetry overlay catches every divergence before it compounds into a retirement.

If the intersection of Rust, Earth observation, and AI-assisted development interests you, the code is all on GitHub. Feel free to ping me with ideas, bug reports, or competing approaches – especially if you have a cleverer way to handle the N-dimensional LUT interpolation. That one was a fun 3 days of Rapid Rust Rewrite fuelled by AI Amphetamine Analogs.

Copilot + Arduino CLI + Saleae: Driver Development for V93XX

2026-02-27T16:00:00.000-08:00

I have been spending time building out the V93XX_Arduino library for Vangotech energy-monitoring ASICs, and this is one of those projects where tooling makes or breaks momentum.

The pairing that worked best for me was:

GitHub Copilot for repetitive protocol code, tests, and workflow glue
arduino-cli for deterministic compile and upload loops
Saleae Logic Analyzer captures to confirm what is actually on the wire

The short version: Copilot gets you to a plausible driver quickly, but the logic analyzer gets you to a correct driver.

V93XX driver workflow walkthrough

Why this trio works

Developing protocol drivers is usually a game of “spec says one thing, silicon does another thing”.

If you rely only on serial logs, you can miss timing and framing issues. If you rely only on captures, you can waste hours writing boilerplate and one-off scripts. Using these three together gives a tighter loop:

Copilot proposes code and tests from your intent
Arduino CLI compiles and flashes quickly from scripts
Saleae confirms framing, parity, baud behavior, and CRC bytes
Copilot helps refactor after you learn what the bus is doing

In practice, this turned the V9381 UART ChecksumMode and waveform/FFT work from a stop-start activity into a repeatable pipeline.

Project context: V93XX_Arduino

Repository: whatnick/V93XX_Arduino

The library currently covers:

V93XX_UART for V9360 and V9381 UART modes (including address pins and checksum modes)
V93XX_SPI for V9381 SPI paths (faster acquisition, up to MHz clocks)
Examples for baseline comms, waveform capture, and FFT

Useful docs in the repo:

Practical workflow

1) Start with a machine-checkable baseline

Before touching hardware, run the local CRC and framing tests.

python tools/test_checksum_mode.py

If this is red, do not flash anything yet.

2) Use Copilot for structured implementation work

I get better results when prompts are specific and constrained. Example prompt:

Implement V9381 UART read path with explicit ChecksumMode behavior.
Keep Clean mode strict and Dirty mode permissive.
Add unit tests for CRC-8 edge cases and frame parsing.
Do not change public method names.

Copilot is strongest here at:

Building test scaffolding quickly
Keeping repetitive register access code consistent
Producing incremental refactors after captures reveal protocol edge cases

3) Consume datasheets with PDF MCP and keep them in-repo

Before generating more driver code, I now ingest the vendor datasheet with pdf-reader-mcp so Copilot can work from extracted register tables and command framing notes.

My workflow is:

Commit the original datasheet PDF to the repository, for example under docs/datasheets/v93xx/.
Use PDF MCP to extract the high-value sections (register map, UART/SPI timing, checksum/CRC rules, waveform buffer details).
Save extracted artifacts back into the repo as text or markdown notes, for example docs/datasheets/v93xx/register_notes.md.
Prompt Copilot using both code and extracted notes so generated changes are tied to concrete datasheet language.

This gives you a versioned paper trail from silicon docs to driver behavior, which is very useful when reconciling captures from the logic analyzer.

4) Compile and flash with Arduino CLI

For ESP32-S3 targets, this keeps the build deterministic and scriptable:

arduino-cli board list
arduino-cli compile --fqbn esp32:esp32:esp32s3 examples/V9381_UART_DIRTY_MODE
arduino-cli upload --fqbn esp32:esp32:esp32s3 --port COM3 examples/V9381_UART_DIRTY_MODE

PowerShell users can run the project helper script:

.\tools\run_automated_tests.ps1 -Port COM3

That pipeline can run unit tests, compile, upload, serial verification, and optional capture/analysis phases.

5) Validate on-wire behavior with Saleae

The logic analyzer phase is where protocol assumptions get tested.

For UART work:

Confirm bus settings (baud, parity, stop bits) match the target mode
Verify frame boundaries and inter-frame timing
Check CRC byte placement and value against expected payload sum/complement logic

If using the helper scripts in this repo:

python tools/capture_v9381_uart.py
python tools/analyze_checksum_captures.py [capture_dir]

This gives a concrete report of expected vs observed CRC and whether behaviour matches Clean vs Dirty semantics.

What changed in my debugging habits

Old approach

Edit sketch
Upload
Stare at serial output
Guess

New approach

Ask Copilot for narrow, test-backed change
Build and flash with Arduino CLI from script
Capture with Saleae and compare against expected frame math
Feed mismatch details back into Copilot for targeted fixes

It feels closer to hardware TDD than trial-and-error firmware hacking.

Trade-offs and cost notes

Arduino CLI is free and excellent for repeatable CI-style local loops
Copilot is a paid productivity tool, but pays for itself when protocol code churn is high
Saleae hardware is not cheap, but it can save days when parity/timing/CRC bugs are subtle

If you are doing serious serial or SPI driver work, a logic analyzer is not optional for long.

Troubleshooting notes that keep coming up

Wrong serial configuration (especially parity) can look like random CRC failure.
Upload success does not imply runtime protocol correctness.
Dirty mode can hide issues by design; keep a Clean mode path for strict validation.
Keep datasheet extracts in-repo so Copilot prompts reference real tables, not memory.
A scripted workflow (pdf-reader-mcp + arduino-cli + capture + analysis) beats ad-hoc manual steps every time.

Closing

For this V93XX work, the winning combination was AI-assisted coding plus old-school instrumentation.

Copilot helps me move faster, Arduino CLI keeps the loop reproducible, and Saleae captures keep me honest.

If you are building protocol drivers, treat those as complementary tools, not alternatives.

Loading FPGA Bitstreams with OpenOCD and Raspberry Pi - From Captain DMA to NeTV2

2026-01-27T16:00:00.000-08:00

A Hot Adelaide Afternoon and a Pile of FPGAs

I was recruited to wire together the NeTV2 to the Raspberry Pi 5 PCI bus by some means. Tim donated a bunch of hardware to make it work. I picked them up from his place, while he sorted through the contents of his container. The ultimate aim is to get them going as part of fpgas.online.

The box contained a treasure trove: an NeTV2 boards from bunnie’s (Andrew Huang) video overlay project, and lots of Raspberry Pi’s. As I went down the rabbit hole I acquired some more hardware for myself, Captain DMA development boards, and various other Artix-7 based experiments (Acorn, LiteFury). These boards share a common trait – they’re designed to appear as network devices on a PCI Express bus, which makes them interesting for all sorts of applications from video processing to… let’s say “creative” system analysis and game hacking using auto-target features.

The challenge with inheriting random FPGA boards is getting new gateware onto them. The original programming cables might be lost, proprietary software might be Windows-only and in Chinese, and documentation can be scarce. This is where OpenOCD and a humble Raspberry Pi come to the rescue. As long as you figure out which pins are the JTAG, and can get a PR merged.

Pi4 and Pi5 with NeTV2

The Problem: DMA Cards and Their Programming Woes

Captain DMA cards (and similar Artix-7 based DMA devices) typically come with a CH347 USB JTAG interface. The vendor provides a GUI tool, but it’s entirely in Chinese and only runs on Windows. For those of us who prefer reproducible, scripted workflows on Linux – or want to program boards headless on a Raspberry Pi – this is less than ideal.

The workflow the GUI implements is actually straightforward:

Connect to the CH347 JTAG interface
Detect the FPGA/JTAG chain
Load a BSCAN SPI bitstream for JTAG-to-SPI access
Program the SPI flash with the target FPGA image
Refresh the FPGA to load the new bitstream

The key insight is that we’re not programming the FPGA directly – we’re using JTAG to access the SPI flash chip that stores the bitstream. The BSCAN primitive acts as a bridge between JTAG and the SPI flash.

OpenOCD Scripts for Captain DMA

I’ve put together a set of scripts at https://github.com/whatnick/fpga-dma-scripts that replicate the Chinese GUI’s functionality using pure OpenOCD commands. The scripts support Linux/macOS and Windows, and handle:

Automatic BSCAN bitstream download from quartiq’s prebuilt collection
CH347 interface configuration installation
Version checking for OpenOCD (needs 0.12.0+dev for CH347 support)
Cross-platform flashing with proper error handling

Captain DMA clone attached to Raspberry Pi

Basic Usage

On Linux/macOS:

./flash.sh 75t /path/to/pcileech_75t484_x1.bin

On Windows:

flash.cmd 75t C:\path\to\pcileech_75t484_x1.bin

The scripts support three Artix-7 variants: 35T, 75T, and 100T. The underlying OpenOCD command is:

openocd -c "set BSCAN_FILE bscan_spi_xc7a75t.bit" \
        -c "set FPGAIMAGE pcileech_75t484_x1.bin" \
        -f flash.cfg

The flash.cfg orchestrates the JTAG chain detection, BSCAN loading, and SPI programming sequence.

Example Output

A successful flash looks like this:

Open On-Chip Debugger 0.12.0+dev-02377-gb9e401616 (2026-01-25-09:47)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
xc7_program
jtagspi_program
Info : CH347 USB To UART+JTAG from vendor wch.cn with serial number 0123456789 found. (Chip version=5.44, Firmware=0x45)
Info : clock speed 7500 kHz
Info : JTAG tap: xc7.tap tap/device found: 0x13632093 (mfg: 0x049 (Xilinx), part: 0x3632, ver: 0x1)
Info : [xc7.proxy] Examination succeed
Info : Found flash device 'win w25q64fv/jv' (ID 0x1740ef)
Info : sector 0 took 234 ms
Info : sector 1 took 250 ms
...
Info : sector 32 took 259 ms
Info : sector 33 took 256 ms

Each sector takes about 230-260ms to program. A full 75T bitstream covers around 34 sectors, so expect 8-10 seconds for the flash operation.

NeTV2 Wired over PCI-e

NeTV2: Bunnie’s Video Overlay Board

The NeTV2 (Network Television version 2) is bunnie’s open-source HDMI video overlay board. It uses the same Xilinx Artix-7 family (XC7A35T or XC7A100T depending on variant) and was designed for real-time HDMI signal processing. The board features:

Artix-7 FPGA (35T or 100T variant)
DDR3 SDRAM (K4B2G1646F)
PCIe x4 interface
RMII Ethernet
HDMI input and output
SD card slot
SPI flash for bitstream storage

The original NeTV2 scripts from bunnie’s repo (https://github.com/bunnie/netv2mvp-scripts) were designed for Raspberry Pi 2/3 using the older sysfs GPIO interface. I’ve submitted a PR (https://github.com/AlphamaxMedia/netv2mvp-scripts/pull/1) that updates these for Raspberry Pi 4 and upstream OpenOCD.

Raspberry Pi 4 GPIO JTAG Configuration

The key changes for RPi4 compatibility involve the bcm2835gpio driver configuration:

# Boilerplate setup for Rpi on Alphamax configuration

adapter driver bcm2835gpio

transport select jtag

set _CHIPNAME xc7a35t

# Raspi4 uses a different peripheral base address
# RPi 2/3: 0x3F000000
# RPi 4:   0xFE000000
bcm2835gpio peripheral_base 0x3F000000

# Speed coefficients tuned for RPi 3B+
bcm2835gpio speed_coeffs 100000 5

# GPIO pin assignments for JTAG
adapter gpio tck 4
adapter gpio tms 17
adapter gpio tdi 27
adapter gpio tdo 22
adapter gpio srst 24

reset_config none

adapter speed 10000

The GPIO pins map directly to the Raspberry Pi’s header:

Signal	GPIO	Physical Pin
TCK	4	7
TMS	17	11
TDI	27	13
TDO	22	15
SRST	24	18

SPI Flash Programming with Raspberry Pi

The cl-spifpga-rpi4.cfg config file handles the actual programming:

# Burn an FPGA image onto the SPI ROM

source [find interface/alphamax-rpi.cfg]

source [find cpld/xilinx-xc7.cfg]
source [find cpld/jtagspi.cfg]

init

jtagspi_init xc7.pld $BSCAN_FILE
jtagspi_program $FPGAIMAGE 0

virtex2 refresh xc7.pld

exit

The command to flash a NeTV2 from a Raspberry Pi:

/opt/openocd/src/openocd \
  -c "set BSCAN_FILE /home/k8s/netv2mvp-scripts/bscan_spi_xc7a100t.bit" \
  -c "set FPGAIMAGE /home/k8s/pcileech_netv2_top.bin" \
  -f /home/k8s/netv2mvp-scripts/cl-spifpga-rpi4.cfg

The output shows the Macronix SPI flash being programmed:

Info : BCM2835 GPIO JTAG/SWD bitbang driver
Info : clock speed 10000 kHz
Info : JTAG tap: xc7.tap tap/device found: 0x13631093 (mfg: 0x049 (Xilinx), part: 0x3631, ver: 0x1)
Info : [xc7.proxy] Examination succeed
Info : Found flash device 'mac 25l6405' (ID 0x1720c2)
Info : sector 0 took 286 ms
Info : sector 1 took 282 ms
...

Note the device ID difference: 0x13631093 for the 100T part versus 0x13632093 for the 75T. The flash chip is a Macronix MX25L6405 (ID 0x1720c2) versus the Winbond W25Q64FV (ID 0x1740ef) on the Captain DMA boards.

LiteX Builds for NeTV2

If you want to run something other than the original gateware, LiteX has excellent support for the NeTV2. The board definition lives at https://github.com/litex-hub/litex-boards/blob/master/litex_boards/targets/kosagi_netv2.py.

Building a Basic LiteX SoC

Install LiteX following the standard setup, then:

cd litex-boards/litex_boards/targets

# Build for XC7A35T variant with Ethernet
./kosagi_netv2.py --variant=a7-35 --with-ethernet --build

# Or for the 100T variant with PCIe
./kosagi_netv2.py --variant=a7-100 --with-pcie --build

The target supports:

DDR3 SDRAM: Using the K4B2G1646F module with LiteDRAM
Ethernet: RMII PHY via LiteEth
PCIe: x4 Gen2 via LitePCIe
SD Card: Both SPI mode and native SDIO
LED Chaser: For the obligatory blinky demo

Platform Definition Highlights

The platform file defines the IO constraints:

_io = [
    # 50 MHz clock input
    ("clk50", 0, Pins("J19"), IOStandard("LVCMOS33")),
    
    # User LEDs
    ("user_led", 0, Pins("M21"),  IOStandard("LVCMOS33")),
    ("user_led", 1, Pins("N20"),  IOStandard("LVCMOS33")),
    # ...
    
    # SPI Flash for bitstream storage
    ("spiflash", 0,
        Subsignal("cs_n", Pins("T19")),
        Subsignal("mosi", Pins("P22")),
        Subsignal("miso", Pins("R22")),
        Subsignal("wp",   Pins("P21")),
        Subsignal("hold", Pins("R21")),
        IOStandard("LVCMOS33")
    ),
]

The platform also defines the programmer configuration, which now supports both OpenOCD and OpenFPGALoader:

def create_programmer(self, programmer="openocd"):
    if programmer == "openfpgaloader":
        return OpenFPGALoader(cable="digilent_hs2")
    elif programmer == "openocd":
        bscan_spi = "bscan_spi_xc7a100t.bit" if "xc7a100t" in self.device else "bscan_spi_xc7a35t.bit"
        return OpenOCD("openocd_netv2_rpi.cfg", bscan_spi)

Provisioning for fpgas.online

The ultimate destination for these NeTV2 boards is fpgas.online – a community project that provides free remote access to FPGA development boards. The service currently has around 10 FPGA boards connected, each with a webcam pointed at the LEDs so you can see your bitstream running in real-time.

Each board is attached to a Raspberry Pi that handles:

JTAG programming via the GPIO bitbang interface
Webcam streaming for visual feedback
Web-based terminal access for interactive development
Queue management so multiple users can share the hardware

This is where the Raspberry Pi JTAG programming approach really shines – you can reflash gateware remotely without any physical access to the boards. Combined with LiteX’s rapid build cycle, it becomes practical to iterate on FPGA designs from anywhere in the world.

If you want to try FPGA development without buying hardware, head over to https://ps1.fpgas.online/fpgas/ and claim a board. There’s usually one free and ready to use.

Why This Matters

These techniques apply to any Artix-7 board with SPI flash storage:

Reproducibility: Scripts in version control beat clicking through GUIs
Automation: CI/CD pipelines can flash test firmware automatically
Headless operation: A Raspberry Pi in a lab can program boards remotely
Cross-platform: The same OpenOCD configs work on Linux, macOS, and Windows
Tool preservation: When vendors disappear, open source tooling remains
Remote labs: Services like fpgas.online can offer FPGA access to anyone

The BSCAN SPI approach is particularly elegant – it uses a small “helper” bitstream that turns the FPGA’s JTAG interface into a SPI master, allowing you to program the flash chip that normally stores the FPGA configuration. Once programmed, a power cycle or refresh command loads the new bitstream.

Resources

fpgas.online Remote Lab: https://ps1.fpgas.online/fpgas/
Captain DMA Scripts: https://github.com/whatnick/fpga-dma-scripts
NeTV2 RPi4 Scripts: https://github.com/AlphamaxMedia/netv2mvp-scripts/pull/1
LiteX NeTV2 Target: https://github.com/litex-hub/litex-boards/blob/master/litex_boards/targets/kosagi_netv2.py
BSCAN SPI Bitstreams: https://github.com/quartiq/bscan_spi_bitstreams
PCILeech FPGA Project: https://github.com/ufrisk/pcileech-fpga
Video Walkthrough: https://youtu.be/CNQnafoOTCM

The next time you inherit a box of mystery FPGA boards, you’ll know what to do. OpenOCD and a Raspberry Pi can breathe new life into almost any Xilinx 7-series board – no proprietary tools, no Windows VM, no Chinese language skills required.

SQRL Acorn boards

Feel free to ping me with other interesting DMA or video processing boards you’ve encountered. There’s always more gateware to explore.

Seven Years to Kubernetes: A Turing Pi 1 Christmas Miracle

2025-12-24T16:00:00.000-08:00

The Seven Year Itch (For Hardware)

Often I buy more hardware than I need. ~~Actually, strike that~~ – I always buy more hardware than I need. It’s a disease, really. This particular affliction manifested in 2018 when I pre-ordered a Turing Pi 1 because I had convinced myself that building a Raspberry Pi cluster would be the perfect way to learn Kubernetes.

It was not the perfect way.

Little did I realize that it would take me seven years to gather all seven Raspberry Pi Compute Module 3+ boards and finally bootstrap a k3s cluster. In that time:

Kubernetes went through approximately 47 major versions
The Raspberry Pi 4 and 5 came out (and experienced their own chip shortages)
I discovered my Turing Pi board had a faulty ethernet switch
I aged visibly, just look at my github profile and videos from recent conference presentations.

The Recipe for Homelab Kubernetes Suffering

In this write-up, I’ll outline what it actually takes to set up a Raspberry Pi 3+ cluster in 2025. Consider this a cautionary tale wrapped in a tutorial. I’ll probably resell this now-functioning cluster to another masochist – er, enthusiast – and use the recouped capital to buy something newer that will sit on my shelf for another seven years.

Step 1: Acquire the Base Board

The Turing Pi 1 was a great option back in the day. It’s a mini-ITX form factor board that accepts up to 7 Raspberry Pi Compute Modules in SODIMM slots. The on-board gigabit ethernet switch was supposed to be the killer feature – no external networking required!

Pro tip: Make sure the on-board switch actually works. Test it before you commit to this path. Mine didn’t, which I discovered approximately 6 years too late.

Step 2: Collect Your Compute Modules (Like Pokemon, But Expensive)

You’ll need Raspberry Pi Compute Module 3+ boards. The Turing Pi 1 can handle up to 7 of them. I sourced mine from Mouser Electronics, though availability has been… variable… over the years.

I really wish there were more alternatives in the SODIMM compute module format. If you’re in the business of making one with a newer processor and more RAM, let’s talk. Seriously. My DMs are open.

Step 3: Flash the OS (The Easy Part, They Said)

The Compute Modules have onboard eMMC storage, which is the preferred boot device. Trying to use SD cards will lead to disappointment, inconsistent boots, and existential questioning of your life choices.

Here’s the gear you’ll need:

A Compute Module IO Board - Something like the Waveshare CM3/CM3+ IO Board or the official Raspberry Pi IO Board to put the module in USB mass storage mode
rpiboot/usbboot - The tool that makes the eMMC appear as a USB drive
Raspberry Pi Imager - The official tool for flashing OS images

Critical step: Bake in your SSH public key during the imaging process. This will save you from having to find 7 spare HDMI cables and keyboards. The Pi Imager has a settings gear icon that lets you configure hostname, SSH keys, and WiFi – use it.


# Generate an SSH key if you don't have one

ssh-keygen -t ed25519 -C "kubernetes-cluster"

Step 4: Network Configuration (Here Be Dragons)

Plug in all the modules and fire up the on-board Turing Pi ethernet. If you’re lucky, the on-board network works and you can access all the nodes. Marvel at how easy this was.

If you’re me, you’ll discover the switch is dead and enter the seven stages of homelab grief:

Denial: “It’s probably just a loose connection”
Anger: Unprintable + emails to Turing Pi support and learning that the board is End-of-Life and unsupported.
Bargaining: “Maybe I only need 4 nodes anyway”
Depression: stares at pile of unused compute modules
Acceptance: “I guess I’m buying USB ethernet adapters”

The Workaround: Get a bunch of USB-to-Ethernet adapters like the TP-Link UE300 and wire them into an external switch.

Unfortunately, only 4 of the compute modules have their USB ports exposed on the Turing Pi 1. For the other 3, you’ll need to do some creative soldering to expose the USB D+/D- and power lines. That’s just 12 more flying wires on the board. What could go wrong?

USB Ethernet Adapters and Working cluster

Step 5: The Case Mod (Optional But Satisfying)

I got a nice acrylic case to put it all in. It has a fan connection on top for cooling, which you’ll need because 7 Pi’s generate surprising heat.

There were no extra slots for the 3 additional USB connections I needed. But I have a Dremel, two weeks of Christmas holidays, and absolutely no fear of voiding warranties.

Step 6: Actually Installing Kubernetes (The Easy Part, For Real This Time)

With SSH keys baked in, installing k3s is delightfully straightforward using k3sup (pronounced “ketchup”, because of course it is).


# Install k3sup

curl -sLS https://get.k3sup.dev | sh

sudo install k3sup /usr/local/bin/



# Bootstrap the first node as the server

k3sup install --ip 192.168.1.101 --user k8s



# Join additional nodes as agents

k3sup join --ip 192.168.1.102 --server-ip 192.168.1.101 --user k8s

k3sup join --ip 192.168.1.103 --server-ip 192.168.1.101 --user k8s

# ... repeat for remaining nodes

k3sup SSHes into each machine, downloads the necessary bits, and bootstraps a low-resource-friendly cluster with etcd (or SQLite) as the datastore. It’s genuinely magical compared to kubeadm.

Reality check: After the k3s install, the Pi 3 doesn’t have much headroom left for actually running applications. We’re talking about 1GB of RAM shared between the OS, kubelet, and your workloads. It’s a great testbed for learning the k3s API and running ARM binaries natively, but don’t expect to run your company’s microservices on it.

K3Sup based cluster setup

The Final Result

After seven years of procrastination, hardware hunting, debugging dead ethernet switches, creative soldering, and Dremel work, I finally have a working 7-node Kubernetes cluster.

It also serves as a rather festive Christmas decoration with the green PCB and red blinking LEDs. Very on-brand for the holidays.

What’s Next?

Hopefully I’ve been a good boy this year and Santa will bring me some newer clustering hardware to play with. The Turing Pi 2.5 looks tempting with its support for CM4, Jetson, and the Turing RK1 modules.

But knowing me, I’ll buy it in 2025 and finally get it working by 2032.

Hardware Shopping List

For anyone brave enough to follow this path, here’s what you’ll need:

Item	Link	Notes
Turing Pi 1 Board	Turing Pi	Check if ethernet works!
Raspberry Pi CM3+ (x7)	Mouser	8GB/16GB/32GB eMMC options
CM IO Board for flashing	Waveshare	Or official RPi IO Board
USB Ethernet Adapters	Amazon	Just in case
Ethernet Switch	Your choice	8+ ports recommended
Acrylic Case	Various	With fan for cooling

Software & Tools

Raspberry Pi Imager - OS flashing tool
rpiboot/usbboot - For eMMC flashing
k3s - Lightweight Kubernetes distribution
k3sup - k3s installer via SSH
etcd - Distributed key-value store

Feel free to ping me with your own homelab Kubernetes horror stories. Misery loves company.

Refactoring the Austender Scraper: From Colly to OCDS

2025-12-12T00:53:47.054-08:00

The AusTender analyser started life as a straight HTML scraper built with Colly, walking the procurement portal page by page. It worked, but it was always one redesign away from a slow death: layout shifts, odd pagination edges, and the constant need to throttle hard so I could sleep at night.

Then the Australian Government exposed an Open Contracting Data Standard (OCDS) API. That changed the whole game. Instead of scraping tables and div soup, I can treat the portal like a versioned data feed.

Part of why I care: I am kind of fascinated by government spending as a system. Budgets read like a mixture of engineering constraints and political storytelling, and I keep wanting to trace the thread from “budget line item” to “actual contract award” without hand-waving. The Treasurer’s Final Budget Outcome release (2022-23, “first surplus in 15 years”) is exactly the sort of headline that makes me want to drill down into the mechanics: Final Budget Outcome shows first surplus in 15 years.

So the redesign in austender_analyser does three things differently:

Fetch via OCDS, not HTML: Reduce breakage by consuming the API’s canonical JSON, not scraped pages.
Persist to Ducklake: Store releases, parties, and contracts in Ducklake so you can query locally without rerunning the whole pipeline. This does not quite work yet; I am treating it as a learning exercise with Ducklake. It is much easier to learn on a real problem than on toy demo datasets.
Treat caching as optional: Counterintuitively, the local cache is sometimes slower than pulling fresh data. Ducklake’s startup and query overhead can outweigh a simple, parallelized upstream call. The new design keeps the cache but makes it opt-in and measurable.

If you prefer Python, the upstream API team ships a reference walkthrough in the austender-ocds-api repo (see also the SwaggerHub docs and an example endpoint like findById).

Why move off Colly?

Scraping HTML is like doing accounting by screenshot. OCDS is the ledger export.
Less breakage: OCDS is documented and versioned; DOM scraping is brittle.
Faster iteration: You model on structured data immediately, not after a fragile extraction layer.
Clear rate behavior: You can respect API limits without guessing at dynamic page loads.

Why keep Ducklake in the loop?

Ducklake is the reproducibility knob. It lets me freeze a snapshot, replay transforms, and run offline queries when I am iterating on analysis (or when the upstream is slow, or when I just do not want to be a bad citizen).

But caches are not free. Ducklake has startup and query overhead, and that can be slower than simply pulling fresh JSON in parallel. So the pipeline treats Ducklake like a tool, not a religion: measure the latency, pick the faster path, keep an escape hatch when you need repeatability.

Current flow

Pull OCDS releases in batches, keyed by release date and procurement identifiers.
Normalize the JSON into Ducklake tables (releases, awards, suppliers, items).
Emit lightweight summaries for quick diffing between runs.

Lessons learned

A stable API beats heroic HTML scraping almost every time. Even in times of AI and (firecrawl)[https://www.firecrawl.dev/].
Caches are not free; measure them. Sometimes stressing the upstream lightly is faster and still acceptable within published rate limits.
Keep exit hatches: allow forcing cache use, bypassing it, and snapshotting runs for reproducibility.

Next steps: Going deeper : tighten validation against the OCDS schema, add minimal observability (latency histograms for API vs cache), and ship a “fast path” mode that only hydrates the fields needed for high-level spend dashboards. Going broader : find sites and build API and Web aggregators for Australian state tender sites (e.g. VicTender and international ones.

Solar Ceilings and Compounding Dreams

2025-12-06T03:36:26.069-08:00

It is fashionable to wave away physical constraints with vague references to solar abundance and human ingenuity. Yet every balance sheet eventually meets a balance of energy. Solar photons may shower Earth with roughly 170,000 terawatts, but financial markets expect growth that compounds on top of itself forever. The math linking those stories rarely appears in the same paragraph—so let’s put them together.

Setting the Stage

I keep coming back to Tom Murphy’s dialogue in Exponential Economist Meets Finite Physicist. In Act One, Murphy plots U.S. energy use from 1650 onward and it traces a remarkably straight exponential line at ~3% per year. Economists in the conversation shrug; after all, 2–3% feels modest. But compounding at that pace means energy demand multiplies by ten every century. Our economic models implicitly assume something even more optimistic : 8–10% returns in equity markets, pension targets, and venture decks; without asking what energy supply function supports that.

Thermodynamic Guardrails

Murphy distills the second law of thermodynamics into plain language:

“At a 2.3% growth rate (conveniently chosen to represent a 10× increase every century), we would reach boiling temperature in about 400 years… Even if we don’t have a name for the energy source yet, as long as it obeys thermodynamics, we cook ourselves with perpetual energy increase.”

That thought experiment matters less for the literal 400-year timer and more because it shows energy growth must decelerate to avoid turning Earth into a heat engine. Solar panels, fusion, space mirrors … pick your technology. The waste heat still has to radiate away. We cannot spreadsheet, app and AI our way around Stefan–Boltzmann and Black Body radiation.

Solar Arithmetic vs Demand Curves

Let’s grant the optimists a heroic build-out: cover 5% of Earth’s land area with 20%-efficient photovoltaic arrays, assume a generous 200 W/m² average output, and we net roughly 20 TW—about the entire human primary energy demand today. That is fantastic news for decarbonization, but it is not a blank check for compounding GDP. If demand keeps growing at 3%, we would need 20 TW × (1.03)ⁿ in perpetuity. Within 250 years we’d be trying to harvest thousands of terawatts—orders of magnitude more land, materials, storage, and transmission than our initial miracle project. Solar abundance is real; solar infinity is fiction.

Finance Is an Energy IOU

Money is a claim on future work, and work requires energy. When pensions assume 7–8% annual returns, when startups pledge 10× growth, and when national budgets bake in permanent productivity gains, they are effectively promising that future societies will deliver 2–3 doublings of net energy per century. If we instead hit a solar plateau—because land, materials, or social license cap expansion—those financial promises become unmoored. We can pretend that virtual goods, algorithmic trading, or luxury desserts (to borrow Murphy’s Act Four anecdote) deliver infinite utility without added energy, but the chefs, coders, and data centers still eat, commute, and cool their CPU’s , GPU’s and Tensor processors. The intangible economy rides on a very tangible energy base.

Rewriting the Business Plan

Accepting a solar ceiling does not doom us to stagnation. It just forces different design constraints:

grow quality, not quantity—prioritize outcomes per unit energy … do proof of useful work rather that roll the dice and gamble.
align finance with expected energy supply rather than mythical exponentials … and I am not talking of wasting energy on crypto.
treat efficiency gains as buying time, not as a perpetual motion machine … if you learnt enough physics in high school to reject the perpetual motion machine, but have been lulled into perpetual 8% returns from the finance markets, there is a serious schizophrenia issue.
embed thermodynamic literacy in economic education so debates start from the same math.

Murphy ends his essay noting that growth is not a “good quantum number.” It is not conserved. Our job is to craft institutions, portfolios, and narratives that can thrive when net energy flattens, because physics already told us that day will arrive long before our spreadsheets hit overflow errors.

Darwin 2022 - Ruminations Compendium

2025-12-06T03:05:00.000-08:00

Collected reflections from the July 2022 Darwin trip, a narrative of adaptation, organisational change, and expansion can live in a single place.

July 19 – Lemmings And Launchpads

There is no exception to the rule that every organic being naturally increases at so high a rate, that if not destroyed the earth would soon be covered by the progeny of a single pair. Even slow breeding man has doubled in twenty five years, and at this rate in a few thousand years there would literally be no standing room for his progeny. – Charles Darwin

Like the lemming marching and diving into the ocean to self‑regulate, humanity plunges itself into vices of its own creation: alcohol, drugs, violence, and greed. Perhaps the next plunge is into the real ocean or into the vacuum of space, chasing more room in which to stand or float. Failure in harsh environments creates room by removing weaker individuals, or greater resilience by rewarding the most adaptable. Colonial Australia itself was founded on such selection—the most adaptable individuals and the strictest rule enforcers reshaped an unforgiving frontier.

July 20 – Organisational Evolution In Flight

Seeing that a few members of such water-breathing classes as the Crustacea and Mollusca are adapted to live on the land, and seeing that we have flying birds and mammals, flying insects of vast diversified types, and formerly had flying reptiles. It is conceivable that flying fish, which now glide far through air, slightly rising and falling by the aid of their fluttering fins, might have been modified into perfectly winged animals. – Charles Darwin

The ability to skim over water for a few metres comes from external tweaks, but the ability to cross the Pacific like a Godwin Tern comes from internal rewiring: hollow bones, high metabolism, and a brain with a built‑in compass. Organisations face the same distinction. A brief digital-transformation spasm can bolt on an app or a website, yet sustaining that flight demands internal metamorphosis and a sense of direction from leadership. Caterpillars become butterflies through wholesale change—so must companies that aspire to be more than flying fish.

July 23 – Questions For The Corporate Naturalist

Where are the transitional forms?
Organisations with no lines on the org chart operate as pure adhocracy. Hidden behind corporate veils, they are like pupae in cocoons, waiting to emerge in a more defined shape.
How can specialised organs evolve?
Marketing machines, technology muscle, sales teeth, enterprise-planning backbone, analyst frontal lobes—each department is an organ honed for a specific survival task.
Is behaviour or instinct inheritable?
Culture answers this. The rituals, stories, and incentives that survive layoffs and leadership changes become the genetic code of the firm.
Why are some species sterile when crossed, while others are fertile?
Some mergers and acquisitions thrive; others fail because the two organisational genomes cannot integrate and diverge instead of hybridising.

July 24 – Conquering New Lands

He who believes in the struggle for existence and in the principle of natural selection, will acknowledge that every organic being is constantly endeavouring to increase in numbers; and that if any one being vary ever so little, either in habits or structure, and thus gain an advantage over some of that inhabitant, however different it may be from its own place, it will seize on the place of that inhabitant. – Charles Darwin

International expansion is a contest for ecological niches. Bringing hard‑won optimisations from one country to another is a bid to displace incumbents. The organisations that vary—by process, by product, by mindset—claim new ground first.

FOSS4G 2025 Auckland: Hazy Hops and Geospatial Heavy Lifting

2025-11-24T16:00:00.000-08:00

Every year, there is an international gathering of geospatial software geeks working in open source somewhere in the world. This has become known as the FOSS4G conference–with the “F” preferentially standing for “Freedom” rather than just “Free”. The last one of these I attended in person was a long while ago in Sydney (2009!). Since then, I have attended a couple of the local Oceania editions and even served on the board of the non-profit that organizes them, but this was my first global one in a long time.

It was great to see the community still alive with a lot of energy, attracting around 400 people from around the world to Tāmaki Makaurau.

Travel and Weather: The Metallica Tax

November in Auckland is very pleasant. Perhaps that is what drew so many events there at once. Accommodation was expensive to unaffordable due to a Metallica concert and an Indigenous education conference happening simultaneously. There were, of course, amazing shows and entertainment on offer as a result, but my wallet definitely felt the “Metallica Tax”. I ended up staying a 30-minute bus ride away from the city centre in Takapuna, which gave me some nice morning views of the Hauraki Gulf before diving into the windowless conference rooms.

Tutorials: Rust, AI, and No-Code Flows

EO with Rust - Leo Hardtke

Leo Hardtke’s tutorial was all about making Earth Observation (EO) processing faster with Rust and without the Python overhead. It was, however, plagued by Nix vagaries and the classic environment distribution issues that seem to follow “modern” build systems everywhere. When it worked, it was blazing fast. The code is available on GitHub for those brave enough to venture into the world of memory safety and zero-cost abstractions.

AI in the Frontend - Felix Palmer

Felix Palmer showed off a cool Claude-enabled frontend to make Deck.gl do things in response to free-text commands. Zooming and searching is just the beginning; we can also do custom frontend processing using the frontend equivalent of GEOS, Turf.js.

I hit a snag here: I could not get my account activation SMS from Claude while in NZ (roaming issues, I suspect). So, in true maker fashion, I ported the code to work with AWS Bedrock instead. It’s a good reminder that in the world of LLMs, being tied to a single API is a recipe for frustration.

Re:earth Flow - Kyle Waite

This was perhaps the most interesting tutorial from an open-source and national policy perspective. I often insist that science developers learn programming, but sometimes for adoption, this is simply not a feasible change. This is what creates room for numerous commercial and open-source tooling for GUIs and no-code/low-code workflow systems like FME, ESRI, and even the QGIS toolbox.

As part of the Plateau project, the Japanese government is putting together the “Flow” framework for prefectures to process and convert their own 3D models. The intro to the UI tools was amazing, and in time, it seems like it can challenge FME. The state management is done by y.js, and we had a fun time getting multiple users to modify the flow until it broke in spectacular fashion.

Presentations: The ones that stuck with me among many excellent ones

For the full roll of presentations check out the 150+ youtube playlist.

The Java Geospatial Ecosystem - Jody Garnett

Jody Garnett gave a deep dive into the state of the art behind the libraries doing the heavy lifting in the Java Geospatial ecosystem: JTS, GeoTools, and the long journey behind getting the Java Imaging library ported to an open-source equivalent supported by the Eclipse foundation. It’s a reminder that while Python gets the hype, a lot of the world’s spatial data still moves through Java pipes. I ended up also going to his talk on Geoserver, which keeps being a force to reckon with in serving GIS data at scale.

GPU Accelerated Zarr Loading - Wei Ji

Wei Ji’s presentation on GPU-native Zarr was a highlight. Optimizing data throughput for large-scale geospatial ML workflows is the new frontier, and moving the bottleneck from the CPU to the GPU is where the real gains are.

Is Zarr the new COG? - Jarrett Keifer and Julia Signell

An excellent presentation dispelling the hype around Zarr. The takeaway? Given the same input and compression, the resulting content is the same size. It’s the usage pattern and the “tyranny of chunking” that really affects performance, along with the fine points about using multiple files, shards, and inodes.

Navara Web3D Engine - Keiya Higuchi

The Re:Earth folks returned with Navara, a re-imagined 3D engine for the web using a modern stack including WebGPU. It aims to create more options for realistic rendering and a functional separation between GIS functions and visualization functions–which are often messily intertwined in current offerings.

STAC Adventures with Matt Hanson

One of the most fun presentations was Matt Hanson’s “Choose Your Own Adventure” session. He let the audience pick the path in D&D fashion to learn more about the STAC ecosystem. I threw in my two cents on the fact that “not all STAC is made equal” and the desperate need for conformance checks on hosting services that claim to be STAC compliant but fail in subtle ways that break downstream libraries.

ESA’s Zarr Foray - James Banting (SparkGeo)

ESA is restructuring the Sentinel satellite imagery archive into DGGS / Healpix Zarr format and publishing the EOPF toolkit. The SparkGeo presentation showed the state of implementation of the user-facing libraries and how they are being integrated into various tools.

After Parties: Hazy Hops and Community

The conference after-parties at the pub are where most of the real action happens. I spent my evenings hanging out with a mix of old friends and new faces, trying the famous New Zealand hazy hops. There’s something about a good IPA that makes discussing coordinate reference systems much more bearable.

Drop shipping products from PCBWay

2022-11-05T06:23:00.006-07:00

Drop shipping products from PCBWay

For a while I have been ordering PCB’s from PCBWay and parts from Mouser and Digikey, then hand assembling them at home. These have been very small scale cottage industry style runs and ultimately time consuming as I focus more on design and evaluation of new energy monitor ASIC’s such as the V9261F. When PCBWay started offering to stock and drop ship my PCB’s directly from the factory using their extensive clout with DHL, I promptly signed up for the service.

Recently I have been getting my ATM90E36 Devkit PCB’s assembled there. The service has been excellent with concierge like parts choice and purchase. Followed by extremely helpful consultation on assembly progress and correctness.

I received the following images after the first stage and confirmed the crystal and LED’s.

Then I received some more inspection photos to allay any doubts.

Looking forward to the stock appearing on the shop front.

NOTE: This is a paid promotion of PCBWay services

Trucks vs Trains as an analogy for Microservices vs Monoliths

2020-12-27T13:51:00.002-08:00

2018 and 2019 was mostly spent obsessing over containers, trucks, trailers and hand written paper invoices for me. I was helping build out the technology stack and engineering team for Lori Systems. Early in 2019 we made our first DevOps hire, getting Clive from Safaricom and getting started on migrating our handrolled Django monolith from EC2 to EKS. We would make jokes around shipping containers using containers. Clive even had a container shaped stressball with the EKS logo on it. This set me thinking on the parallels between shipping code and shipping goods, perhaps also led to the foundations of this post.

Intermodal Shipping in the real-world and in software

Over the almost 2-years of work in Logistics I learnt a lot about how the global logistics system works. Almost like the life-blood of the planet. Large container ships abstract away contents and ship things from Taiwan to Timbukutu. The seminal book on this topic is perhaps, The BOX. Watching global shipping lanes in Marine Traffic and scraping ships arriving in Mombasa from the KPA Sharepoint became a daily ritual. I digress, back to the original point on the importance for containerization in shipping code or machinery.

Docker uses the ubiquitous whale/ship logo, most containers arrive at ports this way from the oceans of developers. I don't quite have an analogy here for the massive ships that land the containers at ports, some 500 or 1000 TEU's at a time. The analogy here covers land transport aspects, somewhat related to how code runs in production and is typically served via datacenters / public clouds to users.

Containers themselves make transfer of goods/code from development (ships) to production (trains/trucks) easy. However even containerized applications can demonstrate tight coupling similar to what a train has, in effect being a distributed monolith, instead of a true suite of microservices. In my opinion, any system that requires a release train approach for new features is most likely to be a distributed monolith masquarding as microservices. The real flexibility comes from the low coupling between containers and the freedom to release each clearly delineated service at its own cadence on the roads.

Trains are awesome

My 5yo is currently obsessed with steam engines, even though they are from an era long gone. There is something magical about a powerful engine pulling everything along smoothly on a set of constraints (rails). It works nicely as long as no quick changes are needed in the carriages and everyone wants to get to the same destination. Trouble arises when something in the closely coupled chain of components goes awry and requires a quick change. I still don't understand the scene in snow piercer where a few wagons were dumped in a siding at speed. If we can do that one neat trick perhaps monoliths would become much more maintainable. In early stages of a product monoliths are a nice simple entry point, especially if the features are narrowly scoped and well coupled. On the reverse the monolith may be a very good idea for a mature product which is not changing rapidly and perhaps needs to be optimised for performance instead by reducing communication overhead between components by introducing tight coupling. In both cases a modular approach and service-oriented designs are still feasible, as long as the implementation and maintenance team is aware of the implications. People are still driving around in classic cars from the 1900's, where as steam locomotives from that era are languishing in museums.

Trucks are flexible

One of the killer advantages of trucks in the logistics business is their ability to deliver right to the factory or warehouse loading bay. It is simply not feasible to build train tracks to serve every address. Even in areas with great railway infrastructure, buffers (known as Inland container depots) have to be placed to cover the last few miles of transport from the rail to the industrial areas. This sort of mode can sometimes be seen in Microservices being layered on older monoliths to provide user facing services, especially in banking systems. The other great advantage trucks have is the ability overtake each other gradually along the road, this manifests itself in software systems as rolling deployment of new features. Such an approach requires careful management of the stateful parts of the system such as storage and database schemas. Otherwise it turns into a Fast and Furious game of stealing a container from a moving platform, aka the Romanian Rollover.

This analogy is not new

The logistics analogies are rife in software engineering, we ship code, we package things, we have release trains. The largest real world container orchestration organization Maersk uses a 7-point logo surprisingly similar to the most popular software container orchestration platform Kubernetes. I will continue updating this post as more ideas and links come together.

You can engage with article via comments or the twitter thread.

Random Roadtrip Rumination : Microservices vs Monotliths is Trucks vs Trains for shipping features.
— Tisham Dhar (@whatnick) December 26, 2020

Desktop Software API's in Python (KiCAD, FreeCAD, Blender, QGIS)

2020-10-04T15:19:00.004-07:00

Python wraps around everything

For the last couple of years I have mostly written Satellite Data Processing code in Python and plenty of Flask/Django web services. However Python is also an excellent automation tool for GUI based applications allowing custom plugins to be written and functionality provided out of the box extended by users.

The first desktop application I seriously looked at Python plugins for was QGIS. It was early days of learning how to wrap C++ code using SWIG/SIP etc. In the old mailing list you can find a much younger me making inane comments about mixing wrapper metaphors in QGIS with SWIG + SIP. We have come a long way since then and SIP based bindings are the mainstay of QGIS plugins.

QGIS

QGIS has so many Python plugins that they need a registry of their own. Occasionally QGIS Python gets twisted around itself due to multiple Pythons in the user enviroment. You can also flip the python API around and instead of building a plugin you can turn QGIS into a custom desktop application. Which is what I have done with my basic Airport Viewer demo.

QGIS being a fairly extensive and complex C++ application which takes hours to compile, being able to make small quick changes in python is invaluable.

KiCAD

At the time of writing KiCAD has an extensive Python API for processing the automating the PCB layout part of the workflow and this has lead to many innovations in automating traditionally laborious hand layout or even performing complex simulations / optimization to set trace lengths. For example Josh Johnson has one for laying parts out in a circle and Greg Davill has several for length matching and rendering file generation. My personal favourite among the KiCAD scripts is the one for generation of Interactive BOM.

I am really looking forward to Python script support in the Schematic Editor. Meanwhile programmatic Schematic generation tools like Skidl provide schematic oriented Python fun.

The rendering of the PCB's is often done in Blender. Which has its own set of Python nicities.

Blender

My first foray in creating a Blender API based application was during the Kinect USB protocol hacking days. The data stream had just been decoded and I wanted an easy pipeline to a commonly installed / open-source 3D display software. The Python API is mature enough for people these days to quickly put together motion capture plugins for Blender. This plugin however demonstrates the challenges for creating native plugins for blender, the .pyd files for Python have to be recreated for different versions of Blender for ABI compaitibility.

Getting the binaries working has had me thrashing about and posting in forums, then sticking to a working Blender build with Python 2.7 for about 5 years since I did not want to touch it and break it. My integration actually reversed the embedding process, i.e. instead of using additional modules in the Blender embedded python I embedded Blender in a 3D GIS automation.

Native plugin weirdness aside, Blender Python API is a really powerful tool for creating procedural objects from waves / fluid simulation to astrophysics with amuse.

FreeCAD

FreeCAD is sort of the third part of my physical electrical / mechnical design triumvirate. I occasionally design parts for KiCAD in FreeCAD, or bring multiple boards together to test enclosure fit. FreeCAD also has an extensive python library which is leveraged by KiCAD part library maintainers to parametrically generate parts.

The scripting in FreeCAD can be used much like the PCB layout scripts in KiCAD to create this with circular symmetry, like ball bearings which are difficult and repetitive to do by hand.

Final words

There are lots of other pieces of desktop software I have used that have started shipping with Python API's to address the never ending demand from users to easily automate repeated tasks. The live process for making this blogpost in somewhat recursive fashion can be found here.

Started a new blog post around #python wrapping around a bunch of desktop applications to ease automation / extension. Live updates here as I collate material to flesh it out. pic.twitter.com/zz8b29kHcO
— Tisham Dhar (@whatnick) October 4, 2020

I have even made videos withs a proprietary one, I will live that here for anyone interested in my attempts at a voiceover.

Compiling QGIS in MSVC in 2020

2020-08-30T00:12:00.004-07:00

Compiling QGIS on Windows in 200x

I don't quite remember when I decided to help compile QGIS on Windows. It was somewhere between compiling GDAL with ECW support for Photoshop on Windows and getting carried into Direct3D and C# land with NASA WorldWind. It was sometime in the 2000's while still working at Apogee Imaging in Lobethal.

At that point I was manually building a database of the footprints of satellite imagery that filled up a wall cabinet with CD's and DVD's. The technique was something like open up the image, go around edges and trace a polygon. This was days before mature boolean thresholding and reliable/easy raster-to-vector logic.

I hopped on IRC on #qgis in Freenode and chatted with luminaries like timlinux, frankw and gsherman. Listened to the automated notifications from sigq, the commits bot. Things were heating up and instead of a Linux cross-compile to windows using MingW, something native to windows say using MSyS+MingW instead of Cygwin was desired. A lot of GDAL and Qt worked in MingW, so presumably QGIS would too. So I set myself to put together an MSYS environment with all the third-party dependencies that could be used to happily build QGIS. Eventually I built a release in NSIS as well.

My MSYS environment got packed in a zip and shared via FTP/HTTP on a VPS I had back then to the rest of the community. I earned myself a pin in the QGIS core contributor map in Adelaide. Something I am very proud of to this day. Eventually the MingW build got deprecated and native MSVC builds were supported. That's how contributions work, nothing lasts forever. In my IRC days, I helped on-board Nathan Woodrow to QGIS, who in turn I believe helped on-board Nyall Dawson. Nyall has surpassed us all in feature contributions and work on QGIS.

Getting started on QGIS on MSVC blog article. Trying to add some historical perspective on my involvement. Found this acknowledgement in the docs. Thanks to @timlinux . Back in the days of sharing mingw zip files via FTP. pic.twitter.com/NTkQgU5ibN
— Tisham Dhar (@whatnick) August 29, 2020

Fast forward to 2020, compiling QGIS in MSVC

I am getting back into doing lots of Open-source work after long hiatus in private industry with Aerometrex and start-up land with Lorisystems. It is great fun working on mostly in the open at Geoscience Australia. There is actually a recently archived opendatacube + qgis repository here. Seeing that repo and speaking to Nathan and LinuxConfAu inspired me to have a go and getting back into actively working on the Qgis code base. It has sprawled out, with lots and lots of new features. The build system is still familiar via CMake and actually much easier now with MSVC. I cast around for a recent guide and found this. The guide mostly works, however I made some refinements.

Ditched bison and flex via Cygwin to using the one available via Msys2. These can be found here. Not needing the while Cygwin system helps in keeping the windows build system light. Simply download the binaries and add them to the Osgeo4W binaries directory.
Captured my CMakeCache.txt to make it easier to reproduce and debug the build environment for others.
Used Incredibuild in demo mode to use a few NUC's I have lying around to speed up the build. Recording while building failed the first time and worked the next. The whole build from scratch still tooks around 35minutes overall.

I am planning to throw some of my day to day DevOps skills towards the QGIS project and start helping again with Raster enhancements and windows release management. Perhaps getting Incredibuild in the hands of the windows maintainers will help tighten up the iteration cycle and make testing easier.

The twitter thread/ stream of consciousness edition of this is available as well.

Moving from babun to msys2 to get flex and bison required for compiling QGIS from source. https://t.co/W866AOM9Kt pic.twitter.com/fUmqpa9YHY
— Tisham Dhar (@whatnick) June 26, 2020

Microservices the hard way - folders in EC2 Instance

2020-07-22T00:12:00.006-07:00

For day to day work I wrangle containers in EKS these days. However when doing personal projects EKS is a luxury (baseline cost being $70 or so per month). So I decided to do microservice development for the rain radar project using no Docker, no Kubernetes but using:

multiple venvs
multiple service folders
environments in .env files (secrets in plain text)
web service start using @reboot in Cron
scheduled services using ... ya Cron

The whole thing started with noble intentions to use lambda's all the way however I got stuck in using S3-SNS to trigger the lambda and decided to scan the S3 bucket using timestamps to find latest files to process. More on the pitfalls of that later.

The major microservices handle are:

Raw radar data preparation using custom hand crafted algorithm, being ported to Rust here.
Inserting prepared data to DynamoDB as a sparse array and serving this via Flask.
Nowcasting using the timeseries of sparse array of rain observations also serving results via Flask.
Capturing rain events and nowcasts and creating text and gif to send to twitter.

Each of these applications consumes the other to some extent and is sort of separated in responsibility. I decided to deploy them with basic a folder per application on the /home/ubuntu directory, with a venv per folder.

I had it like this for a while. Then I got tired for sshing into the box and git pulling in each folder. So I decided to write a fabfile per application which would do this for me and created deployment keys which would be used to pull the code to this folder. Then I got tired of running multiple fabfiles and decided to setup a polled process which run the fabfiles and git synced the code from a master pipeline.

Eventually I got around to bootstrapping the whole VM using Packer + Ansible playbooks. The development work for it was done locally using Vagrant with Hyper-V as the VM provide to test the same Ansible playbooks. I will follow up on this with a few characters on twitter.

Wrangling VM's with Packer, Vagrant and Ansible all at the same time. Helped me figure out issues in Ansible docs and divergence between bento ubuntu box and AWS AMI. One calls ntp ntpd, the other ntp.service. pic.twitter.com/tSbaPuKtVB
— Tisham Dhar (@whatnick) July 18, 2020

Once the initial Packer AMI is established the choice is to either keep building this image or to move away from the whole VM based old-school stuff to a more modern/fun Kubernetes way.

Replicating Databases in RDS

2020-06-01T06:59:00.005-07:00

One Sunday in 2018 I sat for a whole day in Art Caffe at the ground floor of Yaya Centre in Nairobi on the phone to Norman at AWS Support in Cape Town discussing DMS for MSSQL servers. After a whole day of screen sharing and being on call we decided what we were trying to do was no achievable, but AWS was working on it. The next day AWS sent me an NDA (since expired).

Data replication from on-prem Database instances or between cloud database instances is an issue that comes up all the time. I have hands on experience doing this a couple of times now. This post summarizes my 3 or so attempts at doing this with different sources and targets and lessons learnt.

MSSQL to MSSQL using AWS DMS

At the start-up I was working at we adopted a pre-built mini ERP, it covered logistics workflows and finance / billing aspects. It was built in the 2000's in .NET Classic and ran on IIS and MSSQL server. Quickly the MSSQL became the single point of lack of scalability in the system. Since AWS does not natively support read-replicas for MSSQL RDS instances I looked at DMS to create these replicas. DMS did not quite work as expected and led to the conversation alluded to above with Norman. I ended up performing replication using CloudBasic as the SaaS provider for managing the Change Tracking and handling schema changes in the source tables and propagating them to the target replicas. The replication was fine for single replicas, but quickly bogged the source database down as I added more replicas.

As aside the same database was also being replicated to a Redshift cluster for BI usage using tooling provided by Periscope Data.

As part of this excercise I came to appreciate the advantage to write-only / append-only schemas in improving no-lock replication performance (at the cost of storage), also the need for timestamp columns such as update_time to perform incremental data transfer. I spent a lot of time reading the Schemaless articles by Uber Eng around building Schemaless DB's on top of RDBMS's like MySQL. I don't 100% agree with their design choices but it adds interesting perspective. The bottomline CRUD at scale is HARD.

RDS PostgreSQL to PostgreSQL using DMS

Fast forward a year or so, I am now working at Geoscience Australia, with the Digital Earth Australia. Everything runs on Kubernetes and is highly scalable. Single point of lack of scalability is again the database. A pattern seems to be emerging here. We were performing cluster migration in Kubernetes and I offered to investigate DMS again.

In the MSSQL scenario there is a small prologue, I had previously migrated around 1million cells from a massive Google Sheet to the MSSQL database at the start of my tenure at the startup, by the time we hit scalability issues in the single instance MSSQL we were at 10million rows in the largest append-only table. The PostgreSQL migration of the datacube tables featured 8-9 million rows in the largest table. However the database also has lots of indexes and uses PostGIS for some applications, particularly Datacube Explorer. DMS fell down in support for the Geometry columns, however I learnt a lot in setting up DMS using Terraform IAC and fine tuning for JSON Blob columns, which in Datacube design in 1.8.x series can be upto 2MB in size. DMS migrates standard columns separately from LOB columns.

Ultimately DMS was not feasible for datacube DB migration to a new RDS instance. However I believe core datacube can be migrated next time I try with applications depending on Materialized views and PostGIS setup afresh on new DB. Also by the time I try again Amazon may have better PostGIS support. For the cluster migration we ended up using a snaphot of the last DB.

On-prem PostgreSQL to RDS PostgreSQL

There is a Datacube PostgreSQL DB instance at the NCI which needs to be regularly replicated to RDS. It powers the Explorer Datacube application. However DB migration from one server without direct disk access to RDS where we also don't have disk access using pg_dump / pg_restore for a DB with largest tables being around 22 million rows and the compressed dump being around 11GB is a long running task. Ideally we sort something out that is incremental using update_time generated using triggers. The options explored so far are :

Using an Airflow DAG with Kubernetes Executors wrapped around pg_dump/restore with some application specific details.
Using COPY from S3 support for Aurora PostgreSQL, CSV's restored using the COPY command are generated incrementally.
Using PostgreSQL publish / subscribe and Logical Replication. Networking over the internet to maintain connectivity securely to the on-prem instance via SSH Tunnels and to the RDS instance via EKS port-forwarding.

ADE7816 Energy Monitor

2020-05-07T15:11:00.002-07:00

I have been meaning to try out the ADE7816 Single-phase 6 current channels energy monitor for a while. However time has been lacking for the last couple of years. Finally I have a working version with successful board bring-up and a semi-working Micropython driver, with an Arduino driver in the works.

PCB Design

The PCB design process for this was not easy mostly due to a footprint choice mistake on my part. I had placed the 5x5mm QFN part instead of the 6x6mm QFN part in KiCAD. This made the DRC fail everywhere in standard settings. However it ended up being a collaboration opportunity with Greg Davill who loves to practice and photograph bodging stuff. So I now have a work of art at hand instead of a non-functional board.

Practising my bodge skills

This is a QFN where the footprint on the PCB was incorrect.

Just a couple more bodge wires to add. pic.twitter.com/AmZ1CsWqC3
— Greg (@GregDavill) April 22, 2020

I am even debating whether to place the rest of the parts and possibly take away from the dead-bug awesomeness. Next time need to order parts in advance and make sure I do 1-1 prints to verify footprints before pulling the trigger on PCB's.

Energy Montor Details

Now to more about the energy monitor. This ASIC features 3 single-ended and 3 differential current inputs and a single-phase voltage input, all in very compact 40-pin 6x6mm QFN package. In fact the PCB is large on purpose to accomodate ease of use with stereo-jack type current clamps. The main usage would be in standard households where there are typically 3-4 lighting circuits, 1-2 socket circuits and dedicated Air Conditioning circuit. A single energy monitor could be built to monitor all channels using a single-ASIC and leave out fancy NILM stuff from worrying about the lights. The socket circuits could have anything plugged into them and can potentially have point-of-load monitoring instead of breaker board based monitoring. All this translates to more data being generated for IoT platforms and some sensible firmware work needs to be done to handle that.

ADE7816 Driver Development

This is still work in progress. I have done some initial exploration to find prior art. Nothing exists yet from Arduino however there is some register lists from a Javascript driver written for the now defunct Intel Edison.

Intel never quite had the maker market pinned right to market that board, it makes me sad to think of all the engineering ours sunk into a now defunct platform. Open-source software / hardware helps us salvage some of that. I also sped up the register listing by copy-pasting the ubquitous table from the ADE7816 datasheet and dropping it into a Jupyter notebook to parse all the registers, not as fancy as the Pdf parser I had built before, but much more reliable.

My @adafruit #ESP32 + Logic Analyzer test rig. Due in the background is just to keep cables tidy. The ESP32 runs micropython so that I can iterate on SPI modes quickly and use the excellent struct functions to pack/unpack bytes. https://t.co/0pQgAO1Cgl pic.twitter.com/R2imRukrpT
— Tisham Dhar (@whatnick) April 14, 2020

My driver development follwed the now tried and tested Micropython + Jupyter Notebook + Logic Analyzer path. I used an ESP32 feather as host processor with standard micropython loaded and probed the SPI bus with read-write packets for known registers until the protocol gave in and started responding with some values. The ASIC is super versatile in supported protocols - I2C Slave, SPI Master and SPI Slave modes are all viable. So developing a fully functional driver supporting all the possible modes will take a while. The initial work so far is on the SPI slave mode since all my other work in DIN rail and Featherwing formats is linked to the SPI bus, however the I2C mode can be really interesting for host-processors with fewer pins and flaky SPI support (while having solid I2C support - like the Onion).

If anyone is interested in driver development I am happy to send you a board or you can get one yourself from Oshpark, Aisler or PCBWay. Once the drivers mature I will list it for a wider audience on Tindie.

Distributed Locking from Python while running on AWS

2020-04-04T00:12:00.006-07:00

In the day and age of eventually consistent web-scale applications the concept of locking may seem very archaic. However in some instances attempting to obtain a lock and failing to do so within a limited window can prevent dogpile effects for expensive server side operations or prevent over-write of already executing long running tasks such as ETL processes.
I have used 3-basic approaches to create distributed locks on AWS with the help of built-in services and accessed them via Python which is what I build most of my sofware in.

File locks upgraded to EFS

File based locks in UNIX file-systems are very common. They are typically created using the flock command, avalaible in Python under os-specific flock API. Also checkout the platform independent filelock. This is well and good for a VM or single application instance. For distributed locking, we will need EFS as the filesystem on which these locks are held, Linux-Kernel and NFS will use byte-range locks to help simulate locally attached file system type locks. However if the client loses connectivity the NFS lock-state cannot be determined, better run that EFS with enough replicas to ensure connectivity.
File locking this way is very useful if we are using EFS for holding large file and processing data anyway.

Redis locks upgraded to ElastiCache

Another popular pattern for holding locks in Python is using Redis. This can be upgraded in the cloud-hosted scenario to Redis-Elasticache, This pairs well with the redis-lock library.
Using redis requires a bit of setup and is subject to similar network vagaries and EFS. It makes sense when using Redis already as an in-memory cache for accelration or as a broker/results mechanism for Celery. Having data encrypted at rest and transit may require running an Stunnel Proxy.

An AWS only Method - DynamoDB

A while ago AWS published an article for creating and holding locks on DynamoDB using a Java lock client. This client creates the lock and holds it live using heart-beats while the relevant code section executes. Since then it has been ported to Python and I am maintaining my own fork.
It works well and helps scale-out singleton processes run as Lambdas to multiple lambdas in a serverless fashion, with a given lambda quickly skipping over a task another lambda is holding a lock on. I have also used it on EC2 based stuff where I was already using DynamoDB for other purposes. This is possibly the easiest and cheapest method for achieving distributed locking. Locally testing this technique is also quite easy using local-dynamodb in a docker container.
Feel free to ping me other distributed locking solutions that work well on AWS and I will try them out.

Testing the OrangeCrab r0.1

2020-01-24T05:48:00.001-08:00

After hassling Greg Davill for a while on twitter and admiring his OrangeCrab hardware I managed to catch up with him in person in Adelaide. I have been away in Nairobi till October last year, then I spent a brief few days in Adelaide before coming over to Canberra to take up a position in Geoscience Australia. The new gig is much less time commitment than the start-up world and hopefully will allow more time for blogging and board bring-ups like this one.

I caught up with Greg at a Japanese restaurant in Rundle mall and was treated to his now trademark led cube and led icosahedron. They are insanely detailed pieces of work and deserve staring at. However I am most grateful for the care package he left me, an OrangeCrab v0.1. This is an ECP5 board in feather form factor with an ADC built in to respect the Analog In pins on the feather. My aim for this board is to host some energy monitoring code on the FPGA with a fast 4mbps or so ADC and perform power/energy calculation on some parts of LUT's/DSP and have a softcpu push data out.

Greg also left me a home-made FTDI based board to use a JTAG programmer. The whole setup requires 3 USB cables:

To plugin and power the FPGA board (eventually it should be alos programmable via this port)
To attach a USB-Serial converter and watch the console when the gateware comes up
To program the board over JTAG using an FTDI chip

Getting firmware compiled these days is getting easier, but Greg had done his initial testing with Lattice Diamond. I managed to installed it in WSL and promptly ran into a tonne of issues. The weirdest being close coupling to bash, Ubuntu actually uses dash as its default shell. You can get a Diamond licence and help support integration of diamond in litex-buildenv.

I was about to give up then Greg got it working with the opensource toolchain and [NextPNR-ECP5. I had by then setup litex-buildenv to support the orangecrab. So getting some gateware on was relatively easy. Then I got stuck on RAM timing bug in Litex till a few hours ago when I tested out some new gateware.

Also Checkout out Greg's foboot fork and help make programming over the USB possible and reduce 1 USB cable. More work on this including getting MicroWatt running coming soon.

Around Kenya in 4 days and a year ( part 2)

2019-10-06T06:43:00.000-07:00

The New Year adventure continued past the sad mercury laden mines of Migori to the Tanzania border , Isebania in particular. We met up with professor Sangai Mohochi and had a brief gander into Tanzania ( Serere) for a beer. Later we had some Orokore Beer made from millet, we sat around a car tyre sipping still fermenting beer from straws.

We took our leave from the professor and had a long drive out to Rusinga island. A beautiful place without much of the tourist trappings lakeside places have. The Suba culture is being revived with festivals. We woke up early and did a trip around Rusinga Island and visited Tom Mboya's Mausoleum. One of my high school's illustrious alumni.

After another long day of driving and some fiddling with charging phones directly from a car battery ( another story on how many engineers does it take to charge a phone), we ended up in Naiberi River Campsite. It is more like a glamping spot and ideal for a quiet New Year's celebration around a fire in the main hall.

A late lunch/early dinner at Rift Valley Lodge Golf Club. We went for a post dinner walk on the greens and ran into herds of zebras and antelopes. No wonder this golf club is classed as one of the best in the Africa. All the excitement was followed by an uneventful drive back to Nairobi.

We arrived tired but energized the beauty and possibilities in Kenya. The roads need to be built, electricity needs to be channeled to houses, there is a lot to do. We look forward to hanging around and getting it done. Eric is heading back to his PhD at Harvard, Gichini will be doing something about the weather prediction and I have helped put Lori Systems on a solid technical footing and change the Logistics landscape.

Open-source Sustainability (The tale of 2 package managers)

2019-08-14T14:20:00.003-07:00

Last weekend I had the priviledge to attend the 10th PyconAU and listen to some amazing speakers. I went with my I will write markdown on the fly and make a blog-post at the end of the day mindset. Even though I did write a lot of markdown on the fly, I haven't gathered the courage to push these unedited notes into a public post. Excellent examples of live-blogging from conferences here.
What did happen was that the niggling doubts I had around how open-source works in the real world outside of just the code crystallized. This was as a result of 2 very good talks , one about how the PyPi project works and another around Open-source sustainability beyond money.

For the last year I have been writing and reviewing a lot of React Frontend, Python backend (Flask/Django) and Notebooks code. Both frameworks are super easy to buy batteries for where the included ones are running out of juice. Simply via pip install and npm install you can climb onto the shoulders of giants who are library maintainers and the life-blood of lean start-ups everywhere. However the maintainer burnout is a thing and start-ups when building their stack should be highly cognizant of this. Package repository burnout is also a thing. In my time in the software industry I have seen Maven repositories disappear. More recently NPM go through an identity crisis and the left-pad incident.

The PyPi talk gave me great background on a tool I use every single-hour without too much thought. It takes some dedicated volunteers to keep the dream alive. Who according to Dustin Ingram are :

Unemployed and bored and poor (but super talented)
Paid for by their employer (thanks employers who support FOSS)
Not getting enough sleep (or in my case time with the family)

Vicky's talk covers another aspect. Developers and maintainers need more than money to keep going, they need back-up. The community need insurance against the bus-factor and burn-out. I have been guilty of this myself, putting a few dollars behind features I would like to see in BountySource instead of diving in. This has become more so as I have progressed in my career and become increasingly time-poor. Talking about this would anyone at VSCode like to claim the few dollars we put here ?

I love the longevity and discipline of project warehouse and will find some time to contribute to it. I also look forward to a similar alternative to npm, rather than a caching proxy with community behind it.

Sunbeam Coffee Maker Teardown

2019-06-26T11:36:00.005-07:00

The bad Kenyan grid has struck again. There is very little built in surge protection in the system, so all appliances plugged to the wall end up with in-house surge protectors. This time the victim was my beloved coffee machine which had served us well for more than a year.

So I decided to take it home and spend some quality time with Pascal tearing it down. He is always curious about stuff and what is inside, so the Coffee machine is a complex beast capable of keeping him and myself occupied for hours on end. First we took of the top and looked at the heater and the electromechanical assembly controlling the hot-water head and frother was exposed. Also was exposed the heating tank with dual heating elements and a thermistor (I am assuming a thermistor rather than a thermocouple).

When we took the back plate off we found the main controller board. There is to obvious transformer in the set-up so I assume to whole power supply is cap-drop. There is a giant heat-sink attached to a 3-pin device which I assume is a triac or SCR for controlling the heating elements.

The bottom-left corner has a water pump with its own switching mechanism driven by relatively thin signal-wires from the main board.

There are some more interesting details on the main processor board including thick high current traces driven by the silicon switch with exposed copper and extra solder on top to increase thickness and provide a lower current path.

The main CPU is TQFP packaged micro-controller which is covered by the conformal coating the entire PCB is covered in. A macro image on twitter helped establish the lineage of the processor. It is a chinese 8051 variant with the datasheet here. Any help in translating it will be highly appreciated. But most of the schematic is legible.

Next up will be powering on the processor with a suitable 3.3v supply and examining the analog and digital buses. As well as powering on the whole thing to see why it turns of immediately on start. If the machine is unfixable I am planning to scavenge the thermistors, heating elements etc. to build a reflow mechanism.

All the water circulation mechanism is really interesting as well and can possibly be shoe-horned into a water cooler for a 3D printing system.

Nairobi 2nd OSHW meetup

2019-06-22T10:58:00.005-07:00

After a many month hiatus we finally put together the 2nd Nairobi Open-source hardware meetup. This time we were at the Tav Irish Pub near Nairobi Garage on Waiyaki way. The pub was chosen due to proximility to Nairobi garage and possible use of that venue. As it turned out none of us were memebers paying the $60 per-month fee so we ended up co-opting the pub in Chicago 3H fashion.

PCB Lineup

I had collected PCB's over the last few month from Aisler, OSHPark and PCBWay all shipped to Kenya from Germany, US and China respectively. So we lined up to try various soldering techniques on 0603 Jellybeans and some 20pin TSSOP IC's.

Soldering Demo

The techniques we we tried were the tack, finish and reflow one using the iron and an experimental one with paste and dry-iron I had picked up recently. I didn't have a temperature profile controller for the dry-iron method. So we were just going to wing it by the eye.

Part Placement

We pasted up the boards using screwdrivers and tweezers. Started the reflow till the parts settled. Then simply turned off the iron. I am sourcing a K-Type thermocouple and SSR from KTechnics to get a better handle on this for next time. However the dry-iron does reach sufficient temperature for leaded solder reflow.

Dry iron reflow

By the end of the session we had assembled x2 CS5464 Breakouts. Next step from here will be suitable MicroPython and Arduino drivers to ensure these things actually work. We had a lot of fun over the meetup and I hope to continue having these sessions. Next time one of the participants has promised to bring his Weighbridge Automation and IoT set-up.

Scavenging a Coffee Machine

Following the meetup I have been hunting for accessories to make the dry-iron reflow less of a fire hazard and more controllable. (Un)fortunately the coffee machine at work has been fried by the spikes prevalent in the grid in Kenya (especially during the rainy season), so I have a spare K-type thermocouple I can work into the build. I have also co-opted a tissue holder we had on our dining table to stabilize the inverted iron during reflow.

Stand of opportunity

Onwards to the next Meetup.

KiCon 2019 - Chicago in a weekend

2019-06-22T10:57:00.005-07:00

Getting away from a fast paced start-up to attend conferences is a hard ask. However I managed to get away for a few days to attend the inaugural KiCad conference in Chicago a few weeks ago.

The pace has been such that only after about a month I am getting a chance to write it all up. The first night I managed to attend the Hardware Happy Hour at the Ballast Point. I got to see some amazing stuff including pneumatics for an autoplaying piano using layers of PCB.

The huge selection of beers and the chicago dogs were definitely unexpected bonuses.

Tour of MHub assembly area with pick and place machine and large Molex sign. Also got to see Chris Gammell's desk and pick up some Amazon and Mouser shopping to bring back to Nairobi.

I could definitely use the pick-n-place machine in my house and I hope Chris makes use of it whenever he can.

I also got to speak to some companies I have worked with remotely but not met the principles and representatives in person, including CrowdSupply, Aisler, SnapEDA, Hologram and of course DigiKey. It was great to speak to the KiCAD dev team as well, perhaps I can contribute on the MSVC windows native build. I also managed to score credits freebies from all the sources.

Here are pure notes to self on various topics that I picked up on 2-days of super varied and dense conferencing. Some are commands I would like to run some, are techniques I learnt and some are pure stories I picked up.

Schematics are a drug, use code instead

pip install skidl

Auto-routers are evil but useful

Triangles (Topological/ Using TIN mesh)
Maze solving
Shape based ( Rectangles)
Channel Based
FreeRouting
Adaptive heuristics using deep learning
Linear increase in PCB complexity

High-frequency simulation (in GhZ range) for PCB's

OpenEMS ( MEEP)
Wilkinson filter
Important in Radar Design

Prototyping in a few hours

Isolation milling
Midwest Circuit Technologies
Bantam tools. Binary save.
1/32" end mill 0.8mm tool
10 mill trace

Rendering photorealistic PCB's

Render layers to image texture
Use Blender cycles based render with bump maps for silk-screen and traces

Contributing to KiCAD

Development in LaunchPad + Github
Starter bugs are available

More HF PCB's

30MHz to 3Ghz, with narrow bins
5.6GSPS digitizer
Tuned lines. Right angles create problems.
Use footprint, Arc or Trace around edge cuts.

I woke up pretty early on Saturday morning and walked around and almost hopped on a fire-engine tour. Always wanted to do that. Then I checked the radar, another thing missing in Nairobi, and a huge snowstorm was inbound. Snowstorms at this time in Chicago are atypical, but hey everyone stays in for the coference. By the time the conference finished and I headed home from the after-party everything was covered in snow.

The last day (Sunday) was spent taking an Uber Pool , a novelty for me, to a Vietnamese grocery and spending around 2 hours getting additional exotic food-supplies to bring back to Nairobi

I saw a Harvey milk tail fin and took to the skies back to hop across the Atlantic and Mediterrnean to Nairobi. Flying 40 hrs to conference for 48 hours and shop for 6 is a real pain. Will have to bring the KiCAD movement back home to Nairobi and pour more effort into the local hardware scene.

So long Chicago, see you next conference.

Kogan Energy Monitor Teardown - Sonoff-Pow in a wall plug

2019-03-03T07:28:00.001-08:00

After doing a tear-down on the TPLink Wall Plug energy monitor I found mentions in Australia of the much cheaper Kogan Alternative. So I decided to get a couple of them see what makes them tick as well.

General Assembly of the whole unit

TL;DR - The Kogan one is cheaper because it uses a bare metal (non-linux capable) CPU and a PWM output type energy monitor (without intricacies of a SPI protocol and calibration). It is essentially a repackaged Sonoff Pow.

The details of the build are packages in sections as I discovered them.

LV and Buck converter

The LV section consists of two separate cricuits:

Relay to switch the load
Buck converter to rectify and convert 240v to 5v to power electronics

The diagram below presents both of these sections. The Buck converter is basically the toplogy found in many aliexpress products. Perhaps with the difference of being unisolated (notice no slots). The relay is rated at 15A which is nice compared the TPLink product which uses x2 - 5A relays in parallel to achieve 10A.

Low Voltage (240v) Circuitry

Energy Monitor and LDO

This a 3.3v operated section which uses the super low-cost PWM output energy monitor IC made popular by the Sonoff the measure instantaneous voltage, current and hence power. The pulses are channeled to the main processor for forwarding to whatever backend Kogan has put together.

Metering and LDO

Main Processor

The design seems to have taken a standard ESP module and planted it on a basic PCB to fit in the power-plug form factor, sideways. The carries board has lots of markings and test-points making the task of reverse engineering and putting new firmware on this board almost too easy.

Processor module adapter

Marked bottom test pads of processor carrier PCB

Over all this looks like a nice certified unit which you can run your own firmware on thanks with the help of the right triangular security bit.

Around Kenya in 4 days and a year ( part 1)

2019-01-23T11:21:00.000-08:00

2018 was a year on making impromptu decisions and hopping on random flights heading for adventure. In the beginning of the year I left Aerometrex to explore new opportunities at Lori.

It turned out I finished the year in similar markovian motion travelling around Kenya with a couple of high school friends. Thank you so much Eric for driving and showing us the beautiful country.

I received a call late on the 29th December proposing the trip. I booked a ticket of Safarilink and went to bed. Flying out of Wilson Airport is magnitudes easier than flying out of Jomo Kenyatta International. I almost missed the flight, but the empty Christmas streets in Nairobi and low access overheads in Wilson meant I flew into Kisumu with zero dramas. I hopped in a Probox from Kisumu to catch up with the other 2 in Kericho. The Probox are known for being death machines, but is a super cheap and fast way to get around. There is another article somewhere revolving around the impact of the Probox on transport in Kenya.

Tiny Kisumu Airport

Safarilink Flight (Turboprop)

We then travelled via some roads still being built from the tea-gardens of Kericho to the lush sugarcame farms of Awendo. In sections the road meandered into farms and cars had to go in a single file and often backup into fields when a truck came by from the other side.

Road through Kisii

It was a great night of going out to watch football match in Ulanda town and staying at Owuor's Simba. The road from Awendo to Ulanda is awesome. The sugar cane is a mass scale project and produces a lot of bagasse which can be recycled into paper tissue and other products with the right infrastructure.

Mercury laden waters used to extract gold nodules

Sheds housing diesel engines and rock crushers for mining

Next day we took a side trip to Macalder gold mines. It was in interesting experience what people will do to earn a living in far flung places in the country. Migori county is known for its high concentration of mercury in water. The operations here clearly explain why. Rocks are dug up from underground gold mine and manually broken up into smaller chunks and ground into dust using diesel operated octagonal drum crushers. The whole place is noisy. The dust is then filtered to remove solubles and then mixed with mercury to form gold amalgam nodule, women (some of whom were pregnant) then stir the mixture by hand to find gold bits. Mercury is then recovered by retorting in small ovens and reused. At least some work is being done to raise awareness of the harm from Mercury, hopefully sanity will prevail over economic incentives.

We then continued our trip south towards Kenya Tanzania border and I will continue the story in another post.