wamonomicon

Eye @ Electromagetic Fields 2024

2025-05-09T00:00:00+01:00

It has been a while since I've posted anything here, so thought I would put something up about the first project I managed to take in an even vaguely finished state to Electromagnetic Fields. I first attended in 2016 and was lucky enough to be invited to the amazing Hat Village. To get a feeling for the festival I'll refer you to fellow Hat Villager Sam's excellent blog. If you've returned after reading that, I can state that yes indeed my son will be back for the next one (although it may involve more exam revision this time)!

We unsurprisingly make silly hats for Hat Village and this started as an idea by my son wanting a hat like the Las Vegas sphere. While impractical as headwear, the idea remained and grew into this. I'm going to put up more details in the GitHub repository but here are a few details for now:-

It's built from a translucent street light sphere with a 3D printed mount for a micro-projector and fish-eye lens, all designed and assembled by my good friend Charles.
The videos are projected by a Python script running on a Raspberry Pi 5 using OpenGL to warp and move the videos during playback.
Control is via an Akai LPD8 allowing for video clip selection and tuning of the projection mapping parameters.
It has an animation capture mode for recording the eye movement via mouse to be used during playback of each video.

It worked really well but there are definitely some things that could be improved:-

Improve the playback performance by moving the simple OpenGL commands into shaders.
Add sensors to allow the eye to look and react to people around it.
Build a bigger one with more projectors for full coverage.
Build several and have them networked so that they can coordinate when interacting with people.

I'll also get more video of it in action during the further development process, but here are a few clips of it in situ at Hat Village!

Project: Boiler Automation

2018-02-19T00:00:00+00:00

Building a smart central heating and hot water controller with an Arduino, a Pi Zero W and Home Assistant.

The Brief

My study/workspace is in the basement, which can get cold. The central heating button is two floors up at the opposite end of the house. I wanted a cheap, non-invasive device to press the button via a friendly web UI!

The Plan

Actually use one of the many Arduinos I have lying around (a 5V Pro Mini in this case as I was going to add an external regulator).
Press the central heating and hot water buttons with using servos, with the 'fingers' resting in a retracted position so the buttons can still be reached.
Read the central heating and hot water 'on' lights so it can report the status if changed manually.
Read the temperature and humidity of the airing cupboard the boiler controller is in (as I also have plenty of DHT11s lying around).
Communicate with the Arduino using nRF24L01+ 2.4GHz transceiver boards using RF24Mesh, as I was interested in adding other devices to a house mesh network.
Set up Home Assistant as the friendly web UI and write a device plugin for it, running on my Kickstarter-backed UP² home server (yes, I backed a hardware Kickstarter and it actually made it to production).

What went right

To try new tools, I used PlatformIO on VSCode on macOS, which is an excellent setup for development on multiple connected Arduinos.
Controlling the servos using the Arduino PWMs was a doodle, as was reading the lights with LDRs.
Making servo/LDR mounts and button pressing 'fingers' is also a doodle if you have a 3D printer.
Added a beefy 5V regulator to power the Arduino, servos and transceiver.
Managed to get the nRF24L01+ boards registering with RF24Mesh and communicating Arduino to Arduino.

What went wrong

Initially couldn't get the nRF24L01+ boards to work at all, even with the recommended passive components on the supply, and thought I'd managed to pick up a batch of dodgy copies that had the ACK bit flipped. Bought a power breakout adapter for my breakout adapter that came with its own 3.3V regulator and the passive components on a PCB which seemed to solve the issue.
With the nRF24L01+ boards communicating Arduino to Arduino, the boards could not reach the distance required, even with the transmission power at maximum and the bitrate reduced to 250kpbs. Replacing them with nRF24L01+PA+LNA boards (that were thankfully pin-compatible with the power breakout boards) solved the range problem.
RF24Mesh supports running the master node on Raspberry Pi and Linux with a number of IO drivers, so I thought getting it running on the UP² would be simple as it is Pi GPIO compatible. Not so much. Submitted a pull request to get it building correctly with MRAA, but even then it wouldn't remain connected for more than 30 minutes without sporadic GPIO timeouts. Gave up.
As I'd got the Arduino to Arduino communication working, thought the I could attach the nRF24L01+PA+LNA boards to an Arduino set to behave as an I²C node to the UP². Built a 3.3V to 5V level-switching board with pull-ups for the I²C bus between the Pi and Arduino, not strictly needed as I²C pins are high impedance, but didn't want to risk blowing the Pi up if I accidentally set the pins wrong. Tested the I²C communications with a PCF8574P (again, lying around) then implemented the Arduino as an I²C node. Worked. Attached the nRF24L01+PA+LNA. Still unreliable. Gave up.
Was so close to completion but couldn't determine whether the 2.4GHz communication problems were the beta-ness of the mesh network library, electrical noise from the boiler or servos, being overpowered by other local WiFi hubs, dodgy Pi GPIO compatibility on the UP², my ineptitude or a combination of them all.

What went right (finally)

Replaced the nRF24L01+PA+LNA with a Pi Zero W, as I have perfectly good WiFi in my house, communicating with the Arduino via I²C (phew, the code and Pi-to-Arduino board weren't wasted).
Found out you can power the Pi via the GPIO 5V pins, admittedly bypassing the fuses, so could run it off the existing 5V regulator.
Wrote a simple web API for the Arduino with Flask, which I ran as a single-threaded single process so I didn't have to think about shared resource management and locking access to the I²C bus BECAUSE I JUST WANTED IT FINISHED.
Installing Home Assistant on the UP² and writing a switch-derived custom plugin was easy. It has good documentation and being on GitHub you can read the code of the existing plugins.
Configured Home Assistant security and port-forwarded 443 (SSL with Let's Encrypt of course) to the UP² so I can control it on the go!
Even with the unused transceivers, the whole device cost about £20.

Conclusion

It was a fun and practical project that has made my life better (lazier) and even my other half thinks is incredibly useful. So glad to finally find a use for a parts I'd bought thinking 'they would be useful'. Spent way too long with the 2.4GHz transceivers, just because I liked the thought of a house mesh network. Will try them again for future outdoor projects and use a programmable WiFi chip (like an ESP8266) for indoors.

What's left

It's been working well for a while now but there are a few things that can be done:-

tidy and publish the Arduino, Flask and Home Assistant plugin code.
make the cables a bit less chaotic looking.
replace the gaffer tape with a better way to mount (and dismount) the board.
replace the cable ties fixing the controller circuit and Pi to the board with something less dangly.
move the DHT11 so it isn't just taking the temperature of the 5V regulator.
disable the boiler timer and control the schedule with a Home Assistant plugin.
add wireless thermometers around the house to Home Assistant so I can tune the radiator thermostats and possibly get Home Assistant to control the heating to maintain a set temperature.

envassume

2017-05-14T00:00:00+01:00

Assume an AWS IAM role from AWS API credentials in environment variables.

I like to use envdir when selecting AWS API keys for CLI usage. Assuming IAM roles is awkward on the CLI as you have to get the temporary role credentials via aws sts assume-role and manually extract them to an envdir or environment variables prior to running anything. I wrote envassume to give an envdir-like experience when assuming roles. It uses the current credentials to perform the STS request then extracts and replaces the environment variables with the temporary credentials for the child command e.g.

envassume arn:aws:iam::123456789012:role/example aws s3 ls

with an external ID e.g.

envassume -i external_id arn:aws:iam::123456789012:role/example aws s3 ls

The ARN and external ID can be set in the environment variables AWS_ASSUME_ROLE and AWS_ASSUME_ID e.g.

AWS_ASSUME_ROLE=arn:aws:iam::123456789012:role/example AWS_ASSUME_ID=external_id envassume aws s3 ls

Please note, if the ARN is set by environment variable, then no other envassume options can be set.

envassume is available on pypi and github.

3D Printer 2017 Plans

2017-01-02T00:00:00+00:00

I got quite into 3D printing in 2016 after fixing up a borrowed Bits From Bytes printer, then going on to build my own Prusa i3 from scratch.

It's been an enjoyable project that has taken a few upgrades to improve print quality:-

A properly copper etched heated bed as I need >100C and my original could barely make it over 50C.
An E3D V6 as my cheap MK8 seemed to jam at the drop of a hat with ABS.
Updating X and Y axes with DRV8825 stepper drivers, to give me 1/32 stepping rather than 1/16 (and not because I blew the existing A4988s up).
Watching streaming video of a print from work via Raspberry Pi running OctoPrint on Docker.
Watching the stepper motor temperatures on the Raspberry Pi via multiple DS18B20 one-wire thermometers.

There's plenty to do to improve things even more. My list for 2017 so far:-

Fit the printer inside the enclosure I've built.

Fit a heat lamp in the enclosure so I can control the air temperature to improve layer adhesion of my ABS prints.
Fit and test the inductive sensor so I no longer have to bother with Z calibration!

Control the hot end fan and controller fan from the Raspberry Pi so the printer is automatically quiet when not printing.
Control some RGB LEDs from the Raspberry Pi to improve the time-lapse video capture.
Add a second E3D V6 so I can print with a water-soluble support material.

Continuous Lifecycle London

2016-05-31T00:00:00+01:00

As I’ve just updated my Pelican relapse theme, thought I’d better post something.

I was lucky enough to attend the Continuous Lifecycle London conference recently, a great mix of talks on the culture, practices and technology of devops and continuous delivery at scale, all of which I find fascinating. I’ve always been lucky enough to work across the entire process of designing, implementing and deploying interesting software and hardware products. Working on a cloud-native software-as-a-service platform, it’s great to be able to use good software practices to codify, version control, test and deploy everything; software, infrastructure, network, security and even compliance. A long way from burning custom DVDs, cabling racks or flashing firmware!

The points that seemed to resonate most for me were:-

the more you can automate and the faster you can deploy, the more reliable your application can become and the quicker you can respond to feedback or issues.
if part of your deployment process is painful, do it much more often and automate it.
automating your entire deployment pipeline can be hard and laborious, but given how much value it can bring it is worth the investment.

While at SpatialBuzz we already have an effective release process and automated deployment (our current record is eleven deployments to production in one day) there is always more to automate and optimise. As we work towards containerising our application and moving to an orchestrated container cluster, it really is an exciting time to be in software development (something I’m happy to still be saying after all these years).

As I work my way through Continuous Delivery by Jez Humble and Dave Farley, I’ll leave you with the video of the conference keynote.

Keepuppy

2014-11-06T00:00:00+00:00

KeePass database SFTP sync

Keepuppy is a Python script/package I hacked together to keep database files in sync across multiple clients via an SFTP server. It can also be configured to call a script when the local file is updated. A script for restarting KeePassX on OS X is provided.

It's also an experiment in:-

Robust testing with nose and mock.
Configuration via envdir.
Version management using bumpversion.
Continuous integration via github and travis-ci.
Python packaging for PyPi.
Badge-driven development :)

Launchd script

To run it periodically on OS X, here is a sample launchd script. For more information on periodic job management with launchd check out this detailed article.

This script will run keepuppy_sync.py, passing configuration environment variables stored in ~/envdir/keepuppy via envdir, all from a virtualenv called keepuppy. The job runs every 5 minutes and outputs to log files at /tmp/keepuppy.log and /tmp/keepuppy_error.log.

You will need to change the program arguments to match your user, envdir and virtualenv paths.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.wamonite.keepuppy</string>

  <key>ProgramArguments</key>
  <array>
    <string>/Users/warren/.virtualenvs/keepuppy/bin/envdir</string>
    <string>/Users/warren/envdir/keepuppy/</string>
    <string>/Users/warren/.virtualenvs/keepuppy/bin/keepuppy_sync.py</string>
  </array>

  <key>Nice</key>
  <integer>1</integer>

  <key>StartInterval</key>
  <integer>360</integer>

  <key>RunAtLoad</key>
  <true/>

  <key>StandardOutPath</key>
  <string>/tmp/keepuppy.log</string>

  <key>StandardErrorPath</key>
  <string>/tmp/keepuppy_error.log</string>
</dict>
</plist>

To run it as the desktop user when logged in, copy it to ~/Library/LaunchAgents/com.wamonite.keepuppy.plist. To start it and ensure it runs on reboot, use the following command:-

launchctl load ~/Library/LaunchAgents/com.wamonite.keepuppy.plist

IP lookup for Vagrant private networking

2014-07-12T00:00:00+01:00

I haven't posted in ages and given I'm doing so much interesting stuff with SpatialBuzz these days, I felt I needed to post something, however short but sweet.

I spin up a lot of dev instances with Vagrant and VirtualBox on OS X these days and needed them to be host addressable on fixed IPs. I also wanted a single source for managing said IPs. After trying several Vagrant plugins, playing with DHCP and tweaking pf rules, the easiest solution I found was a local DNS server and getting the Vagrantfile to look up its own IP.

30-second guide to local DNS on OS X¹

Install dnsmasq via Homebrew
Add your desired homename and IPs to the end of the default config file /usr/local/etc/dnsmasq.conf in the format address=/<host name>/<IP address> e.g.:-
```
address=/db.vagrant.dev/192.168.2.10
address=/web.vagrant.dev/192.168.2.11
```
Start your local DNS server with the following command (and stop it with unload):-
```
sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
```
Tell OS X to use it for a domain by creating the file /etc/resolver/<domain> e.g. /etc/resolver/vagrant.dev with the contents:-
```
nameserver 127.0.0.1
```
Check the resolver is listed with:-
```
scutil --dns
```
Query it directly with:-
```
dig db.vagrant.dev @127.0.0.1
```

Vagrantfile

Here's a basic Vagrantfile to configure a single instance with the host-only network address read from DNS. As the host name is specified as a parameter, this works for multi-machine setups too. The VM is both addressable via vagrant ssh and SSH directly to the host name.

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

# Set default to virtualbox unless otherwise specified
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'virtualbox'

# hostname.wamonite.com: private IP address read from local DNS
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    config.vm.box = "hashicorp/precise64"

    # Set the hostname
    config.vm.hostname = "web.vagrant.dev"

    # Read the IP address from local DNS and set as private network IP
    ip = Socket::getaddrinfo(config.vm.hostname, 'http', nil, Socket::SOCK_STREAM)[0][3]
    config.vm.network "private_network", ip: ip
end

Guest VM lookups

On VirtualBox the DNS server is available on the .1 address of the specified class C network, so VMs can address each other if the name resolution for the guest OS is configured appropriately. In the above example, for a Linux host /etc/resolv.conf would need to contain:-

nameserver 192.168.2.1
domain vagrant.dev

Another guide with detailed commands can be found at 'Never touch your local /etc/hosts file in OS X again' ↩

Fixed: Automounted DVDs on openSUSE 11.4

2013-01-10T00:00:00+00:00

I had an issue with automounted DVDs on openSUSE 11.4 not appearing in Gnome desktop. It seems that udisks honours UDF-mounted directory permissions even if they aren't readable. ¹

The issue is fixed in udisks 1.0.3 ² however openSUSE 11.4 stopped updating udisks at 1.0.2. As the patch ³ was minimal, I decided to apply it to the contents of the openSUSE source RPM, increment the RPM spec build number and build a new RPM.

I'll put it up for download here (unless I find time to put it somewhere more official like the openSUSE build server).

Download udisks-1.0.2-3.6.2.i586.rpm

To allow the DVD to be mounted without requiring root authentication I had to get handy with polkit. I added the following 10-storage-group-user-override.pkla file to /var/lib/polkit-1/localauthority/50-local.d/ (you may have to create the 50-local.d directory).

[Access to removable media for users]
Identity=unix-group:users
Action=org.freedesktop.udisks.drive-eject;org.freedesktop.udisks.filesystem-mount
ResultAny=yes
ResultInactive=yes
ResultActive=yes

Download 10-storage-group-user-override.pkla

A bug report and patch can be found on the Ubuntu issue tracker #635499. ↩
As listed in the release notes ↩
You can view the patch here ↩

Fixed: OSX Plex Media Server LAN/Wi-Fi sharing

2012-10-29T00:00:00+00:00

I'm sharing this as it caught me out for a few days and the solution wasn't obvious. I run Plex to serve media around the home to my TV and few LAN and Wi-Fi clients. The server runs on a Mac Mini and I keep up with updates. The client running on the server was fine but suddently LAN and Wi-Fi clients could no longer connect, even when manually specifying the server IP. Server logs didn't show any errors but it did look like the clients were no longer managing to connect or query. It seemed that going to http://server-ip-address:32400 should at least show something in a browser or the logs.

I wasn't using myPlex and it didn't appear as if they were forcing it in a recent release. I was eventually tipped off by this comment in a server release changelog:-

(Also, we've updated the Plex for OS X download to fix the code signing issue that was causing the violent disagreement between it and the OS X firewall. Sorry about that!)

I deleted the entries for 'Plex DLNA Server' and 'Plex Media Server.app' from System Preferences -> Security & Privacy -> Firewall -> Firewall Options, restarted the media server and clicked 'allow incoming connections' again for each. As if by magic all my clients could connect again!

Zypper support for Cuisine

2012-10-15T00:00:00+01:00

Fabric is a great Python module for deploying and running software on remote SSH-enabled machines without the need for an array of terminal windows or mad shell-scripting skills.

Cuisine is a Python module adding high-level administrative functions to Fabric (similar to Chef for Ruby).

I'd been using Fabric for a while but wanted to use Cuisine to automate production and build system configuration, but was out of luck with package management dependencies. We use openSUSE which uses Zypper while Cuisine only supported 'apt' and 'yum'. It was too useful not to have, so by the power of open source I remedied that.

Here's an example fabfile.py to ensure FFmpeg is installed from the Packman repository:-

from cuisine import select_package, package_ensure, repository_ensure_zypper

select_package("zypper")

def installFFmpeg():
  repository_ensure_zypper("http://packman.inode.at/suse/openSUSE_11.4/packman.repo")
  package_ensure("ffmpeg")

This could be executed repeatedly without ill effect on the remote hosts peppa and yaffle with the following call:-

fab -u root -H peppa,yaffle installFFmpeg

My Markdown Essentials

2012-10-08T00:00:00+01:00

From the Markdown website:-

Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML).

I've taken to using it wherever I can, including content generation for this site, with these useful set of tools:-

Marked: markdown preview application for OSX.
nvAlt: used in conjunction with Marked for super-charged note taking.
Markdown editor plugin for Eclipse.

Relapse: a Pelican theme using Bootstrap

2012-10-07T00:00:00+01:00

New website, new look and first bit of HTML/CSS design in many years. In the spirit of open source you can find this Pelican theme on Github.