johnmu.com - technical website tips

Automate IndexNow SEO submissions on static sites with Github Actions

Sat, 13 Sep 2025 00:00:00 +0000

Let’s connect a static site generator, like Hugo, to IndexNow’s search engine submissions, using automatic Github Actions whenever you make changes. Will this change how your site appears in all search engines? Will your site’s SEO suddenly explode? No. Also No. I’m just here because I wanted to see if it was possible. And tl;dr: yes, you can. (Well, technically it doesn’t just send the last updates, but read on.)

Embedding robots.txt in WAV files

Fri, 24 Jan 2025 00:00:00 +0000

tl;dr: you can embed a robots.txt file into an audio file, and it can be both a valid robots.txt and a valid audio file.

The robots.txt standard has a nifty feature: it will ignore anything that it doesn’t understand. Random text, ASCII art, and even other data can be added without making the robots.txt file invalid.

You could do something simple.

Roses are red
Violets are blue
user-agent: Googlebot
disallow: /foo

Or do something weird.

Checking for lost URLs after a CMS migration

Wed, 22 Jan 2025 00:00:00 +0000

Sometimes you might find, that after a CMS migration, some things are missing. Obviously nobody would do such a thing on purpose, right? But it can happen. Suddenly, instead of something complete, you’re left with just guitars.

Let’s do a quick & dirty hack to check for important, lost URLs.

curl "http://web.archive.org/cdx/search/cdx?url=colored.house.com*&output=txt&from=20241201&to=20241231" --output - \
| grep " text/html 200 " | awk '{print $3}' | sed 's/\?.*//' | sort | uniq -c | sort -nr \
| awk '{print $2}' | head -n 50 | xargs -I {} curl -s -o /dev/null -w "%{http_code} %{url_effective}\\n" {} \
| grep -v "^[23]"

There you are, the top “important” URLs from the site that worked in December, but which don’t work now (neither redirecting nor returning content). A serious site owner could then check the URLs for things that they either want to maintain (perhaps historical content), if there are new URLs to replace (perhaps one department is now replaced with another), or maybe they just want to keep things 404. SEOs might want to maintain the value of some of the old URLs, even if the replacements are just kinda close.

Updating partition tables via OTA on ESP32 for WLED 0.15.x

Sat, 28 Dec 2024 00:00:00 +0000

WLED 0.15.x needs a bit more app space than some ancient WLED installation. Usually updating the partition table requires a local USB connection. This is an app that you can install via OTA which expands app partitions to 1536KB.

The general approach to creating partitions on an ESP32 device involve connecting it via USB and flashing it with the IDF library (using gen_esp32part.py to create a binary image of the partition table, and idf.py to flash the device - see the ESP docs. …Or using platformio or similar to do it all together). Changing partition tables without having a device locally connected is not really documented. But you can do it.

Midjourney / Discord to Lightroom automation - step by step - for 2023

Sun, 30 Apr 2023 00:00:00 +0000

Midjourney is fun, but the Discord-based interface is somewhat annoying, if you want to make a lot of images. To get all the images you generate, you have to upscale all 2x2 previews, and save them manually. It can get pretty tiring, so instead of making better images, I spent a bit of time to automate the download and import process. With this code, a Discord bot will monitor your server, watch for Midjourney messages, process all images, and upload them to your Lightroom CC cloud account. Run the script anywhere, a Raspberry Pi or a cheap cloud server, and keep making images.

Mastodon - be findable with your domain (on Firebase hosting)

Sat, 11 Feb 2023 00:00:00 +0000

While it’s cool to host your own Mastodon server, it’s not really efficient. A simple thing you can do, however, is to make it easier for others to discover your presence, if they already know your website. Try it out – search for “johnmu@johnmu.com” in your Mastodon instance. Also, click “follow” for SEO tips & other bad takes.

Why is hosting your own instance inefficient? In short, sending updates between users (followers, followees) means connections between their individual servers, and if everyone has their own server, it’s a lot of network activity. It costs you money, it costs them money. It’s better if you share instances (and support your server).

Static, privacy-preserving embedding of Mastodon posts

Sat, 05 Nov 2022 00:00:00 +0000

Now that Mastodon is more of a thing, I thought I’d set up a static embed of Mastadon posts within Hugo, similar to what I did for Twitter posts.

With the static embedding, I wanted to remove all requirements for JavaScript, and remove all third-party requests on behalf of the users here. In short, this speeds things up, and saves you from worrying about cookies & tracking, or needing to make exceptions to the site’s CSP.

ESP8266 wifi connection speed optimization

Sun, 30 Oct 2022 00:00:00 +0000

It’s been a while since I initially looked into what it takes to make an optimal wifi connection with an Espressif ESP8266 / ESP-01 (old post). Using the code from the old post didn’t work … anymore? What the heck. So much for rolling out another kind of battery-powered ESP8266 device. Time to dig into what goes on with the wifi connection timing.

The code is on Github. Most of this post is from the readme file there. Which will rank better in Google, a subdirectory of a well-known domain, or my blog post? Whatever, it’s all the same to me :-).

Using R for people who don't use R

Sun, 30 Jan 2022 00:00:00 +0000

I wanted to do a short post on how to do something with the Search Console API with R. Backing up a bit, I thought I’d include a short summary of how to get started with R, and as you do, I’m now writing a separate post on how to do that. No, I won’t back up more and explain computers or how to connect your printer.

Setup

For today’s excursion we’ll use the programming language “R”. Why do they call it “R”? What happened to all the languages between “C” and “R”? Moving on. But why R? You could do the same things with Python, or most programming languages, but Python is for hipsters, and R is for real data scientists.

Arduino binary USB keyboard

Sat, 14 Aug 2021 00:00:00 +0000

I don’t know many (or any) people who think of characters and words in binary, but that might just be a limitation on my end. If you prefer binary, maybe this is something for you. If you just want to geek out, you’re welcome to stay here too.

Using a 2-button macro-keyboard and some Arduino code, you can tap in binary and have the keyboard enter ASCII characters.

View on YouTube

Standalone MozCast weather report display with ESP32

Tue, 20 Jul 2021 00:00:00 +0000

I have -too many, err- some extra ESP32 boards so I thought it would be fun to make a small weather display for the SEO weather. The obvious choice is to use MozCast. This is Moz’s unique interpretation of how Google’s search results change. This doesn’t reflect what Google does in its algorithms, but it’s a number for the weather, so why not.

It’s hopefully obvious that this does not represent a recommendation of any particular SEO tool. To be honest, I’m not a fan of any SEO weather reports, in particular because of the way the data is collected. The folks from Moz have been smart & friendly in the past, so I hope they’re OK with this hack. (I’m happy to send someone from Moz one of these, if you drop me a note.)

A collection of SMD switches

Fri, 11 Jun 2021 00:00:00 +0000

Who doesn’t love a good button? Click click click, so satisfying.

In the search for SMD switches / buttons, I thought I’d just buy some of “all” and try them out. I realize there are many more, this is just what I found browsing Aliexpress.

This doc is mostly for me, but I’ll probably spent way too much time on it, so I might as well put it up.

Criteria (more or less)

Optimized routing for Zwift's Makuri

Sun, 30 May 2021 00:00:00 +0000

Zwift recently released a new set of routes on a made-up island called Makuri. Different from some of the other maps, this one has a ton of cross-roads, and there’s a lot to see. It’s pretty. So, of course, instead of cycling there, I thought it would be a fun exercise to find the shortest path that lets you see everything at least once.

Without further ado, here’s the optimal path:

Boot, connect to wifi, server requests in under 1 second with ESP-01

Sat, 22 May 2021 00:00:00 +0000

Making a hardware button that connects to wifi and sends a request off to a server is possible using Arduino. Light switches are (almost) immediate – but simple wifi switches easily take 8-10 seconds to connect & switch. How do you get that time down? Here’s my approach.

Measure end-to-end - what’s the default time? Is it really that bad, or just sometimes bad?
Split into parts
Measure the parts
Determine which parts should be optimized, and try options
Combine the best options
Debug why it doesn’t work and try again

The computer we’re using to run on is an ESP8266, model ESP-01. It’s tiny, cheap (< 1 USD), quite limited, and weirdly popular. Wifi buttons are popular projects, so I’m sure all of this isn’t new (it’s just new to me, so I was happy when this all worked out). I use MQTT since it’s one of the common setups for home automation. All measurements were done 10 times to get a sense of how stable they are.