as days pass by: a weblog by Stuart Langridge

Calculating a rolling average without keeping all the numbers around

sil — Sat, 14 Mar 2026 23:23:00 +0000

OK, if you know anything about maths you will read this and shake your head in dismay because it's just obvious and why did I even need to think about it? But... I was quite pleased to think this through, so I thought I'd write it down in case anybody else finds it useful.

So. Your job is to calculate the average of a set of numbers. This is not hard: the average value¹ is the sum of the numbers divided by the count of the numbers. So the average of 2, 7, 8 is

2+7+8   17
----- = -- = 5.666…
  3      3

You know that bit already. But now I say: ok, here's a new number to add to the end of the list: 12. What's the average now?

One obvious way is to say, well, we know the numbers are 2, 7, 8, so I'll just re-do the calculation with 2, 7, 8, 12:

2+7+8+12   29
-------- = -- = 7.25
    4       4

But that means that you have to keep all the numbers around forever. What if the list of numbers is big? That seems like hassle. It would be nice if we were able to calculate the average just from knowing what the previous average was, wouldn't it? So you know 5.666… and you get told that the new number to add is 12: how do you get the new average of 7.25 just from that?

I needed to do this and didn't know how, so I sat and thought about it for a short time and it's actually not hard, which was good; I'd just never considered it. So, as long as you know the "current" average number, and you know how many numbers there were in the list, you're all good. Consider: we know the current average is 5.666… and that there were 3 numbers in the list that made that up. We don't need to know what the 3 numbers were (2, 7, 8) because if the three numbers we'd been given before were all 5.666… -- that is, the list of numbers up to now was 5.666…, 5.666…, 5.666… then when we took the average of that list, it would still be 5.666…, because that's what an average means. So, even though we don't know what the 3 numbers were, we can pretend that we do, and now say that our new list is 5.666…, 5.666…, 5.666…, 12. And we can work out the average of that! Or, put another way, we know that the average is

         sum of all numbers
5.666… = ------------------
          count of numbers

and because we can pretend that all the numbers were the average, we can say that the sum of all numbers is just the current average multiplied by the count:

         5.666…, 5.666…, "count" times
5.666… = -----------------------------
                   count

and so the new average is:

              (5.666…, 5.666…, "count" times) + 12
new average = -----------------------------
                       count + 1

which is

              (current average × count) + new number
new average = --------------------------------------
                          count + 1

So if you know the current average a, how many numbers there have been so far c, and the new number n, then the new a = ((a*c)+n)/(c+1). And you only need to keep a and c around, rather than the whole list of numbers.

Simple? Yes. But it was a useful thought for me that I could actually do this without having to keep the list around, which was handy in order to be able to calculate pi very inefficiently with coin flips on a BBC Micro, a perfectly reasonable thing to do on your Saturday evening in my opinion.

maths people, I mean mean, obvs ↩

Using 11ty's WebC to separate web components out into separate files

sil — Mon, 02 Mar 2026 05:22:00 +0000

For a thing I'm building I'm using web components to give me a convenient way to bundle up functionality of different bits of a web page into separate units. I'm not doing anything particularly clever here with web components; there's no shadow DOM here, the components aren't redistributable or particularly reusable anywhere else, and aren't required. They're simply a useful way to group "what happens when you click on this thing" into its own little section of code. Anyway, what I had was one big page with the styles for the components and the scripting for them all intertwined.

A simple example

Here's a simpler example. In this stripped down "page", you've got a <light-bulb> component which either displays a little coloured dot and "on", or a black dot and "off". There are two of them, a green one and a yellow one. Then there's a <light-bulb-switch> component with three positions: green one on, both off, yellow one on. These aren't intended to be particularly good examples of web components (and you don't need to read or understand¹ the code in any detail), but they demonstrate the point.

See the Pen Untitled by Stuart Langridge (@stuartlangridge) on CodePen.

The HTML there is all one page; CSS and JS to define and style the components all inline. What I wanted was the output to be that, but to be able to split up the components into their own separate files, for ease of editing -- it's hard to keep track of where you are in a big long page. This isn't any sort of improvement to the user experience -- the long page with everything inline was exactly what I intend to serve to the user. This is purely a developer experience improvement.² I could have put the web component parts into their own CSS and JS files, and then added <script> and <link rel=stylesheet> lines for each, of course, and had the browser load them… but I wanted to keep all the code for each component together to make it easy for me to think about, but not have it all in the main file which was too confusing to navigate around in while editing it. Clearly there ought to be something which can do the purely mechanical task of "break this big file up into bits and then glue them back together in a build step of some kind". Helper libraries for web components tend to think of themselves as doing a lot of stuff to aid you in how you do that, either at build time or at run time; stuff like Lit is pretty neat but it does six billion things I don't want, and the client-side half of this equation gives you React and all its cousins and I didn't need or want any of that either. All I needed was something to break a page apart into separate jigsaw pieces and mechanically glue them back together. I could probably write this myself in about twelve lines of python; it's not hard.

WebC

But then I was reminded by a search for "distributing web components in a single file" that WebC exists as part of the 11ty project. And… it sorta looks like it does this? But it's not all that clear; WebC isn't brilliantly documented. (It's fairly clear what the API is, at a technical level, but there's not very much about the vibe: what's WebC for, rather than how it does it.) I am a very big believer indeed in the notion of using software in accordance with its vibe; it's possible (and sometimes quite fun) to contort a bit of software into doing things that its authors don't think that it's for, and would perhaps be horrified to see it used for, but that's for hack value projects. When using a thing as a tool, use it the way it's meant to be used. If you swim against the tide on this, it will at some point bite you in the arse³ and then you'll be sad. So, sure, WebC looks like it could do this mechanical reassemble-the-jigsaw task; is that what it's for? I needed support.

So I got support for WebC in the most scalable and efficient way possible: chat to Zach Leatherman in the pub directly and ask him, after State of the Browser 2026. Isn't this how everyone gets support for everything, just talk directly to the author over a pint?⁴ And Zach said: yup, that's what it's for. I promised to try it, and maybe write something down about it so that future people might not be as confused as I was, and now you're reading that very thing.

(I'm not sure this is useful documentation. But I'm basically incapable of describing how to do anything without turning it into a short story. Insert that Mark Twain(?) quotation about writing a shorter letter here.)

So I took my page above and broke it up into separate files.

shell.html is the "main page". The component names stay in place, but all their inner code is removed, and so are their styles and scripting:

<!doctype html>
<html>
<head>
<title>Combined page</title>
</head>
<body>
<light-bulb data-color="orange">
</light-bulb>
<light-bulb data-color="green">
</light-bulb>
<light-bulb-switch>
</light-bulb-switch>
</body>
</html>

And then each of the components gets put into a separate file. This file contains the HTML for the component, including a <style> and <script> element for its CSS and JS. Here's (a cut-down version of) light-bulb-switch.webc:

    <input type="range" min="0" 
            max="2" value="1" list="bulbs">
    <datalist id="bulbs">
        <option value="0">orange</option>
        <option value="1">neither</option>
        <option value="2">green</option>
    </datalist>

<style>
light-bulb-switch {
    […css snipped…]
}
</style>

<script>
window.customElements.define('light-bulb-switch', class extends HTMLElement {
    […js snipped…]
});

</script>

And there's a similar light-bulb.webc as well. (You can find all the code for this little demo, such as it is, at github.)

Now this separation is done, we need WebC to glue it all back together. And this is a matter of a short bit of scripting:

import { WebC } from "@11ty/webc";
import { writeFileSync } from "node:fs";

// create WebC object
let page = new WebC();

// tell it where our components are
page.defineComponents("*.webc");
// have it collect style and script 
// elements all together
page.setBundlerMode(true);
// this is the page to read and 
// populate with components
page.setInputPath("shell.html");

// actually do the work
let {
    html, css, js, components
} = await page.compile();

// haha who needs webpack? not me!
// just stick the js and css inline
const out_js = `<script>
${js.join("\n")}
</script>`
const out_css = `<style>
${css.join("\n")}
</style>`
const out = html
    .replace("</body>", 
        out_js + "</body>")
    .replace("</head>", 
        out_css + "</head>");

writeFileSync("combined.html", out);

WebC reads in our shell.html and all the component definitions in their own files, and gives back HTML, CSS, and JS; the HTML is a long string, which we can write to a file. The CSS and JS are both arrays of strings, each one being the code in a <style> or <script> element in a web component (and all together because we turned on bundlerMode). Since we still need those styles and scripts in the output page somewhere, we bodge them into place by dropping a new <style> element into the page <head> and similarly for script and body. It's certainly possible to be a lot cleverer than this (and if you wanted to put the scripts in one separate file here and minify it on the way, this is where you'd do it) but for our example here, I don't want any of that cleverness.

And that's all. WebC does quite a lot more than this, but I don't need any of that at the moment. If you do, check out the WebC docs.

or critique! ↩
Obviously there are many things that I could and will be doing to improve the user experience of this whole thing! But that's not what this particular writeup is about. ↩
can tides bite arses? we are in mixed metaphor city. Sorry, Uncle George. Jackboots in melting pots all over the place here ↩
OK, tongue-in-cheek yes, but honestly this is quite a good way to find out about a piece of software! go to conferences and chat to the author. This isn't going to work if what you want help with is, like, Photoshop or something, but most people in the web community at least, and open source projects more generally, are in my experience not just willing but positively eager to talk to you about how they feel about a project they've built ↩

fifty

sil — Thu, 29 Jan 2026 23:19:00 +0000

fifty.

that can't be right.¹ fifty years ago was the war.

no big long writeup this time of musings on a year passing like I normally do. I think I did most of that earlier in the month. ↩

Flipping the cube: 2025 is over

sil — Sun, 04 Jan 2026 16:21:00 +0000

You know that optical illusion with a wireframe cube? Where you can't tell which way around it is, but if you concentrate a bit you can make the cube "flip" from being oriented one way to being oriented another?

So you look at that cube (it's called a Necker cube when you're viewing it as an optical illusion) and with a bit of concentration you can change how you perceive it, from pointing in one direction to pointing in another.

A bit like the dancer who might be spinning one way or the other, or the picture that's a duck or a rabbit. The key point here is: it's either one way, or it's the other. You can switch between ways, but it takes a bit of mental effort. And it can't be both at once.

This is how I feel about 2025. Some stuff happened, some of it amazing, and I am glad the year is over.

There are, I think, four major separate stuffs on that list. Obviously, what actually occupies your mind day to day is the more minor things; you got bolognaise sauce on your good shirt (did that) or your job changed a bit (did that) or you really need to lose some weight or you're nearly fifty or they're still not collecting the bins. But I'll not bore you to tears with all of that; only my head should be pecked by this sort of thing, not everybody's.

Last year, on my birthday, I wrote "Maybe 2025 will be better, although to be honest I doubt it." Oh, sweet summer child. If you only knew.

In March, my dad died.

This, for those of you who do not know, sucks. A very large amount.

It wasn't unexpected. He'd been unwell for a while, and getting worse. And he knew what was happening, and wasn't suffering much, and was content. It could have been considerably worse on that front than it was. We might consider that sort of thing tactical unhappiness: short term, dominates your thinking right now, but you need to reach beyond it. The strategic unhappiness, that's what he could see ahead for himself, and he didn't want it, and I think he was right in that and I'm glad that's what he chose, and I'm glad that he chose it. We tried to express that at the funeral and the wake with eulogies and readings and hugs and memories, and that's where I said everything that I wanted to say, and that went OK -- well, as OK as you can expect for your dad's funeral. Mardi Gras it was not. But the old man left a handprint on the world that made it better, and that's most of what you can hope for.

It's hard with him gone, though. You play all these moody film moments in your head with yourself in the starring role, of monumental grief that takes people's breath away, of sombre melancholy where you're nonetheless handsomely well-dressed in a rainy black-and-white street scene and people whisper under their breath how brave and noble you are for handling it so well. You imagine how you'll be when the next one dies. What you should have done instead. Or, maybe you don't do that at all, which is probably a better and more caring way to be. The things people warn you about -- how you'll think, oh that's cool I need to ask dad about that, and then remember that ah, no, I can't -- that goes away pretty fast. The needle changes direction from sadness to wistfulness, loss rather than bitterness; the thing's still there but the opacity of it drops gradually over time, even though it'll never fully go away. Not entirely.

It's harder on my mum. They were married for over fifty years, for goodness's sake. I only managed ten; I can hardly imagine what it's like for something that's been there for half a century to suddenly be gone. How do you even imagine that? It's been a rough year for her, it really has. Christmas, the first Christmas, was hard. I'm used to living alone; she is not, quite apart from the emotional turmoil. And that's where all the minor things come into play; something tiny that happens each day and feels like so much more a burden. You're supposed to not let the bastards grind you down, but the problem is that a lot of the time there aren't any bastards. There's just... life. Life's hard to deal with, when you're dealing with things.

So, was it all bad? No. There are some bright spots: some that are so great that they're almost blinding.

My daughter is engaged, to a decent guy, and they're pregnant. Next June, I'll be a grandfather.

This is cool.

I think I'm the first, among my friends. This isn't that surprising; I was the first to get married, too, we got started early. But I'm pretty excited about the idea of being a grandad. I still have to work stuff out -- do I need to keep Werther's Originals in my pocket all the time? Buy a flat cap? -- but man, this is going to be great. I liked my grandparents, even the ones I don't really remember; if I can be as good as they were, as good as my dad was for my daughter, I'll be happy with that. And we can read books together! Go look at cool stuff! Make dinners and giggle a lot! Maybe not for the first couple of years, admittedly, but as I understand it when the more, er, brown-stained bits of parenthood come around I get to just hand my grandchild back. I served my time on that sort of thing. I just get the fun bits. Hooray!

It did occur to me that there were times when my daughter was very young that we'd say, hey, we can drop her on the grandparents for a weekend and actually go to the pub again. And now I am the grandad being dropped upon... but I am OK with this. There are so many books we're going to share together. Bring it on. I'm really looking forward to it.

You see, it's not all bad. I have wonderful friends -- we almost all got together in what I am told is now called "Twixtmas" (and which I refuse to call "the merryneum", thank you Bruce), the bit in between Christmas and New Year, and I had such an excellent day. One of the highlights of my year, every year. Open Web Advocacy has had quite some successes this year. I got a neat new cooking pot for Christmas which I'm really looking forward to trying out.

But then there's the fourth thing: we've had death, engagement, grandfatherhood, and finally we have... the world. A world which is getting a little bit worse, every day. A little bit more mean, a little bit more cold, a little bit more gilded and segregated and exclusionary. My industry, my internet, my web, the thing I built (other people were also involved, I admit) is not what we all wanted it to be. Our place, our internet, is no longer ours; it belongs to the arseholes, to everyone who exchanges decency and joy for money and control and considers themselves superior and visionary for doing it. Plenty of people who actually are wise and eloquent have written recently about this, and how bad it is, and what can be done about it, or just how exhausting it is for it to happen: see Cory or Henry or Ethan or Salma or, or, or. But the world is worse, every day, and it makes me dread what's coming.

And so I confront the question: was 2025 a bad year with some good bits, or a good year with some bad bits?

I think that it's up to me. Remember the Necker cube, from one hundred years ago when I started writing this? How you get to choose, with a little bit of effort, which way it points? I reckon that seems right. I could see 2025 as having been a crap year with a couple of bright spots that were drowned by the misery, and that is in fact what I've mostly been doing. But I don't think it helps, and I don't think it's healthy. The world is increasingly shit, and it's considerably more shit for people who aren't as lucky as me, and it's that way because of people who want it that way, and there there really are bastards and I could let those bastards grind me down... but I don't wanna. I'm trying to make a conscious effort to flip the cube, to look forward to the good stuff that's coming and not look down at my feet to avoid the bad. I apologise in advance for all the days to come when I don't quite manage it.

So, that was 2025. Goodbye, dad. Hello, your great-grandchild, whoever they turn out to be. I hope it's just better and I don't need to try to perceive it that way, but... if it isn't, I'll try anyway.

Here's to 2026. Fingers crossed.

Making a Discord activity with PHP

sil — Tue, 08 Jul 2025 08:11:00 +0100

Another post in what is slowly becoming a series, after describing how to make a Discord bot with PHP; today we're looking at how to make a Discord activity the same way.

An activity is simpler than a bot; Discord activities are basically a web page which loads in an iframe, and can do what it likes in there. You're supposed to use them for games and the like, but I suspect that it might be useful to do quite a few bot-like tasks with activities instead; they take up more of your screen while you're using them, but it's much, much easier to create a user-friendly experience with an activity than it is with a bot. The user interface for bots tends to look a lot like the command line, which appeals to nerds, but having to type !mybot -opt 1 -opt 2 is incomprehensible gibberish to real people. Build a little web UI, you know it makes sense.

Anyway, I have not yet actually published one of these activities, and I suspect that there is a whole bunch of complexity around that which I'm not going to get into yet. So this will get you up and running with a Discord activity that you can test, yourself. Making it available to others is step 2: keep an eye out for a post on that.

There are lots of "frameworks" out there for building Discord activities, most of which are all about "use React!" and "have this complicated build environment!" and "deploy a node.js server!", when all you actually need is an SPA web page¹, a JS library, a small PHP file, and that's it. No build step required, no deploying a node.js server, just host it in any web space that does PHP (i.e., all of them). Keep it simple, folks. Much nicer.

Step 1: set up a Discord app

To have an activity, it's gotta be tied to a Discord app. Get one of these as follows:

Create an application at discord.com/developers/applications. Call it whatever you want
Copy the "Application ID" from "General Information" and make a secrets.php file; add the application ID as $clientid = "whatever";
In "OAuth2", "Reset Secret" under Client Secret and store it in secrets.php as $clientsecret
In "OAuth2", "Add Redirect": this URL doesn't get used but there has to be one, so fill it in as some URL you like (http://127.0.0.1 works fine)
Get the URL of your activity web app (let's say it's https://myserver/myapp/). Under URL Mappings, add myserver/myapp (no https://) as the Root Mapping. This tells Discord where your activity is
Under Settings, tick Enable Activities. (Also tick "iOS" and "Android" if you want it to work in the phone app)
Under Installation > Install Link, copy the Discord Provided Link. Open it in a browser. This will switch to the Discord desktop app. Add this app to the server of your choice (not to everywhere), and choose the server you want to add it to
In the Discord desktop client, click the Activities button (it looks like a playstation controller, at the end of the message entry textbox). Your app should now be in "Apps in this Server". Choose it and say Launch. Confirm that you're happy to trust it because you're running it for the first time

And this will then launch your activity in a window in your Discord app. It won't do anything yet because you haven't written it, but it's now loading.

Step 2: write an activity

You'll need the Discord Embedded SDK JS library. Go off to jsdelivr and see the URL it wants you to use (at time of writing this is https://cdn.jsdelivr.net/npm/@discord/embedded-app-sdk@2.0.0/+esm but check). Download this URL to get a JS file, which you should call discordsdk.js. (Note: do not link to this directly. Discord activities can't download external resources without some semi-complex setup. Just download the JS file)
Now write the home page for your app -- index.php is likely to be ideal for this, because you need the client ID that you put in secrets.php. A very basic one, which works out who the user is, looks something like this:

<html>
<body>
I am an activity! You are <output id="username">...?</output>
<scr ipt type="module">
import {DiscordSDK} from './discordsdk.js';
const clientid = '<?php echo $clientid; ?>';
async function setup() {
  const discordSdk = new DiscordSDK(clientid);
  // Wait for READY payload from the discord client
  await discordSdk.ready();
  // Pop open the OAuth permission modal and request for access to scopes listed in scope array below
  const {code} = await discordSdk.commands.authorize({
    client_id: clientid,
    response_type: 'code',
    state: '',
    prompt: 'none',
    scope: ['identify'],
  });
  const response = await fetch('/.proxy/token.php?code=' + code);
  const {access_token} = await response.json();
  const auth = await discordSdk.commands.authenticate({access_token});

  document.getElementById("username").textContent = auth.user.username;
  /* other properties you may find useful:
     server ID: discordSdk.guildId
     user ID: auth.user.id
     channel ID: discordSdk.channelId */
}
setup()

You will see that in the middle of this, we call token.php to get an access token from the code that discordSdk.commands.authorize gives you. While the URL is /.proxy/token.php, that's just a token.php file right next to index.php; the .proxy stuff is because Discord puts all your requests through their proxy, which is OK. So you need this file to exist. Following the Discord instructions for authenticating users with OAuth, it should look something like this:

<?php
require_once("secrets.php");

$postdata = http_build_query(
    array(
        "client_id" => $clientid,
        "client_secret" => $clientsecret,
        "grant_type" => "authorization_code",
        "code" => $_GET["code"]
    )
);

$opts = array('http' =>
    array(
        'method'  => 'POST',
        'header'  => [
            'Content-Type: application/x-www-form-urlencoded',
            'User-Agent: mybot/1.00'
        ],
        'content' => $postdata,
        'ignore_errors' => true
    )
);

$context  = stream_context_create($opts);

$result_json = file_get_contents('https://discord.com/api/oauth2/token', false, $context);
if ($result_json == FALSE) {
    echo json_encode(array("error"=>"no response"));
    die();
}

$result = json_decode($result_json, true);
if (!array_key_exists("access_token", $result)) {
    error_log("Got JSON response from /token without access_token $result_json");
    echo json_encode(array("error"=>"no token"));
    die();
}
$access_token = $result["access_token"];
echo json_encode(array("access_token" => $access_token));

And... that's all. At this point, if you Launch your activity from Discord, it should load, and should work out who the running user is (and which channel and server they're in) and that's pretty much all you need. Hopefully that's a relatively simple way to get started.

it's gotta be an SPA. Discord does not like it when the page navigates around ↩

A (limited) defence of footnotes

sil — Thu, 03 Jul 2025 07:12:00 +0100

So, Jake Archibald wrote that we should "give footnotes the boot", and... I do not wholly agree. So, here are some arguments against, or at least perpendicular to. Whether this is in grateful thanks of or cold-eyed revenge about him making me drink a limoncello and Red Bull last week can remain a mystery.

Commentary about footnotes on the web tends to boil down into two categories: that they're foot, and that they're notes. Everybody¹ agrees that being foot is a problem. Having a meaningless little symbol in some text which you then have to scroll down to the end of a document to understand is stupid. But, and here's the point, nobody does this. Unless a document on the web was straight up machine-converted from its prior life as a printed thing, any "footnotes" therein will have had some effort made to conceptually locate the content of the footnote inline with the text that it's footnoting. That might be a link which jumps you down to the bottom, or it might be placed at the side, or it might appear inline when clicked on, or it might appear in a popover, but the content of a "footnote" can be reached without your thread of attention being diverted from the point where you were previously².

He's right about the numbers³ being meaningless, though, and that they're bad link text; the number "3" gives no indication of what's hidden behind it, and the analogy with "click here" as link text is a good one. We'll come back to this, but it is a correct objection.

What is a footnote, anyway?

The issue with footnotes being set off this way (that is: that they're notes) isn't, though, that it's bad (which it is), it's that the alternatives are worse, at least in some situations. A footnote is an extra bit of information which is relevant to what you're reading, but not important enough that you need to read it right now. That might be because it's explanatory (that is: it expands and enlarges on the main point being made, but isn't directly required), or because it's a reference (a citation, or a link out to where this information was found so it can be looked up later and to prove that the author didn't just make this up), or because it's commentary (where you don't want to disrupt the text that's written with additions inline, maybe because you didn't write it). Or, and this is important, because it's funnier to set it off like this. A footnote used this way is like the voice of the narrator in The Perils of Penelope Pitstop being funny about the situation. Look, I'll choose a random book from my bookshelf⁴, Reaper Man by Terry Pratchett.

This is done because it's funny. Alternatives... would not be funny.⁵

If this read:

Even the industrial-grade crystal ball was only there as a sop to her customers. Mrs Cake could actually read the future in a bowl of porridge. (It would say, for example, that you would shortly undergo a painful bowel movement.) She could have a revelation in a panful of frying bacon.

then it's too distracting, isn't it? That's giving the thing too much prominence; it derails the point and then you have to get back on board after reading it. Similarly with making it a long note via <details> or via making it <section role="aside">, and Jake does make the point that that's for longer notes.

Even the industrial-grade crystal ball was only there as a sop to her customers. Mrs Cake could actually read the future in a bowl of porridge.

Note
It would say, for example, that you would shortly undergo a painful bowel movement.

She could have a revelation in a panful of frying bacon.

Now, admittedly, half the reason Pratchett's footnotes are funny is because they're imitating the academic use. But the other half is that there is a place for that "voice of the narrator" to make snarky asides, and we don't really have a better way to do it.

Sometimes the parenthesis is the best way to do it. Look at the explanations of "explanatory", "reference", and "commentary" in the paragraph above about what a footnote is. They needed to be inline; the definition of what I mean by "explanatory" should be read along with the word, and you need to understand my definition to understand why I think it's important. It's directly relevant. So it's inline; you must not proceed without having read it. It's not a footnote. But that's not always the case; sometimes you want to expand on what's been written without requiring the reader to read that expansion in order to proceed. It's a help; an addition; something relevant but not too relevant. (I think this is behind the convention that footnotes are in smaller text, personally; it's a typographic convention that this represents the niggling or snarky or helpful "voice in your head", annotating the ongoing conversation. But I haven't backed this up with research or anything.)

What's the alternative?

See, this is the point. Assume for the moment that I'm right⁶ and that there is some need for this type of annotation -- something which is important enough to be referenced here but not important enough that you must read it to proceed. How do we represent that in a document?

Jake's approaches are all reasonable in some situations. A note section (a "sidebar", I think newspaper people would call it?) works well for long clarifying paragraphs, little biographies of a person you've referenced, or whatever. If that content is less obviously relevant then hiding it behind a collapsed revealer triangle is even better. Short stuff which is that smidge more relevant gets promoted to be entirely inline and put in brackets. Stuff which is entirely reference material (citations, for example) doesn't really need to be in text in the document at all; don't footnote your point and then make a citation which links to the source, just link the text you wrote directly to the source. That certainly is a legacy of print media. There are annoying problems with most of the alternatives (a <details> can't go in a <p> even if inline, which is massively infuriating; sidenotes are great on wide screens but you still need to solve this problem on narrow, so they can't be the answer alone.) You can even put the footnote text in a tooltip as well, which helps people with mouse pointers or (maybe) keyboard navigation, and is what I do right here on this site.

But... if you've got a point which isn't important enough to be inline and isn't long enough to get its own box off to the side, then it's gotta go somewhere, and if that somewhere isn't "right there inline" then it's gotta be somewhere else, and... that's what a footnote is, right? Some text elsewhere that you link to.

We can certainly take advantage of being a digital document to display the annotation inline if the user chooses to (by clicking on it or similar), or to show a popover (which paper can't do). But if the text isn't displayed to you up front, then you have to click on something to show it, and that thing you click on must not itself be distracting. That means the thing you click on must be small, and not contentful. Numbers (or little symbols) are not too bad an approach, in that light. The technical issues here are dispensed with easily enough, as Lea Verou points out: yes, put a bigger hit target around your little undistracting numbers so they're not too hard to tap on, that's important.

But as Lea goes on to say, and Jake mentioned above... how do we deal with the idea that "3" needs to be both "small and undistracting" but also "give context so it's not just a meaningless number"? This is a real problem; pretty much by definition, if your "here is something which will show you extra information" marker gives you context about what that extra information is, then it's long enough that you actually have to read it to understand the context, and therefore it's distracting.⁷ This isn't really a circle that can be squared: these two requirements are in opposition, and so a compromise is needed.

Lea makes the same point with "How to provide context without increasing prominence? Wrapping part of the text with a link could be a better anchor, but then how to distinguish from actual links? Perhaps we need a convention." And I agree. I think we need a convention for this. But... I think we've already got a convention, no? A little superscript number or symbol means "this is a marker for additional information, which you need to interact with⁸ to get that additional information". Is it a perfect convention? No: the numbers are semantically meaningless. Is there a better convention? I'm not sure there is.

An end on't

So, Jake's right: a whole bunch of things that are currently presented on the web as "here's a little (maybe clickable) number, click it to jump to the end of the document to read a thing" could be presented much better with a little thought. We web authors could do better at this. But should footnotes go away? I don't think so. Once all the cases of things that should be done better are done better, there'll still be some left. I don't hate footnotes. I do hate limoncello and Red Bull, though.

sensible ↩
for good implementations, anyway; if you make your footnotes a link down to the end of the document, and then don't provide a link back via either the footnote marker or by adding it to the end, then you are a bad web author and I condemn you to constantly find unpaired socks, forever ↩
or, ye gods and little fishes, a selection of mad typographic symbols which most people can't even type and need to be copied from the web search results for "that little squiggly section thingy" ↩
alright, I chose a random Terry Pratchett book to make the point, I admit; I'm not stupid. But it really was the closest one to hand; I didn't spend extra time looking for particularly good examples ↩
This is basically "explaining the joke", something which squashes all the humour out of it like grapes in a press. Sorry, PTerry. ↩
I always do ↩
I've seen people do footnote markers which are little words rather than numbers, and it's dead annoying. I get what they're trying to do, which is to solve this context problem, but it's worse ↩
you might 'interact' with this marker by clicking on it in a digital document, or by looking it up at the bottom of the page in a print doc, but it's all interaction ↩

Serving streaming video that adapts to bandwidth from your own website

sil — Wed, 16 Apr 2025 09:26:00 +0100

Recently, I was involved in an event where a video was shown, and the event was filmed. It would be nice to put the video of the event up somewhere so other people who weren't there could watch it. Obvious answer: upload it to YouTube. However, the video that was shown at the event is Copyrighted Media Content and therefore is disallowed by YouTube and the copyright holder; it's not demonetised (which wouldn't be a problem), it's flat-out blocked. So YouTube is out.

I'd like the video I'm posting to stick around for a long time; this is a sort of archival, reference thing where not many people will ever want to watch it but those that do might want to do so in ten years. So I'm loath to find some other random video hosting site, which will probably go bust, or pivot to selling online AI shoes or something. And the best way to ensure that something keeps going long-term is to put it on your own website, and use decent HTML, because that means that even in ten or twenty years it'll still work where the latest flavour-of-the-month thing will go the way of other old technologies and fade away and stop working over time. HTML won't do that.

But... it's an hour long and in full HD. 2.6GB of video. And one of the benefits of YouTube is that they'll make the video adaptive: it'll fit the screen, and the bandwidth, of whatever device someone's watching it on. If someone wants to look at this from their phone and its slightly-shaky two bars of 4G connection, they probably don't want to watch the loading spinner for an hour while it buffers a full HD video; they can ideally get a cut down, lower-quality but quicker to serve, version. But... how is this possible?

There are two aspects to doing this. One is that you serve up different resolutions of video, based on the viewer's screen size. This is exactly the same problem as is solved for images by the <picture> element to provide responsive images (where if you're on a 400px-wide screen you get a 400px version of the background image, not the 2000px full-res version), and indeed the magic words to search for here are responsive video. And the person you will find who is explaning all this is Scott Jehl, who has written a good description of how to do responsive video which explains it all in detail. You make versions of the video at different resolutions, and serve whichever one best matches the screen you're on, just like responsive images. Nice work; just what the doctor ordered.

But there's also a second aspect to this: responsive video adapts to screen size, but it doesn't adapt to bandwidth. What we want, in addition to the responsive stuff, is that on poor connections the viewer gets a lower-bandwidth version as well as a lower-resolution version, and that the viewer's browser can dynamically switch from moment to moment between different versions of the video to match their current network speed. This task is the job of HTTP Live Streaming, or HLS. To do this, you essentially encode the video in a bunch of different qualities and screen sizes, so you've got a bunch of separate videos (which you've probably already done above for the responsive part) and then (and this is the key) you chop up each video into a load of small segments. That way, instead of the browser downloading the whole one hour of video at a particular resolution, it only downloads the next segment at its current choice of resolution, and then if you suddenly get more (or less) bandwidth, it can switch to getting segment 2 from a different version of the video which better matches where you currently are.

Doing this sounds hard. Fortunately, all hard things to do with video are handled by ffmpeg. There's a nice writeup by Mux on how to convert an mp4 video to HLS with ffmpeg, and it works great. I put myself together a little Python script to construct the ffmpeg command line to do it, but you can do it yourself; the script just does some of the boilerplate for you. Very useful.

So now I can serve up a video which adapts to the viewer's viewing conditions, and that's just what I wanted. I have to pay for the bandwidth now (which is the other benefit of having YouTube do it, and one I now don't get) but that's worth it for this, I think. Cheers to Scott and Mux for explaining all this stuff.

Use RSS to read newsletters

sil — Sat, 08 Feb 2025 15:09:00 +0000

Everyone's got a newsletter these days (like everyone's got a podcast). In general, I think this is OK: instead of going through a middleman publisher, have a direct connection from you to the people who want to read what you say, so that that audience can't be taken away from you.

On the other hand, I don't actually like newsletters. I don't really like giving my email address to random people¹, and frankly an email app is not a great way to read long-form text! There are many apps which are a lot better at this.

There is a solution to this and the solution is called RSS. Andy Bell explains RSS and this is exactly how I read newsletters. If I want to read someone's newsletter and it's on Substack, or ghost.io, or buttondown.email, what I actually do is subscribe to their newsletter but what I'm actually subscribing to is their RSS feed. This sections off newsletter stuff into a completely separate app that I can catch up on when I've got the time, it means that the newsletter owner (or the site they're using) can't decide to "upsell" me on other stuff they do that I'm not interested in, and it's a better, nicer reading experience than my mail app.²

I use NetNewsWire on my iOS phone, but there are a bunch of other newsreader apps for every platform and you should choose whichever one you want. Andy lists a bunch, above.

The question, of course, then becomes: how do you find the RSS feed for a thing you want to read?³ Well, it turns out... you don't have to.

When you want to subscribe to a newsletter, you literally just put the web address of the newsletter itself into your RSS reader, and that reader will take care of finding the feed and subscribing to it, for you. It's magic. Hooray! I've tested this with substack, with ghost.io, with buttondown.email, and it works with all of them. You don't need to do anything.

If that doesn't work, then there is one neat alternative you can try, though. Kill The Newsletter will give you an email address for any site you name, and provide the incoming emails to that as an RSS feed. So, if you've found a newsletter which doesn't exist on the web (boo hiss!) and doesn't provide an RSS feed, then you go to KTN, it gives you some randomly-generated email address, you subscribe to the intransigent newsletter with that email address, and then you can subscribe to the resultant feed in your RSS reader. It's dead handy.

If you run a newsletter and it doesn't have an RSS feed and you want it to have, then have a look at whatever newsletter software you use; it will almost certainly provide a way to create one, and you might have to tick a box. (You might also want to complain to the software creators that that box wasn't ticked by default.) If you've got an RSS feed for the newsletter that you write, but putting your site's address into an RSS reader doesn't find that RSS feed, then what you need is RSS autodiscovery, which is the "magic" alluded to above; you add a line to your site's HTML in the <head> section which reads <link rel="alternate" type="application/rss+xml" title="RSS" href="https://URL/of/your/feed"> and then it'll work.

I like this. Read newsletters at my pace, in my choice of app, on my terms. More of that sort of thing.

despite how it's my business to do so and it's right there on the front page of the website, I know, I know ↩
Is all of this doable in my mail client? Sure. I could set up filters, put newsletters into their own folders/labels, etc. But that's working around a problem rather than solving it ↩
I suggested to Andy that he ought to write this post explaining how to do this and then realised that I should do it myself and stop being such a lazy snipe, so here it is ↩

Blog Questions Challenge

sil — Mon, 03 Feb 2025 19:17:00 +0000

The latest thing circulating around people still blogging is the Blog Questions Challenge; Jon did it (and asked if I was) and so have Jeremy and Ethan and a bunch of others, so clearly it is time I should get on board, fractionally late as ever.¹

Why did you start blogging in the first place?

Some other people I admired were doing it. I think the person I was most influenced by to start doing it was Simon Willison, who is also still at it², but a whole bunch of people got on board at around that same time, back in the early days when you be a medium-sized fish in a small pool just by participating. Mark Pilgrim springs to mind as well -- that's a good example of having influence, when the "standard format" of permalinks got sort of hashed out collectively to be /2025/02/03/blog-questions-challenge, which a lot of places still adhere to (although it feels faintly quaint, these days).

Interestingly, a lot of the early posts on this site are short two-sentence half-paragraph things, throwaway thoughts, and that all got sucked up by social media... but social media hadn't been invented, back in 2002.

Also interestingly: the second post on this here blog³ was bitching at Mozilla about the Firefox release schedule. Nothing new under the sun.⁴

What platform are you using to manage your blog and why did you choose it? Have you blogged on other platforms before?

Cor. When it started, this site was being run by Castalian, which was basically "classic ASP but Python instead of VBScript", a thing I built. This is because I was using ASP at work on Windows machines, so that was the model for "dynamic web pages" that I understood, but I wasn't on Windows⁵ and so I built it myself. No idea if it still works and I very much doubt it since it's old enough to buy all the drinks these days.

After that it was Movable Type for a bit and then, because I'd discovered the idea of funky caching⁶ it was Vellum, that model (a) in Python and (b) written by me. Then for a while it was "Thort", which was based on CouchDB⁷, and then it was WordPress, and then in 2014 I switched from WP to a static build based on Pelican, which it still is to this day. Crikey, that was over ten years ago!⁸ I like static site generators: I even wrote 10 Popular Static Site Generators a few years ago for WebsiteSetup which I think is still pretty good.

How do you write your posts? For example, in a local editing tool, or in a panel/dashboard that’s part of your blog?

In my text editor, which is Sublime Text. The static setup is here on my machine; I write a post, I type make kryogenix, and it runs a whole little series of scripts which invoke Pelican to build the static HTML for the blog, do a few things that I've added (such as add footnote handling⁹, make og:image links and images¹⁰, and sort of handle webmentions but that's broken at the moment) and then copy it up to my actual website (via git) to be published.

It's all a bit lashed together, to be honest, but this whole website is like that. It is something like an ancient city, such as London or Rome; what this site is mostly built on is the ruins of the previous history of the city. Sometimes the older bits poke through because they're still actually OK, or they never got updated; sometimes they've been replaced with the new shiny. You should see the .htaccess file, which operates a bewildering set of redirects through about six different generations of URLs so all the old links still work.¹¹

When do you feel most inspired to write?

When the muse seizes me. Sometimes that's a lot; sometimes not. I do quite a lot of paid writing as part of my various day jobs for others, and quite a lot of creative writing as part of running a play-by-post D&D campaign, and that sucks up a reasonable amount of the writing energy, but there are things which just demand going on the website. Normally these days it's things where I want them to be a reference of some kind -- maybe of a useful tech thing, or some important thought, or something interesting -- for myself or for others.

Alternatively you might think the answer is "while in the pub, which leads to making random notes in an email to myself from my phone and then writing a blog post when I get home" and while this is not true, it's not not true either. I do not want to do a histogram of posting times from this site because I am worried that I will find that the majority are at, like, 11.15pm.

Do you publish immediately after writing, or do you let it simmer a bit as a draft?

Always post immediately. I have discovered about myself that, for semi-ephemeral stuff like posts here or projects that I do for fun, that I need to get them done as part of that initial burst of inspiration and energy. If I don't get it done, then my enthusiasm will fade and they will linger half-finished for ever and never get completed. I don't necessarily like this, but I've learned to live with it. If I think of an idea for a post and write a note about it and then don't do it, when I rediscover the note a week later it will not seem anything like as compelling. So posts are mostly written as one long stream-of-consciousness to capitalise on the burning of the creative fire before it gets doused by time or work or everything going on in the world. Carpe diem, I guess.¹²

What’s your favourite post on your blog?

Maybe It's Cold Outside, or Monkey Island 2, for about the fifth time, or Charles Paget Wade and the Underthing for writing, although each of them have little burrs in the wording that I want to polish when I re-read them. The series of birthday posts have been going on since the beginning, one every year, which probably wins for consistency. For technical stuff, maybe Some thoughts on soonsnap and little big details (now sadly defunct) or The thing and the whole of the thing: on DRM in HTML. I like my own writing, mostly. Arrogant, I know.

Any future plans for your blog? Maybe a redesign, a move to another platform, or adding a new feature?

Not really at the moment, but, as above, these things tend to arrive in a blizzard of excitement and implementation and then linger forever once done. But right now... it all seems to work OK. Ask me when I get back from the pub.

Next?

Well, I should probably point back at some of the people who inspired me to do this or other things and keep doing so to this day. So Simon, Remy, and Bruce, perhaps!

In my defence, it was my birthday. ↩
although no longer at simon.incutio.com -- what even was Incutio? ↩
I resisted the word "blog" for a long time, calling it a "weblog", and the activity being "weblogging", because "blog" is such an ugly word. Like most of the fights I was picking in the mid 2000s, this also seems faintly antiquated and passé now. Sic transit gloria mundi and all that. ↩
or "nihil sub sole novum", since we're doing Latin quotes today ↩
and Windows's relationship with Python has always been a bit unsteady, although it's better these days now that Microsoft are prepared to acknowledge that other people can have ideas ↩
you write the pages in an online form, but then a server process builds a static HTML version of them; the advanced version of this where pages were only built on request was called "funky caching" back then ↩
if a disinterested observer were to consider this progression, they might unfairly but accurately conclude that whatever this site runs on is basically a half-arsed system I built based on the latest thing I'm interested in, mightn't they? ↩
tempus fugit. OK, I'll stop now. ↩
like this! ↩
an idea I stole shamelessly from Zach Leatherman ↩
Outgoing links are made to continue to work via unrot.link from the excellent Remy Sharp ↩
I was lying about not doing this any more, obviously ↩

Forty Nine

sil — Thu, 30 Jan 2025 23:04:00 +0000

The sum of the digits of the square of 49 (2401) is the square root of 49.

49 is the first square where the digits are squares. In this case, 4 and 9 are square numbers.

It seems that 49 is an age of squares.

I find myself increasingly OK with this.

It is interesting, although really quite disillusioning, reading through the series of posts I've done on my birthday (follow the links back from last year for 22 years now) and seeing the trend in recent years. The world has got more worrying. A lot more. I hate this.

So I've tried to find joy where I can get it. I'll give you an example. I looked up the number 49 for this, the same as I always do, and apparently 49 is the smallest discriminant of a totally real cubic field. Now, I'm sure that there are maths people out there who understand this. But to me, it is impossible to read this without putting on a mock Valley bro accent: yeah brah, this field is todally real, yah? And that made me smile. Bill dropped a birthday message on me at half seven this morning. Yesterday, I remembered a throwaway joke line that popey dropped on me in 2015 and I literally laughed out loud in my own living room at the thought of it. Two days ago Jess and I went to see Murder on the Orient Express on stage at the Alexandra Theatre. Tonight I've been for beers with Bruce and Matt. My D&D group are about to confront Harazos to let them perform a ritual they've made up. Everyone in my team at Barnardo's was really nice about my birthday. None of this helps fix the world crashing down around us, but all of it certainly makes it easier to cope with, because all of it is a little bit of joy, of niceness, in the midst of unjoy and unniceness. Thank you, all of you.

This has been not a great year. There's been ups and there's been downs. I'm still a lot better off than almost everyone -- I'm not under threat, I'm not more under threat than I was before, I can afford to do what I like as long as what I like isn't too extravagant. But it has nonetheless been tough. Maybe 2025 will be better, although to be honest I doubt it. And I have this sense of regret, not about the world getting worse, but that the idea that the world might get better has basically gone away. When I was younger, the thing was that the world would get better, year on year, as time passed. And I am 100% sure that this was a thing that I got to think because I was privileged to not be being screwed over by the world. But I still had that thought, that things were OK today and would be slightly more than OK tomorrow, and at some point that went away. Things probably won't be better tomorrow. If they're the same, that probably in itself is a victory. This isn't really my tragedy; it's for people younger than me, who haven't experienced this change, who assume that things getting steadily more grey and sad and worse is just the way the world is; that's just normal.

So I look for little bits of joy. Hopefully you have some little bits of joy yourself, as I do. Happy birthday to you, even if it's not your birthday. Eat some cake.