I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Feedback wanted

I’m giving a small number of people access to the guide in exchange for their feedback. If you would like to read the first few chapters and start your implementation send me a message.

A book in two parts

In the interest of releasing early, I’ve split up my original plan for the guide into two parts. The first part is called ‘Running SQL’ and it’s focused on how to parse and execute SQL statements as well as going through database concepts along the way.

I’ve called the second part ‘Speed and Safety’ and it’s more focused on a few database concepts that help make queries faster (indexes and concurrent execution) and keep data safe (persistence, transactions).

I’ve made sure that the first part stands on its own. If I complete the second part it will be a free addition to anyone who has the guide.

Browsing the sample solution

The guide comes with a sample solution in Ruby and I wanted to be able to link to the commits and diffs associated with each chapter. To do this I pushed the solution to GitHub and created branches for every chapter. This means I can link to the state of the sample solution after a specific chapter and show diffs for all the code added in a chapter.

I wanted a way to update all the branches if I had to change an earlier commit. I wrote a Ruby script to help with this that I can run after rebasing the main branch.

The script has a list of commit messages and corresponding chapter names. For each chapter, it finds what should be the last commit on the branch for that chapter. It then resets the branch for that chapter to that commit. Here’s the part that finds the commit and resets the branch:

hash = `git log --grep="^#{message}" --format="%H" -n 1`.strip
`git checkout #{chapter_name}`
`git reset --hard #{hash}`

I did wonder if GitHub was the best tool to use. I looked at other git solutions that I could host myself (including Gogs, GitLab, Gitea, cgit and Stagit) but didn’t find one that had an interface that was as easy to navigate as GitHub and was worth the complexity in setting it up.

Next

There’s a few bits in the first part of the guide that need to be finished and I’ll be making changes based on any feedback.

This might be may last booknotes. When the guide is available to buy I’m going to focus on ways of promoting it that will better show off its contents. I have ideas for a few articles based on topics in the guide that I could write.

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Progress

This week I’ve hit the milestone of having a complete sample implementation!

Grouping and aggregation

Grouping and aggregation were the most complex parts to implement. This complexity comes from how grouping changes the structure of the data it acts on and how that interacts with type checking and the operations that run after the grouping.

I looked at a few databases to see how they implement grouping. SQL database implementations tend to have speed as their goal rather than the simplicity and readability of the code which can make them hard to follow. H2 is one of the few that isn’t written in a low level C-like language.

An insight that helps implement aggregation is how databases that process data in rows usually use an accumulator to calculate the result a row at a time rather than trying to process a whole column of data at once.

The SQL standard

I have wanted to take a look at the SQL standard for a while to clarify some behaviour, but didn’t want to pay the hundreds it costs to purchase the standard. I went to the Library at the IET (The Institution of Engineering and Technology) which lets you view many standards through their network. They are open to the public for free and was a nice place to work from for the day.

The standard was also helpful in confirming the exact meaning of certain terms and making it clear where implementations have the freedom to do their own thing.

HYTRADBOI conference

I spent Friday watching talks from HYTRADBOI. Since the talks were about databases and programming languages, I can’t imagine a more relevant conference for me.

My favourite talks were “A quick ramp-up on ramping up quickly” (about the tradeoff between optimizing for initial run time and later runs in javascript engines), and “A case for feminism in programming language design”.

I also learnt about the pipe-syntax that is now available to use in BigQuery and the process of building things by throwing them away.

My lightning talk on the odd bits of SQL is also available.

Updated chapter list

Going through the sample implementation again has helped me break up the guide into sensible chunks. I’ve updated the chapter list on the landing page for the guide. Most of my time in the next few weeks will be writing the remaining chapters for part one of the guide.

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Progress

This week I’ve continued with my sample implementation so it can handle filtering, sorting, limiting and joins. I’ve also written a chapter about references.

Building web-first

I’m mildly frustrated when technical books are only available in a fixed-size PDF or in a non-so-nicely formatted ebook. My plan for the book is to first publish it as a website. The web is great at looking good at different screen sizes and is very easy to make updates to. To do this, I’m writing the each chapter as a tab in a Google Doc and have a process that exports the document as markdown and then renders it as HTML with Kramdown.

Making a PDF

Although I am building for the web, I did want to make sure I could turn the book into a PDF that looks good in case there was any need for an offline version. I first tried printing the page as a PDF from Chrome. I knew Chrome Dev tools had a way to change the CSS media type to print but it took me a while to find it. It seems like the Chrome engineers do love moving the setting around every few releases. The wording of the menu and setting has even changed since the Stack Overflow answer.

I fixed some visual issues in a new print stylesheet. I made use of break-inside for the first time to keep certain elements together on the same page. I used the Chrome print dialog to save a PDF, I’ll likely automate it if I need to do it again.

Consistent chapters

One thing I’m doing at the moment is making sure the chapters are all of a similar size and can be implemented in a similar amount of code. I’ve moved a few things from the main implementation into optional ‘extensions’ section that each chapter has to help balance things out.

One example is inserting items into explicit columns (e.g INSERT INTO t1 (b, a) VALUES (2, 1); instead of INSERT INTO t1 VALUES (1, 2);). This feature added a bit of complexity early on to the implementation and no later features relied on it- I think it’s much more rewarding to have features you can use and build upon in later chapters.

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Progress

This week I’ve continued with my sample implementation which has helped me to go back and improve the first four chapters.

A new chapter appears

In my last booknotes I said that I’ve restarted my sample implementation so I can go through the test-suite as someone following the guide for the first time would. This has been a great way to evaluate the existing chapters to make sure they have a clear focus and the right amount of complexity.

It was very satisfying to find there was a whole whole chapter related to parsing that was previously muddled into three other chapters. This week I’ve written this chapter which I’ve called ‘Resilient parsing’ and goes into detail about mixed capitalization, extra whitespace and identifiers that overlap with keywords. It suggests using tokenization to help manage this complexity.

Simpler tokenization

And focusing on parsing made me realise that I had overcomplicated the regular expressions I was using in my tokenizer. Previously I used regular expressions to break up the input like this:

if keyword = input.match(KEYWORD_PATTERN)
   Token.new(:keyword, keyword.upcase)
elsif identifier = input.match(IDENTIFIER_PATTERN)
   Token.new(:identifier, identifier)
elsif symbol = input.match(SYMBOL_PATTERN)
   Token.new(:symbol, symbol)
elsif integer = input.match(INTEGER_PATTERN)
   Token.new(:integer, integer)
end

The problem was that keywords (such as SELECT, insert, WHERE) and identifiers (such as my_column, TABLE_B or selected_value) look very similar and the only way to distinguish between them is to use a look-ahead in the regular expression to make sure a keyword isn’t followed by another character or underscore. I also didn’t like how changing the order of the conditions would break the parser because the identifier regular expression didn’t exclude keywords. Put another way, it was hard to make all my regular expressions exclusive of each other.

I found I could avoid this just by matching on a pattern that matched keywords AND identifiers and then checking if the match was in a list of keywords:

if word = input.match(WORD_PATTERN)
   if KEYWORDS.include?(word.upcase)
     Token.new(:keyword, word.upcase)
   else
     Token.new(:identifier, word)
  end
elsif symbol = input.match(SYMBOL_PATTERN)
   Token.new(:symbol, symbol)
elsif integer = input.match(INTEGER_PATTERN)
   Token.new(:integer, integer)
end

Enumerators without next

I was looking at using enumerators in my sample implementation and found that certain kinds of enumerators in Ruby don’t have the next and related methods available.

I submitted a bug to Ruby for the problem. There is likely some reason that product and chain enumerators have been implemented differently to regular enumerators, but I can’t work out why. I’m hoping that they could be made more similar or at least the documentation improved to explain the difference.

Talk

I still would like to submit a lightning talk for February’s LRUG but I’m away for some of next week so don’t think I will have the time. Either way I will be there to watch the other talks.

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Progress

I’ve written more on the chapter on joining data and made improvements to the test runner.

Talks

I’ve submitted a lighting talk to HYTRADBOI. The slightly-silly talk goes some of the shade the PostgreSQL documentation throws on the SQL standard.

I also have an idea for a talk about implementing part of a database server that I might submit to LRUG for February.

Restarting my implementation

I’ve restarted my sample implementation so I can go through the test-suite as someone following the guide for the first time would. I haven’t started completely from scratch since I’m re-using chunks from my previous implementation.

This has led me to add better error handling so even unexpected errors are formatted sensibly and a feature that prints out tests that have passed for the first time. I’ve also added some automated tests for the test suite itself that run as a Github Action against different Python versions.

Since I’ve rearranged a few chapters, this process has also let me build a git history that better matches the order of the guide.

Iterators

I’m deciding when I should introduce iterators in the guide. Many real databases use the iterator pattern to process different parts of the SELECT queries. Iterators let you pull in data as it’s needed. This avoids redundant processing and has lower memory requirements compared to running each part of the query in sequence.

In the first part of the guide I want to discourage people thinking from about performance of queries and working on optimizations that might make their implementation harder to change. In some ways iterators are an optimization but I’m trying to decide if they are also a more helpful abstraction for processing data that could lead to more understandable solutions.

Goal for February

By the end of February I’d like to have the guide at the point where I can share it with a few people that can try it out.

The main things I want to do before then are:

• Write last two chapters on aggregate functions and grouping
• Complete sample implementation for all chapters

The things I’d like to do:

• Have a chapter/section about using iterators to process queries
• Write implementation hints for every chapter

And the things I think I can leave to later:

• Write an introduction about SQL
• Complete all diagrams
• Write an appendixes about working on the test suite and ideas for bigger extensions

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

Progress

This week I’ve been working on two chapters, the first is about implementing filtering, sorting and limiting data (with WHERE, ORDER BY and LIMIT) and the second is about joining data.

Researching

A question I’ve tried to answer this week is why the syntax for CAST is CAST(value AS type) and not just CAST(value, type) like other functions in SQL. I’ve read people suggesting that the AS improves readability, but I’m not sure I believe that as an answer. My reckon is that it’s because most other functions take values and type references such as INTEGER or BOOLEAN aren’t values.

Finding reasons behind decisions like this can be tricky for SQL as many of them happened in the 70s. CAST did come later in the 1992 SQL specification, but was in some implementations before then. It’s not that important to know, but I was curious and I feel that I owe a justification to readers as to why parsing CAST will be that little bit different to other functions.

Surprising behaviour

I was surprised as to how references in ORDER BY clauses get resolved. For example:

SELECT -id AS id FROM orders ORDER BY id;

SELECT -id AS id FROM orders ORDER BY id + 0;

These return rows in different orders in SQLite, PostgreSQL and MySQL. Turning id in the ORDER BY clause from just a reference into an expression causes the column to be used rather than the value from the select list.

MySQL and SQLite allow you to use named items from the select list within ORDER BY expressions when there are no clashes, but PostgreSQL doesn’t (e.g: SELECT id AS x FROM orders ORDER BY -x;). I’m assuming it’s meant to behave this way. I couldn’t find this in the documentation for any of the databases I looked at, I’m yet to delve into the SQL specification and could also check the source code of some databases to confirm the behaviour.

Language

I’m a strong believer of finding the right name. When describing databases, there are lots of overloaded words¹, or words that don’t quite fit in all situations. An example of this is talking about what is being joined with JOIN. The easy answer is tables, but this isn’t always accurate. You could join a view, or join to already joined data.

Relational algebra would describe it as a relation but strictly relations are sets of tuples that can’t contain duplicate rows. Also, in my experience, the word ‘relations’ isn’t commonly used by application developers. I haven’t yet settled on what I should call it. I could stick with ‘table’, but with an explanation. I could be vague and call it ‘data’ or a ‘dataset’. Tabular data? Table-like? A collection of rows? A bag of tuples?

I’m writing a guide to building your own database server. Here’s what I’ve been doing:

• Progress - My aim is to get the first part of the guide in a releasable state by April. I’ve written 6 of the 10ish chapters so far and have a rough outline for the rest.
• Portable code - The book comes with tests to help guide your implementation. So that the tests are as easy to run as possible for most people I’ve chosen to write the test runner in python. Python is widely used, is included with many Linux distributions by default and has an installer for OSX. By having zero dependencies and by targeting a broad range of python 3 versions I hope to reduce any faff that people will have to go through to run the tests. I considered writing the tests in JavaScript/TypeScript and using Bun to make a distributable executable but think I prefer to keep the test runner code more visible and easily editable by people. I’ve spent some time over the last weeks adding to the test runner so that it formats failures better and can run individual test cases.
• Diagrams - I’ve been deciding what tool to draw the diagrams for the guide in. I’d like the tool to be able to export SVGs with a transparent background so that the diagrams look good at different sizes and can work on light and dark backgrounds. The top two contenders are tldraw and FigJam. Both make it easy to create diagrams with arrows that snap to boxes. I like the ‘sketchy’ look that diagrams from tldraw have but it’s very restrictive on customization such as with colours and font sizes. FigJam is a little more customizable, but has to be pasted into Figma before it can be exported as an SVG. Here’s an example that shows how to structure a parser that can work with statements such as SELECT 1, TRUE, FALSE; made in tldraw and FigJam. Let me know if you prefer the look of one over the other or have a suggestion for another tool I should try.
• Typing - The last chapter I wrote was about catching errors and typing statements. In each chapter I call out any interesting differences and implementation choices in different database engines. I learned that SQLite has chosen to be much more flexible when it comes to types compared to other databases, for example "23dogs" / "5" will happily evaluate to 4. SQLite is the JavaScript of databases - it’s everywhere and will do whatever it can to coerce types rather than give an error.

I started using the remote mob and pair programming tool mob with my team. I’ll explain what the tool does and how it helps to maintain momentum and productivity while pairing.

I’m a big fan of pair and mob programming. It’s the quickest way to share knowledge, skills and make progress on many problems. I’m always looking for ways that the pairing experience can be better; how can you remove friction so that it’s easy for engineers to work on the same thing at the same time? And how can you make sure the experience is engaging for everyone and one person it’s hogging the keyboard?

What can make paring frustrating?

There are a few things that can make pairing no fun at all:

Using someone else’s editor

Using a different editor with different extensions and shortcuts is difficult. If you work remotely, waiting for each keystroke to travel across the world will be laggy.

Switching infrequently

If you switch drivers infrequently, people can disengage and won’t have an understanding or ownership of a problem. Skills and power imbalances can make this worse. A senior engineer might hog the keyboard, stopping more junior engineers from being able to explore the problem and contribute.

Breaking your flow

Just like you want fast feedback from running tests so you don’t interrupt your flow, you want to be able to switch drivers quickly without losing focus on your work. And if switching drivers is hard, you’ll do it less often.

So you might have a better time if you can pair while using your own editor with frequent switches and without breaking flow.

The mob tool

Mob is a command line tool that helps you pair and mob programming.

It uses Git to help manage the branch you’re mobbing on. And it works well for a pair of two people up to mobbing in larger groups.

Mob helps solve common frustrations with pairing by helping you have frequent and smooth handovers while letting you use your own editor. Mob is designed to work with remote pairing, so sharing a single machine might work better if you’re in the same room with someone.

Most time you use mob, you’ll be using three commands:

mob start 10 starts or continues a mob session on the current branch with a 10-minute timer. The way that this works is that mob will create a temporary branch prefixed with ‘mob’.

mob next pushes your work to a work-in-progress commit, ready for the next person.

mob done ends the mob session by squashing your commits together, ready for you to review and write a commit message. It also removes the temporary mob branch.

You can learn more about using mob on its project page.

Making sure you switch frequently

Mob reminds you to switch by playing a sound. You choose the handover time. I’ve found 10 minutes works well, and sessions flow best if you try to stick to the timer. One exception is that switching a minute earlier or later can be better if it coincides with you making a test pass or writing a new failing one.

Having a timer solves so many problems - forgetting to switch or feeling embarrassed to ask to switch because you are stuck. The magic of mob is that it’s easy to set the timer as part of switching.

Easy handovers

Mob gives you easy handovers with mob start and mob next. You don’t have to think; just run the commands, and mob will handle it.

Mob will also tell you who’s next by looking at the session history. Useful for keeping track of the rotation if you’re working in larger groups.

And mob will open the last modified file in your editor if you have the MOB_OPEN_COMMAND set, stopping any fumbling to find the right file.

All these things feel small, but having a speedy and reliable handover process means you don’t lose pace when you switch drivers. If switching frequently, an efficient handover process becomes more important.

You could do some of what mob does yourself by managing your own branch in Git, but it won’t be as easy as using the mob command, and you won’t get the other useful features.

What using mob feels like

I’ve found that pairing and mobbing sessions with mob have more momentum. By this, I mean that because everyone gets to type, and each interval is short, it’s more engaging and feels like you’re going faster. Someone I work with described it as ‘Energy boosting, rather than energy sapping’.

I’ve also found that because mob provides structure to your coding sessions, it’s easy for someone to join or leave a mob session without disruption.

Mob’s default is to make a work-in-progress commit when you switch and then squash them together when you finish. If you like making small and well-described commits as you work, you can still do this with mob. Commit on your branch as you’re working on it, and make sure you set MOB_DONE_SQUASH to no-squash or squash-wip so that these commits aren’t lost when a run mob done.

Give mob a try

If you work remotely with others, give mob a go!

Experiment with different timer lengths. The default of 10 minutes is a good amount of time for me. You might find different timings work well for you depending on how many people you are working with and what you’re working on.

I recommend working to optimise your handover time. Using mob is the first step to do this, mob has suggestions on how to configure it and other practices to follow for the smoothest of handovers.

Read more

mob tool on GitHub

Remote Mob Programming - Best practices from the creators of the mob tool

When I gave writing weeknotes a go this year, I liked having a place that I could record and make notes on the the recipes I had made. After I stopped writing weeknotes I wanted for another way I could do this.

I have started to collect recipes in my cookbook. Everything in there comes highly recommended with a five-star/‘I would make this again!’ rating.

The cookbook itself made up of jekyll posts - metadata about the recipes are stored as YAML, and my comments are in markdown. So that I don’t need to fear that my favourite cookie recipe might be lost forever, I made a script that makes a local copy of all the recipes I link to.

Automatic matching

I wanted to include as many different BSicons as possible in my generated maps. I made a copy of all the BSicons from Wikipedia and created a script to analyse them. By examining the pixels in the icon the script would categorize each edge. I used this later to decide if two icons could ‘fit’ together. The strategy for map generation was to:

1. Place a random icon in the centre of the grid
2. For each empty tile in a spiral from the centre, find a tile where the edges of the tile matched with the edges of all adjacent placed tiles

This was a good start and made some interesting maps. However, the way I was determining if icons could fit together wasn’t very reliable. It meant that many icons were never be placed because they had no matches. While icons with curves wouldn’t visually match to the icons they were placed next to.

Less is more

By reducing the number of BSicons I was working with, it became feasible for me to manually categorise what tiles could fit together. I changed to using the wavefunction collapse algorithm to place the tiles. The wavefunction collapse algorithm prioritizes placing tiles in the spaces with the greatest restriction on them. This reduces situations where the algorithm can’t place anything and gets stuck.

I really like the interconnected and overlapping routes the generator can make. You can use it to generate your own maps.

If you go to the Wikipedia page for the Forth and Clyde Canal, the Trans-Siberian Railway or the London Circle Line you may notice that they all have route maps with a similar visual style.

How do the many Wikipedia contributors create all these similar-looking maps? The answer is BSicons.

BSicons

BSicons are the building blocks for these route maps. They are an SVG icon set that Wikipedia contributors use to map railways, subways, footpaths, waterways, cycle paths and roads. Any transport system that they want to produce a simplified schematic for.

The icons were first used on the German Wikipedia which is why most of their names are based on German words. ‘BSicons’ comes from the German word Bahnstrecken (Railway Lines).

Each icon has a shorthand name that follows some conventions:

• A Root picks a given icon such as a station or a crossing.
• Prefixes change it’s status, such as if it’s underground or disused.
• Suffixes denote direction and positioning, such as if it curves to the left or is one way.

Here are some of the icons:

STR

A rail line

Root: STR → STReke (line)

WASSER

A river or other waterway

Root: WASSER (water)

KRZu

Two rail lines crossing with one going under the other

Root: KRZ → KReuZung (crossing)
Suffix: u → unter (under)

STR+l

A rail line that comes from the left

Root: STR → STReke (line)
Suffix: +l → + (from), links (left)

Directions are relative to the line (which is drawn from top to bottom) rather than the page. This is why the suffix says the line is from the left rather than going to the right.

utkBHF+1

A station on an underground light rail line that comes from the first corner

Root: BHF → BahnHoF (station)
Prefix: utk → u-bahn, tunnel, kombination (Compound)

the k denotes that the icon is for drawing compound turns that span four columns of icons.

RP2r

A two-lane paved road with a roundabout

Root: RP2 → Road Paved 2-lane
Suffix: r → roundabout¹

tSTRa

A rail line that goes into a tunnel

Root: STR → STReke (line) Prefix: t → tunnel
Suffix: a → anfang (start)

mhKRZho

An elevated rail line going over a light elevated rail line

Root: KRZ → KReuZung (crossing)
Prefix: mh → mischbetrieb (mixed), hochbahn (high level/elevated)
Suffix: ho → hochbahn (high level/elevated), overpass

These are a few examples. The BSicon Catalogue explains many more conventions.

Building maps

To help put all these icons together into a map Wikipedia has a Routemap template. The template defines a syntax to provide shortcuts to build maps.

Here is the code used to generate the single highlighted row:

{{Subway|Hillhead}}! !\utkBHF+1\\\utkBHF+4\tSTRa~~{{Subway|St George's Cross}}

The syntax is a bit cryptic and uses the shorthand icons names we saw above. Here is what each part does:

• {{Subway|Hillhead}} is a link to the Wkipedia article about this station
• ! ! separates the text on the left from the icons
• \utkBHF+1 is the first icon a light rail station in a tunnel that curves
• \\ adds in two empty icon-sized spaces
• \utkBHF+4 adds in the second icon, similar to the last but towards top-left corner
• \tSTRa adds in the third icon, a start of a stretch of tunnel
• ~~~ separates the icons from the text on the right
• {{Subway|St George's Cross}} a link to the Wikipedia article about this station.

Hundreds of thousands of icons

There are more than 290,000 BSicons out there due to an explosive number of combinations needed for different colours, rotations, intersections, bridges, curves, embankments and tunnels. Many obscure BSicons might only be used a few times across Wikipedia (like BHFABZgl+l)².

Each icon is simple, so simple many of them are marked as ineligible for copyright on Wikipedia. Together the icons can combine to map more complex systems. Seeing how these icons can be combined inspired me to make a random BSicon map generator.

• Mapping - I live close to the Norton Common nature reserve in Letchworth. Over the last few visits I’ve explored more of it so I can improve it on Open Street Map. I’m quite happy with where it’s at now, especially compared to the two paths mapped on Google’s version
• Offsite - Farewill usually has off-sites every couple of months. We had a virtual one this time around and was nice to get to speak to some of the newer joiners in the company. We split into much smaller groups and then split again into teams. It worked so much better than having all of us in a single Zoom meeting would have.
• Immutable objects - This week at work, I wished that none of the functions on JavaScript standard objects modified the receiver. I’ve seen it cause too many bugs
• Plants - I took a cutting from my Chinese Money plant which is doing well in a jar of water. I don’t understand why plants that need well-drained soil can also thrive in just water, I would love to know the answer but searching has lead me nowhere. Less happy is my Aloe Vera. I moved it to a sunnier sunny spot for it’s own benefit to see it go brown in sun-shock.
• Baking - As well as seeing some of the amazing things my sister has baked, I’ve made a Lemon Drizzle Courgette cake and Spiced Pear Muffins. Both of these are butter-less - Salt, Fat, Acid, Heat made me realise that I prefer the moistness and texture of oil-based cakes.
• Wavelength Remote - I adapted Wavelength so I could play it remotely with friends until the creators launch their app. It involved multiple cameras and asking people to close their eyes at certain points. It kinda worked. A friend has made a remote-friendly web-based game that uses a similar mechanic.

• Bird Table - I put a GoPro in the garden and filmed a couple of birds that I tempted on-camera with some dried mealworms. The blue tit is in half-speed since it was barely there at all.
• Hair - MY FRINGE IS OUT OF CONTROL. I’m using two hair clips to keep it out of my eyes. I don’t feel brave enough to trim it myself yet.
• Baking - I made some more port-brownies and a rhubarb crumble with custard. It’s useful being able to use my weeknotes to remember what particular recipe I used a few months ago.
• Isolation Realisations - I miss the act of travelling - going to work, walking to the shops, shuffling between meeting rooms. Never in my adult life have I not been on a train for such a long stretch.
• Easter Egg Hunt - Pauline surprised me with a puzzle-based easter egg hunt which was lots of fun. I was especially impressed with the crossword grid and want to try making one myself.

• Crossword improvements - I’ve had some lovely messages from people who are enjoying using Several People are Solving. I’ve also seen the traffic jump from 10 people a day a month ago to more than 250 people a day now. I’ve spent a bit more time on it and there are now more links to crosswords from the home page. I want to colour-in moves so you can see who made them, but I’m not sure the best way to do it to have some continuity for a user’s colour across games but also avoid clashing. I’d like not to have to create an account system either.
• Lower latency - In pursuit of more reliable video calls and lower-latency Geforce Now I set up a Powerline ethernet connection which works great. I’ve played more Cities Skylines and I’m likely hours away from buying the Sunset Harbour expansion that was released this week.
• Keeping a routine - After spending a whole week either at my desk working or on the sofa watching TV, I felt unenergised and my days began blurring together. I’ve instead tried to go for a walk before starting work to help refresh myself and get me out of bed on time and doing indoor-exercise in the evening using FitOn or a cross-trainer.
• Splitting strings - Skills matter and their library of videos is back online! I made a copy of my Splitting Strings video for safekeeping. I re-read some of the discussion about my talk and post on Reddit. The, perhaps not explicit enough, point of the talk was about the surprising complexity of seemingly simple functions and how there can be logical reasons this complexity and differing behaviour. So much of the discussion devolves into what programming language is ‘right’ or ‘wrong’. Oh well.
• Veg - We found our nearby farm shop has a bountiful supply of vegetables that our local supermarket is missing. I also got to use my drill to help Pauline put together a raised bed so we can grow our own. We have some broccoli, cabbage and bean seedlings that are almost ready to go in.
• Brownies - I wanted to put the last two eggs in the fridge to good use so made brownies. I usually put espresso powder and whisky since they both go so well with chocolate but instead used some port leftover from Christmas. I used this recipe with the addition of a medium wine-glass worth of port into the butter and sugar mix. I was VERY happy with them.

• Toilet roll - I thought I knew how it was made, but then I watched a clip from the BBC and I’m now convinced that they steal toilet paper from giants and then use some kind of shrink-ray to ready it for consumers.
• WFH - Starting Thursday this week we’ve been asked to work from home until further notice. I’m thinking about if there are changes we could try in our team to help us adjust to working remote more. The Maintaining connection & collaboration at a time of isolation email from Tessa’s Collaborative Future arrived in my inbox just in time.
• More multiplayer crosswords - I’ve heard that at least one team is using Multicrosser to do crosswords together while remote. I also had a peep at the Guardian’s new Puzzles App which is provided by Amuse Labs. Excitingly, it lets you share a crossword with one other person to do real-time solving.
• Work area improvements - I’ve spent some time this weekend tidying our spare bedroom and making it a bit more pleasant to work in. I’ve bought some soft furnishings, put up a couple of pictures and ordered posters from the London Transport Museum and Science & Society Picture Library.
• Gaming - Nvidia has made their game steaming service Geforce Now generally available. I jumped at the chance to run Cities: Skylines which I’ve been aching to play for ages but couldn’t on my underpowered laptop. There’s an occasional stutter where everything looks like a 10%-quality JPEG, but otherwise it works really well. I’m going to see if connecting via ethernet instead of wireless improves reliability.
• Eating - I’ve been going through all the veggie items in Morrisons freezer section, I really rate The Best Layered Vegetable Tart - it really is.
• Watching - The last two Inside No. 9 episodes were my favourites of the series. Also, Bon Appétit fans will enjoy an interview with a lot of the Test Kitchen and “Amateur Toy Maker Attempts to Make Claire Saffitz Bootleg Action Figure”.
• Spot my face - My team has rebuilt our careers page to be the nicest looking careers page of anywhere I’ve worked. Can you spot all the places my face appears? (hint: you need to view the page on a larger screen to see them all).

• Magic paste - I’m continuing with an endless stream of house chores. This week I added a towel rail and hooks to the bathroom. Next on my todo list is filling some old drill holes in the bathroom with some magic paste and researching how on earth to fit curtain rails/poles to our bay window.
• SSL Certs - I spent some time managing my Dokku-powered web server. I got an email from Let’s Encrypt telling me that the ACME API that my older version of Dokku was using would stop working in June. For self-hosted sites, the pain of managing SSL certificates has been replaced with the meta-problem of managing software that manages SSL certificates. Still, I’m appreciative of the Let’s Encrypt project and it’s nice that I got the warning email.
• Collecting succulents - I got a pretty String of Pearls plant at our nearby garden centre. I’m still looking for a Chinese money plant and something bigger to live on the floor (suggestions welcome).
• Progression - I spent some of my time at work this week looking at publicly available progression frameworks as it’s something we want to introduce to help us now the engineering team has grown (there’s 10 of us now!). My favourite is Songkick’s Growth framework since it has lots of great examples that aren’t too Songkick specific.
• Chromecast is awful - Our meeting rooms at work all use Chromecast as a way to share your laptop screen with the TV. I find it so difficult to use. If you want to share your screen rather than the current Chrome tab, you have to stop casting and start again. When you want to stop casting it’s way too easy to stop someone else’s cast in another meeting room. And then there’s all the cutting out and glitching to put up with. Jack of all trades and master of none, Google Chrome has become the iTunes of today.
• The roundest animal - Our copy of Wavelength came, me and Pauline tried the 2-player variant. You play by drawing a card which shows two extremes (such as hot & cold). One player has to secretly spin the wheel so it points to somewhere between the two extremes and has to think of a word or phrase that will help others guess where the wheel is pointing too. One of the cards was ‘Roundest animal’ to ‘pointiest animal’ which was fun to debate.

• Gatsby - I’ve been using Gatsby a lot at work for the last few weeks. It’s a flexible static site generator that’s able to create pages from different content sources. We use it a lot with the Contentful API to build pages. The downside is that I find it’s GraphQL functionality less well documented and I’m unsure of what expected behaviour is for some queries. Some query operators that I wanted to use were missing (such as filtering results on the length of a property).
• Overboard - Polygon’s Overboard is one of my favourite YouTube channels. I had the giggles throughout their play of Wavelength and it convinced me to order the game.
• Perfectly balanced - I was around friends this weekend playing Pandemic Iberia. We beat the game, just. The last four moves turned into a complex logic puzzle that we talked about for twenty minutes. Me and Pauline often play Pandemic with two player-cards each to make it more complex when there’s just the two of us. I’m in awe at how well balanced the game is - we often win or lose by a single turn.
• Double helix staircase - Most days on my commute, I change from rail to the Underground at Finsbury Park. I take one of the two spiral staircases that lead to the southbound Victoria and Piccadilly line platforms. The layout of the station never made sense to me and I always struggled to know what way to turn from the stairs to the to tube platform. I found an older plan of the station¹ and it all clicked - what I thought were two stairwells was just one with a set of two stairs inside intertwined in a double-helix. If you take one stair you need to turn left onto the Victoria line at the bottom, you turn right from the other. There’s another double-helix staircase to the northbound platforms too.
• Coding fonts - I like the look of the new Jetbrains Mono font but was put off trying it because of its coding ligatures. I feel similar to the point in this post - symbol-merging ligatures add in ambiguity which programming fonts otherwise try to avoid. I saw there is some demand for a non-ligature version and it might be possible to disable all ligatures in Atom. I decided to stick with Consolas for now.
• WD40 ALL THE THINGS - I’ve been using WD40 to fix lots of stiff or squeaky door handles. I don’t understand how, but it even helped a door that wasn’t closing properly. I aspire to own as many types of WD40 as Paul Mucur.

• Making it easier to rebase from Atom - This week Atom 1.43 was released. This was exciting for me because this included a change that I made in September that added ‘Copy Commit SHA’ and ‘Copy Commit Subject’ actions to Atom’s git tab. I use git’s auto-squashing a lot and thought this would make creating ‘fixup!’ and ‘squash!’ commits easier.
• Free-from Baking - I started the weekend with some gluten free waffles using my oven-top waffle maker and this recipe. I then made Nigella’s vegan chocolate cake which came out great and helped out with a spicy harissa aubergine pie for Sunday lunch. We made it vegan by substituting the feta with some ‘Greek white block’ which is the most utilitarian named foodstuff I’ve ever bought.
• My first day off - I switched to working 9 days in every 10 this year and had my first extra ‘day off’ this week. I had high hopes for how I would spend the time but came downstairs to a leaking boiler and spent most of the day finding an emergency plumber.
• Being a better manager - I’m reading Camille Fournier’s book The Manager’s Path. I enjoyed reading a section that contrasts the ‘imagined’ vs ‘real’ life of individual contributors and mangers which I think is a great way to show why these roles might not always meet peoples expectations. The book also gave me ideas of how to understand my reportees better so I know the best way to work with them. I set some questions for them in our 1:1s based off of the ones suggested by Lara Hogan.