<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
 <title>pitr.ca</title>
 <link href="http://pitr.ca/atom.xml" rel="self"/>
 <link href="http://pitr.ca/"/>
 <updated>2026-02-23T19:59:23+00:00</updated>
 <id>http://pitr.ca/</id>
 <author>
   <name>pitr</name>
 </author>

 
 <entry>
   <title>Solving LinkedIn Queens with APL</title>
   <link href="http://pitr.ca/2025-06-14-queens"/>
   <updated>2025-06-14T00:00:00+00:00</updated>
   <id>http://pitr.ca/queens</id>
   <content type="html">&lt;h1 id=&quot;solving-linkedin-queens-with-apl&quot;&gt;Solving LinkedIn Queens with APL&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;14 Jun 2025 on &lt;a href=&quot;/&quot;&gt;Peter Vernigorov’s blog&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A couple months ago I noticed that LinkedIn now has &lt;a href=&quot;https://www.linkedin.com/games/&quot;&gt;a few simple games&lt;/a&gt;. They’re not much to write home about, but I really enjoy playing Queens.&lt;/p&gt;

&lt;p&gt;This week I saw &lt;a href=&quot;https://ryanberger.me/posts/queens/&quot;&gt;two&lt;/a&gt; &lt;a href=&quot;https://buttondown.com/hillelwayne/archive/solving-linkedin-queens-with-smt/&quot;&gt;posts&lt;/a&gt; about solving the Queens game programmatically. Both were quite interesting to me, so I thought this was a good opportunity to also solve the game in my favourite language - APL - and share my experience. Having been &lt;a href=&quot;https://github.com/pitr/aoc&quot;&gt;using APL for Advent of Code&lt;/a&gt;, I wanted to share my passion for it with others.&lt;/p&gt;

&lt;h2 id=&quot;rules&quot;&gt;Rules&lt;/h2&gt;

&lt;p&gt;The game is pretty straightforward: each colored region must have exactly one queen, and two queens cannot occupy the same row, column, or be adjacent. Multiple queens may share diagonals as long as they’re separated by at least one space.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/images/queens.png&quot; alt=&quot;Queens game screenshot&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;board&quot;&gt;Board&lt;/h2&gt;

&lt;p&gt;First, let’s choose a data structure. LinkedIn sends the initial state as a list of rows, assigning each color a number. We will use 0 to mark queens. Let’s create a 2‑dimensional board representing the image:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;b←⍉⍪1 1 1 1 1 1 1 1
b⍪← 1 1 2 2 2 2 3 1
b⍪← 1 1 2 4 4 2 3 1
b⍪← 1 1 4 4 4 2 3 3
b⍪← 1 5 5 4 2 2 3 3
b⍪← 1 5 6 6 2 7 7 3
b⍪← 5 5 6 6 6 6 7 7
b⍪← 5 5 5 6 8 6 6 7
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And print it, by assigning it to the screen represented by a box:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;⎕ ← b

1 1 1 1 1 1 1 1
1 1 2 2 2 2 3 1
1 1 2 4 4 2 3 1
1 1 4 4 4 2 3 3
1 5 5 4 2 2 3 3
1 5 6 6 2 7 7 3
5 5 6 6 6 6 7 7
5 5 5 6 8 6 6 7
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;breadth-first-search&quot;&gt;Breadth-first Search&lt;/h2&gt;

&lt;p&gt;Next, algorithm choice: &lt;strong&gt;depth-first&lt;/strong&gt; vs &lt;strong&gt;breadth-first&lt;/strong&gt; search. While depth-first is a somewhat obvious choice (and works beautifully for N‑Queens, as seen in &lt;a href=&quot;https://youtu.be/DsZdfnlh_d0&quot;&gt;this video&lt;/a&gt;), I chose breadth-first search, inspired by a &lt;a href=&quot;https://youtu.be/DmT80OseAGs&quot;&gt;Sudoku solution video&lt;/a&gt; I love.&lt;/p&gt;

&lt;p&gt;In our case, the search tree depth equals the number of colors. Each step expands the current solution space by enumerating valid queen positions for a new color. Here’s a simplified mock-up (color order is chosen strategically):&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;initial state
 1 2 2 2 2
 3 4 3 2 3
 3 4 3 3 3
 3 3 3 5 5
 3 3 3 3 3

place all allowed queens for color 5
 1 2 2 2 2   1 2 2 2 2
 3 4 3 2 3   3 4 3 2 3
 3 4 3 3 3   3 4 3 3 3
 3 3 3 ♕ 5   3 3 3 5 ♕
 3 3 3 3 3   3 3 3 3 3

place all allowed queens for color 4
 1 2 2 2 2   1 2 2 2 2   1 2 2 2 2   1 2 2 2 2
 3 ♕ 3 2 3   3 4 3 2 3   3 ♕ 3 2 3   3 4 3 2 3
 3 4 3 3 3   3 ♕ 3 3 3   3 4 3 3 3   3 ♕ 3 3 3
 3 3 3 ♕ 5   3 3 3 ♕ 5   3 3 3 5 ♕   3 3 3 5 ♕
 3 3 3 3 3   3 3 3 3 3   3 3 3 3 3   3 3 3 3 3

place all allowed queens for color 1
notice that solution space is shrinking as some positions are invalid
 ♕ 2 2 2 2   ♕ 2 2 2 2
 3 4 3 2 3   3 4 3 2 3
 3 ♕ 3 3 3   3 ♕ 3 3 3
 3 3 3 ♕ 5   3 3 3 5 ♕
 3 3 3 3 3   3 3 3 3 3

place all allowed queens for color 2, at this point only a single board is valid
 ♕ 2 2 2 2
 3 4 3 ♕ 3
 3 ♕ 3 3 3
 3 3 3 5 ♕
 3 3 3 3 3

place all allowed queens for the last color 3
 ♕ 2 2 2 2
 3 4 3 ♕ 3
 3 ♕ 3 3 3
 3 3 3 5 ♕
 3 3 ♕ 3 3
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s imagine we created an infix function &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;fills&lt;/code&gt; that takes a color and current solution space, and produces a new solution space. We start with the board as an initial solution space, and fill color 5 with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;5 fills b&lt;/code&gt;. Since APL is executed right-to-left, we can then fill color 4 with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;4 fills 5 fills b&lt;/code&gt;. And so forth… the whole solution would be the result of &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;3 fills 2 fills 1 fills 4 fills 5 fills b&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;For those familiar with functional programming, this will look like a use case for a &lt;a href=&quot;https://en.wikipedia.org/wiki/Fold_(higher-order_function)&quot;&gt;fold&lt;/a&gt;, specifically foldr (right fold). In APL this is such a common operation that it is shortened to a single symbol - &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/&lt;/code&gt;. The above becomes &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;fills / 3 2 1 4 5 b&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So far, we’ve been designing the solution using mock output and pseudo code. However, the top-level function for our actual solution will look very similar to the code we used, by first building that list and then folding it. In fact, here it is:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;solve ← {0=⊃fills/(∪,⍵),⊂⊂⍵}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Let’s see how it works when built up slowly. Comments start with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;⍝&lt;/code&gt; symbol (a light bulb).&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;⍝ an anonymous function that just returns the argument
⍝ ⍵ is a symbol for the right argument
⎕ ← {⍵} b
 1 2 2 2 2
 3 4 3 2 3
 3 4 3 3 3
 3 3 3 5 5
 3 3 3 3 3

⍝ flatten the board
⎕ ← {,⍵} b
 1 2 2 2 2 3 4 3 2 3 3 4 3 3 3 3 3 3 5 5 3 3 3 3 3

⍝ unique list of colors
⎕ ← {∪,⍵} b
 1 2 3 4 5

⍝ board again
⎕ ← {⍵} b
 1 2 2 2 2
 3 4 3 2 3
 3 4 3 3 3
 3 3 3 5 5
 3 3 3 3 3

⍝ enclosed twice, making a solution space with one boxed element
⎕ ← {⊂⊂⍵} b
   1 2 2 2 2
   3 4 3 2 3
   3 4 3 3 3
   3 3 3 5 5
   3 3 3 3 3

⍝ all together now, create a list that we will fold over as explained earlier
⎕ ← {(∪,⍵),⊂⊂⍵} b
 1 2 3 4 5  1 2 2 2 2
            3 4 3 2 3
            3 4 3 3 3
            3 3 3 5 5
            3 3 3 3 3

⍝ actually fold using our fills function
⎕ ← {fills / (∪,⍵),⊂⊂⍵} b
   0 2 2 2 2
   3 4 3 0 3
   3 0 3 3 3
   3 3 3 5 0
   3 3 0 3 3

⍝ disclose twice
⎕ ← {⊃⊃fills/(∪,⍵),⊂⊂⍵} b
 0 2 2 2 2
 3 4 3 0 3
 3 0 3 3 3
 3 3 3 5 0
 3 3 0 3 3

⍝ pretty print the result
⎕ ← {&apos;♕1234&apos;[⍵]} {0=⊃⊃fills/(∪,⍵),⊂⊂⍵} b
 ♕ 2 2 2 2
 3 4 3 ♕ 3
 3 ♕ 3 3 3
 3 3 3 5 ♕
 3 3 ♕ 3 3
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;helpers&quot;&gt;Helpers&lt;/h2&gt;

&lt;p&gt;Now we are ready to actually implement the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;fills&lt;/code&gt; function. But first let’s define some helpers.&lt;/p&gt;

&lt;p&gt;The first helper is a &lt;strong&gt;place&lt;/strong&gt; function that places a 0 (queen) on the board. It’s called like this: &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;1 2 place b&lt;/code&gt;. We will use an &lt;a href=&quot;https://aplwiki.com/wiki/At&quot;&gt;At operator&lt;/a&gt;, represented by symbol &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;@&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;⍝ ⍺ is the left argument
place ← {0@(⊂⍺)⊢⍵}

⎕ ← 5 5⍴1
 1 1 1 1 1
 1 1 1 1 1
 1 1 1 1 1
 1 1 1 1 1
 1 1 1 1 1

⎕ ← 1 2 place 5 5⍴1
 1 1 1 1 1
 1 1 0 1 1
 1 1 1 1 1
 1 1 1 1 1
 1 1 1 1 1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Easy enough!&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Note: APL allows overriding &lt;a href=&quot;https://aplwiki.com/wiki/Index_origin&quot;&gt;index origin&lt;/a&gt;. The default is 1-based; however, the &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;place&lt;/code&gt; function doesn’t work there for surprisingly complicated reasons - it involved the behaviour of &lt;a href=&quot;https://aplwiki.com/wiki/Each&quot;&gt;Each&lt;/a&gt; with empty lists, &lt;a href=&quot;https://aplwiki.com/wiki/Prototype&quot;&gt;prototypes&lt;/a&gt;, and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;@&lt;/code&gt; primitive failing when asked to update index (0,0) in 1-based origin system. So this function assumes that the origin is setup with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;⎕IO ← 0&lt;/code&gt;. The rest of the code doesn’t care about index origin.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Now, let’s build the function - we will call it &lt;strong&gt;avl&lt;/strong&gt; - that returns valid queen positions for a specific color. We will call it with color and board as such - &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;4 avl b&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;avl ← {
    ⍝ find positions of all existing queens
    q ← ⍸0=⍵

    ⍝ find positions of a particular color
    p ← ⍸⍺=⍵

    ⍝ boolean mask over p that have row or column conflict with any existing queen
    rc ← ∨⌿1∊¨q∘.=p

    ⍝ boolean mask over p that are next to any existing queen
    nx ← ∨⌿∧/¨1≥|q∘.-p

    ⍝ select color positions that are covered by neither mask using a Nor.
    ⍝ here slash symbol is a select, not fold
    (rc⍱nx)/p
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Last line in this function implicitly returns the result - list of legal positions for a queen in the color region. I leave deeper understanding of this function as an exercise to the reader. For those unfamiliar with APL primitives, APL Wiki has great pages for each, such as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;⍸&lt;/code&gt; - &lt;a href=&quot;https://aplwiki.com/wiki/Indices&quot;&gt;indices function&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;And now we are ready to implement the &lt;strong&gt;fills&lt;/strong&gt; function. We will split it into two functions - &lt;strong&gt;fills&lt;/strong&gt; that works on a solution space and &lt;strong&gt;fill&lt;/strong&gt; that works on a single instance of a board. Left argument for both is always the color.&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;⍝ places a queen on each available position of a board
fill ← {(⍺ avl ⍵) place¨ ⊂⍵}

⍝ calls fill for each board, then merged the results
fills ← {⊃,/⍺ fill¨ ⍵}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;And that’s it!&lt;/p&gt;

&lt;h2 id=&quot;full-solution&quot;&gt;Full Solution&lt;/h2&gt;

&lt;p&gt;The whole solution is this:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;place ← {0@(⊂⍺)⊢⍵}
avl ← {
    q ← ⍸0=⍵
    p ← ⍸⍺=⍵
    rc ← ∨⌿1∊¨q∘.=p
    nx ← ∨⌿∧/¨1≥|q∘.-p
    (rc⍱nx)/p
}
fill ← {(⍺ avl ⍵) place¨ ⊂⍵}
fills ← {⊃,/⍺ fill¨ ⍵}
solve ← {⊃⊃fills/(∪,⍵),⊂⊂⍵}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;11 lines of code that have zero external dependencies, not even APL’s built-in libraries. Just the primitives. &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;avl&lt;/code&gt; can be minified to much fewer lines, but I see no reason to do that.&lt;/p&gt;

&lt;p&gt;There are heuristics that could be used, such as ordering colors by number of positions (fill smaller regions first). However, the current solution runs in a few milliseconds already. Still, there might be edge case boards where this solution doesn’t perform well, but I do not have many data points.&lt;/p&gt;

&lt;p&gt;Let’s solve the large example:&lt;/p&gt;

&lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;⎕ ← board
 1 1 1 1 1 1 1 1
 1 1 2 2 2 2 3 1
 1 1 2 4 4 2 3 1
 1 1 4 4 4 2 3 3
 1 5 5 4 2 2 3 3
 1 5 6 6 2 7 7 3
 5 5 6 6 6 6 7 7
 5 5 5 6 8 6 6 7

⎕ ← {&apos;♕.&apos;[×⍵]} solve board
 ♕.......
 .....♕..
 ...♕....
 .......♕
 .♕......
 ......♕.
 ..♕.....
 ....♕...
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;the-end&quot;&gt;The End&lt;/h2&gt;

&lt;p&gt;My hope is that this has piqued your interest in APL, at least from the perspective of expanding your programming toolbox. APL is a very powerful language that allows you to express complex algorithms in a very concise way. If you want to learn more, I recommend checking out &lt;a href=&quot;https://aplwiki.com&quot;&gt;APL Wiki&lt;/a&gt; and &lt;a href=&quot;https://tryapl.org&quot;&gt;TryAPL&lt;/a&gt;.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;Discussion: &lt;a href=&quot;https://lobste.rs/s/9gjyi0&quot;&gt;lobste.rs&lt;/a&gt; &lt;a href=&quot;https://news.ycombinator.com/item?id=44275900&quot;&gt;news.ycombinator.com&lt;/a&gt;&lt;/p&gt;

</content>
 </entry>
 
 <entry>
   <title>Contributing to Gemini ecosystem</title>
   <link href="http://pitr.ca/2021-05-29-gemini-ecosystem"/>
   <updated>2021-05-29T00:00:00+00:00</updated>
   <id>http://pitr.ca/gemini-ecosystem</id>
   <content type="html">&lt;h1 id=&quot;contributing-to-gemini-ecosystem&quot;&gt;Contributing to Gemini ecosystem&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;29 May 2021 on &lt;a href=&quot;/&quot;&gt;Peter Vernigorov’s blog&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;About a year ago I found out about &lt;a href=&quot;https://gemini.circumlunar.space/&quot;&gt;Gemini protocol&lt;/a&gt;. For those who aren’t familiar with it, it’s a new Internet protocol (which together with its accompanying gemtext format) aims to build an alternative “web, stripped right back to its essence”. It appealed to my love for minimalism and I built a few things for its ecosystem.&lt;/p&gt;

&lt;p&gt;Regardless of your views of Gemini, or familiarity, &lt;strong&gt;the greater lesson here is that even with a simple protocol, useful programs almost never are&lt;/strong&gt;. Similar messages are echoed by others such as &lt;a href=&quot;https://lord.io/text-editing-hates-you-too/&quot;&gt;Text Editing Hates You Too (2019)&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This post requires no knowledge of Gemini protocol specifications. Each section focuses on one project, explaining why and how it was built. Feel free to skip “How” sections as it can go deep into technical details.&lt;/p&gt;

&lt;h2 id=&quot;ios-browser&quot;&gt;iOS Browser&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What&lt;/strong&gt; &lt;a href=&quot;https://github.com/pitr/gemini-ios&quot;&gt;Elaho&lt;/a&gt; is a fully featured iOS browser with features people are used to, such as tabs, bookmarks, history, etc. This is the first project I worked on. While mostly a backend engineer by trade, I’ve delved into the client side of things quite a few times. But this was my first published iOS app.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/images/elaho.png&quot; alt=&quot;Elaho screenshots&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why&lt;/strong&gt; At the time, most clients for Gemini were terminal based, with only one or two focusing on Desktop GUI. I realized that more so than the web, Gemini is almost exclusively meant for consuming content. There is only one way to send information from the client, using something not unlike a one textbox form. Today, around 60% of content consumers online are on mobile/tablet devices, and it’s growing with every year &lt;a href=&quot;https://www.perficient.com/insights/research-hub/mobile-vs-desktop-usage&quot;&gt;[source]&lt;/a&gt;. Within those, while Android has a larger market share than iOS, I’ve chosen the latter as I have the most experience with it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How&lt;/strong&gt; Currently, Gemini-specific code accounts for 2% (1k of 41k) of lines of code. This includes Gemini protocol, certificate helpers, gemtext rendering and ansi escape codes. Most of the other code deals with UI for features such as tabs, session restoration, bookmarks, share extension etc. Foreseeing this, I decided to use an existing browser that already has most of this functionality included - &lt;a href=&quot;https://github.com/mozilla-mobile/firefox-ios&quot;&gt;Firefox for iOS&lt;/a&gt;. It is licensed under Mozilla Public License 2.0, which allows me to fork it as long as copyright and license notices are preserved. Of interest is &lt;a href=&quot;https://github.com/pitr/gemini-ios/commit/a3c3bc1e&quot;&gt;my first commit&lt;/a&gt;, which removed 30k lines worth of telemetry/tracking code: Adjust, Leanplum, Sentry.&lt;/p&gt;

&lt;p&gt;Most of the code is responsible for UI. While I could benefit from view layer code responsible for managing bookmarks, history, and others, the underlying storage layer which relied on Rust-based &lt;a href=&quot;https://github.com/mozilla/application-services&quot;&gt;Firefox Application Services&lt;/a&gt; had to be replaced with &lt;a href=&quot;https://realm.io/&quot;&gt;Realm&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Other mobile browsers for Gemini took a different route. &lt;a href=&quot;https://oppen.digital/software/ariane/&quot;&gt;Ariane&lt;/a&gt; and &lt;a href=&quot;https://oppen.digital/software/phaedra/&quot;&gt;Phaedra&lt;/a&gt; target Android devices, &lt;a href=&quot;https://github.com/snoe/deedum&quot;&gt;Deedum&lt;/a&gt; uses Flutter framework, &lt;a href=&quot;https://git.shadowfacts.net/shadowfacts/Gemini&quot;&gt;Rocketeer&lt;/a&gt; used SwiftUI, and &lt;a href=&quot;https://gmi.skyjake.fi/lagrange/&quot;&gt;Lagrange&lt;/a&gt; is built on a custom-made cross-platform framework written in C. All were built from scratch and (apart from Lagrange, author of which has put in an impressive amount of time into the project) do not support most browser features expected by users. Examples include things like session restoration, URL bar with autocomplete, custom keyboard and without spell check, and horizontal slide gesture to go back/forward. The reason is that the sheer amount of work needed to build all of this auxiliary functionality seems to result in most of these projects being abandoned by their authors.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; Since it’s been in the app store, I’ve seen about 20 weekly installs, and after releasing a new version I get about 3000 updates. It has been mentioned on &lt;a href=&quot;https://www.applevis.com/apps/ios/utilities/elaho&quot;&gt;AppleVis&lt;/a&gt; (online resource for blind and low vision users of Apple products).&lt;/p&gt;

&lt;h2 id=&quot;server-framework&quot;&gt;Server Framework&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What&lt;/strong&gt; &lt;a href=&quot;https://github.com/pitr/gig&quot;&gt;Gig&lt;/a&gt; is a Gemini framework written in Go that helps quickly build dynamic Gemini servers. It is very similar to &lt;a href=&quot;https://github.com/gin-gonic/gin&quot;&gt;Gin&lt;/a&gt;, &lt;a href=&quot;https://echo.labstack.com/&quot;&gt;Echo&lt;/a&gt; and other projects for HTTP servers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why&lt;/strong&gt; At the time, apart from a couple capsules online (search engine and a gardening experience game), all resources were static. Most servers at the time (and this is still true as of 2021) only supported serving static files. I had a few ideas for dynamic resources (see other projects below) and a simple framework would minimize work required later.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How&lt;/strong&gt; Borrowing from other Go frameworks, this project has a zero-allocation router, extensive documentation, lots of rendering helpers, a few builtin middlewares such as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Logger&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;Recover&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;ValidateHasCertificate&lt;/code&gt; and support for custom ones. While HTTP frameworks can rely on &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;net/http&lt;/code&gt; package, Gig needed to implement Gemini protocol specification. Each route handler gets a Context (an interface containing request/response helpers) similar to Gin and Echo, and after the request is done the Context is later reused thanks to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;sync.Pool&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;As I used it to build other projects, I was able to iterate on the framework itself. Context interface was improved, helpers to return an error to client were simplified, a middleware for login/password authentication was added.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; In the beginning Gig was primarily used by me. However, in the last few months a number of servers came up that imported it (see &lt;a href=&quot;https://github.com/pitr/gig#who-uses-gig&quot;&gt;Who uses Gig&lt;/a&gt;). I even found a &lt;a href=&quot;https://www.youtube.com/watch?v=lO51xCvZ3GI&quot;&gt;live stream on YouTube&lt;/a&gt; that features Gig being used to build a microblogging app.&lt;/p&gt;

&lt;h2 id=&quot;link-aggregator&quot;&gt;Link Aggregator&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What&lt;/strong&gt; &lt;a href=&quot;https://github.com/pitr/geddit&quot;&gt;Geddit&lt;/a&gt; is similar to Hacker News, Reddit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why&lt;/strong&gt; In late 2020 new capsules were popping up weekly if not daily, and some were usually posted to the mailing list. These emails were in-between other discussion threads, such as protocol finalization, protocol ideas, questions, etc. I came to a conclusion that a different mechanism to discover and share new capsules/pages was needed. Link aggregation services work relatively well for this in my experience, so I set out to build one for Gemini.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How&lt;/strong&gt; Thanks to Gig, it was quite simple to build using ModelViewController pattern. Controller layer, including CRUD for posts and comments, contains about &lt;a href=&quot;https://github.com/pitr/geddit/blob/master/main.go&quot;&gt;155 sloc&lt;/a&gt;. Model layer, built on GORM+SQLite, is another &lt;a href=&quot;https://github.com/pitr/geddit/blob/master/db/db.go&quot;&gt;100 sloc&lt;/a&gt;. Views use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;text/template&lt;/code&gt; format. The project compiles into a small static binary that can be rsync’ed to the server.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; As of May 2021 there were 160 links submitted and 250 comments posted. There are at least a few submissions a week, and it has been mentioned on the mailing list many times. I continue to check it myself every few days. Other similar aggregators and forum-like capsules went live but none have maintained a consistent usage by the community.&lt;/p&gt;

&lt;h2 id=&quot;wikipedia-proxy&quot;&gt;Wikipedia Proxy&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What&lt;/strong&gt; An interface to Wikipedia from Gemini, available at &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gemini://wp.glv.one&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why&lt;/strong&gt; Capsules are usually built around content and functionality. Most content is produced by authors, with blogs as a prime example. However, there is a lot of content already available elsewhere on the Internet, and Gemini could be a good interface to consume it. Wikipedia content is licensed under &lt;a href=&quot;https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License&quot;&gt;Creative Commons Attribution-ShareAlike 3.0 Unported License&lt;/a&gt; and it can be remixed (in this case, converted to Gemini text) and shared (distributed at wp.glv.one). Gemtext format is a simplified version of Markdown, so the challenge would be to simplify an already content-rich format. Another challenge is to parse the Wikitext format.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How&lt;/strong&gt; &lt;a href=&quot;https://github.com/pitr/wp&quot;&gt;Backend&lt;/a&gt; is built on my Gig framework, &lt;a href=&quot;cgt.name/pkg/go-mwclient&quot;&gt;go-mwclient&lt;/a&gt; for Wikipedia client, and &lt;a href=&quot;https://github.com/d4l3k/wikigopher&quot;&gt;github.com/d4l3k/wikigopher/wikitext&lt;/a&gt; for Wikitext parsing.&lt;/p&gt;

&lt;p&gt;Wikitext package is part of a similar project for gopher space. It contains a PEG grammar and a parser. It was extremely easy to incorporate this into my project. Once an AST (Abstract Syntax Tree) is generated, my code walks the tree and generates Gemtext.&lt;/p&gt;

&lt;p&gt;The first challenge was links in text. Gemtext allows text OR link per line, but not both. Wikipedia articles usually contain a LOT of inline links to related articles so it’s important that these links are kept close to the text. The solution I came up with was to list all links directly AFTER the paragraph, by remembering links as they are parsed and storing them in a Links structure. Once a paragraph break was detected, it was a matter of simply flushing those links and resetting.&lt;/p&gt;

&lt;div class=&quot;language-go highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;type&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;struct&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;buf&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;strings&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Builder&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;func&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;NewLinks&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;{}&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;func&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;f&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Add&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;href&lt;/span&gt; &lt;span class=&quot;kt&quot;&gt;string&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;buf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;WriteString&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;fmt&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Sprintf&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;=&amp;gt; %s %s&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\n&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;href&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;))&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;func&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;f&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;kt&quot;&gt;string&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;return&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;buf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;func&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;f&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;*&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Links&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;Reset&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;f&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;buf&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;Reset&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;()&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;img src=&quot;/images/wikipedia.png&quot; alt=&quot;Wikipedia screenshot&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Anything that is not a paragraph, heading text, list or link is skipped. This includes footnotes, tables, images, complex structures like {Infobox}, etc.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; While there is an active usage of it, I consider this project a failure. Simply looking at the amount of content that is skipped due to inability to represent it in Gemtext format renders it quite useless.&lt;/p&gt;

&lt;p&gt;But perhaps the bigger reason for its failure is the performance factor. This app constantly uses most of the memory and cpu on my server, which profiling showed is due to the parsing library. Latency is also quite large, especially for large pages, again due to slow parsing. Most articles take from a few seconds to more than a minute to load. There are &lt;a href=&quot;https://www.mediawiki.org/wiki/Alternative_parsers&quot;&gt;quite a few Wikitext parsers&lt;/a&gt;, but unfortunately none that are written in Go.&lt;/p&gt;

&lt;p&gt;Another Wikipedia Proxy project by &lt;a href=&quot;https://alexschroeder.ch/&quot;&gt;Alex Schroeder&lt;/a&gt; hosted at &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gemini://vault.transjovian.org/&lt;/code&gt;. It is built in Perl and uses &lt;a href=&quot;https://alexschroeder.ch/cgit/phoebe/tree/contrib/wikipedia.pl?id=c536e0fc9175e33a3b002953019f7d3763356d4b#n163&quot;&gt;a lot of regular expressions&lt;/a&gt; to convert Wikitext to Gemtext. In the end its latency is quite good.&lt;/p&gt;

&lt;p&gt;On a related note, a &lt;a href=&quot;https://portal.mozz.us/gemini/gempaper.strangled.net/mirrorlist/&quot;&gt;few other proxy capsules&lt;/a&gt; have come up for resources like Reddit, Hacker News, and newspapers such as Guardian, Deutsche Welle, CNN, NPR. Much of this content is licensed less permissively, and I think it does more damage than good to the Gemini community.&lt;/p&gt;

&lt;h2 id=&quot;paas&quot;&gt;PaaS&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What&lt;/strong&gt; glv.one is a Platform-as-a-Service, similar to Heroku. Deployment is done by pushing a new docker image to a private registry, and the administration interface is available over Gemini at &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gemini://glv.one&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why&lt;/strong&gt; Having built a few capsules using Gin, I wanted to be able to quickly create new ones. Deploying a new capsule with glv.one is as easy as clicking “New” in the admin interface, and then pushing the image to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;deploy.glv.one/$USER/$APP&lt;/code&gt;. I also saw a potential need in the community to have a place to publish dynamic capsules without paying for your own server. Free static hosting is offered by many capsules such as &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gemini.circumlunar.space&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;flounder.online&lt;/code&gt;, &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;gemlog.blue&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;srht.site&lt;/code&gt;, but there was no place to host dynamic content.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How&lt;/strong&gt; This is the only project that I did not open source, mostly because there isn’t much to open source. It is built by glueing different parts together.&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;
    &lt;p&gt;Docker images are pushed to a private &lt;a href=&quot;https://hub.docker.com/_/registry&quot;&gt;Docker Registry&lt;/a&gt;. Only requests from &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker login&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker push&lt;/code&gt; are allowed. Registry is password protected with &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;htpasswd&lt;/code&gt;, and has a webhook setup to notify a little Go server of pushes:&lt;/p&gt;

    &lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;auth:
  htpasswd:
    path: /.htpasswd
notifications:
  events:
    includereferences: true
  endpoints:
    - url: http://172.17.0.1:8080/event
      ignore:
        actions:
          - pull
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;The tiny Go server parses the webhook event, makes sure that the $USER pushed the new image to &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/$USER/$APP&lt;/code&gt;, an app that he owns. Then it runs &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker stop $APP &amp;amp;&amp;amp; docker rm $APP&lt;/code&gt; (to stop the previous version), and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker run -d --restart=always --name $APP $IMAGE&lt;/code&gt;. There are a few more arguments to limit cpu/memory resources and to mount a volume.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Each app gets its own local port, which is mapped to port 1965 in the container.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Nginx runs on port 1965 and streams requests to the correct capsule’s port based on the request’s &lt;a href=&quot;https://en.wikipedia.org/wiki/Server_Name_Indication&quot;&gt;SNI&lt;/a&gt; (Server Name Indication):&lt;/p&gt;

    &lt;div class=&quot;language-plaintext highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;stream {
  map $ssl_preread_server_name $upstream {
    include /etc/nginx/stream.d/*.conf;
  }

  server {
    listen      1965;
    proxy_pass  $upstream;
    ssl_preread on;
  }
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;    &lt;/div&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;When creating a new app, it simply needs to write &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;&quot;$APP.glv.one&quot; 127.0.0.1:$PORT;&lt;/code&gt; to a file in /etc/nginx/stream.d/ and request nginx to reload its configurations.&lt;/p&gt;
  &lt;/li&gt;
  &lt;li&gt;
    &lt;p&gt;Admin interface offers additional functionality such as viewing logs (by sending back &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;docker logs $APP&lt;/code&gt;) and deploying an old version (by sending a fake push event with that image ID).&lt;/p&gt;
  &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; While there was no community adoption (every new user must be manually whitelisted by emailing me, I received 0 emails), I have used GLV.One for all my apps. Despite being quite hacky (and probably not secure), it works surprisingly well.&lt;/p&gt;

&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;

&lt;p&gt;There were a few smaller projects I built in the Gemini universe that I won’t go into, as this article is long enough. Overall I enjoyed being part of the community and building projects that try to solve real problems.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;Discussion: &lt;a href=&quot;https://lobste.rs/s/rweq1d&quot;&gt;lobste.rs&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 
 <entry>
   <title>Spot instances for personal servers</title>
   <link href="http://pitr.ca/2021-05-23-personal-spot"/>
   <updated>2021-05-23T00:00:00+00:00</updated>
   <id>http://pitr.ca/personal-spot</id>
   <content type="html">&lt;h1 id=&quot;spot-instances-for-personal-servers&quot;&gt;Spot instances for personal servers&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;23 May 2021 on &lt;a href=&quot;/&quot;&gt;Peter Vernigorov’s blog&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;h2 id=&quot;2023-update&quot;&gt;2023 Update&lt;/h2&gt;

&lt;p&gt;I have since moved from AWS to Hetzner as it is significantly cheaper than spot instances on AWS, at least for the workload that I have. 2 instances (&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;t3.nano&lt;/code&gt; and &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;t3.medium&lt;/code&gt;) with an EBS volumes were replaced by a single &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;CAX11&lt;/code&gt; (2x arm64, 4GB RAM).&lt;/p&gt;

&lt;h2 id=&quot;summary&quot;&gt;Summary&lt;/h2&gt;

&lt;p&gt;Use spot instances for personal projects to save ~70% costs, and spot.io to ensure their stability.&lt;/p&gt;

&lt;h2 id=&quot;why&quot;&gt;Why?&lt;/h2&gt;

&lt;p&gt;There was a recent post on lobste.rs asking &lt;a href=&quot;https://lobste.rs/s/p4edt5/what_are_you_self_hosting_2021&quot;&gt;“What are you self hosting?”&lt;/a&gt; and while some people prefer to host as little as possible relying on various services, many are running their own servers.&lt;/p&gt;

&lt;p&gt;There are quite a few cheap/free options for compute instances. Some are listed &lt;a href=&quot;https://github.com/ripienaar/free-for-dev#major-cloud-providers&quot;&gt;here&lt;/a&gt;. However, if one would like to stay on AWS, beyond 1 year of free 750 hours/month of micro instances, there isn’t much that can be done to save costs.&lt;/p&gt;

&lt;p&gt;In this post I describe my setup, that keeps the costs and maintenance as low as possible.&lt;/p&gt;

&lt;h2 id=&quot;minimize-complexity&quot;&gt;Minimize complexity&lt;/h2&gt;

&lt;p&gt;While orchestration systems like Kubernetes can and do simplify the workflow, thereby improving delivery time and other KPIs, &lt;em&gt;they come at a cost&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;My employer has been an &lt;a href=&quot;https://github.com/zalando-incubator/kubernetes-on-aws&quot;&gt;early adopter of Kubernetes&lt;/a&gt;, and now it is as easy to deploy a service as simply pushing to a branch. But it comes at non-zero maintenance costs. There are multiple teams responsible for maintaining and updating Kubernetes clusters, running CI/CD tools, documenting best practices, supporting teams in their work, etc. &lt;a href=&quot;https://srcco.de/posts/how-zalando-manages-140-kubernetes-clusters.html&quot;&gt;See a good write up here&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;For a personal project, unless the goal is to acquire experience with Kubernetes, it is usually a huge overkill both in terms of overhead and maintenance costs.&lt;/p&gt;

&lt;p&gt;I run Amazon Linux 2 distributions, heavily utilizing systemd to run everything.&lt;/p&gt;

&lt;h2 id=&quot;spot&quot;&gt;Spot&lt;/h2&gt;

&lt;p&gt;Every personal server I have runs on a spot instance.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/images/spot.png&quot; alt=&quot;Spot.io dashboard&quot; /&gt;&lt;/p&gt;

&lt;p&gt;Spot instances are unstable by design, and can go down any time. However, in practice I have seen very few terminations. My longest uptime has been above &lt;em&gt;300 days&lt;/em&gt; (in region eu-west-1), and the majority of interruptions was me restarting them for various reasons.&lt;/p&gt;

&lt;p&gt;Running on spot trains one to expect terminations. After all, any instance, be it spot, on-demand, or even a server in your house, can be restarted or terminated at any time.&lt;/p&gt;

&lt;p&gt;Additional benefit of being prepared for terminations is to be able to run instances only when needed. I have a beefy instance dedicated to hosting a Minecraft server, and it’s only up when I need it. If I forget to turn it off, it automatically goes down at midnight.&lt;/p&gt;

&lt;p&gt;This is where &lt;a href=&quot;https://spot.io/&quot;&gt;spot.io&lt;/a&gt; comes in. They help manage spot instances by foreseeing terminations based on spot market shortages, shutting the instance down safely, finding an instance type and an availability zone with better score, and starting your instance up again. They can manage 20 instances for free, which is enough for most people.&lt;/p&gt;

&lt;p&gt;Each instance type in every availability zone gets a score. In case of a termination, spot.io will use a better scoring instance/AZ to boot the instance back up. I use &lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;t(2,3,3a).(micro,small)&lt;/code&gt; instances. In case of low scores across all instance types/AZs, there is an option to automatically go to on-demand until spot market improves.&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/images/spot-market.png&quot; alt=&quot;Spot Market Scoring&quot; /&gt;&lt;/p&gt;

&lt;p&gt;In my experience, instance recycling takes about 5 minutes, which even with a daily termination would be sufficient for most personal project.&lt;/p&gt;

&lt;h2 id=&quot;root-volume-persistence&quot;&gt;Root Volume persistence&lt;/h2&gt;

&lt;p&gt;The killer feature for me is root volume persistence. My instances do not have any additional storage apart from the root EBS volume, which contains OS, all packages, and my applications. Usually, the root volume is deleted on termination, but spot.io creates an AMI from the root volume. So next time my instance boots up using that AMI, it will have all the same content.&lt;/p&gt;

&lt;h2 id=&quot;public-ip&quot;&gt;Public IP&lt;/h2&gt;

&lt;p&gt;There are two options for instance’s public IP. For always on instances I use Elastic IP, which is free as long as the IP is attached to an running instance. And for instances that I only bring up when needed, I use an auto-assigned IP with a simple startup script that updates DNS. For Cloudflare I use the following:&lt;/p&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/home/ec2-user/update-dns.sh&lt;/code&gt; (replace ZONE_ID, DNS_ID and API_TOKEN with correct values from the dashboard)&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;#!/bin/bash&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;IP&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;curl &lt;span class=&quot;nt&quot;&gt;-s&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;http://169.254.169.254/latest/meta-data/public-ipv4/&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;

curl &lt;span class=&quot;nt&quot;&gt;-X&lt;/span&gt; PATCH &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;https://api.cloudflare.com/client/v4/zones/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$ZONE_ID&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/dns_records/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$DNS_ID&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;nt&quot;&gt;-H&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Authorization: Bearer &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$API_TOKEN&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-H&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Content-Type:application/json&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;cat&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;EOF&lt;/span&gt;&lt;span class=&quot;sh&quot;&gt;
{&quot;content&quot;:&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$IP&lt;/span&gt;&lt;span class=&quot;sh&quot;&gt;&quot;}
&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;EOF
&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;code class=&quot;language-plaintext highlighter-rouge&quot;&gt;/usr/lib/systemd/system/update-dns.service&lt;/code&gt;&lt;/p&gt;

&lt;div class=&quot;language-ini highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nn&quot;&gt;[Unit]&lt;/span&gt;
&lt;span class=&quot;py&quot;&gt;After&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;network.service&lt;/span&gt;

&lt;span class=&quot;nn&quot;&gt;[Service]&lt;/span&gt;
&lt;span class=&quot;py&quot;&gt;ExecStart&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;/home/ec2-user/update-dns.sh&lt;/span&gt;
&lt;span class=&quot;py&quot;&gt;User&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;ec2-user&lt;/span&gt;
&lt;span class=&quot;py&quot;&gt;Group&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;ec2-user&lt;/span&gt;

&lt;span class=&quot;nn&quot;&gt;[Install]&lt;/span&gt;
&lt;span class=&quot;py&quot;&gt;WantedBy&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s&quot;&gt;default.target&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;non-24x7-instances&quot;&gt;Non-24x7 instances&lt;/h2&gt;

&lt;p&gt;For managing instances that are up only when needed, I have 2 shell functions:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;# Fish shell syntax, but can be easily changed to support bash/zsh&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;function &lt;/span&gt;spotstart
  curl &lt;span class=&quot;nt&quot;&gt;-X&lt;/span&gt; PUT &lt;span class=&quot;nt&quot;&gt;-s&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;nt&quot;&gt;-H&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Authorization: Bearer &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$API_TOKEN&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;https://api.spotinst.io/aws/ec2/managedInstance/smi-&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$INSTANCE&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/resume?accountId=act-&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$ACCOUNT_ID&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
end

&lt;span class=&quot;k&quot;&gt;function &lt;/span&gt;spotstop
  curl &lt;span class=&quot;nt&quot;&gt;-X&lt;/span&gt; PUT &lt;span class=&quot;nt&quot;&gt;-s&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;nt&quot;&gt;-H&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Authorization: Bearer &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$API_TOKEN&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;https://api.spotinst.io/aws/ec2/managedInstance/smi-&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$INSTANCE&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/pause?accountId=act-&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$ACCOUNT_ID&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
end
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Alternatively, instance can be setup to be up only during specific days/hours:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;/images/spot-cron.png&quot; alt=&quot;Run instance between 9-5 every weekday&quot; /&gt;&lt;/p&gt;

&lt;h2 id=&quot;fine-print&quot;&gt;Fine print&lt;/h2&gt;

&lt;p&gt;I am not affiliated, associated, authorized, endorsed by, or in any way officially connected with spot.io, apart from using their services.&lt;/p&gt;

&lt;p&gt;There are small additional costs associated with using spot.io:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;CloudWatch - spot.io collects data about the health of an instance using CloudWatch, and those API requests add up. With 2 instances, over a month I see about 6 thousand API requests (which are below the 1 million free requests) and 200 thousand metrics requested using GetMetricData API, which costs about $2/month.&lt;/li&gt;
  &lt;li&gt;EBS snapshots - spot.io periodically creates a snapshot of EBS root volumes, which ends up costing additional ~50% of their normal cost. For 2 8GB root volumes this ends up being ~$0.87 in my region.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As can be seen, this is nowhere near close to savings from one instance alone, but worth being aware of.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;Discussion: &lt;a href=&quot;https://lobste.rs/s/ix7ozd&quot;&gt;lobste.rs&lt;/a&gt;&lt;/p&gt;
</content>
 </entry>
 

</feed>