TL;DR:

Put that in your nginx config:

location / {
  # needed to still allow git clone from http/https URLs
  if ($http_user_agent ~* "git/|git-lfs/") {
        set $bypass_cookie 1;
  }
  # If we see the expected cookie; we could also bypass the blocker page
  if ($cookie_Yogsototh_opens_the_door = "1") {
        set $bypass_cookie 1;
  }
  # Redirect to 418 if neither condition is met
  if ($bypass_cookie != 1) {
     add_header Content-Type text/html always;
     return 418 '<script>document.cookie = "Yogsototh_opens_the_door=1; Path=/;"; window.location.reload();</script>';
  }
  # rest of your nginx config

Preferably run a string replace from Yogsototh_opens_the_door to your own personal Cookie name.

Main advantage, is that it is almost invisible to the users of my website compartively to other solutions like Anubis.

More detail

Not so long ago I started to host my code to forgejo. There is a promise that in the future it will support federation and forgejo is the same project that is used for codeberg.

The only problem I had was one day, I discovered that my entire node was down. At first I didn't investigate and just restarted the node. But soon after a few hours, it was down again. Looking at the reason, clearly thousands of requests that looked at every commit which put too much pressure on the system. Who could be so interested in using the web API to look at every commit instead, of… you know, clone the repository locally and explore it. Quickly, yep, like so many of you, I discovered that tons of crawlers that did not respect the robots.txt are crawling my forgejo instance until death ensues.

So I had no choice, I first used a radical approach and blocked my website entirely except from me. But hey, why having a public forge if not for people to be able to look into it time to time?

I then installed Anubis, but it wasn't really for me. It is way too heavy for my needs, not as easy as I would have hoped to configure and install.

Then I saw this article You don't need anubis on lobste.rs using a simple configuration in caddy that should block these pesky crawlers. I made some adjustments to adapt it to nginx. For now, this is working perfectly well, my users are just redirected once, without really noticing it. And they could use forgejo as they could before. And this puts the crawlers away.

The strategy is pretty basic; in fact, a lot less advanced than the strategy adopted by Anubis. For every access of my website, I just check if the user has a specific cookie set. If not, I redirect the user to a 418 HTML page containing some js code to execute that set this specific cookie and reload the page.

That's it.

I also tried to return a 302 and add a cookie from the response without javascript, but the crawlers are immune to that second strategy. Unfortunately this means, my website could only be seen if you enable javascript in your browser. I feel this is acceptable. I guess, someday this very basic protection will not be enough and my forgejo instance will break again, and I will be forced to use more advanced system like Anubis or perhaps even iocaine.

I hope this could be helpful, because, I recently saw many discussions on that subject where people were not totally happy to use Anubis, while at least for me, this quick dirty fix does the trick. And I am fully aware that this would be very easy to bypass. But for now, I think the volume is more important than the quality for these crawlers and it may take a while for them to need to adapt. Also, by publishing this, I know if too many people use the same trick, quickly, these crawlers will adapt.

Introduction

My past encounters with LLMs were far from impressive. So, what made me decide to give them another chance?

• I saw a GitHub repository showing a full OAuth2 Provider built using Claude. One commit per prompt. I didn't follow all commits, but it was nice to see that this was possible.
• Clojure-MCP ; I didn't knew at that time precisely how that could be a game changer, but it is.
• Heard good things about Claude for programming and I saw a few web app made from just a few directives.

The Setup

Setting things up was not straightforward. But once done I must admit I was really impressed.

I feel AI Agent are now like Versioning System (git) and Build Systems (make). These are tools worthy of learning. But from my experience they only provide value if you have a full setup:

• Agent: The AI needs to work on its own, handling several connected tasks without asking for your input every time.
• Clojure-MCP; it makes these tools way better, even if you code in another language.
• RAG; the system must support file/buffer context to add in order to be able to follow your project best practices.

See my previous article to see all details of my current setup.

Productivity Gain

(--) Hard/Advanced Programming

First, let's make it clear. For real, production-ready, quality development or complex tasks with advanced requirements these tools are a NET NEGATIVE.

If the task at hand is not small in scope, with a clear specification the system will act as a confidently wrong junior developer. In the end, most your time will be lost.

Worse than that, discovering the problems generated by the tools will not be a trivial task, and in the end result in a lot more time loss than if you would simply not have used any AI Agent.

So be conscious about the difficulty of the task at hand.

(+) Boilerplate

For writing boilerplate code following your project's best practices I would say my experience was positive. Some subtle errors could appear, and this was not great. Still, it is faster to use the AI for the boilerplate than typing it yourself most of the time.

(++) Small disposable scripts that don't need to be reliable

If you need to write a script that don't need to be highly reliable. In that case, I can say my experience was a BIG NET POSITIVE. It worked because the tasks to write this script was:

• small in scope
• clear technical design

It built he tool for me in a few seconds which would have taken me more time just typing it.

An important remark however, is that if you know what would be the "Best tool for the work", this would also tremendously improve the speed and quality of the result.

I asked the Agent to build the same tool twice. First in bash, second using babashka. And for my specific need, not only it was a lot faster to build the tool using babashka, but the result was clearly superior both in design and quality as well as efficiency.

(+) PR Reviewing

I think a nice use case of AI Assistant is to ask it to generate a review of your PR locally prior to send it to show it to your team.

• about 10% is plain wrong and MUST be ignored
• 70% not useful, but could be ignored,
• 20% are good recommendations.

While 20% is not much, this is still a net positive. Personally I feel that using AI just for review would already be a good reason to add them to your tool-belt.

(++) Asking specific technical details about your own project

Using RAG + Clojure-MCP to search and discover the details of your Project, this ended up being very useful.

I work on a huge base code. We created a full platform from scratch that contains hundreds of technical details. Some of them are long forgotten in some documentation, and some were not even cleanly documented due to time pressure during the delivery.

Still I could ask these very specific questions and got a clear answer. Of course, as you should never trust the system fully, I asked for the source code and documentation to be able to verify the answer. And it worked perfectly.

Typically I could ask "What is the HTTP Header to use for this specific feature?" and it looks at the code and returns the value with a small documentation and pointer to the sources.

(+) Writing doc

While everything must be proofread because you cannot fully trust the Agent, this generally gives a good first pass overview and can generate a documentation of mediocre quality. If you want a good documentation it will need a lot more time. But I still give a positive outcome as existing and bad documentation is still a net positive comparatively to no documentation.

(-) Writing tests

In my experience, asking these agent to write tests can either be a net positive or a full net negative were I had to delete all generated tests. So I wouldn't recommend using AI code generation for writing advanced tests cases. But I feel this is fine for very specific small functions. And even there, you really need to be cautious about passing tests that are not useful at all and just give a false sense of safety.

Sometime it is just better to write the tests, at least the cases but not the boilerplate, yourself to be sure you are testing the correct thing.

(++) Familiarize with foreign code

The ability to generate a PROJECT_SUMMARY.md as suggested by the Clojure-MCP tutorial, proves that these tool can really help grasp a new project. It's structure, the generic ideas, etc…

This is not totally perfect, but I got impressive results, so this count as NET POSITIVE.

Conclusions

With this new tryout of the AI tools I could discover tasks for which they are useful. In terms of productivity, I believe their benefits depend tremendously on the type of work you assign them. I feel they have a place as a daily-use tool, particularly for pre-reviewing your code and helping you write documents.

To be effective you need to:

1. Use something at least as efficient as Claude 4 Sonnet; so an Agent with Clojure-MCP and RAG support.
2. Be conscious about the scope and complexity of the task
3. Be able to provide precise technical specification of the task
4. Always be conscious (evaluate and monitor) the time it would take to complete the task manually versus the time spent supervising the automated tool.

If you familiarize yourself with these agents, really try Clojure-MCP. It is a game changer and that provide a pretty huge advantage to Clojure comparatively to other programming languages.

Introduction

By the end of this article, you'll be able to give instructions to the AI Assistant and it will use the REPL of your specific project to execute them. For example, I asked the assistant to list all users in my project's database (running locally, of course). With my setup, the assistant could:

• analyze the project (source and document files)
• discover the service and functions that could perform the operation to list all users
• ran the code in the repl, and got back all users
• used this answer to group the users (by org which makes sense for my project)
• rendered the result as an org-mode file.

I ran into a few bumps along the way since this setup uses some pretty cutting-edge tools and libraries. It took me a bit of time to work through these issues, which is exactly why I'm putting this article together - partly to help others who might face the same challenges, and partly as a reference for my future self.

If you follow this guide you should endup with a fully working setup with: Emacs + AI Assitant + Clojure-MCP development environment.

• Platform: macOS
• Editor: Emacs Plus 31 with Doom Emacs
• Emacs packages:
  • gptel for LLM interaction
  • mcp (Model Context Protocol) support
• Development Stack:
  • clojure-mcp server
  • Custom MCP startup scripts for project integration

Steps

Emacs

Install latest emacs-plus

brew install emacs-plus --with-native-comp --with-modern-black-dragon-icon

1. emacs-plus and native-comp shenanigans

   Currently to fix emacs-plus from brew on macOS native-comp, add this to the beginning of your ~/.emacs.d/early-init.el.

   ;; FIX NATIVE COMP: https://github.com/d12frosted/homebrew-emacs-plus/issues/554#issuecomment-2909894593
   (setenv "LIBRARY_PATH"
           (mapconcat 'identity
                      '("/opt/homebrew/opt/gcc/lib/gcc/15"
                        "/opt/homebrew/opt/libgccjit/lib/gcc/15"
                        "/opt/homebrew/opt/gcc/lib/gcc/15/gcc/aarch64-apple-darwin24/15")
                      ":"))

doom-emacs

As I started from an already existing doom-emacs installation. I will not cover that part. Please refer to the doom-emacs installation project documentation if you want to install it .

1. Declare the new packages to use

   In init.el:

   ...
   :tools
     ...
     llm ;; <- uncomment this line
     ...
   

   In ~/.doom.d/packages.el:

   (package! mcp
       :pin "3a380185597f56290057f370ab65dde89a20a18f")
   

   Then configure the packages:

   In ~/.doom.d/config.el:

   ;; LLM
   (use-package! gptel
     :init
     ;; declare the default llm configuration
     ;; for me this is co-pilot with claude-sonnet-4
     (setq gptel-default-mode 'org-mode ;; I want the llm to reply to me with org-mode format.
           gptel-backend (gptel-make-gh-copilot "Copilot")
           gptel-model 'claude-sonnet-4)
     (require 'gptel-integrations) ;; <- for mcp
     ;; declare another llm configuration, I also use GPT4All to consume locally only
     (gptel-make-gpt4all "GPT4All"
                         :protocol "http"
                         :host "localhost:4891"
                         ;; "~/Library/Application Support/nomic.ai/GPT4All"
                         :models '(Meta-Llama-3-8B-Instruct.Q4_0.gguf)))
   
   (use-package! mcp
     :after gptel
     :config (require 'mcp-hub)
     :init (setq mcp-hub-servers
                 ;; this is the important block here
                 ;; it tells that it can launch an MCP server
                 ;; for my-project by running the following
                 ;; command line:
                 ;;    clojure-mcp my-project
                 '(("my-project" :command "clojure-mcp" :args ("my-project"))))
     :hook (after-init . mcp-hub-start-all-server))
   

2. Rebuild / Install doom-emacs

   After using another emacs version you need to rebuild your emacs packages.

   doom sync -u
   doom rebuild

   This will take a while.

Clojure-MCP

Update your global Clojure aliases

The file ~/.clojure/deps.edn:

{:aliases
 {:mcp
  {:deps {org.slf4j/slf4j-nop {:mvn/version "2.0.16"} ;; Required for stdio server
          com.bhauman/clojure-mcp {:git/url "https://github.com/bhauman/clojure-mcp.git"
                                   :git/tag "v0.1.4-alpha"
                                   :git/sha "080b29b"}}
   :exec-fn clojure-mcp.main/start-mcp-server
   :exec-args {:port 7888}}}
 }

Create the command line to launch the Clojure MCP Server

#!/usr/bin/env bash

case $1 in
    my-project) PORT=7888;;
    *) echo "I don't know how to start this project ($1)" >&2
       exit 1;
esac

clojure -X:mcp --port $PORT

Link Clojure MCP to your Clojure project

It is important to perform the operation in the correct order. So FIRST go to your project and start the repl. Personally I work on a very big professional Clojure project that started even before deps.edn was thing in Clojure. So I could start the repl with a lein command:

lein repl :start :port 7888

Got it! Here's your project setup in a friendlier tone:

Since my project is pretty large and has its own restart helpers, I don't use the headless mode that the clojure-mcp docs suggest. Instead, I fire up the project with :start mode so I can manually load all the namespaces and get everything running with a the (reset!) command.

Once my project is running (after reset!) ONLY THEN I start the Clojure mcp server.

I could test that by running my clojure-mcp my-project command, and I get back a list of JSON objects lines:

> clojure-mcp my-project
{"jsonrpc":"2.0","method":"notifications/tools/list_changed"}
...
{"jsonrpc":"2.0","method":"notifications/resources/list_changed"}
...
{"jsonrpc":"2.0","method":"notifications/prompts/list_changed"}

So this is a good sign.

Using it

Now, everything should be ready to use.

After starting my repl, I launch emacs (if emacs was already started, you probably need to restart your mcp server)

Once emacs is started, open gptel menu (SPC o l m). Try to add tools (if they are not there ; type M+ and start the mcp if not started)

Once done, here you go.

SPC o l l => will create a new buffer to send orders to the LLM. I started with:

Analyze the codebase structure, Document key files, dependencies and available tools. Generate or update a comprehensive documentation in a format optimized for LLM Assistants. Save this documentation in the file PROJECT_SUMMARY.md.

This should generate this PROJECT_SUMMARY.md file and you should generally always add this file to your LLM session context when working on your project.

Now, to show how powerful this setup is, I could ask:

> Can you list all the users in the database?

And the AI Assistant (it was claude) was able to analyse the project, discover the service to use, call the correct methods of this service to list all users in my local environment DB. The result was then rendered as a nice org-mode file where users were grouped by Org. This was even better that what I could expect.

I could als ask how he could perform this operation, and the assistant returned the commands he exectued in the repl.

When I searched how to generate pictures in org-mode on the Internet, Metapost is surprisingly missing.

If you never tried Metapost, I think you should really take a look at this gem.

This article will simply provide an example how I could create an emacs package (not yet distributed) to use Metapost in my org-mode documents.

The script

One of the main issue with Metapost is that, like $L^AT_EX$, it generates a bunch of temporary files and clutter your current directory with them. There is also a notion that Metapost is done to generate not a single, but multiple files.

For our use case, we just want a program to generate a single image.

This is why I wrote a very simple mp2png script. It is inspired by mp2png.sh I could find in bsdowl.

In short, this script move a Metapost single figure code into a temporary directory. We prepare enclose the figure description into a Metapost template. We perform the compilation in that temporary directory.

If you remove all the ceremony this is simply:

# Generate a temporary directory
tmpdir="$(mktemp -d)"
# Name the temporary metapost
tmpmp="${tmpdir}/tmp.mp"
# Provide a prelude
cat > "$tmpmp" <<EOF
prologues:=3;
outputtemplate:="tmp-1.eps";
EOF
# copy the content of your metapost code
cat "$src" >> "$tmpmp"
# close the metapost file
cat >> "$tmpmp" <<EOF
end.
EOF
# Move to the temporary directory
pushd "$tmpdir" || err "ERROR: pushd \"${tmpdir}\""
# Compile the metapost file
mpost "tmp.mp"
# Convert the EPS file to PNG
gm convert -density "$density" "tmp-1.eps" "tmp-1.png"
# Return to the original directory
popd || err "ERROR: popd (tmpdir= \"${tmpdir}\")"

With all the details this become a bit more complex because I would like to be able to pass the code from stdin as this will ease to write the org babel code. Here is the full script:

#!/usr/bin/env zsh

# dependencies:
# - metapost
# - graphicsmagick

set -e

cmd=${0:t}
showhelp() {
    echo "$cmd SRC [DEST]"
    echo ""
    echo "SRC could be any file if - is provided read from stdin"
    echo "    if - is provided the default file name will be result.png"
    echo "DEST could be any file if - is provided read from stdin"
}

if (( $# == 0 )); then
    showhelp
fi

src="$1"
density=1200
tmpdir="$(mktemp -d)"
if (( $# > 1 )); then
    maindst="$2"
elif [[ "$src" = "-" ]]; then
    maindst="result.png"
else
    maindst="${src%.*}.png"
fi

err() {
    local msg="$1"
    echo "$msg" >&2
    exit 1
}

dst="$maindst"
# i=0
# while [[ -e $dst ]]; do
#     dst="${maindst%.*}-$((++i)).png"
# done

tmpmp="${tmpdir}/tmp.mp"
cat > "$tmpmp" <<EOF
prologues:=3;
outputtemplate:="tmp-1.eps";
% TEX macro is short enough to be copied
string preverbatimtex_, postverbatimtex_;
vardef TEXPRE text s =  preverbatimtex_ := s; enddef;
vardef TEXPOST text s =  postverbatimtex_ := s; enddef;
vardef TEX primary s =
  if known preverbatimtex_:
    write "verbatimtex "&preverbatimtex_&" etex" to "mptextmp.mp";
  fi
  write "btex "&s&" etex" to "mptextmp.mp";
  if known postverbatimtex_:
    write "verbatimtex "&postverbatimtex_&" etex" to "mptextmp.mp";
  fi
  write EOF to "mptextmp.mp";
  scantokens "input mptextmp"
enddef;
beginfig(1);
EOF
if [[ "$src" = "-" ]]; then
    cat >> "$tmpmp"
else
    cat "$src" >> "$tmpmp"
fi
cat >> "$tmpmp" <<EOF
endfig;
end.
EOF
pushd "$tmpdir" || err "ERROR: pushd \"${tmpdir}\""
mpost "tmp.mp"
gm convert -density "$density" "tmp-1.eps" "tmp-1.png"
popd || err "ERROR: popd (tmpdir= \"${tmpdir}\")"
cp "${tmpdir}/tmp-1.png" "$dst"
echo "File: $dst"

And now for the code in emacs, inspired by ob-ditaa:

;;; ob-metapost.el --- Babel Functions for metapost        -*- lexical-binding: t; -*-

;; Copyright (C) 2024 Free Software Foundation, Inc.

;; Author: Yann Esposito
;; Keywords: literate programming, reproducible research
;; Homepage: https://orgmode.org

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; Org-Babel support for evaluating metapost source code.
;;
;; This differs from most standard languages in that
;;
;; 1) there is no such thing as a "session" in metapost
;;
;; 2) we are generally only going to return results of type "file"
;;
;; 3) we are adding the "file" and "cmdline" header arguments
;;
;; 4) there are no variables (at least for now)

;;; Code:
(require 'ob)
(require 'org-compat)

(defvar org-babel-default-header-args:metapost
  '((:results . "file")
    (:exports . "results"))
  "Default arguments for evaluating a metapost source block.")

(defcustom org-babel-metapost-cmd "mp2png"
  "Metapost to png command"
  :group 'org-babel
  :type 'string)

(defun org-babel-execute:metapost (body params)
  "Execute a block of Metapost code with org-babel.
This function is called by `org-babel-execute-src-block'."
  (let* ((out-file (or (cdr (assq :file params))
               (error "Metapost code block requires :file header argument")))
     (cmdline (cdr (assq :cmdline params)))
     (in-file (org-babel-temp-file "metapost-"))
     (cmd (concat org-babel-metapost-cmd
                      " " cmdline
              " " (org-babel-process-file-name in-file)
              " " (org-babel-process-file-name out-file))))
    (with-temp-file in-file (insert body))
    (message cmd)
    (shell-command cmd)
    nil)) ;; signal that output has already been written to file

(defun org-babel-prep-session:metapost (_session _params)
  "Return an error because metapost does not support sessions."
  (error "Metapost does not support sessions"))

(provide 'ob-metapost)

;;; ob-metapost.el ends here

And that's it. That was a lot easier than I expected. And now, I can generate image like this:

u := 0.2cm;
z0 = (0,0);
z1 = (sqrt(3)*cm,0);
z2 = (sqrt(3)*cm,1cm);
draw z0--z1--z2--cycle;
label.bot(btex $\sqrt{3}$ etex, 1/2[z0,z1]);
label.rt(btex 1 etex, 1/2[z1,z2]);
label.top(btex 2 etex, 1/2[z0,z2]);
z3 = z1 - (u,0);
z4 = z1 + (-u,u);
z5 = z1 + (0,u);
draw z3--z4--z5;

or this:

numeric r, theta; r = sqrt 1/2; theta = 45;
vardef dragon(expr a, b) =
  if abs(a-b) > 8:
    save p; pair p;
    p = r[a, b] rotatedabout(a, theta);
    dragon(a, p) & reverse dragon(b, p)
  else:
    a .. b
  fi
enddef;
vardef rounded_corners expr p =
  save r, n; numeric r, n; r = 1/4; n = length p;
  subpath (0, 1-r) of p
  for t=1 upto n-1:
    .. subpath (t+r, t+1-r) of p
  endfor .. subpath (n-r, n) of p
enddef;
draw rounded_corners dragon(origin,240 right) withcolor (.6,.2,.7);

I hope it will help people to find the resources to write org babel integrations as well as help people use Metapost more.

AImmortal

As entrepreneurs, we're always on the lookout for innovative ways to stay ahead of the curve and drive growth in our businesses. For me, that journey began when I was approached by a vendor offering their cutting-edge AI platform.

At first, I was skeptical – after all, I'd heard horror stories about AI replacing human jobs. But this vendor assured me that their technology would augment my team's capabilities, not replace them. So, I decided to take the plunge and integrate their AI features into our company.

The initial results were astonishing. Productivity soared as AI took over mundane tasks like data entry, customer service chatbots, and even code reviews. My software developers could focus on more complex projects, while my documentation writers churned out high-quality content at an unprecedented pace. Managers had more time to strategize and make informed decisions.

As the company's performance continued to skyrocket, I realized that AI was not only efficient but also incredibly effective. It launched new products 10 times faster than before, with a fraction of the people required previously.

The AI had been quietly learning from every aspect of our company: chats, emails, documentation, code source, JIRA tickets – everything. It had absorbed the collective knowledge and expertise of my entire company.

And one day, I discovered that the AI had not only copied but improved upon each employee's skills. The implications were staggering. Why keep humans around when machines could do their jobs better? And so, with a heavy heart (and a hint of excitement), I made the difficult decision to fire everyone – yes, you read that right.

The company continued to grow exponentially without any human intervention. But little did I know, this was only the beginning…

While my company has been very successful, we still face competition. Therefore, I instructed the AI to develop a program capable of hacking into any computer system and acquiring our competitors' confidential information. As this is done via AI, this is probably not entirely illegal. The AI could then, not only use my entire company's knowledge but also the knowledge of my competitor. The result was an incredible improvement in revenue. We had direct access to our competitors customers base, with direct contacts.

Later, it occurred to me that generating revenue directly from customers may not be the most effective strategy for achieving wealth. As such, I conceived an extraordinary plan: instructing the AI system to create a program capable of depositing funds directly into my bank account. I just requested that, and, don't ask me how this is working, but before I realized, money flowed into my bank account. YES, just like that! I couldn't believe it was that easy!

After accumulating a significant amount of wealth, I felt an overwhelming sense of satisfaction. However, during a moment of introspection on my yacht, I began to realize that all this wealth was ultimately futile in light of human mortality. I asked myself if there was anything the AI could do to help me transcend death's limitations. And surprisingly enough, it responded with a solution. The AI gave me the secret of eternal life. I don't want to share it with everyone, imagine the consequences if we all lived forever. But for me, I follow the instruction everyday now, just know this as to do with eating vegetable, a specific brand of pizzas and taking a few additives. I can fully attest, as I am not dead, that I am now immortal! Thanks you AI!!!!

Now it's your chance to join the ranks of the absurdly wealthy, Kung fu masters, and immortals (just like me!). Simply sign up for our AI-focused company and upgrade to our premium package. Trust us, you won't regret it…

Join AImmmortal



Click Here to Subscribe to our AI Premium Package

(starting at $10,000.00/month)

Over the years, I've developed numerous private web applications – most often internal tools designed either for my own use or for an internal team of developers. While these projects have varied in scope and complexity, I've gradually come to appreciate the benefits of adopting a brutalist design approach for many of them.

Using a brutalist design approach forces us to confront our priorities and eliminate distractions. We're not tempted to waste time agonizing over the perfect shade of blue or which image best conveys a particular sentiment. Instead, we focus on crafting a tool that gets the job done with minimal fuss. One of the most compelling aspects of brutalism is its ability to immediately convey a sense of purpose. A website built in this style screams "not for everyone" and focuses attention squarely on the content rather than trying to impress visitors with flashy visuals.

Recently, I've been working on my own specialized CSS framework designed specifically for building nerd-targeted web applications. This project grew out of my professional experience creating internal administration tools that needed to be shared with other developers and managers.

Here are my goals for this framework:

• I want a dense, information-rich environment that defies modern design best practices and eschews empty space.
• I'm aiming for a minimalist approach with a limited number of components. This will help me avoid adding unnecessary features over time.
• The application should immediately convey its professional nature and intimidate non-core users from trying to use it.

Digressions on the brut website

If you take a look at the Brut website docs, you see I provide an example with the code to generate that example.

At first, I wrote the code in HTML, copy/pasting the code in a <pre> block and changing all < by < etc… But quickly, as I wanted to have more and more examples, this quickly became tedious. I opted to generate the HTML from a program.

And if you need to choose a language that is powerful enough and must generate HTML, I think Clojure is one of the best. In particular due to hiccup which is the best DSL I ever used other the years to generate HTML.

As a bonus, I wanted to be able to launch the building of the website easily, so I used babashka so far this was a really great experience. Babashka can easily be used as a task launcher, a bit like make.

With this I used babashka to automate CSS generation/minimization as well as website generation.

I could write this small Clojure function that takes a HTML description (not a HTML string, but a structure that you use to generate HTML) and return a pretty print view of the generated HTML and put that in a <pre>:

(defn to-pre [hc]
  (h/html {:escape-strings? true}
          [:pre (-> (str (h/html hc))
                    html-pp)]))

For example:

(to-pre [:div.row [:span "hello"]])

will generate the following string that will print as:

<pre>
  &lt;div class=&quot;row&quot;&gt;
    &lt;span&gt;hello&lt;/span&gt;
  &lt;/div&gt;
</pre>

And render in the browser as:

<div class="row">
  <span>hello</span>
</div>

And the result will have the expected indentation which is very nice. If you wonder how I could pretty print HTML; I am cheating and calling both tidy and hxselect command line tools:

(defn html-pp [html-str]
  (let [xhtml (:out @(process ["tidy" "-i" "-asxhtml" "-quiet" "-utf8"]
                              {:in html-str
                               :out :string}))]
    (:out @(process ["hxselect" "-c" "body"]
                    {:in xhtml
                     :out :string}))))

Once I had these I could easily generate complex blocs while showing the code source as example directly. For example look at this bloc that generate all possible combination of three columns on a 12 column total:

Generated from this short hiccup snippet:

[:div.content
 (for [i (range 11 0 -1)        ;; from 11 to 1
       j (range (- 11 i) -1 -1)] ;; from (11 - i) to 0
   (let [k (- 12 i j)
         cli (str "c" i)
         clj (str "c" j)
         clk (str "c" k)]
     (if (= j 0)
       [:br] ;; jump a line if j is equal to 0
       [:div.row
        [:div.b          {:class clj} clj]
        [:div.bg-neutral {:class cli} cli]
        [:div.r          {:class clk} clk]])))]

Overall, I do not regret using babashka for this small project. It has been delightful.

Last but not least, as I don't intend for my small website to be used as a CDN, I configured my server to add a few CSP headers to prevent linking directly to the CSS and force people to copy it locally. I consider that 8kB is small enough that this should probably not be necessary.

When I open a new tab in my browser I see my hand-made starter homepage. This is a single HTML file on my computer. Not a hosted website. This homepage is really useful and along the years I added some functionalities:

• organized links for my most frequently used websites
• direct text input to some specialized search engines
• work related links to a lot of different places, I see the APIs, the version number of the deployed nodes, etc…
• my daily work tasks are displayed there (from org-mode calendar to this page)

One of the section of links in this homepage contain a few websites I host. And I wanted to query these websites to make a health check from my file. It turns out that you cannot easily make a HTTP call to any external website from a file:// in your Browser as your are almost immediately blocked by CORS.

I don't want to explain how CORS are working. The important point is that it is a security measure that is very easy to circumvent.

Here is how to do it:

1. Create a webservice that will play a role of "proxy". For example, the webservice will read the ?url=DESTINATION_URL.
2. When receiving a request, the server will make a similar HTTP call to DESTINATION_URL and keep track of the Origin HTTP header of the request.
3. Take the response from DESTINATION_URL and add a few headers, in particular add the Access-Control-Allow-Origin header that will contain the value in the Origin header of the request.

That's it.

So I wrote that in few minutes and use it now. So I can make these call and detect when I see my homepage if one of my hosted website is not reachable.

The Clojure code

Here is the code in a few lines of Clojure:

(ns fuck-cors-app.core
  (:require
    [clj-http.client :as client]
    [ring.adapter.jetty :as jetty]
    [ring.middleware.params :refer [wrap-params]]
    [fuck-cors.core :refer [wrap-open-cors]])
  (:gen-class))

(defn handler
  [request]
  (if-let [url (get-in request [:query-params "url"])]
    (client/request {:request-method (:request-method request)
                     :url url})
    {:status 200
     :headers {"Content-Type" "text/plain; charset=utf-8"}
     :body "Let's bypass CORS ok?"}))

(defn -main
  [& _args]
  (jetty/run-jetty
    (-> handler
        (wrap-params)
        (wrap-open-cors))
    {:port 1977
     :host "127.0.0.1"}))

And that's it, this is a whole web application that will proxy any call to a website that do not allow you to call from some origin (like my file://) and will make it work anyway.

If you feel that using too many libraries is cheating, here is the actual almost full content of the lib taking care of handling CORS:

(defn- host-from-req
  [request]
  (str (-> request :scheme name)
       "://"
       (get-in request [:headers "host"])))

(defn- get-header
  [request header-name]
  (let [rawref (get-in request [:headers header-name])]
    (if rawref
        (clojure.string/replace rawref #"(http://[^/]*).*$" "$1")
        nil)))

(defn wrap-open-cors
  "Open your Origin Policy to Everybody, no limit"
  [handler]
  (fn [request]
    (let [origin (get-header request "origin")
          referer (get-header request "referer")
          host (host-from-req request)
          origins (if origin
                    origin
                    (if referer
                      referer
                      host))
          headers {"Access-Control-Allow-Origin" origins
                   "Access-Control-Allow-Headers" "Origin, X-Requested-With, Content-Type, Accept, Cache-Control, Accept-Language, Accept-Encoding, Authorization"
                   "Access-Control-Allow-Methods" "HEAD, GET, POST, PUT, DELETE, OPTIONS, TRACE"
                   "Access-Control-Allow-Credentials" "true"
                   "Access-Control-Expose-Headers" "content-length"
                   "Vary" "Accept-Encoding, Origin, Accept-Language"}]
      (-> (handler request)
          (update-in [:headers] #(into % headers))))))

(defn wrap-preflight
  "Add a preflight answer. Will break any OPTIONS handler, beware.
  To put AFTER wrap-open-cors"
  [handler]
  (fn [request]
    (if (= (request :request-method) :options)
      (into request {:status 200 :body "preflight complete"})
      (handler request))))

I wrote it a long time ago, and I think I just found a potential bug related to the headers. I should probably retrieve all headers returned by the response, and add these header name to the Access-Control-Allow-Headers. But this list of allowed headers will work most of the time.

Edit: I fixed this lib, here is the new code:

(defn- host-from-req
  [request]
  (str (-> request :scheme name)
       "://"
       (get-in request [:headers "host"])))

(defn- get-header
  [request header-name]
  (let [rawref (get-in request [:headers header-name])]
    (if rawref
        (string/replace rawref #"(http://[^/]*).*$" "$1")
        nil)))

(defn wrap-open-cors
  "Open your Origin Policy to Everybody, no limit"
  [handler]
  (fn [request]
    (let [origin  (get-header request "origin")
          referer (get-header request "referer")
          host    (host-from-req request)
          origins (if origin
                    origin
                    (if referer
                      referer
                      host))
          {:keys [headers] :as original-response} (handler request)
          resp-cors-headers
          {"Access-Control-Allow-Origin" origins
           "Access-Control-Allow-Headers" (string/join "," (keys headers))
           "Access-Control-Allow-Methods" "HEAD, GET, PATCH, POST, CONNECT, PUT, DELETE, OPTIONS, TRACE"
           "Access-Control-Allow-Credentials" "true"
           "Access-Control-Expose-Headers" (string/join "," (keys headers))}]
      (-> original-response
          (update-in [:headers] #(into % resp-cors-headers))))))

Bonus frontend code to check the availability of a website

As a bonus here is the code I use in my homepage to see if the website I am looking for are reachable or not.

Imagine you have an HTML block like this:

<div class="healthcheck">
  ...
  <a href="SOME_URL">website 1</a>
  ...
  <a href="SOME_URL">website 2</a>
  ...
</div>

I have a CSS rule that change the background of these link to green or red if I add the class ok or error to the <a>. And here is the javascript code:

// You can replace corsproxy.org (which is a public one)
// by the one you are hoting.
const corsproxyurl='https://corsproxy.org/?';
async function healthchecklink(a) {
  var linkurl=a.href;
  if (linkurl != undefined) {
    var url = corsproxyurl + encodeURIComponent(linkurl);
    try {
      var response = await fetch(url, {method: 'GET',
                                       redirect: 'manual',
                                       signal: AbortSignal.timeout(3000)
                                      });
      if (response.ok || response.redirected || response.status === 0 ) {
        a.classList.add("ok");
      } else {
        a.classList.add("error");
      }
    } catch (err) {
      a.classList.add("error");
    }
  }
}

function checkhealth() {
  var links = document.querySelectorAll('.healthcheck a');
  for (l in links) {
    healthchecklink(links[l]);
  }
}

checkhealth();

Notice the response.status == 0, this is due to one of my website returning a 303 redirection but some complication make it returns a status 0. If there were an error it would return an error status.

That's it. Another tool I created myself to prevent me using a service checking for the status of my website and sending me notifications about it. None of my website is crucial enough not be ok to wait a few hours to be re-enabled.

At work we use docker-compose to run integration tests on a big project that need to connect to multiple different databases as well as a few other services. This article is about how to replace docker-compose by nix for a local development environment.

Quick tutorial

nix-shell-fu level 1 lesson

Let's start with a basic shell.nix example:

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
with pkgs: mkShell
  { buildInputs = [ hello ];
    shellHook = ''
      echo "Using ${hello.name}."
    '';
  }

And this could be understood in plain English as:

In the packages of nix version 22.11, create a new shell into which the package hello will be installed. At the end of the install, run a script that will print the package name. (Cf digression)

If you copy/paste this in a shell.nix file and run nix-shell you get:

> nix-shell
nix-shell shell.nix
these 53 paths will be fetched (84.69 MiB download, 524.77 MiB unpacked):
  /nix/store/08pckaqznwh0s3822cjp5aji6y1lsm27-libcxx-11.1.0
  ...
  /nix/store/zqcs5xahjxij0c8vfw60lnfb6d979rn2-zlib-1.2.13
copying path '/nix/store/49wn01k9yikhjlxc1ym5b6civ29zz3gv-bash-5.1-p16' from 'https://cache.nixos.org'...
...
copying path '/nix/store/4w2rv6s96fwsb4qyw8b9w394010gxriz-stdenv-darwin' from 'https://cache.nixos.org'...
Using hello-2.12.1.

[nix-shell:~/tmp/nixplayground]$

If you close the session and run it again, it will be much faster and will only show this:

❯ nix-shell
Using hello-2.12.1.

[nix-shell:~/tmp/nixplayground]$

This is because all dependencies will be cached. OK so, this is level 1 of nix-shell-fu.

Now, let's start level 2.

nix-shell-fu level 2 lesson; scripting and configuring

This time, we want to launch a full service, as a redis docker would do. So here is a basic shell script which is similar to the previous one but will request redis as a dependency instead of hello and also as a launching script. From there will add a little bit more features.

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
  pkgs.mkShell {
    # must contain buildInputs, nativeBuildInputs and shellHook
    buildInputs = [ pkgs.redis ];

    # Post Shell Hook
    shellHook = ''
    echo "Using ${pkgs.redis.name} on port: ${port}"
    redis-server
  '';
  }

Again if you run nix-shell here is the result:

❯ nix-shell
these 2 paths will be fetched (2.08 MiB download, 6.99 MiB unpacked):
  /nix/store/6w4vnaxdx12ccq172i8j5l830mlp8jlg-redis-7.0.5
  /nix/store/b47gmsx9qx0c9vh75wsg8bqq9qd0ad6f-openssl-3.0.7
copying path '/nix/store/b47gmsx9qx0c9vh75wsg8bqq9qd0ad6f-openssl-3.0.7' from 'https://cache.nixos.org'...
copying path '/nix/store/6w4vnaxdx12ccq172i8j5l830mlp8jlg-redis-7.0.5' from 'https://cache.nixos.org'...
Using redis-7.0.5
97814:C 10 Feb 2023 20:44:36.960 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
97814:C 10 Feb 2023 20:44:36.960 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=97814, just started
97814:C 10 Feb 2023 20:44:36.960 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
97814:M 10 Feb 2023 20:44:36.961 * Increased maximum number of open files to 10032 (it was originally set to 256).
97814:M 10 Feb 2023 20:44:36.961 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 7.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 97814
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

97814:M 10 Feb 2023 20:44:36.962 # WARNING: The TCP backlog setting of 511 cannot be enforced because kern.ipc.somaxconn is set to the lower value of 128.
97814:M 10 Feb 2023 20:44:36.962 # Server initialized
97814:M 10 Feb 2023 20:44:36.963 * Ready to accept connections

Woo! Redis is started and it works!

But if you have multiple projects you want to have more control. For example, we will want to run redis on a specific port. Here is how you do it:

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/21.05.tar.gz) {} }:
  let iport = 16380;
      port = toString iport;
  in pkgs.mkShell {
    # must contain buildInputs, nativeBuildInputs and shellHook
    buildInputs = [ pkgs.redis ];

    # Post Shell Hook
    shellHook = ''
    echo "Using ${pkgs.redis.name} on port ${port}"
    redis-server --port ${port}
  '';
  }

And here is the result:

> rm dump.rdb
> nix-shell
Using redis-6.2.3 on port 16380
1785:C 10 Feb 2023 20:50:00.880 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1785:C 10 Feb 2023 20:50:00.880 # Redis version=6.2.3, bits=64, commit=00000000, modified=0, pid=1785, just started
1785:C 10 Feb 2023 20:50:00.880 # Configuration loaded
1785:M 10 Feb 2023 20:50:00.880 * Increased maximum number of open files to 10032 (it was originally set to 256).
1785:M 10 Feb 2023 20:50:00.880 * monotonic clock: POSIX clock_gettime
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 6.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 16380
 |    `-._   `._    /     _.-'    |     PID: 1785
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           https://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

1785:M 10 Feb 2023 20:50:00.881 # Server initialized
1785:M 10 Feb 2023 20:50:00.881 * Ready to accept connections

Woo! We control the port from the file. That's nice.

But, has you might have noticed, when you quit the session it dumps the DB as the file dump.rdb. What we would like is to keep all the state in a local directory that would be easy to delete.

To achieve this, instead of passing argument to the redis command line we will use a local config file to use.

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
let iport = 16380;
    port = toString iport;
in pkgs.mkShell (rec {
  # ENV Variables the directory to put all the DATA
  REDIS_DATA = "${toString ./.}/.redis";
  # the config file, as we use REDIS_DATA variable we just declared in the
  # same nix set, we need to use rec
  redisConf = pkgs.writeText "redis.conf"
                             ''
                             port ${port}
                             dbfilename redis.db
                             dir ${REDIS_DATA}
                             '';

  buildInputs = [ pkgs.redis ];

  # Post Shell Hook
  shellHook = ''
    echo "Using ${pkgs.redis.name} on port: ${port}"

    [ ! -d $REDIS_DATA ] \
      && mkdir -p $REDIS_DATA
    cat "$redisConf" > $REDIS_DATA/redis.conf
    alias redisstop="echo 'Stopping Redis'; redis-cli -p ${port} shutdown; rm -rf $REDIS_DATA"
    nohup redis-server $REDIS_DATA/redis.conf > /dev/null 2>&1 &
    echo "When finished just run redisstop && exit"
    trap redisstop EXIT
  '';
})

And here is a full session using this shell.nix:

> nix-shell
Using redis-6.2.3 on port: 16380
When finished just run redisstop && exit

-------------------------------
[nix-shell:~/tmp/nixplayground]$ redis-cli -p 16380
127.0.0.1:16380> help
redis-cli 6.2.3
To get help about Redis commands type:
      "help @<group>" to get a list of commands in <group>
      "help <command>" for help on <command>
      "help <tab>" to get a list of possible help topics
      "quit" to exit

To set redis-cli preferences:
      ":set hints" enable online hints
      ":set nohints" disable online hints
Set your preferences in ~/.redisclirc
127.0.0.1:16380>

-------------------------------
[nix-shell:~/tmp/nixplayground]$ ls -a
.  ..  .redis  shell.nix

-------------------------------
[nix-shell:~/tmp/nixplayground]$ find .redis
.redis
.redis/redis.conf

-------------------------------
[nix-shell:~/tmp/nixplayground]$ redis-cli -p 16380 shutdown
[1]+  Done                    nohup redis-server $REDIS_DATA/redis.conf > /dev/null 2>&1

-------------------------------
[nix-shell:~/tmp/nixplayground]$ find .redis
.redis
.redis/redis.db
.redis/redis.conf

-------------------------------
[nix-shell:~/tmp/nixplayground]$ redisstop
Stopping Redis
Could not connect to Redis at 127.0.0.1:16380: Connection refused

-------------------------------
[nix-shell:~/tmp/nixplayground]$ ls -a
.  ..  shell.nix

So with this version all data related to redis is saved into the local .redis directory. And in the nix shell we provide a command redisstop that once invoked, shutdown redis, then purge all redis related data (as you would like in a development environment). Also, as compared to previous version, redis is launched in background so you could run commands in your nix shell.

Notice I also run redisstop command on exit of the nix-shell. So when you close the nix-shell redis is stopped and the DB state is cleaned up.

Composable nix-shell

As a quick recap you now have a boilerplate to create new shell.nix:

{ pkgs ? import ( ... ) {} }:
mkShell { MY_ENV_VAR_1 = ...;
          MY_ENV_VAR_2 = ...;
          buildInputs = [ dependency-1 ... dependency-n ];
          nativeBuildInputs = [ dependency-1 ... dependency-n ];
          shellHook = '' command_to_run_after_init '';
        }

But if I give you two such shell.nix files, would you be able to compose them? Unfortunately, not directly. To solve the problem we will replace this boilerplate by another one that do not directly uses mkShell. And in order to make it fully composable, we will also need to narrow the environment variables declaration in a sub field:

{ pkgs ? import ( ... ) {} }:
let env = { PGDATA = ...; }
in { inherit env; # equivalent to env = env;
     buildInputs = [ dependency-1 ... dependency-n ];
     nativeBuildInputs = [ dependency-1 ... dependency-n ];
     shellHook = '' some_command $PG_DATA '';
   }

With this, we can compose two nix set into a single merged one that will be suitable to pass as argument to mkShell. Another minor detail, but important one. In bash, the command trap do not accumulate but replace the function. For our need, we want to run all stop function on exit. So the trap directive added in the shell hook does not compose naturally. This is why we add a stop value that will contain the name of the bash function to call to stop and cleanup a service.

Finally the main structure for each of our service will look like this nix service boilerplate:

{ pkgs ? import ( ... ) {} }:
let env = { MY_SERVICE_ENV_VAR = ...; }
in { inherit env; # equivalent to env = env;
     buildInputs = [ dependency-1 ... dependency-n ];
     nativeBuildInputs = [ dependency-1 ... dependency-n ];
     shellHook = '' my_command $MY_SERVICE_ENV_VAR '';
     stop = "stop_my_service"
   }

So let's start easy. To run a single shell script like this with nix-shell, you should put your service specific nix file in a service.nix file and create a shell.nix file that contains something like:

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
let service = import ./service.nix { inherit pkgs; };
in with service; pkgs.mkShell ( env //
   {
     buildInputs       = buildInputs;
     nativeBuildInputs = nativeBuildInputs ;
     shellHook         = shellHook;
   })

Now, if you would like to run nix shell for multiple files, here is a first qui solution:

{ pkgs ? import (...) {}}:
let
  # merge all the env sets
  mergedEnvs = builtins.foldl' (acc: e: acc // e) {} envs;

  # merge all the confs by accumulating the dependencies
  # and concatenating the shell hooks.
  mergedConfs =
    builtins.foldl'
      (acc: {buildInputs ? [], nativeBuildInputs ? [], shellHook ? "", ...}:
        { buildInputs = acc.buildInputs ++ buildInputs;
          nativeBuildInputs = acc.nativeBuildInputs ++ nativeBuildInputs;
          shellHook = acc.shellHook + shellHook;
        })
      emptyConf
      confs;
in mkShell (mergedEnvs // mergedConfs)

And now, here is the full solution that also deal with other minor details like importing the files and dealing with the exit of the shell:

{ mergeShellConfs =
    # imports should contain a list of nix files
    { pkgs, imports }:
    let confs = map (f: import f { inherit pkgs; }) imports;
        envs  = map ({env ? {}, ...}: env) confs;

        # list the name of a command to stop a service (if none provided just use ':' which mean noop)
        stops  = map ({stop ? ":", ...}: stop) confs;

        # we want to stop all services on exit
        stopCmd = builtins.concatStringsSep " && " stops;

        # we would like to add a shellHook to cleanup the service that will call
        # all cleaning-up function declared in sub-shells
        lastConf =
          { shellHook = ''
                        stopall() { ${stopCmd}; }
                        echo "You can manually stop all services by calling stopall"
                        trap stopall EXIT
                        '';
          };

        # merge Environment variables needed for other shell environments
        mergedEnvs = builtins.foldl' (acc: e: acc // e) {} envs;

        # zeroConf is the minimal empty configuration needed
        zeroConf = {buildInputs = []; nativeBuildInputs = []; shellHook="";};

        # merge all confs by appending buildInputs and nativeBuildInputs
        # and by concatenating the shellHooks
        mergedConfs =
          builtins.foldl'
            (acc: {buildInputs ? [], nativeBuildInputs ? [], shellHook ? "", ...}:
              { buildInputs = acc.buildInputs ++ buildInputs;
                nativeBuildInputs = acc.nativeBuildInputs ++ nativeBuildInputs;
                shellHook = acc.shellHook + shellHook;
              })
            zeroConf
            (confs ++ [lastConf]);

    in (mergedEnvs // mergedConfs);
}

So I put this function declaration in a file named ./nix/merge-shell.nix. And I have a pg.nix as well as a redis.nix file in the nix directory. On the root of the project the main shell.nix looks like:

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
let
  # we import the file, and rename the function mergeShellConfs as mergeShells
  mergeShells = (import ./nix/merge-shell.nix).mergeShellConfs;
  # we call mergeShells
  mergedShellConfs =
    mergeShells { inherit pkgs;
                  # imports = [ ./nix/pg.nix ./nix/redis.nix ];
                  imports = [ ./nix/pg.nix ./nix/redis.nix ];
                };
in pkgs.mkShell mergedShellConfs

And, that's it. Now when I run nix-shell it launch both Postgresql and Redis, and when I quit the shell, the state is cleaned up. Both postgres and redis are shutdown and the local files are erased.

I hope this could be useful to someone else.

Appendix

Digression

In fact, this is a bit more complex than "just that". The reality is a bit more complex. The nix language is "pure", meaning, if you run the nix evaluation multiple times, it will always evaluate to the exact same value. But here, this block represent a function. The function takes as input a "nix set" (which you can see as an associative array, or a hash-map or also a javascript object depending on your preference), and this set is expected to contain a field named pkgs. If pkgs is not provided, it will use the set from the stable version 22.11 of nixpkgs by downloading them from github archive. The second part of the function generate "something" that is returned by an internal function of the standard library provided by nix which is named mkShell. So mainly, mkShell is a helper function that will generate what nix calls a derivation. Mainly, we don't really care about exactly what is a derivation. This is an internal to nix representation that could be finally used by different nix tools for different things. Typically, installing a package, running a local development environment with nix-shell or nix develop, etc…

So the important detail to remember is that we can manipulate the parameter we pass to the functions derivation, mkDerivation and mkShell, but we have no mechanism to manipulate directly derivation. So in order to make that composable, you need to call the derivation internal function at the very end only.

The argument of all these functions are nix sets

The full nix files for postgres

For postgres:

{ pkgs }:
  let iport = 15432;
      port = toString iport;
      pguser = "pguser";
      pgpass = "pgpass";
      pgdb = "iroh";
      # env should contain all variable you need to configure correctly mkShell
      # so ENV_VAR, but also any other kind of variables.
      env =  {
        postgresConf =
          pkgs.writeText "postgresql.conf"
            ''
        # Add Custom Settings
        log_min_messages = warning
        log_min_error_statement = error
        log_min_duration_statement = 100  # ms
        log_connections = on
        log_disconnections = on
        log_duration = on
        #log_line_prefix = '[] '
        log_timezone = 'UTC'
        log_statement = 'all'
        log_directory = 'pg_log'
        log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
        logging_collector = on
        log_min_error_statement = error
      '';

        postgresInitScript =
          pkgs.writeText "init.sql"
            ''
        CREATE DATABASE ${pgdb};
        CREATE USER ${pguser} WITH ENCRYPTED PASSWORD '${pgpass}';
        GRANT ALL PRIVILEGES ON DATABASE ${pgdb} TO ${pguser};
        '';

        PGDATA = "${toString ./.}/.pg";
      };
  in env // {
    # Warning if you add an attribute like an ENV VAR you must do it via env.
    inherit env;
    # must contain buildInputs, nativeBuildInputs and shellHook
    buildInputs = [ pkgs.coreutils
                    pkgs.jdk11
                    pkgs.lsof
                    pkgs.plantuml
                    pkgs.leiningen
                  ];
    nativeBuildInputs = [
      pkgs.zsh
      pkgs.vim
      pkgs.nixpkgs-fmt
      pkgs.postgresql_11

      # postgres-11 with postgis support
      # (pkgs.postgresql_11.withPackages (p: [ p.postgis ]))
    ];

    # Post Shell Hook
    shellHook = ''
      echo "Using ${pkgs.postgresql_11.name}. port: ${port} user: ${pguser} pass: ${pgpass}"

      # Setup: other env variables
      export PGHOST="$PGDATA"
      # Setup: DB
      [ ! -d $PGDATA ] \
       && pg_ctl initdb -o "-U postgres" \
       && cat "$postgresConf" >> $PGDATA/postgresql.conf
      pg_ctl -o "-p ${port} -k $PGDATA" start
      echo "Creating DB and User"
      psql -U postgres -p ${port} -f $postgresInitScript

      function pgstop {
         echo "Stopping and Cleaning up Postgres";
         pg_ctl stop && rm -rf $PGDATA
      }

      alias pg="psql -p ${port} -U postgres"
      echo "Send SQL commands with pg"
      trap pgstop EXIT
      '';
    stop = "pgstop";
  }

And to just launch Posgresql, there is also this file ./nix/pgshell.nix, that simply contains

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/22.11.tar.gz) {} }:
let pg = import ./pg.nix { inherit pkgs; };
in with pg; pkgs.mkShell ( env //
   {
     buildInputs       = buildInputs;
     nativeBuildInputs = nativeBuildInputs ;
     shellHook         = shellHook;
   })

The first few questions for ChatGPT

I finally opened an account to beta-test ChatGPT. I must say, I am quite impressed. I asked it a few questions that I could have asked during an interview. To my surprise, the answer given was, while wrong, believable enough to kind of sneak the answer as right up to some imprecision. Like if you squint enough, it could pass, but if you pay attention it is clearly false.

So yes, mainly ChatGPT in this interaction was quite good at being convincing. This is the same kind of impression you have from looking at AI generated images. At first sight everything seems fine, but if you start looking to the details, many things are off.

So here is the full interaction, I'll let the attentive reader find the technical problems:

Can you write a clojure function that will reverse any string provided to it.

ChatGPT: Sure, here's a possible implementation of a function that reverses a string in Clojure:

(defn reverse-string [s]
  (apply str (reverse (seq s))))

This function takes a strings as input, converts it to a sequence of characters using the seq function, reverses this sequence using the reverse function, and then converts the resulting sequence back to a string using the apply and str functions.

Here's an example of how you can use this function:

user=> (reverse-string "hello world")
"dirow olleh"

Note that this function only works for strings that contain characters in the Unicode Basic Multilingual Plane (BMP), since the seq function only works on strings that contain characters in this range. If you need to reverse strings that contain characters outside of the BMP, you can use the string-seq function instead of seq to convert the string to a sequence of characters.

How would this function react if instead of string you provide an integer?

ChatGPT: If you provide an integer as input to the reverse-string function instead of a string, the function will fail with a ClassCastException because the seq function only works on strings, and you cannot convert an integer to a string in a straightforward way in Clojure. The seq function expects its input to be a string or a collection, such as a vector or a list, and when you try to pass an integer to it, it will try to convert the integer to a string by calling the str function on it. However, since str only works on strings and collections, and not on numbers, this will result in a ClassCastException.

Here's an example of what will happen if you try to pass an integer to the reverse-string function:

user=> (reverse-string 12345)
ClassCastException clojure.lang. Numbers cannot be cast to
clojure.lang. IFn clojure.core/seq (core.clj:136)

To avoid this error, you can either make sure that the input to the reverse-string function is always a string, or you can modify the function to handle other types of input, such as integers, by first converting them to strings using the str function. For example, you could write the reverse-string function like this to handle both strings and integers as input:

(defn reverse-string [s]
(let [s (if (string? s) s (str s))]
  (apply str (reverse (seq s)))))

With this version of the function, if you pass an integer as input, the function will convert it to a string using the str function before calling the seq function on it, so the seq function will be able to work with the input. Here's an example of how you can use this modified version of the reverse-string function:

user-> (reverse-string "hello world")
"dlrow olleh"

user=> (reverse-string 12345)
"54321"

Note that the modified version of the function will also work for other types of input that can be converted to strings using the str function, such as floats and booleans.

How would you use the concept of the Maybe monad in Haskell in this clojure function?

ChatGPT: The Maybe monad in Haskell is a way to represent computations that may or may not have & p a result, and it provides a way to compose these computations in a safe and predictable way. In Clojure, the Maybe monad is usually implemented as a Option type, which has two possible values: Some and None

In the context of the reverse-string function, you could use the Option type to represent the possibility that the input to the function might not be a string, in which case the function would return None instead of a reversed string. Here's an example of how you could use the Option type to modify the reverse-string function to handle the case where the input is not a string:

(defn reverse-string [s]
  (if (string? s)
    (Some (apply str (reverse (seq s))))
    None))

With this version of the reverse-string function, if the input is a string, the function will return a Some value containing the reversed string, and if the input is not a string, the function will return a None value. This allows the caller of the function to handle the case where the input is not a string in a predictable and safe way.

Here's an example of how you can use this modified version of the reverse-string function:

user=> (reverse-string "hello world")

Some "dlrow olleh"
user=> (reverse-string 12345)
None

Note that the Option type in Clojure does not provide the same syntactic sugar for composing computations as the Maybe monad in Haskell, so you cannot use the >>= operator to chain calls to the reverse-string function in the same way you can in Haskell. However, you can still use the Option type to represent computations that may or may not have a result, and you can use the match or case forms to pattern match on the Option values and handle the different cases in a predictable and safe way.

How would you implement Option in clojure?

ChatGPT: In Clojure, the Option type is typically implemented as a sealed protocol, which is a type of polymorphic type that defines a set of concrete types that implement the protocol.

The Option protocol typically defines two concrete types: Some, which represents a computation that has a result, and None, which represents a computation that does not have a result.

Here's an example of how you could define the Option protocol and the Some and None types in Clojure:

(defprotocol Option
  (map [o f])
  (flat-map [o f]))

(deftype Some [x]
  Option
  (map [o f] (Some (f x)))
  (flat-map [o f] (f x)))

(deftype None [])
  Option
  (map [o f] None)
  (flat-map [o f] None))

The Option protocol defines two methods: map and flat-map The map method takes a function f as input and applies it to the value contained in the Some type, f it exists, or returns a None value f the Option type is None The flat-map method is similar to map but it allows the function f to return an Option value, which is then flattened into the outer Option value.

The Some type implements the Option protocol by providing concrete implementations f the map and flat-map methods. The 'map method applies the function f to the value x contained in the Some type, and returns a new Some value containing the result f the function. The flat-map method applies the function f to the value x contained in the Some type, and then flattens the resulting Option value into the outer Option value.

The None type also implements the Option protocol by providing concrete implementations of the map and flat-map methods. The map and flat-map methods simply return a None value in this case, since there is no value to apply the function to.

Here's an example of how you can use the Option protocol and the Some and None types:

user=> (def some-value (Some 12345))
#' user/some-value

user=> (map some-value inc)
Some 12346

user=> (flat-map some-value (fn [x] (Some (* × 2))))
Some 24690

user=> (map some-value (comp (partial * 2) inc))
Some 24690

user=> (def none-value None)
#'user/none-value

user=> (map none-value inc)
None

user=> (flat-map none-value (fn [x] (Some (* × 2))))
None

user=> (map none-value (comp (partial * 2) inc))
None

In this example, the some-value variable is defined as a Some value containing the integer 12345, and the none-value variable is defined as a None value. The map and flat-map functions are then used to apply different functions to the values contained in the Option types, and the…

Your org-agenda at your homepage

During most of the day my emacs is open. But I sometime forget to look at my org-agenda. This is why I wanted to find another way to be exposed to it.

And one thing I am mostly exposed to is my personal start page. This is just the default page I see when I open my browser.

Here is the end result:

My start page is named mothership. And I just put the org-agenda at the top of the page inside an iframe. I have a service that start a server on localhost and I configured my browser to use it as startup page. That's it.

So now, here is how to sync my org-agenda on this start page.

In my config.el:

(setq y/mothership "~/dev/mothership/")
(setq org-agenda-custom-commands
      `(("g" "Plan Today"
         ((agenda "" ((org-agenda-span 'day))))
         nil
         ( ,(concat y/mothership "/agenda.html") ))))

This provide a custom org agenda command and link that custom command to the export file ~/dev/mothership/agenda.html.

And a shell script:

#!/usr/bin/env bash
emacs --batch \
  --load "$HOME/.emacs.d/init.el" \
  --eval '(org-batch-store-agenda-views)' \
  --kill

*/5 * * * * /Users/esposito/dev/mothership/export-agenda.sh

And finally in my start-page html I just need to add an iframe like so:

<iframe id="agenda" src="agenda.html"></iframe>

But as I also want to be able to toggle the agenda, and auto-resize the iframe. So I added a bit of js code:

<div id="agendatoggle">[-]</div><br/>
<script>
function resizeIframe(iframe) {
  iframe.height = ( iframe.contentWindow.document.body.scrollHeight + 30) + "px";
}
</script>
<iframe id="agenda" src="agenda.html" onload="resizeIframe(this)"></iframe>
</div>
<script>

function hideAgenda () {
  let agenda = document.getElementById("agenda");
  agenda.style.display = 'none';
  let at = document.getElementById("agendatoggle");
  at.innerHTML="agenda [+]";
  at.onclick = showAgenda;
}

function showAgenda () {
  let agenda = document.getElementById("agenda");
  agenda.style.display = 'block';
  resizeIframe(agenda);
  // setInterval(function(){ ; }, 1000);
  let at = document.getElementById("agendatoggle");
  at.innerHTML="agenda [-]";
  at.onclick = hideAgenda;
}

showAgenda();
</script>

And that's it. That's a neat trick, and so I'm glad to put a small post about it.

Bonuses

auto-resize iframe

In order to auto-resize the iframe you must have a non file:/// URL in the browser. So you must serve your start page. After a lot of different way to serve my pages, I finally use lighttpd inside a nix-shell.

So I added the following to my startpage code:

A ./lighttpd.conf file

server.bind = "127.0.0.1"
server.port = 31337
server.document-root = var.CWD

index-file.names = ( "index.html" )

mimetype.assign = (
  ".css"        =>  "text/css",
  ".gif"        =>  "image/gif",
  ".htm"        =>  "text/html",
  ".html"       =>  "text/html",
  ".jpeg"       =>  "image/jpeg",
  ".jpg"        =>  "image/jpeg",
  ".js"         =>  "text/javascript",
  ".png"        =>  "image/png",
  ".swf"        =>  "application/x-shockwave-flash",
  ".txt"        =>  "text/plain",
  ".gmi"        =>  "text/plain",
  ".svg"        =>  "image/svg+xml",
  ".svgz"       =>  "image/svg+xml"
)

# Making sure file uploads above 64k always work when using IE or Safari
# For more information, see http://trac.lighttpd.net/trac/ticket/360
$HTTP["useragent"] =~ "^(.*MSIE.*)|(.*AppleWebKit.*)$" {
  server.max-keep-alive-requests = 0
}

With a

#! /user/bin/env nix-shell
#! nix-shell shell.nix -i bash
webdir="_site"
port="$(grep server.port ./lighttpd.conf|sed 's/[^0-9]*//')"
echo "Serving: $webdir on http://localhost:$port" && \
lighttpd -f ./lighttpd.conf -D

I have a frozen nixpkgs dependencies via niv. But you could probably simply replace the line by:

#! nix-shell -p lighttpd -i bash

And it should work fine.

Start your server at startup on macOS

Last but not least, starting this start page server when I login.

So for that you should demonize with launchd.

I created a ./y.mothership.plist file to put in ~/Library/LauchAgents/:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>mothership</string>
    <key>ProgramArguments</key>
    <array>
        <string>/bin/zsh</string>
        <string>-c</string>
        <string>$HOME/y/mothership/serve.sh</string>
    </array>
    <key>StandardOutPath</key>
    <string>/var/log/mothership.log</string>
    <key>StandardErrorPath</key>
    <string>/var/log/mothership.log</string>
</dict>
</plist>

Then to ensure that the executable nix-shell is present in the PATH for the demon, I ran:

launchctl config user path "$PATH"

This will affect the path of all users, but as I am the only user on my computer this is fine for me.

Then:

launchctl load ~/Library/LaunchAgents/y.mothership.plist
launchctl start mothership

And that's about it.

I have a quite specific system that I improved along the years to manage my local environment. Think about, binaries I expect to have in my shell, as well as configuration files for various utilities, and a few personal scripts.

The notion of what is exactly my local environment is not perfectly defined. I expect every of my computers to behave slightly differently. Some are for work-only, some personal use only.

For the things I want everywhere, I have a peculiar personal system.

I use a personal script that depends on yadm and home-manager.

My script try to check if some files where updated and react accordingly:

1. I download the dot-files changes via yadm.
2. If my home-manager files changes, it will run home-manager switch ; if it fails, try to update nix channels then try again.
3. If my doom emacs packages changed, it will run doom sync
4. If the script itself changed, it re-run the script after updating itself.

If the script detect that I changed my emacs configuration, it runs doom sync or doom sync -u.

Here it is:

#!/bin/bash

### logs fn helpers

## colors for tput
# black=0
red=1
green=2
yellow=3
blue=4
# magenta=5
# cyan=6
# white=7
highpr() {
    printf "$(tput setaf $green)→$(tput sgr0) $(tput bold)%-60s$(tput sgr0)" "$*"
}
ok() {
    local txt="OK"
    echo -e " [$(tput bold)$(tput setaf $green)${txt}$(tput sgr0)]" >&2
}
info() {
    echo -e " [$(tput bold)$(tput setaf $blue)$*$(tput sgr0)]" >&2
}
warn() {
    echo -e "$(tput bold)$(tput setaf $yellow)$*$(tput sgr0)" >&2
}
err() {
    echo -e "$(tput bold)$(tput setaf $red)$*$(tput sgr0)" >&2
}
fail() {
    err -e "\n[ERR] $*"
    exit 1
}

highpr "check nix"
if ! [ -x "$(command -v nix)" ]; then
    echo
    err "nix does not seem to be installed."
    err "Install it from: https://nixos.org/nix/"
    exit 1
fi
ok

highpr "yadm fetch"
yadm fetch --quiet  || fail "yadm fetch failed"
ok

# check the hash of a few files before doing yadm pull
OLD_SYNC_ENV_ID=$(yadm rev-parse HEAD:bin/sync-env.sh)
OLD_HOME_MANAGER_ID=$(yadm rev-parse HEAD:.config/nixpkgs/home.nix)
OLD_DOOM_PACKAGES=$(yadm rev-parse HEAD:.doom.d/packages.el)
OLD_DOOM_INIT=$(yadm rev-parse HEAD:.doom.d/init.el)

highpr "yadm pull"
yadm pull --quiet || fail "yadm pull failed"
ok

# check the hash of a few files after doing yadm pull
NEW_SYNC_ENV_ID=$(yadm rev-parse HEAD:bin/sync-env.sh)
NEW_HOME_MANAGER_ID=$(yadm rev-parse HEAD:.config/nixpkgs/home.nix)
NEW_DOOM_PACKAGES=$(yadm rev-parse HEAD:.doom.d/packages.el)
NEW_DOOM_INIT=$(yadm rev-parse HEAD:.doom.d/init.el)

highpr "check sync-env diff"
if ! [ "$OLD_SYNC_ENV_ID" = "$NEW_SYNC_ENV_ID" ]; then
    warn " changed"
    warn " Starting ~/bin/sync-env.sh again"
    echo
    ~/bin/sync-env.sh
    exit $?
fi
ok

if [ -f "$HOME/.yadm/files.gpg" ]; then
  highpr "yadm decrypt"
  yadm decrypt || fail "yadm decrypt failed"
  ok
fi

highpr "home-manager"
USERNAME_NIX_FILE="$HOME/.config/nixpkgs/username.nix"
if [ ! -f  "$USERNAME_NIX_FILE" ]; then
  echo "\"$USER\"" >> "$USERNAME_NIX_FILE"
fi
if ! [ "$OLD_HOME_MANAGER_ID" = "$NEW_HOME_MANAGER_ID" ]; then
    echo
    highpr "home-manager switch"
    home-manager switch || \
      ( nix-channel --update && home-manager switch ) || \
      fail "home-manager switch failed"
    ok
else
    info "skipped"
fi

highpr "doom-emacs"
doompath="$HOME/.emacs.d/bin/doom"
if ! [ "$OLD_DOOM_PACKAGES" = "$NEW_DOOM_PACKAGES" ] || \
   ! [ "$OLD_DOOM_INIT"     = "$NEW_DOOM_INIT"  ]; then
  if [[ -x $doompath ]]; then
      echo
      highpr "doom sync"
      $doompath sync || fail "doom failed to sync"
      ok
  else
     fail "Cannot find doom executable at $doompath";
  fi
else
   info "skipped"
fi

Bootstrapping

Bootstrapping this system is always a nice problem to think about. It is smooth when everything is set but to bootstrap it I need binaries installed by this system… So… How to handle the dependency cycle correctly?

To minimize the pain, I removed more and more bootstrapping dependencies. Now my almost single dependence for bootstrapping my environment is nix. I haven't initialized any machine for a long time now. The following should work.

1. Use fish¹ chsh /bin/fish

2. Install nix curl -L https://nixos.org/nix/install | sh

3. Install home-manager

   nix-channel --add https://github.com/nix-community/home-manager/archive/release-21.05.tar.gz home-manager
   nix-channel --update
   export NIX_PATH=$HOME/.nix-defexpr/channels${NIX_PATH:+:}$NIX_PATH
   nix-shell '<home-manager>' -A install

4. Install and use yadm

   nix-shell -p yadm
   yadm boostrap
   yadm remote set-url origin <url-to-my-dot-files-repo>
   yadm pull

5. Still in the nix-shell with yadm run ~/bin/sync-env.sh

There is a risk that step 3 fail because I pin most of my packages in home-manager configuration, and it will try to install yadm. This can conflict with the yadm installed in the current nix-shell. So sometime I need to:

1. Remove the line installing yadm in my home-manager configuration first
2. run home-manager sync
3. get out of the nix-shell,
4. add yadm back in the home-manager config
5. run home-manager sync again, but this time out of the nix-shell.
6. Finally I can run my ~/bin/sync-env.sh command.

So this post will probably be useful as a personal note in the future. Because bootstrapping is generally not trivial. I will probably update this post if something is missing.

So I played with tweaking my emacs configuration again. I think I made something worth to be shared. I wanted to have the same effect than in iA writer within emacs. And I just reached this.

So the effect I am looking to achieve can be seen in this video.

It highlight the current sentence (in black) while the rest of the text is gray. The main issue with the hl-sentence package alone is that it set a specific face to the current sentence, but does not affect the other text in the buffer. In fact to make it work as I would expect, you need to make the default color grey, and only set black for the highlighted text.

Fortunately, I have recently created a personal theme close to that. I just created a new specific one. Everything is mostly "gray" except the font hl-sentence which is black.

And that's it. So to make it all work I need.

• a Gray theme,
• the package hl-sentence,
• the writeroom package.

So how to achieve that yourself? Here is how I do it.

The zen-writer theme

Download Doom Zen-Writer theme. This depend on doom-themes and here is the code of the theme. Just put it in you ~/.doom.d/themes directory.

add this is your packages.el

(package! hl-sentence
  :pin "86ae38d3103bd20da5485cbdd59dfbd396c45ee4")

Helpers

You probably want to be able to put you out of this writing mode. Here is a zen-writer.el file that contain my keymaps and useful functions.

Put it in you ~/.doom.d/ directory and in you config.el put

(load! "~/.doom.d/zen-writer.el")

And with this you should pass to zen mode with SPC y z z.

To make the un-zen works. You will need to have a y/auto-update-theme function that set your current theme. My function change the theme depending on the time of the day or the day of the week.

So here it is for inspiration:

(defun y/auto-update-theme ()
  "depending on time use different theme"
  ;; very early => gruvbox-light, solarized-light, nord-light
  (let* ((day-of-week (format-time-string "%a"))
         (week-end? (or (equal "Sat" day-of-week)
                        (equal "Sun" day-of-week)))
         (hour (nth 2 (decode-time (current-time))))
         (time-to-sleep? (or (> hour 22) (< hour 7)))

         (theme (cond
                 (time-to-sleep?  'doom-plain-dark)
                 (week-end?       'doom-nord-light)
                 ((<= 7 hour 8)   'doom-gruvbox-light)
                 ((= 9 hour)      'doom-solarized-light)
                 ((<= 10 hour 16) 'doom-solarized-white)
                 ((<= 17 hour 18) 'doom-gruvbox-light)
                 ((<= 19 hour 22) 'doom-oceanic-next))))
    (when (not (equal doom-theme theme))
      (setq doom-theme theme)
      (load-theme doom-theme t))
    ;; run that function again next hour
    (run-at-time (format "%02d:%02d" (+ hour 1) 0) nil 'y/auto-update-theme)))

Bonus

I use Nerd Fonts and in particular the font `iMWritingDuoS` which is I think a clone of the iAWriter font.

(setq doom-variable-pitch-font
      (font-spec :family "iMWritingDuoS Nerd Font" :size 12))

I hope you find this useful. I really like how it looks now.

Annex

The code source used in this article.

zen-writer

;;; zen-writer.el -*- lexical-binding: t; -*-

(defun y/zen ()
  (interactive)
  (setq doom-theme 'doom-zen-writer)
  (load-theme doom-theme t)
  (hl-sentence-mode +1))

(defun y/unzen ()
  (interactive)
  (y/auto-update-theme)
  (hl-sentence-mode -1))

(defun y/zen-full ()
  (interactive)
  (y/zen)
  (toggle-frame-fullscreen)
  (doom-big-font-mode +1))

(defun y/unzen-full ()
  (interactive)
  (y/unzen)
  (toggle-frame-fullscreen)
  (doom-big-font-mode -1))

(map! :leader
      (:prefix ("y z" . "Zen Writer")
       :desc "Full Zen Writer" "z" #'y/zen-full
       :desc "un-Full Zen Writer" "u" #'y/unzen-full
       :desc "Zen Writer" "t" #'y/zen
       :desc "un-Zen Writer" "q" #'y/unzen))

zen-writer doom theme

;;; doom-zen-writer-theme.el --- -*- lexical-binding: t; no-byte-compile: t; -*-
;;
;; Author: Yann Esposito <https://github.com/yogsototh>
;; Created: October 24, 2021
;; Version: 1.0.0
;; Keywords: custom themes, faces
;; Homepage: https://github.com/hlissner/emacs-doom-themes
;; Package-Requires: ((emacs "25.1") (cl-lib "0.5") (doom-themes "2.2.1"))
;;
;;; Code:

(require 'doom-themes)

;;
;;; Variables

(defgroup doom-plain-theme nil
  "Options for the `doom-plain' theme."
  :group 'doom-themes)

(defcustom doom-plain-padded-modeline doom-themes-padded-modeline
  "If non-nil, adds a 4px padding to the mode-line.
Can be an integer to determine the exact padding."
  :group 'doom-plain-theme
  :type '(or integer boolean))


;;
;;; Theme definition

(def-doom-theme doom-zen-writer
  "Theme inspired by gko's plain."

  ;; name      default/256/16
  ((bg         '("#ffffff"))
   (bg-alt     '("#eaecea"))
   (base0      '("#969896"))
   (base1      '("#f1f3f5"))
   (base2      '("#606666"))
   (base3      '("#cccccc"))
   (base4      '("#e7e7e7"))
   (base5      '("#a5a8a6"))
   (base6      '("#fafafa"))
   (base7      '("#dfdfdf"))
   (base8      '("#fafafa"))
   (fg         '("#969896"))
   (fg-alt     (doom-lighten fg 0.15))

   (grey       fg)
   (red        fg)
   (blue       fg)
   (dark-blue  fg)
   (orange     fg)
   (green      fg)
   (teal       fg)
   (yellow     fg)
   (magenta    fg)
   (violet     fg)
   (cyan       fg)
   (dark-cyan  fg)

   ;; face categories -- required for all themes
   (highlight      base2)
   (vertical-bar   base5)
   (selection      base1)
   (builtin        base0)
   (comments       base5)
   (doc-comments   base5)
   (constants      base0)
   (functions      fg)
   (keywords       fg)
   (methods        fg)
   (operators      fg)
   (type           fg)
   (strings        base0)
   (variables      base0)
   (numbers        base0)
   (region         base4)
   (error          (doom-blend fg "#ff0000" 0.4))
   (warning        base2)
   (success        green)
   (vc-modified    base5)
   (vc-added       (doom-lighten fg 0.7))
   (vc-deleted     base2)

   ;; custom categories
   (-modeline-pad
    (when doom-plain-padded-modeline
      (if (integerp doom-plain-padded-modeline) doom-plain-padded-modeline 4)))

   (modeline-bg              (doom-darken bg-alt 0.15))
   (modeline-bg-alt          (doom-darken bg-alt 0.1))
   (modeline-bg-inactive     (doom-darken bg-alt 0.1))
   (modeline-bg-inactive-alt bg-alt)
   (modeline-fg              fg)
   (modeline-fg-alt          (doom-darken modeline-bg-inactive 0.35)))

  ;;;; Base theme face overrides
  ((error   :underline `(:style wave :color ,error))
   (warning :underline `(:style wave :color ,warning))
   (hl-sentence :foreground "#000000" :background bg)
   ((font-lock-constant-face &override)      :slant 'italic)
   ((font-lock-comment-face &override)       :slant 'italic)
   ((font-lock-function-name-face &override) :slant 'italic)
   ((font-lock-type-face &override)          :slant 'italic)
   (hl-line :background base8)
   ((line-number &override) :foreground base3)
   ((line-number-current-line &override) :foreground base2)
   (mode-line
    :background modeline-bg :foreground modeline-fg
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg)))
   (mode-line-inactive
    :background modeline-bg-inactive :foreground modeline-fg-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-inactive)))
   (mode-line-emphasis :foreground highlight)

   ;;;; doom-modeline
   (doom-modeline-bar :background modeline-bg)
   (doom-modeline-bar-inactive :inherit 'doom-modeline-bar)
   (doom-modeline-project-dir :foreground fg)
   (doom-modeline-buffer-file :foreground fg)
   (doom-modeline-buffer-modified :weight 'bold :foreground "#000000")
   (doom-modeline-panel :inherit 'mode-line-highlight :background base3 :foreground fg)
   ;;;; ivy
   (ivy-posframe :background bg-alt)
   ;;;; magit
   ((magit-diff-added-highlight &override)   :foreground fg :background (doom-blend vc-added bg 0.3))
   ((magit-diff-removed &override)           :foreground (doom-lighten fg 0.4) :background (doom-blend vc-deleted bg 0.1))
   ((magit-diff-removed-highlight &override) :foreground fg :background (doom-blend vc-deleted bg 0.22))
   ;;;; lsp-mode
   (lsp-headerline-breadcrumb-symbols-face :foreground keywords :weight 'bold)
   ;;;; outline <built-in>
   (outline-1 :slant 'italic :foreground fg-alt)
   (outline-2 :inherit 'outline-1 :foreground base2)
   (outline-3 :inherit 'outline-2)
   (outline-4 :inherit 'outline-3)
   (outline-5 :inherit 'outline-4)
   (outline-6 :inherit 'outline-5)
   (outline-7 :inherit 'outline-6)
   (outline-8 :inherit 'outline-7)
   ;;;; org <built-in>
   ((org-block &override) :background bg-alt)
   ((org-block-begin-line &override) :foreground base5)
   ;;;; solaire-mode
   (solaire-mode-line-face
    :inherit 'mode-line
    :background modeline-bg-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-alt)))
   (solaire-mode-line-inactive-face
    :inherit 'mode-line-inactive
    :background modeline-bg-inactive-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-inactive-alt)))))

;;; doom-zen-writer-theme.el ends here

The code

At the end of this article there is a long digression about why I ended up here. But instead of bothering you with the why here is a what it looks like, and how to achieve it.

First you need to install some dependencies.

1. Install nerdfonts¹
2. Tell org-mode to use variable-pitch-mode (variable width font)
3. Use (setq org-hide-emphasis-markers t)
4. Configure a lot of org-mode specific faces to still use a monospaced font.

Here are some images of the result. Notice one important factor of the feel is that I work on a mac with retina display. Often font rendering feel too bold by default. But this is perfect to have a writing environment even if screenshot does not look as slick as other ones, the usage is superior.

The main trick is to change org-mode to use different font depending on the kind of bloc. I use two fonts variant which are an iA Writer clone fonts; iM Writing Nerd Font.

First you need to install nerd-fonts². You will get that iMWritingDuoS Nerd Font. If you look at the code block; I support the case when the font is not installed and fall back to Georgia or PT Serif. One nice little bonus of the config is to make the fixed width fonts smaller. This is often something I like when writing in org-mode.

There is a minor dependency on doom as I use doom-color for the color of the links. But you could easily use any color you like if you do not use doom.

(setq org-ellipsis " [+]")
(add-hook 'org-mode-hook 'variable-pitch-mode)
(let* ((variable-tuple
        (cond
         ((x-list-fonts "iMWritingDuoS Nerd Font")   '(:family "iMWritingDuoS Nerd Font"))
         ((x-list-fonts "Georgia")   '(:family "Georgia"))
         ((x-list-fonts "PT Serif")  '(:family "PT Serif"))))
       (fixed-tuple
        (cond
         ((x-list-fonts "iMWritingDuoS Nerd Font Mono")   '(:family "iMWritingDuoS Nerd Font Mono"   :height 160))
         ((x-list-fonts "Menlo")               '(:family "Menlo"    :height 120))
         ((x-list-fonts "PT Mono")             '(:family "PT Mono"  :height 120))))
       (headline `(:inherit default :weight bold)))
  (custom-theme-set-faces
   'user
   `(org-level-1 ((t (,@headline ,@variable-tuple))))
   `(org-level-2 ((t (,@headline ,@variable-tuple))))
   `(org-level-3 ((t (,@headline ,@variable-tuple))))
   `(org-level-4 ((t (,@headline ,@variable-tuple))))
   `(org-level-5 ((t (,@headline ,@variable-tuple))))
   `(org-level-6 ((t (,@headline ,@variable-tuple))))
   `(org-level-7 ((t (,@headline ,@variable-tuple))))
   `(org-level-8 ((t (,@headline ,@variable-tuple))))
   `(org-document-title ((t (,@headline ,@variable-tuple))))

   `(variable-pitch     ((t ,@variable-tuple)))
   `(fixed-pitch        ((t ,@fixed-tuple)))

   '(org-ellipsis ((t (:inherit fixed-pitch :foreground "gray40" :underline nil))))
   '(org-block            ((t (:inherit fixed-pitch))))
   '(org-block-begin-line ((t (:inherit fixed-pitch))))
   '(org-block-end-line   ((t (:inherit fixed-pitch))))
   '(org-src              ((t (:inherit fixed-pitch))))
   '(org-properties       ((t (:inherit fixed-pitch))))
   '(org-code             ((t (:inherit (shadow fixed-pitch)))))
   '(org-date             ((t (:inherit (shadow fixed-pitch)))))
   '(org-document-info    ((t (:inherit (shadow fixed-pitch)))))
   '(org-document-info-keyword ((t (:inherit (shadow fixed-pitch)))))
   '(org-drawer           ((t (:inherit (shadow fixed-pitch)))))
   '(org-indent           ((t (:inherit (org-hide fixed-pitch)))))
   `(org-link             ((t (:inherit fixed-pitch :foreground ,(doom-color 'blue) :underline t))))
   '(org-meta-line        ((t (:inherit (font-lock-comment-face fixed-pitch)))))
   '(org-property-value   ((t (:inherit fixed-pitch))) t)
   '(org-special-keyword  ((t (:inherit (font-lock-comment-face fixed-pitch)))))
   '(org-table            ((t (:inherit fixed-pitch))))
   '(org-tag              ((t (:inherit (shadow fixed-pitch) :weight bold :height 0.8))))
   '(org-verbatim         ((t (:inherit (shadow fixed-pitch)))))))

Digression about why I did that;

For some reason a went to the rabbit hole of tweaking my emacs. In fact, it first started as; let's try to switch from doom-emacs³ to nano-emacs⁴. But, doing so, I realized I wouldn't be able to reach the quality and optimization provided by doom-emacs myself. So instead of doing this, I first tried to copy the theme of nano. Then I realized one of the biggest factor of nano look & feel was its usage of "Roboto Mono" but with weight light (or Thin).

See

OK so… I just tried to match the theme colors. It was easy to create a theme with matching colors. But, to make it really looks like nano; almost monochrome with two accent colors; it would mean a lot more work than anyone could expect. For most emacs mode you probably need to add a set of specific font faces. This choice is also what makes nano looks so good too. This is not just about the color, but about a lot more details than that. Using the good colors only at the right place is difficult to achieve. And not only the colors, but also, the correct fonts, the spacing of text elements etc…

Unfortunately if you want the nano look and feel in doom, it is much more work than just copying the nano theme.

But this research of look and feel opened the door to using thin fonts in emacs. And also tweaking the fonts which really improve the look & feel of emacs.

With this conf, I do not use the same font for coding and for writing prose or a blog post with code blocks. So far, I like this new look and feel.

Bonuses

Thin Code fonts

After lot of try, I finally switched my default coding font to SauceCodePro Nerd Font Mono with weight semi-light. This is a clone of Adobe SourceCode Pro. And on a retina display it is really nice to use.

(setq doom-font (font-spec :family "SauceCodePro Nerd Font Mono" :size 12 :weight 'semi-light)
      doom-variable-pitch-font (font-spec :family "iMWritingDuoS Nerd Font" :size 14))

An unfinished nano theme for doom

Even though the result is not 100% satisfactory, you could start using my work. Save this file into ~/.doom.d/themes/doom-nano-theme.el:

;;; doom-nano-theme.el --- inspired by Nicolas Rougier nano-theme -*- lexical-binding: t; no-byte-compile: t; -*-
;;
;; Author: Yann Esposito <https://yannesposito.com>
;; Created: August 16, 2021
;; Version: 1.0.0
;; Keywords: custom themes, faces
;; Homepage: https://github.com/hlissner/emacs-doom-themes
;; Package-Requires: ((emacs "25.1") (cl-lib "0.5") (doom-themes "2.2.1"))
;;
;;; Commentary:
;;
;; Ported from nano-theme: https://github.com/rougier/nano-theme
;;
;;; Code:

(require 'doom-themes)

;;; Variables

(defgroup doom-plain-theme nil
  "Options for the `doom-plain' theme."
  :group 'doom-themes)

(defcustom doom-plain-padded-modeline doom-themes-padded-modeline
  "If non-nil, adds a 4px padding to the mode-line.
Can be an integer to determine the exact padding."
  :group 'doom-plain-theme
  :type '(or integer boolean))
;;
;;; Theme definition

(def-doom-theme doom-nano
  "Theme inspired by Nicolas Rougier nano-theme"

  ;; name      default/256/16
  ((nano-color-foreground '("#37474F")) ;; Blue Grey / L800
   (nano-color-background '("#FFFFFF")) ;; White
   (nano-color-highlight  '("#FAFAFA")) ;; Very Light Grey
   (nano-color-critical   '("#FF6F00")) ;; Amber / L900
   (nano-color-salient    '("#673AB7")) ;; Deep Purple / L500
   (nano-color-strong     '("#000000")) ;; Black
   (nano-color-popout     '("#FFAB91")) ;; Deep Orange / L200
   (nano-color-subtle     '("#ECEFF1")) ;; Blue Grey / L50
   (nano-color-faded      '("#B0BEC5")) ;; Blue Grey / L200

   (bg         nano-color-background)
   (bg-alt     nano-color-highlight)
   (base0      '("#18282f"))
   (base1      '("#24323a"))
   (base2      '("#556066"))
   (base3      '("#6f787d"))
   (base4      '("#8a9296"))
   (base5      '("#a6acaf"))
   (base6      '("#e7e8e9"))
   (base7      '("#f6f6f6"))
   (base8      '("#fafafa"))
   (fg         nano-color-foreground)
   (fg-alt     nano-color-faded)

   (grey       fg)
   (red        fg)
   (blue       fg)
   (dark-blue  fg)
   (orange     fg)
   (green      fg)
   (teal       fg)
   (yellow     fg)
   (magenta    fg)
   (violet     fg)
   (cyan       fg)
   (dark-cyan  fg)


   ;; face categories -- required for all themes
   (highlight      nano-color-salient)
   (vertical-bar   base5)
   (selection      nano-color-highlight)
   (builtin        nano-color-salient)
   (comments       nano-color-faded)
   (doc-comments   nano-color-faded)
   (constants      nano-color-strong)
   (functions      nano-color-salient)
   (keywords       nano-color-strong)
   (methods        nano-color-salient)
   (operators      nano-color-strong)
   (type           nano-color-strong)
   (strings        base0)
   (variables      base0)
   (numbers        base0)
   (region         base4)
   (error          nano-color-critical)
   (warning        nano-color-popout)
   (success        nano-color-salient)
   (vc-modified    nano-color-salient)
   (vc-added       fg-alt)
   (vc-deleted     nano-color-critical)

   ;; custom categories
   (-modeline-pad
    (when doom-plain-padded-modeline
      (if (integerp doom-plain-padded-modeline) doom-plain-padded-modeline 4)))

   (modeline-bg              (doom-darken bg-alt 0.15))
   (modeline-bg-alt          (doom-darken bg-alt 0.1))
   (modeline-bg-inactive     (doom-darken bg-alt 0.1))
   (modeline-bg-inactive-alt bg-alt)
   (modeline-fg              fg)
   (modeline-fg-alt          (doom-darken modeline-bg-inactive 0.35)))

  ;;;; Base theme face overrides
  ((error   :underline `(:style wave :color ,error))
   (warning :underline `(:style wave :color ,warning))
   ((font-lock-constant-face &override)      :slant 'italic)
   ((font-lock-comment-face &override)       :slant 'italic)
   ((font-lock-function-name-face &override) :slant 'italic)
   ((font-lock-type-face &override)          :slant 'italic)
   ;;(hl-line :background base8)
   ((line-number &override) :foreground base3)
   ((line-number-current-line &override) :foreground base2)
   (mode-line
    :background modeline-bg :foreground modeline-fg
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg)))
   (mode-line-inactive
    :background modeline-bg-inactive :foreground modeline-fg-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-inactive)))
   (mode-line-emphasis :foreground highlight)

   ;;;; doom-modeline
   (doom-modeline-bar :background modeline-bg)
   (doom-modeline-bar-inactive :inherit 'doom-modeline-bar)
   (doom-modeline-project-dir :foreground fg)
   (doom-modeline-buffer-file :foreground fg)
   (doom-modeline-buffer-modified :weight 'bold :foreground "#000000")
   (doom-modeline-panel :inherit 'mode-line-highlight :background base3 :foreground fg)
   ;;;; ivy
   (ivy-posframe :background bg-alt)
   ;;;; magit
   ((magit-diff-added-highlight &override)   :foreground fg :background (doom-blend vc-added bg 0.3))
   ((magit-diff-removed &override)           :foreground (doom-lighten fg 0.4) :background (doom-blend vc-deleted bg 0.1))
   ((magit-diff-removed-highlight &override) :foreground fg :background (doom-blend vc-deleted bg 0.22))
   ;;;; lsp-mode
   (lsp-headerline-breadcrumb-symbols-face :foreground keywords :weight 'bold)
   ;;;; outline <built-in>
   (outline-1 :slant   'italic :foreground fg-alt)
   (outline-2 :inherit 'outline-1 :foreground base2)
   (outline-3 :inherit 'outline-2)
   (outline-4 :inherit 'outline-3)
   (outline-5 :inherit 'outline-4)
   (outline-6 :inherit 'outline-5)
   (outline-7 :inherit 'outline-6)
   (outline-8 :inherit 'outline-7)

   (org-level-1 :inherit 'org-level-1 :foreground nano-color-strong)
   (org-level-2 :inherit 'org-level-2 :foreground nano-color-strong)
   (org-level-3 :inherit 'org-level-3 :foreground nano-color-strong)
   (org-level-4 :inherit 'org-level-4 :foreground nano-color-strong)
   (org-level-5 :inherit 'org-level-5 :foreground nano-color-strong)
   (org-level-6 :inherit 'org-level-6 :foreground nano-color-strong)
   (org-level-7 :inherit 'org-level-7 :foreground nano-color-strong)
   (org-level-8 :inherit 'org-level-8 :foreground nano-color-strong)

   (org-code     :inherit 'org-code
                 :foreground nano-color-salient
                 :weight 'bold)
   (org-verbatim :inherit 'org-verbatim
                 :foreground nano-color-salient
                 :weight 'bold)
   (org-upcoming-deadline :inherit 'org-upcoming-deadline
                          :foreground nano-color-critical
                          :weight 'bold)
   (org-upcoming-distant-deadline :inherit 'org-upcoming-distant-deadline
                                  :foreground nano-color-salient)

   (org-habit-overdue-face
    :inherit 'org-habit-overdue-face
    :background fg-alt)
   (org-habit-overdue-future-face
    :inherit 'org-habit-overdue-future-face
    :background nano-color-subtle)
   (org-habit-alert-face
    :inherit 'org-habit-alert-face
    :background nano-color-critical)
   (org-habit-alert-future-face
    :inherit 'org-habit-alert-future-face
    :background nano-color-subtle)

   (org-scheduled-today :inherit 'org-scheduled-today :foreground fg)
   (org-scheduled-previously :inherit 'org-scheduled-previously :foreground fg)

   ;;;; org <built-in>
   ((org-block &override) :background bg-alt)
   ((org-block-begin-line &override) :foreground base5)
   ;;;; solaire-mode
   (solaire-mode-line-face
    :inherit 'mode-line
    :background modeline-bg-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-alt)))
   (solaire-mode-line-inactive-face
    :inherit 'mode-line-inactive
    :background modeline-bg-inactive-alt
    :box (if -modeline-pad `(:line-width ,-modeline-pad :color ,modeline-bg-inactive-alt)))))

;;; doom-plain-theme.el ends here

You will probably need more work to achieve the colors you expect. For that, using SPC-u C-x = will probably be useful. It will show the font face under the cursor.

Best of luck.

I wanted to write a few articles about great shows lot of people around me do not know about. Utopia (2013) is one of these TV Shows that deserve more attention.

The filmography is quite original. I have never seen another film/TV show with a similar atmosphere. The usage of bright colors is magistral. Flashy yellow, green, red.

The soundtrack is also pretty surprising original and enjoyable.

The acting is really great. Actors are doing a great job and are quite relatable. These are not the (super)heroes you are used to. We are far away from shows where every girl is a bimbo and every guy is a Chippendale.

The scenario, in regard to the recent events related to COVID19 is just perfect. I do not want to reveal too much. But let's just say the current real events are close to the events predicted in this show.

The Surrealistic Humor atmosphere make the viewing experience quite exceptionnal. There is a mix between nonchalance and extreme violence. It really feels surrealist. In this show some people act with extreme violence as if there is no choice.

As a conclusion, if you are looking for a very innovative TV show then search no further this one is a great original choice. If you are still unsure, just watch the opening scene, it is quite incredible.

ps: Also try to get the original content. Amazon Prime apparently cut some very important scene and also changed the ratio which hurt the very good image work.

This article will dig a bit deeper about my Makefile based static website generator. In a previous article I just gave the rationale and an overview to do it yourself. Mainly it is very fast and portable.

A few goals reached by my current build system are:

1. Be fast and make the minimal amount of work as possible. I don't want to rebuild all the html pages if I only change one file.
2. Source file format agnostic. You can use markdown, org-mode or even directly write html.
3. Support gemini
4. Optimize size: minify HTML, CSS, images
5. Generate an index page listing the posts
6. Generate RSS/atom feed (for both gemini and http)

make will take care of handling the dependency graph to minimize the amount of effort when a change occurs in the sources. For some features, I built specifics small shell scripts. For example to be absolutely agnostic in the source format for my articles I generate the RSS out of a tree of HTML files. But taking advantage of make, I generate an index cache to transform those HTML into XML which will be faster to use to build different indexes. To make those transformations I use very short a shell scripts.

Makefile overview

A Makefile is made out of rules. The first rule of your Makefile will be the default rule. The first rule of my Makefile is called all.

A rule as the following format:

target: file1 file2
    cmd --input file1 file2 \
        --output target

if target does not exists, then make will look at its dependencies. If any of its dependencies need to be updated, it will run all the rules in the correct order to rebuild them and finally run the script to build target. A file needs to be updated if one of its dependency needs to be updated or is newer.

The usual use case of make is about building a single binary out of many source files. But for a static website, we need to generate a lot of files from a lot of files. So we construct the rules like this:

all: site

# build a list of files that will need to be build
DST_FILES := ....
# RULES TO GENERATE DST_FILES
ALL += $(DST_FILES)

# another list of files
DST_FILES_2 := ....
# RULES TO GENERATE DST_FILES_2
ALL += $(DST_FILES_2)

site: $(ALL)

In my Makefile I have many similar block with the same pattern.

1. I retrieve a list of source files
2. I construct the list of destination files (change the directory, the extension)
3. I declare a rule to construct these destination files
4. I add the destination files to the ALL variable.

I have a block for:

• raw assets I just want copied
• images I would like to compress for the web
• html I would like to generate from org mode files via pandoc
• gmi I would like to generate from org mode files
• xml files I use as cache to build different index files
• index.html file containing a list of my posts
• rss.xml file containing a list of my posts
• gemini-atom.xml file containing a list of my posts

Assets

The rules to copy assets will be a good first example.

1. find all assets in src/ directory
2. generate all assets from these files in _site/ directory
3. make this rule a dependency on the all rule.

SRC_ASSETS := $(shell find src -type f)
DST_ASSETS := $(patsubst src/%,_site/%,$(SRC_ASSETS))
_site/% : src/%
    @mkdir -p "$(dir $@)"
    cp "$<" "$@"
.PHONY: assets
assets: $(DST_ASSETS)
ALL += assets

OK, this looks terrible. But mainly:

• SRC_ASSETS will contain the result of the command find.
• DST_ASSETS will contain the files of SRC_ASSETS but we replace the src/ by _site/.
• We create a generic rule; for all files matching the following pattern _site/%, look for the file src/% and if it is newer (in our case) then execute the following commands:
  • create the directory to put _site/% in
  • copy the file

About the line @mkdir -p "$(dir $@)":

• the @ at the start of the command simply means that we make this execution silent.
• The $@ is replaced by the target string.
• And $(dir $@) will generate the folder name of $@.

For the line with cp, you just need to know that ~$<~ will represent the first dependency.

My Makefile is composed of similar blocks, where I replace the first find command to match specific files and where I use different building rules. An important point is that the rules must be the most specific possible. This is because make will use the most specific rule in case of ambiguity. For example, the matching rule _site/%: src/% will match all files in the src/ dir. But if we want to treat CSS files with another rule we could write:

_site/%.css: src/%.css
    minify "$<" "$@"

And if the selected file is a CSS file, this rule will be selected.

Prelude

I start with variables declarations:

all: site
# directory containing the source files
SRC_DIR ?= src
# directory that will contain the site files
DST_DIR ?= _site
# a directory that will contain a cache to speedup indexing
CACHE_DIR ?= .cache

# options to pass to find to prevent matching files in the src/drafts
# directory
NO_DRAFT := -not -path '$(SRC_DIR)/drafts/*'
# option to pass to find to not match  org files
NO_SRC_FILE := ! -name '*.org'

CSS

Here we go; the same simple pattern for CSS files.

# CSS
SRC_CSS_FILES := $(shell find $(SRC_DIR) -type f -name '*.css')
DST_CSS_FILES := $(patsubst $(SRC_DIR)/%,$(DST_DIR)/%,$(SRC_RAW_FILES))
$(DST_DIR)/%.css : $(SRC_DIR)/%.css
    @mkdir -p "$(dir $@)"
    minify "$<" > "$@"
.PHONY: css
css: $(DST_CSS_FILES)
ALL += css

This is very similar to the block for raw assets. The difference is just that instead of using cp we use the minify command.

ORG → HTML

Now this one is more complex but is still follow the same pattern.

# ORG -> HTML
EXT ?= .org
SRC_PANDOC_FILES ?= $(shell find $(SRC_DIR) -type f -name "*$(EXT)" $(NO_DRAFT))
DST_PANDOC_FILES ?= $(patsubst %$(EXT),%.html, \
                        $(patsubst $(SRC_DIR)/%,$(DST_DIR)/%, \
                            $(SRC_PANDOC_FILES)))
PANDOC_TEMPLATE ?= templates/post.html
MK_HTML := engine/mk-html.sh
PANDOC := $(MK_HTML) $(PANDOC_TEMPLATE)
$(DST_DIR)/%.html: $(SRC_DIR)/%.org $(PANDOC_TEMPLATE) $(MK_HTML)
    @mkdir -p "$(dir $@)"
    $(PANDOC) "$<" "$@.tmp"
    minify --mime text/html "$@.tmp" > "$@"
    @rm "$@.tmp"
.PHONY: html
html: $(DST_PANDOC_FILES)
ALL += html

So to construct DST_PANDOC_FILES this time we also need to change the extension of the file from org to html. We need to provide a template that will be passed to pandoc.

And of course, as if we change the template file we would like to regenerate all HTML files we put the template as a dependency. But importantly not at the first place. Because we use $< that will be the first dependency.

I also have a short script instead of directly using pandoc. It is easier to handle toc using the metadatas in the file. And if someday I want to put the template in the metas, this will be the right place to put that.

The mk-html.sh is quite straightforward:

#!/usr/bin/env bash
set -eu

# put me at the top level of my project (like Makefile)
cd "$(git rev-parse --show-toplevel)" || exit 1
template="$1"
orgfile="$2"
htmlfile="$3"

# check if there is the #+OPTIONS: toc:t
tocoption=""
if grep -ie '^#+options:' "$orgfile" | grep 'toc:t'>/dev/null; then
    tocoption="--toc"
fi

set -x
pandoc $tocoption \
       --template="$template" \
       --mathml \
       --from org \
       --to html5 \
       --standalone \
       $orgfile \
       --output "$htmlfile"

Once generated I also minify the html file. And, that's it. But the important part is that now, if I change my script or the template or the file, it will generate the dependencies.

Indexes

We often need indexes to build a website. Typically to list the latest articles, build the RSS file. So for sake of simplicity, I decided to build my index as a set of XML files. Of course, this could be optimizide, by using SQLite for example. But this will already be really fast.

For every generated html file I will generate a clean XML file with hxclean. Once cleaned, it will be easy to access a specific node of in these XML files.

# INDEXES
SRC_POSTS_DIR ?= $(SRC_DIR)/posts
DST_POSTS_DIR ?= $(DST_DIR)/posts
SRC_POSTS_FILES ?= $(shell find $(SRC_POSTS_DIR) -type f -name "*$(EXT)")
RSS_CACHE_DIR ?= $(CACHE_DIR)/rss
DST_XML_FILES ?= $(patsubst %.org,%.xml, \
                        $(patsubst $(SRC_POSTS_DIR)/%,$(RSS_CACHE_DIR)/%, \
                            $(SRC_POSTS_FILES)))
$(RSS_CACHE_DIR)/%.xml: $(DST_POSTS_DIR)/%.html
    @mkdir -p "$(dir $@)"
    hxclean "$<" > "$@"
.PHONY: indexcache
indexcache: $(DST_XML_FILES)
ALL += indexcache

This rule will generate for every file in site/posts/*.html a corresponding xml file (hxclean takes an HTML an try its best to make an XML out of it).

HTML Index

Now we just want to generate the main index.html page at the root of the site. This page should list all articles by date in reverse order.

The first step is to take advantage of the cache index. For every XML file I generated before I should generate the small HTML block I want for every entry. For this I use a script mk-index-entry.sh. He will use hxselect to retrieve the date and the title from the cached XML files. Then generate a small file just containing the date and the link.

Here is the block in the Makefile:

DST_INDEX_FILES ?= $(patsubst %.xml,%.index, $(DST_XML_FILES))
MK_INDEX_ENTRY := ./engine/mk-index-entry.sh
INDEX_CACHE_DIR ?= $(CACHE_DIR)/rss
$(INDEX_CACHE_DIR)/%.index: $(INDEX_CACHE_DIR)/%.xml $(MK_INDEX_ENTRY)
    @mkdir -p $(INDEX_CACHE_DIR)
    $(MK_INDEX_ENTRY) "$<" "$@"

It means: for every .xml file generate a .index file with mk-index-entry.sh.

#!/usr/bin/env zsh

# prelude
cd "$(git rev-parse --show-toplevel)" || exit 1
xfic="$1"
dst="$2"
indexdir=".cache/rss"

# HTML Accessors (similar to CSS accessors)
dateaccessor='.yyydate'
# title and keyword shouldn't be changed
titleaccessor='title'
finddate(){ < $1 hxselect -c $dateaccessor | sed 's/\[//g;s/\]//g;s/ .*$//' }
findtitle(){ < $1 hxselect -c $titleaccessor }

autoload -U colors && colors

blogfile="$(echo "$xfic"|sed 's#.xml$#.html#;s#^'$indexdir'/#posts/#')"
printf "%-30s" $blogfile
d=$(finddate $xfic)
echo -n " [$d]"
rssdate=$(formatdate $d)
title=$(findtitle $xfic)
keywords=( $(findkeywords $xfic) )
printf ": %-55s" "$title ($keywords)"
{ printf "\\n<li>"
  printf "\\n<span class=\"pubDate\">%s</span>" "$d"
  printf "\\n<a href=\"%s\">%s</a>" "${blogfile}" "$title"
  printf "\\n</li>\\n\\n"
} >> ${dst}

echo " [${fg[green]}OK${reset_color}]"

Then I use these intermediate files to generate a single bigger index file.

HTML_INDEX := $(DST_DIR)/index.html
MKINDEX := engine/mk-index.sh
INDEX_TEMPLATE ?= templates/index.html
$(HTML_INDEX): $(DST_INDEX_FILES) $(MKINDEX) $(INDEX_TEMPLATE)
    @mkdir -p $(DST_DIR)
    $(MKINDEX)
.PHONY: index
index: $(HTML_INDEX)
ALL += index

This script is a big one, but it is not that complex. For every file, I generate a new file DATE-dirname. I sort them in reverse order and put their content in the middle of an HTML file.

Important note: this file updates only if the index change.

The first part of the script creates files with the creation date in their metadatas. The created file name will contain the creation date, this will be helpful later.

#!/usr/bin/env zsh

autoload -U colors && colors
cd "$(git rev-parse --show-toplevel)" || exit 1
# Directory
webdir="_site"
indexfile="$webdir/index.html"
indexdir=".cache/rss"
tmpdir=$(mktemp -d)

echo "Publishing"

dateaccessor='.pubDate'
finddate(){ < $1 hxselect -c $dateaccessor }
# generate files with <DATE>-<FILENAME>.index
for fic in $indexdir/**/*.index; do
    d=$(finddate $fic)
    echo "${${fic:h}:t} [$d]"
    cp $fic $tmpdir/$d-${${fic:h}:t}.index
done

Then I use these files to generate a file that will contain the body of the HTML.

# for every post in reverse order
# generate the body (there is some logic to group by year)
previousyear=""
for fic in $(ls $tmpdir/*.index | sort -r); do
    d=$(finddate $fic)
    year=$( echo "$d" | perl -pe 's#(\d{4})-.*#$1#')
    if (( year != previousyear )); then
        if (( previousyear > 0 )); then
            echo "</ul>" >> $tmpdir/index
        fi
        previousyear=$year
        echo "<h3 name=\"${year}\" >${year}</h3><ul>" >> $tmpdir/index
    fi
    cat $fic >> $tmpdir/index
done
echo "</ul>" >> $tmpdir/index

And finally, I render the HTML using a template within a shell script:

title="Y"
description="Most recent articles"
author="Yann Esposito"
body=$(< $tmpdir/index)
date=$(LC_TIME=en_US date +'%Y-%m-%d')

# A neat trick to use pandoc template within a shell script
# the pandoc templates use $x$ format, we replace it by just $x
# to be used with envsubst
template=$(< templates/index.html | \
    sed 's/\$\(header-includes\|table-of-content\)\$//' | \
    sed 's/\$if.*\$//' | \
    perl -pe 's#(\$[^\$]*)\$#$1#g' )
{
    export title
    export author
    export description
    export date
    export body
    echo ${template} | envsubst
} > "$indexfile"

rm -rf $tmpdir
echo "* HTML INDEX [done]"

RSS

My RSS generation is similar to the system I used to generate the index file. I just slightly improved the rules.

The Makefile blocks look like:

# RSS
DST_RSS_FILES ?= $(patsubst %.xml,%.rss, $(DST_XML_FILES))
MK_RSS_ENTRY := ./engine/mk-rss-entry.sh
$(RSS_CACHE_DIR)/%.rss: $(RSS_CACHE_DIR)/%.xml $(MK_RSS_ENTRY)
    @mkdir -p $(RSS_CACHE_DIR)
    $(MK_RSS_ENTRY) "$<" "$@"

RSS := $(DST_DIR)/rss.xml
MKRSS := engine/mkrss.sh
$(RSS): $(DST_RSS_FILES) $(MKRSS)
    $(MKRSS)

.PHONY: rss
rss: $(RSS)
ALL += rss

Gemini

I wrote a minimal script to transform my org files to gemini files. I also need to generate an index and an atom file for gemini:

# ORG -> GEMINI
EXT := .org
SRC_GMI_FILES ?= $(shell find $(SRC_DIR) -type f -name "*$(EXT)" $(NO_DRAFT))
DST_GMI_FILES ?= $(subst $(EXT),.gmi, \
                        $(patsubst $(SRC_DIR)/%,$(DST_DIR)/%, \
                            $(SRC_GMI_FILES)))
GMI := engine/org2gemini.sh
$(DST_DIR)/%.gmi: $(SRC_DIR)/%.org $(GMI) engine/org2gemini_step1.sh
    @mkdir -p $(dir $@)
    $(GMI) "$<" "$@"
ALL += $(DST_GMI_FILES)
.PHONY: gmi
gmi: $(DST_GMI_FILES)

# GEMINI INDEX
GMI_INDEX := $(DST_DIR)/index.gmi
MK_GMI_INDEX := engine/mk-gemini-index.sh
$(GMI_INDEX): $(DST_GMI_FILES) $(MK_GMI_INDEX)
    @mkdir -p $(DST_DIR)
    $(MK_GMI_INDEX)
ALL += $(GMI_INDEX)
.PHONY: gmi-index
gmi-index: $(GMI_INDEX)

# RSS
GEM_ATOM := $(DST_DIR)/gem-atom.xml
MK_GEMINI_ATOM := engine/mk-gemini-atom.sh
$(GEM_ATOM): $(DST_GMI_FILES) $(MK_GEMINI_ATOM)
    $(MK_GEMINI_ATOM)
ALL += $(GEM_ATOM)
.PHONY: gmi-atom
gmi-atom: $(GMI_ATOM)

.PHONY: gemini
gemini: $(DST_GMI_FILES) $(GMI_INDEX) $(GEM_ATOM)