For a simple Rust project, we could use a configuration that looks like this:

1
2
3
4
5
6

image: "scorpil/rust:stable"

test:cargo:
  script:
  - rustc --version && cargo --version      # Print version info for debugging
  - time cargo test --verbose --jobs 1 --release # Don't paralize to make errors more readable

If you build a project with this, it will work (and it’ll use a Docker image for buzzword compliance), but it’ll be quite slow, several minutes at least. Most of that time is spent downloading your dependencies and compiling them - on every single build, even if they are unchanged. If you run cargo test locally, even if it needs to compile your code, it should take roughly 10 seconds (probably less).

In order to get fast test runs with Rust, some configuration is necessary. Let’s walk through changes to the .gitlab-ci.yml file that’ll speed things up.

1
2
3
4
5
6

test:cargo:
  ...
  cache:
    paths:
      - target/
      - cargo/

This is necessary to ensure that artifacts such as dependencies are retained between builds. Without this, upon every build, you will see lines like this in the build log:

1
2
3

Downloading cookie v0.2.5
...
Compiling cookie v0.2.5

To ensure that dependencies are cached correctly, we need to set the $CARGO_HOME to be inside the build directory (I’m not sure why, but if you try to cache it as is, it doesn’t work):

1
2

variables:
  CARGO_HOME: $CI_PROJECT_DIR/cargo

$CI_PROJECT_DIR is defined by GitLab to be the directory that it unpacks your project in and runs your build.

With this all in place, when you next build your project (after one build to fill the cache with your dependencies), you should see this instead:

1

Fresh cookie v0.2.5

See that? Everybody likes fresh cookies. Cargo doesn’t need to compile this library because the built library was in the cache. This reduced my small project’s build time from ~4 minutes to ~1.2 minutes.

The final .gitlab-ci.yml file looks like this now:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

image: "scorpil/rust:stable"

variables:
  CARGO_HOME: $CI_PROJECT_DIR/cargo

test:cargo:
  script:
  - du -hs target
  - du -hs cargo
  - rustc --version && cargo --version      # Print version info for debugging
  - time cargo test --verbose --jobs 1 --release # Don't paralize to make errors more readable
  cache:
    paths:
      - target/
      - cargo/

Pseudoclassical OO

The following code is in the style referred to as pseudoclassical by Douglas Crockford in Javascript: The Good Parts:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

var Person = function(name, empathy) {
    this.name = name;
    this.empathy = empathy || 0;
};

Person.prototype.get_name = function() {
    return this.name;
};

Person.prototype.get_empathy = function() {
    return this.empathy;
};

Person.prototype.change_empathy = function(amount) {
    this.empathy = this.empathy + amount;
    return this.empathy;
};

Person.prototype.status = function() {
    return this.get_name() + " has " + this.get_empathy() + " empathy";
};

var Player = function(name, empathy) {
    this.name = name;
    this.empathy = empathy || 0;
};

Player.prototype = new Person();

var Mob = function(name, empathy) {
    this.name = name;
    this.empathy = empathy || 0;
};

Mob.prototype = new Person();

And we can use the ‘classes’ like so:

1
2
3
4
5
6
7
8

var player = new Player("Ada", 100);
console.log(player.status()); // 'Ada has 100 empathy'

var mob = new Mob("Bob", -10);
console.log(mob.status()); // 'Bob has -10 empathy'

mob.change_empathy(2);
console.log(mob.status()); // 'Bob has -8 empathy'

There are a few things to note here:

• There are no private fields or methods, it’s not feasible to add them
• There is duplication in constructors that lie in an inheritance chain (i.e. where an object’s prototype is set to another object)
• It’s horribly verbose (a matter of personal taste however)
• Due to the lack of private fields, you can’t have a method called name() that returns a field called name - one will overwrite the other in this style

The limits of inheritance

Even ignoring these issues (and there are other ways to create objects that work around this limitation), object oriented inheritance is not the most flexible way to design software. For many domains of problems, composition or augmentation trumps inheritance. Let’s examine a motivating example. In the following class diagram we have some domain objects for a game. Players and Mobs are Person(s), Person(s) and Rucksacks are Objects. All is well with this view of the world, no code is duplicated between objects and it all works as intended.

Here are some unit tests of these objects to illustrate their functionality:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

var JS = require("jstest"),
    components = require("../lib/components");

JS.Test.describe("components", function() {
    this.before(function() {
        this.rucksack = components.rucksack({name: "Bag of Holding"});
        this.another_rucksack = components.rucksack({name: "Bigger bag"});
        this.spell = components.spell({name: "Potion of empathy"});
    });

    this.describe("container", function() {
        this.it("can contain other containers", function() {
            this.rucksack.add(this.spell);
            this.another_rucksack.add(this.rucksack);
            this.assertEqual(1, this.another_rucksack.quantity());
            this.assertEqual("Bag of Holding",
                             this.another_rucksack.items()[0].name());
        });
        this.it("can contain other objects", function() {
            this.rucksack.add(this.spell);
            this.assertEqual(1, this.rucksack.quantity());
            this.assertEqual("Potion of empathy",
                             this.rucksack.items()[0].name());
        });
    });
});

Now say we want to extend our player objects to have an inventory (or container). We could approach this in a number of ways but inheritance is not one of them. We already have rucksack and person inheriting from object so we can’t just create container as a superclass of those. We need a way to extend both those objects and extract the container-related behaviour and state so we don’t have code duplication. In Ruby, we might create container as a Module and include it in our classes.

The method pattern in JS allows us to do something quite similar, but first, let’s write a failing test:

1
2
3
4
5
6
7
8

this.describe("player", function() {
    this.it("has an inventory", function() {
        this.assertEqual(0, this.player.quantity());
        this.player.add(this.spell);
        this.assertEqual(1, this.player.quantity());
        this.assertEqual("Potion of empathy", this.player.items()[0].name());
    });
});

The Module Pattern

Now we can go ahead and examine how to construct the relationship between objects that we need. This diagram approximates the idea (I’m not exactly clear what options to pass GraphViz to make the container obviously indicate it is included as a module):

The basic ‘shape’ of an object using ‘functional composition’ looks like the following:

1
2
3
4
5
6
7
8
9

var object = function(spec) {
    var that = {};

    that.name = function() {
        return spec.name;
    };

    return that;
};

The spec argument is the object to be augmented (sometimes that’s as simple as being the parent object). The that object which gets returned constitutes the public interface that this object returns. Because spec is passed in and that is returned, this object constructor is a base class.

The container constructor function looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

var container = function(that) {
    var inventory = [];

    that.add = function(item) {
        inventory.push(item);
    };

    that.quantity = function() {
        return inventory.length;
    };

    that.items = function() {
        return inventory;
    };

    return that;
};

Unlike the previous example, container exists to decorate the provided object with more functionality. If you needed to provide defaults for this object specifically, you can pass a spec argument in also but it wasn’t necessary here. Crucially, the inventory variable is lexically scoped to the container constructor, meaning no code gets access to it except that inside the container constructor. This is in contrast to Modules in Ruby where everything, fields and all, gets included into a Class and becomes part of that class.

The person object is a little different:

1
2
3
4
5
6
7
8
9
10
11
12
13

var person = function(spec) {
    var that = container(object(spec)),
        empathy = spec.empathy || 0;

    that.change_empathy = function(amount) {
        empathy = empathy + amount;
    };
    that.empathy = function() {
        return empathy;
    };

    return that;
};

We’re back to using a spec argument here so that we can instantiate a person with some default amount of empathy. The that = container(object(spec)) stands in for traditional object inheritance.

After we’ve created the container object and wired it up to person and rucksack, the code for concrete objects in our toy example looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

module.exports.rucksack = function(spec) {
    var that = container(object(spec));
    return that;
};

module.exports.mob = function(spec) {
    var that = person(spec);
    return that;
};

module.exports.player = function(spec) {
    var that = person(spec);
    return that;
};

module.exports.spell = function(spec) {
    var that = object(spec);

    that.affect = function(obj) {
        if (that.inRange(obj)) {
            obj.change_empathy(1);
        }
    };

    return that;
};

Although the diagram above implies a strict inheritance hierarchy, that is not necessarily the case. The difference between object and container is subtle, with object hiding the spec object and returning a new object (just as person and rucksack do), whereas container just adds methods to an object.

One last thing! It might be apparent, but it’s worth calling out the other advantage this gives us: dynamic runtime extensions. If you want the spell object to be able to contain items, you could do so at any stage, running:

1
2
3
4

var my_spell = spell({name: "Confusing potion"});
// my_spell doesn't have an inventory
container(my_spell);
console.log(my_spell.quantity()); // '0'

This code modifies the my_spell variable, but no other variables that use the spell constructor function. This could be useful in games where items can interact, giving each other different functionality (e.g. imagine a potion that allows bookcases to float or drinking a flying potion adds the flying component to a player). In those cases, there’s no need for complex if/else/switch-style programming, just augment the object with the desired module and that’s it!

time java -cp cljs.jar:src clojure.main build.clj

On the Raspberry Pi 1 Model B this takes 61.59 seconds, whereas on the new Raspberry Pi 2 Model B this takes 18.42 seconds!

For comparison, on my MacBook Air (11-inch, Mid 2013, 1.4 GHz Intel Core i5, 8GB RAM), while lots of software was running, so not a complete fair test (the Pis were booted up into the console, not even running X) this takes 5.75 seconds.

Considering the difference in specifications of these machines, the Raspberry Pi 2 is remarkably fast. I could even load up and play with Sonic Pi pretty quickly (on the original Model B that took long enough to make a cup of tea).

Looks like I’m going to be drinking less tea in the future.

But the emphasis on the finished product doesn’t interest me as much. I’d rather open source the process by which art is made rather than the result.

The new wave of digitally assembled music opens the way to truly open source music - when music can have source code, can have a version history - we can all learn from how it gets made.

Put your music in the public domain if you like, but don’t forget to release the things you learned along the way.

It all started when I borrowed the book Simple Composition by Charles Wuorinen from the Sussex University library (I was studying Computer Science & Artificial Intelligence at the time, some years ago).

More recently I tracked down a copy so that I could have a go at writing twelve-tone music. This book explains the basics of composing in the twelve tone technique created by Arnold Schoenberg. Some days ago I started playing around with the concepts from that book in Overtone. It was a lot of fun.

If you’re not familiar with twelve-tone composition or serialism in music, you could do a lot worse than to watch this awesome video by ViHart:

The code

I used a library called Leipzig written by Chris Ford to build up the composition from an initial tone row (a set of 12 integers), via several functional transformations of the tone row and into 3 instrumental parts: piano, violin and cello.

Since this piece uses a randomly-generated tone row and randomly-generated sequence of both note duration and playing style (vibrato, non-vibrato or pizzicato), it’s not actually reproducible from the code. The code is useful mainly for encoding the basic form of the piece and the parameters of the work (such as instruments, length of the piece, which elements are random and which aren’t etc.).

In future I will work out some way to capture the notes being generated. The simplest way to do this would be to save all the structures produced by the piano-trio function into an EDN file, which could be trivially played back at a later date. Transformation of that data into traditional musical notation wouldn’t be too tricky either.

The structure of this piece consists of 3 instrumental parts played in parallel. The main function is piano-trio and is inline below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

(defn piano-trio
  [bpm tone-row]
  (in-tempo
   bpm
   (with
    (->> (total-serial tone-row)
         (play-on :cello)
         (serial-style :cello))
    (->> (total-serial tone-row)
         (play-on :violin)
         (serial-style :violin))
    (->> (total-serial tone-row)
         (play-on :piano)
         (serial-style :piano)))))

The most interesting aspect as it relates to musical transformation is the definition of retrograde, inversion and retrograde-inversion:

1
2
3
4
5
6
7
8
9
10

(defn retrograde
  [row]
  (reverse row))

(defn inversion
  [row]
  (map - row))

(def retrograde-inversion
  (comp retrograde inversion))

It was for this reason - the applicability of pure functions and functional composition - that I felt overtone and Clojure were such a good fit for writing serialist music. It’s even more applicable to 12-tone composition than tonal (diatonic) music since the rules are so much more mechanistic.

The recording

I had a lot of fun exploring serialism through Overtone and the Leipzig library. I was inspired to release this little track and to spend more time on code like this when I watched a great talk called Make Art, Not Apps by @ThisIsJohnBrown.

Each example plugin from the source code of this book has a build script that looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

#!/bin/sh
# Set the variable MCSERVER to ~/Desktop/server
# unless it's already set
: ${MCSERVER:="$HOME/Desktop/server"}
MODS="$MCSERVER"/CanaryMod.jar

# Make the build directories if they aren't there.
# Throw away any error if they are.
mkdir bin 2>/dev/null
mkdir dist 2>/dev/null

# Remove any previous build products
rm -f bin/*/*.class
rm -f dist/*.jar

# Get the name of this plugin
# from the directory it's in
HERE=`pwd`
NAME=`basename "$HERE"`

# 1. Compile
echo "Compiling with javac..."
javac -Xlint:deprecation src/*/*.java -d bin -classpath "$MODS" -sourcepath src -g:lines,vars,source || exit 2

# 2. Build the jar
echo "Creating jar file..."
jar -cf dist/"$NAME.jar" *.inf -C bin . || exit 3

# 3. Copy to server
echo "Deploying jar to $MCSERVER/plugins..."
test ! -d "$MCSERVER/plugins" && mkdir "$MCSERVER/plugins"
cp dist/$NAME.jar "$MCSERVER"/plugins || exit 4

echo "Completed Successfully."

The above is a perfectly good build script especially as the intended audience of the book doesn’t need to learn about build tools like Make or Maven or Ant.

tup is an interesting build tool for a number of reasons. Particularly the fairly straightforward configuration and file monitoring rebuilds (only on Linux). The other intriguing feature of tup is you can run tup upd anywhere in your source code and it’ll build everything that needs building, which is pretty convenient for large source trees with multiple build targets/libraries - just like the example plugins for this book!

So here’s my first attempt at a Tupfile:

1
2
3
4
5
6
7

MCSERVER   = /Users/will/Downloads/canary-server
MODS       = $(MCSERVER)/CanaryMod.jar
JAVAC_OPTS = -Xlint:deprecation
DEBUGGING  = -g:lines,vars,source

: foreach src/helloworld/*.java |> javac $(JAVAC_OPTS) %f -d bin -classpath "$(MODS)" -sourcepath src $(DEBUGGING) |> bin/helloworld/%g.class
: *.inf bin/helloworld/*.class |> jar -cf %o *.inf -C bin . && cp %o $(MCSERVER)/plugins/ |> dist/%d.jar

I have a few reflections about this:

• I don’t know how environment variables work with tup. Just like Make, it has its own variables but it doesn’t appear to let any shell variables in (that’s why my full home directory is hard-coded at the top of the file).
• Each line in the Tupfile consists of inputs |> commands |> outputs which is very functional-like and intuitive to me. There doesn’t appear to be an easy way to have outputs that are outside your source tree because I had errors when I tried to copy the JAR into my server/plugins directory as a separate step, so I collapsed it into the previous JAR-building step.
• %g refers to a glob in the input files, but alas only the first glob, so I couldn’t write a general rule for building any package regardless (like: foreach src/*/*.java |> javac %f -d bin |> bin/%g.class). The other options for output flags include %b which is the input base filename (e.g. HelloWorld.java) and %B which is the same without the extension (e.g. HelloWorld). I couldn’t work out how to translate the src/a/b.java into bin/a/b.class for any a and b. In Make you can do something like $(source_files:%.java=bin/%.class) or similar and I can’t find an analogue in tup so far.

I tried to tweak the colours to vaguely match the magenta powerline theme of an Emacs theme called moe-theme (Emacs doesn’t work well with the powerline-patched fonts, so the unicode arrows look funny):

I have acheived something close and it looks like this (you need powerline-patched fonts for the unicode characters to work):

Here is the final config for the status colours and content (I decided to ditch the segment that shows my username, since it’s always the same and I know who I am). This goes in your ~/.tmux.conf:

1
2
3
4
5
6

set -g status-fg black
set -g status-bg colour231
set -g status-left '#[fg=colour234,bg=colour162,bold] ❐ #S #[fg=colour162,bg=colour231,nobold]⮀'
set -g window-status-format "#[fg=black,bg=colour231] #I #[fg=black,bg=colour231]#W "
set -g window-status-current-format "#[fg=colour231,bg=colour141]⮀#[fg=colour231,bg=colour141,bold] #I ⮁ #W #[fg=colour141,bg=colour231,nobold]⮀"
set -g status-right '#[fg=colour162,bg=colour231]⮂##[fg=colour231,bg=colour162] %H:%M %d-%b-%y'

Emacs themes and tmux have different colour palettes. Here’s how moe-theme specifies some powerline colours for the modeline:

1

(set-face-attribute 'powerline-active1 nil :background "#ffafff" :foreground "#ff1f8b")

Which is easy enough - they are simply hex colour codes, the same as can be used in CSS. However tmux uses 24-bit xterm colour codes (for a generous total of 256 colours). In order to map between them and find the closest colour I found a handy python script which you can call with a hex code (e.g. python colortrans.py ff4ea3) and it’ll print the nearest xterm colour code. If you call the script with no arguments it’ll print a mapping table of xterm colour codes to hex codes, so you can tweak the value you’re using. Or you can simply use this colour chart. One thing that stalled my progress for a while was I kept writing [fg=color183,bg=color213] sort of thing and tmux is unusual in using the French/British spelling of colour, which is unusual in programming. At least it’s otherwise straightforward, unlike bash color codes which look like somebody fell on the keyboard.

If you get an error like this

Symbol's function definition is void: apropos-macrop

When editing CSS files with Emacs on Debian (testing or unstable at the time of writing), then this is because the package css-mode includes code that no longer works with Emacs version 24 and has been abandoned (emacs contains its own css-mode).

The solution is simply to uninstall css-mode:

sudo apt-get remove css-mode

That’s it.

Add core.incubator to your dependencies in project.clj of your ClojureScript project:

1
2
3
4

(defproject some-project "1.0.0-SNAPSHOT"
  :dependencies [...
                 [org.clojure/core.incubator "0.1.3"]
                 ...]

Now you can require the strint macros in your ClojureScript source (e.g. in src/cljs/app/core.cljs):

1
2

(ns app.core
  (:require-macros [clojure.core.strint :as strint]))

Now you can use the fast string interpolator thus:

1
2
3
4
5
6
7
8

(enable-console-print!)

(let [name "Ethel Smyth"
      profession "Composer"
      born 1858]
  (println
   (strint/<< "The person named ~{name} works as a ~{profession}
and was born in ~{born}")))

That’s it!

There are two main ways to build strings in Clojure: str and format. str essentially does string concatenation like this:

1

(str "This is a sentence with " some " variables ")

I find str forms somewhat unreadable, especially on one line. They require the reader to mentally keep track of quote marks and spaces around variables.

On the other hand, format offers a fully-featured string interpolation function using Java’s Formatter class:

1
2
3

(format "This string has another string: %s in it and a number: %.2f"
         "hello!"
         30.1)

In ruby we might use string interpolation thus:

1
2
3
4
5

def str_interpolate name, profession, born
  puts "The person named #{name} works as a #{profession} and was born in #{born}"
end

str_interpolate "Ethel Smyth", "Composer", 1858

The advantage of string interpolation like this is how readable the code can be.

The problem with prefering format over str is the difference in performance. format is a lot more complex and if all you’re doing is string concatenation, then it’ll not do the job as quickly as str does (which uses a StringBuilder under the hood).

Benchmarks!

The following code uses the Criterium library aliased to bench.

First let’s define a function using str and benchmark it:

1
2
3
4
5
6
7
8
9
10

(defn str-concat-fun
  [name profession born]
  (str "The person named "
       name
       " works as a "
       profession
       " and was born in "
       born))

(bench/bench (str-concat-fun name profession born))

Which results in (256ns):

1
2
3
4
5
6
7
8
9
10

Evaluation count : 241104540 in 60 samples of 4018409 calls.
             Execution time mean : 256.236563 ns
    Execution time std-deviation : 4.617404 ns
   Execution time lower quantile : 250.226629 ns ( 2.5%)
   Execution time upper quantile : 266.339191 ns (97.5%)
                   Overhead used : 1.166050 ns

Found 5 outliers in 60 samples (8.3333 %)
        low-severe       5 (8.3333 %)
 Variance from outliers : 7.7764 % Variance is slightly inflated by outliers

Now let’s check the format version:

1
2
3
4
5
6
7
8

(defn format-fun
  [name profession born]
  (format "The person named %s works as a %s and was born in %d"
          name
          profession
          born))

(bench/bench (format-fun name profession born))

Which results in (1.7µs):

1
2
3
4
5
6
7
8
9
10

Evaluation count : 34997760 in 60 samples of 583296 calls.
             Execution time mean : 1.703759 µs
    Execution time std-deviation : 36.732362 ns
   Execution time lower quantile : 1.663752 µs ( 2.5%)
   Execution time upper quantile : 1.779579 µs (97.5%)
                   Overhead used : 1.166050 ns

Found 2 outliers in 60 samples (3.3333 %)
        low-severe       2 (3.3333 %)
 Variance from outliers : 9.4397 % Variance is slightly inflated by outliers

That’s not good, but not entirely unexpected. An order of magnitude slower to use format for string contatenation tasks like this.

One of the advantages of Clojure is the promise of powerful, expressive abstractions and not having to compromise on those abstractions to achieve performance. In a blog post about string interpolation in Clojure, Chas Emerick proposes a macro for simple string interpolation that would behave much like Ruby’s does. This has made its way into the core.incubator project and can be used in projects today.

To require it, add core.incubator to your project’s dependencies and add the following to any namespace that needs it:

1
2

(ns example...)
  (:require [clojure.core.strint :refer [<<]]))

So now we can define a function like the others using this new macro:

1
2
3
4
5

(defn interpolation-fun
  [name profession born]
  (<< "The person named ~{name} works as a ~{profession} and was born in ~{born}"))

(bench/bench (interpolation-fun name profession born))

And this results in (272ns):

1
2
3
4
5
6

Evaluation count : 222317940 in 60 samples of 3705299 calls.
             Execution time mean : 271.580867 ns
    Execution time std-deviation : 2.117763 ns
   Execution time lower quantile : 267.593081 ns ( 2.5%)
   Execution time upper quantile : 274.826087 ns (97.5%)
                   Overhead used : 1.166050 ns

Not bad! Ever so slightly slower than the str version but a performance penalty well worth paying for to get more expressive string interpolation I feel.

Kindred by Octavia Butler

Kindred is classified as sci-fi, but that doesn’t do it justice really. It’s more of a sci-fi conceit (time-shifting) to allow Butler to immerse the reader in the world of 1815 on a plantation with slaves. The protagonist is black and the abrupt time shift lands her in palpable danger. The relationships between characters was always nuanced and interesting and the tension was high throughout - tension from real danger. I especially appreciated the subtle way Butler weaved in historical names and events into the narrative. I will definitely be reading more of Octavia Butler’s books.

Nothing O’Clock by Neil Gaiman

Part of a series of short stories, one for each Doctor, released on the 50th anniversary of Doctor Who. This was an undemanding entertaining read and I intend to read more of them.

Flat-Earth News by Nick Davies

We all know the media are dreadful, but just how dreadful and why? This book attempts to shine some light on the shady world of British media (although for the most part the same problems exist elsewhere). It is somewhat unscrupulous in aiming its critical eye at broadsheet and tabloid alike and generally castigates broadsheets for pumping out unchecked nonsense because we expect them to be a cut above the tabloids (spoiler: they’re not). Shocking, sickening, depressing - exactly what you’d expect from a book about newspapers.

Clipping Through by Leigh Alexander

This book by my favourite games journalist - Leigh Alexander - takes you into the surreal world of games journalism itself. Part stream of consciousness and part short biography this is a world most of us have no idea about. It kept me entertained on a few train commutes. You can and should buy it here

A Natural History of Dragons by Marie Brennan

The first part of a series of books about the main character - Lady Trent. A cross between Pride and Prejudice and Reign of Fire, sort of. Thoroughly entertaining and I shall be reading the rest of the series.

The Long ships by Frans Bengtsson

This Swedish classic about Vikings in the 10th century in Denmark was a surprisingly good read. The humour was super dry and for some considerable time it passed me by. My main criticism of it is that it’s rather overly long and the novelty of the sarcasm wore out well before the end for me.

Jane Eyre by Charlotte Brontë

Quite simply a must read. I particularly enjoyed cross-referencing with King Lear and the various other literary works Brontë weaves in.

To Boulez and Beyond by Joan Peyser

The first half of this book is a biography of composers from Debussy to Boulez, roughly speaking. Lots of time is spent covering Schoenberg, Berg and Webern as you might expect. The second half of the book is a biography of Boulez himself. I found the combination fascinating since it put the musical trends and works of Boulez in perspective wonderfully. This book lead me to seek out and appreciate a number of compositions which makes it my favourite of the books I read in 2014.

Multi-machine setup

Let’s begin by defining a Vagrant environment that we will play with (you will need VirtualBox, Vagrant and Ansible installed):

1
2
3

mkdir multi-vagrant-ansible
cd multi-vagrant-ansible
vagrant init

This will create a Vagrantfile in the current directory with commented contents. Let’s cut it back to the essentials and add in a URL for the base box (Ubuntu Trusty is the latest LTS release so that’s what I’ll use):

1
2
3
4
5
6
7
8

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.box = "trusty64"
  config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/trusty/current/trusty-server-cloudimg-amd64-vagrant-disk1.box"
  config.vm.network "private_network", type: "dhcp"
end

If we run Vagrant now, it’ll clone that base box (downloading it first if it hasn’t already done so) and boot it up. This is already quicker than downloading an ISO, creating a new VirtualBox instance, booting that up and going through the installation procedure.

Let’s define some machines and set them up to be provisioned by Ansible. We’ll have two web servers and one load balancer, because that’s boringly conventional:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

  config.vm.define "ariadne" do |ariadne|
    ariadne.vm.provision "ansible" do |ansible|
      ansible.playbook = "loadbalancer.yml"
      ansible.sudo = true
    end
  end

  config.vm.define "minos" do |minos|
    minos.vm.provision "ansible" do |ansible|
      ansible.playbook = "webserver.yml"
      ansible.sudo = true
    end
  end

  config.vm.define "pasiphae" do |pasiphae|
    pasiphae.vm.provision "ansible" do |ansible|
      ansible.playbook = "webserver.yml"
      ansible.sudo = true
    end
  end

So in the above, minos and pasiphae are web servers (i.e. they will be running nginx) and ariadne is the load balancer. The location of the ansible playbooks are relative to the Vagrantfile so in the same directory we will create webserver.yml with the following contents:

1
2
3
4
5
6

---

- hosts: all
  tasks:
    - apt: name=nginx state=present
    - service: name=nginx state=started

Which ensures that nginx is not only installed but also running (it’ll also mean that if that server is rebooted, it’ll still run nginx).

Now for the load balancer, loadbalancer.yml:

1
2
3
4
5
6

---

- hosts: all
  tasks:
    - apt: name=haproxy state=present
    - service: name=haproxy state=started

Which ensures haproxy is installed and running in the same way.

These two playbooks are not aware of each other, they act independently and you could use ansible-playbook to provision any server you liked with them.

If you run vagrant up at this point (assuming you’ve not done that with this Vagrantfile before), it’ll boot up new VirtualBox instances and provision them with ansible, installing the necessary software etc. All well and good so far.

Ansible facts

Ansible starts off by collecting facts about the nodes it’ll run on. It does this so that you can use information about the node in your playbooks, roles and tasks.

To see the kind of facts that ansible collects about a node, you can run ansible’s setup module like this (for the minos instance):

1

ansible -i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory -m setup -u vagrant --private-key=.vagrant/machines/minos/virtualbox/private_key minos

The above command should print out a large JSON structure of all the facts ansible has collected about that node. Ansible facts are somewhat extensible so it can include information gathered using the Ohai or Facter tools.

The facts relevant to our example are under the ansible_eth1 key and they include an IPv4 address - which will come in handy in a moment.

Facts & templates

Now let’s create a template for the haproxy configuration (in templates/haproxy.cfg.j2):

1
2
3
4
5
6

backend web-backend
  balance roundrobin
  mode http
  {% for host in groups['webservers'] %}
     server {{ host }} {{ hostvars[host].ansible_eth1.ipv4.address }}:80 check
  {% endfor %}

We’ll also need to ensure that template gets used in the loadbalancer playbook:

1
2
3
4
5
6
7
8

---

- hosts: all
  tasks:
    - apt: name=haproxy state=present
    - service: name=haproxy state=started
    - name: Configure haproxy
      template: src=templates/haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg

If we run this now, we’ll get a cryptic error:

fatal: [ariadne] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute 'webservers'", 'failed': True}

One possible reason for this is that we haven’t defined any groups for our vagrant instances, let’s do that now. We’ll start by defining the groups at the top of the Vagrantfile, before anything else (but after the emacs/vi mode comments):

1
2
3
4
5

groups = {
  "webservers" => ["minos", "pasiphae"],
  "loadbalancers" => ["ariadne"],
  "all_groups:children" => ["webservers", "loadbalancers"]
}

This correlates to the playbooks we’ve assigned for each node in the Vagrantfile. Then we need to refer to that variable in each of our machine definitions, adding a line that says ansible.groups = groups, so the modified ariadne definition should now be:

1
2
3
4
5
6
7

  config.vm.define "ariadne" do |ariadne|
    ariadne.vm.provision "ansible" do |ansible|
      ansible.playbook = "loadbalancer.yml"
      ansible.sudo = true
      ansible.groups = groups
    end
  end

If we run vagrant provision now we get a different error! Ah Ha! Progress:

fatal: [ariadne] => {'msg': "AnsibleUndefinedVariable: One or more undefined variables: 'dict object' has no attribute 'ansible_eth1'", 'failed': True}

Oh no!

It would be useful at this point to examine what we do have in that dictionary object. Maybe I mistyped the key? In order to do that, we can add a debug line above the haproxy configuration line in the loadbalancer.yml file, like this: - debug: var=hostvars['minos'].

When we run vagrant provision now, we will get the facts about minos printed in JSON to the console. It’ll look something like this:

1
2
3
4
5
6
7
8
9
10
11
12

{
  "hostvars['minos']": {
    "inventory_hostname_short": "minos",
    "inventory_hostname": "minos",
    "group_names": [
      "all_groups",
      "webservers"
    ],
    "ansible_ssh_port": 2200,
    "ansible_ssh_host": "127.0.0.1"
  }
}

Clearly all those facts gathered aren’t here. Why? The reason for this is that Vagrant runs provisioning separately on each virtual machine - so each ansible run is not aware of anything from another ansible run. If you look this up online, you will find apparent answers to this problem that reconfigure vagrant to connect to all hosts when doing an ansible run. Let’s do that now.

For each ansible block in the Vagrantfile, add the line: ansible.limit = 'all'. Let’s try vagrant provision again now that’s in place.

The error that I get after making this change is that SSH is failing. If we add ansible.verbose = 'vvvv' to each ansible block in the Vagrantfile then with a lot of scrolling around we can deduce that ansible is attempting to connect to each machine in the inventory using the same private key as it would for the machine it’s currently provisioning. In other words, while provisioning ariadne, it uses the ariadne private SSH key to log on to both the other servers. This won’t work of course because those SSH keys are generated by Vagrant per machine. Not only that but the private key is on the host machine, not on the guests so it’s a fools errand.

I’m not sure what kind of SSH key setup would allow ansible.limit = 'all' to work at all, but it’s hardly straightforward.

Potential workaround: Redis

The only way I’ve discovered to have ansible and Vagrant work well together is to use Fact Caching. This allows ansible to cache all facts from a node in Redis (or memcached) so that nodes can refer to each other without requiring an extra ssh connection for every node.

In order to enable fact caching, you will need Redis installed and running. Then create an ansible.cfg file in the same directory as your Vagrantfile, with the following contents:

1
2
3

[defaults]
fact_caching = redis
fact_caching_timeout = 86400

You will need to provision minos and pasiphae first so that their facts are stored in Redis before provisioning ariadne (because it refers to those other nodes):

1
2

vagrant provision minos
vagrant provision pasiphae

Now that those facts have been gathered, we can run vagrant provision and it should complete without trouble this time.

Now to verify that the haproxy config has been written as we expect, we can run vagrant ssh ariadne -- cat /etc/haproxy/haproxy.cfg and get something akin to:

1
2
3
4
5

backend web-backend
  balance roundrobin
  mode http
    server minos 172.28.128.4:80 check
    server pasiphae 172.28.128.5:80 check

It worked! So although fact caching is intended for use in large organisations with thousands of nodes (possibly in disparate data centres) to speed up deployment, it can be handy working around weaknesses in the vagrant+ansible combination.

Simple Secret Santa

Your common or garden Secret Santa consists of putting names in a hat and picking them out again, ensuring you don’t pick yourself. Suitable for offices and other groups of awkward strangers and acquaintances.

We need core.logic imported, so here’s the namespace declaration (note that both clojure.core and clojure.core.logic define == for totally different purposes. I have never used clojure.core/== so I’m a bit hazy as to what it’s for - we exclude it so they don’t clash):

1
2
3
4

(ns secret-santa.core
  (:refer-clojure :exclude [==])
  (:require [clojure.core.logic :refer :all]
            [clojure.core.logic.pldb :refer :all]))

We’ll start by defining a relation:

1

(db-rel santa p)

This is so that we can constrain values to only those names we’ve defined are santas (otherwise you get weird logic variables and nobody wants to deal with that noise).

Now for the Secret Santa function. It takes a list of friend names (not actually a requirement that they are friends, they may not be after exchanging presents) and returns a list of lists. Each inner list is a pair of giver and receiver of presents.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

(defn simple-secret-santa
  [friends]
  (let [facts (coll->db (for [friend friends]
                           [santa friend]))
        num (count friends)
        givers (repeatedly num lvar)
        receivers (repeatedly num lvar)]
    (->> (with-db facts
           (run* [q]
             (everyg santa givers)
             (everyg santa receivers)
             (distincto givers)
             (distincto receivers)
             (pairupo givers receivers q)))
      (map sort)
      distinct
      rand-nth)))

If we call it thus:

1

(simple-secret-santa ["Tommen" "Gregor" "Daenerys" "Arya"])

Then we should get the following as a result:

1
2
3
4

(["Arya" "Gregor"]
 ["Daenerys" "Tommen"]
 ["Gregor" "Daenerys"]
 ["Tommen" "Arya"])

Please note my chronic lack of imagination evidenced by my lazily using characters from Game of Thrones who would likely not be involved in this kind of tomfoolery.

We need a helper function for pairing people up now:

1
2
3
4
5
6
7

(defne pairupo
  [givers receivers pairs]
  ([() () ()])
  ([[g . gs] [r . rs] [p . ps]]
   (!= g r)
   (== p [g r])
   (pairupo gs rs ps)))

The mind-bending nature of that function can be rather confusing. In core.logic (as with miniKanren and others), it’s common for an “output value” to be one of the parameters, in this case the 3rd parameter (pairs). This function defines a relationship between the parameters provided so you can call it with givers and pairs and it’ll fill in the blanks for the receivers value (effectively unzipping the pairs value), which is pretty neat.

Unify what?

It’s crucial to understand the clojure.core.logic/== unification function to understand how any of this works. It looks like an equality test from one of those other programming languages we shall not mention here, but it isn’t like that. Sometimes I thought of it as a kind of wishful-thinking-equality - a sort of “wouldn’t it be just lovely if these things were the same”.

Taking an example from the core.logic wiki:

1
2
3
4
5

(run* [q]
  (fresh [a]
    (membero a [1 2 3])
    (membero q [3 4 5])
    (== a q)))

In the above we are asking core.logic for the possible values of q (the output variable). We start using a new logic variable a which we unify with q using the == unification function. That is to say, all solutions must satisfy the equation a = q, they must have the same value. Core.logic tends to add ‘o’ to the end of common function names so (membero a [1 2 3]) isn’t a boolean predicate function for testing for list membership, it attempts to make that statement true. With simply that statement - a can be 1 or 2 or 3. The next line asserts that q is a member of the list [3 4 5] so its value can be 3 or 4 or 5. The last line unifies a and q so they must have the same value. We can tell that the only cross-over between the 2 lists already mentioned is the value 3, so q can only be 3. run* returns a list of all possible solutions so in fact we will get a single element list: (3) as the result from that code.

The input of our Secret Santa function will be a list of names (strings), so it’d be useful to turn that list of names into a database that we can constrain our logic functions on:

1
2
3

(defn coll->db
  [coll]
  (apply db coll))

clojure.core.logic.pldb/db takes a variable number of parameters and returns a database with those facts contained. We want to call it with a collection so apply comes in handy here. Effectively you can use apply to turn this: (apply somefn [arg1 arg2 arg3]) into (somefn arg1 arg2 arg3)

How does it work again?

Let’s concentrate on the core part of the function:

1
2
3
4
5
6
7

(with-db facts
  (run* [q]
    (everyg santa givers)
    (everyg santa receivers)
    (distincto givers)
    (distincto receivers)
    (pairupo givers receivers q)))

We are asking for all the solutions where every item in the givers collection is a santa and every item in the receivers collection is a santa (lines 3 and 4). This ensures that those collections contain only valid santa names and not logic variables or numbers or anything else. There’s something subtle about this - it’s not simply an assertion. core.logic will make those statements true in every way they can. One possible version of that (assuming there were 4 names input to the function) is that givers = ["Arya" "Arya" "Arya" "Arya"]. That’s nonsensical for our purposes (remember we mean to zip up these collections so the first giver is giving to the first receiver, the 2nd giver to the 2nd receiver and so on). So then we ensure that the collection of givers has distinct elements in it (line 5), saying the same about receivers on line 6. If you imagine core.logic has assembled all the combinations of values that are santas first - imagine now it throws away any where values repeat. One of the remaining values for givers will be [“Arya” “Gregor” “Tommen” “Daenerys”]. There will be more. How many?

Derangements, subfactorials, oh my!

The mathematical name for the number of unique permutations of elements in a set (that are different from their original arrangement) is the number of derangements or the subfactorial.

Roughly speaking, Secret Santa is a special case of finding a derangement of names. When you have 4 names, there are 9 derangements.

Here is some clojure that calculates that:

1
2
3
4
5
6
7
8
9

(defn derangements
  "Calculate the number of derangements of n elements."
  [n]
  (condp = n
    0 1
    1 0
    (* (dec n)
       (+ (derangements (dec n))
          (derangements (- n 2))))))

This algorithm is primarily useful for checking your results in unit tests for example. Another way to use it is as the parameter to run - you can ask for all the possible derangements of friends. I didn’t have much luck with that because I misunderstood the values that my logic code was producing, also there is no need to know beforehand how many combinations are possible since you only want one.

Further improvements

One weakness of the solutions described above is that they model the relationship between the gift giver and receiver as a binary - allowed or not. It would be useful to be able to first attempt to solve the problem under ideal conditions before falling back to less and less ideal solutions. For example, it’s not ideal to have people paired up symmetrically - it’s just a bit boring that way. It’s more optimal to assign Santas in a circle, which makes it slightly more difficult to identify who’s assigned who.

Finding all possible solutions is extremely slow in core.logic (at least, the way I’ve written it) so this could do with a fair amount of optimisation.

I punted the problem of picking a solution out of possible solutions to the combination of (map sort), distinct and rand-nth. This isn’t really necessary, I could have told core.logic what constitutes a distinct solution (it doesn’t realise that order of pairings doesn’t matter) and then simply picked one. My brain hurt so much by that point that I decided to call it a day and move on to more interesting problems, like

Final thoughts

I had a great deal of difficulty writing the pairupo function, largely because all the various defne/defnu/defna confused me - I still couldn’t tell you what they do. This was partly due to me moving from a real problem (Secret Santa) to the implementation in core.logic on the basis of sketchy logic programming knowledge, so I missed a lot of subtlety related to conde which is crucial for understanding this stuff. The official documentaion for core.logic is extremely sparse also, you are very much on your own if the problem you want to solve isn’t Sudoku or the Typed Lambda Calculus (it boggles my mind that that is on the Examples wiki page, I’m not sure if the intention is to educate or obfuscate there).

I hope that a bit more blogging from mere mortals like myself might help others grok this mind-mending area of programming.

Lack of planning

It was only after Ludum Dare was completed and I filled in a Trello board with the tasks necessary to complete my game that I realised the immensity of the project. I was reading Amy Hoy’s new book Just Fucking Ship and it emphasises working backwards from a goal to a set of tasks that need to be done. Hoy also advocates Mise en place style of working, which I’ve never identified as a thing before but it’s obvious in hindsight. Ludum Dare advice usually contains something about starting on paper, getting all the high-level details worked out before diving into code and that’s absolutely right. The other massive advantage of having a todo list (e.g. in Trello) is that it’s super easy to order the list of things in terms of what needs to be done and what is optional. Does it matter that the resolution is fixed at 800x600? Nope. But I’ve totally sunk hours/days into coding around the idea I need to have a scalable interface on the web for my games (and it’s always really painful).

Case in point - this Ludum Dare, I decided to use Elm, which was entirely new to me. The conventional wisdom has it that one should never learn tools while participating in Ludum Dare, just use whatever you have at hand (i.e. programming languages/IDEs/libraries etc. that you have used extensively before). I only make games during Ludum Dare though and I’ve never reached a level of proficiency with gamedev that I feel leaves me with a default technology choice, so I used a new thing. However, this time round, using Elm, the web game ends up getting drawn using the 2D canvas and image elements positioned on top of it basically. You can’t control a lot of how Elm does any of this and there was a minor wrinkle - there’s no way to use CSS’s image-rendering option to optimise for pixel art (crisp-edges is what you want - which is essentially nearest-neighbour scaling up/down). The result is you get blurry pixel art in your games and this sucks. It’s not at all a blocking problem for Ludum Dare games though and I feel silly for spending any more than 5 minutes on it.

Overly ambitious game design

I’m not a regular gamedev, this is something I do very occasionally. I’m a software engineer and therefore I completely and wildly underestimate the complexity and difficulty of programming tasks. Not to mention my total lack of ability to gauge time requirements. In the world of work, many of us software engineers avoid the near-impossible task of time estimation by using Agile-based practice and estimating complexity instead. It’s entirely based on teamwork and iteration of process and makes no sense for an individual working towards a goal on a deadline.

Aside: within the confines of Agile processes and teams, individuals can and do use more “traditional” time estimation and tracking tools to live up to their own expectations of themselves, so we do think “this will probably take half a day” but the discipline of working in an Agile team is never externalising that because it becomes an expectation and a commitment.

All my weaknesses as a software engineer aside, I made the fundamental error of dreaming up a cool game that was not within my reach of a 48 hour deadline. This is a completely avoidable self-imposed problem. The solution seems to be, sketch out the game in its entirety - all the objects in the game, all the components and all the behaviour. Make sure you have all the artwork at least approximated (I did this in Paper on the iPad because I could doddle on that while watching TV and it didn’t even feel like work). At this point you can identify elements that can be left out - think “what could I postpone if I was forced to launch this in an hour?” - or some other mind trick to force yourself to be ruthless with the scope of your game. One advantage of sketching everything on paper or in Paper on an iPad is that if time gets really tight and you don’t have super great pixel artwork ready - just chop up/scan in/photograph the sketches and use them as your assets. There will be people playing your game who think the hand-drawn nature of the artwork was a stylistic choice you made from the beginning and you know you ran out of time.

Onwards

So now I have a Trello board that makes it embarrassingly clear how unrealistic my game was for a 48 hour competition - where do I go from here? I think the best answer to that is to carry on with it, attack each task in the list and then realise the game is done (that sounds obvious but it’s impossible to realise when you’re ready to ship something unless you define up-front what you actually want to ship).

In Agile circles, we talk a bit about what it means to be Done. The “definition of done”, which can sound rather alien to somebody who’s not familiar with specifying that concretely. It’s essential to working progress otherwise you end up with tasks that linger on for days and weeks because they are quite simply never finished. Those aren’t tasks, they’re a waking nightmare. Sometimes there is no definition of done. This blog post doesn’t have a definition of done because I made an agreement with myself that I would write until I felt hungry then I would order some food. In Agile that’s usually called time-boxing and it usually isn’t hunger-based but might as well be. “Let’s time-box this for an hour” is a common phrase in Agile teams. It’s how you manage investigation tasks (otherwise they go on forever). We all need more, clearer definitions of done.

So Ludum Dare is over. I’m done with this blog post. I have plenty to practice and improve before next Ludum Dare and with any luck I’ll ship a game next time.

Javascript (Chrome, Firefox etc.)

Javascript has the JSON.stringify function, which can pretty-print JSON according to the number of spaces you pass as the 3rd parameter. For example:

1
2

var example = {user: {name: "Ada Lovelace", email: "ada@example.com"}, group: "programmers"}
JSON.stringify(example, undefined, 4);

If you try the above in a console in a web browser (or a Node.js console), you should see this:

1
2
3
4
5
6
7

{
    "user": {
        "name": "Ada Lovelace",
        "email": "ada@example.com"
    },
    "group": "programmers"
}

json_reformat

That’s all well and good, but relying on a javascript console is not the most efficient way to do things, so an easier and more scritable tool is the json_reformat command (part of the Yajl). To get this tool installed, do the following:

• brew install yajl - OS X
• apt-get install yajl-tools - Debian/Ubuntu Linux

Now you can just pipe JSON text to json_reformat and it will pretty-print it to standard output. This makes it very useful for use in other tools, such as vim. Here is some vim config for using json_reformat:

1
2

au BufRead,BufNewFile *.json set filetype=json
au FileType json setlocal equalprg=json_reformat

Now let’s edit a file that was generated by Berkshelf and has been spat out on the filesystem in ~/.berkshelf/config.json. Initially it looks like this:

1

{"chef":{"chef_server_url":"https://api.opscode.com/organizations/acmecorp","validation_client_name":"acmecorp-validator","validation_key_path":"/Users/ada/Work/.chef/acmecorp-validator.pem","client_key":"/Users/ada/Work/.chef/ada.pem","node_name":"ada"},"cookbook":{"copyright":"Ada Lovelace","email":"ada@example.com","license":"MIT"},"allowed_licenses":[],"raise_license_exception":false,"vagrant":{"vm":{"box":"Berkshelf-CentOS-6.3-x86_64-minimal","box_url":"https://dl.dropbox.com/u/31081437/Berkshelf-CentOS-6.3-x86_64-minimal.box","forward_port":{},"network":{"bridged":false,"hostonly":"33.33.33.10"},"provision":"chef_solo"}},"ssl":{"verify":false}}

This passive-aggressive attempt by the machines to subjugate us humans just will not stand and thankfully we have the tools to take back control of our JSON configs. Position the cursor anywhere on the single line of JSON and press ‘==’, vim will pipe the buffer to json_reformat and replace it with the output of that command resulting in this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

{
    "chef": {
        "chef_server_url": "https://api.opscode.com/organizations/acmecorp",
        "validation_client_name": "acmecorp-validator",
        "validation_key_path": "/Users/ada/Work/.chef/acmecorp-validator.pem",
        "client_key": "/Users/ada/Work/.chef/ada.pem",
        "node_name": "ada"
    },
    "cookbook": {
        "copyright": "Ada Lovelace",
        "email": "ada@example.com",
        "license": "MIT"
    },
    "allowed_licenses": [

    ],
    "raise_license_exception": false,
    "vagrant": {
        "vm": {
            "box": "Berkshelf-CentOS-6.3-x86_64-minimal",
            "box_url": "https://dl.dropbox.com/u/31081437/Berkshelf-CentOS-6.3-x86_64-minimal.box",
            "forward_port": {

            },
            "network": {
                "bridged": false,
                "hostonly": "33.33.33.10"
            },
            "provision": "chef_solo"
        }
    },
    "ssl": {
        "verify": false
    }
}

This trick with vim and json_reformat is one of the many answers to a StackOverflow question about pretty-printing JSON, there are so many more.

On OS X, if I use tmux (or more usually, wemux for pairing) then I end up with oddly named windows like this:

This is because I’ve followed the awesome instructions from Dr Bunsen’s Text Triumvirate and in order to have the tmux and system (OS X) clipboards interacting, it requires a hack called reattach-to-user-namespace which proxies the running of zsh in tmux.

One quick way to get around this and save time renaming my tmux windows, was to use autoenv and add a file named .env to projects with content similar to:

tmux rename-window "project-name"

This is piling hacks on top of hacks, but at least it saves a bit of time for frequently visited directories.

/log

I.e. git will ignore everything in that directory, however The Silver Searcher doesn’t understand that and it’ll still show matching lines from files in log/. For git and ag (The Silver Searcher) to both ignore that directory and its contents, change /log to log.

A common pattern with webapps in these exciting times is the transition-to-new-thing-you-just-created pattern. The way it works is this: you fill in a form to create a new thing (like, a widget or something) and then when you hit ‘save’, you are transitioned to the page for that widget. This is so you can confirm that it worked and so you could make further changes on that widget. The assumption being, when you create a widget, it is the focus of your task.

Whatever, anyway, I had to make something along those lines in an EmberJS app at work and hit an ember-data bug. Surprise!

The bug is that when you try to do this (code speaks louder than widget metaphors):

1
2
3
4
5

var model = App.Widget.createRecord({name: "special_widget"});
model.one('didCreate', function() {
  this.transitionToRoute('widgets.show', model);
});
model.get('store').commit();

This doesn’t work with ember-data currently - what happens is the model object that gets sent to the didCreate callback there won’t have an id from the server (it hasn’t been filled in for some reason).

To work around this issue, you need to do something akin to this:

1
2
3
4
5
6
7
8
9

saveTheWidget: function() {
  var model = App.Widget.createRecord({name: "very_special_widget"});
  model.addObserver('id', this, this.showWidget);
  model.get('store').commit();
},
showWidget: function(model) {
  model.removeObserver('id', this, this.showWidget);
  this.transitionToRoute('widgets.show', model);
}

In that example we trigger the page transition off the change of value of id. This works because for a short while, the id will be null until Ember-data fills in the new id from the server. Then your callback is called and the page transition happens. The difference in timing is not noticeable to the user - it’s just a race condition within Ember-data itself. Other than that - in this example, the observer is removed, which is good practice for observers and you should always remove observers you’ve added (as I understand it).

Other alternative workarounds include wrapping the transitionToRoute call in a setTimeout() block - to make it pause just long enough for the model’s id to be filled in.

Most of the above came from Stackoverflow and the bug report and workaround on ember-data.

If you are trying to read SVGs and convert them (through STDIN/STDOUT, from a script for example), like this:

cat test.svg | convert svg: png:- > test.png

And you get this error:

1
2
3
4
5

Error reading SVG:
convert: delegate failed `"rsvg-convert" -o "%o" "%i"' @ error/delegate.c/InvokeDelegate/1058.
convert: unable to open image `/tmp/magick-Oj094Z9n':  @ error/blob.c/OpenBlob/2587.
convert: unable to open file `/tmp/magick-Oj094Z9n':  @ error/constitute.c/ReadImage/571.
convert: missing an image filename `png:-' @ error/convert.c/ConvertImageCommand/3011.

Then you need to install (on Ubuntu anyway):

apt-get install libmagickcore4-extra

I had to strace the convert command to find out that it was looking for a missing file. This is from the strace output:

1

stat("/usr/lib/ImageMagick-6.6.9/modules-Q16/coders/svg.la", 0x7fffe6cace90) = -1 ENOENT (No such file or directory)

Words cannot describe how I feel about Imagemagick right now.

Note to self: install this required package (stupidly called “-extra”).

Also note to self: avoid Imagemagick like the plague.

How to check whether Ember can ‘see’ your templates

In your browsers console, type:

Ember.TEMPLATES

You should see something like:

Object {application: function, domains: function}

In the above example I have two templates defined: ‘application’ and ‘domains’. If something is stopping ember-rails from putting your templates in that object, it will simply be a blank object.

Why is this happening?

This happened to me because both handlebars_assets and ember-rails were registering with the asset pipeline to process .handlebars files (and .hbs files). They fight and, at least in my case, handlebars_assets won.

The fix

The only way to get this to work reliably right now is to use a file extension for your Ember handlebars templates that isn’t recognised by handlebars_assets. I used .hjs. It’s not exactly a fix, but things are all working right now so that’s groovy.

Why are you using both?

There’s probably a way round this, but I’m using handlebars_assets to compile handlebars templates for some javascript (that ends up not being at all related to EmberJS - it’s embedded in other pages). I need those templates to be available in the HandlebarsTemplates object so I can run them in the javascript. I include handlebars.runtime in the javascript to run them. ember-rails includes that (I just looked) so I can probably ditch handlebars_assets. That’ll be the next thing to try.

This might not make a lot of sense, but if you’ve hit this problem (i.e. nothing renders in your EmberJS app but your templates are all compiled and you get no errors in the browser console) then this is one possible gotcha.