Steven Gravell – mokele.co.uk

More Learning... and More Beer!

2013-06-09T00:00:00+01:00

Here I am, a few weeks into homebrewing, and I've already brewed 3 times – and hoping to fit them in a lot more often post-thirtieth birthday shenanegans. The first beer I made Gyle Zero was an amber style citrusy hoppy beer with a lot of crystal malt flavour (which I totally winged), my second which I am now drinking the first bottle of is a hoppy pale ale, and the third once it's finished fermenting will be a 2.something% simple dark mild. I have some simple things to say regarding homebrewing for this short amount of time:

just start brewing / wing it
get to know your kit through brewing with it
it you don't hit the gravity you want, it doesn't matter. You can either, do nothing about it or if you have the option add more water before pitching yeast. It'll be fine.
get to know hops through using them, it's more than likely it will not be undrinkinable

Here’s a list of things I changed after Gyle Zero

Use the Raspberry-Pi to do temperature regulation in both kettles
Less Mash liqour and more Sparge liqour
After Run-Off and Sparge clean out Mash and distribute Wort evenly between both the original Mash Kettle and my second Kettle to do 2 Boils rather than one to prevent dangerous bubbling over. All additions are evenly distributed among both Kettles.
For the second brew I adopted a HERMS (Heat-Exchange Recirculated Mash System) to regulate and even raise mash temperature. You can see evidence of this in the graph below.
For the third brew I recirculated after the Boil through the heat-exchanger to lower the temperature faster. This cut down transfer time to Fermentor by around half. You can see evidence of this in the graphs below.
Liqour back after boil to hit desired gravity.
Use the pumps I have and lines to put a bottle on the end of a tube to more effectively pressure clean the bottle with alkaline based detergent

Here’s some pics and graphs and recipes:

Gyle One

12L Mash
26L Sparge
5kg Maris Otter Pale
500g Cara
500g Wheat
33g Galaxy 14.4α @ 90m
20g Challenger 7.12α @ 90m
25g Galaxy 14.4%α @ 20m
25g Simcoe 11.9%%α @ 20m
25g Galaxy 14.4%α @ 5m
25g Simcoe 11.9%%α @ 5m
Liqour Back to hit 1.048 post-boil
Safale US-05 Yeast

Gyle Two

12L Mash
28L Sparge
2.5kg Maris Otter Pale
250g Wheat
250g Munich
250g Crystal
150g Chocolate Malt
100g Toasted Chocolate Malt (I toasted in my dry frying pan)
30g Fuggles 4.35α @ 90m
10g Fuggles 4.35α @ 5m
10g Willamette 6.3%α @ 5m
Accidently rinse your lines too much that ends up in the beer leading to just 1.030 post-boil
Safale S-04 Yeast

What I've Learnt From My First All-Grain Homebrew

2013-05-28T00:00:00+01:00

My last post about Gyle Zero was basically a "werhoo! it sortof worked!" post, along with a fun graph of the temperatures and major events during the mash, boil, and transfer. Now I'm going to go through all the various parts of the process that I either cocked-up entirely, or semi-cocked-up, like having the mash be too wet, and doing a one-pass counterflow heat exchange, which took AGES. I better also write out the recipe! It's down to 1.025 from 1.060 OG now and it's tasting great :)

3kg Pale
1kg Munich
1kg Crystal
500g Wheat
30g Chinook 13.3%α @ 90m
20g Cascade 7.5%α @ 15m
20g Citra 14.4%%α @ 15m
20g Cascade 7.5%α @ 0m
20g Citra 14.4%%α @ 0m
Safale S-04

I was planning on hopping the secondary over night while I make sure the krausening is definitely working before bottling the following day, but it tastes rather good right now, so am happy to leave the recipe the way it is.

Here's a list of things I've learnt with some images:

If your mash is too wet, adding the grist won’t make much or any difference to the current temperature

I had planned for the Mash to be around 63C so hit around 71 strike temperature assuming the grist would lower this, it did not. It stayed stable, only coming down quickly when I stirred and removed the lid. Rob the head brewer at work later mentioned I could have recirculated via the heat exchanger to cool this down a lot faster, but I think I was too scared of too many variables of me messing about, so instead left it to lower down to 63 over 90 minutes. Next time I’ll use less mash liquor, more sparge liqour, and a closer strike temperature to what I want for my first mash rest.

Boiling 25-30L of wort is dangerous

With the kettle lid on the rolling boil would get a bit too violent and splurt out the sides, this meant I had to have the lid partially on leading to a large amount of loss to evaporation. Next time I am going to do 5L less volume, liqour back during/after transfer to FV, and also attempt to put the lid back on after the initial violent boil stage.

There is plenty of hopped wort left over in the bottom of the boiler to set aside for krausening

Once the boiler level got somewhat low I decided to take 1L of hopped wort to put in the freezer ready to sanitise before bottling (called krausening). Next time once the boiler starts to run a lot slower I will call it a day and then pour the remainder from the top of the boiler (by tipping it to the side, just before I clean it), staight into the recycled polypin bags I’m using to store it in the freezer.

After the transfer via the heat exchanger to the fermentation bucket there can be a bit of foam making it more difficult to get an accurate hydrometer reading

I had to use the hydrometer to push some bubbles aside to try and get a clear head to get a reading, it was not the best. Next time I’ll be taking the reading from the hopped wort that was left over in the bottom of the boiler that I’ll then set that aside for my future krausening wort.

Accurate measurements of final volume after the boil is important

I thought I could wing it with rough estimates based on the size of the fermentation vessel. I’ve made marks on the side of the vessel for this first gyle so will fill up with water once it’s empty to get the accurate reading. I will be making marks on a long plastic spoon that correspond to various volumes in the 25L fermentation buckets I have.

One-pass counterflow heat exchange is SLOW

The small heat-exchanger and cold water supply I have doesn’t cool the hopped wort quick enough. It was boring and a huge waste of time. Next time I will be recirculating the kettle through the heat exchanger rather than going straight into the fermentation bucket – this way it will cool quicker and I can have the flow at a higher rate due to it all mixing in with the rest to cool all at once rather than solely on the super slow flow rate.

My friends who have chickens will have to come fetch the spent grain afterwards

Because I’ll have had a few beers while brewing so won’t be able to drive to drop it off before it spoils. I can also put it in the fridge so it’s alright for the next day.

Gyle Zero – My First Attempt at Homebrewing

2013-05-24T00:00:00+01:00

So here I am. I feel kinda stressed out, kinda relieved, a tad anxious. It's all a bit of fun right? Good to hear. I (hopefully) brewed some beer today. The Safale S-04 should be doing its part of the bargain, I'll sneak a peak before I go to bed and try and not disturb them.

Graphs! (if you wait for them to load)

If you’ve read my previous entry on my homebrewing venture you’ll see all of the fancy contraptions and things that I’ve been working on… well, I didn’t really use much of that other than the digital thermistor readouts. It was taking too long to get something I was happy with, so decided to bite the bullet and get a brew done with a lot more manual intervention, which turns out to be quite boring.

I made a lot of mistakes and learnt a great deal about how I need to change my kit to improve on this zeroth gyle. I’ll post more about that soon. This is an end-of-brew-day signing-off post. Enjoy the graph! :)

Intro: Brewing Beer with Erlang, Bootstrap, and a Raspberry Pi

2013-04-17T00:00:00+01:00

I've started working on a homebrewing controller and interface for the purposes of:

learning how to program things that interact with the real world
learn more about the processes involved in brewing beer
manage my homebrewing accurately as to easily measure and reproduce any changes I make between brews
brew good beer consistently and with minimal effort - this is the most important bit right? :D sure is!

I started off with an Arduino, Erlang, and some custom Javascript, but have now moved to Raspberry Pi and Bootstrap. The Erlang side has stayed the same other than the Raspberry Pi related libraries :D

What’s Involved

Hardware

Raspberry Pi
RF Transmitter to control 3 wireless plug sockets
3 Photocells to monitor the current state of the plug sockets
A number of OneWire Waterproof Thermistors
2 Cygnet Boilers (one for Mash and one for Hot Liquor Tank and for the Boil)
some [false buttoms][] for the boilers
a counterflow heat exchange from The Malt Miller
Loads of tubing bought from The Homebrew Forum
Loads of brewing buckets
and a pump

Doesn’t it look a mess right now! :)

Software

I define my kit, process, and recipe with a Javascript object dialect defined with Protocol Buffers for easily understandable configuration and simple interaction between Javascript and Erlang. The kit repo looks a bit like this where I define what each point does and its neighbours:

(function() {
   Gyle.kit({
       points: [{
           id: -1,
           neighbours: [-7],
           hasHose: true,
           hose: {}
       }, {
           id: -2,
           name: "Mash Tun",
           neighbours: [3],
           hasKettle: true,
           kettle: {
               temperatureSensor: 0,
               powerController: 1
           }
...

This allows me to:

Attribute actions and processes to specific vessels, pumps, intersections, valves, and pipes.
Use the point graph to tell when a route includes a pump leading to it being automatically turned on when necessary.
Draw a diagram of the kit on the screen labeling each point.
Track changes in my kit, processes, and recipes via git repositories. Fork this kit/process/recipe!

The backend is written in Erlang and I can do things like this:

11> gyle_hardware_io:power(on, 1).
20:34:33.044 [info] PowerState 1 on
ok
12> gyle_hardware_io:power(on, 2).
20:34:50.191 [info] PowerState 2 on
ok
13> onewire_therm_manager:subscribe("w1", "28-000004753e42").
{ok,17.437,{1366,233345,32572}}
14> flush().
Shell got {therm,{"w1","28-000004753e42"},17.5,{1366,233378,22579}}
Shell got {therm,{"w1","28-000004753e42"},18.062,{1366,233381,682590}}
Shell got {therm,{"w1","28-000004753e42"},19.312,{1366,233385,342572}}
ok

Show me! show meeee!

It looks like this! My old interface ended up being horrible, and this new Bootstrap based one is awesomely lovely.

The Power indicators show the state of the sockets that are either on, or should be on and are currently off. In this case both 1 and 2 are on. Socket 0 is not on and is not supposed to be on, otherwise it would be visable and red.
The Temperature indictors show the current states of each thermistor. They will eventually show the rate and direction of change as well as target temperatures e.g. for efficient boils and mash temperatures
The list to the left is a step-by-step list of each process involved, all defined in the process Javascript describing the flow of liquid and other ingredients (and even cleaning solution!) in the system.

Migrating from the Arduino

The Hardware side of things

This is the Arduino I was previously using (^above).

And this is the new Raspberry Pi along with a lot more neat wiring than before, since now I have the experience to know exactly what I’m doing.

The Software side of things

The code I had on the Arduino simply routed data directly down the serial port to the Erlang code, so was super easy to migrate to Raspberry Pi, well… sortof. I had to write:

pie Erlang bindings to [pihwm][] Raspberry Pi C interface
pt2262.erl Erlang code to produce pt2262 encoded signal sequences where previously I used the Arduino code made available by rayshobby.net
onewire_therm Erlang code to grab data from OneWire Thermistors
a bugfix for pihwm

Well that’s the intro. Hopefully I’ll show it all working and release some more code soon. I haven’t even brewed with it yet! Been too busy and enthused getting it all connected and designing the whole thing. It’s great fun! I feel like a beer now but need to get to bed ready to brew more beer at work tomorrow morning :) Bottoms up!

Property testing Javascript Go match variations

2012-10-30T00:00:00+00:00

The past couple of days I've managed to churn out proper.js – some middleware between Javascript and Erlang's PropEr property-based testing framework (Now supports QuviQ QuickCheck). And it finally has a real-world use-case for a project I'm actually working on! Games of Go! What else could it have been!

The reason why I wrote proper.js in the first place was for this exact set of tests that was becoming a massive pain in the ass to debug. Hold on tight, or jump straight to the juicy property-based tests

Match and moves data-structure background on the data-structure I'm testing
Traversing more on how I want to traverse the data-strcture
The juicy property-based tests get straight to the juicy bits
The actual property test I wanted to write
Interpreting failed tests

Match and moves data-structure

You can skip straight to the juicy property-based tests, but here is some background on the data-structure I am testing.

I have a data-structure in Javascript where a match has 0-many moves in an array, and each of those moves may also have its own list of moves. Each move has a move number that starts off at 1, and each list of moves cannot have a move who’s next item is not the same number or 1 greater. Added to that, child moves must be of a greater number to the parent.

The reason I designed the data-structure this way rather than simply adding all move number 2s into the moves list of the 1s, and 3s into the moves of the 2s, and so on… was that my variations data structure is designed for games with less variations than there are numbers of moves. In an average game of Go there are more than 200 or 300 moves, so in that case I do not want a 300 depth nested array, it would be better to have a single depth array. This means the depth of my data-structure is proportional to the number of variations rather than just the number of moves.

For example these are valid:

var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = []},
    {number: 3, moves = []},
    {number: 4, moves = []}
  ]
}
var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = [
        {number: 3, moves = [
            {number: 4, moves = []},
            {number: 4, moves = []},
            {number: 5, moves = []}
          ]},
        {number: 3, moves = []}
      ]},
    {number: 2, moves = []},
    {number: 2, moves = []}
  ]
}

and these are invalid:

var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = []},
    {number: 1, moves = []}, // 1 is out of sequence
    {number: 4, moves = []}
  ]
}
var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = [
        {number: 3, moves = [
            {number: 3, moves = []}, // 3 cannot also contain 3
            {number: 4, moves = []},
            {number: 5, moves = []}
          ]}
      ]},
    {number: 2, moves = []},
    {number: 2, moves = []}
  ]
}

This data structure allows the storage of games of [Go][] and move variations/scenarios during a game that someone may wish to play out and review.

Traversing

To traverse the match variations I create a VariationNavigator to encapsulate this behaviour. It has 3 functions I’m going to test in this post, they are:

move(move)

Add a move to the navigator and update the move pointer to that move. If any moves with a larger move number are before it, push them all into the moves list of the last child of the parent moves with the same number of the one being inserted. This sounds weird, and is one of the reasons I want property tests! This is strange logic and it needs it.

For example inserting another move number 2 to the root match.moves list which change this:

var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = []},
    {number: 3, moves = []},
    {number: 4, moves = []}
  ]
}

into this:

var match = {
  moves: [
    {number: 1, moves = []},
    {number: 2, moves = [
        {number: 3, moves = []}, // <- move into previous 2
        {number: 4, moves = []}  // <- also moved into previous 2
      ]},
    {number: 2, moves = []} // <- the new entry
  ]
}

next(variation)

Move to last of a specific move number, and if the index of the item we are at exeeds the variation pointers move to the move.moves level and continue moving forward from there.

previous()

Decrement the pointer, and if we’re at the beginning of a list of moves, move to parent move in its parent’s move list.

Now the juicy property-based tests!

Right, now down to business.

Here we define what it is to be an empty match, a stone, a point, a board_size, and a move.

var emptyMatch = function() {
    return {
        move_number: 1,
        moves: []
    };
};
var stone = function() {
    return oneof('BLACK', 'WHITE');
};
var point = function(size) {
    return oneof(Array.seq(0, size*size - 1));
};
var board_size = function() {
    return oneof(9, 13, 19);
};

A move is the joining of a stone() and a point(size) that we’ve already defined above, so let’s use LET to do this.

You can read this type as the sentence:

Given any stone
and any point from a board of a give size
construct a move object with those properties

var move = function(size) {
    return LET(stone(), point(size),
         function(stone, point) {
             return {
                 moves: [],
                 stone: stone,
                 point: point
             };
         }
    );
};

The FORALL_MATCHES function below is more complicated, but helps define the most important property, that of all matches with all move variations. Well not entirely, it doesn’t yet insert moves into anywhere other then the match.moves level, but I have yet to implement that part of the property before posting this entry.

You can read this property as the sentence:

For all board sizes and
all lists of possible moves
add all those moves with a possible move number to the VariationNavigator for that match.
Then finally return the results of the function f of the generated match and navigator.

var possible_move_numbers = function(length) {
    var a = [];
    for(var i=0; i<length; i++) {
        a.push(integer(2, i+1));
    }
    return a;
};
var FORALL_MATCHES = function(f) {
    return FORALL(board_size(),
        function(size) {
            return FORALL(array(move(size)),
                function(moves) {
                    return FORALL(possible_move_numbers(moves.length),
                        function(move_numbers) {
                            var match = emptyMatch();
                            var navigator = VariationNavigator(match);
                            var maxNumber = 1;
                            for(var i=0; i<moves.length; i++) {
                                var move = clone(moves[i]);
                                move.number = Math.min(maxNumber, move_numbers[i]);
                                maxNumber = move.number + 1;
                                navigator.move(move);
                            }
                            return f(match, navigator);
                        }
                    );
                }
            );
        }
    );
};

note:

In another language (e.g. Erlang) you may write this as a separate property named ‘match’ and use LET instead of FORALL and not take a function as an argument. I have not done it this way because proper.js does not yet support yielding an object property with javascript functions, since this get stripped by erlang_js. I have plans to create a work around to solve this issue, but it was easy enough to create this FORALL_MATCHES function instead for now.

Now the actual property test I wanted to write to find the bug I’m looking for

This property can be read as the sentence:

Given all possible matches with variations
and all possible variation pointers to moves in those matches
- e.g. [0, 3, 2] -> match.moves[0].moves[3].moves[2]
call navigator.next(variation) until it returns false
- assert the next move is always 1 more than the previous
assert that we have not exceeded the indexes of the variation pointers
call navigator.previous() until it reaches the start again
- check the move we’ve at is the same as the one while we moved forward
finally assert that moving forward and moving backwards traversed the same number of moves

Variation.props = {
    variation_walking: function() {
        return FORALL_MATCHES(
            function(match, navigator) {
                navigator.reset();
                return FORALL(variations(match),
                    function(variation) {
                        var lastMove = null;
                        var next = null;
                        var forwards = [];
                        // FORWARDS
                        while(next = navigator.next(variation)) {
                            var move = next.move;
                            forwards.push(move);
                            if(lastMove) {
                                assert.equal(lastMove.number + 1, move.number);
                            }
                            lastMove = move;
                        }
                        forwards.pop();

                        // assert that we've not gone beyond each variation pointer
                        for(var i=0; i<variation.length; i++) {
                            assert.ok(variation[i] >= navigator.pointers()[i]);
                        }

                        var previous = null;
                        // BACKWARDS
                        while(previous = navigator.previous()) {
                            var move = previous.move;
                            if(!move) {
                                break;
                            }
                            var forwardMove = forwards.pop();
                            //
                            assert.deepEqual(move, forwardMove);
                            assert.equal(lastMove.number - 1, move.number);
                            //
                            lastMove = move;
                        }
                        // no more forward walked moves left
                        assert.equal(forwards.length, 0);
                    }
                );
            }
        );
    }
};

Interpreting failed tests

$ pjs variation.js Variation
property Variation.props.variation_walking()
......................!
Failed: After 23 test(s).
[19]
[[{"moves":[],"stone":"BLACK","point":122},{"moves":[],"stone":"BLACK","point":22},{"moves":[],"stone":"WHITE","point":15},{"moves":[],"stone":"WHITE","point":146},{"moves":[],"stone":"WHITE","point":13},{"moves":[],"stone":"BLACK","point":217}]]
[[1,2,2,4,4,2]]
[[2,0]]

Shrinking ..........(10 time(s))
[9]
[[{"moves":[],"stone":"BLACK","point":0},{"moves":[],"stone":"BLACK","point":0},{"moves":[],"stone":"BLACK","point":0},{"moves":[],"stone":"BLACK","point":0},{"moves":[],"stone":"BLACK","point":0},{"moves":[],"stone":"BLACK","point":0}]]
[[1,2,2,4,4,2]]
[[2,0]]

What PropEr did was take the values it generated when the test failed, and shrink them down to the smallest example that it can find that still fails the same test. The first failure is after the “Failed: After 23 test(s)” line and the shrank example is after “Shrinking ..”. Each one of the 4 values represents each LET or FORALL list of properties, and is the value that was generated that you can use to reproduce that specific failure.

So:

[9] is [board_size()]
The list of moves [[{“moves”:[],”stone:”BLACK”..},..]] was generated with [array(move(size))]
[[1,2,2,4,4,2]] was the list of which move numbers to try [possible_move_numbers(moves.length)]. Note that it does not contain a 3, the test uses the minimum of the possible move number and what the maximum the next move number can possibly be so far.
and [[2,0]] was the variation to traverse

This means the match looked like the code below, and traversing forward to move numbers 3 (at pointers [2, 0]) and back again did not peform correctly.

var match = {
    {number: 1, moves = []},
    {number: 2, moves = []},
    {number: 2, moves = [
        {number: 3, moves = []},
        {number: 4, moves = []}
      ]},
    {number: 2, moves = []}
}

Beats writing contrived examples right?

In Summary

The most important thing to note about this now is that given the example above and many other examples re-running the tests, I can now start debugging and hopefully fix the bug for all possible matches, and not just one contrived example I can think up.

If you’d like to help out with proper.js fork it and start hacking away!

Javascript property-based testing

2012-10-29T00:00:00+00:00

Having utilised property-based testing using the Erlang applications PropEr and QuviQ QuickCheck, I now have a weird drive to use the same library in my Javascript code. So I decided to write some middleware between erlang_js and PropEr/QuviQ QuickCheck, that at the time of this post has ?LET, ?FORALL, and and oneof support.

The project lives over on GitHub at https://github.com/mokele/proper.js

To test the array properties first build it:

make

and run:

./properjs priv/array.js Array

This example was written before pjs was installable. Installation instructions are on http://mokele.github.com/proper.js.

Here’s the above array example that adds a remove function to Array’s prototype (from stackoverflow) and tests an Array of incrementing positive integers does not include an item after it is removed and that its length always decrements by 1.

Array.prototype.remove = function() {
    var what, a = arguments, L = a.length, ax;
    while(L && this.length) {
        what = a[--L];
        while((ax = this.indexOf(what)) != -1) {
            this.splice(ax, 1);
        }
    }
    return this;
}

Array.seq = function(from, to) {
    var a = [];
    while(from <= to) {
        a.push(from++);
    }
    return a;
}

function min_max_pos_integer_pair() {
    return LET(pos_integer(), pos_integer(),
        function(n1, n2) {
            var min = Math.min(n1, n2);
            var max = Math.max(n1, n2);
            return {min: min, max: max};
        }
    );
};

Array.props = {
    min_max_pos_integer_pair: function() {
        return FORALL(min_max_pos_integer_pair(),
            function(pair) {
                assert.ok(pair.min <= pair.max);
            }
        );
    },
    remove: function() {
        return FORALL(min_max_pos_integer_pair(),
            function(pair) {
                var n = pair.min;
                var size = pair.max;
                var seq = Array.seq(1, size);
                var length = seq.length;
                seq.remove(n);
                assert.equal(seq.indexOf(n), -1);
                assert.equal(seq.length, length - 1);
            }
        )
    }
}

So there it is, a preliminary version of property-based testing in Javascript plugged directly into PropEr.

While writing this post I found out qc.js exists, which is all well and good, but oh well! I’m sure it’s great but I’m already set on integrating directly with PropEr/QuickCheck rather than supporting an entire fork.

Round Games of Go on a Round Goban

2012-10-10T00:00:00+01:00

I’m working on a web based Go client and server, and to aide in abstracting out all the different rules I decided to implement Round Go as “popularised” by Harald Schwarz. Well, the game is pretty messed up to play and my brain gets pretty confused rather quickly and has no idea yet what constitutes good opening strategy.

I’ve open sourced the drawing of this board via HTML5 Canvas Element over at https://github.com/mokele/roundgoban – go nuts. I have.

I recently asked my friend Maribel over at http://maribelmade.com/ for a new Sock Monkey for a another friend’s new baby girl, and was pleasantly surprised when she asked for a hi-res version of the board to hang on her wall. Well, that’s the main reason why this repository got life. By adding some functional options to the board constructor you can now do arty things like this:

var size = 9;
var outer = function(from, to) {
    return from <= size && to < size;
};
var inner = function(from, to) {
    return from >= size*(size-1) && to >= size*(size-1);
};
var spiral = function(from, to, mod, s) {
    return (from % size) % mod == s && to == from+size;
};
var goban = RoundGoban('#goban', {
    labels: false,
    outerCircle: true,
    lineWidthMultiplier: function(from, to) {
        if(outer(from, to)) {
            return 3;
        } else if(inner(from, to)) {
            return 2;
        } else {
            return 1;
        }
    },
    strokeStyle: function(from, to) {
        if(outer(from, to)) {
            return '#BBBBBB';
        } else if(inner(from, to)) {
            return '#BBBBBB';
        } else if(spiral(from, to, 3, 0)) {
            return '#555555';
        } else if(spiral(from, to, 3, 1)) {
            return '#AA0000';
        } else if(spiral(from, to, 3, 2)) {
            return '#FF0000';
        } else {
            return '#BBBBBB';
        }
    }
});

Erlang, Protocol Buffers, and JSON... epbj

2012-10-05T00:00:00+01:00

I got fed up generating and manipulating proplists in my new Erlang project, and realised I needed a generic Protocol Buffers to JSON encoder decoder. Well, there was nearly already one, and it’s called the erlang_protobufs application. It parses .proto files and generates erlang records, so I’ve made it also do record <-> proplists conversion. It’s the ‘json’ branch of my fork of erlang_protobufs and the full demo is at the epbj demo repo, with the .proto file here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41 1> rr(pbj_pb).
[ingredient,jelly,peanut_butter,sandwich,sandwich_request]

2> PBJ = #sandwich{
    bread = 'WHOLEMEAL',
    slices = 2,
    ingredients = [
      #ingredient{
        peanut_butter = #peanut_butter{style = 'COARSE'}
      },
      #ingredient{
        jelly = #jelly{flavor = 'STRAWBERRY'}
      }
    ]
  }.
#sandwich{...}

3> JSON1 = pbj_pb:json_ready(PBJ).
[{<<"bread">>,<<"WHOLEMEAL">>},
 {<<"slices">>,2},
 {<<"ingredients">>,
  [[{<<"peanutButter">>,[{<<"style">>,<<"COARSE">>}]}],
   [{<<"jelly">>,[{<<"flavor">>,<<"STRAWBERRY">>}]}]]}]

4> JSON2 = pbj_pb:json_ready(PBJ, true).
[{<<"sandwich">>,
  [{<<"bread">>,<<"WHOLEMEAL">>},
   {<<"slices">>,2},
   {<<"ingredients">>,
    [[{<<"peanutButter">>,[{<<"style">>,<<"COARSE">>}]}],
     [{<<"jelly">>,[{<<"flavor">>,<<"STRAWBERRY">>}]}]]}]}]

5> pbj_pb:from_props(sandwich, jsx:decode(jsx:encode(JSON1))).
#sandwich{bread = 'WHOLEMEAL',slices = 2,
          ingredients = [#ingredient{peanut_butter = #peanut_butter{style = 'COARSE'},
                                     jelly = undefined},
                         #ingredient{peanut_butter = undefined,
                                     jelly = #jelly{flavor = 'STRAWBERRY'}}]}

6> pbj_pb:from_props(jsx:decode(jsx:encode(JSON2))).
#sandwich{...}

There are a few current design decisions and notes, so here they are:

encoding named ‘json_ready’ because it itself is not yet json, it’s also a very specific proplist that json encoders expect. This will surely change, this is the first thing I came up with.
decoding named ‘from_props’
Uses named fields and enums and not integer representations due to not yet wanting to generate any .js includes to more easily parse on the javascript end. This will surely change as it evolves due to wanting to save on precious bandwidth.
JSON names are camel cased
My .proto files name messages with _ naming convension (which is not what is advised by the spec) because that is currently how I prefer my erlang records to be named, and that is also how the erlang_protobufs app currently handles those names. This is a work-around.
extensions are supported and saved in the “$extensions” key of the resulting JSON. The parent message however is only ‘from_props’ decodable by the _pb module that defined the extension. This is due to erlang_protobufs having two types of value in the ‘$extensions’ dict {raw, binary()} and an actual value. JSON decoding would require a third and a lot of other changes I am not yet free to make.
I have changed string decoding to return binaries or else JSON encoding gets confused and spits out lists of integers.
I’m sure there are more, watch this space…

The new project I’m working on is in no way related to sandwiches.

Rebar release upgrade caveats / gotchas (and fixes)

2011-07-01T00:00:00+01:00

I’ve had to change a few things in rebar to make release upgrades a little easier and less error prone for our projects here at Smarkets.com. They should be beneficial to everyone, but specifically those new to OTP release upgrades or those who’ve not read all the release documentation before jumping in at the deep end with rebar. Rebar is great at getting those new and experienced with Erlang on the right track, but the release upgrades have (read had) a number of caveats that can be big gotchas to those who haven’t built their own releases from scratch without rebar before going hell for leather.

The caveats that I’ve found with the existing copy of rebar are thus:

The default in the reltool.config node template is to use archived libs which prevents you from being able to manage release upgrades
The default environment config in the node template is named app.config and is copied to your TargetDir/etc/ whereas a release upgrade requires this file be named sys.config and be located in TargetDir/releases/Vsn/ (see erlang/doc/man/config) - if this is not the case then the entire application environment (application:get_env) will be wiped.
Hipe may get built and distributed into your release regardless of whether you use it and will cause errors when generating your first upgrade.
No TargetDir/releases/RELEASES file is generated causing your first release to have an empty application libs lists. If you upgrade from a version with an empty application libs list and then downgrade back to it and release_handler:remove_release the newer of the two then all applications directories along with erts will be deleted from the release libs directory.
temporary directories “releases” and “libs” are created when running generate-upgrade which prevents there from being existing directories with these names in the same directory as a reltool.config
sasl error_logger_mf can cause VM crashes when logging possibly large terms.
generate-upgrades looks in the same directory as reltool.config for the new and old versions instead of looking in target_dir and its parent directory. This causes an enoent file error if you have a target_dir that points elsewhere.
the default generate-appups instruction for module changes is {update..}, which is incorrect

These have been addressed by:

adding the instruction to simplenode.reltool.config to not use archived libs.
updating the simplenode templates to refer to sys.config instead of app.config requiring the {{rel_vsn}} default overlay var addition used in simplenode.reltool.config
exclude hipe application by default in simplenode.reltool.config
call release_handler:create_RELEASES in generate-upgrade command
“releases” and “libs” temporary directories are placed in _tmp before being cleaned.
Remove error_logger_mf* from simplenode.sys.config. You should be using [riak_err][] or another alternative. There should be no default that has any possibility of crashing the VM.
generate-upgrades now looks for new and previous versions in the parent directory to target_dir instruction in reltool.config
changed {update..} to {load_module..} in generate-appups

These patches are available on the Smarkets github account and have also been split up into separate branches ready for pull requests to the main basho repo. You can follow the comments there once I’ve made those pull requests if you are interested to chime in / see whether any of this ever goes up-stream.

Release Upgrades with Rebar

2010-11-23T00:00:00+00:00

Update: This post was a bit overly complex for rebar release upgrades. The official rebar code now has upgrades in it’s master branch and a wiki page, so read that instead

I originally wrote my own build-tool for Erlang/OTP projects, it was an amazing exercise in learning Erlang and specifically getting my head around OTP. If you’re starting to learn this stuff now, then I highly recommend spending a few weeks as I did working with everything manually instead of having the world magically work for you.

If you don’t understand how release upgrades/installs work already, then do not read on and do not use what I’m writing to aide you in doing anything of the sort, it’s probably a bad idea.

That aside, what I’ve done is taken the release upgrade helpers from my original build tool and have started rewriting them into basho’s rebar build-tool. The outcome of this is I’ve added the gen-relup command which fetches a release’s boot_rel’s .rel file from a git ref specified by a tag along with the associated .app files so that it can then run systools:make_relup and give you a nice relup file containing the necessary instructions gathered from your .appup files. I’ve also patched generate so that it copies this to your new releases/Vsn directory along with start.boot and sys.config (have renamed and moved this from app.config due to more accurately fitting the OTP design principles for releases.

The code is available on the release_updates branch of my github fork, and here’s how to use it (my example project is named ‘pointless’ and is exactly that, and useless too):

mokele:pointless steve$ ./rebar compile gen-relup upfrom=v3 generate
==> rel (compile)
==> pointless (compile)
==> rel (gen-relup)
==> rel (generate)

What this will do is:

compiled as normal
git show refs/tags/v3:rel/pointless.rel > rel/pointless-v3.rel
mkdir ebin/v3/
git show refs/tags/v3:ebin/pointless.app > ebin/v3/pointless.app
<pre>systools:make_relup("rel/pointless.rel", ["rel/pointless-v3.rel"], ["rel/pointless-v3.rel"], [{path, ["ebin", "ebin/v3"]}])</pre>
generate release as normal
cp rel/sys.config {{target_dir}}/releases/4/
cp rel/relup {{target_dir}}/releases/4/
cp {{target_dir}}/releases/4/pointless.boot {{target_dir}}/releases/4/start.boot
cp {{target_dir}}/releases/4/pointless.rel rel/pointless.rel

At the moment this requires that you add the .rel file associated with the boot_rel value from your reltool.config into git under your reltools directory (the one that was last copied in the list above), and also add the .app files that might have been generated from .app.src files into git also, otherwise it would have to check out the .src file and regenerate the older version’s .app file, which I didn’t feel was worth it, plus I’d like to keep a snapshot of the exact .app and .rel files associated with a release anyways, incase the a future version of rebar changes something I am unaware of. These files must exist in your git repository for the tag specified in the upfrom argument, otherwise it’ll most probably fail to create a useful relup file or create one at all.

If you’re happy with the relup file then you can go ahead and do a test run of your install. One thing I do for this is git stash my changes for the new version then compile and generate my older release and copy it to rel/test_pointless (again maintaining my pointless project and release names). Then after this I can git stash pop my potential new version, compile, gen-relup, and generate it, then I can do the ugly business.

Another update I have made to the generate command is to have it also generate me a RELEASES file with relative paths to ebin directories. What this means is you’ll still be able to move your generated release’s directory around without a problem, but the downside is that you can’t use release_handler:unpack_release anymore, since that will put full path’s in the RELEASES file for your new release. You can if you really want and don’t mind pull paths once you’ve got your release running somewhere where it’s never going to get moved. In that case you can zip up your new release directory and go nuts.

But if you want to maintain relative paths in your RELEASES file then you’ll need to use release_handler:set_unpacked and then install like so (I’d like to automate this stuff too at some point):

mokele:pointless steve$ cp -r rel/pointless/lib/pointless-4 \
                              rel/test_pointless/lib/pointless-4
mokele:pointless steve$ cp -r rel/pointless/releases/4 \
                              rel/test_pointless/releases/4
mokele:pointless steve$ cd rel/test_pointless/
mokele:pointless steve$ bin/pointless attach

1> release_handler:set_unpacked("./releases/4/pointless.rel",
2>        [{pointless,"4","./lib/"}]).
{ok, "4"}
3> release_handler:install_release("4").
{ok,"3",[]}
4> release_handler:make_permanent("4").
ok

well that’s it for now. Hopefully I keep improving on this and end up putting a pull request into basho’s repository, but for now you can check it out on my github account.

You can do release upgrades with inet booted clients

2010-11-20T00:00:00+00:00

Update – The patch discussed in this post has since been accepted into OTP/R14B03.

While learning Erlang/OTP I came across these two posts by Joel Reymont, which helped a great deal in getting started using -loader inet to remotely boot a client node which doesn’t have any of the necessary code available to it directly on its own disk, it all gets magically downloaded from the master server. How nice!

A problem that I had though was the follow code snippet that Joel offered:

1
2
3
4
5
6
7
8 load_modules([]) ->
    ok;

load_modules([Mod|T]) ->
    code:purge(Mod),
    code:soft_purge(Mod),
    {module, Mod} =  code:load_file(Mod),
    load_modules(T).

which I really didn’t feel happy using, considering it side-stepped the OTP release_handler design principles, so made me feel rather dirty. So I went about figuring out how to do a proper release_handler:install_release whenever I wanted to push new live code upgrades to nodes that don’t necessarily have a disk of their own. Wouldn’t that be nice.

After finding and fixing the few places in [release_handler.erl and release_handler_1.erl][] that incorrectly tried to access the local disk for your code upgrades as apposed to using rpc or [erl_prim_loader][] like it was supposed to, I actually found it to be rather easy, and very similar to a regular release_handler:install_release that you might be used to.

The first thing to note is that when referring to the nodes that have been booted using -loader inet we need a unique releases directory for each one, or else OTP will get very confused about which running release it’s upgrading. For the boot_server you have started with erl_boot_server:start or using the kernal app config you’ll have the regular releases directory for your node. I suggest using rebar to make sure you’re doing this right, but this is not the releases directory your booted clients should be using.

Directory structure of your master node

Following the Release Structure Design Principles under Disk-Less and/or Read-Only Clients, you’ll need something like the following (where the ClientNames are your node names e.g. diskless1@hostname, diskless2@hostname, etc):

$ROOT/...
    /clients/ClientName1/bin
                        /releases/Vsn
            /ClientName2/bin
                        /releases/Vsn
            ...
            /ClientNameN/bin
                        /releases/Vsn

If you’re doing this manually you can copy the releases and bin for your diskless release, otherwise I propose writing an automated solution for when you connect/boot new nodes, or maybe this is something rebar could be doing in a special case when generating new node skeletons. You could make the diskless release the same as the boot server release, but again, be clear not to get confused between your clients’ releases directory and that of your boot server. The reason for this is because release_handler needs to keep track of each of your clients and boot server independently, it does this using the files start_erl.data and RELEASES for each node, which you’ll need to create.

Starting up your release

Now that we’ve got the right directory structure for our clients we can start them up:

HOST=127.0.0.1 # the ip address of your boot server
ID=diskless1@hostname
erl -loader inet -hosts $HOST -id $ID -name $ID -setcookie samecookie \
  -boot /full/path/to/your/node/clients/diskless1@hostname/releases/1/start \
  -config /full/path/to/your/node/clients/diskless1@hostname/releases/1/sys

Where our sys.config looks like so (and boot_server@hostname is our inet boot server):

[{sasl, [
  {masters, [boot_server@hostname]},
  {client_directory, "/full/path/to/your/node/clients/diskless1@hostname/"},
  {releases_dir, "/full/path/to/your/node/clients/diskless1@hostname/releases/"}
]}]

Read about these sasl app configuration options on erlang.org

release_handler:install_release/1

Now you’re good to go! By manually unpacking the releases/NewVSN directory to your client’s path (containing the required relup file!) and each of the new libs required to the boot server’s lib path (with the necessary app files!), you can now run release_handler:set_unpacked then release_handler:install_release on your inet booted node. It’s also possible to keep release specific libs inside a lib directory within each client_directory.

(diskless1@hostname)1> release_handler:set_unpacked(
  "/full/path/to/your/node/clients/diskless1@hostname/releases/2/myrelease.rel",
  [myapp, "2", "/full/path/to/your/node/lib/"]).
{ok,"2"}
(diskless1@hostname)2> release_handler:install_release("2").
{ok,"2",[]}
(diskless1@hostname)3> release_handler:make_permanent("2").
ok
(diskless1@hostname)4> release_handler:which_releases().
[{"myrelease","2",
  ["kernel-2.14.2","stdlib-1.17.2","myapp-2", ...], permanent},
 {"myrelease","1",
  ["kernel-2.14.2","stdlib-1.17.2","myapp-1", ...], old}]

sync_nodes

Actually, we’re not quite done yet. There’s one brilliant feature OTP provides that we haven’t yet made use of, and that’s release installs with sync_nodes. When we have more than one node running the same version of a release, we need to synchronise the release install between all of these nodes. This appup and relup instruction is briefly explained in the appup docs and relup docs. What this does is to halt all the nodes specified in the nodes list when they each reach this instruction, all of which then contact one another and wait until the very last one has also reached this point. After this they all resume and the installation is sync’d like magic.

In the example above the myapp.appup and subsequently the relup for myrelease vsn 2 will contain an instruction along the lines of the following (along with the other load_modules and other instructions your update requires):

{sync_nodes, shared_id_atom, 
  [diskless1@hostname, diskless2@hostname, diskless3@hostname]}

When your relup involves this instruction it will halt forever unless all nodes also reach this point, so be careful. I’m not entirely sure of the best practice in dealing with nodes that are down or don’t reach there, so if someone knows then that’d be a great addition to this point.

ok, there we go. Something that might be described as an explanation of how to do release upgrades on and across inet booted client nodes. If only there was something to automate the creation and upgrading of possibly a lot of nodes. *faints*

Post Revisions with Jekyll

2010-11-19T00:00:00+00:00

Note: this post is now hosted by GitHub so this extension to Jekyll is not available and visible on this post

As you can see to the right of this text, there’s a link to a page that shows the difference between the current version of this post and that of the previous version(s). It will include the addition of the introductory paragraph you are reading right now that I added after committing the rest below.

Yesterday after getting some of my previous posts imported and rendered onto mokele.co.uk using Jekyll, I realised I’d never felt more happy setting up a new blog engine (of sorts). If you haven’t already, you already know you’ve put it off too long, so do it now. I can’t explain this entirely, but I think it might be down to the confidence I didn’t realise I had in hosting a site made entirely of static files; it’s quite a weight of my shoulders.

That said, an obvious downside to Jekyll is the absence of dynamic content. If one was to add the possibility of on-request dynamic content it would definitely kill its elegance, and we’d have to think very carefully about how to do it without simply doing it wrong.

It’s still an interesting thing to think about, and after you realise it’s dynamic already through the git commits you’re making (followed by the static file generation), then maybe it isn’t so hard to think about how best to extend it to meet more needs.

Following on from this idea (I’m not sure why I decided to do it, but it was still fun), I’ve forked Jekyll and have made it possible to embed the git revision histories of your posts into their rendered pages, something that would have previously been developed as truly on-request dynamic functionality. (it also requires my fork of ruby-git)

jekyll --revisions

What this does is to add the list page.revisions to the templates of each post, and to generate pages for each commit involving each post. So what I’ve done is added the following snippet to my post.html template:

{% if page.revisions %}
<div class="revisions">
  <strong>Changes to this Post</strong>
  <ul>
  {% for revision in page.revisions %}
    <li>
      <span>{{ revision.date | date: "%d %B %Y" }}</span><br />
      <a href="{{ revision.url }}">{{ revision.message }}</a>
    </li>
  {% endfor %}
  </ul>
</div>  
{% endif %}

As well as this it will need the file _layouts/revision.html to define the layout for each changeset diff page. By accessing diff.patch, diff.message, and diff.date in this template you can display a nice little diff page. You should look on github at my revision.html.

In Summary

Having a visible revision history for each post is an interesting idea, considering it would enable you to treat your blog as more of a wiki than a static set of posts, and could also be used to do things such as update the updated tag in your atom.xml or other generated feed. It’s simply an extension to Jekyll that doesn’t break the concept and makes use of the git log for your posts, and maybe there’s more to be done around making it more dynamic, but I know what I’m not going to be doing, making it require server-side logic for page requests.

Last.fm Semantic URI Routing / Rewriting

2009-03-02T00:00:00+00:00

We’ve always been big on semantic URIs here at Last.fm, and I don’t think that’ll ever change (if not just due to not wanting to crush our google rank by screwing up any missing permanent redirects). Apache mod_rewrite has been a staple in the Audioscrobbler and Last.fm dev toolkit for many years, and we still use it now and again for webservices and some legacy bits and bobs, but it was getting rather annoying in cases where we wanted to reuse controllers for many different routing paths.

Consider these 2 URLs:

http://www.last.fm/music/Dananananaykroyd/+events
http://www.last.fm/user/mokele/events

Although the URLs are completely different apart from the word “events” the resulting controller action and template we want to end up on is actually the same. All that changes in this example is the request context which is written back to the Request along the path of parsing the URI from left to right.

The way this is done is by controller delegation through the concept of Routes and Rewriters.

Below is a massive flow diagram of Routing and Rewriting for the Last.fm Radiohead Wiki editing page, and after that we’ll talk more indepth about what’s actually going on in plain English.

http://www.last.fm/music/Radiohead/+wiki/edit

<?php
// from /www/index.php
new Route('music', array('namespace' => 'music'));

// from /music/musiccontroller.class.php
new Rewriter('$noredirect$artistname/', array(
        'noredirect' => '(\+noredirect\/|)',
        'artistname' => '((?!\+)(.[^\/]*))',
        'controller' => 'artist',
     ),
    array('artistname' => 3)
);

// from /music/artistcontroller.class.php
new Rewriter('+');
new Route('wiki', array('action' => 'view'));
?>

Now here’s the same with loads of weird lines in describing the flow of terms

<?php
// -> /www/index.php

There are no Rewriters on the FrontController level, just Routes.
 
   /music/Radiohead/+wiki/edit
   |____|
    | action
    |_________
           __|__
          |     |
new Route('music', array('namespace' => 'music'))
          |_____|                       |_____|
              |                             |
          _____\____________________________|
         |     /
       __|__ _|___
      |     |     |
// -> /music/musiccontroller.class.php
 
/**
 * The default behaviour for naming a controller is the name of the matched route.
 * You can also override this behaviour by supplying a 'action' => 'my' which would
 * end up finding mycontroller.class.php.
 * From this point on the logic of 'find first Rewriter match then first Route match'
 * repeats each time a controller file changes. (indicated by ->)
 */
 
   /Radiohead/+wiki/edit
   |_________|
     action
 
The action is always defaulted to the next / separated part of the URI
 
new Rewriter('$noredirect$artistname/', array(
        'noredirect' => '(\+noredirect\/|)',
                         |_______________|
                             |noredirect request param

        'artistname' => '((?!\+)(.[^\/]*))',
                                |_______|
                                  |artistname request param
        'controller' => 'artist', |
                        |______|  |
                               |  |
     ),                        |  |
                            ___\__|
                          |    /
                          |    |
    array('artistname' => 3)   | 3rd () part of the regex above
),                             |
                _______________|
               |
             __|___
            |      |
// -> /music/artistcontroller.class.php
 
   /+wiki/edit
   |_____|
     action
 
new Rewriter('+')
 
/**
 * Rewriters take the full URI, match it and save it back without what it matched
 * so this basically just strips the +
 */
 
   /wiki/edit
   |____|
     | action
     |______
           _|__
          |    |
new Route('wiki', array('action' => 'view'))
          |____|                    |____|
        ___|                          action
       _|__
      |    |
// -> /wikicontroller.class.php
 
/**
 * If /edit was *missing* from the url then the current action would be 'view'.
 * The default action when it runs out of url to parse is 'overview'
 */
 
   /edit
   |____|
     action
?>

Now that we’ve got the amazingly awesome looking crazy ASCII flow diagram out of the way let’s explain a little more about what the hell we’re actually doing!

The first step is done by the front controller which has a list of global Routes that it will try and match against the first ‘/’ separated token of the request URI. When it finds it the Route will tell it which namespace and controller to delegate to.

<?php
$routes = array(
    new Route('user', array('namespace' => 'user')),
    new Route('group', array('namespace' => 'group')),
    new Route('about'),
    //... etc
);
//construct front controller etc here
?>

Now when a request comes in to any URI starting with /about it will match the 3rd of our Routes and get delegated to the about controller.

Now that we’re out of the front controller the next steps repeat until the whole URI has been parsed. Each controller has it’s own list of Rewriters and Routes of which it may have none. We then run through each Rewriter and see if it matches the URI. The whole URI is used for this, which is unlike Routes which only match against the next URI token, and Rewriters also use a regular expression to match the URI as apposed to a static string for Routes. Everything matched will be chomped off of the URI before any subsequent controller delegation. This is the time at which request parameters can be grabbed and assign to the request with a given name.

<?php
// usercontroller.class.php
$rewriters = array(
    new Rewriter('$username/'),
);
?>

For URIs such as /user/mokele this Rewriter will end up matching and assigning the ‘username’ request parameter since ‘mokele’ matches the default regex ([^\/]+), and we always shove a slash on the end of URIs after the request comes in when using Routes and Rewriters.

Now that we’ve got the name of the user the user controller will construct that user by it’s name and assign it back to the request for itself and possible controller delegates to fetch.

If a Rewriter matches and it has a ‘controller’ parameter then this next Routing step is skipped. Now that we’ve done the Rewriting step we move back to the Routing step for the user controller. We do exactly the same as the front controller did and match against the first token of the rest of the URI that we have left to parse. The difference here is that if nothing is matched then this token will become our controller action at which point delegation ends. If there is no URI left to parse then the action will be the default which is ‘overview’.

/user/mokele ‘overview’ action

/user/mokele/overview ‘overview’ action again

/user/mokele/tags matches no routes so goes to the ‘tags’ action

/user/mokele/events match the ‘events’ route and delegate to the the events controller with action ‘overview’ and a context of the User ‘mokele’

This is the end of the delegation logic. If we’ve delegated at any point we start the process again until we have no URI left to parse.

In Summary

By doing URI parsing this way we’ve created clear separation of logic dependent on each token or segment of the URI. The front controller decided the first bit, the user controller decided that ‘mokele’ is a username and that a User object should be constructed with it, and so on. It also doesn’t require an Apache restart for changes to take effect, definitely a big win.

Finally, the one Apache mod_rewrite configuration we need to add is this: (and before you start trying… index.php is only one of a few files in our document root)

RewriteCond %{SCRIPT_FILENAME} !-f
RewriteCond %{SCRIPT_FILENAME} !-d
RewriteRule ^(.*)$ index.php [L]