deferred until inspiration hits

Confusing git log behaviour

Chris Roos — Fri, 20 Nov 2015 16:20:25 +0000

I was confused by the behaviour of git log when using the before and after options. After some playing I realised that it was because the default git log output shows the author date but the before and after options operate on the commit date.

I can demonstrate the behaviour in an empty repository.

Create a new git repo

$ cd /tmp
$ mkdir testing-git-log
$ cd testing-git-log
$ git init

Create a file and add it to the repo

$ touch file1
$ git add file1
$ git commit -m "Add file1"

Edit the author and commit date of the commit

$ GIT_COMMITTER_DATE="2015-01-02 09:00" \
git commit --amend \
--date "2015-01-01 09:00" \
--reuse-message HEAD

The --date option sets the author date of the commit.

Setting GIT_COMMITTER_DATE affects the commit date of the commit when it’s amended.

Check that the author and commit date have been set as expected

$ git log --pretty=fuller
Author:     Chris Roos <chris@seagul.co.uk>
AuthorDate: Thu Jan 1 09:00:00 2015 +0000
Commit:     Chris Roos <chris@seagul.co.uk>
CommitDate: Fri Jan 2 09:00:00 2015 +0000

    Add file1

Use git log to show commits on 1st Jan 2015

$ git log --after "2015-01-01 00:00" --before "2015-01-01 23:59:59"
# Nothing shown

Use git log to show commits on 2nd Jan 2015

$ git log --after "2015-01-02 00:00" --before "2015-01-02 23:59:59"
commit 5f2c92e2b44ab697f92cb4336fd66c6206f56be2
Author: Chris Roos <chris@seagul.co.uk>
Date:   Thu Jan 1 09:00:00 2015 +0000

    Add file1

Manually uploading activities from Garmin Fit app

Chris Roos — Sat, 14 Nov 2015 08:06:40 +0000

I had two activities recorded on 12 Nov 2015 that weren’t syncing to Garmin Connect from the Garmin Fit app on my phone. I signed out and back in a number of times to try to force the sync (as per Garmin’s support page), but to no avail.

I eventually used Airdroid to download the two .fit files (stored in /SD Card/Android/data/com.garmin.android.apps.fitness/files/) from my phone to my laptop.

I was able to manually import those files into Garmin Connect and can now see the two activities as expected.

After signing out and back in to the Garmin Fit app I noticed that the 2 activities on 12 Nov 2015 were now duplicated - presumably the versions on the phone were slightly different from the versions on the website. I fixed this by removing all the Garmin Fit app data from my phone and signing in one final time.

Garmin Fit app has been removed

While writing this I tried to find a link to the Garmin Fit app in the Google Play Store. While I can see it in my list of installed apps, it no longer appears in the public store. I also followed the link to the app in iTunes but that reports that it’s no longer available in the UK store. I can’t find any official information about the app being removed but this thread on the Garmin forums suggests that it was removed in early 2015!.

Getting my Webby powered website running on Mac OS X El Capitan

Chris Roos — Thu, 29 Oct 2015 17:20:00 +0000

This is what I needed to do to get Webby powered website working on my laptop.

$ cd ~/Code/github/chrisroos.co.uk

$ cat .rbenv-version
1.8.7-p352

$ rbenv install 1.8.7-p352

ERROR: This package must be compiled with GCC, but ruby-build couldn't
find a suitable `gcc` executable on your system. Please install GCC
and try again.

DETAILS: Apple no longer includes the official GCC compiler with Xcode
as of version 4.2. Instead, the `gcc` executable is a symlink to
`llvm-gcc`, a modified version of GCC which outputs LLVM bytecode.

For most programs the `llvm-gcc` compiler works fine. However,
versions of Ruby older than 1.9.3-p125 are incompatible with
`llvm-gcc`. To build older versions of Ruby you must have the official
GCC compiler installed on your system.

TO FIX THE PROBLEM: Install Homebrew's apple-gcc42 package with this
command: brew tap homebrew/dupes ; brew install apple-gcc42

You will need to install the official GCC compiler to build older
versions of Ruby even if you have installed Apple's Command Line Tools
for Xcode package. The Command Line Tools for Xcode package only
includes `llvm-gcc`.

BUILD FAILED (OS X 10.11.1 using ruby-build 20150818)

$ brew tap homebrew/dupes ; brew install apple-gcc42
==> Tapping homebrew/dupes
Cloning into '/usr/local/Library/Taps/homebrew/homebrew-dupes'...
remote: Counting objects: 42, done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 42 (delta 0), reused 4 (delta 0), pack-reused 0
Unpacking objects: 100% (42/42), done.
Checking connectivity... done.
Tapped 39 formulae (103 files, 420K)
==> Installing apple-gcc42 from homebrew/homebrew-dupes
==> Downloading http://r.research.att.com/tools/gcc-42-5666.3-darwin11.pkg
######################################################################## 100.0%
==> /bin/pax --insecure -rz -f usr.pkg/Payload -s ,./usr,/usr/local/Cellar/apple-gcc42/4.2.1-5666.3,
==> Caveats
NOTE:
This formula provides components that were removed from XCode in the 4.2
release. There is no reason to install this formula if you are using a
version of XCode prior to 4.2.

This formula contains compilers built from Apple's GCC sources, build
5666.3, available from:

  http://opensource.apple.com/tarballs/gcc

All compilers have a `-4.2` suffix. A GFortran compiler is also included.
==> Summary
🍺  /usr/local/Cellar/apple-gcc42/4.2.1-5666.3: 104 files, 75M, built in 23 seconds

$ rbenv install 1.8.7-p352
Downloading ruby-1.8.7-p352.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/2325b9f9ab2af663469d057c6a1ef59d914a649808e9f6d1a4877c8973c2dad0
Installing ruby-1.8.7-p352...
Installed ruby-1.8.7-p352 to /Users/chrisroos/.rbenv/versions/1.8.7-p352

Downloading rubygems-1.6.2.tgz...
-> https://dqw8nmjcqpjn7.cloudfront.net/cb5261818b931b5ea2cb54bc1d583c47823543fcf9682f0d6298849091c1cea7
Installing rubygems-1.6.2...
Installed rubygems-1.6.2 to /Users/chrisroos/.rbenv/versions/1.8.7-p352

$ rbenv rehash

$ gem install bundler
Fetching: bundler-1.10.6.gem (100%)
Successfully installed bundler-1.10.6
1 gem installed

$ bundle install
Fetching gem metadata from http://rubygems.org/..........
Fetching version metadata from http://rubygems.org/..
Resolving dependencies...
Installing rake 0.9.2.2
Installing RedCloth 4.2.7 with native extensions
Installing activesupport 3.0.0
Installing addressable 2.2.6
Installing diff-lcs 1.1.3
Installing directory_watcher 1.4.1
Installing haml 3.0.25
Installing hpricot 0.8.5 with native extensions
Installing launchy 2.0.5
Installing little-plugger 1.1.3
Installing logging 1.6.1
Installing loquacious 1.9.1
Installing rdiscount 1.6.8 with native extensions
Installing rspec-core 2.7.1
Installing rspec-expectations 2.7.0
Installing rspec-mocks 2.7.0
Installing rspec 2.7.0
Installing webby 0.9.4
Using bundler 1.10.6
Bundle complete! 5 Gemfile dependencies, 19 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.

$ webby autobuild
WARNING: Global access to Rake DSL methods is deprecated.  Please include
...  Rake::DSL into classes and modules which use the Rake DSL methods.
WARNING: DSL method Webby::Apps::Main#import called at /Users/chrisroos/.rbenv/versions/1.8.7-p352/lib/ruby/gems/1.8/gems/webby-0.9.4/lib/webby/apps/main.rb:177:in `import_default_tasks'
WARNING: DSL method Webby::Apps::Main#import called at /Users/chrisroos/.rbenv/versions/1.8.7-p352/lib/ruby/gems/1.8/gems/webby-0.9.4/lib/webby/apps/main.rb:177:in `import_default_tasks'
WARNING: DSL method Webby::Apps::Main#import called at /Users/chrisroos/.rbenv/versions/1.8.7-p352/lib/ruby/gems/1.8/gems/webby-0.9.4/lib/webby/apps/main.rb:177:in `import_default_tasks'
WARNING: DSL method Webby::Apps::Main#import called at /Users/chrisroos/.rbenv/versions/1.8.7-p352/lib/ruby/gems/1.8/gems/webby-0.9.4/lib/webby/apps/main.rb:177:in `import_default_tasks'
WARNING: DSL method Webby::Apps::Main#import called at /Users/chrisroos/.rbenv/versions/1.8.7-p352/lib/ruby/gems/1.8/gems/webby-0.9.4/lib/webby/apps/main.rb:177:in `import_default_tasks'
[17:22:46]  INFO: starting autobuild (Ctrl-C to stop)
[17:22:47]  INFO: running the build
[17:23:18]  INFO: running the build

Exploring my Google Location History

Chris Roos — Sat, 01 Nov 2014 09:44:26 +0000

I used Google Takeout to export my Location History as JSON. I wanted to know what date range the file covered (it’s not obvious from the Location History site) and figured I’d use command line tools rather than parsing the JSON.

This is the command I ended up with to display the earliest timestamp in the file:

$ cat LocationHistory.json | \
grep timestampMs | \
grep -o -E "\d+" | \
sort | \
head -n1 | \
cut -c1-10 | \
xargs -J% date -j -f "%s" %

#=> Sun  4 Mar 2012 12:33:29 GMT

Building the command

Use grep to extract all the lines containing timestamps.

$ head LocationHistory.json | \
grep timestampMs

#=> "timestampMs" : "1414831041235",

Use grep to extract the timestamp from each line.

$ head LocationHistory.json | \
grep timestampMs | \
grep -o -E "\d+"              # Find 1 or more digits and print the matching string

#=> 1414831041235

Use cut to strip off the milliseconds from the timestamp.

$ head LocationHistory.json | \
grep timestampMs | \
grep -o -E "\d+" | \
cut -c1-10                    # Ignore the last three digits (milliseconds) of the timestamp

#=> 1414831041

Use date to display the timestamp in a human friendly format.

$ head LocationHistory.json | \
grep timestampMs | \
grep -o -E "\d+" | \
cut -c1-10 | \
xargs -J% date -j -f "%s" %   # Use `-j` to avoid setting the date
                              # and -f "%s" to tell date to parse the string as unix time

#=> Sat  1 Nov 2014 08:37:21 GMT

The full command adds sort to ensure all timestamps are ordered and then either head -n1 or tail -n1 to print the earliest and latest dates respectively.

Notes from The First International Conference on Software Archaeology

Chris Roos — Fri, 07 Feb 2014 12:25:17 +0000

James and I attended The First International Conference on Software Archaeology last week.

I’ve spent some time going through the notes I made and have categorised the things I took from the presentations into three broad themes. I’m not sure how effective a picture this paints for anyone that wasn’t there but hopefully there’s something useful in here. For an alternative overview, please see Jeff Foster’s TICOSA writeup or the TICOSA sessions page which contains some of the presentation slides.

The three broad themes are analysing legacy code (the past), working on legacy code (the present) and helping our future selves (the future).

Analysing legacy code

I think the main thing I took away from this theme was that there’s quite a lot of work going into what still appears to be a fairly new idea.

Michael Feathers demonstrated the use of his delta-flora tool to analyse Ruby projects over the history of the git repository.

Dmitry Kandalov demonstrated his code-history-mining IntelliJ plugin that was inspired by Michael’s delta-flora.

Robert Smallshire showed us a number of ways of visualising complexity, including maps of complexity density, and explained how we can use these to identify areas of concern in a system.

Stéphane Ducasse told us that having to maintain software means that the software is useful and successful. So, maintenance is a good thing but we can often benefit from dedicated tools that help us understand our specific problems.

Working on legacy code

The main thing I took away from this theme was the reinforcement of my belief that we shouldn’t be so quick to think that we can improve something by throwing it away and rewriting it.

Avoiding the temptation to throw away what’s there

Stéphane Ducasse reminded us that, “Early decisions were certainly good at that time”

Keith Braithwaite explained that different types of people approach problems in slightly different ways: Computer scientists might prefer a single longer method that contains all the logic, while OO advocates might prefer many smaller methods. There are merits to both approaches and we should remember that projects coded in a style different to our own aren’t wrong; they’re just different.

Chris Oldwood’s lightning talk contained a paraphrased quote (coined by either Robert Frost or G.K. Chesterton) that really resonated with me:

Don’t ever take a fence down until you know why it was put up

Stuart Curran reminded us of George Santayana’s quote, “Those who cannot remember the past are condemned to repeat it”

Using tools to get an overview of the code we’re working on

Michael Feathers (using delta-flora), Dmitry Kandalov (using his IntelliJ plugin) and Robert Smallshire (in his dependency structure matrices) all demonstrated ways of visualising which parts of the system are tightly coupled.

Helping our future selves

The main thing I took away from this theme was that code will be around for longer than we expect and we should do our best to help our future selves.

Stéphane Ducasse reminded us that software is a living entity and will probably survive for longer that we might imagine. Stéphane gave examples of military and insurance systems that have been around for 15/20 years, which seemed like a long time until I realised that Mocha has been around for about 8 years.

Robert Smallshire showed some graphs that suggest code is likely to outlast most other aspects of an organisation. By the time you’re working on a 20 year old codebase, you’re probably working in a team where only 20% have been around from the beginning.

Michael Feathers suggested we be more aware of how seemingly unrelated factors can affect the complexity of our code. He used the fascinating example of how parole is more likely to be granted at certain times of day to illustrate the sort of things we could be looking for.

I loved Matt Wynne’s metaphor that he uses to explain refactoring to non-developers: Think of developers as chefs, their output as meals and refactoring as keeping the kitchen clean. Matt also talked about analysing the commits in the new cucumber-ruby-core project and saying that he thinks that about 75% of the commits are refactoring commits.

Daniel Brolund described how to use the Mikado method to ensure that we’re always reshaping the system to get it into a state that’s ready to accept new changes.

Ivan Moore gave us some practical advice about being kind to our future selves in his lightning talk.

Deploying Rails apps using Recap, Apache and Passenger

Chris Roos — Tue, 15 Oct 2013 14:49:07 +0000

This assumes that you already have Apache and Passenger installed; and that you’ve run recap setup and configured your Capfile with the address of your server.

I’m going to use ‘thames-time-lapse’ as an example application name but you’ll want to replace that with the name of your app in the instructions below.

The first step is to set-up the server and get the code deployed.

$ cap bootstrap
$ cap deploy:setup
$ cap deploy

We now want to make Recap’s configuration options available to Passenger. We can do this by changing the application user’s shell to bash and loading Recap’s environment variables in that user’s .bashrc.

# Set the app user's shell to bash
# So that Passenger can pick up environment variables from .bashrc
$ sudo chsh -s /bin/bash thames-time-lapse

# Create a .bashrc that loads the recap environment variables
$ sudo su - thames-time-lapse
$ echo 'source ~/.recap-env-export' > ~/.bashrc

Add an Apache virtual host for our application. Note that I’m setting the PassengerRuby option as it defaults to the user that owns config.ru, which happens to be the user that deployed rather than the application user.

$ sudo vi /etc/apache2/sites-available/thames-time-lapse.chrisroos.co.uk.conf

# Paste the following content
<VirtualHost *:80>
  # Set PassengerRuby if you're using a non-system Ruby
  PassengerRuby /usr/local/bin/ruby
  # We have to set the user because config.ru is owned by the user that deployed
  PassengerUser thames-time-lapse
  ServerName thames-time-lapse.chrisroos.co.uk
  DocumentRoot /home/thames-time-lapse/app/public
  <Directory /home/thames-time-lapse/app/public>
    Allow from all
    Options -MultiViews
  </Directory>
</VirtualHost>

# Make the site available to Apache
$ sudo a2ensite thames-time-lapse.chrisroos.co.uk.conf

# Restart Apache
$ sudo service apache2 reload

All being well, your Rails app has now been deployed with Recap and is being served with Apache and Passenger.

The final step is to define the deploy:restart Capistrano task so that the app is restarted after deployment. Add the following task to your Capfile.

namespace :deploy do
  task :restart do
    as_app 'touch tmp/restart.txt'
  end
end

And that should be everything you need to get your app deployed using recap, and served by Apache and Passenger.

Moving my timelapse app from Heroku to AWS

Chris Roos — Sun, 13 Oct 2013 17:08:54 +0000

I love Heroku and tend to use the free resources to deploy apps when they’re in their very early stages of development. I have a few things that I continue to use Heroku for but unfortunately ran into some of the limits with the Thames Time Lapse project.

The first limitation was the number of rows in the free development database. The dev plan gives you 10,000 rows for free but I managed to get up to around 150,000 before receiving a warning email from Heroku. I only create around 1,000 rows a day, and I’m nowhere near 150 days in, so I’d obviously screwed up somewhere along the line.

Another limitation was in my use of the S3 gem to find the new images that had been uploaded to S3. As far as I can tell the gem doesn’t understand S3 folders which meant that I had to scan the entire S3 bucket looking for new images. As more were uploaded, so this took longer and longer which just wasn’t sustainable. Moving to AWS has allowed me to use the excellent s3cmd command line interface to greatly improve the performance of looking for new images.

Finally, having the app running on AWS should also make it easier to use imagemagick to create smaller versions of the images, and ffmpeg to automatically create a timelapse of each day of images.

Instructions for settings up an AWS instance (I’m using a free micro instance) are in the README of the application should you be so inclined to play along.

Time slice images of the Thames

Chris Roos — Sun, 15 Sep 2013 17:23:24 +0000

This image was created by taking 24 images (well, I think it might actually be 23 because my maths was off) from the set I captured on the 30th August 2013.

I played around with different numbers of images making up the final image, but felt that one per hour produces the best results.

I’ve also had a bit of a play with creating a movie from a set of 27 of these time-slice images. I really like the undulating effect of the water. You can also view it on Vimeo if you can’t see the video below.

Your browser does not support the video tag.

Why split the subject and body in git commit messages?

Chris Roos — Mon, 09 Sep 2013 15:53:06 +0000

Tim Pope’s, “A Note About Git Commit Messages” suggests that we should split our git commit message into subject and body, with a newline between the two. Tim lists some tools that make use of this split and I thought I’d follow that with some screenshots showing how this convention is used in four fairly common (well, I use them) git tools.

GitHub for Mac

With a long commit message on a single line

With the commit message split into subject and body

Gitk

With a long commit message on a single line

With the commit message split into subject and body

GitX

With a long commit message on a single line

With the commit message split into subject and body

Git cli

With a long commit message on a single line

With the commit message split into subject and body

Automatically updating my desktop background from my Thames time lapse pictures

Chris Roos — Mon, 09 Sep 2013 10:39:55 +0000

One of the things I wanted to do with the pictures of the Thames that I’ve been capturing is to periodically update my desktop background to show the most recent image.

I was investigating how best to hook it all up when I came across this article that describes how to Set the Desktop Picture to NASA Image of the Day. I figured that I could add a feed of the recent images to the thames-time-lapse website and follow the same workflow to set my desktop background.

Once I had the workflow downloading images and automatically setting my desktop background, I added a cron job to execute that workflow every 15 minutes.

Another thing I’ve been meaning to do is try to improve the quality of some of the images: They’re being taken through a pretty dirty window and suffer from both the dirt and reflections as the light changes during the day. I’m hoping that the increased visibility of the images will prompt me to do something about this sooner rather than later.