Gnuplot is an incredibly powerful and useful utility for presenting data in graphical format. In this post I’ll cover a very simple usecase that involves graphing the size of a database over a period of time.

Installing

On Linux use your normal package manager, e.g.

# Debian based...
apt-get install gnuplot

# Redhat based...
yum install gnuplot

The easiest way to install gnuplot on OS X is using brew. Gnuplot requires X11 to display graphs so to install it use,

brew install gnuplot --with-x11

If X11 support isn’t available gnuplot will fail to display graphs with the error,

WARNING: Plotting with an 'unknown' terminal.
No output will be generated. Please select a terminal with 'set terminal'.

If this happens uninstall and then reinstall with X11 support.

Basic Usage

In the example below I’ll use a data file called ‘db_size.dat‘. Each line in this file records the size of a database at an hourly interval.

The most basic gnuplot command to graph this data would be,

gnuplot -p -e "plot 'db_size.dat'"

The -p parameter (short for persist) leaves the window containing the graph open after gnuplot finishes.

The -e parameter is the command to pass to gnuplot. The command “plot ‘db_size.dat'” means plot a graph using the data in db_size.dat.

Running this command will open a a window with this graph.

Depending on your requirements this simple graph may be absolutely fine. However if you’d like to polish it up there are a few changes we can make.

First of all lets give the graph a title.

gnuplot -p -e "set title 'Database Size'; plot 'db_size.dat'"

At this stage the command line is getting a little long. This is where gnuplot command files come into play. Each file is basically a list of commands.

This command file is called ‘db_size.gnuplot’,

set title 'Database Size'
plot 'db_size.dat'

The command file name is passed to gnuplot as a parameter,

gnuplot -p db_size.gnuplot

Lets add axis labels and give the key a name.

set title 'Database Size'
set xlabel "Time"
set ylabel "Size (bytes)"
plot 'db_size.dat' title 'Database Size'

The dots can be difficult to follow when the they’re spaced out so lets use a line instead.

set title 'Database Size'
set xlabel "Time"
set ylabel "Size (bytes)"
plot 'db_size.dat' title 'Database Size' with line

The last change I’d like to make is to make the Y-axis scale more readable by converting bytes to gigabytes.

set title 'Database Size'
set xlabel "Time"
set ylabel "Size (GB)"
plot 'db_size.dat' using ($1/1000000000) title 'Database Size' with line

Finally lets save the graph to a PNG rather than displaying it on screen.

set term png
set output "db_size.png"
set title 'Database Size'
set xlabel "Time"
set ylabel "Size (GB)"
plot 'db_size.dat' using ($1/1000000000) title 'Database Size' with line

This saves the graph to a file called db_size.png in the current directory.

Going a Little Further

In the previous example there was one data value recorded at fixed intervals, i.e. hourly. But what if the same data is collected at irregular intervals? To reliably graph this data we need to record the time each data point is recorded, db_size_with_time.dat. In this file the first column is seconds since the epoch. Gnuplot can parse this using the ‘set xdata’ command. Here’s the full command file,

set title 'Database Size'
set xlabel "Time"
set ylabel "Size (GB)"
set key off
set xdata time
set timefmt "%s"
set xtics format "%H"
plot 'db_size_with_time.dat' using 1:($2/1000000000) with line

‘set key off’ turns off the key. Its not very useful in this graph.

‘set xdata time’ parses the X axis data as a time.

‘set timefmt “%s”‘ means read time values as seconds since the epoch.

‘set xtics form “%H”‘ means write the hour to the X axis

This post barely scratches the surface of what can be achieved using gnuplot. It’s really an amazing piece of software.

For more information:

• Demo scripts for gnuplot version 5
• Official gnuplot Documentation

Each amendment describes the change being proposed like this (from the second amendment in respect of a change being proposed to article 25),

In sub-section 1° of section 2, the deletion of the word “five” and the insertion in its place of the words “the fifth”, and the deletion of the words “seven days” and the insertion in their place of the words “the seventh day”

While its accurate it’s not very easy to see the impact of the change in the context of the document.

In the software engineering world we store code, which is just text, in source control. Using the tools provided by the source control system we can compare two versions of a document to see how they differ.

To get to the point where I could compare different versions of the constitution I assembled a text version of the document and created a commit for each amendment. The end result is this git repository, https://github.com/adrian/irish-constitution.

Every amendment up to and including amendment thirty-three is included.

For the sake of completeness I included the transitory articles 51 to 64 and 34a. Rather than including the transitory sub-sections 29.7.3, 29.7.4 and 29.7.5 (from amendment 19) I made the changes directly to articles 2 and 3.

Each commit of constitution.md represents an amendment. By opening any of these commits we can see how that amendment changed the constitution. For example, amendment thirty-three looks like this.

The left column is the constitution before amendment thirty-three, the right is after amendment thirty-three. Red lines are deleted or changed with dark red words deleted. Green lines are added or changed with dark green words added. By showing the two versions of the document, before and after, and highlighting the changes the effect of the amendment should be easier to understand.

To round it off there are two pending changes (pull requests) for each of the upcoming referendums on the 22nd May 2015,

• Thirty-fourth Amendment of the Constitution, Act 2015
• Thirty-fifth Amendment of the Constitution, Act 2015

Android Studio requires Oracle Java 1.6 or later. It won’t work with OpenJDK.

Installing Oracle Java 1.7

These instructions describe how to install Oracle Java using java-common. java-common only works with Java 1.6 or 1.7. It doesn’t work with Java 8.

• Download the latest 1.7 JDK from http://www.oracle.com/technetwork/java/javase/downloads/index.html. To find the tarball follow the links “Previous Releases – Java Archive” -> “Java SE 7” -> “Java SE Development Kit 7u..”. You’ll need the tarball not the RPM, e.g. jdk-7u67-linux-x64.tar.gz. Note: you’ll need an Oracle account to download archive versions of Java (those < Java 8).

• Install java-common
  

  # apt-get install java-common

• Create a deb file using the Java tarball,
  

  $ make-jpkg <downloaded tarball>

• Install the resulting package (your deb file name will likely be different),
  

  # deb -i oracle-j2sdk1.7_1.7.0+update67_amd64.deb

• Update your java links to point at this new Oracle Java package. To find the name of this package take a look in /usr/lib/jvm/. There should be a directory named j2sdk1.7-oracle (or something similar). Use this directory name in a call to update-java-alternatives.
  

  # update-java-alternatives -s j2sdk1.7-oracle

• Test your system is using the correct version of Java using,
  

  $ java -version
   java version "1.7.0_67"
   Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
   Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

  $ javac -version
   javac 1.7.0_67

Install the Android SDK Tools

The Android SDK Tools package contains the SDK Manager. You’ll need this to download one or more SDKs.

Download “SDK Tools Only” from https://developer.android.com/sdk/index.html. Extract the downloaded file to your applications directory, e.g. ~/apps/.

Install Android Studio

Download Android Studio from http://developer.android.com/sdk/installing/studio.html.

Unzip the downloaded file to whereever you keep your apps, e.g. ~/apps/.

To start Android Studio run <install dir>/android-studio/bin/studio.sh.

Configure the location of your Android SDK by selecting “Configure” -> “Project Defaults” -> “Project Structure” -> “Android SDK Location”. Enter the directory where you extracted the Android SDK Tools.

I find it useful to create a launcher icon. This way I can launch Android Studio by clicking Alt-F1 and entering the first few characters of the app name. To create a launcher icon from the Welcome splash screen select “Configure” -> “Create Desktop Entry”.

References

• https://wiki.debian.org/Java/Sun
• http://developer.android.com/sdk/installing/studio.html

Feed2me monitors your favourite RSS/Atom feeds and sends you an email when new entries are posted. It runs on Google App Engine.

My Hardware

Video: Advanced Micro Devices [AMD] nee ATI RV710 [Radeon HD 4350]
Sound: Advanced Micro Devices [AMD] nee ATI RV710/730 HDMI Audio [Radeon HD 4000 series]
Monitor: Dell SP2208WFP (with built in webcam)
WebCam: OmniVision Technologies, Inc. Monitor Webcam

Video and sound devices can be determined using lspci. The Webcam is a USB device so it can be determined using lsusb.

Video

For regular use the opensource Radeon display driver is considered adequate. If you’re a gamer or just want to squeeze every bit of performance out of your card then you might consider the AMD proprietary display driver. For my needs the opensource driver is fine.

On the first boot GNOME 3 failed to load. The installer recognized my video card and installed the correct package, xserver-xorg-video-radeon, but as documented on the wiki my card requires proprietary firmware. This firmware is available in non-free. The steps to install it are,

1. Open up Synaptic Package Manager and select Settings, Repositories from the menu
2. Tick the checkbox next to Non-DFSG-compatible Software (non-free)
3. Click Close and then the Reload toolbar button
4. Search for and install firmware-linux-nonfree
5. Reboot

GNOME 3 should start successfully after the reboot.

Audio

When I didn’t hear any sound I thought I was going to be in for a world of pain with sound drivers, ALSA and PulseAudio. Luckily all I needed to do was select the right output device. My system has two sound devices, an onboard Intel device and the sound device on the AMD video card. My monitor is connected by HDMI. I have a soundbar on my monitor drawing it’s signal from the HDMI cable. All I needed to do was,

1. Open up the System Settings
2. Click the Sound icon
3. In the Output tab select the HDMI audio device, i.e. RV710/730 HDMI Audio [Radeon HD 4000 series] Digital Stereo (HDMI)

Update (Aug-2014): HDMI audio is disabled by default in the kernel/audio driver. To enable it,

1. Edit /etc/default/grub and append “radeon.audio=1” to the GRUB_CMDLINE_LINUX_DEFAULT variable.
2. Open a terminal execute sudo update-grub
3. Reboot

Source: No sound on HDMI with Radeon driver

Webcam

My monitor has a built in Webcam. It’s an OmniVision Technologies, Inc. Monitor Webcam. I managed to get it working but it’s a bit hit and miss. When then uvcvideo kernel module is loaded it should create the device /dev/video0. Sometimes it does, sometimes it doesn’t. To get it to work is a matter of removing the module and reloading. After two or three attempts it usually works. Not very satisfying but I don’t use it very often so I’m not that bothered.

$ sudo modprobe -r uvcvideo     # unload the module
$ sudo modprobe uvcvideo        # load the module

Once the module loads successfully and creates the /dev/video0 device you can use the Cheese application to test it.

Microphone

The mic didn’t appear to be working initially but as with audio out all I needed to do was select the right device and turn up the volume. I use the mic on my monitor (attached to the webcam).

1. Open up the System Settings
2. Click the Sound icon
3. Select the Input tab
4. Select the Monitor Webcam Analog Stero device
5. Turn the volume up to 100%

After getting the hardware working these are the applications I installed.

Flash Player

The Flash Player package is avaiable from contrib.

1. Open up Synaptic Package Manager and select Settings, Repositories from the menu
2. Tick the checkbox next to DFSG-compatible Software with Non-Free Dependencies (contrib)
3. Click Close and then the Reload toolbar button
4. Search for and install flashplugin-nonfree
5. Restart your browser if you

Reference: FlashPlayer on the Debian Wiki

Java Browser Plugin

Simply install the package icedtea-6-plugin.

gThumb

gThumb is a photo manager. I prefer it over the default, Shotwell, because it feels quicker and more lightweight. I already organise my photos in a year, month, day folder structure so I have no need for anything fancy.

While gThumb itself is very easy to install (the gthumb package) it was failing to properly list all the photos and videos in a directory containing one or more videos. The problem was it couldn’t find the image it uses to surround videos (to give the reel effect). The fix is simple,

$ sudo ln -s /usr/share/gthumb/ui/filmholes.png /usr/share/filmholes.png

Reference: Redhat Bug 842183 – Gthumb breaks file list on movies

Android Tools

The Android Tools are used when developing Android applications. After installing the Android SDK I found adb wouldn’t work. When I ran it it gave a “file not found”. The problem is down to missing shared libraries. Here are the steps I took to fix it,

$ sudo dpkg --add-architecture i386
$ sudo aptitude update
$ sudo aptitude install libstdc++6:i386 libgcc1:i386 zlib1g:i386 libncurses5:i386

Reference: Installing Android SDK tools on Debian Wheezy

IFTTT is a web service that allows you to create little “programs”. These programs, or recipes as they’re called, carry out an action following some trigger event. Each recipe has the same structure; “if this happens then do that”. The “this” piece is called the trigger. The “that” piece is called the action.

Example triggers,

• I tweet a message
• I update my Facebook status
• I upload a photo to Instagram
• A new entry is posted to a blog

Example actions,

• Send me an SMS
• Upload a photo to Facebook
• Add an event to my Google Calendar
• Send me an email

See About IFTTT for a more comprehensive description.

Actions and triggers are organized into channels, e.g. Facebook, Dropbox, Email, Feed (i.e. blog feed), etc.

Using the “New feed item” trigger in the Feed channel and the “Send me an email” action in the Email channel I put together a recipe for each blog I wanted to subscribe to. Here’s what my XKCD recipe looks like,

The email subject has the blog name and entry title. For the body I chose to include a link to the entry rather than the full text. Here’s what an email from IFTTT looks like,

I’ve subscribed to number of blogs this way. What I really like is getting email notifications pushed to me rather than having to use a blog aggregation tool. I use email as a sort of TODO list. This way I can treat new blog posts as something I need to do. Once I have the email I can come back to it whenever I want.

As good as IFTTT is there are some rough edges for my particular use case. I wouldn’t say they’re problems though because IFTTT is meant to be a general service.

1. If I want to subscribe to a new blog it’s a bit of a nuisance having to create a whole new recipe from scratch. It would be great if I could create a new recipe based on another one. That way all I’d need to change is the recipe’s name, the blog feed URL and the mail subject (which includes the blog title). The positive side of this is that it stops me subscribing to blogs willy-nilly. I prefer a few quality subscriptions.
2. I’d like to be able to export my recipes. Why? I think it’s the programmer in me. Now I have all these recipes I feel like they should be under version control.
3. An IFTTT API would be great. If there was an API to create a recipe I could create a browser extension to automatically create a “Subscribe to RSS” recipe while on a blog.

So that’s my experience. It works for me.

I think it was all this building stuff that led me into a career in programming. Weather you’re using your mind or your hands it’s the process of making something out of nothing I really love.

I miss using my hands and so after hearing recently about Lego’s new EV3 platform I pulled out my old Mindstorms Robotics Invention System 2.0. This is pretty old set but it’s still perfectly useabable. It comes with two motors, two touch sensors, a light sensor, an RCX (the computer) and more bits of Lego than you can shake a stick at.

I run Ubuntu so the Windows based program that came in the kit would have to stay there (besides, the cover says it only supports Windows 98/ME so what are the chances it’ll work on Windows 7).

What I needed was to do was get the infrared tower used to transmit programs to the RCX working. I wasn’t all that hopeful but after plugging it in I was more than surprised to see a new device appear, /dev/usb/legousbtower0. Apparently the Lego USB tower kernel module has been part of base Linux for some time now. How cool is that.

Now I needed a programming language. A quick google and I came across Not Quite C. It has a C-like syntax but given the restrictions of the RCX it’s pretty straight forward. What’s even better is it’s available from the Ubuntu repositories.

$ sudo apt-get install nqc

The next step was the trickiest, getting nqc to recognize the device. The first step was to give my user access to the /dev/usb/legousbtower0 device. This can be done with the command sudo chmod 666 /dev/usb/legousbtower0. For a more permanent solution create the file /etc/udev/rules.d/90-legotower.rules with the following contents,

ATTRS{idVendor}=="0694",ATTRS{idProduct}=="0001",MODE="0666",GROUP="<group>"

You can use any group you’re a member of in place of <group>. You can find a list of the groups you’re a member of using ‘id -a‘. I used the group “adrian”. On a lot of systems there’ll be a group with the same name as your used id. This is fine so long as you’re the only one who’ll need access to the device. Otherwise you’ll need to find a common group or perhaps create a new one. By the way, the vendor and product ids in the udev rules file came from running lsusb.

The first time I tried to transfer a program to the RCX I got an error saying there was no firmware on the device. Apparently it only lasts as long as the batteries are powered. The nqc command can be used to install the firmware so all I needed to do was locate the firmware file. A few sites talk about the file /firm/firm0309.lgo on the system CD. My CD has no such file. I knew the firmware had to be on there somewhere though. Eventually I found it in the file data2.cab. This is an InstallShield archive. To extract the file install unshield and then expand the CAB file.

$ sudo apt-get install unshield
$ unshield -d /tmp/lego x <path to CD>/RIS2/data1.cab

The firmware file will be located at /tmp/lego/Script/Firmware/firm0328.lgo.

To install the firmware run,

$ nqc -Susb -firmware /tmp/lego/Script/Firmware/firm0328.lgo

Needless to say the USB tower has to be connected and pointing towards your powered on RCX. The transfer takes a few minutes to complete.

Finally, here’s how to transfer a simple program (source file prog1.nqc) to the RCX and have it run automatically,

$ nqc -Susb -d prog1.nqc -run

Now that everything is up and running I can get on with building something.

Links

• Not Quite C – Wikipedia
• Not Quite C
• Programming Lego Robots using NQC
• MINDSTORMS SDK2.5 An archive of RCX tools no longer available on the Lego site. Not relevant on Ubuntu but may be of use to someone.

For security reasons Javascript running in a browser is not allowed to make requests to domains other than the one from where it came from. This is referred to as the Same Origin Policy. CORS is specification allowing browsers and application servers work out an agreement whereby these types of requests are allowed.

Swift 1.7.5 introduced CORS support. This means a Javascript application running in a browser and hosted outside of a Swift cluster can still query that cluster’s API. I expect to see lots of Swift based applications over the coming months thanks to this great new feature.

The ETag

Every object written to a Swift cluster has an ETag associated with it. The value of this ETag is the MD5 digest of the file’s contents. What makes this useful is that you can use it to make a conditional request for the object using the If-Match, If-None-Match headers.

For example, lets say your Swift cluster contains the object “movie1.mp4”. You already have a version of the file locally but you’re not sure if it’s exactly the same. You don’t want to have to download it unnecessarily because it would take too long and/or might incur bandwidth charges. What you can do is invoke a conditional download request, i.e.

$ md5sum movie1.mp4
d41d8cd98f00b204e9800998ecf8427e  movie1.mp4

$ curl -i -H 'X-Auth-Token: xxx' -H 'If-None-Match: d41d8cd98f00b204e9800998ecf8427e' http://swift.17od.com/movies/movie1.mp4
HTTP/1.1 304 Not Modified
...

The 304 response code tells us that our local version is the same as the remote version. If it wasn’t the same it would be downloaded.

Object Versioning

With object versioning, each PUT request to an object will result in the existing object being archived to a special “versions” container.

Versioning is controlled at the container level by setting the header “X-Versions-Location” to the name of the container where you want to archive object versions.

For more information see Object Versioning in the developer documentation.

Renaming or Moving an Object

Renaming or moving objects isn’t supported in the classic sense. However the same result can be achieved by downloading the existing object and re-uploading it to the new container and/or with a new name. Of course if the object is large this can be a time consuming operation. Luckily Swift supports server side copies. What this means is this object is copied from it’s source to destination all within the confines of the Swift cluster.

Either PUT or COPY can be used to perform a server side copy. Neither has an advantage over the other.

For example, given the container and object “photos/sunset.jpg”, here’s how to move it to “holiday-pics/sunset_glow.jpg”.

$ curl -X PUT -H 'X-Auth-Token: xxx' -H 'X-Copy-From: /photos/sunset.jpg' http://swift.17od.com/holiday-pics/sunset_glow.jpg

See Copy Object for more details.

Expiring Objects

Objects can be given an expiry time. When that time is reached Swift will automatically delete the object. Object’s are given an expiry time by setting either the X-Delete-At or X-Delete-After headers.

The value given to X-Delete-At is a Unix Epoch timestamp. There are many ways of converting a time to an epoch integer. For example on UNIX you can to,

$ date +%s
1355945201

The epoch converter website allows you to convert any time to an epoch and vice-versa.

X-Delete-After is a convenience header allowing you to give the number of seconds from now when you want the object deleted. Swift will use this value to calculate an epoch time this number of seconds into the future and set an X-Delete-At header on the object.

# delete the object on Sat, 19 Dec 2015 19:18:52 GMT
$ curl -X POST -H 'X-Auth-Token: xxx' -H 'X-Delete-On: 1450552732' http://swift.17od.com/holiday-pics/sunset_glow.jpg

# delete the object 24 hours from now, 24*60*60
$ curl -X POST -H 'X-Auth-Token: xxx' -H 'X-Delete-After: 86400' http://swift.17od.com/holiday-pics/sunset_glow.jpg

See Expiring Object Support in the developer documentation for more information.

Segmented Objects

Swift has an object size limit of 5Gb. Larger objects must be split up on the client side and the segments uploaded individually. A manifest object is then used to create a logical object built up from the segments.

It’s not just large objects that can be segmented. Any sized object can be broken up. Having said that I can’t think of a good reason why you’d want to do this?

On UNIX the split command can be used to split an object up into segments. There’s two ways of using split, either give it the number of segments or the size of the segments you want to split the object up into.

# split an object up into 5 segments
$ split -n 5 at_the_beach.mp4 at_the_beach.mp4-

# split an object up where each segment is at most 5GB
$ split -b 5G at_the_beach.mp4 at_the_beach.mp4-

In each case the result will be a number of files called at_the_beach.mp4-xx where xx is an alphabetically ordered sequence of characters, e.g. aa, ab, ac and so on. This ordering is important because when Swift rebuilds the object it sorts the segments by name before concatenating them. Each of these segments should be uploaded into the same container.

The manifest object is an object with no content. Instead it has the header X-Object-Manifest. The value of this header is the container name and common prefix of the segments making up the object. Assuming the segments were uploaded into a container called ‘holiday-pics’ and the prefix was ‘at_the_beach.mp4-‘ the header value would be ‘holiday-pics/at_the_beach.mp4-‘.

$ curl -X PUT -H 'X-Auth-Token: xxx' -H 'X-Object-Manifest: holiday-pics/at_the_beach.mp4-' http://swift.17od.com/holiday-pics/at_the_beach.mp4

A GET for the manifest object will return the reassembled source object. Swift will stream each segment in sequence so from the client side it will appear as one continuous object.

See Large Object Support in the developer documentation for more information.

Metadata

Accounts, Containers and Objects can all have custom metadata headers associated with them. These headers are simple name/value pairs. Custom headers are distinguished from system headers with a prefix, X-Account-Meta, X-Container-Meta or X-Object-Meta.

Headers can be set on an existing account, container or object using the POST method. Alternatively the headers can be set when the container or object is being created using PUT.

# Set a header on an existing object
$ curl -X POST -H 'X-Auth-Token: xxx' -H 'X-Object-Meta-Location: Dublin' http://swift.17od.com/holiday-pics/at_the_beach.mp4

# Set a header on a container when creating it
$ curl -X PUT -H 'X-Auth-Token: xxx' -H 'X-Container-Meta-Year: 2012' http://swift.17od.com/holiday-pics/

Metadata can be retrieved using the HEAD method.

$ curl -i -X HEAD -H 'X-Auth-Token: xxx' http://swift.17od.com/holiday-pics/
HTTP/1.1 204 No Content
Content-Length: 0
X-Container-Object-Count: 3
Accept-Ranges: bytes
X-Timestamp: 1355861803.81992
X-Container-Bytes-Used: 2
X-Container-Meta-Year: 2012
Content-Type: text/plain; charset=utf-8
Date: Wed, 19 Dec 2012 20:07:03 GMT

Permissions

Permissions in Swift are controlled at the container level. Only users classified as administrators can create containers. By default regular users can’t create or access containers.

Users can be given read or write permissions for all the objects in a container using the X-Container-Read and X-Container-Write headers.

# Give bob and alice read access to the holiday-pics container
$ curl -X POST -H 'X-Auth-Token: xxx' -H 'X-Container-Read: bob, alice' http://swift.17od.com/holiday-pics/

# Give bob write access to the holiday-pics container
$ curl -X POST -H 'X-Auth-Token: xxx' -H 'X-Container-Write: bob' http://swift.17od.com/holiday-pics/

For more information see the ACLs section of the developer docs.

Pseudo-Hierarchical Directories

While containers can be compared with regular filesystem directories, it’s not possible to nest them. A container with thousands of objects can become extremely difficult to navigate and manage. With that in mind Swift supports a feature called pseudo-hierarchical directories. These are directory structures derived from the names of objects themselves. For example, lets say we have a container with the following six objects,

photos/2010/001.jpg
photos/2011/001.jpg
photos/2011/002.jpg
photos/2012/001.jpg
photos/2012/002.jpg
photos/2012/003.jpg

By using a delimiter character Swift can be asked to list the objects as if they were in a directory structure similar to,

/photos/
|-- 2010
|   `-- 001.jpg
|-- 2011
|   |-- 001.jpg
|   `-- 002.jpg
`-- 2012
    |-- 001.jpg
    |-- 002.jpg
    `-- 003.jpg

For example,

$ curl -X GET -H 'X-Auth-Token: xxx' http://swift.17od.com/adrian/?delimiter=/
photos/

$ curl -X GET -H 'X-Auth-Token: xxx' "http://swift.17od.com/adrian/?delimiter=/&prefix=photos/"
photos/2010/
photos/2011/
photos/2012/

$ curl -X GET -H 'X-Auth-Token: xxx' "http://swift.17od.com/adrian/?delimiter=/&prefix=photos/2012/"
photos/2012/001.jpg
photos/2012/002.jpg
photos/2012/003.jpg

For more information see Pseudo-Hierarchical Folders/Directories in the Developer Guide.

Swift All in One

Like anything, the best way to learn more about Swift it to play around with it. Luckily that’s relatively easy thanks to Swift All in One, a set of instructions describing how to setup a fully functional Swift cluster on a single machine (ideally a VM).

Take the activity lifecycle for example. Under normal circumstances activities are only destroyed when their finish() method is called or the system needs to free up some memory. From a testing perspective this isn’t very deterministic so testing what happens when activities are destroyed is either forgotten about or not given the attention it deserves.

Luckily there are tools to help in these scenarios. The Dev Tools App is particularly useful. It’s automatically installed with every virtual device created using the Android Virtual Device Manager in Eclipse.

Testing Activity Destruction

To test what happens to your app when activities are destroyed tick the Immediately destroy activities checkbox under Development Settings in Dev Tools. This instructs Android to immediately destroy activities that are no longer visible. Under normal conditions you might never get to this test scenario if you don’t explicitly call finish() on your activities. It’s amazing the subtle bugs this can throw up.

Testing Process Termination

Another very useful test is to see what happens when your application’s process is terminated. When I stated out with UPM I received quite a number of crash reports that initially baffled me. The stack traces suggested situations that I just couldn’t recreate. It wasn’t until I started thinking about the impact of process termination that I found the root of the problems.

When a terminated application is restarted Android will try and return the user to whatever activity was previously in the foreground. This can cause problems if the activity assumes too much about the application’s state, e.g. what static variables are initialized. Restarting the application means there’s a new process so the static state of the previous instance is gone. Here’s how I test this scenario.

First I start the application in an emulator as normal. Once I’ve made it to the activity under test I hit the Home button to send the application into the background. Now I need to kill the application’s process. There’s a few ways to do this. My favourite is to use adb. adb is located in the Android SDK platform-tools directory. Running

$ adb shell

starts a shell on the emulator. From there I use

$ ps

to find the application’s process and

$ kill -9 <pid>

to kill the process. Once I’ve killed the process I go back to the emulator and hit the application’s icon to restart it. Android will try to resume the activity that was previously in the foreground. Ideally there should be no problems but if there are they’ll usually present themselves as a NullPointerException with a “Force Close” dialog for the user.

Android can be a difficult, confusing platform to work with. This is particularly true if you’re coming from a regular desktop application environment. There’s a paradigm shift in thinking required to fully appreciate the transient nature of the Android environment. Thankfully, once you’ve made that leap things do get easier. The most useful resources I’ve found are the Android Developer’s Guide and Stackoverflow.

The PRTB publish a list of all registered rental properties in Ireland on their website. The data is split up into 47 separate spreadsheets (xls), one for each county and Dublin postcode. There’s a lot of interesting information in there but it’s very difficult to get at it when it’s in so many files. For this reason I’ve loaded all the information into a database.

Before looking at the data it’s worth bearing a few things in mind,

• the data is effective 1-Apr-2011
• there are a lot of properties included in “County Dublin” that are really part of an area code. this has the effect of bumping up the number of properties in “County Dublin”
• some properties appear to have their floorspace given in square meters while others have it in square feet. this makes it difficult to do a consistent analysis of the full dataset
• there seems to be a few properties with duplicate entries

I’ve included some high level numbers below. If you’d like to have a look at the data yourself I’ve published it in searchable form here. The code for this application is available on my GitHub page.

Total number of properties: 221,315

Number of properties per county:

Number of properties per Dublin area code: