Robbie Mackay

This week #7

2021-08-20T00:00:00+12:00

A collection of technical, management and other thoughts from this week.

Thinking

I’ve been on-boarding to a new company the last month, and learning a bunch of new codebases. As I’ve started to contribute to code reviews, I was reflecting a bit on code review for knowledge sharing (a heated debate sometimes, I know!). I think t while code review as a teaching tool might be of dubious value, it can be a great learning tool.

In particular, I find that I use code review as a learning tool myself and find it really effective. It points me at parts of a codebase I might never have seen before and in the process of figuring out if a change makes sense, I have to read and understand what’s happening.

For the musicians out there - it feels a bit like sight-reading a new piece of music. It’s not the same as playing and coding within that codebase, but it goes a long way to being able to reason about what’s going on there now and in future.

Reading

I’m diving into single-table design and DynamoDB, so I’ll share what I’ve read so far there:

And up next…

This week #7 was originally published by Robbie Mackay at Robbie Mackay on August 20, 2021.

This week #6

2019-05-17T00:00:00+12:00

A collection of technical, management and other thoughts from this week.

Life happened and I’ve missed a couple weeks but I’m back at writing these.

Thinking

I changed roles in January, moving from a management role to a developer role. I choose an engineering role partly because I wanted to learn and grow my technical skills again, but partly I just wanted to feel like I’d done something at the end of each day.

As a manager, I often finished the day wondering what I’d done or feeling like maybe I didn’t actually do anything useful at all today. I only recently realised thats a common experience for many new managers. It’s harder to see your impacts as a manager, and part of becoming a good manager is figuring out how to judge your impact.

I’ve recently found myself in a similar place again. I’m not managing a team, but we’re in a phase of open-ended exploratory work. Once again, it’s hard to tell if what I did today was useful. Did I read anything that gave me insight into the product, or customers needs? Even if I did, was that enough? Is it actually moving us closer to a profitable business? I’m just … not sure. In abscence of a concrete deliverable ie. a working bit of code, it’s hard to see my impact.

I’ve read a couple of good pieces talking about this, and I’ve started trying out some personal processes. I’ve set high level goals: pick an idea in 2 weeks, build and test a prototype in a month. My daily tasks are partly reactive: follow up X, and partly driven by those goals: sketch a lean canvas for Y idea or answer a question about idea Z. It’s early days but it’s helped a little. I have some idea if I’m seeing progress now.

Reading

Satisfaction and progress in open ended work.

How to Make Sense of Your Impact When You’re No Longer Coding.

Hiring and the market for lemons. An interesting analysis of broken hiring practices. I’m not sure I agree with all of it but it’s worth a read.

How not to disagree. A good piece about how to handle managing a team through changes you don’t agree with.

Fire fixation. I’ve definitely had teams stuck in a place of fighting so many fires we couldn’t get any new work done. It’s a hard place to get out of.

Remote mob programming. A good guide, not something I can use right now but saving it for future reference.

Listening

a16z on healthcare and fintech.

Quick and Cognitive Behavioural Therapy on Changelog.

This week #6 was originally published by Robbie Mackay at Robbie Mackay on May 17, 2019.

This week #5

2019-04-30T00:00:00+12:00

A collection of technical, management and other thoughts from this week.

A bit late and short this week. I tried several times to turn my thoughts into a couple of coherent paragraphs but I’m going to call time and publish this as-is.

Reading

Meritocracy doesn’t exist, and believing it does is bad for you. The title says it all really.

Why software projects take longer than you think. A great write up a possible reason why we’re so bad at estimating software projects: People estimate the median completion time well, but not the mean. It’d love to see this confirmed with more data.

How to Track Your Kids (and Other People’s Kids) With the TicTocTrack Watch. A good break down of the security failures of a watch for tracking your kids. I’m with Germany on this one - ban them - the security risks are too high to consider any device that tracks children.

A way to do CSS for applications. Nice and simple. I might try this on my next project.

Listening

Start With This. A new podcast from the creators of Welcome to Nightvale about writing and the creative process. Each episode has assignments to help push you to actually create something.

This week #5 was originally published by Robbie Mackay at Robbie Mackay on April 30, 2019.

This week #4

2019-04-18T00:00:00+12:00

A collection of technical, management and other thoughts from this week.

I’m on vacation this week and having mixed success disconnecting from work. Mostly I’m still going down rabbit-holes of interesting tech which at least gives me something to write about.

Thinking

I’m slowly poking at a couple of new things. The first is building progressive web apps in Vue, using Vuetify and Cordova. I’m fairly convinced that this is the stack I would use to build a new frontend today, but only if it needs a mobile app or offline support. I’m now stuck needing a real project to experiment on before I can get any further.

The second thing is Phoenix, a web framework built in Elixir. I love the promise of super fast requests, built-in support for web sockets, easy scaling. At first glance, it looks like the early learning curve isn’t too bad, although it’s clear there’s a lot of complexity to uncover eventually. My biggest worry is the deployment story. It’s just not as well supported as something like PHP or Python. You can deploy Elixir to Heroku but lose some of the great features in the process.

Reading

The 4 ‘Attachment Styles,’ and How They Sabotage Your Work-Life Balance. An interesting and possibly useful bit of psychology context.

UX Design For Startups. I’ve only just started this. So far it looks like a nice overview, but I haven’t seen a lot of depth. Not surprising for a free ebook.

Timestamps and time zones in Postgresql. Time is hard.

One Neat Trick To Avoid Rewriting Your Software Product. Yet another take on why you shouldn’t rewrite, and how better alternatives. I particularly appreciated the points about the need to limit a new products runway. It’s too easy to let a new spin-off product kill the old one slowly without actually proving itself.

Listening

I had a long drive by myself recently so I caught up on Welcome To Nightvale and got through a few more episodes of Good Morning Nightvale. If you don’t already know about Nightvale you should really start listening.

This week #4 was originally published by Robbie Mackay at Robbie Mackay on April 18, 2019.

This week #3

2019-04-11T00:00:00+12:00

A collection of technical, management and other thoughts from this week.

Thinking

I haven’t had a clear focus for my reflections this week, but there are some rough themes.

Building new things

What’s my ideal prototyping tech stack? Is it worth learning Sketch, Figma or XD or should I just prototype in code? What should I learn now that will help with product development later?

I have a lot of experience with PHP and Laravel, such that I could very quickly through something together. But I feel like I have to balance that with how much PHP would appeal (or not) when hiring other developers. If I go towards Python, which is having a bit of a resurgence and pair that with Vue or React, that seems more likely to appeal to good developers. But there are also those who would be wary of that stack too.

What should I learn next?

I could double down of areas I’ve already dabbled in: front-end dev, React, Vue, etc. I could focus on new technical areas: machine learning, graph databases, modelling tricky problems. I could dig into something new outside my field: UX design, user research, or product management.

Right now I’m tending towards doing some practice building Progressive Web Apps, and trying to use it as practice for more UX and product thinking. How that all fits together is still very much a work-in-progress.

Reading

Meritocracy Stinks. Criticisms of meritocracy aren’t new but I thought this was a really good dig into it, particularly in a New Zealand context.

The Feedback Fallacy. This is definitely worth reading. It challenged my assumption that critical feedback is necessary and good for growth.

Another of our collective theories is that feedback contains useful information, and that this information is the magic ingredient that will accelerate someone’s learning. Again, the research points in the opposite direction. Learning is less a function of adding something that isn’t there than it is of recognizing, reinforcing, and refining what already is.

Field Guide to UX Research For Startups. A little lightweight, but a good quick start to answering: what kind of research am I doing right now?

How To Get Startup Ideas. From 2012 but relevant for me right now.

Listening

Should This Exist?. I’m just getting started on this now but it looks promising.

This week #3 was originally published by Robbie Mackay at Robbie Mackay on April 11, 2019.

This week #2

2019-04-05T00:00:00+13:00

A random collection of thoughts from this week.

Thinking

I’ve been pondering learning skills from other disciplines. It’s potentially a killer feature. My hunch is that knowing unexpected things outside my field gives me an edge, and helps me remain a high performer.

Think of the project manager who codes and has a little more context around technical decisions, the developer who can give critical feedback on design, or help do user research.

I could be a function of the smaller organisations I’ve been in but it’s often been useful to work outside my role, and I’ve seen other succeed doing the same. It might be supporting your coworkers when they’re swamped, or picking up jobs no-one on the team actually has experience doing. It can earn respect as a high performer, and potentially make the whole team faster too. In the best case it can build a sense of cohesion and a team where everyone helps each other out.

The pitfall - which I have seen too - is that people with a little bit of knowledge start to second guess their coworkers. Imagine the developer with a little bit of product knowledge repeatedly second guessing the product direction. Judgement and humility are essential.

Reading

Some from further back:

A minimal viable product needs to actually be viable.

How not to screw up hiring.

Research questions are not interview questions.

Thanks to @tmcw for inspiration to just do something simple each week

This week #2 was originally published by Robbie Mackay at Robbie Mackay on April 05, 2019.

This week #1

2019-03-28T00:00:00+13:00

A random collection of technical, management and other thoughts from this week.

Thinking

I’ve been reflecting a little on how much time I’ve spent on operations work in previous roles - wrangling servers, configuring everything from DNS to job queues.

I think in many projects I’ve worked on the companies involved would have got much better value from managed / off the shelf solutions. In the smaller ones it might be that they could have just used a managed Drupal or Wordpress host. In others they needed a full continuous development pipeline but might have been able to get 90% of that from Heroku.

It’s easy to look at Heroku (and similar providers) on paper and label it ‘too expensive’, but when I think of the hundreds of hours spent setting up a deployment pipeline and automating server configurations it would probably have balanced out. The worst factor in roll-your-own solutions is that they potentially slow down the development, so the cost compounds over time.

It does seem like there might be a gap between a Heroku-style solution which is expensive but basically does everything for you, and a DIY deployment pipeline shipping to EC2 or a Kube cluster. As a company grows, and Heroku gets expensive but the dev team is still < 20 people, it’s not clear to me where those teams should land.

Reading

Your work is not your worth. Trying to square what I know (I am not my work) with how I feel.

This twitter thread from Ryan Hoover. Getting paid as an remote contractor outside the US feels messier than necessary, and having a bunch of remote employees in many countries seems nearly impossible for small start ups right now.

Calculating the mean of a list of numbers. Turns out this is much harder than you’d think.

Born for it. A write up of how the image of a socially-awkward, white, male programmer came about, including some good take-aways about how the women and people of colour are disadvantaged by idea of intellectual ability as a fixed gift vs something you can improve over time.

Listening

A lot of Keeno and other dnb. I’ll try to pay more attention for next week!

This week #1 was originally published by Robbie Mackay at Robbie Mackay on March 28, 2019.

Psychological safety

2017-05-19T01:05:29+12:00

I spent a little time reading and thinking about safety recently: what it means to feel ‘safe’ in a team or an organisation, what impact that has on your and your team.

I haven’t really formed this into coherent thoughts yet but I’ll leave some links as a placeholder.

The first I really read about this was looking at Google’s work on building successful teams. One their five keys to a successful team is Psycological Safety, in fact they list as the most important.

In a team with high psychological safety, teammates feel safe to take risks around their team members. They feel confident that no one on the team will embarrass or punish anyone else for admitting a mistake, asking a question, or offering a new idea.

And then someone posted a couple of podcasts about safety in the Rands leadership slack:

Both of these are well worth a listen.

I’m still reflecting a bit on how this applies to me, to my team and to how I manage them. I’ll post a link if I have future thoughts or updates.

Psychological safety was originally published by Robbie Mackay at Robbie Mackay on May 19, 2017.

A little less coffee

2017-05-19T00:57:51+12:00

I’ve been experimenting over the last week with reducing my coffee habit, and seeing how that effects my mood, productivity, sleep etc.

I started at about 3 cups / day. I realised a long time ago that much more that 3 cups a day is probably my limit. Much more and I don’t sleep properly and sometimes get jittery.

What I’ve noticed so far:

I slept much better. I took a few days with no coffee at all but then I got much better nights sleep.
I don’t know what to do with my break time, or how to start the day, without my coffee making ritual.
Coffee out, is not the same as coffee at home. Part of how I reduced my intake was by letting my coffee supply at home run out. After a week, and a few coffee’s in cafes - I found what I missed most was the quiet moment with a cup of coffee at home.
I’ve built rituals around coffee. ie. writing and coffee
I think more clearly in the morningwithout the coffee. At least some of the time
Tiredness and coffee are a bad combination. Where tiredness by itself I can work through. Also if I haven’t had coffee, I can usually nap.
I think - though I’m not sure yet - that I’m less grumpy and irritable without coffee.

All in all, I’d like to keep my coffee in take lower. I’m going to try and avoid the coffee absolutely every day habit, and have days I just have none.

Update

It’s been a couple more weeks and I’m mostly back to a daily coffee habit but only 1 a day. I don’t have any beans in the house right now, which means that its not the first thing I grab in the morning. I’m finding that makes it a more conscious choice… but also leaves me a bit unsettled sometimes when I want something hot to drink and I don’t have time to go out.

I’m still sleeping better, probably partially due to reduced caffeine intake, and partially because I’m heading to bed earlier.

A little less coffee was originally published by Robbie Mackay at Robbie Mackay on May 19, 2017.

Why I run

2017-04-24T08:49:12+12:00

This is far from complete but here’s a few thoughts on why I run.

I run for clarity

When I run my mind runs too. Its not empty but theres only one thing to do: run. So as I run it sort through the day, through the problems and often the obvious solutions finally dawn on me, or sometimes just clarity about the actual problem.

I spent on day stressed out feeling like things just weren’t working and I wouldn’t possibly get everything done. Within 2 mins of feet moving I realised the obvious things I missed: I have a team, why am I even doing some of these things myself when I have a team?!

I run for calm

Running is meditative. To start with my mind chases random thoughts and ideas. After a while I can start to see past those. In a sitting meditation I focus on my breathing and watch it come and go. When I run its similar: I focus on my breathing and sometimes just the pain. The burn in my legs, and effort to get oxygen demands my attention.

I’ve learnt over years that when something happens that leaves me angry and fuming: it might be what happened or it might just be the day. Sometimes I can channel the anger into fixing something that really should have been dealt with a long time ago, but often trying to do that might upset someone and create a bigger mess. I never know. I run to calm down. Sometimes when I’m done there’s no anger left. Other times there’s still a spark but usually with more idea why I’m annoyed and what I can do about it.

I run to go further

If I head out to hike there are plenty of spots that would take me all day to get there. I have to carry food, and more gear. But if I run I cover a 3 hour hike in maybe 1.5 hrs. I live near Waitakere Ranges, running means I can disappear into the hills for a few hours and get to beautiful spots I’d otherwise miss.

I run to explore

There’s a real joy to exploring. Just heading out of a trail with only a vague idea which path I’ll take. Sure I take a map and do some prep, but I have 3 or 4 options and I can just pick whatever looks most interesting when I get there.

I run to see how far I can go

I think this actually motivates me more than I notice. I love to just see how far I can get, how much it will hurt, where I’ll end up. This is the temptation to do an ultra, to do a triathlon, or just to enter that next race. Can I do it? Can I do it faster this time?

Why I run was originally published by Robbie Mackay at Robbie Mackay on April 24, 2017.

Adding optional parameters in Lumen routes

2017-04-24T05:52:15+12:00

I’ve been working on migrating Ushahidi from Kohana to Lumen recently. Lumen uses a FastRoute instead of Laravels core router. This is faster than the core route, but also more limited.

Initially reading the documentation ~~it appears not to support optional URL parameters~~ Optional URL parameters are now included in the routing docs. But digging into the Fast Route docs it turns out there is some support:

parts of the route enclosed in [...] are considered optional, so that /foo[bar] will match both /foo and /foobar. Optional parts are only supported in a trailing position, not in the middle of a route.

So you can do something like

$app->get('/config[/{id}]', 'SomeController');

It also turns out that Lumen doesn’t strip trailing slashes when parsing URLs so the optional parameters were a useful workaround. I have a few routes like

$app->get('/config[/]', 'ConfigController@index');

Adding optional parameters in Lumen routes was originally published by Robbie Mackay at Robbie Mackay on April 24, 2017.

Handling login events with Sentry and AngularJS

2017-03-20T00:00:00+13:00

I added Sentry event tracking to RollCall this morning. As part of that I added some simple handling of login and logout events to set the current user in the Sentry meta data.

First we simple add Raven and - if we have a DSN set - configure it:

import Raven from 'raven-js';

// Load raven if configured
if (RAVEN_DSN) {
  Raven
    .config(RAVEN_DSN, {
      release: GIT_COMMIT,
      environment: ENVIRONMENT
    })
    .addPlugin(require('raven-js/plugins/angular'), angular)
    .install();
}

There’s some minor sugar in there to add both the environment and current git commit. Both of these are set during the build process.

To wire up log in/out events I added this little service

class RavenService {
  constructor(user, $rootScope, Raven) {
    "ngInject";
    this.user = user;
    this.$rootScope = $rootScope;
    this.Raven = Raven;
  }

  init() {
    this.$rootScope.$on('auth.login', this.handleLogin.bind(this));
    this.$rootScope.$on('auth.logout', this.handleLogout.bind(this));

    if (this.user && this.user.isLoggedIn) {
      this.handleLogin();
    }
  }

  handleLogin() {
    this.Raven.setUserContext({
      username: this.user.name,
      email: this.user.email,
      id: this.user.id
    });
  }

  handleLogout() {
    this.Raven.setUserContext({});
  }

}

export default RavenService;

And tied it all together in a simple module:

import Raven from 'raven-js';

// Load raven if configured
if (RAVEN_DSN) {
  Raven
    .config(RAVEN_DSN, {
      release: GIT_COMMIT,
      environment: ENVIRONMENT
    })
    .addPlugin(require('raven-js/plugins/angular'), angular)
    .install();
}

let ravenModule = angular.module('app.raven', [
  'ngRaven',
  userModule
])

.factory('Raven', () => {
  return Raven;
})

.service({
  ravenService
})

.run(['ravenService', (ravenService) => {
  if (RAVEN_DSN) {
    ravenService.init();
  }
}])

.name;

export default ravenModule;

A few things to note:

I’m calling ravenService.init() in a run() block because that way we know that user should be ready and we can check it starting state.
We already fire auth.login and auth.logout events elsewhere in our code so I know those just work.

Handling login events with Sentry and AngularJS was originally published by Robbie Mackay at Robbie Mackay on March 20, 2017.

Auto deploying middleman via Travis-CI

2016-07-16T13:02:00+12:00

I spun up a Today I Learned site based on Today Icelab Learned. But I wanted a simple set up where any push would auto deploy, and I didn’t have to run some other service to do that. So a bit of quick Travis hacking with the help of this article and Travis is now building the site and deploying to the gh-pages branch.

The main additions are were adding NodeJS (as well as Ruby) and running bin/setup during install

Adding NodeJS to our ruby build

It turns out Travis already include NodeJS and nvm. So getting the version we want is super simple:

before_install:
  - nvm install node

Here’s the final .travis.yml:

rvm:
- 2.3.0
branches:
  only:
  - master
cache: bundler
before_install:
  - nvm install node
install:
  - bin/setup
before_script:
- git config --global user.name '${GIT_NAME}'
- git config --global user.email '${GIT_EMAIL}'
- git remote set-url origin https://rjmackay:${GH_TOKEN}@github.com/rjmackay/til.git
script:
- bundle exec middleman deploy
env:
  global:
  - GIT_NAME: Your name
  - GIT_EMAIL: you@email.com
  - secure: # Insert encrypted GH_TOKEN here

Auto deploying middleman via Travis-CI was originally published by Robbie Mackay at Robbie Mackay on July 16, 2016.

Remote Working @ Gather

2014-07-14 19:49:07 +1200T00:00:00-00:00

The 3rd Gather conference happened this Saturday. Despite having been to the last two and many of the previous Barcamp Auckland’s I’ve never run a session myself.. so I finally did: “Remote working and distributed workplaces. Lets talk about remote working. Good, bad and otherwise. Bring your problems and suggestions”

I’ve been working remote for 2 1/2 years now, to start with from a coworking space, then later from home. But always very remote, the only Ushahidi staff member in NZ. I’ve had some great moments, and some low ones, and I think (hope) I’ve got better at remote work over time ..

The general idea for this session was to share some of the common issues and wisdom around working remote. And hopefully get people to help each other out.

All in all it went pretty well.. we were in the first slot of the day, not much of a hallway track so plenty of people showed. I talked for about 2 mins then left people to discuss. We spent a bit too long on the tools discussion.. and I didn’t pull that back in. But there was some good advice offered.

Thanks to Lance Wiggs for taking notes during the session. I grabbed some photos so here is a rough transcribed version..

Whiteboard notes: Secrets of Remote Working

Get you timezones right
- Do a proper shift handover
- Use video for depth, context
- Design teams with time zone crossover
- Know your handover times
Have explicit conversations/processes on how you wokr
Have a visit schedule to ensure constant interchange
- Get everyone together periodically
  - Annual dev day
  - Second people to other offices - short and long times
  - Local, regional global gatherings
  - Functional gatherings
  - Training groups that are diverse
Be aware & work constantly on culture
- Make sure people know others cultures
Pre-book flights so you know who is coming when
Have a system for passive comms
Break into manageable teams, groups
Create processes that work for you

Tools

Check timezones: time.is
Collab
Team chat
- Slack
- Hipchat
- IRC
- Sqwiggle
Audio
- Skype
- Telephone
Video
- Sqwiggle (maybe more team chat)
- Zoom.us
- Google Hangouts
Meeting / measurements

What else?

I knew we wouldn’t manage to cover everything. I realised as soon as the session finished I’d missed something I wanted to know about: how do people plan remote?

Every year Ushahidi brings our team together for a week. Largely to touch base with each other, but also to do high level planning. This works alright but I’ve often wondered if doing a big sprint of planning in person prevents us figuring out how to plan remote. Things always change during the year, without notice, so some form of continuous planning while remote is necessary. How does anyone else do this?

Coming back from breakdown

Chatting at the pub after Gather another good question came from wombleton: has anyone experienced broken, failing remote work and resolved it? what happened? how did they bring it back? Breakdown is easy to achieve: miscommunication, not including remote workers, silo’d info. There’s a lot of opportunity for making things difficult in remote work.. but I haven’t seen much about how to get things back to workability again if you hit that point.

I’m pretty happy to have started discussion. Looking forward to doing it again at Gather next year, or elsewhere

Remote Working @ Gather was originally published by Robbie Mackay at Robbie Mackay on July 14, 2014.

Remote working

2014-04-21T09:00:17+12:00

My alarm goes off early.. I jump out of bed, a bit groggy and head for a quick 2 min shower. Grab breakfast - a bowl of cereal. Make coffee. Head straight out to my office. Open up the laptop, sign in and wait..

Check a few emails, start to catch up on skype chats from last night.. keep waiting.

Ping someone else: Is this call on?

5 mins longer.. still no call.

Check the calendar: Yup it’s right there in mine, good good.

Check the team calendar.. OH it was 5 hours ago..

This is remote working..

Sometimes this is just how it goes. There over 20 of us, spread over 7 countries, we hit at least 7 timezones. Most of us work from home or a coworking space. We can’t all talk daily. Not even in chat. Some days you just don’t overlap. We can’t run video on our team calls: the bandwidth just doesn’t hold up to that.

Our teams aren’t always grouped together: Crowdmap has 1 person in the US and 1 in South Korea. Ushahidi 3.0 has developers in Atlanta, Greensboro, Nairobi, Cape Town and Auckland.

Sometimes I think we’re mad. But then I couldn’t work on what I do if we didn’t work this way.

I get to live in New Zealand, close to family, often working at home with my 2 year old son. I get to work on open source software every day, With amazing users mapping corruption, environmental pollution, tracking violence and much more.

Maybe we’d be more effective if we were all in the same place, or even in just one country, but we also wouldn’t be the same company. Our odd mix of staff from around the world, all passionate and dedicated to the work we’re doing.

Remote working was originally published by Robbie Mackay at Robbie Mackay on April 21, 2014.

Automating Behat and Mink tests with Travis CI

2013-05-03T00:00:00+12:00

I was excited to see Github now shows build status branches. Even more so when I submitted code to an upstream library and was surprised by the tests results appearing in my pull request.

So I jumped in to get it running for Ushahidi 3.0

We’ve been running some tests through Jenkins already, but I’ve largely been doing manual runs locally. Travis-CI integrates with Github automatically, so I figured that was the easiest path to integration.

First impression: its really really easy to get started, and the docs were great.

Getting started

Log in with Github, turn on builds for the repo
Add .travis.yml file, configure the language, and PHP version to use
Push to github
Wait for builds to run

Installing Behat, Mink and Guzzle.

We use composer to install behat, mink and guzzle. I had just been doing this locally anyway so that turned out to be easy. Just add:

before_script:
  - composer install

Running PHPUnit and Behat

By default Travis-CI would just run PHPUnit. We use both PHPUnit and Behat so we have to run both. That’s as simple 2 extra lines in the config

script:
  - ./bin/behat --config application/tests/behat.yml
  - phpunit -c application/tests/phpunit.xml

Further config

The next steps get little trickier.

Bootstrapping the DB wasn’t too bad. Add mysql to ‘services’ and add a couples of ‘before_script’ steps:

services:
  - mysql
before_script:
  # db setup
  - mysql -e 'create database lamu_test;'
  - mv application/config/database.travis application/config/database.php
  - ./minion --task=migrations:run --up

API tests

We’re running API tests by hitting a web server with Guzzle. This means I need a web server running in my Travis-CI build. Theres a few docs on doing this for other projects but not so much PHP, and the examples I found were ugly at best.

In the end I’m running the built in PHP dev server, and restricting builds to PHP 5.4.

Final script

A few other bits of bootstrapping and I had working builds:

language: php
php:
  - "5.4"
services:
  - mysql
before_script:
  # install behat, mink etc
  - composer install --no-interaction --prefer-source # Have to prefer source or hit github rate limit
  - git submodule update --init --recursive           # not really needed but makes sure submodules are there
  - mkdir application/cache application/logs          # create cache and log dirs
  - chmod 777 application/cache application/logs      # set permission
  # db setup
  - mysql -e 'create database lamu_test;'
  - mv application/config/database.travis application/config/database.php
  - mv application/config/auth.travis application/config/auth.php
  - ./minion --task=migrations:run --up
  # webserver setup
  - php -S localhost:8000 httpdocs/index.php &
  - sleep 3
script:
  - ./bin/behat --config application/tests/behat.yml  # run behat
  - phpunit -c application/tests/phpunit.xml          # run php unit

Not quite done..

I quickly realised that my code actually had some issues:

PHP 5.4

I’ve been building on PHP 5.3, so immediatey hit a bunch of ‘Array to string conversion’ bugs in PHP 5.4. The main culprit for this was a bug in Kohana 3.3, so I’ve ported a rough fix from the develop branch.

Case sensitivity

I develop on OS X using a Vagrant box that mounts project code over NFS. The side effect of this is that my filesystem isn’t really case sensitive. Kohana uses PSR-0 autoloading, so the filename and class names need to match. The Travis builds run on a standard Ubuntu box: I suddenly hit errors about missing classes, etc.

Fixing this was a bit slow and painful but actually a win in the long run. The first linux user to fire up the code base would have hit these issues immediately!

Done

In the end getting this set up took longer than expected, but I’m pretty pleased with the result. I’ve merged these changes into all my work-in-progress branches and fixed a few bugs as a result already. I’ll call that success.

Automating Behat and Mink tests with Travis CI was originally published by Robbie Mackay at Robbie Mackay on May 03, 2013.

API wrangling

2013-04-11T00:00:00+12:00

Following on from my Building 3.0 blog post, here’s another update on Ushahidi 3.0.

We’re currently still pushing on finishing the 3.0 API, or at least the first cut at it. The original due date for this was March 30, but thats slipped to mid April. Conversations have quieted down a little as we get to work on getting things done. Thats good, but means we need to keep working hard to keep the community up to date on our progress.

API design primer

Designing an API seemed simple.. but once you get into the weeds and start building things there are a lot of questions to answer. I’ve done a chunk of reading as I tried to figure out 1. what a proper RESTful approach would look like, 2. what’s normal practice, ie. where do developers often cut corners? why? is this good or bad?

A few valuable resources:

Web API Design from apigee
Whitehouse API standards

Progress

The last couple of weeks have been dominated by the thousands of tiny decisions made at each step of building the API. I’ve tried to keep the wiki updated with the general patterns of our API. This gives a few guidelines

What methods to use, and how to accomodate more complex queries (ie. search)
What HTTP response codes to use
What to cover in functional tests
How to expose relations and links beteen resources

We’ve got the basics in now: forms, with their attributes and group, and posts. This gives us enough to start experimenting, but there are still a lot of loose ends. The next things we’re looking at are: sets, tags, users - along with their roles and permissions - and extensions to posts - adding revisions, translations and updates. I’ve built out most of the translations and revisions support already, but it still needs polish and documentation.

Authentication

This is probably the next major decision. How do we handle API authentication? do we use OAuth? 1.0 or 2.0? Where do we handle actual user login/registration?… the entire app is going to be built on the API.

At the moment I’m leaning towards OAuth 2.0, primarily because its what Swiftriver uses, and consistency between products is important. However OAuth 2.0 has some issues:

the editor for the spec withdrew saying OAuth 2 had failed
there are a few ways to mess up an implementation security wise
and one implementation isn’t guaranteed to be interoperable with another

That said the many almost-oauth APIs out there probably have more, and similar problems: rolling our own is not an option.

OAuth 2 is probably still going to get a lot of use, and there are good resources appearing on how to do it right, and a couple of good oauth2 clients.

One recommendation that stuck out was “If your API can require HTTPS, use OAuth 2.0 with bearer tokens. Otherwise, use OAuth 1.0a”. I need to dig into the details of this, but given deployers won’t always use SSL this is worth considering.

Stumbling blocks

In my last post I mentioned adding support for other databases, particularly PostgreSQL/PostGIS. I’ve already hit the first snag with this: minion migrations doesn’t handle other database engines at all. There are a few forks on github that try to solve this, but its mixed in with some other major changes. This isn’t a show stopper, but there’s some unpicking to be done and for now we’re concentrating on building a working API.

If anyone has working PostGIS knowledge and whats to have a go at getting this working, grab the code, leave a note and hopefully send a pull request. I’m happy to walk you through the existing code and answer questions.

API wrangling was originally published by Robbie Mackay at Robbie Mackay on April 11, 2013.

Improving Ushahidi Bug Triage

2013-04-09T00:00:00+12:00

I’ve been thinking around how we manage our bug and features backlogs. The current Ushahidi 2.x backlogs have 144 features, and 47 Open bugs (even more last week before our bug fix sprint). At this size these start to get a little unweildy and feel overwhelming.

My previous experience with Kanban and other agile methods has demonstrated that limiting work-in-progress, can make life for developers and customers much easier. It reduces the number of things you need to think about, and the amount of time spent on work that hasn’t yet shipped.

I recently read joelonsoftware talking about software inventory and I’ve been thinking about how this could be applied to Ushahidi. We’re building open source software, and delivering a long lived product, not just a one off project, so not everything transfers perfectly.

For instance “Do not allow more than two weeks (in fix time) of bugs to get into the bug database” just doesn’t work for us. Our users can add bugs directly themselves and we have no desire to change that, and our pace is slower as our resources are stretched.

However we still have a large amount of what could be described as ‘software inventory’ thats built or managed by the core team. Bug queues, feature queues, partially complete code branches, completed work waiting for QA, completed work waiting for release. We need all of these, but when they grow too large they become significantly less useful. Looking at a list of 144 features.. its much harder to know which ones are important, similarly with bugs: we still have issues in the system from 6 months ago.

For bug tracking: what matters is are the bugs in github valid, fixable, and current.

Some issues which have been in the system for many months, with little activity fail on both being not current, and probably no longer valid. Other bugs are not fixable, not because its not technically possible, but because they’re low priorities (can be worked around) and our resources are limited - we could leave these bugs in the system, but a more honest response might be to close these and mark them as ‘wont fix’.

Feature backlogs are a little different: if our users are happy to invest time in detailing a feature they need thats great. If that leads to large detailed backlog thats ok. What we still need to avoid is expending developer time scoping a feature that we don’t have time to build. Ushahidi 2.x is now a feature freeze so this is less of an issue, however I have no doubt we’re going to have to look at this again with Ushahidi 3.0.

I’ve also been guilty of expanding the feature backlog myself by dropping every idea I have into a 2 sec feature ticket. This is useful initially, but over time its just filling the backlog with things that aren’t really useful, and maybe just unneeded by our users.

Those watching the github issues will have noticed a recent flurry of closing tickets.. this isn’t a major change in behaviour. But it reflects a recent bug fix sprint and a push to close invalid bugs as mentioned above.

You closed my bug, but its not fixed. What should I do?

If you think we’ve closed an issue thats still important for you, just let us know: comment on the ticket (and maybe reopen it). The worst thing you could do here would be to ignore it, and decide we don’t care. We simply don’t always know if you’re waiting patiently on an issues, or is you’ve forgotten about it and moved on. Give us an update!

Improving Ushahidi Bug Triage was originally published by Robbie Mackay at Robbie Mackay on April 09, 2013.

Building custom blocks in Ushahidi

2012-12-24T00:00:00+13:00

I’ve been meaning to write this up for a while. Adding custom blocks to Ushahidi is a fairly common request but generally requires a developer, or at least some basic coding skills. However for some basic use cases it should be possible to add a block with a bit of cut and paste scripting.

To add a block you’ll need to create a plugin. This is actually less complicated than it sounds. In your Ushahidi install find the directory ‘plugins’ and create a new folder. The structure is like this:

/plugins
  /my-custom-blocks
    readme.txt
    /hooks
      register_category_blocks.php
    /views
      category_block.php

In more complicated plugins we might also have libraries, controllers, helpers or anything else that exists in the usual Kohana module structure. But for this simple case lets still to just hooks and views.

readme.txt

The readme file is required for Ushahidi to pick up the plugin at all. The first section of this file is parsed for plugin name, description and other details used in the Ushahidi admin.

=== About ===
name: Category Block
website: http://www.ushahidi.com
description: Adds custom category blocks
version: 0.1
requires: 2.5
tested up to: 2.5
author: Ushahidi Team
author website: http://www.ushahidi.com

Add any other docs here

hooks/register_category_blocks.php

<?php defined('SYSPATH') or die('No direct script access.');

// Start wildlife category block
class category_wildlife_block { // CHANGE THIS FOR OTHER BLOCKS

	public function __construct()
	{
		// Array of block params
		$block = array(
			"classname" => "category_wildlife_block", // Must match class name above
			"name" => "Wildlife Reports",
			"description" => "List the 10 latest reports in the wildlife category"
		);
		// register block with core, this makes it available to users 
		blocks::register($block);
	}

	public function block()
	{
		// Load the reports block view
		$content = new View('blocks/category_wildlife_block'); // CHANGE THIS IF YOU WANT A DIFFERENT VIEW
		
		// ID of the category we're looking for
		$category_id = 7; // CHANGE THIS

		// Get Reports
		$content->incidents = ORM::factory('incident')
			->with('location')
			->join('incident_category', 'incident.id', 'incident_category.incident_id')
			->where('incident_active', '1')
			->where('category_id', $category_id)
			->limit('10')
			->orderby('incident_date', 'desc')
			->find_all();

		echo $content;
	}
}

new category_wildlife_block;

views/category_block.php

<?php blocks::open("reports");?>
<?php blocks::title(Kohana::lang('ui_main.reports_listed'));?>
<table class="table-list">
	<thead>
		<tr>
			<th scope="col" class="title"><?php echo Kohana::lang('ui_main.title'); ?></th>
			<th scope="col" class="location"><?php echo Kohana::lang('ui_main.location'); ?></th>
			<th scope="col" class="date"><?php echo Kohana::lang('ui_main.date'); ?></th>
		</tr>
	</thead>
	<tbody>
		<?php
		if ($incidents->count() == 0)
		{
			?>
			<tr><td colspan="3"><?php echo Kohana::lang('ui_main.no_reports'); ?></td></tr>
			<?php
		}
		foreach ($incidents as $incident)
		{
			$incident_id = $incident->id;
			$incident_title = text::limit_chars(strip_tags($incident->incident_title), 40, '...', True);
			$incident_date = $incident->incident_date;
			$incident_date = date('M j Y', strtotime($incident->incident_date));
			$incident_location = $incident->location->location_name;
		?>
		<tr>
			<td><a href="<?php echo url::site() . 'reports/view/' . $incident_id; ?>"> <?php echo html::specialchars($incident_title) ?></a></td>
			<td><?php echo html::specialchars($incident_location) ?></td>
			<td><?php echo $incident_date; ?></td>
		</tr>
		<?php
		}
		?>
	</tbody>
</table>
<a class="more" href="<?php echo url::site() . 'reports/' ?>"><?php echo Kohana::lang('ui_main.view_more'); ?></a>
<div style="clear:both;"></div>
<?php blocks::close();?>

So how does this work?

The hook ‘register_category_blocks.php’ is included by Ushahidi (and Kohana). This hook registers the ‘Wildlife Reports’ block with Ushahidi core by calling blocks::register().

<?php
// Array of block params
$block = array(
  "classname" => "category_wildlife_block", // Must match class name above
  "name" => "Wildlife Reports",
  "description" => "List the 10 latest reports in the wildlife category"
);
// register block with core, this makes it available to users
blocks::register($block);
?>

The array passed to blocks::register() tells Ushahidi the plugin name, description, and where to find the plugin content.

When the plugin is rendered, Ushahidi will call category_wildlife_blocks::block().

<?php
  // Load the reports block view
  $content = new View('blocks/category_wildlife_block'); // CHANGE THIS IF YOU WANT A DIFFERENT VIEW

  // ID of the category we're looking for
  $category_id = 7; // CHANGE THIS

  // Get Reports
  $content->incidents = ORM::factory('incident')
    ->with('location')
    ->join('incident_category', 'incident.id', 'incident_category.incident_id')
    ->where('incident_active', '1')
    ->where('category_id', $category_id)
    ->limit('10')
    ->orderby('incident_date', 'desc')
    ->find_all();
  echo $content;
?>

This loads the view blocks/category_wildlife_block.php and passes it a list of incidents: the last 10 incidents with category ID = 7.

Tweaking it

Change the category ID - just change line 24 of register_category_blocks.php. You might also want to change the name and class for the block too.
Change the html in category_wildlife_block.php - if you want to tweak the output slightly you can easily change it here
Filter the incidents by something else - change where() call on line 30 of register_category_blocks.php
Load images or similar - I’ll post an example for this in the new year…

Installing Ubuntu 12.04 on a Macbook Pro 8,2

2012-08-13T00:00:00+12:00

This has been waiting around for a while so I’ll give it a quick clean up and hit publish. Unfortunately because its been waiting a while I might have missed some details but I wouldn’t remember now.

Thankfully the notes on help.ubuntu.com have vastly improved since I first tried this, so check those out first.

Weirdest part of the whole thing

Use both USB and CD installer. It took me a while to find this out, but they only way to get the installer to boot properly is to use the same install image on both a USB key and CD at the same time.

What I did:

Resize the OSX partition with disk utility
Create an MSDos partition
Install rEFIT
Boot the installer using both USB and CD
Install ubuntu as usual<ul>

Delete msdos partition and replace with ext4

Install the MBR onto sda5 (ie. the linux ext4 partition)

You can probably skip creating the swap, I had to delete it later, but with 8GB of ram it wasn't an issue.

</ul>

Sync MBR/partition table (using refit)
fails to boot
Boot LiveCD (+USB) again
Delete swap partition (Probably could have skipped this to start with and it would have gone better)
Restart
Sync partition table (using refit)
Restart
Boot linux
works Successful boot, yay!
Add ppa for wireless driver (installing using ethernet)
reboot
wireless works (no extra hacking)

Extra packages to install/configure:

macfanctld Absolutely essential unless you want to cook your CPU.
natural scrolling
other synaptics wrangling - definitely worth it
enable extra drivers (?)

I’ll have to go find more info to document these last bits properly.. however I’m back on OSX so it’ll need to wait for a reboot.

Installing Ubuntu 12.04 on a Macbook Pro 8,2 was originally published by Robbie Mackay at Robbie Mackay on August 13, 2012.

Robbie Mackay

This week #7

Thinking

Reading

This week #6

Thinking

Reading

Listening

This week #5

Reading

Listening

This week #4

Thinking

Reading

Listening

This week #3

Thinking

Building new things

What should I learn next?

Reading

Listening

This week #2

Thinking

Reading

This week #1

Thinking

Reading

Listening

Psychological safety

A little less coffee

Why I run

I run for clarity

I run for calm

I run to go further

I run to explore

I run to see how far I can go

Adding optional parameters in Lumen routes

Handling login events with Sentry and AngularJS

Auto deploying middleman via Travis-CI

Adding NodeJS to our ruby build

Remote Working @ Gather

Whiteboard notes: Secrets of Remote Working

Tools

What else?

Coming back from breakdown

Remote working

This is remote working..

Automating Behat and Mink tests with Travis CI

Getting started

Installing Behat, Mink and Guzzle.

Running PHPUnit and Behat

Further config

API tests

Final script

Not quite done..

PHP 5.4

Case sensitivity

Done

API wrangling

API design primer

Progress

Authentication

Stumbling blocks

Improving Ushahidi Bug Triage

Building custom blocks in Ushahidi

readme.txt

hooks/register_category_blocks.php

views/category_block.php

So how does this work?

Tweaking it

See also

Installing Ubuntu 12.04 on a Macbook Pro 8,2

Weirdest part of the whole thing

What I did:

Extra packages to install/configure: