Here is the talk description:

Massive, long-lived pull requests are often such a headache. In this session we go through a move into a trunk-based development source-control workflow which eliminates the need for these “code avalanche” pull requests.

After short introduction of trunk-based development and the fears people have for implementing it, we will go through concrete examples, lifted from a production system, of code, architecture, and workflows to tackle those fears when working on improving the development and deployment happiness of the team working on the application.

After this talk the attendee are emboldened to start their own trunk-based development journey or excited to apply some of the techniques from the talk to their existing development processes and codebases.

And video here:

And slides here:

vx includes a network graph component for showing graph data which I needed for a project. Unfortunately it doesn’t automatically calculate the positions for the graph nodes, so I decided to use the D3.js force simulation to calculate the X and Y coordinates for the nodes.

React component’s componentDidMount callback sets up d3 forceSimulation which takes care of setting the x and y properties for the passed nodes.

The full code for a simple network graph with automatic layout is below. You can also view how it looks and play around with it in CodeSandbox.

import React from "react";
import ReactDOM from "react-dom";

import { Graph } from "@vx/network";
import {
  forceCenter,
  forceLink,
  forceManyBody,
  forceSimulation
} from "d3-force";

// The node rendered by the graph
class NetworkNode extends React.Component {
  render() {
    return <circle r={10} fill={"#9280FF"} />;
  }
}

class Network extends React.Component {
  constructor(props) {
    super(props);

    const links = props.network.links;
    const nodes = props.network.nodes;

    this.state = {
      data: {
        nodes,
        links
      }
    };
  }

  // Update force if the width or height of the graph changes
  componentDidUpdate(newProps) {
    if (
      newProps.width !== this.props.width ||
      newProps.height !== this.props.height
    ) {
      this.force = this.force
        .force("center", forceCenter(
          newProps.width / 2,
          newProps.height / 2
        ))
        .restart();
    }
  }

  // Setup D3 force
  componentDidMount() {
    this.force = forceSimulation(this.state.data.nodes)
      .force(
        "link",
        forceLink()
          .id(function(d) {
            return d.id;
          })
          .links(this.state.data.links)
      )
      .force("charge", forceManyBody().strength(-500))
      .force(
        "center",
        forceCenter(this.props.width / 2, this.props.height / 2)
      );

    // Force-update the component on each force tick
    this.force.on("tick", () => this.forceUpdate());
  }

  render() {
    if (!this.force) {
      return null;
    }

    return (
      <div style={{ width: "100%", height: "100%" }}>
        <svg width={this.props.width} height={this.props.height}>
          <rect
            width={this.props.width}
            height={this.props.height}
            fill="#f9fcff"
          />
          <Graph graph={this.state.data} nodeComponent={NetworkNode} />
        </svg>
      </div>
    );
  }
}

function App() {
  const nodes = [{ id: 1 }, { id: 2 }, { id: 3 }, { id: 4 }, { id: 5 }];
  const links = [
    { source: 1, target: 2 },
    { source: 1, target: 3 },
    { source: 1, target: 4 },
    { source: 2, target: 4 },
    { source: 3, target: 4 },
    { source: 4, target: 5 }
  ];
  return (
    <div className="App">
      <Network
        width={400}
        height={400}
        network={{
          nodes: nodes,
          links: links
        }}
      />
    </div>
  );
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

Configure Bundler to use HTTPS for GitHub sources:

bundle config --global github.https true

And on Heroku (note the double underscore):

heroku config:set BUNDLE_GITHUB__HTTPS=true

And on Travis (put it to your .travis.yml file):

before_install:
  - "bundle config --global github.https true"

I managed to hit my reading goals and they were also good reading years in quality.

In 2015 I read 82 books (24 of them comics) and in 2016 read 65 books (20 of them comics). In 2016 I also started doing lot more re-reading of previously read books or parts of them.

Books with 5 stars

I’m not a literary critic and don’t know which books are good or bad so I give my stars according to my own feeling on how much I got out of the book and how big of an impact I estimate it having on my life.

Here’s the list:

• Leap First: Creating Work That Matters by Seth Godin
• Moominpappa at Sea by Tove Jansson
• Joustava mieli - Vapaudu stressin, uupumuksen ja masennuksen yliotteesta by Arto Pietikäinen in Finnish
• The Road Less Traveled: A New Psychology of Love, Traditional Values and Spiritual Growth by M. Scott Peck
• Shoe Dog: A Memoir by the Creator of NIKE by Phil Knight
• Siddhartha by Hermann Hesse

You can view all the books I read in my Goodreads profile for 2015 and for 2016.

Check out also previous Year in books posts for 2013 and 2014.

You can follow the books I read on Goodreads. It gets updated when I get reading done. I’m also interested in following people with similar taste.

Bird by Bird: Some Instructions on Writing and Life by Anne Lamott

This is a book about writing and writer’s life. Even though I don’t write that much I could find plenty of good advice. Encouraging and inspiring (even though some people might find the honesty off-putting) read that also reminded me of the importance of the shitty first draft.

The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph by Ryan Holiday

Meditations by Marcus Aurelius

I’ve been interested about Stoicism and read two books related to it. Great advice for the rough times and one-liners to spice up your day. I already bought couple other Stoic books so will read more during 2015.

Foundation and Empire & Second Foundation by Isaac Asimov

On the fiction front I read the second and third installment of the science fiction series Foundation. I enjoy the dry writing style of Asimov. The books are low in action and very dialogue heavy so not for everyone. I love the plots and uncovering the mysteries of psychohistory. I will continue reading the series in 2015.

World War Z by Max Brooks

Another fiction pick is World War Z. It is a fictional collection of stories about lives of people after the Zombie war. The stories encompass various nationalities in different life situations told as dialogues with the interviewer of the book. Stories blend together social, economic, religious, political, and environmental themes and there is a clear critique and commentary on the current state of the world.

After reading the book I returned back to the stories in my head and marvelled on all the view points the book included. Event if you don’t like zombies, I would recommend it. It’s sometimes categorized as horror but I wouldn’t label it as such.

Functional JavaScript: Introducing Functional Programming with Underscore.js by Michal Fogus

I didn’t read that many technical books this year but from the ones I did, this was my favorite. I’ve being dabbling with Clojure and this book linked many of those functional programming ideas to JavaScript.

Waking Up: A Guide to Spirituality Without Religion by Sam Harris

Still the Mind by Alan Watts

On the spirituality front these books were my favorites.

The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution by Walter Isaacson

This is the book I have talked about the most this year, so I guess that makes it my favorite. It tells the stories of people and teams that have invented the ideas and built the technologies we rely on in today’s digital world: programming, microprocessor, personal computer, software, internet and so on. It also observes what are the commonalities of the innovators and tries to lift themes from their stories. While reading the book I also read some of the papers written by the early pioneers. Reading papers written in the 60’s and 70’s you see how so many parts of the technology puzzle took decades to go from idea to wide usage.

I also found it interesting that when the first interactive computers were invented, the first user interface designers were psychologists. The teams building these interactive machines thought of it as a man-machine symbiosis so it only made sense to put engineers, the experts of the machine, and psychologists, the experts of the human, to the same team to work on the machines.

Here is a trailer of the book:

You can view all the books I reading in 2014 on my GoodReads profile. I wrote similar post also a year ago so you can see what I enjoyed in 2013.

My plan of writing notes or mini-reviews on the books I read failed miserably so I’m doing it properly this in 2015. See you in a year!

Here is an example migration class that follows all the guidelines reviewed in this post:

class AddAreaToPlots < ActiveRecord::Migration
  class Plot < ActiveRecord::Base; end

  def up
    add_column :plots, :area, :float

    Plot.find_each do |plot|
      plot.area = plot.width * plot.height
      plot.save!
    end
  end

  def down
    remove_column :plots, :area
  end
end

Redefine ActiveRecord classes

If possible, ActiveRecord models should not be used in migrations. Migrations should be simple and failproof transformations of database schema and data. Using complex model objects that complicate the migration should be avoided.

If you have strong SQL skills, you can use it for data transformations. Well written SQL is often faster than using ActiveRecord. For schema changes I would use the ActiveRecord methods which are readable and give extra features provided by the framework (most notably the migration/rollback support talked later in this post).

If you want to use the familiar ActiveRecord API to do the data transformations, the ActiveRecord classes should be redefined. As shown in the example, the Plot class has been redefined inside the migration class. The redefinition overrides the model class from the application code and makes it a bare ActiveRecord class. This means it has no validations, associations or callbacks. If some of the callbacks should be called in the migration, those can be reimplemented in the redefined class. Same goes for methods in the model class.

Why go through this trouble? When the application changes, the models change with it. It’s possible that the models change in a way that break the migrations. Or the models can be renamed or removed which will also break the migrations.

It might feel weird but it just works, makes your migrations more robust, and protects you from trouble in the future.

Use find_each

When modifying records, Model.find_each should be used to fetch the records in batches. If Model.all is used on a big database table, ActiveRecord will instantiate objects for all rows in the table which will consume all the available memory and the process starts swapping memory like crazy slowing the migration.

By default find_each loads the records in batches of 1000 but that can be configured. Only drawback of find_each is that you can’t use order, offset or limit. This is because find_each orders the records by id and then uses offset and limit to control the loading of the batches. Other than these caveats, there are no drawbacks on using find_each. Filtering the records using where works just fine.

Don’t change the data using the change method

The change method for migration classes was introduced in Rails 3.1 to make the writing of migrations simpler. Using change and methods that support inversing themselves, you can write migrations like this:

class AddAreaToPlots < ActiveRecord::Migration def change add_column :plots, :area, :float end end

When the migration are run by ActiveRecord, it executes the “up” version of the command adding the area column. On rollback it runs the code and executes the “down” version of the command for all the methods that have it. In the case of adding a column, rollback would remove it.

If you modify your data within the change method, ActiveRecord will run that code on both migration and rollback which is probably not what is wanted. Here is an example:

class AddAreaToPlots < ActiveRecord::Migration
  class Plot < ActiveRecord::Base; end

  def change
    add_column :plots, :area, :float

    Plot.find_each do |plot|
      plot.area = plot.width * plot.height
      plot.save!
    end
  end
end

On rollback the migration would first remove the column and then try to populate the column which would of course fail due to the missing column breaking the rollback.

In addition to the following these guideline it doesn’t hurt to understand how the ActiveRecord migrations work. Read the official Active Record Migrations guide to get more information.

Do you have any best practices you follow to make your migrations better? I’d be interested hear about those in the comments.

Addition: Migrations should not contain seed data

Seed data refers to the initial data the database must have for the application to function correctly at all. Rails provides a [mechanism for populating the database with seed data] which should be used. Migrations are meant only to provide the structure of the database, not the contents.

If migrations are used for seed data, there is a chance that a database is created without the seed data. When the incorrect migrations get older and a new database is bootstrapped using rake db:schema:load it uses the db/schema.rb file to get the right structure, but the data from the migrations won’t be created in the database.

Thanks for Christian Hellsten for the addition!

So why should you be interested about test duration?

Test duration is important because if running your tests takes too much time it builds friction and you run the tests less often. When you don’t run your tests regressions get committed to version control which slows down your other team members. Also if you do test-driven development (TDD) the feedback loop provided by the tests gets too slow to be useful.

If you do the test suite duration comparison often enough you can pinpoint tests that slow the suite down disproportionately to the benefit the added test provides. When you know certain test is especially slow you can think of ways to improving the speed of the test or find a way to test the functionality with another, faster way.

Pulling test suite durations from Travis

Fetching Travis data is easy using Ruby and the official Travis Ruby library. These instructions are for the pro version of Travis which is meant for private repositories. You can also use the same instructions for the open source version of Travis but you need to change the commands and library usage to remove the references to pro.

First install the Travis RubyGem:

gem install travis

Then login to Travis using it:

travis login --pro --auto

Here is the Ruby file for fetching the data:

require 'travis/pro/auto_login'

repo_name = "your-organization/your-project"

builds = Travis::Pro.repos.find(repo_name).first.builds
finished_builds = builds.select(&:finished?)
durations = finished_builds.map {|build|
  [build.started_at, build.duration, build.commit.sha]
}
File.open("stats.txt", "w") {|f|
  durations.each do |duration|
    f.puts duration.join("\t")
  end
}

The program fetches all finished builds for a repository, gets the durations and writes those to a file stats.txt separated by tabs.

To make the data a bit more readable for large projects, lets skip some data points to get to less than 100:

require 'travis/pro/auto_login'

repo_name = "your-organization/your-project"

builds = Travis::Pro.repos.find(repo_name).first.builds

finished_builds = builds.select(&:finished?)
every_n_build = (finished_builds.size / 100.to_f).ceil

smaller_dataset = finished_builds.each_slice(every_n_build).map(&:first)
durations = smaller_dataset.map {|build|
  [build.started_at, build.duration, build.commit.sha]
}

File.open("stats.txt", "w") {|f|
  durations.each do |duration|
    f.puts duration.join("\t")
  end
}

Now you can use your favorite charting tool to get something visual.

Here is a screenshot of a data from one of my apps created using Numbers.app. It was created by opening the file in Numbers.app, selecting the data and selecting bar chart from the Chart menu.

You can download the episode and view the shownotes at the Frontend Friday website.

Things like flu, small aches and stiffness in the body are normal in human life. Urban life and stationary jobs strain the body. Usually these smaller things pass on their own and you don’t pay too much attention on them. But sometimes those are a signal of something bigger. A disease that needs treatment or overstress that you should react to.

For me exercise works great on separating the signals that are meaningful and need to be reacted to. Exercise stresses the body in different way than home and work life. I do CrossFit exercise twice a week and have been doing for three years. If for many workouts in succession I don’t feel normal, it is usually a sign that it is not the normal everyday things going on. I notice it from being out of breath easily and muscle pain after workouts being greater and taking longer to pass.

I also follow my workout results (using our own workout journal app WODconnect). My memory for results is horrible so having a place for the results that makes them comparable over time is very valuable. Sometimes not being able to get the results I have been previously been capable of is also a sign of something more severe (sometimes it is just being tired from previous workouts or something else).

Listening to your body is never stupid but it is especially important when exercising.

Overall I read 55 books (of which 10 were comic books). I use GoodReads to track my reading so you can view my full reading list there and even follow my “reviews” if you are really interested.

The books

So Good They Can’t Ignore You: Why Skills Trump Passion in the Quest for Work You Love by Cal Newport

The title sounds rather self-helpish but reading the author’s Study Hacks blog made me interested. Newport has been studying remarkable careers and the book formulates rules based on the that.

Perhaps the most controversial bit is the idea that you should not follow your passion but start learning valuable skills. Newport says that passion is dangerous because having passion is so rare that it being a precursor for a great career is depressing and confusing. After you have advanced skills and knowledge on an area the passion will come automatically. Waiting for the passion to appear is not an effective strategy. Working right trumps finding the right work.

Book offers a models for improving your skills, steering your work choices to maximize control, and on formulating an underlying mission for your career. I had bunch of light bulb moments with this book and would recommend it to anyone pondering where she is going next with her work life.

Buddhism Plain and Simple by Steve Hagen

I’ve been reading some Buddhist material during the past few years and I found this book to be great summary of many Buddhist ideas. It is stripped from any extra layers and presents the ideas in a clear way. I would recommend it to people who are interested in Buddhism on all knowledge levels of the subject.

Surely You’re Joking, Mr. Feynman! by Richard P. Feynman

Great stories from one of my favorite scientist. I’ve watched videos and read stories about Feynman earlier but this was the first book I read about him. I also read the second book What Do You Care What Other People Think? but didn’t enjoy it as much because it was more fragmented collection of random events from his life.

Design Is a Job by Mike Monteiro

Mike Montero has a unique speaking and writing voice which I enjoy. Book is about his learnings from running his design consultancy Mule Design. Book has good points to be applied on my own work, especially on client relationships. I would recommend it to anyone running a business that serves clients with professional services.

Thinking Fast and Slow by Daniel Kahneman

The book is a summary of research done by the author on the area of cognitive biases, his work on prospect theory and on happiness. Underlying idea of most of Kahneman’s research is based on separation of two modes of thinking: faster, intuitive system and slower, logical system. He goes through how the systems work in unison and what kind of behavioral patterns emerge from that.

The book is a bit long to my taste but it is easy to follow and I think knowing about the biases of our own thinking has is beneficial to everyone.

Comics

Fok_It by Joonas Rinta-Kanto (books 1-4)

Only available in Finnish. I enjoyed the whole available catalogue of Fok_It albums. It is just my kind of humour.

Habibi by Craig Thompson

A love story set in Islamic world. The visual design is really great and with over 600 pages you get a lot to see. Anyone who enjoys great illustration and graphical storytelling should grab this one.

Extra pick

DJ-kirja by Matti Nives & Iina Esko

I actually finished reading this on 2014 but I’ll include it anyway. This one is also only available in Finnish. The book includes essays about DJ culture, interviews of DJ’s and a section on club flyers from Finnish point of view. As a music nerd I really enjoyed all the view points and stories from familiar faces.

Next year I’m going to aim for 50 read books and try to start a habit of writing notes or mini-reviews of the books I read. It takes some work but in the long run you get a quite nice library of good notes you can read to remind you on the important points. For example checkout out Derek Sivers’ book notes with short summaries and longer notes.