a.blog.about.code

Productivity Engineering

2017-05-26T00:00:00+00:00

On June 24th I’m giving a talk titled Developer Productivity Engineering (DPE) at GORUCO. DPE is a framework that applies the principles of Site Reliability Engineering, developed at Google, to improving productivity through automation.

While putting together my intro for this talk I realized that there are a few terms in the orbit of “productivity engineering” and it can often be confusing to know what’s what.

You may have seen any one of these terms in a job description recently:

DevTools Engineer
Platform Engineer
Release Engineer

Companies all tend to define these roles a little differently. I’m not going to try to clearly demarcate each title, but I think I’ve got a pretty good working definition and some distinctions that might help.

Platform engineers build systems on which other engineers build applications. Some canonical project examples might include implementing an event or queueing framework; or a scaling and maintaining a container orchestration cluster.

DevTools engineers build tools that help other engineers do their work. Some canonical project examples might include CLI scripts or IDE extensions; or a continuous integration and build service.

Release engineers build systems and create processes that enable engineers to ship code. Some canonical project examples might include automating a mobile release pipeline; or building a service to enable blue-green deployments.

Some organizations do not distinguish between these roles, they will often have a single Platform team. Other organizations are so large that they will have multiple siloed teams for each of these roles.

So where does productivity engineering fit in?

Productivity engineers build systems, create processes, and facilitate development efforts that enable other engineers to be more productive.

In a smaller company a productivity engineering team might own all of the example projects I listed above. In a larger organization the owned projects list may be smaller, but the team’s contributions as facilitators and contributors to other teams will likely be much broader.

Productivity engineering is less focused on a single part of the development pipeline, and is instead focused on the bottlenecks in an organization. If the most pressing productivity slow-downs happen to be in a particular platform service that is not scaling, then productivity engineers have a role to play there.

At least thus far, this is how I see productivity engineering fitting into an engineering organization. The DPE framework is really a way of identifying, measuring, and prioritizing the work that will make everyone more productive.

Deploy Env Vs Rails Env

2017-04-07T00:00:00+00:00

Reading through Config: Behavior versus Credentials I was struck that, at Strava, we ran into this very same problem in our early Rails days. It didn’t take us long to realize:

RAILS_ENV is meant to configure the behavior of the Rails framework - not your application.

The clearest example of this distinction, as Richard Schneeman points out, is credentials. You do not want to tie credentials to a Rails environment. Credentials, regardless of how you manage them, should be configured based on the deployment environment.

On the other side of this spectrum is ActiveSupport autoloading. Autoloading is a framework responsibility. You should tie autoloading behavior to the Rails framework environment - not to your deployment environment.

In practice, what does this mean? You effectively should specify two environment variables when running your application:

    RAILS_ENV=production DEPLOY_ENV=production ./run

    RAILS_ENV=production DEPLOY_ENV=staging ./run

In reality the DEPLOY_ENV might be a set of environment variables or a pointer to a configuration store. The core concept is that DEPLOY_ENV is a different dimension of configuration than RAILS_ENV.

How Strava Merges Code

2017-03-28T00:00:00+00:00

I wrote a post for Strava’s Engineering Blog about how we merge code:

Butler Merge Queue - How Strava Merges Code

For close to a year, a large portion of our repositories on GitHub have been using a fast-forward-only master branch that is guaranteed to be green (all CI tests have passed) and that is guaranteed to have only Pull Request (reviewed) code in it. Developers fire-and-forget a Slack slash command that squashes their commits, runs all tests, merges their Pull Requests, and cleans up their Git branch. This is the story of how we got there.

Overwriting Console Output Using Curses in Ruby

2013-04-01T00:00:00+00:00

Καλό μήνα! A while back I wrote about how to overwrite console output in Ruby. Originally the project I used this technique for only required overwriting a single line of output. That project has now grown and I need to be able to maintain a mini-dashboard on the console.

Fortunately, Ruby rocks and has a standard library wrapper for Curses. I would encourage you to read through the documentation. I’ve written a simple script to show-off the most basic use-case. In this example I initialize ten workers. Each worker is responsible for keeping track of the work she has completed, @percent, and then reporting that progress on her own line, @index, of the console.

#!/usr/bin/env ruby

require 'curses'

Curses.noecho
Curses.init_screen

class Worker
  def initialize(index)
    @index = index
    @percent = 0
  end

  def run
    (1..10).each do
      work
      report
      sleep(rand())
    end
  end

  def to_s
    "Worker ##{'%2d' % @index} is #{'%3d' % @percent}% complete"
  end

  private

  def work
    @percent += 10
  end

  def report
    Curses.setpos(@index, 0)
    Curses.addstr(to_s)
    Curses.refresh
  end
end

workers = (1..10).map{ |index| Worker.new(index) }

at_exit do
  workers.each{ |worker| puts worker }
end

workers.map{ |worker| Thread.new{ worker.run } }.each(&:join)

Curses.close_screen

Because Curses clears the screen once it is complete, the at_exit block ensures that the last known state of the workers is echoed to the screen one last time.

Another command I found useful was the Curses.clear command. This function clears the entire screen before rewriting output. This is particularly helpful when you want to overwrite a long line of text with a shorter one.

I hope this was helpful! Have fun making some cool console applications. If you build something worth sharing, please leave a comment that links to your code.

Context Driven Refactoring

2012-12-16T00:00:00+00:00

I got excited about this refactoring website called refactr.it. Users post refactoring questions and vote on the best answers. The website is kind of a simplified version of the Programmers Stack Exchange. But as I explored the ruby section, which is definitely still in it’s infancy, I was a little disappointed.

Here is an example of a question and solution that was titled “Simplifying Nested If Statements”.

Question Code

@problems =
    if @tags.blank?
        unless @language.nil?
            Problem.where(language: @language).desc(@sort).page(@page)
        else
            Problem.desc(@sort).page(@page)
        end
    else
        unless @language.nil?
            Problem.in(tags: @tags).where(language: @language).desc(@sort).page(@page)
        else
            Problem.in(tags: @tags).desc(@sort).page(@page)
        end
    end

Solution Code

@problems = Problem
@problems = @problems.in(tags: @tags) unless @tags.blank?
@problems = @problems.where(language: @language) unless @language.nil?
@problems = @problems.desc(@sort).page(@page)

The solution is definitely a start. In fact it “simplifies the nested if statement” perfectly. And I would be a much happier programmer if I happened upon the solution code than I would be if I came across the question code. My disappointment is that the refactoring lacks context. It’s just an example of a refactoring formula:

# Nested

if A
  if B
    Z.X(A).Y(B)
  else
    Z.X(A)
  end
else
  if B
    Z.Y(B)
  else
    Z
  end
end

# Not Nested

chain = Z
chain = chain.X(A) if A
chain = chain.Y(B) if B

The solution is “correct”, and it’s a valuable technique, but it misses the mark. Refactoring often falls short because people blindly apply a selection of these transformations and then call it a day. Refactoring isn’t just about rewriting code. Let’s take a look at a couple of questions that might help us devise a better strategy for approaching this problem.

Question One: Why should I refactor this code?

Refactoring is fun, but if the code is a mess, and you don’t need to touch it, leave it alone. As Sandi Metz explains, if you can’t see the mess, it doesn’t exist. Refactoring should always happen for a reason. In this case there are multiple potential reasons we might need to refactor. But leaving that reasoning out of the discussion can lead to a misguided solution.

Do I need to add a new search parameter? Then I’m probably in need of a solution that reduces complexity.

Am I replicating this code somewhere else? Then I’m probably in need of a solution that DRYs up my existing implementation.

Step One: Establish a context for the refactoring.

Let’s start with, “I need to add a new search parameter.” This helps us establish that the code is too complex, and that we need to address this issue because we need to change the code.

Question Two: How am I going to test this code?

Imagine, first without using a database, how you would go about testing the question code, or even the solution code for that matter. You would definitely have to provide stubs for #in, #where, #sort and #page. You would have to test conditionals for @language, @tags, and your new search parameter. And all those tests have to combined with the rest of the controller tests for the search action.

Even if you allowed yourself use of the database, you would still have to construct objects for each of the test scenarios and verify that the conditionals were working properly.

That’s a lot of stuff (otherwise know as responsibilities). The “always write tests” mantra isn’t just meant to increase code confidence, it helps illuminate design flaws and unwanted complexity.

Step Two: Read/Write Tests.

Assuming the tests don’t already exist, write them. Even though they are brutal to write after the fact, you need to have test coverage before you start moving code around.

Question Three: What did the tests teach us?

Probably that we have a “Fat Controller”. You’ve probably heard of “Fat Models, Skinny Controllers”, but this heuristic does not really mean Fat ActiveRecord::Base classes and Skinny ApplicationController classes. It means that objects responsible for connecting business logic and views, should do only that. It’s the “Single Responsibility Principle” in Rails terms.

The reason we should strive for single responsibility objects is that they are easier to extend and test. In this case we’ve seen how difficult it is to test the code (in both the question and the solution) which indicates that the proposed solution should probably not be our first refactoring step.

Step Three: Refactor Your Code

Because the tests indicated an unnecessary complexity, we know that we should probably introduce a new object this is responsible for handling some of the logic, without involving the controller.

@problems = ProblemSearch.new(@language, @tags, @sort, @page).find

So what goes into ProblemSearch#find? Probably the original solution, but at this point it really doesn’t matter because it’s completely isolated. The original non-nested code is definitely simpler and easier to extend than the nested code, but that’s the transformation part of refactoring, not the reason we are refactoring in the first place.

You should always simplify your programming constructs so that they are more intention revealing, but the meat of this refactoring is really in the domain modelling and the tests. The original code had too many responsibilities, it was hard to test, and it was hard to extend.

When we needed to add a new search parameter and tried writing tests, we realized how complex that was going to be and identified the need for a refactoring. Allowing that context to guide us, resulted in an intention revealing, testable and extensible solution.

Actually Always Refactoring

2012-09-26T00:00:00+00:00

I apologize, it’s been a while since my last blog post. After being sick, doing a lot of climbing, and some traveling, I attended GoGaRuCo 2012. It was my first Ruby conference and it really opened my eyes to the active and friendly community of Ruby. It gave me the boost I needed to sit down and write another blog post.

While on my hiatus, some comments trickled in on old blog posts. I do want to go back and write updated versions of a few of my older articles. But today I want to talk about Refactoring. I watched Therapeutic Refactoring by Katrina Owen the other day and it got me thinking. While I was at GoGaRuCo I made a point of talking to people, and I noticed the sly smile on their faces when I told them that, “I love refactoring.” They loved it too. So why don’t we do more of it? If it’s therapeutic, if it’s exciting and makes our code better, why aren’t we always refactoring?

The reason, I think, is that along with that feeling of making our code awesome, and coding purely to write code - no feature requests, there is a very real feeling of impending doom. What if I pull on a thread that unravels the system? What if I find out that refactoring the mailer actually means I need to refactor the entire user object?

We’ve all experienced that elated feeling after watching a great presentation about refactoring. We head back to our keyboards, determined to make our code better. But in reality the systems we encounter are never as simple as the examples we’ve just seen. The same tricks don’t apply. Our objects aren’t easy to isolate, they aren’t easy to test and are often so complex that it hurts to even think about them.

So what are we to do? Here are my thoughts.

Change your mindset. The goal is not to make the system perfect and easy to understand. The goal is to make the system better and easier to understand.
It’s okay to stop. Start small - even if it’s not the final solution. Pick one function, a few lines or even one line of code, and work on that first. Think about creating a pull request with the changes you’ve just made. Would your peers be able to reason about your changes?
Don’t be afraid to replace two messes with one mess (of equal and approximate size). It’s an evolving process and each piece of code does not need to be final or pretty, it just needs to make your program better in one way. Take another stab at refactoring duplication or complexity once the first step is done. Too many times have I been sucked down the rat hole of trying to fix all the messy parts at once.
Find someone to share the experience with. Pair refactoring is a great way to tackle the larger messes.
Refactor all the time. When a new feature request comes along, see if you can refactor first, in a separate commit. Then implement the feature. The goal is to make the feature commit simpler.
Demand refactoring. If you don’t have the support to refactor then get that support or find a company/client that values it. Refactoring makes code better. Better code makes better products and better jobs.

I hope that helps. And keep your spirits up. Refactoring isn’t easy to do the first time. Soon enough though, you’ll begin to view it as “your feature request.” And it will be a regular part of your awesome day.

Fixtures and Factories

2012-06-21T00:00:00+00:00

I recently converted a test suite that only used factories to a hybrid that uses a combination of factories and fixtures. While I really like the simplicity and cleanliness of factories, fixtures are simply unbeatable in some cases. I had never really used both together in the same project, so I wanted to outline the pros and cons of each, as well as how they work together.

Loading and Creating Data (Performance)

If you’re loading all of your fixtures at the beginning of every test you’re missing out on one of their major benefits - loading a lot of data only once. Instead, switch to loading all of your fixtures at the beginning of the entire test suite, and then using transactions around each test for cleanup. That way the base fixture data hangs around and each test takes care of cleaning up its own changes. Since generating data can be more expensive than saving it to the database, this simple change can result in dramatic performance gains.

When using factories you have the benefit of creating only the data you want. You may face some degraded performance caused by create operations, since you’ll have to process all those pesky callbacks, but you will benefit from having more isolated tests and data.

Fixtures often make customizations to data more complex. It’s certainly a lot harder to change fixtures because other tests depend on that very same data. Certainly a slew of update statements is going to offset some of that performance gain of having data pre-loaded as well.

I often find that fixture based tests end up doing a lot of pre-assertions to ensure that data is in the correct state. If that’s the case in your tests it’s probably worth asking yourself if you could be using factories instead.

One way to combat the fixture modification data is to combine fixtures and factories. For instance, if you need to parse a large XML file and then create a model from that data, do the parsing ahead of time, store it as a fixture, and then use that model as a base for your factories.

Factories should generally be an MVO, or minimally-viable-object. Data sharing becomes a non-issue because each object adds only what it needs without worrying about conflicting with other objects or tests.

Proximity of Data and Assertions (Ease of Use)

In the fixture approach it’s often harder to find the data related to a test because it’s usually contained in another file. Sometimes this is unavoidable, even with factories, but it’s more often the case that fixture data lives in some other file. If you are making an assertion about data, it’s best to have that data near the assertion.

In the factory approach the relevant data declarations and modifications are all contained in the same test file which makes the test cases easier to read and follow.

Rule of Thumb

No approach is ever 100% better. In fact you’ll probably see the greatest performance gains and have the most fun by using both factories and fixtures. It feels great to use the right tool for the job. If you use transactions to clean your tests then the two approaches work very well together. Fixtures will load data before the entire suite, and then each test will automatically rollback any changes to fixture data, and remove any factories it created.

As a rule of thumb, if data is expensive to create, it’s probably better to use fixtures. If data requires many small customizations, it’s probably better to use factories.

Benchmark Your Bundle

2012-05-03T00:00:00+00:00

Your Rails app takes anywhere from 20 seconds to a minute to boot. Maybe you’ve even tried to optimize your gem loading. But which gems are actually causing you all that trouble? How can you pinpoint the gems to delay, replace or optimize?

Here’s a simple script I wrote to benchmark the loading of each of the gems in your Gemfile. A gist can be found here.

#!/usr/bin/env ruby

require 'benchmark'

REGEXPS = [
  /^no such file to load -- (.+)$/i,
  /^Missing \w+ (?:file\s*)?([^\s]+.rb)$/i,
  /^Missing API definition file in (.+)$/i,
  /^cannot load such file -- (.+)$/i,
]

def pull(dep)
  required_file = nil

  begin
    Array(dep.autorequire || dep.name).each do |file|
      required_file = file
      Kernel.require file
    end
  rescue LoadError => e
    if dep.autorequire.nil? && dep.name.include?('-')
      begin
        namespaced_file = dep.name.gsub('-', '/')
        Kernel.require namespaced_file
      rescue LoadError
        REGEXPS.find { |r| r =~ e.message }
        raise if dep.autorequire || $1.gsub('-', '/') != namespaced_file
      end
    else
      REGEXPS.find { |r| r =~ e.message }
      raise if dep.autorequire || $1 != required_file
    end
  end
end

require 'rails/all'

# If you would prefer gems to incur the cost of autoloading
# Rails frameworks, then comment out this next line.
ActiveSupport::Autoload.eager_autoload!

$VERBOSE = nil

Benchmark.bm do |x|
  Bundler.setup.dependencies.each do |dependency|
    x.report(dependency.name[0..20].ljust(21)) do
      pull(dependency)
    end
  end
end

All of the code in #pull has been plucked right out of the Bundler require function. It’s responsible for loading a gem depending on its Gemfile configuration. It considers custom require paths, and generally converts dashes into slashes if it can’t load the file.

The script then loads rails (require 'rails/all') - something you won’t be able to avoid either way. It also turns warnings off ($VERBOSE = nil) so that the benchmarking output looks a little nicer.

Once that setup is complete, it loops through each of the gem dependencies and requires them. Your output will probably look something like this:

rails                  0.000000   0.000000   0.000000 (  0.000215)
aws-s3                 0.160000   0.030000   0.190000 (  0.182991)
declarative_authoriza  0.850000   0.120000   0.970000 (  0.970152)
fog                    0.530000   0.060000   0.590000 (  0.589278)
mysql2                 0.010000   0.000000   0.010000 (  0.012908)
nokogiri               0.000000   0.000000   0.000000 (  0.000285)
oauth2                 0.140000   0.010000   0.150000 (  0.156330)
rack                   0.000000   0.000000   0.000000 (  0.000232)
rails_autolink         0.000000   0.000000   0.000000 (  0.000854)
rpm_contrib            0.450000   0.030000   0.480000 (  0.482697)
timezone               0.010000   0.000000   0.010000 (  0.007698)
jquery-rails           0.010000   0.000000   0.010000 (  0.008963)
hike                   0.000000   0.000000   0.000000 (  0.001602)
sass-rails             0.370000   0.020000   0.390000 (  0.400510)
coffee-rails           0.120000   0.020000   0.140000 (  0.136651)
uglifier               0.000000   0.000000   0.000000 (  0.002760)
capybara               0.010000   0.010000   0.020000 (  0.006783)
capybara-webkit        0.050000   0.000000   0.050000 (  0.053145)
database_cleaner       0.000000   0.000000   0.000000 (  0.007077)
gherkin                0.070000   0.010000   0.080000 (  0.068361)
jasmine                0.020000   0.000000   0.020000 (  0.025752)
jasmine-headless-webk  0.000000   0.000000   0.000000 (  0.010595)
launchy                0.060000   0.010000   0.070000 (  0.060992)
rspec                  0.350000   0.030000   0.380000 (  0.385097)
rspec-rails            0.000000   0.000000   0.000000 (  0.000903)
sham                   0.030000   0.000000   0.030000 (  0.025905)
spork                  0.000000   0.000000   0.000000 (  0.003553)
timecop                0.040000   0.000000   0.040000 (  0.041003)
guard                  0.000000   0.000000   0.000000 (  0.003354)
rb-fsevent             0.010000   0.000000   0.010000 (  0.004126)
guard-spork            0.000000   0.000000   0.000000 (  0.008679)
test-unit              0.130000   0.010000   0.140000 (  0.132788)
webmock                0.150000   0.020000   0.170000 (  0.167813)
vcr                    0.060000   0.000000   0.060000 (  0.068362)

I’ve condensed the output, but the total load time was 5 seconds. You can verify that by benchmarking Bundler.require in your own application. You’ll also notice that the declarative authorization gem took up approximately 1 second (20% of the total time) to load. Maybe it’s a good candidate for an upgrade, replacement or optimization.

Overwriting Console Output in Ruby

2012-04-16T00:00:00+00:00

I’ve always liked the way some long running console utilities display their status without having to print a new line. For example the little spinner that alternates displaying |, /, and \. Or various internet utilities that display the percentage of the file that has been downloaded.

I recently wrote a utility to kick off parallel RSpec tasks on remote instances and wanted to report the results back to the console after pinging the utility every few seconds. I accomplished this with a cool little trick - the \r.

(1..10).each do |percent|
  print "\r#{percent*10}% complete"
  sleep(0.5)
end

This little scripts begins by printing 10% complete and works up to 100% complete in 10 percent increments. It’s a cool way to notify the user of your Ruby utility that the script is still progressing and that it hasn’t simply stopped dead in its tracks.

One thing to keep in mind is that you are overwriting the previous line, so if your previous line is of a longer length than the next one, you might need to pad it with whitespace to cover up the last message.

Respect the Active Record

2012-03-22T00:00:00+00:00

Here’s some bad Rails code I often come across:

Author.where(public: true).where('score > ?', 50)

The problem isn’t that we’re chaining. Because we’re only handling ActiveRecord::Relation objects, we aren’t increasing the complexity or responsibility of our code by the mere act of chaining. Chaining is a useful programming pattern. However, we are increasing the complexity of our code by making assumptions about the internal workings of Author - we’re breaking encapsulation. Even if you feel that ActiveRecord makes this easy to do, it doesn’t mean you should do it.

We shouldn’t know that authors have internal score and public attributes. Keep in mind that this is different from being able to call Author#score and Author#public, because in the former case, we actually “know” a lot more about Author and we make some unhealthy assumptions.

In particular we make the assumption that these two statements are logically equivalent:

Author.all.select{ |u| u.score == 50 }

Author.where(score: 50).all

In short, this code is asking Author to perform an operation based on an assumed internal state. We are asking - not telling. That’s not good.

Telling Author to give us this data looks very different:

Author.with_public_account.with_score_above(50)

Having this kind of interface for our objects benefits us in a couple of really awesome ways.

Readability

Using intentionally crafted public interfaces for your objects makes them easier to read and more descriptive. This is usually a by-product of the simple fact that you are naming your methods instead of using what is already there. Keep in mind that ORMs were created as a general infrastructure for many different kinds of applications. There is no need to expose that infrastructure to your own application. It makes more sense to write Author.with_public_account than it does to write Author.where(public: true). The intent is clearly communicated.

Searchability

Searching for where.*score in a code base can be tedious and might not result in catching all the possible use cases. Code structured like this can easily be missed:

conditions = {score: 50}
Author.where(conditions)

By contrast, searching for with_score_above is much more likely to yield usable and accurate results. Obviously having a full test suite helps alleviate this kind of a refactoring, but that doesn’t mean you should make it difficult for others to track down your handiwork.

Changeability

Have a more explicit interface makes it easier to change our ORM from ActiveRecord if we ever choose to. It makes it easier to make small attribute level changes, like renaming score. That’s because we now only need to modify the Author object, not every piece of code that filters authors on score. You could imagine a future implementation that looks something like this:

module AuthorRelation
  def initialize(authors)
    @authors = authors
  end

  def with_public_account
    @authors = @authors.select{ |a| a.public? && a.paid? }
    self
  end

  def with_score_above(score)
    @authors = @authors.select{ |a| a.cached_score > score }
    self
  end
end

If you’re thinking - why would I ever want to change something like that? Then you’ve missed the point. This isn’t a pre-optimization, this is a philosophy for creating persistence objects that behave like simple Ruby objects, without exception. It’s a slippery slope to assume that some objects just aren’t going to change and that kind of thinking causes headaches for the rest of your code and for other developers working with code you’ve written.

What if we didn’t change the entire object; what if we simply denormalized and indexed a flag that combined score and public? Our carefully crafted public interface would allow us to make use of this new column, whereas being tied to the where way of doing things would mean having to modify a lot more code.

Testability

It’s easier to begin building an application without being tied to the ActiveRecord way of doing things. This frees us up to write isolated tests and hide ActiveRecord as a true implementation detail, rather than making it a central part of our application. No object or collection of objects should dominate your code. Some objects might be more or less connected, but that doesn’t mean they should be required to test one another. Isolation is a great thing.

Summary

Respect the ActiveRecord. Let it do what it does best, as a piece of infrastructure and as a 3rd party library. Its way of doing things and its method signatures should not infect the rest of your application. It does what it does best when it’s limited to and hidden inside of your ActiveRecord::Base objects.