This last trick is not specific to Pry, but is very useful nonetheless.

Stuck request

I had a problem recently where one particular discussion in my application wouldn't load. It looked like the error was a timeout, and as result Nginx was returning a 500. Since this was a timeout and not an exception, the error did not show in New Relic, which made it difficult to debug.

To the console!

My first check was to verify whether the same request would eventually run in the console, and if it was just really slow and needed optimization:

> app.get "/discussions/problems/1234"
................ 10min later, still nothing ..............

So the request was stuck somewhere, but where? set_trace_func to the rescue!

set_trace_func proc { |event, file, line, id, binding, klass|
    if event == "line" && file[RAILS_ROOT]
      puts "#{file}:#{line}"
    end
}

Run the request again:

> app.get "/foo/bar/baz
/data/APP/current/config/initializers/action_controller.rb:8
/data/APP/current/config/initializers/action_controller.rb:11
/data/APP/current/config/initializers/action_controller.rb:11
[...]
/data/APP/current/config/app/models/comment/concerns/admin.rb:34

And there I had it. The exact line where the request was stuck. It turned out to be a bad string/regex combination, but I'm not sure how else I could have figured it out. set_trace_func definitely was handy.

Logging SQL requests

Also very useful when debugging requests in the console:

# Rails 2
ActiveRecord::Base.connection.instance_variable_set :@logger, Logger.new(STDOUT)

# Rails 3 & 4
ActiveRecord::Base.logger = Logger.new(STDOUT)

Conclusion

So that concludes our short serie on Pry. I hope you enjoyed it :)

That's it for today, cheers!

Last week, we took a look at the basics of using Pry in production as a replacement for the standard IRB console. Today, let's look at a practical scenario and see how Pry can help us debug a misbehaving request.

Why use Pry?

After all, we already have New Relic which gives us a stack trace of every error in our application.

Well, many errors do not result in an exception. Oftentimes the application responds, but with the wrong data. And oftentimes as well, the error only occurs in rare cases dependent on production data: it's hard to reproduce an error locally if you don't even know what makes the error occur in the first place.

There are 2 ways developers deal with this problem:

• Have a full copy of the production environment (with up-to-date production data). While this may be feasible for small datasets, it becomes harder and harder as the data grows. Customers will also often complain about an error that just appeared, making it more likely that the important data does not yet exist in the test environment.

• Connect a local development environment to a remote database. I find this method too dangerous: it's a disaster waiting to happen. You should not trust yourself with a direct connection to the production database on your machine. Ever.

In these situations, I find that Pry offers just enough functionality to do some actual debugging in production, while keeping you relatively safe.

PryNav

I like pry-nav better than pry-debugger or pry-byebug because it works with every version of Ruby, and being pure ruby, doesn't require me to include the debugger gem in production. I can thus safely include pry and pry-nav in my Gemfile, keeping them unloaded, and only require them when I start the Rails console manually with my pryrc. You get the best of both worlds: no impact on running production servers, but the flexibility of a basic debugger when you need one. And it works everywhere.

# Gemfile
gem "pry", :require => false
gem "pry-nav", :require => false

Scenario

One of my apps is a customer support app, and so it deals a lot with discussions, comments, support staff, etc. I had a case recently where a customer could not assign a particular discussion to a particular user. It worked (and had been working) for every other discussion and every other user, so it was fairly obvious that there was something peculiar about the data that triggered a bug. A first exploration of the code involved did not reveal any obvious clue, so this was thus a perfect use case for Pry.

Step 1: Authenticate

Rails provides you with a few methods to interact with your app directly from the console (see this blog post for a good introduction). The first step of every debugging session is often to authenticate as the user with something like that:

[1] pry(main)> app.post "/login",  :user => {:login => 'joe', :password => 'password'}

For real applications however, this is unpractical as passwords are not stored in clear text in the database. We still need to login as our user though... While we can't use a login/password, most apps will have a current_user method or similar, that we can patch.

First, let's edit our current_user method:

[1] pry(main)> edit Authenticated#current_user -p

And overwrite it:

def current_user
  return User.find(1234)
end

From now on, every request will be made as user 1234.

Step 2: Set our breakpoint

We know our entry point: the controller action. So let's start there:

[2] pry(main)> edit DiscussionsController#assign -p

and add a call to Pry:

def assign
  binding.pry
  [...]
end

Now, call our action:

[3] pry(main)> app.post "/discussions/problems/1234/assign", :user_id => 768

From: /data/APP/current/app/controllers/discussions_controller.rb @ line 72 DiscussionsController#assign:

    71: def assign
 => 72:   binding.pry
    73:   @discussion.assign(user)
          [...]
    89: end

[1] pry(#<DiscussionsController>)>

So let's try the basics: run the line and verify if it works or not.

[1] pry(#<DiscussionsController>)> @discussion.assign(user)
=> nil

[2] pry(#<DiscussionsController>)> @discussion.assignments
=> []

That doesn't fair well.. Let's step inside the method instead:

[3] pry(#<DiscussionsController>)> step

From: /data/APP/current/app/models/discussion/concerns/assignments.rb @ line 87 Discussion#assign:

    86: def assign(user)
 => 87:   unless assigned?(user)
    88:     assignment = Assignment.create(:discussion => self, :user => user)
    89:   end
    90: end
    
[4] pry(#<DiscussionsController>)>

Something looks fishy:

[4] pry(#<DiscussionsController>)> assigned?(user)
=> true

[5] pry(#<DiscussionsController>)> assignments
[]

Let's look inside assigned?

[6] pry(#<DiscussionsController>)> step

From: /data/APP/current/app/models/discussion/concerns/assignments.rb @ line 123 Discussion#assigned?:

    122: def assigned?(user)
 => 123:   REDIS.sismember(RedisKey.assignments(self.id), user.id)
    124: end
    
[7] pry(#<DiscussionsController>)>

And there is our bug! The assigned? method looks in our Redis cache to see if the discussion is already assigned to this user, but assignments looks in the database. Somehow the data in Redis has been corrupted, and is no more in sync with the database.

Step 3: Fix it

Now that we understand the bug, we can go back to our development environment and start writing tests, a fix, etc.. We can also start looking at logs to figure out how the Redis data got corrupted in the first place: we will have to fix that as well.

Conclusion

While Pry should not be used as the "be-all and end-all" for production problems, it does come in handy to debug issues related to data, that are hard or impossible to reproduce in other environments. By selectively patching code in the session, and stepping into requests, we can quickly figure out the source of problems, and then go back to our normal workflow to write the fix.

I hope I was able to convince you that Pry is totally awesome. But don't take my word for it, try it for yourself: you'll be hooked!

That's it for today, cheers!

If you don't know about Pry yet, you're missing out. It's a replacement REPL for Ruby with tons of features and many plugins, which makes it a perfect replacement for IRB, and as a result, for the standard Rails console.

A little while ago, the fine folks at Bugsnag wrote an article about their usage of Pry in production. I've been a Pry convert for quite some time as well, so today I'd like to share my own tips and tricks.

Running Pry

I think the Bugsnag article nails it pretty well, though I tweak some of the steps:

• Use a dedicated machine: it's pretty easy for a console session to start eating up memory and resources, and you don't want that to disrupt your running application. The best machine is usually a job runner machine: all configured with the app, but not running any webserver.

• Setup your Gemfile: instead of using pry-rails, I prefer to use the pryrc technique which I'll get back to in a minute. So my Gemfile looks like this:

  # Gemfile
  gem 'pry', :require => false
  gem 'pry-nav', :require => false
  

• Use a shortcut: mine is called t1. This is only for quick edits though, not for long running tasks. If you are going to leave a session running for a while, processing data, you definitely want to run it inside a screen or tmux session.

  # ~/.bashrc
  alias t1='ssh -t X.X.X.X -C "cd /data/APP/current && RAILS_ENV=production bundle exec pry"'
  

Configure Pry

Since we are using a dedicated machine, we can have a shared pryrc for everyone that offers some Rails goodies. We are not using pry-rails, so we also need some of the code in ~/.pryrc to do the Rails initialization:

# ~/.pryrc
# MISSING

Play

The first benefit of using Pry is that you get an actual editor for your code. I tend to do a lot of support related tasks, and so I often need to run one off snippets of code. They are slightly more complicated than a one-liner, but at the same time quite straight forward to write and modify. So my usual workflow is this:

PRODUCTION [1] pry(main)> def find_ips; end
PRODUCTION [1] pry(main)> edit find_ips

Since Pry is configured with vim as the default editor, this drops me in vim where I can start writing my code:

require "set"

# Check all the IP addresses used by one user
def find_ips
  list = Set.new
  user = User.find 1234
  user.comments.each {|comment| list << comment.ip}
  list.each {|ip| puts ip}
end

The code is simple, but being able to work with it in a real editor is a big improvement. We can now run it, make adjustments, and iterate until we get it just right:

PRODUCTION [1] pry(main)> find_ips
X.X.X.X
X.X.X.X
X.X.X.X

PRODUCTION [1] pry(main)> edit find_ips
# do some changes

PRODUCTION [1] pry(main)> find_ips
X.X.X.X on Feb 1st, 2014 at 8:14:45 AM
X.X.X.X on Mar 2nd, 2014 at 10:01:32 PM
[...]

The other nice thing is that it makes it a lot easier to extract useful code. As I try to solve a particular problem, or extract data, I will often end up with a method 10 to 20 lines long that solves a particular support use case. If I want to save it for later, or integrate it into the main codebase, I can simply edit it again, and save it to a safe location from vim, or copy/paste it somewhere else. This would be a lot harder if all I had was a bunch of one-liners that only existed in the history of my session.

Patch

One of the most useful feature of Pry is it's ability to easily monkeypatch existing code. Let's say that we have a job that generates a CSV export of some reporting data. The export is great and all customers love it, but this one customer asked you to add another column to it. They have a somewhat narrow use case, and including the column for all customers would not necessarily be an improvement, so you decide to simply run the report once with this extra column, just for them. With Pry, this is super easy!

First, we take a look at the CSV export itself:

PRODUCTION [1] pry(main)> show-source Report#generate_csv

def generate_csv
  [...]
  documents.each {|doc| csv << doc.csv}
  [...]
end

So we know that we need to modify the Document#csv method. It's a simple method that returns an array. We are going to patch the method: the modification will only exist within our Pry session and won't be written on disk. To do that, we use edit -p instead of the standard edit:

PRODUCTION [1] pry(main)> edit Document#csv -p

Add the extra data:

def csv
  [ title,
    content,
    created_at,
    updated_at,
    custom_data ] # LINE ADDED
end

We can now generate the report for this customer, without affecting anyone else:

PRODUCTION [1] pry(main)> Report.find(1234).generate_csv

And we're done!

Conclusion

Pry is a perfect replacement for IRB, and can greatly improve your life when working on production applications. Just for the ability to edit code in vim, Pry is absolutely worth it. I definitely recommend spending some time installing it, configuring it, and getting it working on your application.

And if you've never used Pry before, take a look at this great talk: Ruby Conf 2013 - REPL driven development with Pry by Conrad Irwin. It will make you an instant convert.

That's it for today, cheers!

I have 2 Github accounts: a personal one, and a professional one. If you have more than one account on Github, you've probably experienced this before:

ERROR: Permission to user2/repo.git denied to user1.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Why does this happen?

To better understand what's going on, let's assume we have 2 users:

• user1 is the main profile, associated with ~/.ssh/id_rsa
• user2 is a secondary profile, associated with ~/.ssh/id_github_rsa

First, let's get some debugging info. In ~/.ssh/config, add:

Host github.com
  LogLevel DEBUG

Then try again a git push in a repository owned by user2:

debug1: Connecting to github.com [192.30.252.131] port 22.
debug1: Connection established.
[...]
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: ~/.ssh/id_rsa
[...]
debug1: Authentication succeeded (publickey).

The key used for authentication is id_rsa attached to user1. This is the problem. SSH tries keys in a certain order, and id_rsa occurs first. The authentication succeeds at the SSH level (there IS a user attached to this key), but fails at the git level because user1 is not a member of the repository we are trying to push to, owned by user2.

Now that we understand what the problem is, let's try to fix it.

First attempt

Since SSH allows for custom configuration per host, the first idea that comes to mind is to create a special host with a preferred key. In ~/.ssh/config, let's add a section for user2:

Host github-user2
  User git
  Hostname github.com
  IdentityFile ~/.ssh/id_github_rsa

We will also need to update our github repository. We need to change the current origin from its default git@github.com to github-user2:

cd ~/projects/user2/project
git remote set-url origin github-user2:user2/project.git

Great! Let's try to push again:

ERROR: Permission to user2/project.git denied to user1.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Mmmm.. Looks like that didn't do it. Let's increase our log level to the maximum:

Host github-user2
  LogLevel DEBUG3
  User git
  Hostname github.com
  IdentityFile ~/.ssh/id_github_rsa

And push again:

debug2: ssh_connect: needpriv 0
debug1: Connecting to github.com [192.30.252.128] port 22.
debug1: Connection established.
[...]
debug2: key: ~/.ssh/id_rsa (0x700100200300),
debug2: key: ~/.ssh/id_github_rsa (0x700200300400), explicit
[...]
debug1: Next authentication method: publickey
debug1: Offering RSA public key: ~/.ssh/id_rsa
[...]
debug1: Authentication succeeded (publickey).

And so here is where weirdness happens: id_github_rsa key seems to be recognized from our git config (marked as explicit), but somehow id_rsa is still used first. Why?

It took me a minute to figure this one out. I spent some time searching and eventually found this blog post by Drew Crawford. The problem comes from the way IdentityFile works. From the man:

Specifies a file from which the user's DSA, ECDSA or RSA authentication identity is read. The default is ~/.ssh/identity for protocol version 1, and ~/.ssh/id_dsa, ~/.ssh/id_ecdsa and ~/.ssh/id_rsa for protocol version 2. Additionally, any identities represented by the authentication agent will be used for authentication. [...]

So despite specifying a preferred key for authentication, SSH will first try keys already loaded in the agent. Since id_rsa is our main key and we use it everywhere, chances are it's loaded in the agent first, therefore overriding our configuration.

With this new information in hand, let's update our SSH config.

Second attempt

So we need to tell SSH to ignore any keys already loaded in the agent, and just use the one we specify. Thankfully, there is an option for that, IdentitiesOnly:

Specifies that ssh(1) should only use the authentication identity files configured in the ssh_config files, even if ssh-agent(1) or a PKCS11Provider offers more identities. The argument to this keyword must be yes or no. This option is intended for situations where ssh-agent offers many different identities. The default is no.

Let's update our ~/.ssh/config:

Host github-user2
  User git
  Hostname github.com
  IdentitiesOnly yes
  IdentityFile ~/.ssh/id_github_rsa

And try one last push:

$ cd ~/projects/user2/project
$ git push
Everything up-to-date

Hooray \o/

Recap

So to successfully authenticate on Github with a secondary user, you need to:

1. Create a dedicated SSH config with IdentityFile and IdentitiesOnly
2. Update the origin URL of every repo to use the SSH config: github-user:user/repo

Not that complicated, but not easy to figure out either. It certainly took me a minute, especially since my keys were not always loaded in the same order in the SSH agent, which confused things a bit. So I hope this will avoid you some headaches :)

That's it for today, cheers!

OS X Mavericks introduced a new tab in Activity Monitor: Energy. The nice thing about this new pane is that it includes an average. If you run on battery a lot, and are looking for ways to maximize your battery life, this gives you instant insight into the power hogs on your system:

I'm definitely not going to kill Chrome or Propane, but it's interesting to notice that the simple Little Snitch Network Monitor is using so much battery, compared to Evernote for example (which is also running all the time). A quick change of settings and you get an instant battery win:

Looking at this section regularly, I was able to tweak the settings of a few applications (I'm looking at you Crashplan) and got a noticeable improvement in overall battery life. I've also been enjoying FruitJuice, which tells you when to plug and when to use the battery.

So if you are running Mavericks and looking for ways to maximize your time away from an outlet, take a look at Activity Monitor: it's fast and easy.

That's it for today, cheers!

Continuing on last week theme of downloading videos, youtube-dl is a nifty little utility that allows you to download videos to watch later, or for safekeeping.

Install with Homebrew:

brew install youtube-dl

You can now download videos from YouTube, BlipTV, Brightcove, Dailymotion, TED and many other supported sites:

$ youtube-dl https://www.youtube.com/watch?v=fK4mokbaaAU
# [youtube] Setting language
# [youtube] fK4mokbaaAU: Downloading webpage
# [youtube] fK4mokbaaAU: Downloading video info webpage
# [youtube] fK4mokbaaAU: Extracting video information
# [download] Destination: XXY (Collision Course).mp4
# [download] 100% of 29.55MiB in 01:36

Enjoy!

I love my commute: I love catching up on tech news in the morning, and indulge in silly entertainment in the evening. It is my moment of piece, my transition between work and home.

While Feedly has long solved the problem of text content, I've often wanted a simple way to catch up on all my favorite youtubers on the way back home. Unfortunately, the time I want to consume videos is also the time where it is the less practical to do so.

iTunes podcasts and RSS Handler

The nice thing about iTunes podcasts is that iTunes downloads them for you. Just like RSS feeds, there is no work for me to do: just let the content come to you. It also solves the bandwidth problem when commuting: just sync before leaving, and you're good to go.

RSS Handler is a standalone application that allows you to subscribe to YouTube users, playlists, etc., in iTunes. Just type in the username:

Copy it in iTunes

And iTunes will now download those videos for you:

Improving the worflow

While RSS Handler is great, finding the username, pasting it in the app, generating the URL and pasting it in iTunes, is just too much work. So I wrote a simple user script to simplify the process.

1st, make sure you have either Greasemonkey (Firefox), or Tampermonkey (Chrome), installed. Then go and install this user script:

Once you're done, you should see a small "iTunes" button at the bottom of user pages:

Choose a title for the podcast:

And you will be automatically subscribed in iTunes:

Conclusion

That's it for today, enjoy!

NCurses Disk Usage, or ncdu for short, is a graphical disk space analyzer:

Why you should use it

• It's incredibly fast
• It's available on every environment
• You can delete files directly within the app
• You can limit your search to one filesystem with -x (no /dev, etc)

Where you should use it

While I regularly use ncdu on my Mac, ncdu shines on servers. More than once I found myself with a hard drive close to full and no idea why? ncdu allowed me to scan the disk, find the source of the problem and fix it, all within minutes. Often times the issue was a renegade process creating huge amounts of data at a totally unexpected place, and finding the culprit without the "drill-down" approach of ncdu would have amounted to searching a needle in a haystack.

So now, I make sure it's installed on every server I work with. You never know when you will need it to save the day.

That’s it for today. Cheers ;)

mrd creates a virtual disk in memory, and runs an instance of MySQL on it. The main application is running tests: no matter how fast your hard drive is, it will never be as fast as RAM. Since test data is volatile anyway, why not use the fastest medium available. As an added bonus, because it prevents all these volatile writes from going to disk, it can increase the lifespan of your SSD.

While mrd is great, it's only great if you use it all the time. So in this article I'll share a few tips that have made using mrd a much more pleasant experience for me.

Installing it

Similar to our previous article on Mailcatcher, If you have a reasonably recent version of Linux or Mac OS X, you probably have Ruby installed. If not, you will have to install Ruby first. Once you have Ruby installed:

# If you are using the system's Ruby:
sudo gem install mrd

# If you use rbenv/rvm/etc:
gem install mrd

Running it

If you use rvm, rbenv or chruby, you will first need to create a wrapper script. I put mine in ~/.bin/launchd-mrd:

#!/bin/bash
export PATH="$PATH:/usr/local/bin"
source /usr/local/share/chruby/chruby.sh
chruby 2.1
mrd

Make the script executable:

chmod +x ~/.bin/launchd-mrd

Launch it:

~/.bin/launchd-mrd
# Created Ramdisk at /dev/disk4
# Formatted Ramdisk at /dev/disk4
# Mounted Ramdisk at /Volumes/MySQLRAMDisk
#
# [...]
#
# Starting MySQL server
# MySQL is now running.
# Configure you client to use the root user, no password, and the socket at '/Volumes/MySQLRAMDisk/mysql.sock'.
# Just close this terminal or press ^C when you no longer need it.

Great! It works perfectly. Let's now create a daemon to automatically start it on boot.

Starting it automatically

Create the startup script:

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
  <dict>
    <key>Label</key>
    <string>com.$USER.mrd</string>
    <key>Program</key>
    <string>$HOME/.bin/launchd-mrd</string>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/tmp/com.$USER.mrd.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/com.$USER.mrd.out</string>
  </dict>
</plist>
" > ~/Library/LaunchAgents/com.$USER.mrd.plist

Launch it:

launchctl load ~/Library/LaunchAgents/com.$USER.mrd.plist

You should see the new shiny icon right on your desktop:

If you run into any trouble, look at the logs:

/tmp/com.$USER.mrd.out
/tmp/com.$USER.mrd.err

Configuring Rails

You can now open your config/database.yml and update the test section:

test:
  adapter: mysql2
  encoding: utf8
  collation: utf8_unicode_ci
  host: localhost
  database: NAME
  username: root
  password: 
  socket: /Volumes/MySQLRAMDisk/mysql.sock

Run the tests:

cd ~/project
RAILS_ENV=test bundle exec rake db:create
bundle exec rake db:test:prepare
bundle exec rake test
..............................................................
..............................................................
...

Yeah \o/ This works!

Automating the database creation

Wouldn't it be better if we could automate the database creation/preparation as well? After all, you might have multiple projects, and you don't want to have to recreate the test database each time, load the schema, etc... So let's automate all that.

Create a new script ~/.bin/prepare-test-dbs:

#!/bin/bash

# Load your preferred Ruby Version Manager
export PATH="$PATH:/usr/local/bin"
source /usr/local/share/chruby/chruby.sh
source /usr/local/share/chruby/auto.sh

# Wait until 'mrd' is actually loaded, and MySQL running
until [[ -S /Volumes/MySQLRAMDisk/mysql.sock ]]; do
  sleep 1
done

# Go through each project, and:
# - create the database
# - load the schema
# - clear logs/*
# - clear tmp/*
for dir in ~/projects/project1 \
           ~/projects/project2 \
           ~/projects/projectN; do
  cd "$dir"
  RAILS_ENV=test bundle exec rake db:create
  bundle exec rake db:test:prepare
  bundle exec rake log:clear
  bundle exec rake tmp:clear
done

Make it executable:

chmod +x ~/.bin/prepare-test-dbs

And create the startup script:

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
  <dict>
    <key>Label</key>
    <string>com.$USER.mrd-prepare</string>
    <key>Program</key>
    <string>$HOME/.bin/prepare-test-dbs</string>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/tmp/com.$USER.mrd-prepare.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/com.$USER.mrd-prepare.out</string>
  </dict>
</plist>
" > ~/Library/LaunchAgents/com.$USER.mrd-prepare.plist

Now, each time your start your computer:

• The RAM disk will be created
• mrd will launch a MySQL instance
• The script will prepare a test DB for each of your projects

Run the tests:

cd ~/projects/project1
bundle exec rake test
..............................................................
..............................................................
...

You're all set!

That's it for today. Cheers ;)

One of the great features coming in Rails 4.1 is Action Mailer Previews. This will offer a quick and simple way to preview the emails of your application as your develop them.

Unfortunately, not everyone will be able to jump to Rails 4.1 right away, and not everyone uses Rails. Your application might be written in PHP, Java, .Net, etc. Wouldn't it be great if you could easily preview all your emails without having to use Gmail as your SMTP?

Well rejoice my friend: here comes mailcatcher!

What is mailcatcher?

The premise of mailcatcher is really simple: it runs a SMTP server on your machine, on port 1025, and saves all the emails it receives. It also comes with a web interface where you can inspect said emails:

Why is it awesome?

• It catches all emails (but doesn't actually send any). This means that you can not only verify that each email looks ok, but also that the right number of emails was sent, to the right people. This also means that you can use real data, like an existing customer list, without fear of spamming those users.

• It is compatible with pretty much every language/framework. If your application sends emails, chances are it supports SMTP. So this provides you with a one-stop solution for all your email needs.

• It gives you the relevant information: HTML, plain text, source. Quick and easy.

Installation

The mailcatcher site provides a lot more details but the gist of it is really simple. If you have a reasonably recent version of Linux or Mac OS X, you probably have Ruby installed. If not, you will have to install Ruby first.

Once you have Ruby installed:

# If you are using the system's Ruby:
sudo gem install mailcatcher

# If you use rbenv/rvm/etc:
gem install mailcatcher

You can then run mailcatcher with:

mailcatcher     # As a daemon
mailcatcher -f  # In the foreground

If mailcatcher is not in your path, look at your ruby environment:

gem environment
# RubyGems Environment:
# - RUBYGEMS VERSION: 2.2.2
# - RUBY VERSION: 2.1.1 (2014-02-24 patchlevel 76) [x86_64-darwin13.0]
# - INSTALLATION DIRECTORY: /Users/Julien/.gem/ruby/2.1.1
# - RUBY EXECUTABLE: /Users/Julien/.rubies/2.1.1/bin/ruby
# - EXECUTABLE DIRECTORY: /Users/Julien/.gem/ruby/2.1.1/bin
# - SPEC CACHE DIRECTORY: /Users/Julien/.gem/specs

Look in the EXECUTABLE DIRECTORY:

# It should be here
/Users/Julien/.rubies/2.1.1/bin/mailcatcher

Configuring Rails

# In config/environments/development.rb
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = { 
  :address => "localhost", :port => 1025 
}

The mailcatcher site provides information for other platforms.

Starting it automatically

You will probably want to automatically start mailcatcher when you start your computer. It's pretty easy on Mac OS X:

# Path to mailcatcher. You might want to try 
# 'which mailcatcher' in your terminal first
# to make sure it returns a result.
# If it doesn't, and you know the path to the
# executable, just set it here.
MAILCATCHER_PATH=`which mailcatcher`

# Create the daemon file
echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
  <dict>
    <key>Label</key>
    <string>com.mailcatcher</string>
    <key>ProgramArguments</key>
    <array>
      <string>$MAILCATCHER_PATH</string>
      <string>-f</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardErrorPath</key>
    <string>/tmp/com.mailcatcher.err</string>
    <key>StandardOutPath</key>
    <string>/tmp/com.mailcatcher.out</string>
  </dict>
</plist>
" > ~/Library/LaunchAgents/com.mailcatcher.plist

You can then start it for the current session with (make sure it's not already running):

launchctl load ~/Library/LaunchAgents/com.mailcatcher.plist

If you run into any trouble, look at the logs:

/tmp/com.mailcatcher.out
/tmp/com.mailcatcher.err

Caveat

If you use rbenv, rvm, etc., you might need a wrapper script for the environment:

#!/bin/bash
export PATH="$PATH:/usr/local/bin"
source /usr/local/share/chruby/chruby.sh
chruby 2.1
mailcatcher -f

Conclusion

Mailcatcher is absolutely awesome and I don't know what I would do without it.

That's it for today. Cheers!

Animated GIFs have seen a resurgence over the last few years. It's easy to see why...

While I'm grateful for puppies and kittens, the reason I like animated GIFs, and I think the reason they won over the web, is because GIF is a universal format. Point anyone to a URL, and no matter their platform/device, it's pretty much guaranteed they can see it.

Screencasts

Because GIF is such a universal format, it makes sense to use it for short screencasts. LICEcap is a tiny utility for Mac and Windows that allows you to quickly select a section of your screen and record it directly to a GIF file:

Editing

While LICEcap makes it super easy to quickly record GIFs, it lacks any editing capabilities. Sometimes, all you need to make your GIF perfect is remove a few frames, for errors, typos, dead time, etc. Keeping the smallest number of frames possible also helps with size.

GIF being a standard format, you can open it in Preview to edit the frames. Simply click on a frame and press delete to delete it. When you're done, you can save the file.

Another alternative, if the GIF gets really big, is to use ImageMagick:

# Generate individual frames
convert -coalesce input.gif frames.png

# Do some editing, and then put the frames back together
convert frames*.png output.gif

Framerate

Your GIF is almost ready. Since your removed a few frames, and sticked to the essential, it's possible it's now a bit too fast. So let's reduce the framerate with ImageMagick:

convert     \
  -loop 0   \  # number of loops. 0 is infinite
  -delay 20 \  # ticks between frames. 1 tick is 10ms, so 5FPS
  input.gif \
  output.gif

Optimizing

Finally, we can use ImageOptim to optimize the file size:

And you're done! Your screencast is ready to be shared with the world.

Conclusion

While you won't be producing hours of videos with LICEcap, it's definitely a useful tool for one off screencasts and other small demos. In association with Preview, ImageMagick and ImageOptim, it makes for a fast and powerful workflow.

Other alternatives and tools:

• GIF Brewery
• Recordit
• Screenr

That's it for today. Cheers!

There are many ways to develop on a Mac, and many stack to choose from. One common and recurring need however is to access your local websites through a named domain, ie using example.dev. Thankfully, there is a really simple way to do this using Dnsmasq as a local resolver.

Install Dnsmasq

This is straightforward with Homebrew:

# Install it
brew install dnsmasq

# Create the etc dir if needed
mkdir -p /usr/local/etc

# Create a simple configuration
# This forces the .dev domain to respond with 127.0.0.1
# You can find more information in the default config file:
#   /usr/local/opt/dnsmasq/dnsmasq.conf.example
echo "address=/.dev/127.0.0.1" > /usr/local/etc/dnsmasq.conf

# Install the daemon startup file
sudo cp -fv /usr/local/opt/dnsmasq/*.plist \
  /Library/LaunchDaemons

# Start the daemon
sudo launchctl load \
  /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist

Configure Mac OS X

All we need to do is tell the resolver to use Dnsmasq for .dev domains:

# man 5 resolver
sudo mkdir -p /etc/resolver
sudo sh -c 'echo "nameserver 127.0.0.1" > /etc/resolver/dev'

You can now use any .dev domain and it will always resolve to 127.0.0.1:

ping google.com   # this still works
# PING google.com (74.125.239.38): 56 data bytes

ping foo.dev      # you can use any domain
# PING foo.dev (127.0.0.1): 56 data bytes

ping bar.baz.dev  # or subdomain
# PING foo.dev (127.0.0.1): 56 data bytes

Conclusion

This is very useful in particular for applications that use subdomains as account identifier: you can easily create new accounts on the fly, and never have to worry about your /etc/hosts file again.

Finally, you may also want to look at Pow (and powder). Pow will automatically start your Rails/Rack applications, and provides a port proxying feature for apps written in other languages. Definitely a great alternative.

That's it for today.

Cheers!

In our last article, we saw how to use Little Snitch to stay secure by forcing the use of a VPN at unsafe locations. Let's go a little further and see how we can save on bandwidth and improve the quality of our connection.

Why save bandwidth?

I started thinking about restricting internet access to a few applications for 2 reasons:

• Free Wifi at crowded places can be less than stellar. When you can't SSH to a remote server because iTunes is downloading your podcasts, and Mac OS X checking for updates, you have a problem.

• I have a hotspot from FreedomPop, and I get 500MB of free data every month. This may not be enough to watch Netflix, but it's plenty when you're on call and all you need is a browser, and a terminal. Why pay extra for data when you don't need to?

Thankfully, Little Snitch offers a simple and elegant solution.

Creating a profile

Let's start by creating a new profile. We'll call it "Low Bandwidth":

Now let's connect to our hotspot:

Building the rules

If you followed the previous article, you should have Little Snitch in "Silent Mode: Deny". This makes it really easy to quickly enable the 2 or 3 applications we care about when on the go.

Open the Rules screen:

Then, go to "Silent Mode Connections" and allow Chrome, Safari and Terminal:

Since you're doing this while on the Low Bandwidth profile, the new rules are automatically associated with it:

And there you have it! Safari now works:

But the App Store won't start updating all your applications behind your back:

Conclusion

This works really well and has allowed me to enjoy my browsing experience on questionable networks. It has also allowed me to stay within my free 500MB while using my hotspot regularly.

Also, if you're keen on saving bandwidth, I highly recommend Bandwidth+. It's a simple little app that sits in your menu bar and monitor your bandwidth usage. Priceless.

I hope these 2 articles were helpful and gave you a better understanding of the power of Little Snitch. It's a great tool, simple to use and configure, and absolutely worth the money. Definitely a must have for anyone working remotely.

That's it for today.

Cheers!

There are a couple reasons why you would want to access the internet through a VPN: your work requires it, you regularly work from open Wifi networks, at hotels, coffee shops or libraries, and you want to prevent eavesdropping.

This article presents a simple technique that allows you to force internet access through a VPN when using unsafe networks. I will use Private Internet Access as an example, but this applies to any VPN connection, on any network.

TL;DR

• Modify rules in default profile to deny all connections.
• Create a "VPN" profile that allows all connections.
• Let "Automatic Profile Switching" do the rest.

Little Snitch

Little Snitch is a firewall that allows you to control connections from your computer to the internet. One of its greatest features, introduced in version 3, is "Automatic Profile Switching": the ability to automatically apply different rules depending on which network you're connected to.

We are going to use this feature to provide unrestricted access to the internet when connected to a VPN, and automatically cut off access as soon as we are disconnected (or before we are connected).

Step 1: Install Little Snitch

Go ahead and install Little Snitch.

Step 2: Stop the filter and switch to silent mode

Little Snitch has a tendency to be a bit verbose, and will pester you with questions as soon as any application attempts a connection, which can rapidly get annoying.

Fortunately, there is a "Silent Mode", which will automatically allow/deny any connection, and offer us some peace while we work on the configuration. So:

• Stop the network filter
• Set "Silent Mode" to "Deny"

Step 3: Delete all default rules

Little Snitch comes with a couple of default rules. They are mostly harmful, but if you are worried about your privacy, it can't hurt to be cautious. So let's start from an empty environment.

Open the "Rules" screen:

Delete or disable all the rules. You may get a few warnings, but just go ahead and do it anyway (you can always restore the factory defaults later).

I only keep 3 main rules:

• DNS
• Outgoing connections to local network
• Incoming connections (though it's safe to disable those as well)

When you are done, your rules should look like this:

Let's simplify the view a bit by hiding disabled rules:

Step 4: Create a new profile

First, we need to enable "Automatic Profile Switching":

Now, let's create our VPN profile:

Finally, we restart the network filter:

Turn Wifi on and off:

And now Little Snitch wants you to choose a profile. If this your home connection, you could choose the newly created VPN profile. If you are at an unsafe location, or if you simply prefer to have VPN activated at all times, select "Deactivate Active Profile":

Since the default rules do not explicitly allow any connection, and we have set "Silent Mode" to "Deny", we basically lost internet access:

That's what we wanted :) Let's now configure the VPN.

Step 5: Creating new rules

We succeeded in stopping access for all applications, but the truth is: the VPN itself needs access. So we need to create a few rules for that.

Try to start the connection:

At this time, the VPN won't be able to connect, but since we activated Silent Mode, the connection attempts will appear in Little Snitch and we can create new rules:

As soon as the rules are created, the VPN connection will succeed and you will be prompted with the familiar dialog. Choose "VPN & Safe Networks":

Finally, now that the association has been made between the VPN network and the VPN profile, we need to restore access to all applications. Go back to the rules window, and click "New". You will need to create 2 rules. One for all applications owned by you, and one for all system applications:

If all goes well, you now have full internet access:

Caveats

• I left access open for DNS. Depending on your VPN configuration and your needs, you may want to restrict that as well, and just allow access to the IP of your VPN server. It's up to you.

• The rules required to make your VPN work will of course depend on your VPN. It is usually pretty simple to figure out once you look at the connection attempts, but you may have to play with it a bit until you find the right set. You can double click any rule to adjust its parameters (expand it, restrict it).

• Some hotels/hotspots require that you accept their terms and conditions before getting access, in which case your VPN won't be able to connect. Because of this, I have a special rule to always allow access for Safari. I don't use it as a browser normally, but if the VPN fails and I'm on a free network, I just open it and try apple.com. If there is an authentication phase, I'll get redirected. Once I've accepted the terms and conditions, I'll get the full access and the VPN will be able to connect normally.

Conclusion

That's it! You're pretty much set :) Now, every time you connect to a new network, Little Snitch will ask you to choose a profile and you can either choose the safe one (for work and home), or deactivate the current profile and launch your VPN (for coffee shops, etc).

I hope this was helpful. Until next time, Cheers!

If you are a heavy user of vim, chances are that over time you added a bunch of plugins, syntax files and other goodies to your vim configuration. This makes your editing experience smoother, but if you installed all these scripts manually, maintaining them can be a pain. Keeping up-to-date with the latest versions, or simply removing one you don't use anymore becomes difficult as files from the different scripts get mixed up in your .vim directory.

Vimballs and pathogen can both ease the process, but you still have to download the scripts manually, and take care of all the updates.

Vundle

Vundle is basically a package manager for vim plugins. It will search, install, update and delete plugins for your while maintaining a nice directory structure. To install it:

git clone git://github.com/gmarik/vundle.git ~/.vim/bundle/vundle

Then, update your .vimrc with the following lines:

set rtp+=~/.vim/bundle/vundle/
call vundle#rc()

You're good to go.

Installing a plugin

Let's say that you are want to install Align. You can do a search for it in vim:

:BundleSearch align

And you will be greeted by a new window listing all the search results:

" Keymap: i - Install bundle; c - Cleanup; ...
" Search results for: align
Bundle 'right_align'
Bundle 'align--Ahmad'
Bundle 'Easy-alignment-to-column'
Bundle 'Align.vim'
Bundle 'AutoAlign'
Bundle 'feralalign.vim'
Bundle 'Align'
Bundle 'Lineup--A-simple-text-aligner'

Press i to install a plugin, and then copy the relevant line to your .vimrc to have it loaded automatically when you launch vim:

" Initialize Vundle
set rtp+=~/.vim/bundle/vundle/
call vundle#rc()

" My plugins
Bundle 'Align'

Sometimes, the result of the search isn't very useful and you won't be sure which plugin to install. I find that an easier way is to go directly to the list of scripts where you will find a short description and links to the plugins' pages.

Other commands

Updating all your plugins:

:BundleInstall!

Removing unused plugins:

:BundleClean

Where to find plugins?

You can look on vim.org for the list of all plugins, sorted by ratings or downloads. This will give you an idea of what most people are using: chances are some of them will be useful to you too.

Beginners might also want to look at the janus distribution, which makes starting with vim easier. More advanced users will enjoy the Thoughtbot dotfiles.

That's it for today. Hope you enjoyed it.

Until next time, Cheers!

As a developer, chances are you spend a lot of time every day in your terminal. Whether you use it for building, debugging, inspecting, deploying or administering, the shell is part of your life.

Since you spend such a large amount of time using it, it makes sense to get comfortable with it. You can treat your shell as a temporary mean to an end, an impersonal hotel room of sorts, functional but never yours, or you can make it your home, a warm and cozy place, with your things, your habits and your tools.

One way to achieve that goal, to make your shell your home, is to customize it. And what better place to start than the prompt? Most Linux distributions have a slightly customized prompt, designed to work in most situations. If you're lucky, it looks like this:

user@host $ ...

If you're not, it looks like this:

$ ...

Wouldn't it be better if you could have a little more information? Maybe the time of the day, the current directory, or just some colors? Thankfully you can, and it's easy.

PS1

The variable that contains the definition of the prompt format is PS1. You can experiment with it directly in your shell:

$ PS1="% "
% PS1="! "
! ...

Apart from using a simple string, you can also use special variables such as:

\n     newline
\r     carriage return

\j     number of jobs managed by the shell

\d     date in "Weekday Month Date"
\t     current time in 24-hour HH:MM:SS format
\T     current time in 12-hour HH:MM:SS format
\@     current time in 12-hour am/pm format
\A     current time in 24-hour HH:MM format

\H     hostname
\h     hostname up to the first '.'
\u     username of the current user
\w     current working directory

There are more available, which you can find in the man (search for PROMPTING). For example, in my shell, I use the following format:

$ PS1="\u \h \t \w \\$ "
user hostname 20:46:05 ~/folder $

Colors

The shell has special escape codes for colors:

FG=COLOR_CODE
BG=COLOR_CODE

# '\033[' + FOREGROUND + '\033[' + BACKGROUND
CODE="\033[$FG\033[$BG"

The Bash Prompt HOWTO provides a script to quickly list all the colors:

Together

So you can now create your own prompt, just the way you want it :) In my ~/.bashrc I have:

# We need to encapsulate with "\[ \]" to use in a prompt
BLUE1='\[\033[1;34m\]'
BLUE2='\[\033[34m\]'
GREEN1='\[\033[1;32m\]'
GREEN2='\[\033[32m\]'
CYAN1='\[\033[1;36m\]'
CYAN2='\[\033[36m\]'
RED1='\[\033[1;31m\]'
RED2='\[\033[31m\]'
PURPLE1='\[\033[1;35m\]'
PURPLE2='\[\033[35m\]'
BROWN1='\[\033[1;33m\]'
BROWN2='\[\033[33m\]'
GRIS1='\[\033[1;37m\]'
GRIS2='\[\033[37m\]'
YELLOW1='\[\033[1;33m\]'
YELLOW2='\[\033[33m\]'
WHITE='\[\033[1;37m\]'
NOCOLOUR='\[\033[0m\]'

# Don't forget to use double quotes "
PS1="$RED1\u $BLUE1\h $PURPLE1\t $GREEN2\w $BROWN2\\$ $NOCOLOUR"

Which gives:

Going further

This was just a simple customization, but you can do much more: you can run complex commands to create your prompt, using all the tools available in your system. If you want to know more, check out:

• Color Bash Prompt
• The Bash Prompt HOWTO
• Crazy Powerful Bash Prompt

That's it for today. Hope you enjoyed it.

Until next time, Cheers!

Last time we introduced local port forwarding with SSH. Today let's look at related features: remote port forwarding and socks proxying.

Remote port forwarding

Remote port forwarding works exactly like local forwarding, but the other way around: while local forwarding transfers local connections to the remote machine, remote forwarding transfers remote connections to you (or another machine you have access to).

When is this useful? Well most of the time, your network setup is such that you can access the server through SSH, because it has a public IP address, but the server can't access you because you are behind a firewall or a router.

So an example use case would be remote debugging an application. A debugger is installed on a remote development server, but the debugger can't reach you, and therefore your IDE/editor of choice. You thus setup things as follow:

• Set the debugger on the server to connect to localhost (default)
• Forward the remote port (9000) to your machine (same port)
• Set your IDE on listen mode, ie listening for debugger connections
• Start a debugging session

The SSH command is :

ssh -R 9000:127.0.0.1:9000 example.com

And the scenario:

http://example.com?START_DEBUG=1 ----+
                                     |
+-- SSH <- server:9000 <- debugger <-+
|
+-> SSH -> local:9000 -> IDE

Apart from the direction of the forwarding, it works exactly like local forward, and thus offers the same options: you bind to localhost, an external address, or all interfaces, and you can forward to a local port, or to another machine you have access to.

In a complex scenario, you could have something like this: a remote server B talks to a webservice on server A (also remote), but A doesn't have a webservice. The webservice is on your network, so you use your machine and machine A as bridges to make everybody communicate:

webservice <---------> you <----> A <---------> B
            network 1       SSH      network 2

with:

ssh -R 9000:webservice:9000 machine-a

That's it for local and remote port forwarding. Sometimes however, you need a little more than one or two ports, and you need to redirect a large part of your connections through a 3rd party: this is what SOCKS is for.

SOCKS Proxying

SOCKS is a protocol specifically designed to proxy network connections. It works by setting up a server which acts as a gateway/proxy to all connections. SOCKS aware applications can connect to it, send their packets, and the SOCKS server will take care of sending them to their destination.

As a general proxying mechanism, SOCKS is much more powerful than simple port forwarding. Through one single SOCKS server, you can proxy multiple applications and protocols, as long as they support SOCKS, which many applications do. Most web browsers and instant messengers for example natively support it.

So how is that related to SSH and how do you use it? Well SSH can act as a SOCKS server. It is as simple as:

ssh -D 7000 example.com

and you have a SOCKS server running on port 7000 of your machine, proxying all client connections through example.com. Now all you have to do is configure your applications to use the SOCKS server.

Configuration: Mac OS X

If your computer is running on Mac OS X, setting up a SOCKS proxy for all your applications can easily be done in one place. You need to go to your Network Preferences:

Select the Advanced button for your connection:

Click the SOCKS Proxy option and fill out the parameters:

You're done. All the applications on your Mac will now use this configuration (at least those who do it right, which is most).

Configuration: Firefox

If your operating system doesn't provide such configuration, you can generally configure it on a per-application basis. With Firefox, go to Preferences -> Advanced -> Network and click on Settings

There you can fill out the SOCKS proxy information:

Other browsers have similar configuration screens. If you are gonna switch between proxy and no proxy pretty often, it might be useful to add a plugin/extension to your browser to be able to switch easily without having to go through all these configuration screens all the time :

• QuickProxy and FoxyProxy for Firefox
• Proxy Switchy! for Chrome

Final note: DNS

The SOCKS protocol was built to proxy any kind of network connection through a server. When you enable a SOCKS proxy in some browsers however, there is one type of connection that doesn't get proxied: DNS.

When you request a web page at http://example.com, the browser will first ask the operating system to resolve example.com to a known IP address, and then go through the SOCKS proxy to connect to that IP. What this mean is that if you are using a proxy for privacy reasons, you are leaking information to the DNS resolver. Depending on your situation, that may or may not be an issue, but it's good to be aware of it. If you want to have full privacy, you need to go through an HTTP proxy instead, because in this case the HTTP proxy is in charge of the DNS resolution, not you (which of course only works if you are in control of the HTTP proxy, otherwise you just moved the problem somewhere else). Or you can just change the default setting.

Conclusion

That's it for today. Hope you enjoyed it.

Until next time, Cheers!

What is tunneling? To put it simply, tunneling is the mechanism by which one port on your local machine redirects to another port on another machine. To understand what it means and why it's useful, let's look at a few use cases.

Local forward, simple case

Imagine you are working on a remote server, example.com. This server has a pretty standard configuration: Varnish, Apache, ...

• Port 22 : SSH, publicly accessible
• Port 80 : Varnish, publicly accessible
• Port 8000 : Apache, localhost only (private)

Let's now say that for some reason you are having troubles with the web server but you are not sure whether the problem is coming from Varnish or from the application itself. So you need to access Apache, but since Apache is listening on localhost, you can't just go to http://example.com:8000, you won't be able to connect. Here comes SSH tunneling to the rescue:

ssh -L 8000:127.0.0.1:8000 example.com

What does this do? It creates a tunnel between the local port 8000 on your machine to the IP address 127.0.0.1 and port 8000 on the remote machine. So, when you go to http://127.0.0.1:8000, your connection enters the tunnel and comes out on the other side knocking on Apache's door:

Browser -> 127.0.0.1:8000 -> SSH --+  # your machine
                                   |
Apache <-  127.0.0.1:8000 <- SSH <-+  # the server

Local forward, binding

The full syntax of the local forward is this :

ssh -L bind:port:host:hostport example.com

By default, SSH will bind to all your network interfaces. So if you are connected to a network, and have an IP address (192.168.0.10 for example), anyone can connect to port 8000 on your machine and access the remote server:

Developer 2                          # other machine
  |
  +--> 192.168.0.10:8000 -> SSH --+  # your machine
                                  |
Apache <- 127.0.0.1:8000 <- SSH <-+  # the server

Maybe the access to this server is sensitive, and you don't want anybody to come and fiddle with it. In this case, you can specify which network interface you want to bind to, and maybe restrict it to localhost:

ssh -L localhost:8000:127.0.0.1:8000 example.com

Local forward, advanced case

Let's go a little further. Imagine now that you want to access a web server located on example2.com. The problem is, example2 is inside a private network and you don't have direct access to it. However, example1 and example2 are on the same network, and you have access to example1. This is the typical situation where you have access to a bridge machine which then gets you access to other machines. This is useful for us because SSH can redirect not only to a local port, but to any port on any machine we have access to on the remote server:

ssh -L 8000:example2.com:80 example1.com

Here is what happens : you enter the tunnel on port 8000 on your machine, get out on machine example1.com and then ask the operating system: "get me to machine example2.com on port 80".

Browser -> 127.0.0.1:8000 -> SSH --+  # your machine
                                   |
         example2.com:80 <- SSH <--+  # example1

With SSH config

You can use aliases for this as well. If you regularly need to forward ports to/from the same host, you can do this in your ~/.ssh/config

Host ex1
  User user1
  Hostname example1.com

Host ex1proxy
  User user1
  Hostname example1.com
  LocalForward 8000 127.0.0.1:80

Which translates to:

ssh ex1       # ssh user1@example.com
ssh ex1proxy  # ssh user1@example.com -L 8000:127.0.0.1:80

Multiple forwards

You can of course use multiple forwards at the same time. So the following is valid:

ssh user1@example.com     \
  -L 8000:127.0.0.1:80    \
  -L 8080:127.0.0.1:8080  \
  -L 8000:example2.com:80

As well as:

Host ex1proxy
  User user1
  Hostname example1.com
  LocalForward 8000 127.0.0.1:80
  LocalForward 8080 127.0.0.1:8080
  LocalForward 8000 example2.com:80

Daemon mode

Finally, if you just want to use SSH to redirect ports, but do not need to launch commands on the remote server, you can do so with -N:

ssh ex1proxy -N &  # launch SSH in daemon mode
[1] 30604          # PID of the process

When you want to kill it:

jobs
[1]+  Running ssh ...  # list background jobs

fg 1                   # bring job to the foreground
^CKilled by signal 2.  # stop it with CTRL+C

Or:

kill 30604
[1]+  Done  ssh ...

Caveats

While I used the same port 8000 for the local and remote machine throughout this post, you don't have to. So the following is just as valid:

ssh -L 9000:127.0.0.1:8000 example.com

Privileged ports still apply. So binding to port 80 for example requires you to sudo:

$ ssh -L 80:127.0.0.1:8000 example.com
Privileged ports can only be forwarded by root.

$ sudo ssh -L 80:127.0.0.1:8000 example.com
Password: ********

# SUCCESS

Conclusion

Tunneling is an invaluable tool in your arsenal, and you should definitely get familiar with it. As usual, you can find more information in the man:

man ssh
man ssh_config

Until next time, Cheers!

SSH is a wonderful tool. It has many advanced features which can make your life easier as a developer, but they are not always well known or understood. So today, let's introduce one of these: aliases.

When you are working on a project, you often have to switch between different servers: a local server, a test/staging server and a production server. So you find yourself typing this all day:

ssh user@virtual-machine
ssh user@staging.server.com
ssh user@production.server.com

Some people find a workaround with shell aliases:

alias test="ssh user@staging.server.com"

That works alright in most cases, but what happens for example when you have to use scp or a rsync: you have to type the full command again...

# SSH to production server, quick and easy
prod

# Copy a file from test server. #FAIL
scp user@user.staging.server.com:/path .

The solution

Thankfully, SSH offers aliases too. They are defined in ~/.ssh/config:

Host foo
  User user
  Hostname example.com

With that configuration in place, you can now do all of the following:

# ssh user@example.com
ssh foo

# scp user@example.com:/some/path /bar
scp foo:/some/path /bar

# rsync user@example.com:/some/path /bar
rsync foo:/some/path /bar

All servers in a project

I follow a convention for all my servers and use suffixes to identify which server I'm connecting too. So let's say I want quick access to all the environments of my project example. Here is what my config would look like:

Host example.dev
  User deploy
  Hostname virtual-machine

Host example.test
  User deploy
  Hostname test.example.com

Host example.prod
  User deploy
  Hostname www.example.com

This way, I can quickly go to any of those servers, and I don't have to remember their exact name, IP address, login, etc.:

ssh example.dev    # local vm
ssh example.test   # testing server
ssh example.prod   # production server

Of course, the following now works too:

rsync -avz example.test:/var/www path

Other options

You don't have to limit yourself to User and Hostname, you can use the full range of SSH options. Here is a more advanced configuration:

Host example
  Port 678
  User special
  Hostname 88.88.88.88
  LocalForward 8000 127.0.0.1:80
  PreferredAuthentications publickey
  IdentityFile ~/.ssh/id_example_dsa
  Compression yes

You can find more information about the options available and the general format of the file in the man:

man ssh_config

Conclusion

SSH aliases can dramatically ease your life. Use them!

Until next time, Cheers!