Stories From The Field - The moocode blog

Build a Simple Twilio Customer Support Line in 10 minutes

2011-10-28T10:36:53+01:00

Twilio recently launched in the UK supporting 0800 (freephone) and 0207 (central london) numbers so I thought I'd have a go at setting up a customer support line for my company. This post walks you through a simple implementation.

Setting up

If you want to follow along with this example you'll need a free demo Twilio account, free Heroku account, git and the sinatra and heroku ruby gems.

Hello Moocode

To test the setup we'll create a very simple implementation and deploy it. This implementation will just say hello and then hang up.

In a twilio folder I created app.rb with the following:

require 'rubygems'
require 'sinatra'

get '/hello' do
  response =<<EOF
<Response>
  <Say>Hello from moocode</Say>
</Response>
EOF
end

To get this to run on Heroku we also need two other files, a config.ru rack configuration file

require './app'
run Sinatra::Application

and Gemfile so Heroku knows it needs to load the sinatra gem.

source 'http://rubygems.org'
gem 'sinatra'

If you want to test this locally I recommend using thin. You can start the application with thin -R config.ru start.

We should be ready to deploy this to Heroku now, but first we will initialise a git repository and add all the files

git init
git add .
git commit -m 'initial commit with hello twilio implementation'

now create the application on Heroku and do our first push

heroku create
git push heroku master

Heroku will give you your application url in response to the create command so you can now test it out it should be something along the lines of

http://superior-scallops-2020.heroku.com/hello.

Assuming that is all working, we add that URL to the sandbox Voice URL in your Twilio account, make sure to set the method to GET instead of POST. You can now try it out using the twilio client and you should hear your welcome message!

There is a lot of good information in the request from Twilio that we can use in our system implementation like what number was called and what country and number the person is calling from.

Adding some meat

Everything is now working end-to-end so we can focus on the implementation of our support system. The goal is to have Twilio call my support phone number during normal UK business hours and ask the user to leave a message out of hours.


get '/start' do
  case Time.now.utc.hour
    when 9..19 then redirect '/support'
    else redirect '/voicemail'
  end
end

get '/voicemail' do
  response =<<EOF
<Response>
  <Say>
    Welcome to moocode. Sorry our support team is not available.  
    Please leave a message including your phone number or email address 
    after the beep.
  </Say>  
  <Record action="/recording" />
</Response>
EOF
end

get '/support' do
  response =<<EOF
<Response>
  <Say>Welcome to moocode. Please wait while we connect you.</Say>
  <Dial>+440123456789</Dial>
</Response>
EOF
end

post '/recording' do
  recording_url = params['RecordingUrl']
  # email the recording url to the support team via sendhub.net ;)
  response =<<EOF
<Response>
  <Say>Thank you.  We'll be in touch shortly.  Goodbye.</Say>
  <Hangup/>
</Response>
EOF
end

Don't forget to update the Voice URL in your Twilio account to point to /start and try out your new Customer Support switchboard!

The main commands in use there are Dial and Record. You will need to update the telephone number to your own. We pass in the action to Record so it will post the recording URL to our server and we can then send that on to the support team, or store it in our customer database.

If you want to go live with your new system you just need to purchase a telephone number (from $1/month!) and point it at your URL.

Further ideas

This is a simple yet effective implementation but we could add a lot more logic to the example like redirecting the user to a different support team based on the country they are calling from, calling a fallback support number if the first one doesn't answer and call different people depending on the time of day if you have a support team in a different time zone.

You could also improve your support process by looking up the customer by their phone number and present that information to the support team.

This example just touches the surface of what is available via the Twilio API. It's quite fun to play with so let me know below if you do anything great.

Code Your Own Multi-User Private Git Server in 5 Minutes

2011-09-24T16:19:05+01:00

Following on from last weeks post about Simple Two Factor SSH Authentication this post shows you how to use the same SSH trick to create a multi-user private git server. I believe the principles here can also be applied to mercurial or subversion.

I was recently working on a client project that we converted to git, we hired an agency to work on the front-end for the project and they had four users that needed access. I didn't really want to create them individual accounts on the server so I started thinking how I could securely manage multiple-user access to a git repository running under a single git user without giving them shell access.

After a bit of research I identified two possible candidates gitosis and gitolite but they seemed overkill for what I was trying to achieve.

Setting up the environment

We'll assume we have a git user on the server and create a test repository in the home directory

git init --bare testing.git

Now we need to add my SSH key in authorized_keys with rw (read-write) permissions

command="/usr/bin/gitserve richard rw",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa AAA...zzz me@example.com

The first part of that line holds all the magic, firstly we will be invoking the script /usr/bin/gitserve with two parameters, a user identifier and the permissions that anyone authorising with this key has.

Then there are some extra options to disable port-forwarding, X11 forwarding, agent-forwarding, and finally deny the user access to a shell (we'll only be executing git commands for this user).

The implementation

I've chosen to implement this in ruby but any language would do. Put the following in /usr/bin/gitserve

#!/usr/bin/env ruby

# user and permissions are passed from authorized_keys

user = ARGV[0]
permissions = ARGV[1]
command = ENV['SSH_ORIGINAL_COMMAND']
abort unless user and permissions and command

# check the supplied command contains a valid git action 

valid_actions = ['git-receive-pack', 'git-upload-pack']
action = command.split[0]
abort unless valid_actions.include? action

# check the permissions for this user

abort "read denied for #{user}" unless permissions =~ /r/
abort "write denied for #{user}" if action == 'git-receive-pack' and permissions !~ /w/

STDERR.write "user #{user} authorized\n"

# user made a valid request so handing over to git-shell

Kernel.exec 'git', 'shell', '-c', command

When you execute git against a remote SSH repository it uses one of two commands git-receive-pack or git-upload-pack so we first verify those commands since we don't want our git users to be able to execute anything else.

Next we check what permissions the user has, read-only (r) or read-write (rw) and grant access depending on what action they are trying to perform.

Finally if the user is valid we pass execution to git-shell with the original command.

Testing it out


$ git clone git@myserver:testing.git
Cloning into testing...
user richard authorized
warning: You appear to have cloned an empty repository.
$ cd testing
$ touch README
$ git add .
$ git commit -m 'added readme'
$ git push origin master
user richard authorized
Counting objects: 3, done.
Writing objects: 100% (3/3), 213 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@myserver:testing.git
 * [new branch]      master -> master

That seems to work! We can now easily add more users with read-only or read-write permissions by just adding a new line to the authorized_keys file with the corresponding users public key.

Wrapping up

This solution is very simple and works for my scenario. It doesn't allow you to decide per-repository who has access to what but since you know which user has authorized in the script and what repository they are trying to access (in the command) it would be trivial to add support for per-repository access.

That's all for now, if you have any tips and tricks please leave them in the comments below and if you're looking for a great hosted private git solution please check out repodrop.

Simple Two-Factor SSH Authentication with Google Authenticator

2011-09-22T17:24:37+01:00

In a two-part post I'm going to show you some tricks you can do with SSH logins. This post covers setting up two-factor SSH authentication with the Google Authenticator app.

I was recently getting some servers in shape so I can pass the Payment Card Industry standards questionnaire and one requirement was two-factor authentication access to the server. I queried whether SSH key + passphrase was acceptable but didn't get a clear answer so I figured I'd explore setting up another authentication factor myself, plus it piqued my interest.

After a bit of research I found it was possible using a PAM module but it doesn't work along with SSH key authentication (only password authentication) and I only use SSH key logins for my servers.

The magic

I wanted to find the simplest method of implementing this so I started looking at what we can do with SSH itself. There is an option in the authorized_keys file that allows you to run a command when a user authorizes with a particular key eg.

command="/usr/bin/my_script" ssh-dsa AAA...zzz me@example.com

The command="..." part invokes a different command upon key authentication and runs the /usr/bin/my_script instead. Now we've got a starting point to work on the Google Authenticator logic.

UPDATE A number of comments on hacker news suggest the use of ForceCommand in sshd_config to apply this globally.

Simple implementation

I've chosen ruby to implement this simple example but in theory you could use anything you want. This is a naive implementation but it will prove the concept. You're going to need the rotp library as well for this to work gem install rotp.

We put the following in /usr/bin/two_factor_ssh

#!/usr/bin/env ruby
require 'rubygems'
require 'rotp'

# we'll pass in a secret to this script from the authorized_keys file
abort unless secret = ARGV[0]

# prompt the user for their validation code

STDERR.write "Enter the validation code: "
until validation_code = STDIN.gets.strip
  sleep 1
end

# check the validation code is correct

abort "Invalid" unless validation_code == ROTP::TOTP.new(secret).now.to_s

# user has validated so we'll give them their shell

Kernel.exec ENV['SSH_ORIGINAL_COMMAND'] || ENV['SHELL']

The secret is in Kernel.exec which, upon successful validation, replaces the two_factor_ssh script process with the original command the user was attempting or their default shell so it is a completely seamless experience from that point on.

Generating the secret

We need to generate a secret token that is shared between the Google Authenticator app and the server.

Here's a little script that will spit out a new token and a link to a QR code that can be scanned into the Google Authenticator application.

#!/usr/bin/env ruby
require 'rubygems'
require 'rotp'

secret = ROTP::Base32.random_base32
data = "otpauth://totp/#{`hostname -s`.strip}?secret=#{secret}"
url = "https://chart.googleapis.com/chart?chs=200x200&chld=M|0&cht=qr&chl=#{data}"

puts "Your secret key is: #{secret}"
puts url

Running this produces:

Your secret key is: 4rr7kc47sc5a2fgt
https://chart.googleapis.com/chart?chs=200x200&chld=M|0&cht=qr&chl=otpauth://totp/myserver?secret=4rr7kc47sc5a2fgt

We can scan the QR code directly into Google Authenticator and then update our authorized_keys file as follows:

command="/usr/bin/two_factor_ssh 4rr7kc47sc5a2fgt" ssh-dsa AAA...zzz me@example.com

That should do it!

UPDATE @js4all has identified a security problem with a certain version of OpenSSH which prints out the command (including parameters!) in the debug when connecting. To work around this problem instead of using the secret above, pass through an identifier and then put the security key inside the script and perform a lookup there.

Testing it out


[richard@mbp ~]$ ssh moocode@myserver
Enter the validation code: wrong
Invalid
Connection to myserver closed.
[richard@mbp ~]$
[richard@mbp ~]$ ssh moocode@myserver
Enter the validation code: 410353
moocode@myserver:~$

Great, that seems to work as expected.

Wrapping up

I've got a slightly more involved example that adds in support for 'remember me' by IP address for a fixed period of time so you don't have to reach for the phone on every single login from the same IP.

The extended example also does some primitive logging but I'd like to add in a better auditing system (another PCI compliance requirement) as this would allow us to know which key is used to log into the server and whether they validated.

We should also probably have a fallback mechanism (a master key or 5 one-time codes like Google does) so we don't inadvertently lock ourselves out of the server.

If you have any tips or ideas please leave them in the comments below and if you liked this post follow us on twitter so you'll get notified about next weeks follow-up post.

Comments on Hacker News

Launch: RepoDrop - Private Git Repository Hosting

2011-09-03T15:38:02+01:00

We're proud to announce RepoDrop - Private Git Repository Hosting our first paid product! RepoDrop offers developers and designers everywhere a simple, secure, affordable service to host their Git repositories remotely.

We have tried to keep it as simple as possible while still providing enough features but if there is anything else you need please let us know in the comments below.

We've also tried to keep the subscription plan as simple as possible, you pay only for storage space, can host an unlimited number of repositories and can share each repository with an unlimited number of collaborators.

Try it free for 14 days with no commitment.

Using the Google Authenticator App with Rails

2011-06-29T10:18:17+01:00

Following on from last weeks post on Two-factor Authentication with Rails this post adds support for the Google Authenticator app for Android, iPhone and Blackberry. The full source code is available on github and there is now a live demo available on heroku.

Integration

The integration is very straight-forward thanks to the ROTP library. Add it to your Gemfile.

gem 'rotp'

The authentication mechanism works by having a per-user shared key between your web application and the app on the phone so we're going to add one to our User model:

add_column :users, :auth_secret, :string

And assign a random base32 string when you create a new User.

class User < ActiveRecord::Base
  ...
  before_validation :assign_auth_secret, :on => :create
  ...
  def assign_auth_secret
    self.auth_secret = ROTP::Base32.random_base32
  end
end

Now we can present that auth_secret to the user when they sign up and they can add it to Google Authenticator. The app can also scan a QR Code to save the user having to enter the secret manually. This helper uses Google Charts to generate the QR Code:

module SessionsHelper
  def google_authenticator_qrcode(user)
    data = "otpauth://totp/two_factor_demo?secret=#{user.auth_secret}"
    data = Rack::Utils.escape(data)
    url = "https://chart.googleapis.com/chart?chs=200x200&chld=M|0&cht=qr&chl=#{data}"
    return image_tag(url, :alt => 'Google Authenticator QRCode')
  end
end

All that is left to do is validate the code in our Session model when the user enters it.


def validates?
  return true if self.validation_code == ROTP::TOTP.new(self.user.auth_secret).now.to_s
end

That's it. You can try it out on the demo site.

Two-factor Authentication with Rails

2011-06-24T12:13:06+01:00

Web application security is a hot topic at the moment with the recent high-profile hacks and publication of user email/password lists so any measures that can be taken to further protect users should be considered depending on the nature of your web application. This post walks through setting up two-factor authentication for a Rails application.

Two-factor authentication helps to protect users by requiring two pieces of information to log in; one that the user knows (their password usually) and another that is random, unique, time-sensitive and only generated on demand.

Typically this second code is sent to the user in real-time either by Email, SMS, Automated Voice or by the use of a 'code generating' mobile application (like Google Authenticator). Using this mechanism ensures that someone who has access to your login and password still can't use the web application unless they also have access to your secondary authentication mechanism (email account or mobile phone in this case).

This implementation uses Email for simplicity but adding other delivery mechanisms is very easy to do. Rather than requiring the user to perform two-factor authentication every time they log in we use a permanent cookie to remember the 'validation' of the client for 30 days. The source code for the full application is available on github.

The implementation

First we're going to create a table to store each unique client (or device) the user logs in from and some other useful information to identify the client and track validation.

rails g model session

Then complete the migration like this:

class CreateSessions < ActiveRecord::Migration
  def change
    create_table :sessions do |t|
      t.references :user
      
      t.string   :client_id, :null => false
      t.string   :ip_address, :null => false
      t.string   :user_agent
      t.integer  :login_count, :null => false, :default => 0
      
      t.string   :unique_key
      t.datetime :unique_key_generated_at
      
      t.integer  :confirmation_failure_count, :null => false, :default => 0
      t.datetime :client_confirmed_at
      t.datetime :authenticated_at
      t.datetime :finished_at
      
      t.timestamps
    end
    add_index :sessions, [:user_id, :client_id], :unique => true
  end
end

Our ApplicationController adds some filters to control access to your resources, we're using a simple User authentication model here but this should fit in nicely with any user authentication you're using:


class ApplicationController < ActionController::Base
  protect_from_forgery
  helper_method :current_user, :current_session
  before_filter :user_required, :session_required, :confirmed_session_required
  
  private
    def current_user
      @current_user ||= User.find_by_id(session[:user_id]) if session[:user_id]
      session[:user_id] = nil unless @current_user
      @current_user
    end
    
    def current_session
      @current_session ||= Session.find_by_id(session[:user_session_id])
      session[:user_session_id] = nil unless @current_session
      @current_session
    end
    
    def user_required
      redirect_to :new_session unless current_user
    end
    
    def session_required
      redirect_to :track_sessions unless current_session
    end
    
    def confirmed_session_required
      redirect_to confirm_session_url(current_session), 
        :alert => "This device is not recognised" unless current_session.confirmed?
    end
end

The authentication flow is forced via the SessionsController#track method which is the meat of the implementation:

def track
  Session.perform_housekeeping
    
  # assign a new id for this client if it isn't recognised
  client_id = cookies.signed[:_client_id] ||= UUIDTools::UUID.timestamp_create.to_s
  @session = Session.find_or_initialize_by_user_id_and_client_id(current_user.id, client_id)
    
  # check if we have already authenticated this session
  redirect_to params[:next] || :root and return if @session == current_session
    
  @session.update_attributes(
    :ip_address => request.remote_ip,
    :user_agent => request.user_agent,
    :client_id =>  client_id,
    :login_count => @session.login_count + 1,
    :authenticated_at => Time.now.utc,
    :finished_at => nil
  )
    
  # sign up session is trusted
  @session.confirm! if session[:first_visit]
    
  session[:user_session_id] = @session.id
    
  # remember this client
  cookies.permanent.signed[:_client_id] = {
    :value => @session.client_id, 
    :secure => Rails.env.production? ? true : false,
    :httponly => true
  }
    
  @session.send_confirmation_code unless @session.confirmed?
    
  flash.keep # pass on any flash messages
  redirect_to params[:next] || :root
end

Unless the session is valid, the unique token is sent via email and the user is forced to a page where they can enter the code.

The SessionController#validate method also tracks the number of failed attempts to enter the validation code to prevent brute-force attacks and if the user exceeds 3 attempts a new code is generated and sent. We could potentially implement a delay here as well if required.

def validate
  @session.validation_code = params[:session][:validation_code]
  if @session == current_session and @session.validates?
    @session.confirm!
    redirect_to :root, :notice => 'This device is now validated'
  else
    @session.increment! :confirmation_failure_count
    if @session.too_many_failures?
      @session.send_confirmation_code
      flash[:alert] = "Too many validation failures. We've sent you another code."
    end
    redirect_to :action => :confirm
  end
end

Finally the Session model contains all the business logic for managing the session, an abbreviated version is below:



class Session < ActiveRecord::Base
  attr_accessor :email, :password, :validation_code
  
  validates_uniqueness_of :client_id, :scope => :user_id
  belongs_to :user
  default_scope :order => 'authenticated_at DESC'
  
  scope :confirmed, where('client_confirmed_at IS NOT NULL')
  scope :expired, lambda { confirmed.where("client_confirmed_at < ?", Time.zone.now - 30.days) }  
   
  def validates?
    return false if self.unique_key_generated_at < (Time.now.utc - 10.minutes)
    return false unless self.validation_code == self.unique_key
    return true
  end
  
  def send_confirmation_code
    assign_unique_key!
    self.update_attribute :confirmation_failure_count, 0
    # Could also easily do SMS or Automated Voice call here
    Mailer.session_confirmation(self).deliver
  end
  
  def self.perform_housekeeping
    # invalidate any expired sessions (30 days old)
    self.expired.update_all :client_confirmed_at => nil
  end
  
  private
    def assign_unique_key!
      assign_unique_key
      self.save!
    end
    
    # Assigns a time-sensitive random validation key
    def assign_unique_key
      # generate zero padded random 5 digits
      self.unique_key = ActiveSupport::SecureRandom.random_number(10 **5).to_s.rjust(5,'0')
      self.unique_key_generated_at = Time.now.utc
    end
end

Wrapping up

This implementation is relatively straight-forward and not too intrusive for the end-user. An added bonus is that the user can review and invalidate any unused sessions.

I thought about trying to package this up into a gem but I find that with this kind of thing a one-size-fits-all approach usually doesn't work. I hope it helps anyone else looking to do two-factor authentication and welcome any feedback on this implementation.

The next post continues with this theme and adds support for Google Authenticator.

Deploying a Rails 3.1 application to production

2011-05-26T13:32:03+01:00

There is a big change to the handling of assets in Rails 3.1 compared to previous versions called the asset pipeline and this can have an effect on your production deployments, more so than any previous Rails update I can remember. Here is a list of the gotchas I came across and how to fix them.

My deployment tools consist of capistrano, nginx and unicorn so the solutions listed here may be specific to those but you'll get the general idea if you are using something else. Information is correct as of the Rails 3.1.0.rc1 release, I'll update it if further releases change any of this (updates are now inline).

Javscript Runtime

The thing that has surprised me most is the requirement to have a javascript runtime present on the production server. To me this isn't a small thing, it's a rather large new dependency that I have to manage.

Update: Further release candidates have made the javascript runtime optional in production if you precompile assets before you deploy. If you want to precompile as part of your deployment process (like I do) then you will still need to follow these instructions.

If you deploy your application without a suitable runtime you are likely to see the following error message:

Could not find a JavaScript runtime. 
See https://github.com/sstephenson/execjs for a list of available 
runtimes.

To fix this, I chose one of the available ruby-wrapped runtimes, therubyracer, and configured it in my Gemfile:

group :production do
  ...
  gem 'therubyracer'
end

Be prepared to sit back and wait while it compiles and installs when you do your first deployment.

Update: If you're deploying to Heroku please read this article.

Asset Compilation

In production you generally won't want to use the asset pipeline directly, instead you will run the assets:precompile rake task so your webserver can serve the generated assets. My capistrano configuration in /config/deploy.rb is:

after 'deploy:update_code' do
  ...
  run "cd #{release_path}; RAILS_ENV=production rake assets:precompile"
end

This will copy all the assets to /public/assets/ and encode the MD5 hash of the contents into the path name. This is a great improvement as far as caching is concerned but has some side-effects (see below).

One interesting thing I haven't discovered yet is what happens if you have two assets with the same name in different folders? I guess one of them will win and the other will be ignored, or it will throw an error.

CSS Assets

A frustration I had with the new asset pipeline code and the production environment is inconsistent behaviour. One thing I liked about the asset pipeline in development is that it will 'find' the assets for you. In your CSS you can specify something like:

body { 
  background: #00ff00 url('rails.png') no-repeat fixed center;
}

As long as your rails.png asset exists somewhere in the /app/assets/ folder it will be served.

However, when you run the assets:precompile rake task as described above the asset you were referencing now becomes:

/public/assets/rails-9c0a079bdd7701d7e729bd956823d153.png

but the compiled CSS isn't changed accordingly to point to this generated asset. I thought this was a bug but apparently it is expected behaviour. You can see the comments on this rails ticket.

In short, you should either not use the asset pipeline (ie. reference your images directly using /images/rails.png as before) or use erb to generate your asset paths by appending .erb to the CSS filename and using the asset_path helper as follows:

body { 
  background: #00ff00 url(<%= asset_path 'rails.png' %>) no-repeat fixed center;
}

Asset Caching

If you've got this far and all your assets are compiled and are being referenced correctly you'll want to update your web server configuration to take advantage of this new cachability. My nginx configuration was updated with:


location ~* ^/assets {
  expires max;
  add_header Cache-Control public;
  break;
}

Other Bits & Pieces

At the time of writing I wasn't able to deploy with the latest version of rake. I was getting undefined method `task' errors in production so ended up hard coding gem 'rake', '0.8.7' in my Gemfile. I expect this will get fixed before the final release.

Update: The rake issue above is now fixed in the latest versions of Rake and Rails.

I'm also having some trouble with the include declaration of javascript assets. I ended up just including all of them and I believe some of them may be getting included more than once, but I'm assuming it is my fault.

That's all for now, I hope you find this information useful when deploying your Rails 3.1 applications. Please feel free to leave your war stories in the comments below.

Comments on Hacker News
Comments on Reddit