code

Storing IP addresses as integers

Fri, 25 Sep 2009 21:44:45 +0000

Tying long sessions to an IP address is a good way to ensure some security for your users. But inevitably, you’ll want to store that IP. While you could, of course, store it as a string, such as "123.125.126.127" (called a “dotted quad”), some developers might prefer saving some space and storing it as a compact integer. There’s a right way and a wrong way to do this. The wrong way is as follows:

'12.34.56.78'.sub('.', '').to_i

This is a bad idea because two IP addresses can result in the same integer: 12.34.56.78 and 123.4.56.78 are both valid IPs. The correct way is as follows:

'12.34.56.78'.split('.').collect(&:to_i).
              pack('C*').unpack('N').first

To understand this seemingly obfuscated train wreck, let’s look at the data in this chain after each method call:

'12.34.56.78'.split('.'). #=> ['12', '34', '56', 78']
  collect(&:to_i).        #=> [12, 34, 56, 78]
  pack('C*').             #=> "\f\"8N"
  unpack('N').            #=> [203569230]
  first                   #=> 203569230

Many developers are unfamiliar with pack and unpack. These allow you to create and extract data into and out of binary-packed strings. In the third method call, you’ll see the result is "\f\"8N". This strange-looking string is really just 4 bytes of data, the numbers 12, 34, 56, and 78 in binary form, put into a string. We then unpack that 4-byte string into a 4-byte integer (with network byte order).

This is also the same problem solved by the C library method inet_aton, which is implemented as part of Ruby’s IPAddr class. Thus, there is a much simpler alternative:

require 'ipaddr'
IPAddr.new('12.34.56.78').to_i   #=> 203569230

But what fun is that!

PS. Yes, you can have multi-line method chains simply by leaving the dot at the end of the line. Ruby then knows to look for a method call on the next line. Both snippets are valid Ruby! You should, of course, indent appropriately.

Snow Leopard: Upgrading for Rails Developers

Sat, 29 Aug 2009 00:36:26 +0000

Upgrading to Snow Leopard is not as easy as Apple would lead you to believe; at least not if you’re a Rails developer. Here are a few select errors you might have encountered:

uninitialized constant MysqlCompat::MysqlRes
dlopen(/Library/Ruby/Gems/1.8/gems/nokogiri-1.3.3/lib/nokogiri/nokogiri.bundle, 9): no suitable image found

These errors seems to stem because OS X is now fully 64-bit, and unfortunately, all your compiled libraries are 32-bit. Whoops!

So, let’s fix them.

Important: These instructions are tailored for Intel 64-bit machines, as only 64-bit machines should have these issues. Any Intel Core 2 machines including recent Macbooks, Macbook Pros, and iMacs, and all Mac Pros should work fine with these instructions.

Install 64-bit MySQL

To fix the MySQL problem, you need install the x86_64 version of MySQL. This will uninstall your old MySQL version, but will not migrate your databases. In order to migrate your data, first make sure MySQL is not running, then:

$ sudo mv /usr/local/mysql/data /usr/local/mysql/data.default
$ sudo mv /usr/local/mysql-oldversion/data /usr/local/mysql/data

Start up MySQL and your databases should be in tact.

Reinstall the MySQL gem

Now that you have the correct version of MySQL, you need to reinstall the latest MySQL gem. Make sure to uninstall your current MySQL gem first:

$ sudo gem uninstall mysql
$ sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Re-install MacPorts

To fix nokogiri (and other gems with external dependencies), you’ll need to fix your Macports install. Unfortunately, the only way to fix Macports is to completely reinstall it. You need to completely delete (or at least move) your /opt/local directory. Once that is done, download and install the latest version of MacPorts for Snow Leopard.

You’ll probably want to install two things pretty quickly: libxml2 for nokogiri and git-core:

$ sudo port install libxml2
$ sudo port install git-core +bash_completion +doc

Re-install nokogiri (and other gems)

Any gem that has a compiled component will need to be reinstalled. nokogiri is just one of the libraries; others include ruby-debug, ruby-prof, and a lot of others. To fix them, just run this command:

$ sudo gem pristine --all

Symbol vs String performance

Sun, 15 Feb 2009 23:00:20 +0000

A more interesting metric to this discussion is the use of strings versus symbols. Fortunately, these types of discussions can easily be solved by benchmarks:

Results under Ruby 1.8.6:

                           user     system      total        real    less no op
String instanciation   9.050000   0.010000   9.060000 (  9.057162)   5.921219
Symbol use             5.130000   0.000000   5.130000 (  5.131844)   1.995901
new String#to_sym     14.550000   0.020000  14.570000 ( 14.567466)   11.431523
const String#to_sym   11.960000   0.010000  11.970000 ( 11.967217)   8.831274
String const lookup    6.350000   0.010000   6.360000 (  6.358697)   3.222754
Symbol const lookup    6.400000   0.010000   6.410000 (  6.416395)   3.280452
No op                  3.130000   0.010000   3.140000 (  3.135943)   n/a

Results under Ruby 1.9.1:

                           user     system      total        real    less no op
String instanciation   9.170000   0.010000   9.180000 (  9.174451)   4.736163
Symbol use             4.920000   0.010000   4.930000 (  4.930031)   0.491743
new String#to_sym     18.080000   0.030000  18.110000 ( 18.087386)   13.649098
const String#to_sym   13.940000   0.030000  13.970000 ( 13.954099)   9.515811
String const lookup    4.920000   0.000000   4.920000 (  4.927497)   0.489209
Symbol const lookup    4.910000   0.020000   4.930000 (  4.921151)   0.482863
No op                  4.440000   0.000000   4.440000 (  4.438288)   n/a

What do these results mean? Well, first you need to subtract out the “no op” results from all the others, which I’ve added as a column above. We can now see that string instantiation takes about 90 nanoseconds, which means about 11000 string instantiations per millisecond. Are symbols faster? Considerably so. But the real lesson here is that these numbers are so small that no one in there right mind should spend time worrying about them.

So please, use symbols when you should use symbols, and otherwise use strings.

Garbage collection thresholds in Ruby

Sun, 15 Feb 2009 22:39:31 +0000

I’ve put together a somewhat extensive collection of scripts and results from looking at how often objects get garbage collected in Ruby 1.8 and Ruby 1.9. Performance metrics are also provided running the script under various different environments, which I found quite fascinating. irb, for instance, is ridiculously slow. Granted, the vast majority of these scripts times are spent performing the very inefficient ObjectSpace calls, so the raw numbers should be taken with a grain of salt. The metrics are only interesting for comparison, and are of questionable use.

This was instigated by a String versus Symbol discussion in #ruby on Freenode. The individual was primarily worried about memory usage, and these scripts confirm that strings will get garbage collected often and quickly.

One thing to note is how Ruby 1.8 changes its thresholds for garbage collection under each environment, which may be based on the amount of objects in global space that it cannot garbage collect. Under Ruby 1.9, the results are bit more consistent, although this may be due to better metrics being available in Ruby 1.9.

Now shipping internationally

Sun, 15 Feb 2009 22:23:25 +0000

By popular demand, we’ve added shipping support for 32 additional countries to RubyRags:

Our full country list is: Argentina, Australia, Austria, Belgium, Brazil, Canada, Denmark, Finland, France, Germany, Hungary, Iceland, India, Ireland, Italy, Japan, Luxembourg, Malaysia, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Russian Federation, Singapore, Slovakia, Spain, Sweden, Switzerland, UK, USA

If your country isn’t on the list, contact us and let us know of your dissent!

Remembering lighttpd, nginx, and the Internet as a pipe

Sun, 15 Feb 2009 09:44:37 +0000

We asked ourselves a year ago if lighttpd 1.5.0 was vaporware. It seems that was nearly true. At that time, nginx, apache’s mod_proxy_balancer, and haproxy were flourishing as Rails proxy solutions. The more recent introduction of Phusion Passenger (mod_rails) and Ruby Enterprise Edition, both excellent, free, and open-source products, has now driven most deployments (including our own) away from proxying altogether.

There is—or was—generally two counter-arguments to Passenger. The first is the stability and performance argument, which is well understood and has been discussed at length. I believed it was well summarized by Engine Yard’s discussion on the topic. But this is not the argument I’m not interested in.

My early argument against mod_rails was complexity: first, by using Apache instead of a smaller, simpler web server, and second, using a platform that I don’t comprehend and can’t debug. This argument relates to an old article I wrote: that Rails applications—and applications in general—should act like a pipe. (The original article, by James Duncan Davidson, is so old its only accessible through archive.org). My presumption was that the same argument should apply to a 3rd-party mod_rails solution: I don’t know anything about the app server, so if I have problems, I’m screwed. How is this better than FastCGI?

It is ironic, then, to note the success of Passenger in light of Rails’ history with FastCGI. The reason, as it turns out, basically comes down to two answers:

Passenger is easy to set up under Apache, and requires less configuration than under nginx
Passenger always works, in my experience, and thus debugging problems is non-existent.
As a bonus, if you’re using something like monit to ensure your app stays up, monitoring apache2 is a lot easier than individual mongrel processes.

So, better products are better, and users (including myself) will flock to them once they realize it.

The Future

I do wonder, though, whether history will continue to repeat itself in this regard. Where will the next improvements be? What will be our theory (or justifications, as it seems to turn out) behind those improvements? Personally, I’ve been eying threaded solutions for a long time, and given that mod_rails is a multi-process solution, perhaps threads will move things forward in the future. However, given that Ruby 1.9 is still limited by the Global Interpreter Lock, we might not see a threaded answer for a while. The closest we’ll come in the short term is JRuby. Could it be possible that JRuby is the future? Only time will tell.

ls, colors, and Terminal.app

Sat, 14 Jun 2008 21:12:21 +0000

This isn’t a Ruby thing but many of us spend a lot of time in Terminal.app, and I suspect few of you have taken the time to both enable colors and change your LSCOLORS, the setting which affects what colors ls uses when in color mode.

Enable Colors in ls

In order for ls to use colors at all, you need to set up an alias to turn colors on. To do this, open (or create) .profile file in your home directory using your favorite text editor and add:

alias ls="ls -G"

Now open a new Terminal window and type ls. You will see colors, hurray!

Make Colors Linux-like

If you’re used to Linux-like colors, you will appreciate this setting. This is what I use and it works particularly well on dark Terminal backgrounds (I use the “Pro” theme). I also check off “Use bright colors for bold text” under Terminal > Preferences > Settings. Again, add this to your .profile:

export LSCOLORS="ExGxBxDxCxEgEdxbxgxcxd"

Customize Your Colors

The values in LSCOLORS are codes corresponding to different colors for different types of files. The letter you use indicates which color to use, and the position in the string indicates what type of file should be that color. Each color comes in pairs – a foreground color and a background color. Here is a list of color values:

a = black
b = red
c = green
d = brown
e = blue
f = magenta
g = cyan
h = grey
A = dark grey
B = bold red
C = bold green
D = yellow
E = bold blue
F = magenta
G = cyan
H = white
x = default

And here is a list of the positions in LSCOLORS:

directory
symbolic link
socket
pipe
executable
block device
character device
executable with setuid set
executable with setguid set
directory writable by others, with sticky bit
directory writable by others, without sticky bit

The default is “exfxcxdxbxegedabagacad”, which indicates blue foreground with default background for directories, magenta foreground with default background for symbolic links, etc.

Interview on the Ruby on Rails podcast.

Sat, 03 May 2008 02:29:31 +0000

The interview I did with the incredibly gracious and awesome Geoffrey Grosenbach (of Peepcasts) on the official Ruby on Rails podcast just went up.

I talk a little bit about consulting at Norbauer Inc, a bit about RubyRags, and I spend a bit of time kicking social networks in the nuts.

Incidentally, I’m proud to announce that we’re now selling Peepcode shirts at RubyRags:

Alternative Ruby Implementations

Mon, 28 Apr 2008 20:17:03 +0000

Sorry to just link drop on ya’ll, but if you haven’t seen it, Charles Nutter of the JRuby team has published a writeup titled Promise and Perils for Alternative Ruby Impls. It provides an excellent analysis of 1.8, 1.9, JRuby, Rubinius, and the other Ruby interpreter projects out there, if only slightly biased for JRuby (understandably). If you have an interest in the future of Ruby, it is a recommended read.

Rails 2.0 development finally meets Web 2.0

Sat, 19 Apr 2008 04:16:11 +0000

I know I am late to the show, but I’d just like to say it is about time that Rails move off of Trac – with the move to Lighthouse and Github, Rails development itself is now hosted on awesome Rails-based services. I know such migrations are not easy for such a large project so I just want to say thanks to the Rails team, and nice choices!

Strange analogy

Thu, 17 Apr 2008 15:36:32 +0000

Creating software is like eating a grapefruit (Randall Munroe’s favorite fruit). Sure, you can rush through it and you’ll get some enjoyment out of the grapefruit bits, but if you take your time and patience, you will get much more out of it. Like all things, you reap what you sow.

Gitsplosion

Sat, 12 Apr 2008 22:44:20 +0000

So Git has this charming habit of leaving directories intact when you use it to delete their content. If you try to delete, say, vendor/rails, it will therefore leave a barren ghost-town of EDGE directories intact.

This will cause the following, not-particularly-elucidating error whenever you try to do anything:

(in /Users/ryan/Sites/app_name) rake aborted! no such file to load -- /blah/blah/lib/initializer /Users/ryan/Sites/app_name/rakefile:4 (See full trace by running task with --trace)

The solution is just a simple, friendly:

rm -rf vendor/rails

Then you’ll be able to freeze again, and whatnot.

UPDATE I have submitted a patch to Rails to make it less dumb about this (and actually check that Rails really exists rather than just checking for its directories.) If you think this is a worthwhile patch, please go put a +1 in the ticket comments.

Then once we can set it to “Verified,” maybe it’ll be accepted in three years, just in time for Git to be supplanted by some other new nerd bauble.

Yet another Ruby framework and HTTP adapter

Sat, 12 Apr 2008 10:49:36 +0000

It’s an exciting time to be Ruby or Rails developer as there are so many new and exciting things going on – in fact, it’s a lot to keep tabs on. Let’s look at two issues: Ruby web frameworks and HTTP adapters.

So, web frameworks, some of these you’ve probably heard of:

Ruby on Rails – You’ve probably heard of this one.
Camping has been around for a while, and aims to be tiny and simple – great when you’re only building a few pages. Developed by the infamous why.
Merb was created by Ezra, which now has many of the core Rails features, but less of the frills. It’s fast and, best of all, its thread-safe.

But, oh, there are so many more:

Nitro – Been around a while now, but development stalled for a year or two.
Ruby Waves, which claims to be highly flexible.
Wuby, a tiny server+framework, which serves RHTML files, in a manner similar to PHP.
Sinatra, a REST-centric framework which uses a DSL to make life simpler.
Ramaze, which is also very flexible, supporting a variety of ORMs, template engines, HTTP adapters, etc.
Mack is brand new, claiming to be highly modular and agile, as well as speedy. Their site is down but their documentation is up.
Maveric
Vintage
Cerise – Not recently maintained (2006).
IOWA – Not recently maintained (2004).

I’m sure I still missed some. Moving on the HTTP adapters – these are the likes of WEBrick and Mongrel. It’s good to keep up on these as they can often mean a boost in performance for your app, often regardless of your framework, thanks to the common interface provided by the Rack library.

Ebb is the newest to the show, touting fantastic performance as most of it is written in C. It can be a little tough to set up. It has support for Rails; for other frameworks, you’ll have to roll your own interface.
Thin is quickly becoming the defacto standard (at least until ebb is further along)
Swiftiply and Evented Mongrel was the first to improve Mongrel’s performance.
Mongrel started it all.
Fuzed allows your Rails app to run with Yaws.
Passenger is “mod_rails” for Apache, which could be a big news. They are also working on Ruby Enterprise Edition, which will be launched soon.

And you have already learned and adopted git, right? Can’t fall behind the times…

git: revert (reset) a single file

Tue, 08 Apr 2008 07:38:10 +0000

This one is hard to find out there so here it is. If you have an uncommitted change (its only in your working copy) that you wish to revert (in SVN terms) to the copy in your latest commit, do the following:

git checkout filename

This will checkout the file from HEAD, overwriting your change. This command is also used to checkout branches, and you could happen to have a file with the same name as a branch. All is not lost, you will simply need to type:

git checkout -- filename

You can also do this with files from other branches, and such. man git-checkout has the details.

The rest of the Internet will tell you to use git reset --hard, but this resets all uncommitted changes you’ve made in your working copy. Type this with care.

Generating access keys

Sun, 06 Apr 2008 09:15:49 +0000

This is a somewhat obscure but very efficient way to generate access keys. Look ma, no loops!

Array.new(16) { rand(256) }.pack('C*').unpack('H*').first

This will generate a 32-length key, like b3512f4972d314da94380e1a70e6814a.

This may look strange unless you’ve used Array.new, Array#pack, and String#unpack many times in the past. In the gaming world, I tended to deal with binary data on a regular basis, so pack and unpack became commonly-used weapons in my repertoire. They allow you to operate on binary data – pack creates binary data (stored in a regular string) from an array of Ruby objects, and unpack takes binary data and creates Ruby objects in an array. If you think about it, you are dealing with binary data here: you’re creating a random 128-bit (16-byte) key which you want in hex-format, so using pack and unpack do make sense. The parameters (C* and H*, in this case), are formats which indicate how to process the data.

Very cool cats may want to create more-compact Base64 keys. Be aware that Base64 is case sensitive:

[ Array.new(16) { rand(256) }.pack('C*') ].pack('m').chop

This generates keys like Gv5CJ68ptOZKVRvAFdzGpg==.