ELC Code Feed

Friday late night hacking

Gregor Martynus — Fri, 06 Nov 2009 21:57:07 +0000

Pairing with Heineken on some cool stuff we will show you soon - live from Paris

Add Solr Searching to External Gem and Third Party Database

Nicholaus — Fri, 06 Nov 2009 00:49:09 +0000

Problem

Say you have a third party database that you need your rails applications to access. But of course you want it to be modular and use it again for other applications, so you write a gem to provide access to the non-rails standard database. Now you want to add in acts as solr to search across the database from your application.

Solution

Acts as solr is a rails plugin and requires a solr server instance be running and therefore its not necessarily correct to add in the acts_as_solr plugin into your gem library. The consuming applications should be responsible for indexing what they and and how they want it. So a common trick is used to add in the application specific functionality into the gem. Initializers are used to inject the desired functionality into the gems classes. Given you have your gem named third_party_db_gem, create a file in config/initializers named something along the lines of third_party_db_gem_extensions.rb. And put in some code like:

require 'third_party_db_gem'

ThirdPartyModel.class_eval do 
  include ActsAsSolr
  acts_as_solr :fields => [:name, :email, :phone, job]
end

Explanation

So first we had to require our gem, because our gem hasn't been loaded yet. Then we do a class_eval to inject our solr definition into one of the models. You would repeat that block for every class you wanted to index. Here's Jay Fields article more on class and instance eval.

batch finding composite primary keys [SOLVED]

Max Murphy — Thu, 05 Nov 2009 19:14:05 +0000

Issue with composite_primary_keys and find_in_batches. In short, they don't play nicely together.

Issue:

Rails 2.3+ find_in_batches will not work if your model has composite primary keys.

Solution:

I found this gist:
https://gist.github.com/105318/13fe7c42c95c820fadbad84d54e00680ce9d110f
which solved the issue. I just dropped it into a file in RAILS_ROOT/config/initializers to extend active record to be able to do batch finds on those models which use composite_primary_keys

[SOLVED] 64bit MySQL and 32bit Ruby: FAIL

David Palm — Thu, 05 Nov 2009 14:22:27 +0000

For the people out there using RVM to rock multiple Ruby versions, here's a gotcha with REE and Snow Leopard that took me hours to solve: if you happened to boot your mac in 32-bit mode when building ree with the oh so awesome:

rvm install ree

And then install MySQL 64bit and try to install the gem, ruby will positively hate you. It will also do it's best not to help you understand the issue whining about missing libraries and whatnot.

No, it's not a library issue. Your ruby is built for 32bit and the mysql libs are 64bit. I know for sure nokogiri has the same issue and I think many others as well.

What kernel version am I running?

~# uname -m
x86_64

~# uname -m
i386

To force 64bit mode, reboot holding down the "6" and "4" keys on the keyboard.

Recompiling your rubies is fun for sure, but if you get tired of it, here's how to make the 64bit switch permanent:

~# sudo vim /Library/Preferences/SystemConfiguration/com.apple.Boot.plist

Change the last key to arch=x86_64. You'll end up with:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Kernel</key>
        <string>mach_kernel</string>
        <key>Kernel Flags</key>
        <string>arch=x86_64</string>
</dict>
</plist>

To check if a lib is 64bit, run:

~# lipo -info /usr/local/mysql/lib/libmysqld.a 
input file /usr/local/mysql/lib/libmysqld.a is not a fat file
Non-fat file: /usr/local/mysql/lib/libmysqld.a is architecture: x86_64

What about my ruby version?

~# lipo -info ~/.rvm/ree-1.8.7-2009.10/lib/libruby-static.a 
input file /Users/david/.rvm/ree-1.8.7-2009.10/lib/libruby-static.a is not a fat file
Non-fat file: /Users/david/.rvm/ree-1.8.7-2009.10/lib/libruby-static.a is architecture: x86_64

~# lipo -info ~/.rvm/ruby-1.9.1-p243/lib/libruby-static.a 
input file /Users/david/.rvm/ruby-1.9.1-p243/lib/libruby-static.a is not a fat file
Non-fat file: /Users/david/.rvm/ruby-1.9.1-p243/lib/libruby-static.a is architecture: i386
</code

Looks like I'll have to nuke my Ruby 1.9 install too. Sigh.

Give us the fix already!

Ok ok. Here it goes:

# One
echo "rvm_archflags='-arch x86_64'" > ~/.rvmrc

Now rvm will now know what we want.

That's not enough though and we need to make sure REE doesn't build with tcmalloc (at least until the good people at Google solve the non-64bit-ness of it). Only thing is rvm as of v0.64 has a little bug stopping params to be passed along. The soon to be released v0.75 fixes this but for now you need head:

# Two
rvm rvm update --head

Now we're ready to reinstall REE:

# Three
rvm --force install ree --ree-options --no-tcmalloc

Many thanks to wayneeseguin for helping with this issue! Rvm is the best thing since sliced bread.

iPhone Tech Talk World Tour - San Jose

ELC — Mon, 02 Nov 2009 20:56:27 +0000

Last week I went out to San Jose, CA to go to Apple's iPhone Tech Talk World Tour!

Apple once a year holds 9 single day "Tech Talks" across the world. This year only 3 were in the US and I was lucky enough to get a spot at the San Jose meetup. The conference was focused on getting developers to use best practices in all regions of their applications. Apple's stance was even if the app is small, it might as well be designed as well as possible. The day started with a nice overview of the iPhone platform. They sell tons of these things and people seem to love the App store. No real news there. Nice overview of the creative apps in the store. Certainly got everyone's brains working for an exciting day. After looking over all the sessions I decided to spend my whole day in Room A which seemed to focus on the performance, threading and data management. The main tech point to take away from the conference was how threading is done on the iPhone. Apple's goal is to abstract away the complexities of classical threading. I went to an hour session on it and the only times I heard the words semaphore or lock was the speaker saying that they didn't exist anymore. The iPhone has may build in calls which are blocking calls. This means that when performed on the main thread functionality will halt until they are done. A very commonly used blocking call is initWithContentsOfURL:. This method is used commonly for XML parsing and grabbing images from the internet. Apple has created a solution for this that essentially automizes your thread management with a single line of code. Lets say for example I am creating an app that on launch parses an XML source. If I have a containing method called parseXML() that does all the parsing I can call this method on a managed thread simply by using.

NSOperationQueue *queue = [[NSOperationQueue alloc] init];
NSInvocationOperation *op = [[NSInvocationOperation alloc] initWithTarget:self selector:@selector(parseXML) object:nil];
[queue addOperation:op];
[op release];

That's it. Apple has made a general class called NSOperation that gives users the power to create their own automated threading classes. In this case I am using the built in NSIvocationoperation object that takes in a method and runs it on a secondary thread. Throw the operation into an operation queue and you are done. Very powerful stuff. Aside from the fun tech stuff it was a great opportunity to meet people in the developer community and share ideas.

Great Resource For Testing Regular Expressions

Brandon Trebitowski — Thu, 29 Oct 2009 16:43:10 +0000

Through my travels through the internets, I have stumbled upon a very useful resource. It is a site (written in rails of course) for testing your regular expressions. It has a very clean interface and uses fancy ajax to give you real time results about your regular expressions. The URL of the this site is http://rubular.com/

Not only does it provide real-time feedback on your regular expressions, it also will give you information about your matches. This is much more useful than trial and error in your code. Also, if you have an error in your regex, rubular will let you know about it.

A little rusty on your regex skills? No problem, the site even includes a box at the bottom detailing all of the most important regex strings. So be sure to check it out

Executing SQL commands in Rails

Alex Chee — Thu, 29 Oct 2009 01:19:47 +0000

Most of the times, ActiveRecord's helpers to access database info is all you need; sometimes, you want to do some hacky stuff.

For example, I had to figure out a database's timezone and schema but I had no shell access to the server. So I ran this used Base.connection.execute and fetch_row to get the result

q=ActiveRecord::Base.connection.execute 'SELECT NOW();'
=> #<mysql::result:0x2b783735c4f8>
>> q.fetch_row
=> ["2009-10-20 17:30:49"]
>> Time.now
=> Tue Oct 20 10:32:02 -0700 2009
>>q.free
</mysql::result:0x2b783735c4f8>

You can basically run any SQL query you want with these two methods. Now go crash some servers!

Note: You should also free your result to free up memory. Thanks Emmanuel.

Can't add git remote branch

Daniel LaBare — Thu, 29 Oct 2009 00:30:01 +0000

I love git, but it's full of gotchas. Here's one I ran into recently when trying to create a remote branch to do some major code refactoring.

20:22:03>git push origin origin:refs/heads/new_remote_branch
error: src refspec origin does not match any.
fatal: The remote end hung up unexpectedly
error: failed to push some refs to '<repo here>'
</repo>

Hmm. This has always worked for me before, what gives? When I took a look at my remote branches:

20:35:33>git branch -r
  origin/master

I noticed that there's no HEAD pointer. I'm actually not certain of how that came about, but let's add it:

git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master

Now this looks a little better:

20:38:10>git branch -r
  origin/HEAD -> origin/master
  origin/master

And now git knows what do do when you try to branch off origin to a new remote branch:

20:41:14 >git push origin origin:refs/heads/new_remote_branch
Counting objects: 429, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (257/257), done.
Writing objects: 100% (370/370), 54.03 KiB, done.
Total 370 (delta 247), reused 167 (delta 110)
To git@<repo_url_here>.git
 * [new branch]      origin/HEAD -> new_remote_branch
</repo_url_here>

Great. Add it to your evernote and move on with your life :D

Good Ruby Times

David Palm — Tue, 27 Oct 2009 13:01:21 +0000

In the last few weeks I've had the chance to glance into several different codebases, some written years ago by devs in their Rails-infancy (rainfancy?).

Doing the travelling codes-man like that is a golden opportunity to see what people's stumbling blocks are and how a few easy tricks can improve the code substantially. What follows is not your new cutting edge coffe making kitchen sink script-fu, but a rather dull list of everyday Ruby that I learned people need to learn.

One-stop requires

This is a short one: requires go into the environment.rb.

Really, don't use it elsewhere. I've seen people reason along the lines of "I keep the 'require hpricot' in my xe.com currency rates scraping class as to keep things local and closely knit". There is some truth to that argument and if a gem is only used by a rake task, maybe it even saves us a second during app startup and some RAM. Still, load order issues, poor readability and maintainability makes it a real pain to deal with down the road.

Trust me: don't. Put it in environment.rb with a comment describing its purpose in your project.

Whenever possible use the "config.gem". Advantages:

allows you to specify the lib name even if the gem name is different (e.g. for github gems)
allows you to require sub libs

Not used very often perhaps but put to good use by e.g. the right_aws gem where you can choose to load just the SQS part with:
config.gem 'right_aws', :lib => 'sqs/right_sqs'
or maybe you just wanted the SDB part?
config.gem 'right_aws', :lib => 'sdb/right_sdb_interface'
I personally believe this particular feature is more confusing than helpful, but it's good to know it's there.
allows you to specify the library version

Towards the end of the development cycle, before rolling out to production, gems should generally be frozen. An alternative is to define exact versions in the config. I generally advise doing both as having the environment.rb as the one-stop source for gem requirements is a boon for everybody involved, capistrano scripts or new devs on your team.
all in one place, easy to find out requirements

Singleton methods (creation)

Ruby provides three (common) ways to define a singleton method:

Explicitly name the class

class RapidInterventionGroupTeam < ActiveRecord::Base
  def RapidInterventionGroupTeam.load_group_allocations_and_teams(project_id, curr_user)
    # Code
  end
end

PROs: hmm, dunno. Makes it easy to know what class you're looking at if the source code file is very long?

CONs: long, ugly

Use self
class RapidInterventionGroupTeam < ActiveRecord::Base def self.load_group_allocations_and_teams(project_id, curr_user) # Code end end
PROs: easy to spot here-comes-a-singleton-marker. Some people like to use the 'self' keyword. I call them self-ish people. Sorta short.

CONs: I think it's ugly. For more than two singletons that repeating 'self' annoys me.

Use the metaclass

      
class RapidInterventionGroupTeam < ActiveRecord::Base
  # ==========================
  # = Class methods go here! =
  # ==========================
  class << self
    def load_group_allocations_and_teams(project_id, cur_user)
      # Code
    end
  
    def team_allocations
      # Code
    end
  
    def group_allocations
      # Code
    end
  end

  # ============================
  # = Instance methods go here =
  # ============================
  def beef_cow_and_fowl
    # Code
  end
end

PROs: shortest. Encourages devs to gather singletons in one spot, and perhaps they will notice when it gets out of hand and start creating mixins. Maybe. Also allows devs to show off their Ruby metaclass knowledge at bars. Maybe.

CONs: that class << self is funky and if funky doesn't rub well with you (or your boss) then I guess it's no good.

I personally favor c) whenever the number of class methods goes above two or three; b) is fine when there are just a few of them.

Invoking singleton methods

In Ruby there are many ways to invoke a class method. A few common ones:

Name the class explicitly:

user = User.find(123)
Predators.feed_to_wolves(1, user)

When used from an instance of the class, use the context: self.class

class Predators < ActiveRecord::Base
  def self.feed_to_wolves(pack_id, meat)
    # code
  end
  
  # we know how we were born...
  def feed_us!(meat)
    self.class.feed_to_wolves(id, meat)
  end
end

Use a helper instance method
class Steak def self.logger @logger ||= Logger.new('log/steak.log') def def logger self.class.logger end def beef logger.debug "MONDAY BEEF!" beef_for!(:monday) end end
Using an instance method this way to invoke a class level method is of course useful only if you invoke it very often and readability becomes very important. In the above example we're also memoizing (caching) the class level Logger object for fast access.

I personally try 1) to avoid singletons altogether, 2) use method b) whenever possible and 3) start asking myself hard and awkward questions when doing a) more than 3 times a day. Too many singletons might be a codesmell.

Multiple return values

First an example of how people often fake multiple value returns in Ruby:

class Something
  def many
    [calc_monday_beef, calc_total_beef]
  end
end

s = Something.new
retval = s.many

DON'T: when coding a method and you realize you need to return more than one value from a method, you should stop and think. You're probably doing something wrong.
This is actually the reason why Ruby does not provide a way to return multiple values.

If you're sure you know what you're doing and want to return more than value here are some tips:

do NOT do what I did in the example.

use descriptive variable names. "retval" above sucks. Use Ruby multiple value assignments like so:

>> cows, sheep, fish = [1, 5, 3]
=> [1, 5, 3]
>> cows
=> 1
>> sheep
=> 5
>> fish
=> 3

Use this to assign multiple return values to local variables with helpful names:

      
s = Something.new
monday_beef, total_beef = s.many

c) if you're returning many values and you don't know beforehand how many values are coming back, use the splat operator (*):

>> me, *others = [:abe, :bob, :caesar, :donald]
=> [:abe, :bob, :caesar, :donald]
>> me
=> :abe
>> others
=> [:bob, :caesar, :donald]

d) avoid using array indexes and prefer using Array#first, Array#last:

BAD:

  
s = Something.new
retval = s.many
Beef.find(retval[0], :limit => retval[1])

STILL BAD SOMEWHAT BETTER:
s = Something.new retval = s.many Beef.find(retval.first, :limit => retval.last)
Why better? Because you communicate to the reader that that 'retval' variable only have two values. (Yeah, it still sucks I know)

ALSO BAD BUT SOMEWHAT-ER BETTER STILL:

s = Something.new
monday_beef, beef_count = s.many
Beef.find(monday_beef, :limit => beef_count)

GOOD:

s = Something.new
Beef.find(s.monday_beef, :limit => s.total_beef_count)

Drop the return

Use the fact that Ruby methods return the value of the last expression. The only place where I think an explicit return is legit is when we want to return early and that's a performance optimization and as we know, premature optimization is the root of all evil, so unless you have benchmarks at hand proving the need for speed, just don't user 'return'.

Instead of:

def course_description
  retval = course_shortname 
  retval = course_shortname + "(#{project.shortname})" if project
  return retval
end

Do:

      
def course_description
  course_shortname + (project ? "(#{project.shortname})" : '')
end

Enumerable

Learn to use and love ruby Enumerable module. It's really really useful.

Instead of:

      
def Vacation.holidaytype_selectbox
  types = Array.new
  for vt in VacationTypes
    types << vt[:name]
  end
  @holidaytype_selectbox_model ||= types
end

Use:

    
def self.holidaytype_selectbox
  @holidaytype_selectbox_model ||= VacationTypes.map(&:name)
end

Instead of:

for result in @candidate.results do
  result.destroy
end

Use:
@candidate.results.each(&:destroy)

Instead of:

@approved = 0
unless @do_it.nil?
  @events = Event.hourreporting_entries(@person.user_id, params[:projectid], @week.first.date, @week.last.date)

  for e in @events
    if e.approved == 1
      @approved += 1
    end
  end
end

Do:

    
if @do_it
  @events = Event.hourreporting_entries(@person.user_id, params[:projectid], @week.first.date, @week.last.date)
  @approved = @events.sum{|e| e.approved ? e.approved : 0}
end

The Array#sum method is a Rails addition.

In the app where I spotted the above, the default value for the 'approved' field is set to NULL. Had the default value been 0 instead, we could have written the above in an even shorter way:

@events.sum(&:approved)

Truth

In Ruby, everything except false and nil evaluates to true. For realz.

Instead of:

  
if options[:add] == true && check_id.to_i != 0
  meanie = MeanGuy.find(check_id)
  if meanie.reason != '' && !meanie.reason.nil? && meanie.type == MeanGuy.allowed_types[0]
    # Code
  end
end

Do:

  
if options[:add] && !check_id.zero?
  meanie = MeanGuy.find(check_id)
  if !meanie.reason.blank? && meanie.type == MeanGuy.allowed_types.first
    # Code
  end
end

When chaining conditional checks like above with "&&", remember that Ruby will stop the checks at the first failure, so always put the 'cheapest' checks first (e.g. put any checks that require a database query last).

In the above snippet we see the very common "!something.blank?" idiom. I personally really don't like the prepended "!" and always try to avoid it. Hurts my eyes and makes me stumble while reading. Ideally the above should be:

if meanie.reason && meanie.type == MeanGuy.allowed_type

but if the meanie reason is allowed to be the empty string -- which Ruby will consider True -- then we need the explicit blank?-check.

Unrelated neat trick using &&:

Instead of:

def safe_death(dude)
  if dude
    if dude.destroy
      if DudeMailer.deliver_destruction_notification
        Call.his_mum
      end
    end
  end
end
safe_death(Dude.first)

Do:

def safe_death(dude)
  dude && dude.destroy && DudeMailer.deliver_destruction_notification && Call.his_mum
end
safe_death(Dude.first)

This works well only if you don't care about handling any error conditions and success depends on *all* method calls being successful.

To improve readability in general create as many predicate methods as possible. They are short, cheap and really helps readability.

Instead of:

  
if @person && @person.active == true && @person.projects.include?(:beef_on_monday) && (@person.type == :human || @person.type == :dog)
  # Code
end

Do:

class Person
  def active_monday_beefer?
    actvive? && member_of?(:beef_on_monday) && (human? || dog?)
  end
end

if @person.active_monday_beefer?
  # Code
end

Method naming

When naming your methods, avoid prepending "get_" or "set_". The Ruby convention is that getters are just the value name and the setter has the "=" postfix. Avoid the quasi-get_'s too: load_, retrieve_, store_, put_. Sometimes they're ok, but just stop a second and think if:

store_incoming_beef()

is really really better than

more_beef_in_the_holds()

beefers_keepers()

not_pork_not_sheep_not_fowl_but?()

As always rules are made to be broken, judiciously. ;)

DON'T:

#get_premium, #get_unix_time_stamp, #load_dates, #retrieve_mums etc

Do:

#premium, #unix_timestamp, #dates, #mums, #beefs

DON'T:

Server.set_clock(Time.now)
@doc.store_signer(Dude.new)
@tellus.put_fire(:wild)
Time.set_back(1.year.ago)

Do:

Server.clock = Time.now
@doc.signer = Dude.new
@tellus.fire!(:wild)
Time.now = 1.year.ago

Keep method names short: e.g. #shorten_timespaces_because_of_special_holiday is too long. Long method names are often a sign of complex methods. Complex methods are often a sign we should refactor the code into smaller pieces.

Sometimes the work done by a method is simply too complicated to be expressed properly by a short and self-explanatory method name. If so, don't even try. Leave the method name short and cryptical. It becomes a marker for the reader of "hey, wanna understand this one? Sorry but you really really have to go read the code!".

DON'T:

#shorten_timespaces_because_of_special_holiday

DO:

#holiday_adjustment

Use the "?" for predicates (should *always* return true/false or at least something that evaluates to true false, such as "abc"/nil)

class Session
  def finished?
    finished_at
  end
end

sess = Session.new
sess.finished_at = Time.now
sess.finished?

GOOD:

if sess.finished? 
  coffee!
end

BAD:

if sess.finished_at
  coffee!
end

Use rdoc!

User rdoc for your rails apps. It's actually nice. When your mum asks you what it is you actually do you can show her the rdocs with links and text and colors and lists and it will make her happy too. And very useful for people trying to read your code (ok, they can run the freakin' rake task themselves, but you should do it too!).

  rake doc:app
  open doc/app/index

Amazon's MySQL in the Cloud (RDS)

Jonathan Siegel — Tue, 27 Oct 2009 10:08:41 +0000

Amazon just released the "public beta of Amazon Relational Database Service (Amazon RDS), a new web service that makes it easy to set up, operate, and scale relational databases in the cloud."

Great! So what is it?

RDS today is an EC2 instance running MySQL 5.1. Instead of starting an EC2 instance with RightScale or EC2 commandline tools, you start an RDS instance with the RDS commandline tools (and I'm sure RightScale will support RDS shortly).

The RDS instance accepts parameters such as root database username and password, db space to allocate, initial instance size (note there are new instances), db backup periods, among others. Amazon's RDS layer configures MySQL in a proven configuration based on the parameters you specify.

RDS instances cost 10% more than a vanilla EC2 instance for Small, Large and XL instances. For Double and Quadruple DB instances the pricing is higher:
Note that EC2 pricing is about to change and you will want to recalculate that 10% if RDS does not shift as well.

And why do we want it?

If you run MySQL in EC2 for reasonably large properties, this service effectively provides a substantial reduction in undifferentiated system administration overhead within your organization. That is, if your deployment is small--you probably run your DB on the same machine as your application--you could use RDS to reduce your SysAdmin costs, but you probably value your time less than AWS costs at this stage of deployment. If your deployment is huge--you probably run your DB on EBS (RAID striped) with hot replication to other instances (for performance and failover) and automated offsite backups. You have a large AWS bill and a large SysAdmin cost to implement, monitor and maintain the configuration. RDS could potentially greatly reduce your SysAdmin bill, but today you cannot hot replicate nor run an RDS DB across multiple underlying instances--and therefore RDS is not for you.

So who is it for?

RDS as it stands today is ideal for deployments that have scale to warrant a separate DB. These are midsize deployments--or smaller deployments that follow best-practices in deployment and scalability over instance cost savings. These deployments may have mid-to-large EC2 instances powering their DB today, and don't need hot replication for performance nor uptime. Note that with the new double and quad instance sizes, the performance ceiling on a single instance has been greatly raised.

These deployments will benefit from a proven DB configuration, out of the box automated backups, one command time-and-date restores, auto instance failover (some downtime would be associated), monitoring (accessible via CloudWatch), and autoscaling of their DB instance (yes--as load increases, your instance is seamlessly migrated to a larger instance).

Our team routinely deploys the midsize and huge DB configurations described here. We have the highest SysAdmin cost on our huge deployments, but also the highest budgets :-). We have been using RDS as the staging DB for several of our midsized deployments. So far we have seen no real usage difference in using an RDS DB vs our own EC2-hosted DB. We do notice that we don't worry about whether our DB backup script is working. That alone is worth 10% of our instance cost. The reduction in DB instance deployment time and SysAdmin cost is also a win--but we won't see those savings until our next deployment--as all our deploys are RightScripted and already include a configured DB. Finally--the autoscale, fine-grained restore and other features sound great, but we won't have more to say until we use them in practice.

Other Observations

Logs are accessible within a MySQL table exposed within the RDS insance.
In the future it appears that other DB types (non MySQL 5.1) will be supported.
DB allocated storage must be declared in advance. Think of this as allocating the EBS volume connected to your instance. Going to need 50GB for your DB? Allocate 50GB. If it grows? Monitor DB size on CloudWatch and make sure you bump the allocated space. Will the RDS team email you to let you know this is an issue? Try it and find out. :-P
New instances were released coincident with the launch. 64GB and 32GB high-memory instances.
A new Small 64 bit instance is available, but only for RDS. This would be a nice-to-have for EC2 as well to minimize rebuilding templates when going from Small (32bit) to other sizes (64bit).

RightSignature hits the Press on ReadWrite Enterprise

ELC — Wed, 21 Oct 2009 22:17:55 +0000

ReadWrite Enterprise says "RightSignature user interface is pretty elegant. It has a flow that makes it simple to complete the signing of documents through the automation of tasks such as adding initials to each page, a requirement of most contracts." They also comment that "Part of RightSignature's appeal is in its overall change to the workflow and the efficiencies that result." RightSignature is entering the Software-as-a-Service market and is headed to the top with other ecosystems like Salesforce.com and Google Wave.

ELC Technologies continues to launch cutting edge products on the web.

How to remove trailing slashes to get better analytics on your page views

Daniel LaBare — Wed, 14 Oct 2009 01:42:54 +0000

I noticed when viewing my google analytics report that I had several duplicate entries for the same that looked like /page, and then /page/ (a trailing slash).

Obviously I'd like these two to be combined to get one statistic for those page views. There are a couple of ways to do this.

1) The particular project in question is using nginx. We can use a rewrite to get rid of the slash.

server {
    server_name example.com;

    ...

    rewrite ^/(.*)/$ /$1 permanent;

    location / {
    ...
    }
}

2) You can also get creative with a little rack fun.

sudo gem install rack-rewrite

# then slip the following in your environment.rb

config.gem 'rack-rewrite', '~> 0.1.2'

require 'rack-rewrite'
config.middleware.insert_before(Rack::Lock, Rack::Rewrite) do
  r301 %r{^/(.*)/$}, '/$1'
end

3) Other ways? Leave me some feedback.

Now you should start to see only /page hits in your google analytics and no more /page/

Cool.

Custom Validations for Polymorphic Associatied Models

Nicholaus — Tue, 13 Oct 2009 18:25:31 +0000

I recently needed custom validations for a polymorphically associated model, but only for certain polymorphic parents. Lets looks at an example, take the acts_as_addressable plugin, an oldie but a goodie, it lets you attach addresses to all sorts of models. That is a great and extremely useful pattern, I use it all the time. But requirements can't always stay nice and polymorphic freindly. So we have to know when and how to extend the associations features and when to split the models. But all I needed was different validations for different models, so no need to split. Its simple, just utilize the if option on the validations and the model name stored in the addressable_type column.

SKIP_VALIDATIONS_FOR =['Widget', 'Sprocket']

validates_presence_of :street1, :city, :postal_code, :country, :if=> Proc.new {|obj| !SKIP_VALIDATIONS_FOR.include?(obj.addressable_type)}

This is a pretty simple example, but could easily be expanded to fit different requirements.

Floodgates opened to iPhone development

Rick — Mon, 12 Oct 2009 18:35:54 +0000

For as active as iPhone application development community is, achieving success in iTunes has been an elusive affair for those who participate in this vertical. The two main obstacles presented to anyone who wants to create an iPhone applications are: one, finding resources/developers with the right skill set; and two, marketing the application after the application has been submitted. The bad news is Apple keeps iTunes a black box. Unless your application has been reviewed or mentioned on review sites or blogs, no one will be able to find your application outside of iTunes. This makes marketing your application relatively difficult. Here's the good news: the cost to build an iPhone application should come down substantially as it no longer requires a developer with an exclusive knowledge to a specific technology to build an application for iPhone.

When Apple first announced to openly accept applications from developers, the prerequisite for the developer is a somewhat extensive knowledge in a language called "Objective-C." For a short while, it would seem as though the developers who could produce Objective-C codes were superstars that also came with a superstar price tag. Such stardom, however, did not last. When PhoneGap was introduced as an open source development tool for iPhone via JavaScript, the web development community devoured it like salmon to a hungry bear. Shortly after PhoneGap's success, Mono framework was released in the commercial sector that provided the necessary development tools to the vast number of C# developers across multiple platforms. And to unhinge the final bar from the floodgates, Adobe has just announced that the next release of Flash is capable of compiling a flash project directly into native iPhone application. Simply put, a project can go from design to finish without even being touched by a developer.

The implication for this phenomenon is a curious one: how will Apple respond to the rush of new applications when the floodgates are finally open? Will Apple still be able to keep its manual review process intact? When the market is saturated with developers and applications, will Apple be able to to maintain iTunes exclusive distribution channel and continue to motivate merchants to participate?

How all of this will affect Apple or iPhone developers is yet to be seen. However, one thing that seems to be true is that when given enough demands, people will find ways to liberate a technology regardless of how businesses are structured around it.

Make your buttons look super awesome

Gregor Martynus — Fri, 09 Oct 2009 16:24:41 +0000

Back in April, ZURB published the super awesome buttons to showcase what can now be done with CSS and RGBA. They are so awesome! They still look like images but loading speed and maintainability continues to benefit from normal HTML buttons. Below is what they look like:

I started using these buttons in some of our projects and extended them to improve the cross browser compatibility. I added a nicer design for when the buttons get clicked, and removed the necessity to set to different background colors for different states of the buttons.

Here is how the even more awesome buttons look like, click'em!

| I'm just a link - Click me

This is how the css magic looks like

/* set an awesome color for the buttons */
button.awesome, 
input.awesome, 
a.button.awesome {
  background-color: #111;
}
/* Touch the rest at your onw risk. */
button.awesome,
input.awesome,
a.button.awesome { 

  font: inherit;
  background-repeat: repeat-x; 
  color: #fff;
  text-decoration: none; 
  position: relative; 
  cursor: pointer; 
  border: 0; 
  font-style: normal; 
  font-weight: bold; 
  line-height: 1; 
  
  padding: 5px 10px 6px;
  font-size: 13px;
  
  /* IE only stuff */
  border-bottom: 1px solid transparent\9;
  
  
  /* not all browser support these, but who cares? */
  text-shadow: 0 -1px 1px rgba(0,0,0,0.25), -2px 0 1px rgba(0,0,0,0.25); 
  border-radius: 5px; 
  -moz-border-radius: 5px; 
  -webkit-border-radius: 5px; 
  -moz-box-shadow: 0 1px 2px rgba(0,0,0,0.5); 
  -webkit-box-shadow: 0 1px 2px rgba(0,0,0,0.5);
  
  /* one image for all states 
     see http://www.alistapart.com/articles/sprites */
  background-image: url(awesome-overlay-sprite.png);
  background-position: 0 0;
  
  /* cross browser inline block hack 
     see http://blog.mozilla.com/webdev/2009/02/20/cross-browser-inline-block/ */
  display: -moz-inline-stack;
  display: inline-block;
  vertical-align: middle;
  *display: inline !important;
  position: relative;
  
  /* help IE to calm down a bit */
  zoom: 1;
  
  /*disable text selection (Firefox only)*/
  -moz-user-select: none;
}
/* hide selection background color */
.awesome::selection {
	background: transparent;
}

button.awesome:hover,
input.awesome:hover,
a.button.awesome:hover {
  background-position: 0 -50px; 
  color: #fff;
}
button.awesome:active,
input.awesome:active,
a.button.awesome:active	{ 
  background-position: 0 100%; 
  -moz-box-shadow: inset 0 1px 2px rgba(0,0,0,0.7); 
  /* unfortunately, Safari seems not to support inset yet */
  -webkit-box-shadow: none;
  
  /* IE only stuff */
  border-bottom: 0\9;
  border-top: 1px solid #666\9;
}

button.awesome.small,         input.awesome.small,          a.button.awesome.small 	        { padding: 4px 7px 5px; font-size: 10px; }
button.awesome.small:active,  input.awesome.small:active,   a.button.awesome.small:active	  { padding: 5px 7px 4px; }
button.awesome.medium,        input.awesome.medium,         a.button.awesome.medium         { /* default */ }
button.awesome.medium:active, input.awesome.medium:active,  a.button.awesome.medium:active	{ padding: 6px 10px 5px; }
button.awesome.large,         input.awesome.large,          a.button.awesome.large 	        { padding: 8px 14px 9px; font-size: 14px; }
button.awesome.large:active,  input.awesome.large:active,   a.button.awesome.large:active	  { padding: 9px 14px 8px; }

Turn a or a Link just by adding the 'awesome button' classes:

<button>button before</button> &rarr; 
<button class="awesome button">button after</button><br />
<input class="awesome button" type="submit" value="submit before" /> &rarr; 
<input type="submit" value="submit after" /><br />
<a href="#">Link before</a> &rarr; 
<a href="#" class="awesome button">Link after</a>

→
→
Link before → Link after

Awesome Blueprint Plugin

These super awesome buttons play well together with the blueprint css framework. Simply get the files from github and put them in you blueprint/plugins folder. Works awesome.

Demo

You can find the latest version of the demo in the git repository or you can open it here.

Let us know what you think!

grepping the logs with your ears

John Eberly — Thu, 08 Oct 2009 17:45:04 +0000

This is a simple obvious idea, but for some reason I had never done it until recently. My problem, I had a callback that I thought was firing way too often, but I didn't want to grep the logs constantly looking back and forth from the app to the console. So I took advantage of the OSX 'say' command.

system('say "in here"')

Just jam that in the code you want to monitor and listen for the noise. There are many voices. Also, it can also be a fun easter egg for your teammates by checking git author name and presenting them with a special message... but I would never recommend that :)

Copying files between S3 accounts

Alex Chee — Thu, 08 Oct 2009 01:42:52 +0000

Recently, I had to transfer a all files from one S3 account to another one. Since I didn't like to bother Amazon with my petty problems. I decided to use a ruby script to do it. Here's the script and some steps I took to do it.

Setup

First thing we need to do is to get a list of the buckets we're using. Since each bucket has to be uniquely named, we need to figure out what to name our corresponding buckets in the new S3 account.

Start up irb. Time for some digging.

>> require 'rubygems'
=> true
>> require 'right_aws'
=> true
>> s3=RightAws::S3Interface.new(old_aws_id, old_aws_key)
=> #<rightaws::s3interface:0x1a56490 ...stuff...>
>> buckets=s3.list_all_my_buckets.collect{|b| b[:name]}
=> ["old_bucket1", "old_bucket2"]
</rightaws::s3interface:0x1a56490>

Looks like we have 2 buckets to copy: old_bucket1 and old_bucket2. Let's make new buckets for our new account: new_bucket1 and new_bucket2.

Back in console:

>> s3_new=RightAws::S3Interface.new(new_aws_id, new_aws_key)
=> #<rightaws::s3interface:0x1a5649a ...stuff...>
>> s3_new.create_bucket('new_bucket1', :location => :us)
=> true
>> s3_new.create_bucket('new_bucket2', :location => :us)
=> true
</rightaws::s3interface:0x1a5649a>

Now, we need to peek at the ACL settings for the files so we'll know how to modify the permissions for the new bucket.

In console:

>> key=s3.list_all_my_buckets.first
=> {:owner_display_name=>"my.old.name", :creation_date=>"2009-08-06T23:32:38.000Z", :name=>"old_bucket1", :owner_id=>"123abcdefghijklmnoprxyz45600000000000000000000000000000000000001"}

Make note of the old owner_display_name and owner_id: my.old.name, 123abcdefghijklmnoprxyz45600000000000000000000000000000000000001. We will need to replace these with our new ones during the process.

Now we get our new owner_id and owner_display_name, the same way.

>> s3_new.list_all_my_buckets.first
=> {:owner_display_name=>"my.new.name", :creation_date=>"2009-08-06T23:32:38.000Z", :name=>"new_bucket1", :owner_id=>"123abcdefghijklmnoprxyz45600000000000000000000000000000000000002"}

Here's our new owner_display_name and owner_id: my.new.name, 123abcdefghijklmnoprxyz45600000000000000000000000000000000000002.

The Code

Now here's the script to run. I recommend doing this on an EC2 machine, because Amazon won't charge for bandwidth within the EC2 and S3 networks. I also suggest running it on a screen session so you can leave the script alone on the server.

The script will loop through each item in your buckets.
Copy the item over to the new corresponding bucket.
Strip out the ACL from the old one and reformat it for the new owner.
Update the ACL permissions on the new file.

#!/usr/bin/ruby
require 'rubygems'
require 'right_aws'

oldAWS = RightAws::S3Interface.new(old_aws_id, old_aws_key)
newAWS = RightAws::S3Interface.new('new_aws_id, new_aws_key)
newS3=RightAws::S3.new(new_aws_id, new_aws_key)

bucket_mapping={"old_bucket1" => "new_bucket1",
"old_bucket2" => "new_bucket2"
}

# ACL property differences
old_owner_id='123abcdefghijklmnoprxyz45600000000000000000000000000000000000001'
new_owner_id='123abcdefghijklmnoprxyz45600000000000000000000000000000000000002'

old_disp_name='my.old.name'
new_disp_name='my.new.name'

bucket_mapping.each do |old_bucket, new_bucket|
  # get all keys for old bucket by looping through sets of max keys (1000) amazon gives
  newS3Bucket=newS3.bucket(new_bucket)
  oldAWS.incrementally_list_bucket(old_bucket) do |key_set|
    # loop through content of key_set which contains keys
    key_set[:contents].each do |key|
      # if key already exists, don't copy over
      if newS3Bucket.key(key[:key]).exists?
        puts "#{new_bucket} #{key[:key]} already exists. Skipping..."
      else
        # download data and header from old bucket
        puts "Copying #{old_bucket} #{key[:key]}"
        retries=0
        begin
          data=oldAWS.get_object(old_bucket,key[:key])
        rescue Exception => e
          puts "cannot download, #{e.inspect}\nretrying #{retries} out of 10 times..."
          retries += 1
          retry if retries <= 10
        end
     
        retries=0
        begin
          headers=oldAWS.head(old_bucket,key[:key])
        rescue Exception => e
          puts "cannot get header, #{e.inspect}\nretrying #{retries} out of 10 times..."
          retries += 1
          retry if retries <= 10
        end

        # upload key to bucket
        puts "Putting to #{new_bucket} #{key[:key]}"
        retries=0
        begin
          newAWS.put(new_bucket, key[:key], data, headers)
        rescue Exception => e
          puts "cannot put object, #{e.inspect}\nretrying #{retries} out of 10 times..."
          retries += 1
          retry if retries <= 10
        end

        # copy permissions to new Bucket
        retries=0
        begin
          acl_prop=oldAWS.get_acl(old_bucket, key[:key])
        rescue Exception => e
          puts "cannot get ACL, #{e.inspect}\nretrying #{retries} out of 10 times..."
          retries += 1
          retry if retries <= 10
        end
     
        # Replace Owner ID and Display name for new bucket
        puts "old ACL #{acl_prop[:object]}"
        acl_prop[:object].gsub!(old_owner_id,new_owner_id)
        acl_prop[:object].gsub!(old_disp_name,new_disp_name)
        puts "new ACL #{acl_prop[:object]}"
   
        puts "changing ACL"
     
        retries=0
        begin
          newAWS.put_acl(new_bucket, key[:key], acl_prop[:object])
        rescue Exception => e
          puts "cannot update ACL, #{e.inspect}\nretrying #{retries} out of 10 times..."
          retries += 1
          retry if retries <= 10
        end
      end
     
    end
  end
end

That's basically all there is to it. If you want to do a check if all the files are there, use this script. Now I'm going to explain each part of the script. So you probably don't want to stay for this part.

Breakdown

#!/usr/bin/ruby
require 'rubygems'
require 'right_aws'

oldAWS = RightAws::S3Interface.new(old_aws_id, old_aws_key)
newAWS = RightAws::S3Interface.new('new_aws_id, new_aws_key)
newS3=RightAws::S3.new(new_aws_id, new_aws_key)

Initialize connections to S3. I used RightAws::S3 object to able to access the bucket individually.

bucket_mapping={"old_bucket1" => "new_bucket1",
"old_bucket2" => "new_bucket2"
}

Map the old buckets to the new corresponding ones.

old_owner_id='123abcdefghijklmnoprxyz45600000000000000000000000000000000000001'
new_owner_id='123abcdefghijklmnoprxyz45600000000000000000000000000000000000002'

old_disp_name='my.old.name'
new_disp_name='my.new.name'

Store the ACL credentials for the old and new accounts.

bucket_mapping.each do |old_bucket, new_bucket|
  # get all keys for old bucket by looping through sets of max keys (1000) amazon gives
  newS3Bucket=newS3.bucket(new_bucket)
	...

We loop through each of the buckets in our hash, grab the new corresponding bucket

  oldAWS.incrementally_list_bucket(old_bucket) do |key_set|
    key_set[:contents].each do |key|
	...

Since Amazon only lets you get 1000 keys max at one time, we use right_aws's incrementally_list_bucket method to get eventually loop through all the keys in that bucket. Then we loop through each set of keys.

	  if newS3Bucket.key(key[:key]).exists?
	    puts "#{new_bucket} #{key[:key]} already exists. Skipping..."
	  else
	    # download data and header from old bucket
	    puts "Copying #{old_bucket} #{key[:key]}"
	    retries=0
	    begin
	      data=oldAWS.get_object(old_bucket,key[:key])
	    rescue Exception => e
	      puts "cannot download, #{e.inspect}\nretrying #{retries} out of 10 times..."
	      retries += 1
	      retry if retries <= 10
	    end

	    retries=0
	    begin
	      headers=oldAWS.head(old_bucket,key[:key])
	    rescue Exception => e
	      puts "cannot get header, #{e.inspect}\nretrying #{retries} out of 10 times..."
	      retries += 1
	      retry if retries <= 10
	    end

	    # upload key to bucket
	    puts "Putting to #{new_bucket} #{key[:key]}"
	    retries=0
	    begin
	      newAWS.put(new_bucket, key[:key], data, headers)
	    rescue Exception => e
	      puts "cannot put object, #{e.inspect}\nretrying #{retries} out of 10 times..."
	      retries += 1
	      retry if retries <= 10
	    end
	...

Now, we only copy over the file if it's not there, and we copy over the header from the old file to the new one.

       retries=0
       begin
         acl_prop=oldAWS.get_acl(old_bucket, key[:key])
       rescue Exception => e
         puts "cannot get ACL, #{e.inspect}\nretrying #{retries} out of 10 times..."
         retries += 1
         retry if retries <= 10
       end

       # Replace Owner ID and Display name for new bucket
       puts "old ACL #{acl_prop[:object]}"
       acl_prop[:object].gsub!(old_owner_id,new_owner_id)
       acl_prop[:object].gsub!(old_disp_name,new_disp_name)
       puts "new ACL #{acl_prop[:object]}"

       puts "changing ACL"

       retries=0
       begin
         newAWS.put_acl(new_bucket, key[:key], acl_prop[:object])
       rescue Exception => e
         puts "cannot update ACL, #{e.inspect}\nretrying #{retries} out of 10 times..."
         retries += 1
         retry if retries <= 10
       end
...

Finally, we get the old ACL, which is an XML file. We then gsub the old values with the new values and replace the new file's ACL.

Wash, rinse, repeat.

RightSignature Wins VentureNet "Best of Show"

ELC — Thu, 08 Oct 2009 00:44:03 +0000

Daryl Bernstein, CEO of RightSignature, presented at VentureNet in Los Angeles today and won "Best of Show" for 2009. This follows another win by RightScale in 2007. ELC Technologies continues to launch cutting edge products on the web.

[Solved] RightScale CentOS 5.0 Rails 2.3.4 upgrade

Jonathan Siegel — Wed, 07 Oct 2009 13:12:15 +0000

Here's the issue:

[prompt] sudo gem install rails --no-ri --no-rdoc --version=2.3.4
ERROR:  could not find gem rails locally or in a repository

Huh?

[prompt] gem -v
1.2.0
[prompt] cat /etc/issue
CentOS release 5.0 (Final)
Kernel \r on an \m

OK--no problems. Let's look at the gem sources...

[prompt] sudo gem sources
*** CURRENT SOURCES ***

http://mirror.rightscale.com
http://gems.github.com

RightScale's gem mirror is stale!

So let's add rubyforge too...

[prompt] sudo gem sources -a http://gems.rubyforge.org
http://gems.rubyforge.org added to sources

And then update our gems en masse:

[prompt] sudo gem update
Updating installed gems
Updating RedCloth
Building native extensions.  This could take a while...
Successfully installed RedCloth-4.2.2
Updating RubyInline
Successfully installed RubyInline-3.8.3
Updating ZenTest
Successfully installed ZenTest-4.1.4
Updating erubis
Successfully installed erubis-2.6.5
Updating eventmachine
Building native extensions.  This could take a while...
Successfully installed eventmachine-0.12.8
Updating highline
Successfully installed highline-1.5.1
Updating hoe
ERROR:  While executing gem ... (Gem::InstallError)
    hoe requires RubyGems version >= 1.3.1

Our RubyGems is too old, but upgrading gives us:

[prompt] sudo gem update --system
Updating RubyGems
Updating rubygems-update
Successfully installed rubygems-update-1.3.5
ERROR:  While executing gem ... (NameError)
    undefined local variable or method `remote_gemspecs' for #<gem::commands::updatecommand:0xb776f4f0>
</gem::commands::updatecommand:0xb776f4f0>

Ouch! Let's try the rubygems update helper:

[prompt] sudo gem install rubygems-update
Successfully installed rubygems-update-1.3.5
1 gem installed
Installing ri documentation for rubygems-update-1.3.5...
Installing RDoc documentation for rubygems-update-1.3.5...

[prompt] sudo update_rubygems
/usr/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': Could not find RubyGem builder (>= 0) (Gem::LoadError)
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:134:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem'
	from /usr/bin/update_rubygems:18
[prompt] sudo gem install builder --no-ri --no-rdoc
Successfully installed builder-2.1.2
1 gem installed
[prompt] sudo update_rubygems
/usr/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': Could not find RubyGem session (>= 0) (Gem::LoadError)
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:134:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem'
	from /usr/bin/update_rubygems:18
[prompt] sudo gem install session --no-ri --no-rdoc
Successfully installed session-2.4.0
1 gem installed
[prompt] sudo update_rubygems
/usr/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': Could not find RubyGem hoe-seattlerb (>= 0) (Gem::LoadError)
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:134:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem'
	from /usr/bin/update_rubygems:18
[prompt] sudo gem install hoe-seattlerb --no-ri --no-rdoc
Successfully installed hoe-seattlerb-1.2.1
1 gem installed
[prompt] sudo update_rubygems
/usr/lib/ruby/site_ruby/1.8/rubygems.rb:578:in `report_activate_error': RubyGem version error: hoe(1.8.2 not >= 2.3.3) (Gem::LoadError)
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:134:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:158:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `each'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:157:in `activate'
	from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:49:in `gem'
	from /usr/bin/update_rubygems:18
[prompt] sudo gem install hoe --no-ri --no-rdoc
ERROR:  Error installing hoe:
	hoe requires RubyGems version >= 1.3.1

[prompt] echo "lost in a maze of tiny dark passages"
lost in a maze of dark tiny passages

So it appears that we can neither satisfy the requirements for installing hoe (need RubyGems >= 1.3.1) nor update rubygems (need hoe >= 2.3.3)... but let's take a deeper look and find the solution.

[prompt] sudo gem list
...
rubygems-update (1.3.5, 1.3.1)
...

The Solution:

[prompt] sudo gem uninstall rubygems-update --version=1.3.5
Successfully uninstalled rubygems-update-1.3.5

And now we're unblocked and back on track:

[prompt] sudo update_rubygems
Installing RubyGems 1.3.1
mkdir -p /usr/lib/ruby/site_ruby/1.8
mkdir -p /usr/bin
install -c -m 0644 ubygems.rb /usr/lib/ruby/site_ruby/1.8/ubygems.rb
install -c -m 0644 rubygems.rb /usr/lib/ruby/site_ruby/1.8/rubygems.rb
--snip--
install -c -m 0755 /tmp/gem /usr/bin/gem
rm /tmp/gem
rm -f /root/.gem/source_cache
rm -f /usr/lib/ruby/gems/1.8/source_cache
Removing old RubyGems RDoc and ri
rm -rf /usr/lib/ruby/gems/1.8/doc/rubygems-1.2.0
rm -rf /usr/lib/ruby/gems/1.8/doc/rubygems-1.3.1
Installing rubygems-1.3.1 ri into /usr/lib/ruby/gems/1.8/doc/rubygems-1.3.1/ri
Installing rubygems-1.3.1 rdoc into /usr/lib/ruby/gems/1.8/doc/rubygems-1.3.1/rdoc

------------------------------------------------------------------------------

= Announce: RubyGems Release 1.3.0

NOTE:  RubyGems 1.1 and 1.2 have problems upgrading when there is no
rubygems-update installed.  You will need to follow the second set of update
instructions if you see "Nothing to update".

Release 1.3.0 fixes some bugs.

Bugs fixed:

* Disregard ownership of ~ under Windows while creating ~/.gem.  Fixes
  issues related to no uid support under Windows.
* Fix requires for Gem::inflate, Gem::deflate, etc.
* Make Gem.dir respect :gemhome value from config.  (Note: this feature may be
  removed since it is hard to implement on 1.9.)
* Kernel methods are now private.  Patch #20801 by Stefan Rusterholz.
* Gem::location_of_caller now behaves on Windows.  Patch by Daniel Berger.
* Silence PATH warning.

Deprecation Notices:

* Gem::manage_gems will be removed on or after March 2009.

For a full list of changes to RubyGems and the contributor for each change, see
the ChangeLog file.

Special thanks to Chad Wooley for backwards compatibility testing and Luis
Lavena for continuing windows support.

== How can I get RubyGems?

NOTE:  If you have installed RubyGems using a package system you may want to
install a new RubyGems through the same packaging system.

If you have a recent version of RubyGems (0.8.5 or later), then all
you need to do is:

  $ gem update --system   (you might need to be admin/root)

NOTE:  RubyGems 1.1 and 1.2 have problems upgrading when there is no
rubygems-update installed.  You will need to follow the second set of update
instructions if you see "Nothing to update".

NOTE: You may have to run the command twice if you have any previosly
installed rubygems-update gems.

If you have an older version of RubyGems installed, then you can still
do it in two steps:

  $ gem install rubygems-update  (again, might need to be admin/root)
  $ update_rubygems              (... here too)

If you don't have any gems install, there is still the pre-gem
approach to getting software ... doing it manually:

1. DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126
2. UNPACK INTO A DIRECTORY AND CD THERE
3. INSTALL WITH:  ruby setup.rb  (you may need admin/root privilege)

== To File Bugs

The RubyGems bug tracker can be found on RubyForge at:
http://rubyforge.org/tracker/?func=add&group_id=126&atid=575

When filing a bug, `gem env` output will be helpful in diagnosing the issue.

If you find a bug where RubyGems crashes, please provide debug output. You can
do that with `gem --debug the_command`.

== Thanks

Keep those gems coming!

-- Jim & Chad & Eric (for the RubyGems team)


------------------------------------------------------------------------------

RubyGems installed the following executables:
	/usr/bin/gem

If `gem` was installed by a previous RubyGems installation, you may need
to remove it by hand.

Now what we were after again? Right--Rails 2.3.4. Let's do some general cleanup:

[prompt] sudo gem update --system
Updating RubyGems
Updating rubygems-update
Successfully installed rubygems-update-1.3.5
:0:Warning: Gem::SourceIndex#search support for String patterns is deprecated
Updating RubyGems to 1.3.5
Installing RubyGems 1.3.5
RubyGems 1.3.5 installed

=== 1.3.5 / 2009-07-21

Bug fixes:

* Fix use of prerelease gems.
* Gem.bin_path no longer escapes path with spaces. Bug #25935 and #26458.

Deprecation Notices:

* Bulk index update is no longer supported (the code currently remains, but not
  the tests)
* Gem::manage_gems was removed in 1.3.3.
* Time::today was removed in 1.3.3.


------------------------------------------------------------------------------

RubyGems installed the following executables:
	/usr/bin/gem

And a full update as well:

sudo gem update

This completes while reporting one error with Hoe resolvable via this post.

And finally install Rails:

sudo gem install rails --no-rdoc --no-ri 
Successfully installed rails-2.3.4
1 gem installed

Ruby Daemons and Angels

David Palm — Tue, 06 Oct 2009 14:05:35 +0000

Unix is pretty good at managing processes. Fork is a simple yet powerful means of achieving parallelism for workloads that are reasonably well self-contained.

Ryan Tomayko just published an awesome overview of the technology and philosophy of Unicorn, where the *nix kernel and process management is put to very good use. A very good read, highly recommended! In addition to that, the Unicorn sources are a treasure trove of good ruby code applied to POSIX programming.

The web application development racket has a higher tier where the load can be so insane that the standard "receive request, process it, format a reply and send it back" just doesn't cut it. I/O contention, traffic spikes and subsystem failures becomes the real hurdle for a successful website. As all human-machine interaction studies show, latency is one of the top characteristics of a successful software. Quick and accurate feedback to users is paramount to communicate solidity and high quality.

At ELC, we're ever more often confronted with helping clients and partners to provide not only applications with all the needed features, but also to help cope with some crazy workloads.

As many others, we often enough look to queues and asynchronous workers for help. Instead of processing a request right off the HAProxy plate, we throw it off to a queue where a battery of worker processes will take care of the real work. This allows the app server(s) to respond quickly and provide that stern look in the eyes of the user, saying "Trust us, you're request is in good hands."
:)

Queues and workers

Queues is a big topic. Very interesting, go google it. Today I want to talk about the other, often neglected part: the workers.

When decoupling the web tier from the processing tier you often end up with code that all in all is pretty simple and straightforward. Throwing together a worker script is pretty easy, especially if you already have a working version of it to steal bits and pieces from.

The pesky thing about workers is that they run headless and they must somehow be monitored. Process monitoring is also a big topic I'm not going to spend time on. What I do want to talk about is how important it is not to underestimate the difficulty of writing a decent worker, and to realize how your focus should probably be on making it debuggable, introspectable and as a good Unix citizen as possible.

Daemon Kit

Cron and script/runner is a useful combo for some kinds of asynchronous worker, but the issues that can arise are many. Each time your job runs it has to instantiate a full Rails stack and open up all the IO channels needed. Lot of fat there.

Plain ruby scripts and the Daemon class is somewhat better and given the knowledge it's possible to write awesome daemons where your workers can live and thrive. It does take a lot of knowledge and fiddling to get it right though.

Enter Daemon Kit by Kenneth Kalmer. Tired of reinventing the wheel he wrote a collection of libraries and generators to facilitate the lives of daemon summoners like yourselves.

Go get it!

gem install kennethkalmer-daemon-kit -s http://gems.github.com

What follows is an example daemon built with daemon-kit, but with an additional twist inspired by the unicorn project: spawn a configurable number of workers to attack your workload in parallel. We're going to start with a standard daemon-kit daemon and see how easy it is to extend to fit you own needs.

Step one: Generate a daemon

Easy as pie. Run:

$ daemon-kit foff

A bunch of files are generated. Open the foff/ directory in your IDE and have a look around. You'll find a bunch of useful readme files in the various subdirs, pointing you in the right direction.

Step two: command line options

We need to be able to pass in the number of worker processes to start. Open config/arguments.rb. Add the following:

  @options[:worker_count] = 1 # Default
  opts.on('-w', '--workers WORKER_COUNT', 'Number of worker processes to spawn') do |worker_count|
    @options[:worker_count] = worker_count.to_i
  end

Try your daemon!

  $ bin/foff

See that? Already up and running.

Step three: Signals&traps

Let's move on. Open libexec/foff-daemon.rb. You'll find a configuration block that is executed once the daemon is initialized but before the workers arrives to the plant. Here's where we'll set up our traps to catch the signals.

DaemonKit::Application.running! do |config|
end

Signals? Signals is another big topic I'm in no way competent to speak in depth about. For now, suffice to say that you want to make sure that when you stop your daemon you send it a signal, usually the INT signal. There are several interesting signals you can send your processes and when properly implemented they allow the user to do many nice things. Check out nginx or Unicorn for examples of good signal processing.

When your daemon goes down, the workers probably need time to finish up what they were doing. If you don't do this carefully, one Ctrl + C later you might just have interrupted a recurring billing worker just after it sent off a request to the inventory dudes but before billing the client. Don't go there.

For now just trust that mr Kalmer knows what he's doing and add:

# encoding: utf-8
  
DaemonKit::Application.running! do |config|
  config.trap( 'INT' ) do
    DaemonKit.logger.info "\nINT  #{Process.ppid} > #{Process.pid} GOING DOWN. WORKERS: #{WORKERS.keys.inspect}"
    WORKERS.each_pair do |pid, foff|
      DaemonKit.logger.info "\nNotifying child process #{pid} it's time to go."
      kill_worker(:QUIT, pid)
    end
    Process.waitall
  end
end

The above code intercepts the INT signal, logs a message and then tells each of your running workers to stop gracefully (using the QUIT signal). The last line, Process.waitall, makes the master daemon process wait for all children (your workers) to finish.

This is example code and the above is not enough for production use. If an error occurs inside your trap block, the INT signal will go unheeded, or worse, the master will die but the worker process will steam on uncontrolled. You need to add all kinds of checks. For instance, what happens if evil Eve just issued a kill -9 to one of your worker processes, leaving your master daemon with a dangling worker. Tread carefully.

Step four: GO KILL THAT DAEMON

Next up is the kill_worker method. Plain and simple (and stolen from the awesome Unicorn sources):

  # Delivers a signal to a worker and fails gracefully if the worker
  # is no longer running.
  def kill_worker(signal, wpid)
    begin
      Process.kill(signal, wpid)
    rescue Errno::ESRCH
      worker = WORKERS.delete(wpid) rescue nil
    end
  end

"Errno::ESRCH"?!? I bet that's obvious to Matz, but to me it's even hard to read... It's good I don't need to know exactly what that part does. So, kill_worker signals the worker with PID wpid and if Eve already killed it (or, it died by itself) we stop trying to deal with it and move on.

Step five: Trapping workers on a fork

Now let's set up the workers. I choose to put that in its own method, fork_and_trap:

  # Fork, listen for QUIT signals and run()
  def fork_and_trap(wrkr)
    fork do
      trap(:QUIT) do 
        wrkr.stopit!
        wrkr.log "aawwwright, time for a break. Hang on, lemme finish up..."
      end

      wrkr.run
    end
  end

Given a worker instance ("wrkr"), we fork() and setup the "listener" for the QUIT signal that we just setup above. The master daemon, when receiving an INT, will send QUIT to all known workers which in turn will run worker specific cleanup code, the #stopit! method in our case, and gracefully exit.
We also call the #run method on the worker and off he goes!

Step six: do it already!

The final piece is the actual worker spawning and accountant code. Again, there are probably a lot of pieces missing, but it boils down to:

# ===========================
# = Set up and fork workers =
# ===========================
WORKERS = {}

(0...DaemonKit.arguments.options[:worker_count]).each do
  foff = Foff.new
  WORKERS[ fork_and_trap(foff) ] = foff
end

DaemonKit.logger.info "PIDs: #{WORKERS.keys.inspect}"
Process.waitall

Remember the first step, when we set up the arguments processing? We can access the value passed from the command line in the DaemonKit.arguments.options collection. If we launched with:

  $ bin/foff -w 12

we'll spawn 12 worker processes and the master daemon can keep tabs on them in the WORKER constant, where the PIDs are keys and the values are the now running workers.

We finish up with a call to Process.waitall so the master daemon doesn't exit.

Step seven: the Working Class

The worker class is of course where all the meat is going. For this article I just cooked up a barebones worker that executes 5 steps and sleeps for 1 or 2 seconds for each step. The "steps" are application specific, something like "1. Pull from inventory", "2. Call mum", "3. Tell FedEx to pick up the stuff", "4. Bill client". At the completion of each work unit we check if we have received a QUIT in the meantime. If so, exit the loop.

Put the following in lib/foff.rb

  # encoding: utf-8

  class Foff
    def log(msg)
      (@logger ||= DaemonKit.logger).info "#{Process.ppid} > #{Process.pid} #{msg}"
    end

    def stopit!
      @shutting_down = true
    end

    def shutting_down?
      @shutting_down
    end

    def run
      loop do
        log "START @ #{Time.now}"
        (0..5).each do |step|
          log "workee workee at step #{step}"
          sleep rand(2)
        end
        log "END @ #{Time.now}"

        if shutting_down?
          log "TERMINATING @ #{Time.now}"
          exit
        end
      end
    end
  end

Not much to say here. Note that the only reference to DaemonKit is in the #log method and that is easy to remove. We're also pleasantly free of process management code in here. All that stuff is taken care of in the master daemon. Loose coupling ftw!

Step eight: profit off of the work of others

Time for a test run:

  $ bin/foff -w 3

You can see your three workers start up and go about their business. When the signal comes (Ctrl + C), the master daemon notifies the workers who in turn finishes off whatever they were doing and then, finally, our daemon can rest in peace.

You might notice how some log messages are printed weirdly on screen. I'm not sure why this happens, but I'd put my money on the Ruby standard logging library not coping with parallel access very well. It's a good thing the logging backend of DaemonKit is pluggable...

As said above, writing daemons is harder than you'd think but with tools like DaemonKit to help it's not that bad.

Stop wasting time - create Bug Tickets with Screenshots using Freshlog

Gregor Martynus — Sat, 03 Oct 2009 12:01:05 +0000

I love bug reports with screenshots because you take one look and you know exactly what the problem is. There is no need to leave your ticket environment to reproduce the bug, you see it right there. You see the browser and you see the current site, so let's start fixing!

But: I hate to attach screenshots to bug reports (don't you?). Reason being, you need two tools to work. First, you have to create the screenshot, then you need to create a ticket and attach the screenshot to it.

Because creating and attaching screenshots is so frustrating, clients may not see the benefit in doing it, which results in tickets lacking screenshots and causes developers to waste time trying to understand a ticket, before being able to fix one.

The guys from freshlog.com created a neat solution for this problem: they simply combined ticketing and taking screenshots in one easy to use tool:

Welcome to agile bug reporting!

Take a screenshot, crop it, annotate it and upload it to a new OR existing ticket/to do within your favorite team collaboration tool like Basecamp, unfuddle or lighthouse. Simply enter you account credentials and ticket creation with attachments is only a keyboard shortcut away!

How to setup Freshlog for unfuddle

1. Add your unfuddle Account

This is what Freshlog looks like when you open it for the first time. It offers image hosting for your screenshots, however, you don't need this feature for unfuddle as the screenshots will be attached directly to the tickets.

Click on the Issue Tracker Tab
Select https:// in the dropbox next to URL: (if you have a paid unfuddle account)
Enter the subdomain of the unfuddle account you want to login
Enter username and password of your personal account and hit save

2. Setup Preview as your default Image Editor

Click on the Application Tab and hit "Chose Editor". You can use any editor you want but the build in Mac OS X preview application fits perfectly for screenshot annotations.

Note: You may need to enable annotations in your preview app:

3. Take and attach a screenshots to a new or existing Ticket

Simply follow the video tour provided by Freshlog. The shown steps are the same for unfuddle.

Creating PDFs with MacRuby

David Palm — Fri, 18 Sep 2009 12:08:13 +0000

Macs are cool. Cocoa is cool. Ruby is cool. Matz is nice so we are nice. Sansonetti&Co at Apple are like the master drinkmixers, joining together the best of the best in one neat package. I'm talking about MacRuby.

The latest MacRuby dev branch, 0.5, is using LLVM and packs speed and features enough to make any geek teary eyed. This morning I wanted to take it for a test run on a problem domain that is sort of close to what my current project is dealing with: PDFs.

We all know Mac OS X is crazy good at reading, writing and printing PostScript and PDFs. It's right in there in the OS core graphics libraries. So, I thought, wouldn't it be cool to see how to use MacRuby and Quartz to read and parse PDFs directly, rather than that pesky old beast that is Ghostscript? (GS is an awesome piece of work, very capable and I'm not meaning to sound negative here. It's just kinda... old style?)

So, here's some example code. You never knew you needed it, but here it is: you know all of those PDFs that you have lying around on your Desktop? Wouldn't it be fantastic to have one big PDF with the first page of all of those?

No? Really? C'mon!

So, listen, it's like the internet. You didn't know you needed that either back in the day. This is exactly the same. Keep reading.

Get the goods

Go read this: http://redartisan.com/2009/9/1/macruby-intro. Awesome intro, instructions that work. Takes hours, so it's a good thing you now know you really really need your first-page-of-all PDF.

Got quartz?

Done? Sweet. So here goes:

framework 'Quartz'
mother_pdf = PDFDocument.new
Dir.glob(File.expand_path('~/Desktop/*.pdf')).each do |file_path|
  url = NSURL.fileURLWithPath(file_path)
  pdf = PDFDocument.alloc.initWithURL url
  puts "Loaded PDF from: \"#{file_path}\""
  mother_pdf.insertPage(pdf.pageAtIndex(0), atIndex: mother_pdf.pageCount)
end
puts "Done"
mother_pdf.writeToFile('./mother.pdf')

See how short that was? I was just blown away by the ease of it. The only gotcha was that it took me a while to a) find the API docs for PDFKit and b) learn which framework I needed (quartz, not PDFKit).

Mixing and matching standard Ruby (File, Dir.glob etc) and Cocoa is just... seamless! Very very cool. The PDFKit API is straightforward to use. Let's add an annotation to each page. There are a ton of different kinds of annotations you can add, but let's keep it simple: PDFAnnotationText. Add this method on top of the code above.

def annotation(text)
  rect  = NSRect.new(NSPoint.new(100,100), CGSize.new(150,100))
  annot = PDFAnnotationText.alloc.initWithBounds(rect)
  annot.shouldDisplay = true
  annot.contents = text
  annot.color = NSColor.whiteColor
  annot
end

Notice how we can use annot.color= instead of the "real" message name, annot.setColor(). That's a MacRuby extra.
Next, modify the PDF generating code like so to get this:

framework 'Quartz'
def annotation(text)
  rect  = NSRect.new(NSPoint.new(100,100), CGSize.new(150,100))
  annot = PDFAnnotationText.alloc.initWithBounds(rect)
  annot.shouldDisplay = true
  annot.contents = text
  annot.color = NSColor.whiteColor
  annot
end

mother_pdf = PDFDocument.new
Dir.glob(File.expand_path('~/Desktop/*.pdf')).each do |file_path|
  url = NSURL.fileURLWithPath(file_path)
  pdf = PDFDocument.alloc.initWithURL url
  puts "Loaded PDF from: \"#{file_path}\""
  page = pdf.pageAtIndex(0)
  page.addAnnotation(annotation("Origin: \"#{file_path}\""))
  mother_pdf.insertPage(page, atIndex: mother_pdf.pageCount)
end
puts "Done"
mother_pdf.writeToFile('./mother.pdf')

See? Pretty cool right? :)

Also cool:

Lazer Battles in Nerd World: pew pew pew

Nicholaus — Thu, 17 Sep 2009 17:05:53 +0000

For one week, some of the worlds smartest people gather in the desert and its not for a conference. Engineers from all disciplines, artists, craftsmen, writers, thinkers, cat herders and of course pyrotechnicians come to show off their inventions, create art, and to collaborate on a new society; a society based on one nerdy principle: Lazer Battles. Purely metaphorical lazer battles of course, nobody's actually playing lazer tag, because that would be lame. Its more about who can drag the largest lazer possible to the desert and mount it on top of the largest geodesic dome possible and fire it across the night at other geometric shapes or even other nerds. Those other nerds could possibly be trying to solve a turn based collaborative giant floating Rubik Cube or going head to head on a 4 story high Tetris game with player cam. One camp of nerdy rocket surgeons took the fight to the heavens with their Raygun Gothic Rocketship.

Nerdy meets Mad Max and the computer controlled propane cannons were born.
But in Nerd World™, its the giant propane explosion that trumps all others.

(skip to the 1:10 mark for the goods)

I'm not sure what you've all heard about Burning Man. But I am hear to let you know that the rumors are true, its more than geek friendly, its built by geeks for geeks. Its not all about being a hippie and dancing the night away, like so many people who haven't gone think; its about community and collaboration. And if you can only attend one conference next year, it should be in Black Rock City.

Rake Routes Review Repeat

Victor Christensen — Fri, 04 Sep 2009 22:52:09 +0000

If you have a large and active project with many, many nested routes you'll find that listing routes using rake:routes usually ends up a mass of info spewed into terminal that is painful to wade through even with grep.

It became a pain for me and I decided to take a stab at extending rake:routes to add a task to simplify the output and add some new information. Most of the time I grep rake:routes to confirm a path name or to see if I'm missing a route. Some of the time I wonder about whether there are orphaned routes or whether there are missing controller methods. It would be nice to get some help on these too.

So I put together a rake task for routes that you can drop into lib/tasks/routes.rake to get more succinct output.

>>rake routes:by_controller

With this I'm able to get a brief listing by controller of the action, method and path along with some information to identify any routes that have been orphaned. Perhaps some of the resources can be removed or some paths should be filtered (ie. :only =>[:index, :show]).

For example: running "rake routes:by_controller pattern=com.e" will result in output like the following (except better in terminal; color coded):

Attempting to match routes controller methods to app controller methods

Match controllers on pattern ~= /com.e/

Controller methods implemented
*Controller methods not implemented


CONTROLLER: user/tickets/comments
  *index    GET     user_ticket_comments                                                   
  create    POST                                                                           
  *new      GET     new_user_ticket_comment                                                
  *edit     GET     edit_user_ticket_comment                                               
  *show     GET     user_ticket_comment                                                    
  *update   PUT                                                                            
  *destroy  DELETE                                                                         


CONTROLLER: comments
  *index    GET     comments                                                               
  create    POST                                                                           
  *new      GET     new_comment                                                            
  *edit     GET     edit_comment                                                           
  *show     GET     comment                                                                
  *update   PUT                                                                            
  *destroy  DELETE                                                                         


Number of methods: 14
Controller methods implemented: 2
*Controller methods not implemented: 12

How does it help?

It becomes easy to copy and paste a path.
You'll see 'create' is defined in the controller but that the other actions aren't. Maybe they should be implemented (or maybe not) and just filtered from routes (ie. :only => [:create]).
A succinct, uncluttered view by controller.

Any way, grab the code below and have fun.

namespace :routes do

  desc 'Brief listing of routes info plus matches to app controller methods'
  task :by_controller => :environment do
    task_setup "Attempting to match routes controller methods to app controller methods"
    methods = find_routes_controller_methods_matched_to_app_controller_methods(@pattern)
    index_width, controller_width, action_width, path_width, method_width, segment_width, app_controller_action = get_array_elements_max_width(methods)
    last_controller = ''
    methods.each do |index, controller, action, path, method, segment, app_controller_action|
      @methods_count += 1
      @missing_methods_count += 1 unless app_controller_action 
      puts "\n\nCONTROLLER: #{controller.ljust(controller_width).strip}\n" unless last_controller == controller
      puts "  #{(app_controller_action ? green(action) : yellow("*#{action}")).ljust(action_width+colorize_width)}  #{method.ljust(method_width)}  #{path.ljust(path_width)}"
      last_controller = controller
    end
    display_counts
  end

  def get_array_elements_max_width(array)
    array.first.enum_with_index.map {|element,idx| array.map {|element| element[idx]}.map {|n| n ? n.length : 0}.max}
  end

  def list_directories(directory, pattern)
    result = []
    Dir.glob("#{directory}/*") do |file|
      next if file[0] == ?.
      if File.directory? file
        result.push(*list_directories(file, pattern))
      elsif file =~ pattern
        result << file
      end
    end
    result
  end

  def find_routes_controller_actions(pattern='')
    routes = []
    ActionController::Routing::Routes.routes.each do |route|
      path = ActionController::Routing::Routes.named_routes.routes.index(route).to_s
      method = route.conditions[:method].blank? ? 'GET' : route.conditions[:method].to_s.upcase
      segment = route.segments.inject("") { |str,s| str << s.to_s }
      segment.chop! if segment.length > 1
      controller = route.requirements.empty? ? "" : route.requirements[:controller]
      action = route.requirements.empty? ? "" : route.requirements[:action]
      route.requirements.delete :controller
      route.requirements.delete :action

      if controller =~ /#{@pattern}/
        unless routes.assoc "#{controller}:#{action}"
          routes << ["#{controller}:#{action}", controller, action, path, method, segment] unless controller.blank?
        end
      end
    end
    routes
  end

  def find_app_controller_actions(pattern='')
    controller_actions = []
    controllers = list_directories "#{RAILS_ROOT}/app/controllers", /_controller.rb$/
    controllers.each {|a| a.gsub!("#{RAILS_ROOT}/app/controllers/",''); a.gsub!('.rb','')}
    controllers.each do |controller_path|
      if controller_path =~ /#{@pattern}/
        controller = controller_path.camelize.gsub(".rb","")
        (eval("#{controller}.public_instance_methods") -
            ApplicationController.public_instance_methods -
            Object.methods).sort.each {|method| controller_actions << ["#{controller.underscore.gsub!("_controller",'')}:#{method}"]}
      end
    end
    controller_actions
  end

  def find_routes_controller_methods_matched_to_app_controller_methods(pattern='')
    controller_actions = find_app_controller_actions(@pattern)
    find_routes_controller_actions(@pattern).map! {|p| p << controller_actions.find {|i| i[0] == p[0]}}
  end

  def task_setup(title)
    @pattern = ENV['pattern'] ? ENV['pattern'].downcase : ''
    @methods_count = 0
    @missing_methods_count = 0
    puts "\n#{title}\n\n"
    puts "Match controllers on pattern =~ /#{@pattern}/\n\n"
    puts green "Controller methods implemented"
    puts yellow "*Controller methods not implemented"
  end

  def display_counts
    puts "\n\nNumber of methods: #{@methods_count}"
    puts green "Controller methods implemented #{@methods_count - @missing_methods_count}"
    puts yellow "*Controller methods not implemented #{@missing_methods_count}"
  end

  def colorize(text, color_code)
    "#{color_code}#{text}\e[0m"
  end
  def colorize_width; 10;end
  def red(text); colorize(text, "\e[31m"); end
  def green(text); colorize(text, "\e[32m"); end
  def yellow(text); colorize(text, "\e[33m"); end

end

Counting total number of objects in S3

Alex Chee — Tue, 01 Sep 2009 19:45:25 +0000

Sometimes it comes in handy to get the total number of objects you have in S3 but it is not as straight forward. Here's a snippet I use to get total number of objects using right_aws gem.

require 'rubygems' 
require 'right_aws' 

AWS_ID='id'
AWS_KEY='mykey'

s3=RightAws::S3Interface.new(AWS_ID, AWS_KEY) 
count=0

buckets=s3.list_all_my_buckets.collect{|b| b[:name]}
buckets.each do |bucket_name|
    s3.incrementally_list_bucket(bucket_name) { |k| 
        count += k[:contents].size
    }
end
puts count

The script gets all your buckets, count the contents of the buckets incrementally because AWS only allows a max of 1000 items to be returned per request.

Get total size of data being used

You can also modify this script to sum up the amount of space you're using:

require 'rubygems' 
require 'right_aws' 

AWS_ID='id'
AWS_KEY='mykey'

s3=RightAws::S3Interface.new(AWS_ID, AWS_KEY) 
size=0

buckets=s3.list_all_my_buckets.collect{|b| b[:name]}
buckets.each do |bucket_name|
    s3.incrementally_list_bucket(bucket_name) { |k| 
        k[:contents].each do |content|
            size += content[:size]
        end
    }
end
puts count

Note: I just made up this script right now and haven't tested it out. Please tell me if it works.