There’s two types of tools that are generally built on top of ParseTree: code analytics and custom DSLs. I covered 3 of them flog, roodi, and ambition. The first two being code analytics libraries and the second being a custom DSL.

My slides can be found up at SlideShare:

And if you attended you can rate my talk on SpeakerRate.com: http://speakerrate.com/talks/594-ruby-ast-tools

And here’s some more resources for those interested in learning more about the Ruby AST:

• Mark Chung’s slides from his RubyConf2008 talk: “Hacking with Ruby2Ruby”
• Ilya Grigorik’s blog post “Ruby AST for Fund and Profit”
• Chris Wanstrath’s (aka defunkt) presentation at Goruco 2008 on ParseTree

Tags: Ruby, AST, ParseTree, Programming, slides, talks, Raleigh.rb

I quickly worked up a hack that gets me both:

1. Keep your Twitter email address as my gmail address.
2. Then add a filter in Gmail that takes all emails from “twitter-follow-GMAILUSERNAME=gmail.com@postmaster.twitter.com” (replacing GMAILUSERNAME with your username of course) and with “following” in the subject, and archives them, marks them as read, and forwards them to your Twimailer email address.

This works perfectly I keep my email address associated with my twitter account and get the enhanced follow emails.

If you don’t use gmail you could easily adapt this to work with whatever email service you use. You just have to get the service to forward the appropriate emails to the correct email address.

Twimailer is a wonderful service and getting to keep your email address findability on twitter with this trick takes out any negatives from the service leaving just the good stuff.

Tags: twitter, gmail, tips, tricks

I found a couple of conversions of Facebox for Prototype. After finding these it didn’t take too much work to convert it to Lowpro.

The usage is simple:

Event.addBehavior({
  'a#fbox' : Facebox
});

The code can be found on GitHub.

Take this code put it in a file named inflectors.rb in your config/initializers directory, and you’re good to go.

Tags: ruby, slugs, URLs, rails, activesupport

error: pathspec '2eed4e6a42d0a50bb6587930659f943f9fae6c44' did not match any file(s) known to git.
Unable to checkout '2eed4e6a42d0a50bb6587930659f943f9fae6c44' in submodule path 'vendor/extensions/file_system'

This normally means you’ve changed a submodule and not properly handled updating it.

In this case though the repository would check out correctly on other users machines just fine. Odd thing was we were all using the same OS (MacOS X 10.5), the save version of git (1.6.0.1) from the same source (MacPorts), and most of the same external libraries installed via MacPorts. There was no real discernible difference we could find between machines.

After beating our heads enough times and enough fiddling around with my .profile, we found the cause. In my .profile I had the following lines:

GREP_OPTIONS="--color=always"
export GREP_OPTIONS
export GREP_COLOR='1;34;43'

These lines give me colored grep output which I personally find more useful than the non-colored variant. Problem is git uses grep for all sorts of functionality, and the colored output was throwing it off. The solution was to remove (or comment out) those lines from my .profile.

My problem here is just that if you simply run grep with the option “--color=never” the colored output would be disabled, resulting in no problems at all. That being said this clearly seems like a bug in git that will hopefully be fixed soon so I can go back to my colored grep output.

Tags: git, grep, color, submodules

A while back I set out to try and duplicate this functionality in a PHP function. Over the course of the past few months I feel it’s slowly matured and become something extremely valuable that I can’t live without.

It returns true when ever it is sure a value is blank, and false otherwise. It considers NULL, and false to be blank, and true to be not blank at the start. It then considers integers of 0 to be blank, and floats of 0.0 to be blank. It considers any string that is only composed of blank space characters, or “0″’s to be blank. I have considered adding a string of “false” or “null” but for now I considered this not to be generalizable enough. I feel that a string just of “0″’s almost always counts as a blank value when I’m processing data, but I could possibly see a string of “false” or “null” representing a non-blank value.

Then we get to the difficult part, arrays. The function of course regards an empty array as blank, but it will also recursively (this is performed by calling its self) go through all elements and if they are all blank declare the whole array blank.
There is also the special case of Objects that developers wish to be considered enumerable/iterator/array-like objects. These are created by implementing the SPL iterator interfaces. So the function checks to see if the necessary interfaces are implemented, and if they are treats the object, exactly like it would an array.

The last rule is it considers all other objects to be not blank.

This function can of course be expanded or refined to meet your particular needs. If, for instance, you have a self-defined Null object that you use you get set that to return true. Or if you don’t want strings with just “0″ in them to return true, you can change that. The function is very simple and it should be obvious where you change things to get the result you want.

Here’s the code:

function is_blank($value){
   $type = gettype($value);
   switch ($type) {
      case 'NULL':
      return true;
      break;
      case 'boolean':
      if ($value === false)
         return true;
      break;
      case 'integer':
      if ($value == 0)
         return true;
      break;
      case 'double':
      if ($value == 0.0)
         return true;
      break;
      case 'string':
      if (preg_match('/^\s*0*\s*$/', $value))
         return true;
      break;
      case 'array':
      if (count($value) == 0)
      {
         return true;
      }
      else {
         $all_blank = true;
         foreach ($value as $item) {
            if (!is_blank($item)){
               $all_blank = false;
            }
         }
         if ($all_blank){
            return true;
         } else {
            return false;
         }
      }
      break;
      case 'object':
      $implements = class_implements($value);
      if (
         in_array('ArrayAccess', $implements) && 
      (
         in_array('IteratorAggregate', $implements)  || 
         in_array('Iterator', $implements)           || 
         in_array('SeekableIterator', $implements)   || 
         in_array('RecursiveIterator', $implements)
         ) && 
         in_array('Countable', $implements)
         )
      {
         if (count($value) == 0)
         {
            return true;
         }
         else {
            $all_blank = true;

            foreach ($value as $item) {
               if (!is_blank($item)){
                  $all_blank = false;
               }
            }

            if ($all_blank){
               return true;
            } else {
               return false;
            }
         }
      }
      break;
      default:
      return false;
      break;
   }
   return false;
}

Pastie: http://pastie.textmate.org/169940
Pastie code download: http://pastie.textmate.org/pastes/169940/download

Tags: PHP, blank, code, programming, null

module ActionView::Helpers::FormOptionsHelper
         
   def options_for_select_with_include_blank(container, selected = nil, include_blank = false)
      options = options_for_select_without_include_blank(container, selected)
      if include_blank
         options = "<option value=\"\">#{include_blank if include_blank.kind_of?(String)}</option>\n" + options
      end
      options
   end
   alias_method_chain :options_for_select, :include_blank
   
   def options_from_collection_for_select_with_include_blank(collection, value_method, text_method, selected = nil, include_blank = false)
      options = options_from_collection_for_select_without_include_blank(collection, value_method, text_method, selected)
      if include_blank
         options = "<option value=\"\">#{include_blank if include_blank.kind_of?(String)}</option>\n" + options
      end
      options
   end
   alias_method_chain :options_from_collection_for_select, :include_blank
   
end

Tags: ruby, rails, programming, code

I’ve also created a large number of specs (with RSpec) for testing the plugin. Amazingly, testing plugins is actually really hard task, which I’ll probablly cover in a future post.

I’ve created a wiki page for Documentation and updated the RDoc’s.

If your using the plugin, and have any questions or concerns, or bug reports feel free to contact me.

Tags: rails, ruby, plugin, datepicker, ujs

I wanted to take the YouTube model and allow property owners to upload videos in any format they had them in and for us to show them in Flash video so that as many users as possible would be able to view it. So I set out constructing the process via which this would occur.
It would start out with a user uploading a video. The file would be moved to a temporary location (although a Tempfile object could not be used as these files get deleted automatically, i.e. before the background process would get to it). This path would have to be saved, and status value given. Then a background process would have to be passed this information to convert the movie, make a preview image, and upload both to Amazon S3, then saving the record as finished. Additionally, this background process would need to notify the property owner if either an error processing the video occurred or it had been successfully converted and uploaded.

The video model class I created is very similar in many ways to the functionality offered by the attachment fu plugin. It handles taking the uploaded file and moving it to a temporary location, it deletes the files when the record is deleted, it keeps track of the preview image and video file paths on S3. It also has a status attribute for the different statuses a video can have: queued, processing, ready, and failed.

The most complicated part of the whole process is how to handle the file once it’s been uploaded and temporarily stored. This needs to be handled by a background process, for Rails the obvious choice is the BackgrounDRb plugin. The background worker I created starts by grabbing the related video record, and sets it’s status to “processing”. It then uses the RVideo library, which in turn uses ffmpeg and flvtool2, to scale the movie and convert it to flash video. Finally, it uses RVideo to make a preview jpeg image, and uploads both to S3.
If anywhere along the way an error occurs the video record’s status is set to “failed” and a notification email is sent to both the property owner and the administrator using ActionMailer. On the other hand if the whole process succeeds the worker sends an email to the property owner only, and marks the video record as “ready”.

The last step was to display the video on a given page. To do this I used SWFObject and the JW FLV Media Player. I created a simple helper to create the SWFObject javascript for the video.

All of the external libraries I can say were very easy to work with, and made creating this functionality much easier. But the one I loved the most was RVideo. This library is a little gem for anyone wanting to script out ffmpeg or flvtool2 commands in Ruby. I highly recommend it.

Tags: Flash, video, rails, ruby, backgroundrb, flv, swfobject, rvideo, webdeb, ffmpeg

First off while not very well documented in my opinion is the ability of rails to have and understand sub controllers, such as the URI “/admin/users” which would be the controller in “app/controllers/admin/users_controller.rb”. To do this just requires some code like this:

app/controllers/admin_controller.rb

class AdminController < ApplicationController
   …
end

module Admin
end

app/controllers/admin/user_controller.rb

class Admin::UsersController < AdminController

   active_scaffold :user do |config|
      …
   end

end

Of course the main thing we wanted to do was to style it to match other administrative interfaces we were using. In case you don’t know already the ActiveScaffold team really thought ahead on this one and allows customization at pretty much every level. For sweaping changes that effect style most likely you’re looking at Template Overrides. Since we were looking to have all administrative forms look the same we just had to provide new template in the “app/views/active_scaffold_overrides/” directory. The templates that I found were needed for really doing what I we needed to do to change the look of the forms were the “_form.rhtml”, “_form_association.rhtml”, “_form_association_record.rhtml”, and “_form_attribute.rhtml” templates.

Of course you also often want to change the specifics of how a specific form input is generated for a given column is generated. And for this there’s Form Overrides. These are just little helpers in the helper file for the given controller. So say you want to change the type of input used for your password column on the User form, just add a method called “password\_form \_column” in the “user.helper.rb” file.
But let’s say you want to change all columns that are strings to function in a specific way, like say setting the size and maxlength attributes, well here ya go:

Article.columns.each do |column|
   if column.type == :string
      size = (column.limit > 59) ? 60 : column.limit + 1
      src = <<-end_src
         def #{column.name}_form_column(record, input_name)
            text_field(:record, '#{column.name}', :size => #{size}, :maxlength => #{column.limit}, :name => input_name)
         end
      end_src
      class_eval src, __FILE__, __LINE__
   end
end

Well there’s really lots more I could cover about active scaffold but I’ll let you sophisticats check it out for yourself. The documentation is top notch. All in all a great addition to your Rails toolbox.

Tags: ruby, rails, programming, code, ActiveScaffold

But of course like with anything else I decided the “basic” setup needed to be customized. I started by building and installing the nginx server with the Fair Proxy Balancer module, and set up appropriate logrotate and init.d scripts for it.
I then decided, based on numerous recommendations and benchmarks out there, to install the Swiftiply gem and use Evented Mongrel. The is done by installing the gem sudo gem install -y swiftiply and then using env EVENT=1 mongrel_rails instead of just mongrel_rails.

I then needed to pick an image processor for resizing uploaded images, rmagick is of course the obvious and almost “default” choice (particularly at RailsMachine). But I had heard reports about it eating memory and being a performance hog. So I decided to go with FreeImage and the ImageScience Gem instead.

Then of course there’s the almost ubiquitous memcached. How could I forget that! I installed that, the memcache-client gem, and the mem_cache_fragment_store gem. We use memcache for sessions, and fragment caching (partly via the Interlock plugin and partly via the built-in Rails fragment cache).

Lastly, since videos can be uploaded and I didn’t want to force property owners to upload a specific format, or have to support embedding different formats I installed ffmpeg, to convert them all to flash.

We also decided to go with Amazon S3’s service for hosting the image and video uploads, as well as serving as an asset host for our stylesheets, and images. And besides the monumental failure they had the other week it’s been a great success. In case you don’t know it’s really easy to map a sub-domain of your site to S3 by naming the bucket that domain name (in our case “assets.heelshousing.com“), and then setting a CNAMErecord with your DNS that points that sub-domain to the appropriate S3 domain name (in our case “assets.heelshousing.com.s3.amazonaws.com“).
It’s also a good idea when uploading images to S3 to set the EXPIRES and CACHE-CONTROL headers for them. Here’s a sample of the code I use to do that:

   AWS::S3::S3Object.store(
      key, File.open(path), BUCKET_NAME, 
      :content_type => content_type,
      :expires => Time.now.years_since(10).httpdate,
      :cache_control => "max-age=315360000",
      :access => :public_read 
   )

I think that should be self explanatory, but if not: I set the key variable to the name/path I want to use on S3, I set the path variable to the path of the file on the local machine, I set content_type to the appropriate MIME type, and of course I set the BUCKET_NAME constant to the name of the bucket I’m using.

Overall these choices were meant to hopefully increase speed of response for the user and performance on the server while also allowing everything to scale well as the sites user base grows.

Update:
The title refers to how Rails apps should not be hosted in shared environments.

Tags: Ruby, Rails, nginx, webdev, server, mongrel, Rails Machine, memcached, Evented Mongrel, Amazon S3

Another one of the sites great features is it’s overall integration with Google Maps. You’ll find on every property page a map of where the property is. Also a little javascript pop-up box (not window) with the map will be displayed if you click the “Map It!” link in the search results.
And just to put the proverbial whip-cream on the sundae, If a property owner or administrator specifies that a property is close to certain Chapel Hill bus routes, those routes are overlaid on top of the map.

The Daily Tarheel, who commissioned this site from us, run the day to day operations of it. This includes adding and updating current properties, but also selling property listings to local property owners. They can specify 2 levels of paid owner user accounts with different levels of permissions. The owners can then specify different types of units for each property, and upload unit floorplans as well as property pictures and a video.

In my next post I’ll go over some of the back-end features used to create and host the site.

Tags: webapp, webdev, Rails, UNC, Chapel Hill, Daily Tarheel

While getting started on that I wanted to try the new method for Active Record DB migrations. So I decided to rewrite all my old migrations (or at least the ones adding tables). I ended up crafting this regex to allow for a quick find and replace of the old style to be like the new style:

Find:

   \.column\s+(("\w+")|(\:\w+))\s*,\s+(\:(\w+))

Replace:

   .$5 $1

Tags: rails, regex, find-replace, migrations, code, activerecord

The idea from the beginning was to make very nearly identical in usage and in output to the built-in commands so as to make it as much of a drop-in replacement as possible. I also wanted to use the 12 hour time standard since most of us don’t go by a 24 hour clock. Thus similar or identical options are available, and the output will be almost identical to that made by the built-in commands, it will of course work with a form builder and take data from a passed in model object to set the selected options. In other words all the Rails goodness one expects!

• Unobtrusive Javascript Date-Picker Plugin

Tags: ruby, rails, programming, datepicker, javascript

The meetup in general was nice, it was my first one. There were some recruiters that came at the beginning to say hi and also that they had no Ruby or Rails jobs to offer. There was discussion of favorite text editors that actually didn’t become a fight and some brief Mac-talk.

Hopefully I’ll get time to go again after Lillia arrives!

Tags: rails, ruby, javascript, webdev, calendar, datepicker, raleigh.rb

It’s probably not production ready (although I haven’t found any immediate or glaring problems with it so far), and it desperately needs some tests written. But other than that it’s good to go!

It can be found as at http://rubyforge.org/projects/mongrel-compres/.

Tags: compression, HTTP, mongrel, Ruby, rubygems

So, I’ve begun work on a Mongrel plugin for offering this functionality.
More to come…

Tags: compression, HTTP, mongrel, Ruby

Pictures:

• My Chumby Flickr pictures

Tags: Chumby, cool, fun, photos, silly, widgets

#!/usr/bin/env ruby -KU

require 'optparse'
require 'ostruct'
require 'iconv'

SCRIPT_NAME = "File Character Encoding Converter"
SCRIPT_VERSION = "0.1"

class FileEncodingConverterOptionParser

   def self.parse(args)
      options = OpenStruct.new
      options.to = 'UTF-8'
      options.verbose = false

      opts = OptionParser.new do |opts|
         opts.banner = "Usage: convert_encoding.rb [options] directory|file..."

         opts.separator ""
         opts.separator "Specific options:"

         opts.on("-f", "--from ENCODING", "Character Encoding converting from") do |encoding|
            options.from = encoding
         end

         opts.on("-t", "--to ENCODING", "Character Encoding converting to") do |encoding|
            options.to = encoding
         end

         opts.separator ""
         opts.separator "Common options:"

         opts.on("-v", "--[no-]verbose", "Run verbosely") do |v|
            options.verbose = v
         end

         opts.on_tail("-h", "--help", "Show this message") do
            puts opts
            exit
         end

         opts.on_tail("-V", "--version", "Show version") do
            puts SCRIPT_NAME + ' ' + SCRIPT_VERSION
            exit
         end

      end

      opts.parse!(args)
      options
   end

end

options = FileEncodingConverterOptionParser.parse(ARGV)
VERBOSE = options.verbose ? true : false

class FileEncodingConverter

   def initialize(from, to = 'UTF-8')
      @to = to
      @from = from
   end

   def fconv(filename)
      begin
         contents = File.open(filename).read
         output = Iconv.conv(@to, @from, contents)
         file = File.open(filename, 'w')
         file.write(output)
      rescue Iconv::IllegalSequence
         puts "Could not be processed: #{filename}"
      else
         if VERBOSE
            puts "Processed successfully: #{filename}"
         end
      ensure
         if defined? file
            if file
               file.close
               file = nil
            end
         end
      end
   end

   def process_dir(dir)
      Dir.foreach(dir){ |filename| 
         if !filename.match(/^\./)
            absolute_path = dir + '/' + filename
            if File.file?(absolute_path)
               self.fconv(absolute_path)
            elsif File.directory?(absolute_path)
               self.process_dir(absolute_path)
            end
         end
      }
   end

end

converter = FileEncodingConverter.new(options.from, options.to)

ARGV.each { |arg| 
   wd = Dir.getwd
   entryname = arg.match(/^\//) ? wd + '/' + arg : arg
   if File.directory?(entryname)
      converter.process_dir(entryname)
   elsif File.file?(entryname)
      converter.fconv(entryname)
   end
}

• Script file download

Tags: ruby, iconv, utf8, encodings, scripting

I’ve included the script code below and posted the app and workflow with this posting (to use them you will have to first follow the instructions over at subtleGradient to install Safari 2 on top of Safari 3).
(*the workflow should be installed in “~/Library/Workflows/Applications/Finder“)

#!/usr/bin/env bash

IFS=" "
filename=${*##/*/}
appname=${filename%%.*}
binpath="$*/Contents/MacOS/$appname"

env DYLD_FRAMEWORK_PATH="/Applications/Safari2.app/Contents/Resources" \
WEBKIT_UNSET_DYLD_FRAMEWORK_PATH=YES "$binpath" >&/dev/null &

• Application and Workflow download

Tags: Safari, WebKit, MacOSX, bash, shellscripting, download, Automator

Tags: feeds, pipes, yahoo, ZFS

Tags: email, web apps, web forms, contact

Tags: mii, wii, avatar, images, photoshop

#!/usr/bin/env bash

v=''
# process the option flags
while getopts ":vh" opt; do
   case $opt in
      v  ) v='-v ' ;;
      h  ) echo "usage: renameproj.sh  [-v] [-h] old new" ;;
      \? ) echo "usage: renameproj.sh  [-v] [-h] old new" ;;
   esac
done
shift $(($OPTIND - 1))

# Get the old and new project names
old=$1
new=$2

# Function to rename files
renamepfiles(){
v=''
# process the option flags
while getopts ":vo:n:" opt; do
   case $opt in
      v  ) v='-v ' ;;
      o  ) old=$OPTARG ;;
      n  ) new=$OPTARG ;;
      \? ) echo "usage: renamepfiles  [-v] [-o oldproject] [-n newproject] args..." ;;
   esac
done
shift $(($OPTIND - 1))

# Make the move
for filename do
   oldfile=${filename}
   # the next line transforms the old filename into the new one by replacing the
   # old porject name with the new one.
   newfile=${filename/$old/$new}
   # first rename it with the move command
   mv -i ${v}$oldfile $newfile
   # then rename it with the hg command
   hg ${v}rename -A $oldfile $newfile
done
}

# perform an mercurial clone of the project
hg ${v}clone $old $new
# change the working directory to our new project
cd ${new}

# call the function over all the files with the "old" project name in their name
renamepfiles ${v}-n $new -o $old $(find . -name "*$old*" -maxdepth 2 -print | xargs echo -n)
echo "- Project Moved"

Tags: shell scripting, bash, mercurial, webdev, code